Assessing the reliability and validity of a shorter version of RIASEC in Turkish

Holland’s RIASEC types are being frequently utilized in commercial vocational profiling tools for various human resources purposes. On the other hand, the length of the RIASEC scale and the copyright restrictions put by the publishers, are important barriers to application. In the present study, a RIASEC scale consisting of 41 items and adapted to Turkish language and culture, was developed. Each RIASEC type was represented with 6 or 7 items. Responses were obtained from a sample of 364 business professionals. Survey results indicated a good reliability for the scale, with a Cronbach’s alpha of 0.889. However, reliability analysis pointed out to the need for revision of certain scale items when each RIASEC facet was separately analysed. Then, feedback regarding scale composition, wording and structure were gathered from 20 PhD students. Lastly, feedback of 7 HR professionals were sought, regarding scale items’ expression and application of the scale in regular HR processes of companies. Results from face and content validity have been that for some items of the scale, more descriptive and specific expressions in Turkish are required. Moreover, some of the items would need to be reallocated to another facet where they would be more relevant. In line with findings from face and content validity, construct validity through confirmatory factor analysis also indicated that the short version of RIASEC must be revised substantially in order to become a valid tool for vocational profiling in Turkish context.


Introduction 1.RIASEC: an integrated model for determining vocational profiles
Vocational profiles gather information about individual personality traits, skills, interests, values and other behavioral aspects that are assumed to be affecting the strengths and limitations of individuals when pursuing their educations, vocations or careers.Literature on vocational research increasingly emphasize the significance of developing integrated models representing the inter-relations between constructs for a more comprehensive understanding of the relation between vocational profile and job performance and career development, [2], Junk, 2012).A commonly used model for integrating personality traits to other individual characteristics such as a person's interests to determine job performance is RIASEC.According to Holland, [10], [12] there are six types of personality which can also be used to describe six types of work environment: Realistic (R), Investigative (I), Artistic (A), Social (S), Enterprising (E), and Conventional (C).Hence, RIASEC is the acronym for the Holland's six classifications of personality and job interests, [2].This classification is used in matching people with appropriate work environments and employment opportunities, [6].Each of RIASEC typologies can be described in terms personality traits and also preferences of occupations.For instance, Realistic types would have characteristics such as "compliant, rigid, genuine etc…", as well as high vocational preferences such as working with tangible things and nature (e.g., tools, machinery, animals), and lower preference for activities related to educating or helping people, [17].

Work values: contextual elements
Holland's model assumes that people look for and select the environments where they can express their interests.Once the work environment is selected, there is further interaction between the individual and the work-place, which results in further personalization of work environment according to the individual's type.Holland emphasizes that the congruence between the person's characteristics and the environment (e.g.workplace or field of study) may affect his or her contentment, performance and continuity in that occupation, [11], [12].However, definition of congruence is controversial, [13] since to achieve a successful fit of individual to workplace requires other factors such as coherence of skills and work values with the work environment, [4].Work values are important contextual elements complementing material rewards, [15].They are not exactly the same as needs but are created as a result of prior awareness of the subject need / value area.They represent an evaluation at the current time and do not involve expectations, [15].Work values can be considered as the rewards one gets from the job that are considered and contemplated by the employee and given a meaning.There are several factors that influence the development of work values: First, socialization and other types of life experiences prior to entry to workforce shape person's overall values.Second, social roles influence the perceptions and meanings given by the individual.Last, work experiences affect a person's evaluation of certain rewards, [15].Work values are important determinants of job satisfaction and performance.Person -organization match delivers desirable work attitudes and outcomes, resulting in higher job satisfaction.Hence, work values interact with vocational interests measured by RIASEC, [23].

Vocational interests and job performance
Vocational interests affect work performance through motivation.According to social cognitive career theory, [16] interests drive goals related to career, the actions taken to achieve them, and the performance in achieving those goals, [26].Interests mainly drive people to gain job-related knowledge and skills, which brings superior job performance, [1], [3], [16], [22], [26].Individuals whose career interests are matching with the work environment will be motivated to obtain appropriate knowledge and skills, which will help them to build intelligence in their area of interest and perform better than others who do not have matching interests.Moreover, interests may also drive the productivity of training, and better training outcomes, during learning of necessary knowledge and skills, [18], [26].Similar to job performance, job turnover is also driven by interests.According to Holland, [11], [12] individuals with interests fitting to those provided by the work environment, will be more likely to enter, be content, and stay longer.Hence, congruence of person's interests and the workplace will affect organizational commitment, turnover, as well as behaviours such as absenteeism.This in turn creates positive job attitudes, job satisfaction, and likelihood to stay compared to those who have less positive attitudes, [26].Lastly, Schneider's, [20] attraction-selection-attrition (ASA) theory states that as the employees and workplaces seek for congruence of interests, organization may eventually offer a more homogeneous environment in terms of personalities, competencies and interests, [21].

Validity of scales across different contexts
Although Holland's theory states that individuals' job interests are shaped by their personalities, construct validity studies consistently indicate that there is small to moderate correlations between personality and RIASEC vocational interests, [9].Tracey and Rounds, [25] found no support to suggest RIASEC dimensions as indicative of personality types.Deng et al. [5] suggested that Holland's six classifications represent a limited range of occupational interests, and therefore, is not a fit for the United States.The application of Holland-based interest measures may have limitations due to its length of 180 items and copyright restrictions.Goldberg [8] has mentioned concerns about copyright restrictions in personality testing.Separately, Holland [12] has argued that the completion of RIASEC scale can be considered as a career-related intervention in itself due to a wide range of job opportunities being prompted to the respondent when completing the scale, [2].Some people naturally may like or enjoy most of the items on the scale, because of their psychological disposition.On the other hand, others may have a more critical personality and so may dislike most items or tend to evaluate more negatively, [27].Also, RIASEC scales may be confounded with topics of prestige and gender, [5].Situational factors and their impact on expression of personalities are also missing elements.Tett and Burnett, [24] point out to certain situational factors influencing trait expression that might be used to clarify the conditions favouring personality use in selection efforts.
An important question concerning RIASEC is its validity over different cultural and social contexts.Since Holland's RIASEC model is widely accepted in the United States, researchers and practitioners from other countries have adopted this model without any reserves, [19].It is translated to many languages and used in cross-cultural vocational studies.However, the generalizability of the test remains questionable, [19].Walsh, Savickas and Hartung [27] argue that the scale may deliver varying results according to different samples.Hence, a methodological adaptation might be required to apply RIASEC in other countries or cultures.In the rest of the paper, we will examine the reliability and validity of a shortened version of RIASEC scale to modify it for using in the Turkish career environment.After analysing the fit of its items to each dimension in terms of reliability, face and content validity, we will complete our analysis with a confirmatory factor analysis testing the discrimination of RIASEC dimensions from work values dimensions.

Reliability and validity of RIASEC short version
Cronbach's alpha is a commonly used way of testing reliability, and is a measure of internal consistency.For all dimensions alpha is higher than 0.700 which is above the acceptable threshold, except for Investigative dimension which improved after removing one item, [7].
Validity refers to the degree the scale measures the intended construct.Validity evaluation is crucial in order to minimize the errors caused by measurement, and without validity evaluation the outcome of a scale might be erroneous and must not be used, [7].

Face validity
Face validity is the extent a measure appears related to a specific construct as perceived by the target group to whom the test may be applied, [7].Generally, respondents of a scale give feedback on scale items whether they provide a good and comprehensive representation of the key concept.Face validity of the short version of RIASEC has been evaluated by twenty PhD students with a variety of educational and carreer backgrounds.According to the feedback, some facets of shorter version of RIASEC needs to be supported with more items.For instance, Artistic dimension should have additional items reflecting more contemporary interests such as media or graphics.Investigative dimension should have additional items related to topics other than hard sciences, such as "analysing other cultures", "journalism" and "reaching information sources".In the Realistic dimension, expressions need to be clarified, i.e. "working close to plants and animals" should be replaced with "agricultural work or raising livestock".Moreover, handcrafts might be added to realism dimension.In the social dimension, NGO type of voluntary activities needs to be included.Lastly, Enterprising dimension was considered as having sufficient number of items.
Respondents also stated that the shorter scale includes more generalized descriptions of occupations and does not go into details of granular job types, and are not accurate translations of the original RIASEC scale.Moreover, some of the questions are found to express more work values or personality aspects -e.g.creativity, independence -instead of job interests.Some other questions might have loaded meanings and it is highly probable that they would have social desirability which is reflected to high mean scores and low standard deviations of those scale items.Respondents also stressed strong biases that might affect scale responses such as those caused by social pressure, employer pressure, and biases due to social status and sex.For instance, women may rate items such as "understanding how machines and gadgets work" quite low since it is seen culturally as a masculine type of job.Finally, shorter RIASEC scale items are found to be expressing more generalized interests which have to be revised to have more stable and concrete meaning in local context.Like face validity, content validity is about how much a scale truly reflects its intended construct, [7].However, rather than lay people, the scale is evaluated by experts on the particular subject area.Experts provide feedback on meaning of scale items, if they are well linked to the key concept, and the construct of overall scale.In this study, content validity has been evaluated by seven senior HR professionals and recruitment specialists coming from finance, construction and ceramics sectors.According to the experts, the original RIASEC scale is being widely used in Turkey for career planning, and provides correct guidance for an individual's career development.HR experts state that job interests is the correct point to start when determining the right career path for a person.

Content validity
Most of the respondents have found the shorter RIASEC scale was to be less professionally developed and insufficient with respect to the number of items used.They contended that for a better adaptation of RIASEC to local culture, number of items, dimensions' names, item wording and meaning should be handled in a more comprehensive way.Almost all of them were critical about word by word translation of English items.Another comment was if RIASEC intends to measure job satisfaction and job stability, then the items should be designed in a way to provide responses consistent under various situations.These need to be evaluated with additional criterion validity assessments.There has also been a recommendation for inclusion of cross-check questions on personality traits for better validity assessment.Moreover, some questions have been described as loaded, so they would attract more positive responses.

Construct validity-discriminant validity
Construct validity is assessed through evaluation of discriminant validity.
Thinking behind discriminant validity is that the scale representing a certain concept must have minimum overlap with scales of distant concepts.Consequently, correlations between the facets of two distant scales must be low.
As for discriminant validity, shorter RIASEC scale has been compared against Work Values scale through Confirmatory Factor Analysis (CFA).CFA is a prominent method of measuring construct validity.Unlike to the Exploratory Factor Analysis (EFA) where data indicates possible set of latent variables and structure of the construct, in CFA there is hypothesized model structure, based on the scientist's interpretation of available knowledge and theories, regarding what he or she aims to conceptualize.CFA has been found superior to other methods of construct validity, such as Multi trait -Multi method (MTMM) methodology.This is because MTMM aims to confirm the the latent variables of a construct via degree of similarity to relevant or irrelevant constructs, but does not provide any improvements to the construct in hand.On the other hand, CFA analyzes totality of the construct -the latent variables and their explanatory factors -, allows the scientist to impose further constraints and adjust the model, and finally suggests modifications in order to have a better fit of the hypothesized model to real situation conveyed by the data.
In CFA, the aim is that the hypothesized model captures covariance between all items in the sample data.If this is not achieved, a poor model fit results, and changes in factors' connection with latent variables would be required.

Findings
To test discriminant validity we modelled RIASEC and Work Values as two different latent variables with no correlation.Although the items are significantly loaded to their

Discussion
Our findings suggest that short version of RIASEC needs substantial revisions to offer a valid test for vocational profiling in the Turkish context.Among the possible changes we have reported in our previous comments, special care needs to be given not to confuse when selecting Realistic and Conventional; Enterprising and Recognition items.Moreover, Social and Conventional dimensions of RIASEC appear to be correlated with the Work Values, hence revising the items within these dimensions to be more relevant to interests than values will substantially improve discriminant validity.
presumed constructs, model fit measures are far beyond acceptable thresholds, suggesting that RIASEC largely overlaps with work values and we cannot provide enough evidence for discriminant validity (Table1

Table 1 .
Discriminant validity; CFA model between RIASEC and WORK VALUES Root mean square error of approximation (RMSEA) = 0.147; Upper limit of the 90% confidence interval for RMSEA = 0.171; Correlation between factors = 0.000; N = 130.Following the suggestions in the modification indices, we allowed covariance between RIASEC and Work Values, error terms of Realistic to Conventional, Enterprising to Recognition and Social and Conventional to values.The fit indices improved substantially coming closer to acceptable threshold (Table2).

Table 2 .
Discriminant validity; CFA model between RIASEC and WORK VALUES after modification Root mean square error of approximation (RMSEA) = 0.72; Upper limit of the 90% confidence interval for RMSEA = 0.104; Correlation between factors = 0.436; N = 130.