Monitoring and Assessment of the Effectiveness for Cultural Competence Development at University

This article is devoted to the research of the cultural competence development at the university level. It considers various interpretations of the concept of “cultural competence”, its componential criteria to assess students’ progress through their studying. The literature review revealed the redundancy of interpretations that can be explained by multiculturality of the concept as well as the lack of information, concerning assessment tools for the cultural competence monitoring. As a result of sifting and practical testing of a number of methods, the development of proprietary methodology eligible for the competencybased approach in the multi-level system of education is represented here. This study is intended to suggest compact and less time-consuming methods approbated by educators among the students of the Institute of Mathematics and Computer Science at the University of Tyumen, Russia.


Introduction
Integration of Russian education in higher schools into the world multi-level educational system resulted in some changes in the contents of almost all the programs and courses, related to the introduction of the competency-based approach, covering technical expertise and cultural competence. The term Cultural Competence (hereinafter CC) used in Russian educational standard is referred to the common-accepted general competence describing the combination of abilities, motivations, and traits required to be performed effectively in a wide range of jobs.
Until quite recently the main body of methodological and pedagogical researches was devoted to the development of Technical Expertise (hereinafter TE), sufficiently investigated at present. Modern scientific papers let educators have full-featured arsenal, ranging from meaningful contents for every sphere and learning path, various teaching methods and approaches to precise monitoring and assessment tools. Since the situation, concerning the development of CC in the scope of professional education is not the same because of some considerations articulated below; we need to focus our attention on studying of those aspects of the competency-based approach as development and improving CC, the problems of assessment of the students' progress in this path. Therefore, this paper considers various leading ideas of the competency-based approach at university, analyzes the concept of competence with particular attention to CC as a standard requirement of educational programs in higher schools.
Based on the scientific literature review, concerning the above competence, we distinguished a great variety of components and some levels of development in realization of the competency-based approach [1][2][3][4][5][6][7]. We also concluded that this approach combined TE+CC, contributed to some changes of the teacher's role in the educational context and proved to be important for teaching staff of all the programs, since educators must take over and adopt, evaluate and implement innovative projects and technologies in education, the effectiveness of which should be monitored and assessed through all the levels of education. Being competent today means that along with regular improvements in teaching a subject it's necessary to keep up with progress in pedagogy, psychology, and methodology to analyze comprehensively own teaching strategies and finally students' progress in the development of the competencies. Such multidimensional activities imply mountain of effort and time-consuming analyses caused by the shortage of definite methods and tools dealing with the effectiveness of the CC development as well.
Changes in paradigm of higher education in Russia resulted in some methodological and didactic problems, in particular, all the academic programs of both the Bachelor and Master levels contain the assessment criteria and requirements for TE that are abundantly information-packed and carefully developed for each, while examining academic requirements for CC we find them eclectic and presented indistinctly that can be explained by multiculturality and even redundancy of interpretations of the CC concept and its components.
Moreover, the CC criteria are duplicated through the Bachelors' to Masters' programs without taking into account the levels of training and development of the future specialists that make it impossible to assess the progress of students' CC. Besides, in contrast to comprehensively developed assessment system of TE (both midpoint and summative assessment), the assessment system of CC is absent or presented fragmentarily.

Purpose of the Research
Thus, the purposes of this research were, first, to distinguish basic components that at the same time are the criteria of CC. Second, they present our interpretation of educational experience, which covers along with TE the development of CC (set of skills) in the chosen learning paths in the Institute of Mathematics and Computer Science at the University of Tyumen with due account for in consecutive stages. Third, the aim is to develop and suggest to professional association of educators our methodology of monitoring and assessment of the effectiveness for the CC development of the university students according to the componential criteria and complex assessment; and finally, to represent the results of approbation of the set of methods carried out among the students of the Institute of Mathematics and Computer Science in 2017-2018.

Materials and methods
The competency-based approach is still in the focus of many researchers since the socio-professional requirements to specialists have been changing through the time, consequently, educational aims and outcomes have been changed and at present, the concept of CC, containing progressively more components, is currently used instead of readiness and education.
Competence was primarily concerned with the use of language, thinking skills, and human experience [8]. Personal characteristics including motivation were added to the concept of competence [9]. The detailed definition was given by Prof. Raven J. in "Competence in Modern Society: Its Identification, Development, and Release" in 1984. When considering the nature of competence, his definition comes complete with the concepts of "motivation", "goal setting", and "behaviour", so he concludes that it is impossible to assess competence without values. He adds "characteristics and abilities that enable people to achieve personal goals regardless of the nature of these goals and the social environment in which these people live and work" to the components of competence [10]. Thus, competence is based on intelligence and effective behavior as well as abilities and intrinsic motivation determined by human values.
Later general competence was regarded as fundamentals, the "four pillars on which to base education: learning to know, learning to do, learning to live together, learn to live" (the report in the international Commission on education for the XXI century "Educationhidden treasure" by Jacques Delors, 1996) [5]. In the same year (at the Symposium in Berne 27-30 March 1996), the educational reform suggested defining a set of core competencies required by students for successful work and for getting higher education in the future [11]. Russian researchers studied CC consider it as "the property of the individual" [12], integral individual merit [13]. Cultural competence determines the sustainable ability to mobilize knowledge, skills, and experience in any meaningful social or professional situations using critical thinking and the most effective solutions [14]. Therefore, CC is defined as some alienated, reassigned requirement for a job as well as self-regulation, self-reflection, and selfevaluation [15].
Thus, the reviewing and analysis of the research papers devoted to the concept of competency elucidated a wide variety of componential composition of this concept, depending on the interpretation of its essence by different authors. Despite the indisputable value of the competency-based approach in teaching, its implementation is deficiently applicable to educational practice.
Here, in the scope of this approach, we discuss another way of interpretation of educational experience covering integration of both CC and TE (i.e., a set of skills) in a certain field. It is the very combination that finally develops competent graduates. Considering CC as a personality trait, continuously developing through the entire university course, we should determine its level to be sure whether we are on the right track to developing competent in every respect specialists. For this purpose, we defined some criteria of CC to ensure the students' progress in this field and continuity through the Bachelor's and Master's degrees as well. We distinguished the following componential criteria of CC that seemed to be the most significant, informative, quite convenient and compact to be assessed by the methods to follow: Component 1 is a system of motives, goals, and values in cultivating a positive attitude toward any professional activity; appreciate the importance of CC in relation to career objectives. Component 2 assumes the presence of a certain amount of cultural skills and knowledge about CC that are necessary for the future professional activity of a bachelor or master's degree graduate.
Component 3 is significant because of reflection on the ability to develop self-consciousness and personality, analysis and evaluation skills, problem-solving and other marketable skills that allow correlating the results achieved with the goal and feeling responsible for the work done.

Criteria and indicators
Diagnostics Component 1: -internal motives for a positive relationship to CC in professional activities; -awareness of the importance of CC for educational and professional activities.
"Diagnostic test of the subject importance for the development of the student's personality" [16];; "Educational motives at the University", "Development of cultural competence in studying mathematical disciplines", and the questionnaire about the desired level of education [17]. Aim: Determine the level of motivation while forming a positive attitude toward the development of CC through the study of mathematical disciplines. Component 2: -possession of CC at the level of a bachelor's or master's degree; -productive educational activities in the educational environment at University; -manifestation of activity and independence in the class.
Aim: Determine the level of CC based on mutual and expert self-evaluation. Component 3: -involvement in cognitive activity; -self-assessment of readiness for independent implementation of educational and professional tasks; -development of skills for self-analysis and selfassessment while solving educational problems and correlating the result with the goal.
Aim: Determine the level of the ability to selfreflection.
Based on our criteria and the degree of severity of the relevant indicators, it is possible to monitor the dynamics of the students' progress up to a higher level of CC development. In accordance with the ordinal criterion-level scale, we can give general characteristics for the levels of CC development. The following diagnostic methods, used to assess the criteria of values and motivation, knowledge, skills and experience, as well as reflection, are presented in Table 2.

Results and discussion
In accordance with the principle of comparative importance, consideration of the integrated assessment should be based on the fact that the criteria and their indicators are not equal in their significance and impact on the final result. Therefore, we chose the method of qualimetry ("Qualimetry" from Latin cval -'quality' and metros -'to measure'), since it is based on the isolation of the component parts of the certain characteristics, diagnostics and assessment of the components and the elimination of the integrated index.
The comprehensive assessment of CC development was determined by the qualitative rule:  We conducted both componential analyses of CC development individually and comprehensive assessment to monitor and rate the levels of CC development in the student groups of the Bachelor's programmes.
Diagnostics of CC development for undergraduate students at all stages of the research was carried out according to the compiled data for each of 93 students. Personal data collected in the personal diagnostic sheet can be presented like in Table 4. Table 4. The sample test-sheet for componential diagnostics of student N.

Characteristics Point
Component 1 N. is interested in the competencies; realizes the importance of their possession both in the class and in the further professional activities.
2 Component 2 N. has a minimum level of knowledge about how to use CC in the educational and professional activities; midlevel intellectual potential and some general skills that he uses only in standard situations (after the pattern/model).  The generalized results of the ascertaining phase of the research are presented in Tables 5-6. The threshold level of CC development: students show unstable interest in СС. They attach great importance to some competences in educational activities, but they believe that they won't use most of them in the future. A minimum level of competences or only a part of them has been mastered. Practical readiness to use competencies in business is formed at the level of reproduction. Students are ready to perceive and analyze, communicate and visualize information, but processing, critical evaluation, and application of information are possible only with the teacher's help.
The basic level of CC development: students are well aware of the importance of competencies in future professional activities. They are challenged with the need to improve their already acquired skills and abilities, strive to gain new knowledge, but they lack the depth and strength of this knowledge and systematic replenishment. They have an idea of all competencies, and ready to perceive, analyze, and critically evaluate information in solving the problems with a little guidance.
The advanced level of CC development: students have a persistent need for the development of competences. Students have much subject knowledge, possess all competencies, and they are active and independent in the class, often offer creative, nonstandard solutions. They are able to evaluate critically the results of their own activities as well as those of the other students.
The presented study discusses the problem of Cultural Competence Development of the University Students, particularly monitoring and assessment tools. Information and literature review resulted in the conclusion that such tools for the CC were worked out rather fragmentarily, i.e. without the system-wide approach in contrast to the thoroughly developed methods for the TE improvements. Since both the Bachelors' and Masters' program requirements include CC, it should be assessed in order to conclude whether the students' and educators' activities are effective enough. Besides, some requirements to the CC skills are duplicated through the Bachelors' to Masters' programs, so it implies that the level of CC for the Masters' students must be higher, and again, that can be monitored by means of the methods suggested above.
Another difficulty we faced with was multiculturality and multivariance of the CC concept, interpreted in many ways and including plenty of components, therefore, to make our research methods more compact and less time-consuming we distinguished three main componential criteria of CC. In the course of the present research on the diagnostic stage, we sorted and approbated various tests, the most instrumental of which turned out to be those ones presented in Table 2.
For the comprehensive assessment of CC development, the method of qualimetry was chosen as it allowed complying with the qualitative rule. Despite the apparent at first sight complexity of such calculation, we should notice that the above algorithmic formula can be performed by computer-aided estimating so that it presents no problems.
The test results while approbating our methodology made it possible to define the levels of the CC development of the university students at the beginning of training and showed that the criterion of motivation, goals, and values is evaluated by higher indexes in percentage correlation than the criteria of knowledge, skills, experience, and ability to reflection according to the data from Tables 5-6. On the bases of these data, we concluded that most of the students have got the threshold (to a greater extent) and basic levels of CC and decided on the case study methods that could be appropriate and chosen at the definite stage of a program as a component of the general pedagogical strategy. The students' personal diagnostics sheets (Table 3 and 4) can be helpful in the implementation of both a studentcentered approach and realization of learning paths in general.

Conclusion
Modern strategies in the university education tend to develop both TE and CC of students through all levels of their studies and consequently must imply monitoring and assessment of students progress. After having reviewing worldwide teaching experience it was found that the monitoring and assessment tools for the CC were not presented to the full extent and varied greatly, therefore, our research aimed to select the appropriate set of methods, test and try out the most effective of them. The developed methodology includes the componential criteria of CC, the rating of the levels with their detailed description, the ways of the data compilation. The methodology combines the qualimetry assessment, comparative analysis, generalization and specification, testing and questionnaire. It could be used as an assessment tool at the beginning and completion of the Bachelors' to Masters' degree programs, but if there's necessity to check up the effectiveness of the teaching methods to improve and correlate learning paths in the course of a program, especially a new one. The methodology can be recommended as an informative monitoring tool at least three times, i.e. additionally as a midpoint assessment.