Business Simulators in the Educational Process: Comparative Evaluation of the Effectiveness of the Use

There is considerable mobility in the educational services market: stakeholders’ requirements, the forms of organizing the educational process, and the training tools used are changing. Testing of changes requires monitoring and evaluating the effectiveness of improvements and the impact on changes in content parameters, the quality of learning materials and the demand for formed managerial competencies. There is a surge in researchers ‘interest in developing methods for evaluating and justifying the increase in improvements from introducing changes to the educational process. This work reflects the results of an experiment conducted by a research group to expand the tools for teaching students of management training programs. The hypothesis is put about a higher rate of information assimilation and the formation of managerial competencies in groups of students in which classes are organized using business simulators, in contrast to those groups of students who master the course using traditional means - problem solving and case presentation. The authors believe that simulators based on the principles of gamification, to a greater extent satisfy the Generation Z students’ needs for mobility, rapid feedback, change of impressions and locations. The proof of the hypothesis is based on the differentiation of the student group into two subgroups, with a preliminary assessment of students ‘ academic performance in economic and management disciplines and input testing of knowledge in Unit Economics.


Changing stakeholders' requirements for educational technologies
Evaluating the effectiveness of various forms of organizing the educational process, the training tools used, information transmission channels, methods of communication and interaction with students over a long period of time is an urgent demand for many stakeholders. Heads of educational areas are testing new tools for implementing educational trajectories, investing resources in modernizing the equipment of the educational process. Students test new ways of communication and gaining knowledge-online formats and action learning; try coaching and mentoring programs [1]. Teachers are also active in developing original programs [2] and new educational tools. Public authorities form national projects for the implementation of modern popular initiatives. The support and interest of stakeholders allow the education system to develop and be equipped with up-todate highly effective practices. The contribution of the scientific community is to understand and evaluate the impact of the practices used on the resulting process -the quality of professional competence formation.
All stakeholders actively discuss the changes introduced in training systems, since changes affect everyone and are associated with obtaining a social good -education, which is the foundation of professional viability and competitiveness in the labor market. The interest of stakeholders is stable.
We note that the quality of education and students' motivation to study in a particular specialty, as well as to the quality of knowledge and skills development are very heterogeneous, they depend on many factors: individual characteristics of students, the nature of the students group, the level of the student body development and the teaching staff of the educational institution. Creating motivation for learning is a rather complex and ambiguous process of changing the attitude of a person both to a particular subject of study and to the entire educational process. This determines the feasibility to continue searching for new learning tools. This work is aimed at presenting the results of the experiment of introducing new tools into the educational process and substantiating the hypothesis that the impact of using business simulators in the educational process of students belonging to Generation Z is higher than traditional cases and simulated tasks.
To achieve this goal, the research group consistently solved the following tasks: -first, the rationale for the frequency of use and breadth of distribution of specialized business simulators in the training of representatives of generation Z; -second, the formation of experimental students' groups, marking the input parameters of academic performance and motivation to learn; -third, conducting an experiment in the form of classes using business simulators in one group and using traditional forms in the form of solving problems and cases in another; -fourth, processing and analyzing the results, checking their statistical significance; -and, finally, validation of conclusions and recommendations on replication of experience and development of business simulation tools in education.
This sequence of actions will allow us to work out the hypothesis and confirm/refute the positive impact of new tools used in the educational process on the quality of material acquisition and the formation of professional competencies.

Trends in the development of society and features of young professionals: review of literature
Development transforms the principles of economic functioning and social interaction. The qualitative composition of the labor force is changing: an increasing part of the workingage population is made up of employees belonging to generation Z. The essential characteristics of the labor force are changing. Key provisions of the Stauss -Howe generational theory [3] prove the influence of socio-economic and political events of society on the formation of value attitudes of the population in General and labor resources in particular. Representatives of the modern generation who actively join the work-force-Generation Z-were formed during the period of digitalization of society under the influence of changing technological patterns. The largest sociological schools (M. Castels [4]) record the distinctive characteristics of this period: -informationality -the ability to generate, process and effectively use information based on knowledge; this determines the productivity and competitiveness of agents in the economy; -globality -organization of the main types of economic activity (production, consumption and circulation of goods and services), as well as their components (capital, labor, raw materials, management, information, technology, markets) on a global scale directly or using an extensive network connecting economic agents; -development of electronic networks as a dynamic, self-expanding form of organization of human activity [4].
Therefore, young professionals, the formation and socialization of their personalities during this period, are more likely to have the following qualities and characteristics [5]: -continuous connection with gadgets and social networks; -high involvement in the digital and technological world; -high focus on entrepreneurship and self-centeredness; unstable partnerships and a wide range of choice; -propensity to get rapid results; -a more relaxed attitude with money and no targeted focus on material success; -preferred formats of learning through action and practice; -immunity to educational technologies with a focus on theory; -a tendency to fragmentary (clip) thinking and processing a large amount of input information.
The changes outlined above also lead to a change in educational concepts, methods of presenting material, motivation, and retention of attention. Figure 1 shows the causal relationships of the above arguments. Therefore, educational technologies are intensively updated with active practiceoriented ways of interaction between the teacher and the student. These interaction models include the use of business simulators. Since their use in the educational process allows you to attract attention and arouse interest from the first seconds, otherwise the representatives of the considered generation will switch to something else.

Forms and tools for organizing the educational process for representatives of Generation Z
The changes recorded above in the typological and qualitative characteristics of Generation Z students suggest that gamification may be one of the appropriate forms of learning and communication. This position is also confirmed by the active introduction of gaming tools into the educational process over the past five years [6][7][8]. Gamification is an imitation of reality, which allows you not only to remember, assimilate and apply knowledge by 90-100%, but also not to forget them in the future. Gamification is based on the use of game tools in management training processes [9][10][11]. Examples of gemification tools include trainings, business games, strategy sessions, case and project presentation, and modeling. The use of gamification in learning leads to a more solid assimilation of information, since everything learned is associated with specific actions and events, rather than with abstract symbols and General principles [12][13]. Gamification tools allow you to apply the acquired knowledge "here and now" to explain the phenomena observed in life. And unlike in real life, getting results during gamification is not associated with long expectation, high risks, and significant costs. In addition, gamification provides some real help to the teacher in stimulating interest in the subject. And one of the most striking examples of gamification in education is business simulators.
Interest in the development of educational simulators has been stable for a long period and affects many areas of training. In the beginning, high-risk industries and those with a high price of error were equipped (medicine [14], construction [15], air and rail transportation [16][17], power supply [18], engineering [19][20]), then the development of gaming tools began to find application in management activities. Initially, the use of simulators was carried out for activities that are difficult to reproduce in a real environment, as well as works that have a high cost of error due to the human factor. The development of technical capabilities for the development of simulators and the development of solutions for the instrumental support of modeling phenomena and processes -allow you to expand the range of tasks solved using simulators. Simulators are a kind of bridge between learning in the classroom and the real environment, because they allow you to form skills through actions.
The description and characteristics of the use of business simulators in the educational process require factual confirmation, including an assessment of the resulting effect. It is for this purpose that an experiment was organized aimed at a comparative assessment of the effectiveness of students ' training in management disciplines based on business simulators and traditional means of information transmission.

Organization of experimental work and methods for evaluating the effectiveness of simulators in the educational process
To confirm the refutation of the initial theses about the best performance of students ' assimilation of educational material using business simulators and justify the feasibility of their introduction into the educational process, an experiment was organized with the following distribution of work: First, the choice of a specialized business simulator aimed at mastering and developing knowledge, skills and calculations based on the Unit-economy methodology is justified.
Secondly, the analysis of the initial level of training of students and segmentation of students into two groups was carried out.
Third, differentiated classes were organized: in one group, classes during the semester were conducted using a business simulator, in the other-based on solving traditional problems and cases.
Fourth, the level of training of students was checked based on the results of training. Finally, the results are processed and the conclusions of the study are formulated. This sequence of actions was chosen before the start of the experimental work.

Research results and empirical verification of results
The study was conducted in the framework of classes in the discipline "Business planning "in the course of studying the topic "Unit-Economics". 64 students took part in the experiments. In Group A (37 people), a business simulator was used during the classes, while in group B, traditional forms were used in the form of tasks and cases. Initially, information was collected on students ' academic performance in disciplines with topics from related fields of Economics (Microeconomics, Statistics; Methods of Managerial Decision-making, Financial Management, Investment Management, Econometrics, Industry Economics and Economic Analysis).
Each of the presented disciplines is evaluated on a 100-point rating scale. After collecting the information, the average values of students ' academic performance were calculated. Before conducting the study, an initial level of students ' knowledge of Unit Economics was tested and the relationship between student performance and test results was determined (Fig. 2).

Fig. 2.
Relationship between the mean score in the disciplines and the results of the initial level of knowledge tests in Unit Economics.
The value of the correlation coefficient was 0.20, while tcalc = 1.18 < ttable = 2.03, so with 95% probability it can be argued that the relationship between the value of the average score in economic disciplines and the results of tests of the initial level of knowledge in Unit Economics is absent or is statistically insignificant.
Within 4 hours several groups were taught the basics of Unit Economics: when working with one subgroup of students, you can use a business simulator, and with another grouptasks and cases. After classes, students took a test of their final level of knowledge in Unit Economics.
We tested the hypothesis that the initial level of knowledge in Unit Economics influenced the results of the final test. The results of the correlation analysis were calculated separately for group " A "(0.12, tcalc = 0.45 < ttable = 2.13), for group" B " (0.38, t calc = 1.29 < ttable = 2.18), and for the entire sample (0.34, tcalc = 1.83 < ttable = 2.05). Thus, it can be stated with 95% probability that the initial level of knowledge in Unit Economics did not affect the final level of knowledge in this area after completing training using both a business simulator and tasks/cases.
As a result, for further analysis, you can use statistical data of students ' learning outcomes not only for those who completed both tests, but also for those who only completed the training and completed only the final level of knowledge in Unit Economics. Thus, the statistical series for further analysis was 34 paired values (Fig. 3)   Fig. 3. Relationship between the average score in the disciplines and the results of the initial level of knowledge tests in Unit Economics.
The correlation coefficient is 0.44, with tcalc = 2.76 > ttabl = 2.03, which means that there is a 95% probability that the relationship is statistically significant.
Further, using regression analysis and the least squares method, the equation of the detected dependence f(x) = 1.01 x -26.59 was obtained.
The analysis of individual coefficients yielded the following results: the multiplier for x (a): tcalc = 2.76 > ttable = 2.03, so we can say with 95% probability that this coefficient is different from 0 and is statistically significant. In this case, the confidence interval of the coefficient value with 95% probability is within the following values 0.27 < a < 1.76.
The significance of the model as a whole was also analyzed. The value of the coefficient of determination was R 2 =0.19, while Fcalc= 7.62 > Ftable = 4.15, so we can say with 95% probability that the regression model is statistically significant.
The presence of this relationship indicates, most likely, the ability of a person to learn. Higher past scores indicate that people with higher mean scores are more likely to learn new knowledge and skills, so they get better results in the final test.
The average values of the test results of the final level of knowledge and other important parameters of the analysis are presented in Table 1. Therefore, in further analysis, it is necessary to take into account possible differences in the average score in disciplines with subjects from related areas of Economics.
For correct analysis, you need to adjust the value of the average test result of the final value for the difference in the average score. After appropriate adjustment based on the model obtained as a result of regression analysis, the calculated average value of the test results of the final level of knowledge for group "A" was 49.4 (the calculated value after adjusting for the difference in the average score for disciplines).
Calculations to analyze the reliability of differences in the average values of the two groups showed that tcalc= 1.57 < ttable= 2.04, so it is 95% likely that there are no significant differences between the results of training when using various forms (using a business simulator and using tasks and cases).
Additionally, a detailed analysis of the relationship between the average score in the disciplines and the results of the final test was carried out. It is noted that there are participants who received 0 and 10 points, which is significantly lower than the results of all other participants. A detailed analysis of their average scores in the disciplines showed that the scores in several disciplines at once for these participants are 50 points -the minimum threshold score for passing the test / exam in the discipline. There was an assumption that these scores could have been obtained with lower knowledge of students than they actually are. It was also hypothesized that there are two different groups of students who have significantly different results of mastering disciplines in different forms of education. Subgroup 1-students whose knowledge is assumed to correspond to the grades, subgroup 2-students who have a large number of grades of 50 points, and their real level of knowledge learning ability is significantly lower than the level of grades.
For additional analysis, a hypothesis was formulated that the form of training has a significant impact on the learning outcomes of students from Subgroup 1. At the same time, subgroup 1 included students who had no more than 2 grades of 50 points in the disciplines.
The adjusted statistical series for analyzing the relationship between the results of the initial and final level of knowledge tests is shown in Figure 4. The calculations allowed us to conclude that there is a 95% probability that the initial level of knowledge in Unit Economics did not affect the final level of knowledge in this area after completing training using both a business simulator and tasks/cases.
As a result, for further analysis, you can use statistical data of students ' learning outcomes not only for those who completed both tests, but also for those who only completed the training and completed only the final level of knowledge in Unit Economics. Thus, the statistical series for further analysis was 29 paired values (Fig. 5)   Fig. 5. Relationship between the average score in disciplines and the result of the final level of knowledge test in Unit Economics.
The calculated correlation coefficient was 0.22, with tcalc= 1.17 < ttable= 2.05, which means that there is a 95% probability that the relationship is absent or statistically insignificant. Thus, when analyzing the differences in the average values of the final level of tests between subgroups with different forms of training, the average score for the selected disciplines can be ignored.
The average values of the test results of the final level of knowledge and other important parameters of the analysis are presented in Table 2. Calculations to analyze the reliability of differences in the average values of the two subgroups showed that tcalc= 1.58 < ttable = 2.05, so it is 95% likely that there are no significant differences between the results of training when using different forms (using a business simulator and using tasks and cases).
In addition to the effectiveness of the training, the participants ' own opinions on the process were also collected. During the study, participants answered 3 questions: -Question 1: Evaluate how much you liked the organization of this seminar session compared to traditional forms of organizing seminars (1-much less liked, 4-no differences, 7-much more liked); -Question 2: Evaluate how much from your point of view this form of organizing classes helps better understand the topic being studied compared to traditional forms of organizing seminars (1-much worse assimilation of the material, 4-a similar level of assimilation, 7 -much better assimilation of the material); -Question 3: Would you like all seminars to be organized in this way? (1 -I would not like it at all; 7-I would like it very much).
The average values of the results of the estimated parameters are shown in Table 3. Thus, we can conclude that the satisfaction and interest of participants in the class when using a business simulator is significantly higher than when using traditional forms of education in the form of problem solving and case studies. Of particular interest is the fact that the participants themselves believe that a business simulator is a more effective form of training, in which they learn the material better.

Conclusions and final provisions
During the experiment, a business simulator for Unit Economics was developed. The business simulator is based on a common business model of the company that attracts consumers through the use of contextual advertising. Thus, the developed business simulator closely imitated part of the business model of most modern companies. In addition, a printed version of most elements was developed, which increased the involvement of students. Thus, it can be arguable that this business simulator has met the world level of such simulation models.
"Entrance" testing allowed us to assess the students initial level of knowledge in the field of Unit Economics, as well as to further assess the degree of influence of initial training on the final level of knowledge. The calculation of the mean score for the selected disciplines allowed us to assess further the impact of previously acquired knowledge on the initial level of knowledge in the field of Unit Economics, as well as to assess the ability of students to learn. The final testing allowed us to assess the level of mastering the material by students.
In the course of analyzing the study results, the following conclusions were obtained: -the absence (with a 95% probability) of a statistically significant difference between the level of material development during the educational process when using business simulators and when using traditional forms in the form of solving problems and cases, despite the fact that the differences in the results of the final test values of the two subgroups for different analysis options were about 20%; -significantly increased interest in classes on the part of students when using the business simulator; -students' subjective perception of classes that used the business simulator as more effective, which better contribute to the assimilation of the material being studied.