Research on prediction of critical decision-making behavior in online examination

. Starting with the study of the positive and negative factors affecting candidates' decision-making in the critical answer period of online examination, this paper preliminary establishes the prediction model of critical answer decision-making behavior of online examination. Taking the computer-based examination practice of academic level examination in a domestic city as an example, this paper makes an empirical study on the prediction model of critical answer decision-making behavior through data analysis and questionnaire survey, and verifies the relevant assumptions.


Introduction
With the development and popularization of the Internet, online examination is more and more widely used. A large number of qualification examinations and academic level examinations are realized in the form of computer examination. Generally speaking, the links and processes of online examination include: login, understanding the rules, topic selection, reading the stem of the question, seeing the answer option, analyzing and thinking, submitting the answer and other steps. We call the critical answer making period (state) of online examination after the examinee selects the question and completes reading the option under time constraints. In the critical answer making period, candidates' decision-making activities are affected by many factors, including the relevant information factors of question stems and options in the online examination system and the influencing factors of candidates' relevant knowledge points or abilities. Based on the background data of the computer-based academic proficiency test in a domestic city, we drew the examiner's examination behavior model graph (Examination Behavior Model Graph, EBMG) of the online test, as shown in Figure 1: It can be clearly seen from Figure 1 that the probability of choosing from the previous question to the next question is only 65%, while 15% of the candidates go back to see the answer options, and 20% of the candidates go back to see the question. In other words, in the critical answer period, 35% of the candidates did not answer directly or did not submit directly. From critical answer to option submission, what factors affect candidates' decision-making?
Different from the traditional item response theory (IRT), the purpose of this paper is to obtain the prediction model of candidates' decision-making behavior in the critical answer

Overview of existing relevant research methods
The existing research methods of critical decision-making with strong time constraints mainly include three aspects: 1) Research from the perspective of behavioral science Fishbein and Ajzen [2] first proposed that behavior intention is the main antecedent variable of behavior decision-making; Svenson O and edland A [3] gave an effective model for selection and judgment under the background of time pressure. They believe that human behavior in critical decision-making will be significantly affected by time and pressure; Steel and Ovalle (1986) reached the same conclusion by using the method of Meta-analysis.
2) Research from the perspective of value (Utility) Researchers believe that the key to the correctness of critical decision-making lies in the object's evaluation of the decision-making target value (utility). If the object believes that the decision-making target value (utility) increases, the critical decision tends to be made quickly, otherwise it is the opposite. In terms of model research, Dwyer F. R [4] proposed a value based decision model, and Payne J W [5] analyzed the changes of critical decision from the perspective of long-term value.
3) From the perspective of Human-Computer Interaction Psychology (HCIP) The time boundary and the examiner's test pressure will certainly affect the examiner's critical decision-making behavior, thus affecting the examiner's submission probability of answer choice in the online test. The relevant theories of human-computer interaction psychology also give the different time constraints and examination environmental pressure brought by traditional examination and online examination from another perspective. The relevant response models based on these theories are helpful to deduce the behavior model of critical decision-making in online examination. The basic idea is that variables such as the level of examinee, the difficulty of investigating the test questions, time constraints and the importance of the test itself lead to the prolongation of the critical decision-making time, which eventually leads to the re-inspection behavior. Relevant models in this regard include Benson l III. Svenson O. (1993) [6] Model, Weenig M W H, Maarleveld M. Model (2002) [7] , Rieskamp J, Hoffrage U. (2007) Policy Model [8] , etc.
The domestic research on the critical decision-making behavior of online examination is still in the primary stage, and the research on the model of critical decision-making is even less. In the research on the behavior of examinee in traditional examinations, Zhang Bei (2013) Analysis of the causes of college students cheating in examinations. From the impact of the social environment, college education management deviation, students to improve the self-management of three aspects of detailed analysis [9] . Liu Jing (2014) studied the identification of abnormal behaviors in Chinese examinations and proposed an interface model for identifying abnormal behaviors of examinee based on Kinect [10] ; In terms of online examination, Hu Senbo (2019) conducted in-depth research on the detection method of online examination cheating based on video in his master's thesis, and proposed relevant methods and models [11] . Fan Zijian etc. (2018) [12] studied the online examination abnormal behavior monitoring based on multi-dimensional somatosensory information, and judged whether the examiner's behavior was abnormal by analyzing the frequency and duration of abnormal events in a time window. Generally speaking, their research has not established a relatively complete path model, and there is less analysis of critical decisionmaking.
Based on the above relevant research methods, we intend to verify the critical decision prediction model through empirical methods on the basis of relevant assumptions.

Research objectives
Based on the online data of academic level examination in a city in China, this paper conducts an electronic questionnaire survey on the candidates with significant critical answer making period through mobile phone short message or telephone, in order to study the impact of various information or influencing factors on the decision-making behavior of the candidates in the critical answer making period, and establishes a prediction model of critical answer making behavior of online examination through empirical research, It provides a theoretical basis for the system management of online examination and related proposition management.

Structure of prediction model
The model involves two concepts: one is the confirmation behavior of making answer, and the other is the critical decision-making of making answer.
The so-called online examination answering behavior is actually the process of information interaction between candidates on the network through mouse, keyboard and other digital sound or image input devices. It is the embodiment of candidates' thoughts in the online examination network. The behavior in the online examination network is affected by many factors. As a kind of online examination network behavior, answering confirmation behavior is also affected by many factors. For example, in the example of computer-based academic level examination in this paper, answering confirmation behavior is affected by students' learning level, difficulty of test questions, complexity of test presentation methods, examination environment, training, operation proficiency, software interface, examination duration, input method The complexity of submitting answer results, candidates' review time, hardware and network stability, the ease of use of drawing tools and formula editors, examination rules, candidates' physical condition, candidates' computer-based examination experience and omissions. The decision-making process in the critical response behavior is a complex process. It is a comparison between the score and the focus and effort expenditure of the examinee under the condition of time constraints. Because the examinee will always be affected by all or part of various information, in fact, the final decision-making result is the comprehensive result of the examiner's subjective judgment and intuitive feeling, In essence, it is a psychological process. In view of this, the establishment of critical answering decisionmaking behavior prediction model can also refer to Slovic's psychometric model [13] , as shown in Figure 2. It is proposed to analyze the critical answering decision-making behavior prediction model from the following three aspects: 1) Relevant information: This is the data source for candidates to make critical answer making decisions. According to the impact of information on answer making behavior, information can be divided into positive information and negative information, such as students' learning level, training, operation proficiency, hardware and network stability, ease of use of drawing tools and formula editors, examination environment The examiner's computer test experience is included in the positive information; The difficulty of test questions, test rules, complexity of test question presentation, input method, complexity of submission of answer results, omission and negligence are included in negative factors.
2) Decision making behavior: candidates' decisions on critical answering behavior are mainly based on the timeliness and integrity expectation of answering. In the twodimensional space composed of these two factors, any critical answering decision has a corresponding position, which can reflect the guidance of critical answering behavior.
3) Decision making indicators of critical answering behavior: including the usual test scores, review preparation, difficulty of test questions, answering methods, input method settings, test duration, test importance, test substitutability, remaining make-up times, test site environment, new question types, test presentation methods, cheating possibility, candidate credit level, computer-based test adaptability, omission and negligence, etc.

Research hypothesis
Hypothesis 1: In the critical answer making period, negative information will accelerate the candidates to cancel the answer making choice submission behavior, and even cause abnormal operation; The positive information will promote the final completion of the selection and submission of answers, and the smooth progress of the online examination and the improvement of candidates' scores.
Hypothesis 2: The factors of examiner's learning level and the difficulty of the question are the key factors affecting the success of the answer making choice submission behavior in the critical answer making decision prediction model. Hypothesis 3: In the critical answer making period, candidates will use, including the usual test results, review preparation, difficulty of test questions, answer making methods, input method settings, test duration, test importance, test substitutability, remaining makeup times, test site environment, new question type, test question presentation method, cheating possibility, candidate credit level, computer test adaptability Omission, negligence and other effective decision-making indicators to make critical answer decision.

Time and scope
The data analysis lasted three months. Firstly, 50000 answer process data of candidates with critical answer behavior were exported from the background database in the online examination system. The pause time of 20 seconds in multiple-choice questions was taken as the boundary of the critical answer decision-making period. It was confirmed that 26368 answer records were not submitted immediately, and there may be decision-making information in the critical answer period, Among them, 5690 pieces of answer information are finally blank. Through the analysis of the answer data of candidates in the critical answer decision-making period, it is found that there are 2235 pieces of repeated information (the same topic question, the same item question and the same candidate ID). After removing the repeated critical answer data, 24133 records are compared with the candidate ID data, and they are divided into three groups, The first group is fresh students and the first group to take the computer test (1218 items in total), the second group is former students and the first group to take the computer test (6539 items in total), and the third group is former students but not the first group to take the computer test (16376 items in total). We then use the stratified random sampling method to randomly select 1000 items from the first group and 2000 items from the second group 3000 items were randomly selected from the third group and 1000 items were randomly selected from the blank group. A total of 7000 database records were taken as our research population (initial sample), and then the data were collected in two steps through the primary key content after the combination of item questions, topic questions, candidate ID and other information. The first step is to directly analyze the background data. The second step is to obtain the contact information of candidates through the student status system, and then obtain the contact information of candidates such as QQ, Wechat or E-mail through telephone, and then send the decision questionnaire of critical answer period to the effective information to obtain the relevant data of candidates' decisions during the critical answer period.

Questionnaire
It mainly includes three parts: 1) Investigation on the internal information acquisition of online examination: it investigates the acquisition and response of examinee households to the information provided by the questions, and mainly designs three types of questions, that is, the evaluation data of examiners on the existing information provided by the test question web page or page; Candidates want other information provided on the test question presentation page; Among the above information, there are 31 top 10 ranking information that affect the critical response decision, which are measured by Likert 5-point scale.
2) Critical answer making decision evaluation questionnaire survey: To investigate the decision-making process of candidates during the critical answer making period, two measurement indicators of timeliness and integrity are introduced, including the usual test results, review preparation, difficulty of test questions, answer making methods, input method settings, test duration, test importance, test substitutability, remaining make-up times, test site and test environment Likert 5-point scale is used to measure whether there are new question types, presentation methods of test questions, possibility of cheating, candidates' credit level, adaptability of computer-based test, omission and negligence, etc.
3) Information investigation on the external influence of online examination on decision-making: it mainly investigates the content channels for candidates to obtain relevant information outside the online examination system and the ranking of factors affecting critical answer making decision in these information, as well as the investigation of relevant personal information, with a total of 24 items, which are measured by Likert 5point scale.

Formal survey sample
There are 2623 officially returned statistical electronic questionnaire texts, because each person may be involved in the critical decision-making of multiple questions, and the final effective sample is 2321 after removing the wrong information and repeated information. Relevant information is shown in Table 1:

Statistical analysis method
It includes three parts: system data and questionnaire data processing, factor analysis and structural equation modeling.
1) System data and questionnaire data processing Firstly, the coding table is established according to the system data and questionnaire questions and answers, the returned data information of the questionnaire information is coded, and the corresponding questionnaire is established in EpiData software, the data file is set, the field control is added, then the original data of the questionnaire is input, and the validity and continuity of the data are checked by using the inspection module of the software, Finally, the data is transferred out to the data file for the call of SPSS analysis software.
2) Factor analysis At present, the main factor analysis methods include principal component analysis and common factor analysis. Principal component analysis is to linearly combine variables into a group of independent new variables, and make the new variables have the ability to explain the maximum deviation; Common factor analysis is to find out the hidden common factors from a group of variables. In this paper, the principal component analysis method is used for factor analysis, and the factor analysis data table shown in Table 2 is established. The main steps are as follows: Table 2 data sheet of factor analysis Step 1: in the case of parameter standardization, import the questionnaire data processed by EpiData software into the data table to obtain its covariance matrix (Correlation Coefficient Matrix), which reflects the correlation between research variables.
Step 2: extract new variables (Factors) based on the covariance matrix, and seek a group of factors that are small in number and can reflect the overall variance and are not related to each other, that is, the linear combination of the original variables. For the population with K variables, X = { 1 2 k x ,x , ,x  }, for example, find the linear combination of variables with the largest variance Z1 is called the main component 1, and then the deviation caused by the main component 1 is removed, and then the optimal linear combination of variables independent of the linearity of Z1 is found to obtain the main component 2, and then the main components 3, 4, 5, 6, etc. are obtained by analogy.
Because there are many variables set in this paper, SPSS software is used to analyze and calculate the 55 items that affect the trading behavior of online customers in critical state. The combination of orthogonal and skew rotation is used to obtain five factors, and the common variance, eigenvalue and cumulative interpretable proportion of these factors are obtained through their factor load matrix, The software calculation results show that the cumulative interpretable proportion of these five factors reaches 87.32%. These five factors are: A) Examiner's individual factors: including 14 items, including examiner's learning ability, examiner's middle school, type of middle school, examiner's identity, examiner's gender, examiner's age, examiner's eyesight, examiner's physical condition, examiner's family economic condition, examiner's entrance goal, examiner's usual performance, examiner's review preparation, examiner's credit level, examinee careful degree and so on.
B) Test question related information: including 10 items, such as test question type (picture and text, table, multiple-choice, single choice, etc.), test question difficulty, picture and text ratio, test paper structure, test question quantity, score distribution, font size, font type, knowledge point coverage, new question type or not. C) Human computer interaction factors: including the presentation form of test questions, font or graphic format, answer method (Simple multiple-choice questions, Drawing questions, Keyboard complex input questions, etc.), page turning convenience, input method setting convenience, screen presentation and paper presentation difference. D) Examination environment factors: including 8 items, including the examination site environment, invigilator mode, invigilator, examination room rules, time reminder, camera distribution, examination room interference, possibility of discipline violation and cheating. E) Examination attribute factors: including examination grade (top secret, secret, secret), examination time, examination duration, examination method, examination importance, examination passing rate, examination substitutability, whether it can be made up and the number of make-up examinations allowed, the number of remaining make-up examinations, and whether it is linked to graduation or promotion.
In addition, it also includes some other factors, such as weather factors and candidates' emotional factors. Due to the low factor load, it will not be considered separately here.
3) Structural equation modeling Structural equation modeling (SEM) is a statistical data analysis tool formed by the comprehensive use of multiple regression analysis, path analysis and confirmatory factor analysis. SEM includes four variables: exogenous explicit variable, endogenous explicit variable, exogenous latent variable and endogenous latent variable. Its model also includes measurement model and structural model.
In the analysis of the measurement model, according to the results of the above factor analysis, we take five factors as the latent variables of the model: individual factors, test question related information, human-computer interaction factors, test environment factors and test attribute factors, and take the timeliness and integrity in the critical answer making decision-making behavior as the intermediate variables (endogenous explicit variables). The decision-making variables include the examiner's usual performance, the examiner's review preparation, the difficulty of the test question, whether the new question type, the presentation method of the test question, the way of answering, the convenience of setting the input method, the length of the test, the importance of the test, the substitutability of the test, the number of remaining make-up tests, the environment of the test site, and the possibility of discipline violation and cheating, Using orthogonal rotation, it can be integrated into two types of factors: positive factors (examiner's usual performance, examiner's review preparation, test question presentation, answer method, convenience of input method setting, test duration, test importance, test substitutability, test site environment) and negative factors (test question difficulty, new question type, possibility of discipline violation and cheating, and remaining make-up test times), The overall interpretation rate was 59.22%. On the basis of Figure 2, we establish a conceptual model for prediction and analysis of critical answering behavior, as shown in Figure 3, Based on the conceptual model, we use Amos21 software for further analysis, due to Amos21 has the function of automatic detection of structural model. On the basis of establishing the path map of conceptual model, we use this function to automatically detect and optimize the structural equation model after importing data. The prediction model between dependent variables and independent variables is shown in Figure 4. In Figure 4, we add the correlation between individual factor F1 and negative expected index E1, It is mainly considered that the learning ability and usual performance in individual objective factors have a direct impact on the candidates' critical answer decision. At the same time, the results of data analysis also show that the correlation between F1 and E1 can increase the fitting degree of data. It can be seen from Figure 4 that different factors have different effects on the critical decision-making. Among them, the factors of test questions, individual candidates and human-computer interaction have the greatest impact on the critical decision-making. The more difficult and complex the test questions are, the longer the critical decision-making time is, and the more complex and complete the decision-making is; The better the individual learning ability of the examinee, the higher the usual score, and the shorter the critical time to make answer decisions. In addition to individual factors and test attribute factors, other factors have a negative impact on the timeliness and integrity of critical response decision-making. From the perspective of online examination development, in order to better promote online examination, we can start from at least the following three aspects:

Submit
1) Continue to expand the way of converting papers to electronic test papers, so that more question types can be presented more scientifically and optimized on the computer screen. At the same time, actively explore new question types and expand new investigation methods, so that candidates can minimize the impact of factors other than the test questions on the critical decision-making of the examination.
2) Improve the reminder function of the computer test system to help candidates answer reasonably and scientifically, so as to avoid omission and neglect.
3) The computer-based examination system is developed and designed with a safe, reliable and user-friendly human-computer interface, and the HCIP theory is fully applied to optimize the layout and design of relevant modules of online examination, so as to reduce the impact of other content on candidates' critical decision-making.

Hypothesis verification and data interpretation
The prediction model shown in Figure 4 basically verifies hypothesis 1. However, since the factor classification method in Figure 4 is only one of many classification methods with high interpretation rate, and there are also positive and negative factors within the factor category, the influence of the factor category may vary with different combinations of factors.
For hypothesis 2, the model in Figure 4 realizes partial verification, because the test question factors and examinee individual factors include the examiner's learning level factors and the difficulty factors of the questions, which are the main factors affecting the critical answer making decision.
The model in Figure 4 also partially verifies hypothesis 3. The data show that the overall interpretation rate of the optimized index reaches 59.22%.
In addition, through the analysis of variance of the above five factors in different decision-making subjects (middle school type, age group and gender), it is found that these five factors have significant differences in the impact of different subjects. In terms of the type of middle school, the better the examiner's middle school, the smaller the influence of individual factors on decision-making, and the greater the influence of test question factors and human-computer interaction factors on decision-making, which shows that the higher the examiner's quality, the more sensitive he is to test question factors; In terms of the age group of examiners, the decision-making behavior of examiners under the age of 16 is most affected by individual objective factors, learning ability and human-computer interaction. This may be due to the fact that this age group is mainly low-grade students or technical secondary school and higher vocational students, and has less contact with standardized Online examinations; The influencing factors of decision-making in the age group of 17-24 are relatively scattered. This age group is basically senior three or previous students, and has strong adaptability to the number of online examinations and computer-based examinations. On the other hand, in the critical decision-making, the influence of boys is more significant than that of girls, and the dispersion of boys to the five factors is significantly higher than that of girls. In terms of middle school distribution, in terms of the influencing factors of critical decision-making, the coefficient of test question factors should be significantly higher than other factors, and also significantly higher than the relevant data of examiners of other types of middle schools. In terms of human-computer interaction, the ranking of former students is higher than that of new students; In terms of examination environment factors, boys rank higher than girls, which shows that boys are less sensitive to the examination environment than girls. In terms of test attribute factors, candidates in municipal key middle schools and older candidates' critical decision-making ranking is significantly higher than other candidates.

Conclusion
Using the method of stratified random sampling analysis and investigation, this paper studies the relevant data of 2321 candidates' critical decision-making behavior in the computer-based examination of academic level in a city in China. Through the analysis of the data, this paper puts forward a prediction model of critical decision-making behavior in online examination. In the empirical study of the model, we find that: 1) The difficulty of test questions has an important impact on the critical decisionmaking of online examination; 2) The friendly human-computer interaction of the online examination system also plays a good role in promoting the candidates' critical decision-making; 3) Examiner's individual factors also have an important impact on the critical decisionmaking of online examinee. We should strive to improve the examiner's learning ability and usual performance.