Development of diagnostic procedures and tools for assessing information culture and competencies of cadets

In modern conditions of digitalization of society, the presence of information competencies and information culture is the key to successful professional activity in all areas, including in the military sphere. The studies carried out in recent decades indicate the need and possibility of the formation and development of information communication and information technology skills in the process of integrated development of computer science, humanitarian, socioeconomic, general professional and special disciplines. At the same time, the methods for assessing the level of information culture formation as an integral personal education are not sufficiently developed. In this regard, it is necessary to study the existing methods, to develop diagnostic procedures and tools for assessing the information and technological competencies of military educational organizations cadets, as well as to test statistical hypotheses. The concept of the information support system for assessing the level of competence formation and the intermediate results of a large-scale experiment on the development of cadets' information culture are presented.


Introduction
Numerous studies [1][2][3][4][5], including military teachers' ones, confirm the significant role of information culture of students in the professional development of specialists in various fields of activity, information security and social adaptation of the individual in the conditions of computerization and digitalization of the modern information society. The conducted studies allow us to determine the information culture of the military engineering specialty cadet "as a part of the general culture, which is an integrative professional and personal quality, characterizing his ability and readiness for information activities aimed at meeting information needs using traditional, information and communication technologies, and presented the unity of cognitive (universal competences, including system and critical thinking, knowledge and ideas about the informational picture of the world), communicative (universal competencies, including the ability to comply with the principles and rules of personal behavior in the information society, the use of communication technologies), operational-content (general professional and professional competencies, including the ability to apply information and communication technologies, practical skills and skills related to information processes of military professional activity), value-reflective (information worldview, including life attitudes, assessments and attitude to the information society) components" [6]. Thus, information culture can be considered as a set of specialist's key competencies. According to the researchers, "key competencies are the ability of a learner to act independently in uncertain situations when solving problems that are urgent for them. Information competence includes search, sorting, reproduction, storage, effective use of media sources, ensuring their safety, as well as the development of media culture" [7].
In the scientific and pedagogical literature, various terms are used to characterize and assess information skills. It is emphasized that "digital competencies, computer skills, information literacy and related abilities are a critical element in ICT (information and communication technologies) education" [8]. At the same time, it is noted that the analysis of the Web of Science (WOS) and Scopus databases shows that the main terms defining these clusters prevail in different periods: "Computer" -these are earlier terms, followed by "Information", and now -"Digital". In this study, we adhere to the terms "information culture" and "information competencies", which, in our opinion, most accurately reflect the essence of the phenomena in question.
In order to substantiate the effectiveness of the methodological system ( Fig. 1) during the pedagogical experiment, the following steps are necessary. First to assess the current state of the cadets' information culture level in control and experimental groups. Then to identify changes resulted from the realization of pedagogical conditions of information culture development, that influence the cadets in the information and educational environment of military educational organization. The level of information culture is understood as the degree of quantitative and qualitative social and professional acquisitions (knowledge, abilities, skills, competencies, modus operandi and behavioral patterns) that determine the effectiveness of a cadet's information activity. A qualitative measure of information culture is the level of its development. Dynamics is an aggregate indicator showing the change in the state (level) of information culture development.

Methods for the qualitative and quantitative assessment of information culture
When developing methods of qualitative assessment, many studies were considered [9][10][11] and dozens of types of questionnaires, designed to measure the level of information culture and competencies of schoolchildren, students, managers, teachers and other categories of specialists, were studied. The researchers refer to the main criteria of information culture as the ability to formulate the need for information, to find the necessary information from various sources, to select and to evaluate information, to analyze information, to create new information, to maintain catalogs and card files. Though these skills, acquired by students in libraries and in the school informatics lessons, are undoubtedly the basis of information culture, they are not sufficient for specialists with higher engineering education, especially Armed Forces specialists. In order to determine the level of information competence, information and legal culture, as well as the ethics of information interaction of future military specialists in automated special-purpose systems, a questionnaire was developed by the authors. This questionnaire comprises the groups of questions related to: general concepts of computer science and information technology; the field of data search and selection in bibliographic sources and of information dissemination on the Internet; data structuring, storage and protection; data processing and transmission. In [12] it is noted that along with the advantages, the questionnaire method also has the significant disadvantages, such as "the subjectivity of respondents when answering the questionnaire; the subjectivity of the researcher when drawing up the questionnaire itself; uncertainty of the representative audience of respondents; the random nature of the answers to the questionnaire and the possible uncertainty of their interpretation" [12], etc.
For a quantitative assessment of the level of development of the cadets information culture at the ascertaining stage, the assessments of the school certificates in the disciplines "Informatics" and "Russian language" were taken into account. At the control stage, the list was determined of twenty-six disciplines aimed at the formation of universal, general professional and professional ICT&IT competencies. These disciplines include ones of the humanitarian and social-economic cycle, of natural science, as well as of the general professional and special blocks.
As a result of the theoretical analysis [13], the following criteria for the levels of development of the future military engineers' information culture were identified: 1) «the degree of formation of the motivation for information activity» [13]; 2) «the degree of formation of cognitive and communicative competencies» [13]; 3) «the degree of formation of IT competencies» [13]; 4) «the degree of formation of value attitudes» [13]; The degree of formation of the motivation for information activity is expressed by «a conscious responsible attitude to information activity» [13].
The degree of formation of cognitive and communicative competencies is expressed by the fallowing abilities: «to carry out a critical analysis of problem situations based on a systematic approach; to develop an action strategy; to use modern communication technologies, including in a foreign language, for academic and professional interaction; to organize and manage team work, to develop team strategy to achieve a goal» [13].
The degree of formation of IT competencies is expressed by the following abilities: «to understand the essence and meaning of information; to be aware of dangers and threats; to comply with the basic requirements of information security and protection of state secrets; to use modern software and tools for computer modeling to solve various research and professional problems; to master the basic methods and means of obtaining, storing, processing information; computer skills related to professional activity» [13]; The degree of formation of value attitudes is expressed by the following abilities: «to determine and implement the priorities of one's own activity and ways to improve it on the basis of self-esteem and education throughout life; to master the culture and logic of thinking, generalization, analysis, systematization, forecasting, setting research tasks for professional activities and choosing ways to achieve them; to show respect for the value of information, copyright, security issues, verification of the accuracy of information» [13].
The scale of cadets information culture development in terms of content is represented by the low, medium and high levels. The low level is characterized by «the lack of formation» [13]. The middle level means «the partial formation of the information culture components» [13]. The high level is characterized by «the formed high motivation to study humanitarian, socio-economic and IT disciplines, deep knowledge and confident skills in information activities, readiness to using ICT in solving professional problems, to improvement and self-education in the information society formation environment» [13].

Diagnostic procedures and tools for assessing information culture and competencies
One of the key problems in identifying the level of information culture is the development of diagnostic procedures and tools for assessing students' information competencies. In this regard, a pedagogical experiment is currently being carried out at the Department of Automation (and Computing) of the Yaroslavl Higher Military School of Air Defense. Within the framework of this experiment a zero statistical hypothesis was put forward that the control and experimental groups can be considered as homogeneous. The initial level of information culture was determined on the basis of samples obtained on the basis of the cadets' certificate points in computer science and the Russian language and the author's questionnaire. The questionnaire includes 26 questions divided into eight categories, with three answer options, estimated from 0 to 2 points. Consequently, the total number of points in the questionnaire varied from 0 to 52. The interval from 0 to 52 points was divided into three intervals from 0 to 17 points, from 18 to 34 points and from 35 to 52 points, corresponding to low, medium and high levels of information culture development. Statistical testing of the hypothesis was carried out according to Pearson's χ2 test of agreement at the level of statistical significance α = 0.05. As a result, we obtained empirical values of Pearson's χ2 statistics (1.05; 4.65; 0.67) less than 2cr, = 5.99, which allowed us to accept the zero statistical hypothesis as plausible.
As a result of the ascertaining stage of the pedagogical experiment, an insufficient level of information culture among cadets of control and experimental groups, as well as problematic issues in the process of cadets' information culture developing, were revealed. The conclusions based on the results of the ascertaining stage served as the basis for planning, preparing and conducting the formative stage of the pedagogical experiment. At this stage, classes with cadets enrolled in programs of the specialty "Application and operation of automated systems for special purposes" were carried out in accordance with the developed model of the methodological system for the formation of information culture of engineering specialties cadets. Some of the classes were conducted using electronic interdisciplinary educational and methodological complexes according to the developed teaching methods of the disciplines aimed at the formation of information culture. These disciplines include methods and technologies for critical thinking development, ICT and project method. In the educational process, the current, midterm and final control of the cadets' knowledge and skills was carried out for each module of academic disciplines. The author's system of information support for assessing the level of competence formation was used [14]. These system is based on the following principles: the tests on previous disciplines' individual topics are repeated when studying subsequent interrelated disciplines; the history of test results of each student is stored in an information database; analytical reports reflect the level of competence formation and problematic topics and serve as the basis for developing individual corrective actions. In order to implement the system, a relational database was developed, the logical model of which in the IDEF1X notation is shown in Fig. 2. The database organization allows one to accumulate information to track the dynamics of current academic performance and the level of cadets achievement of competencies during the study of interrelated disciplines. It also allows to interact with users through an application that is an integral part of the information support system» [16].
In order to test the methodology, the results of intermediate certification of control and experimental groups were analyzed in the context of the formation of one of the competencies -the ability to use programming languages and systems, general-purpose software, computer modeling tools for solving various research and professional problems. A statistical hypothesis was put forward that the control and experimental groups can be considered as heterogeneous in terms of the formed level of IT competencies. Statistical testing of the hypothesis was carried out on the samples obtained according to the estimates of the cadets' intermediate attestations in computer science and programming. The Pearson χ2 criterion of agreement at the level of statistical significance α = 0.05 was used.
The data obtained shows ( Table 1) that initially the cadets of the experimental groups had lower grades (discipline "Informatics", certificate). As the study of information technology disciplines ("Programming", 3-4 semesters) in the experimental groups there is a tendency to increase the average score compared to the control groups (Table 2). The values of Pearson's χ2 statistics also have an upward trend (0.19; 1.57; 1.63; 1.92). However, the differences in the levels of training do not exceed χ2cr, which indicates the need to continue the formative experiment.

Conclusions
The complexity of assessing the dynamics of cadets information culture development is associated with the experiment scale. The duration of the experiment is more than six years. The experiment envelopes training materials of 26 academic courses aimed at the formation of cadets information culture. The control and experimental groups belong to different years of recruitment. Information culture formation occurs throughout the entire period of the study and is closely related to the formation of universal, general professional and professional competencies. The methods of measuring the achievement of these competencies are discussed by many researchers, but have not been sufficiently developed yet.