Multilayered balanced indicators system for evaluating the results of innovation activities within the framework of cooperation

. A modified system of indicators for assessing the innovation activities effectiveness in the region based on data on patent and licensing activities in the framework of scientific and industrial cooperation is described. Also in the article we presented a general mechanism for innovation process effectiveness managing of such cooperative interaction. They are intended to be used as tools to support decision-making, allowing one to coordinate the cooperative indicators management. The proposed system of indicators provides transfer and decomposition of goals for planning operational activities by levels and actors of the innovation process in the cooperative formation, as well as monitoring their achievement. Taking into account the peculiarities of the innovation environment, the proposed balanced indicators system (hereinafter referred to as the BSC) focuses on non-financial logistics performance indicators and patent`s information. The structure of the BSC assumes a pyramid-layer structure, the number of elements of each layer is determined by the structural decomposition of the cooperative interaction corresponding level. The article describes stages of forming such BSC-maps process. As a tool for improving the balance of strategic maps for generating management decisions, a transition to "horizontal dynamics" is proposed, according to which at each level of logistics management, among all goals there is the greatest deviation from the target values, i.e., from the target values achieved to the least extent. These goals are the "target limitations" of the enterprise effective operation and they take greatest importance for further analysis of critical situations. For this purpose, a procedure for managing innovation environment by deviations is proposed.


Introduction
In the modern literature, a lot of attention is paid to the analysis and evaluation of the innovative processes' implementation at the regional level.There are a large number of different approaches, models and methods of its assessment.Such close attention to the analysis and evaluation of innovation activity results on the part of Russian and foreign scientists is due to its essential importance for making operational and, above all, strategic decisions.In addition, technological progress, the growth of dynamism and environmental changes uncertainty, with the increasing globality of competition, significantly shorten the life cycles of products and services and forcing production and economic entities to quickly implement innovative technologies in their work experience.The complexity of technological processes makes innovations more expensive and risky, which emphasizes the importance and significant contribution of the evaluation and analysis stage in the innovative management.The need for a clear, transparent and logical system for analysing and evaluating the results of such activities is also due to the significant role of regional authorities in their implementation, the significant contribution of budget financing and co-financing, which involve strict reporting and ensuring flexibility in the allocation of resources to budgets of various levels.The size of Russian territory and the presence of industrial locations specialization separated by large distances lead to an increase in logistics costs when implementing the processes of material and technical support for the production and sale of finished products.This circumstance determines the feasibility of forming closed supply chains localized in one territory, as well as an integrated innovation process [1].In addition, the climatic and resource conditions in each of the Russian regions differ quite significantly, which determines the need to implement an approach to concentrate efforts to catalyse integration processes within their localization territories and extract maximum benefits for them and the region.It should also be noted that the crisis consequences caused by measures to prevent the spread of coronavirus had a significant impact on national innovation systems, which caused a reduction in budgets for innovative development programs and even more clearly raised the question of tools for assessing innovation activities effectiveness for making management decisions in this area.On the other hand, it is noted in [2] that these circumstances only further emphasized the need for innovative and information development to ensure the sustainable development of society and world economies that were not ready for global challenges.However, most of the existing methods and tools for assessing innovation activity results practically do not take into account such aspects as inter-organizational interaction and the synergistic effects generated by such cooperative formations.At the same time, these formations become an integral element of modern production and economic systems and have a significant potential for innovation development [3,4].The level of such cooperation is most likely not taken into account for the existing objective problems in its analysis.In this case, it is assumed to analyse a significant number of qualitative characteristics, subjective and relative variables that have a probabilistic nature of implementation.This work is devoted to the development of tools for analysing innovation activity results in the region, which differ from others by using a multi-layer balanced indicators system for evaluating scientific and industrial cooperation based on data on patent and licensing activities.

Literature review and analysis
Among the statistical and information-analytical materials that reflect quantitative assessments of new forms strategic alliances, various partnerships and other cooperative relations implementation, we can distinguish the HSE collections.The indicators presented in them make it possible to identify and establish the existence of cooperative relations, research and development organization cooperation with other organizations, regardless of whether the organization is the customer or the performer of the relevant works and services.In particular, the following groups of indicators are presented: the underdevelopment of cooperative ties, technological partnership, the number of joint projects and in the context of innovations types, cooperative ties types, as well as by type of economic activity.It is worth noting that the methodological materials for the development of innovative development programs for Russian organizations and enterprises (approved by the decision of the Interdepartmental Working Group on the Implementation of Innovation Development Priorities of the Presidium of the Presidential Council for Russia Economic Modernization and Innovative Development (No. 1 of February 27, 2018)) include a section on assessing cooperation.In particular, this is interaction development analysis with third-party organizations, the implementation of "open innovation" principles, the development of partnership in education and science fields, the level of interaction with the subjects of innovation infrastructure in general, as well as the development of foreign economic activity and international cooperation in the innovation sphere.In [5], a comparative characteristic of the indicators of the analysis of innovative development, proposed by the European Commission, is given.Among all the statistical indicators that are most widely used in domestic and foreign practice and characterize business entities innovation activity and competitiveness, the following groups can be distinguished: cost, time and structural [6,7,8,9].Modern scientists also assess the effectiveness of innovative development, including financial development, at the microeconomic level, from the point of view new technologies development of by industrial enterprises, the introduction of new technologies into economic circulation, the functioning of research and development departments of innovative business entities, etc.On the other hand, there is no unified methodology for assessing the effectiveness of innovation financing at the regional level, and the actual assessment of innovation financing results effectiveness of investment is based on a system of monitoring indicators that characterize the state of Russia's scientific potential.The assessment of cooperation at the regional level is also reflected in the works of foreign and Russian scientists.So, Kuznetsova E. P. in [10] describes an approach to assessing the development of industrial cooperation in the region, based on the calculation of an integral indicator that reflects the "scale" of cooperation between corporate structures, small and medium-sized businesses and the "cost-effectiveness" of their interaction.The author defines the stages of assessing industrial cooperation in the region, and, in particular, at the first stage, identifies three groups of indicators: -indicators that characterize change in the cost parameters of production activities in the region; -indicators that characterize "efficiency" of production cooperation; -indicators that characterize "scale" of production cooperation.Aleshin A.V. [11] as the main areas of inter-organization interaction evaluation offers an assessment of the cooperation effectiveness both for individual business entities and for the regional economy as a whole, as well as a methodology for assessing the level of partnership relations development in regional business environment.A significant contribution to the theory of interorganizational interaction evaluation is taking into account the fact of participation in the innovation process of various scales subjects.However, the author considered only business subjects and this assessment is not applicable for research organizations, educational institutions, government agencies, etc., which have mainly a different range of interests.Fedosova T. V. and Morozova T. V. [12] investigated the form of interaction "university-enterprise", the proposed assessment methodology identifies three vectors describing the state of the process according to three corresponding ones, namely: the university development vector, enterprise development vector, mutual development vector.Their spatial intersections give an idea of the level of integral interaction.The advantage of this method is the ease of visual representation, but there is no direction of interaction with government agencies.Ivashchenko N. S. [13] offers an assessment of the partnership innovation process subjects relations depending on the stage of their life cycle and defines the following groups of criteria, evaluated on a 4-point scale: the level of partnerships organization (the nature of goals and objectives, motivation, etc.), the level of their development (reliability, dynamics, trust) and the level of such partnerships effectiveness (depth, durability, etc.).Measuring the effectiveness of network forms of organization is a fairly new direction in research [14,15].In [16], network modeling is used for evaluating interoperable networks.Tsiteladze D. D. [17] with regard to subjects' interaction features in the innovation process notes that it is largely determined by the nature and direction of transaction costs, while interaction is considered from the side of transaction costs of various subjects: business, state, population, expert community, financial institutions, etc. Interaction of subjects within clusters is reflected in the work of Stepnov I. M., Kovalchuk Yu. A., Gorchakova E. A. [18].The authors describe an original model for evaluating the effectiveness of intra-cluster interaction, which includes a specially developed criterion "the degree of cluster leverage".Simachev Yu.V., Kuzyk M. G., Feigina V. V. [19] note that large research organizations are more often involved in interaction with business for R&D, and non-state research organizations are more often involved in one-time cooperation with business entities.An important common factor in the presence of integration interaction, as well as further plans for this interaction, as the authors note, is a relatively high scientific potential.Quite often, coefficients and indicators groups that reflect patent activity are included in the methods for evaluating innovation activities results and determining the degree of security of an enterprise with intangible intellectual resources.Most enterprises in the annual reports on innovation activities provide results on such indicators as: the number of RID registration applications, the number of registered certificates, patents for an invention, utility model, and so on.Patents and computer programs registration certificates secure the right to the result of intellectual activity and thus are the final research stage of the innovation life cycle.At the same time, the uniqueness of patent data lies in the fact that they characterize both the effectiveness of research and development and the potential for innovation, i.e. they are a kind of bridge between science and production.Patents not only perform the function of legal protection, but also are a unique source of technological information, since the information contained in patents is usually not presented anywhere else.In addition, patenting, usually several years ahead of scientific and technological achievements introduction, allows to take into account the possibility of technological innovations in advance.It is also necessary to note the strengthening of trends towards "open" borders between the world's economies, and the practice of open innovation is becoming increasingly popular.And the more such borders are open, the more acute the issue of patent protection becomes.For analytical purposes, absolute and relative indicators of patent activity are used, the information for the calculation of which is publicly available and has an objective quantitative assessment, which can serve as an objective basis for evaluating the innovative activity results of cooperative interaction at the regional level.
Thus, the presented analysis allows us to say that the currently existing methods and tools for innovation activities effectiveness assessing in the region have a number of limitations, which makes it difficult to apply them for a comprehensive analysis of modern integrated entities characterized by multidirectional interests of participants and high dynamism of changes.

Results and Discussion
Building an effective system of economic entities cooperative interaction in the innovative processes implementation at the regional level is impossible without evaluating and analysing the effectiveness of interaction processes, taking into account the interests of all its participants.The practice of network organization forms strategic management shows that to solve this problem in the planning and control of the organization's activities, it is advisable to use a balanced system of indicators (BSC), proposed in the works of Norton D. and Kaplan R. and developed in the works of Vetter M., Leonard D. [20].The BSC for cooperative formations acts can be used as a tool to support management decision-making, allowing for coordinated management of indicators that characterize the activities of both individual entities and the group as a whole.At the same time, the existing methodological apparatus for the construction and application of the BSC does not fully take into account the balance of interests of all participants in the innovation regional process in various functional areas and at different levels of management.At the same time, there is also a shortage of mathematical expressions that allows one to control a balance degree of both individual indicators and groups of BSC indicators, taking into account their importance.In this regard, as a basis for managing interorganizational interaction in the process of implementing innovation activities, a multi-level modified BSC and its processing process are proposed, which provides for the transfer and decomposition of goals by levels and actors of the innovation process in the integration formation for planning operational activities, as well as monitoring their achievement.The structure of the proposed BSC has a pyramid-layer structure, the number of each layer elements is determined by the structural decomposition of the cooperative interaction corresponding level.In order to build the structure of the intra-company base level of such BSC, taking into account the features of innovative processes, it is proposed to focus on non-financial logistics performance indicators.The structure of the proposed matrix-hierarchical BSC is shown in Table 1.
It is possible to distinguish the following indicators that are part of BSC, taking into account the specifics of the cooperative innovative processes' implementation in the region.The system for evaluating innovation activity results in the region should be based on well-known indicators that characterize the main results of the functioning of and organizations.For example, it is proposed to include such indicators based on patent and licensing information as: coefficient for assessing the involvement of universities and research organizations in the patent and licensing activities of the industry, the indicator for assessing long-term interaction, the number of registered RID created within the framework of research contracts between an industrial enterprise and a university or research institute.It is also proposed to analyze the number of applications submitted for registration of RID, patents obtained and publications of a university or research institute, in publications included in the list of the Higher Attestation Commission and foreign bases, in the subject area of an industrial enterprise according to the IPC classifier.The calculation and analysis of these indicators will allow us to determine the presence, nature and level of scientific and industrial cooperation, which will thereby make it possible to significantly supplement the assessment of its innovative potential.To characterize the innovation component at the strategic planning level, it is proposed to use such indicators as the predicted ratio of the degree of applied novelty technologies in relation to their global counterparts, indicators of the long-term innovative projects' implementation, indicators of the energy and resource saving strategy implementation, etc.The process of forming BSC is proposed to be divided into the following stages: determination of control actions (within the framework of local innovation cycles); determination of external and internal innovation environment states, as well as the general economic state and development trajectory of the region; determination of diagnostic parameters that characterize the external manifestations of the innovation system and the effectiveness of its functioning; determination of structural parameters that characterize the internal state of the innovation system; establishing the relationship between the structural and diagnostic parameters of the system; determining the vertical structure of BSC.As a tool for improving BSC balance generating management decisions, a transition to "horizontal dynamics" is proposed, according to which at each level of logistics management, among all the goals, those that, as a result of the analysis of their indicators, have the greatest deviation from the target values, i.e., have been achieved to the least extent: these goals are the "target limitations" of enterprise effective activity and are of the greatest importance for further analysis of critical places.To do this, we propose a procedure for managing interorganizational interaction in the implementation of the innovation process in the region by deviations, consisting of the following steps: description of the desired state of cooperation innovation potential, i.e., the establishment of those qualitative and quantitative requirements for the state of the potential in all parameters that ensure the achievement of a specific goal and the implementation of the strategy; step-by-step establishment of the actual state of complex innovation environment in all parameters and the construction of the BSC; analysis of the discrepancy between the normative, target and actual values of the parameters, while for each BSC is proposed to calculate the imbalance separately using the formula of the form: where ǻi -local imbalances (normalized differences) between identical indicators in different categories of BSC maps (in the simplest case for parametric indicators between planned and actual, planned and normative, and actual and normative indicator i), knorm -weight coefficient of group normalization for indicators different categories, m -the number of layers of the global BSC map, n -the number of management indicators groups.This is followed by determining the total imbalance for individual contours, identifying the relationships between the unbalanced parameters and identifying possible causes of the discrepancy; defining a structured action plan for managing deviations.The methodology for applying this mechanism includes the following main steps.a) Using the results of external and internal cooperation environment analysis, strategic goals and corporate strategy of the participant are developed.The first line of the BSC is filled in.b) The BSC is being built.The degrees of importance (weight coefficients of group rationing) of the BSC indicators for different planning levels are determined.A preliminary control of imbalance values is carried out.c) Development of appropriate detailed strategic, medium-term and operational plans aimed at achieving the targets included in the BSC.In the event of an imbalance, we adjust the medium-term and operational plans according to the procedure proposed in paragraph b.This methodology allows us to take into account the degree of importance of individual indicators that are part of the proposed matrix-hierarchical BSC for different groups of subjects when implementing corporate governance procedures for regional innovation environment.The evaluation of innovation activity effectiveness of individual subjects can be carried out in the form of calculation and comparison of the obtained indicators with the established basic values.Depending on the current state of the region, the results achieved to support and develop its innovation component, the basic values can be indicators for the last or other reporting year, the average values for the country or the federal district, or the corresponding indicators of other regions.At the same time, data from the survey of the innovation infrastructure of other subjects' activity or foreign experience can be used as threshold criteria.These coefficients, calculated for subterritorial entities within the region (districts, cities, municipal units) on a territorial basis, also lay the basis for comparative comparisons.When calculating each of the above-mentioned indicators in the framework of comparative procedures or calculating dynamic series in a certain time interval, the result is a significant amount of quantitative economic parameters estimates of innovation financing in the region.Such a comprehensive analysis and the formation of evaluation indicators significant number on its basis make it possible to obtain, on the one hand, a multidimensional tool for evaluating the results of innovation activities, but, on the other hand, it makes it difficult to make decisions on determining the final dynamics of processes due to the significant amount of data being compared.Also the relevant one is the issue of comparative efficiency assessment based on territorial differences in the structure of the organization of the support infrastructure (economic and geographical analysis).It is based on a comparative analysis of innovative entrepreneurship development levels in regions/territorial entities with similar characteristics of socio-economic development, but different configurations of support infrastructure organizations.

Conclusion
Thus, calculation and analysis of indicators reflecting innovation activity results in the region according to the described methodology will allow us to determine the presence, nature and level of scientific and industrial cooperation, which will thereby make it possible to significantly supplement the assessment of its innovation potential.The breadth of the set of hierarchically arranged indicators delineating the zones, coupled with the possibility of transforming the described system of indicators for specific tasks, makes it possible to widely use integral indicators not only in the process of evaluating innovative potentials, but also in determining the key indicators of individual subjects functioning in the scientific and educational space.The use of such a tool will reduce the time for making decisions on the selection and implementation of innovative projects, reduce the level of uncertainty in the analysis and evaluation of scientific and industrial cooperation processes by using quantitative data available in the public domain and a more visual and simple system.And will also make it possible to predictively identify imbalances in the resulting economic indicators of innovation activities functioning in the region.

Table 1 .
The structure of the proposed BSC.