Digitalization of Global Economy: A Qualitative Study Exploring Key Indicators use to Measure Digital Progress in the Public Sector

Research background: The digitalization of the global economy is the most common phenomenon in the 21st century. Most Public sector organizations have already started their journeys towards digitalization, and many of them have dealt with their contemporary challenges with success. At the center of these transformations are metrics and indicators used for evaluating digital progress. Already existing measures focus on numerical measures of the presence of digital technology in the public sector (output measures), but do not evaluate the quality of the digitalized public sector (outcome measures). Purpose of the article: This paper attempts to evaluate metrics and indicators used for measuring the digital progress in the public sector. Methods: Three particular aspects of digitalization in the top five leading countries (Finland, Sweden, Denmark, Netherlands, and Malta) were examined: I. success rate of public sector IT projects (measured by how often projects are scrapped); II. The price comparability between the public sector and private sector IT projects; and III. The relative modernity of government IT systems (compared to private sector systems). Findings & Value added: The findings of this paper present key metrics and indicators that can be used to evaluate public sector digital progress. Policymakers will need to redefine digitalization goals and areas of investments, while researchers can contribute more insights to the individual impact of these metrics and indicators on the development of a digital public sector. To this end, the paper contributes to a better understanding of the essential metrics and indicators to measure digitalization progress in the public sector.


Introduction
A profound technological transformation is sweeping the world, impacting our work, and lives more than anything that has come before. New technologies that seemed unimaginable in the past are commonplace now and are playing an increasingly important role both in the public and private sectors. More and more organizations are incorporating connectivity, cloud, big data, Internet of Things (IoT), and artificial intelligence (AI) into their core processes and management systems, aiming to boost efficiency and competitiveness. These developments reflect some of the most creative and futuristic thinking in today's organizations.
Digitalization in the public sector is an urgent need for many governments around the world today [21,23]. Digital transformation is no longer just about technology; it requires a complete overhaul of how employees work, how we provide services for customers and citizens, and how departments within the public sector operate.
The expectation of benefits from digitalization sparks a superfluity of investment programs in public organizations and governments [8,24]. However, digitalization is not a sea free from navigational problems [1]. Organizations involved in digitalization face multiple issues related mainly to the: 1) Prioritization of investments (needs exceed available funding), and 2) Understanding the true value of digital (measurable results and clarity of business cases). A growing number of governments are claiming to "go digital" by incorporating digital technology into their internal administration and service delivery-but measure their efforts woefully imprecisely [1]. Less than 15% of companies can quantify the impact of their digital initiatives, according to McKinsey's Digital Quotient analysis [7].
For every government that is aiming to "digitalize", one can find a different and often competing definition of what that means. Consequently, measures of digitalization are often sporadic and poor [1].
Digital technologies increasingly place new demands and expectations on the public sector. Traditional KPIs are poor indicators of the effectiveness of ongoing digital efforts; they are best suited for measuring long-term impact, revealing improvements annually or quarterly. Existing indicators focus on numerical measures of the presence of digital technology in the public sector but do not evaluate the quality of digitalized government. Realizing the full potential of appropriate measuring indicators is key for the public sector's digitalization progress.

Methods For Assessing digitalization
There are many frameworks for assessing public sector digitalization progress and maturity, including efforts by the United Nations, World Bank, European Commission, Waseda University, The Economist Intelligence Unit, Brown University, and the Open Knowledge Network. Examples of these frameworks are listed in Table 1 below. Taking, for example, the UN Department of Economics and Social Affairs (UNDESA)'s E-government Development Index (EGDI) and the World Bank's Digital Adoption Index (DAI)-the most comprehensive measures currently available. They cover all countries, unlike other regional or sector-focused frameworks [16,18].
Another, deficiency of EGDI is that the data is not collected every year and in regular periods. This deficiency complicates the statistical analysis since the nature of the data is unbalanced panel data [15]. Study of [15] also, identify some deficiencies in EGDI. According to the studies the deficiencies can be summarized as: Mobile subscribers per 100 inhabitants should be dropped from EGDI, EGDI is biased toward developed countries that have better telecommunication infrastructure and more developed human capital, and EGDI is unbalanced panel data which complicates statistical analysis.

Other public-sector digitalization indexes
Apart from EGDI, there have been other efforts to measure the e-government development. However, none of these methodologies have the consistency and longevity of UN EGDI [1,3,19]. The Economist also has created an e-government ranking but barely for two years (2009 and 2010). This benchmark forms the e-government index based on six categories with different weights: Connectivity and technology infrastructure -20%, Business environment -15%, Social and cultural environment -15%, Legal environment -10%, Government policy and vision -15%, and Consumer and business adoption -25%. The limitations of this index are that it was measured only for two years and the methodology of creating the index is shallow [4,18].
Waseda-IAC International E-Government Ranking, the last e-government ranking that will be analyzed in this section. This e-government ranking is performed from Waseda University in cooperation with the International Academy of CIO (IAC). This survey is based on 9 major indicators and 33 sub-indicators in the public sector, as well as the relationship between governments and their stakeholders. They include: Network Preparedness; Management Optimization; Online Service; National Portal/Homepage; Government Chief Information Officer; E-Government Promotion; E-Participation/ Digital Inclusion; Open Government/Data and Cyber Security [16]. The 2014 survey base on this index includes 62 countries. This survey is another major effort to measure the egovernment maturity or rank countries based on the aforementioned criteria. The limitation of this survey is the number of countries ranked by this survey compared to UN EGDI [16].

Public sector IT projects
Public sector organizations have invested large sums in IT and their budgets continue to increase. Naturally, they are interested in knowing the return on these investments. Recent surveys suggest that as much as 10-15% of all software projects are stopped before they SHS Web of Conferences 9 2, 0 (2021) Globalization and its Socio-Economic Consequences 2020 deliver anything at all [5]. In 2019, the Swedish Project Review which is Sweden's leading index on project-related capabilities in collaboration between KPMG Project Advisory and PMI Sweden Chapter, provided benchmarks, points out trends and areas for improvement in public sector projects [16]. The survey shows that organizations in the public sector have more than 40 percent higher probability of failure compared to private organizations. This calls for a review of areas and indicators used in accessing or benchmarking public sector progress towards digitalization. European Union is heading towards a digital single market based on its Digital Single Market strategy accepted in 2015. European countries have started their national digitalization strategies from very different levels and the gap is considerable between frontrunners and challengers. According to the Digital Economy and Society Index (DESI) 2020 report [6], Finland, Sweden, Denmark, Netherlands, and Malta have been at the forefront of digital government development and implementation [6].
A careful analysis of the digital blueprint of these countries, for instance, suggests several key performance indicators by which digital policies and programs should be benchmarked and evaluated-these include and expand on the major indicators suggested so far by both policymakers and analyzed by Aaron M. [1]. The indicators suggested in this paper are believed by far the best representative measure of public sector digitalization progress.

Indicators for measuring public sector digitalization progress 4.1 Assessing online services
Citizens in all nations always expect the creation and delivery of high-quality services from their respective governments [2]. This expectation creates pressure for the government to perform. Public sector digitalization initiatives are results of such pressures by citizens combined with the capabilities achieved through the use of ICT.
Service Quality Measurement literature provides a basic understanding of the aspects of service quality measurement like 'what to measure' and 'how to measure.
There have been suggestions of five dimensions to measure service quality namely tangibility, assurance, responsiveness, reliability and the purchasing process [14]. These measures may include: the degree to which a website simplifies effective and active shopping, purchase and delivery; the total customer assessment regarding the service quality excellence and delivery in the virtual Marketplace; perceptions of the service along with recovery perceptions if problems occur; assessment of the service quality during online business transactions; the complete stages of a customer's interactions with the internet website.

Effectiveness of public sector IT systems
Research of [7] consider the "relative modernity" of public sector IT systems, compared to private sector adoptions. This includes the quality of back office systems, large databases, front office software, desktop systems, web-compatible systems, network speed and bandwidth, the development level of digitalized public sector services. The existence of legacy problems, recency of transition to fully Web-enabled networks, the pace of generating government websites, support to citizens in navigating e-bureaucracy, and adoption of up-to-date technical standards. According to [1]. Whether government IT systems are old or new, their capacity to deliver output and outcomes is key. Hence much attention should be placed on the "effectiveness" of a system. Effectiveness entails several aspects:  Effectiveness as a digital system: including the reliability of hardware, efficiency of software, and sufficiency of bandwidth.
 Effectiveness as a specific system: e.g., a digital tax system might focus on results like higher revenue collection, higher compliance levels, and lower evasion, reduced costs, increased audit efficiency, reduced occurrence of activities like money laundering, more expeditious license and identification, etc.; and  Dynamic effectiveness: e.g., through the use of technology-neutral approaches and regular reviews, to "future-proof" systems and mitigate against over-reliance on particular platforms. In some cases, there may be trade-offs in different dimensions of effectiveness. For instance, the customization of digital systems for government needs might also create structural "lock-in", which could impede future flexibility and system effectiveness [1].

Usability and usefulness of government digital platforms
Usability is a well-known and well-defined concept in human-computer interaction (HCI) research, referring to the extent to which the user and the computer can "communicate" clearly through the interface. Fernandez et al. [9,11] defined usability as the capability of the software product to be understood, learned, operated, and attractive to the users.
The usability and usefulness of government digital systems signify the difference between "white elephant" systems and those genuinely serving citizen needs. They are measured by the number of users of government digital platforms, the proportion of such users relative to the total number for a particular service (if the service has an "offline" delivery option), and user experiences of the systems (including time spent on particular functionalities and the ease of use of web services).

Security of government data
As concerns over individual privacy and collective data integrity rise, the overall security of data will be a key new dimension of any successful public sector digitalization. The Finnish Government published its first National Cyber Security Strategy as a Government Resolution in January 2013. The strategy was drafted by the Defence and Security Committee to showcase the preparedness of governments towards data protection.
People trust the government to protect the sensitive information it collects. Data breaches or other security failures damage this trust and expose governmental agencies to penalties and other sanctions. Even more importantly, stolen data can threaten the country's stability and security.
Heightened security standards and evolving threats have intensified the need for a sophisticated approach to data protection. This indicator measures the quality of governments' internal data governance processes, not just their citizen-facing delivery capacity. Observable dimensions include the existence of protocols for data governance (e.g., classification, extraction) [1,10]; the use of stewardship models, change-control mechanisms, enhanced encryption, increased identification, identity and rights management, secure ID, and multi-factor authentication, including biometrics; and how effectively a government uses platforms like cloud computing or hybrid cloud.

Timeliness of completion of Public sector IT projects
This measure indicates governments' capacity to translate an initial idea for a digital project into tangible outcomes within a reasonable timeframe. The planned time of completion is an essential measure of the success of the public sector IT project. It is likely to be influenced by the size of each project, as well as the presence of expertise able to anticipate and pre-empt potential obstacles early. Such expertise could be internal or outsourced to contractors with a direct link to internal decision-makers.

Data use in policy and organizational redesign
This measure indicates how government agencies themselves use the opportunities created by digitalization. This includes harnessing digitally collected and synthesized data to understand broad fiscal patterns and generate insights specific to companies and key citizen segments.
Potential responses to such data include restructuring an agency, crafting strategies or developing key performance indicators around citizen and business needs, rather than supply-driven factors (e.g., by function, sector or region).

Success and Price competitiveness of government digital projects
This measure refers to the money cost of government projects, compared with similar efforts in other governments and other sectors. It measures whether governments are overpaying or obtaining value for money in their expenditure. Price competitiveness is likely to be influenced by:  the presence of incentives for civil service leaders to secure defensible prices (e.g., "value for money" audits or requirements for smart commissioning);  the occurrence of forced outsourcing (creating a closed and concentrated market);  the nature of relationships between firms and governments;  firms specifying contractual terms in ways that protect them from future market changes; and  Long-term contracts that lead to only incumbent firms being able to deliver midcontract modifications. Following [7,11,12] a digital system's success can be measured inversely, by the number of projects scrapped. This indicator matters because of the loss of investments involved during such write-offs. Project scrappage is likely to be affected by factors including:  size, scale and specificity (modularity) of projects;  existence of rigorous techniques and/or institutional processes for IT sector planning;  whether projects proceed in well-defined stages;  whether projects are backed by senior leadership (either political or from key central agencies like Finance Ministries);

Digital literacy and proficiency
This measures how effectively individuals and organizations engage with digital technology for both internal and external stakeholder benefits [18,20]. The various aspects that can be considered include:  Percentage of the marketing spend that is digital  Brand value in the market  Reach of the organization in the market  Digital maturity quotient of the employees including board and senior leaders  Percentage of revenue through digital channels  Contribution to digital initiatives from each department, such as purchasing, finance, HR, IT, Sales & Marketing To determine the level of public sector digitalization, equal emphasis needs to be placed on digital skills development as to infrastructure development. Digital literacy involves more than the ability to use software or operate a digital device, it also includes a large variety of complex cognitive, sociological, and emotional skills that end-users need to function effectively in a digitally-driven environment [20]. Studies of [12] noted that digital literacy has changed continually with rapid advances in technology. General society indicators that can be looked at for digital literacy as suggested by the EU are listed in Table 2 below. None Basic (if one or more "basic" items but none of the "above basic" items) Above basic (if at least one of the "above basic" items) SHS Web of Conferences 9 2, 0 (2021) Globalization and its Socio-Economic Consequences 2020

Conclusion
Alone, none of these proposed indicators sufficiently capture the progress and quality of public sector digitalization effort. Together, however, they paint a rich composite picture and could contribute to filling the substantial measurement gap in the literature on public sector digitalization progress. This paper has refined the discussion of the overall indicators used for measuring digitalization progress in the public sector. While these indicators may not specifically apply to the private sector, this paper focus and addressing the gap in literature on the appropriate indicators used for measuring public sector digitalization progress. Some avenues for future research include looking more critically into how the stated indicators can individually measure the digitalization progress in the public sector. This article was supported by grant No. 2020_018 supported by the Student Grant Competition.