Challenge

As we can measure the degree of digital maturity of an entity and a territory so that almost any administration can accelerate its transformation to improve people's lives?

Problematic 

The digital transformation of an organization is a complex process that requires a constant evaluation of the progress made. However, developing an instrument to measure the degree of digital maturity involves a series of difficulties:

  • there is no precise definition of the concept of digital government that unequivocally determines its dimensions and the indicators that define it;
  • the processes of collection and integration of the data necessary to develop these indicators present difficulties;
  • sufficient or quality data is not always available, which leads to having to discard some indicators from the analysis;
  • it must meet the needs of all councils equally and allow comparisons to be made, regardless of their size. This is not trivial, since the population conditions the set of services to be provided and the potential volume of activity of the digital services it provides.

In addition to the complexity involved in measuring the degree of digital transformation of an entity, the fact that most municipalities in Catalonia are small or medium-sized and have all kinds of limitations to face this challenge, such as:

  1. very limited financial resources;
  2. data access difficulties;
  3. lack of capabilities on how to define and generate digital key indicators;
  4. impossibility of being able to make comparisons with other similar administrations due to difficulty in accessing the data and lack of data standards; i
  5. public workers without advanced analytical skills.

Local administrations therefore need a turnkey solution that allows them to evaluate their change process with minimal effort and make the most informed decisions based on data. To respond to this need, the AOC has created theDigital Maturity Index (hereinafter, IMD), a tool that evaluates the degree of digital transformation of Catalan local administrations and allows administrations to compare themselves with each other.

Proposed solution

The IMD is a tool that provides us knowledge to design a digital transformation strategy for all Catalan public administrations, as well as plan the necessary actions that will need to be carried out.

It's not just another indicator. It is an information system (“relates indicators”) that allows:

  • Know the degree of digitization of a public body (strong points and weak points of each body) and understand the progress of its transformation process.
  • Compare and gain perspective. The digital maturity of each of us becomes more evident when it is compared (contrasted) with others. In addition, it makes it possible to know the degree of digital maturity of a territory, be it the whole of Catalonia, a province or a county.
  • Get an overview (who is leading the way and who needs help) and understand the context of public sector digitization (strengths and areas for improvement).
  • Others: create a data-driven culture, identify best practices, value the work done.

In addition, it is especially useful for organizations that must provide support in the digital transformation of local entities (such as the AOC!), since it allows the identification of objectives on which to act and facilitates the design and analysis of the effectiveness of digitalization policies.

Until 2018, the AOC calculated the degree of digitalization of local entities based on indicators of the Electronic Administration Map. These indicators were obtained from the implementation data of the AOC services and the analysis of the municipal websites. All were “attribute” type indicators, that is, they indicate qualities or states that the entity analyzed may or may not meet (e.g. has a citizen folder, e-headquarters, contractor profile, etc.; offers online procedures, sends e-notifications, accepts e-invoices, etc.; exchanges data with other administrations, is a T-CAT registration entity or idCAT, has approved regulations on e-administration, has joined the FUE service, etc.). Over time, we realized that the adoption of digital services by local entities was becoming more widespread and that evaluating the degree of implementation was not enough. It was necessary to look for new indicators that reflected progress in digital transformation, such as the use that entities made of these digital services.

In 2018, activity indicators were introduced, i.e. “variables” that can be quantified, such as:

  • Inquiries to Via Oberta
  • Procedures EACAT (or SIR Procedures)
  • e-TRAM procedures (including the generic transfer and the FUE channel)
  • e-FACT invoices
  • e-NOTUM notifications
  • Use of T-CATs (Validator)
  • Publications in the e-TAULER
  • Publications on the contractor's profile (electronic tenders)
  • Publications/updates on the portal transparència
  • Submissions from the IVTM and/or TESTRA

However, in a context as changing as the current one, the way to evaluate electronic government must continuously evolve to remain effective and incorporate new aspects that respond to the reality of the moment, based on empirical evidence. The e-Government reports and maps project thus entered a new phase at the end of 2019 and was replaced by an index that we named theDigital Maturity Index.

Thanks to the implementation of Business Analytics in the management of AOC services, it was possible to automate the collection and integration of the index's activity indicators and present the information through an interactive visualization of the data. In addition, in the 2020 edition, a significant improvement was made in the evaluation of the use of digital services: instead of assessing the use of a digital service based on the absolute number of transactions, we began to measure the digitalization associated with the use of the service; that is, the calculation as the number of digital transactions compared to the total number of transactions, including face-to-face ones (for example, electronic entries compared to total entries, or electronically tendered contracts compared to total tendered contracts).

The IMD, a collaborative tool in constant evolution

In order for the IMD to be useful and informative, it must always be kept in mind that only the most relevant indicators of the moment in which we find ourselves should be included, those that mark the difference between an entity that we consider digitally mature and another that is not. And it is necessary to determine very carefully the weightings of each attribute or variable within the index. To provide the IMD with a rigorous and flexible calculation methodology (capable of adapting to the present moment), a working group of experts in public management was created and the result was shared and debated with the provincial councils, regional councils and city councils.

In addition, activities and meetings of staff from the local world are periodically organized to review and evolve the IMD, such as the workshop that took place at the Digital Government Congress 2021, in which more than 50 participated representants of Catalan local administrations and where improvement proposals were collected, some of which were incorporated into the 2023 IMD, or the workshop held in November 2023 with representaof municipalities and provincial councils, to revise and improve the IMD.

Therefore, the IMD is the result of collaborative work, always applying a continuous improvement approach with feedback from public staff and continuous investment in the creation of a data warehousing which integrates data from different sources.

IMD calculation methodology 

The IMD is a composite index or indicator consisting of more than 30 indicators organized into three dimensions and nine sub-dimensions that are aggregated in a weighted manner to obtain a comparable value. It is currently calculated for 947 councils and 41 county councils.

The IMD is conceptually divided into three dimensions:

Digital rights. We assess the availability of digital public services that allow citizens to exercise their digital rights and compliance with the regulations governing these rights.

Digital services activity. We evaluate the use of digital services by citizens, companies, associations and public employees. Specifically, we measure digitization in the relationship with users, the digitization of internal processes and interoperability between public administrations.

Open governmentWe evaluate the practical application of open government principles: transparència, access rights, open data and citizen participation.

The following figure schematically shows the three dimensions of the IMD, as well as their sub-dimensions and the indicators that make them up:

Map of the subdimensions and indicators of the Digital Maturity Index

The IMD is the weighted sum of the values ​​of the indicators in each of the three dimensions:

Mathematical formula of the IMD

Each indicator is assigned a weight and a maximum score based on its importance. The different weights have been assigned to 1) recognize (or penalize) the dimensions and sub-dimensions that contribute more (or less) to the digital transformation, according to people's opinion; and 2) reflect the statistical quality of the data.

See APPENDIX Data sources and types of indicators.

Presentation of the results

The IMD is presented through a set of dashboards and reports intended for the different profiles involved in the decision-making of a local body.

  • The "Let's make a digital country" dashboard shows the key indicators of a specific territory in aggregate form (all of Catalonia, a province or a region). In addition to the IMD's own indicators, it shows others that are derived from them, such as the percentage of entities that process electronically or that have a portal. transparència.
Dashboard "Let's make a digital country"
  • The dashboard "Your entity - Global vision" provides a quick and clear view of the state of each entity with the aim of enabling municipal elected officials to make data-based decisions. In addition to the total score of the IMD and its dimensions, it shows the value of key indicators and compares them with the average value of local entities in the same population band.
Dashboard "Your company - Global view"
  • The report "Your Ens – Detailed View" provides the complete picture of the IMD. It shows the value of all the indicators of a local body and also places them in the context of the municipalities in its population band. It is intended for managerial staff and municipal technical staff so that they can identify the strong points and areas of improvement on which to act.
Report "Your us - Detailed view"
  • The "General comparison" report offers a global view of the degree of digital maturity of a territory. It allows supra-municipal bodies to identify where digital transformation policies need to be deployed for each of the three main dimensions of the index.
"General comparison" report

Acknowledgments Open Administration

We use the score obtained at the IMD to award the Open Administration Awards each year to the leading administrations.eres in digital transformation, divided into seven categories according to population range.

More information about the Acknowledgments Open Administration.

Conclusion

The sustainability of this initiative is based on promoting that all the information required to produce the Index is available in open data formats based on standards, so that they can be easily obtained and integrated. Currently, 95% of the required information meets these requirements.

The IMD is a collaborative initiative involving different levels of government. The four Provincial Councils of Catalonia and all the Catalan regional councils participate in the validation of the MD's methodology and results. Good governance is essential in this project. Regarding the usefulness of the IMD, we have verified that the publication of the results, and especially the annual recognitions, serve as a stimulus for local administrations to identify their areas of improvement and advance in the digital transformation process.

Status of the project

In production. Every year, since 2019, the AOC analyzes the digital maturity of local public entities using the IMD methodology. The results can be consulted in the Digital Maturity Index visualization reports.

More information

APPENDIX Data sources and types of indicators

Data sources

The IMD is calculated from public, objective and verifiable data obtained from the following information sources:

  • The web pages of the local administrations themselves.
  • The activity data of the digital services of the AOC available at Open Data Portal of the AOC.
  • The activity data of the AGE services available on the Data OBSAE portal of the Electronic Administration Observatory (General Secretariat of Digital Administration).
  • The activity data of digital services provided by the local entities themselves and by the providers of market platforms, which are subjected to a review process to contrast their quality.
  • Data from the Infoparticipa Index of the Autonomous University of Catalonia and the Cybersecurity Exposure Index of the Cybersecurity Agency of Catalonia.
  • The data published in the official information and documentation search engine of the Diputació de Barcelona (CIDO).
  • The recruitment data provided by the Catalan Recruitment Consultative Board.
  • Citizen participation data provided by providers of public electronic participation platforms (Diputacions de Barcelona and Girona) and market (supplier companies).

In the case of missing data, intermediate variables are used (for example, if we do not have the total number of registration entries of a City Council, we obtain the data from the number of inhabitants and a correlation coefficient) . All missing data imputation methods used have been validated based on their statistical properties.

IMD indicators

Below is a breakdown of what is analyzed and how each indicator is scored:

Digital rights

  • Generic instance:
    • If the us has a generic instance to do procedures online
    • 3,80 points if he offers it to us and 0 points otherwise
  • Citizen folder:
    • If it is available to us from the citizen folder
    • 2,28 points if we have it and 0 points otherwise
  • Web search engine:
    • If it is available to us as a search engine on your municipal website
    • 1,52 points if we have it and 0 points otherwise
  • Urban planning viewer:
    • If it is available to us from the town planning viewer on the municipal website
    • 0,76 points if we have it and 0 points otherwise
  • Map viewer:
    • If you have a cartographic viewer on the municipal website
    • 0,76 points if we have it and 0 points otherwise
  • Finestreta empresarial:
    • If you have a Business One-Stop Shop
    • 1,52 points if it is offered by the maximum number of procedures and the proportional score otherwise
  • Registration Entity idCAT:
    • If the entity is a registration entity idCAT
    • 2,28 points if the entity is a registration entity and has issued certificates idCAT during the year of calculation of the index, and 0 points otherwise
  • Register in Via Oberta:
    • If you allow us to carry out interoperability queries on your register
    • 3,80 points if he offers it to us and 0 points otherwise
  • Cybersecurity level:
    • Cybersecurity Level score calculated by the Cybersecurity Agency of Catalonia.
    • 1,14 points if the indicator is 1 (advanced security performance) and proportional score otherwise (the relationship between the cybersecurity level calculated by the ACC and the indicator value in the IMD is as follows: Advanced=1, Intermediate= 0,75, Basic = 0,25)
  • Own ENS certificate:
    • If the local entity has a certificate of compliance from the ENS on the website of the Centro Criptológico Nacional
    • 1,14 points if the entity has a certificate of compliance from the ENS and 0 otherwise
  • Data Protection Officer:
    • If the entity has a data protection officer and has communicated this to the APDCAT
    • 0,76 if the entity has a data protection officer and has communicated this to the APDCAT and 0 otherwise
  • Service MUX:
    • If the entity is integrated with the service MUX AOC registration
    • 2,28 points if the entity is integrated with MUX Register and 0 points otherwise
  • Citizen folders hub:
    • If the entity is integrated with El meu espai
    • 2,28 points if the entity has integrated its case manager with El meu espai, 0,81 if it has integrated AOC services and 0 points if it has no integration
  • Electronic archive:
    • If the entity is integrated with iArxiu (or own system) effectively, with transfers made
    • 2,28 if the entity is effectively integrated with the electronic archive and 0 points otherwise
  • Accessibility mailbox:
    • If we have an electronic processing mailbox for accessibility issues
    • 0,76 points if we have it and 0 points otherwise
  • Accessibility seal:
    • If an accessibility statement has been published on the municipal website
    • 0,76 points if it has been published on the municipal website and 0 points otherwise
  • Regulatory ordinance:
    • If it has been approved by an electronic administration ordinance
    • 1,52 points if it has been approved by us and 0 points otherwise
  • Power of attorney registration:
    • If it is available to us from the power of attorney register
    • 0,76 if the company has a power of attorney register and 0 otherwise
  • ENI compliance:
    • If the e-administration platform has a certificate of compliance with the ENI
    • 2,28 points if an ENI compliance certificate has been issued to us and 0 points otherwise
  • ENS Compliance:
    • If the electronic administration platform has a certificate of compliance from the ENS on the website of the Centro Criptológico Nacional
    • 2,28 points if the ENS has published a certificate of compliance and 0 points otherwise

Service activity

  • Input digitization:
    • Percentage of electronic entries compared to the total number of entries (Online procedures e-TRAM (or own system) + e-Invoices e-FACT + Submissions EACAT total receipts / entries)
    • 8,10 points if the percentage is 80% and the proportional score otherwise
  • Output digitization:
    • Percentage of electronic outputs compared to the total number of outputs (e-Notifications e-NOTUM + Submissions EACAT sent / total outputs)
    • 5,40 points if the percentage is 100% and the proportional score otherwise
  • Publications in the Register of Contracts:
    • Rate of validated publications in the public register of contracts per inhabitant
    • 0,68 points if the ratio is in the highest percentile of the entity's population band and the proportional score otherwise (3 percentiles are established for each population band)
  • E-tenders:
    • Percentage of electronic tenders for open contracts with respect to the total
    • 4,05 points if the percentage is 100% and the proportional score otherwise
  • Publications on the Contractor's Profile:
    • Rate of publications on the contractor's profile per inhabitant
    • 2,03 points if the ratio is in the highest percentile of the entity's population band and the proportional score otherwise (3 percentiles are established for each population band)
  • Services of Via Oberta assets:
    • Percentage of services Via Oberta active with respect to the mandatory
    • 5,40 points if the percentage is 100% and the proportional score otherwise.
  • Inquiries Via Oberta:
    • Rate of queries made to Via Oberta per inhabitant
    • 12,60 points if the query ratio of Via Oberta is in the highest percentile of the entity's population range and the proportional score otherwise (10 percentiles are established for each population range)
  • Automatic queries:
    • Percentage of queries in Via Oberta carried out automatically with respect to the total number of queries carried out by the entity
    • 5,40 points if the percentage is 100% and the proportional score otherwise
  • Internal e-signatures:
    • Ratio of public worker signatures with respect to the number of entries
    • Indicator excluded from the IMD 2025 due to lack of automated data for all entities
  • Automation (Automated Administrative Actions):
    • If the entity has published a rule that regulates automated administrative actions.
    • 1,35 points if the entity has published a rule that regulates automated administrative actions and 0 otherwise

Open Government

  • Alert channel:
    • If we have an internal alert channel.
    • 2,50 points if we have it and 0 otherwise
  • Right of access to public information:
    • If the entity has a specific electronic procedure for requests for access to public information
    • 1,67 if the entity has a specific electronic procedure for requests for access to public information and 0 points otherwise.
  • Compliance with transparència:
    • Score received in the Infoparticipa Index
    • 6,67 points if infoparticipa score is 100% and proportional score otherwise
  • Portal of transparència:
    • If the entity has a portal transparència
    • 3,33 points if it is available to us and 0 otherwise
  • Citizen participation:
    • Number of participatory processes by electronic means carried out by the entity
    • 1,67 points if the entity has carried out more than 7 participatory processes, 1 if it has carried out between 4 and 6 processes, 0,5 if it has carried out between 1 and 3 processes and 0 if it has not carried out any
  • Account performance:
    • If the entity has sent its accounts to the Auditor General within the established deadline
    • 1,67 points if the entity has presented its accounts within the established deadline, 0,835 if it has presented them incompletely, 0,4175 if it has presented them after the deadline and 0 if it has not presented them at all.
  • Electronic board:
    • If the entity has an electronic bulletin board and announcements
    • 0,83 points if we have it and 0 otherwise
  • Open data:
    • If you publish your own open data to us
    • 1,67 points if we publish it and 0 points otherwise