Challenge

How can we improve the attention and support to users of the AOC's digital services to achieve a better user experience and reduce the workload of the human support team, through the use of generative artificial intelligence technologies?

 

Problem that is intended to be resolved by the end of 2023 

The intensive use of the AOC's digital services generates a very high volume of queries and incidents, with more than 91.000 requests annually (2023) handled by the User Service Center (CAU). This sustained increase encourages the complementation of current channels —especially telephone service and the contact form— towards 24×7 availability through an AI chatbot.

This situation is aggravated by the digital and social divide: many users have difficulty finding the information they need, do not know how to formulate searches or do not correctly interpret the results of traditional search engines. All of this generates confusion, frustration and a limited digital experience, which further increases the number of queries that reach the CAU.

Although the AOC allocates 1 million euros annually to the support service, these allocations are not sufficient to absorb all the growing demand from citizens and administration personnel.

In this context, the AOC had already opted in 2019 for a first “conventional” chatbot based on predefined attempts. Although it allowed to offer a service available the 24 hours a day at a reduced cost, it presented important limitations: only answered simple queries, followed static flows, required intensive training and maintenance, and didn't maintain the context of the conversation, which severely limited its efficiency and flexibility. (See sheet of transparència algorithmic nature of AOC conversational chatbots – first generation or traditional chatbots).

This set of factors—high demand, need for continuous assistance, digital divide, limitations of traditional chatbots, and limited human resources—evidenced the need to adopt more advanced, flexible and scalable support solutions that would allow improving the user experience and optimizing available resources.

Proposed solution

The AOC has made a qualitative leap in user support service by deploying a new generation of conversational chatbots based on generative AI, developed jointly with the specialized company 1MillionBot.

These chatbots:

  • They are based on a large language model (LLM), specifically GPT-4o, through an instance ofAzure OpenAI located in Europe.
  • They use an architecture of retrieval augmented generation (RAG), which combines the generative model with the AOC's own knowledge base (frequently asked questions, recurring support tickets and user feedback).
  • Permeten natural and contextual conversations: they maintain the thread of the conversation, allow for follow-up questions and adapt to each person's way of expressing themselves.
  • They have been deployed on cloud infrastructure (Google Cloud Platform, data center in Belgium), with an architecture segmented by use case to ensure security, data isolation and scalability.
  • They are designed so that they don't look like a person: unlike old chatbots (Rita, Paula...), proper names are not used to avoid confusion about the automated nature of the assistant.

Chatbots with generative AI have been put into production on May 2024 and are currently fully integrated within the services interface, so that support is immediate, accessible and available 24×7.

Finally, the solution is accompanied by a sheet of transparència algorithmic published on the portal of transparència of the AOC (accessible from the chatbot itself in the option “Transparència algorithmic"), where the operation of the system, the identified risks and the measures adopted to protect the fundamental principles and rights of people are explained in plain language.

Current use cases

Chatbots with generative AI support citizens and businesses in several key AOC services, in which they resolve the most frequent doubts and accompany the user step by step:

  • VALID: assistance during the identification process.
  • idCAT Phone: obtaining, using and renewing.
  • idCAT Certificate: obtaining, using and renewing.
  • e-NOTUM: queries about electronic notifications.
  • Representa: doubts about representations and powers.
  • e-FACT: queries about the issuance and management of electronic invoices.

The segmented architecture allows these use cases to be progressively expanded to other services and channels, with the possibility of multi-channel and multilingual support.

Results and impact

Since the deployment of the chatbot with generative AI, the main results are:

  • Increased support capacity
    • the year 2024 were managed around 208.000 support requests (between chatbots and human agents).
    • Of these, 143.000 were resolved in a way automated through chatbots.
    • About half of these automated queries are already managed with the generative AI chatbot.
  • Reduction of the burden on the human team
    • Support tickets managed by human agents:
      • 2023: 92.532
      • 2024: 64.564
    • This means a 30% reduction in tickets handled by people, despite the very significant increase in the use of services (of 557 million transactions in 2023 a 724 million in 2024). However, this reduction cannot be attributed exclusively to bots, since during 2024 improvements were made to the support portal and to the services themselves, which contributed to the reduction in requests.
  • Generative AI chatbot performance
    • Average of11.575 monthly users attended to.
    • 88% hit rate, with clear, relevant and contextualized answers.
    • Availability of the service 24 × 7 with a service level of 99,8%.

This data shows that the generative AI solution allows serve more people, with better quality and without increasing human resources, freeing up time for more value-added tasks.

Benefits and conclusions

For the citizens

  • Continuous access: assistance available 24 hours a day, every day of the year.
  • Immediate and autonomous response: users can resolve most queries without having to wait or contact telephone support.
  • Natural and adapted conversation: the chatbot understands different formulations, allows you to refine the question and maintains the context, improving the user experience, especially for people with fewer digital skills.

For the administration

  • Optimization of resources: chatbots absorb repetitive and simple queries, allowing the human team to focus on complex incidents and continuous improvement of services.
  • Significant economic savings: a preliminary analysis estimates a savings of 96% in the cost of answering a query resolved by the chatbot (approximate average cost of 15€ for human attention vs. 0,60€ via chatbot).
  • Continuous improve: conversation analysis and daily supervision of the Support team allow us to update the knowledge base, detect new needs and adjust the service.

Conclusion

The implementation of chatbots with generative AI under a framework of transparència, security and human supervision, has been consolidated as a key piece in the AOC's support strategyThe solution demonstrates that it is possible to combine innovation, efficiency and respect for people's rights, while enhancing the capacity for attention and contributing to a more inclusive digital transformation.

Status of the project

In production. From May 2024.

More information

AOC Blog: