Algorithmic transparency: Chatbots with generative AI from the AOC

Overview 

Problem to be solved

• Need for constant assistance: AOC users need continuous, 24/7 support, but face-to-face or telephone support cannot guarantee this full availability.
• Increase in inquiries and incidents: The growing use of digital services and the AOC transparency portal has increased the demand for information and assistance, with more than 91.000 requests annually handled by the User Service Center (CAU). To manage this increase efficiently and maintain the CAU's commitment to citizens, this is a good time to consider incorporating new tools that help optimize resources.
• Digital divide and search difficulties: Many users, affected by the digital and social divide, often have difficulty knowing what to search for or correctly interpreting the results of traditional search engines. This can lead to confusion and frustration in your digital experience, necessitating more tailored support that allows easy access to relevant information.

Adopted solution

• Chatbot Development with Generative AI: To facilitate self-resolution of queries in an accessible and immediate way, AI-based conversational assistants have been created. These chatbots make it possible to answer the most common questions of users by using large amounts of data and algorithms that interpret natural language.
• Why Opt for Generative AI Chatbots? The AOC began experimenting with chatbots almost a decade ago, gradually introducing them to improve user support. Initially, conventional chatbots were implemented such as Paula (21/01/2021), which provided support in the identification process with VÀLid; Rita (29/04/2021), who helped in obtaining, using and renewing the idCAT Certificate, and the chatbot of the transparency portal (19/4/2023), dedicated to solving queries in this area. Despite their usefulness, these chatbots operate with rules and structured flows of predefined questions and answers, which requires intensive manual training, makes their implementation complex and expensive, and also delivers answers with relative success. (See transparency sheet Conventional Chatbots of the AOC)
  With this experience, the AOC is now committed to a new generation of chatbots based on data and generative AI, which need much less training and can provide more tailored and contextualized responses, thus improving the user experience. In addition, this technology allows re-asking, that is, completing the initial question while maintaining consistency with the previous conversation, while in conventional chatbots, each question is treated as if it were the first.

Who is it addressed to?

• Citizens who use the services of the AOC, to offer them accessible, fast and 24-hour support.

Use cases

Chatbots with generative AI are applied in the following AOC services:

• VALID: assists citizens with the most frequent queries during the identification process on the VALid page.
• idCat Mobile: helps resolve doubts about obtaining, using and renewing the Mobile idCat.
• IDCat Certificate: offers support with obtaining, using and renewing the Certified idCat.
• e-NOTUM: resolves doubts and provides support in the management of electronic notifications.
• It represents: helps citizens and companies resolve doubts about representations and powers of attorney. 
• e. FACT: provides support in the preparation and management of electronic invoices.

Status

• Implemented and in use since May 2024, with an average of 11.575 monthly users. 

Level of Service Agreement

• Available 24×7 with an availability level of 99,8%.

Main benefits of the service

 For citizens:

• 24/7 access to information: Citizens can obtain assistance at any time, without time restrictions, facilitating the resolution of doubts when they need it.
• Immediate response: Chatbots offer instant responses, avoiding the waiting times of traditional channels such as face-to-face or telephone support.
• Self-resolution of queries: Users can resolve their issues autonomously, reducing their reliance on direct support and improving the experience.
• Improved user experience: Chatbots with generative AI allow for a more natural and close interaction, facilitating an authentic "conversation" with the user. These chatbots can adapt to each person's way of expressing themselves, allowing them to specify or refine their question using their own words.

For administration:

• Optimization of resources: Chatbots handle repetitive queries, freeing up staff for more complex tasks, which improves time and resource efficiency. 
• Cost reduction: Each consultation via chatbot has a much lower cost than telephone support.
• Increased efficiency: Chatbots handle multiple queries at once, avoiding bottlenecks and improving responsiveness.
• Less training effort: Generative AI chatbots require less initial setup and training, starting from a general pre-trained model that adjusts with minimal tweaks. The adjustment process consists of reviewing and correcting detected incorrect answers, rather than working exhaustively on the classification and structuring of answers for each The intent [1].
• Continuous improvement of services: Conversation analysis makes it possible to identify patterns, improve key areas and adapt services to user needs.

Contact information 

More detailed information about the service 

Familiarize yourself with the data and information used by the system, the operating logic of the algorithms and its governance. 

Data sets 

The knowledge base of chatbots is based on three main sources:

• Frequently asked questions (FAQs): The chatbots are initially trained with a database of frequently asked questions available on the support portal. These FAQs collect the most common queries from citizens and are updated with the most recent information about the service.
• Recurring tickets: Chatbots also learn from the analysis of historical support tickets that contain information about users' questions, doubts and incidents and how they have been resolved.
• Continuous feedback: The chatbot system presents a form of constant learning where each generative attempt has a series of training sentences nurtured through user conversations. In case a person asks a new question, the chatbot will try to find similarities in its existing training sentence base. If the chatbot generates an incorrect answer to this question, during the conversation review period, the sentence will be included within the corresponding attempt and the AI ​​will generate variations of these to have a more extensive knowledge base.

Other input data:

• User data: No personal data from users is used to train the model.

Considerations on the data processing and conservation

Data processing 

The operational logic of automatic data processing and the reasoning carried out by the system is based on the following model and methodology: 

User-chatbot interaction process
The user-chatbot interaction process takes place through a digital interface accessible from devices such as web, mobile or tablet. The user enters a question in text format ("prompt") that describes the information he wants to obtain. The chatbot processes this information and generates a text response to provide the requested information.

Service architecture

The service uses a generative artificial intelligence engine [2] that processes natural language.

• The AI ​​engine used by the AOC chatbot is GPT-4o, from the OpenAI company.
• An Azure OpenAI instance located in Europe is used.
• The engine works on the basis of a large language model (Large Language Model, LLM), which is made up of deep neural networks trained with huge data sets extracted basically from the Internet (web pages, blogs, social networks, articles scientists, books, etc.)

The service has been adapted by 1MillionBot, which has been responsible for:

• Design the instructions that allow the AI ​​model to guide and follow directions.
• Prepare datasets, collection and cleaning of data that will serve as a reference for the Retrieval Augmented Generation (RAG) system
• Incorporation of datasets in the RAG system of 1MillionBot, which is able to improve the quality of the answers using specific contexts.
• Test cases and validate the system's ability to handle errors and complex queries.
• Model optimization based on user interactions.

The entire infrastructure of the service is hosted in the cloud of Google Cloud Platform (GCP), specifically in the Google Cloud data center in Belgium, within the European Economic Area (EEA). This location ensures compliance with European privacy and security regulations.

The chatbot has a system to collect the opinion of the users and to be able to assess the satisfaction of the overall conversation.

Main characteristics of the architecture:

• Safe and certified accommodation:
  • GCP installations comply with security standards ISO 27001 i SOC 2/3, ensuring a robust and reliable infrastructure.
  • All infrastructure of the provider, 1MillionBot, including webchat, the Millie platform (frontend) and logic and data (backend), is hosted in this environment.
• Isolation by use cases:
  • Each use case of the service is isolated in a specific network and development environment, avoiding interference between applications and maximizing security.
• Protected storage:
  • Conversation questions and answers are stored in databases hosted on Google Cloud virtual machines, which are protected by advanced firewalls and isolation mechanisms.
  • Access to this data is exclusively through the backend, ensuring strict control.
• Protection against attacks:
  • The system incorporates several layers of security, such as:
    • Request limiters to prevent overloads.
    • Strict validations of data and access.
    • Specialized firewalls to protect the infrastructure.

This architecture ensures the robustness, security and adaptability necessary to offer a quality service to users.

Algorithm performance 

91,33% success rate. This rate is reviewed monthly (last review: 15/04/2025)

Human supervision 

The algorithmic system of the chatbot acts directly, but it is under ex post surveillance of the staff responsible for the final decision, especially intensified in the initial period of operation to compensate for algorithm biases and reduce wrong answers. Chatbot responses are fixed and cannot be modified.

Human supervision of chatbots ensures the quality and accuracy of responses through a rigorous and regular process:

1. Weekly reviews: Experts review chat logs to identify incorrect answers. For each detected error, the ideal answer is noted and user sentences with the wrong answer are reclassified, assigning them to the most appropriate generative attempt.
2. Monthly analysis: 150 randomly selected conversations are evaluated, along with those already identified as erroneous, to detect error patterns and correct inappropriate responses.
3. Attempt management: When errors cannot be fixed by simply reclassifying sentences, experts adjust the prompts of existing attempts or create new attempts.

This process combines the traditional methodology of chatbots with the flexibility of generative AI, guaranteeing agile management and a reliable service, aligned with users' needs.

Regulatory compliance of the system

The system does not process personal data. Even so, the system complies with current regulations on data protection and security. In particular:

• The principles of the General Data Protection Regulation (RGPD) apply:
  • Principle of data minimization: only the data necessary to fulfill the purpose of the system is collected.
  • Purpose limitation principle: the data collected is only used for the purpose communicated to the interested party.

• From a technical point of view, the solution complies with the requirements established by the National Security Scheme (ENS) for LOW level systems.

✨Voluntary application of European Union Artificial Intelligence Regulation (AI Regulation) [3]

The system complies with the requirements of the AI ​​Regulation required in the case of risk AI systems limited i generative AI models without systemic risk. That's why:

• Users are informed that they are interacting with an AI from the start. For this reason, the service is presented as "AOC virtual assistant" and not with a personal name.

• As generative AI, AOC chatbots also meet the following requirements:
  • It is revealed that the content is generated by AI.
  • A summary of the training data is published.
  • Information about the AI ​​model is provided in clear and accessible language.
    The last two points are met with this algorithmic transparency sheet.
• These chatbots are not subject to the requirements for generative AI with systemic risk. This means that there is no need to pass model evaluations or contradictory tests, nor does it have to report on energy efficiency.

This is a Automated Administrative Action (AAA)? No.

In this case the AOC is not subject to the legal obligation to publish the AAA linked to the service on its website, accompanied by a technical file (See article 11, letter i, of Royal Decree 203/2021 , which approves the Regulation on the performance and operation of the public sector by electronic means, which deploys Law 40/2015 on the Legal Regime of the Public Sector)

Risk management 

Know the more likely risks in relation to the fundamental principles and rights that must be protected, and the measures applied in each case to mitigate or minimize them.

A. Equality and non-discrimination

• Identified risks: biases in the results, risk of exclusion and discrimination.
  • The LLM model of the GPT-4o engine, which forms the basis of the AOC chatbots, is pre-trained with large volumes of data from the Internet. This data includes a wide variety of information and may contain biases related to gender, ethnic origin, language, among others. As a result, the system can reproduce discriminative patterns present in the training data. In addition, the complexity of neural networks makes it difficult to identify and prevent these biases.
  • On the other hand, Catalan is not one of the main languages ​​used in model training. This can result in responses with a predominant use of masculine forms and less linguistic diversity, affecting the inclusiveness and richness of interactions in that language.
• Measures applied
  • Representative training data: The data used to make the final training and fine-tune the chatbot has been carefully selected to include various population groups and avoid explicit biases in order to ensure that the responses are as representative and fair as possible.
  • Equality in linguistic access: The service can be used in Catalan and Spanish, with the option of choosing the language at the beginning of the conversation or changing it during use. This allows all users to interact with the system in the language they prefer.
    • Currently, the chatbot asks the user in which language they prefer to be served, as this improves the accuracy of the answers. However, tests are being conducted to assess its performance without the need to ask the language.
  • Tolerant interpretation of language: The system is designed to understand natural language, even if there are spelling or linguistic errors, ensuring that users are not penalized for small mistakes in their queries.
  • Inclusion and diversity in design: Worked on a chatbot interface that can be easily used by anyone, regardless of their digital skill level, to ensure that everyone can access the service without technological barriers.

B. Data protection, privacy and freedom of choice

• Identified risks: possible threat to privacy, misuse of personal data by third parties and loss of freedom of choice.
  • The service never asks for personally identifiable information. However, the service involves a risk related to the processing of personal data if the user enters unnecessary personal data.
• Measures applied
  • To protect data and privacy:
    • The provider (1MillionBot) has signed a Processing Order with the location of the data within the European Union.
    • The Data Controller (AOC) guarantees the ARCO rights (Access, Rectification, Cancellation and Opposition) of the processed data.
    • Consent and cookies are managed.
    • Conversation logs are regularly reviewed by experts who remove any personal data not detected by the system.
    • In the next version, a feature will be implemented that will automatically delete any social security number or email entered in the chat, and the user person will be able to request that their conversation be removed from the system.
    • The user person is informed that the conversation will be saved. If you don't agree, you can't continue.
    • The user person can delete the conversations.
  • To guarantee freedom of choice:
    • We offer alternatives to help citizens without the use of AI algorithms, such as telephone and email support. If the user specifically asks to “contact support”, “talk to an agent”, “I want more support”, or similar, the bot automatically redirects to the AOC support contact pages. In addition, the chatbot is located within the AOC support portals themselves, where you can also directly access the “Contact Support” option from the footer.

C. Safety and robustness

• Identified risks: dissemination of incorrect information, errors and inconsistencies in responses, unavailability of service and unauthorized access to data.
  • Models based on neural networks and probabilistic calculations make it difficult to interpret how the final result is reached, which limits control over the responses generated. In addition, the nature of generative AI may occasionally cause the system to draw information beyond its specific knowledge base, including unverified external sources or data, which may result in out-of-context responses. or on topics that do not belong to their field.
  • To mitigate these risks, the AI ​​engine must be installed in a controlled and protected environment, with strict configuration and security to ensure protection and service integrity.
• Measures applied
  • The system has been installed in a closed and secure environment, based on a certified architecture (see the main characteristics of the architecture). This configuration ensures:
    • Robustness and reliability, ensuring the correct operation of the system.
    • Security and confidentiality of managed data.
    • Adaptability, to offer quality answers aligned with users' needs.
  • Security risk analysis: the AOC has carried out a security assessment following the guidelines of the Catalan Cybersecurity Agency and the National Security Scheme (ENS). The system has been classified with a LOW security level, and in accordance with this categorization, specific measures have been implemented to mitigate the identified risks. Specifically, to guarantee the availability and correct operation of the system:
    • Service Level Agreements (SLAs) have been established with the supplier, which must be complied with and reviewed periodically, to ensure system capacity, availability and incident management.
    • Fortnightly ANS compliance reports are carried out to ensure that the service maintains agreed standards.
  • To guarantee the right to a truthful and adequate information: protocols are applied to review the system, minimizing errors and inconsistencies, and improving the quality of responses. Additionally, to ensure responses within the chatbot's scope of competence:
    • Keyword and topic filters have been implemented (e.g. Certificate of Registration, Payment of a fine, Pre-registration at school, etc.). These filters generate "attempts outside the catalog", with which the chatbot informs that it does not have information on the subject and recommends contacting the relevant administration.
    • The model is trained exclusively on data from the AOC domain, ensuring as much as possible that the information is relevant and fit for purpose.

D. Transparency and explainability

• Identified risks: mistrust, misinterpretation of the results by users (inability to understand the result), confusion in thinking that it is an interaction with a human, difficulties in submitting complaints or inquiries about the results obtained
• Measures applied
  • Clear identification as AI: From the beginning, the service is presented as a "virtual assistant" to avoid confusion about the nature of the interaction.
  • User Consent: Users are informed that the conversation will be saved, so they can decide whether they want to continue.
  • Algorithmic Transparency Sheet: It is published on the AOC website, with information on:
    • Problem to be solved
    • With the AI ​​system implemented
    • Target audience
    • Data used to train the AI ​​system
    • Identity and contact details of the supplier
    • Responsible body and contact address for complaints, inquiries and suggestions
  • Easy access to the file: The transparency sheet can be consulted from the home page of the service.

E. Retention of accounts and auditability

• Identified risks: Limitations in the traceability of the results, lack of periodic verification and review, difficulties in the interpretation of the model even by specialists, absence of assigned responsibility.
  • The risks associated with the accountability and auditability of generative AI chatbots can jeopardize the reliability of the system if there is no means to accurately trace the response generation process, verify quality on a regular basis, and ensure that the operation is understandable even by specialists. These factors make it difficult to properly supervise the service and make it difficult to attribute responsibility for mistakes or unfair decisions, which can affect trust in the public service.
• Measures applied
  • To ensure accountability of the AI ​​system and its results:
    • Quality and impact monitoring: Bi-weekly reports are made with quality and impact indicators to assess whether the service meets the established objectives.
  • To guarantee responsibility for the system and its results:
    • It is informed who is responsible for the service (AOC) and the procedure to follow to submit a complaint in case of adverse impacts.
  • To ensure auditability:
    • The AOC reserves the right to conduct additional audits to review quality indicators and Service Level Agreements (SLAs) at any time, thereby ensuring continued monitoring of system reliability.

F. Sustainable development and solidarity

• Identified risks: negative environmental impact, inequalities in access to this technology due to its high cost.
  • The use of chatbots with generative AI poses significant challenges in terms of sustainability and equity. On the one hand, its environmental impact is high, as it requires large computational resources for data processing and storage, which implies high energy consumption and CO₂ emissions. On the other hand, its implementation and maintenance costs are significant, which may limit access to this technology to organizations with more resources.
• Measures applied
  • Chatbot training tuning and optimization to achieve maximum efficiency. Studies indicate that one ChatGPT query can be equivalent to ten Google searches; therefore, if this is confirmed, the high energy cost of generative AI could be offset by its greater effectiveness compared to cheaper methods.
  • Promoting transparency on usage costs to contribute to a more responsible use of this technology.
  • Promoting transferability: The AOC is working to offer the chatbot as a “turnkey” service to Catalan local administrations, facilitating access to this technology in an equitable manner. This replicability strategy will allow the adoption of advanced generative AI without requiring large investments.

More information

• Article "Generative AI and ChatGPT. Impact on university learning". By Climent Nadeu Camprubí. Signal and Communications Theory Department (TSC). TALP and IDEAI Research Centers. Polytechnic University of Catalonia (UPC).

[1] Un The intent refers to the intention or purpose that the user person has when asking a question or making a query to the chatbot. Attempts help identify what the person wants to achieve, such as obtaining specific information, solving a problem, or completing a procedure. In traditional chatbots, attempts must be predefined and manually trained so that the chatbot can recognize them and provide the appropriate response. In contrast, chatbots with generative AI can more flexibly identify and interpret attempts by generating responses in real-time from the context of the conversation, without the need for exhaustive attempt classification up front. This makes the interaction more natural and adapted to the needs of the user.

[2] Some of the main capabilities of the generative artificial intelligence engine are:

• Identify and correct typographical and spelling errors to improve communication.
• Detects message intent, even with ambiguous sentences or multiple meanings.
• Maintains short-term memory to connect questions and ensure consistent answers.
• It evaluates possible answers, chooses the best one, and offers suggestions for clearer conversations.
• It adapts to several languages ​​(Catalan, Spanish, English), recognizes slang and emojis.

[3] El Regulation of the European Parliament and the Council establishing harmonized rules on artificial intelligence and amending certain legislative acts of the Union (better known as the EU Artificial Intelligence Law or AI Act) entered into force on 2 August 2024 and will apply from 2 August 2026.