Clarifying concepts about AI: LLM, RAG, AI agents or agentic AI
Share
If you've read news about artificial intelligence in recent months, you've probably come across a soup of acronyms that all sound the same: LLM, RAG, AI agents, agentic AI... It seems like a tongue twister, and it's normal to feel a little lost.
The good news is that you don't have to be a computer scientist to understand this. These terms are not different competing modes: they are layers that are added to each other, and each layer solves a different problem. If you understand what each one brings, you will also better understand which one can be useful for your work and, above all, where the risks lie.
This publication is intended as a basic guide for anyone in public administration who wants to get up to speed, without the need for prior technical knowledge.
Layer 1. LLMs: the basis of everything
Els LLM (from English Large Language Models, i.e. large language models) are the engine behind tools like ChatGPT, Claude or Gemini. They are systems trained on a huge amount of text and have learned to do something very specific: generate language coherently.
What are they good at? Writing, summarizing, translating, explaining concepts, giving ideas, reformulating a text to make it sound clearer... In their day-to-day work, they can help you draft a circular, summarize the conclusions of a long report, or adapt a technical text to plain language.
But there is an important limit that must be kept in mind: an LLM does not know your data. He doesn't know what the internal regulations of your department say, nor what the specific procedures of your entity are, nor what was decided in the last committee. He only knows what he learned during training, and that may be outdated or too generic.
Therefore, LLMs are useful for general tasks, but by themselves they do not solve the specific problems of administration.
Layer 2. RAG: add knowledge
This is where the RAG (Retrieval-Augmented Generation, or augmented generation by recovery). The name is very technical, but the idea is simple: connect the LLM with your documents and data so that I can respond based on real information from your organization.
Imagine an assistant who, before answering a query about subsidies, goes to look for the regulatory bases published in the DOGC, the internal criteria of your department and the frequently asked questions on the website. This is, in essence, what a RAG system does: it retrieves the relevant information and uses it to construct a substantiated response.
This is where many administrations start to see real value. Some examples from the public sector where a RAG approach makes sense:
An internal assistant who answers staff questions about administrative procedures, based on internal manuals.
A citizen consultation tool that responds to municipal procedures using the official documentation on the website.
Support for technicians who must resolve cases by consulting regulations and precedents.
However, there is an essential condition: the quality of the source information is everythingIf the documents are messy, duplicated, contain contradictory versions, or have a chaotic structure, the answers will be equally confusing. The maxim in this world is clear: garbage in, garbage outIf "garbage" data is put in, "garbage" answers come out.
Therefore, before investing in a RAG system, it is worth asking a more basic question: Is our information organized enough for a machine to read it well? If the answer is no, perhaps the first step is not AI, but putting the documentation in order.
Layer 3. AI agents: moving from thinking to doing
So far, AI helps us understand and write. But the AI agents they go one step further: actThey use tools to perform tasks: they can create a document, send an email, update a record, compare two databases, or follow a series of predefined steps.
The change is substantial. Until now, AI helped you think; now it helps you do.
Let's think about it applied to administration. An AI agent could:
Receive a request, check if the attached documentation is complete and, if it is not, generate an amendment request.
Search a database to see if the applicant already has open files and attach them to the new file.
Prepare the notification, fill it in with the relevant data and leave it ready for someone to review and sign.
This leap opens up enormous possibilities for efficiency, but it also changes the nature of the risk. When a tool only suggests text, the error remains in a draft that someone else reviews. When a tool actua, the error can have real consequences: a request sent to the wrong person, data updated incorrectly, a procedure initiated without basis.
Therefore, in this layer the key question is no longer whether the tool responds well, but whether can act safelyAnd that means defining very well what the agent can do, what it can't do, when it has to stop and ask for human validation, and how each action is tracked.
Layer 4. Agentic AI: several agents that coordinate
If an agent is already a jump, the Agentic AI it is as follows: multiple agents working together within the same workflow, each with a specific role.
In such a scenario, we could have an agent who detects an incident in a service, another who checks the related data, a third who notifies the responsible team and a fourth who prepares an initial response to the affected person. All of this in a coordinated and, to a large extent, autonomous manner.
It sounds impressive, and it is. But it is also considerably more complex. Every handover between agents is a point where errors can occur. The more autonomy, the more difficult it is to track exactly what happened, who made which decision and why.
And here a question appears that, in the public sphere, is no less important: who is responsible when a system made up of several agents makes a decision that affects a person? Because in the administration, unlike the private sector, it is not enough for the system to work most of the time: it is necessary to be able to explain, justify and, if necessary, review each decision that affects a citizen.
Ask yourself the right question before moving forward.
When you receive a proposal or demo of an AI tool, instead of looking at whether it works in general, pay attention to what question needs to be answered according to the layer:
LLMDo you write well enough? Is the text it generates clear, useful and well-constructed?
RAGFind what you need? Do you know how to find the correct information in our documents?
AgentsIs it acting correctly? When performing a task, does it stay within expected limits?
Agentic AICan we respond as an organization? Do we know what each agent does, at what time, and whose responsibility it is if something goes wrong?
This last question is the one you should keep most in mind if you have to decide whether to go ahead with an initiative. Because the day a citizen complains about a decision that has affected them, someone will have to be able to explain what happened, who decided it and with what criteria. If the answer is “the system”, we have a problem.
The practical advice: start with the problem, not the technology
If you have to take away just one idea from this post, let it be this: don't start by asking yourself what AI you want to use, start with the problem you want to solve.
Too often the conversation goes the other way around: “we want to do something with agent-based AI.” And the right question is: “do we have a specific job where there are people wasting time, recurring errors, or citizens waiting too long, and can technology help us improve that?”
From here, choose the simplest pattern that solves the problem. Sometimes an LLM will be enough to help draft communications. Sometimes a RAG will be needed for the system to understand your regulations. And sometimes, yes, an agent will be worth it. But complexity is only justified if it brings real value.
