Futura AI
it
All articles
  • Architecture
  • Method

Agentic AI: when a system stops answering and starts acting

An assistant suggests, an agent acts: it reads, writes, decides the next steps. The question that matters isn't what an agent can do, but where it has to stop.

by Daniele Grotti6 min readUpdated on
Futura AI — Agentic AI: when a system stops answering and starts acting

An assistant suggests. An agent performs actions: it reads a document, queries a case-management system, fills in a file, opens a ticket.

That’s the difference between a system that produces text and a system that changes the state of something. The potential value changes, and, to a much greater degree, so does the risk.

The term is widely used today. It’s worth defining it operationally, because the distance between the theoretical formulation and what can actually be put into production in an organization with formal accountability is considerable.

An operational definition, without the hype

An agent is a system that, given a goal, autonomously decides which steps to take to reach it, using the tools it has been connected to.

Three elements make the definition concrete.

It doesn’t receive a question, it receives a task. Not “which documents are missing from this file” but “check this file for completeness and request what’s missing.”

It doesn’t execute a fixed sequence. It decides step by step, based on what it finds. If the document is unreadable, it takes a different path than when it’s readable.

It produces effects outside itself. It writes to a system, sends a communication, updates a status.

The third point is the one that changes everything. An assistant that gets something wrong produces an answer to correct. An agent that gets something wrong produces an action to undo, and not all actions are reversible.

The three ingredients

The available tools. An agent can only do what the tools it’s connected to allow. Reading a certified-email inbox, querying a database, writing a record, generating a document, opening a ticket.

Tool design is the first level of control, and the most effective one. A tool that only allows reading can’t cause direct harm. A tool that writes requires validation, limits and reversibility. Many choices that look like application security are really choices about which tools to expose and with what parameters.

Context memory. To carry out a multi-step task, the agent has to remember what it has already checked, what information it has gathered, and which attempts have failed. Without this continuity, it repeats the same operations or loses track on complex cases.

Memory has to be designed: what to keep, for how long, with what visibility. In a context with sensitive data, this is also a compliance question, not just a functional one.

Stopping criteria. When the agent has to stop. Because the task is complete, because it has hit a limit on attempts, because the confidence level is below the threshold, because it’s about to perform an action that requires human authorization.

An agent without stopping criteria isn’t an autonomous system: it’s a system whose behavior in unforeseen cases is unknown.

The hard part isn’t getting it to act

Building an agent that carries out a task in a favorable case takes relatively little effort today. The technical tools are mature and well documented.

The hard part is deciding where it has to stop.

That decision isn’t technical. It requires establishing, together with whoever owns the process, which actions can be executed without confirmation, which require authorization, and which remain entirely human. And it requires defining what happens in doubtful cases.

In our experience, this is the part of the project that absorbs the most analysis time, and the one that determines whether the system will be usable.

Three criteria help set it up.

The reversibility of the action. A wrong classification can be corrected. A communication sent to a citizen can’t.

External impact. Anything that leaves the organization — a certified email, a quote, a notice of outcome — deserves a higher control threshold than what stays internal.

The confidence level. The system has to be able to state that it doesn’t have enough to go on. An agent that completes every task, always, isn’t a reliable agent: it’s an agent that doesn’t know how to recognize its own limits.

An example on an administrative process

An agency receives applications by certified email. The process, from arrival to intake, today involves several manual steps.

An agent designed around this workflow operates as follows.

It reads the incoming communication and classifies its type. It extracts the identifying data and cross-checks it against the records in the case-management system. It checks which attachments are required for that type and which are actually present. If a document is missing, it drafts the request for additional documentation. It logs the file in the system with the correct status and produces a summary for the case officer, with a reference to the exact point in the document each piece of information comes from.

And then it stops.

Sending the request for additional documentation requires human confirmation, because it’s a communication going outside the organization. Assessing the merits isn’t the agent’s job. Cases where the classification is uncertain are routed to an operator instead of being resolved automatically.

The value isn’t in the autonomy. It’s in the fact that the operator receives a file that has already been read, checked and structured, and spends their time on the decision instead of on gathering information.

What an agent doesn’t solve

It doesn’t make up for an undefined process. To decide where the agent has to stop, you need to know how the process works, and in several cases that understanding has to be reconstructed first.

It doesn’t eliminate the work of checking: it concentrates it on the cases that deserve it. That’s a real gain, but it has to be communicated correctly to the people involved, because the promise of total automation creates expectations the system won’t meet.

It’s not suited to processes where every case is an exception. Repetitiveness is what makes the cost of design and maintenance sustainable.

And it doesn’t reduce security requirements. A system that performs actions has a wider attack surface than one that only answers: content retrieved from emails or external documents has to be treated as untrusted input, and adversarial testing is part of validation, not of after-the-fact monitoring.

In closing

The right question before starting a project of this kind isn’t what the agent can do. It’s which actions you’re willing to delegate, with what controls and what possibility of intervention.

The architecture follows from that answer. And it’s an answer that belongs to the organization, not to the vendor.

If you have a process in mind with repetitive steps and defined rules, we’re available for a conversation about that case: what tools it would take, where the stopping points would sit, and what part of the work would stay human.

If this topic touches a real process in your organization, let's talk about it with a focused AI Assessment.

Request an AI Assessment