In an AI system the first release is the cheapest part. Models change, documents change, processes change.
This asymmetry is well known across software in general, but it’s more pronounced in systems built on language models. A traditional application, if nobody touches it and the environment stays stable, keeps behaving the same way. An AI system anchored to documents and to models supplied by third parties degrades even while standing still.
The decay is gradual and doesn’t produce obvious errors. That’s why it gets recognized late.
The three recurring line items
Model updates. Vendors release new versions and retire older ones, with notice periods that rarely match the organization’s timeline. A model change isn’t a transparent swap: behavior varies, and instructions calibrated on one version can produce different results on another.
Every change requires rerunning the evaluation tests and, generally, retuning the instructions. On a production system in a regulated context, it also requires updating the technical documentation and logging the active version.
Those running open-weight models on their own infrastructure have more control over timing, but take on the burden of updating and sizing it themselves.
Updating the document base. Circulars change, procedures get revised, manuals get new revisions, new acts enter the archive every day.
If indexing isn’t automatic and continuous, the base drifts out of alignment. The symptom is characteristic: the system correctly answers what was true six months ago. Nobody notices until a dispute comes in on a specific case.
On top of this comes version governance, which is an organizational activity: someone has to establish and maintain which document is current.
Process evolution. A regulation changes, an office reorganizes, a new type of case is introduced. The system was designed around the previous process.
This is the hardest line item to predict and the one that generates the heaviest intervention requests. A system that doesn’t keep up with process evolution gets progressively bypassed by operators, who fall back to the previous way of working for new cases.
Why an unmanaged system gets worse
The decline isn’t a figure of speech. It has three measurable causes.
The document base drifts out of alignment with reality, and accuracy drops on everything that’s changed since the last indexing run.
Exceptions pile up. Every unforeseen case the system handles badly generates an informal workaround. After a few months, the share of work that goes through the system has shrunk without anyone deciding it should.
Permissions drift out of alignment. People change roles, offices reorganize. If rights don’t automatically follow the sources, you end up with both improper access and an inability to reach documents that are actually needed.
There’s also a less technical phenomenon. If nobody collects and analyzes the outputs corrected by operators, the most valuable information about how the system actually behaves gets produced every day and lost.
Sizing internal oversight
Oversight doesn’t necessarily require a dedicated structure, but it does require time that’s assigned and recognized.
There are five recurring activities: periodic accuracy checks against the test set, collecting and analyzing discarded outputs, updating the index when the base changes, handling reports from users, and vendor relations for future changes.
The first three are largely automatable in execution; analyzing the results is not.
Sizing depends on the number of systems, the document volume and the criticality. In our experience, for a single production system on a defined process, this amounts to a stable fraction of one person’s time, with spikes around model or regulatory changes. That figure still needs to be estimated case by case during the project, not assumed.
Two recurring mistakes in sizing. Assigning oversight to someone already at capacity, with the result that checks get postponed. And assigning it to someone who doesn’t know the process: evaluating a wrong output requires domain expertise, not just technical skill.
What to put in the vendor contract
Seven elements, to define before signing.
Handling of model changes: who detects them, with what notice, who runs the new evaluation, and under what commercial terms.
The frequency and method of document updates, distinguishing what’s automatic from what requires intervention.
Service levels on accuracy, with the metric, the reference test set and the review cadence. A service level expressed only in terms of availability says nothing about the quality of the answers.
The boundary between corrective and evolutive maintenance, with an operational definition that avoids the recurring dispute over where a defect ends and a new requirement begins.
Ownership of data, indexes, configurations and instructions, and the format they’re returned in at the end of the contract.
Exit terms: notice period, migration support, duration of the handover period.
The documentation to keep up to date and who’s responsible for it.
What maintenance doesn’t solve
It doesn’t make up for inadequate architecture. A poorly designed system needs more maintenance and stays fragile.
It doesn’t replace document governance. If the organization doesn’t establish which version is current, no update frequency fixes that.
And it doesn’t justify a marginal project. If the expected benefit is modest, the recurring cost of oversight can wipe it out. That’s a legitimate reason not to do a project, and it should be assessed beforehand.
In closing
Maintenance needs to be estimated alongside development and budgeted from the start, with a horizon of at least three years. It isn’t a minor line item: in many projects it exceeds the initial cost over the system’s lifecycle.
When evaluating a project, the question to ask alongside “how much does it cost to build” is “how much does it cost to keep running, and who will do it.”
If you’re evaluating a project or already have a system in operation, we’re available for a conversation about sizing the oversight effort and what’s worth defining in the contract.
