The right question isn’t which option is more secure. It’s which perimeter you’re able to govern and demonstrate.
The wording matters. “More secure” suggests an absolute hierarchy that doesn’t exist: an internal infrastructure managed without dedicated expertise is less secure than a service run by a specialized operator, and an excellent service located outside the applicable regulatory constraints isn’t usable regardless of its technical quality.
The choice is one of governance, not technology preference.
The three possible configurations
Public cloud. The model is delivered as a service by a provider, and data flows to its infrastructure. It’s the fastest configuration to stand up, with the most immediate access to the latest models and costs tied to actual usage.
The points to check are where processing takes place, the contractual terms on data use, how long logs are retained on the provider’s side, and the rules that apply if a foreign authority requests access.
Dedicated or sovereign cloud. The service runs on reserved infrastructure, with data location contractually guaranteed and isolation from other customers. The cost is higher, the flexibility lower, the perimeter far more definable.
In our experience, this is the configuration that resolves most cases in banking, insurance and the public sector, when the constraint is about data residency more than physical control of the hardware.
On-premise. Models run on the organization’s own infrastructure. The data doesn’t leave the perimeter.
It’s the configuration with the maximum control and the maximum burden. It requires specialized hardware, operational expertise, a plan for updating models, and capacity planning that has to be done in advance. With the open-weight models available today, the achievable quality is adequate for many document-based use cases, though it generally remains behind proprietary frontier models on the most complex tasks.
The real decision criteria
Four criteria, in the order it makes sense to apply them.
The nature of the data. Not of the project: of the data. A system working on publishable technical manuals has different constraints from one handling health data or credit files. The first task is to classify the data the system will touch, including information that appears incidentally in attachments.
Regulatory and contractual constraints. Sector requirements, data-location rules, obligations toward the supervisory authority, clauses already agreed with your own customers. This is an assessment to run together with the legal function before the technical evaluation, because in some cases it directly rules out one or two configurations.
Internal expertise. An on-premise architecture requires stable, ongoing oversight. If that oversight doesn’t exist and there’s no plan to build it, the configuration that looks most controlled on paper becomes the least secure in practice: systems left unpatched, logs left unmonitored, dependency on a single external vendor.
Lifecycle cost. Not the initial cost. Hardware, energy, model updates, staff, evolving workloads. Over a three-year horizon, the comparison between configurations often flips relative to a first-year assessment, in either direction depending on volumes.
Hybrid architectures
In most real projects the choice isn’t binary, and that’s the most useful part of this discussion.
A hybrid architecture separates components based on the sensitivity of the data they handle. Confidential documents stay within the internal perimeter, together with indexing and search. The model that generates the final answer can be internal or external depending on the use case.
A common variant classifies the request automatically: questions involving sensitive data are routed to the internal model, others to the external service. This requires reliable, verifiable routing logic, and it needs careful design, because a misclassification has immediate consequences.
Another approach keeps identifying data internal and sends only pseudonymized content externally. It works for some use cases and not others: pseudonymization needs to be verified, not assumed, because the remaining context can be enough to re-identify.
The general rule is that deployment follows the organization’s constraints, not the other way around. When data residency isn’t negotiable, the architecture stays internal or hybrid, and the downstream choices adapt.
The questions to ask a vendor
Six questions, to ask before deciding and to put in writing.
Where does the data physically reside during processing and afterward, and for how long is it retained.
Is the data used to train or improve models, whether the vendor’s own or a third party’s, and on what contractual basis.
What gets logged on the vendor’s side, who can access it, and through what process.
How is effective deletion of a piece of data achieved, in what timeframe, and with what evidence.
What happens if the service is discontinued or the model retired: what notice is guaranteed and what migration path exists.
What rules apply if a third-country authority requests access.
If an answer is vague, the question needs to be put in writing again. At the contract stage, the difference between “data is not used for training” and “data is not used for training unless the service is configured otherwise” is substantial.
What the infrastructure choice doesn’t solve
No configuration protects against poorly designed permission management. An internal system that shows a user documents they wouldn’t have been authorized to open constitutes an incident, regardless of where the server sits.
No configuration protects against application-level vulnerabilities. Defending against prompt injection, treating retrieved content as untrusted input, controlling which actions can be executed: these are application architecture topics, not infrastructure ones.
And no configuration replaces the audit trail. Knowing the data stayed internal doesn’t tell you who accessed what.
In closing
The decision should come after classifying the data and checking the applicable constraints, not before. In our experience, when that order is respected, the right configuration emerges with little ambiguity.
If you’re evaluating a project involving sensitive data, we’re available for a conversation on security requirements and perimeter: what data is involved, what constraints apply, and what architecture would hold up under review.
