How financial institutions deploy AI without sending regulated data to third parties: isolation patterns, audit needs, and model-risk governance.
Private AI for fintech exists because of a specific constraint: in most regulated financial contexts, customer data cannot be processed by a third party without contractual, jurisdictional, and audit conditions that public API endpoints do not satisfy by default.
The engineering question is not whether to isolate. It is how much isolation the specific obligation requires, because the options differ enormously in cost and capability and teams routinely over-buy.
Four deployment patterns, honestly compared
1. Public API with a data processing agreement. The standard commercial endpoint under enterprise terms — no training on your data, defined retention, contractual security commitments.
Adequate for: internal tooling on non-customer data, code assistance, document drafting, anything where the input contains no regulated material.
Insufficient for: most customer-data processing under strict interpretations, because data still leaves your control boundary and crosses jurisdictions you may not control.
2. Cloud provider's isolated AI service. Models running inside your cloud tenancy — Bedrock, Azure OpenAI, Vertex — where data stays within your account boundary and region.
This is the pattern most fintechs land on, and usually correctly. You get frontier-class models with data residency, VPC isolation, existing compliance certifications, and no GPU operations. Your cloud provider is already a processor under your existing agreements, which is a meaningful contractual simplification.
Limits: you depend on the provider's model availability and deprecation schedule, and specific regulators occasionally reject even in-tenancy processing for particular data classes.
3. Self-hosted open-weight models in your infrastructure. Weights running on your hardware, in your network. Complete control over data flow and model version permanence.
Necessary when regulation forbids processing outside your infrastructure, or when model permanence is contractually required — a validated model that a vendor cannot deprecate.
Costs: GPU capacity, a serving stack, and genuine ML-operations capability. Realistically 0.3–0.5 FTE ongoing plus hardware. Capability trails frontier models on the hardest reasoning tasks, though the gap has narrowed substantially for extraction, classification, and summarisation — which is most of what fintech actually needs.
4. Air-gapped deployment. No external network path. Reserved for genuinely classified environments. Everything — model updates, dependency patches, monitoring — becomes a manual, audited process. Choose this only when explicitly mandated, because the operational cost is severe and permanent.
Matching the pattern to the obligation
The common failure is defaulting to maximum isolation for everything. That produces a slow, expensive programme delivering less value than a tiered approach.
Classify workloads by the data they touch:
Public or internal non-customer data — product documentation, public filings, internal engineering material. Public API under a DPA is fine. Do not spend isolation budget here.
Pseudonymised customer data — transaction patterns with identifiers stripped, aggregate behaviour. Cloud-isolated services are typically appropriate. Verify your pseudonymisation genuinely resists re-identification; transaction data is notoriously re-identifiable through combination, and a weak pseudonymisation scheme means you are in the next tier without realising it.
Identified customer data — names, accounts, balances, KYC material. Cloud-isolated at minimum, self-hosted where the regulator or contract demands it.
Regulated processing with explicit locality requirements — where law or licence conditions specify processing location and control. Self-hosted or air-gapped, with the specific obligation documented.
Most institutions find the majority of their AI use cases fall in the first two tiers, where cloud-isolated services are sufficient. Reserve self-hosting for the cases that genuinely require it and the economics become defensible.
Requirements that apply regardless of pattern
Isolation is necessary and not sufficient. These are demanded in audit and are frequently missing:
Complete inference audit trail. Every request logged with timestamp, requesting identity, input reference, model version, output, and the downstream decision. Not sampled — complete, for the full regulatory retention period. This is a storage and schema decision to make before launch, because retrofitting it means losing history you cannot reconstruct.
Model version pinning with change control. You must be able to state which exact model version produced a decision on a given date, and demonstrate that changes went through review. Auto-updating endpoints are incompatible with this in most regulated contexts — a silent provider-side model change invalidates your validation evidence.
Documented human oversight. For decisions materially affecting customers, a defined human review point with recorded reviewer identity and rationale. "The model suggested and a human approved" requires evidence of genuine review, not a rubber-stamp workflow that approves everything within seconds.
Explainability proportionate to impact. Credit and adverse-action decisions carry explanation obligations in many jurisdictions. A language model producing an unexplainable decision creates regulatory exposure regardless of accuracy. Common resolution: models assist analysts rather than deciding autonomously, with the analyst's reasoning forming the recorded basis.
Bias testing and documented fairness assessment. Where decisions affect credit access or pricing, disparate-impact testing across protected characteristics, repeated on a schedule rather than once at launch.
Prompt injection controls. Any system processing customer-supplied text — support messages, documents, transaction memos — must assume that text may contain instructions. Treat model output as untrusted input to downstream systems, never as authorisation.
Model risk governance
Institutions under model-risk frameworks such as SR 11-7 must treat AI systems as models subject to validation, independent review, and ongoing monitoring.
This is the requirement that most delays fintech AI deployment, and it is usually discovered late. Practical implications:
Documented development process — data lineage, design rationale, alternatives considered.
Independent validation by a party not involved in development, with authority to block deployment.
Ongoing performance monitoring with defined thresholds triggering revalidation.
A defined inventory listing every model in production, its owner, validation date, and risk tier.
Engage model risk in design, not at launch. A system built without validation evidence frequently cannot be retrofitted with it, and the rebuild is more expensive than doing it correctly initially.
Where these projects actually fail
Underestimating the data layer. The model is rarely the constraint. Getting clean, correctly permissioned, lineage-tracked data to the inference point is most of the work. Institutions with fragmented data estates spend the majority of the project here.
Ignoring the retrieval boundary. RAG over internal documents inherits the permission model of those documents — or fails to. If retrieval can surface a document the requesting user is not entitled to see, you have created a data leak with an approachable interface. Permission filtering must occur at retrieval, before content reaches the model, and must be tested adversarially.
Treating output as authoritative. A model producing a plausible account balance is a correctness incident. Anything factual should be retrieved from systems of record, not generated. Use the model for language, not for facts.
No rollback path. Model behaviour changes after deployment as inputs shift. A defined rollback to a previous validated version, exercised in testing rather than assumed, is a basic operational requirement that is routinely absent.
A sequence that works
Classify the workload by data sensitivity, and choose the least isolation that satisfies the actual obligation.
Engage compliance and model risk in design. Their requirements shape architecture and cannot be added afterwards.
Build audit logging first. Before the feature works, before evaluation. It is unrecoverable retroactively.
Start with human-in-the-loop. Model assists, human decides. This satisfies most oversight requirements and generates the performance evidence validation needs.
Measure against a documented baseline — current human performance on the same task, so improvement is demonstrable rather than asserted.
Expand autonomy only where evidence supports it, tier by tier.
Frequently asked questions
Do we need self-hosted models for regulated data? Often not. Cloud-isolated services within your tenancy satisfy most data residency and processing requirements, with existing compliance certifications. Self-hosting is required when regulation mandates processing within your own infrastructure or when model version permanence is contractually necessary.
What is the most commonly missed requirement? Complete inference audit logging. It cannot be reconstructed retroactively, and it is required for the full regulatory retention period — not sampled.
Can we use commercial APIs for any customer data? Depends on jurisdiction, data classification, and your agreements. Many institutions use them for pseudonymised or internal data under a DPA and restrict identified customer data to isolated deployments. Verify your pseudonymisation actually resists re-identification.
How does model risk governance apply? Under frameworks like SR 11-7, AI systems are models requiring documented development, independent validation, and ongoing monitoring. Engage model risk during design — retrofitting validation evidence is frequently impossible.
What about prompt injection? Any system processing customer-supplied text must assume it contains instructions. Never let model output authorise actions directly; treat it as untrusted input, and enforce permissions at retrieval rather than relying on the model to respect them.
Further reading
Related
Written by
Moueen Togarvi
Founder & CEO at Voquarn Code, focused on product engineering, search growth, and practical AI systems.