Private AI is a set of choices, not a single setting.
Organisations want AI close to their data for good reasons: confidentiality, regulation, latency or simply control. But “private” can describe several different architectures, each placing the boundary somewhere else and moving different work onto your team.
We help choose between them, write the data-flow boundary down and test it — without promising more isolation than the design actually provides.
01Four deployment options
Where processing happens in each option.
The same application can call a model in very different places. What changes is which parts of the system sit inside infrastructure your organisation controls.
Managed service, enterprise terms
- Your environmentUsers & application
- Your environmentYour data & retrieval
- ProviderModel inferenceProvider's shared service
- ProviderRequest logsRetention per contract
Dedicated tenant or private cloud
- Your environmentUsers & application
- Your environmentYour data & retrieval
- ProviderModel inferenceIsolated capacity, region you choose
- ProviderRequest logsPer tenancy terms
Self-hosted open-weight model, your cloud
- Your environmentUsers & application
- Your environmentYour data & retrieval
- Your environmentModel inferenceIn your cloud account
- Your environmentLogs & monitoringRun by your team
On-premises
- Your environmentUsers & application
- Your environmentYour data & retrieval
- Your environmentModel inferenceYour hardware, your facility
- Your environmentLogs & monitoringOn your own servers
- Your environment
- Provider
02Trade-offs
More control means more to operate.
Hybrid designs are common — for example, documents and retrieval stay in your environment while selected, less sensitive requests use a managed model.
| Consideration | Managed service | Dedicated tenant | Self-hosted in your cloud | On-premises |
|---|---|---|---|---|
| Data boundary | Prompts leave your environment under contract, retention and region terms | Isolated capacity in a chosen region, still operated by the provider | Stays in your cloud account | Stays in your facility |
| Model choice | Widest, including the largest proprietary models | The provider's catalogue for that offering and region | Open-weight models you are licensed to run | Open-weight models that fit your hardware |
| Hardware | None to buy | None to buy; capacity may be reserved | Cloud GPUs you rent and size | GPUs you buy, house, power and cool |
| Latency | Network round trip; shared capacity can vary | More predictable with reserved capacity | Depends on instance type and region | Shortest network path; bounded by local hardware |
| Operations | Provider runs the model; you run the application | Shared: the provider runs the model service | Your team, or a partner under contract | Your team, including the hardware |
| Patching & monitoring | Model updates managed by the provider | Mostly provider-managed | You patch runtimes and models and watch capacity | You patch everything from firmware up |
| Cost profile | Pay per use; low fixed cost | Higher committed spend | Instances cost money busy or idle | Capital up front; sized for peak demand |
03How it works
From data classification to a tested boundary.
Classify the data
Agree which data the system will touch and how sensitive each class is. That — not a general preference — decides which deployment options are acceptable.Choose the option and the model
Match the acceptable options against the quality the task needs, tested on representative cases, and against the hardware or budget available.Write the boundary down
Document every data flow: what enters, where it is processed and stored, what is logged, which outside services are called — including updates and telemetry — and who can reach each part.Build with controls
Network segmentation, identity integration, encryption in transit and at rest, and logging with retention rules agreed with your IT and security teams.Test the boundary
Check the running system against the written flows — outbound connections, access paths, where logs land — before real data is introduced.Agree how it is operated
Who patches runtimes and models, who watches capacity and quality, and how updates and incidents are handled — in writing, before go-live.
04Limits and security
What we will — and will not — promise.
Trade-offs to plan for
- Open-weight models that fit modest hardware can be weaker than the largest hosted models on some tasks; quality is tested on your cases before committing.
- Self-hosting moves availability, scaling and upgrades onto someone in your organisation, and that effort needs a budget.
- GPU supply, power and cooling can set the timeline for on-premises work.
Security implications
- No design is described as “100% secure” or as guaranteeing that no data ever leaves; every statement is scoped to the architecture that was verified.
- Isolation is only as strong as its weakest flow: update channels, telemetry, vendor support access and backups are all part of the boundary.
- Model files and the software that serves them have a supply chain, and get the same scrutiny as any other software you run.
- Private is not the same as open to everyone inside: access control still applies within your network.
05How success is measured
A boundary that is verified, and a system that is good enough.
Quality is evaluated the same way whichever option is chosen — see evaluation and governance.
- Boundary verification
- Observed network flows and access paths match the written design.
- Quality parity
- The chosen model meets the task's bar on the same evaluation set used to compare the options.
- Performance
- Latency and throughput at expected peak load, not only on a quiet afternoon.
- Operability
- Patching, backup, restore and failover rehearsed rather than only documented.
- Cost
- Actual running cost against the estimate, including idle capacity.
06Related
Related services.
Next step
Start with the data, not the hardware.
Tell us which data the system would touch and the constraints you work under. The right deployment option follows from those answers.