Security and data residency
What leaves your network, said before you ask.
You are going to find this in review either way. Volunteering it is the only version of this page worth writing.
The short version
- The KV cache stays on your GPU. It is built from your prompt, read and freed on the card, and is never serialised, sent anywhere, or written to disk. This is the derived data that matters most, since it is built from what your users actually type.
- Model weights and inference execution stay on your hardware too. No SovFleet component has a code path that transmits them off your GPUs. The agent itself ships as a GPG-signed release with published checksums, so you can verify what runs there before you run it.
- Your gateway is dedicated to your account. You choose the cloud and region it runs in at account creation, and no other customer's traffic shares it.
- Prompts and responses transit that gateway. They are held in memory for the life of the request and are never persisted, logged, or used for training.
- Metadata does leave and is stored: GPU health, utilisation metrics, token counts, endpoint names and model names.
- Connections are outbound only. No inbound firewall rules, no VPN, no listener on your network.
- A customer-hosted gateway is on the roadmap. With it, nothing transits SovFleet at all. It is not shipped today.
The part most vendors bury
Prompts transit our gateway.
In the architecture we ship today, a request goes from your client to our gateway, and our gateway forwards it to your node agent. That means the prompt, and the response, passes through infrastructure we operate. That gateway instance is dedicated to your account: no other customer's requests share the process, the memory, or the host it runs on.
While it is there: it exists in process memory for the duration of the request. It is not written to disk. It is not written to an access log, an error log, or a trace. It is not sampled for quality review. It is not used to train or evaluate anything. And it is gone when the connection closes.
That is a meaningful set of controls, and it is still not the same thing as the data never leaving. Those are different claims and they get different answers in a review, so we make the weaker, true one.
On the phrase you will see elsewhere
“Your data never leaves your environment” is the line the rest of this category uses. For SovFleet today it would not be true, so it is not on this site.
What is true: your model never leaves your environment, and your prompts pass through one hop that we operate and do not retain.
The exceptions a security review checks for, addressed directly rather than left for you to ask about:
- A gateway process crashes.
- Core dumps are disabled on every gateway host, so a crash cannot write process memory, and therefore prompt content, to disk.
- Memory pressure forces a page to disk.
- Swap is disabled on every gateway host. This is the exception that breaks a “never written to disk” claim silently, with no log line marking it, so we close it instead of asserting around it.
- Someone debugs the gateway in production.
- Attaching a debugger or capturing a heap snapshot against a live gateway requires break-glass access, and break-glass notifies the account it was used against.
- A support engineer is working your incident.
- They see what is in the inventory below: metadata, status codes, latency. Not prompt or response content. There is no support tool that shows it to them.
If that hop is a hard no for your workload, and for some regulated workloads it should be, say so on the first call. We will tell you honestly whether the roadmap timeline works for you, and if it does not, we would rather not waste your procurement cycle.
Data-flow inventory
Every class of data, and where it ends up.
This is the table your security architect is going to build anyway. Here it is already built, and we would rather argue with you about a row in it than about whether it exists.
Model weights
Your hosts, pulled from a registry you nominate
Never transit our infrastructure. If you point the agent at an internal mirror, nothing is fetched from the public internet either.
KV cache
GPU memory on the serving host
Built on the card, read on the card, freed on the card. It is never serialised, never sent anywhere and never written to disk.
Inference execution
Your GPUs
Every forward pass runs on your hardware. There is no remote execution path and no fallback that moves a request off your fleet.
Prompts (request bodies)
Through the SovFleet gateway, in memory
Held for the life of the request so the gateway can route it, then dropped. Not written to disk, not written to logs, not used for training or evaluation.
Responses (completions)
Through the SovFleet gateway, in memory
Streamed back through the same path under the same rules. Response bodies do not appear in our logs, including error logs. A failed request records the status and the reason, not the content.
Request metadata
SovFleet control plane
Timestamp, team, API key ID, endpoint name, model name, prompt and completion token counts, latency, status code. This is what quota enforcement and chargeback are computed from. Retention is fixed at 13 months and is not currently configurable per account.
GPU and node telemetry
SovFleet control plane
Utilisation, memory in use, temperature, ECC errors, driver and engine versions, hostname, node labels. Retention is fixed at 13 months and is not currently configurable per account.
Deployment configuration
SovFleet control plane
Model names, precisions, replica counts, node labels, quota allocations. These reach our control plane as routing metadata, the same as a hostname would: if a name would disclose something on its own, do not put that in the name.
API keys
SovFleet control plane
Stored as a salted hash. The full value is shown once at creation and cannot be retrieved afterwards. The visible prefix exists so you can identify a key in an audit log without holding it.
Console user identities
SovFleet control plane
Email address, display name, and the subject identifier from your IdP. No password if you use SSO, because we never receive one.
Audit log
SovFleet control plane
Who changed which deployment, quota, key or policy, and when. Retention is fixed at 24 months and is not currently configurable per account. Exportable to your SIEM.
Network posture
Outbound only
The agent dials out on 443 to the control plane and holds the connection open. Nothing dials in. There is no inbound rule to request from your network team, which for most banks is the difference between a two-week change and a two-month one.
Egress allowlist
Two hostnames: the control plane, and whichever registry you nominate for weights. If your registry is internal, the second one never leaves your network. Both are configurable and both are documented in the agent config.
Mutual TLS
The agent authenticates with a certificate issued at enrolment and rotated automatically. A stolen enrolment token is single-use and short-lived; a stolen agent certificate can be revoked from the console and takes effect on the next connection.
TLS 1.3 in transit
Client to gateway, gateway to agent, agent to control plane. No plaintext hop anywhere in the path, including inside our own infrastructure.
No inbound path to your GPUs
The engine binds to localhost on the serving host. It is not reachable from your own network either, let alone ours, which removes a whole class of lateral-movement questions.
Your choice of cloud and region
The gateway can run in AWS, Azure or GCP, in the region you choose, and it is dedicated to your account rather than shared. Requests are only ever routed to that instance. Metadata is stored in the same region. Choose it before the trial rather than after.
Access and key management
Three roles, not one flat login
Admin holds full configuration access. Operator can create services, adjust quota and manage user access, but not billing. Developer can use existing quota and services, but cannot create or change them. Every console user gets one of the three.
Encrypted at rest
Metadata, audit logs and deployment configuration are encrypted at rest using our cloud provider's managed keys. That is provider-managed encryption today, not a key you hold yourself.
API keys are hashed
We store a salted hash, not the key itself. The full value is shown once at creation and cannot be retrieved afterwards. A compromise of the control plane's database does not hand out usable keys.
Compliance
What we do not have.
No certifications, stated plainly
SovFleetdoes not hold ISO 27001, ISO 27017, SOC 2 Type I or Type II, PCI DSS attestation, or any other certification. We are not “in the process of” anything we can put a date on, and we are not going to imply a pending audit to get through your first gate.
We know what that costs us. It rules us out of some procurement processes outright, and it should. A bank with a hard ISO 27001 gate is not doing anything unreasonable by applying it.
Also not done yet: an independent penetration test, a published vulnerability disclosure policy, and a software bill of materials for the agent. If any of those is a hard requirement, say so and we will tell you honestly where we stand against it rather than let you find out in review.
What we can give you instead, today, on request:
- A complete data-flow inventory, which is the table above, with anything your reviewer wants added to it
- The current subprocessor list, by name and by function, with the region each operates in
- Our architecture, including the parts that are awkward, in a call with an engineer rather than a questionnaire team
- Answers to your own security questionnaire, with “no” written in the boxes where the answer is no
- A written commitment on breach notification timelines in the contract, which is enforceable in a way a badge is not
If a certification is a hard requirement rather than a preference, tell us on the first call. We will say so if we cannot meet it and neither of us will spend a quarter finding out.
Roadmap
The customer-hosted gateway.
The shipped architecture has one hop we would rather not have. The fix is to let you run the gateway yourself, inside your own network, so that prompts never cross a boundary you do not control. The control plane would keep doing what it already does: inventory, quota, policy, metrics, over the same metadata-only channel.
We are not putting a date on this page, because a date on a marketing site is a promise made to someone who cannot hold us to it. Ask on a call and you will get our current honest estimate, including the confidence attached to it.
What we will commit to in writing: if you sign while the hosted gateway is the only option, moving to the customer-hosted one when it ships is a configuration change, not a migration, and it is not a paid upgrade.
For your RFI
The operational questions, and where they are answered.
Most of these are already documented, just not on this page. This is the index, so your platform team does not have to go looking.
What privileges does the agent need on the host?
No root beyond what your container runtime already requires, and no kernel module.
InstallationWhat egress do we have to allow?
Two hostnames: the control plane, and whichever registry you nominate for weights. Both configurable, and the second can be internal.
Agent configWhat happens when a component fails?
Node death, control plane unreachable, gateway unreachable, registry unreachable, quota exhausted and a full fleet, each answered separately.
Failure modesCan we run this air-gapped?
No. The agent needs outbound connectivity, so a genuinely air-gapped fleet is a case where self-managed vLLM wins outright.
Where not to buyWhat does the audit log capture, and can we export it?
Who changed which deployment, quota, key or policy, and when. Exportable to your SIEM, with a fixed 24-month retention.
Data inventoryCan we pin the cloud and the region?
You choose both at account creation, and the gateway instance is dedicated to your account, not shared. Metadata is stored in the same region. Tell us the jurisdiction before the trial rather than after.
Network postureWhat's the availability commitment on the gateway?
99.95%, best-effort today rather than backed by service credits. There is no public status page yet.
Network postureWhat are your support hours and response time?
9am to 9pm IST, Monday to Saturday, with a 4-hour response target on production-down issues.
PricingNot published here yet
Whitepaper 02 · Security and compliance
Four Questions, Not One
"Does our data leave?" is the question every AI security review asks, and it is too coarse to have a useful answer. A data-flow reference for RBI-regulated institutions, with a template you can hand any vendor.
CISO, security architect, compliance and risk, CIO · 16 minutes read
Questions from actual security reviews
- Do prompts leave our network?
- Yes. Prompts and responses transit the SovFleet gateway, which runs in our cloud, on their way to and from your node agent. They are held in memory for the life of the request and are never written to disk, never logged and never used for training. Model weights, the KV cache and inference execution never leave your hardware.
- Can we run the gateway ourselves so nothing transits SovFleet?
- Not today. A customer-hosted gateway is on the roadmap and is the single most requested item on it. Until it ships, the transit described above is unavoidable, and we would rather you knew that in the first meeting than discovered it in review.
- What inbound firewall rules do we need?
- None. The node agent opens an outbound TLS connection on port 443 to the control plane, and the gateway reaches your nodes over that existing connection. There is no listener on your side, no inbound rule and no VPN.
- Do you hold SOC 2 or ISO 27001?
- No. We hold no certifications today and we will not imply otherwise. We can give you the architecture, a data-flow inventory, our subprocessor list and direct answers from an engineer, which is what an early-stage vendor honestly has to offer.
- Where does the gateway run, and can we pin the region?
- You choose the cloud, AWS, Azure or GCP, and the region at account creation. That gateway instance is dedicated to your account, not shared with any other customer, and requests are only ever routed to it. Metadata is stored in the same region. Tell us the jurisdiction you need before the trial rather than after.
- What happens to our data if we leave?
- Weights and KV cache were never ours to delete. They are on your hardware and stay there. Metadata, telemetry and audit logs are deleted on request within 30 days, and exportable before that.
If your question is not here, it is probably a good one. Send it to us and it will be routed to the engineer who owns that part of the system.
Bring your security architect to the first call
Not the second one. The awkward questions are cheaper to answer now, and if the answer rules us out, both of us would rather know in week one.
An engineer replies within one working day. No qualification call before the technical one.