Product

What installs where, and what each part can see.

Three components. Two of them run in our cloud and hold no model data. One runs on your machine and holds all of it. The rest of this page is the detail behind that sentence.

The gateway is drawn outside your boundary because that is where it runs today. See security and data residency for what that means in review.

Component by component

Each Atlas component, where it runs, what it can see and what network access it needs.
Component	Runs	Can see	Never sees	Network
Control plane	Atlas cloud	Fleet inventory, model catalogue, quotas, metrics, audit log	Prompts, responses, weights, KV cache	Accepts the agent's outbound connection
Gateway	Atlas cloud	API keys, request and response bodies in memory, token counts	Weights, KV cache; nothing written to disk	Accepts client HTTPS; reuses the agent's connection outbound
Node agent	Your machine	Everything on that host: weights, prompts, responses, KV cache	Other customers' anything — it is a single-tenant process	Outbound 443 only. No listener, no inbound rule
Engine (vLLM)	Your machine	Weights, KV cache, the forward pass	Any network destination outside your host	Localhost only

The request path

Four hops, two of them inside your network.

1
Client to gateway
Standard HTTPS to your subdomain. The request is an OpenAI-format chat completion; nothing about it is proprietary to us.
2
Gateway resolves and routes
The key resolves to a team. The team's quota is checked, the model name is resolved to a running deployment, and a node with a free sequence slot is chosen. If none is free, the request queues or is rejected according to your policy.
3
Gateway to node agent
Over the connection the agent opened outbound. There is no inbound listener on your side and nothing to allow through a firewall.
4
Agent to engine
Localhost to the vLLM process. Prefill, then decode. Tokens stream back along the same path, and the KV cache stays where it was built.

On latency

Two of those hops cross the public internet, and that costs you round-trip time. How much depends entirely on where your GPUs sit relative to the gateway region, so we do not publish a number — anyone who does is quoting their own lab. Measure it on your own network during the trial; it is the first thing we help you set up.

Installation

One binary, one token, one config file.

The agent is a single static binary. It needs a host with the NVIDIA driver and container runtime you already use, outbound 443, and a registry it can pull weights from. That registry can be yours — an internal mirror, an S3-compatible bucket, or a filesystem path where the weights already sit.

It does not need root beyond what your container runtime already requires, and it does not install a kernel module.

on each GPU host

curl -fsSL https://get.atlas.example.com/agent | sh
atlas-agent enroll --token ATL_ENROL_...
systemctl enable --now atlas-agent

/etc/atlas/agent.yaml

control_plane: https://control.atlas.example.com

# Where weights come from. Point this at your own mirror and
# nothing is fetched from the public internet.
registry:
  type: s3
  endpoint: https://artifacts.internal.acme.net
  bucket: model-weights

# Which cards this host offers, and to whom.
resources:
  gpus: [0, 1, 2, 3]
  labels:
    zone: dc-fra-1
    compliance: pci

# Egress allowlist, if your policy requires one.
egress:
  allow:
    - control.atlas.example.com:443
    - artifacts.internal.acme.net:443

Placement

How a model ends up on a particular card.

You declare a deployment: a model, a precision, a context length, a target number of replicas, and which node labels it may land on. The control plane solves for placement using the same arithmetic the capacity calculator uses — weights first, then whether the remaining memory holds enough KV cache to be worth running.

Labels are how compliance boundaries get enforced in practice. A deployment marked compliance: pci is only ever placed on hosts carrying that label, and the scheduler will leave a GPU idle rather than violate it.

Tensor-parallel groups are placed on cards within a single host unless you explicitly allow otherwise, because cross-host TP over anything short of InfiniBand is usually a mistake.

deployment

model: meta-llama/Llama-3.1-70B-Instruct
quantization: int4-awq
max_model_len: 32768

placement:
  requires:
    zone: dc-fra-1
    compliance: pci
  tensor_parallel: 4
  replicas: 2

quota:
  teams:
    fraud-ml:      { reserved_slots: 48 }
    support-tools: { reserved_slots: 24 }
  overflow: fair-share

Failure

What breaks, and what happens when it does.

Any review worth passing asks this. Here it is without being asked.

A GPU node dies

In-flight requests on that node fail and are not retried automatically, because a partially streamed completion cannot be safely replayed. New requests route to the remaining replicas. If that drops you below your reserved slots, the affected teams are told which ones.

The control plane is unreachable

Running deployments keep serving. The agent holds its last known configuration and continues. What you lose is the ability to change anything — new deployments, quota edits, and placement decisions all wait.

The gateway is unreachable

Requests fail. This is the honest answer and the sharpest edge in the current architecture: there is no direct client-to-agent path today. It is the same failure surface as any hosted API, and it is the main thing the customer-hosted gateway on the roadmap is meant to remove.

Your registry is unreachable

Already-loaded models keep serving from GPU memory. New model pulls fail with the registry error rather than a generic one, because a scheduling failure that does not name the cause is worse than useless.

A team exhausts its quota

Requests beyond the reservation queue or receive a 429 with a Retry-After, depending on the policy you set. Other teams are not affected — that is the point of the reservation.

The fleet is genuinely full

Requests queue by policy. Queue depth and wait time are recorded per team, which is the evidence you will need if the answer turns out to be more hardware.

Scope

What this does not do.

It does not train or fine-tune. Serving only. Bring the weights you have.
It does not supply GPUs. There is no marketplace and no burst-to-our-cloud option. If your fleet is full, it is full, and we will show you the evidence for that rather than sell you capacity.
It does not write a custom inference engine. It supervises vLLM. When vLLM ships a faster kernel you get it by upgrading the engine, not by waiting for us to reimplement it.
It is not a RAG framework, an agent framework, or a prompt manager. It terminates an OpenAI-compatible API and puts tokens on GPUs.

Install the agent on one node

A demo is thirty minutes and uses one of your machines. We enrol a node, deploy a model, point a client at it, and you watch the traffic path with your own tooling.

Request a demo Security and data residency

Priya Raghavan replies within one working day. No sequence, no SDR, no calendar tennis.