Is Atlas faster than running vLLM myself?

No. Atlas runs vLLM, so single-model throughput and latency are whatever vLLM gives you on that hardware. What changes is utilisation across a fleet with several models and several teams, because idle capacity gets allocated rather than sitting behind someone's static port assignment.

When should we not buy this?

One model, one team, one owner who enjoys the work, and no compliance boundary between users — keep running vLLM. You would be paying for multi-tenancy you do not have and quota you do not need. The same is true for a genuinely air-gapped deployment, which Atlas does not support today.

How long would it take us to build this ourselves?

The routing proxy is a week. The quota system is a month. The placement solver, the per-team observability, the engine version management and the audit trail are the part that never quite finishes, because each one is a quarter of somebody's time forever rather than a project with an end date.

Your weights are on your hardware in a standard format, your clients speak the OpenAI API, and your deployments are YAML in your own repository. Turning Atlas off leaves you with GPUs running vLLM, which is where you started. That is deliberate — a platform whose main retention mechanism is difficulty of exit is not a platform we would want to sell.

vLLM production deployment

You could just run vLLM yourself.

Many of you already do, and it works. This page is about the specific point at which it stops working, what you would have to build to get past it, and the cases where the right answer is to keep doing what you are doing.

First, the true part

vLLM is very good and you were right to use it.

Continuous batching and paged attention were the two ideas that made self-hosted inference practical, and vLLM shipped both. A single command gets you an OpenAI-compatible server that saturates an H100 properly. It is actively maintained, the kernels are current within weeks of a paper, and it is free.

If someone on your team stood this up in an afternoon and it has been serving production traffic since, that was good engineering and nothing on this page is an argument against it.

We run vLLM too. Every throughput number you would get from us is a vLLM number. We are not a faster engine and we are not going to claim to be one.

# This is genuinely most of the way there for one model.
vllm serve meta-llama/Llama-3.1-8B-Instruct \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.90

Qualification, in the other direction

Do not buy this if any of these describe you.

We would rather lose the deal in week one than in month four.

One model, one team

There is no multi-tenancy to manage, so there is nothing here for you to buy. Come back when the second team asks.

You are air-gapped

Atlas needs outbound connectivity from the agent. If your fleet has none, we cannot serve you today and we will tell you that on the first call.

You have a hard SOC 2 gate

We do not hold it. If that is a requirement rather than a preference, we fail your review and both of us should find out now.

Prompts cannot leave your network at all

They transit our gateway today. The customer-hosted gateway that fixes this is not shipped. If that is a blocker, it is a real one.

You are chasing a benchmark number

Tune vLLM yourself. Full control of every flag will beat a managed default, and it should.

Your fleet is one card

Placement across one GPU is not a problem worth paying to solve. Use the calculator, run vLLM, spend the money on the second card.

Where it turns

Nothing breaks at model two. Everything breaks at model five.

The second team

The first team owned the box, so capacity questions were internal. The second team turns every capacity question into a negotiation, and there is no shared unit to negotiate in. In practice this gets settled in a channel by whoever asks loudest.

The fifth model

Four models on eight cards has a hand-drawn answer. Five models, two of which need tensor parallelism, one of which is only allowed on the PCI-labelled hosts, does not. The placement is a constraint problem, and constraint problems solved by hand drift silently.

Quota between teams

Someone eventually writes a proxy with a token bucket. It works. Then a team needs a burst allowance for a batch job, and another needs a floor guarantee during market hours, and the proxy grows a policy engine nobody owns.

Who gets capacity when it is full

Without an admission policy, a full fleet degrades for everyone at once: latency climbs, timeouts fire, and every team files the same ticket on the same afternoon. The answer is not more GPUs, it is deciding in advance who yields.

Version pinning

A researcher pins an engine version to keep a benchmark comparable. Now the fleet cannot be upgraded without invalidating their work, so it is not upgraded, and six months later you are three releases behind on kernels you would actually benefit from.

The bus factor

The most common way a working self-hosted stack fails is that the person who built it changes jobs. The flags are in their shell history. The reason for --enforce-eager on one node is in their head. This is the risk that never makes it onto the slide, and it is the one that actually bites.

Job by job

The comparison, with the losses included.

Two of the rows below say DIY is better. One says it wins outright. A table where the vendor wins every row is not a comparison, it is a brochure.

Job-by-job comparison of self-managed vLLM against Atlas, including the jobs where self-managed vLLM is the better choice.
The job	vLLM, self-managed	Atlas
Serve one model quickly	Excellent. One command, continuous batching, paged attention. Genuinely one of the best pieces of infrastructure software of the last few years.	Identical, because we run vLLM. We make this no faster and we will not pretend otherwise.
Squeeze the most out of one GPU	Excellent, and better than us if you tune it. Full access to every flag, every engine version, every experimental kernel the day it lands.	We expose the same flags, but a managed default is a default. If your job is to win a benchmark, tune it yourself.
Add a second model	Fine. A second process, a second port, a second entry in the load balancer, a second systemd unit.	A line in a deployment file. Marginal difference at n=2, which is why n=2 is a bad moment to buy anything.
Add a fifth model across three teams	This is where it turns. Which model is on which card, who may call it, what happens when two teams want the same card, and whose job it is to know.	Placement is solved against declared constraints. Teams get keys and quotas rather than hostnames.
Stop one team starving another	You build it. Usually a proxy in front, a token bucket, some Redis, and a policy nobody wrote down. It works until the policy needs to change.	Reserved sequence slots per team, with an overflow policy. Teams see their own ceiling rather than a timeout.
Answer 'why was it slow at 4pm'	Possible. Prometheus, Grafana, and a week of building the dashboards that make vLLM's metrics mean something per team.	Queue depth, wait, admission and preemption by team and model, without you building it.
Bill it back to cost centres	You build it. Token counts per key, mapped to teams, exported monthly.	Included, because the gateway is already counting for quota.
Upgrade the engine without fear	Hard once anyone depends on a pinned version for a benchmark. Usually deferred, sometimes for a year.	Deployments pin engine versions independently. Upgrade one, leave the other, roll back the one that regressed.
Survive the owner leaving	This is the real risk, and it is rarely on the slide. The flags live in one person's head and in a shell history file.	Configuration is declarative and in your repo. Someone else can read it.
Run air-gapped	Straightforward. It is your network and there is no vendor in the path.	Not supported today. The gateway and control plane need outbound connectivity, so a genuinely air-gapped fleet is a case where DIY wins outright.

The build

What you would be building, if you built it.

This is not a list designed to look intimidating. Every item is buildable and some of you have built three of them. The point is that none of them is finishable — each one is a permanent fraction of an engineer, not a project with a completion date.

Routing proxy
Key to team to model to a healthy replica. A week to working, then forever to keep current with health semantics.
Quota and admission
Token buckets, reservations, burst allowances, an overflow policy, and a way to change all of it without a deploy.
Placement
Which model on which cards, respecting TP groups, memory arithmetic and compliance labels. Re-solved every time anything changes.
Per-team observability
vLLM's metrics are per-process. Turning them into 'what did fraud-ml experience yesterday' is a dashboard project of its own.
Engine lifecycle
Rolling restarts that drain rather than drop, version pinning per deployment, and a rollback that works at 2am.
Weight distribution
Pulling tens of gigabytes to N hosts without saturating the network, with verification, and with a cache that survives a reboot.
Audit and chargeback
Who called what, under which key, at what token cost, exported monthly in a form finance accepts.
The runbook
The document that means the person who built it can go on holiday.

If your platform team has the headcount and wants this to be a core competency, build it. Some organisations should, and the ones that do usually end up with something better fitted than we could sell them.

The other side

Where we are worse.

There is a hop in the path that you do not control

Self-managed vLLM has no vendor between your client and your GPU. We do, today. The security page is explicit about it. On this axis DIY is simply better, and it will stay better until the customer-hosted gateway ships.

We cost money and vLLM does not

vLLM is free. We are not. If the operational cost you are carrying is one afternoon a month, our price will not clear that bar, and the arithmetic should be done before the call rather than during it. The bands are published so you can do it now.

A managed default is a default

You keep access to the engine flags, but the whole value of a platform is that most teams do not touch them. If your workload needs an unusual configuration on one deployment, that works — if it needs one on every deployment, you are fighting the product.

We are an early company

Fewer people, less operational history, and a shorter list of things that have already gone wrong and been fixed. That is a real risk to take on a production dependency, and any procurement process that flags it is doing its job.

Questions

Is Atlas faster than running vLLM myself?: No. Atlas runs vLLM, so single-model throughput and latency are whatever vLLM gives you on that hardware. What changes is utilisation across a fleet with several models and several teams, because idle capacity gets allocated rather than sitting behind someone's static port assignment.
When should we not buy this?: One model, one team, one owner who enjoys the work, and no compliance boundary between users — keep running vLLM. You would be paying for multi-tenancy you do not have and quota you do not need. The same is true for a genuinely air-gapped deployment, which Atlas does not support today.
How long would it take us to build this ourselves?: The routing proxy is a week. The quota system is a month. The placement solver, the per-team observability, the engine version management and the audit trail are the part that never quite finishes, because each one is a quarter of somebody's time forever rather than a project with an end date.
Are we locked in?: Your weights are on your hardware in a standard format, your clients speak the OpenAI API, and your deployments are YAML in your own repository. Turning Atlas off leaves you with GPUs running vLLM, which is where you started. That is deliberate — a platform whose main retention mechanism is difficulty of exit is not a platform we would want to sell.

Bring the person who built your vLLM setup

They will have the sharpest objections, and they are the ones worth answering. If they conclude you should keep what you have, that is a legitimate outcome of the call.

Request a demo See the pricing bands

Priya Raghavan replies within one working day. No sequence, no SDR, no calendar tennis.