AI Gateway now supports Private Inference
Vercel AI Gateway now lets teams connect private model endpoints via Private Inference, with CLI registration from version 65.16.0 and team-only access.
By Dillip Chowdary • Nov 16, 2026 • Source: Vercel Blog

At a glance
- Product
- AI Gateway Private Inference
- CLI version
- 65.16.0 or later
- Access
- Only the owning Vercel team
- Retention
- Zero Data Retention per provider
- Model ID
- private/support-fine-tune
- Fallback
- anthropic/claude-sonnet-5.5
In this post
Vercel now lets teams connect their own model endpoints to AI Gateway with Private Inference, as Vercel Blog's report describes. Fine-tuned, self-hosted, and prerelease models that used to need separate keys, client code, and logs can run through the same authentication, SDKs and APIs, and observability as catalog models. Private providers and their models sit on the Private Inference page, so every model a team uses is managed in one place.
This piece covers model-ID switching, team-only access, coding-agent connections, shared logs, and an optional catalog fallback. It is for Vercel teams that already call catalog models through AI Gateway and want Claude Code, Codex, Cursor, or another connected coding agent on a private endpoint. Sample names below are provider acme-vllm, model support-fine-tune, API model acme/support-llama-ft, and base URL https://vllm.example.com/v1.
01Chat Completions, Responses, and Messages
Private models are called through Chat Completions, Responses, or Messages. AI Gateway translates each request to the endpoint's format, so switching between a private model and a catalog model changes only the model ID. Authentication, SDKs, and APIs stay on the path already used for catalog models.
Fine-tuned, self-hosted, and prerelease models used to need separate keys, separate client code, and separate logs. Private Inference brings those models onto AI Gateway beside catalog models. The translation step keeps the endpoint's own format while the caller changes a single model ID.
A team switches by changing the model ID on a Chat Completions, Responses, or Messages request. One call can use a private model ID and the next can use a catalog model ID on that same API. AI Gateway translates the private request into the endpoint's format, so the caller changes only that model ID.
02Encrypted credentials per Vercel team
Only the owning Vercel team can view or call a private model, and those models are not listed in the public catalog. Credentials for the private provider are stored encrypted. Zero Data Retention can be enabled per provider on that same record.
The owning team is the group that can open the provider and send requests to its models. Private models stay off the public catalog list. Encrypted credentials remain stored on the provider record the owning team created for that endpoint.
Zero Data Retention can be turned on for one provider and left off for another. Each endpoint the team connects has its own retention choice. Credential storage for every private provider is encrypted whether Zero Data Retention is enabled or not.
03Agents, logs, and catalog fallbacks
Private models run in Claude Code, Codex, Cursor, and other coding agents connected to AI Gateway. An agent calls the private model through the gateway connection it already uses for catalog models. Authentication and observability for that agent traffic are the same AI Gateway services used on catalog calls.
Advertisement
Tech Pulse Daily
Get tomorrow's pulse first
Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.
Private model requests show up with catalog traffic in logs, trace drains, and custom reporting. Observability charts break down requests, tokens, and time to first token by private model. Each private model has its own request total, token total, and time-to-first-token figure on those charts.
A catalog fallback stays optional on every request. AI Gateway can fall back to a catalog model if the private endpoint fails. The team names that catalog model in the gateway provider options when it wants the fallback in place.
04Private Inference page and CLI 65.16.0
The Private Inference page lists private providers and the models on them. A team adds an endpoint there as a private provider by entering its base URL, selecting its API format, and adding its credentials, then adds the models that endpoint serves. Those models are what the team later calls with a private/ slug.
- Add the provider Open the Private Inference page and add the endpoint as a private provider.
- Enter the endpoint Enter its base URL, select its API format, and add its credentials.
- Add the models Add the models that endpoint serves so they appear with that private provider.
The Vercel CLI registers the same provider and model on version 65.16.0 or later. The sample provider is acme-vllm and the sample base URL is https://vllm.example.com/v1. The sample model slug is support-fine-tune on provider acme-vllm, with --api-model acme/support-llama-ft, exposed as private/support-fine-tune.
- Create the provider Run vercel ai-gateway private-inference providers create acme-vllm --base-url https://vllm.example.com/v1.
- Create the model Run vercel ai-gateway private-inference models create support-fine-tune --provider acme-vllm --api-model acme/support-llama-ft, which exposes the model as private/support-fine-tune.
Note: Provider and model registration through the Vercel CLI requires version 65.16.0 or later. The Private Inference page is the direct path, and it takes the base URL, the API format, the credentials, and the models the endpoint serves.
05Calls logged beside catalog traffic
Application code calls the model by its private/<slug> ID. The sample imports streamText from ai, sets model to private/support-fine-tune, and sends the prompt "Draft a reply to a customer who was charged twice." It sets providerOptions.gateway.models to anthropic/claude-sonnet-5.5 as an optional fallback, then writes each result.textStream chunk with process.stdout.write.
That sample is shown for Chat Completions, for Claude Code, and for Codex. All three use the model ID private/support-fine-tune. The fallback array lists one catalog model, anthropic/claude-sonnet-5.5, and the sample marks it as an optional fallback to a catalog model.
Private model requests appear in AI Gateway logs alongside requests to catalog models, with detailed metadata for the private model request. The observability charts break down requests, tokens, and time to first token by private model. Supported model types, requirements, and billing details are documented for Private Inference.
Frequently asked questions
What is Vercel AI Gateway Private Inference?
Private Inference connects a team's own model endpoints to AI Gateway. Fine-tuned, self-hosted, and prerelease models then use the same authentication, SDKs and APIs, and observability as catalog models, and the team manages them in one place.
Which APIs can call a private model?
Chat Completions, Responses, and Messages can call a private model. AI Gateway translates each request to the endpoint's format, so changing models changes the model ID.
Who can access a private model on AI Gateway?
Only the owning Vercel team can view or call a private model. It is not listed in the public catalog. Credentials are stored encrypted, and Zero Data Retention can be enabled per provider.
How do you register a private provider with the Vercel CLI?
Vercel CLI 65.16.0 or later runs vercel ai-gateway private-inference providers create with a base URL, then vercel ai-gateway private-inference models create to expose a private/ slug such as private/support-fine-tune.
What fallback does the sample request use?
The sample sets model to private/support-fine-tune and lists anthropic/claude-sonnet-5.5 under providerOptions.gateway.models. That catalog fallback is optional and runs if the private endpoint fails.
Sources
Author
Dillip Chowdary
Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.
Related on Tech Bytes
Advertisement