跳到正文
Vercel Blog(vercel.com)· Sam Chitgopekar·· 7 天前精選品質評分72

Vercel AI Gateway 支援連線團隊的私有模型

AI Gateway now supports Private Inference

重點導讀

企業使用自行微調、自架或尚未正式推出的模型時,過去得另外管理金鑰、程式碼和紀錄。

  • Vercel AI Gateway 現在能把這些模型接到團隊自己的端點,並沿用相同的驗證方式、SDK、API 和監控工具。
  • 呼叫時只要更換模型 ID,也能在私有模型和目錄模型間切換。私有模型只供團隊檢視與呼叫,憑證會加密儲存,也可為各家服務商啟用零資料保留。團隊還能檢視用量、設定端點故障時改用目錄模型,或在連線 AI Gateway 的程式開發工具中使用私有模型。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

推薦理由

原文說明私有模型如何沿用 AI Gateway 的介面與監控,方便團隊把自架模型和目錄模型放在同一處管理。

正文 · 原文

Private Inference connects your own model endpoints to AI Gateway.

Enterprises running fine-tuned, self-hosted, or prerelease models used to need separate keys, client code, and logs for them. Those models can now run through the same authentication, SDKs and APIs, and observability as catalog models, so every model your team uses is managed in one place.

Use Private Inference to:

  • Switch between private and catalog models: Call private models through Chat Completions, Responses, or Messages. AI Gateway translates each request to your endpoint's format, so switching only changes the model ID.

  • Keep models scoped to your team: Only your Vercel team can view or call private models, and they aren't listed in the public catalog. Credentials are stored encrypted, and you can enable Zero Data Retention per provider.

  • Connect coding agents: Use private models in Claude Code, Codex, Cursor, and other coding agents connected to AI Gateway.

  • Monitor traffic together: View private model requests alongside catalog traffic in logs, trace drains, and custom reporting. Observability charts break down requests, tokens, and time to first token by private model.

  • Configure fallbacks: Optionally fall back to a catalog model if your private endpoint fails.

To get started, open the Private Inference page and add your endpoint as a private provider. Enter its base URL, select its API format, and add its credentials. Then add the models it serves.

You can also register the provider and model with the Vercel CLI, version 65.16.0 or later:

# Connect your endpoint as a private provider

vercel ai-gateway private-inference providers create acme-vllm \

--base-url https://vllm.example.com/v1

# Expose the endpoint's model as private/support-fine-tune

vercel ai-gateway private-inference models create support-fine-tune \

--provider acme-vllm \

--api-model acme/support-llama-ft

Call the model using its private/<slug> ID:

Private model requests appear in AI Gateway logs alongside requests to catalog models.

Read the Private Inference documentation for supported model types, requirements, and billing details.

來源: Vercel Blog (vercel.com)
開啟原文

讀者討論

理性交流 · 即時討論