AI Gateway 新增信心度決策回退功能
AI Gateway adds confidence-based decision fallbacks
AI Gateway 現在支援基於置信度的決策回退,可根據 Choice、Score 或 Boolean 問題的置信度閾值決定是否切換到備援模型。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
AI Gateway can now escalate a decision request to a fallback model when the primary model's answer trips a confidence condition you set.
Conditions can be combined, so an escalation can depend on more than one signal. The plain model names you already list in models keep catching outright errors, so existing fallbacks are unaffected. Confidence conditions cover Choice and Score questions. Boolean questions escalate on a probability range instead.
Set the fallback by adding one conditional object to providerOptions.gateway.models. Requests without it keep their existing behavior.
{
"gateway": {
"models": [
{
"model": "openai/gpt-6-astra",
"when": {
"question": "intent",
"confidenceBelow": 0.6
}
}
]
}
}
Escalating an uncertain answer to a fallback model with a confidence condition
Leave out question and the condition checks every question of the matching type, so { "confidenceBelow": 0.6 } escalates when any Choice or Score answer falls below 0.6.
Because a triggered fallback runs a second decision, it bills both stages.
Decision fallbacks are in beta. Learn more in the decision fallbacks documentation.