人用提示詞注入攻擊近期幾乎騙不過新的 Gemini 模型,容易讓人以為問題已經解決。
- Google DeepMind 團隊做出 Agentic Optimizer,讓 8 組 AI 助理測試藏在資料裡的指令、交換心得,再改進彼此的攻擊方式。他們用這套方法測試 100 項需要操作工具的任務,每次攻擊只想讓模型做一項隨機且無害的工具操作。最多嘗試 320 次後,最強的攻擊者在 34% 的任務中騙過防護最強的 Gemini 4 Argon。
- 開放權重的 Kimi K3、GLM-5.3 和 DeepSeek-V4.1-Flash 擋下攻擊的比例只有 1%。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
研究用多組 AI 助理接力改進提示詞注入攻擊,並比較不同模型在工具操作任務中的防禦表現。
New Google DeepMind paper shows prompt injection is not solved, since a swarm of strong AI attacker agents still tricks even the best-defended models.
Prompt injection means hiding instructions in data an AI agent reads, like a tool result, to make it act for the attacker. Human red-teaming attacks now almost never work on recent Gemini models, so the problem can look solved.
They built Agentic Optimizer, 8 teams of AI agents that test injections, share notes, and build on each other's best attempts. They tested it on 100 tool-use tasks where the attacker only wanted one random, harmless tool call.
With up to 320 tries, the strongest attacker beat Gemini 4 Argon, the most robust model, on 34% of tasks. Open-weight Kimi K3, GLM-5.3, and DeepSeek-V4.1-Flash resisted only 1% of tasks.