研究發現,小型開放式 LLM 使用簡短直接的提示詞與少量範例,就能在自然語言推論(NLI)中媲美較複雜的提示方法。
- 測試方式:研究評估 5 個小型開放式模型,包含 Gemma 2 9B、Mistral 7B 和 Llama 3.1 8B,在 3 個 NLI 資料集上使用 0 至 15 個範例,不加入推理步驟。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
This paper finds that a short, direct prompt with a few examples (In-Context Learning) lets small open LLMs match more complex prompting methods on natural language inference.
The task, called NLI, asks whether one sentence follows from another, contradicts it, or is unrelated. Earlier LLM work on it leaned on complex prompting and sampling tricks, assuming more examples help.
They tested 5 small open models, including Gemma 2 9B, Mistral 7B and Llama 3.1 8B, on 3 NLI datasets. Each got 1 short, direct prompt with 0 to 15 examples and no reasoning steps.
Mistral 7B beat the 63.4% MNLI accuracy that a published in-context sampling method got with the same model. Gemma 2, Llama 3.1 and Mistral scored about the same with 1 example or 15 on MNLI and SciTail.