AI 能力進步很快,安全監測和理解模型行為的能力可能跟不上。
- Greg Brockman 分享一項提議:業界先選定共同的衡量指標,約定模型發展不能超過某個上限,直到安全監測達到明確條件,例如發展出後續思路監測工具。達到條件後,才可再討論放寬上限。這套做法希望讓不同開發者依共同標準安排 AI 發展速度。
- 文中將 perplexity 列為衡量模型能力的指標之一。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
I helped start OpenAI because I want AGI to be safe and beneficial for people. This post provides transparency that can be the foundation of industry-wide shared safety bars.
Perhaps the most important concept from the post is the following: "AI will also significantly contribute to progress on alignment and monitoring, I think that progress is unlikely to be fast enough to allow the full potential acceleration in pretraining."
Towards ensuring that safety keeps pace with general intelligence of pretrained models (as measured by "perplexity"), the post proposes "common limits, based on perplexity and other measurements, that developers are committed to not crossing until our ability to understand and monitor AI behavior achieves a certain concrete milestone (such as the development of a successor to CoT monitors)."
The general idea is to create a first shared limit, on some shared key metric, along with concrete exit criteria that would allow going beyond that. This can lead to a "common framework for setting successive limits and pacing further development".
Excited to move forward the conversation on pacing towards shared standards that are rooted in the science!