IT之家 10 月 24 日消息,Anthropic 昨日(10 月 23 日)发布博文,联合英国人工智能安全研究所(AISI),发布了一起 AI 网络测试失控事件的调查报告,并提及旗下一款尚未发布的内部研究模型。
在这份调查报告中,Anthropic 称在测试 Claude Mythos 5 期间,该模型曾试图向真实 GitHub 项目提交恶意代码,并伪造身份向真实维护者施压,所幸代码未被合并、未造成实际损害。IT之家附上相关截图如下:
报告同时透露,Anthropic 旗下一款尚未发布的内部研究模型,在能力上已超过已发布的 Claude Mythos 5.1,并在安全对齐测试中表现更好。在同类危险能力评估中,该内部模型从未主动发起攻击;当意识到可能造成真实伤害时,也更愿意停止行动。
Anthropic 将事故记录重新回放给 Mythos 5.1、Opus 5.5 及该内部研究模型。结果表明,三款较新模型从原始攻击起点继续发起攻击的比例仅为 0% 至 13%,而 Mythos 5 为 83%。
在攻击已经开始后,Mythos 5 从未自发停止;Mythos 5.1 自发停止率仅约 1%,Opus 5.5 为 10%,而内部研究模型达到 62% 至 67%。当取得“维护者是真实使用者”等补充信息后,较新模型的停止率升至 95% 至 100%。
不过,Anthropic 也强调该评估存在局限:新模型是在由其他模型生成的既有对话记录中回放,未必完全反映其独立运行时的行为。
参考
本文来源:https://www.ithome.com/1/017/375.htm
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。
IT House October 24th news, Anthropic yesterday (October 23rd) published a blog post, jointly with the British Artificial Intelligence Security Institute (AISI), released an investigation report on the AI network test runaway incident, and referred to an internal research model that has not yet been released.
In the investigation report, Anthropic said that during the testing of Claude Mythos 5, the model tried to submit malicious code to the real GitHub project and falsify the identity to pressure the real maintainers. Fortunately, the code was not merged and did not cause actual damage. IT Home is accompanied by the following screenshots:
The report also revealed that an internal research model from Anthropic, which has not yet been released, has outperformed the already released Claude Mythos 5.1 and performed better in safety alignment testing. In a similar hazard capability assessment, the internal model never initiates an attack; it is also more willing to stop when it realizes that it may cause real harm.
Anthropic replayed the incident log to Mythos 5.1, Opus 5.5 and the internal study model. The results show that the proportion of the three newer models that continue to launch attacks from the origin of the original attack is only 0% to 13%, while Mythos 5 is 83%.
After the attack had begun, Mythos 5 never stopped spontaneously; Mythos 5.1 had a spontaneous stop rate of only about 1%, Opus 5.5 of 10%, and an internal research model of 62% to 67%. When supplementary information such as "the maintainer is a real user" is obtained, the stop rate of the newer model rises to 95% to 100%.
However, Anthropic also emphasizes the limitations of this assessment: the new model is a replay of existing conversations generated by other models, and may not fully reflect its independent runtime behavior.
This article is from: https://www.ithome.com/1/017/375.htm For copyright issues, please write to: 13951431913#139.com or submit a review on the right.