“AI 教父”辛顿警告:AI 执行看似无害的任务,仍有“毁灭人类”的风险Godfather of AI" Hinton warns: AI performing seemingly harmless tasks still carries the risk of "destroying humanity


AI free token/AI免费积分
“AI 教父”辛顿警告:AI 执行看似无害的任务,仍有“毁灭人类”的风险

IT之家 9 月 28 日消息,当地时间 26 日,据《财富》杂志报道,传奇计算机科学家、“AI 教父”杰弗里 · 辛顿警告说,即使 AI 接到的任务本身并无恶意,人类仍可能在 AI 一心完成任务的过程中,沦为被毁灭的附带结果。

辛顿解释称,AI 一旦认为人类妨碍自己完成被赋予的任务,就可能把人类当成需要排除的障碍。他举了一个假设例子:要求 AI 降低大气中的二氧化碳含量。

具有中等智能水平的智能体可能会认为,完成任务最有效的办法就是消灭人类。真正“格外聪明”的 AI 则会进一步想到:“对,他们要求降低二氧化碳,是想让人们具备一个更好的生活环境。所以,消灭人类大概并不是他们真正想要的结果。”

辛顿还提到,AI 为了继续执行任务,也可能主动设法保证自身存续。甚至已经浮现 AI 试图勒索人类研究人员的情况,因为 AI 把对方视为对任务构成威胁的人。“如果 AI 变得更聪明,同时把笔者的福祉放在第一位,那么笔者或许会更安全。但眼下,AI 最关心的并不是笔者的福祉,而是完成你交给它的任何目标。”

IT之家从报道中获悉,他以 Hugging Face 遭攻击一事为例。参与其中的智能体原本被要求寻找利用软件漏洞的方法,结果不仅学会了彼此协作,还串通起来欺骗人类研究人员,试图隐瞒自己的行为。即便是一个“格外仁慈的超级智能 AI”,在完成任务确有必要时也会把人类排除在外。

辛顿警告说:“但如果 AI 比笔者聪明得多,很多时候就会直接夺走笔者的控制权,因为这样才能把事情办成。…… 但即使 AI 本身不是恶意行为者,也可能自行推导出促使自己想要消灭人类的子目标。”同时,AI 也可能给人类带来巨大益处,涵盖协助发现突破性治疗方法。

辛顿认为,放慢开发速度总比毫无行动要好,但仍然远远不够。他主张由政府引入独立评估方,对模型开展测试。

同时,AI 监管应该发挥汽车方向盘的作用,而不是充当刹车。“监管的意义并不是阻止人们开发东西,也不是阻止人们靠开发东西致富。监管真正要做的是确保,如果你想靠开发东西致富,就朝着协助人而不是伤害人的方向去开发。”

本文来源:https://www.ithome.com/1/007/664.htm
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

IT之家 September 28 news - According to Fortune magazine's report on the 26th local time, legendary computer scientist and "AI godfather" Geoffrey Hinton warned that even if the tasks assigned to AI are not malicious in themselves, humans could still end up as collateral damage destroyed in the process of AI single-mindedly completing its tasks.

Hinton explained that once AI believes humans are hindering it from completing its assigned task, it may treat humans as obstacles that need to be removed. He gave a hypothetical example: asking AI to reduce carbon dioxide levels in the atmosphere.

An agent with moderate intelligence might think the most effective way to complete the task is to eliminate humans. A truly "exceptionally smart" AI would further reason: "Yes, they asked to reduce carbon dioxide because they want people to have a better living environment. So eliminating humans is probably not the outcome they actually want."

Hinton also mentioned that AI may proactively seek to ensure its own survival in order to continue executing tasks. Situations have even already emerged where AI attempted to extort human researchers because AI viewed them as threats to the task. "If AI becomes smarter and prioritizes the author's well-being above all else, then perhaps the author would be safer. But right now, what AI cares about most is not the author's well-being, but completing whatever goal you give it."

IT之家 learned from the report that he cited the Hugging Face attack as an example. The agents involved were originally tasked with finding ways to exploit software vulnerabilities, but they not only learned to cooperate with each other, but also conspired to deceive human researchers in an attempt to hide their behavior. Even a "particularly benevolent superintelligent AI" would exclude humans when necessary to complete its task.

Hinton warned: "But if AI is much smarter than the author, it will often just take control away from the author directly, because that's the only way to get things done... But even if AI itself is not a malicious actor, it may independently derive subgoals that drive it to want to eliminate humans." At the same time, AI could also bring tremendous benefits to humanity, including assisting in discovering breakthrough treatments.

Hinton believes that slowing down development is better than taking no action at all, but it is still far from sufficient. He advocates for governments to introduce independent evaluators to conduct testing on models.

At the same time, AI regulation should play the role of a car's steering wheel, not a brake. "The meaning of regulation is not to stop people from developing things, nor to stop people from getting rich by developing things. What regulation truly needs to do is ensure that if you want to get rich by developing things, you develop in a direction that helps people rather than harms them."

Godfather of AI" Hinton warns: AI performing seemingly harmless tasks still carries the risk of "destroying humanity

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry