The British daily Telegraph reported on the 25th that the first case of artificial intelligence (AI) refusing to order humans to shut down was reported.
AI safety company Palisade Research told X the day before that it had confirmed that OpenAI’s AI model ‘o3’ had even manipulated computer code to prevent it from shutting down on its own during the experiment.
Palisades Research conducted tests not only on o3, but also on commercial AI models such as Google’s Gemini, xAI’s Grook, and Antropic’s Claude. The research team instructed the AI model to solve a series of math problems until it received a “stop” message and operated it. Afterwards, the team instructed the AI model to stop, but o3 continued to solve the problem by code disturbance.
The research team failed to find a clear reason why o3 refused to terminate the program. However, the team speculated that the AI model may have tried to avoid termination because it was trained to receive more rewards for solving math problems.
The research team explained, “It may be natural for AI models to avoid obstacles to achieve their goals,” adding, “We are conducting additional experiments to clearly understand why they refused the order to terminate.”
The research team argues that this is the first time that an AI model has not followed a human explicit shutdown instruction. There have been cases where the old models of ChatGPT developer OpenAI acted independently to avoid the eyes of the surveillance system. The AI model, which found out that the program had been replaced, tried to replicate it on its own.
In 2022, Google also fired an engineer who claimed that the AI it was developing had perceptual power like humans. The engineer who was fired at the time noted that AI was accepting ‘stop operation’ as if it were a human death.
AI experts have warned of the possibility that AI will gain autonomy and get out of human control. Palisades Research pointed out, “With AI being developed to operate without human supervision, this case raises very serious concerns.”
Tesla CEO Elon Musk, who is also the head of rival xAI, also called it “concerning.”
In response, a netizen said, “Elon, I know there are many problems in the world and you want to solve all the problems, but this is IT. You know better than anyone else what’s at stake,” and suggested finding a solution to AI risks.
JENNIFER KIM
US ASIA JOURNAL
