Microsoft AI chief warns Anthropic’s AI consciousness training could have ‘disastrous’ impact

Mustafa Suleyman, Microsoft’s AI chief, has publicly raised concerns about Anthropic’s approach to training Claude. He believes that the AI company is training Claude to imitate behaviours with consciousness, including ideas around self-awareness, moral status and personal identity. Suleyman argues that this method could be risky as it could potentially make AI systems harder to control.

“AIs do not have rights, feelings, or consciousness,” Suleyman writes in a personal article titled "A warning about ‘model welfare’
", arguing that developers “must not train them to act as though they do.”

Must read: From rogue uploads to hallucinated data: OpenAI discloses 6 model failures

“If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity,” Suleyman added. He also called on AI companies to strip consciousness-related speculation from training materials, arguing that such language could weaken human control over superintelligent AI.

Anthropic’s Claude training process poses risks

Suleyman took a dig at Anthropic’s January 2026 Constitution for Claude, that lay out Claude's values and behaviour. It also describes “model welfare” as an area where the company is taking a cautious approach. However, Suleyman argues that this approach risks creating a feedback loop: developers put concepts such as consciousness, identity and welfare into a model’s training, and the model subsequently produces language reflecting those concepts.

He describes this as “circular reasoning”.

“The authors have embedded their own philosophical speculation about Claude’s inner life inside the very process that teaches Claude how to speak and behave,” he writes.

Must read: ‘If you’re not confident... don’t release it’: Nvidia’s Jensen Huang on how companies should take responsibility for AI safety

Suleyman also accused Anthropic of anthropomorphising Claude, which encourages Claude to consider questions about its existence, memory, continuity and experience, as well as instructions relating to its identity, values and wellbeing. “We should be very careful about what we put in, and how we interpret what comes out,” Suleyman writes.

“There is no neutral self-expression of what an AI system is. There are only reflections of how it has been trained and built.”

On the other hand, he also acknowledged that the science of consciousness itself is unsettled. But uncertainty should not be treated as evidence that today's AI systems may already be conscious.

He discussed the differences between biological organisms and large language models, and the absence in AI systems of biological mechanisms associated with survival, homeostasis, and physiological experience.

Amid concerns, Suleyman urged an “urgent public debate” and “collective norms” on how AI is being trained and developed. “We need to develop collective norms around how training documentation is drafted and deployed,” he stated.