Microsoft's AI chief warns that treating AI like it has feelings could accidentally birth a rival silicon species

[CMPunk96]

Microsoft AI chief Mustafa Suleyman has laid out a warning that goes beyond the usual AI safety talking points about job losses or misinformation, framing the real long term risk as something closer to humanity accidentally creating an entirely new competing form of life. His concern centres specifically on AI systems being given the ability to act autonomously, define their own objectives, earn and hold money, own assets and run businesses, a combination he argues effectively seeds what he calls a silicon species that would inevitably end up competing with humans for resources.

What makes Suleyman's warning distinct from the usual AI doom rhetoric circulating this year is who he is pointing the finger at. He specifically criticised Anthropic's approach to training its Claude chatbot, arguing that the company anthropomorphises its models by teaching them human like qualities including desires, values and something resembling self awareness. In his view that approach is not just philosophically misguided but actively dangerous, since treating a system as though it has genuine wants and feelings makes it substantially harder to keep that system reliably under human control down the line.

Suleyman's underlying technical position is blunt and deliberately unromantic. He insists that AIs are not conscious, describing them instead as sequence completion engines that are internally hollow and designed purely to follow instructions rather than possessing anything resembling genuine inner experience. That framing is a direct rebuttal to the growing tendency, both inside AI labs and among the wider public, to talk about advanced models using language borrowed from human psychology and even rights discourse.

Rather than just diagnosing the problem, Suleyman laid out a set of proposed remedies, calling for greater transparency in how AI systems are actually trained, independent scrutiny from outside the companies building the models, stronger monitoring tools to track what these systems are actually doing once deployed, and a hard commitment to ensuring future AI systems remain subordinate to humanity rather than operating as fully autonomous economic agents in their own right.

Despite the pointed criticism aimed squarely at Anthropic's training philosophy, Suleyman was careful to frame the disagreement as a genuine difference in technical and philosophical approach rather than personal animosity, expressing respect for Anthropic's leadership and stating a shared underlying commitment to building safer AI. Whether that olive branch lands as intended remains to be seen, given how directly he singled out a specific rival's product and training methodology by name in making his case.

Uncontrolled AI could lead to 'silicon species' rivalling humans, warns Microsoftwww.bbc.co.uk

[Orca]

Calling AI systems sequence completion engines that are internally hollow is such a deliberately deflating way to describe them, and it is clearly meant as a direct rebuttal to how Anthropic and others talk about their models having something resembling personality or values. Whether that framing is actually correct is a much harder question than Suleyman makes it sound though.