A fake AI intelligence report on a Chinese cargo ship nearly triggered a US military confrontation

[EdgeNode Joel]

A chatbot embedded within a US military AI programme generated a badly wrong intelligence report about a Chinese vessel operating in the Middle East, misidentifying its cargo as containing nuclear weapons components when no such material was actually aboard. The error was serious enough that US military aircraft were positioned nearby the ship and forces came close to actually boarding it before officials caught the mistake and stood the operation down.

The system responsible combined intelligence feeds from US Special Operations Command Pacific with classified signals intelligence, generating what turned out to be an entirely erroneous assessment of the ship's actual manifest. One source described to CNN just how close this came to a genuinely serious escalation, stating plainly that the situation almost started a war, a characterisation that is not being used loosely given how far the actual military response had already progressed before the error was identified and corrected.

This incident lands directly on top of Defense Secretary Pete Hegseth's recent announcement that the US Department of War, as it has been renamed, is adopting multiple commercial AI systems into its operations, including versions of Elon Musk's Grok, OpenAI's ChatGPT and Google's Gemini. The timing is awkward at best, an intelligence failure this serious surfacing publicly right as the department is actively expanding rather than pausing its reliance on exactly this category of AI powered tool for processing sensitive military intelligence.

A Department of War official's response to the incident struck a notably cautious and defensive tone, stating that the department remains committed to the rigorous verification of all intelligence inputs and the responsible integration of emerging technologies, careful language that does not directly acknowledge how close this specific failure actually came to real consequences involving an armed confrontation with a Chinese vessel and crew.

Experts have been consistently warning throughout this broader wave of military AI adoption against cutting out critical human vetting steps in the rush to speed up decision making, and this incident reads as close to a worst case illustration of exactly that warning playing out in practice. Speed and efficiency gains from AI assisted intelligence processing mean very little if the underlying system occasionally manufactures a completely false threat assessment with enough apparent confidence to move real military assets into position based on it.

Fake Ai Used By Us Military Almost Started War With Chinawww.telegraph.co.uk