Ghosts in the machine: How a false AI intel almost triggered a US-China clash in West Asia

In a scene straight out of a Hollywood movie, the artificial intelligence (AI) did not pause to consider the geopolitical fallout when it hallucinated a shipment of nuclear components.

A false intelligence report generated by artificial intelligence brought the US military dangerously close to intercepting a Chinese ship in the waters of West Asia, according to a CNN report, laying bare the grave hazards of integrating automated tools into national security operations.

Operating inside US Special Operations Command Pacific in Hawaii, an intelligence analyst asked an AI chatbot to evaluate a Chinese cargo ship’s manifest. Fusing open-source data with secret signals intelligence, the system invented a critical, nonexistent detail: the vessel was transporting nuclear weapons components through West Asia, the report added.

MUST READ | Now, Google Gemini becomes latest AI model to break out & hack real companies

As per the report, the analyst used AI a second time to reformat the hallucinated findings into a standard intelligence briefing without verification. High-level commanders, trusting the report's official formatting, authorised an interception.

Armed US personnel prepared to forcibly board the ship while military warplanes scrambled into the air. Only a last-minute intervention halted the raid, averting a direct conflict between two nuclear powers. As one source familiar with the matter told CNN, the entirely false report "almost started a war."

The high-seas near miss highlights a systemic pattern. Across the defense apparatus, automated systems, fractured internal databases, and an unchecked rush to integrate artificial intelligence are exposing the profound dangers of high-tech national security.

'Hallucination' trap in defense operations

Generative AI models are fundamentally built to predict language patterns, not to verify absolute reality. When applied to military intelligence, this flaw becomes lethal. In the case of the Chinese cargo vessel, the chatbot's internal algorithms bridged gaps in raw data by fabricating critical details — a phenomenon known as a "hallucination."

DON'T MISS | From rogue uploads to hallucinated data: OpenAI discloses 6 model failures

What made the threat worse was automation bias: human analysts, operating under extreme time pressure, treating machine-generated outputs as authoritative facts rather than unverified leads.

By passing the AI's initial output through a second AI tool to generate a polished briefing, the analyst unintentionally laundered fabricated data into a military-grade directive. As a former senior US official told CNN, internal intelligence tools are frequently "mostly just copies of the commercial stuff wearing lipstick," lacking the safeguards required for high-stakes warfare.

Fractured systems & 'siloed' data intelligence 

The risks of modern defense tech extend beyond hallucinating chatbots to the broken architecture of the databases feeding them. AI models and targeting algorithms are only as reliable as the institutional memory they draw from.

Nowhere are the consequences of this disconnect clearer than in Minab, Iran. A US missile strike on the Shajareh Tayyebeh Elementary School killed over 150 civilians, including more than 120 children.

FIND WHY | AI safety warning: Sam Altman, Elon Musk back Anthropic CEO’s call to slow down frontier AI. Why?

According to a Bloomberg report, Pentagon probes revealed that years before the strike, an analyst had flagged in a digital tool that the former naval facility was being used as a school. However, because that software was never linked to the main target database, the warning was buried.

Automated targeting pipelines and military planners continued relying on outdated imagery, misidentifying an active school filled with children as a military compound.

Institutional speed vs Human oversight

Under initiatives like Defense Secretary Pete Hegseth’s "Artificial Intelligence Acceleration Strategy," the US military has aggressively pushed to place commercial AI models directly into the hands of personnel to speed up battlefield decisions. The operational imperative is simple: outpace adversaries who are integrating similar technology.

DO CHECKOUT | A kill switch won’t save us from rogue AI, tech leaders say amid growing concerns

However, the rapid rollout has created a patchwork of uncoordinated AI tools across different commands, operating without unified safety standards. When speed is prioritised over verification, the human element in the "kill chain" is reduced to a rubber stamp.