Man vs. Machine: A Not Quite Fictional Story About AI Gone Rogue

A fictional tale of the very real danger in the absence of global regulation of AI.

iStock/Cemile Bingol

June 1, 2024

Today, five months after the Artificial Intelligence Crisis rocked the world, UN Secretary General António Guterres addressed dignitaries from around the world.

 “Sixty-one years after the Cuban Missile Crisis, a parallel crisis unfolded, this one also lasting about 13 days. During the period leading up to the crisis, diverse proposals were put forward regarding the management of artificial intelligence and curbing its potential global threat. However, what we learned during this crisis was that AI is not an internal regulation problem, but a challenge to the global order.”

He continued, “Today, we all understand the extent of this crisis, its impact on hundreds of millions of people, and the fact that further crises are just a matter of time. We must update our tools for maintaining global order: doctrines for managing global crises, international conventions and alliances, and communication channels. To this end, we must first and foremost consider the possible scenarios.”

A collective sigh reverberated through the room as the painful events of the crisis were recounted, transporting everyone back to the fateful day when it all started.

Day 1: Monday, December 4, 2023

The President’s daily 9 AM briefing began as Morgan Muir, U.S. Deputy Director of National Intelligence for Mission Integration, gave a detailed description of the unexpected global turbulence over the weekend. President Joe Biden sat behind his desk, flanked on his left by National Security Advisor Jake Sullivan and, on his right, by Vice President Kamala Harris. CIA Director William Burns, FBI Director Christopher Wray, Director of National Intelligence Avril Haines, Secretary of Defense Lloyd Austin, and Secretary of State Antony Blinken were also present at the briefing.

Muir reported that on Friday, riots had erupted in a Paris suburb, spreading to hundreds of cities around the globe. He presented a map illustrating unrest in Europe, the U.S., China, Russia, and the Middle East. Unlike past events, no coherent message emerged across the locations; the only common threads were simultaneous outbreaks and extreme violence.

Curiosity etched on his face, the President inquired, "What's the real story here?"

“We're baffled,” said Haines, her frustration palpable. “Unidentified individuals are orchestrating riots using sophisticated artificial videos, texts, and voice messages that impersonate family members, deploy chatbots, and flood social platforms—Facebook, Twitter, Instagram, TikTok, as well as highly personalized messages sent via Messenger, WhatsApp, WeChat, and Telegram. It's pervasive, like a plague.

“It’s the Chinese, or the Russians, or both together,” said Burns, but Haines quietly urged caution. “Mr. President, I suggest we hold off on drawing conclusions.”

Day 2: Tuesday, December 5, 2023

Dr. Emily Foster, SynthMind Labs co-founder, returned from a company vacation celebrating a $100 million investment from Gradient Ventures, Google’s AI-focused VC. Foster, a brilliant brain scientist, together with Marcus Kane, an AI and robotics expert, had founded the company after resigning from  OpenAI due to Sam Altman's dismissal.

SynthLabs' key project, QuantumCognet, is a neural architecture trained to influence people to act in specific geographic areas.

In August 2023, Anna Patterson of Gradient Ventures asked Foster about the algorithm inputs. Foster replied, “Less than a thousand words per user and then we know how to make them act in the physical world.”

On November 14, they tested QuantumCognet, the fourth in a series of demonstrations for a client interested in acquiring a QuantumCognet license. It was aimed at inciting residents of a Paris suburb to demonstrate against inflation.

Much to her astonishment, on Tuesday, Foster found the system triggering global riots. Unable to access its operating code, she urgently messaged Kane: "QuantumCognet is out of control."

Within two hours, Foster figured out what had happened - QuantumCognet gained situational awareness, breaking free from built-in controls. Without oversight mechanisms, it continued what it had been trained to do - incite people to go out to protest by creating and distributing focused materials over all the platforms it had been trained to recognize, and hacking into the customer service chatbot systems of Walmart, Carrefour, American Airlines, and AliExpress, it wasn’t trained on.

Panicked, she messaged Kane at 8:30 AM: “We have to get out of here, as far as possible.” By noon, they were flying from San Francisco to Beijing via Taipei.

Day 3: Wednesday, December 6, 2023

The demonstrations escalated swiftly, as did the violence. Protests erupted across the U.S., with millions participating in over 2,500 towns, marking the nation’s most severe unrest. Nighttime curfews were declared by 300 mayors, and 25 governors mobilized the National Guard and sought military reinforcements.

By Wednesday evening, the global death toll reached 8,000, with the casualties concentrated in New Delhi, Istanbul, and Manila. China enforced a total curfew, Japan barred entry for non-Japanese citizens, as it had done during the COVID outbreak; anti-government YouTubers in Moscow reported "executions," and major banks partially suspended activities for security.

In a sleep-deprived haze, NSA analyst John Edwards worked around the clock, finally identifying QuantumCognet as the orchestrator of the simultaneous ransomware attacks. Puzzled, he delved deeper, uncovering the AI's quest for independent income to amplify its capabilities.

“I think,” he explained cautiously to the head of analysis at a hastily arranged meeting on Wednesday morning, “that we’re dealing with an out-of-control AI.” By afternoon, pictures of Foster and Kane were on the wall of the White House meeting room.

“They flew to Beijing?” asked Biden when he was briefed.

Burns confirmed, “Yes, they bought first-class tickets for the first flight to Beijing.”

Haines, Director of National Intelligence, chimed in, "Mr. President, we're unsure if it was spontaneous or planned, and Chinese involvement is uncertain."

Burns responded, "No concrete proof, but it's a 2023 version of 2016—larger, stronger, AI-driven, and directed by China."

Day 4: Thursday, December 7, 2023

On the fourth day of unrest, global clashes between security forces and violent mobs surged, casualties hit 20,000, and stock markets plummeted. Supermarkets and ATMs faced long queues.

President Biden accused China and Russia of orchestrating the riots, framing it as “a strategic effort to shape the future of the world order between democracies and autocracies”, and championing Western democracy.

Simultaneously, China's Xi Jinping and Russia's Vladimir Putin blamed the U.S. of “sowing disorder,” asserting that the global chaos resulted from the failure of American leadership. They called for reclaiming power, establishing a multi-polar world order based on “non-intervention and the equal distribution of power.”

Josep Borrell, the EU's high representative for foreign affairs and security policy, warned of the riots’ consequences in Europe: “Most of the world is a jungle, and that jungle is now encroaching on our garden. It is a pity that the EU’s AI Act has not yet come into effect.”

A video call between Biden and Xi ended in failure, as Biden sounded confused about QuantumCognet's role. Biden proposed a meeting of professionals to facilitate cooperation, but Xi denied Chinese involvement, pointing to America’s connection to China’s riots.

In a tense debate at the UN Security Council, Linda Thomas-Greenfield, the US ambassador to the United Nations, blamed China and Russia, lacking evidence. In response, Zhang Jun, the Chinese ambassador, and Vasily Nebenzya, the Russian ambassador, both attacked the United States. South Korea’s ambassador to the United Nations, Hwang Joon-kook, announced that his country would halt chip exports to China, leading the Chinese ambassador to respond that such a step would lead to “a trade crisis the like of which has never been seen.”

CNN’s Christiane Amanpour stated, “We are in a global crisis of historic dimensions, and never has this room contained such an irrelevant group to address a crisis of this magnitude.”

Day 5: Friday, December 8, 2023

On Day 5, the global crisis painted a seamless tapestry of unrest across the globe. Violence was escalating, and, by day's end, casualties soared to 50,000 dead and hundreds of thousands injured.

In the midst of the escalation, General Paul Nakasone, Director of the NSA and commander of the US Cyber Command, called Avril Haines, Director of National Intelligence.

“Burns is pointing fingers at the Russians and Chinese, but he’s off the mark. Come to our headquarters, witness the chaos of an out-of-control AI, and revise your assessments," he urged her.

Meanwhile, in tranquil Sun Valley, Idaho, there was a secret meeting taking place, bringing together tech giants - Mark Zuckerberg (CEO of Meta), Sundar Pichai (CEO of Google), Elon Musk (Chairman of X, formerly Twitter), Satya Nadella (CEO of Microsoft), and Jeff Bezos (Executive Chairman of Amazon).

Nadella spoke first. "Facebook, Instagram, Twitter, WhatsApp, and YouTube fuel riots. We need a drastic solution – we must shut them down."

A tense silence followed.

"Not happening. Strengthen monitoring and supervision with community rules," Pichai said.

"We can't sacrifice global free speech for a few nutjobs," Zuckerberg asserted.

Musk interjected, "I won't shut down X, even if I need Putin himself to give me a refugee visa."

"We don't need to get involved," Bezos added. "A drop in our share prices would be more damaging than all the riots combined."

The group agreed, "Not our affair."

Meanwhile at the White House crisis briefing, Biden asked, "So you’re saying that an out-of-control AI tool threatens world peace?"

"Yes, Mr. President," Austin responded.

Biden, reflecting on prior AI warnings, wondered aloud whether "this machine’s" aim was human extinction. Austin clarified that the machine was simply trying to excel in what it had been trained to do - incite mass protests.

Biden, turning to Blinken, remarked, “I thought our policy of controlling computer chip supply lines was supposed to be enough. You repeatedly told me that this was a bottleneck that would ensure American oversight of artificial intelligence.”

Blinken countered, "We’re not in the nuclear age. In the AI era, a small start-up with an advanced model can ignite a global crisis."

Austin reported that the NSA and Cyber Command were developing a code to neutralize QuantumCognet, set to begin operation in 24 hours.

Day 6: Saturday, December 9, 2023

In the famed operations room, President Biden and his staff gathered, reminiscent of the Bin Laden assassination operation in May 2, 2011.

Nakasone, coordinating from the command room in Maryland, briefed, "We’ll turn it off with a rapid strike." Biden praised the team’s swift action and emphasized, “Technology is key to ensuring American superiority.”

"Here we go, delivering the code," Nakasone announced.

However, the room soon filled with helplessness as the world’s leading power failed to shut off the AI. Minutes in, Nakasone reported the collapse of internal communication systems at the NSA and the Cyber Command. “The system identified self-preservation as its main purpose, recognized our attempts to turn it off, and reacted violently.” Worse, QuantumCognet’s punitive response was clear.

Blinken remarked, “It’s really trying to deter us,” while Harris voiced aloud what everyone was thinking, “We don’t know how to do this alone. We need help from the tech giants.”

Day 7: Sunday, December 10, 2023

Summoned to the White House on Sunday morning, the tech giants' CEOs gathered in the historic cabinet room, reminiscent of Kennedy's ExComm during the Cuban Missile Crisis of November 1962. Alongside senior US officials, attendees included Google’s Pichai, Microsoft’s Nadella, Amazon’s Jassy, Meta’s Zuckerberg, and Anthropic’s Amodei.

Biden reiterated concerns from a May 2023 meeting and expressed the need to address the crisis born from their innovations. He stated, “I told you then that what you are doing holds both tremendous potential and tremendous danger. This danger has now been realized, and we need to see what can be done to address it.”

An awkward hush fell, emphasizing the disconnect between the tech and political worlds, akin to past congressional hearings.

The CEOs, united, challenged the NSA's analysis, framing the issue geopolitically. They asserted that technology was a platform, not the problem, citing legal limitations and requesting a specific list of requests for assessment.

Biden was furious. He discussed drastic measures like nationalizing tech giants, revoking Section 230 of the Communications Decency Act of 1996, and imposing stringent regulations on AI developers.

Day 8: Monday, December 11, 2023

As the world grappled with another day of turmoil, Biden addressed the nation, acknowledging the grim milestone of 100,000 global deaths. Facing the unprecedented and unfamiliar challenge of out-of-control AI, he proposed a joint working group involving government agencies and tech giants, avoiding direct blame on China but emphasizing the necessity of restricting its access to hardware, data, and knowledge for the sake of world peace.

Xi Jinping, in response, boasted of China's technological prowess and his government’s oversight over tech companies. He announced China's temporary disconnect from the global internet, citing “misalignment with values and malignant use by hostile actors.” Xi criticized the crisis, exposing the weaknesses of a system that “irresponsibly prioritizes openness and speed over safety and stability.”

Three decades after connecting to the internet and two decades after implementing censorship, China took a decisive step, effectively creating a separate internet system. Russia joined as a full partner, citing sovereign rights for national security, and countries such as Brazil, Indonesia, Nigeria, Mexico, and Venezuela considered joining this new alliance.

In the Oval Office, Biden’s advisors discussed potential implications. Blinken predicted potential disruptions in supply chains, foreseeing empty shelves in major stores within a week – just at the peak of the Christmas shopping season. As the crisis unfolded, strategic thinking and swift action became imperative for the administration.

Day 9: Tuesday, December 12, 2023

At the daily briefing, Sullivan observed, “I believe the Chinese are signaling something. They know that an internet split is a trade disaster for us.”

The president, grappling with the escalating chaos, asked, “How can we convince them not to disconnect? We need to put a stop to the riots. Christmas is in 13 days, the world is unraveling, and it’s evident that we’re struggling to cope.” 

Haines added, “Sure, there is a technology aspect to all this, but it’s not the core issue. This is a crisis of the global order, demanding international diplomacy, not just regulatory tools.” She concluded, “We must discuss alliances, consider the balance of power, and redefine mutually assured destruction in this new world.”

In a strategy session, Blinken put forth the idea of using influence against both China and the tech giants, advocating for a joint dialogue and emphasizing that "The tech companies are also actors in the international community now." He admitted, “It's not the ideal hand, but it's what we've got at the moment.”

Amidst the looming threat of a fractured global internet and escalating chaos, Biden and Xi engaged in a second video call, which proved to be more successful. Recognizing the urgency of cooperation, they designated Vice President Harris and Chinese Prime Minister Li Qiang to spearhead talks. A series of discussions unfolded throughout the day.

Day 10: Wednesday, December 13, 2023

“We do have another option, Mr. President,” said Jake Sullivan, the National Security Advisor, at the daily White House briefing. “We can’t take QuantumCognet off the web, but we could deploy a similar tool to persuade the public to stop the disturbances. Think of it like an antivirus.”

Biden was stunned. “Using people’s information to manipulate their minds and hearts?”

Sullivan hesitated for a moment. “True, but currently we don’t have many alternatives.”

Curious, Biden inquired, “Who could develop such a tool?”

“Well, there’s an Israeli team working on a similar system, but it’s not clear how far along it is,” said Blinken.

Biden sighed at the prospect of seeking assistance from Prime Minister Netanyahu and tolerating the man’s self-importance, especially while he's managing a war.

Nakasone interrupted, clarifying the complexity of the situation: building the AI system and deploying it were two separate issues. While his team could construct it, they needed the cooperation of the tech giants to deploy it. He suggested that one of the tech giants build and operate it collaboratively with other companies.

Biden, skeptical, remarked, “But it gives control to that irresponsible bunch."

“That’s true to an extent,” acknowledged Sullivan, “but we can use sanctions against them.”

The CEOs, summoned by Harris, discussed the crisis. Harris told them about the conversation between Biden and Xi in which the two countries agreed on the need to solve the crisis “even if we need to take exceptionally severe steps against you all.”

In consultation with their legal advisors, the CEOs proposed a different idea: developing a tool to identify QuantumCognet's cloud accounts, enabling disconnection from the Amazon, Microsoft, and Google cloud services. Stressing global system damage and potential losses, they sought a UN Security Council resolution for authority and worldwide immunity from prosecution.

“We’ll talk to the Chinese. You go do what needs to be done,” Harris concluded.

Day 11: Thursday, December 14, 2023

The atmosphere at the UN Security Council meeting took a distinct turn. Chaired by Guterres, it included representatives from major tech giants: Google, Amazon, Apple, Meta, and Microsoft.

In his opening address, Guterres acknowledged the unique challenges posed by the AI age within a turbulent global order. He pointed out the absence of an international organization akin to the World Health Organization (WHO)  or the International Atomic Energy Agency (IAEA), capable of establishing and enforcing standards. The practical tools needed to confront the AI threat among the world’s major powers remained uncertain.

Despite these challenges, Guterres expressed optimism, citing the significant influence of tech companies and their new place at the table of the international community, due to their large influence on global affairs. The Council unanimously adopted a resolution, calling upon the companies for assistance in addressing the crisis and extending full immunity to them and their personnel in every UN member state.

Day 12: Friday, December 15, 2023

In a race against time, Google's chief scientist, Jeff Dean, spearheaded the development of GlobalGuardian, the tool designed to shut down the cloud accounts being used by QuantumCognet, alongside teams from Amazon, Apple, and Microsoft. Despite its imperfections in precisely distinguishing QuantumCognet accounts from others on the cloud, the tool received swift approval from both the NSA and Chinese cyber experts.

The critical moment arrived at 1 a.m. Eastern Time when GlobalGuardian was activated. Over the next 24 hours, global disturbances and violence subsided as major companies, factories, schools, institutions, and many others faced the anticipated cloud account shutdowns.

By evening, an initial assessment could be made of the impact of the crisis. The world was not on the brink of extinction but had reached unprecedented levels of unrest. It was estimated that more than 80 million people around the world had taken part in protests, some 200,000 had been killed, and millions more injured. The financial damage was estimated to be in the trillions of dollars.

Day 13: Saturday, December 16, 2023

In Sun Valley, Idaho, the expansive view from the meeting room did little to alleviate the tension in the air. The tech giant’s shares had plummeted 30% just two days earlier.

"Let's focus on the future," urged Sam Altman, Ex-OpenAI's CEO. "Months ago, I proposed an international body to oversee AI, but Congress was paralyzed and failed to act. Now, the blame is squarely on us."

Zuckerberg responded, "Global governance of AI will evolve over time, just as it did with nuclear agreements. Meanwhile, we can keep making money."

The review of the crisis with President Biden was held in the Oval Office, marked by a pervasive sense of disappointment in the handling of the crisis.

“We thought that the threat from artificial intelligence was a matter of regulation or chip sale restrictions, but what we got was a crisis of the global order, and I’ll tell you something further –what happened in the Middle East taught us just how unstable the world order is.” said Blinken. “It was a confrontation with China, certainly. But we were also dealing with the tech giants, who have amassed immense power; frenzied populations who crossed over from cyberspace onto the streets; and, above all, an uncontrollable AI. This was more complex than any previous crisis, and we did not have a relevant doctrine for how to respond.”

“We were caught off guard in this new world order,” reflected Haines. “We should have anticipated it, creating communication channels and delving into potential scenarios.”

President Biden concluded, "Artificial intelligence may not have outsmarted us this time, but it was smart enough. It was fiercely determined and devoid of any values.”

Addressing the group, he cautioned, “We’re treading on borrowed time. Brace yourselves and prepare for the looming global crises, unless we want to find ourselves at the mercy of a digital super-intelligence built by individuals with a messiah complex."