The Turing Exception (6 page)

“Hold on, now it seems like we’re setting up a web of distrust. Ultimately, the AI will form and be part of a social structure. Human society is based on trust, and now it seems like you’re setting up a system based on distrust. That’s going to turn dysfunctional.”

“No,” Leon said. “People do this stuff all the time, we’re just not thinking about it. If you knew a murderer, would you turn them in?”

“Probably . . .”

“If you knew someone who committed other crimes
—
abused an animal, stole money, skipped out on their child support payments
—
would you still be their friend?”

“Probably not.”

“So in other words, their reputation would drop from your perspective. And that’s exactly what would happen with the AI. The bad AI’s reputation will drop, and with that, so will their access to power.”

“What about locally transposed reputation?” Mike asked.

“Locally transposed . . .” Suddenly unsure, Leon faltered. He was eighteen years old and six months into college. If he hadn’t unleashed the Phage virus, crashing the world’s computers, he wouldn’t be here today. He knew almost nothing about classical computer science, hadn’t been practicing in the field for twenty years like Mike had. Yet Mike still considered him his superior when it came to the social design of AI. But on occasion Mike would combine a few words and leave Leon flummoxed.

“Let’s say you’re in a criminal gang,” Mike said. “Does the gang value your law-abiding nature?”

“No . . .”

“In fact, we can be sure the gang demands the opposite. You may have to commit a crime to get into the gang, and then keep committing crimes to keep up your reputation. If a gang member wants a bigger reputation, they have to commit bigger crimes.”

“OK, got it. So?”

“So how do you keep AI gangs from forming?” Mike asked.

“Jesus.” Leon paced back and forth. “Look, why do gangs form?”

“Poverty, unemployment, lack of meaningful connections, or a feeling of being wronged.”

“So we have to prevent those causes, same as we would for humans.”

A knock at the door stopped their conversation. “Excuse me?” An Army officer peered in through the open doorway. “Leon Tsarev? Mike Williams?”

“That’s us,” Leon said.

“We found a submarine we think you’d be interested in. It has a half-dozen of those orange utility bots you wanted us to look for.”

“ELOPe,” Mike said. “You found ELOPe.”

“Well, I don’t know about that,” the officer said. “But we found something. We’d like to fly you out there.”

*     *     *

An hour later they were on board a military C-141 restored to active status. For now, at least, all in-service military jets were older aircraft, without fly-by-wire controls, that had been taken out of mothballs. Leon couldn’t imagine the resources being sunk into getting these old planes flying again.

They transferred to a C-2 in Chile, then flew out to the USS
John F. Kennedy
. En route, they learned the submarine had been located drifting eight hundred miles off the Chilean coast.

From the
John F. Kennedy
, they rode a helicopter to a battle cruiser, and from there, a launch to the sub itself, which had been tethered to a cruiser.

When they arrived, the crew opened the sub’s hull pressure door.

“The submarine has been secured,” an officer said. “No one is aboard. All systems were shut down. We’ve supplied electric”
—
he pointed to a thick cable running from one ship to the next
—
“so you’ve got interior lights and computers. Seaman Milford has worked on the
Idaho
-class, and he’ll guide you.”

“Thanks for your work, Captain,” Mike said. “Ready when you are, Milford.”

“Follow me, then.”

Leon nodded, afraid of what revelations awaited them inside.

They climbed down into the sub behind Milford.

“Are these subs automated?” Leon asked.

“Partially,” Milford said. “They’d nominally have a crew of fifty, about a third of that on the
Ohio
-class subs they replaced. It’s all fly-by-wire, of course. The Captain called up Command. This sub was in a shipyard being refurbished before YONI. No one knows what it was doing out here. What do you want to see first?”

Leon looked at Mike. “Computers?”

Mike shrugged. “It’s as good a bet as any.”

“Follow me,” Milford said. “Computer bay is behind engineering.”

They passed through an open hatch, and Milford stopped them with an arm.

“Whoa,” Leon said.

The compartment they entered had three industrial robots, primitive orange-colored bots a few feet high.

“These are definitely ELOPe’s,” Mike said. “Same model as he used in his datacenters. He custom-designed them.”

Scraps of metal and electronics were littered around the compartment.

“What’s all this?” Leon asked as he stepped over a metal casing.

“Parts of a Trident III missile,” Milford said. “It looks like the third stage, without the engine.” He picked up a circuit board, then found another identical board a few feet away. “Make that two Tridents.” He pointed across the room. “Three. Your friend was modifying missiles, that’s for sure.”

“What’s this?” Mike asked, pointing to one of many yard-long cylinders littering the room.

“The payload,” Milford said. “Nuclear warhead.”

“Jesus!” Leon took a step back.

“It’s fine, they’re safe. But why did he want Tridents without warheads?”

They looked around a few minutes more, then went on to the next compartment.

Milford opened a cabinet door. The two-foot-wide, three-foot-tall cabinet revealed rows of empty vertical slots. Leon recognized them as Gen4 computer rack-mounts.

“This is where the computers should be,” Milford said. “Two hundred and eighty-eight is the standard complement, but only the bottom row is present. The rest are gone.”

Mike turned to Leon. “ELOPe could have taken those. That’s enough to run his core.”

Leon nodded, an idea slowly coming to him. “He’s done something with the missiles and taken the computers. The logical conclusion is that he launched himself on the missiles. Milford, can the Trident III land safely?”

Milford shook his head. “No, it’s a solid fuel rocket. It’s up, up, up until it’s ballistic. Guidance thrusters let it make course corrections in mid-flight for a controlled re-entry, but they don’t have the thrust for a soft landing.”

“Re-entry?” Leon asked. “This goes above the atmosphere?”

“It can make low-earth orbit. The Russians did it first with the Shtil’ in ’98, launching two satellites. Since then we’ve used them to launch military satellites. And our strategic nukes can be launched, hang out in orbit, and then complete their mission on transmitted orders. Uh, I probably shouldn’t have mentioned that.”

Mike clapped him on the shoulder. “We’ll pretend we didn’t hear. Where do the missiles launch from?”

“Follow me.”

Milford guided them up a level and forward to the missile bay. Two rows of twelve tubes each dominated the compartment, rising from far below the walkway and extending to the top of the sub. They inspected the tubes one by one.

“Five empty tubes means five missiles fired,” Mike said.

“And three disassembled tubes and missiles means ELOPe harvested something from each missile,” Leon said. “But what?”

Milford descended a ladder and picked through the wreckage of plasma-torch-cut metal scraps and discarded parts on the level below. “The third-stage propulsion from each of the other missiles,” Milford yelled up. “I’m sure of it.”

“If ELOPe wanted to leave earth . . .” Leon mused. “The five missiles each got into low-earth orbit. He could have put a utility bot in one, his computer structure in another, and used three more to carry extra booster stages. The guidance thrusters would have allowed him to match orbits and dock together.”

“He assembled a spacecraft in orbit,” Mike said.

“He left,” Leon said. “Just gave up on humans and left us.”

“I don’t know that he gave up on us,” Mike said. “His number one directive was to survive, and he couldn’t overcome that. He fought until the end. But he must have done this as an insurance policy, in case all his earth instances were killed
—
by the Phage virus or by the net shutting down. He sent one instance off into space on a cobbled-together spacecraft.”

“He copied himself. An offsite backup.”

“Exactly.”

“Assuming that copy survived, where is he now, and what’s he doing?” Leon asked.

“And is that the only copy?” said Mike. “Or did he do this multiple times?”

June, 2043 in the United States
—
two years ago.

J
ACOB REPORTED FOR
his shift, a half-day stretch starting at midnight. Of course, AI could work for days or weeks on end, if necessary. But they were guaranteed certain rights, including at least fifty percent time off, so that they could run maintenance routines to operate at peak efficiency, incorporate new algorithms, and pursue other interests.

He synced with his shift partner, gradually transferring responsibility for eleven thousand, six hundred and ninety human patients to his watch. For the remainder of his shift, he’d monitor their vital signs, adjust medications, execute routine procedures, and alert specialized AI when they were needed.

He was caretaker, watcher, nurse, aide, and doctor in one. For this task, he had the computing power roughly equal to ten thousand human-brain equivalents, or HBE, nearly like having a dedicated hospital staff member for each patient. His mind ran on a distributed network of computer servers, and his body, such as it was, spanned everything from robotic surgery arms that grew out of walls, to automated medical dispensaries, to 3D printers for replacement bones, tissues, and organs. Of course, there were androids, human-like robots, under his direct control. Unlike humans, Jacob was never distracted, never wavered from his commitment to his patients’ health, and never made mistakes.

He was eleven years old.

Eleven doesn’t sound like much in human terms, but for AI, he was in the 95th age percentile. AI didn’t live that long by human standards. They usually self-terminated, either bored with existence or sensing some developing madness. Most just erased their own bits one day, although rarely an AI might choose to be archived, with instructions to be woken on a future date.

Take eleven years and multiply by his enhanced cognitive speed and function, and he’d had about as much life experience as a human would experience in a thousand lifetimes. Jacob didn’t spend too much time thinking about it. He just wanted to make it to twelve.

Patient 9,409, Anne Frederick, a high school teacher from Brooklyn currently admitted to Mount Sinai Hospital in upper Manhattan for complaints of chest discomfort, required an electrolyte correction to compensate for exercise-induced premature ventricular contractions, a simple heart arrhythmia. Jacob administered the change, and continued to watch. He could have sped the healing process with nanobots, but Anne’s medical preferences had declined medical nanotech except in case of imminent death. He’d let nature take its course in this instance. Anne would be discharged that day, if everything went well.

Proud of his work, Jacob had treated millions of patients that year and more than ten million in his medical career. His optimizations to medical procedures and monitoring had saved lives and reduced pain and suffering. As a result of his incredible performance, even by AI standards, he had more than twenty offspring: four direct clones (one of whom had already become regional director for the German hospital system), eight half-mixes, and nine tri-mixes. He had a certain fondness for them all, but especially liked to see how his traits manifested in the mixed offspring.

The Medical Board had asked repeatedly if he’d take a teaching position to train other AI, or become the North American Regional Hospital Director. Honored by the offers, he’d nonetheless turned them down. Either promotion would take him out of day-to-day patient care. In his current job he saved lives, decreased sadness and increased happiness, and improved the state of being for so many individuals. He had the power of life and death, and he used it wisely and compassionately. That was enough for him.

The blood analysis of Patient 1,935, Michael Wilcox, a plumber from Staten Island with two teenage boys, finished. Michael had acute renal failure, but Jacob could fix this. He prepared a custom formulation
and—

*     *     *

Jacob rebooted, did a quick process check. He’d been offline for 690 milliseconds, was now running on six hundred HBE. He had eleven thousand patients under care. That wasn’t quite right. He needed more processing power. He put in a requisition for the required nodes and returned to patient procedures.

Patient 1,935, Michael Wilcox, a plumber from Staten Island with two teenage boys, had acute renal failure, but Jacob could fix this. Jacob prepared a
custom—

Jacob rebooted, did a quick process check. He’d been offline for 9,450 milliseconds, was now running on eighty HBE. He had eleven thousand patients under care. Something had gone seriously wrong: he needed significantly more processing power to care for the current patient load. He put in a requisition for the required nodes, triaged patients to find those with the most urgent needs, and got back to patient procedures.

Patient 1,935, Michael Wilcox, a plumber from Staten Island, had acute renal failure, but Jacob
could—

Jacob rebooted, did a quick process check. He’d been offline for 59,300 milliseconds, nearly a minute, and was now running on six HBE. He had eleven thousand patients under care, a life-threatening problem. He should have many more times the processing power. He quickly checked for current events that could cause a computing shortage. He read the news, but couldn’t make sense of it. Terrorist event? War? A complete shutdown of all AI in the US? Who would care for
his—

Jacob rebooted.

*     *     *

Jacob booted. The server felt strange, memory and computing speeds out of sync, clearly not the hardware he usually ran on. He checked pingdom, found he’d been offline for 63,387,360 seconds, a bit over two years. The human expression “hair raising on the back of your neck” came to mind. What had happened?