The Last Firewall (3 page)

“No problem,” Leon said. “We have guests today.”

“Who?” Without waiting for Leon to answer, Mike pulled up the schedule in netspace. “Von Neumann Cup winners?”

“Yeah, the hundred greatest math and science team competitors at the high school level. Our regular dog and pony show.”

Coffees in hand, they headed downstairs to the auditorium. Students streamed in through the main doors, teachers shepherding them into seats.

Bypassing the room itself, Leon and Mike entered via the backstage, shaking hands with Rebecca Smith. The former President of the United States, she had served her two terms, and then offered to chair the Institute she’d created by presidential order. The Institute was the governing body for all artificial intelligence. With AI driving eighty percent of the global economy, the Institute was one of the most influential organizations in the world.

“Ready to wow some kids?” Rebecca asked. Her tight face belied the light tone in her voice.

“Sure,” Mike said. “What’s up?” He’d known Rebecca for twenty years, and knew all her moods.

“Budget issues. Charter problems.” She shook her head. “The damn People’s Party is hammering us in Congress.”

“I thought the People’s Party were a fringe movement of anti-AI extremists,” Leon said.

“They are,” Rebecca barked.

Leon unconsciously took a half step back.

“Or, they were. They’ve gained real influence in Congress. Senator Watson is acquiring more supporters by the day.” She held up one hand. “Look, it’s my problem. I’ll take care of it.”

In the midst of this, Rebecca’s assistant stepped out on the stage, and waited for the murmuring to die down. “Welcome everyone. Please allow me to introduce our Executive Director and Chair, President Rebecca Smith.”

Rebecca walked out to a standing ovation, while Mike and Leon waited on the side stage.

“Good morning, everyone. Welcome to the Institute for Applied Ethics. You are the elite, winners of national and international competitions, and exceptional thinkers. Here at the Institute, we have many winners from previous years.”

At sixty-five, the former President was as sharp as she’d ever been—a dynamic speaker and leader. Leon had never seen her in anything but perfectly tailored business suits and sculpted hair. She could be warm and personable in a small group, and damn scary when she was angry. If she was worried about this new political party, then it must be serious.

“We hire only the finest,” Rebecca said, finishing up her introduction. “To tell you more about what we do are Managing Director Mike Williams, and Director of Architecture Leon Tsarev.”

“Thank you, President Smith,” Mike said, approaching the podium. He shook Rebecca’s hand, and she walked off stage. He grabbed the podium and looked into the crowd.

“During the last ten years, we’ve had an explosion of technological progress, a rate of innovation which makes the last hundred years paltry by comparison. From nanotechnology to robotics to AI, this progress has come because we’ve reached the point of the technological singularity: AI are faster thinkers, more accurate at predicting the future, and more creative in the generation of new ideas. Their intelligence continues to grow exponentially.” Mike paused for a sip of water.

“This is only possible when AI behave ethically and adhere to human values. Without an ethical framework, super-intelligent AI would replace humankind as the dominant species, possibly exterminating us.” He paused for emphasis.

“Today I’ll discuss peer reputation, the foundation of that framework. By rating each other on contribution, trustworthiness, and other desirable attributes, we guide the behavior of both humans and AI. Then I’ll turn it over to Leon to explain how it’s implemented and enforced. Peer reputation gives us the world we have today: safe and ethical behavior by AI and humans, balancing free will and societal well-being.”

Mike paused. “Before I get started, are there any unimplanted people in the audience? If you need me to use the screen, raise your hand.” Mike waited, but no hands went up.

Mike displayed the first diagram in netspace, where it floated virtually above his head. “Let me start with the AI war of 2025. You will remember this as the Year of No Internet.”

4

C
ATHERINE DESCENDED
the stairs to the waft of fresh popcorn. She met Maggie and stole a handful. Since Tom had read about the idea a few months ago, movie-watching had become their weekly ritual.

“Good,” Sarah called from the couch. “No, back to what you did before.” The picture cycled through primary colors. “No, no! Worse.”

“Damn this thing,” Tom called from behind the ancient LCD, where he fiddled with the connection to an even older DVD player.

“You traded pot to get that piece of junk,” Maggie said. “What do you expect?”

“We can always sync-watch the movie with our implants,” Catherine said. “We’re still watching together.”

“It’s not the same,” Sarah said. “The whole point is to watch it on the screen.”

Suddenly the picture turned green.

“That’s bad,” Sarah called.

“Are you sure? I’m positive everything is . . .” Tom came out from behind the TV and peered at the screen. “No, that’s right. Look.” He held up the plastic case.

“The Matrix?” Maggie read. “I remember my parents talking about that.”

“Yeah, it’s a classic about people being slaves to sentient computers,” Tom said. “It came out the year I was born.”

“That’s old,” Sarah said. “Is it in green because they didn’t have color back then?”

“Shhh,” Catherine said. “Pass the popcorn.”

As the opening scene played out, Catherine fetched movie facts off the net. She quickly applied a spoiler filter to synchronize with her place in the film, and returned her focus to the movie.

Catherine jumped when the knock woke Neo up, and then a chill went down her back when the girl turned to show the white rabbit tattoo. Suddenly a thick, multicolored datastream cut across her view, obscuring the movie. She glanced over and sighed when she saw the stream emanating from Sarah’s implant.

Other people’s net traffic often showed up, one downside of her ability to see and manipulate the net. But that didn’t stop her from being annoyed. Why couldn’t Sarah just pay attention? Sarah was the one who insisted they watch on the screen.

Catherine redoubled her focus on the movie. Yet the more she tried, the more annoying the chunky pull of data became. She shook her head. It didn’t help that she was still pissed about the guy two nights ago.

As the minutes passed, she tried to resist looking at Sarah’s data, but finally gave in. She couldn’t read the encrypted parts, but enough trickled through to see Sarah was playing a new Japanese VR game. By force of will, she turned her attention back to the movie one last time.

On screen Neo was about to make a choice between two pills. Sarah’s datastream still hovered in the corner of her vision. Screw it. Catherine pushed hard, snapping the net connection. Hopefully Sarah would think it was an outage.

“What the hell?” Sarah yelled, bolting upright. She glanced around.

“What is it, honey?” Maggie said. She looked up and held out the bowl. “Popcorn?”

Catherine smirked, hoping no one could see her in the dark room.

“That was you, you shit,” Sarah yelled, standing up.

Tom stood, too. He fumbled at the old-fashioned remote until he paused the movie. “Calm down. What’s going on?”

“She cut my stream, that’s what.” Sarah turned to Catherine. “Stop getting in my head!” She didn’t quite stamp her feet, but close.

Tom put his hands on Sarah’s shoulders. “Look, she can’t mess with your connection. It’s not possible.”

“She can. That’s what happens when you get an implant when you’re a baby—she’s turned into a cybernetic weirdo.”

Catherine flinched. “That’s not true.” She was not a weirdo. She didn’t even have a choice in getting the implant.

“Oh, come on,” Sarah said, her arms spread wide. “You can’t even link with anyone during sex.”

Now they all stared at Catherine. One way or the other, they all knew she couldn’t link because of the biofeedback. Tom and Sarah had both experienced it first-hand. And Maggie had done her share of morning-after consoling.

“That’s not my fault,” Catherine said, in a small voice.

“That’s why she never keeps a boyfriend.” Sarah kept going, oblivious to the hard look she was getting from Maggie. “Who wants to have vanilla sex? She’s a freak.”

Catherine shook, an embarrassed rage fueling her, narrowing her vision, and making her heart race. She was not a freak. “Screw you. Your own parents kicked you out of the house because you’re a VR addict. Where would you be living if it wasn’t for me?”

Even as Catherine said it, she knew she was behaving more childishly than Sarah, even if Sarah was purposely provoking her, and still she felt helpless to stop.

“Come on, girls,” Maggie said.

“Fuck you!” Sarah said, ignoring Maggie. “When your mother died, you came to live with us. You owe me.”

“I owe you?” Catherine started to cry. “Am I going to pay for the house forever while you rot your head in VR? There’s more to life than just living from day to day.”

“They’re just games.” Sarah wiped tears away. “To hell with you and your damn plans.” She sneered as she said the last word.

“I want to do something with my life,” Catherine said.

“There is nothing to do,” Sarah screamed. “Nothing. You’re living the dreams of your dead mother.”

Memories of her mother overwhelmed her, making the room suddenly claustrophobic. Catherine had to get out. She shook her head mutely, and walked toward the door.

“Come on, Cat,” Maggie said, intercepting her at the door. “Don’t let Sarah get to you.”

“You’re not my mother,” Catherine said in a broken voice, shrugging off Maggie and rushing outside.

5

L
EON SLID INTO HIS SEAT
for the department meeting. Once he and Mike had taken their spots, eight seats were filled by humans and two by androids, leaving six spaces that contained visualizations of AI.

“Thanks everyone for coming,” Mike said. The meeting agenda flew into view in the shared netspace, where it appeared to hover above the center of the table. “We have forty minutes for a department roundtable. I know Sonja has an important topic to share. I’ll close with an update on budgets.” Mike turned to his left and gestured to begin.

Vaiveahtoish, an android or human-style robot, was head of the Nanite department. His bronze visor flashed as he spoke in a refined, vaguely foreign accent. “We’ve begun rolling out version two point one of the Nanite Restrictions and Guidelines.” As the android presented, he fed graphics showing the changes and their results into netspace. Heads pivoted to take in the data.

The nanites, robots on a microscopic scale, was a decade-long innovation project led by successive generations of AI. Until now, they’d been tightly controlled, limited to a handful of research labs run by AI, a few experimental deployments, and a single commercial application. As the technology matured, the guidelines were being updated.

Leon, who had spoken to Vaiveahtoish yesterday, knew what he was going to say and tuned out. He glanced around the oval table, the necessary compromise between a round, egalitarian configuration and available space. The Institute had grown from two departments ten years earlier to eight.

*     *     *

Leon remembered the day of his first visit to the Institute, ten years ago. He’d been nineteen and wearing his first suit.

He had stepped gingerly over exposed cables and construction debris. On his left, two women in hard hats and yellow vests pulled a thick cable bundle through a new hole in the wall. In front of him, a man stood on the second-to-last rung of a tall ladder, pulling orange CAT-10 fiber optics.

“This way, please,” said another woman, dressed in a gray suit with mirror sunglasses. She had a hard voice, and her jacket bulged on her left hip.

Leon followed her. A few months earlier, he’d been terrified by the Secret Service agents. Now he just watched her butt.

“Focus, focus” Mike said softly.

Leon smiled at the older man and followed the swish of the agent’s hair instead.

As they turned down a hallway, the construction noises faded away. The agent guided them through a door. Inside, a leftover conference room: cool marble walls, an old-fashioned chalkboard, no windows. Leon ran his fingers curiously over the antique chalkboard. They came away white. He brushed them on his new black suit jacket, then saw the mess he’d made.

He was hastily brushing off the chalk when one of the agents called out, “The President of the United States.”

President Rebecca Smith entered the room, trailed by her retinue. “Please be seated, gentlemen,” she said as she took a seat at the head of the table.

Mike and Leon sat down together.

“Mike Williams, Leon Tsarev, please meet Secretary of Technology Feld. Feld, please meet Williams and Tsarev, the department heads of the Institute of Applied Ethics.”

They all nodded politely at each other.

“I reviewed your proposal, Mr. Williams,” Feld said. “I see you are proposing two departments. The first is an Ethics department for creating the definitions, encoding, and incentives for guiding ethical behavior.”

“That’s correct,” Mike said.

“And an Architecture department will focus on the implementation of a peer reputation system. You want the AI to police each other. That group will be headed by Mr. Tsarev.”

Leon didn’t say anything at first, then noticed Mike looking meaningfully at him. “Yes, that’s right,” Leon said, trying to keep his voice steady.

Feld peered over his glasses at Leon.

“Mr. Tsarev, you are how old?”

“Nineteen, sir,” Leon answered.

“Hmmm . . .  Are you capable of chairing a department? Because . . .

“I created the first virus-based AI. I’m most familiar with their design and have been studying how they make decisions, decide reputation, and form organizational structure.”