Cyber Rogues (9 page)

Hector propelled himself across the floor of the kitchen, stopped in front of the broken window and paused while FISE considered the situation.

“What happened?” Dyer asked. Ron, who was standing with his elbows resting on the opposite side of the tank, raised his head.

“We told him that the garbage pail had to go out in the yard.” he explained. “So he threw it through the window.” Dyer grunted and returned his gaze to the tank.

Hector reached out and grabbed hold of one of the jagged fragments of glass that remained around where the window had been. PROPS immediately caused a vivid red line to appear across Hector’s hand. The gash proceeded to ooze blood profusely but Hector ignored it and continued tugging experimentally at the piece of glass in an effort to remove it.

“Hold it. Hold it there, Chris,” Ron called out. The figure in the kitchen froze. “Now FISE,” Ron said, adopting his stoic voice. “There are a few more things that you have to get straight about Hector. Glass cuts. Hector does not like being cut. You don’t cut bits off him or permit him to be cut by anything if you can avoid it. Okay? You have to find a way to fix the window without cutting Hector in the process.” A few seconds elapsed while Chris completed keying in the last addition to FISE’s growing store of information.

“Question,” FISE’s voice said from the grille.

“What?” Ron inquired.

“When Hector was shaving, his hair got cut. Why was that okay?”

“Oh yeah, I forgot that,” Ron agreed. “When any part of Hector’s body starts to get cut it hurts, just like you already know for things that are too hot. When he feels that, he’ll respond with a reflex that overrides everything else he’s doing. Hair is an exception. It doesn’t hurt when it’s cut. An unshaven face is not a nice thing. Shaving in the morning is okay.”

“Thanks,” FISE acknowledged.

“Before you go any further, let’s just try something,” Dyer suggested. “I’d like to see how well it understood what Ron just said. Chris, could you reset to the point just before where Hector grabbed at the glass, and force the same action.” Chris took a while to compose the commands. Dyer and Ron watched intently as Hector flashed back to his previous starting posture and approached the window once more. He grabbed at the glass as before but this time his hand jerked back again instantly. PROPS could justify no more than a slight scratch.

“Not bad,” Ron conceded, sounding impressed.

“I’m almost tempted to suggest that we might be safe in upgrading his IQ to one,” Chris murmured, leaning back in his chair to stretch his arms.

Dyer felt a sudden urge of excitement They were getting there! It was a slow and tedious business, certainly, but the first signs were there. It was all beginning to come together. To cut it off at this point would be tragic.

“Reset again, Chris, and let FISE handle it himself,” he said. “Let’s see if he can figure out a better way.”

Hector tried several approaches, including wrapping his hand in the tablecloth and then in a towel, but Ron vetoed all of them. Eventually FISE gave up and Ron supplied a hint by suggesting that if Hector looked in the utility closet, he might find something with which to knock out the pieces of glass.

“The hammer is used for knocking things, but it would break the glass,” FISE protested. “You told me that breaking glass isn’t okay. What am I supposed to do?” Ron got excited again and delivered a lengthy exposition on the profound insights required, after which Hector made a reasonable job of clearing and cleaning out the window frame. Chris told PROPS to materialize a pane of glass and Hector placed it squarely in the frame after first, on his own initiative, stopping to put on a pair of gloves that just happened to be in the utility closet.

“This is interesting,” Dyer commented. “Look. He didn’t just turn away. He’s waiting and watching the glass. FISE has connected it with something else he’s learned somewhere that’s telling him it might not be very stable.” Sure enough, PROPS weighed up the shape and angle of the pane, couldn’t make up its mind and flipped a random number. The pane began to fall inward. Hector stepped forward, caught it in one hand and repositioned it more carefully.

Three enthusiastic roars of approval greeted the performance. For once, Ron treated FISE to a jubilant stream of ungrudging congratulations. Chris reconsidered his earlier statement and suggested that the machine might qualify for whatever IQ category lay above one. Although the thought had been half in his mind, Dyer decided it was not the time to mention the things that had been said in Hoestler’s office earlier in the afternoon. After all, he told himself, Lewis had not gone further than using the word “probably” several times. Nothing firm had been decided yet.

“Chris and I are gonna go eat out while we’re deciding where to go later,” Ron called across the lab as he pulled on his coat. “Want to join us, Ray?” Dyer looked up from the console, where he was studying some of the new coding linkages that FISE had constructed in the course of the afternoon’s exercise.

“Huh? Oh, no . . . I think I’ll stay on a while. This looks interesting. I’ll have to take a rain check on it. Thanks all the same.”

“Okay. See ya tomorrow.” Ron moved away toward where Chris was standing waiting by the door. “Chris, great idea!” he said as they began moving out into the corridor. “Why don’t we catch a game later? There’s one on tonight that I promise ya is going to be terrific.”

“Netball or rounders?” Chris’s voice inquired disdainfully.

“Hey, you’ve said that before. What is all this stuff?”

“Oh, just a couple of English games,” Chris told him matter-of-factly.

“Really? Big-league stuff and all that, eh?”

“No. Actually they’re normally played by schoolgirls.”

“Schoolgirls?” Ron’s voice rose in sudden outrage. “SCHOOLGIRLS! Hey, exactly what the hell are you getting at? If you’re telling me what I think you’re tel—” Dyer grinned to himself as the exchange faded away. Judy left with Betty and a few minutes later Allan tossed across a sheepish goodnight and went out to join Pattie, who was waiting in the corridor. Dyer returned to the displays glowing on the console.

One of the basic objectives of FISE was to investigate ways of enabling the computer to make generalized inferences from a few specific experiences, in much the same way as a child learns. The incident with the window bothered Dyer despite its encouraging aspects. FISE hadn’t been able to generalize sufficiently. If he knew enough not to allow Hector to burn himself, he should have been capable of generalizing to the extent of not allowing Hector to harm himself at all. Dyer had an idea where the problem lay and he spent the next two hours building a new section of system code and modifying some key parameters. Then he switched on the tank again to set up some situations to test out what he had done.

He gave Hector a few simple tests to check that he had not inadvertently introduced any major bugs, and then left Hector to put the tools away while Dyer thought about the next part. He wanted to find out if Hector would deduce for himself that an object in contact with a hot object would itself get hot and should therefore be avoided.

At that moment the kitchen door was nudged open and Brutus ambled in. Brutus was Hector’s comical white dog, a mischievous scamp with a black patch around one eye. He tended to appear from time to time when PROPS was getting bored. Dyer smiled faintly as he watched. Brutus wandered on into the room and Hector’s head suddenly jerked around to look directly at him, which meant that FISE had just become aware of the dog’s presence.

Nothing unusual happened for a while. Brutus sniffed here and there as PROPS commanded and Hector looked around every now and again to keep track of his movements, giving an uncanny simulation of human behavior in the process. And then Brutus moved over to the window. At once Hector rushed over from the utility closet, scooped Brutus up in his arms, hauled him across to the door and threw him outside.

Dyer froze the image and blinked at it in amazement. Then he became excited. He hastily composed a message requesting an explanation for the action and played it into the console. FISE’s voice responded at once.

“There are pieces of broken glass on the floor by the window. Glass cuts. I must not allow Brutus to get cut. It’s not okay.”

Dyer was astonished. Nobody had ever said anything about Brutus getting cut; they had told FISE only about protecting Hector. Dyer queried the point accordingly.

FISE replied, “I am hurt if I get burned or if I get cut. Brutus is hurt if he gets burned. Therefore Brutus is probably hurt if he gets cut too. Brutus is like me. Things like me and Brutus do not like being hurt, I must stop us from getting hurt if I can. Allowing things like me and Brutus to get hurt is not okay.”

Dyer was stunned. FISE had made the gigantic step of generalizing Hector’s basic attributes to include other animate objects . . . and without being told to!

He stared across at the blank cubicle that housed FISE and shook his head in wonder. To kill it now would be madness, he thought to himself. Just twelve more months at this rate . . . He sat up with a start.

Kill it? Christ! He’d caught himself thinking about it as if it really were a living being. He clicked his tongue in self-reproach. Getting sentimental about a machine. That would never do. What’s the time? Hell! Eight already. Time to go home and fix something to eat. Sharon will probably call. Not really interested . . . Eat out and see if any of the guys are around tonight.

He shut down the system and walked over to the door to collect his windbreaker from the stand. At least, he thought as he doused the lights and turned to survey the deserted offices, he was now certain in his own mind of one thing: Ripping HESPER out of TITAN was not the way to go. Backward was never the way to go. Given some improvements, FISE would never make the kind of error in judgment that the HESPER machines at Tycho had made. The way to get things right was to go full speed ahead on perfecting FISE and getting it into the net, not shrinking back from it and running the other way.

When he was halfway down the corridor he remembered something else: Three years previously he had been just as certain about HESPER. Suddenly he didn’t feel so sure.

None of the guys were in town that night but he found that he really didn’t care very much. He had enough on his mind to occupy him.

CHAPTER SIX

“A lot of people are starting to say that TITAN could go just that way. What do you think, Ray? Could it evolve the capacity for feeling emotions? Could it develop an awareness of its own existence?” Dr. Jacob Manning, one of the three who had arrived from Princeton earlier in the day, put the question while they were summing up in Dyer’s office after seeing Hector in action. The subject of the discussion was the notion that TITAN might integrate its capabilities on a global scale sufficiently to emerge as an intelligence in its own right.

“Obviously we can expect to see its behavior becoming more coordinated worldwide as time goes by,” Dyer replied from where he was sprawled in leisurely fashion behind his desk. “If you take as a working definition of intelligence: ‘A measure of a system’s ability to learn from experience and to modify its own behavior appropriately to what it has learned,’ then we’d have to concede that TITAN has taken a rudimentary step in that direction already. So yes—it could become intelligent. But I think it would be a mistake to draw too many inferences based on the human model.”

“How do you mean—emotions and that kind of thing?” Sally Baird, also from Princeton, spoke from the far corner.

Dyer nodded. “What are emotions?” he asked. Without waiting for an answer, he continued. “I’d submit that an emotion is a stereotyped behavioral pattern that’s been reinforced through natural selection because it has demonstrated a survival value. Obviously an
animal
that gets mad and fights or gets scared and runs stands more chance of staying healthy than one that feels nothing, does nothing and gets eaten. You buy that?”

“I’ll buy that,” Sally agreed.

“Good. So if you take a more general view of it, an emotion is a behavioral tendency that a self-modifying system evolves because it is beneficial in helping the system to accomplish whatever its basic programming compels it to want to accomplish. In the case of organic systems that have arrived via the mechanics of organic evolution, the ‘whatever’ happens to be survival.”

“Yes, I think I see what you’re driving at,” Steve Conran came in from beside Manning. “You’re saying that an inorganic intelligence might well evolve its own compulsive traits but there’s no reason why they should bear any resemblance to human emotions. Our emotions derive from the survival needs and wouldn’t have any inherent value to a system that came from origins that were totally different.”

“That’s what makes the most sense to me,” Dyer affirmed.

“Interesting,” Manning mused, half to himself. “I wonder what traits they’d be. As Ray says, they’d probably reflect whatever basic function the system had been designed to perform.”

“Well, in the case of something like TITAN, it might turn out to be insatiably curious . . . or compulsively rational, or efficient or something,” Conran suggested. “Certainly, if you look at it that way, there’s no reason why it should become a threat to anybody, is there?”

“So how about awareness?” Sally asked. “What’s the likelihood of it getting to the point where it not only knows, but it knows that it knows?” Dyer was nodding even before she had finished asking the question.

“Again, I think that if it ever came to possess anything like that, it would be radically different from a human understanding of awareness.” He paused to collect his thoughts for a second while the three visitors waited expectantly. “A man is aware of himself as existing in the localized region of space that’s defined by the focal point of his senses. He has evolved the ability to construct mental models of extensions to that space, which he and other objects move around in. But something like TITAN will perceive the universe through billions of sensory channels distributed all over the surface of the Earth and beyond. On top of that, its ‘senses’ cover the whole spectrum from high-power proton microscopes in research labs to the big orbiting astronomic telescopes . . . from galactic gravity-wave detectors to the infrared sensors lowered into the ocean trenches.” Dyer swept his eyes across the three faces in front of him and spread his hands expressively. “TITAN moves pieces of itself around in millions of places at once—the ISA probes that are nosing around Jupiter . . . robot freighters under the Arctic ice caps . . . all kinds of things. How can we even begin to imagine how an awareness as totally alien as that would perceive itself and the universe around it?”