Final Jeopardy (28 page)

Tesauro's first goal was to run millions of simulated
Jeopardy
games, just as he had with backgammon. For this he needed mathematical models of three players, Watson and two humans. Modeling Watson wasn't so hard. “We knew all of its algorithms,” he said, and the team had precise statistics on every aspect of its behavior. The human players were more complicated. Tesauro had to pull together statistics on the thousands of humans who had played
Jeopardy
: how often they buzzed in, their precision in different levels of clues, their betting patterns for Daily Doubles and Final Jeopardy. From these, the IBM team pieced together statistical models of two humans.

Then they put them into action against the model of Watson. The games had none of the life or drama of
Jeopardy
—no suspense, no jokes, no jingle while the digital players came up with their Final Jeopardy responses. They were only simulations of the scoring dynamics of
Jeopardy
. Yet they were valuable. After millions of games, Tesauro was able to calculate the value of each clue at each state of the game. If Watson was in second place, trailing by $1,500 with $14,400 left on the board, what size bet on a Daily Double maximized its chance of winning? The answer changed with every move, and Tesauro was mapping it all out. Humans, when it came to betting, only had about five seconds to make a decision. They went with their gut. Watson, like its number-crunching brethren in advertising and medicine, was turning its pile of data into science.

The science, it turned out, was a bit scary. Watson's model was based on the record it had established following the simple heuristics. And studies showed that the machine, much like risk-averse humans, had been dramatically underbetting. In many stages of the game, according to Tesauro's results, the computer could maximize its chances by wagering nearly everything it had. (This wasn't always the case. If Watson enjoyed a big lead late in the game, it made sense to minimize a bet.) When Tesauro adjusted Watson's strategy toward a riskier blend of bets, it started winning more of the simulated games. He and Gondek concluded that in many scenarios, Watson should bet the farm. “When we first went to Dave Ferrucci about this,” Tesauro recalled, “he turned pale as a sheet and said, ‘You want to do what?'”

“We showed him all the extra wins we were getting with this,” Gondek said. “But he looked at the colossal bets we were making and said, ‘What if you get them wrong?'”

The conflict between rational analysis and intuition were playing out right in the IBM War Room. And Ferrucci, much like the humans who placed small, safe bets every evening on
Jeopardy
, was turning away from statistics and focusing on a deeper and more primitive concern: survival. Watson was going to be playing only one game on national television. What if it bet big on that day and lost?

“That would really look bad for us,” Tesauro said. Perhaps it would be better to sacrifice a win or two out of a hundred and protect Watson a bit more from prime-time catastrophe. It wasn't clear. The strategy team continued to crunch numbers.

The numbers flowing in from the real matches, where Watson was playing flesh-and-blood humans, were improving. Through the autumn season, the newer, smarter Watson powered its way past scores of
Jeopardy
champions. It won nearly 70 percent of its matches; its betting was bolder, its responses more assured. It still crashed from time to time, of course, and routinely made crazy mistakes. On one Daily Double, it was asked to name the company that in 2002 “came out with a product line featuring 2-line Maya Angelou poems.” Watson missed the answer (“What is Hallmark?”) and appeared to pay tribute to its creators, responding: “What is IBM?”

Watson's greatest weakness was in Final Jeopardy. According to the statistics, after the first sixty clues, Watson was leading an astounding 91 percent on the games. Yet that final clue, with its more difficult wording and complex wagering dynamics, lowered its winning percentage to 67 percent. Final Jeopardy turned Watson from a winner to a loser in one-quarter of the games. This was its vulnerability going into the match, and it would no doubt rise against the likes of Ken Jennings and Brad Rutter. The average human got Final Jeopardy right about half the time, according to Gondek. Watson hovered just below 50 percent. Ken Jennings, by contrast, aced Final Jeopardy clues at a
68 percent
rate
.
That didn't bode well for the machine.

Brad Rutter, undefeated in his
Jeopardy
career, walked into the cavernous
Wheel of Fortune
studio. It was mid-November, just two months before he and Ken Jennings would take on Watson. Rutter, thirty-two, is thin and energetic, with sharply chiseled features. His close-cut black beard gives him the look of a vacationing television star. This is appropriate, since he recently moved from his native Lancaster, Pennsylvania, to L.A.'s Beechwood Canyon, right under the famous Hollywood sign. He's trying to make it as an actor.

On this autumn day, Rutter and Jennings were having their orientation for the upcoming match. They were shuttling back and forth between meetings in the Robert Young Building and interviews in the empty
Wheel of Fortune
studio. Rutter, clearly fascinated by television, spotted a rack of men's suits by the stage. “Are those Pat Sajak's?” he asked, referring to the longtime
Wheel of Fortune
host. Told that they were, he went over to check the labels. For years, the show announced every evening that Sajak's wardrobe was provided by Perry Ellis. Rutter, a stickler for facts, wanted to make sure it was true. It was.

The previous evening, Rutter had been given a Blu-ray Disc featuring five of Watson's sparring rounds. He studied them closely. He noticed right away that Watson hopped around the board, apparently hunting for Daily Doubles. He also focused on Watson's buzzer speed and was relieved to see that humans often managed to beat the machine. This was crucial for Rutter, who viewed speed as his greatest advantage. He said he was no expert on computers and had only a vague idea of how Watson worked. But he had expected the IBM team to give Watson an intricate timing program to anticipate the buzz. (This was a frightfully complex option that Ferrucci had decided not to pursue.) “That scared me,” Rutter said.

Rutter's speed is legendary. It fueled his 16-0 record on
Jeopardy
, including his decisive victories over Jennings. It was such an advantage that IBM's Gondek referred to Rutter as “Jennings Kryptonite.” Rutter said he wasn't sure what made his thumb so fast, but he had a theory. “I used to play a lot on the Nintendo entertainment system when I was a kid,” he said. “And if you played Super Mario Brothers or Metroid, you had to hit the button to jump at exactly the right time. It was not about speed but timing. And that's what the
Jeopardy
buzzer is about.” This meant that a computer game trained the human, who would later use those same skills to take on another computer.

But Rutter boasted strengths beyond mere speed. In a Tournament of Champions match that aired in May 2005, he found himself in a most unusual position—third place—heading into Final Jeopardy. The category was People and Places, and the clue: “This Mediterranean island shares a name with President Garfield's nickname for his wife.”

“I started scanning Mediterranean islands,” Rutter said. “OK. Sardinia? No. Corsica? No. Sicily? No. Menorca? Mallorca? Malta?” He figured, “Malta could be a girl's name,” and wrote it down. But he knew it was wrong. As the theme music played, he continued to think of islands. Lesbos, Rhodes, Ibiza . . . “With about five seconds left,” he said, “I got to Crete.” All at once the pieces came together. “Crete could be short for Lucretia. That's a very nineteenth-century name. And then it was an apparition in my head. I'd looked at a list of First Ladies, and somehow Lucretia Garfield popped out at me. I can't explain it. So I scribbled down Crete. It was barely legible. I was the only one to get it right, and I ended up winning by a dollar.”

A timely spark of human brilliance had saved him. It featured a series of insights and connections Watson would be hard-pressed to match. Indeed, in the coming showdown, Rutter's competition on that type of clue was more likely to come from the other human on the stage. This led to a question: Was it fair that the two humans had to battle each other in addition to the machine? Mightn't it be easier for one player to face two machines?

Rutter thought so. “I've seen Ken play seventy-four matches,” he said. “I know his strengths and weaknesses pretty well. They're different than Watson's. So when I'm picking different categories or clues off the board, who do I attack? Whose weaknesses do I try to get to? That's a tough question. I haven't really figured it out yet. I'm going to be thinking about it a lot.”

Jennings, sitting in the same studio, elaborated on the point. “I don't mean it to sound like I'm making excuses already,” he said, “but there is some inherent disadvantage that there are two humans and one Watson.” The way he saw it, Watson's algorithms would master “a certain percentage of the
Jeopardy
canon.” And if the computer was fast on the buzzer, it would dominate in those areas. That left two players to battle over the clues “that only humans can do.”

The thirty-six-year-old Jennings, with his featherweight build, is far smaller than Rutter. He has an easy laugh and a self-effacing style. He hadn't yet found a Blu-ray player, he said, to watch the video of Watson in action. But he had clearly been reading everything he could find about the computer, including technical articles. Jennings double-majored in computer science and English at Brigham Young University and later worked as a computer programmer in Salt Lake City. When he heard about the match against Watson, he said, it excited him. “I said, ‘Wow, we get to see if a computer can play
Jeopardy,
'” he said. “I was more interested in the geeky part.”

As Jennings studied up on Watson's algorithms and its massive parallel processing, he couldn't help comparing the computer to his own mind. “Many of the tricks that I used in
Jeopardy
are things that I read Watson does,” he said.

He gave an example. One
Jeopardy
clue asked for the name of two of Jesus' disciples whose names are both top-ten baby male names and end in the same letter. “I remember thinking,” he said, “that that's not the kind of thing you can know. The only way to do it is to break it down, make a list, do the Venn diagram of it. And I come to find out that Watson does exactly that. It's very good at decomposing questions, so it does the two fact sets in parallel. Then it does the Venn diagram to see if there's anything on both lists.” Jennings paused for a moment, then said, “Matthew and Andrew, by the way. I got that one right at the last minute. I was about to put James and Judas, but I don't think Judas is a popular baby name, for some reason . . .”

Jennings reflected on traveling across the country, to IBM's lab in Yorktown, to take on Rutter and Watson. “It's a little different to be the road team,” he said. “I'm not playing in the familiar studio where I have the muscle memory and the good times and the million-dollar check. I'm picturing a very sterile lab from the fifties, people running around in white coats. . . .”

Jennings had been following Watson's record against its sparring partners, and the trend looked worrisome. In the beginning of the matches, he said, Watson was winning 64 percent of the time against standard
Jeopardy
players. “Now they've fine-tuned it, and it's 67 percent against Tournament of Champions players. I know it can still be beat,” he said. “But I think to myself: Could I win 67 percent of my games against Tournament of Champions players? That's not something I've ever done. I rattled off a very long streak, but it was against rookie players.” He said that he was going into the match feeling, for the first time, like an underdog.

Jennings is well known for his disarming modesty. In previous games, it could be argued, it may have benefited him as a psychological tactic. Rivals encountered a likable and unassuming young man who seemed almost surprised at his own success. By the time they looked at the scoreboard, he was annihilating them.

Such tactics wouldn't mean much in the coming showdown. Jennings and Rutter would be facing a foe impervious to nerves and psychological maneuvering. And while millions tuned in to what promised to be an epic knowledge battle between two men and a thinking machine, the drama would leave Watson unmoved. The machine, unlike everyone else, had no stake in the outcome.

DAVID FERRUCCI HAD
driven the same stretch hundreds of times from his suburban home to IBM's Yorktown labs, or a bit farther to Hawthorne. For fifteen or twenty minutes along the Taconic Parkway he went over his endless to-do list. How could his team boost Watson's fact-checking in Final Jeopardy? Could any fix ensure that the machine's bizarre speech defect would never return? Was the pun-detection algorithm performing up to par? There were always more details, plenty to fuel both perfectionism and paranoia—and Ferrucci had a healthy measure of both.

But this January morning was different. As he drove past frozen fields and forests, the pine trees heavy with fresh snow, all of his lists were history. After four years, his team's work was over. Within hours, Watson alone would be facing Ken Jennings and Brad Rutter, with Ferrucci and the machine's other human trainers reduced to spectators. Ferrucci felt his eyes well up. “My whole team would be judged by this one game,” he said later. “That's what killed me.”

The day before, at a jam-packed press conference, IBM had unveiled Watson to the world. The event took place on a glittering new
Jeopardy
set mounted over the previous two weeks by an army of nearly a hundred workers. It resembled the set in Culver City: the same jumbo game board to the left, the contestants' lecterns to the right, with Alex Trebek's podium in the middle. In front was a long table for the
Jeopardy
officials, where Harry Friedman would sit, Rocky Schmidt at his side, followed by a line of writers and judges, all of them with monitors, phones, and a pile of old-fashioned reference books. Every piece was in place. But this East Coast version was plastered with IBM branding. The shimmering blue wall bore the company's historic slogan, Think, in a number of languages. Stretched across the shiny black floor was a logo that looked at first like Batman's emblem. But closer study revealed the planet Earth, with each of the continents bulging, as if painted by Fernando Botero. This was Chubby Planet, the symbol of IBM's Smarter Planet campaign and the model for Watson's avatar. In the negotiations with
Jeopardy
over the past two years, IBM had lost out time and again on promotional guarantees. It had seemed that Harry Friedman and his team held all the cards. But now that the match was assured and on Big Blue's home turf, not a single branding opportunity would be squandered.