Rise of the Robots: Technology and the Threat of a Jobless Future (21 page)

Algorithmic grading, despite the controversy that attaches to it, is virtually certain to become more prevalent as schools continue to seek ways to cut costs. In situations where a large number of essays need to be graded, the approach has obvious advantages. Aside from speed and lower cost, an algorithmic approach offers objectivity and consistency in cases where multiple human graders would otherwise be required. The technology also gives students instant feedback and is well suited to assignments that might not otherwise receive detailed scrutiny from an instructor. For example, many communications courses require or encourage students to maintain daily journals; an algorithm can evaluate each entry, and perhaps even suggest improvements, at the click of a button. It seems reasonable to assume that automated grading will, at least for the foreseeable future, be relegated to introductory courses teaching basic communication skills. English professors have little reason to fear that the algorithms are poised to invade upper-level creative writing seminars. However, their deployment in introductory courses might eventually displace the graduate teaching assistants who now perform these routine grading tasks.

The uproar over robotic essay grading represents only a small example of the backlash that is certain to arise as the full force of accelerating information technology finally falls upon the education sector. Thus far, colleges and universities have largely been immune to the substantial increases in productivity that have transformed other industries. The benefits of information technology have not yet scaled across the higher-education sector. This, at least in part, explains the extraordinary increase in the cost of college in recent decades.

There are strong indications that things are about to change. One of the most disruptive impacts is sure to come from online courses offered by elite institutions. In many cases, these courses attract huge enrollments, and they will, therefore, be an important
driver of automated approaches to both teaching and grading. EdX, a consortium of elite universities founded to offer free online courses, announced in early 2013 that it will make its essay-grading software freely available to any educational institutions that want to use it.
⁴
In other words, algorithmic grading systems have become yet another example of an Internet-based software building block that will help accelerate the inevitable drive toward the increased automation of skilled human labor.

The Rise—and Stumble—of the MOOC

Free Internet-based courses like those offered by edX are part of the trend toward massive open online courses—or MOOCs—that exploded into the public consciousness in the late summer of 2011, when two computer scientists at Stanford University, Sebastian Thrun and Peter Norvig, announced that their introductory artificial intelligence class would be available to anyone at no cost over the Internet. Both of the course’s instructors were celebrities in their field with strong ties to Google; Thrun had led the effort to develop the company’s self-driving cars, while Norvig was the director of research and co-author of the leading AI textbook. Within days of the announcement, more than 10,000 people had signed up. When John Markoff of the
New York Times
wrote a front-page article
⁵
about the course that August, enrollment rocketed to more than 160,000 people from over 190 countries. The number of online students from Lithuania alone exceeded the entire undergraduate and graduate student enrollment at Stanford. Students as young as ten and as old as seventy signed up to learn the basics of AI directly from two of the field’s preeminent researchers—an extraordinary opportunity previously available only to about 200 Stanford students.
⁶

The ten-week course was divided into short segments lasting just a few minutes and modeled roughly on the enormously successful videos for middle and high school students created by the Khan
Academy. I completed several units of the class myself and found the format to be a powerful and engaging learning vehicle. The production employed no visual wizardry; instead, it consisted primarily of either Thrun or Norvig presenting topics while writing on a notepad. Each brief segment was followed by an interactive quiz—a technique that virtually guarantees that key concepts are assimilated as you proceed through the course. About 23,000 people completed the class, took the final exam, and received a statement of accomplishment from Stanford.

Within months, an entirely new industry materialized around the MOOC phenomenon. Sebastian Thrun rounded up venture capital and formed a new company named Udacity to offer free or low-cost online classes. Across the country and the globe, elite universities rushed to get in on the game. Two other Stanford professors, Andrew Ng and Daphne Koller, founded Coursera with a $22 million initial investment and built a partnership with Stanford, the University of Michigan, the University of Pennsylvania, and Princeton. Harvard and MIT quickly invested $60 million to form edX. Coursera responded by adding another dozen universities, including Johns Hopkins and the California Institute of Technology, and within eighteen months it was working with over a hundred institutions throughout the world.

By early 2013, the hype surrounding MOOCs was exploding as rapidly as course enrollments. The online classes were widely believed to be poised to usher in a new age in which elite education would be accessible to all at little or no cost. The poor throughout Africa and Asia would soon be attending Ivy League colleges via cheap tablets and smart phones. Columnist Thomas Friedman of the
New York Times
called MOOCs a “budding revolution in global online higher education” and suggested that the online courses had the potential to “unlock a billion more brains to solve the world’s biggest problems.”
⁷

Reality struck in the form of two studies released by the University of Pennsylvania in the final months of 2013. One of the studies
looked at a million people who had enrolled in classes offered by Coursera and found that MOOCs “have relatively few active users, that user ‘engagement’ falls off dramatically—especially after the first 1–2 weeks of a course—and that few users persist to the course end.”
⁸
Only about half of the people who signed up for classes viewed even a single lecture. Course completion rates ranged from 2 to 14 percent, and averaged about 4 percent. MOOCs were also largely failing to attract the poor and undereducated students whom everyone thought stood to benefit the most; about 80 percent of the people who signed up for the classes already had a college degree.

Several months earlier, a high-profile partnership between Udacity and San Jose State University had likewise failed to measure up to expectations. The program, intended to offer disadvantaged students inexpensive online classes in remedial math, college algebra, and introductory statistics, was announced at a press conference by Sebastian Thrun and California governor Jerry Brown in January 2013 and touted as a possible solution to soaring tuition costs and overcrowding at state colleges. When the first groups of students completed the courses, which cost just $150 and offered online mentors to provide individual assistance, the results were dismal. Three-quarters of the students taking the algebra class—and nearly 90 percent of those coming directly from high school—failed the course. In general, the MOOC students did significantly worse than students enrolled in traditional classes at San Jose State. The university has since suspended the program at least temporarily.
⁹

Udacity is now deemphasizing broad-based education and is instead focusing on more vocational classes designed to give workers specific technical skills. Companies like Google and
Salesforce.com
, for example, are underwriting courses that teach software developers how to work with their products. Udacity has also partnered with the Georgia Institute of Technology to offer the first MOOC-based master’s degree in computer science. Tuition for the three-semester program will cost just $6,600—about 80 percent less than a traditional
on-campus degree. The program’s setup costs are being funded by AT&T, which plans to send many of its employees through the program. Initially, Georgia Tech will enroll about 375 students, but the goal is to expand the program so it can serve thousands.

As MOOCs continue to evolve and improve, the hope that they will drive a global revolution that will bring high-quality education to hundreds of millions of the world’s poor may ultimately be realized. In the near term, however, it seems evident that these online courses are most likely to attract students who are already highly motivated to seek further education. In other words, MOOCs are poised to compete for the same people who might otherwise enroll in more traditional classes. Assuming that potential employers see MOOCs as offering a valuable credential, this could eventually unleash a dramatic disruption of the entire higher-education sector.

College Credit and Competency-Based Credentials

When Thrun and Norvig tallied up the results from their 2011 artificial intelligence class, they found that 248 participants had achieved perfect scores in the course; these students had never answered an examination question incorrectly. They also discovered that not a single Stanford student was among that elite group. In fact, the highest-scoring on-campus student was outperformed by at least 400 online participants. None of those stellar performers, however, received formal Stanford credit or even a traditional certificate of completion for their work.

Months earlier, when Stanford administrators had first learned of the course’s soaring enrollment, they had repeatedly called the professors into meetings to negotiate the nature of any credential that might be awarded to the online participants. The concern was not just that the Stanford cachet would potentially be diluted across tens of thousands of people—none of whom was being charged the roughly $40,000 in annual tuition paid by regular on-campus
students—but also that the identity of students in remote locations could not be verified. The administrators eventually agreed that a simple “statement of accomplishment” could be offered to students who completed the course over the Internet. Stanford officials were so concerned with this precise terminology that when a journalist used the word “certificate” in a column about the course, they immediately called to request a correction.

The Stanford officials’ worries about verifying the identity of online students were not unfounded. Indeed, ensuring that credit is awarded to the same person who actually completes the course and takes the exams is one of the most significant challenges associated with offering college credit or official credentials for MOOCs. Without a robust identification process, a vibrant industry would soon spring up around the fraudulent completion of courses and exams. In fact, a number of websites have already appeared offering to take online courses for other people in return for a fee. In late 2012, journalists from the website
Inside Higher Ed
posed as students and requested information from some of these sites about completing an introductory online economics course offered by Penn State. They were quoted fees ranging from $775 to $900 and guaranteed at least a “B” in the course. And this was for a class at Penn State’s traditional, degree-granting online branch, where verifying student identities ought to be far less of a challenge than it would be for an open class with enormous numbers of participants.
¹⁰
Total enrollment for the entire Penn State program is only about 6,000 graduate and undergraduate students—a tiny fraction of the number of people likely to sign up for a single popular MOOC.

Cheating has also been a significant problem with massive online classes. In 2012, dozens of complaints were filed about plagiarism in humanities courses offered through Coursera. These courses relied on peer grading, rather than algorithms, to assess student performance, so course administrators responding to complaints had to deal with both the possibility of rampant plagiarism and the likelihood that at
least some of the accusations were erroneous. In one class on science fiction and fantasy writing, claims that student essays were being copied from Wikipedia or other previously published sources prompted Eric Rabkin, the University of Michigan English professor teaching the course, to send a letter to all 39,000 students warning them against appropriating others’ work, but also pointing out that “an accusation of plagiarism is a deeply serious act and should be made only with concrete evidence behind it.”
¹¹
The remarkable thing about such incidents is that no academic credit is being offered for any of these classes. Apparently, some people will cheat “just because they can” or perhaps because they don’t understand the rules. In any case, there can be little doubt that associating formal academic credit with such courses would dramatically increase the incentive to misbehave.

There are a number of possible technical solutions to the identification and cheating problems. A simple method is to pose challenge questions requesting personal data at the beginning of each session. If you’re planning to cheat by hiring someone to take a class in your name, you might think twice before giving them your social security number. That type of strategy would be difficult to implement globally, however. A remote proctoring solution requires that a camera be active on the computer so administrators can monitor the student. In 2013, edX—the MOOC consortium founded by Harvard and MIT—began offering ID-verified certificates to students who pay an additional fee and take the class under the watchful eye of a webcam. Such certificates can be presented to potential employers but generally cannot be used for academic credit. Monitoring by proctors is expensive and obviously not extensible to tens of thousands of people taking a free course, but it seems likely that facial recognition algorithms of the type currently used to label photos on Facebook may eventually step into that role. Other algorithms may soon be able to identify students by analyzing the cadence of their keystrokes, or root out plagiarism by automatically comparing written assignments to vast datasets of existing works.
¹²