Read My Life as a Quant Online
Authors: Emanuel Derman
A little like Feynman diagrams, which allowed workaday physicists to compute unthinkingly the detailed quantum mechanical probabilities that had formerly demanded the genius of Schwinger or Feynman, these parsing tools allowed regular programmers to prosaically create languages that would previously have required magnificent exertions.
I used to always equate computation with numerical evaluation. Now, having been exposed to the linguistic aspects of computing, I regretted I had not recognized them before. I fantasized about escaping the Business Analysis Systems Center to become a true research computer scientist in Area 10, and made an attempt to transfer. But I had neither the credentials nor the background, so it was quite impossible.
Nevertheless, for most of my five years at the Labs, I worked on building compilers. I spent several years designing and implementing a language I named
HEQS
, a “Hierarchical EQuation Solver,” an equation-solving language for businesspeople who knew enough to specify the equations they wanted to solve but lacked either the mathematical ability or the time to solve them. The name had the obligatory UNIX cuteness, but it was also a homonym of “hex,” which succinctly reflected my secretly morbid view of life in Building 5.
Like
lex
or
yacc
, HEQS was a nonprocedural languageâusers could state what they wanted done (“Solve these equations!”) without having to specify a procedure for accomplishing it. (FORTRAN and C, in contrast, are
procedural
languages that demand that the programmer specify, in excruciating detail, how to execute a task.) In its final incarnation, you could give HEQS a collection of thousands of algebraic equations (linear, nonlinear or simultaneous) and have it solve them or tell you why some error you had made in specifying the equations made a solution impossible. HEQS also provided tools for analyzing the set of equations; users could examine the relation between input and output, and understand the chain by which altering one variable's value affected other variables. In essence, it allowed business users at AT&T to spend their time specifying the relationships in their business or accounting model without worrying about how to solve it.
Six or seven years later, after the widespread advent of PCs, spreadsheets like Visicalc and Lotus provided tools for doing the same thing. Until then, various businesspeople at AT&T headquarters used HEQS for model solving, and programmers in our center employed it as an equation-solving engine in many of the applications we built. I was pleased to keep one foot in the research world by describing it in an issue of the
AT&T Technical Journal
.
4
I developed HEQS on my own, delving into
lex
and
yacc
to design the language and build its compiler. Once users had entered the equations they wanted to solve, my program regrouped them into sequences of smaller sets of simultaneous equations, ordered so that the output from the solution of one set provided the input you needed to solve the next. Stimulated by the courses on computer science I had taken at the Labs, I realized that I could represent each variable in the set of equations by a node in a directed graph, and that reordering the equations was equivalent to decomposing the graph into its strongly connected components. I was naively proud to be doing real math.
There were helpful resources wherever I turned. I found that Chris Van Wyk, an MTS in Computer Science in Area 10, had written a set of UNIX tools for solving simultaneous equations. He had started this project while working on his PhD thesis at Stanford under Donald Knuth, the famous author of the four-volume
The Art of Computer Programming
and the inventor of
TeX
, a widely used language for mathematical typesetting and word processing that has become standard among scientists. Steve Blaha, my particle-physicist friend at the Labs, told me that Knuth had been his college roommate. As we worked together I was impressed by Chris's professional programming skills; I was an amateur, living by my wits, while Chris was the real thing, a researcher working in his area of expertise. I felt doomed to remain a mere dilettante.
HEQS was a good idea, and soon, as people in the Center started using it for solving larger sets of equations, it needed a more skillful and efficient implementation than I could provide. My version of HEQS allowed users only single numbers (scalars) and one-dimensional vectors to represent financial time series. A colleague, Ed Sheppard, was assigned to work with me, and we planned to rewrite the system to incorporate multidimensional array variables in order to represent more general financial time series. While I was away on a two-week beach vacation at Fire Island with my family, Ed suddenly threw himself into redesigning and then rewriting the entire systemâwithout giving me advance notice. I returned to a
fait accompli
, a completely new, enhanced, and almost unrecognizable APL-flavored version of the language. Ed's version now incorporated vastly complex dynamically linked data structures, whose details I knew I would not live long enough to master. Ed had also cleverly modified HEQS so that, once you had used it interactively to develop and solve a financial model, you could then use it generate a C program that would solve your equations many times faster.
Programming came naturally to Ed in a way it never would to me, and his proficiency daunted me. Sometime in late 1984 he left to join Asymetrix, a Seattle-based company founded by Paul Allen. For the remainder of my stay at the Labs, I was a victim of our success: I often had to spend days at a time diving into his code to fix the residual bugs that showed up in algorithms I had never written and never fully understood.
It was around this time, seeing Ed write and design code, that I realized how many physicists misperceived the nature of jobs and careers in the nonacademic world. Physicists tended to think they were so smart that, once they descended to a job in the “outside world,” their talent would allow them to work in a 9-to-5 mode and still outperform their colleagues. But, in any nonacademic job, there are people for whom that particular work is not a compromise but a passion and dedication, taken seriously. They, rather than the smart but coasting physicists, set the standards of excellence.
HEQS ended happily for me. Chris Van Wyk and I eventually wrote an article on HEQS that appeared in the software journal
Programming: Practice and Experience
in 1984. I was overjoyed to publish again rather than perish, to be “doing science.” Even now, I get a small kick out of seeing occasional references to that paper on the Web, though most of them undoubtedly occur because Chris continues to work in the field of nonprocedural languages. I was pleased to discover recently that Lucent, the descendant of Bell Labs, still offers HEQS for sale at $89.00 on its website at
https://www.lucentssg.com/heqs.html
. In this era of Mathematica and Excel, however, I cannot imagine who buys it.
I learned almost nothing about business or finance in the Business Analysis Systems Center between 1980 and 1985. In contrast, the software engineering skills I learned there stood me in good stead, and formed the basis for much of the fixed-income financial modeling infrastructure I later built at Goldman Sachs in 1987.
Despite the excellent education, I always felt subservient and demeaned at the Labs. Only a month after starting work there I took Joshua, barely three years old, to play on the lawns at Rockefeller University where we still lived. He loved to take off his shoes and run barefoot through the grass. While he did so, I sat there ruminating about what I was doing. Suddenly, he came over, looked at me, and asked: “Daddy, why you sad?” I knew even then that my daily reverse commute to the Labs would be merely temporary. I just didn't know how I would find a way to end it.
1
On Wall Street until the late 1990s, PhDs s rarely listed their degrees on their business cards, because to do so would have outed them immediately as nonbusiness people. For the same reason, it took a long time before quants and programmers routinely put their email addresses on their business cards. A rise in the prestige of PhD degrees occurred slowly but steadily between 1996 and 1999 as dot-coms prospered.
2
Writing operating systems in a high-level language like C was a new idea. Operating systems used to be written from scratch, arduously, in the idiosyncratic, low-level, hard-to-read, and primitive
assembler
or
machine code
that came native on each new machine. By using standard C rather than each machine's particular machine code to write the operating system, you could quickly create a version of UNIX and all its tools to run any newly manufactured computer, simply, by first getting C itself to run on that machine.
3
Pairs trading is the search for statistically significant oscillatory patterns in the spread between pairs of similar stocks. If you believe you have detected such a phenomenon, you short the expensive stock and buy the cheap one when the spread is large, and then reverse the trade when/if the spread narrows. Since Tartaglia's renowned but temporary successes at Morgan Stanley, trading houses, hedge funds, and the scientists they employ have regularly and hopefully attempted to build model-driven, so-called “statistical arbitrage” money machines of this type.
4
At this time I also began attending various computer science research seminars and conferences, where I was always struck by the difference in quality between computer science research and physics research. In physics, seminar speakers described completed achievements. In computer science, however, the majority of the talks were about plans for systems, sketches of new languages, and unimplemented ideas. The hurdle for declaring accomplishment seemed much lower.
Wall Street beckons
Interviewing at investment banks
Leaving the Labs
How to get out of the Labs? That was my daily obsession for five years. Meanwhile, always the complainer, I came home every day and droned on indignantly about my plight, burdening anyone willing to listen to my own Passover Seder, the story of how I came to leave the land of academic milk and honey and live under Pharoah in the land of the Business Analysis Systems Center. I recounted the heavy labors that were inflicted upon me and strategized about how to get free. Every day was
Paradise Lost
and
In Search of a Lost Time
. I know I drove my wife half crazy.
When I took three-year-old Josh to the Center's Christmas party, I worried that he would give me awayâI had spent so many evenings at home disparagingly describing my frustration with bosses and coworkers within his earshot that I feared he would recognize someone's name when I introduced them and repeat one of my remarks. Before driving out to Murray Hill that morning, I cautioned him not to repeat anything I had said about anyone. Of course, he didn't.
At that party, Mark Koenigsberg told me about a date who had taken him to see a movie he hated intensely. We knew that our tastes in movies were diametric, and the more he described the plot, the more I was certain I would like it. As soon as the party was over, I drove back to New York and went to see the filmâLouis Malle's
My Dinner With André
, with André Gregory and Wallace Shawn. The self-indulgent dinner conversation about the contrast between a mystical search for fulfillment and the mundane pleasures and disappointments of everyday life resonated deeply in me. It reminded me of the sense of possibility about the future I had once experienced when I had first wanted to be a physicist; it also reminded me of the potentialities for happiness that I had sensed while meditating at the Dharmadhattu in Boulder. On that December afternoon,
My Dinner With André
imbued me with a such a vast sense of hope and salvation that I felt uplifted. Its aftereffects dissipated astonishingly slowly, trickling out of me for about a week. Two months later, wondering if its salutary effects had been merely a fluke, I went to see the movie once again, and once again felt better, though this time only for several days. Malle, Gregory, and Shawn depicted the middle ground between hope and despair so well; a few years ago I saw their
Vanya on Forty-Second Street
and felt similarly moved, at the first viewing and the second.
I was naive and felt demeaned by the “moneyness” that underpinned everything we were supposed to work on at the Labs. Despite the vicissitudes of life in physics, I had worked out of love, and now I was working for money. Though Bell Labs was ranked as one of the top 100 places to work in America in 1985, it was at the top of my own selfish list of the worst places I'd worked at. My karmic destiny, I suppose, was to have my vain illusions about life and work painfully sandpapered off me as I brushed against the rough surface of the world. Later, when that process was complete, I would be able to enjoy life at Goldman, Sachs and Co. Had I come to work at Goldman straight out of graduate school, I might well have hated it for its intensely pecuniary focus just as much as I hated the Labs.
Meanwhile, I obsessed about changing my life. There were a variety of excellent programmers who worked in a software research group within the Business Analysis Systems Center. Dave Korn worked a few offices down from me, busy creating the now ubiquitous UNIX Korn shell; Emden Gansner and Jonathan Shopiro were both enthusiastic contributors to object-oriented programming in general and to Stroustrup's C++ environment in particular. I tried to transfer into their group, but Downs insisted that it was narrow-minded of me to think that I had to be located with them to do interesting work. Perhaps it was just as well. Though I was envious of Emden's calm, undistracted ways, I was quite incapable of emulating his temperament. He never asked for help with debugging, but read whatever manuals were necessary and then patiently dug deeper, struggling for as long as it took to solve the problem. I simply ran for help at the first panicky opportunity.