Moon Lander: How We Developed the Apollo Lunar Module (20 page)

I spent more time on GSE, often attending at least part of their weekly meeting with Lanzkron as well as continuing to hold my own biweekly technical staff meeting with GSE Engineering. As I became familiar with the GSE program, I soon found myself doing what the GSE “natives” did: thinking of and referring to the GSE end items by their five- or four-digit numbers. In some cases the dialogue at the GSE meetings sounded like the legendary comedians’ convention where someone calls out the number of a joke and everyone laughs.

In late 1966 LM GSE Manufacturing was consolidated in a refurbished building in Syosset, Long Island, known as the Davega building for its former occupant, a sporting-goods chain. With the GSE Engineering and Manufacturing staffs greatly expanded and intense concentration and effective leadership
by Coursen and his team, plus increased support from the corporate ILS Department, the LM GSE program gradually turned the corner. The end-item list stabilized, and our ability to make schedules and hold to them gradually improved. At its peak in early 1967, LM GSE Engineering and Manufacturing employed more than fifteen hundred people. The total cost of LM GSE was greater than our 1962 negotiated estimate for the entire LM program, an indication of how GSE was underestimated by all at the start of the program.

Lanzkron was reassigned by Shea to key trouble spots elsewhere on the program, but he returned to Grumman for a time in 1967 to help us resolve pervasive manufacturing assembly and quality-control problems. Although his brand of assistance was painful to endure, it was effective and contributed substantially to both Grumman’s success and that of the Apollo program.

Configuration Management and Scheduling

The management of the Apollo program was a staggering task, more complex in its scope, number of participants, and interrelations between program elements than anything yet attempted by the aerospace industry. The activities and outputs of more than 175,000 people in thousands of organizations across the United States had to be coordinated, scheduled, and provided with technical interface data wherever their products interacted with someone else’s. Interlocking schedules for each program participant had to be prepared and scheduled against actual progress. The whole array of schedules had to be capable of rapid revision whenever new developments, such as unforeseen test failures or delivery slippages, caused a change in technical approach or plans.

As a model of how to manage such complexity, NASA used the ballistic missile programs then under development by the air force and the navy. They adopted major techniques from each of them: from the air force, the configuration management system, and from the navy, the program evaluation and review technique (PERT).

Configuration management, developed and used on the Atlas, Titan, and Minuteman programs, was a formal, structured method for defining and controlling the detailed configuration of aerospace flight vehicles and GSE. Rigorous control over the millions of parts in a missile or spacecraft was essential to prevent unexpected failures due to the actual vehicle differing from what the engineering drawings showed it to be. The leaders of NASA’s manned spaceflight program had learned this lesson early, through a terrifying incident that occurred as they all watched the first launch of an unmanned Mercury capsule on a Redstone booster, the last flight prerequisite to a manned suborbital mission. MR-1, as it was called, failed ignominiously and dangerously on the launch pad on 20 November 1960. The Redstone rocket fired briefly and lifted a few inches off the pad, then abruptly shut down. The Redstone settled back on the launch pad, but the Mercury spacecraft continued to
go through its preprogrammed sequence of events for liftoff and landing; firing the escape tower rocket, then the radar chaff dispenser, the drogue parachute, and the main parachute. The Redstone was left standing on the pad, unsecured, with its propellant tanks pressurized and no connection to ground control from the blockhouse. Fortunately there was no fire or explosion, and NASA technicians eventually went out to the pad and successfully disarmed the live Redstone rocket.

The cause of this mishap was determined to be a failure of configuration management: the “as built” system was not “as designed” in the engineering drawings. Subsequent to the prior launch from the pad, a technician filed about one-quarter inch from one of the two prongs in the tail plug, an electrical connector at the base of the launch umbilical tower that plugged into the base of the Redstone, and did not tell anyone what he did. When the Redstone lifted off about an inch, the tail plug was supposed to pull out of the base of the rocket, switching the Redstone over to internal power. However, if one of the two prongs was to disconnect before the other, the circuitry was designed to shut the Redstone’s engine down immediately. This is what happened when one connector prong had been made shorter than the other. NASA was extremely lucky that MR-1 did not explode on the pad in a spectacular failure that might have caused cancellation of the whole manned spaceflight program. This narrow escape made all those who witnessed it, including Gilruth and von Braun, determined to have a disciplined configuration management program throughout NASA.
¹

The air force’s configuration management system was thorough and rigorous and was documented in five manuals known as the “375 series.” With minor revisions NASA adopted it completely and educated its contractors on how to apply it. I listened to an air force lecture on configuration management, after which Bill Rector and I discussed how we would apply it to LM. We agreed that the M-5 mockup review would also be the preliminary design review (PDR) as defined by the configuration management manuals, marking the completion of the definition phase of the program. After PDR no major configuration changes could be made without prior approval by the LM Change Control Board (CCB), staffed by both NASA and Grumman. The design and development phase for unmanned LMs would end shortly before LM-1 delivery with a critical design review (CDR), and for manned LMs before LM-3 delivery
²
After CDR no changes that affected form, fit, or function of any part or component could be made without prior CCB approval. The nomenclature implies the rigor and discipline that this system imposed. Making it happen would require continuous education and repetitive indoctrination of all those on the program, a major task of training and communications. The LM program was Grumman’s first exposure to rigorous air force 375 configuration management, as the navy had not yet imposed it on our aircraft programs.

The PERT scheduling and control system was first put into major program usage by Admiral Rayburn on the Polaris Fleet Ballistic Missile program and improved by other users, including the Minuteman program. It was adopted by the Department of Defense as a standard tool of program management, and numerous software programs were developed to provide computer-assisted generation of the PERT schedules and networks. NASA adopted a version of PERT as its standard across all programs, including Apollo. As in configuration management, the LM program was Grumman’s first use of PERT, so we had to train our people in the system and its application to LM. During 1964 we conducted many training sessions for our people in the use of CM and PERT.

Many of PERT’s features were well suited to the Apollo program. It provided flexible, computer-aided scheduling of program activities and allowed for uncertainties by inputs of “best, worst, and most likely” completion dates for each event. It showed the relationships between events through graphic network diagrams in which the activities or events were positioned and coded to show their relationship to other events, whether as prerequisite constraints or as parallel nonconstraining activities. Using computer software, PERT allowed the program to be scheduled at a low level of detail that could directly be related to measurable actions and outputs in Engineering, Manufacturing, or test. It used the computer to “roll up” the highly detailed network into summary schedules sorted by spacecraft, by test program, or whatever views were meaningful at the overall program level.

Rathke and Mullaney established the LM program Scheduling Group to learn the PERT system and help us apply it. The group was headed by Larry Moran from the Scheduling Group in corporate engineering. Larry was tall, thin, and energetic, with a tanned and freckled face, black hair brushed back in a low pompadour, and a ready laugh and smile. A heavy smoker, he looked unhealthy hunched over the networks and schedules on his drafting board, wreathed in smoke with a cigarette dangling from his lips. He was a natural leader and had a wonderful sense of humor, which helped his hardworking group endure the long hours they put in.

Larry Moran built a group of program schedulers who were far more than just PERT technicians. They understood interactions between program events better than anyone else on LM, and as scheduling problems developed they were very constructive in helping program and engineering management devise “workarounds” to recover lost time or accommodate unforeseen events and requirements. The schedulers roamed freely throughout the LM program, determining the actual status of events scheduled on the PERT networks and posting them on sliding schedule wallboards displayed in the Plant 25 main conference room. Gavin and Mullaney conducted the weekly LM program meeting using Moran and his charts to show the schedule problem areas and discuss recovery options. Summary schedules and actual status
were rolled up from our PERT networks and fed into the computers at NASA-Houston for integration into the total Apollo spacecraft schedules and networks that NASA maintained.

Moran’s group and their PERT diagrams became the principal tool that Rathke, Carbee, Whitaker, Coursen, and I used to plan and schedule Engineering’s activities. The networks included all the drawings and the analyses required from Whitaker’s Systems Engineering groups that were constraints on drawings. The flow of major test activities was added as the networks matured. The drawing estimates and scheduled delivery promises came from the leaders of the individual Engineering design sections based upon their past experience, knowledge of the current design, and their workload and staffing. Bob Carbee for the flight and ground test spacecraft and John Coursen for GSE, probed, challenged, and demanded performance against schedule from their people—day after day, week after week. It took time, but the results of their persistent efforts gradually became visible as the growth in our estimated number of drawings required and slippage against the established delivery schedules decreased throughout 1965 and 1966.

Mountains of Work

We faced a mountain of required drawing outputs that built up rapidly throughout 1965 and plateaued in 1966 at more than four hundred drawings a week. The pressure of schedule was relentless. The expansion of our Engineering groups continued in 1965 and 1966, completely filling Plant 25. Engineers worked in large bullpen areas on the open floor with drawing boards and desks jammed five abreast. (To reach the aisle, the person working at the center board had to pass behind and disturb two people on either side.) Group leaders went from board to board several times a day, collecting nearly finished drawings to be turned over to Drafting or the Release Group and updating the status inputs for PERT. Problems involving lack of information from other Engineering groups, or anything that held up completion of a drawing were taken to the group leaders for action or resolution. There was a corner of the first floor of Plant 25 where a large plywood table had been constructed for preparing oversized drawings and “white masters.” The table was three feet high and about ten feet square. Several engineers or draftsmen worked on their hands and knees on the table, drawing directly on large sheets of vellum paper or on the white painted aluminum sheets on which the “white master” tooling templates were drawn and later cut out to fit into the tool that defined LM skin contours or structure. The three large Engineering floors in Plant 25 were like throbbing frames in a giant beehive, bustling with purposeful activity, visible output, and the hum of accomplishment.

Plant 35, a two-story building with almost the same floor area as Plant 25, was completed in 1965. It provided some relief to the overcrowding in Plant
25 and space for the continuing growth in Engineering staff. By 1966 LM Engineering also occupied substantial office areas in Plants 5 and 30 as the number of engineers neared its peak of almost three thousand. Art Gross and the Facilities Management Group were busy with office moves, expansions, and reconfigurations, which had to be carried out with a minimum of disruption to the ongoing work.

The Drafting Group was a veteran operation in corporate Engineering, dating back to Vice President of Engineering Dick Hutton, a draftsman-engineer who was the first person hired by Chief Engineer Bill Schwendler when Grumman was founded. Draftsmen took drawings after the engineer had drawn and specified the basic design and added the finishing touches, such as dimensions, notes, and additional views, that were needed to make them intelligible and useful to manufacturing. They created detailed parts drawings from assembly drawings and engineering instructions. They did preliminary work to create a drawing by preparing cross sections or views upon which the engineer could work out his design additions or modifications. We often had more than four hundred draftsmen on LM, a number that could be readily be changed by bringing in “job shoppers,” contract draftsmen from manpower supply companies. Extensive use of overtime in Engineering and Drafting evened out the peaks and valleys in manpower demand.

Corporate Drafting was run by Howard Krier and the LM Drafting Group by Ross Chandler. They were a hard-bitten pair of aircraft design veterans and not about to take any nonsense from a young project engineer like me, or from an upstart government agency like NASA for that matter. Krier was a fatherly looking fellow with salt-and-pepper hair and clear-rimmed glasses who considered himself the final authority on anything pertaining to drafting. He had a high-pitched voice that became increasingly annoying to me in debates when he would stubbornly refuse to give ground. Chandler, black-haired and usually toting a large cigar in his mouth, was somewhat younger and had the patient but exasperated look of one who believes he has seen it all.