NASA: project costs no longer heading for the moon
7 May 2014|

Crescent Moon (NASA, International Space Station Science, 11/03/07)

Maybe it’s because I’m a child of the 1960s, but for me ‘NASA’ is a synonym for ‘cool’. You can’t be a science-obsessed lad of six (yes, I was really that sad) in July of 1969 without forming some indelible associations. NASA seemed to be the masters of cutting-edge technology and could pull off anything. It never occurred to me to wonder how much it all cost, or how solid their planning assumptions had been—and, frankly, I wouldn’t have cared.

How things change. A couple of weeks ago I was unreasonably pleased to find the US Government Accountability Office’s annual review of NASA’s portfolio of large scale projects. (Apparently I’m still a sad lad, but with a different focus.) Apart from the fact that anything with NASA in the title is cool (right?), I’ve been closely following their progress in costing and scheduling major projects.

The reason I’m writing about NASA here isn’t just because space science and technology is of high importance to Australia’s civil and military infrastructure and capabilities. Rather, it’s because the NASA experience tells us something about managing high-technology projects, and close management of the mega-project heavy Defence Capability Plan will be vital in ensuring that all of the important ‘joining up’ bits of ADF capability don’t get squeezed out.

Back in the ’60s, with the backing of a President who wanted to do things ‘not because they are easy but because they are hard’, NASA could take a ‘pick a number, add five, and double it‘ approach to budgeting. Today, faced with shrinking budgets and Congresses and Administrations considerably less enamoured with space science than their counterparts fifty years ago, every dollar sent NASA’s way is seen as an opportunity cost elsewhere—as it should be. They need to do better than rough guesses.

The good news is that NASA is doing better:

NASA’s total portfolio of major projects saw cost and schedule growth that remains low compared to GAO’s first review of the portfolio. Some projects in this year’s portfolio launched within their cost and schedule baselines…

Still, we also read that ‘several other [projects] are undergoing replans, which could temper the portfolio’s positive performance’.

But NASA at least seems to be on the right track, and it’s worth understanding how they got there. The answer isn’t romantic; nor is it simple.

What they’ve done is to approach project costing as an engineering problem. Projects are broken down into component parts, each of which has its own technical risk assessment and cost band. In other words, they’ve taken their systems engineering methodology (PDF) seriously and have applied it consistently and rigorously. In that approach, there’s no place for optimistic guesses, or for assuming that immature technologies can be relied upon to ‘come right’, or that they’ll work as hoped when bolted onto other systems. To the greatest extent possible, mature technologies are preferred:

NASA projects have continued to make progress in maturing technologies prior to the preliminary design review, which GAO’s past work has shown is important to decrease the likelihood of cost and schedule growth. This year, 63% of projects met this standard, up from only 29% in 2010. For example, in preparation for its upcoming confirmation review, one project has matured all 10 of its critical technologies.

They’ve also made sure that designs are stable before entering production:

… the agency has also increased its focus on design stability. …over the past several years projects have consistently reported higher percentages of drawings releasable at the critical design review and lower percentages of drawing growth after that time

Hardly rocket science (which NASA is also good at) but our Air Warfare Destroyer project shows what happens when re-design and production overlap. And even when the proper processes are in place, cutting corners leads to bad outcomes:

… in 2013, two projects experienced significant issues immediately after being confirmed, indicating that neither project had completed an adequate assessment of risk.

Finally—and apposite to the DCP—is the simple observation that the biggest projects have the biggest impact on the overall portfolio if they go off the rails. The GAO gives figures for NASA’s performance over the past six years, with and without the inclusion of the James Webb Space Telescope (JWST)—a project that has almost doubled in cost from $4.5 billion in 2006 to $8.8 billion today.

Average launch delay (months) with JWST

Average launch delay (months) without JWST

Average cost growth (%) with JWST

Average cost growth (%) without JWST

2009

11

11

12

12

2010

11

12

13.4

13.6

2011

8

9

14.6

17.8

2012

11

8

46.5

14.6

2013

8

4

46.4

3.9

2014

6.5

3

37.8

3.0

The overall portfolio performance has been steadily improving if you ignore the elephant in the cleanroom. With it, the average cost growth figures look, well, pretty average. The implications for the big projects in the DCP are clear.

Andrew Davies is senior analyst for defence capability and director of research at ASPI. Image courtesy of Flickr user NASA’s Marshall Space Flight Center.