It’s time to recalibrate the debate over defence innovation. Nothing illustrates how far we’ve drifted like the current passion for ‘picking winners’. The phrase was once pejorative. It has become a mission statement. Recent contributions to the debate read like products of Gosplan, the Soviet Union’s central planning agency: by becoming all-seeing and all-knowing, we can prevent the ‘valley of death’ from killing off innovation in the gap between development and production.

I’ll say it: the valley of death is, on balance, a good thing. It’s not just a place where good ideas go to die; it’s also a place where bad ideas go to die.

We celebrate the outcomes of Silicon Valley but forget that failure is exactly what makes it successful. One recent estimate from the Massachusetts Institute of Technology put the economy-wide success rate for new technologies as low as 5%. The genius of Silicon Valley is that entrepreneurs are empowered to fail fast, learn and try again. They don’t avoid the 95% of projects that fail; they burn through them quickly and emerge from the experience stronger.

The traditional innovation and technology development pipeline is not keeping up with the threat. If we accept that forcing high percentages of projects through to production isn’t the answer, where does that leave us? The only way to increase throughput is to widen the base. We need more investment and more new firms participating in defence innovation. But legacy defence primes grow ever larger by merging with, acquiring and squeezing out dynamic small businesses. Defence-focused, private-sector research and development budgets wither along with their public-sector counterparts.

These trends should come as no surprise; they are the direct result of deliberate choices. These choices fall into two broad categories. First is the size of the prize available to defence innovators. Second is the environment created for them by public investment patterns.

Procurement officials face a dilemma: the basic structure of the market in which they operate is one of natural monopoly. Thus, the US government has Federal Acquisition Regulation Part 15 (FAR 15), which allows it to open the books of defence contractors and dictate ‘fair’ profit margins. FAR 15 is one of many mechanisms that squeeze defence-industry profitability, from the open-systems movement to export controls to the commoditisation of sustainment tails.

Dictating ‘fair’ profit margins has predictable effects on the market. Set them too low and you encourage consolidation and disinvestment. Companies grow and merge, attempting to cover fixed costs through volume rather than margin. They restrict in-house R&D since they won’t be able to recover its costs. Capital shifts towards industries with higher rates of return.

According to data collected by NYU Stern, net margins in the aerospace and defence industry have fallen consistently below that of the broader economy since the series started at the turn of the century. In 2022 they stood at 4%, less than half the economy-wide average. And yet we wring our hands as defence primes gobble up competitors and refuse to invest their own money in R&D.

The flipside of FAR 15 is FAR 12, which exempts commercial products from the sweeping disclosure requirements of Part 15. Even as FAR 15 drives defence-industry consolidation and crushes R&D, FAR 12 pushes innovation out of defence-focused technologies altogether. Increasing reliance on dual-use technology is not a feature; it is a bug. It’s exactly what we would expect from the incentives we’ve been providing to the market for decades.

Governments also lay the foundations of the entrepreneurial environment through their investment patterns. Early-stage private R&D sees chronic underinvestment due to the difficulty businesses face in capturing the value of their innovation. It is for good reason that theoretical physics is the province of non-profit universities and not the Fortune 500. Governments must fill this gap.

Incentives offered to defence R&D organisations push them to do exactly the opposite. Budgets increase with big and obvious wins, which are easier with proven technologies. At the turn of the century, 25% of US funding for research, development, testing and evaluation went to early-stage research (defined as budget activities one through three). The proportion fell to 15% by 2008 and, apart from a blip during the sequestration period as late-stage projects were cut, has hovered in the high teens ever since. The 2024 budget request allocates only 13% of RDT&E funding to early-stage projects.

What are the lessons for Australia?

First, our measure of success shouldn’t be quick wins in publicly funded development. If it’s easy, we’re wasting money on projects that would be perfectly well addressed by the private sector—or worse, forcing technology through to production that doesn’t have a sound business case. Keep public money in early-stage R&D where it belongs. Lay the foundations for tomorrow’s world-beating technologies.

Second, accept that organisation and information flows will get you only so far. Capital markets are efficient at identifying profit opportunities. If there really are billions in unexploited profits languishing in underutilised intellectual property and projects stalled at middle technology-readiness levels, financiers will find them. This is happening in the US through vehicles such as America’s Frontier Fund.

Third, understand how market mechanisms operate in highly regulated industries. A positive rate of return is not good enough; a market-beating rate of return will cause new firms to enter the innovation ecosystem. No amount of browbeating will push defence primes to foster the next generation of competitors. If you want new champions, set the conditions for profitability and feed the innovation pipeline with basic research. For a mid-sized economy like Australia’s, profitability also means minding the addressable market. A healthy export regime is vital.

And above all, learn to love the valley of death.