Can Pentagon Bridge Artificial Intelligence’s ‘Valley of Death’?

With a $2 billion investment in artificial intelligence research and development from the Defense Advanced Research Projects Agency, the Pentagon signaled its commitment to outspend the U.S.’s international rivals in an effort to maintain its technological advantage.

DARPA director Steven Walker announced the initiative, known as “AI Next,” at the agency’s 60th anniversary symposium Sept. 7 in National Harbor, Maryland. “With AI Next, we are making multiple research investments aimed at transforming computers from specialized tools to partners in problem-solving,” he said. “We want to explore how machines can acquire human-like communication and reasoning capabilities, with the ability to recognize new situations and environments and adapt to them.”

DARPA’s move follows the Pentagon’s June decision to launch a $1.7 billion Joint Artificial Intelligence Center, or JAIC (pronounced “Jake”), to promote collaboration on AI-related R&D between military service branches, the private sector, and academia.

While some experts are applauding increased investment in AI, others have raised concerns that more funding won’t prevent the U.S. edge from slipping away. That is, unless the Pentagon can find better ways to bridge the tech industry’s “valley of death” and translate innovation in the lab to innovation on the battlefield.

The ‘Valley of Death’

For technology startups, the “valley of death” refers to the painful gap between a product’s development and launch, and the point when it achieves wider adoption and commercial success. It’s where most companies, even those flush with venture capital, perish.

The valley of death also offers an apt metaphor to describe the Pentagon’s challenges with innovation. In recent years, the Pentagon has experimented with a number of ways to accelerate its adoption of emerging technologies, including establishing an outpostsin Silicon Valley, Boston, and Austin – now known as the Defense Innovation Unit, or DIU – and expanding its use of “other transaction” agreements, which bypass some regulatory hurdles to help fast-track small deals with nontraditional suppliers.

This has produced dozens of prototypes, but few, if any, production-scale programs, says Amir Husain, chief executive officer of Austin-based AI startup SparkCognition Inc. “What we have in place right now are relatively smaller contracts and some prototype systems development, dispensed through organizations such as DIU and a few others,” he told Bloomberg Government. “The ability to connect these projects with large scale field deployment is broken. That connection is not scaling.”

SparkCognition was one of a handful of AI companies to receive what’s known as a commercial solutions opening, or CSO, through DIU. In July 2017, DIU arranged a partnership between SparkCognition and the U.S. Air Force to design a prototype that could use machine learning to provide decision support. That way, leaders could simulate the outcomes of various choices, and presumably, arrive at better decisions.

SparkCognition’s venture with the Air Force has earned the company about $795,000, little more than a rounding error for the Pentagon, which has a 2019 budget exceeding $700 billion. The question is, what comes next? The Pentagon doesn’t seem to have a good answer.

Of the dozens of companies that have worked with DIU, just two have received multimillion-dollar “prototype-to-production” OTA awards that would enable them to scale their solutions across an entire agency or department-wide. Yet this approach has attracted controversy.

In February, the Pentagon issued a $950 million fast-track award to REAN Cloud LLC, only to scale it back to $65 million one month later, amid outcry from competitors and reports of poor coordination between DIU and Pentagon leadership. The contract was revoked altogether in late May after the Government Accountability Office sustained a bid protest from Oracle Corp., ruling that the Pentagon should put the contract to a competitive bid.

GAO’s decision has left the Pentagon’s nontraditional suppliers in a state of limbo. Officials remain reluctant to issue production-scale OTA awards out of a fear of inviting scrutiny from GAO or Congress. Meanwhile, few startups possess the experience and financial flexibility to navigate a months-long competitive bidding process.

“If you ask me to rank Russia, China, and DOD procurement in terms of the threats we have, DOD procurement and the difficulty of sourcing the cutting-edge technology that’s available in this country is the biggest threat,” said Husain. “I know that sounds like a harsh statement, but sometimes you can’t produce change without making harsh statements.”

The Defense Department’s attempts to strike AI partnerships with larger, more financially mature technology companies have proven similarly challenging. In June, Google Inc. withdrew from a high-profile contract with the Pentagon’s Project Maven program after employees rebelled over the perceived “militarization” of artificial intelligence. Google’s contract, which involved adapting its image recognition algorithms to improve the accuracy of Pentagon targeting systems, was expected to grow from $15 million to $250 million, according to emails obtained by the New York Times.

A Homegrown Approach to Artificial Intelligence?

Some commentators have interpreted DARPA’s Sept. 7 announcement as a rebuke of companies like Google, promising a more homegrown approach to AI R&D. In reality, DARPA’s AI Next program more or less reflects continuity. The $2 billion will fund both new and existing research over a five-year period. When you consider that DARPA already expected to invest $448 million on AI-related R&D in fiscal 2019, according to the Pentagon’s budget request, it’s not clear how much new spending there will be.

There’s also little indication that this funding will help the Pentagon overcome perhaps its greatest obstacle: fielding and scaling working prototypes. A DARPA spokesperson confirmed to Bloomberg Government that the majority of the $2 billion will remain focused on basic and applied research.

But if the goal is to promote U.S. technological competitiveness in the long run, AI Next is a step in the right direction. Since fiscal 2017, DARPA has stepped up its investment in artificial intelligence by almost 50 percent, from $307 million to $448 million. Dozens of the world’s top tech entrepreneurs, including Google’s Larry Page and Sergey Brin, have benefited from DARPA funding.


The Pentagon’s overall trend is positive as well. Between fiscal years 2017 and 2019, total research, development, test, and evaluation (RDT&E) is expected to grow by about 30 percent, from $1.4 billion to about $1.9 billion. The bulk of that growth is taking place in downstream RDT&E activities, such as advanced component development and prototyping and system development and demonstration. These figures appear to indicate that the Pentagon is investing more than ever to cross artificial intelligence’s valley of death.

What’s Ahead

As part of AI Next, DARPA plans to issue multiple broad agency announcements (BAAs) for artificial intelligence research over the next 12 months, according to the Sept. 7 news release. The agency envisions “streamlined contracting procedures and funding mechanisms” that will allow it to “move from proposal to project kick-off within three months of an opportunity announcement.”

Two such solicitations have already been posted. The first seeks “innovative basic or applied research concepts in the technical domain of automated knowledge discovery, curation, and application,” according to the Aug. 17 posting. DARPA plans to issue an OTA award valued at $1 million. The second is a BAA seeking research proposals for so-called “third wave” AI techniques that address the limitations in rule-based and statistical learning-based models. DARPA will issue multiple OTA awards under this solicitation. Both solicitations are expected to generate interest from universities and innovative companies, both large and small.

Bloomberg Government subscribers can click here to set an alert for AI research opportunities under AI Next.

Chris Cornillie is a federal market analyst with Bloomberg Government.

To contact the analyst: Chris Cornillie in Washington at

To contact the editors responsible for this story: Daniel Snyder at; Jodie Morris at