INNOVATIONS: Making the Pentagon an Even More Attractive Customer for AI Upstarts

 Welcome to Innovations—a column designed to help you navigate this time of vibrant change by bringing you inspiring ideas, approaches, and methods you can apply.

It’s nearly a truism these days that startups and emerging technology firms find selling to government unattractive, burdensome, and risky to their intellectual property. But has anyone really pulsed those companies for evidence of these “facts” or investigated their underlying causes?

Well, as it turns out, yes. Andrew Bowne, U.S. Air Force Judge Advocate, undertook an exploration of the sentiments of AI companies, most of them nontraditional government suppliers, and came up with some surprising findings, which you’ll find in this month’s Innovations column.

It seems that at least in the case of the Department of Defense (DOD), government isn’t as off-putting a customer as has been portrayed, at least not among AI companies. Picking apart motivations and turn-offs shows that it’s not DOD, but its contracting process that AI firms eschew. Bowne’s findings also reveal ways that DOD can counter companies’ perceived negatives in selling to government.

Federal agencies seeking the innovativeness of rambunctious, young, and small companies could use more of this kind of original research. Picking apart what appears to be true can reveal real strengths on the government side of the table.

—Anne Laurent, NCMA Director of Professional Practice and Innovation

Making the Pentagon an Even More Attractive Customer for AI Upstarts

BY ANDREW S. BOWNE

Despite appearances to the contrary, the Department of Defense (DOD) is an attractive customer for artificial intelligence (AI) companies ¹.  A new survey reveals that even the DOD contracting attributes that AI companies view negatively—intellectual property requirements, lengthy and cumbersome processes, limited collaboration, impenetrable jargon and lack of understanding of startup and emerging firms—can be overcome within the DOD system. Further, DOD remains an attractive customer due to AI companies’ highly positive views of the opportunity to contribute to national security, work challenging projects, and have access to large and stable budgets.

AI is a critical element in discussions of military modernization and features prominently in defense strategies, budgets, and congressional testimony. However, with years of ongoing acquisition reform, it is widely accepted that the DOD contracting process, from engagement to close-out, is far from ideal²,  particularly in the context of contracting for disruptive technology such as AI³.  

There are several reasons why the DOD should examine how it can better attract the AI industry. Unlike during the Cold War, when disruptive technologies such as global positioning system (GPS) satellites and the Internet were invented under DOD-funded R&D programs, today, the commercial sector is the primary driver of advancements in AI⁴.  Additionally, DOD is accustomed to building large systems under the DOD 5000 series, soliciting contracts with 100-plus page requests for production, and working with traditional DOD contractors. However, the AI industry is composed mostly of nontraditional companies. Most innovation in the field of AI originates with tech giants, small start-ups, and academia⁵,  accustomed to the fast-paced, high-risk/high-reward deals of Silicon Valley. Few of the most innovative companies in the field of AI have extensive, if any, experience competing or performing in the comparatively slow and rigid DOD procurement system⁶.  DOD must understand how to convince these innovative companies to compete and perform contracts to help the military realize the promise of AI-enabled capabilities in achieving national security objectives. 

Understanding the preferences, opinions, and perceptions of AI firms can inform DOD’s decisions on acquisition strategy, industry engagement, requirement generation, source selection, funding mechanisms, negotiations, and which contract vehicles to use to acquire AI capability. This insight will help develop best practices in attracting and working with innovative AI firms and target acquisition reform efforts.

Attracting AI Firms

DOD must be attractive enough for AI firms to dedicate their finite resources to DOD contracts, foregoing many other revenue-generating opportunities. Thus, the fundamental research question is whether DOD is an attractive customer to commercial AI firms. Several related questions must also be asked to understand and contextualize the answer to the primary question:

• What drivers (customer, supplier, competitors, technology maturity, prior experience, size, culture) and dimensions (bureaucracy, process, cost, time, regulations, complexity, unique terminology, intellectual property law) affect the perceived attractiveness of DOD as a customer for commercial AI?
• To what extent is there a cultural gap between the high-tech AI industry and DOD?
• Which considerations, business-related or cultural, have more weight in the calculus to engage with DOD?
• To what extent is DOD’s procurement process a barrier to engagement with the AI industry?
• Which contract features are perceived as barriers, and which features are preferred by the AI industry?
To help answer these questions, representatives from 111 AI firms volunteered to participate in a comprehensive survey⁷.

Survey Targets

A total of 370 U.S.-based firms were identified as potential members of the survey population based on their publicly available information on AI products or services with possible defense application⁸.  One hundred fifty-one firms agreed to participate with 111 completing the survey⁹.  Most survey participants were C-Suite executives, vice presidents, or directors of sales or business development. For DOD contractors with separate government divisions or employees (about 37% of firms surveyed), the head of the public sector or defense division often completed the survey. Participation was confidential, though demographic information about each company was collected.

While most firms came from the tech hubs of Silicon Valley (32), the DC region (20), Boston (11), New York (9), Seattle (5), and Austin (5), dozens of firms from outside these areas were surveyed as well. Survey participants were closely split between 49 early-stage startups—i.e., founders, seed, angel funding—and 51 firms with later-stage financing—i.e., Series A, B, C, D, and E. The sample also includes 11 public companies. Approximately 84% of the firms surveyed would qualify as a small business for a DOD contract¹⁰.  

Roughly 23% of surveyed companies have never performed a DOD contract. Half get less than 30% of their revenue from DOD contracts; 25% receive 70% or more of earnings from Pentagon deals. Thirty-four identified as traditional contractors, having performed a contract or subcontract subject to full coverage under the Cost Accounting Standards within the past year; the other 77 are nontraditional defense contractors¹¹.  Eighteen never competed for a government contract, while the other 93 have competed for at least one contract under the Federal Acquisition Regulation, Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs, or other transaction authority (OTA)¹².  A third of companies are members of at least one OTA consortium.

Survey participants were asked their opinions about the degree of importance their firms place on several contract factors using a five-point scale from extremely important to not at all important. (See FIGURE 1 for the factors and the responses.) 

Collaborate, Communicate, Negotiate

The ability to collaborate, communicate, negotiate intellectual property rights, and the process’s timeliness were the most important factors to survey participants. The potential for the commercial success of the product or service and the consistency of business practices between the commercial and government markets were some of the least important overall. These findings indicate that contracting processes that are collaborative, timely, and take into consideration IP concerns are likely to be attractive to the AI industry, even if the DOD contracting process differs from the commercial market.

The ability to contribute to national security was considered at least somewhat important by 93% and extremely important by 41% of AI firms. Most companies believe it is the AI industry’s responsibility to engage with and support the DOD’s national security mission. In fact, the ability to contribute to national security was selected by 71% as a factor that influences their firm’s decision to engage with DOD. This strong feeling toward an attribute naturally and uniquely associated with DOD appears to provide an advantage to DOD over other potential customers.

Several factors make DOD a unique customer. The AI industry’s strong opinions and preferences can help inform a contracting team’s AI contracting strategy to best attract competition and innovation. DOD can help break down barriers with the AI industry by avoiding jargon and defense-unique terminology. The AI industry strongly disagrees with the notion that DOD uses easy-to-understand terms and conditions. Another way to align a solicitation with the industry’s preferences is to use platforms such as LinkedIn or trade magazines such as Wired or TechCrunch to advertise contract opportunities. More than any other factor, using the website where government contract opportunities are advertised (formerly known as “Federal Business Opportunities” or FedBizOpps, and now found at beta.sam.gov) to review DOD contract opportunities is strongly associated with whether an AI firm already performs significant DOD work.  

Solutions, Not Specifications

 While AI firms are willing to write proposals, most (more than 3 to 1) prefer direct, face-to-face interactions such as pitches. When choosing a new project, AI firms overwhelmingly prefer opportunities to develop solutions to open-ended problems rather than following pre-set specifications written by the customer (78% to 5%). This preference indicates the vast majority prefer solicitation methods such as commercial solutions openings (CSOs)¹³,  broad agency announcements (BAAs)¹⁴,  or other approaches more in line with the commercial marketplace than traditional FAR Part 15 requests for proposals. Reinforcing this inference, 70% of the firms surveyed prefer commercial contracting; only 8% prefer the DOD process. 

Survey participants strongly prefer contracting vehicles that permit agile methods and iterative steps as opposed to the fixed requirements and milestones associated with waterfall software development¹⁵.  Only 11% preferred fixed requirements when developing and deploying their product or service. Consistent with a preference for agile development methods, 86% of AI firms surveyed agreed that starting with a prototype, pilot, minimum viable product, or proof of concept is essential before integrating or scaling any new use case.

The question of whether firms prefer FAR contracts over OTs produced an unexpected response. While companies generally preferred OTs over FAR-based contracts, a notably large portion of respondents (47%) had no preference between the two contract types. Importantly, 75% of those with no preference had limited or no experience working with DOD, so their lack of strong opinion might be explained by lack of knowledge of the two contracting approaches. Seventy-four percent of firms expressing a preference favored OTs, many of them (41%) strongly.

Firms that have competed for or been awarded an OT clearly prefer them over FAR contracts, yet even firms that have never competed for or been awarded one slightly prefer OTs. Nontraditionals had a clear preference for OTs on average, while defense contractors slightly preferred FAR contracts. The contract vehicle preference findings are inconclusive due to lack of relevant experience among nearly half the survey sample. However, participants strongly preferred contract attributes offered by OTs, such as flexibility, collaboration, agility, negotiation, prototyping, and speed, which they rated as essential factors in their calculus for pursuing DOD contracts¹⁶.  

Strong Positives

About three-quarters of respondents had a positive or very positive opinion of DOD as a customer; less than 9% had a negative opinion. Notably, the statement that received the highest degree of agreement (by far) was that DOD “is an attractive customer to my company”—91% agreed, 47% strongly agreed, and none strongly disagreed. (See FIGURE 2.)

Though most AI firms surveyed believe DOD is an attractive customer, they hold far more negative views of the defense procurement process. Only four companies saw barriers to entry in the defense marketplace as low and only five found the competition process simple and straightforward. Almost all believed that competing for a contract with DOD was more time consuming and difficult than with a commercial customer. The majority (66%) believe performing a DOD contract is more difficult than a commercial contract. Only five believed that performing a commercial contract was more costly than a defense contract. Slightly more felt that winning a defense contract was more important to their business than snagging a commercial contract of the same size. However, more than 70% believed that winning a defense contract is a good path to maximizing profits. Perhaps the strongest indictment of defense contracting is the near-universal disagreement that the commercial marketplace should model its contracting practices on DOD—only six firms agreed.

These findings indicate that the majority of AI firms surveyed believe DOD is an attractive customer despite its contracting practices. Respondents agreed nearly 4 to 1 that if the contract terms and processes were identical, DOD would be a more attractive customer than a commercial buyer.

IP Rights Represent the Biggest Divide

 It is widely assumed that DOD and innovative tech companies, particularly those based in Silicon Valley, hold disparate values and have unique cultures. Additionally, the agile, collaborative nature of AI development seems to clash with traditionally rigid DOD contracts governed by regulations and oversight. Finally, the concept of ethical development and deployment of AI capabilities has led to public clashes between DOD and the AI industry¹⁷.

 This survey indicates that the cultural divide is narrower than is assumed. Generally, the AI industry does not believe commercial customers are more trustworthy than DOD. Most agree that DOD culture, beliefs, behavior, and values are consistent with their own, while only 10% disagreed. All but 8% trust DOD to use AI ethically. (See FIGURE 3.)

The majority said their companies were comfortable knowing that DOD customers might use their product or service for lethal purposes (75% agreed; 11% disagreed). However, many of those uncomfortable with DOD using their AI for lethal purposes were Silicon Valley firms (seven, or about 22% of the 32 Silicon Valley-based firms). Only 9% of all respondents said potential use of their technology on a weapon system would prevent their firm from working with DOD. Even with this concern, of the 9% of companies that would not work with DOD if their tech was used on a weapon system, most already provide AI solutions to DOD and have a positive opinion of DOD as a customer overall. Many companies are more than willing to engage with DOD in spite of cultural differences and ethical concerns.

The factors that influence a company not to engage with DOD include the length, complexity, and rigidity of the procurement process; lack of communication with the customer; and concerns about obtaining security clearance. The perceived long timeline ranks high as an absolute barrier, but the biggest concern was intellectual property (IP). More than half (55%) of companies surveyed would not work with DOD if unlimited rights in technical data or computer software were required as part of the contract. Twenty-eight percent balked even at government purpose rights, making concerns about IP the most significant barrier for DOD in attracting AI firms.

However, the factors that influence AI firms to engage with DOD make it a uniquely attractive customer. The ability to contribute to national security is the top factor influencing AI firms to engage with DOD, joined by the ability to work on challenging projects (71% of respondents). Just behind these two is patriotism (70%). These are powerful factors in attracting AI firms. Second-tier influential factors include the perception that DOD offers favorable funding and profits, listed by 59% and 57% of respondents, respectively.

Conclusion

 While contract law and defense-unique procurement regulations act as barriers to engagement with the AI industry, the primary reasons DOD is still perceived as an attractive customer are related to the unique nature of DOD’s mission. This offers reason for optimism for DOD acquisition professionals. Contract tools exist to improve communication and collaboration between the AI industry and end users. IP terms can be negotiated. Contract opportunities can be advertised using plain language and publicized in fora actually seen by innovative tech firms.

Requirements can be fulfilled with open-ended problem statements rather than dictated specifications. By understanding the AI industry’s perceptions, preferences, and opinions, DOD can better identify best practices available under existing law and target areas for reform. Adopting these best practices to better attract commercial AI firms is critical for DOD to meet its national defense goals. CM

Andrew S. Bowne
• Ph.D. candidate at the Adelaide Law School, University of Adelaide, Australia. This article is based on research conducted as part of his Ph.D. studies.
• Active-duty U.S. Air Force Judge Advocate. The views expressed in this article are solely those of the author and do not necessarily represent the views of the Judge Advocate General, Department of the Air Force, or any other department or agency of the U.S. federal government.

Endnotes 

¹Employees at several high-profile companies have cited ethical concerns about DOD use of AI as reason to refuse to create technology for the military. A mass protest by Google employees led the company to end its role in Project Maven, a full-motion video analysis system for intelligence analysts. A similar plea from Microsoft employees to withhold its augmented reality HoloLens 2 headsets from the U.S. Army shortly after led to growing concern about tension between the AI industry and DOD. (See Jack Corigan, “Microsoft Stands By Its $480 Million Pentagon Contract,” Nextgov (February 25, 2019), available at https://www.nextgov.com/emerging-tech/2019/02/microsoft-stands-its-480-million-pentagon-contract/155120/.)

²Eighty-two percent of national security experts believe DOD contracting requirements are unnecessarily burdensome. (National Security Institute, The U.S. Defense Industrial Base: Can it Compete in the Next Century?, 21 (2020).)

³Cong. Res. Serv., R45178, Artificial Intelligence and National Security, 17-18 (updated November 10, 2020), available at https://fas.org/sgp/crs/natsec/R45178.pdf.

⁴Ibid., at 2; see also Andrew Ilachinski, AI, Robots, and Swarms: Issues, Questions, and Recommended Studies, 231 (CNA, 2017). Part of the reason for the shift from public to private innovation is certainly attributed to the levels of funding coming from the private sector. While the post-Sputnik public funding of R&D outmatched industry funding 2 to 1, currently the gap is 1 to 3 and growing. (Innovation and National Security: Keeping Our Edge, Council on Foreign Relations, 19 (2019), available at https://www.cfr.org/report/keeping-our-edge/pdf/TFR_Innovation_Strategy.pdf.)

⁵Asa Fitch, “Tech Giants Hunt for AI Startups—and the Brains Behind Them,” Wall Street Journal, available at https://www.wsj.com/articles/tech-giants-hunt-for-ai-startupsand-the-brains-behind-them-11577365201.

⁶Comparing the top 100 AI companies in 2020 by innovation and the top 100 Defense contractors by Defense revenue, there is not a single company on both lists. (Compare Andy Patrizio & James Maguire, “Top 100 Artificial Intelligence Companies 2020,” Datamation (July 2, 2020), available at https://www.datamation.com/artificial-intelligence/top-artificial-intelligence-companies.html, with “Top 100 for 2020,” DefenseNews, available at https://people.defensenews.com/top-100/.)

⁷This survey was part of a broader research effort. The survey findings informed the questions asked in interviews of representatives from 15 AI firms in an explanatory sequential design mixed methods research project. (See John W. Creswell and J. David Creswell, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (SAGE Publications, Inc., 5th ed., 2018), 218.) Findings and conclusions from the interviews will be published separately.

⁸There may be many more firms that provide AI-enabled capabilities and solutions that could be relevant to DOD. Firms that were not listed or identified in the databases previously described were omitted. AI firms that appear to only work in nondefense industries, such as advertising or entertainment, were not invited to participate in this study. All firms surveyed were U.S.-owned and operated, with one exception of a foreign owned, but U.S.-headquartered firm. With multiple layers of subjective judgment used in identifying the target population, it is likely the field of AI firms that make up the target population is even larger.

⁹Seventeen participants did not clear the screening questions, and an additional 23 did not complete the survey after clearing the screening questions. Data from those participants who did not complete the survey for any reason was not analyzed in the study in accordance with the terms of participation and consent.

¹⁰Based on size standards for NAICS code 541715 (up to 1,000 employees). U.S. Small Business Association, Table of Small Business Size Standards Match to North American Industry Classification System Codes (effective January 1, 2017). A review of FPDS data on DOD contracts for AI solutions revealed that the most common NAICS code used by the contracting agency was 541715 (Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology)).

^{10 U.S.C. § 2302(9) defines “nontraditional contractor” with respect to a procurement or other transaction.}

¹²OTA provides DOD with the ability to enter into flexible contract arrangements called other transaction (OT) agreements that generally exempt from procurement statutes and regulations that are required for FAR-based contracts or grants. These contracts are generally for prototypes with the potential to enter into production without competition and are intended to reduce barriers to the defense market for nontraditional defense contractors. DOD’s other transaction authority is codified in 10 U.S.C. § 2371b.

¹³A commercial solutions opening is a solicitation method that can be used to acquire innovative commercial solutions through competition. This process allows DOD to obtain solutions or new capabilities to close capability gaps, fulfill requirements, or provide technology advances through an open-ended area of interest rather than predefined specifications. (See DAU Contracting Cone, “Defense Commercial Solutions Opening (CSO) Pilot Program,” available at https://aaf.dau.edu/aaf/contracting-cone/defense-cso/.

¹⁴A broad agency announcement is a solicitation method that is used to obtain proposals for research and development in science and technology initiatives under FAR 35.016, though the process can be used for non-FAR contracts, such as OTs. These proposals are not related to a specific requirement but rather open topics of strategic importance and technical capability that are used to build knowledge in basic, applied, and advanced research and development. (See DAU Contracting Cone, “Broad Agency Announcement (BAA) – FAR Part 35.016,” available at https://aaf.dau.edu/aaf/contracting-cone/baa/.

¹⁵Traditional project management methods such as “waterfall” are used to develop software in a sequential, linear process with discrete phases. “Agile” methods differ from waterfall by completing steps in development in an iterative, collaborative, and fluid environment. (See Villanova University, “Waterfall vs. Agile in Project Management,” available at https://www.villanovau.com/resources/project-management/waterfall-vs-agile-project-management/.

¹⁶The interviews conducted after the surveys indicated that OTs are strongly preferred over FAR contracts by AI firms, with the notable exception of legacy defense contractors that appreciate the barriers to new entrants posed by traditional defense contracts as limiting the strength of their competition.

¹⁷Billy Mitchell, “The ‘Silver Lining’ in the Fallout between Google and Project Maven,” FedScoop (December 10, 2019), available at https://www.fedscoop.com/project-maven-silver-lining-peter-thiel/.