INNOVATIONS: Making the Pentagon an Even More Attractive Customer for AI Upstarts
It’s nearly a truism these days that startups and emerging technology firms find selling to government unattractive, burdensome, and risky to their intellectual property. But has anyone really pulsed those companies for evidence of these “facts” or investigated their underlying causes?
Welcome to Innovations—a column designed to help you navigate this time of vibrant change by bringing you inspiring ideas, approaches, and methods you can apply.
It’s nearly a truism these days that startups and emerging technology firms find selling to
Well, as it turns out, yes. Andrew Bowne, U.S. Air Force Judge Advocate, undertook an exploration of the sentiments of AI companies, most of them nontraditional government suppliers, and came up with some surprising findings, which you’ll find in this month’s Innovations column.
It seems that at least in the case of the Department of Defense (DOD),
Federal agencies seeking the innovativeness of rambunctious, young, and small companies could use more of this kind of original research. Picking apart what appears to be true can reveal real strengths on the government side of the table.
—Anne Laurent, NCMA Director of Professional Practice and Innovation
Making the Pentagon an Even More Attractive Customer for AI Upstarts
BY ANDREW S. BOWNE
Despite appearances to the contrary, the Department of Defense (DOD) is an attractive customer for artificial intelligence (AI) companies 1. A new survey reveals that even the DOD contracting attributes that AI companies view negatively—intellectual property requirements, lengthy and cumbersome processes, limited collaboration, impenetrable jargon
AI is a critical element in discussions of military modernization and features prominently in defense strategies, budgets, and congressional testimony. However, with years of ongoing acquisition reform, it is widely accepted that the DOD contracting process, from engagement to close-out, is far from ideal2, particularly in the context of contracting for disruptive technology such as AI3.
There are several reasons why the DOD should examine how it can better attract the AI industry. Unlike during the Cold War, when disruptive technologies such as global positioning system (GPS) satellites and the Internet were invented under DOD-funded R&D programs, today, the commercial sector is the primary driver of advancements in AI4. Additionally, DOD is accustomed to building large systems under the DOD 5000 series, soliciting contracts with 100-plus page requests for production, and working with traditional DOD contractors. However, the AI industry is composed mostly of nontraditional companies. Most innovation in the field of AI originates with tech giants, small start-ups, and academia5, accustomed to the fast-paced, high-risk/high-reward deals of Silicon Valley. Few of the most innovative companies in the field of AI have extensive, if any, experience competing or performing in the comparatively slow and rigid DOD procurement system6. DOD must understand how to convince these innovative companies to compete and perform contracts to help the military realize the promise of AI-enabled capabilities in achieving national security objectives.
Understanding the preferences, opinions, and perceptions of AI firms can inform DOD’s decisions on acquisition strategy, industry engagement, requirement generation, source selection, funding mechanisms, negotiations, and which contract vehicles to use to acquire AI capability. This insight will help develop best practices in attracting and working with innovative AI firms and target acquisition reform efforts.
Attracting AI Firms
DOD must be attractive enough for AI firms to dedicate their finite resources to DOD contracts, foregoing many other revenue-generating opportunities. Thus, the fundamental research question is whether DOD is an attractive customer to commercial AI firms. Several related questions must also be asked to understand and contextualize the answer to the primary question:
• What drivers (customer, supplier, competitors, technology maturity, prior experience, size, culture) and dimensions (bureaucracy, process, cost, time, regulations, complexity, unique terminology, intellectual property law) affect the perceived attractiveness of DOD as a customer for commercial AI?
• To what extent is there a cultural gap between the high-tech AI industry and DOD?
• Which considerations, business-related or cultural, have more weight in the calculus to engage with DOD?
• To what extent is DOD’s procurement process a barrier to engagement with the AI industry?
• Which contract features are perceived as barriers, and which features are preferred by the AI industry?
To help answer these questions, representatives from 111 AI firms volunteered to participate in a comprehensive survey7.
Survey Targets
A total of 370 U.S.-based firms were identified as potential members of the survey population based on their publicly available information on AI products or services with possible defense application8. One hundred fifty-one firms agreed to participate with 111 completing the survey9. Most survey participants were C-Suite executives, vice presidents, or directors of sales or business development. For DOD contractors with separate government divisions or employees (about 37% of firms surveyed), the head of the public sector or defense division often completed the survey. Participation was confidential, though demographic information about each company was collected.
While most firms came from the tech hubs of Silicon Valley (32), the DC region (20), Boston (11), New York (9), Seattle (5), and Austin (5), dozens of firms from outside these areas were surveyed as well. Survey participants were closely split between 49 early-stage startups—i.e., founders, seed, angel funding—and 51 firms with later-stage financing—i.e., Series A, B, C, D, and E. The sample also includes 11 public companies. Approximately 84% of the firms surveyed would qualify as a small business for a DOD contract10.
Roughly 23% of surveyed companies have never performed a DOD contract. Half get less than 30% of their revenue from DOD contracts; 25% receive 70% or more of earnings from Pentagon deals. Thirty-four identified as traditional contractors, having performed a contract or subcontract subject to full coverage under the Cost Accounting Standards within the past year; the other 77 are nontraditional defense contractors11. Eighteen never competed for a government contract, while the other 93 have competed for at least one contract under the Federal Acquisition Regulation, Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs, or other transaction authority (OTA)12. A third of companies are members of at least one OTA consortium.
Survey participants were asked their opinions about the degree of
Collaborate, Communicate, Negotiate
The ability to collaborate, communicate, negotiate intellectual property
The ability to contribute to national security was considered at least somewhat important by 93% and extremely important by 41% of AI firms. Most companies believe it is the AI industry’s responsibility to engage with and support the DOD’s national security mission. In fact, the ability to contribute to national security was selected by 71% as a factor that influences their firm’s decision to engage with DOD. This strong feeling toward an attribute naturally and uniquely associated with DOD appears to provide an advantage to DOD over other potential customers.
Several factors make DOD a unique customer. The AI industry’s strong opinions and preferences can help inform a contracting team’s AI contracting strategy to best attract competition and innovation. DOD can help break down barriers with the AI industry by avoiding jargon and defense-unique terminology. The AI industry strongly disagrees with the notion that DOD uses easy-to-understand terms and conditions. Another way to align a solicitation with the industry’s preferences is to use platforms such as LinkedIn or trade magazines such as Wired or TechCrunch to advertise contract opportunities. More than any other factor, using the website where government contract opportunities are advertised (formerly known as “Federal Business Opportunities” or FedBizOpps, and now found at beta.sam.gov) to review DOD contract opportunities is strongly associated with whether an AI firm already performs significant DOD work.
Solutions, Not Specifications
While AI firms are willing to write proposals, most (more than 3 to 1) prefer direct, face-to-face interactions such as pitches. When choosing a new project, AI firms overwhelmingly prefer opportunities to develop solutions to open-ended problems rather than following pre-set specifications written by the customer (78% to 5%). This preference indicates the vast majority prefer solicitation methods such as commercial solutions openings (CSOs)13, broad agency announcements (BAAs)14, or other approaches more in line with the commercial marketplace than traditional FAR Part 15 requests for proposals. Reinforcing this inference, 70% of the firms surveyed prefer commercial contracting; only 8% prefer the DOD process.
Survey participants strongly prefer contracting vehicles that permit agile methods and iterative steps as opposed to the fixed requirements and milestones associated with waterfall software development15. Only 11% preferred fixed requirements when developing and deploying their product or service. Consistent with a preference for agile development methods, 86% of AI firms surveyed agreed that starting with a prototype, pilot, minimum viable product, or proof of concept is essential before integrating or scaling any new use case.
The question of whether firms prefer FAR contracts over OTs produced an unexpected response. While companies generally preferred OTs over FAR-based contracts, a notably large portion of respondents (47%) had no preference between the two contract types. Importantly, 75% of those with no preference had limited or no experience working with DOD, so their lack of strong opinion might be explained by
Firms that have competed for or been awarded an OT clearly prefer them over FAR contracts, yet even firms that have never competed for or been awarded one slightly prefer OTs.
Strong Positives
About three-quarters of respondents had a positive or very positive opinion of DOD as a customer; less than 9% had a negative opinion. Notably, the statement that received the highest degree of agreement (by far) was that DOD “is an attractive customer to my company”—91% agreed, 47% strongly agreed, and none strongly disagreed. (See FIGURE 2.)
Though most AI firms surveyed believe DOD is an attractive customer, they hold far more negative views of the defense procurement process. Only four companies saw barriers to entry in the defense marketplace as low and only five found the competition process simple and straightforward. Almost all believed that competing for a contract with DOD was more time consuming and difficult than with a commercial customer. The majority (66%) believe performing a DOD contract is more difficult than a commercial contract. Only five believed that performing a commercial contract was more costly than a defense contract. Slightly more felt that winning a defense contract was more important to their business than snagging a commercial contract of the same size. However, more than 70% believed that winning a defense contract is a good path to maximizing profits. Perhaps the strongest indictment of defense contracting is the near-universal disagreement that the commercial marketplace should model its contracting practices on DOD—only six firms agreed.
These findings indicate that the majority of AI firms surveyed believe DOD is an attractive customer despite its contracting practices. Respondents agreed nearly 4 to 1 that if the contract terms and processes were identical, DOD would be a more attractive customer than a commercial buyer.
IP Rights Represent the Biggest Divide
It is widely assumed that DOD and innovative tech companies, particularly those based in Silicon Valley, hold disparate values and have unique cultures. Additionally, the agile, collaborative nature of AI development seems to clash with traditionally rigid DOD contracts governed by regulations and oversight. Finally, the concept of ethical development and deployment of AI capabilities has led to public clashes between DOD and the AI industry17.
This survey indicates that the cultural divide is narrower than is assumed. Generally, the AI industry does not believe commercial customers are more trustworthy than DOD. Most agree that DOD culture, beliefs, behavior, and values are consistent with their own, while only 10% disagreed. All but 8% trust DOD to use AI ethically. (See FIGURE 3.)
The majority said their companies were comfortable knowing that DOD customers might use their product or service for lethal purposes (75% agreed; 11% disagreed). However, many of those uncomfortable with DOD using their AI for lethal purposes were Silicon Valley firms (seven, or about 22% of the 32 Silicon Valley-based firms). Only 9% of all respondents said
The factors that influence a company not to engage with DOD include the length, complexity, and rigidity of the procurement process; lack of communication with the customer; and concerns about obtaining
However, the factors that influence AI firms to engage with DOD make it a uniquely attractive customer. The ability to contribute to national security is the top factor influencing AI firms to engage with DOD, joined by the ability to work on challenging projects (71% of respondents). Just behind these two is patriotism (70%). These are powerful factors in attracting AI firms. Second-tier influential factors include the perception that DOD offers favorable funding and profits, listed by 59% and 57% of respondents, respectively.
Conclusion
While contract law and defense-unique procurement regulations act as barriers to engagement with the AI industry, the primary reasons DOD is still perceived as an attractive customer are related to the unique nature of DOD’s mission. This offers reason for optimism for DOD acquisition professionals. Contract tools exist to improve communication and collaboration between the AI industry and
Requirements can be fulfilled with open-ended problem statements rather than dictated specifications. By understanding the AI industry’s perceptions, preferences, and opinions, DOD can better identify best practices available under existing law and target areas for reform. Adopting these best practices to better attract commercial AI firms is critical for DOD to meet its national defense goals. CM
Andrew S. Bowne
• Ph.D. candidate at the Adelaide Law School, University of Adelaide, Australia. This article is based on research conducted as part of his Ph.D. studies.
• Active-duty U.S. Air Force Judge Advocate. The views expressed in this article are solely those of the author and do not necessarily represent the views of the Judge Advocate General, Department of the Air Force, or any other department or agency of the U.S. federal government.
Endnotes
1Employees at several high-profile companies have cited ethical concerns about DOD use of AI as
2Eighty-two percent of national security experts believe DOD contracting requirements are unnecessarily burdensome. (National Security Institute, The U.S. Defense Industrial Base: Can it Compete in the Next Century?, 21 (2020).)
3Cong. Res. Serv., R45178, Artificial Intelligence
4Ibid., at 2; see also Andrew Ilachinski, AI, Robots, and Swarms: Issues, Questions, and Recommended Studies, 231 (CNA, 2017). Part of the reason for the shift from public to private innovation is certainly attributed to the levels of funding coming from the private sector. While the post-Sputnik public funding of R&D outmatched industry funding 2 to 1, currently the gap is 1 to 3 and growing. (Innovation and National Security: Keeping Our Edge, Council on Foreign Relations, 19 (2019), available at https://www.cfr.org/report/keeping-our-edge/pdf/TFR_Innovation_Strategy.pdf.)
5Asa Fitch, “Tech Giants Hunt for AI Startups—and the Brains Behind Them,” Wall Street Journal, available at https://www.wsj.com/articles/tech-giants-hunt-for-ai-startupsand-the-brains-behind-them-11577365201.
6Comparing the top 100 AI companies in 2020 by innovation and the top 100 Defense contractors by Defense revenue, there is not a single company on both lists. (Compare Andy Patrizio & James Maguire, “Top 100 Artificial Intelligence Companies 2020,” Datamation (July 2, 2020), available at https://www.datamation.com/artificial-intelligence/top-artificial-intelligence-companies.html, with “Top 100 for 2020,” DefenseNews, available at https://people.defensenews.com/top-100/.)
7This survey was part of a broader research effort. The survey findings informed the questions asked in interviews of representatives from 15 AI firms in an explanatory sequential design
8There may be many more firms that provide AI-enabled capabilities and solutions that could be relevant to DOD. Firms that were not listed or identified in the databases previously described were omitted. AI firms that appear to only work in
9Seventeen participants did not clear the screening questions, and an additional 23 did not complete the survey after clearing the screening questions. Data from those participants who did not complete the survey for any reason was not analyzed in the study in accordance with the terms of participation and consent.
10Based on size standards for NAICS code 541715 (up to 1,000 employees). U.S. Small Business Association, Table of Small Business Size Standards Match to North American Industry Classification System Codes (effective January 1, 2017). A review of FPDS data on DOD contracts for AI solutions revealed that the most common NAICS code used by the contracting agency was 541715 (Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology)).
10 U.S.C. § 2302(9) defines “nontraditional contractor” with respect to
12OTA provides DOD with the ability to enter into flexible contract arrangements called other transaction (OT) agreements that generally exempt from procurement statutes and regulations that are required for FAR-based contracts or grants. These contracts are generally for prototypes with the potential to enter into production without competition and are intended to reduce barriers to the defense market for nontraditional defense contractors. DOD’s other transaction authority is codified in 10 U.S.C. § 2371b.
13A commercial solutions opening is a solicitation method that can be used to acquire innovative commercial solutions through competition. This process allows DOD to obtain solutions or new capabilities to close capability gaps, fulfill requirements, or provide technology advances through an open-ended area of interest rather than predefined specifications. (See DAU Contracting Cone, “Defense Commercial Solutions Opening (CSO) Pilot Program,” available at https://aaf.dau.edu/aaf/contracting-cone/defense-cso/.
14A broad agency announcement is a solicitation method that is used to obtain proposals for research and development in science and technology initiatives under FAR 35.016, though the process can be used for non-FAR contracts, such as OTs. These proposals are not related to a specific requirement but rather open topics of strategic importance and technical capability that are used to build knowledge in basic, applied, and advanced research and development. (See DAU Contracting Cone, “Broad Agency Announcement (BAA) – FAR Part 35.016,” available at https://aaf.dau.edu/aaf/contracting-cone/baa/.
15Traditional project management methods such as “waterfall” are used to develop software in a sequential, linear process with discrete phases. “Agile” methods differ from waterfall by completing steps in development in an iterative, collaborative, and fluid environment. (See Villanova University, “Waterfall vs. Agile in Project Management,” available at https://www.villanovau.com/resources/project-management/waterfall-vs-agile-project-management/.
16The interviews conducted after the surveys indicated that OTs are strongly preferred over FAR contracts by AI firms, with the notable exception of legacy defense contractors that appreciate the barriers to new entrants posed by traditional defense contracts as limiting the strength of their competition.
17Billy Mitchell, “The ‘Silver Lining’ in the Fallout between Google and Project Maven,” FedScoop (December 10, 2019), available at https://www.fedscoop.com/project-maven-silver-lining-peter-thiel/.
Share Your Stories of Innovation! |
---|