Practical Advice for Acquiring Artificial Intelligence
By John Ferry
Government acquisition professionals more and more often face the challenge of acquiring cutting-edge technologies such as cloud computing and big data. Artificial intelligence (AI) is just like all new technology in that it presents new operational challenges for contracting and buying programs. Yet, at the same time, AI is nothing like anything government has acquired before.
AI comes with a mountain of new questions and concerns. Some are familiar, such as security, performance, and supportability. Others are less so, including ethical considerations, underlying data rights, bias, and where humans should be involved in its functioning.
To make matters more challenging, an overwhelming number of vendors offer AI solutions wrapped in jargon-filled briefs, obscuring which are genuinely new and which are the same old products sprinkled with AI fairy dust. To simplify acquiring AI, let’s start with a basic framework.
Inductive and Deductive Reasoning in AI Acquisition
In more than eight years supporting technology transition at the Defense Advanced Research Projects Agency (DARPA), it became clear to me that there are essentially two ways government goes about implementing AI: Start from users’ needs and work back to the available AI capability or start with what AI can do and work back to the challenges users face. Without getting overly philosophical, these two different approaches highlight two different ways of looking at the world: deductive and inductive reasoning.
1. Inductive Reasoning: Inductive reasoning starts with specific observations or interactions, which are then synthesized into broader insights or hypotheses.
• Application in AI Acquisition: Engaging with end users, technical experts, and stakeholders through interviews, focus groups, or field observations can provide specific insights. These insights can inform the understanding of the desired capabilities, ethical considerations, or operational requirements of the AI system.
• Benefits: Uncovering nuances, unanticipated needs, preferences, and concerns related to the AI system. Inductive reasoning provides a rich understanding of the context in which the AI system will be deployed.
• Limitations: If the end users or the acquisition professionals are unaware of the realm of the possible, then both are limited by their knowledge and imagination in expressing requirements. This came into play often at DARPA. As Henry Ford once said, when end users are not aware of what's possible, they ask for a faster horse.
2. Deductive Reasoning: Deductive reasoning starts with a hypothesis or theory, which is then tested through structured analysis to arrive at specific conclusions.
• Application in AI Acquisition: Formulating hypotheses based on existing data or materials on AI capabilities, market research, or previous acquisitions, and testing these hypotheses through structured analysis such as surveys, bench-marking studies, or controlled experiments. For instance, hypothesizing that AI could help identify duplicate applicants for government programs, or that generative AI could help publish regulations or acquisition documents faster. The recently publicized Artificial Intelligence Use Case Inventory published by AI.gov provides much fodder for the deductive approach. (See the article, “Innovations: Inventory Shows How Federal Agencies Are Using Artificial Intelligence,” on page 10 of this issue.) (2)
• Benefits: Provides a structured and objective way to validate assumptions, ensuring that the AI system meets predefined criteria and standards. It’s useful for confirming specific requirements and ensuring compliance with standards and policies.
• Limitations: Might overlook unexpected insights or the nuanced understanding that can be derived from inductive methods. The accuracy of conclusions is contingent on the validity of the initial hypotheses. In other words, you might end up trying to hammer a round AI peg into a square operational use case hole.
The Hybrid Approach
There is a third hybrid approach that combines inductive and deductive reasoning, harnessing the strengths of both methods. It identifies user needs and AI capabilities, increasing the chances of finding the most effective solutions:
1. Investigate user and stakeholder requirements (inductive): Interview, conduct focus groups, and observe the field to gain understanding of user requirements, preferences, and the operational environment. Discover the required capabilities, ethical concerns, and operational needs of users and stakeholders.
2. Investigate AI possibilities (deductive): Gain an understanding of current AI offerings by studying industry research, prior purchases, and comparison studies. Create hypotheses about how those AI capabilities can meet user requirements and difficulties.
3. Verify predictions (deductive): Utilize research strategies such as polls, regulated trials, or comparative testing to confirm the established hypotheses. Make sure the AI capabilities are consistent with established criteria, regulations, and rules.
4. Combining insights (inductive): Bring together the findings from the validation stage with the initial user and stakeholder feedback. Improve comprehension of how AI technology can be adjusted to meet the user requirements and the operational context.
5. Create a continuous cycle of refinement: Continuously collect feedback from end users, stakeholders, and technical professionals to guarantee the acquisition process remains in line with the changing needs and evolving AI capabilities. Use feedback to refine hypotheses and knowledge of end-user needs.
This hybrid model is a departure from the top-down/bottom-up requirements and capabilities development process that isn’t a good fit for fast-evolving AI acquisition. If you’re familiar with software development methodologies, this is very similar to the DevOps and Agile approaches; put the users and developers in a loop and iterate until you get to the right outcome.
Test Novel Approaches to AI Acquisition
Acquiring AI for government agencies will require contract managers to continually innovate procurement methods to accommodate rapid advancements in AI technology. Traditional methods likely will fall short in the face of the evolving digital landscape. Fostering collaboration among stakeholders and industry experts through novel approaches will be essential.
Consider leveraging public-private partnerships to enable agencies to collaborate with private sector organizations specializing in AI development. Adopt open-source software for cost-effective access to advanced AI tools. Actively participating in open-source communities and building custom solutions based on existing frameworks fosters transparency, encourages collaboration among developers, and enables customization to meet specific agency requirements.
Test novel approaches to developing and acquiring AI such as hosting capability development challenges or hackathons. These techniques allow agencies, and end users in particular, to evaluate AI offerings firsthand, encourage innovation among participants, and pro-mote transparency and collaboration between government and companies. During interactive competitions, participants are given access to agency data to demonstrate their AI technologies, providing valuable feedback on potential improvements.
“Try before you buy” methods allow agencies to test potential solutions before making a commitment, thus mitigating the risk of investing in costly AI systems that don’t turn out to match government needs. This strategy empowers decision-makers with firsthand experience in assessing the performance and compatibility of AI technologies.
The user-in-the-loop development approach engages users throughout the AI acquisition process, yielding valuable insights about their preferences and expectations, and enhancing their satisfaction. Continuous user involvement significantly increases the likelihood of successful AI adoption, facilitates smooth integration into existing workflows and processes, and allows for early identification and efficient solution of challenges during development.
Looking Ahead
AI is coming to government quickly and in a big way. Programs are demanding AI capabilities, and industry is more than ready to provide them. Acquisition and contracting professionals must be ready to handle this new wave of technology adoption. Readiness includes frameworks for thinking about how users want to implement AI, such as those we explore in this column.
This "AI in Practice" column will also help readers stay up to date on increasingly rapid advanced in AI with frequent "AI News to Know" (see below). Additionally, it provides readers a basic understanding of how AI works, and tips for using AI in contracting and acquisition. Stay tuned!
AI News to Know
• IBM’s NorthPole Chip: A Potential Game-Changer: IBM unveils NorthPole, a novel chip set to turbocharge AI operations with its unified memory and processing architecture. Although in nascent stages, NorthPole promises a leap towards more efficient, potent AI systems. (3)
• AI’s Answer to Meeting Overload: Microsoft researchers are exploring AI to ease the “meeting creep” of remote work. Utilizing asynchronous tools like automated transcripts and meeting recordings, AI can transform meeting transcripts into structured, easy-to-digest documents allowing for after-the-fact interaction and reducing the burden on meeting organizers. This innovation holds the promise of more efficient meetings and new ways to have discussions asynchronously, potentially changing meetings altogether. (4)
• NSA to Inaugurate AI Security Center: A Leap Towards Secure AI Adoption: The coming National Security Agency AI Security Center aims to foster best practices and risk frameworks for AI utilization in national security realms. Collaborating with industry and academia, this initiative seeks to ensure responsible AI deployment. (5)
• Elevate Google Searches with Generative AI on Chrome: Generative AI, through the Search Generative Experience (SGE) in Chrome can refine Google search results. (6) CM
John Ferry is the President of Trenchant Analytics, LLC, creator of AcqBot which is AI for the Department of Defense Chief Data and Artificial Intelligence Office (CDAO) with Tradewinds Solutions Marketplace, a digital environment of post-competition, readily awardable, technology solution pitch videos. He is also the Chief Executive Officer of UseRogue.com, a company applying AI to government sales.
ENDNOTES
1 https://framerusercontent.com/modules/assets/JUxAlI4DcQw4ZjpQVYFqoCt90Y~xG3T-svQNQGYfVmyjBhhaxo3GTEYvFWeJrMAbGgrMZQ.mp4
2 https://ai.gov/ai-use-cases/
3 https://www.nature.com/articles/d41586-023-03267-0
4 https://www.microsoft.com/en-us/worklab/how-ai-could-change-having-too-many-meetings
5 https://www.defense.gov/News/News-Stories/Article/Article/3541838/ai-security-center-to-open-at-national-security-agency/
6 https://www.zdnet.com/article/how-to-supercharge-your-google-searches-with-generative-ai-in-chrome/