Foreword

The Department of Defense (DoD) and civilian agencies are attempting to bring in the best, most innovative, and capable bidders for federal contracts using solicitations that are difficult or very difficult to read—fewer than 3 percent are written in plain English. Small businesses asked to read solicitations that matched their capabilities lifted from beta. sam.gov were appalled. “That was SUPER painful.” “Super annoying.” “I’m struggling mightily.” And not only are solicitations dauntingly difficult to read, but the government gives bidders almost no time to digest them and respond. Every year for the past ten, 70 percent or more of Defense solicitations sought responses within 21 days; 30 percent gave just ten days. Amanda and Alex Bresler deserve our thanks for crunching the real data to prove just how hard government makes it for companies to play in the federal market. So now we know without a doubt why so many firms avoid it like the plague. Read on to learn about more hurdles, and solid advice for DoD and all agencies about how to fix these problems. These are urgent times. The Pentagon is making it abundantly clear that it needs new suppliers at the cutting-edge of capability. Civilian agencies are no different, and a leading few are joining DoD in battling barriers. They all should apply the Breslers’ evidence-based recommendations.

Anne Laurent
NCMA Director of Professional Practice and Innovation

 

To Lure Innovators, Give Them Clarity and Time to Avoid Redundancy

By Amanda and Alex Bresler

 

Over the last 20 years, companies outside the traditional defense industrial base (nontraditionals) have emerged at the forefront of innovation in areas critically important to national defense, making it vital that the military attracts and engages these suppliers.

Last year,^[i] we sought to evaluate whether the Department of Defense (DoD) has been accomplishing this goal. We analyzed the number of new vendors that worked with the DoD annually over the last decade. Despite billions in investment for innovation initiatives, we found that the number of new vendors contracted by the military has steadily declined: in 2010, approximately 19 percent of DoD vendors had no prior defense business, compared with just 8 percent in 2019. Furthermore, exploring new vendors’ Product Service Codes, we realized that most of these companies were not contracted for goods or services related to commercial innovation.

We posited that one reason the DoD does a poor job of attracting innovative new vendors is because it fails to market requirements in ways that reach and engage suppliers outside the traditional defense industrial base (DIB). Thus, we used qualitative and quantitative research—including analyzing millions of archived solicitations from beta.sam.gov (beta.sam)—to explore four basic criteria related to how the DoD communicates with industry:

• Content/Readability: The extent to which requirements are written clearly and contain necessary information
• Response Time: The number of days between when an opportunity is posted and the response due date
• Redundancy: The extent to which multiple DOD/government entities are simultaneously seeking similar solutions
• Discoverability: The extent to which nontraditionals can identify relevant DOD opportunities 

In each area, we demonstrated that the DOD’s methods of marketing its requirements substantially inhibit access to nontraditionals. We offer concrete recommendations for how the DOD can course-correct to reach and engage a broader, more diverse audience of potential suppliers, thereby ensuring that warfighters have access to cutting-edge technologies necessary to fight and win.

Content/Readability

To attract a broad audience, DOD requirements must be written clearly so that potential suppliers can grasp what customers seek and evaluate which opportunities are worth pursuing. Therefore, we first sought to gauge how readable and understandable DOD solicitations are.

We aggregated archived solicitation data from beta.sam in each year from 2002 through 2020 and filtered and cleaned the data to isolate 1,050,933 unique solicitations issued by DOD. We assessed each solicitation using the Flesch-Kincaid (F-K) Reading-Ease test and the F-K Grade Level test, which weigh total words, total sentences, and total syllables to indicate how difficult a passage is to understand. The two tests’ scores correspond to one another: Text classified as “Difficult to Read” is equivalent to the “College” grade level, “Very Difficult to Read” is equivalent to “College Graduate” grade level, and so forth. 

As Figure 1 shows, most DOD solicitations were “Difficult” or “Very Difficult” to read. Nearly 59 percent of the solicitations required some college-level education, and another nearly 20 percent were suited for college graduates. By comparison, fewer than 3 percent were written in plain English.

Figure 1: DOD Solicitations, Scored by Reading-Ease and Grade Level

 

The fact that most requirements are not written in concise, accessible language limits competition. If a company cannot understand what the customer is looking for, it cannot and will not participate in a requirements process. We also found that the DOD often omits critical pieces of information altogether. For instance, there are no required fields for contract value or performance period. Nontraditionals, especially those with robust private sector investment and revenue, are unlikely to invest time and resources to explore an engagement without some sense of the potential upside or when the work might begin.

We also surveyed 23 small businesses, all participants in the Air Force’s Small Business Innovation Research (SBIR)^[ii] program, to gather their feedback on solicitation readability. We identified opportunities via beta.sam that appeared to be relevant to each company’s capabilities, shared the opportunity links with the companies, and asked for feedback on each match. The fact that these companies already worked with DoD implied that they would be comfortable deciphering government opportunities. On the contrary: They were frustrated by how challenging it was to make sense of the solicitations, and commented that the government tends to bury critical information in cumbersome attachments:

• “I cannot tell from the AOIs [Areas of Interest] what they are asking.”
• “I'm struggling mightily to find the AOIs that say what the DoD really wants.”
• “Super annoying that I had to comb through attachments to find the AOI’s topic.”
• “BAAs [Broad Agency Announcements] are complex.”
• “These BAAs take quite a while to go through and communicate.”
• “That was SUPER painful...because of the opacity with which those SAM postings are written. There are a couple—even AFTER downloading the documents from SAM— that remain mysterious.”

Readability Recommendations

We recommend that the DoD require that solicitations be written in plain English, suitable for an eighth- to ninth-grade reading level. The government can incorporate a feature that automatically reads the text of every new solicitation, calculates its F-K scores prior to publication, and automatically recommends simpler language if requisite F-K levels are not met. Opportunities should explicitly state the customer’s primary areas of interest and contain estimated or actual contract value and performance periods. Estimates can be generated using an algorithmic approach, to include aggregating and weighing factors such as average contract size awarded by the contracting office over the last five years, average contract size for the product or service, and budget estimates.

Response Time

For the DoD’s supplier outreach efforts to be effective, companies also need ample time to identify, assess, and respond to an opportunity. We next calculated the response timeframe for each of the 1,050,933 DoD solicitations.

As Figure 2 shows, in every year over the last decade, 70 percent or more of all DoD solicitations had a response time of 21 days or less, and in each year, with the exception of 2020, at least 30 percent of solicitations required responses within 10 days.

Figure 2: DOD Solicitation Response Time by Year

Exploring response time by notice type, as Figure 3 shows, we also found that most Special Notices and Sources Sought and nearly half of all Presolicitations have turnarounds of 21 days or less.

Figure 3: Response Time by Notice Type

These notices are intended to inform future requirements by gathering information from a broad range of suppliers about their capabilities and how they would approach solving the DoD’s stated problem(s). It is especially important that nontraditionals participate in them; otherwise the military’s view of how problems can be solved is shaped exclusively by entrenched suppliers. Yet, based on response time alone, companies unfamiliar with the DoD’s supplier outreach methods are effectively precluded from these critical calls for market research.

DoD opportunities with aggressively short turnaround times often are referred to by companies as “wired,” meaning the customer already has identified a preferred vendor and the bid process exists only for compliance purposes. We recognize the importance of being able to engage with suppliers swiftly—in fact, allowing companies to contract quickly is critical for attracting innovators. However, the volume of opportunities with anticompetitive turnaround times indicates a disconnect with the intent of regulatory standards requiring real competition. We recommend that the DoD be required to make solicitations active for at least 30 days or justify circumventing this requirement. 

Redundancy

Many of the same technologies are in high demand across all branches of the military. In cases where multiple DoD stakeholders are seeking similar solutions, a company must not only identify and decipher relevant opportunities, but also decide which ones to pursue.

To assess the scope of the redundancy problem, we explored DoD demand for capabilities related to two military modernization priorities, Unmanned Aerial Vehicles (UAVs) and Artificial Intelligence (AI), in 2020. Exploring text contained in the title and description fields of the 69,933 DOD solicitations issued in 2020, we identified 42 requirements that corresponded to UAVs and/or AI.

Because the military often outlines needs in attachments, we searched attachment data for 2,519 broad agency announcements and sources sought from 2020 using optical character recognition (OCR) and other methods and identified an additional 22 opportunities.

When assessing the hurdles nontraditionals face in trying to prioritize DoD customers, it is also worth considering demand from non-DOD federal customers. In beta.sam, companies discover DoD and non-DoD opportunities concurrently. Thus we aggregated all open federal opportunities, including attachment data, from a single day—October 8th, 2020—and searched for UAV terms across this data set. On that single day, 132 open opportunities corresponded to UAVs.

Assessing redundancy using any one of these methods shows that a company with a high-priority capability can encounter anywhere from dozens to hundreds of potential opportunities. Realistically, it cannot pursue all of them. As a result, DoD customers only receive feedback from a small number of firms, and not necessarily those with the most applicable capabilities. Likewise, companies must decide which customer(s) to engage with using the information they have at hand. They do not necessarily choose the customers whose use cases align best with their capabilities, or the customers that most urgently need their technology. 

To maximize exposure of their requirements, military stakeholders must improve intra-government communication and coordinate their outreach and communication efforts. For priority verticals, the DoD should consider issuing requirements jointly. In addition to helping the DoD gather information from a wider pool of suppliers, this approach would allow companies to market their capabilities to multiple prospective customers simultaneously—a major advantage over the current stove-piped system.

Discoverability

In our 2020 research, we argued that one reason legacy contractors continue to receive most DoD contracts is the general lack of awareness among companies outside the traditional DIB about how to identify and engage with military customers. Thus, we would be remiss not to discuss the design and marketing of beta.sam, where government agencies must publicly post all opportunities above $25,000 (note: the site was relaunched May 24, 2021).^[iii]

The FAR stipulates that open opportunities must be publicly available, but companies like GovWin, Bloomberg Government, and GovShop nevertheless charge firms a subscription fee in exchange for repackaged beta.sam opportunity data. The robust secondary market for this public data suggests that beta.sam is an inadequate resource. Furthermore, this pay-to-play market for solicitation data means that only companies willing and able to pay will receive analyzed requirements data. This could exclude companies unable and or unwilling to pay to play.

Conclusion

Conditions that must be met for the DoD’s marketing and outreach methods to be effective include:

 

• Companies must know where to go to search for DoD opportunities, and the search process must be user-friendly and intuitive.
• Companies need sufficient time to identify, assess, and respond to opportunities.
• DoD customer requirements must be easily understood, and all solicitations should include estimated value and period of performance. 
• The DoD needs to coordinate its marketing and outreach efforts, especially for capabilities in high demand government-wide. 

The absence of these conditions not only fails to meet FAR objectives, but also creates bottlenecks that limit industry participation in market research. The military then operates with an incomplete picture of how its problems could be solved and what capabilities exist in the market. Our recommendations are intended to help the DoD clear these bottlenecks as efficiently as possible, and to make the process of engaging with the military more seamless for nontraditional companies accustomed to commercial practices. The military and civilian agencies need access to the best and brightest suppliers. To attract them, the DoD must become a better customer.

Amanda Bresler

Chief Strategy Officer, PW Communications.

Manages a strategic initiative focused on democratizing the U.S. federal marketplace for innovative solution providers.

In 2019, she was awarded Phase I and Phase II SBIR contracts for Project SHELDON to enable military stakeholders to understand and leverage vetted government suppliers using large-scale, publicly available data sets.

Alex Bresler

Chief Data Officer, PW Communications.

Data-driven technologist, investor, and advisor to world-class high-tech companies.

Specializes in delivering mission-critical data analytics and visualizations to clients in finance, law, real estate, and professional sports.