“Favor Scope Changes When Possible.” —Mike Cohn (2010)
Times have changed. When I first saw this statement in Mike Cohn’s book1 about Agile software development, I was at least puzzled, if not a little shocked. I had seen instances when this philosophy got companies and government agencies in trouble. Constructive change claims, for example, sometimes arise out of informal, ill-advised scope tradeoff decisions and other-than-stellar contract management practices.
So, what is the implication for public procurement professionals and contract managers?
As the use of technology in public-sector management grows exponentially, the practices in IT procurement are evolving. Three major associations of state and local government procurement and IT professionals publish best practice standards, research briefs, and whitepapers about the changes.2
Agile delivery methods are challenging the historical “waterfall” method of IT procurement—i.e., sequential project delivery with lengthy planning, complex contracts with a single supplier, and often years of implementation. Also, the federal government’s promotion of modular procurement—i.e., breaking large system acquisitions into smaller pieces—creates opportunities for more suppliers, but adds its own dimension of risk.
State and local government procurement also is facing the evolution of IT delivery approaches. The National Institute of Government Purchasing (NIGP) has undertaken a revision of its Legal Aspects of Public Procurement, the textbook used to teach public procurement professionals nationally. The 2010 edition included a chapter on software licensing. The new edition will look forward toward the challenges that procurement professionals will face concerning these new methods of providing IT solutions.
Over the past few years, troubled IT projects have been in the news. A 2016 Indiana Supreme Court decision3 illustrated the litany of legal issues in a large system implementation to replace face-to-face applications for supplemental food programs and Medicaid benefits. In 2013, on the heels of two failed IT projects worth over $500 million, the California State Controller published a comprehensive task force report4 that identified issues in large IT system implementations. The task force found, with the time between planning and completion of implementation often measured in years, the clear majority of IT implementations face changing requirements. Indiana and California are not alone: It’s been widely reported that 80–90% of large IT implementations are not achieving expectations, with one-third or more being outright failures.
Research has found that requirements in large IT projects change at a rate of about 3% per month.5 So, over the course of a one-year implementation, scope changes become sizable. These changes are in addition to those that arise because an agency does not fully or adequately describe its own business processes or the environment in which the implementation will occur.
As previously mentioned, a companion development has been a move toward modularity. Modular procurement breaks down the size of implementations, increasing opportunities for small companies, making contracts shorter, and reducing risk by not relying on single-solution, “all-or-nothing” contracts and implementations. The Office of Management and Budget has promoted modular procurement,6 and Centers for Medicare and Medicaid Services (CMS) has passed rules that encourage modular programming in state Medicaid management information systems.7 The CMS policy also promotes procurement streamlining and identifies some alternative, mature project delivery methodologies like Agile.
But—with this changing landscape comes challenges. As this article dives more deeply into Agile, ask yourself, “What are the implications for contract management?”
Agile was born as an iterative software development method used in projects having significant uncertainty about requirements and solutions. Agile includes planning, execution, testing, and delivery of usable code within rapid, successive iterations, phases, or cycles as short as two to six weeks.8
While traditional project delivery is still used when requirements are reasonably certain, Agile is promoted where there is a likelihood of significant change. Agile project delivery reflects the fluidity of requirements development and places schedule at the forefront. The “Agile Manifesto” frames the philosophy behind Agile development.9 Among the central guiding principles in Agile are:
Among the various Agile methods, “Scrum” may be the most popular. Under this method, threshold challenges for contract management include the varying personnel roles and different glossary compared to traditional project management. There is a product owner having overall responsibility for the product and a backlog of user stories that frame requirements and need to be satisfied (i.e., developed) during sprints. The customer is expected to be integrally involved with the development, even attending periodic (often weekly) scrum meetings to assess progress. A Scrum Master is part of the team and satisfies the coaching and other team leader roles. There usually is a project manager for both parties to the contract. For procurement officials, it is especially important to clarify who speaks for the supplier from a contractual perspective, as many of these roles are defined in the context of software development, not contract administration.
Decisions about features and other scope tradeoffs happen quickly. So, the government customer must “clothe” contract administrators with adequate authority and ensure that contractually defined approaches exist for elevating decisions efficiently to approval authorities where needed.
While many contracting issues remain the same—liability allocation terms, for example—pricing may be particularly challenging. The whole premise for Agile development is that requirements in most respects are unknown and are clarified during project delivery. Consequently, estimating is more of an art than a science, and the concept of “fixed price” is more fluid.
This method of contracting requires an adjustment in historical ways of viewing contracts. The remainder of this article forecasts some contract management challenges inherent in Agile.
Selecting the correct vendor is paramount. California is procuring its Child Welfare Information System using Agile. The state collaborated with the U.S. General Services Administration’s 18F—an office well versed in Agile methods—to help with request for proposals (RFP) strategy and development.10 The state first established a prequalified vendor pool. Companies on the state’s multiple award schedules were eligible to be prequalified.
A prototype repository of supplier documents was available for state review. The documents included source code and associated documentation. The depository documents described the collaborative approach to development using Agile processes in the U.S. Digital Services Playbook.11 The process included a periodic refresh of the qualified vendor pool.
After vendor qualification and selection, departments could develop and release RFPs to the pre-qualified vendors. The offers are firm for not less than 90 days. An updated responsibility determination is performed by the state agency and a contract executed.
This method of vendor selection was different from the traditional RFP process. Both technical and legal expertise were needed to adapt the process to the statutes and regulations governing procurement.
At the core of the Agile approach is an acknowledgement of the role of trust, sometimes including a written agreement about cooperation. Some contractual terms and conditions addressing trust read much like team norms. These contract terms can sound like precatory language—e.g., “we agree to work together and build trust in one another and each other’s expertise.” They are intended to express the importance of the customer being intimately involved in setting priorities, agreeing on estimates, and signing needed change addenda.
Customers are expected to be available for “sprint” planning and weekly progress meetings that assess progress. The intimate involvement of the customer addresses one of the problems with large sequential development projects identified by the California IT Procurement Task Force.
However, it has been noted that there is a likely cultural impediment among attorneys in these kinds of contract provisions.12 If the relationship breaks down, it is likely that a party’s satisfaction of the “trust” obligations would be legally measured only by the obligation of good faith and fair dealing if specific cooperation obligations are not spelled out.
Agile contracts often use a hybrid contract type, not a firm-fixed-price contract in the classic sense. The contract types are closer to time-and-materials (T&M) contracts with a not-to-exceed (NTE) price where scope is not clearly defined. One issue that often occurs with a contract NTE price is its interpretation—i.e., whether it is only a control mechanism on spending or the parties expect a satisfactory result below that amount (a ceiling price). Agile uses a framework for achieving greater certainty about and control over the price. This is sometimes referred to as an “indicative fixed price” contract.13 The indicative fixed price is adjustable based on the unanticipated complexity of the project.
Further, in any agreement, it is important to relate how costs and complexity should be estimated. Generally, “user stories” are estimated based on their complexity—not in traditional labor hours, but in “points.” Points estimates are based on the IT professionals’ (i.e., scrum team’s) experience with like features and programming. More points are given for higher complexity, with points estimated based on agreed criteria and using other “sprints” and “user stories” as analogies. At some point, of course, points need to be turned into cost figures that are useful for budgeting, and this is a central discussion that should occur up-front before contract execution.
The provisional or indicative price is established at the time of award based on the information, such as user stories, that are disclosed in the solicitation. Workshops can be held during the solicitation process to gain a fuller understanding of requirements before award. After award, some learning occurs during early sprints, and a fixed-price range is agreed to.
Under this model,14 a risk-share percentage is negotiated for the project. If the project cost exceeds the upper limit of the fixed-price range, the customer and contractor agree to split costs (50/50 or another agreed risk share). Moreover, the model uses a shared savings ratio as an incentive. If costs are less than the lower limit of the agreed fixed-price range, the contractor can receive an agreed incentive share. The Agile contract can permit price adjustments (to the indicative price or fixed-price range) when unexpected complexity is encountered or requirements change.
T&M contracts may be acceptable for some work, as when a bounded requirement is not suitable for the Agile framework, or the agency can adequately steer the T&M effort to achieve a successful result. However, the agency bears the cost risk in T&M contracting. Early Agile sprints used before an early exit checkpoint can be used to gain learning before establishing the fixed-price range. This approach to pricing will require different mindsets during contract management.
Contract scope is central to Agile development, like any other IT implementation. The objective of a scrum sprint is to deliver functional software at each iteration. However, because the specifications are not developed in the traditional sense until the sprints are conducted to satisfy user stories, there may be unanticipated complexity. The Mike Cohn quote that opened this article expresses the preferred way to address change: adjust the scope rather than schedule.
Agile practitioners maintain that they welcome changes because they are consistent with the underlying philosophy of Agile. A “change for free” clause is used in Agile contracts to permit new features to be added if others of equal scope are removed. Some “change for free” clauses state that if the parties cannot agree on work item estimates, the work reverts to T&M billing. Some caution is needed here, however, because remedy clauses triggering T&M pricing may end up not actually being “change for free.”
While a “change for free” clause may be consistent with the flexibility of Agile, documentation should track the tradeoffs that are being made and any adjustments to price. Documenting these changes is particularly challenging with traditional contract modification practices in many public agencies.
Essentially, in Agile development, the documentation strategy is “just enough.” A contractual means of identifying and prescribing the use of these documentation standards is important, both for product documentation and handoff documentation if different suppliers will be involved. Suppliers need to see the code they have to integrate with.
One advocated approach15 is a post-development documentation workshop to identify documentation that is required. In modular development, documentation approaches and standards would have to be coordinated with other affected suppliers. Obviously, IT professionals and the business client must steer project documentation requirements.
A recurring challenge in IT implementations is knowing when performance is complete and ready for acceptance. The acceptance approach in the Agile methodology known as “Scrum” is built around a “definition of ‘done’” in the contract that includes code quality (i.e., completion of testing) and functionality as components.
A fundamental assumption in Agile is that usable, quality code is delivered at the end of each sprint. In large system integrations, however, there must be some ability to ensure that independently developed software works in an integrated way. Some thought needs to be given to ensuring that the project modules are integrated before final acceptance.
Agile contracts contemplate periodic or partial payments. Agile practitioners acknowledge that “done” software that is useful within the principles of Agile development is not necessarily deployable at that time. So, contracts may use payment retention until final acceptance. Payment milestones would have to be defined contractually and tied to one or a combination of sprints.
Incremental delivery also adds complexity to warranties. When do they begin? Waiting until the end for the warranty to begin may make sense in a fully integrated solution. Where individual components are being used by the public entity after sprint deliveries, however, there may be a need to begin warranty periods at different times.
One can see the need for continuous communication in this type of delivery approach. Obviously, there can still be disagreements about, for example, scope, price/schedule adjustments associated with changes, or acceptance of software. Agile contracts may use a scope-steering group above the customer and contractor project teams, as well as an executive steering committee as a second level. The steering group is used for escalation of issues that cannot be resolved at the project level. The escalation process advocated by some Agile experts involves eventual use of a third party “expert” serving not as an arbitrator, but more as a third-party mediator to provide expertise and opinions to the steering groups/committees regarding an issue not resolved at the steering group/governance body level. Some clarity is required about the role of the expert and whether there are other remedies in the event of a dispute.
This escalation approach addresses an issue identified in the California IT Procurement Task Force report. Formal disputes clauses may not foster the more informal communication that is needed in projects of this nature. Issue elevation ideally mirrors the transparency in the way that the supplier and public entity project teams communicate. The contract provisions in Agile may look more like a relationship than a traditional contract disputes provision.
Agile contracting uses a series of exit strategies. Agile contracts are typically not multi-year, committed relationships. There is usually a “checkpoint” milestone at which a customer (or even contractor) can elect to terminate. However, a minimum number of short sprints can be identified as the committed period, before which the customer cannot terminate. This provides a period of mutual learning before decisions to terminate are made. After the checkpoint, exit points may be defined (with negotiated payment provisions) to permit the customer or contractor to terminate the relationship if either is not realizing the expected value. Typical termination for convenience clauses probably are inadequate to define the parties’ relative rights at early termination.
As can be seen, the Agile method of project delivery has potential to address fundamental shortcomings of the historical “waterfall” approach to large system implementations, but to adapt the model to public procurement, some innovative and nontraditional contract management practices will have to be used. Perhaps most important, government customers must understand and commit to a level of engagement not seen before.
Agile is a new and exciting frontier! NIGP has realized that once one gets beyond purchasing off-the-shelf software, it becomes even more important that procurement professionals collaborate with IT professionals, business process owners, and legal counsel. The recent evolution of modular and Agile project delivery is illuminating different challenges in contract management at speeds never before encountered. CM
Richard Pennington, JD, LLM, CPPO, C.P.M.
1 Mike Cohn, Succeeding with Agile: Software Development Using Scrum (Addison-Wesley Professional, 2010).
2 See, e.g., National Institute of Governmental Purchasing (NIGP), Information Technology (IT) Procurement Series—Nos. 1 and 2 (2018), available at www.nigp.org/home/find-procurement-resources/guidance/global-best-practices; National Association of State Procurement Officials (NASPO), Modular Procurement: A Primer (2018), available at www.naspo.org/Publications/ArtMID/8806/ArticleID/4597; and National Association of State Chief Information Officers (NASCIO), Agile IT Delivery: Imperatives for Government Success (2017), available at www.nascio.org/Publications/ArtMID/485/ArticleID/556/Agile-IT-Delivery-Imperatives-for-Government-Success.
3 State of Indiana v. International Business Machines, 51 N.E.3d 150 (Ind. 2016).
4 California Task Force on Reengineering IT Procurement, “Recommendations to Improve Large Information Technology Procurements: A Road Map for Success in California” (August 2013), available at www.sco.ca.gov/Files-EO/0813_IT_Task_Force_Recommendations.pdf.
5 A. Opelt, B. Gloger, W. Pfari, and R. Mittermayr; Agile Contracts: Creating and Management Successful Projects with Scrum (John Wiley & Sons, 2013).
6 Office of Management and Budget, “Contracting Guidance to Support Modular Development” (June 14, 2012).
7 CMS, U.S. Department of Health and Human Services, “Enhanced Funding Requirements: Seven Conditions and Standards” (April 2011).
8 Agile is on the list of NASCIO’s “State CIO Top Ten Priorities for 2018.”
9 “Manifesto for Agile Software Development” (2001), available at http://agilemanifesto.org/.
10 V. Zvenyack and A. Franciso, “From 1,500 Pages to 10: Helping California Buy a New Child Welfare System” (March 22, 2016), available at https://18f.gsa.gov/2016/03/22/helping-california-buy-a-new-child-welfare-system/.
11 Office of Management and Budget, U.S. Digital Service, Digital Services Playbook, available at https://playbook.cio.gov/.
12 See T. Arbogas, C. Larman, and B. Vodde; Agile Contracts Primer (from chapter 14 of C. Larman and B. Vodde, Practices for Scaling Lean & Agile Development (Addison Wesley Professional, 2010)), available at www.agilecontracts.com; see also Opelt, et al., note 5.
13 As coined within Opelt, et al., see note 5.
14 I.e., the one described by Opelt, et al. (see note 5).
15 I.e., one advocated by Opelt, et al. (see note 5).