What the Pathway Actually Requires
The SWP is a separate pathway in the Adaptive Acquisition Framework, not a modification of the Milestone A–B–C sequence used by major capability acquisitions. The AAF's own change log records the pathway's launch in January 2020 and the integration of DoDI 5000.87 in December 2020. DoDI 5000.87 §1.2 states that SWP programs are not subject to JCIDS and will not be treated as major defense acquisition programs even if they exceed the statutory thresholds.
The instruction defines two paths. The applications path covers software running on commercial hardware and cloud platforms; the embedded software path covers upgrades to software in weapon systems and other military-unique hardware, and aligns its testing and delivery with the host system's schedule (DoDI 5000.87 §3.1 and §3.3).
The planning phase is guided by a draft Capability Needs Statement (CNS) from the operational community. To enter execution, "the approved artifacts required to enter the execution phase include the CNS, user agreement (UA), acquisition strategy, test strategy, and cost estimate," and the Decision Authority documents the decision in an acquisition decision memorandum.
During the execution phase, DoDI 5000.87 §3.3 requires that:
- The sponsor and program office maintain a product roadmap, and the product owner and program office maintain prioritized backlogs.
- For applications, the minimum viable capability release (MVCR) is deployed to an operational environment within one year after funds are first obligated, "including appropriate operational test," and subsequent capability releases are delivered at least annually.
- Developmental and operational testing are "integrated, streamlined, and automated to the maximum extent possible to accelerate delivery timelines based on risk strategies."
- Each program develops and tracks metrics on performance, progress, speed, cybersecurity and quality.
- The sponsor and user community perform a value assessment at least annually, informed by test and evaluation results, which the Decision Authority, sponsor and PM use to update strategies and resourcing.
The instruction also tells programs to use modern iterative practices that, among other things, "create effective modular open systems approaches to support future capabilities." That is an architectural expectation, not a delivery cadence.
Two artifacts are easy to mischaracterize. The AAF glossary defines the User Agreement as "a commitment between the sponsor and PM for continuous user involvement and assigned decision making authority in the development and delivery of software capability releases." DoDI 5000.87 §3.2 says the decisions it assigns include "defining and prioritizing required capabilities, tradeoffs of software features and cadence, user acceptances, and readiness for operational deployment." The UA says who decides; it is not a performance specification. The needs themselves live in the CNS, which the glossary describes as "a high-level capture of mission deficiencies, or enhancements to existing operational capabilities, features, interoperability needs, legacy interfaces and other attributes."
Six Years of Oversight: What GAO Has Reported
GAO has examined the pathway's implementation more than once. In GAO-21-105298 (September 30, 2021), it summarized its June 2021 finding that DoD had started implementing the pathway but "did not have a data collection strategy for it," and recommended that DoD automate data collection; DoD concurred. The same report noted that as of June 2021 "only six of 36 weapon programs that reported using Agile also reported delivering software to users in less than 3 months."
In GAO-23-105867 (July 20, 2023), GAO found that "DOD developed policies and guidance for oversight of programs on the software pathway, including using metrics and value assessments that measure Agile development outcomes," but had not issued corresponding guidance for weapon programs using Agile on other pathways, and had no plan or identified resources for adopting modern engineering tools across programs. GAO made three recommendations; DoD partially concurred.
Neither report is about AI, and neither found AI-specific governance failures. What they establish is that the pathway's oversight rests on metrics, value assessments and data collection, and that GAO has repeatedly looked at whether those mechanisms exist and work. For an AI program, those are exactly the places where model governance has to live, because they are the instruments the Decision Authority actually uses.
Where Machine Learning Breaks the Assumptions
DoDI 5000.87 does not address machine-learning models specifically. NIST's AI Risk Management Framework (AI RMF 1.0, January 2023) explains why that matters: "AI systems, for example, may be trained on data that can change over time, sometimes significantly and unexpectedly, affecting system functionality and trustworthiness in ways that are hard to understand." Its Appendix B lists risks that set AI apart from traditional software, including that AI systems "may require more frequent maintenance and triggers for conducting corrective maintenance due to data, model, or concept drift," and "underdeveloped software testing standards and inability to document AI-based practices to the standard expected of traditionally engineered software for all but the simplest of cases."
Read against the SWP's artifacts, that produces four gaps. These are our analysis, not findings from an audit:
- Model versioning and traceability. The product roadmap and backlog track capabilities and features. Neither requires a record that links each fielded model version to its training data, evaluation results and approved operating envelope. Without that record, a value assessment can judge whether the software was worth the investment without knowing which model produced the outcomes it is judging.
- Evaluation currency. Functional testing confirms that a function returns the right output for a given input. Model evaluation measures performance across a distribution of inputs that can shift after fielding. A model that passed evaluation at one release may perform differently a year later if the operating environment has changed and the model has not been re-evaluated.
- Drift detection. The test-strategy provisions of DoDI 5000.87 §3.2 say "continuous runtime monitoring of operational software will provide health-related reporting (e.g., performance, security, anomalies), as well as additional data collection opportunities to support test and continuous operational test." That is a natural hook for drift monitoring, but nothing in the instruction requires model-performance baselines, so a program can satisfy the language with uptime and error rates alone.
- Data governance. The pathway's strategies address intellectual property, cybersecurity and product support. None of the required artifacts is a training-data provenance and access plan. Programs that leave data management to the development team lack the documentation needed to explain model behavior to oversight bodies or recover from a data-quality incident.
Putting AI Governance Into the Artifacts That Exist
The SWP gives programs room to tailor, and the planning-phase guidance says a Decision Authority may use the acquisition decision memorandum to "capture unique direction for the program to address high risk areas" (AAF planning phase). Our recommendation is to use that room deliberately, mapping each AI control to an artifact that already has an owner and a review cycle:
Evaluation thresholds in the CNS performance attributes and the test strategy. The CNS is where needs and attributes are stated, and DoDI 5000.87 says the test strategy "defines the streamlined processes by which capabilities, features, user stories, use cases, etc., will be tested and evaluated." Writing an accuracy floor, a false-positive ceiling and operating-condition bounds into those two documents gives the program an objective test of whether a model update is a material change. The UA then names who on the user side accepts or rejects a model release, which fits its defined role of assigning decision-making authority over user acceptance and readiness for deployment.
A model register maintained alongside the product roadmap. The roadmap is the planning artifact for regular deliveries. A register that records each fielded model version, its training-data version, evaluation results, approved envelope and known limitations, kept with the roadmap and updated at each release, supplies the traceability the roadmap does not.
Model-performance metrics in the program metrics plan. Since every SWP program already has to develop and track metrics, adding model-level measures on representative recent operational data (performance against baseline, input-distribution shift, override or rejection rates by operators) turns runtime monitoring into drift detection.
AI-specific criteria in the value assessment. The value assessment is the sponsor's at-least-annual judgment of whether delivered capability was worth the investment, and GAO-23-105867 cites value assessments, with metrics, as part of the pathway's oversight guidance. Asking whether the model still meets its thresholds on current data makes that judgment honest for an AI capability.
A named owner for AI assurance. Someone in the program office needs the authority and expertise to review training-data provenance, evaluation methodology and scope limitations. If that work sits informally with a developer, documentation quality will vary and nobody will be accountable when an oversight body asks why the model changed.
The Oversight Gap, Illustrated
DoDI 5000.87 requires delivery at least annually and encourages "more frequent updates and deliveries" where practical, with testing integrated and automated "based on risk strategies" rather than a fixed operational-test gate before every release. For deterministic software, an automated regression suite is a reasonable proxy for that gate. For a model, a release can pass every functional test and still change behavior.
A hypothetical example shows the risk. A logistics AI program on the applications path ships model updates monthly. Its CNS calls for prioritizing supply requests by urgency and availability. One update shifts how the model ranks a handful of rare supply categories. The change is within the stated need and passes functional tests. Weeks later a shortage is traced to the new ranking, and the program has no record of what changed in the model, which data it was trained on, or who accepted it. Nothing in that sequence violates DoDI 5000.87. It is the gap the controls above are meant to close.
The Alternative and When It Makes Sense
The approach above keeps the SWP intact and adds AI controls inside its existing artifacts. It needs no new milestone reviews or statutory authority, only the formality programs already give their test strategy and metrics.
The credible alternative is a hybrid acquisition. The AAF FAQs on the 2025 directive note that "hybrid acquisitions combining the use of SWP with other pathways can be used to de-couple hardware and software acquisition, as applicable and appropriate." That makes sense when an AI capability is tied to new sensors or other hardware that the SWP is not meant to develop. It does not solve model governance by itself; the software portion still needs the controls above.
The tradeoff runs both ways. Adding thresholds and a register costs time in planning and creates a trigger that will sometimes slow a release. Skipping them keeps velocity but leaves the Decision Authority judging value without knowing what changed. Evidence that would change this recommendation includes future DoD guidance that sets AI-specific SWP requirements, or program data showing that standard SWP metrics catch model regressions on their own.
Questions to Settle Before the Execution Decision
- Do your CNS performance attributes and test strategy state evaluation thresholds for the AI component, or only functional requirements? If only functional, a model update has no objective trigger for added scrutiny.
- Does your UA name who on the user side accepts a model release, and on what evidence? The UA assigns user-acceptance authority; make sure it covers model changes explicitly.
- Can you trace any fielded model version to its training data and evaluation results? If the product roadmap is your only record, you cannot reconstruct why an operational outcome occurred.
- Does your metrics plan include model-performance measures on current operational data? If it tracks only uptime and defects, drift will surface as a mission problem first.
- What makes a model change material, and where is that written? If it is not in a program document, the call will be made informally and inconsistently.
- Who owns AI assurance in the program office? If no one is named, there is no single point of accountability when oversight bodies ask questions.
Settle these with the first draft CNS and UA, not before the first value assessment.
Spartan X builds AI governance frameworks covering data provenance and explainability alongside program management support, and that pairing fits this problem: tying model evaluation, data lineage and release acceptance to the CNS, test strategy, metrics and value assessments that SWP decision authorities already review.
Sources and further reading
- DoD Instruction 5000.87, Operation of the Software Acquisition Pathway (October 2, 2020)
- Adaptive Acquisition Framework: Software Acquisition Pathway, Planning Phase
- Adaptive Acquisition Framework: Software Acquisition Pathway, Execution Phase
- Adaptive Acquisition Framework: Software Acquisition Pathway Glossary
- Adaptive Acquisition Framework: Software Acquisition Pathway FAQs, including the SW Lethality Directive FAQs
- Adaptive Acquisition Framework: Software in NDAAs, including FY2020 NDAA Section 800
- Secretary of Defense, Directing Modern Software Acquisition to Maximize Lethality (March 6, 2025)
- GAO-21-105298, DOD Software Acquisition: Status of and Challenges Related to Reform Efforts (September 30, 2021)
- GAO-23-105867, Defense Software Acquisitions: Changes to Requirements, Oversight, and Tools Needed for Weapon Programs (July 20, 2023)
- NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (January 2023)
