GAO’s 2018 weapon system tests showed a control list is not evidence of security
GAO found testers taking control of weapon systems with basic tools while program offices cited their security controls. The gap had four parts: controls defeated in use, alerts nobody read, findings left open and tests narrower than the threat.
Looking Back revisits past weeks one at a time, up to the most recent, and asks what it meant for the banks, health systems and public agencies we serve. The series was written in 2026, with the benefit of hindsight, and each article says which week it looks back on.
Two-minute summaryThe dek and the first sentence of each section, quoted from the article
GAO found testers taking control of weapon systems with basic tools while program offices cited their security controls. The gap had four parts: controls defeated in use, alerts nobody read, findings left open and tests narrower than the threat.
- The week On Tuesday, October 9, 2018, the Government Accountability Office published and publicly released a report to the Senate Armed Services Committee titled “Weapon Systems Cybersecurity: DOD Just Beginning to Grapple with Scale of Vulnerabilities” (GAO-19-128).
- Controls that existed and were defeated GAO was direct on this point: test reports made it clear that simply having cybersecurity controls does not mean a system is secure, and how the controls are implemented can significantly affect security.
- Detection that nobody acted on The second failure was in detection and response.
- Findings that stayed open The third failure was remediation.
- Tests narrower than the threat The fourth failure was in the evidence itself.
- Why the confidence persisted The four failures explain how a control list can mislead.
- The four failures in a bank, a hospital and an agency The report’s findings come from weapons programs, but each of the four failures has a recognizable form in the institutions this series writes for.
- Objections: realism, metrics and relevance The first objection, the program officials’ own, was realism.
- Turning test results into evidence a board can use The four failures suggest a different way to report security to leadership: by evidence type, not by control count.
- With hindsight GAO returned to the subject in March 2021 with “Weapon Systems Cybersecurity: Guidance Would Help DOD Programs Better Communicate Requirements to Contractors” (GAO-21-179), published March 4, 2021.
The week
On Tuesday, October 9, 2018, the Government Accountability Office published and publicly released a report to the Senate Armed Services Committee titled “Weapon Systems Cybersecurity: DOD Just Beginning to Grapple with Scale of Vulnerabilities” (GAO-19-128). It was GAO’s first report specific to cybersecurity in weapon systems acquisitions. GAO had been asked to review the state of Defense Department weapon systems cybersecurity, and the report noted that the department planned to spend about $1.66 trillion to develop its current portfolio of major weapon systems.
GAO analyzed weapon systems cybersecurity test reports, policies and guidance. It interviewed officials from defense organizations with weapon systems cybersecurity responsibilities and program officials from a nongeneralizable sample of nine major defense acquisition program offices. The department conducted a security review and cleared the report for public release; GAO said it would provide a classified briefing to Congress. To avoid identifying specific systems, GAO illustrated some concepts with fictitious depictions and was deliberately vague about some details.
GAO said multiple factors contributed to the state of weapon systems cybersecurity, including the increasingly computerized and networked nature of the systems, the department’s past failure to prioritize their cybersecurity and its nascent understanding of how best to develop more secure systems. It said GAO and others had warned of cyber risks for decades, yet until recently the department did not prioritize cybersecurity in weapon systems acquisitions, in part because it had historically focused on the cybersecurity of its networks rather than of the weapon systems themselves.
The central finding was stark. In operational testing, GAO said, the department routinely found mission-critical cyber vulnerabilities in systems under development, yet program officials GAO met with believed their systems were secure and discounted some test results as unrealistic. Using relatively simple tools and techniques, testers were able to take control of systems and largely operate undetected, due in part to basic issues such as poor password management and unencrypted communications. GAO said the vulnerabilities the department knew about likely represented a fraction of the total, because not all programs had been tested and tests did not reflect the full range of threats.
The report gave examples. In one case, a two-person test team took one hour to gain initial access to a weapon system and one day to gain full control. One test team guessed an administrator password in nine seconds. Several systems used commercial or open source software without changing the default password, which testers looked up on the internet. One team caused a pop-up message to appear on users’ terminals telling them to insert two quarters to continue. Another downloaded 100 gigabytes of data. In some cases, simply scanning a system caused parts of it to shut down, and one test had to be stopped for safety reasons.
GAO said the department had taken steps, issuing or updating at least 15 department-wide policies, guidance documents and memorandums since 2014, and that it and Congress had begun initiatives to understand and address vulnerabilities. But it identified barriers, including difficulty hiring and retaining cybersecurity staff and difficulty sharing information about vulnerabilities and threats across programs. GAO noted that federal information security had been on its high-risk list since 1997. It made no recommendations in the report and said it would continue evaluating the issue.
GAO described most weapon systems as cyber-physical systems such as aircraft, missiles and ships, and it deliberately obscured the details of the ones tested. The findings were not about weapons. They were about the distance between what an organization believes about its security and what a test shows, and about why that distance survives even when the test results are on file. The same distance can open in any organization that relies on a list of controls as evidence that it is secure.
GAO reported that program officials cited the security controls they had applied as the basis for believing their systems were secure. Officials from one defense agency expressed confidence but could not point to test results; they offered a list of controls instead. Officials told GAO that controls are necessary but not sufficient, and that “penetration test results—rather than compliance documentation—are better indicators of a system’s security.”
The report is also unusual as evidence. Where a compliance review asks whether policies exist and are followed, this report described what happened when people tried to break the systems, drawn from the department’s own test reports. That gave it a kind of authority that a compliance review lacks, and it is why its findings are worth applying outside the setting it examined.
Our argument is that the report describes four distinct ways a control list fails as evidence, and that each needs its own fix. We take them in order:
- Controls that existed and were defeated: the control was on the list, but its implementation let the testers through.
- Detection that nobody acted on: the system recorded the testers, but no person looked or responded.
- Findings that stayed open: earlier tests found the problem, and the program did not fix it.
- Tests narrower than the threat: the testing itself was limited in time, scope and method, so even a clean result would have proved less than it seemed.
The four are distinct. The first is about implementation, the second about operations, the third about remediation and the fourth about the evidence itself. A program could solve any three and still fail on the fourth. After the four parts we take the objections, including the program officials’ own, and then set out how a leader can turn test results into evidence a board or an oversight body can use. The report’s subjects were defense programs, but the four failures map closely onto the systems banks, health systems and civilian agencies run, and that is where we apply them.
Controls that existed and were defeated
GAO was direct on this point: test reports made it clear that simply having cybersecurity controls does not mean a system is secure, and how the controls are implemented can significantly affect security. Its examples are a catalog of controls that would appear on any compliance checklist.
Authentication was present. But one test team guessed an administrator password in nine seconds, and several systems that used commercial or open source software had not changed the default password at installation, so testers found the password online and gained administrator privileges. Multiple test teams used free, publicly available information or software downloaded from the internet to avoid or defeat security controls.
Access control was present. One system had implemented role-based access control, but its internal communications were unencrypted. A regular user could read an administrator’s username and password as they crossed the system and use them to gain greater access. The control on the list was real. The design around it made it irrelevant.
Basic resilience was missing in ways that no control list would show. In some cases, simply scanning a system caused parts of it to shut down. One test had to be stopped because of safety concerns after the team scanned the system. GAO noted that scanning is a basic technique that requires little knowledge or expertise.
Isolation was assumed rather than shown. GAO, citing the department’s test and evaluation office, said any exchange of information is a potential access point for an adversary, and that even air-gapped systems could potentially be reached through USB devices and compact discs. A control list that records a system as isolated says nothing about the maintenance laptop, the update media or the vendor’s remote session that crosses the gap. Isolation is a claim that has to be tested like any other, by tracing every way data reaches the system.
Implementation is the control
The point GAO made is that the evidence of a control is its behavior under test, not its presence on a list. For a bank, a health system or an agency, the same distinction separates a policy that requires changing default passwords from an inventory that shows every installed product has had its default credentials changed. It separates a statement that access is role-based from a test showing that credentials cannot be read in transit between components.
The practical consequence is a different kind of control record. Next to each control, the record carries the date and method of the last test that exercised it and the result. A control without such an entry is a claim. A control with a failing entry is a known gap. Only a control with a recent passing test is evidence, and only for what the test covered.
Default credentials deserve particular attention because they recur across sectors. GAO illustrated the risk with the 2016 attack on Dyn, a company it described as a key intermediary in directing internet traffic, in which malware searched the internet for devices using factory-default usernames and passwords and used them to flood targets with traffic. Hospitals with connected clinical and building systems, banks with appliances in branches and data centers, and agencies with operational technology may all run commercial products that ship with default credentials. The control that matters is the inventory that proves each one was changed.
Configuration baselines as evidence
The unencrypted internal traffic example points to a second kind of implementation evidence: the configuration baseline. A control described as role-based access says nothing about whether credentials travel in clear text between components. A documented baseline for each system, covering how components authenticate to each other, whether internal traffic is encrypted, which services are exposed and which accounts hold administrator rights, can be compared against the running system. Differences between the baseline and the system are findings before any tester arrives.
Baselines also make tests more useful. A test team that knows the intended configuration can report where the system departs from it, as well as where it can be broken. The program officials GAO interviewed objected that testers asked for design information. A baseline is that design information, and handing it over lets a short test go further, which matters when the test has 41 hours.
Detection that nobody acted on
The second failure was in detection and response. GAO reported that test team activities were not detected at all during some assessments, including one case in which a team operated for several weeks without being detected. One test report said the team was not discovered even though it was deliberately “noisy” and was not trying to hide.
In other cases, detection tools worked but people did not act. Intrusion detection systems correctly flagged test activity as suspicious, but users were unaware of the detection. In one test, the intrusion detection system identified the team’s activity but did not improve operators’ awareness because it was always “red”: warnings were so common that operators were desensitized to them. Multiple test reports said team activity was recorded in system logs that operators did not review. One system had no documented procedures for reviewing logs.
Response failed even when operators knew. One test team emulated a denial of service attack by rebooting the system. Operators reported that they did not suspect a cyberattack because unexplained crashes were normal for that system. In another test, operators identified intrusion attempts and took steps to block the team, which easily circumvented them. In another, operators needed outside help to restore a compromised system.
Alerts are a staffing commitment
Each of those findings describes a control that would pass a documentation review. The system had intrusion detection. It produced logs. It had operators. What it did not have was a person whose job was to read the alerts and logs, a threshold that made alerts meaningful, a procedure for what to do next and practice in doing it.
The always-red console is the most transferable example. A detection tool that alarms constantly trains its operators to ignore it. The fix is tuning, which takes time from skilled staff, and a measure of whether alerts lead to action. Banks, hospitals and agencies that run security operations centers or outsource monitoring face the same question: of the alerts generated last month, how many were reviewed by a person, how many led to an action, and how long that took.
The crash that nobody suspected has a clinical parallel. A device or system that fails often enough for failures to seem normal hides an attack in plain sight. The organization’s incident process needs a way to ask, for repeated unexplained failures, whether a cyber cause has been ruled out, and someone with the knowledge to rule it out.
The finding that one system had no documented procedure for reviewing logs is the simplest to check elsewhere. For each critical system, an organization can ask who reviews its logs, how often, against what criteria and where the review is recorded. Collecting logs may satisfy a requirement on paper. Reviewing them is what detects anything. GAO’s finding that test activity sat in logs nobody read is a reminder that the two are different controls with different costs.
Findings that stayed open
The third failure was remediation. GAO said program offices were aware of some of the vulnerabilities that test teams exploited because earlier assessments had found them. One test report said only 1 of 20 cyber vulnerabilities identified in a previous assessment had been corrected, and the test team exploited the same vulnerabilities to gain control. Asked why, program officials said they had identified a solution, but for some reason it had not been implemented, and they attributed it to contractor error. Another test report said the team exploited 10 vulnerabilities identified in previous assessments.
The report’s barriers section explains part of why findings stayed open. Programs were not sharing vulnerability and threat information across programs, so flawed designs could be repeated in new systems by designers who did not know the flaws had caused vulnerabilities elsewhere. Officials from one heavily connected system said it was only as secure as its weakest link, but they did not have information on the vulnerabilities of the systems it connected to because of classification. GAO said that because programs were not sharing this information, they were unaware of their full risk exposure.
Remediation needs an owner outside the test
A test produces findings. Whether they get fixed depends on what happens after the test team leaves. In the GAO examples, a solution had been identified and not implemented, and the blame went to a contractor. That pattern points to three gaps: no single owner tracked the finding to closure, the contract may not have required the fix, and nobody retested to confirm.
For banks, health systems and agencies, the equivalent check is simple to state and harder to pass. Take the findings from the last two penetration tests or assessments. For each, name the owner, the planned fix, the date it was confirmed fixed and how it was confirmed. Findings that were accepted as risks should show who accepted them and why. Any finding that the next test found again is evidence that the remediation process, not the original control, is the problem.
The contractor attribution matters too. When a supplier builds or runs a system, the supplier implements the fix. If the contract does not require fixes to test findings within a set time, the organization depends on goodwill. GAO did not examine contracting in the 2018 report; it said so. Its 2021 follow-up did, as the hindsight section describes.
Retesting closes the loop. A fix that has not been tested is another claim on the control list. The cheapest point to confirm it is soon after the fix, by the people who found the problem or by an automated check that looks for the same weakness. Waiting for the next scheduled test means the organization learns late, as GAO’s example showed, that only 1 of 20 findings had been corrected. A short retest window written into every engagement, and a rule that a finding closes only when the retest passes, would have changed that example’s outcome on paper and, more to the point, in the system.
Risk acceptance belongs in the same record. Not every finding should be fixed; some cost more to fix than the risk they carry. But an accepted risk is a decision, and it needs a decision-maker with the authority to accept it, a reason and a date to revisit it. A finding that is neither fixed nor formally accepted is the case GAO described: known, unaddressed and exploited again by the next test team.
Tests narrower than the threat
The fourth failure was in the evidence itself. GAO said the department did not know the full extent of its weapon systems vulnerabilities because of limitations in the tests. Test teams reported that they portrayed realistic threats, but the nature of testing imposed limits that adversaries do not face. Department officials said cybersecurity assessments were generally conducted over a few days to a few weeks. One assessment was cut short by external factors, leaving the team 41 hours with the system. Officials said a determined adversary could spend months or years.
Because of the time limits, test teams looked for the easiest or most effective way in, and did not identify all vulnerabilities an adversary could exploit. The test reports GAO reviewed tended to portray nascent to moderate threats and generally did not target special components such as industrial control systems and non-internet devices. Counterfeit parts were outside the scope of the tests reviewed.
Scope limits were also imposed by the systems’ owners and their circumstances. In at least one case, portions of a system’s networks and data were proprietary, and the system used the contractor’s corporate networks, which the test team was not allowed to attack. In several tests, connections to external systems were limited or simulated. One team was not allowed to use classified networks to attack a system. Another worked in a lab and simulated external communications.
And some systems had not been tested at all, because they were not far enough along, had been fielded before the department’s emphasis on penetration testing, or because of concern that testing would interfere with operations. GAO noted that untested systems are not necessarily more or less secure than tested ones; the department simply did not know.
Reading a clean test correctly
A test result is evidence about what was tested, for as long as it was tested, by the methods used. The GAO examples show how each limit shrinks what a result proves. A test that excluded the contractor’s network says nothing about the path through it. A test that simulated external connections says nothing about the real ones. A test of 41 hours says little about an adversary with months.
For a bank, a health system or an agency, the same limits appear in ordinary penetration tests. The scope letter excludes production systems, a vendor’s hosted environment, the clinical device network or the branch equipment. The test runs for a fixed number of days. The rules of engagement forbid techniques that might disrupt operations. Each of those choices may be sensible. Each should also appear in the summary that goes to leadership, so that a clean result is not read as proof that the excluded areas are safe.
The untested systems point applies directly. Every organization has systems that have never been tested, because they are too old, too critical to disturb or too recently deployed. A list of them, with the reason and the plan, is part of the evidence. Its absence leaves leaders with the same false confidence GAO described.
Time deserves a separate note. GAO reported that the department’s test and evaluation office had noted that longer-term tests generally identify more cyber vulnerabilities than shorter ones, and department officials said the number of long-term assessments had increased in recent years. A short test finds the easiest way in and stops. A longer one, or a standing arrangement in which testers return over months, comes closer to the patience of a determined attacker. For an organization that cannot afford long engagements on every system, the choice of which systems get the long test is itself a risk decision worth recording.
Why the confidence persisted
The four failures explain how a control list can mislead. GAO’s report also explains why the program offices kept believing it. Three of its reasons apply well beyond defense.
The first was where attention had been. GAO said the department had historically focused on the cybersecurity of its networks, not of the weapon systems themselves, and was in the early stage of understanding how to apply cybersecurity to weapon systems. Several officials told GAO it would take time, and possibly some missteps, to learn what works. An organization that has secured its corporate network for years can carry the same assumption into a newer class of systems: clinical devices, building controls, branch equipment, operational technology in a utility. The controls that work on the corporate network become the list, and the newer systems inherit the confidence without the testing.
The second was connection. GAO said weapon systems connect to the department’s networks and sometimes to external networks, such as those of defense contractors, and that business systems sometimes share networks with weapon systems. Some systems that do not connect directly to a network connect to other systems, such as electrical systems, that do. Even air-gapped systems, it said, citing the department’s test and evaluation office, could potentially be reached through USB devices and compact discs. GAO illustrated the point with the 2013 Target breach, in which attackers first entered through a heating and air conditioning subcontractor and reached a payment system that sat on the same network as other systems. A program office that judges its system by its own controls, and not by everything it connects to, sees only part of the exposure.
The third was people. GAO said the department struggled to hire and retain cybersecurity staff, particularly those with weapon systems expertise. Program officials from a majority of the programs and test organizations GAO met said they had difficulty hiring and keeping people with the right skills, citing a shortage of qualified personnel and private sector competition. Test officials said that once their staff gained experience in the department, they left for the private sector, where they could command much higher salaries. Every one of the four failures depends on skilled people: to implement controls correctly, to read alerts, to fix findings and to design tests. A shortage of those people makes a control list the cheapest available evidence, and so the evidence a short-staffed organization reaches for.
Information sharing compounded all three. Programs that could not learn from each other’s test results repeated each other’s flaws. Officials at a heavily connected program did not know the vulnerabilities of the systems theirs connected to. Inside a bank or a health system, the equivalent is a penetration test report that stays with the team that commissioned it, while other teams running similar systems never see it.
The four failures in a bank, a hospital and an agency
The report’s findings come from weapons programs, but each of the four failures has a recognizable form in the institutions this series writes for. The examples below are our illustrations of how the patterns can appear, not findings from the report.
In a bank, defeated controls can look like a core banking or payments platform where multi-factor authentication protects the front door but service accounts between components use shared, unchanged credentials. Unacted detection can look like a fraud or security monitoring service whose alerts go to a mailbox that nobody watches overnight. Open findings can look like the same privileged-access weakness appearing in successive annual penetration tests. Narrow testing can look like a test scope that excludes the outsourced core processor’s environment, so that a clean report says nothing about the path that matters most.
In a health system, defeated controls can look like clinical devices and imaging systems running commercial software with default administrator passwords. Unacted detection can look like a device that reboots unexpectedly and is logged as a maintenance fault without anyone asking whether the cause could be malicious, the hospital version of GAO’s operators who did not suspect an attack because crashes were normal. Open findings can look like a vulnerability in a supported device that waits on the manufacturer’s patch with no compensating control and no owner. Narrow testing can look like a penetration test that excludes the clinical device network because testing might disrupt care.
In an agency, defeated controls can look like a citizen-facing application with role-based access whose internal service calls are unencrypted. Unacted detection can look like logs that are collected centrally to meet a requirement but reviewed only after an incident. Open findings can look like assessment results recorded in a plan of action that rolls forward from year to year. Narrow testing can look like an assessment of an agency system that stops at the boundary of the contractor’s cloud service, mirroring GAO’s test team that was not allowed to attack the contractor’s corporate network.
In each case, the control list would show a control in place. The evidence would show something else. The point of the four-part split is that each pattern has a different owner and a different fix: engineering for implementation, operations for detection, the risk or program office for remediation, and whoever commissions the testing for its scope.
Objections: realism, metrics and relevance
The first objection, the program officials’ own, was realism. GAO reported that some officials questioned the validity of test results because testers had more system information and access than an adversary would have, or asked for detailed design information. Test organizations and National Security Agency officials dismissed those concerns, GAO said, noting that adversaries are not subject to testers’ limits on time and funding, and that information and access are granted to testers to simulate moderate to advanced threats more closely. The same objection arises whenever a test team is given credentials or network diagrams. The answer is the same: the shortcut stands in for time an attacker would spend.
A second objection concerns metrics. GAO itself included a caution from a department test official that the time it took to break a password or access a system is not a useful measure of cybersecurity, because a determined adversary could spend months or years and there is no meaningful difference between hours and days on that horizon. GAO said it used the nine-second example to illustrate ease of access, not as a measure of overall security. We agree, and it supports the argument: the useful measures are whether a control was defeated, whether the activity was detected and acted on, and whether the finding was fixed.
A third objection is that weapon systems are cyber-physical systems with unusual constraints and that the lessons do not carry to business systems. GAO did describe weapon systems as cyber-physical, with physical consequences that may include loss of life. Hospitals run cyber-physical systems in the form of connected clinical devices and building controls. Agencies run them in utilities, transport and public safety. Banks’ core systems share other features GAO described: heavy software dependence, connections to other systems and dependence on contractors.
A fourth objection is that GAO’s sample of nine program offices was small and not generalizable. GAO said so. Its conclusions about programs’ confidence rest on that sample, but its findings about test results rest on the department’s own test reports. For an organization deciding whether to trust its control list, the specific numbers matter less than the four failure patterns, each of which can be checked directly.
A further point concerns the report’s own limits. GAO said it did not look in depth at related issues such as the security of contractor facilities, internet-connected devices, microelectronics, contracting and industrial control systems, and that test findings were as of a specific date, so vulnerabilities found in development might no longer exist when a system was fielded. Those limits are the same kind GAO criticized in the tests: they bound what the report proves. Applying its patterns elsewhere means checking them locally, which is the method this article recommends.
A fifth objection is cost. Longer tests, broader scope, testing of untested systems and retesting of fixes all cost money, and an organization with a fixed budget has to choose. That is true, and it argues for spending the budget where the four failures are most likely to hide, not for keeping the control list as the evidence. A shorter test of the systems never tested, or a retest of last year’s open findings, may tell leadership more than a repeat of last year’s full-scope test of the corporate network.
Turning test results into evidence a board can use
The four failures suggest a different way to report security to leadership: by evidence type, not by control count. For each critical system, a report can answer four questions with dates.
- Implementation: which controls were exercised by a test in the last year, and which failed. Controls never tested are listed as untested, not as in place.
- Detection and response: in the last test or exercise, was the activity detected, by what, how long after it began, and what did a person do about it.
- Remediation: how many findings from the previous test remain open, who owns each, and how many reappeared in the latest test.
- Coverage of the test: what was out of scope and why, how long the test ran, and which systems have never been tested.
A report in that form is harder to make look good. That is its value. GAO’s report described program offices that believed their systems were secure because the controls were listed. A board, an audit committee or an agency head who receives evidence in these four columns is less likely to make the same mistake.
The report should also change over time in a visible way. If the number of untested controls falls, the number of findings that reappear falls and the time from intrusion to detection in exercises shortens, leadership is seeing improvement in the things GAO found weak. If the control count rises while those measures stay flat, the organization is adding paper. Plotting both side by side, once a quarter, keeps the distinction in front of the people who fund the work.
The same structure works for oversight of suppliers. A vendor that offers a certification or a control list as proof of security can be asked the same four questions about the systems that hold the organization’s data. The answers, or the refusal to give them, are more informative than the certification.
What to ask before the next test
The four failures also shape how a test is commissioned. Before signing a scope letter, a leader can settle four things: whether the scope includes the systems that matter most, including those run by suppliers, and how any gap will be reported; whether the test will check that last year’s findings were fixed; whether the test team will try to trigger a response, so that detection and response are measured as well as prevention; and how long the team will have, compared with how long a capable attacker would.
The answers will not always be the ones a leader wants. A supplier may refuse testing of its environment. Operations may refuse testing that risks an outage. Those refusals are acceptable choices, but they belong in the report as limits on the evidence, so that nobody later reads a clean result as covering ground the test never reached.
With hindsight
GAO returned to the subject in March 2021 with “Weapon Systems Cybersecurity: Guidance Would Help DOD Programs Better Communicate Requirements to Contractors” (GAO-21-179), published March 4, 2021. It found that since the 2018 report, the department had taken action, with officials highlighting more access to expertise, enhanced cyber testing and more guidance. Selected programs had conducted, or planned to conduct, more cybersecurity testing during development than past programs had.
The follow-up turned to the gap the 2018 report had left aside: contracts. GAO quoted department guidance that cybersecurity requirements should be treated like other system requirements and, “more simply, ‘if it is not in the contract, do not expect to get it.’” It said requirements should be defined in contracts, with criteria for accepting or rejecting the work and for how the government verifies that requirements are met. Yet GAO found contracts that omitted requirements, acceptance criteria or verification processes; contracts for three of the five programs it reviewed did not include any cybersecurity requirements when they were awarded. Among the four military services, only the Air Force had issued service-wide guidance on defining cybersecurity requirements and putting them in contracts. GAO recommended that the Army and the Navy develop such guidance, and that the Navy ensure the Marine Corps did as well. As shown on GAO’s page in an Internet Archive copy captured November 11, 2026, the Army recommendation was closed as implemented after the Army published guidance in March 2022. The Navy and Marine Corps recommendations were open; in May 2026, Navy officials said they expected to finalize new guidance by the end of fiscal year 2028.
CISA’s red team reports to critical infrastructure organizations echoed the 2018 findings outside defense. In a February 28, 2023 advisory, CISA said that during a 2022 assessment of a large critical infrastructure organization with a mature cyber posture, the organization did not detect the red team’s activity throughout the assessment, including when the team tried to trigger a security response. CISA said multifactor authentication prompts stopped the team from reaching one sensitive business system. In a November 21, 2024 advisory on another assessment, CISA said the red team gained initial access through a web shell left from a third party’s previous security assessment, that the organization discovered evidence of the initial activity but did not act promptly, and that leadership had deprioritized a vulnerability its own cybersecurity team identified. CISA also said that organization relied too heavily on host-based endpoint detection and response tools and did not implement sufficient network-layer protections, and that its staff needed continuous training, support and resources.
Read beside the 2018 report, the record since then shows the four failures persisting in new settings. The defense follow-up found more testing during development, which bears on the narrow-test failure, and found contracts that omitted cybersecurity requirements, acceptance criteria or verification processes, which in our reading is where the remediation failure starts in contractual form. CISA’s assessments found an organization with a mature cyber posture that did not detect a red team, and another that saw evidence of the team’s initial activity but did not act promptly, both forms of the unacted-detection failure, and a known vulnerability left untreated, the open-finding failure. In our reading, none of those later findings depended on weapon-specific technology.
Spotted an error? Tell us and we will check it and correct the article where needed.