SolarWinds and the three assumptions behind every update policy
The SolarWinds Orion compromise broke three things agencies took for granted: that a signed update is a safe update, that monitoring tools are harmless plumbing, and that federal sensors would spot a national-scale intrusion. Only the second was quick to fix.
Looking Back revisits one week at a time, from January 2020 to today, and asks what it meant for the banks, health systems and public agencies we serve. The series was written in 2026, with the benefit of hindsight, and each article says which week it looks back on.
Two-minute summaryThe dek and the first sentence of each section, quoted from the article
The SolarWinds Orion compromise broke three things agencies took for granted: that a signed update is a safe update, that monitoring tools are harmless plumbing, and that federal sensors would spot a national-scale intrusion. Only the second was quick to fix.
- The week On December 8, 2020, the security firm FireEye disclosed that a highly sophisticated attacker had breached it.
- Poisoned at the factory The incident makes sense only once you follow how software reaches an organization.
- Assumption one: signed updates are safe Patching quickly is among the most consistent pieces of security advice.
- Assumption two: monitoring tools are low risk Network monitoring software is useful because it can see and reach a great deal of an environment.
- Assumption three: government sensors would see it The federal government had invested for years in perimeter sensing for civilian agencies.
- Four steps from download to backdoor Following an update like the tainted Orion release through a typical organization shows where a defense could fit at each step.
- A government breach with lessons for every board SolarWinds was a government story, but its lessons apply well beyond Washington.
- Days to an answer, or weeks Teams with an accurate software inventory, months of retained DNS and proxy logs, and a working security contact at the vendor answered within days.
- What to ask before the next supplier breach Each question below maps to one of the three assumptions or to the evidence that settles them.
- With hindsight The scale came into focus over the following weeks.
The week
On December 8, 2020, the security firm FireEye disclosed that a highly sophisticated attacker had breached it. FireEye was itself a company that investigates breaches for others, so the news was striking on its own. The attackers had taken FireEye’s red team tools, the software its consultants use to simulate attacks for clients. FireEye published countermeasures so that defenders could detect those tools if they were misused. Within days it turned out that this breach, unlike most, had begun with organizations doing exactly what they had been told to do: installing a software update from a trusted supplier.
On Sunday, December 13, the wider scope emerged. Attackers had inserted malicious code into updates of SolarWinds’ Orion network monitoring software. Network monitoring software is the tool an IT team uses to keep watch over its computers, servers and network equipment, alerting staff when something slows down or fails. To do that job, it needs to see and reach a great deal of the organization. In this case, the very updates customers installed to keep Orion current carried a hidden backdoor: a secret way for the attackers to get in later. FireEye and Microsoft published technical details the same day, and news reports named the Treasury and Commerce departments among the victims.
The emergency directive
That evening CISA, the Cybersecurity and Infrastructure Security Agency, which leads the defense of federal civilian networks, issued Emergency Directive 21-01. An emergency directive is a binding order to federal civilian agencies. This one ordered them to disconnect or power down affected Orion products, covering versions 2019.4 through 2020.2.1 HF1. Agencies with the expertise were to capture forensic images of system memory and disks first. All agencies had to block traffic to and from outside hosts running Orion and remove any accounts or persistence the attackers had created.
The directive set a deadline of noon Eastern time on Monday, December 14, for agencies to report whether they had found the malicious file or related indicators. Department chief information officers had to attest by the same deadline that affected devices were disconnected. Agencies were told not to rejoin the affected servers to their networks, or reinstall patched software, until CISA said so. It was an unusually blunt instruction, and it gave agencies hours rather than days.
The scale
On December 14, SolarWinds told the Securities and Exchange Commission that fewer than 18,000 customers may have installed the affected versions, released between March and June 2020. The tainted updates had been circulating for months before anyone noticed.
Over the next days, defenders learned more about how the backdoor behaved. Researchers found that infected servers announced themselves through internet directory lookups to a domain the attackers controlled. Microsoft, FireEye and GoDaddy then took control of that domain to help disable the backdoor in some circumstances. That step could not undo access the attackers had already used.
During the week, news reports and official statements confirmed that several federal departments had been affected. On December 16 the FBI, CISA and the Office of the Director of National Intelligence formed a unified coordination group to lead the federal response. That step signaled how seriously the government was treating the incident. The Energy Department confirmed it had been affected, and Microsoft said it had found the malicious Orion code in its own environment.
Beyond Orion
On December 17 CISA published a public alert. It said the actor posed a grave risk to federal, state, local, tribal and territorial governments, critical infrastructure and the private sector. The activity had begun at least as early as March 2020. CISA also said it was investigating other ways in besides the Orion updates, including abuse of legitimate accounts and poorly secured administrative credentials.
The alert described one technique in particular. Once inside, the attackers had stolen the certificate an organization uses to sign its login tokens. With it, they could forge valid tokens and enter cloud services that trusted those tokens. That turned an on-premises compromise into access to email and documents in the cloud.
Organizations far from Washington drew an uncomfortable conclusion. If an attacker could reach thousands of organizations through one trusted supplier, the same could be true of the suppliers every organization depends on.
Large intrusions into government networks were not new in 2020. The route was. The attackers did not break into each target. They compromised one vendor’s build process and let thousands of customers install the result through normal, approved update procedures.
Most customers who installed the tainted update were never pursued further. The attackers chose a much smaller set of targets for follow-on activity. That pattern broke three assumptions on which agency security programs rested. One was that signed updates from a trusted vendor are safe to install. Another was that monitoring tools are low-risk infrastructure. The third was that the government’s own sensors would detect an intrusion of this scale.
The three assumptions were held by different people. Patch policy belonged to IT operations and was measured by compliance reports. The status of management tools was set by network teams who chose and ran them. Detection belonged to security operations and, for federal civilian agencies, partly to CISA. Each group was acting reasonably within its own frame. The attack ran through the seams between them.
The sections below first explain how the software was poisoned, then take each assumption in turn. They follow a single update from download to backdoor, consider what the incident meant beyond government, and end with how prepared organizations got to an answer quickly.
Poisoned at the factory
The incident makes sense only once you follow how software reaches an organization.
A software company writes its product as source code, the human-readable instructions programmers write. Before customers can run it, that code goes through a build process: an automated assembly line that compiles the code into a finished program, packages it and prepares it for distribution. Then the company applies a digital signature, a cryptographic stamp that lets a customer’s computer confirm the file really came from that company and was not altered on the way.
A software supply chain attack targets that assembly line rather than the customer. It works like tampering with food at the factory rather than breaking into each kitchen: every jar then carries the genuine brand and seal and is bought by customers doing nothing wrong. The seal proves the jar came from the factory, not that the factory was clean.
Orion was poisoned in just this way. The malicious code was inserted during the building of Orion updates, so it was signed and shipped like any other release. Customers’ systems checked the signature, found it genuine and installed the update.
How the build was compromised
Investigators later pieced together how the code got in. SolarWinds said in January 2021 that the earliest suspicious activity it had found dated to September 2019, and that the attackers had tested their ability to insert code in an October 2019 release. The real backdoor went into releases in early 2020.
The security firm CrowdStrike, working with SolarWinds, described the tool used on the build server. It watched for the moment Orion was being compiled and swapped one source file for a malicious version, then put the original back. The source code in the company’s repository stayed clean. A developer reviewing it would have seen nothing wrong. Only the compiled output carried the backdoor.
That detail matters for defenders. Code review, the most common software quality control, looks at source code. An attack that lives only in the build step passes code review by design. Defending against it means protecting the build environment itself and checking that what was built matches what was written.
How the backdoor behaved
The backdoor was built to be patient. After installation it waited for up to about two weeks before doing anything. It checked whether it was running in a test environment or alongside certain security tools, and stayed inactive if so. Then it sent encoded information about its host to the attackers through internet directory lookups that looked like routine traffic.
The attackers read those lookups and decided which victims were worth pursuing. For the few they chose, the backdoor fetched further tools. FireEye described a second-stage dropper that ran in memory, leaving little on disk. From there, the attackers used stolen credentials and legitimate administrative tools to move toward the data they wanted, often email.
Two more terms help. Privilege is the level of access an account or tool has: the more systems it can see and change, the more privileged it is. A foothold is the first point of access an attacker gains, from which they try to move further. The attackers in this case gained a foothold inside a highly privileged tool, which is the most valuable kind.
Assumption one: signed updates are safe
Patching quickly is among the most consistent pieces of security advice. Patching means installing the fixes that vendors release, often to close known weaknesses. Agencies were measured on how fast they applied vendor updates. The Orion compromise turned that discipline against them. The malicious code arrived in updates that were digitally signed and delivered through official channels, so the controls designed to confirm authenticity confirmed it.
A signature proves which organization produced a file. It says nothing about whether that organization’s build environment was clean, and the whole incident turned on that distinction.
Stopping or slowing patching would have been the wrong response, because known vulnerabilities in unpatched software remained a far more common route in. A leader who reacted to SolarWinds by telling teams to delay updates would have traded a rare, sophisticated risk for a common, everyday one. The useful responses lay on either side of the update.
- Upstream, at the supplier. Buyers began asking how software is built. Are build systems isolated, and signing keys separated from developers? What third-party components go into a product, and does the supplier follow a recognized secure development framework?
- Downstream, after installation. Trusted software can be watched for behavior its function does not need. A monitoring tool that starts contacting unfamiliar internet domains is a signal, whatever its signature says.
The upstream questions were hard for most agencies to verify in 2020. Few contracts gave them the right to ask, and few suppliers had answers ready. That gap shaped much of the federal policy that followed.
What a signature can still do
Signatures did not become useless. They still stop the far more common attack in which a criminal distributes a fake update from a look-alike site. They also gave the industry a way to respond. SolarWinds later revoked the certificate it had used to sign the affected releases, so systems would stop trusting anything signed with it. Customers then had to install versions signed with a new certificate.
Revocation has its own cost. Every legitimate release signed with the old certificate becomes untrusted at once, and organizations that have not upgraded find tools failing. A buyer that tracks which certificate signs each critical product, and which versions it runs, can plan for a revocation. One that does not finds out when the software stops working.
Why testing windows did not help
Many organizations install updates in rings: a small test group first, then wider groups after a few days. Rings catch updates that crash systems or break features. They were never designed to catch a backdoor, and this one was built to defeat them. A dormant period of up to two weeks meant a test server could run the update cleanly through a typical evaluation window.
Rings can still help if the test group is watched for behavior as well as stability. A monitoring server in the first ring with tightly restricted outbound access would have generated a blocked connection, sooner or later, when the backdoor tried to call out. That turns a ring from a quality gate into a detection opportunity. It costs little beyond keeping egress rules and alerting in place.
What buyers can ask of suppliers
The supplier questions have become more concrete since 2020. A buyer can ask whether builds run on isolated, short-lived systems that are rebuilt for each release. It can ask whether signing keys are held in hardware security modules and whether release requires more than one person. It can ask whether the supplier can show that a given binary was built from a given set of source code. It can ask for a software bill of materials, a list of the components in the product.
None of those answers proves a supplier is clean. Each makes a SolarWinds-style compromise harder, and each tells the buyer whether the supplier has thought about the problem. A supplier that cannot answer is giving useful information too.
Bargaining power differs by buyer
How far a buyer can push depends on who it is. A large federal department can write requirements into contracts and, after 2021, could rely on government-wide rules. A state agency or county often buys through cooperative contracts or statewide vehicles negotiated by someone else, with little room to add terms. A bank is subject to supervisory expectations on third-party risk that already require due diligence on critical vendors, so it has both the obligation and some bargaining power. A small credit union or rural hospital buying commercial software off the shelf has almost none.
For buyers with little bargaining power, the downstream controls matter more. They cannot change how a supplier builds software. They can control what that software is allowed to reach once it is installed.
Assumption two: monitoring tools are low risk
Network monitoring software is useful because it can see and reach a great deal of an environment. It often runs with broad credentials and wide network access. In many agencies it was run by network operations teams, outside the scrutiny applied to identity systems or financial applications. It was treated as plumbing.
Orion showed that management tools are among the most valuable footholds an attacker can get. Nobody regards the facilities office as the most sensitive room in a building, yet whoever holds its master keys can open every door.
The same is true of remote management agents, backup systems, configuration management, endpoint management and directory synchronization services. Any tool that can see or change many systems at once is a concentration of privilege. On-premises systems are those an organization runs in its own buildings or data centers; cloud services are run by a provider over the internet. Many organizations link the two so that one login works in both, and that link became a bridge for the attackers.
The bridge into the cloud
The bridge worked through federated login. Many agencies and companies ran an on-premises service that issued signed tokens to prove a user had logged in. Cloud services such as hosted email trusted any token carrying the right signature. CISA’s alert described attackers who gained high privileges in the on-premises directory, stole the token-signing certificate and forged tokens for any user they chose.
The attackers also added credentials to existing cloud applications that already had broad permissions, such as the ability to read mailboxes. That gave them access that did not depend on any user’s password and survived password resets. Many victims did not routinely review which applications had such permissions, or when their credentials changed.
The defenses are specific. Token-signing keys belong in hardware security modules or in the cloud provider’s managed service, not on an ordinary server. Changes to credentials on privileged cloud applications should raise an alert. Administrative accounts for the cloud should be separate from on-premises accounts, so that a compromise of one does not automatically become a compromise of the other. Each of those steps was available in late 2020.
Inexpensive steps
This was the assumption most open to quick action, and the steps were inexpensive:
- Inventory the tools. List every product that holds broad credentials or can push changes to many systems, with its owner.
- Restrict outbound traffic. Most management servers need to reach a short, known list of destinations. Blocking or alerting on everything else, including unusual DNS lookups, makes a backdoor’s call home harder to complete and easier to spot. DNS is the internet’s directory service, which turns names like a company’s web address into the numeric addresses computers use; a record of those lookups shows who a machine has been trying to reach.
- Treat service accounts as privileged. A service account is a login used by software rather than a person. Put their credentials in privileged access management, a system that locks away and monitors the most powerful passwords, scope them to what each tool needs, and alert on use outside expected patterns.
- Segment the management plane. Place these servers where a compromise cannot reach everything else directly, so that one compromised server does not open the rest of the network.
The friction, and who pays it
The trade-off is operational friction. Egress restrictions, limits on outbound traffic, break vendor features that phone home for updates or licensing, and network teams resist changes to tools they rely on daily. Agencies that made the changes stick usually gave the tools an explicit security owner alongside the operational one.
Remote vendor support is the hardest case. Many management products are supported by the vendor reaching in, or by the product reaching out. Cutting that link can slow support during an outage. A workable compromise is a documented, time-limited path for vendor access, opened on request and logged, instead of a standing connection. Service accounts raise a similar tension: scoping them tightly means some monitoring features stop working until someone grants the specific rights they need. That work is tedious, which is why it was often skipped.
Assumption three: government sensors would see it
The federal government had invested for years in perimeter sensing for civilian agencies. The perimeter is the boundary between an organization’s network and the internet, and perimeter sensors inspect traffic crossing it. The SolarWinds intrusion was found by a private company investigating its own breach.
The attackers used legitimate update channels, kept their traffic close to normal patterns, and moved slowly and selectively. Signature-based inspection at the network boundary, which looks for known patterns of malicious traffic, was a poor fit for that behavior. Checks for known contraband do not stop an intruder who carries legitimate paperwork and looks like every other traveler.
What the sensors could and could not see
The federal perimeter program, known as EINSTEIN, compared traffic against indicators of known threats. A new backdoor has no known indicators until someone finds it. Much of the traffic was also encrypted, or looked like ordinary directory lookups and web requests to cloud services. The attackers even chose infrastructure in the United States, which made their traffic look less unusual to analysts.
Visibility on the hosts themselves was thin. Many agencies had not deployed endpoint detection and response tools, which record what programs do on each computer, across their estates. Where such tools existed, CISA did not generally have access to their data. The central team could see the border, and little inside it.
What hunting requires
Hunting across many organizations needs three things the 2020 model lacked. The first is legal authority to look inside networks the hunter does not own, without negotiating each time. The second is access to rich data from endpoints and identity systems, rather than network traffic at the border alone. The third is skilled analysts with time to follow weak signals that may lead nowhere.
Each has a cost. Agencies give up some autonomy when a central team can search their systems. Endpoint data is large and sensitive, and sharing it raises questions about privacy and about who may see what. Skilled hunters are scarce and expensive. A department with its own capable security team gains less from central hunting than a small agency with two security staff, which gains a great deal.
Who found it
FireEye’s discovery came from inside its own environment. Its chief executive later told Congress that the investigation began when a security analyst noticed an unexpected device registered to an employee’s account for multi-factor login. Pulling that thread led back, over weeks of work, to the Orion server. A single alert about identity, investigated by people with the skills and time to follow it, did what national sensors had not.
The detection gap pointed toward visibility inside networks and on endpoints, the individual computers and servers, longer log retention, and the authority for a central team to hunt across agencies rather than inspect boundary traffic alone. Hunting means actively searching for signs of an intruder who has already slipped past the defenses, rather than waiting for an alarm.
It also showed how much national cyber defense depended on private-sector visibility and on victims’ willingness to disclose quickly. A private company’s investigation of its own breach, not a government sensor, was what began the wider discovery.
Outside the federal enterprise
State and local governments were in a weaker position still. Many relied on shared services, such as those offered through the Multi-State Information Sharing and Analysis Center, which also worked largely from known indicators at the network edge. Most had small security teams and short log retention. Banks and large health systems usually had their own security operations centers, but their detection was also tuned to known threats. Across sectors, the organizations that found traces quickly were those with endpoint visibility and months of retained logs.
The case that nothing would have worked
A fair counter-argument is that this was an elite intelligence service, and no realistic defense would have stopped it. That is partly true. Few organizations could have prevented the tainted update from arriving, and the most careful targets were still breached.
Prevention was never the only measure, though. Restricted egress would have stopped or exposed the first call home in many networks. Alerts on new credentials for cloud applications would have exposed a common later step. Retained logs would have let organizations establish quickly whether they had been touched. Those controls would not have stopped the campaign. They would have shortened it, narrowed it and made the cleanup faster.
Four steps from download to backdoor
Following an update like the tainted Orion release through a typical organization shows where a defense could fit at each step.
First, the vendor announces a new version. The IT team responsible for the tool reads the release notes, perhaps tests the update on a small number of systems, and schedules it for installation. Their performance is often measured, in part, by how quickly they keep tools current, so there is a natural pressure to move promptly.
Second, the update is downloaded from the vendor’s official site or delivered automatically. The organization’s systems check the digital signature. It matches. The file is genuine in the only sense the check can measure.
Third, the update is installed on the monitoring server, which already holds broad credentials and network access because that is what monitoring requires. From that moment, whatever is inside the update runs with all the access the tool already had.
Fourth, the tool goes back to its normal job. It polls devices, collects performance data and sends alerts. If hidden code inside it also tries to reach an outside server, that traffic is mixed in with the tool’s ordinary activity. In an organization that allows its management servers to reach anywhere on the internet, and keeps only short records of where they connect, nobody is likely to notice.
Laid out this way, the opportunities become clear. Before the update, a buyer can ask the supplier how its build process is protected. At installation, little can be done, because the signature is genuine. After installation, the organization can limit where the tool is allowed to connect, watch for connections it does not need, and keep records long enough to look back. The most practical defenses are on either side of the update, not at the moment of installation.
The selectivity of the attackers matters too. Because most customers who installed the update were never pursued further, many organizations could reasonably conclude they had not suffered harm. But they could only reach that conclusion with evidence. Without logs, “we were probably not targeted” is only a hope.
Owners for each step
Each step has a natural owner. Procurement and vendor management own the questions asked before purchase and renewal. IT operations owns the update schedule and the test ring. Network engineering owns egress rules for management servers. Security operations owns the alerts and the logs. Identity teams own service accounts and the trust between on-premises and cloud. Mapping those owners to a single tool, such as the monitoring platform, is a quick way to find the step that nobody owns. In many organizations in 2020, that step was outbound traffic from management servers.
A government breach with lessons for every board
SolarWinds was a government story, but its lessons apply well beyond Washington.
Supplier risk is your risk. Every organization trusts dozens or hundreds of software suppliers with privileged access. The incident showed that a single supplier’s internal weakness can become a customer’s breach, without the customer making any mistake.
Cost of uncertainty. For state and local agencies and regulated institutions that ran Orion, the week brought an immediate task. They had to establish whether they had installed the affected versions, whether the code had called out, and whether there was follow-on activity. Many lacked the logs to answer with confidence. That gap, more than the software itself, decided how long their uncertainty lasted. Uncertainty is expensive: it consumes staff time, delays answers to regulators and customers, and can force an organization to assume the worst.
Regulators and disclosure. A breach through a supplier still has to be explained to regulators, auditors and, in many cases, customers. Being able to say quickly and confidently what happened is itself a form of protection.
Staff and accountability. Tools that fall between teams, owned operationally by one group and examined for security by nobody, are exactly the ones attackers favor. Clarifying ownership is an organizational decision rather than a technical one.
How the question reached each sector
The same question reached different organizations through different channels. Federal agencies answered to CISA under a binding directive with hard deadlines. State and local governments faced no binding federal order, but received CISA’s alerts and questions from their own leadership and auditors. Banks could expect their examiners to ask whether institutions ran Orion and what they had found. Health systems had to weigh whether any access to patient data could amount to a reportable breach under HIPAA. Publicly traded companies had to consider whether the exposure was material for securities disclosure.
In each case the answer depended on the same three facts: which version ran where, what it had connected to, and what the attackers had done next. The format of the report differed. The evidence behind it did not.
The board’s part
Boards cannot evaluate build pipelines. They can ask whether management has a list of the suppliers whose compromise would hurt most, and what the organization would know within a week if one of them were compromised. They can ask how long logs are kept and what that retention costs. They can ask whether critical supplier contracts include security notification terms. These are questions about preparation, not technology, and they suit a board’s role.
Days to an answer, or weeks
Teams with an accurate software inventory, months of retained DNS and proxy logs, and a working security contact at the vendor answered within days. Others spent weeks rebuilding basic facts. A proxy is a gateway through which an organization’s web traffic passes; its logs, like DNS logs, record where systems have been trying to connect.
Take two organizations that both ran an affected product. Suppose a regional bank keeps a list of every tool with broad access, each with a named security owner. Its management servers can reach only a handful of approved internet destinations. It retains DNS and proxy logs for many months. When the news broke, its team confirmed within hours which version it ran and checked its logs for unusual outbound connections from the monitoring server. It reported a clear answer to its board and examiners within days.
Now say a county government ran the same product. Its monitoring server could reach any destination on the internet. Its DNS logs were kept for only a short period, so the months when the tainted update was active had already been overwritten. Nobody outside network operations had been responsible for the tool’s security. The county spent weeks reconstructing basic facts and, in the end, could not say with confidence whether anything had happened.
Neither organization was at fault for the supplier’s compromise. The difference lay entirely in preparations made before it.
The evidence that settles the question
Prepared teams worked from a short list of evidence. They pulled the software inventory showing which Orion versions ran on which servers and when each was installed. They searched DNS logs for lookups to the attackers’ domain and its subdomains. They reviewed proxy and firewall logs for outbound connections from the Orion servers. They checked identity logs for new accounts, new credentials on cloud applications and unusual token use. Each item answered one link in the chain from installation to follow-on activity.
Working with the vendor
The vendor relationship mattered as much as internal records. Prepared organizations knew who their security contact at the supplier was and how advisories reached them. They knew which customer portal carried hotfix notices, and who internally was subscribed to it. In December 2020 SolarWinds issued guidance and hotfixes in quick succession, and CISA updated its own requirements several times. Teams that had to discover where those notices lived lost days. A named contact, a monitored advisory feed and a contract clause on prompt security notification cost little to set up in advance.
The cost of keeping logs
Log retention is not free. Storing months of DNS, proxy, endpoint and identity logs costs money, and the cost grows with volume. Many organizations kept short retention because nobody had made the case for more. SolarWinds made that case: the backdoor had been circulating since the spring, and it was found in December. A retention period shorter than the time an intruder can go undetected leaves an organization unable to answer the most basic question.
Tiered storage eases the cost. Recent logs can be kept in fast, searchable systems, while older ones move to cheaper storage that is slower to query. For an investigation like this one, slow access to old logs is far better than none.
The usual errors follow from the three assumptions:
- Overreacting on patching. Slowing all updates in response to a supply chain attack swaps a rare risk for a common one.
- Treating management tools as plumbing. Leaving powerful tools without a security owner, unrestricted network access and unmonitored service accounts.
- Short log retention. Keeping logs for days or weeks when compromises can go undetected for months.
- No contractual rights with suppliers. Discovering, mid-crisis, that the contract gives no right to ask how the software is built or to get timely security information.
- Relying on the perimeter alone. Assuming that anything dangerous will be caught at the border, when careful attackers arrive looking legitimate.
What to ask before the next supplier breach
Each question below maps to one of the three assumptions or to the evidence that settles them. The first two concern privileged tools. The third concerns detection and logs. The fourth concerns suppliers and signed updates. The last tests all of them together.
- Which of our tools can see or change large parts of our environment, and who is responsible for the security of each one?
- Where are the passwords for our service accounts kept, and would we notice if one were used in an unusual way?
- How many months of DNS, proxy and endpoint logs do we keep, and would that be enough to investigate something that began half a year ago?
- Do our contracts with critical software suppliers give us the right to ask how their software is built and what components it contains?
- If a trusted supplier announced tomorrow that its updates had been tampered with, could we say within days, with evidence, whether we were affected?
The SolarWinds compromise was a reminder that trust in a supplier is a decision with consequences. The organizations that came through it best were not those with the most sophisticated defenses. They were those that knew what they ran, kept the records to prove what had happened, and had decided in advance who was responsible.
With hindsight
The scale came into focus over the following weeks. In February 2021 the White House said that about nine federal agencies and about 100 private-sector companies had been compromised in follow-on activity. That confirmed the pattern visible in December: thousands of organizations installed the update, and a small number were chosen.
In April 2021 the U.S. government formally attributed the campaign to Russia’s foreign intelligence service and announced sanctions. Attribution, the formal public naming of who was responsible, is a significant step, because it signals that the government is confident enough in its evidence to act on it. The same service remained active. In January 2024 Microsoft disclosed that the group, which it now called Midnight Blizzard, had accessed some of its corporate email accounts.
Congress moved first on detection. The annual defense authorization act that became law on January 1, 2021 gave CISA authority to hunt for threats on federal civilian networks. Executive Order 14028, signed in May 2021, launched federal work on software supply chain security, endpoint detection, logging standards and zero trust, an approach to security that assumes no user, device or tool should be trusted automatically just because it is inside the network. That work led to software bills of materials, which are ingredient lists for software showing which components a product contains, secure development attestations from software suppliers and expanded CISA authority to hunt on agency networks. Each maps directly onto the gaps the incident exposed: what is inside the software, how it was built, and whether anyone was looking inside the network.
The follow-on guidance was detailed. NIST published its Secure Software Development Framework in early 2022. OMB told agencies in September 2022 to collect attestations from software suppliers that they followed it, and CISA finalized a common attestation form in March 2024. An August 2021 OMB memo on logging set retention expectations of 12 months in active storage and 18 more in cold storage, a direct answer to the log gaps of December 2020. A January 2022 OMB memo set a federal zero trust strategy with deadlines running to the end of fiscal 2024.
A Cyber Safety Review Board was created to study major incidents, though its members were dismissed in January 2025. CISA formally closed Emergency Directive 21-01 in January 2026, having concluded its purpose was met.
The legal aftermath was long. The SEC’s 2023 enforcement case against SolarWinds and its chief information security officer was sharply narrowed by a federal judge in July 2024. The SEC had alleged that the company misled investors about its security practices and risks, and most of those claims did not survive the court’s review. The SEC dropped the remaining claims in November 2025. Security chiefs and boards had watched the case closely as a test of personal accountability for security disclosures, and its outcome is part of the incident’s legacy.
The pattern recurred. In July 2021 a ransomware gang exploited Kaseya’s remote management software to reach the customers of managed service providers that used it. In 2023 a trojanized version of the 3CX desktop phone application was distributed through the vendor’s own channels. Both cases ran through tools that many customers trusted with wide access, as Orion had.
Of the three assumptions, the second has changed most: management tools are now widely treated as privileged infrastructure. Verifying a supplier’s build process and seeing inside every network remain incomplete. Attestations depend on suppliers describing their own practices accurately, and software bills of materials are useful only to buyers able to act on them. Endpoint coverage and logging improved most at large federal agencies and lagged at smaller agencies and local governments.
That ordering is instructive. The fixes that changed fastest were the ones an organization could make on its own, within its own walls, at modest cost. The ones that lagged required suppliers, contracts and entire industries to change. Leaders who want to be ready for the next supply chain incident can start with what is in their own hands.
Spotted an error? Tell us and we will check it and correct the article where needed.