ProxyLogon: the patch was the easy part

The week

On March 2, 2021, Microsoft told the organizations that ran their own email servers that the software had four serious, previously unknown holes in it. Someone had already been using them to break in. For the IT manager of a mid-sized county, the morning had started with email working and nobody complaining.

That day, Microsoft released what it called out-of-band security updates for on-premises Exchange Server. “Out-of-band” means the fixes could not wait for the regular monthly patch cycle. “On-premises” means Exchange running on servers the organization owns and manages itself, in its own building or data center, rather than Microsoft’s hosted version. The four flaws were what security people call zero-days: weaknesses that attackers discovered and used before the vendor knew about them, leaving defenders zero days of warning. As a group, they later became known as ProxyLogon.

The updates covered Exchange Server 2013, 2016 and 2019, the versions then in support. Microsoft also shipped an update for the older Exchange 2010 as a precaution. The four flaws were tracked as CVE-2021-26855, CVE-2021-26857, CVE-2021-26858 and CVE-2021-27065. The first was the most important, because it let an attacker who had no password reach parts of the server that should have required one.

The flaws had not been found that week. A Taiwanese security firm, DEVCORE, had reported the core bug to Microsoft in early January. The security firm Volexity separately detected attacks using it in early January and also reported them to Microsoft. For about two months, then, a small number of attackers had a way in that defenders did not know existed. Exploitation appears to have widened sharply in the last days of February, before the fixes shipped.

Microsoft attributed the first wave of break-ins to a group it called HAFNIUM, which it assessed to be state-sponsored and operating from China. Its account described the attackers using the access to steal mailbox contents and to dump credentials from server memory. Exchange Online, Microsoft’s hosted email service, was not affected. The problem lived only in the copies organizations ran for themselves.

The next day, the Cybersecurity and Infrastructure Security Agency (CISA), the federal government’s lead civilian cyber defense agency, issued Emergency Directive 21-02. Emergency directives are rare, binding orders to federal civilian agencies. This one told them to do three things. First, collect forensic images: exact, preserved copies of a server’s contents, the digital equivalent of sealing a crime scene. Second, look for indicators of compromise, the telltale traces an intruder leaves behind. Third, patch or disconnect affected servers. It gave agencies days, not weeks, and told them to report back to CISA.

The timing was hard on federal security teams. CISA’s previous emergency directive, 21-01, had been issued in December 2020 over the SolarWinds compromise, and agencies were still working through that investigation. Many of the same small teams that were hunting for SolarWinds intruders now had to image, examine and patch their mail servers as well, against a deadline measured in days.

The directive bound only federal civilian agencies. States, counties, cities, schools, hospitals and businesses were not covered by it, although CISA urged everyone to follow the same steps. It also issued a joint advisory with the FBI describing the attackers’ techniques. For most of the organizations that ran Exchange, the federal directive was a template to borrow, not an order to obey.

The story did not stay contained. Within days, researchers reported that several other groups had joined in, using automated tools to sweep the internet for unpatched Exchange servers and break into them at scale. What began as a targeted espionage operation became a free-for-all. On March 7, the European Banking Authority said its Exchange servers had been attacked and took its email systems offline as a precaution. Before the month was out, researchers had found ransomware, including a strain named DearCry, being dropped onto servers through the same holes.

The response also went political. The White House National Security Council set up a Unified Coordination Group, drawing on the FBI, CISA and the intelligence community, to run the federal response. Microsoft followed with a one-click mitigation tool in mid-March, aimed at organizations without dedicated security staff.

Widely reported estimates put the number of compromised organizations in the United States in the tens of thousands. Many were not large enterprises with security teams on staff. They were local governments, schools and small businesses that ran their own mail servers, often looked after by one or two people, or by a contractor who visited occasionally. For them, the week raised an uncomfortable possibility: if the attackers were already inside, installing a patch would not be enough.

Most coverage of ProxyLogon treated it as a race to patch. That framing held for the first few days and then misled. Once exploitation spread, the patch only stopped new break-ins. It did nothing about the intruders who had already arrived.

For public agencies, the outcome turned on three gaps that sat around the patch, each with a different owner. The first was an exposure gap: who knew which servers the internet could reach. The second was a persistence gap: who looked for intruders already inside. The third was an ownership gap: who had the authority and budget to retire the platform. In many agencies, none of the three had an owner on March 2.

The argument applies across the public sector, but unevenly. A federal civilian agency was under a binding directive and had CISA on the phone. A state agency usually had a central IT department or a state security operations center to call. A county, city, school district or court often had neither, and a single administrator who also handled printers and phones. The three gaps were the same everywhere. How wide each one was depended on where in that hierarchy an organization sat.

The sections below explain the technology briefly and take each gap in turn. They then follow the costs, compare two neighboring governments, and end with the questions a CIO should be able to answer.

What Exchange does, and why these flaws were so dangerous

Microsoft Exchange Server is the software that runs an organization’s email, calendars and contacts behind the scenes. When a staff member opens Outlook, sends a meeting invitation or checks mail on a phone, Exchange is the engine doing the work. For decades, organizations ran it themselves on their own hardware. Over time, many moved to Microsoft’s hosted version, Exchange Online, where Microsoft runs the servers and keeps them updated. Plenty did not, for reasons we will come to.

A vulnerability is a flaw in software that lets someone make it do something its designers never intended, such as hand over data or run the attacker’s own instructions. A patch or security update is the vendor’s fix. If the vulnerability is a faulty lock on the front door, the patch is a replacement lock. It keeps new burglars out. It does not tell you whether anyone walked in while the old lock was broken, or whether they left a spare key under the mat.

That spare key has a technical name in this story: a web shell. It is a small script, often only a few lines long, placed in a folder that a web server makes available to the outside world. Anyone who knows its address can use it to send commands to the server. An email server has to be reachable from the internet so people can read mail from home or on their phones. A web shell on an Exchange server gives an intruder a discreet way back in at any time.

Email servers are an especially valuable target. They hold years of internal correspondence, legal advice, personnel matters, procurement discussions and constituent data. They also sit near the heart of an organization’s network and its directory of user accounts. An attacker who controls the mail server is well placed to read sensitive material and to move deeper.

How the four flaws fitted together

Each of the four flaws was serious. Chained together, they were worse, because they removed the need for a password at all.

The first, CVE-2021-26855, was a server-side request forgery. Exchange’s front end passes web requests to back-end services on the same server. The flaw let an outsider craft a request that the back end treated as if it came from Exchange itself, which is trusted. In effect, the attacker borrowed the server’s own identity.

With that foothold, the other three flaws did the rest. CVE-2021-26857 was a deserialization flaw in the Unified Messaging service, which handles voicemail, and could let an attacker run code with the highest privileges on the machine. CVE-2021-26858 and CVE-2021-27065 let an authenticated user write a file to any location on the server. Combined with the first flaw, that meant an unauthenticated outsider could place a file wherever they liked.

The file of choice was a web shell, written into a folder the web server publishes. CISA’s advisory with the FBI described web shells, including variants of a long-known one called China Chopper, as a common result. Microsoft’s account described what the first attackers did next: dumping credentials from memory, exporting mailboxes and compressing data for theft.

That sequence explains why the response could not end with the patch. The patch closed the first step. By the time most organizations installed it, many servers had already completed the later steps.

Why patching took longer than it looked

“Apply the update” sounds like a ten-minute job. On Exchange it often was not. Microsoft’s security updates install only on recent cumulative updates, the large periodic releases that bring Exchange up to a current baseline. A server several cumulative updates behind had to be brought current first. That can take hours per server, needs a maintenance window and sometimes requires directory changes.

Microsoft recognized the problem and released stopgap fixes for some older cumulative updates within about a week. Its mitigation tool and an automatic mitigation pushed through Microsoft Defender followed, both described as interim measures rather than replacements for the update. Each of those steps helped organizations that were behind. Each also signaled how many organizations were behind.

The servers furthest behind tended to belong to organizations with the least capacity. A county whose Exchange server had not been touched since a contractor installed it was likely to be several versions behind, with nobody on staff who had run an Exchange upgrade. The patch race was hardest for the people least equipped to run it.

The exposure gap: knowing what faces the internet

Exchange servers were attractive targets because they are reachable from outside by design. Outlook on the web, mobile sync and autodiscover all require it. Outlook on the web is the browser version of email. Mobile sync is how phones keep mail up to date. Autodiscover is the feature that lets a new device find the right server automatically when a user types in an email address. Each one is a doorway that has to face the street.

Many smaller agencies had stood these servers up years earlier, often through a contractor, and they had become part of the furniture. Some agencies also had forgotten hybrid servers left over from a partial move to the cloud, kept only to manage directory attributes. They had moved mailboxes to Microsoft’s hosted service but kept one Exchange server on site to manage user account details, and then stopped thinking about it. A server nobody thinks about is a server nobody patches.

When the advisory landed, the first task was simple: count the servers and find them. In many organizations the honest answer took days, pieced together from firewall rules, DNS records and purchase orders. Firewall rules are the settings that decide which traffic may enter the network. DNS records are the internet’s address book, matching names like a county’s mail address to machines. Purchase orders are, well, purchase orders. Reconstructing your own estate from those three sources is slow and error-prone work.

That delay mattered more than the time needed to install the update. Mass scanning began within days of disclosure, so an agency that needed three days to find its servers had already lost the race on some of them. Scanning, here, means automated programs knocking on every address on the internet and noting which ones answer like a vulnerable Exchange server. Attackers do this cheaply and constantly. They often know what you have facing the internet before you do.

Closing this gap is cheap compared with what it prevents. It means an external attack surface inventory, a continuously updated list of everything the outside world can reach, compared against the internal asset register, and owned by a named person. The useful test is whether the agency can answer, within an hour, which systems expose a login page or service to the internet, what version each runs, and who maintains it. An inventory that leaves out what the internet can see fails exactly when it is needed.

What the inventory has to hold

A list of server names is not enough. For each internet-facing system, the record that would have helped on March 2 holds a small set of fields:

• Addresses and names. Every public IP address and DNS name that reaches the system, including old ones still pointing at it.
• Product, version and build. For Exchange, the cumulative update level, because that decided whether the patch would even install.
• Exposure path. Whether traffic reaches the server directly, through a load balancer or through a reverse proxy, and which ports are open.
• Technical owner and business owner. The person who maintains it and the executive who answers for it, by name.
• Data held. A rough classification, so the response team knows whether a compromise could trigger notification duties.
• Last verified. The date someone last confirmed the record matched reality.

The record has to be checked from the outside as well as the inside. Internal registers record what was bought and installed. External scans record what actually answers. The gap between the two is where forgotten hybrid servers, test systems and contractor installs live.

Federal civilian agencies had tools for this that smaller governments often did not use. CISA’s Continuous Diagnostics and Mitigation program gave agencies dashboards of their assets. CISA also offered a free external vulnerability scanning service, called Cyber Hygiene, to state, local, tribal and territorial governments as well as federal agencies. Many local governments had never signed up. The capability existed. The habit of using it did not.

How the gap differed across governments

In a federal agency, the gap was mostly about scale and sprawl: many components, many networks, and systems run by contractors on the agency’s behalf. In a state with a consolidated IT agency, the central team could often answer for executive-branch departments but not for the legislature, the courts or independent boards.

Counties had a distinctive problem. Many county functions are run by separately elected officials, such as the sheriff, the clerk, the assessor and the treasurer. Each may have its own IT staff or vendor, and some run their own mail. A county IT director asked on March 2 for a list of every Exchange server in “the county” might honestly not know what the sheriff’s office ran.

School districts and special districts, such as water or transit authorities, were often the thinnest of all. Many relied on a single managed service provider. Their exposure inventory, if it existed, lived in the provider’s records, and the question of who owned the response depended on what the contract said.

The objection: inventories never finish

The strongest objection is practical. Asset inventory projects have a long history of running over budget and never reaching completeness. Why would this one be different?

The answer is scope. The internet-facing slice of an estate is far smaller than the whole, and it can be discovered from the outside by the same method attackers use. An agency does not need a perfect register of every laptop to know which addresses answer on the public internet. Starting from the outside and working in produces a bounded, useful list in weeks, and it covers the systems that matter first in an event like this one.

The persistence gap: patching did not remove intruders

The second gap was more dangerous because it was invisible. Attackers who reached a vulnerable server commonly dropped web shells in web-accessible folders that let them return later without the original flaw. Installing Microsoft’s update closed the door but did nothing about anyone already inside. CISA’s directive recognized this by requiring forensic collection and hunting, and by telling agencies to assume compromise if they had been exposed.

“Assume compromise” is a mindset worth pausing on. It means that if your server was reachable and unpatched during the window when attackers were active, you should treat it as broken into until evidence shows otherwise. That reverses the usual burden of proof. Instead of looking for signs that something is wrong, the team has to show that the machine is clean.

For agencies without a security operations function, the response stalled here. A security operations function, often called a security operations center or SOC, is the team or service that watches systems for signs of attack and investigates them. Patching is an operations task with a clear end. Hunting is an investigation with no obvious end, and it needs skills many small IT teams lacked. They include reading IIS logs (the records the web server software keeps of every request it receives), checking for unexpected files in Exchange directories, and reviewing mailbox export activity, which can reveal whether someone copied out whole mailboxes.

Microsoft released a one-click mitigation tool in mid-March aimed at exactly these organizations. In April, the Justice Department announced a court-authorized FBI operation that removed web shells from hundreds of servers whose owners had not done so. A federal law enforcement agency cleaning servers remotely shows where the persistence gap sat. Those were organizations that, weeks after the alarm, still had an intruder’s back door in their mail system.

What a compromise assessment involved

For a team that had never run one, the work followed a recognizable sequence. Each step produced evidence that the next step depended on.

1. Preserve first. Capture a disk image and, where possible, a copy of memory before changing anything. Rebuilding or rebooting first destroys the record of what happened.
2. Read the right logs. Microsoft published the specific Exchange and web server logs that would show each of the four flaws being used, and a script, Test-ProxyLogon, that searched them automatically.
3. Look for planted files. Check the web-published folders for scripts that had no business being there, and compare them against a known clean installation.
4. Follow the credentials. Check for signs that memory was dumped or that administrative accounts were used from the mail server. If credentials were stolen, the problem has left the mail server.
5. Check what left. Review mailbox export requests and unusual outbound transfers, to estimate what data was taken.
6. Decide and document. Record whether the server is clean, rebuilt or retired, who decided, and on what evidence.

Step four is where small agencies were most exposed. Exchange servers have historically held broad permissions in Active Directory, the directory that controls who can log in to what. A compromised mail server can therefore be a route to the whole network. Resetting passwords on privileged accounts, and checking for new accounts or changed permissions, turned a server problem into a domain problem. That is slow, disruptive work, and it is the step most often skipped.

Closing the gap means separating two jobs. The person who patches should not also be the only person who decides the system is clean, for the same reason the person who writes the checks should not reconcile the bank statement. Agencies that had a retainer with an incident response firm, or access to a state-run security operations center, could start hunting the same week. Others waited, and some never hunted at all.

A useful control is a written rule: any critical vulnerability exploited before the patch was available triggers a compromise assessment, with evidence preserved, before the server is declared clean. Writing it down in advance matters, because in the middle of an incident the pressure to declare victory and get back to normal is intense.

Where help came from, and who paid

The routes to outside help varied by type of government. Federal civilian agencies reported to CISA and could draw on its incident response teams. Many states ran security operations centers or fusion centers that offered help to their own agencies and, in some cases, to local governments.

Local governments had other routes. The Multi-State Information Sharing and Analysis Center, then federally funded, offered state and local members threat alerts and incident response support. Many cities and counties held cyber coverage through an insurer or a public-entity risk pool, and those policies typically came with a hotline and a panel of approved response firms. The catch was that the insurer’s route works best when it is set up before the incident. An agency reading its policy for the first time during a crisis loses days.

A retainer sounds like a luxury for a small county. The alternative is negotiating a contract, a scope and a rate with an unfamiliar firm during the busiest week those firms had seen in years. In March 2021, response firms were stretched. Customers with existing agreements went to the front of the queue.

The objection: just rebuild the server

Some practitioners argued for a simpler path: assume the server is compromised, rebuild it from clean media, restore the mailboxes and move on. That is faster than a forensic investigation and needs less specialist skill.

Rebuilding has real merit, and for a server with no sign of credential theft it may be the right call. It fails in two ways if done alone. First, a rebuilt mail server does nothing about stolen administrator passwords or new accounts elsewhere in the network. Second, without preserved evidence, the agency cannot say what data was taken. Many state breach notification laws cover public bodies as well as businesses, and an agency that cannot answer “what was accessed” may face a harder notification decision, not an easier one. Agencies should check with counsel on what their state requires. The defensible approach preserves evidence first, then rebuilds.

The ownership gap: who decides to retire the platform

The third gap was strategic. By 2021, the case for moving agency email to a hosted service was well understood, yet many agencies still ran Exchange on premises. The reasons were rarely technical. Some agencies had data residency concerns, meaning worry about where, physically, the data would be stored. Procurement rules made recurring subscription costs harder to approve than one-time capital purchases. Hybrid configurations were needed for directory synchronization. And, simply, nobody owned the decision.

The procurement point deserves attention from finance leaders. Many public budgets are built so that buying a server every five years is a familiar capital request, while paying a monthly subscription forever looks like a new, open-ended operating cost. The accounting structure favors the option that carries more security risk. No one chose that outcome; it happened because nobody looked at the whole cost.

ProxyLogon turned that indecision into a measurable cost. The servers that were hardest to patch quickly were the ones nobody was accountable for. The incident also gave CIOs a rare, concrete budget argument. The cost of running mail infrastructure includes staff time for emergency patching, forensic work and notification when something goes wrong, and those costs had never appeared in the business case for keeping servers in-house.

Why the servers stayed

Several specific constraints kept public-sector Exchange on site, and each needed a different owner to resolve.

• The hybrid “last server.” For years, Microsoft’s supported setup for organizations that synchronized their directory to the cloud kept an Exchange server on site to manage mailbox attributes. A supported way to remove that last server arrived only in 2022. Many agencies that had “moved to the cloud” still had an internet-facing Exchange server in March 2021.
• Criminal justice data. Law enforcement agencies handling criminal justice information must meet the FBI’s CJIS Security Policy. Some sheriffs and police departments kept mail on premises because they had not confirmed that a cloud offering met it.
• Records and discovery. Public bodies answer public records requests and litigation holds. Retention and search processes built around on-premises journaling and archiving tools were hard to move.
• Telephone integration. Exchange 2019 dropped the Unified Messaging voicemail feature. Agencies whose phone systems relied on it had a reason to stay on older versions, which carried the vulnerable service.
• Budget shape. As above, capital money was easier to find than recurring money.

None of those is a security decision. They belong to the records officer, the sheriff’s technology lead, the telecom manager and the budget director. A security team cannot retire a platform whose blockers sit in four other departments. That is why ownership, rather than technical skill, was the gap.

Who holds the decision in each kind of government

In federal agencies, the Federal Information Technology Acquisition Reform Act of 2014 gave agency CIOs formal authority over IT budgets and investments. The decision had an obvious owner on paper, though large departments with many components often struggled to exercise it.

In states, it depends on whether IT is consolidated. A consolidated state can move executive-branch mail in one program. A federated state, where each department runs its own systems, has as many decisions as departments.

In counties and cities, the decision usually needs the governing board, because it changes the budget. For a county with separately elected officials, a single migration may require agreement among people who answer to different voters. The ownership gap is widest exactly where the exposure gap was widest.

Closing this gap requires a decision record for each legacy platform: a named executive owner, a target state, a date and a funded plan. For a city or county, that may mean moving mail to a government cloud tenant within a budget cycle, with the hybrid server reduced to a management role and placed behind strict network controls. A government cloud tenant is an organization’s own walled-off space inside a cloud service built to meet public-sector security and data-handling requirements.

The objection: the cloud has zero-days too

Hosted email is not risk-free, and the agency still owns identity, configuration and monitoring. A serious critic would go further. Moving every agency to one provider concentrates risk, and a flaw in the hosted service would hit everyone at once. Outages are shared too.

Those points are fair, and they argue for managing the provider, not for running the servers yourself. The patching race is a contest of speed and staffing. A three-person IT team cannot win it reliably against attackers who scan the whole internet within hours. A provider operating at scale can patch its own service without waiting for each customer. The agency’s remaining job, identity, configuration, logging and oversight of the provider, is real work. It is work a small team can plan for, rather than a race it will lose on someone else’s timetable.

The argument here also does not say every agency should have moved its mail on March 3. It says someone should have owned the decision, whatever it was. An agency that kept Exchange on premises deliberately, funded it, staffed it and monitored it was in a defensible position. An agency that kept it by default was not.

The bill that never appears under “email”

ProxyLogon is less a story about servers than a story about accountability, which puts it on the desks of people who never touch a server.

Consider the money. An agency that discovers an intrusion faces forensic investigation fees and possible legal review of what data was exposed. It may have to notify residents or employees. It pays overtime for staff, and it bears the opportunity cost of every project paused while the team responds. Those costs never show up in the line item labeled “email,” but they are real all the same.

Some of those costs fall on budgets that never approved the original decision. Overtime comes from the IT operating budget. Legal review comes from the county attorney’s office. Notification comes from whoever owns the affected data. Insurance claims can raise the next year’s premium or narrow the next policy. The decision to keep an old server running was made in one place, and its costs land in four.

Consider the risk and the regulators. A public body’s mailboxes hold information it has legal duties to protect. A breach can trigger notification obligations, auditor findings and difficult questions from elected officials. For a business, the equivalent questions come from regulators, insurers and customers. In both cases, “we patched promptly” is a weak answer if nobody checked whether the attackers had already arrived.

Public bodies also face an exposure businesses do not. Records about the incident itself may be requested under public records laws, and the agency’s handling of it may be discussed in an open meeting. A decision log that shows who knew what and when protects the agency in that setting. Its absence is hard to explain afterward.

Consider reputation and trust. Residents who learn that a county’s email was compromised rarely care about the technical root cause. They care whether the people in charge knew what they were running and acted sensibly. The three gaps map neatly onto that public judgment: whether you knew what you had, whether you checked that it was safe, and whether anyone had decided what to do about it.

And consider staff. A small IT team asked to patch, hunt and plan a migration all at once, during a crisis, with no outside help arranged in advance, is being set up to fail. Burnout and turnover after major incidents are common, and losing the one person who understands the mail system is its own risk.

The same accounting applies to organizations outside government that ran Exchange themselves. A community hospital, a credit union or a small broker-dealer faced the same three gaps, under different rules. A HIPAA covered entity had to weigh breach notification for any protected health information in mailboxes. A broker-dealer had its own recordkeeping and supervisory obligations around email. The regulators differed. The gaps did not.

Two neighbors, one advisory

Suppose a county and a neighboring city of similar size both run Exchange on premises in early 2021.

At the county, a security analyst owns an external inventory, refreshed weekly by an automated scan. On March 2 she pulls the list within the hour: two Exchange servers, one main and one hybrid, both on known versions. The county has a retainer with an incident response firm, and a written rule says that any exploited-before-patch flaw triggers a compromise assessment. The patch goes on the same day. The firm starts hunting the next morning, preserves images of both servers, and finds and removes a web shell on the hybrid server. The county administrator already has a decision record showing mail moving to a government cloud tenant next fiscal year. The incident costs money and sleep, but it ends.

The county’s hunt also checks the domain. The response firm confirms that no privileged accounts were used from either mail server, resets the service account passwords anyway and records the evidence. When the county attorney asks whether residents must be notified, the answer rests on preserved logs, not guesswork. The board gets a two-page summary within two weeks.

The neighboring city spends three days working out what it has. It patches the main server on day four, misses the hybrid server entirely because nobody remembered it existed, and declares the problem solved. Months later, it learns from an outside party that someone has been reading mail through a back door the whole time.

By then the city’s choices are worse. The server logs from March have rolled over. Nobody can say which mailboxes were read or whether passwords were taken. The city must reset every credential, review months of activity and decide on notification without evidence. The migration it had been deferring now happens under pressure, priced by vendors who know the city is in a hurry.

The difference between the two is not budget or brilliance. The county had decided, before the crisis, who owned each part of the answer. The city’s errors are the familiar ones, and each follows from one of the gaps:

• Treating the patch as the finish line. The patch stops the next intruder, not the last one.
• Relying on memory rather than an inventory. Hybrid and contractor-built servers are exactly the ones people forget.
• Letting the same person patch and certify. The person under most pressure to declare success should not be the only judge of it.
• Destroying the evidence. Rebuilding a server quickly can feel responsible, but without a forensic image first, you may never know what was taken.
• Leaving the platform decision to drift. “We will migrate eventually” has no owner, no date and no money behind it.
• Assuming the cloud ends the job. Moving to hosted email shifts who patches, but identity, configuration and monitoring remain the organization’s responsibility.

Each of the three fixes is mainly an assignment of responsibility, backed by a test that shows whether it is working, rather than a technology purchase. Agencies that came through ProxyLogon well were usually not the ones with the most tools. They were the ones where someone already knew that the internet-facing inventory, the hunt and the platform decision were their job.

What the CIO should be able to answer before the next advisory

You do not need to understand web shells or IIS logs to hold your organization to account. You need clear questions and an expectation of clear answers:

1. If a serious flaw were announced tomorrow in a system we run, how long would it take you to tell me every place we run it that the internet can reach? Can we test that this quarter?
2. Who, by name, owns the list of our internet-facing systems, and does it include the servers left over from past migrations or set up by contractors?
3. When we patch an urgent flaw that attackers were already using, who decides whether we were broken into, and is that a different person from the one who installed the fix?
4. For each older system we still run ourselves, who is the executive owner, what is the target state, what is the date, and is it funded?
5. When we compare keeping a system in-house with moving it to a hosted service, does the cost include emergency patching, forensic work and notification, or only hardware and licenses?

Good answers will be specific: a name, a time, a document. Vague answers, such as “we’d figure it out” or “the vendor handles that,” are the gaps themselves, speaking out loud.

Evidence worth asking for

Answers in a meeting are a start. The stronger test is evidence that exists before anyone asks. Three artifacts cover the three gaps:

• For exposure: the current external inventory, with the date of its last outside scan and the number of entries that have no named owner.
• For persistence: the written trigger for compromise assessments, the response retainer or state service agreement, and the record of the last time either was exercised.
• For ownership: a one-page decision record for each legacy platform, signed by the executive owner, with a date and a funding source.

Two measures make progress visible over time. The first is the time from a public advisory to a complete list of affected internet-facing systems, tested in a drill at least once a year. The second is the share of legacy platforms with a signed decision record. Both are easy to report to a board or council, and both move only when the underlying ownership is real.

A tabletop exercise built on ProxyLogon’s timeline is a cheap way to test all three. Give the team the March 2 advisory, then reveal the March 5 news of mass scanning, then the discovery of a web shell on a server nobody listed. Each reveal tests one gap. Where the team stalls is where the next incident will stall.

With hindsight

ProxyLogon started a pattern rather than ending one. The ProxyShell chain later in 2021 and further Exchange flaws kept on-premises mail servers on attackers’ target lists. For many organizations, the March scramble turned out to be the first of several.

The federal response kept tightening through the spring. CISA issued supplemental direction under the emergency directive on March 31, 2021, requiring agencies to run Microsoft’s new detection tools and to harden their servers. On April 13 it issued a second supplement, after Microsoft’s April updates fixed further serious Exchange flaws that the March patches did not cover. The same day, the Justice Department announced the FBI’s court-authorized removal of web shells from hundreds of servers.

Attribution for the first wave was settled formally in July 2021, when the United States and its allies attributed the Exchange campaign to actors affiliated with China’s Ministry of State Security. That attribution mattered for diplomacy, but for a county IT manager it changed little. The servers had been opened up to criminals of every kind in the days that followed.

The policy response went wider than Exchange. Executive Order 14028, signed in May 2021 after both SolarWinds and the Exchange campaign, pushed federal agencies toward endpoint detection, better logging and zero trust architecture. Those measures speak directly to the persistence gap: they are the tools that let an agency find an intruder after the door has been shut.

Microsoft changed Exchange itself in response. Later in 2021 it added an emergency mitigation service to supported Exchange versions, designed to apply interim protections automatically when a new threat appears. That shortened one part of the patch race for organizations that kept their servers current. It did nothing for servers that had fallen behind, which were the ones that suffered most in March.

The exposure gap eventually got its own federal rule. CISA formalized the point for federal agencies in Binding Operational Directive 23-01 on asset visibility, issued in October 2022. In everyday terms, it turned “know what you have and what faces the internet” from good advice into a requirement, about as clear a signal as government gives that a gap was real.

Money for smaller governments followed, slowly. The Infrastructure Investment and Jobs Act of November 2021 created the State and Local Cybersecurity Grant Program, about $1 billion over four years, administered through states. It required states to pass much of the money to local governments. It was modest against the need, but it was the first federal program aimed squarely at the kind of county that struggled in March 2021.

Hosted email faced its own scrutiny. The Cyber Safety Review Board’s 2024 report on the Storm-0558 intrusion criticized Microsoft’s security culture, a reminder that moving to the cloud shifts risk without removing it. Organizations that had moved mail off premises still had to pay attention. They traded one set of responsibilities for another and became more dependent on how well their provider did its job.

Then the ownership gap closed itself, whether or not anyone decided to close it. Exchange Server 2016 and 2019 reached end of support in October 2025, meaning Microsoft stopped providing routine fixes for them. Microsoft had released a subscription edition of Exchange Server that July for organizations staying on premises, and offered a limited, paid extended security update period for those not ready. That forced the retirement decision many agencies had deferred. Those that had already built a plan made the move on their own timetable. Those that had not made it under a deadline.

The forgotten hybrid server also came back. In August 2025, CISA issued Emergency Directive 25-02 over a flaw in hybrid Exchange configurations, where on-premises Exchange and Microsoft’s cloud service trust each other. An attacker with administrative access to the on-premises server could use that trust to escalate into the cloud environment. CISA also urged agencies to disconnect the “last Exchange server” many kept after moving to the cloud, and required end-of-life servers to be disconnected. More than four years on, the leftover server that the city in our comparison forgot was again the subject of a federal emergency order.

Some things did not hold up. The hope that one painful incident would push every public body off on-premises mail within a year or two proved optimistic; many held on until the support deadline forced the issue. And the idea that a hosted service would remove the need for in-house security attention did not survive Storm-0558 or the hybrid directive.

What held up best was the narrowest finding: organizations that could see their internet-facing assets and had someone accountable for hunting were better placed, wherever their mail ran. The habit matters more than the email system. Know what the outside world can reach. Assume that a fix is not the same as a clean bill of health. And make sure every aging system has a named person who owns its future, before a crisis assigns one for you.