Oldsmar’s disputed hack, undisputed weaknesses

Nobody could later prove whether an outsider or an employee changed Oldsmar’s chemical setting. That uncertainty was itself the finding: small utilities need evidence that tells the two apart, safeguards that work whoever is at the controls, and public statements that survive revision.

Looking Back revisits one week at a time, from January 2020 to today, and asks what it meant for the banks, health systems and public agencies we serve. The series was written in 2026, with the benefit of hindsight, and each article says which week it looks back on.

Two-minute summaryThe dek and the first sentence of each section, quoted from the article

Nobody could later prove whether an outsider or an employee changed Oldsmar’s chemical setting. That uncertainty was itself the finding: small utilities need evidence that tells the two apart, safeguards that work whoever is at the controls, and public statements that survive revision.

  1. The week On February 8, 2021, the Pinellas County sheriff, Bob Gualtieri, held a press conference with city officials.
  2. Setpoints, screens and remote access A water treatment plant takes in raw water from a river, lake or well, cleans it, adjusts its chemistry and pumps it into the pipes that serve homes and businesses.
  3. Why plant computers are different from office computers Leaders sometimes ask why a plant cannot simply be secured the way the town hall’s email is.
  4. Evidence: telling an error from an intrusion The advisories described remote desktop software on control system computers and a password shared among the people who used it.
  5. Safeguards that work whoever acts Officials said the change would have been caught before it affected drinking water, by alarms, water quality monitoring and the time chemicals take to move through the system.
  6. Communication before the facts are settled The first announcement was made quickly, with confidence about what had happened.
  7. Decisions for mayors and boards For a mayor, a county commissioner or a utility board member, Oldsmar can feel remote: a technical problem for technical staff.
  8. A utility that could answer the question A well-run small utility can show that each person who touches the control system has their own login, and that remote access arrives through one recorded entrance.
  9. Proving who changed a setpoint Two checks belong alongside them.
  10. With hindsight In March 2023 Oldsmar’s former city manager said publicly that the event was likely the result of an employee error.

The week

On February 8, 2021, the Pinellas County sheriff, Bob Gualtieri, held a press conference with city officials. He announced that on Friday, February 5, someone had remotely accessed the control system of the water treatment plant serving Oldsmar, Florida. Oldsmar is a city of about 15,000 people near Tampa. The sheriff said the FBI and the Secret Service were assisting his office.

The account he gave sounded like a scene from a thriller. Around 8 a.m. that Friday, a plant operator noticed someone connecting remotely to the computer he was monitoring. He thought little of it, because supervisors routinely used the same software to check on the plant. Early that afternoon, according to the sheriff, it happened again. This time the operator watched the cursor move on its own.

The mouse pointer drifted across the controls and changed a setting for one of the chemicals used to treat the town’s drinking water. According to the public account, the setting for sodium hydroxide was raised from 100 parts per million to 11,100. Sodium hydroxide, also known as lye, is used to manage acidity, so that water does not corrode pipes. In small amounts it is routine. At the higher level it would have been dangerous.

The operator reversed the change at once. Officials said the public was never at risk because of other safeguards, the checks and alarms that stand between a setting on a screen and the water that reaches a kitchen tap. The sheriff said it would have taken 24 to 36 hours for treated water to reach the distribution system, and that alarms would have caught a change in the water’s chemistry before then. The city said it had disabled the remote access program.

A “control system” in this context is the set of computers and equipment that run a physical process: pumps, valves, chemical feeds and sensors. Engineers call this world operational technology, to distinguish it from the information technology of email and spreadsheets. In many plants, an operator manages the process from a workstation that shows the plant as a diagram, with numbers that can be adjusted by clicking. The software that draws that diagram and passes commands to the equipment is often called SCADA, for supervisory control and data acquisition.

Federal and state advisories in the following days described conditions common across the water sector. On February 11, the Cybersecurity and Infrastructure Security Agency (CISA) published a joint advisory with the FBI, the EPA and the multi-state information sharing center. It said the intruders had likely exploited weak password security and an outdated operating system, and that desktop sharing software such as TeamViewer may have been the route in. It did not confirm that.

A notice from Massachusetts officials to that state’s public water suppliers was more specific about the Florida plant. It described remote access software installed on several control system computers, a single password shared among the people who used it, and machines running a 32-bit version of Windows 7. Microsoft had ended support for Windows 7 in January 2020, meaning it no longer fixed security flaws in it. The notice also said the computers appeared to be connected directly to the internet without a firewall, the digital barrier that normally filters what can come in. An FBI notice to private industry pointed, less specifically, to poor password security.

The CISA advisory paired familiar office advice with recommendations specific to water. It urged multifactor authentication, logging of remote connections, and isolation of computers that could not be updated. For water systems, it recommended independent cyber-physical safety systems. These are physical limits such as pump sizing, valve gearing and pressure switches, so that a compromised control system cannot drive conditions to dangerous levels. For remote control software, it advised against unattended access and in favor of requiring the person at the plant to approve anything beyond view-only access.

The incident drew national attention to the security of small water utilities, many of which serve towns and counties with very small teams. It also landed in a sector with known gaps. Federal law already required larger community water systems to assess their risks, including to electronic and control systems, under the 2018 America’s Water Infrastructure Act. Systems the size of Oldsmar’s were due to certify those assessments to the EPA by June 30, 2021.

Oldsmar became shorthand for a frightening scenario: someone on the internet trying to poison a town’s water. For a while it featured in nearly every conversation about critical infrastructure security, in congressional hearings, vendor presentations and board briefings alike.

Then the story shifted. In March 2023 the city’s former manager told a public administration conference that the event was more likely an employee’s mistake than an outside intrusion. The FBI told reporters it had not been able to confirm that a targeted cyber intrusion had occurred. Many people concluded the story had been overblown. Some treated the revision as evidence that worries about utility security were exaggerated.

A more useful conclusion is that two years on, nobody could show conclusively which version was true. A plant where several people share one remote access password, and where actions are not tied to individuals, cannot tell an error from an attack. That finding applies to far more than one Florida plant. It applies to any organization that runs a physical process through shared logins, from a county wastewater system to a hospital’s building controls.

Oldsmar raised three questions: what evidence a utility needs to know what happened, which safeguards work regardless of who acts, and how officials should speak before the facts are settled. Two sections of background come first, on how plants are run and why their computers resist office-style security. The three questions follow in turn. The last three sections turn them into decisions for the people who govern small utilities. They cover what to fund, what a utility that has done the work can show, and how a board can test itself.

Setpoints, screens and remote access

A water treatment plant takes in raw water from a river, lake or well, cleans it, adjusts its chemistry and pumps it into the pipes that serve homes and businesses. Much of that work is automated. Sensors measure things like flow, pressure, acidity and chlorine levels. Small industrial computers, often called programmable logic controllers, read those measurements and open valves, start pumps or adjust chemical feeds to keep everything within target ranges.

Operators supervise the process from a screen that shows the plant as a diagram. Engineers call this screen the human-machine interface. Each target value, such as how much of a chemical to add, is a “setpoint.” Change the setpoint, and the equipment works to reach the new value.

How a setpoint becomes a dose

The chain from screen to tap has several links, and each one is a place where a mistake can be caught or missed. An operator types a new value. The SCADA software writes it to a controller. The controller changes the speed or stroke of a metering pump that injects the chemical. The dosed water then flows through mixing, clearwells and storage tanks before reaching the distribution system.

Along the way, analyzers measure the result, such as the water’s pH. If a reading leaves its normal band, an alarm sounds on the operator’s screen and, in many plants, on an on-call phone. The time water spends in tanks gives operators hours to respond. That delay is why Oldsmar officials could say a dangerous dose would have been caught before it reached homes.

The weakness is that several of these links often run through the same computers. If the workstation that accepts the setpoint also displays the alarms, an intruder or a malfunction on that workstation can affect both. The more independent each link is, the harder it is for one failure to defeat them all.

Why remote access spread

Small utilities rarely have round-the-clock engineering staff. When something goes wrong at night, or when a supervisor wants to check on the plant from home, remote access is convenient. Remote access software lets an authorized person see and control the plant’s screen from a laptop elsewhere, as though sitting in front of it. The same convenience creates the risk. If the same doorway lets in an operator from home, it can let in anyone else who has the key.

Consumer-grade desktop sharing tools made this easy. They are designed to work through firewalls without configuration, so a single install on a plant computer can make it reachable from anywhere. Many offer “unattended access,” which lets a remote user connect without anyone at the plant clicking to accept. That is the feature CISA’s advisory told utilities to switch off.

Vendors added more doorways. A controls integrator that built the plant’s SCADA system often keeps its own remote connection for support. A chemical supplier may monitor tank levels. Over a decade, a small plant can accumulate several remote paths that nobody has listed in one place.

Who does the work

In a large organization, there would be separate teams for networks, security and plant operations. In a town of modest size, the person who runs the plant may also be the person who installs the software, manages the passwords and talks to the vendor. The arrangement reflects how most of the country’s water is delivered rather than any failing of the people involved.

Ownership varies too, and it shapes what is possible. Oldsmar’s plant is run by the city. Other systems are special districts with their own boards, rural water associations owned by their members, or investor-owned utilities regulated by a state public utility commission. A municipal department competes for money with police and parks inside the city budget. An investor-owned utility can ask its commission to recover security spending through rates. A rural association may have no technical staff at all and rely on a circuit rider from a state association.

Why plant computers are different from office computers

Leaders sometimes ask why a plant cannot simply be secured the way the town hall’s email is. The answer lies in how differently the two worlds work.

Lifecycles and change control

Office computers are replaced every few years, updated automatically every month, and can be restarted at lunchtime without anyone noticing. Plant computers often run for a decade or more. They are tied to specialized equipment and software that the vendor has certified for one particular version of an operating system.

Updating them can mean scheduling downtime, testing with the vendor and accepting a small risk that the process misbehaves afterward. For a facility that must deliver water every hour of every day, that is a real constraint. It explains, without excusing, why plants end up running software past its end of support. Upgrading the operating system can also mean upgrading the SCADA software, which can mean new licenses, new screens and retraining. A ten-minute patch in the office becomes a capital project at the plant.

Safety before confidentiality

Priorities differ too. In an office, the first concern is usually keeping information confidential. In a plant, the first concern is keeping the process safe and running. A security measure that locks out an operator at the wrong moment can itself be a safety problem.

For that reason, plant security leans heavily on separating networks, controlling the few ways in, and building physical limits, rather than on frequent updates alone. Account lockouts after failed passwords, automatic screen locks and aggressive endpoint software are routine in offices. On an operator console they need care, because a locked screen during a chemical overfeed is a hazard of its own.

The slow drift onto the internet

Finally, there is the internet. Over the years, many plants were connected to it for good reasons: remote monitoring, vendor support, reporting to regulators. Each connection was a small convenience. Together, without a firewall or a single guarded entrance, they can leave the control system reachable by anyone who finds it.

Attackers routinely scan the internet for exactly these systems, using freely available search engines that list devices answering on the ports industrial equipment uses. A controller or workstation exposed this way is found within days of going online. Nobody needs to target the town by name.

The counter-argument from plant engineers

Plant engineers sometimes argue that office security people overstate the danger. Controllers have run for decades, operators watch them closely, and physical processes have natural limits. There is something to this. Oldsmar’s own outcome shows that human attention and process time are real defenses.

The argument weakens once a plant is reachable from the internet. Natural limits protect against slow, visible drift. They protect less well against a deliberate change made at three in the morning, when the console is unattended and the alarm goes to a phone that is on silent. The right conclusion from the engineers’ point is to formalize those natural limits, which the later sections do, rather than to rely on them unexamined.

Evidence: telling an error from an intrusion

The advisories described remote desktop software on control system computers and a password shared among the people who used it. In that setup, the record of a session says that the shared account connected. It does not say who was behind it, or from where.

The same holds for a building with one key that five people share. If something goes missing overnight, the log of the door opening tells you only that the key was used. It cannot tell you whether it was a colleague who forgot something or a stranger who copied the key. Give each person their own key card, and the question answers itself.

Insiders, former staff and outsiders look alike

The months around Oldsmar showed why the distinction is hard and why it matters. In March 2021, federal prosecutors in Kansas indicted a former employee of a rural water district. They said he had remotely accessed the district’s system in 2019, after leaving the job, and shut down cleaning and disinfection processes. He pleaded guilty later in 2021.

In June 2021, NBC News reported a separate case from January 2021, based on a regional intelligence center report. Someone had used a former employee’s remote access login to reach a water treatment plant serving part of the San Francisco Bay Area and deleted programs used to treat water. The plant was not named, and the report said nobody was harmed.

Each case involved remote access tools and credentials that outlived someone’s employment. In a shared-password plant, a disgruntled former employee, a careless current one and a foreign stranger leave the same trace.

What the investigation itself turned up

The difficulty showed up in the Oldsmar investigation in a second way. In 2021 the industrial security firm Dragos reported that a computer on the city’s network had visited a compromised website on the day of the incident. The site belonged to a Florida water infrastructure contractor, and it carried malicious code. Dragos concluded the visit was a coincidence and unrelated to the plant intrusion.

A utility with poor logs is exposed to exactly that kind of false lead. Without good records, investigators chasing one explanation can find circumstantial evidence for another. Only clear records narrow the field.

Three changes that create the evidence

Three changes give a small utility the evidence it lacked:

  • Individual accounts. Every operator, engineer and vendor technician signs in as themselves, with multifactor authentication for remote sessions. Multifactor authentication means a second proof of identity beyond a password, such as a code from an app on a phone. Staff turnover argues for this, because shared passwords outlive the people who knew them.
  • One remote access path. All remote sessions go through a single gateway that records who connected, from where and when, instead of through software installed on individual control computers. A gateway is a single, guarded entrance in place of several unlocked side doors.
  • Logs kept off the plant network. Session records and changes to control settings are copied to a location an intruder on the plant network cannot alter, and kept for at least a year. Logs stored on the same computers an intruder can reach can be altered by that intruder.

How the pieces fit

The most common pattern is a virtual private network that ends in a small, separate zone, sometimes called a demilitarized zone, between the business network and the plant. Inside that zone a hardened “jump host” is the only machine allowed to talk to the control network. Remote users connect to the jump host, not to the operator workstations, and the jump host records the session.

Vendor access is switched off by default. When the integrator needs to work, an operator enables the account for a set window, watches the session and disables it afterward. The approval, the session recording and the list of setpoint changes together tell the full story.

Setpoint changes need logging in their own right. Most SCADA packages can keep an audit trail of who changed which value, when and from which console, but the feature is often off or keeps only days of history. Turning it on and forwarding it to the off-network store closes the gap between “someone connected” and “someone changed the dose.”

The objections

The control system does not need replacing. The changes do require someone to set it up, and in a utility with a handful of staff, that person may also be running the plant. Shared services help here. A county, a regional authority or a state program can provide a standard managed gateway and logging service that each small system joins, so no plant has to engineer its own.

Operators raise a fair objection about emergencies. At two in the morning, a second login step can feel like an obstacle between an on-call operator and a failing pump. The usual answer is a documented emergency procedure: a sealed credential kept on site, used only when the normal path fails, and reviewed every time it is opened. The exception is allowed, but it is visible.

There is a second benefit. Good evidence also protects staff. An operator accused of a mistake, or suspected of worse, is better served by a clear log than by an argument about what probably happened. In a small community where everyone knows everyone, that protection matters.

Safeguards that work whoever acts

Officials said the change would have been caught before it affected drinking water, by alarms, water quality monitoring and the time chemicals take to move through the system. The operator also noticed in real time. Human attention and engineered limits were the parts of the story that worked, and they work whether the person at the controls is an intruder, a careless employee or a confused one.

Many security programs focus on keeping bad actors out. For a process that can hurt people, the more important question is what stops a dangerous outcome no matter how the instruction arrived. A typo, a misread screen and a criminal all produce the same wrong setpoint.

Three kinds of barrier

Those safeguards deserve the same deliberate design as the digital ones:

  1. Hard limits. Chemical dosing setpoints are bounded in equipment that cannot be overridden remotely, so no screen can command a dangerous value.
  2. Independent sensing. Water quality sensors and alarms that do not depend on the same computers as the controls, so a compromised workstation cannot hide what is happening.
  3. Confirmation for large changes. Procedures requiring a second person, or a local confirmation, before a setpoint moves beyond a normal range.

Where a hard limit can live

Hard limits come in grades. The weakest is a range check in the SCADA software, which rejects values outside a band. It helps against typos but not against anyone who can change the software’s configuration. A stronger limit lives in the controller’s own logic, so the controller refuses or clamps an out-of-range command whatever the screen sends. That logic can still be changed, but only by someone with engineering access to the controller.

The strongest limits are physical. A metering pump sized so that even at full stroke it cannot deliver a dangerous dose for the plant’s flow cannot be talked into one. A chemical day tank that holds less than a harmful quantity limits how much can enter the water in any period. These are the “independent cyber-physical safety systems” CISA’s advisory recommended, and they are the hardest for any digital failure to defeat.

Physical limits have costs. A pump sized tightly for normal flow may struggle during a main break or a spike in demand, when operators need more chemical. The design work has to account for the full operating range, which is why it belongs with the utility’s engineers and its consultants rather than with IT.

Starting from the consequence

The method is to start from the dangerous outcome, here an unsafe chemical dose reaching customers, and work backward through every barrier that prevents it. Idaho National Laboratory has formalized a version of this approach as consequence-driven, cyber-informed engineering. A small utility can apply the core of it without the formal method.

The steps are short. List the handful of outcomes that would harm the public, such as a chemical overfeed, loss of disinfection or loss of pressure. For each, trace the path from a command to the harm. Mark every barrier on that path, and note which barriers depend on the same computer or the same person. Each barrier should be tested on a schedule; one that has never been tested cannot be relied on.

Trade-offs with operations

The trade-off is operational. Limits and confirmations slow legitimate adjustments, so operators need to help set the ranges. A limit set too tight will be bypassed by frustrated staff; one set too loose protects nothing. The people who run the plant every day know where the sensible boundaries lie.

Alarm design carries a similar trade-off. A plant whose alarms sound constantly trains its operators to ignore them. Fewer, better-chosen alarms on the outcomes that matter, routed to someone who will act, beat a screen full of warnings. A pH alarm that reaches an on-call phone at night is worth several that only flash on an unattended console.

Hygiene still matters

The advisories named an unsupported operating system and a direct internet connection. Where replacement takes years, isolating legacy machines from the internet and from business networks reduces exposure while the engineered barriers protect the public. CISA offers free vulnerability scanning to water utilities that ask for it, which shows what an outsider can see from the internet. For a small system, that report is often the fastest way to find a forgotten connection.

Communication before the facts are settled

The first announcement was made quickly, with confidence about what had happened. When that account was questioned two years later, the revision fed doubt about the whole issue of utility security, including the parts that were never in dispute.

Why early statements overshoot

Oldsmar’s incident was announced by the county sheriff, with city and state officials alongside. Several agencies speaking at once makes a confident early account more likely, because each wants to reassure the public. A law enforcement agency naturally frames an event as a crime with a perpetrator. An elected official wants to show control. A utility wants to show its staff did well. Each framing is reasonable on its own, and together they can harden a single explanation before anyone has examined the logs.

Agreeing in advance who leads public statements for a utility incident, and what they will and will not say in the first days, removes that pressure. The agreement can be a page in the utility’s emergency response plan. Plans of that kind were already required of larger community water systems under the 2018 water infrastructure law, so the cost of adding a communications annex is small.

A simple discipline

Officials facing an operational technology incident can protect credibility with a simple discipline:

  • Say what is known.
  • Say what is not yet known.
  • Say which protections worked.
  • Describe the cause as under investigation until the evidence supports a conclusion.
  • Commit to a follow-up statement.

That restraint costs a day of headlines. It preserves trust for the next incident, which may be clearly malicious, and it is far easier when the utility holds logs that can settle the question. Evidence and communication are linked: a utility that can prove what happened can speak with confidence; one that cannot should speak with care.

A holding statement

For a small town, a holding statement on day one might read: “This morning our operators detected and reversed an unexpected change to a treatment setting. Our alarms and monitoring worked, and water quality remained within safe limits throughout. We do not yet know how the change was made, and we have asked state and federal partners to help us find out. We will provide an update by Thursday.”

It is calm, honest and true whichever explanation turns out to be correct. Nothing in it would need to be retracted two years later. Compare that with a statement that names an outside attacker on the first day. If the evidence later points elsewhere, the retraction becomes the story, and the town’s credibility suffers on every later security issue, including real ones.

The case for speaking boldly

There is a serious counter-argument. Naming a possible attack loudly and early can push other utilities to check their own systems, and Oldsmar did exactly that across the country. A cautious statement might have drawn less attention and produced less action elsewhere.

The two aims can be separated. A utility can describe the conditions it found, such as shared passwords and an exposed computer, and urge peers to check for the same, without asserting who was behind the change. The warning travels just as far. The claim that might need retracting is never made.

Who else needs to hear

Public statements are one audience among several. The state drinking water program, which oversees the utility under the Safe Drinking Water Act, expects early notice of anything that could affect water quality. The utility’s insurer usually requires prompt notice of a suspected cyber incident and may assign its own investigators. Sector channels such as WaterISAC exist so that the technical details reach other utilities quickly. Each of these needs facts, not conclusions, in the first days.

Decisions for mayors and boards

For a mayor, a county commissioner or a utility board member, Oldsmar can feel remote: a technical problem for technical staff. Yet the decisions that shape these risks are budget, staffing and governance decisions.

  • Public safety. Drinking water is one of the few services where a technology failure can directly harm residents.
  • Trust. Residents expect their water to be safe without thinking about it. A frightening headline, accurate or not, erodes that trust quickly.
  • Liability and regulators. Officials who cannot show what happened, or who made claims they later had to retract, face hard questions from regulators, insurers and the public.
  • Staff. Without good evidence, employees can end up under suspicion for events they did not cause, or unable to show what they did.
  • Cost. Individual accounts, a shared gateway, off-network logs and tested limits are modest expenses, especially when shared across several small systems.

What size changes

The legal duties differ by size. Community water systems serving more than 3,300 people must assess their risks, including to electronic and control systems, and keep an emergency response plan under the 2018 water infrastructure law. They certify to the EPA every five years. Smaller systems are not required to certify, though nothing stops them from doing the same work. Wastewater systems fall outside that requirement altogether, even though many share staff and control networks with the drinking water side.

A board should know which category it falls in, when it last certified, and whether the assessment actually covered remote access and control systems or treated them in a paragraph. The assessment is the natural home for the questions Oldsmar raised.

Paying for it

Funding paths differ by ownership. A city department will usually need the council to approve the spending, either from the water enterprise fund or a grant. A special district’s board can set rates, but must justify increases to ratepayers. An investor-owned utility can ask its state commission to approve the costs as prudent investments.

Shared services change the arithmetic for all of them. A gateway, a log store and an engineer who knows how to run them cost roughly the same whether they serve one plant or twenty. A county or regional authority that operates them for several small systems spreads the fixed cost. It also gives each plant access to skills it could never hire alone.

A hypothetical district

Suppose a water district serves a few thousand homes with four operators. Its board asks one question after reading about Oldsmar: if a setpoint changed tonight, could we prove who did it? The honest answer is no.

Over the next year, it joins a county-run remote access gateway and gives each operator and its vendor technician their own login. It sends logs to the county and asks its controls vendor to add a hard upper limit on chemical dosing. The total effort is a few weeks of staff time and a modest service fee. The district’s risk of an unexplained incident falls sharply.

The district also gives something up. Operators can no longer check the plant from a personal phone app. The vendor must ask before connecting. A few routine adjustments now take an extra step. The board accepts those costs knowingly, because it has weighed them against the cost of not knowing.

A utility that could answer the question

A well-run small utility can show that each person who touches the control system has their own login, and that remote access arrives through one recorded entrance. Its logs are kept somewhere safe for at least a year. Its dangerous setpoints are physically bounded and alarmed independently. It has agreed who speaks for it in an incident and has a short holding statement ready.

The evidence an inspector would ask for

The proof is in documents, not assurances. A state inspector or an EPA reviewer would expect a current list of remote access paths and who can use each. They would want an account list that matches current staff and vendors, with leavers removed on their last day. Session logs and setpoint change logs should reach back at least a year. A record of the last test of each critical alarm and limit belongs in the file, along with the emergency response plan that names a communications lead.

A utility that keeps these up to date will find its five-year risk assessment far easier. It will also find insurance renewals simpler, since cyber insurers have increasingly asked about remote access and multifactor authentication.

The usual weak points

  • Installing remote access software directly on control computers for convenience, then forgetting it is there.
  • Sharing one password among staff and vendors, and never changing it when people leave.
  • Keeping logs only on the systems an intruder would reach first.
  • Relying on an operator happening to look at the screen as the main safeguard.
  • Announcing a cause in the first hours because the public wants an answer.
  • Assuming a small utility is too obscure to be a target.

Most of these persist for understandable reasons. Remote software was installed during an emergency and never removed. A shared password survived because changing it meant calling every vendor. Each weak point is easy to fix once someone is assigned to it and given the time.

Proving who changed a setpoint

The board’s first question, whether the utility could prove who changed a setpoint tonight and from where, breaks down into five more:

  1. How many people and vendors can reach our control systems remotely, and does each have their own login?
  2. Where are our logs kept, could an intruder change them, and how long do we keep them?
  3. Which dangerous settings are limited in equipment that no screen can override?
  4. When did we last test our water quality alarms, and do they depend on the same computers as the controls?
  5. Who speaks for us if an incident happens, and what would we say in the first 24 hours?

Two checks belong alongside them. The first is whether all remote access runs through one recorded entrance rather than through software installed on individual computers. The second is which control computers run software past its end of support while still reachable from the internet.

Running the test

The test works best as an exercise rather than a questionnaire. A board can ask staff to pick a real setpoint change from last month and reconstruct it from the records. The reconstruction should show who made it, from which console or remote session, and what the process did afterward. If the reconstruction takes an afternoon and ends with a name, the utility is in good shape. If it ends with a shared account and a shrug, the gaps are now visible.

A second exercise tests the barriers. With the vendor present and the plant in a safe state, staff try to enter an out-of-range dosing value and record what stops it. They also confirm that the high and low pH alarms reach the on-call phone. Both exercises take hours, not weeks, and both produce evidence an inspector would accept.

The results belong in a short annual report to the board. Each year the report says which gaps closed, which remain, and what it would cost to close them. Over a few cycles the question Oldsmar raised turns from an alarming hypothetical into a routine line of governance.

With hindsight

In March 2023 Oldsmar’s former city manager said publicly that the event was likely the result of an employee error. The FBI said it had not been able to confirm a targeted cyber intrusion, and the sheriff’s office described the case as still open. Whatever the cause, the conditions the advisories described were real, and later events showed they were widespread.

The federal response came in stages. In January 2022 the White House, the EPA and CISA extended an industrial control systems initiative to the water sector. Its 100-day action plan centered on voluntary monitoring pilots and information sharing. The infrastructure law signed in November 2021 created a state and local cybersecurity grant program, which some states used for shared services to small governments, including utilities.

In late 2023, actors linked by U.S. agencies to Iran’s Islamic Revolutionary Guard Corps compromised internet-exposed industrial controllers at several U.S. water facilities, including one at a water authority in Aliquippa, Pennsylvania. CISA’s advisory said the actors likely used default passwords. A default password is the one a device ships with from the factory. Those incidents involved exactly the kind of exposure Oldsmar had highlighted: control equipment reachable from the internet with weak protection.

In January 2024, a water system in Muleshoe, Texas, saw a storage tank overflow after its control interface was accessed remotely. A Russia-linked hacktivist group claimed responsibility, and the U.S. Treasury sanctioned two of its alleged members that July. In September 2024, Arkansas City, Kansas, moved its water treatment plant to manual operations after a cyberattack, and officials said the water supply was not affected.

Regulators took note. The EPA’s attempt to require cybersecurity review through sanitary surveys, the periodic inspections that drinking water systems already undergo, was issued in March 2023. It was withdrawn that October, after Missouri, Arkansas and Iowa sued, joined by two water industry associations, and a federal appeals court stayed the memo. In March 2024 the EPA administrator and the national security adviser wrote to governors asking states to press their water systems to assess and reduce cyber risks.

The EPA issued an enforcement alert on drinking water cybersecurity in May 2024. It said that more than 70 percent of systems it had inspected since September 2023 did not fully comply with the risk assessment and emergency planning requirements of the Safe Drinking Water Act. Among the weaknesses it named were default passwords, single logins shared by all staff, and access not removed when employees left. Those are the Oldsmar conditions, three years on.

Much of the improvement was left to existing law, voluntary action, state programs and individual utility boards. The five-year cycle of risk assessments came around again in 2025 and 2026, giving boards a scheduled moment to check whether remote access and control systems were covered properly.

By this point in 2026 the gap between what small utilities need and what they can staff remains the central problem. The technical fixes are well understood and not especially expensive. What is scarce is the time and expertise to put them in place in an organization with a handful of people. Shared services run by counties, regional bodies and states matter so much for that reason.

Oldsmar’s lasting value is as a test any utility can apply to itself: if something changed a setpoint tonight, could you prove who did it? A utility that can answer yes has evidence for investigators, protection for its staff and a firm footing for whatever it tells the public. A utility that cannot is relying on luck, and on an operator happening to be watching the screen at the right moment.

Spotted an error? Tell us and we will check it and correct the article where needed.

  • water utilities
  • operational technology
  • remote access
  • logging
  • crisis communication