Meltdown and Spectre turned one class of chip flaw into fixes at four layers, each with a different owner
On January 3, 2018, CERT/CC’s first note on Meltdown and Spectre said fully removing the flaw required replacing the processor. The fixes that shipped landed, as we count them, at four layers, each with a different owner and its own conditions.
Looking Back revisits past weeks one at a time, up to the most recent, and asks what it meant for the banks, health systems and public agencies we serve. The series was written in 2026, with the benefit of hindsight, and each article says which week it looks back on.
Two-minute summaryThe dek and the first sentence of each section, quoted from the article
On January 3, 2018, CERT/CC’s first note on Meltdown and Spectre said fully removing the flaw required replacing the processor. The fixes that shipped landed, as we count them, at four layers, each with a different owner and its own conditions.
- The week Wednesday, January 3, 2018, was the day a class of processor flaws became public.
- The processor: a fix that arrived as microcode, through someone else The first layer was the chip itself.
- The operating system: page table isolation and the conditions on installing it The second layer was the kernel.
- The virtualization and cloud layer: a hypervisor someone else might own The third layer was the one between the operating system and the hardware in a virtualized estate.
- The application and browser: where outside code gets in The fourth layer was the code that lets someone run instructions on the machine.
- How exposure differed across banks, health systems and agencies The four layers applied everywhere.
- Objections, in order We take five objections, numbered in the order we address them.
- What a leader could check today These checks follow from the four layers.
- With hindsight The class of flaw did not end in January 2018.
The week
Wednesday, January 3, 2018, was the day a class of processor flaws became public. Google’s Project Zero published a post titled “Reading privileged memory with a side-channel.” It said CPU data cache timing could be abused to leak information out of mis-speculated execution. At worst, it said, that meant arbitrary virtual memory reads across local security boundaries.
The post named three variants. Variant 1 was bounds check bypass, CVE-2017-5753. Variant 2 was branch target injection, CVE-2017-5715. Variant 3 was rogue data cache load, CVE-2017-5754. The first two became known as Spectre and the third as Meltdown. Project Zero said it had reported the issue to Intel, AMD and ARM on June 1, 2017.
The post was specific about what its own proofs of concept did. One variant 1 proof of concept, run with normal user privileges on a modern Linux kernel, could read kernel memory on the Intel Haswell Xeon processor it tested. A variant 2 proof of concept, run with root privileges inside a KVM guest, could read host kernel memory on the same processor, with a specific older host kernel. A variant 3 proof of concept could read kernel memory with normal user privileges, under a precondition the researchers described. Project Zero also said that processor vendors were better placed than it was to design mitigations, and that it expected them to be the source of authoritative guidance.
The CERT Coordination Center at Carnegie Mellon University published Vulnerability Note VU#584653, “CPU hardware vulnerable to side-channel attacks,” dated January 3. The version captured early on January 4 had a short Solution section with two headings. The first was “Replace CPU hardware.” It said the underlying vulnerability was primarily caused by CPU architecture design choices, and that fully removing it required replacing vulnerable CPU hardware. The second heading was “Apply updates.” It said operating system updates mitigated the underlying hardware vulnerability.
That first version was short. Its Description section named the attacks and pointed to the write-ups by Project Zero and by the Institute of Applied Information Processing and Communications at Graz University of Technology. Its Impact section said an attacker able to run code with user privileges could read otherwise protected kernel memory and bypass kernel address space layout randomization. It listed the three CVE numbers but did not yet describe the variants or say which systems carried the highest risk.
Project Zero’s post filled in some of that. It said variants of the issue were known to affect certain processors by Intel, AMD and ARM. It listed the processors it had tested: an Intel Haswell Xeon, two AMD processors and an ARM Cortex A57 core in a phone. It also said other researchers, whom it named, had also reported the issues before public disclosure, and linked their write-ups on Spectre and Meltdown.
The National Cybersecurity and Communications Integration Center, part of the Department of Homeland Security, issued US-CERT alert TA18-004A on January 4. It said the center had become aware of the vulnerabilities on January 3. The copy captured on January 6 told administrators to refer to their operating system vendors. It said that, because the vulnerability existed in CPU architecture rather than in software, patching might not fully address it in all cases.
That January 6 copy also gave two operational warnings. After patching, it said, performance might be diminished by up to 30 percent, so administrators should monitor critical applications and services. It also said impacts to availability at some cloud service providers had been reported as a result of patches to host operating systems. Users who relied on cloud infrastructure should work with their provider on the effects of host patching and mandatory rebooting. The alert carried a table of vendor advisories, which grew through the week.
Microsoft added its own condition. A support article titled “Important: January 3, 2018, Windows security updates and antivirus software” said Microsoft had found a compatibility issue with a small number of antivirus products. Those products made unsupported calls into Windows kernel memory, which could cause stop errors that left a device unable to boot. So Microsoft was offering the January 3 security updates only to devices whose antivirus vendor had confirmed compatibility, by setting a specific registry key. The article said devices without that key would not receive the January 2018 updates or any later security updates.
A hardware flaw sounds like a problem for the hardware maker. In our view, Meltdown and Spectre showed something more useful for a technology leader. A single flaw in the processor became a set of separate fixes, each shipped by a different party and each needing its own decision.
The documents made that visible. CERT/CC first said full removal required new processors; later versions of its note point readers to updates. Project Zero said the processor vendors should be the source of authoritative guidance. US-CERT pointed administrators to their operating system vendors and their cloud providers. Microsoft gated its own operating system fix on a declaration by antivirus vendors. Each party was right about its own piece.
We take the response in four layers. Each is a separate place where a fix had to land. Together they cover the path from the silicon to the user:
- The processor: microcode and firmware from the chip maker, delivered through the system maker.
- The operating system: kernel changes such as page table isolation, and the conditions vendors put on installing them.
- The virtualization and cloud layer: the hypervisor and the host, owned by the organization or by its provider.
- The application and browser: the code that lets an outsider run instructions on the machine, and the compilers that build it.
Our argument is that the response was an ownership problem before it was a technical one. An organization that knew, for each important workload, who owned each of the four layers could move. An organization that tracked the flaw as three CVE numbers on a patch list could not, because no single patch closed it. After the four layers, we look at how this differed across banks, health systems and agencies, at objections, and at what a leader could check today.
Why insist on owners rather than patches? Vulnerability management is built around identifiers. A scanner finds a CVE, a ticket opens, a patch closes it. That model assumes one fix per identifier, from one supplier, applied by one team. Each of the three identifiers here broke at least one of those assumptions. Variant 2 needed two suppliers. Meltdown needed one supplier’s fix and another team’s reboot. Variant 1 needed changes in code the organization might never have thought of as its own.
A note on sources. We rely on the researchers’ own post, CERT/CC’s note and the US-CERT alert in their January 2018 and current versions, Microsoft’s January 2018 support article, and the Linux kernel’s own documentation. We do not assess any vendor’s patch quality, and we name products only where an official document did.
The processor: a fix that arrived as microcode, through someone else
The first layer was the chip itself. CERT/CC’s first version of its note was blunt that the cause was in CPU architecture design choices. A processor in a server cannot be redesigned in place. What can change is the microcode, the low-level instructions the processor loads to control parts of its own behavior.
Two owners for one component
Microcode has an awkward path to a machine. The chip maker writes it. The system maker packages it into firmware updates for its own boards. The operating system may also load microcode at boot. For a server bought from one company with a processor from another, the update could come from either or both routes.
This made the processor layer the hardest to own. A bank’s server team might own the hardware lifecycle. Firmware updates on those servers might sit with the vendor’s support contract. The operating system team might load microcode at boot without knowing which version the firmware already provided. Each team could reasonably assume another had it.
What microcode was needed for
The current version of CERT/CC’s note sets out which fixes depended on the processor. For Spectre variant 2, it says the software mitigation requires CPU microcode updates. For Meltdown, it lists kernel page table isolation, a software change. For Spectre variant 1, it lists compiler changes and browser updates. So the processor layer was required for one variant and not the other two.
That split mattered for prioritization. An organization that waited for microcode before doing anything would leave the Meltdown fix, available as an operating system update, uninstalled. An organization that installed operating system updates and declared victory would leave variant 2 open where microcode was missing.
Embedded and vendor-managed hardware
Some processors were not in machines the organization could update at all. Appliances, network devices, storage controllers and medical or industrial equipment can contain general-purpose processors. Their firmware comes from the device maker, on the device maker’s schedule. US-CERT’s vendor table in January 2018 included storage and network vendors, which signaled that the question reached beyond servers and laptops.
For those devices, the organization’s lever was the contract and the support relationship, not its own patch tooling. The useful question was whether the device ran code supplied by anyone other than its maker. A sealed appliance with no third-party code had a different risk from a general-purpose server, as CERT/CC’s own risk language implied.
The replacement question
CERT/CC’s first advice, to replace CPU hardware, is not in later versions of its note. That reversal is easy to read as a correction. We read it as a statement that two time horizons were in play. In the short run, nobody could replace every processor, so updates were the only route. In the long run, hardware choices would decide how much mitigation an organization had to carry in software.
The current version of the note keeps that second horizon. It says replacing CPUs in deployed systems is not practical. It also says organizations acquiring new systems should evaluate CPU selection in light of the expected longevity of the vulnerability and the performance impact of software patches. That is a procurement statement, and it belongs to whoever writes hardware requirements.
The trade-off is not simple. Replacing servers early costs capital and migration effort. Keeping them means carrying the software mitigations, and their performance cost, for the rest of the servers’ life. For a firm planning a refresh anyway, processor mitigation features became one more line in the requirements. For a firm with years of life left in its estate, the cheaper path was to measure the mitigation cost and plan capacity for it.
Evidence that the layer was done
The record for this layer is a microcode version per physical host, compared with the version the chip maker published for that processor model. For a firmware route, it is the firmware release installed on each server and the maker’s statement of what that release contains. Without that record, the organization can say it applied operating system updates, but not that the processor layer is closed.
Firmware changes also carry a different risk from software patches. A server whose firmware update fails may not boot, and rolling firmware back is harder than removing an operating system update. That argues for staging firmware on a small group of hosts first. It also argues for knowing, before the first update, how to recover a host that does not come back.
The operating system: page table isolation and the conditions on installing it
The second layer was the kernel. For Meltdown, the fix was to separate kernel and user memory more strictly. CERT/CC’s note says the Linux mitigations are known as KAISER and then KPTI, for kernel page table isolation. Windows and other operating systems shipped their own changes.
A fix with a cost
Isolation had a price in performance, and the January documents said so. US-CERT’s January 6 copy said performance might be diminished by up to 30 percent after patching. It told administrators to monitor critical applications and services. CERT/CC’s current note warns that the software fixes can have a negative effect on system performance.
We would not expect the cost to be uniform. A workload that crosses between user and kernel mode at a high rate, through input and output or system calls, should pay more than one that stays in user space. That meant the operating system team could not judge the cost alone. It needed the application owners to say which systems had headroom and which did not.
The antivirus condition
Microsoft’s January 3 article added a condition no patch list would have shown. The January updates would be offered only where the antivirus vendor had set a registry key confirming compatibility. Without the key, a Windows device would receive neither the January updates nor later security updates, according to the article.
This is a clear example of the layer problem. The operating system vendor made its own fix depend on a statement from a third party, the security software vendor. An organization’s endpoint team owned the antivirus product. Its server or desktop team owned Windows updates. A device could fall out of patching entirely because one team’s product had not told another team’s product that it was ready.
The article also covered machines with no antivirus at all. In a default installation of Windows 7 SP1 or Windows Server 2008 R2 SP1, it said, no antivirus application was installed. Microsoft recommended installing a compatible product, or setting the key manually where antivirus could not be run. That put a decision on administrators of older servers that had run without endpoint software.
Server registry settings
The current version of the US-CERT alert adds a later point. For Windows Server, it says registry changes had to be made in addition to installing the patches, and points to Microsoft’s server guidance. So on servers the fix could be installed and not switched on. An operating system team that measured success by patch installation would overstate its coverage.
Sequencing the work
The layers did not have to be fixed in the order they sit in the machine. A sensible order follows exposure. Browser updates were fast to deploy and cut the route by which an outsider’s code reached a desktop. Kernel and hypervisor fixes on shared hosts came next, because those hosts carried the greatest risk in CERT/CC’s own description. Microcode followed as makers published it, staged carefully because of its recovery risk. Single-user servers that ran no outside code could wait for their normal windows.
That order is our suggestion, not a rule. Its value is that it gives each team a place in the queue and a reason for it. Without an order, every team waits for the others, or every team rushes at once into the same maintenance windows.
Rebooting the estate
Kernel fixes need a restart. For an organization with long-running systems, the reboot was the real constraint. Mainframe-adjacent systems, clustered databases, clinical systems and public-facing services all have maintenance windows negotiated with their owners. The kernel fix could not be faster than the slowest window an application owner would grant.
Evidence that the layer was done
On Linux, the kernel’s own documentation describes files under /sys/devices/system/cpu/vulnerabilities that report, per host, whether the system is vulnerable to each Spectre variant and which mitigations are active. A collection job that reads those files across the estate turns a patch claim into a measured state. On Windows servers, where the fix had to be switched on through settings, the equivalent record is the configuration state, not the update history.
The record also has to show exceptions. A host deliberately left without a mitigation, for performance or compatibility, should appear as an accepted risk with an owner and a date. An exception that exists only in an engineer’s memory is not visible to the next audit, or to the next person who inherits the system.
The virtualization and cloud layer: a hypervisor someone else might own
The third layer was the one between the operating system and the hardware in a virtualized estate. CERT/CC’s note, in its current version, says multi-user and multi-tenant systems, including virtualized and cloud environments, likely face the greatest risk. Project Zero’s variant 2 proof of concept had read host kernel memory from inside a guest virtual machine.
Why the host mattered more than the guest
In a virtualized system, several guests share one physical processor. A flaw that lets code read across a boundary on that processor turns a neighbor into a potential reader. Patching the guest operating system protects the guest’s own kernel. It does not, by itself, protect the host or other guests from a hostile guest.
So the hypervisor and the host operating system had to be fixed, along with microcode on the physical machine. In an organization’s own data center, that was the virtualization team. In a public cloud, it was the provider. The customer’s guest patch was necessary and not sufficient.
The provider’s maintenance became the customer’s outage
US-CERT’s January 6 copy named the consequence. It said impacts to availability in some cloud service providers had been reported as a result of patches to host operating systems. It told users who relied on cloud infrastructure to work with their provider on host patching and mandatory rebooting.
For a customer, that meant the provider’s security fix could arrive as the customer’s unplanned restart. A bank running a customer-facing service on cloud instances, or an agency running a benefits portal, could see instances restarted on the provider’s schedule. The applications that recovered cleanly from a restart were the ones designed to.
Shared responsibility, tested in one week
The cloud model had always divided duties between provider and customer. Meltdown and Spectre tested the division at a layer customers do not see. The provider owned the host, the hypervisor and the physical microcode. The customer owned the guest kernel, the applications and the decision whether to keep sensitive workloads on shared hardware.
That last decision is an architecture choice. Dedicated hosts, isolated tenancy options and the placement of sensitive workloads were all choices an organization could make. In our reading, before January 2018 they were priced as compliance or performance options. After it, they were also side-channel options.
Containers on a shared kernel
Containers raised the same question in a different shape. Containers on one host share that host’s kernel. Meltdown let a process in user space read kernel memory, as CERT/CC’s note describes it. So a container’s isolation from its neighbors depended on the host kernel’s fix, not on anything inside the container image. Rebuilding images did nothing for that boundary. Patching and restarting the host did.
For an organization that ran several applications, or several customers, on shared container hosts, that put the platform team on the critical path. The owner of each application could not fix the boundary from inside its own container.
Hyperthreading and shared cores
The kernel’s current documentation adds a point about processor threads. On systems with simultaneous multi-threading, it says, attacks are possible from the sibling thread, because some processor structures may be shared between hardware threads in a core. It describes mitigations an administrator can apply, including turning off a guest’s indirect branch speculation. That turns a performance setting into a security setting, and someone has to own it.
Evidence that the layer was done
For an organization’s own virtualization platform, the record is the hypervisor version and host microcode, per cluster. For a cloud provider, the organization cannot inspect the host. Its record is the provider’s published statement, the date it was read, and any notice of host maintenance the provider sent. A contract that obliges the provider to answer such questions in writing makes that record possible.
Regulated firms have a reason to keep that record even when they trust the provider. An examiner or auditor asking how the firm responded to a processor flaw will expect to see the firm’s own steps, including how it satisfied itself about the provider’s layer. A copy of the provider’s public statement, filed with a note of who read it and when, is a modest piece of evidence. It is better than none.
The application and browser: where outside code gets in
The fourth layer was the code that lets someone run instructions on the machine. CERT/CC’s current note says an attacker would need a valid account or an independent compromise to execute code locally. It also says attacks using JavaScript in web browsers are possible. US-CERT’s current alert describes initial access through malicious JavaScript, malvertising or phishing.
The browser as an execution environment
A browser runs code from any site a user visits. That made it a direct way for an outsider to execute instructions on a desktop, which is what these attacks needed. CERT/CC’s current note lists web browser updates to help prevent exploitation from JavaScript among the variant 1 mitigations.
For a health system or an agency, that put the desktop browser in the same conversation as the data center. Clinical workstations, call center desktops and caseworker laptops all browse. The browser team, if there was one, joined the response alongside the kernel and virtualization teams.
Compilers and the code you build
Variant 1 had a mitigation in the toolchain. CERT/CC’s current note lists compiler changes for it. The kernel’s documentation describes return trampolines, known as retpoline, as a variant 2 mitigation built into the compiled kernel. Software an organization built itself, or bought from a vendor that built it, might need recompiling to benefit.
That reached the software supply chain. An application vendor that shipped binaries built before the compiler changes would need to rebuild them. A bank with in-house trading or payment software would need its own build pipeline updated. The owner here was the development organization, not operations.
Sandboxes and interpreters
Any component that runs untrusted code inside a process is in this layer. The current CERT/CC note describes a Spectre scenario that used the kernel’s eBPF facility to read kernel memory, since eBPF lets user programs run code in kernel space. Scripting engines, plug-in hosts and document renderers raised similar questions. The fix for each came from the owner of that component.
Asking suppliers the right question
The obvious question to a supplier was whether it was affected. For this flaw, nearly every supplier whose product ran on a modern processor could truthfully say yes, or say the fix belonged to someone else. A more useful question named the layer. It asked whether the product runs code from outside parties, and whether it ships its own kernel or firmware. It asked whether the product was built with a compiler carrying the variant 1 and variant 2 mitigations, and when the supplier would confirm all this in writing.
Questions framed that way produce answers an organization can file and compare. They also reveal which suppliers know how their own products are built, which is worth knowing before the next flaw of this kind.
Reducing who can run code
A low-cost mitigation in this layer was reducing who can run code at all. CERT/CC’s current note says single-user systems that do not readily provide a way for attackers to execute code locally face significantly lower risk. Application control, removing unneeded interpreters and limiting browsing on sensitive servers all cut exposure without waiting for any vendor. Those were decisions the organization could make that week.
Evidence that the layer was done
For browsers, the record is the version deployed across the desktop estate, compared with the version that carried the vendor’s mitigation. For software the organization builds, it is the compiler version and flags used for each release, and the date of the first release built with them. For vendor software, it is the vendor’s statement on whether its binaries were rebuilt, and when.
That last record is the hardest to collect. A software contract that did not oblige the vendor to say how its product was built left the customer without an answer. A flaw that needed a rebuild exposed that gap. The fix was a contract clause for the next renewal, which would not help that month and would help with the next flaw of the same kind.
How exposure differed across banks, health systems and agencies
The four layers applied everywhere. How they landed depended on who owned each layer in a given organization.
Regional banks and credit unions
A smaller institution that runs its core systems through a service provider has three of the four layers with that provider. The bank’s work was to ask the provider, in writing, what it had done at each layer, and to record the answer. The bank’s own estate, branch desktops, web servers and back-office servers, still needed its own operating system and browser work.
A side-channel flaw in shared infrastructure tested the institution’s oversight of that provider. The evidence an examiner might ask for was the vendor’s statement and the bank’s follow-up, not a patch report from the bank’s own tools.
Broker-dealers and trading firms
Firms with latency-sensitive systems had a sharper version of the performance question. A kernel fix that added overhead to system calls could matter to a trading or market data system. Those firms had a reason to test before deploying, and a reason to document the test. The risk decision, to delay a fix on a specific system for a stated period, belonged to a named business owner. Where a broker-dealer’s market-facing systems run in dedicated environments, managed by the firm, with little outside code, their exposure is lower under CERT/CC’s own risk description. The firm could document that reasoning alongside any deferral.
Health systems
Hospitals have a large population of devices whose software they do not control. Imaging systems, laboratory analyzers and other connected equipment can contain general-purpose processors. Updates to them come from the manufacturer. A health system’s inventory had to say which devices were vendor-managed, and its biomedical engineering team had to ask each vendor for a position.
Clinical workstations and virtual desktop servers were in the organization’s own hands. Virtual desktops, which put several users’ sessions on shared hosts, sat squarely in CERT/CC’s higher-risk category of multi-user systems. That made the virtual desktop host a priority over a single-user laptop.
Federal, state and local agencies
Agencies run a mix of their own data centers, shared service centers and cloud services. A state that hosts systems for several agencies on shared virtualization faced the multi-tenant case directly. The hosting agency owned the host layer for all of its customers. Each customer agency owned its own guests and applications.
A federal agency on a commercial cloud service is in the same position as a bank: the provider owns the host, and the agency owns its guests and configuration. The January alert’s advice to work with the cloud provider applied equally to a federal tenant and a commercial one.
A county or city with a small IT staff sits closer to the small bank’s position. Its email, records and public-facing systems may run with a managed service provider or a state shared service. Its own work was the desktops, browsers and any servers it ran itself. Its main tool was the question to each provider, in writing, about the layers that provider owned, and a file of the answers.
Fintechs and software vendors
Fintechs and other software firms sat on both sides of the problem. As cloud customers, they owned their guests and depended on their provider for the host. As software suppliers to banks, they owned the fourth layer for their customers: the build of their own code, and the answer to a bank’s question about it. A fintech that could not say how its product was built would face that question from every bank client.
Research and life sciences computing
Research organizations run shared computing clusters where multiple users submit their own code to common hardware. It is the multi-user case in its plainest form. A cluster that accepts jobs from researchers in different projects, or different institutions, relies on the boundary these flaws weakened. Its operators had to weigh mitigation cost against the sensitivity of the data on the cluster, such as patient-level research data.
Organizations with legacy systems
Older operating systems were a separate problem. Microsoft’s article singled out Windows 7 SP1 and Windows Server 2008 R2 SP1 installations without antivirus. Systems that could not take the update, or whose vendors no longer supported them, could not be fixed at the operating system layer. For those, the remaining controls were in the fourth layer: restricting who could run code on them.
Objections, in order
We take five objections, numbered in the order we address them.
1. No attack was reported in the wild, so the response was overblown
The first objection is that the January documents described proofs of concept, not attacks. That is true of what they said. But the response did not need an attack to justify it. The flaw was in the boundary that separates tenants and privilege levels. Organizations that relied on that boundary had to decide what they would do if it failed. Waiting for an attack would also have meant waiting for detection, and we would not expect a side-channel read to leave much trace in ordinary logs. The layer map was the same whenever exploitation came.
2. The vendors would handle it
The second objection is that chip makers, operating system vendors and cloud providers would fix their own layers. Each did ship fixes. But each fix had conditions on the customer’s side: a registry key, a reboot, a microcode load, a recompile, a maintenance window. No vendor could meet those conditions for the customer. The vendors could not see the customer’s estate either. Only the customer knew which of its systems sat on which hosts, which ran old antivirus, and which ran software it had built itself.
3. Patching three CVEs is ordinary patch management
The third objection is that this was three CVEs, handled like any others. The difference was that no single patch closed any of the three everywhere. Variant 2 needed microcode and software together. Meltdown needed the kernel and the reboot. Variant 1 needed compilers and browsers. A process that closed a CVE when one patch was installed would have closed these tickets too early. The tickets would have shown green while the exposure remained. A green ticket over a live exposure is worse than an open ticket, because no one is looking for it.
4. Performance cost made the fix impractical
The fourth objection is that the cost was too high for some systems. For a specific system, that may be a sound decision. It is still a decision, and it needs an owner, a record and a compensating control. CERT/CC’s later language treats the cost as something to weigh against risk, not a reason to skip the weighing. The weighing also depends on the system’s exposure. A server that runs no untrusted code and hosts no other tenants may reasonably carry less mitigation than a shared host that does.
5. Hardware replacement was the real fix, so software work was wasted
The fifth objection takes CERT/CC’s first advice at face value. Replacement could not happen that month at any scale. Software mitigations were the only route available. The current kernel documentation also shows that newer processors carried hardware features to make mitigation cheaper, not that they removed the need for it.
What a leader could check today
These checks follow from the four layers. They are our suggestions, not requirements.
- For each critical workload, can the inventory name the owner of the processor firmware, the operating system, the host or hypervisor, and the code that runs on it?
- Does the patch process close a vulnerability only when every layer that needs a change has one, rather than when the first patch installs?
- Do cloud and hosting contracts say how the provider will notify customers of host-level maintenance, and do critical applications survive an unplanned restart?
- Do vendor-managed devices have a recorded position from their maker on processor-level flaws, with a date?
- Where a fix is deferred for performance, is there a named business owner, a date to revisit and a compensating control?
Each check is cheap to ask and hard to answer for an organization that has never mapped its layers. That gap is the finding.
The checks also need owners. The inventory belongs to the configuration management or asset team, with input from each platform team. The patch process belongs to vulnerability management, but its closing rule needs the agreement of the server, virtualization and desktop teams. Cloud and hosting contracts belong to procurement and vendor management, with the architecture team writing the technical clauses. Device positions belong to whoever runs third-party risk, and in a hospital to biomedical engineering. Deferred fixes belong to the business owner of the system, not to the engineer who deferred them.
With hindsight
The class of flaw did not end in January 2018. The Linux kernel’s administration guide now has a section on hardware vulnerabilities. Alongside Spectre side channels, its index lists L1 Terminal Fault, Microarchitectural Data Sampling, TSX Asynchronous Abort, iTLB multihit and Special Register Buffer Data Sampling. It also lists Processor MMIO Stale Data, Cross-Thread Return Address Predictions, Speculative Return Stack Overflow, Gather Data Sampling and Register File Data Sampling. The list also includes Indirect Target Selection and VMSCAPE, along with a page on old microcode.
That list fits the core of our argument. The flaws kept arriving as processor behavior that software had to compensate for. The guide describes, for these flaws, mitigations that can be selected at compile, boot or run time, and it has a separate page on old microcode. So the layer map stayed relevant.
The kernel’s Spectre page also shows how hardware changed. It lists processors from Intel, AMD, IBM, ARM, Apple and MIPS lines as vulnerable to Spectre. It says x86 processors with Enhanced Indirect Branch Restricted Speculation available in hardware should use that feature instead of retpoline, and that Enhanced IBRS is more efficient. So some later processors carried part of the mitigation in hardware.
The kernel reports mitigation status directly. Files under /sys/devices/system/cpu/vulnerabilities report whether a system is vulnerable to each Spectre variant and which mitigations are active. That gives an organization a machine-readable record per host, which is the evidence an auditor would want.
CERT/CC’s note changed again. Its current version was last revised on January 7, 2022. It no longer tells readers to replace hardware. It says replacing processors in deployed systems is not practical, and that organizations buying new systems should weigh CPU selection against the expected longevity of the flaw and the performance cost of patches. That is the procurement point from our first layer, now in the note itself.
The US-CERT alert, now on CISA’s site and marked as archived content, was revised through May 1, 2018. Its current text no longer gives the “up to 30 percent” figure from the January 6 copy. It says performance impacts may vary depending on use cases. The revision history shows the alert was updated repeatedly in January, with vendor links added as advisories appeared.
One revision bears directly on the operating system layer. On April 27, 2018, the alert added a section on what it called “Total Meltdown,” CVE-2018-1038. It said users running Windows 7 64-bit or Windows Server 2008 R2 64-bit on Intel processors, who had installed Microsoft’s fix in January or February 2018, should install the latest patch immediately. Citing a researcher, it said those earlier patches contained a vulnerability that could allow users and applications to read and write kernel memory. A fix in one layer had opened a hole in the same layer.
The alert also added a note on Spectre variant 2 and Microsoft’s separate update to enable mitigation against it. That reflected the processor and operating system layers moving at different speeds.
For a CIO today, the practical point is that processor flaws now arrive as a series. Each new entry in the kernel’s list asks the same four questions about firmware, kernel, host and code. An organization that can answer them for one entry can answer them for the next with the same map. One that treats each entry as a new emergency pays the coordination cost every time.
What held up is that the response was an ownership problem across layers. What did not hold up was the first week’s sense of a single event. The kernel documentation shows a continuing series. An organization that keeps the layer map and the per-host status current is better placed for each new entry than one that rebuilds them every time.
Spotted an error? Tell us and we will check it and correct the article where needed.