Rogers: the bank outage that started at someone else’s carrier

The week

On the morning of Friday, July 8, 2022, Canadians across the country picked up their phones and found nothing there. Rogers Communications, one of the country’s largest telecommunications providers, had suffered a nationwide outage of both its wireless network (the mobile phones) and its wireline network (the cables that carry home and business internet and phone lines). It began before dawn in the east and lasted most of the day, and some customers waited longer still.

Rogers attributed the failure to a maintenance update in its core network. The core is the central part of a carrier’s system, the part through which all of its traffic passes. The update caused routing equipment to malfunction. Routers are the machines that decide where each piece of data should go next. When they stopped doing that job properly, traffic simply had nowhere to go. Because Rogers ran its wireless and wireline services through a shared core, both failed together.

What turned a telecom problem into a national banking story was who else depended on Rogers. Interac runs Canada’s domestic debit network, the system that carries a payment from a shopper’s debit card at the till to their bank and back, as well as the popular e-Transfer service for sending money between people. Interac relied on Rogers for connectivity. When Rogers went down, debit payments failed across the country. Shops that could not take debit fell back to cash or credit cards, and customers without either were stuck at the checkout.

The damage did not stop at the till. Some bank branches, ATMs and contact centers were affected. Government services, hospitals and transit systems reported disruptions to phones and connections. More seriously, some calls to emergency services could not connect, which turned the outage into a public safety matter as well as a commercial one.

The response in the weeks that followed

The reaction from authorities was swift. The federal regulator, the Canadian Radio-television and Telecommunications Commission (CRTC), ordered Rogers to explain in detail what had gone wrong. On Monday, July 11, the Minister of Innovation, Science and Industry, François-Philippe Champagne, gave the major carriers 60 days to reach an agreement. It was to cover emergency roaming, which lets one carrier’s customers use a rival’s network during an outage. It was also to cover mutual assistance in a crisis and how to keep the public informed.

Interac said the same day that it would add a second network provider to its system, although it did not name one. Rogers announced it would credit customers the equivalent of five days of service. Its chief executive published a letter committing to separate the company’s wireless and wireline networks so that one failure could not take down both. Later that month, Rogers executives appeared before a House of Commons committee to answer for the outage. Rogers filed its answers to the CRTC over the following weeks, with substantial portions redacted as confidential.

For banks, the weekend after the outage was spent counting the cost and fielding questions. Interac debit is the everyday way Canadians pay in person with money from their bank accounts, and e-Transfer is how many of them pay each other, their landlords and small businesses. Losing both for a business day touched almost every household. To customers, the failure looked like their bank’s, though the fault lay two steps away. Banks had to explain an outage they had not caused and could not fix, using channels that were themselves partly down.

The timing added scrutiny. Rogers was then seeking approval to acquire Shaw Communications, a deal already under review by the Competition Bureau and federal authorities. Critics used the outage to argue that concentration in Canadian telecommunications carried resilience risk as well as pricing risk. It was also not Rogers’ first national disruption: a wireless outage in April 2021 had been tied to a software update.

The banking regulator’s calendar coincided with the event. On July 13, five days after the outage, the Office of the Superintendent of Financial Institutions (OSFI) published its final Guideline B-13 on technology and cyber risk management. It applies to federally regulated banks and insurers and took effect in January 2024. It covers technology operations and resilience, including the dependencies that keep services running. The guideline had been drafted and consulted on before the outage, but the outage gave its resilience expectations an obvious example.

The striking fact for banks was that customers of every Canadian bank lost debit payments, including customers of banks that had never bought a single circuit from Rogers. No amount of carrier diversity on their own side would have prevented that.

Take the finance director of a mid-sized grocery chain on that day in July. Her stores are open, the shelves are full and the staff are on shift. Her bank’s systems are running perfectly. Yet at every checkout, the debit terminal shows an error, and a line of shoppers is deciding whether to leave their carts behind. Every company she pays for technology is working. Something two steps removed from her, and from her bank, has failed.

That is the experience the Rogers outage forced onto Canadian banks and their customers, and it is why the event deserves a careful look from any board or executive team responsible for a financial institution, or for any business that depends on payments.

Bank resilience programs have long listed telecommunications as a critical dependency. In practice that usually meant the bank’s own circuits, and the standard control was to buy them from two carriers. The Rogers outage showed the limit of that control. Customers of every Canadian bank lost debit card payments for much of a business day, including banks with no Rogers circuits at all, because a shared payment utility depended on one carrier. A second carrier on the bank’s data center would not have helped.

Telecom exposure hides in three places that the dual-carrier policy does not reach, and each has a different owner inside a bank. The first is the shared utilities the bank relies on but does not run. The second is the bank’s own lower-tier channels, which often have thinner connectivity than its data centers. The third is the customer’s phone, which modern banking treats as part of the authentication system.

Carriers, circuits and a shared highway

A telecommunications carrier is a company that owns and runs the networks that carry voice and data: the cell towers, the fiber-optic cables under streets, and the equipment in between. When a bank says it has “two carriers,” it means it pays two separate companies to connect its buildings to the outside world, so that if one connection fails, the other keeps working.

A circuit, in this context, is simply a dedicated connection that a bank rents from a carrier, for example a line between a data center and the internet, or between a branch and head office. Banks typically insist on two circuits from two different carriers for their most important sites.

A payment utility is shared infrastructure that many banks use together because it would make no sense for each to build its own. A domestic debit network such as Interac is a good example. When a customer taps or inserts a debit card, the shop’s terminal sends a message through the network to the customer’s bank, which checks the balance and replies with an approval or a decline. The whole round trip takes a second or two. Card schemes, clearing houses and real-time payment systems play similar roles for other kinds of payments. They work like a shared highway: each bank builds its own on-ramp, but everyone drives on the same road.

The weakness the Rogers outage exposed is easy to state. Each bank had checked its own on-ramp carefully. Far fewer had asked what the highway itself depended on. The debit network relied on Rogers for connectivity, so when Rogers failed, the highway closed for everyone at once, regardless of how well-built any individual on-ramp was.

Engineers have a phrase for this: a single point of failure, meaning one component whose failure brings down the whole system. The uncomfortable insight from July 2022 is that a single point of failure can sit outside your organization, outside your vendors and inside an institution you share with your competitors.

What failed inside the carrier

The detailed cause became public later, through an independent review commissioned by the CRTC. Rogers had been upgrading its core network in phases. In the sixth phase, staff removed a routing filter on a set of distribution routers as part of a cleanup. Without the filter, those routers sent a flood of routing information into the core. The core routers had no limit on how much they would accept, so they were overwhelmed and stopped working.

The review also found weaknesses in how the change was approved. Because the first five phases had gone smoothly, the risk rating for the sixth phase had been lowered, and the change skipped extra scrutiny that a higher rating would have required. The audit of the change’s parameters did not flag the error. Rogers afterward set up a separate network for managing and troubleshooting its systems, so that recovery would not depend on the network being recovered.

Bank technology leaders should recognize every element of that story. Change risk scores that fall after a run of successes appear in bank incident reviews too. So do cleanup changes treated as low risk, missing safety limits on critical systems and recovery tools that depend on the system being recovered. The Rogers failure is a supplier’s failure, but its mechanics are a checklist for a bank’s own change management.

Why a shared utility ends up on one carrier

A national payment network has sound reasons to concentrate its connectivity. A single managed network is easier to secure, monitor and support. One provider means one service agreement, one escalation path and one team that knows the whole design. Carriers price large managed networks attractively, and a second full network can come close to doubling the cost.

True diversity is also harder than adding a contract. The utility needs separate physical routes into each of its data centers, equipment that can switch traffic between providers automatically, and routing designs that work across both. It needs regular failover tests in which it moves live traffic from one provider to the other, which carries its own risk of disruption. Those are reasonable engineering choices that a utility’s operators weigh against cost and complexity. The outage showed that the members who depended on the utility had a stake in that choice but little visibility into it.

Shared utilities two layers down

Banks depend on networks they do not operate: card schemes, domestic debit networks, clearing houses, real-time payment systems and the processors that connect to them. Each of those depends on carriers, cloud regions and data centers. A bank’s third-party risk program usually assesses its direct providers. It rarely asks which carrier a shared utility relies on, and it has little power to change the answer.

That is not negligence so much as a natural blind spot. Third-party risk management, the discipline of checking that suppliers are sound, is organized around contracts. A bank sends questionnaires to the companies it pays, reviews their audits and negotiates their service levels. A shared utility often sits in a different category: the bank is a member or participant rather than a customer, and the utility’s own suppliers are two layers away from anything the bank signed.

What a bank can do is plan for the utility’s failure as a scenario rather than a vendor finding. A vendor finding is an item on a risk register that says “this supplier has a weakness.” A scenario is a concrete story that says “this service is gone for a full business day; what do we do at 9 a.m., at noon and at 5 p.m.?” For a domestic debit outage lasting a full business day, the plan covers four things.

• Cash. Whether branches and ATMs can dispense more cash, and whether enough is on hand to meet a surge.
• Authorization. Whether card transactions can fall back to another network for cards that carry one.
• Customer relief. Whether fee waivers or overdraft relief apply automatically for customers caught out, and who approves them.
• Messaging. How the bank will tell customers what is happening through channels that do not share the failed dependency.

Each of those items hides real operational choices. On cash, someone has to know how much is in each branch and ATM, how quickly armored transport can replenish it, and whether withdrawal limits should be raised for the day. On authorization, the card team needs to know in advance which cards carry a second network and whether merchants’ terminals will try it. On relief, the decision about waiving fees should be made before the crisis, with a named approver, so that customers are not charged for a failure that was not theirs. On messaging, the team that runs the website, social media and text alerts must know which of those channels might themselves be down.

How the exposure differs by institution

The shared-utility exposure looks different depending on how an institution reaches the payment systems. A large bank typically connects directly to the main networks and runs its own card processing. It has engineers who can talk to the utility’s engineers and some influence over its design. A credit union usually reaches the same networks through a shared service provider. In Canada many credit unions rely on a central or a cooperative service provider for payment access. In the United States credit unions and community banks commonly rely on core processors, card processors and credit union service organizations.

For those smaller institutions, the dependency chain is one link longer. The credit union depends on its processor, which depends on the network, which depends on its carriers. The scenario planning is the same, but the levers are different. The credit union’s questions go to its processor, and its fallback options are often whatever the processor supports. Contract reviews and processor due diligence are where a small institution can ask about carrier dependency and failover.

Card design matters as well. In the United States, the Federal Reserve’s Regulation II requires debit cards to be enabled on at least two unaffiliated networks. That gives the payment system a second route by design, although both routes may still pass through the same processor and the merchant’s terminal must be set up to use either. Many Canadian debit cards carry a Visa or Mastercard debit function alongside Interac, mainly for online and cross-border use. Whether a co-badged card can complete an in-store purchase when the domestic network is down depends on the card, the terminal and the merchant’s acquirer. Banks should know the answer for their own card portfolio before they need it.

What a member can ask of a utility

A participant in a shared utility is rarely in a position to demand changes alone. It can still ask for information and support collective requests. A useful first request is a description of the utility’s carrier arrangements, at a level of detail that does not compromise security. Others are the results of its most recent failover tests, its recovery time objectives for the services the bank relies on, and its plan for communicating with participants during an outage.

Industry associations and user groups can turn those individual requests into a standard. When several institutions ask for the same evidence, the utility can produce it once. Supervisors are increasingly interested in this kind of collective assurance, because they see the same concentration risk across every institution they oversee.

The merchant end of the same chain

Banks that provide merchant services face the chain from the other side. A merchant’s card terminal needs a connection to reach the acquirer, the bank or processor that handles the merchant’s card payments. Many small merchants, food trucks and market stalls use terminals with a built-in mobile connection on a single carrier. Larger stores often connect through their own broadband. On July 8 some merchants lost the ability to take any card at all, because their own connection ran through Rogers, even where the card networks were working.

An acquiring bank can reduce that exposure for its merchants. Terminals can be supplied with SIMs that can roam onto a second network, or with both a fixed and a mobile connection. Merchant agreements and onboarding guides can explain offline options, such as deferred authorization for low-value transactions where the card scheme rules and the merchant’s risk appetite allow it. Those options carry fraud and credit risk, so they need limits set in advance by the acquirer’s risk team rather than improvised by a merchant at the counter.

Industry bodies are the right venue for the upstream question. A shared utility’s carrier diversity is a collective concern, and members can ask for it to be assessed and reported. Interac’s decision after the outage to add a second network provider is the kind of change that collective pressure produces.

Operational resilience rules were already moving toward this view. Operational resilience is the regulatory term for an organization’s ability to keep delivering its most important services through a disruption, rather than merely recovering its computer systems afterward. The UK’s regime, in force from March 2022, asks firms to map important business services end to end and to stay within impact tolerances in severe but plausible scenarios. An impact tolerance is the maximum disruption a firm says it can allow before customers or markets suffer intolerable harm, for example “no more than a few hours without the ability to pay at a shop.” A shared utility failure is exactly such a scenario.

Canadian supervisors were moving the same way. OSFI was working toward operational resilience expectations, which it later set out in a revised operational risk guideline. Its technology and cyber guideline, published days after the outage, asked institutions to manage technology operations and resilience across their dependencies. Those documents did not single out shared utilities, but a bank applying them honestly would have to account for the debit network.

The bank’s own thinly connected channels

The second place is closer to home. Many banks had dual carriers for data centers but single carriers for smaller branches, ATM networks, back-office sites and contact center telephony. Some circuits that looked diverse on paper shared an upstream provider or a physical route. Remote staff depended on home broadband and mobile hotspots from whatever carrier they happened to use.

The “diverse on paper” problem is worth dwelling on, because it catches experienced teams. A bank may buy one circuit from Carrier A and one from Carrier B and believe it is protected. But Carrier B may rent part of its network from Carrier A, or both cables may run through the same trench under the same road. In that case one backhoe, or one faulty software update, can cut both. Diversity has to be checked at the level of the actual wires and the actual upstream providers, not the names on the invoices.

A carrier diversity review asks, channel by channel, which carriers carry the traffic, whether one carrier failure would take the channel down, and how quickly it could recover. The answers often reveal a few inexpensive fixes, such as cellular backup from a second carrier for branch routers and ATMs. They also reveal a few expensive ones that need a business decision, such as true route diversity for a large operations center.

Consider a regional bank with eighty branches and a few hundred ATMs. Its two data centers each have circuits from two carriers, and its board has been told, accurately, that the bank has “carrier diversity.” A review finds that most branches connect through a single carrier, chosen years ago on price, and that the ATM fleet uses cellular modems from that same carrier. The cheap fix is a small backup modem from a different carrier in each branch and ATM. The expensive question is whether the main operations center, where payments are processed, needs a second physical cable route into the building. The first is a procurement task. The second is a board-level trade-off between cost and risk.

What a diversity review collects

A useful review gathers evidence rather than assurances. For each important site, it asks each carrier for a route map showing the physical path of the circuit from the building to the carrier’s network. It identifies where each cable enters the building, since two carriers sharing one conduit or one entrance room are not diverse at that point. It asks which carrier owns the last mile, because smaller providers often lease the final connection from a larger one.

For cellular backup, the review checks which network the backup actually uses. Some mobile brands are resellers that run on a larger carrier’s network, so a backup modem from a different brand can still depend on the primary carrier. Dual-SIM routers that can switch between two mobile networks reduce that risk. Branch and ATM routers increasingly use software-defined networking that can send traffic over whichever connection is working, which makes failover automatic but needs testing to confirm it behaves as designed.

The review should also cover power and equipment. A backup circuit is of little use if the router that terminates it shares a power supply with the primary, or if the backup has never been switched on in production. Failover should be tested on a schedule, during business hours at low-risk sites, so that the bank knows it works rather than assuming it does.

US banks had long been told to think this way. The FFIEC’s business continuity management guidance, updated in 2019, asks institutions to address telecommunications diversity and the risk that apparently separate services share infrastructure. What the Rogers outage added was scale and visibility: a real case in which the shared infrastructure was the carrier of a national payment utility.

Contact centers deserve particular care because they absorb the customer impact of every other failure. If debit is down and the contact center’s inbound lines run through the failed carrier, customers cannot reach the bank at the moment they most need to. Inbound numbers should be routable across carriers, and agents should be able to work from sites or homes on different networks.

Remote work adds a twist that many continuity plans written before the pandemic do not cover. When agents work from home, the bank’s contact center is only as resilient as the mix of home internet and mobile plans its staff happen to have. If most agents in one city use the same provider, a single carrier failure can remove most of the bank’s phone capacity at once.

ATMs deserve their own line in the review. A bank’s fleet often mixes machines at branches, which share the branch connection, with off-site machines in shops, malls and transit stations. Off-site machines may connect over a cellular modem or over the host location’s network, and some are run for the bank by a third-party operator. Each arrangement has a different carrier dependency and a different owner. On a day when cards fail at the till and customers turn to cash, an ATM that cannot reach the bank is a second failure layered on the first. The review should list every machine, its connection type and its carrier, and give priority to backup connections at the machines customers rely on most.

The fixes for contact centers are well understood. Inbound toll-free numbers can be configured so that calls are delivered over more than one carrier. Cloud contact center platforms can route calls to agents on any connection, which helps if those agents are spread across providers. Some banks issue agents a mobile hotspot on a different carrier from their home broadband. Each of these adds cost and administration. The decision about how much to spend belongs to the business owner of the contact center, informed by how much capacity the bank could afford to lose for a day.

The customer’s phone as an authentication device

The third place received the least attention. Banks had moved customers to mobile apps and to text message one-time passcodes for login and transaction approval. During the outage, Rogers customers could not receive codes, could not use mobile banking away from Wi-Fi, and in some cases could not call the bank. For those customers the bank’s digital channels were down, even though the bank’s own systems were working.

A word on terms. Authentication is how a bank confirms that you are who you say you are. A one-time passcode is the short number a bank texts to your phone when you log in or approve a payment, which you then type in to prove you hold the phone. It is a sensible control, but it also makes the customer’s mobile carrier part of the bank’s security system. If the carrier cannot deliver the text, the customer cannot get in.

That argues for authentication methods that do not depend on a single carrier. App-based authenticators and passkeys work over any internet connection, including Wi-Fi. An app-based authenticator generates codes inside an app on the phone itself, without needing a text message to arrive. A passkey is a newer method in which the phone holds a secret digital key and unlocks it with a fingerprint, face or PIN; there is no code to send at all. Hardware tokens, small physical devices that generate codes, suit some business customers. Fallback procedures in branches and contact centers should let staff verify a customer securely without a text message, using methods that resist social engineering, since fraudsters also exploit outages.

Social engineering means manipulating people rather than breaking into systems: a fraudster calls a contact center, claims to be a customer who “can’t get the text because of the outage,” and asks the agent to bypass the usual checks. Outages are fertile ground for this, because agents are busy, customers are frustrated and everyone wants to be helpful.

There is a trade-off. Every fallback path is also a potential bypass. Recovery procedures need the same fraud controls as primary ones, and staff need clear rules about when the fallback applies. The goal is a second path that is safe enough to use, rather than an emergency exception that becomes a standing weakness.

Inventorying the journeys that depend on a text

Most banks have more text-dependent journeys than they realize. Login and step-up authentication are the obvious ones. Others include adding a new payee, confirming a wire transfer, resetting a password, activating a card, approving an e-Transfer to a new recipient and responding to a fraud alert. A useful exercise lists each journey, the authentication methods it accepts and the share of customers who have only the text option enrolled.

Outbound messages need the same review. Many banks send fraud alerts, low-balance warnings and service notices by text. During a carrier outage, the customers most affected are exactly those whose texts will not arrive. A bank’s outage communications plan should rely on channels that do not share the failed dependency: push notifications to the app over Wi-Fi, email, a banner on the website and login page, messages through the media, and signs in branches. The plan should also say who approves the wording, because a delayed message is often worse than an imperfect one.

Text message codes carry a second weakness alongside carrier dependency. Fraudsters can take over a victim’s phone number through a SIM swap, persuading a carrier to move the number to a new SIM card, and then receive the codes. NIST’s digital identity guidelines had already flagged codes delivered over the phone network as a restricted method with known risks. Moving customers to app-based methods and passkeys addresses both problems at once, which strengthens the business case.

Who cannot move off text codes

The migration has limits. Some customers do not have smartphones, or share a device with family members. Some older customers find app-based methods confusing. Some customers with disabilities rely on text messages as the most accessible option. A bank that removes text codes too quickly will lock out exactly the customers least able to switch.

The practical answer is layered. Text codes remain available but are no longer the only option, and customers are encouraged to enroll a second method. Branches and contact centers can help customers set up alternatives. For high-risk actions, the bank can require a stronger method where the customer has one. Commercial and wealth clients, who often move larger sums, can be offered hardware tokens or passkeys as standard.

Business banking adds its own dependencies. Treasury management platforms often require dual approval for payments, with each approver confirming on a phone. If both approvers use the same carrier and that carrier is down, the business cannot release payroll. Advising business clients to enroll approvers on methods that work over any connection is a simple, valuable conversation for a relationship manager.

Where the consequences land

None of this belongs to the technology department alone, because the consequences land elsewhere.

Customers and reputation. Customers do not distinguish between the bank’s systems, the debit network and their phone carrier. If their card fails at the till, the bank failed them. The reputational cost of a day like July 8 falls partly on institutions that did nothing wrong.

Money. A lost day of debit payments means lost fee income, higher cash handling costs, surge staffing in contact centers and, potentially, refunds and goodwill payments. For the bank’s business customers, such as merchants, it means lost sales.

Regulators. Supervisors increasingly expect firms to understand their critical services end to end and to show how they would stay within impact tolerances. A board that has only been told “we have two carriers” may struggle to answer a regulator who asks what happens when the debit network itself goes down.

Staff. Branch and contact center staff carry the human weight of these events. Clear plans, authority to waive fees and safe fallback procedures make the difference between a hard day and a chaotic one.

Fraud risk. Every outage creates pressure to relax controls. Without pre-agreed rules, that pressure is resolved case by case, by tired people, in the middle of the crisis.

The case for accepting some shared risk

There is a strong argument that banks should accept this exposure. Shared utilities exist because duplicating them would be wasteful and would undermine the interoperability that makes debit cards useful. A bank cannot rebuild the national debit network, and pushing it to diversify adds cost that every participant pays in the end. Day-long national outages are rare. Resilience budgets are finite, and money spent on a rare shared failure is money not spent on more frequent risks such as ransomware.

Much of that is right, and the argument here does not ask banks to duplicate shared utilities. It asks for three things that cost little. The first is a plan for the day the utility fails, because the customer impact lands on the bank regardless of fault. The second is collective pressure for the utility to remove its own single points of failure, which the industry can apply together. The third is attention to the exposures the bank does control: its own channels and its customers’ authentication. Those three steps leave the shared utility shared, while making its failure survivable.

One map, rehearsed

The three places come together in a single map. For each important business service, it shows every telecom dependency across shared utilities, the bank’s channels and the customer’s device, with an owner and a tested fallback for each.

In a well-run bank, that map is a living document rather than a slide. It names a person responsible for each dependency, records when the fallback was last tested and notes what the test found. The business owner of a service, such as the head of cards or the head of retail branches, understands the map and can explain it as well as the technology team can.

Rehearsal matters too. A tabletop exercise, in which executives walk through a scenario around a table and make the decisions they would make on the day, is an inexpensive way to discover gaps. “The debit network is down nationwide from 8 a.m. and nobody knows for how long” is a scenario worth an afternoon of senior time.

Running the exercise

A useful tabletop has the right people in the room: the heads of retail, cards, branches, contact centers, digital channels, fraud and communications, plus technology, risk and a senior executive with authority to make decisions. It runs on a timeline. At 7 a.m., early reports of failed debit transactions arrive. At 9 a.m., branches open to queues. At noon, the outage has no end time and social media is filling with complaints. At 5 p.m., payroll for business clients is due.

At each point, the facilitator asks what the bank does and who decides. The answers expose gaps quickly. Nobody may know the cash position across branches by mid-morning. The fee-waiver decision may need a committee that cannot convene. The customer message may depend on a text alert service that runs on the failed carrier. Each gap becomes an action with an owner and a date.

A few measures make the exercise comparable from year to year. How long it took to agree the first customer message. Whether the bank knew its cash position across branches and ATMs by mid-morning. How quickly the fee-relief decision was made and by whom. Whether the contact center could keep answering calls if the failed carrier also carried its lines. Tracking those few numbers shows whether the bank is getting faster and better prepared, which is what a board and a supervisor will want to know.

The exercise also produces evidence. Supervisors applying operational resilience expectations want to see that scenarios have been tested and that lessons have been acted on. A short report, with the scenario, the decisions, the gaps and the follow-up actions, is exactly that evidence. Repeating the exercise a year later, and showing the gaps closed, is stronger still.

What supervisors will ask to see

Supervisors applying resilience expectations tend to ask for the same artifacts. The first is a list of important business services, such as paying at a store, sending money to another person or reaching the bank by phone. The second is a map of each service’s dependencies, including third parties and the parties those third parties rely on. The third is a stated tolerance for disruption of each service. The fourth is evidence of testing against severe but plausible scenarios, and of the actions taken afterward.

A nationwide debit outage tests all four. Consider an institution whose service list includes in-store payments and whose map reaches through the debit network to its carrier. If it has also set a tolerance and run the scenario, it is in a strong position. One that can show only its own data center diversity is not. The differences by jurisdiction are in the details, such as UK impact tolerances, EU requirements under DORA and OSFI’s critical operations, but the evidence a supervisor asks for is broadly similar.

The scale should match the institution. A large bank may run a full simulation with live systems. A credit union with a dozen branches can run the same scenario in two hours around a table, with its processor on the phone. The value comes from making decisions before the day, not from the size of the exercise.

Banks that come up short tend to do so in the same ways.

• Confusing contracts with resilience. Two carrier contracts do not guarantee two independent paths. Diversity must be verified physically and upstream.
• Stopping at the first layer. Assessing direct vendors while ignoring what shared utilities depend on leaves the largest exposures unexamined.
• Protecting the data center and forgetting the edges. Branches, ATMs, contact centers and remote staff are where customers actually meet the bank.
• Treating the customer’s phone as outside scope. If authentication runs through the customer’s carrier, that carrier is part of the bank’s service.
• Writing fallbacks without fraud controls. An emergency bypass that anyone can invoke is an invitation to fraudsters.
• Planning without deciding. Plans that say “consider waiving fees” rather than naming who decides, and on what basis, fail when the moment comes.

Owners for every point on the resilience map

The map and the exercise come back to a short set of questions that a board or executive committee can ask without technical background. They cover the three hiding places and the decisions that fall between them. Each answer should have a named owner and a date when it was last tested.

• If the domestic debit network were down for a full business day starting tomorrow morning, what would we do about cash, card authorization, fees and customer messaging, and who would decide?
• Which shared payment utilities do our most important services depend on, and do we know which carriers those utilities rely on?
• Which of our branches, ATMs, operations sites and contact centers depend on a single carrier today?
• What share of our customers can log in or approve payments only by text message, and what are we doing to offer alternatives such as app authenticators or passkeys?
• When a customer cannot receive a code, how do our staff verify them, and would that procedure stand up to a determined fraudster?

These questions need no technical background to ask, but each needs someone in the organization to own the answer.

With hindsight

In the months after the outage, Canada’s major carriers signed an agreement on emergency roaming, mutual assistance and public communication during major outages. Thirteen providers signed it in early September 2022, and it took effect on September 9. Interac said it would add a second network provider. Both were sensible, collective fixes to a collective problem: the carriers agreed to help each other’s customers in a crisis, and the payment utility removed its own single point of failure.

Rogers completed its acquisition of Shaw in April 2023, after the federal government approved it with conditions. Rogers committed to physically separating its wireless and wireline cores and told the CRTC the work would cost more than a quarter of a billion dollars. In July 2024 the CRTC published the executive summary of its independent review. It traced the outage to the removed routing filter and to weaknesses in change management and risk assessment, and the CRTC said Rogers had implemented the review’s recommendations. The regulator also introduced outage reporting: interim requirements took effect in 2023, and in September 2025 it set final rules requiring providers to notify authorities of major outages within two hours and to file a report within 30 days.

The episode joined a growing list of shared-infrastructure failures. In November 2023, a nationwide Optus outage in Australia disrupted payments, hospitals and emergency calls after routers were overwhelmed during a software upgrade. The Australian government announced a post-incident review, a Senate committee held hearings, and the company’s chief executive resigned within weeks. In February 2024, a misconfigured network change at AT&T blocked more than 92 million calls in the United States, including over 25,000 attempts to reach 911, according to the FCC’s report. In July 2024, a faulty security software update crashed Windows systems at airlines, hospitals and banks worldwide, again harming organizations that had done nothing wrong themselves.

The parallel is hard to miss. In each case a routine change at a single provider spread outward through organizations that had never thought of that provider as a critical risk to them. And in each case the provider’s own change controls, rather than an attack, were the cause. Recovery also took far longer than the fault itself: a change made in minutes cost customers hours or days.

Regulators continued to formalize operational resilience. OSFI’s Guideline B-13 took effect in January 2024, and OSFI finalized its revised operational risk and resilience guideline, E-21, in 2024, asking institutions to map critical operations and set tolerances for disruption. US banking agencies issued joint guidance on third-party risk management in June 2023. The EU’s Digital Operational Resilience Act applied from January 2025, and UK firms faced a March 2025 deadline to show they could stay within impact tolerances. The UK also put in place a regime for directly overseeing critical third parties to the financial sector, with rules in force from January 2025. The direction of travel is clear: supervisors no longer accept “our systems were fine” as a complete answer when customers could not pay.

On the customer side, passkeys spread across banking apps, reducing reliance on text message codes. That addresses the third hiding place by taking the customer’s mobile carrier out of the login path, and it also closes the SIM-swap route that fraudsters had used against text codes.

By 2026, supervisors expect banks to understand critical dependencies end to end, including those inside shared utilities and on customers’ own devices. Resilience is no longer a property of the systems you own but of the whole chain that delivers a service to a customer, including the parts you share with competitors and the phone in your customer’s pocket. The boards that will handle the next shared failure best are the ones that have already asked, plainly, what happens if the highway closes, and insisted on a tested answer.