How the AI Act proposal tied risk to the use, whatever the model
The European Commission’s April 2021 AI Act proposal regulated what AI is used for, not how it is built. Its four tiers sorted the AI in banks, hospitals and agencies by business process, so compliance belonged to process owners as much as to data scientists.
Looking Back revisits one week at a time, from January 2020 to today, and asks what it meant for the banks, health systems and public agencies we serve. The series was written in 2026, with the benefit of hindsight, and each article says which week it looks back on.
Two-minute summaryThe dek and the first sentence of each section, quoted from the article
The European Commission’s April 2021 AI Act proposal regulated what AI is used for, not how it is built. Its four tiers sorted the AI in banks, hospitals and agencies by business process, so compliance belonged to process owners as much as to data scientists.
- The week On April 21, 2021, the European Commission, the executive arm of the European Union that drafts its laws, published its proposal for a regulation laying down harmonised rules on artificial intelligence: the AI Act.
- What the proposal counted as AI In everyday business, most of what gets called AI is machine learning : software that learns patterns from historical examples instead of following rules a person wrote down.
- Prohibited practices: narrow, but aimed at the public sector The prohibited list was short.
- High-risk uses: where the work concentrated The high-risk tier carried the heavy requirements, and it was defined by function, meaning what the system does in the business, not how it is built.
- The hard part: drawing process boundaries Because the tier followed function, the hard classification questions were about process boundaries.
- Why Brussels reached Chicago A proposal in Brussels can feel remote from a boardroom in Chicago or a county office in Ohio.
- Transparency duties: telling people they are dealing with AI The third tier imposed disclosure without the full high-risk regime.
- Everything else: unregulated, not unimportant Most AI in use fell into the fourth tier and faced no new obligations.
- An operating model that follows the law’s logic The use-based design suggests an operating model.
- With hindsight The use-based core survived the legislative process largely intact while much else changed around it.
The week
On April 21, 2021, the European Commission, the executive arm of the European Union that drafts its laws, published its proposal for a regulation laying down harmonised rules on artificial intelligence: the AI Act. It was an attempt to write a general law for AI across every sector, and it took a risk-based approach. Rather than regulating the technology as such, it sorted uses of AI by how much harm they could do.
Two members of the Commission presented it: Margrethe Vestager, the executive vice-president responsible for digital policy, and Thierry Breton, the commissioner for the internal market. The package also included a review of the EU’s coordinated plan on AI with member states. It added a proposal for a new machinery regulation, covering the safety of physical products that might contain AI.
The proposal had a long run-up. In April 2019 a high-level expert group appointed by the Commission had published ethics guidelines for trustworthy AI. In February 2020 the Commission published a white paper on AI and consulted on options, including whether to limit binding rules to high-risk uses. A draft of the proposal leaked the week before publication, so its outline was already being debated when the official text arrived.
Take two teams in the same bank, using the same piece of software. One uses it to decide which customers should receive a credit card promotion. The other uses it to decide who gets a loan. Under the proposal, the first team would face no new obligations. The second would face some of the most demanding requirements in the whole law.
At the top, a short list of practices would be prohibited outright, such as social scoring by public authorities, the idea of a government rating citizens on their behavior and treating them accordingly.
Below that came a defined list of high-risk uses. These would face requirements on risk management, data governance, technical documentation, record-keeping, transparency, human oversight, accuracy and robustness. The list, in an annex to the proposal, was grouped into eight areas. They ran from biometric identification and critical infrastructure through education, employment and access to essential services, to law enforcement, migration and the administration of justice. It included AI used to evaluate the creditworthiness of individuals and to decide access to public benefits. It also covered AI in products such as medical devices that already needed third-party conformity assessment, meaning an independent body had to check the product met safety rules before it could be sold.
For most of the listed uses, the developer would check its own system against the requirements and attach the CE mark familiar from European product safety law. It would also register the system in a new EU database. Remote biometric identification was the main exception, with an outside body involved.
A third group of systems, such as chatbots, would face transparency duties: essentially, an obligation to tell people they are dealing with a machine. And most other AI would face no new obligations at all.
Enforcement would run through national authorities, coordinated by a new European Artificial Intelligence Board. The most serious breaches could draw fines of up to €30 million or 6 percent of worldwide annual turnover, whichever was higher. The proposal also invited member states to set up regulatory sandboxes, controlled settings in which new systems could be tested under supervision.
Banks got specific treatment. For credit institutions already supervised under the EU’s banking directive, the proposal folded parts of the new duties into existing internal governance rules. It named the financial supervisors as the authorities responsible for enforcing the AI rules on those institutions.
The same week, regulators across the Atlantic were working from a different starting point. On April 19, the US Federal Trade Commission published business guidance on AI. It warned companies that biased or deceptive uses could breach laws it already enforced, including the FTC Act, the Fair Credit Reporting Act and the Equal Credit Opportunity Act. Weeks earlier, five federal financial regulators had issued a request for information on how financial institutions used AI. The American approach applied existing laws to AI; the European one proposed a new law built around uses.
This was a proposal, not yet law. It would now go through negotiation between the Commission, the European Parliament and the member states. But its basic architecture, regulate the use and not the tool, told every organization operating in Europe where the work would fall.
Most early commentary on the proposal focused on its definition of AI, which was broad enough to cover machine learning, logic-based and statistical approaches. Lawyers and technologists argued over whether ordinary spreadsheets with formulas might somehow count.
That breadth mattered less than it seemed, because the definition alone triggered nothing. Obligations attached to the use. The same gradient-boosted model could be unregulated in a marketing team and high-risk in a lending team.
For regulated organizations, that design had a practical consequence: classification had to be done process by process, by people who understand the processes. The proposal’s four tiers give the structure: prohibited practices, high-risk uses, transparency duties, and everything else. First, a word on what the technology actually is. After the high-risk tier, we look at the boundary problem it created and at why the law reached organizations far from Brussels. We close with an operating model that follows the law’s logic.
What the proposal counted as AI
In everyday business, most of what gets called AI is machine learning: software that learns patterns from historical examples instead of following rules a person wrote down. Show a machine learning system thousands of past loan applications and whether each borrower repaid, and it learns which combinations of facts tend to predict repayment. Then it can score new applicants.
A model is the result of that learning: a kind of recipe, captured in software, that takes inputs (income, existing debts, payment history) and produces an output (a score, a category, a yes or no). A “gradient-boosted model,” the example above, is simply one popular way of building such recipes, by combining many small decision trees, each a short chain of yes-or-no questions, into a single stronger predictor. Executives do not need the mathematics. What matters is that the same kind of model can be pointed at almost any question.
The proposal also counted older logic-based approaches, where experts write rules into software, and statistical approaches, the methods that banks and insurers have used for decades. That is why the definition looked so wide. A lot of software already in use in regulated industries arguably fit.
Why the definition debate was a distraction
The proposal listed these techniques in an annex: machine learning approaches, logic- and knowledge-based approaches, and statistical approaches including Bayesian estimation and search and optimization methods. The Commission would be able to update the annex as technology changed. That made the definition both broad and movable, which is why it attracted so much attention.
For a regulated organization, the sensible response was to inventory broadly and classify narrowly. A bank’s model inventory, kept for model risk management, already listed most statistical and machine learning models. In the United States, supervisory guidance on model risk from 2011, known as SR 11-7, had shaped those inventories for a decade. In Europe, the European Banking Authority’s guidelines on loan origination, published in 2020, set expectations for automated models used in creditworthiness assessment. Those lists were the obvious starting point. The definition question only became decisive for the small number of systems that sat both at its edge and in a listed use.
The proposal’s central insight was that the technical detail does not decide the risk. Like a kitchen knife, a model is harmless or dangerous depending on where it is used and to what end, and that is where the law looks.
Prohibited practices: narrow, but aimed at the public sector
The prohibited list was short. It covered AI that manipulates people through subliminal techniques and AI that exploits the vulnerabilities of specific groups to cause harm. It also covered social scoring by public authorities, and real-time remote biometric identification in publicly accessible spaces for law enforcement, subject to exceptions. That last item means, roughly, live facial recognition by police scanning crowds in public places.
For banks and hospitals, this tier was mostly a check, not a program. Few commercial organizations were doing anything close to these practices.
For public agencies operating in or selling into the EU, it was more pointed. Social scoring by public authorities covered systems that rate people on social behavior or personal characteristics and then treat them unfavorably in unrelated contexts. A benefits fraud model that draws on data far removed from the benefit in question sat uncomfortably close to that line. A model might flag someone for a housing benefit investigation partly because of their record with an unrelated public service. The agency then needs to be able to explain why that is not social scoring. Agencies needed a documented view on where their risk-scoring systems stood.
The ban on live biometric identification carried exceptions that mattered for police forces: targeted searches for victims of crime, including missing children, prevention of a specific and imminent threat such as a terrorist attack, and finding suspects of serious crimes. Each use would generally need prior authorization from a judicial or independent authority. A police force planning to use such a tool would need a process for obtaining and recording that authorization, as well as the tool.
The prohibitions also reached suppliers. A company outside Europe selling a facial recognition service to European police, or a manipulative design feature to European consumers, would be affected even if its customers were the ones deploying it. For a US technology vendor serving public-sector clients on both sides of the Atlantic, product roadmaps would need to account for the European line.
The practical response for an agency was modest but specific. For each risk-scoring system, it would write down what data the system used, why each source was relevant to the decision, and what happened to a person the system flagged. That memo would be the evidence if anyone alleged social scoring. It would also be useful under existing data protection law, which already asked for a purpose for each use of personal data.
High-risk uses: where the work concentrated
The high-risk tier carried the heavy requirements, and it was defined by function, meaning what the system does in the business, not how it is built.
In financial services, creditworthiness assessment of individuals was listed. In healthcare, AI in medical devices fell in through existing product safety law, which tied AI obligations to the device approval process. In government, AI that decides eligibility for public assistance, prioritizes emergency dispatch, or supports certain law enforcement decisions was listed. Recruitment and worker management were listed for every sector, which meant a hospital or bank using AI to screen applicants was in scope even if it used no AI with customers or patients. That last point surprised many organizations: the human resources department turned out to be one of the most exposed.
Healthcare is worth a closer look, because it shows how the proposal piggybacked on existing law. Medical devices in Europe already go through a conformity assessment before they can be sold. Software that diagnoses or guides treatment can itself be a medical device. The proposal said, in effect: if your AI is part of a product that already needs independent checking, the AI requirements will be checked through that same process. For a hospital, whether a tool is a medical device now carries even more weight than before. The answer belongs with clinical and regulatory specialists rather than the IT department alone.
How the list landed sector by sector
For banks, the creditworthiness item had a narrow exception for systems put into service by small-scale providers for their own use. Most credit unions and small lenders in Europe buy scoring tools rather than build them, so the exception mattered less than it might seem. For larger banks, the interaction with banking supervision was the bigger story. Their AI duties would be examined by the supervisors who already examined their credit models.
For health systems, timing was striking. The EU’s Medical Device Regulation began to apply on May 26, 2021, a month after the proposal, having been postponed a year because of the pandemic. Under that regulation, much medical software needs a notified body to assess it before it can be sold. Through the proposal’s product route, AI in such software would become high-risk. Hospitals that build tools for their own use had a narrow exemption under the device regulation, subject to conditions, and needed advice on whether their tools qualified.
For public agencies, the listed uses read like a description of core services: eligibility for benefits, emergency dispatch, migration and asylum processing, and certain law enforcement work. A national benefits agency and a municipal social services department could both be in scope. The municipal one would usually have far fewer staff to do the work.
Employment: the exposure nobody planned for
The employment item was broad. It covered AI used to advertise vacancies, screen or filter applications and evaluate candidates in interviews or tests. It also covered AI used to decide on promotion and termination, to allocate tasks, and to monitor and evaluate the performance and behavior of workers. Many of those functions were embedded in applicant tracking systems, workforce scheduling tools and productivity analytics that human resources teams had bought without thinking of them as AI.
Employers in some member states already faced a second layer. In Germany, for example, works councils have co-determination rights over technical systems capable of monitoring employees. A bank or hospital introducing an AI-based monitoring tool there would need to negotiate with employee representatives as well as classify the system. Human resources leaders were often the last to learn that the proposal concerned them.
The requirements fell mostly on whoever developed the system. They are worth translating one by one:
- A risk management process: a continuing, documented effort to identify what could go wrong and reduce it, not a one-time review.
- Training data that is relevant and representative: the historical examples the model learned from must fit the people it will be used on. A model trained mostly on one population may perform badly, and unfairly, on another.
- Automatic logging: the system keeps its own record of what it did, so decisions can be traced later.
- Technical documentation detailed enough for an authority to assess: a regulator should be able to read how the system works, what it was tested on and where its limits are.
- Human oversight designed into the system: people must be able to understand, monitor and if necessary overrule it, and the design must make that realistic rather than theoretical.
- Declared levels of accuracy and robustness: the developer states how well the system performs and shows it holds up under errors, unusual inputs and attempts to manipulate it.
Users of high-risk systems, the organizations that buy and operate them, had narrower duties, such as following instructions for use, monitoring operation and keeping logs. But many large banks, insurers and health systems build their own models, which makes them developers as well as users.
What the requirements cost to build in
Each requirement translates into engineering and process work. Logging means deciding what the system records for each decision, how long the logs are kept and who can query them. Data governance means documenting where training data came from, what it represents and how it was checked for gaps and bias. Technical documentation becomes a living file, updated whenever the model is retrained.
The proposal also required providers to run a quality management system and to monitor their systems after release. Serious incidents and malfunctions that breach obligations protecting fundamental rights would have to be reported to authorities. For a bank, much of this overlaps with model validation and ongoing monitoring under existing supervisory expectations. For a vendor, it changes the product: documentation and logs become deliverables that customers will ask for in every procurement.
Provider or user: a role that can change
The proposal called developers “providers.” An organization could become a provider by putting its own name on a high-risk system, changing its intended purpose, or making a substantial modification to it. That rule catches buyers who adapt what they buy. A bank that retrains a vendor’s scoring model on its own data may take on the full set of provider duties without noticing. So may an agency that repurposes a vendor’s tool for a listed use.
The role question therefore belongs in procurement and in change control. Before a team modifies a bought system, someone should ask whether the change shifts the organization’s role, and record the answer.
The hard part: drawing process boundaries
Because the tier followed function, the hard classification questions were about process boundaries. Is a model that pre-fills a loan officer’s recommendation a creditworthiness assessment? Or is it decision support outside the list? Is a triage tool in a hospital a medical device, or administrative software?
These are not questions a data science team can answer alone. They need the credit policy owner, the clinical lead and counsel. The data scientist knows what the model calculates. The process owner knows how its output is used, whether staff routinely accept it, and what happens to the customer or patient as a result. That second kind of knowledge decides the tier.
Hospitals face their own version. An emergency department triage score that ranks patients by urgency shapes clinical care directly, and may be medical device software. A bed management model that predicts discharge times affects operations but not the clinical decision about any patient. Between them sit tools such as deterioration alerts, which prompt a nurse to look again. Classifying the middle cases needs the clinical lead and the device regulatory specialist in the same conversation.
Evidence that settles a boundary
Process owners can answer boundary questions with evidence rather than opinion. The first piece of evidence is how often staff change the model’s output. If underwriters accept a suggested credit limit almost every time, the model is effectively making the decision, whatever the procedure says. The second is whether the output reaches the customer or patient directly, or only through a person with real discretion. The third is how much the outcome for the individual would change if the model were wrong.
Override rates, approval workflows and complaint data are the material a supervisor would look at. A human signature on a decision does not by itself take a system out of a listed use. People tend to defer to confident software, a tendency known as automation bias, and regulators know it.
Take a regional lender with three models in its consumer business. One predicts which customers will respond to a mailing. One suggests a credit limit that an underwriter can accept or change. One detects suspicious card transactions. Read against the proposal, the first looks like ordinary marketing, the third looks like fraud detection, and the second is the difficult one: it shapes a credit decision about an individual even though a person signs off. The lender’s data scientists could describe all three models perfectly and still not be able to say which tier the second falls in. Its head of consumer credit could.
The stakes were set high enough to force those conversations. The proposal allowed fines of up to 6 percent of global annual turnover for the most serious breaches. Its reach also extended to providers and users outside the EU where the system’s output was used in the Union. A US lender scoring EU residents, or a life sciences company screening applicants for European sites, could be in scope without any EU infrastructure. As with Europe’s data protection rules, the law follows the people affected, wherever the company sits.
Why Brussels reached Chicago
A proposal in Brussels can feel remote from a boardroom in Chicago or a county office in Ohio. It was closer than it looked, for five reasons.
Money. A fine pegged to global turnover is designed to matter to the largest companies in the world. Even the possibility puts AI governance on the same footing as data protection and anti-money-laundering, issues boards already track.
Customers and citizens. The high-risk list reads like a map of the moments when people are most exposed to an institution: applying for a loan, applying for a job, applying for public help, waiting for an ambulance, receiving medical care. Errors or unfairness in those moments do lasting damage to individuals and to trust in the institution.
Operations. The developer requirements, logging, documentation, oversight, testing, are not paperwork bolted on at the end. They shape how systems are built and run. A model built without logs cannot easily be given them afterward. Organizations that build AI in-house needed to change their engineering habits as well as their policies.
Suppliers. Many organizations buy rather than build. The proposal pushed most obligations onto developers, which made vendor selection a compliance decision. A vendor that cannot produce documentation, explain its training data or support human oversight becomes a liability to its customers.
Reputation. Even outside Europe, the proposal set a reference point. Regulators, journalists and advocacy groups began to ask whether an organization’s use of AI would count as high-risk under the European test, whether or not the law applied.
What the supplier contract should say
Because providers carried most duties, buyers needed contracts that made those duties visible. Useful clauses included access to technical documentation and instructions for use, and a description of the training data and its known limitations. Buyers also needed notice before material changes to the model, the ability to export logs, cooperation with regulators and prompt notification of serious incidents. The contract should also say which party is the provider if the buyer configures or retrains the system.
Many vendors in 2021 could not offer all of this. That was useful information in itself. A buyer could weigh it in selection or set a date by which the vendor would close the gap. It could also decide that a particular use was too important to rest on a supplier that could not explain its own product.
The case for waiting, and why it was weak
The strongest argument against acting in 2021 was that the text would change. Proposals of this size take years to negotiate, the definition was contested, and the list of high-risk uses could grow or shrink. An American institution with modest European exposure could reasonably wonder whether it would be in scope at all.
That argument had force on detail and little on architecture. The work the proposal implied, knowing which systems exist, what each is used for and who owns that use, was reusable under almost any version of the rules. It was also what existing law was starting to demand. The FTC’s April 2021 guidance and the banking agencies’ request for information both assumed an institution could explain how its models affected people. An inventory organized by use served both.
Organizations differ in how far to go. A global bank with European retail customers had every reason to build the full register. A US credit union with no European members needed little beyond what fair lending and model risk practices already required. A US county or state agency was outside the law but faced similar questions from its own legislators and auditors.
Transparency duties: telling people they are dealing with AI
The third tier imposed disclosure without the full high-risk regime. People had to be told when they were interacting with an AI system such as a chatbot, unless it was obvious, and when emotion recognition or biometric categorization was applied to them. Synthetic images, audio or video depicting real people or events had to be labeled; these are what are often called deepfakes.
For customer service and citizen service teams, this was the tier most likely to touch daily operations. A bank’s virtual assistant, a health system’s symptom-checker chat and an agency’s benefits helpline bot would all need clear disclosure.
The cost was low. The design question was how to disclose without degrading the service, and how to offer a route to a person. A clumsy legal notice at the top of a chat window helps no one. A plain line, “You’re chatting with an automated assistant; type ‘agent’ at any time to reach a person,” meets the spirit of the rule and tends to improve trust rather than damage it.
The duties applied to channels a compliance team might overlook. A voice assistant on a contact center phone line is as much a chatbot as one in a web window, and the disclosure has to be spoken. An agency serving residents in several languages needs the disclosure in each of them. Accessibility matters too: a notice that a screen reader skips is not much of a notice.
Emotion recognition deserved separate attention. Some contact center analytics products claimed to infer a caller’s mood from their voice. Under the proposal, people subjected to such a system would have to be told. Many organizations decided the feature was not worth the conversation, and turned it off.
A health system’s symptom-checker was a reminder that the tiers could stack. The chatbot needed a disclosure. If it also suggested what condition a patient might have, it could be medical software under the device regulation, which would put it in the high-risk tier as well.
Everything else: unregulated, not unimportant
Most AI in use fell into the fourth tier and faced no new obligations. Fraud detection, document classification, IT operations analytics and marketing models generally sat here, though the proposal encouraged voluntary codes of conduct.
Two cautions applied. First, systems move between tiers when their use changes. A document classification model reused to route benefits applications can drift into a listed use without any change to its code. Nobody rewrote anything; someone simply found a new job for a tool that worked. Second, other law still applied. Data protection rules, sector supervision and anti-discrimination law did not depend on the AI Act’s tiers. An organization that treated the fourth tier as a safe harbor misread the proposal.
The European data protection regulation was the most important of those other laws. It already gave people rights concerning decisions based solely on automated processing that significantly affect them, and it required a lawful basis and a purpose for every use of personal data. A fraud model outside the AI Act’s high-risk list could still produce a decision that triggered those rights.
The fourth tier still needed some governance. Fraud and anti-money-laundering models generate alerts that lead to frozen accounts and closed relationships. Those consequences are serious for the people affected, and banking supervisors already expected the models to be validated and monitored. The proposal also encouraged voluntary codes of conduct for lower-risk systems. A light version of the high-risk discipline, with documentation, testing and a named owner, cost little and made later reclassification easier.
An operating model that follows the law’s logic
The use-based design suggests an operating model. Each business process owner registers the AI used in their process and states its purpose, and a central team classifies uses and tracks any change of purpose. That puts accountability where the knowledge sits, and it survives changes to the legal text, because every later version kept the link between use and obligation.
In practice, the register does not need to be elaborate. For each system it should record what the system does, which process uses it and who owns that process. It should also record whether the organization built or bought it, which countries’ residents it affects, and its current tier with the reasoning. The central team, usually a mix of risk, legal and data specialists, reviews new entries and any change of use.
Good practice also includes a trigger: any proposal to use an existing model for a new purpose goes back through classification before it goes live. That single habit catches the drift problem described above.
For systems bought from vendors, the register should also note what the vendor has committed to provide: documentation, information about training data, logs, support for human oversight, and notice of significant changes to the model. Those commitments belong in the contract. A buyer that waits until a regulator asks for documentation, and only then asks the vendor, may find it does not exist in a usable form.
Finally, the register is only useful if it is kept current. A quarterly review, with each process owner confirming that their entries still describe how the system is actually used, takes little time and keeps the picture honest.
Who does what
The register works best when it plugs into structures that already exist. In a bank, the business line that uses a model is the first line of defense and owns the entry. The risk function, as second line, challenges the classification. Internal audit tests whether the process works. Model risk management already validates many of the same models, so the AI classification can be added to its existing inventory rather than kept in a parallel spreadsheet.
A health system has no direct equivalent of model risk management, so it usually needs to create one. A clinical AI committee, drawing on clinical informatics, quality, legal, privacy and the medical device specialists, can own classification. Human resources needs a seat for hiring and workforce tools. In a public agency, the program office owns the use, while the chief information officer and the legal office share the central role. Procurement must be involved, because so many agency systems are bought.
Wiring the trigger into change control
The reclassification trigger works only if it is attached to processes people already follow. Most organizations route significant system changes through a change management process and new purchases through procurement. Adding a single required question to each, asking whether the change alters what an AI system is used for or whom it affects, turns the trigger into routine. The answer routes the request to the central team when it is yes, and costs seconds when it is no.
Recording which text you classified against
In 2021 the legal text was a proposal, and everyone knew it would change. A register entry should therefore record which version of the rules its classification relied on, and when it was made. When the final law arrived, the organization could then re-run its classifications against the new text systematically, starting with the entries most likely to move, instead of rediscovering its reasoning from scratch.
Measures a board can follow
A board or risk committee needs a handful of measures, reported quarterly. They include the number of systems registered and the share with a named owner. They add the share of systems with a recorded classification and reasoning, and the number of reclassifications triggered by a change of use. Vendor documentation coverage for bought systems belongs in the set, as do open findings from audit or validation. Trends matter more than levels. A register that never changes is more likely stale than stable.
Scaling to the organization
The effort should match the exposure. A small credit union might have a dozen models, most supplied by its core banking and card vendors, and could keep a short register with vendor documentation attached. A large bank might have thousands of models and need workflow tooling, automated discovery and regular attestation. Neither needs to start with technology. Both need named owners and a classification rule that a business owner can follow.
The failures ran the other way. Some organizations asked the data science team to produce the inventory alone, and got a list of models with no reliable picture of how each was used. Some focused on the definition debate and never started classifying. Some assumed that buying a system from a vendor transferred the compliance problem along with the invoice. And some ignored HR, procurement and other back-office functions where high-risk uses were hiding in plain sight.
What the AI register should already show
- Do we have a single list of the AI systems we use, and does each one have a named business owner?
- Do any of our systems assess individuals’ creditworthiness, screen job applicants, manage workers, decide access to public services or sit inside a medical device?
- Where we are unsure of the tier, who decides, and is the reasoning written down?
- Do any of our systems affect EU residents, even if we have no operations in Europe?
- What happens when someone wants to reuse an existing model for a new purpose?
With hindsight
The use-based core survived the legislative process largely intact while much else changed around it. The idea that obligations follow what AI is used for, rather than how it is built, is still the backbone of the law.
The legislative path took three years. EU member states agreed their position in December 2022. The European Parliament adopted its position in June 2023, after the arrival of ChatGPT had reshaped the debate. The institutions reached a political agreement in December 2023, the Parliament approved the final text in March 2024 and the Council adopted it in May 2024. The AI Act entered into force on August 1, 2024.
Generative AI, the kind that writes text and creates images, prompted obligations for general-purpose AI models that were not in the 2021 text. These are the large, flexible models that can be adapted to many tasks, which the original proposal had not anticipated. They are regulated at the level of the model, a partial exception to the use-based design, while the systems built on them are still classified by use.
The definition changed shape. The final law dropped the 2021 annex listing techniques and adopted a definition close to the one used by the OECD, built around systems that infer from inputs how to generate outputs. The spreadsheet debate faded with it.
The vocabulary shifted too. “User” became “deployer,” with added duties such as fundamental rights impact assessments for public bodies, private entities providing public services, and deployers of credit scoring and certain insurance pricing systems. As a result, some organizations that only operate a high-risk system, rather than build it, must now assess its effect on people’s rights before using it. People affected by certain high-risk decisions also gained a right to an explanation of the role the AI system played.
The final high-risk list added life and health insurance risk assessment and pricing, and carved fraud detection out of the creditworthiness category. A new filter let a provider argue that a listed system is not high-risk when it performs only a narrow procedural or preparatory task, provided it does not profile individuals. That filter made the boundary questions described above more consequential, and it requires the reasoning to be documented.
The prohibitions grew. The final list added, among other things, emotion recognition in workplaces and schools, with limited exceptions, and untargeted scraping of facial images to build recognition databases. The maximum fines rose to €35 million or 7 percent of worldwide turnover for prohibited practices, with lower tiers for other breaches.
The schedule was staggered. Prohibitions applied from February 2025, together with a duty on providers and deployers to ensure adequate AI literacy among their staff. Obligations for general-purpose AI models applied from August 2025, supported by a code of practice published that summer. The high-risk rules were due from August 2026 for stand-alone systems such as credit scoring and from August 2027 for AI inside regulated products such as medical devices.
Those dates then moved. In November 2025 the Commission proposed a digital omnibus, a package of simplifying amendments. In 2026, EU lawmakers agreed to push the main high-risk deadlines back to late 2027 for stand-alone systems and 2028 for product-embedded ones. Organizations should confirm the current dates and transition rules with counsel.
The United States took a different route but arrived at similar questions. There is still no general federal AI law. Federal agencies were instead directed by White House budget office memoranda, in 2024 and again in 2025, to apply extra safeguards to AI uses with significant effects on people’s rights or safety. Colorado enacted a state law in 2024 aimed at high-risk AI used in consequential decisions, and its start date was later pushed back. In each case, the trigger was the use.
Organizations that classified by use early are managing a schedule change, not starting from zero. Their registers already show which systems touch credit, hiring, benefits or medical devices. Adjusting to the final list, insurance pricing in, fraud detection out of the credit category, was an update rather than a project.
Those that waited for the final text, or for the final deadlines, face a harder task: building an inventory under time pressure, while also absorbing new duties for deployers and general-purpose models. The delay to the high-risk deadlines gives them room, but not much.
The broader point holds well beyond Europe. Rules about AI keep changing, but they keep returning to what a system is being used to decide, and about whom. An organization that can answer that for every system it runs is ready for most versions of the rules it is likely to meet.
Spotted an error? Tell us and we will check it and correct the article where needed.