Five principles, three committees: the AI blueprint inside a hospital

The week

On October 4, 2022, the White House Office of Science and Technology Policy released the Blueprint for an AI Bill of Rights. It set out five principles for automated systems that affect people’s rights, opportunities or access to critical services. An “automated system” here simply means software that makes, or helps make, decisions about people: who gets flagged, scored, approved, prioritized or contacted.

The five principles were:

• Safe and effective systems: test them before use and keep checking that they work.
• Algorithmic discrimination protections: make sure they do not treat groups of people unfairly.
• Data privacy: protect the personal information they use.
• Notice and explanation: tell people when such a system is involved and how it works.
• Human alternatives, consideration and fallback: let people reach a human who can review or override the result.

A technical companion described how to put the principles into practice, with examples from health care, employment, lending and public benefits.

The office had announced the effort a year earlier and said the blueprint came out of extensive consultation with the public, industry and federal agencies. A White House fact sheet released the same day listed agency actions to go with it. For health care, HHS pointed to its proposed rule under Section 1557 of the Affordable Care Act, published that August, which would prohibit covered health programs from discriminating through clinical algorithms. HHS also promised an evidence review of health care algorithms and racial and ethnic disparities, and the Department of Veterans Affairs described an ethics framework for the use of veteran data. Other agencies covered credit decisions, hiring, tenant screening and education.

The Section 1557 proposal mattered more to hospitals than the blueprint did, because it would bind. Section 1557 applies to health programs and activities that receive federal financial assistance, which reaches most hospitals through Medicare and Medicaid. The proposal did not ban clinical algorithms. It would make a covered entity responsible if its decisions, made using an algorithm, discriminated on a protected basis. Comments on the proposal closed that October, so health system leaders reading the blueprint were also weighing a rule that could make one of its principles enforceable.

Reaction to the blueprint itself split along familiar lines. Some industry groups worried it would chill useful technology or confuse developers with a second set of expectations. Some advocates said a voluntary document without enforcement would change little. Both readings assumed the blueprint’s value lay in its legal force, which it did not have.

A week earlier, on September 28, the Food and Drug Administration had issued final guidance on clinical decision support software. It explained which of these tools fall outside the FDA’s device rules under criteria Congress set in the 21st Century Cures Act. The guidance took a narrower view than many developers had hoped. Software that alerts clinicians to a time-critical condition, such as possible sepsis, was generally treated as a regulated device. Hospitals therefore received two documents in the same fortnight that bore on the same tools from different directions: one on safety regulation, one on rights.

By 2022, software was already helping to run American hospitals. Programs scanned patient records and raised alerts when someone might be developing sepsis. Others scored which patients were likely to deteriorate overnight, which might be readmitted, or which might miss an appointment. Most patients had no idea. Many executives did not either.

The blueprint was explicitly nonbinding and created no legal obligations. It was guidance, not regulation: a statement of expectations, without penalties for falling short. That drew criticism, but it also gave organizations room to decide how to respond.

The timing matters. Large language models, the kind of AI that can write fluent text and hold conversations, were not yet in wide public use. ChatGPT was released at the end of the following month. The blueprint was written for the predictive models and scoring tools of the time, before generative AI changed the public conversation.

Take the chief medical officer of a mid-sized health system reading the blueprint in October 2022. Her inbox already holds proposals for an “AI steering committee,” an “AI ethics board” and an “AI center of excellence.” Each would need members, a charter, meeting time and a budget. Each would run alongside the committees she already chairs. She wonders whether the hospital really needs another layer, or whether it already knows how to do most of this.

That second instinct turned out to be the more useful one. This article first describes what is being governed, then shows how the blueprint’s principles map onto disciplines hospitals already practice. It takes the three functions that do the work in turn: validation, equity monitoring, and notice with a human fallback. It follows one model from purchase to bedside, sets out what is at stake, and ends with the one genuinely new artifact the approach needs.

Sepsis alerts, risk scores and no-show models

First, what is being governed. The tools in question in 2022 were mostly predictive models. These are programs that look at patterns in large amounts of past data and use them to estimate the likelihood of something happening to a current patient.

A sepsis alert, for example, watches a patient’s vital signs, lab results and notes, compares them with patterns seen in past patients who developed sepsis, and raises a flag when the resemblance is strong. A readmission risk score estimates how likely a patient is to be back in hospital soon after discharge. A no-show model predicts which patients might miss appointments so staff can send reminders.

A model works like a weather forecast. It does not know what will happen; it estimates likelihood based on patterns. And like a forecast, a model built in one place can be less accurate somewhere with a different climate. A model trained on one hospital’s patients may perform differently on another’s.

By 2022 many health systems ran predictive models embedded in their electronic health records, the digital systems that hold patient charts, or bought from vendors: sepsis alerts, deterioration scores, readmission risk, no-show prediction. These tools are often called clinical decision support: software that offers information or suggestions to help clinicians decide, without making the decision for them.

Where the models come from

Models reach a hospital by three routes, and each needs a different kind of oversight.

• Features inside the EHR. Large record vendors ship predictive models as part of their software. A hospital can switch them on with configuration rather than a purchase, which is why they often escape procurement review.
• Third-party products. Specialist vendors sell models for imaging, deterioration, documentation or operations, connected to the EHR through interfaces. These go through purchasing, but contract terms rarely cover performance.
• Homegrown models. Academic medical centers and larger systems build their own, often in research groups that later hand them to operations without a clear owner.

Administrative models matter as much as clinical ones. Tools that prioritize collections, flag claims, schedule appointments or screen for financial assistance shape who gets what, even when no clinician sees them.

What the evidence showed by 2022

Concern about performance was not theoretical. In 2021, researchers at Michigan Medicine published an external validation of a widely used proprietary sepsis prediction model in JAMA Internal Medicine. Across more than 38,000 hospitalizations, the model’s ability to distinguish patients who developed sepsis from those who did not was modest. It missed about two-thirds of sepsis cases and generated alerts on about 18 percent of hospitalized patients. The study did not show the model was useless everywhere. It showed that a hospital could not know how a model performed on its own patients without testing it.

Different organizations, different capacity

The governance problem looks different by type of organization. An academic medical center has data scientists, informaticists and research infrastructure, and can validate models itself. A community hospital may have one analyst and rely on its EHR vendor for everything. An ambulatory network runs many outpatient tools, such as no-show prediction and outreach lists, with less clinical risk per tool but large numbers of patients. A critical access hospital may run whatever comes with its record system. The principles are the same for all of them. The practical question is which parts each can do alone and which need vendors, peers or partners.

An old discipline under a new name

The blueprint drew criticism from several directions: toothless because it was voluntary, too vague to implement, too broad for the range of systems it covered. For health systems, a more useful reading was available. Most of its principles described disciplines clinical medicine already practiced under other names.

• Drugs and devices are evaluated before use and watched afterward: “safe and effective.”
• Quality programs compare outcomes across patient groups: the core of discrimination protection.
• Clinicians can override decision support: “human fallback.”
• Patients are told what is being done to them: “notice.”

That suggested a governance design. Rather than building a new AI committee from scratch, a health system could assign the principles to the bodies that already owned the matching discipline. Data privacy belongs with the existing HIPAA program, the system of policies and staff that already protects patient information under federal health privacy law. The other four principles fall to three functions: validation, equity monitoring, and notice with a human fallback. Each has a natural home and a different set of skills.

The appeal for executives is practical. Existing committees already have authority, trust among clinicians, meeting rhythms and reporting lines to the board. A new body has to earn all of that from nothing, and often ends up debating principles while deployment decisions are made elsewhere.

The bodies that already exist

Most hospitals already have the relevant bodies. The pharmacy and therapeutics committee governs medicines. A value analysis or technology assessment group reviews new devices and equipment. Many systems have a clinical decision support committee that approves alerts and order sets. The quality and patient safety function tracks outcomes and investigates harm. The institutional review board governs research. The medical executive committee holds final clinical authority under the medical staff bylaws. Each of these already makes decisions about things that affect patients, with procedures clinicians recognize.

Where oversight reports

Extending existing bodies also settles the reporting line. Clinical committees answer to the medical executive committee, which answers to the board through its quality committee. Hospital boards already receive quality and safety reports and are accountable for them under accreditation and Medicare conditions of participation. Model oversight that flows through the same channel reaches the board in a form directors already know how to read. A separate AI council needs its own reporting line, and boards then receive two pictures of patient safety that may not agree.

The strongest objection

The best argument against this design is that AI really is different. Models can be opaque in a way drugs are not. They change when the data changes. They scale instantly across thousands of patients. Existing committees lack data science skills and may move too slowly for software that updates every quarter.

Each point is fair, and each argues for adding skills to existing bodies rather than replacing them. A pharmacy committee did not know pharmacogenomics when it emerged either; it added members and expertise. The authority to approve, pause and retire is the scarce resource, and it already lives in the existing structure. A data science lead can sit on the committee. A new AI board without that authority can only advise.

Validation: the formulary model

The safe and effective principle asked that automated systems be tested before deployment and monitored afterward in the setting where they are used. That last phrase matters. Published studies had shown that some widely used models performed worse in practice than their developers reported. A model can look excellent in the developer’s tests and disappoint on a particular hospital’s patients.

Borrowing from the pharmacy committee

The closest existing analogue is the pharmacy and therapeutics committee. It decides what goes on the formulary, the hospital’s approved list of medicines, under what conditions, and when something comes off. Clinicians trust it because it is rigorous, clinically led and accountable.

A model governance function can work the same way. It:

• approves a model for a defined use, not for any use someone might think of later;
• requires local validation on the health system’s own patients before go-live, checking that the model works here as well as in the brochure;
• sets thresholds that trigger review, such as a drop in accuracy beyond an agreed level;
• has authority to pause or retire a model.

Performance drifts as populations, practices and documentation habits change, so validation is a schedule, not an event. If clinicians start recording a symptom differently, or the patient population shifts, a model that once performed well can degrade without anyone noticing. This is sometimes called model drift, and it is why a one-time approval is not enough.

What to measure

A single accuracy figure hides what clinicians care about. A useful validation report covers several distinct measures:

• Sensitivity: the share of patients who go on to have the event that the model flagged in advance.
• Positive predictive value: the share of alerts that turn out to be right.
• Alert rate: how many alerts fire per day on a typical unit.
• Lead time: how much warning the model gives before clinicians would have noticed anyway.
• Calibration: whether a predicted 20 percent risk means roughly one in five patients.

The alert burden question is the one most often skipped. A model that is accurate on paper but fires on one in five patients will be ignored on a busy ward. The committee should approve a threshold and an expected alert rate together, because changing one changes the other.

Silent running and its limits

Running a model in the background, with results hidden from clinicians, is the standard first step. It shows how the model’s predictions compare with outcomes on local patients without affecting care. It cannot show how clinicians will respond to the alerts, which is where much of the benefit or harm arises. A model that predicts well can still fail if alerts arrive too late, reach the wrong person or prompt no clear action.

A staged rollout fills that gap. The model goes live on one or two units first, with agreed measures of process and outcome, such as time to antibiotics for a sepsis alert or rapid response calls for a deterioration score. Only when those measures look right does it spread. That costs time, and it is the closest a hospital can get to the trials that drugs go through.

What the function needs

The function needs data science staff, clinical champions for each model, and access to outcome data, the information about what actually happened to patients, so predictions can be checked against reality.

It also needs vendors to cooperate. Contracts and selection reviews should ask for:

• performance evidence;
• a description of training data and known limitations;
• support for local testing;
• advance notice of model changes.

Without those, the committee is reviewing a black box: a system whose inputs and outputs are visible but whose workings are not.

Regulatory status belongs in the review as well. After the FDA’s September 2022 guidance, some decision support tools were regulated devices and others were not. The committee should know which is which, because a device carries its own labeling, intended use and reporting obligations, and using it outside its cleared purpose shifts risk to the hospital.

Keeping it fast enough

The formulary model has a cost. Committees meet monthly, and product teams want to deploy faster than that. A tiered approach helps. Low-risk tools, such as a model that predicts no-shows to send reminders, can be approved by a small subgroup against a checklist. Tools that influence diagnosis or treatment go to the full committee.

The tiering itself should be written down, so that a vendor feature switched on in an EHR upgrade does not bypass review because nobody classified it. This is a common blind spot. Major software vendors regularly add new predictive features in upgrades, and unless someone is watching, a new score can appear in clinicians’ screens without anyone having decided it should.

Useful tiering criteria include whether the output reaches clinicians or patients directly and whether it can change a treatment decision. Others are how time-critical it is, how many patients it touches, and whether a human reviews every output before action. A tool that scores every inpatient every hour and prompts a rapid response is high tier. A model that drafts a reminder list that staff review is low tier.

Equity monitoring: the quality program’s method

The algorithmic discrimination principle asked that systems be checked for unjustified differences in treatment based on protected characteristics such as race, sex or age. In health care the concern was concrete.

A widely cited 2019 study found that an algorithm used to select patients for care management underestimated the needs of Black patients because it used past health care spending as a proxy for need. A proxy is a stand-in: a measure used because the thing you really care about is hard to measure directly. Spending seemed a reasonable stand-in for sickness. But if some groups have historically had less access to care, and so less spent on them, a model using spending will conclude they are healthier than they are.

Two checks

Quality and health equity teams already stratify outcomes by race, ethnicity, sex, age, language and payer. “Stratify” simply means breaking results down by group to see whether some groups fare differently. Extending that method to algorithms means two checks:

1. Performance across groups. Whether a risk score misses more true cases in one population than another.
2. Inputs. Whether the model relies on proxies such as cost, zip code or prior use of services that carry historical inequities into the prediction.

Suppose a hospital’s quality team takes its deterioration score and looks back at a period of admissions. For each patient group, it asks: of the patients who actually deteriorated, what share did the score flag in advance? If the score caught most cases in one group but noticeably fewer in another, the team has found something to investigate. The cause may lie in the model, the data or the way care is delivered.

Rules as well as models

The same review applies to clinical rules that are not AI at all. Medicine had already been reexamining equations that adjusted for race. In 2021, the National Kidney Foundation and the American Society of Nephrology recommended a new equation for estimating kidney function without a race coefficient. In 2022 the national organ transplant network moved to require race-neutral kidney function calculations for waiting list purposes. The Section 1557 proposal of August 2022 used the broad term “clinical algorithms,” which covers scoring rules and calculators as well as machine learning. An equity review limited to products labeled AI would miss much of the exposure.

Choosing what fairness means

Fairness has more than one technical definition, and they can conflict. A model can catch the same share of true cases in every group but raise more false alarms in one. Or its risk estimates can be equally accurate across groups while the share of cases caught differs. Improving one measure can worsen another.

The quality team should not settle that choice alone. The committee should decide, for each high-tier model, which kind of error matters most for the clinical purpose. For a sepsis alert, missing true cases in one group is usually the most serious failure. For an outreach list with limited slots, who gets offered the service matters most. Writing down that choice, and the reason, is what an auditor or regulator would later ask to see.

Acting on a finding

Finding a gap is the start of the work. The options include recalibrating the model or adjusting its threshold for affected groups, and removing or replacing a proxy input. Others are adding human review for the group where the model is weaker, asking the vendor to retrain it, or retiring it. Each option has costs, and some raise legal questions of their own, so counsel should be involved in choosing among them.

Whatever the choice, the committee should record the finding, the decision and the reason, and set a date to check whether the fix worked. A documented, reasoned response to a known gap is far easier to defend than a gap nobody looked for.

The data problem underneath

Data quality limits both checks. Race, ethnicity and language fields in many records are incomplete or inaccurate, which makes subgroup analysis noisy. Improving that data is slow, unglamorous work. It is also a precondition for knowing whether a model treats patients fairly, and it helps the quality program in other ways.

Small groups make the statistics harder still. A hospital may see too few patients from some groups in a year to detect a difference with confidence. Pooling several years of data, or comparing results with peer institutions using the same vendor model, can help. So can treating an unexplained gap as a reason to look closer rather than as proof.

Notice and fallback: the informatics and patient experience job

The notice and explanation principle and the human alternatives principle belong together in operation. People affected by an automated decision should know a system is involved, understand roughly how, and be able to reach a person who can review or override it.

For clinicians

For clinicians, override exists in principle. Whether it is realistic depends on alert design, time pressure and workflow, which clinical informatics, the team that designs how technology fits into clinical work, owns. A doctor can always ignore an alert, but if the alert interrupts every few minutes, ignoring becomes automatic rather than considered.

Tracking overrides and the reasons for them shows where a model and clinical judgment disagree, which feeds back into validation. If clinicians consistently override a model for a particular kind of patient, either the model has a blind spot or the clinicians do, and either way the committee should know. Alert fatigue is a sign that the fallback has turned into a reflex.

Explanation clinicians can use

Notice for clinicians means more than a label saying a model produced the alert. It means showing what drove the score, such as rising heart rate and falling blood pressure, so the clinician can judge whether it makes sense for this patient. It means a short reference card on what the model was trained on, what it predicts, and where it is known to perform poorly. Clinicians who understand a tool’s limits use it better than those told only to trust it.

Design choices shape behavior. An interruptive pop-up forces a response but adds to fatigue. A passive score in a patient list is easier to ignore but less disruptive. Some systems route high-risk alerts to a dedicated nurse or rapid response team who screens them before they reach the bedside. Each choice changes what override means, and the committee should approve the design along with the model.

For patients

For patients, the questions arise in scheduling, triage chat, outreach lists and financial assistance screening. Do patients know an algorithm shaped the outcome, and can they easily reach a person?

Patient experience and access teams can write those requirements into design reviews and vendor contracts, and test them by trying to reach a human through each channel. That test is revealing and cheap: someone on the team simply tries, as a patient would, to get past the automated system to a person, and reports how long it took and whether it worked.

The test should cover patients who need language interpretation, who use assistive technology, or who do not use the patient portal. A route to a person that works only in English, only online, or only during business hours is a partial fallback. Health systems that receive federal funds already have language access and disability obligations, so this work extends existing compliance rather than starting new.

When a patient challenges a result

Hospitals that participate in Medicare must already have a process for patient grievances. That process is the natural route for a patient who believes an automated tool treated them unfairly, such as a scheduling system that kept offering distant appointments or a financial assistance screen that denied them. Grievance staff need to know which tools are in use, so they can recognize a complaint about one. Logging complaints by model, and sending the log to the committee that owns it, turns individual grievances into evidence about how a tool behaves.

Following one model from purchase to bedside

The three functions fit together around a single tool. Take a health system considering a vendor’s deterioration score, a model that estimates which hospitalized patients are at risk of getting sicker in the next day or so.

Selection. The vendor’s sales materials report strong accuracy. Before signing, the health system’s contracting team uses a checklist from the model governance function. It asks for the evidence behind those figures, a description of the data the model learned from, and its known limitations. It also asks for support for testing it locally, and a commitment to give notice before the model changes. The vendor’s answers go into the file the committee will review.

Classification. Because the score will prompt nurses and doctors to reassess patients, and so influences treatment, it is placed in the highest risk tier and goes to the full committee rather than the fast-track subgroup. The team also records whether the product is a regulated device and, if so, its cleared intended use.

Local validation. The data science team runs the model in the background on recent patients, without showing results to clinicians, and compares its predictions with what actually happened. A clinical champion, a respected hospitalist, reviews cases where the model and reality disagreed.

Equity check. The quality team breaks the validation results down by patient group and examines the model’s inputs for stand-ins that might carry historical inequities.

Design. Clinical informatics decides how the alert will appear, how often, to whom, and how clinicians record a reason when they disagree. Patient experience staff decide what, if anything, patients should be told.

Approval and monitoring. The committee approves the model for inpatient wards only, sets a review date and a performance threshold that would trigger an earlier look, and adds it to the inventory with named owners. Months later, the override reports show nurses on one ward dismissing most alerts. That finding goes back to the committee, and the investigation begins.

Change and retirement. A year later, the vendor announces a new model version trained on different data. Under the contract, the health system gets notice in advance. The new version runs in the background alongside the old one before it is switched on. If it performs worse on local patients, the committee can decline the update, or retire the tool altogether and record why.

Not one of these steps required a new institution. Each was done by people who already did something similar for drugs, devices or quality measures. What was new was the inventory that tied them together and the habit of treating a model as something that needs the same care as a new medicine.

Safety, equity and money at stake

The stakes fall into familiar categories:

• Patient safety. A model that performs worse than expected can lead to missed warnings or unnecessary interventions.
• Equity and reputation. A tool that underserves some patient groups, as the 2019 study showed can happen, damages trust with the communities a health system serves.
• Money. Models bought from vendors cost money to license and staff time to use. Local validation shows whether that money is buying what was promised.
• Clinician trust and workload. Poorly tuned alerts add to burnout and teach staff to ignore warnings, including the important ones.
• Regulatory readiness. Nonbinding principles often foreshadow binding ones. An organization that already governs its models is better placed when rules arrive.

Who carries the legal risk

When a model contributes to harm, liability does not stop with the vendor. A hospital that deploys a tool, sets its thresholds and designs its alerts has made clinical choices of its own. Contracts often limit vendor liability, leaving the provider with most of the exposure. The August 2022 Section 1557 proposal pointed in the same direction for discrimination: it would hold covered entities responsible for decisions made using clinical algorithms, whoever built them. Organizations should check with counsel how these duties apply to them, but the direction was clear in October 2022.

Patient data in vendors’ hands

The blueprint’s data privacy principle maps onto the HIPAA program, but models raise a specific question. Vendors that host or tune models often want to use client data to improve their products. Under HIPAA, a business associate may use protected health information only as its business associate agreement permits. Privacy officers should check whether agreements allow use of patient data for training models, and whether de-identification is required first. That review belongs in the selection checklist alongside the performance questions.

The cost of doing it

The approach is not free. Local validation needs analyst time and data access for every high-tier model. Equity reviews need cleaner demographic data. Committee review adds weeks to deployment. For a community hospital, the honest answer may be that it cannot validate complex models alone. It can still demand evidence from vendors, run the simpler checks, join peer groups that compare results, and decline tools it cannot evaluate. Declining a tool is a legitimate governance outcome.

The one new artifact: an inventory

The three functions need a light coordinating layer: a single inventory of models in use, each with an owner in each function, reviewed by a senior clinical and operational leader. That inventory is the one genuinely new artifact. Everything else extends what the organization already does.

What the inventory records

Each entry should record the model’s name, source and version, its approved use and risk tier, and whether it is a regulated device. It should list the inputs, including any demographic variables or proxies. It should hold the date and results of local validation and subgroup analysis, the owners for validation, equity and notice, the next review date, and what patients and clinicians are told. A spreadsheet is enough to start. What matters is that it is complete and that someone is accountable for keeping it so.

Finding what is already running

The first inventory is the hardest. Useful sources include the EHR vendor’s list of available predictive features and which are switched on, and purchasing records for software with predictive functions. Add the data science team’s deployed models and department-level tools bought on a credit card. Revenue cycle and patient access teams often run more models than clinical teams realize. A short survey of department heads, followed by a check against system configuration, catches most of them.

What the board sees

A quarterly summary drawn from the inventory gives the board what it needs without technical detail. It shows the number of models by risk tier, reviews that are overdue, models paused or retired and why, open equity findings with their owners, and any safety events where a model played a part. Directors can then ask the same questions they ask about any other clinical risk.

Around that inventory, a working program has:

• One list of every predictive model in use, including vendor features inside the EHR, each with named owners for validation, equity and notice.
• A formulary-style approval process with written risk tiers.
• Local validation before go-live and on a schedule afterward.
• Regular breakdowns of model performance by patient group.
• Override tracking that feeds back into review.
• Vendor contracts that require evidence, transparency and notice of changes.

Programs go wrong in predictable ways:

• Building a parallel bureaucracy. A new AI committee with no authority over existing clinical decisions tends to produce principles rather than outcomes.
• Trusting the vendor’s numbers. Performance elsewhere is not performance here.
• Approving once and forgetting. Models drift.
• Missing the features that arrive unannounced in upgrades.
• Counting overrides as success. A clinician who can override but always does so reflexively is not exercising judgment.

Counting, validating and checking the hospital’s models for fairness

1. How many predictive models are running in our clinical and administrative systems today, and who owns each one?
2. Which of them were validated on our own patients before going live, and when was each last checked?
3. Have we broken down each model’s performance by race, ethnicity, sex, age, language and payer? What did we find?
4. How often do clinicians override each alert, and what do the reasons tell us?
5. If a patient wants a person to review an automated decision, how easy is that, and have we tested it ourselves?

With hindsight

The blueprint itself had a short formal life. Executive Order 14110 in October 2023 and OMB’s 2024 AI guidance for agencies set binding federal requirements. The OMB memo required agencies to apply minimum practices, such as impact assessments, testing and human review, to AI that affected people’s rights or safety. Both were rescinded or replaced in 2025: the executive order on the first day of the new administration, and the OMB memo by new guidance that April.

Its ideas moved into rules that do bind health care. The ONC’s HTI-1 rule, finalized at the end of 2023, added transparency requirements for predictive decision support in certified health IT, the electronic record systems that meet federal certification standards. Developers had to let users see information about how a predictive tool was built and tested, and to manage its risks. HHS’s 2024 Section 1557 rule requires covered entities to make reasonable efforts to identify and mitigate discrimination risks in patient care decision support tools, with that duty taking effect in May 2025. Several states passed laws on AI in health care and insurance. Colorado enacted a broad law in 2024 on high-risk AI systems used in consequential decisions, including health care services, and later postponed its start. California, for example, began requiring disclaimers on patient communications generated by AI from 2025.

Coverage decisions drew scrutiny too. CMS rules and guidance for Medicare Advantage plans, issued in 2023 and 2024, made clear that plans may use algorithms to assist but must base coverage decisions on the individual patient’s circumstances. That is the human fallback principle, written into payment rules.

For hospitals, the effect of these rules was practical. The Section 1557 duty turned the equity check described above from good practice into a compliance obligation for tools whose inputs include protected characteristics. The HTI-1 transparency requirements gave hospitals a firmer basis for asking their record vendors for the information a validation function needs.

The FDA kept refining its side. It finalized guidance in December 2024 on predetermined change control plans, which let makers of AI-enabled devices describe in advance how a model will be updated. It issued draft guidance on AI-enabled device software in January 2025, and it revised its clinical decision support guidance again in January 2026.

Generative AI then spread quickly into clinical documentation and patient messaging, with tools that draft notes and replies rather than simply producing a score. That raised the stakes and widened the range of tools needing oversight. A drafted reply to a patient is a different kind of output from a risk score. The governance needs are the same: evidence that it works here, evidence that it works for everyone, and a human who stands behind it.

That is where the choice made in 2022 matters. A health system that had already extended its existing committees to algorithms was better placed to take each new tool in its stride, for a simple reason. When a new kind of tool arrived, it already had an inventory to add it to and a committee with authority to approve or pause it. It had a quality team used to checking results by patient group, and informatics staff watching how clinicians actually used it.

A standalone body, by contrast, has to win authority, clinical trust and a place in the approval process before it can use them, and fast-moving tools do not wait for that to happen.

The broader pattern holds beyond health care. Guidance that carries no penalties is easy to dismiss, but it often previews what later becomes binding, as transparency and nondiscrimination duties did for health care. Federal policy on AI swung sharply between administrations from 2023 to 2025, which is one more reason to anchor governance in disciplines that do not change with each administration. And the most durable way to govern a new technology is frequently to ask which existing disciplines already do the job, and extend them, before inventing something new.