Before the bias audit, the inventory: what New York City’s hiring-tool law asked of employers

New York City’s hiring-tool law made bias audits the headline. For banks, insurers and asset managers hiring in the city, the harder first step was finding every tool that scores candidates, deciding which were covered, and naming an owner for each.

Looking Back revisits one week at a time, from January 2020 to today, and asks what it meant for the banks, health systems and public agencies we serve. The series was written in 2026, with the benefit of hindsight, and each article says which week it looks back on.

Two-minute summaryThe dek and the first sentence of each section, quoted from the article

New York City’s hiring-tool law made bias audits the headline. For banks, insurers and asset managers hiring in the city, the harder first step was finding every tool that scores candidates, deciding which were covered, and naming an owner for each.

  1. The week On July 5, 2023, New York City’s Department of Consumer and Worker Protection began enforcing Local Law 144, which regulates what it calls automated employment decision tools.
  2. What an automated hiring tool actually is The phrase “automated employment decision tool” sounds like a robot in a hiring office.
  3. Five ways the law reaches a financial firm Once a bias audit is posted, its numbers are public, and a large gap between groups can follow a financial firm into court, into the press and into its own recruiting.
  4. Finding the tools A bank or asset manager rarely buys one hiring system.
  5. Deciding scope The law covers a tool that uses machine learning, statistical modeling, data analytics or artificial intelligence to produce a simplified output, such as a score or ranking, that substantially assists an employment decision.
  6. Owning the audit An audit needs data, and the data rarely lives in one place.
  7. Giving notice Candidates who live in the city must be told, at least ten business days before use, that a tool will be used and which qualifications or characteristics it assesses.
  8. Keeping the register alive The steps share a requirement.
  9. What to ask before the next audit The answers belong in the register itself, dated and signed, so the next review starts from evidence rather than memory.
  10. With hindsight Local Law 144 turned out to be a weak enforcement regime and a strong signal.

The week

On July 5, 2023, New York City’s Department of Consumer and Worker Protection began enforcing Local Law 144, which regulates what it calls automated employment decision tools. These are software systems that score, rank or filter job candidates or employees, such as a program that grades an online assessment or sorts a pile of applications into “strong” and “weak.”

The law was aimed at a familiar scene. A job applicant in the city finishes a recorded video interview, closes her laptop and waits. She assumes a recruiter will watch it. In some firms, software might watch it first and give her a score that helped decide whether any human ever did. From that week, the city insisted that employers be open about it.

The road to enforcement had been long. The City Council passed the law in late 2021, and it took effect on January 1, 2023. The department published proposed rules in September 2022, revised them in December after a public hearing, and held a second hearing early in 2023. Advocates argued that the draft definitions left too many tools outside the law. Employers and vendors argued that the audit requirements were unclear and costly. The department adopted final rules in April 2023 and moved enforcement from April 15 to July 5. In late May it held roundtables for employers, for worker advocates and for the emerging audit industry.

The law had technically taken effect in January, but enforcement waited while the department finalized its detailed rules. From July 5, employers and employment agencies using such a tool to substantially assist hiring or promotion decisions in the city needed three things:

  • an independent bias audit of the tool, conducted within the prior year;
  • a public summary of the audit’s results; and
  • advance notice to candidates that a tool was in use.

The bias audit is a statistical check. It compares how often the tool selects or favorably scores people in different sex and race or ethnicity categories. If one group passes much less often than another, that gap shows up in the published numbers for anyone to read, including candidates, journalists and lawyers.

The final rules filled in the mechanics. The auditor had to be independent: not involved in building, selling or using the tool, not employed by the employer or vendor, and without a material financial interest in either. The audit had to report results for sex, for race and ethnicity, and for combinations of the two, using the categories employers already use in federal workforce reporting. It had to rely on the employer’s historical data where enough existed, with test data allowed only where it did not. The summary had to show the number of people assessed and how many fell into an unknown category. Notices had to reach candidates who live in the city at least ten business days before the tool was used. Penalties ran from $500 for a first violation to $1,500 for later ones, with each day of use counted separately.

The city’s law did not arrive alone. In April 2023, the Equal Employment Opportunity Commission, the Consumer Financial Protection Bureau, the Justice Department’s Civil Rights Division and the Federal Trade Commission had issued a joint statement. It said existing anti-discrimination and consumer protection laws applied to automated systems. On May 18, the EEOC published technical assistance on how federal employment discrimination law applies to software used in selection. It made clear that employers can be responsible for the effects of tools they buy from vendors. The city’s audit requirement thus landed on top of federal liability that already existed.

The law was widely described as the first of its kind in the United States. For financial firms, which hire in large numbers in New York, it was the first time a regulator had asked them to account publicly for how software shaped their hiring. And the first question it raised, “which tools do we actually use?”, is harder to answer than it sounds.

Most commentary on Local Law 144 focused on the bias audit: who could perform it, what statistics it required, and whether its numbers meant anything. That was understandable. The audit was the novel part, and it produced figures that could be published, compared and criticized.

For financial firms with large New York hiring programs, though, the audit came last in a sequence. It could not start until the firm knew which tools it used, which of those the law covered, who owned each one, and how candidates would be told. Those steps are less glamorous and much more revealing. They expose how little many organizations know about the software making early judgments about the people they hire.

The sections below take them in order. First, what these tools are and why the law reaches a financial firm at all. Then the four working steps: finding the tools, deciding scope, owning the audit and giving notice. Last, keeping the register alive, and the questions to ask before the next audit.

What an automated hiring tool actually is

The phrase “automated employment decision tool” sounds like a robot in a hiring office. The reality is more mundane, which is exactly why these tools are easy to overlook.

Most of them work in a similar way. A vendor collects examples of past candidates and outcomes, such as who was hired, who performed well, or how experienced recruiters rated answers. Software then looks for patterns that predict those outcomes. This is what is meant by machine learning: rather than a programmer writing every rule, the system learns its rules from historical examples. Simpler tools use statistical models, formulas built from data, or straightforward scoring rules. The output is usually something simple: a score from 1 to 100, a ranking, a traffic-light rating or a “recommended” flag.

That simplicity is both the appeal and the risk. A recruiter facing hundreds of applications naturally starts at the top of a ranked list. The score shapes the decision even if no one formally hands the decision to the machine.

The fairness concern comes from the learning step. If the historical examples reflect past patterns in who was hired, the tool can learn to reproduce them. It might, for instance, favor features that correlate with particular schools, neighborhoods or styles of speech, and those features can track sex or race without anyone intending it. A bias audit is the check for whether that has happened.

What a financial firm’s hiring stack contains

In a bank, insurer or asset manager, the tools fall into a few recognizable families:

  • Résumé parsing and matching. The applicant tracking system extracts skills and experience from a résumé and compares them with the job description, producing a match score or ranking.
  • Online assessments. Cognitive tests, situational judgment tests, personality questionnaires and game-based exercises, scored against a model of successful employees.
  • Video interview scoring. Recorded answers are transcribed and rated, sometimes against criteria set by the employer, sometimes by a vendor model.
  • Technical tests. Coding exercises graded automatically, common in technology and quantitative hiring.
  • Chat and scheduling assistants. These mostly book interviews, but some ask screening questions and pass only some candidates forward.
  • Generative AI features. Newer tools that summarize résumés or interviews and suggest which candidates to advance.

Not all of them fall under the law. A fixed knock-out question, such as whether a candidate holds a required license, is a rule written by a person rather than a statistical model. A chatbot that books interviews without filtering anyone does not assist a decision. The definitions do the sorting, which is why the scope step matters.

The audit’s central measure is the impact ratio. Take the rate at which each group is selected or scored favorably, and compare it with the rate for the most-favored group. If the most-favored group passes at a given rate and another group passes at half that rate, the second group’s impact ratio is 0.5. A ratio near 1 suggests similar treatment; a low ratio suggests a gap worth investigating. The law requires the ratios to be calculated and published. It does not say what number is acceptable.

What the impact ratio does not tell you

For tools that produce a score rather than a pass or fail, the rules use a scoring rate instead. That is the share of each group scoring above the median score of everyone assessed. The impact ratio is then computed the same way.

Practitioners usually read the ratios against the four-fifths rule of thumb from the federal Uniform Guidelines on Employee Selection Procedures, adopted in 1978. A ratio below 0.8 is traditionally treated as a sign of possible adverse impact. Federal agencies have long said that it is a rule of thumb rather than a legal test. Courts may look at statistical significance as well, and a small sample can produce a low ratio by chance.

The audit also leaves things out. It covers sex and race or ethnicity, but not age or disability, which are among the most common grounds of hiring complaints. The rules let an auditor exclude categories that make up less than 2% of the data, so smaller groups may never appear. And the audit measures outcomes, not validity. A tool can treat groups similarly and still predict job performance poorly, or show a gap and be well justified by the job’s requirements.

A firm that reads its published ratios as a clean bill of health is reading them for more than they say.

Five ways the law reaches a financial firm

Once a bias audit is posted, its numbers are public, and a large gap between groups can follow a financial firm into court, into the press and into its own recruiting.

Legal exposure. The city’s own enforcement turned out to be light, as later events showed. The bigger risk lies in long-standing federal and state discrimination law. A published audit with a large gap between groups is a ready-made exhibit. And a firm that cannot say which tools screened a rejected candidate is poorly placed to defend any hiring claim, whether or not a tool was involved.

The federal framework is well established. Under Title VII’s disparate impact doctrine, a plaintiff shows that a practice produces a significant disparity. The employer must then show that the practice is job-related and consistent with business necessity. Even then, the plaintiff can win by showing a less discriminatory alternative that would serve the employer’s needs. A published impact ratio supplies the first step. The employer’s ability to answer the second and third depends on validation evidence and records that many firms never asked their vendors for.

State and city law add more. New York State’s and New York City’s human rights laws prohibit discrimination in employment, and the city’s law is generally read more broadly than federal law. A firm hiring in the city faces all three layers at once.

Reputation. Banks, insurers and asset managers spend heavily to present themselves as fair employers and to recruit diverse graduate classes. An audit summary is a public document. A journalist or advocacy group can compare one firm’s numbers with another’s. The story writes itself if the numbers are poor and the firm has no explanation.

Talent. A screening tool that systematically filters out capable people is a recruiting problem as well as a fairness one. The firm pays to attract candidates and then discards some of the best before a person sees them. Understanding what the tools do is part of hiring well.

Regulators beyond the city. Financial regulators increasingly expect firms to govern their models, meaning to know what automated systems they run, who owns them, how they were tested and how they are monitored. Banks already do this rigorously for credit and risk models. Hiring tools have often escaped that discipline because they sit in HR rather than in risk. Local Law 144 was a reminder that a model is a model, wherever it sits.

For banks, the relevant standard is supervisory guidance on model risk management, issued by the Federal Reserve and the OCC in 2011. It defines a model broadly, as a quantitative method that turns inputs into estimates. A résumé ranking engine fits that definition comfortably. Bringing hiring tools into the model inventory, even at the lowest risk tier, gives them an owner, a validation record and a periodic review, using machinery the bank already runs.

Insurers and asset managers have no direct equivalent for HR tools, but many have built AI governance programs for customer-facing models. Extending the same intake form and review to hiring tools is cheaper than building a parallel process in HR.

Cost. Audits, notices and alternative processes cost money and staff time. The cheapest route is to know early which tools are worth keeping. A tool that adds little value and carries real risk may be better retired than audited every year.

How the exposure differs by firm

The same law lands differently across financial services.

  • Large banks run high-volume campus, contact center and branch hiring, often with several assessment vendors. They also have mature model risk and compliance functions. Their challenge is volume and coordination across many recruiting teams.
  • Regional banks and credit unions hire fewer people in the city and usually rely on features inside their applicant tracking system and on staffing agencies. Their exposure is less visible, which makes the inventory more important.
  • Broker-dealers hire registered representatives and run graduate programs that attract many applicants per place. Automated screening is tempting at the top of that funnel, where gaps between groups would be most visible.
  • Insurers hire at volume for claims and customer service roles, where online assessments are common and the case for a staffed alternative process is strongest.
  • Asset managers and wealth managers hire in smaller numbers and lean on executive search firms. Their tools are often the search firm’s, which puts the weight on contracts rather than internal controls.

Finding the tools

A bank or asset manager rarely buys one hiring system. Campus recruiting may use a video interview platform with automated scoring. Experienced-hire recruiting may use the ranking feature in the applicant tracking system, the software that holds job postings, applications and candidate records. Operations and contact center hiring may use online assessments from a different vendor. Technology hiring may use coding tests that grade automatically. Staffing agencies and executive search firms add their own tools, often invisible to the employer.

These tools arrive through different budgets. Talent acquisition buys some, business lines buy others, and some come bundled as features in software bought for another purpose. Procurement records will not find a scoring feature switched on in an existing contract. Counting purchase orders will miss every tool that arrived as a feature of something else.

The practical method is a structured survey of every recruiting team and vendor, asking one question: does any software rank, score, filter or recommend candidates before a person reviews them? Each answer goes into a register with the vendor, the feature, the roles it is used for and the volume of candidates.

Running the survey well

A survey that relies on memory will miss things, so its answers should be checked against records. Four sources catch most of what surveys miss:

  • Applicant tracking system configuration. An administrator can export which scoring, matching and AI features are switched on, by business unit and requisition type.
  • The identity provider’s application catalog. The single sign-on system lists the software recruiters log into, including tools bought outside procurement.
  • Expense and card data. Small assessment subscriptions are often paid on corporate cards.
  • Agency and search firm contracts. These show who is screening candidates on the firm’s behalf.

What the register records

For each tool, the register should hold the vendor and product, the specific feature, and the roles and locations it is used for. It should note whether candidates in the city are assessed, annual candidate volume, the form of the output, and how recruiters use it. It should also hold the scope decision and its date, the owner, the latest audit date, where the notice appears, and the data retention terms. A spreadsheet is enough to start. What matters is that one team owns it and that changes reach it.

Vendors keep adding features. Applicant tracking systems began adding AI-generated summaries and match scores to existing products, sometimes switched on by default in an upgrade. The register needs a trigger tied to vendor release notes and contract renewals, so a new feature is reviewed before recruiters start relying on it.

The register also needs the staffing agencies. The law applies to employment agencies as well as employers, but a firm that relies on an agency’s screening still carries reputational and legal exposure. If a candidate is screened out unfairly by a search firm’s tool, it is the employer’s name that appears in the complaint and the headline. Contracts with agencies and search firms should ask whether they use automated tools on the firm’s requisitions, and require them to share audit results.

What the agency clause should say

A short set of contract terms covers most of the exposure from staffing agencies and search firms:

  • A duty to disclose any automated tool used to screen, rank or recommend candidates for the firm’s roles, and to notify the firm before introducing a new one.
  • A duty to meet the city’s audit, summary and notice requirements where they apply, and to provide copies of audit summaries on request.
  • Access to candidate-level outcome data for the firm’s requisitions, so the firm can run its own analysis if a concern arises.
  • Cooperation with accommodation requests and with any investigation or litigation involving candidates the agency screened.

Smaller agencies may resist. The firm then has a choice between accepting less visibility and moving the work to agencies that will sign. Either way, the decision should be made knowingly and recorded in the register.

Suppose a mid-sized insurer believes it uses one hiring tool: the assessment platform its talent team bought two years ago. The survey turns up four more. The applicant tracking system has a “match score” feature that a regional team switched on. The graduate program uses a video interview product with automated ratings. A contact center manager pays for a typing and situational judgment test on a corporate card. And two staffing agencies use their own screening software on the insurer’s roles. The tools were not hidden; no one had asked the teams to list them.

Deciding scope

The law covers a tool that uses machine learning, statistical modeling, data analytics or artificial intelligence to produce a simplified output, such as a score or ranking, that substantially assists an employment decision.

The department’s rules read “substantially assist” narrowly. A tool is covered when its output is the sole factor, outweighs other factors, or overrules a human decision.

That definition invited firms to conclude they were out of scope, because a recruiter nominally reviews every candidate. The conclusion may be defensible for some tools and weak for others. If recruiters in practice only look at the top-ranked quarter of applicants, the ranking is doing the deciding, whatever the process manual says.

A scope decision should therefore rest on how the tool is actually used, observed and documented, not on what the policy says. In practice, that means asking recruiters to show how they work, looking at whether low-scored candidates are ever reviewed, and checking whether anyone ever advances a candidate the tool ranked poorly.

Testing practice with data

Interviews tell you what recruiters think they do. Data tells you what they actually do. A simple analysis compares advancement rates by score band. If nearly everyone advanced comes from the top band, and almost no one below a certain score is ever reviewed, the score is functioning as a cutoff whatever the manual says. That finding is uncomfortable, but it is better discovered internally than in a deposition.

Geography also needs a written decision. The law concerns jobs in the city and candidates who live there, and hybrid and remote roles complicate both. Counsel should confirm how the department’s guidance treats each role type, and the register should record the answer.

The case for a narrow reading, and its limits

A firm can make a reasonable legal argument that most of its tools fall outside the rules’ narrow definition. Many did. The argument has limits. Federal and state discrimination law applies whether or not a tool is covered by the city’s law. A firm that concludes a tool is out of scope still needs to know what it does to different groups, because that is what a plaintiff will ask. The narrow reading can avoid the publication requirement. It does not avoid the underlying risk, and it does not remove the need for the inventory that supports every later rule.

Firms should record each scope decision, its reasoning and who made it, and check it with employment counsel. A tool judged out of scope in 2023 could move into scope if a team changed how it used the output. A busy hiring season, when recruiters lean harder on rankings to get through volume, is exactly when that drift happens.

A scope memo need not be long. It should describe the tool and its output, how recruiters were observed using it, the data on advancement by score band, the conclusion against each limb of the rules’ definition, and the conditions that would change the answer. It should name who decided and when, and set a review date. A memo of that kind is also what a plaintiff’s lawyer will ask for, so it should be written as if it will be read by one.

Owning the audit

An audit needs data, and the data rarely lives in one place. Selection rates by sex and race or ethnicity require demographic information. That is usually collected voluntarily for equal employment reporting and held in the applicant tracking system. Scoring data is held by the vendor. An independent auditor needs both, matched together.

Getting the data

Demographic self-identification is voluntary, and many candidates decline. A large unknown category weakens the audit and must be disclosed. Firms can improve response rates by explaining why the data is collected, but cannot require it.

Handling matters as much as volume. Demographic data should be held separately from the information recruiters see, with access limited to the people running the analysis. Sharing it with an auditor needs a data sharing agreement covering use, security and deletion. Vendors should be contractually obliged to provide candidate-level scores in a usable format within a set time. Without that clause, the audit timetable belongs to the vendor.

Choosing the auditor

The rules set minimum independence standards. Beyond them, firms should ask what the auditor will actually do. Some audits calculate the required ratios and nothing more. Others add statistical significance testing, a review of the tool’s validation evidence and an analysis of which features drive any gap. The first satisfies the city. The second is far more useful if the results ever reach a courtroom.

Vendors offered audits based on pooled data from many customers. The rules allow that in some circumstances, but a pooled audit says little about whether a tool behaves fairly for one firm’s candidate population. A firm recruiting heavily from particular universities or regions may see different results from the pool. It is like judging a restaurant chain’s food safety by the national average when what you care about is the kitchen down the street.

The rules allow an employer to rely on a vendor’s multi-employer audit only in limited cases, generally where the employer contributed its own historical data or has never used the tool before. A firm relying on a pooled audit should confirm which case applies and keep the evidence.

Each in-scope tool therefore needs an owner, usually in talent acquisition, with authority to obtain vendor data, commission or review the audit, publish the summary and act on the results.

Acting on results is the part most often left undefined. The law requires an audit, not a particular outcome. But a firm that publishes an impact ratio showing one group selected at a much lower rate has created evidence that plaintiffs and other regulators can use under existing discrimination law. The audit summary is public. Anyone considering a claim can read it.

The summary has to be posted on the employment section of the firm’s website. It must show the date of the audit, the data source, the number of people assessed and the unknown category, along with the selection or scoring rates and impact ratios for each group. It must stay up for a period after the tool’s last use. A firm that swaps tools needs a process to keep old summaries available for that period.

The owner needs a pre-agreed path: investigate, adjust the tool or its thresholds, or stop using it. Legal, HR and the business should agree that path before the first audit arrives, not in a hurried meeting after an unwelcome number appears. Deciding in advance also protects the owner, who otherwise faces pressure to explain away a result rather than respond to it.

Suppose a regional bank’s audit of its contact center assessment shows that one group passes noticeably less often than the most-favored group. Because the path was agreed in advance, the owner does not need to convene a crisis meeting. She asks the vendor which parts of the assessment drive the gap. It turns out that a timed section penalizes candidates who take longer to read the questions, a skill the job barely uses. The bank removes the time limit, re-runs the analysis on recent candidates, and records what it found and what it changed. The next published summary tells a better story, and, more importantly, the bank is hiring from a wider pool of capable people. Not every investigation ends this neatly. Some tools cannot be fixed and should be retired. What matters is that the decision is made on evidence, by someone with the authority to make it.

Beyond the impact ratio

A firm that sees a low ratio needs more than the ratio. It needs validation evidence showing that the tool measures something the job requires, ideally a study linking scores to later performance in comparable roles. It also needs to show that it looked for alternatives with less impact. Vendors vary widely in the evidence they can supply. That should be a selection criterion when buying a tool, not a discovery after the first audit.

Giving notice

Candidates who live in the city must be told, at least ten business days before use, that a tool will be used and which qualifications or characteristics it assesses. Firms must also explain how to request an alternative selection process or accommodation where available, and must make information about data retention available.

The rules allow several channels: the employment section of the website, the job posting itself, or mail or email to the candidate. Information about the type and source of data collected and the retention policy can be posted on the website or provided within 30 days of a written request. Employees being considered for promotion are covered as well as external candidates.

Notice is an operational task for the career site and the recruiting workflow. Job postings, application confirmations and assessment invitations all need consistent language. The ten-business-day window has to be built into the recruiting timeline, which matters for roles filled quickly.

The simplest approach is a standing notice on the careers site, linked from every posting for roles in the city. It satisfies the timing requirement for anyone who reads the posting before applying. The harder cases are internal promotions and roles filled through agencies, where candidates may never see the firm’s careers page. Each needs its own notice path in the register.

Recruiters need a script for candidates who ask how the tool works or ask to be assessed another way. That second request is the hard one. If the firm offers an alternative, someone has to run it, which means interviewers and capacity. If it does not, the reason should be documented. Offering an alternative on paper that nobody is staffed to provide is worse than not offering one, because it turns a compliance statement into a broken promise.

Alternatives and accommodations

Two different things are easily confused. The city’s rules require the notice to explain how to request an alternative process or an accommodation, but they do not oblige the employer to offer an alternative. Reasonable accommodation for disability is a separate duty, under the Americans with Disabilities Act and state and city law. A candidate with a visual impairment who cannot complete a timed online test is entitled to have an accommodation considered whatever the city’s audit rules say.

The operating design follows from that. Accommodation requests go to the team that already handles them, with a defined turnaround. Requests for an alternative process go to the tool’s owner, who decides according to a written policy. Both are logged. The logs show whether candidates are using the routes, and a sudden rise in requests from one group can be an early sign that a tool has a problem.

Keeping the register alive

The steps share a requirement. Someone must own the register of tools and keep it current, because the hiring stack changes every year. That role sits naturally with HR operations, with a second line of review from compliance. Without it, the audit is a document produced once and outdated by the next hiring season.

A firm doing this well has a register it trusts, scope decisions written down with reasons, and an owner for each tool. It has audits based on its own candidates where possible, a plan for bad results, notices that match practice, and a staffed alternative process. It reviews the register at least yearly and whenever a team adopts something new.

Wiring it into existing processes

The register lasts only if other processes feed it. Procurement and third-party risk reviews should ask whether a new HR product scores or ranks people, and route it to the register if so. Change management for the applicant tracking system should flag any change to scoring features. Contract renewals with agencies should trigger a fresh question about their tools. The audit calendar should come from the register, so that each audit is commissioned in time to stay within the one-year window.

Second-line compliance should test the register periodically, sampling requisitions and checking that the tools used match the register entries. Internal audit can then test the whole chain: register, scope decisions, audits, summaries and notices. Reporting to the HR or risk committee of the board can be brief: number of tools, number in scope, audits current, results outside tolerance, and actions taken.

Programs tend to fail in the same places:

  • Starting with the audit. Commissioning an audit before knowing which tools exist means auditing the wrong things.
  • Scoping by policy rather than practice. “A human always reviews” is often true in the manual and false at the desk.
  • Accepting the vendor’s pooled numbers without question. They may not describe your candidates.
  • Forgetting the agencies. Their tools are screening your candidates under your name.
  • Publishing without a plan. A public gap with no response is a gift to litigants.

What to ask before the next audit

  • Can we list every piece of software that ranks, scores, filters or recommends candidates for our roles, including features inside larger systems and tools used by our agencies, and when did we last update that list?
  • For each tool, have we decided whether the law covers it, based on how recruiters actually use it? Who made that call, and is it written down?
  • Who owns each in-scope tool, and do they have the authority to get the vendor’s data and act on the audit?
  • What will we do if an audit shows a large gap between groups? Has legal, HR and the business agreed that in advance?
  • Do our job postings give the required notice in time, and if a candidate asks for an alternative assessment, who runs it?

The answers belong in the register itself, dated and signed, so the next review starts from evidence rather than memory.

The bias audit drew the attention. The inventory, the ownership and the honest scope decisions are what made it meaningful.

With hindsight

Local Law 144 turned out to be a weak enforcement regime and a strong signal. Both halves of that verdict matter.

The weak enforcement came first. The narrow definition of “substantially assist” let many employers conclude they were out of scope. In December 2025 a New York State Comptroller audit found the city’s enforcement weak. It covered July 2023 to June 2025. The department received only two complaints in that period, and its intake process did not reliably route complaints to the right place, including those made through the city’s 311 line. Where the department’s own review of 32 companies found one problem, the auditors found at least 17 potential instances of non-compliance. The department had not drawn on the city’s technology office for technical help in judging whether tools were covered, though an agreement with that office provided for it. The auditors recommended proactive research into compliance rather than reliance on complaints. A law that relies on complaints works poorly when candidates rarely know a tool was used, and when complaints that do arrive go astray.

Federal enforcement took its own course. In August 2023, a month after the city’s law took effect, the EEOC settled what was widely described as its first case involving AI in hiring. The tutoring company iTutorGroup paid $365,000 over software that automatically rejected older applicants. Private litigation grew too. In Mobley v. Workday, a federal court in California allowed claims against an HR software vendor to proceed in 2024 on the theory that it acted as an agent of the employers using it. In 2025 the court allowed an age discrimination claim to proceed as a collective action. That case put vendors in the frame alongside employers.

Then federal priorities shifted. After the change of administration in January 2025, the EEOC removed its AI technical assistance from its website. In April 2025 an executive order directed federal agencies to deprioritize disparate impact enforcement. Federal enforcement risk fell. Private plaintiffs could still bring disparate impact claims under Title VII, and state and city laws were unaffected. The risk moved from agencies to courts and to the states, rather than going away.

The signal proved more durable. The wider pattern took longer to settle, but it moved in one direction. Colorado passed a broad AI law in 2024 covering employment decisions, among others. In a special session in August 2025 it pushed the start date back to June 30, 2026. Illinois amended its Human Rights Act to address AI in employment decisions, effective January 1, 2026. California’s Civil Rights Council adopted regulations on automated decision systems in employment that took effect in October 2025. California’s privacy agency adopted rules on automated decision-making technology, including in employment, that took effect in January 2026, with compliance phased in from 2027.

The EU’s AI Act, which entered into force in August 2024, treats many employment uses as high-risk, which brings heavier obligations for firms hiring in Europe. Those obligations were scheduled to apply from August 2026, and in late 2025 the European Commission proposed delaying them. Generative AI, the kind of software that writes text and summarizes documents, spread through recruiting faster than governance did, adding tools that draft job descriptions, screen résumés and summarize interviews.

For a financial firm hiring across New York, Colorado, Illinois, California and Europe, the details of each rule differ. The starting point does not. Each rule asks, in its own way, which tools are making or shaping decisions about people, how they are used, and who is accountable for them.

Some parts of the 2023 approach aged less well. The city’s focus on sex and race left age, the ground of the most visible early cases, outside the audit. Firms that audited only what the city required learned little about the exposure that materialized first. The impact ratio also proved a blunt instrument for generative tools, whose outputs are summaries rather than scores.

A current register of tools that score candidates is the starting point for answering each of these rules. Firms that built one for New York in 2023 had a head start on everything that followed. Firms that concluded they were out of scope, and stopped there, had to begin the inventory again when the next rule arrived. The practical rule is simple: know what software is judging your candidates, whatever the law currently requires.

Spotted an error? Tell us and we will check it and correct the article where needed.

  • Local Law 144
  • AI hiring
  • bias audit
  • HR technology
  • New York City