Which back-office jobs are a bad first test for AI?

A first test of AI on back-office work is a comparison. Drafts made from old documents sit beside what your team actually did with those same documents, and the person who knows the work best decides whether they match. Every sign of a bad first test is a way that comparison breaks: nothing to compare against, nothing on paper to draft from, too few cases to judge, a decision no draft should make, or files nobody outside may see.

Our free AI proof of concept is that comparison: one job, copies of 20 to 30 documents your team already finished, sent after an NDA is signed, and nothing logging in to a live system. The owner names your reviewer, the success criteria are agreed first, and the reviewer's call decides it. So the job has to give that reviewer something real to judge.

NIST's AI Risk Management Framework (January 2023) puts this kind of check before anything is built. Its MAP function maps the setting a system would work in, and NIST writes that afterward an organization should know enough to make an initial go/no-go decision on whether to design, develop or deploy the system at all. A no at that stage is an answer, not a failure. The five signs below are a short version of that check, for one job.

There are no finished examples. The job is new, or its output isn't kept: a vendor file cleaned up in place with no record of what it looked like before, a forecast rebuilt each month on top of last month's. NIST lists this among the ways AI risk differs from ordinary software: the ground truth "may either not exist or not be available". Without it, your reviewer grades drafts against a fresh opinion of their own.

The answer lives in someone's head or on a call. The documents show what came in, not why the team acted. A credit hold released after the controller rang the customer. A deduction written off because of something a sales rep promised. A draft built from the paper can't know the deciding fact, and your reviewer can't fairly mark it wrong for missing it. Under MAP 2.2, NIST asks that a system's knowledge limits be written down along with how people will oversee its output. Here the limit sits in the job, because the knowledge never reached a page.

Both usually have a fix close by. Where the decision was spoken, the paperwork around it often wasn't: the credit application, the claim packet, the deduction backup. Those parts have finished versions on file.

Some jobs come up a handful of times a year: an annual insurance questionnaire, a rare kind of contract amendment, year-end confirmations to auditors. Even when 20 to 30 past examples exist, they may span years of changed forms, rules and staff, and so describe several jobs at once. NIST's MAP 2.3 names the availability, representativeness and suitability of the data among the things to document before testing. A rare job tends to fail all three.

Low volume also weakens the reason to test. MAP 1.4 asks that the business value of a use be clearly defined, and a job that takes a few afternoons a year is rarely where the hours go, so even a clear yes changes little. Start with a job that runs every day or every week.

Some back-office work is mostly decisions: whether to deny a warranty claim, whether a worker is an employee or a contractor, whether to grant a leave or an accommodation, whether to cut off a customer's credit. Each can carry legal consequences for a person or the company, and there is no clean answer to score. Two experienced people can decide the same file differently, and a wrong draft is not a corrected field but a decision someone may later have to defend.

OMB's April 2025 memo to federal agencies, M-25-21, draws a useful line. It treats AI as high-impact when its output serves as a principal basis for decisions or actions with a legal, material, binding or significant effect on rights or safety, and it says that can be true whether or not a person oversees the decision. Its list of uses presumed to be high-impact includes decisions on the terms of federal employment: hiring, pay, promotion, reasonable accommodation, discipline and termination. A private company isn't bound by it, but the reasoning carries over: a person reading every draft doesn't turn a judgment job into a safe first test.

The memo also notes that AI can sit inside a high-impact process without being high-impact, when its output isn't the basis of the decision. That is where a test can go: assemble the claim file, list what is missing, put the dates in order, and leave the decision with the person who owns it.

Some files are fenced by law or contract. Under the ADA rules, medical information an employer gets from employee medical examinations and inquiries is kept on separate forms, in separate medical files, and treated as a confidential medical record, with narrow exceptions. A defense contractor holding covered defense information under DFARS clause 252.204-7012 must keep it on systems held to the security requirements of NIST SP 800-171, and the clause goes into any subcontract whose work will involve that information. Either can mean the files stay where they are. None of this is legal advice; your counsel decides what may be shared.

A fenced file usually still allows a test, because look-alikes can take its place: documents built with your reviewer in your real layouts, with invented values and a written answer for each. That breaks down when the difficulty is the real content itself. A decades-old contract whose odd clauses are the whole problem, the handwriting of one long-serving foreman, a drawing whose controlled details are exactly what the job reads. A look-alike can copy a layout; it can't copy what nobody outside may see. When both halves are true, pick a different job.

NIST's framework is plain about the far end of the scale: where a system presents unacceptable negative risk, development and deployment should cease in a safe manner until the risk can be managed. Most back-office jobs are nowhere near that line. A few, though, fail the checks above for reasons that won't change, and a first AI test should leave them alone:

• Decisions about people. Hiring, firing, discipline, pay and accommodation. Typing up the paperwork after a person has decided is a different job; drafting the decision is not one to start with.
• Anything a law or a license says a named person must decide or sign. A professional stamp, an attestation, a sworn statement. AI can gather the file for that person, but it shouldn't be tested as the one who decides.
• Files you may not share, even as copies, that can't be rebuilt as look-alikes. Controlled defense information, records under a court's protective order, data a customer contract keeps inside your walls. With neither copies nor look-alikes, there is nothing to test.

When the job you wanted fails, look one step before or after it, or at the job next door. You want the part that is on paper, repeats every week, has a finished answer on file and lets your reviewer say right or wrong in a minute. Swaps that usually work:

• Instead of the credit decision, the account setup that follows it: the application, the tax exemption certificate and the terms keyed into the customer record.
• Instead of deciding warranty claims, assembling them: the claim form, serial number, proof of purchase and a list of what is missing, ready for the person who decides.
• Instead of HR files, vendor invoices coded to job, property or GL account, each with a posted entry on file.
• Instead of a job nobody wrote down, one that leaves a trail: customer POs and the sales orders your team entered from them.