What can we safely send for an AI test, and what should stay out?

A test of one job needs two things for each document: the document as it arrived, and what your team did with it. For an order desk, that is the customer's PO and the sales order keyed from it. For payables, the vendor invoice and the way it was coded. For certificates of insurance, the request and the certificate that went out. Everything else on those pages is along for the ride, and the extra material is where the risk sits.

Federal guidance gives every business the same rule. The FTC's guide to protecting personal information rests on five principles, and the second is scale down: keep only what you need. NIST's guide to protecting personally identifiable information (SP 800-122, 2010) tells organizations to hold the use, collection and retention of that information to what is strictly necessary for the purpose. A test is a narrow purpose, so what it needs is a short list.

So before anyone opens a folder, write down the fields the job reads. Your reviewer, the person on your team who knows this work best, is the right one to write it. Anything that isn't on that list is a candidate to remove.

Send copies or exports. The originals stay in your systems, untouched, and your records stay complete whatever happens to the copies. A PDF saved from the email it arrived in, a scan of a paper form, or a spreadsheet export of the order as entered all work. A login to the system that holds them is never part of it: we don't ask for one.

Choose finished documents, ones your team already processed, because the finished work is the answer key. Your reviewer compares each draft with what the team actually did, so a document with no outcome attached tells the test nothing.

Keep a short log on your side while you assemble the set: the file name, what it is, and who approved sending it. If you later need to know exactly what left the building, the answer is already in your hands.

Some information comes out even when the job seems to touch it. No desk job we test needs it, and the cost of a slip is far larger than anything the test gains from it.

• Full card numbers and card security codes. The PCI Security Standards Council defines cardholder data as, at minimum, the full card number, and treats the card verification code, full track data and PINs as sensitive authentication data. Its standard prohibits storing that sensitive data once a payment is authorized, even in encrypted form. Card numbers turn up in odd places: phone order notes, a deposit form stapled to a PO, a customer who typed theirs into an email. Cut the number, or leave the page out.
• Health records. Federal privacy rules name an employer among those who create or receive health information that identifies a person. A leave request, a fitness-for-duty note or a workers' compensation file can carry it at a company with no connection to health care. None of it belongs in a test of an order desk or a payables queue.
• Passwords, logins and keys. They hide in vendor setup forms, onboarding packets and old IT tickets. A test that never logs in to anything has no use for them.
• Personal identifiers the job doesn't read. Social Security numbers, birth dates, driver's license numbers and personal bank details. The FTC's guide says to use Social Security numbers only for required and lawful purposes, such as reporting employee taxes, and never as an ID for an employee or a customer.
• Files that belong to someone else. A drawing a customer gave you under its own NDA, or anything export-controlled, needs its owner's answer before it goes anywhere, our way included.

A black box drawn over a name in a PDF often hides it only from the eye. The National Security Agency's guide to sanitizing documents, written for analysts in 2005 and still a sound checklist, calls covering text with black the most common mistake: it works on paper, but in general not in a file passed around electronically. The same guide warns that turning a Word file into a PDF does not strip all of its metadata.

Three habits close the gap. Use a redaction tool that deletes the text under the mark, then check it: search the file for a name you removed, or copy the page into a blank document and read what comes across. Clear comments, tracked changes, hidden rows and sheets, and the author fields in the file's properties. Where the job needs to tell records apart, replace instead of deleting, so Customer 14 stays Customer 14 on every page and the draft can still match it.

A name is not the only thing that points to a person. The federal rule for de-identifying health data lists eighteen kinds of identifiers, from names and phone numbers to any date tied to a person except the year, and any place smaller than a state. Most companies aren't bound by that rule, but the list makes a good prompt for anything that could single someone out.

Some sets can't be cleaned well enough. With a handful of customers, the mix of items, dates and delivery addresses on one order can point to a single buyer even with the name gone. NIST's guide makes the same point: information stripped of identifiers can sometimes be re-identified.

Then we build look-alikes together with your reviewer: documents that copy your real layouts, field positions and the exceptions your team handles, filled with invented values. Your reviewer writes the right answer for each one, the way the team would have, and judges the drafts against those answers.

Look-alikes test only what is in them. Ask your reviewer to build the hard cases in on purpose: the scan that came in sideways, the line that could be two different items, the request that took a phone call to sort out.

The NDA is signed before any document moves. The owner names the reviewer, and the reviewer, with whoever looks after privacy or IT at your company, approves the final set before it goes.

An NDA limits what the other side may do with what you send. It does not change which laws or customer contracts cover a file, so the keep-out list above applies with or without one. The FTC's guide also tells businesses to put security expectations in writing in their contracts with service providers, and to check that they are met. Ask any vendor, us included, where the copies will be kept, who can open them and when they will be deleted, and have the answer in writing before the first file goes.

None of this is legal advice. When a document falls under a law or a contract you aren't sure about, leave it out of the first set and ask your counsel.