Do we need to clean up our data before we try AI?

"Our data is a mess" usually describes the whole ERP: years of items, vendors and accounts nobody has looked at closely. A document job touches a thin slice of it. Drafting one sales order or one coded invoice means looking up a handful of records and nothing else.

• Order entry and quoting read the item master (item numbers, descriptions, selling units and pack quantities), the customer part number cross-reference, and each customer's account, ship-to and price records.
• AP and invoice coding read the vendor master (vendor IDs, names, remit-to addresses, payment terms and any tax reporting flag), the chart of accounts or the job cost codes, and the open POs, receipts or subcontracts each invoice bills against.
• Billing reads the customer master, the contract or rate sheet, and the tickets, timesheets or work orders that record the finished work.

So the useful question isn't whether your data is clean, but whether the records one job reads are good enough for that job. The UK government's Data Quality Framework (December 2020) describes data quality as fitness for purpose, says the level needed depends on the purpose, and adds that data is unlikely to be equally fit for every use.

ISO 8000, the international series on data quality, takes the same line. Its part on concepts and measuring (ISO 8000-8, 2015) splits quality into syntactic, semantic and pragmatic kinds, and says that judging whether quality is sufficient takes a threshold suited to the business for each thing measured; it leaves those thresholds to you. Its part on exchanging master data (ISO 8000-110, 2021) calls its own requirements necessary but not sufficient, and puts the accuracy of master data outside its scope. Neither asks you to perfect every record before using any.

In a proof of concept on old documents, every draft sits beside what your team actually did. When your reviewer rejects a draft, the reason usually falls on one side of a line: the document was read wrong, or the record the draft needed was missing, out of date or there twice. The second kind is a data finding, and the test hands you a list of them, each tied to a real document and the fix someone made by hand that day.

The UK framework's six dimensions give that list a vocabulary. Read your reviewer's rejections against them:

• Completeness, whether the records are present: a customer part number with no cross-reference row, a cost code opened mid-year that never made it into the export.
• Uniqueness, whether anything is duplicated: one supplier under two vendor IDs, one product entered twice with different units.
• Consistency, whether values contradict each other: an item stocked by the box in one record and sold by the each in another.
• Validity, whether values are in the range and format expected: a cost code retired years ago that people still pick, a ship-to with half an address.
• Timeliness and accuracy, whether the record still matches reality: last year's price, a remit-to address the vendor stopped using.

Work through the list starting with the gaps that came up most:

• Duplicates on the records the job reads. Two vendor IDs for one supplier, or two items for one product, split the history and invite double payments and wrong prices. Each needs a person to decide which record survives.
• Cross-references for the customers who send the most documents. A match your reviewer confirmed while scoring the test is already the fix; your team enters it the way it enters any other.
• Units and pack quantities on the items those customers order. One wrong conversion turns a correctly read line into a wrong order.
• Codes your team stopped using but never closed. Close them, so no draft, and no new hire, can pick them.

What can wait is most of it: descriptions written in three styles, inactive items nobody orders, vendors idle for years, classification codes and catalog attributes the job never reads. They matter for other work, but none of them stops a document job, and cleaning them first delays the test that would tell you what does.

A cleanup with no job attached also has no finish line. It treats every record alike, including the many no document will ever read, and it ties up the people a test needs as reviewers. Meanwhile new items, vendors and codes keep arriving, so a file cleaned in spring has drifted by autumn.

Some drift comes from outside. GS1, the standards body behind GTIN product numbers, sets guiding principles for when a product change needs a new GTIN: when trading partners are expected to tell the changed product from the old one, when there is a regulatory or liability disclosure, or when shipping, storing or receiving it substantially changes. A GTIN is never reused, so when a maker changes a pack and issues a new number, a cross-reference built on the old one quietly stops matching. That needs a standing check, not a one-time scrub.

The UK framework names the better habit: use root cause analysis to fix data quality problems at their source rather than applying temporary fixes, because the symptoms often get treated instead of the cause. A test finds causes cheaply. If most rejected drafts trace to one customer's part numbers, fix how those numbers reach the cross-reference. The framework also concedes that perfect data quality may not be achievable, one more reason not to wait for it.

Two exceptions. If your own team can't work from the records a job reads, the desk runs on memory, and writing that memory down comes first. And some fixes, such as merging vendors with payment history, change accounting records; your accountant decides how. None of this is accounting advice.

The vendor master deserves care beyond any AI test, because it decides where payments go. A 2018 SEC report on public companies hit by cyber-related frauds describes criminals who broke into real vendors' email accounts, asked company staff to change the vendors' banking information and attached doctored invoices. Procurement passed the request to the people who maintain vendor data, and payments went to the criminals; several companies learned of it only when the real vendor asked about unpaid invoices. Their procedures already required vendor data changes to be verified, yet staff treated the emails alone as enough, and afterwards each company set out to strengthen that verification.

Two more public sources add habits. The NHS Counter Fraud Authority's quick guide to mandate fraud (version 2.0, October 2022) warns that fraudsters may first ask for the contact details on file to be changed, so check whether those details changed recently, and it calls for periodic reviews of the supplier database to weed out expired or obsolete records. The IRS's TIN Matching service lets payers that file information returns check name and TIN combinations before filing. On a mismatch the payer gets a CP2100 or CP2100A notice, and a payee listed a second time within three years must provide a copy of a Social Security card or an IRS Letter 147C, not just a fresh W-9. How your company handles changes and notices is for your accountant and counsel; this is not tax or legal advice.

In a test, and in any live run after it, the drafts read the vendor master and never change it. A draft can notice the things below and flag each one for the person on your team who already owns vendor changes. It never edits a vendor, never approves a change and never contacts the vendor:

• A remit-to address or bank detail on the invoice that differs from the vendor record.
• A letter or email asking to change payment details, attached to an invoice or sent along with it.
• One supplier that matches two vendor records by name, address or phone number.
• An invoice from a vendor that has been dormant for years, or from one set up recently with little history behind it.

With copies of 20 to 30 documents your team already finished, send a spreadsheet of only the master records those documents touch: for orders, their items with units, pack quantities and those customers' cross-reference rows; for invoices, those vendors, the cost codes or GL accounts in use that period and the open POs. Leave out tax IDs and bank account numbers. Send it as it is today, uncleaned, so the test sees what a live run would. Nothing connects to your systems, and the NDA comes before anything moves. After a yes, the job runs live for about 30 days, with a person approving each item before it posts.