When an AI says it can't tell

By The Pantry Pro team · 6 September 2026 · Updated 8 September 2026 · 8 min read

A grocery receipt goes into a phone camera and comes back as a list. One line on the paper reads GV WHL MLK GAL. Software reading that line has two honest options: decide it means a gallon of whole milk, or say it is not sure. Most systems take the first option, because a list with a hole in it looks broken and a list with no holes looks finished.

For anything you are going to act on, that preference is backwards.

A wrong answer costs more than a missing one

The two kinds of error are not equally expensive.

A missing answer announces itself. A blank line, a row marked unresolved, a note saying three of forty lines could not be read — each is something a person can look at, and looking takes seconds, because the receipt is still in your hand.

A wrong answer announces nothing. Suppose a line that should read 1.2 lb of ground beef is read as 12. Nothing about the number looks unusual on a screen of forty others. It goes into the inventory; a week later a recipe check says there is plenty of beef, so beef stays off the shopping list, and the shortfall is found at the stove. By then the number has been copied through three screens, and none records that it started as a guess about a smudge.

That is the shape of the problem. An error inherits the authority of every system it passes through, while a gap keeps pointing at itself until somebody closes it.

The review screen is where a draft becomes a record

That is what a review step is for. Photographed into Pantry Pro, a receipt is read into a draft purchase, and every line shows two things side by side: what was printed, and what it was matched to. GV WHL MLK GAL sits next to the item the app thinks it means.

The work is in the numbers. Tap a line to correct its quantity or unit or match it to something you already own; the ✕ beside a line excludes it. Lines that matched nothing are marked new and become items when you confirm. Confirming does three things at once: the new items are created, every line teaches that store's shorthand so the same abbreviation matches on its own next month, and — if you leave the box checked — the food is added to your inventory, dated by the receipt's own purchase date rather than today.

None of that is what happens when a barcode is scanned. Those digits are decoded on the device and looked up, with no model involved and nothing to review, which is a different kind of certainty.

What the standards bodies say about this

The idea has a name in the federal literature. In 2020 NIST proposed four principles of explainable artificial intelligence, and the fourth is knowledge limits. The announcement puts it plainly: "The idea is that if a system has insufficient confidence in its decision, it should not supply a decision to the user" (NIST, 18 August 2020).

Its AI Risk Management Framework says the same from the risk side. Under valid and reliable it warns that deploying systems "inaccurate, unreliable, or poorly generalized to data and settings beyond their training" increases risk, and that risk management "may need to include human intervention in cases where the AI system cannot detect or correct errors" (NIST AI RMF 1.0, section 3.1).

Neither document is a regulation. The principle does not get smaller as the stakes do; it only gets cheaper to follow.

Two different things "can't tell" can mean

They are worth separating, because they have different fixes.

The first is a reading failure. Thermal paper fades, a fold crosses a line, the light was bad, the abbreviation belongs to one store and appears nowhere else. The system was asked something it could answer and the input was not good enough; a second photograph often fixes it.

The second is not a failure of the reading at all. A recipe asks for 4 cups of broth. The cupboard holds 2 cans. Both facts are exactly right, and whether 2 cans is enough cannot be answered from them, because nothing on hand says how many cups are in that can. No amount of better vision closes that gap; what closes it is a person supplying the conversion once.

Software that reports both cases the same way throws away the more useful half of the information, because the two need different repairs: the first needs a better look at the paper, the second needs a fact nobody has written down yet — how much one can holds. In Pantry Pro the first is a correction on the review screen, stored so it is made once; the second is the can's size, recorded on the item, after which cans convert to cups for every recipe that asks by volume.

A confidence score is a claim, not an audit

Extraction tools often attach a number to each line, or a word like low. That number is the system's own estimate of how likely it is to be right, produced by the same machinery that produced the answer. Useful for deciding what to check first, it is not independent evidence, and a system can be confident and wrong at once — the failure the NIST framework describes when a model meets data unlike anything it was trained on.

Treat confidence as a sort order for your attention, not a verdict. The lines it does not flag are not correct because it did not flag them.

Reading extracted data as a draft

A review step only helps if it is used as one. Four habits make the difference.

Check numbers before names

A wrong name is usually obvious, because you know whether you bought anchovies. A wrong quantity or unit looks exactly like a right one. Read the quantity column first and the units with it, because 12 oz and 12 lb look alike in a list of numbers.

Check what is not there

A photograph of a shelf is weak evidence of absence. The jar may be behind the cereal box. Anything that says an item is gone because a camera did not see it is a question rather than a fact, and it should never remove anything on its own. A shelf scan in Pantry Pro marks such rows Expected but not visible and leaves them where they are.

Make correcting as cheap as confirming

If accepting the whole list is one tap and fixing one line is five, the list gets accepted. That is a design property, not a discipline problem, and it is the thing worth judging a tool on: how many taps it takes to disagree with it.

Keep the original until you have checked

A stored receipt photograph, or the paper in a drawer for a week, is what turns a suspicious number back into a fact. Extraction without a retrievable source is a claim that cannot be settled.

What a useful "can't tell" looks like

Reporting uncertainty is not the same as shrugging. A blank that says nothing is almost as bad as a guess, because it gives a person no way to close it. A useful one names what could not be done in the specific case — the amount could not be converted, not "an error occurred" — shows the evidence it does have, gets counted rather than quietly producing a shorter list, and stays out of automatic actions by default while being easy to add back.

Pantry Pro gives every required ingredient in a recipe one of four verdicts — have it, not enough, need, and can't tell — and the fourth is what makes the other three worth believing. An unresolved line shows its working: two cans of broth, which location they are in, and that cans do not convert to cups. A shopping list leaves those lines off and says how many; a checkbox adds them anyway, noted as unverified, so the cupboard gets checked in the shop instead of guessed at — the same check a planned week gets in planning meals around what expires.

None of this is specific to food. Any system that reads documents, transcribes speech or classifies images faces the same choice on every item, and the answer is the same. A tool that will not guess produces shorter lists and looks less impressive in a demonstration. What it gets in exchange is that its gaps carry information: a blank is a real question, put where a person can answer it, rather than a number nobody will go back and check.

Check what your next recipe needs

Opens Pantry Pro, which is free to start. It asks you to sign in, or to start a pantry if you do not have one yet.