When an AI says it can't tell
A grocery receipt goes into a phone camera and comes back as a list. One line on
the paper reads GV WHL MLK GAL. Software reading that line has two honest
options: decide it means a gallon of whole milk, or say it is not sure. Most
systems take the first option, because a list with a hole in it looks broken and
a list with no holes looks finished.
For anything you are going to act on, that preference is backwards.
A wrong answer costs more than a missing one
The two kinds of error are not equally expensive.
A missing answer announces itself. A blank line, a row marked unresolved, a note saying that three of forty lines could not be read — each of those is something a person can look at, and looking takes a few seconds, because the receipt is still in your hand.
A wrong answer announces nothing. Suppose a line that should read 1.2 lb of ground beef is read as 12. Nothing about the number looks unusual on a screen of forty other numbers. It goes into the inventory. A week later a recipe check says there is plenty of beef, so beef stays off the shopping list, and the shortfall is discovered at the stove. By then the number has been copied through three screens, and none of them records that it started as a guess about a smudge.
That is the shape of the problem. An error inherits the authority of every system it passes through, while a gap keeps pointing at itself until somebody closes it.
What the standards bodies say about this
The idea has a name in the federal literature. In 2020 NIST proposed four principles of explainable artificial intelligence, and the fourth is knowledge limits: "The system only operates under conditions for which it was designed or when the system reaches a sufficient confidence in its output." The same article puts it plainly — "The idea is that if a system has insufficient confidence in its decision, it should not supply a decision to the user" (NIST, 18 August 2020).
NIST's AI Risk Management Framework, released in 2023, makes the same point from the risk side. Under valid and reliable it says: "Deployment of AI systems which are inaccurate, unreliable, or poorly generalized to data and settings beyond their training creates and increases negative AI risks and reduces trustworthiness." The same section adds that risk management "may need to include human intervention in cases where the AI system cannot detect or correct errors" (NIST AI RMF 1.0, section 3.1).
Neither document is a regulation. The principle they describe does not get smaller as the stakes do; it only gets cheaper to follow.
Two different things "can't tell" can mean
They are worth separating, because they have different fixes.
The first is a reading failure. Thermal paper fades, a fold crosses a line, the light was bad, the abbreviation belongs to one store and appears nowhere else. The system was asked something it could answer, and the input was not good enough. A second photograph often fixes it.
The second is not a failure of the reading at all. A recipe asks for 4 cups of broth. The cupboard holds 2 cans. Both facts are exactly right, and the question of whether 2 cans is enough cannot be answered from them, because nothing on hand says how many cups are in that can. No amount of better vision closes that gap. What closes it is a person supplying the missing conversion once.
Software that reports both cases the same way is throwing away the more useful half of the information. The first says take another photo. The second says answer one question and this will never come up again.
A confidence score is a claim, not an audit
Extraction tools often attach a number to each line, or a word like low. It is worth knowing what that number is: the system's own estimate of how likely it is to be right, produced by the same machinery that produced the answer. It is genuinely useful for deciding what to check first. It is not independent evidence, and a system can be confident and wrong at the same time — which is the failure the NIST framework describes when a model meets data unlike anything it was trained on.
So treat confidence as a sort order for your attention, not a verdict. The lines it flags are worth checking first. The lines it does not flag are not correct because it did not flag them.
Reading extracted data as a draft
A review step only helps if it is used as one. Four habits make the difference.
Check numbers before names
A wrong name is usually obvious, because you know whether you bought anchovies. A wrong quantity or unit looks exactly like a right one. Read the quantity column first, and read the units with it, because 12 oz and 12 lb look alike in a list of numbers.
Check what is not there
A photograph of a shelf is weak evidence of absence. The jar may be behind the cereal box. Anything that says an item is gone because a camera did not see it should be treated as a question rather than a fact, and it should never remove anything on its own.
Make correcting as cheap as confirming
If accepting the whole list is one tap and fixing one line is five, the list gets accepted. This is a design property more than a discipline problem, and it is the one worth judging a tool on: how many taps it takes to disagree with it.
Keep the original until you have checked
A stored receipt photograph, or the paper in a drawer for a week, is what turns a suspicious number back into a fact. Extraction without a retrievable source is a claim that cannot be settled.
What a useful "can't tell" looks like
Reporting uncertainty is not the same as shrugging. A blank that says nothing is almost as bad as a guess, because it gives a person no way to close it. A useful one does four things.
It names what could not be done, in the specific case rather than in general: the amount could not be converted, not "an error occurred".
It shows the evidence it does have. For the broth: two cans, in the pantry, and cans do not convert to cups. That is often enough for a person to answer instantly.
It gets counted. A shopping list built from a recipe should say how many ingredients it left off because their stock could not be verified, rather than quietly producing a shorter list.
It stays out of automatic actions by default, and is easy to add back. Excluding an unverifiable line from a shopping list, while saying so and offering a box to include it anyway, is the version that fails safely in both directions.
Pantry Pro treats "can't tell" as one of four verdicts on every recipe ingredient, alongside have it, not enough and need, for the reason above: the fourth one is what makes the other three worth believing.
None of this is specific to food. Any system that reads documents, transcribes speech or classifies images faces the same choice on every item, and the answer is the same. A tool that will not guess produces shorter lists, admits more often that a person is needed, and looks less impressive in a demonstration. What it gets in exchange is that its gaps carry information. A blank is a real question, put somewhere a person can answer it, rather than a number nobody will ever go back and check.
Sources
- NIST, NIST Asks A.I. to Explain Itself, August 2020
- NIST, AI Risk Management Framework 1.0, section 3.1 Valid and Reliable
- Pantry Pro, User manual
More guides
- A freezer inventory you will actually keep
- First in, first out, at home
- How long food really keeps
- Planning meals around what expires
- What date labels on food actually mean
- What food waste costs a household
- What groceries cost per person
Keep track of it
Pantry Pro keeps a real inventory of the food in your house — what you have, where it is, and when it runs out — from a photograph of a receipt or a shelf, and tells you what you can cook from it tonight.