Skip to content
Interface language: English
Audit / Signal

Twelve boxes in a photograph: the limits of AI object counting

Define the unit, compare object identifications and state what an image cannot establish when assessing AI object counts.

Article language: English

Illustration of two people at a table before a large circular structure and mountains.
In this article

The result is twelve. The reference answer is also twelve. Before accepting the test as correct, however, check what was counted. A model could miss one box and identify another twice. A correct total does not repair that recognition; two errors have simply met at the same answer.

We can show this without a photograph or AI. A simple calculation of our own uses twelve synthetic identifiers, p01 through p12. The reported list contains p01 through p11 and p01 again. It has twelve entries, but only eleven distinct identifications. p12 is missing. This is a logical example, not a model result or evidence of photograph-counting quality.

Accepting object counting therefore requires the count, the meaning of one unit and a checkable basis for the result. That applies even to a simple working question about a photograph. An image of a warehouse may show a certain number of visible packages while still not revealing the full state of things behind them.

What is one object in this task?

Imagine a workshop that photographs sets of paper decorations before preparing material descriptions. Boxes of sets sit on a table, beside one open package containing several pieces. "How many products are there?" does not decide whether we count packages, sets for sale or individual parts. Someone familiar with the offering may answer in a different unit from the model analysing the image.

Name the object and the frame's scope. "Count the visible closed boxes on the table" means something different from "How many sets were prepared for dispatch?" The latter may require information the photograph does not contain. An empty package and a complete set can look alike from outside. Image counting should not invent the state of their contents.

A partially visible object also needs an agreed rule. If a fragment of a lid appears at the edge, do you count the box? The answer may depend on the task and whether it can be recognised. Do not decide only after seeing the model's answer. Such a change could give two identical identifications different assessments depending on the desired total.

The authors' documentation for Learning To Count Everything from CVPR 2021 shows counting using boxed examples of objects. This is a method with a particular input, not a universal promise to count anything in a photograph. It is a useful example of the need to specify which kind of distinct objects is to be counted.

In an ordinary business process, a clear definition and marked examples in an acceptance set can serve that role. Your reference should indicate each included object. If two people disagree about a fragment at the edge, record the unresolved question. A randomly chosen number does not become a strong basis merely because it is called an expert answer.

Do not confuse recognising a product type with recognising its individual instances. Visual product search (in Polish) (in Polish) may identify a package model. Counting still requires establishing how many separate instances are actually visible. One class label "box" does not mean one item, and several labels on the same box do not make several items.

Reflections and photographs printed on packaging are separate cases. A drawing of a box on a poster is not another box on the table. A reflection may show the same instance rather than an additional one. The acceptance set should contain such cases if they occur in real material. Prepare the expected decision for the task instead of assuming every similar visible shape must be added.

A small control series can be prepared from a physically known set. First arrange the objects separately, then obscure some, and then add a picture of similar packaging to the background. Record each change. The aim is to compare conditions, not present one representative frame. No such photo series was performed for this article; it is a proposed acceptance test. Its reference should distinguish the known number of things placed on the table from the number whose presence can be confirmed in that particular image.

From a number to identifications that can be compared

OpenAI notes among vision-model limitations that object counts can be approximate. This is a warning about capability, not a measurement of our process. If a company needs an exact count, it must separately accept that condition on representative material.

Keep the image, question and result from the same test. If the tool can mark objects, ask for recognisable identifications and compare them with the reference. In the identifier example, only that check reveals missing p12 and repeated p01. The difference between totals is zero even though the basis of the answer is defective.

Not every solution returns separate identifications. If you receive only a number, you can compare it with the reference but cannot assign the error to a particular box. Do not then invent a list of detected objects from a later model description. The report should preserve that observation limit. A method providing locations can be considered when checking them is part of the task. Its selection also requires acceptance; adding boxes to a screen does not prove they match the process from which the total came.

Hugging Face describes object detection as recognising specified classes with boxes and labels, with numerical scores also present in the API response. That is a different kind of result from a sentence containing only a total. Boxes may aid checking, but their accuracy must also be evaluated rather than automatically treating each one as a separate object.

Check whether several boxes identify the same instance, one covers two neighbours, or a partially visible object was omitted. Changing the threshold for retaining identifications can change their count. Record the setting if available; if processing is unknown, state that limit. Do not compare two totals as though produced under identical conditions when one list was additionally filtered.

Aeneia conceptual photography

Include an image with no objects of the chosen class. Zero should mean that a properly performed analysis found none within its scope. A file read error, an empty response or lack of support for the needed class should not receive the same value. In our example a photograph of an empty table differs from an image of a full table that failed to load. The team should recognise that distinction before using the result in later description work.

Also prepare two similar frames differing by one clearly visible item. Leave the other objects in a similar arrangement. The expected result should change by one according to the agreed definition. Such a pair helps expose an answer based on a general stack pattern or a constant, frequently repeated number. Keep both images and their own labels; do not tell the model the expected difference in the question if it is absent from the real task.

Overlaying boxes requires correctly transferring coordinates to the output image (in Polish) (in Polish). A list of numbers can be correct for a reduced image but drawn wrongly on the original. The reviewer then cannot check which object was indicated. The box total may still look tidy, but the visual evidence points elsewhere.

The input frame should retain the area needed for counting. Preparing an image before a model (in Polish) (in Polish) includes resizing and cropping, which can remove necessary details. If part of the table is outside the analysed material, the result does not automatically describe the entire photographed scene. Do not compare it with a reference for a wider image without noting the difference.

For several images, decide whether you expect a sum of appearances across images or a count of unique physical objects. Two views of the same parcel can yield two correct readings of visible boxes. Adding them does not establish how many things are in the workshop. Recognising a repeated instance requires a separate basis, such as an agreed marking that can actually be read.

If a label is used for that purpose, distinguish counting from barcode decoding and product recognition (in Polish) (in Polish). A model number shared by many items is not the identifier of each individual instance. Seeing the same writing on two boxes does not establish whether one thing was photographed twice or two similar things were photographed.

Aeneia Audit can examine what information a process derives from an image and what supports the next decision. The material is a limited set of photographs with a clearly marked reference and an intended scope of use. This helps assess whether an approximate suggestion is needed or an accuracy condition that the current solution has not yet demonstrated.

Aeneia Audit

Let us establish the unit counted, visibility scope and how individual identifications will be checked. Prepare images like those used at work and your own object markings so the result can be compared with an actual reference.

Discuss acceptance of object counting from photographs

A visible arrangement does not establish a hidden count

Occlusion may leave insufficient basis for an exact count. A regular arrangement of boxes does not prove that a hidden row is full. An estimate can be built on an explicit assumption, but it must be named as such. "At least ... are visible" or "the full number cannot be determined" should match the agreed task and what the image actually reveals.

A lower bound also needs a basis. If three visible fragments might belong to one package, they cannot be counted as at least three items without additional information. First establish that they are distinct. For an estimated interval, show where its endpoints come from. A range from ten to twenty does not become a cautious result merely because it is wide. It can be just as unsupported as one confident number if the image lacks the necessary information.

Appropriate AI refusal when there is no basis (in Polish) (in Polish) is useful here too. Lack of a view should not turn into a confident number inferred from a pattern. At the same time, refusal on a clear, unambiguous image can be a process error. Acceptance should include both countable cases and those requiring a limited result.

Show total errors and errors in recognising distinct items separately. Also retain frame scope and cases needing manual assessment. A result from an open shelf photograph should not receive the same interpretation as one from a dense stack with many obscured items. One combined accuracy figure can hide precisely that difference.

Record the number of tests in each group. A large collection of easy photographs of single objects can dominate results from a few difficult frames, even if the latter correspond to the team's actual work. Also state whether assessment concerns an exact total, an allowed error or agreement of separate identifications. The p01 through p12 calculation shows why these criteria are not interchangeable. If the process is only for preliminary photo organisation, that scope can be accepted deliberately; it should not later be presented as confirmation of the full number of physical things.

Repeat the test after changing how photographs are taken. A new background, angle and distance can change the material the solution uses. Do not attribute improvement only to a new prompt if the boxes were also arranged separately. A clearer arrangement can make the task easier; name that change in the report rather than attributing it solely to model ability.

In our example, one difficult image where a correct number could hide two opposing errors is worth including in acceptance. Only comparison of identifications can establish the basis of the result. Twelve list entries still need to be connected to twelve distinct boxes in the photograph.

Aeneia / Aeneia editorial

Editorial article. Source links are included in the text.

About Aeneia

Brands we have worked with

  • L’Oréal
  • Garnier
  • Netflix
  • Carrefour
  • Warner Bros. Discovery
  • Biedronka
  • HBO Max
  • Wolt
  • Żabka
  • COSRX
  • Embassy of China in Poland
  • Mixa
Aeneia / Initium

From knowledge to a decision

Tell us what you want to put in order. We will agree on where to start.

Contact Aeneia

We partner with and use solutions from:

  • Google
  • OpenAI
  • Microsoft
  • Stripe
  • Anthropic
  • Linear