This comes up in almost every first conversation: "how many images do we need to collect?" Answering "it depends" sounds like a dodge, but it is honest — what decides the outcome is not the number of images, but how many different situations they cover. Here is what that means in practice.
Why the count alone says little
A thousand images of the same specimen, in the same position and under the same lighting, teach the model exactly one situation. Two hundred images spanning different material batches, different shifts and the real spread of positioning teach it something much harder — what normal process variation looks like.
And that is the core of the task. The model does not learn "what a defect looks like" in isolation. It learns to tell a defect apart from what is still an acceptable deviation. To do that, it first has to see how wide that band of acceptable deviation is in your process.
NOK examples are the bottleneck, not OK ones
Images of good products are easy to collect — an hour of line time is enough. The problem is defective examples, because a well-running process by definition produces few of them. At a scrap rate of a fraction of a percent, accumulating a few dozen specimens of one specific defect naturally can take weeks.
So the cheapest thing you can do today — before making any decision about automation — is to start archiving defects. Not just photographing them, but setting the physical parts aside. A phone photo taken in the office has limited value, whereas a stored part lets you photograph the defect later under the target imaging conditions, once the station is roughly configured.
What "representative dataset" means
A dataset is representative when it covers the variation the system will face after deployment. In practice it is worth checking whether the data spans:
- different material batches and different suppliers, if the material is ever switched
- different shifts and operators — because how the product is fed and positioned is never identical
- different cavities, moulds or tools, if the product is made on several in parallel
- the full severity range of the defect — from borderline to unambiguous, not just the most obvious cases
- colour variants and product versions meant to run on the same station
- real drift in imaging conditions: optics getting dirty, lighting ageing, changes in ambient light
A test set the model has never seen
Part of the data has to be set aside and left out of training. If accuracy is measured on the same images the model learned from, what you measure is its memory, not its ability to handle a new part on the line. Numbers from that kind of measurement always look excellent and are always worthless.
That is why the pilot separates the training set from an independent test set from the start, and the figures in the qualification report — recall, precision, false-accept and false-reject rates — come only from the latter.
Borderline cases weigh more than obvious ones
A defect visible from two metres away will be caught by practically any solution. Inspection quality is decided by borderline cases: a scratch at the edge of tolerance, flash just under the acceptance threshold, contamination at the size where your own people start to hesitate. Those are what determine where the decision threshold really belongs.
A dataset built only from extreme examples produces a model that fails precisely where the decision actually matters. That is why acceptance criteria — what still counts as OK and what does not — are agreed before training, rather than fitted to the result afterwards.
How much data fits in a pilot
A standard pilot covers one product, up to three main defects and up to 500 annotations, over 2–4 weeks. That bound is deliberate: the pilot answers whether the case is detectable at all under the available conditions — before you invest in a full production station. It is a feasibility test, not the final dataset.
If the pilot goes well, the dataset grows later in a controlled way: difficult cases from production go back into Trainer, a new model version is produced, and Server publishes it only to the selected stations. Data is not something you collect once.
What to do if you have no data yet
The worst option is waiting until "enough has accumulated". Start archiving defects from next week, recording at least the date, the shift and the defect type for each one — that record often turns out to be more valuable than the photo itself.
If a particular defect occurs so rarely that it cannot be gathered naturally, specimens are sometimes produced deliberately. That is acceptable, on one condition: such parts must be labelled as artificially made. A deliberately created scratch rarely looks exactly like one from the process, and mixing the two without distinction distorts validation and produces a result you cannot trust.
If you are unsure whether your current material is enough, that is a good topic for an initial consultation — assessing the available data and imaging conditions is the first step of a pilot, before any training takes place.