AI Quality Control in Manufacturing: Find the Cost of Missed Defects and False Rejections

An inspection report can show how many parts were rejected without showing whether the inspection made the right decisions. Some rejected parts may meet the specification. Some accepted parts may contain a defect that appears only at a later operation or at the customer. Both mistakes matter, but they create different costs and require different evidence.

That is the starting point for AI quality control in manufacturing. Before choosing a camera or a model, establish what the current process misses, what it rejects unnecessarily, and who can verify the difference. The useful opportunity may be in existing inspection images, machine records or quality history rather than a new installation.

This article develops the questions raised in Embiggen X’s French-language webinar on quality control. It adds a practical framework for an English-speaking operations team: define the decision, connect the evidence, test alongside the existing process and keep a named person responsible for the outcome.

01 Watch the quality-control webinar

The discussion brings together Camille, Stéphane and Nicolas to explore hidden quality costs, visual judgement, process correlations and the relationship between inspectors and AI. Watch the original French recording below. The pilot framework in this article is an editorial application of those themes, not a transcript or a promise of results.

02 Measure both kinds of inspection error

A defective part accepted as good is an escape. A conforming part rejected as defective is a false rejection. If a model labels defects as positive, these correspond to false negatives and false positives. Agree on that convention before comparing dashboards, because teams sometimes use the same terms for different outcomes.

A lower rejection rate does not, by itself, demonstrate an improvement. It could mean that the process produces fewer defects. It could also mean that inspection has become less sensitive and more defective parts are passing. Equally, a higher rejection rate could reflect a genuine production problem or a test that is rejecting acceptable variation.

Start by separating the inspection decision from the verified condition of the part. The decision is what the current system said. The verified condition requires an agreed reference: a measurement, a suitable follow-up test or a documented expert adjudication. Neither the inspector’s first judgement nor the model’s confidence is automatically ground truth.

Use the same reference method when comparing people and models. Sample accepted parts as well as rejected ones where practical, and record the limits of that sampling. Looking only in the scrap bin cannot tell you how many defects escaped. Looking only at customer complaints misses defects that have not yet been reported.

03 Find opportunities hidden inside the rejection total

Consider the illustrative shift below, drawn from the webinar’s supporting slide example. Two hundred parts are inspected. The initial report says 188 passed and 12 were rejected. A follow-up review finds that three of the rejected parts were good, while one defective part passed. The totals did not change; the understanding of the process did.

The opportunity is to make both error paths visible. The three good parts suggest avoidable loss worth investigating. The escaped defect suggests a different failure with a potentially different consequence. It would be misleading to subtract one from the other and call the net result an acceptable trade.

There is also a denominator problem. In this illustration, 190 parts are actually good and ten are defective. Three false rejections out of 190 good parts measure something different from three out of all 12 rejects. For each metric, write down the numerator, denominator and verification method. Keep the raw counts alongside percentages, especially when defects are uncommon.

This example does not show an AI improvement. It shows the baseline an improvement claim needs. A model should earn its place by improving a defined decision under an agreed constraint, such as reducing unnecessary reinspection without increasing escapes beyond the quality team’s accepted limit.

04 Build a cost picture beyond scrap

The ASQ cost-of-quality framework distinguishes prevention and appraisal from internal and external failure costs. That distinction helps explain why the scrap total is an incomplete business case: inspection effort, rework and failures discovered after delivery sit in different parts of the picture.

For a proposed pilot, build a small cost ledger for the chosen part family. Record the material and processing already invested in scrapped parts, the time spent reworking or reinspecting, and the cost of investigating an escape. Add documented disruption where it can be traced to the problem. Keep uncertain estimates separate from costs the team can substantiate.

Avoid counting the same loss twice. If a rework estimate already includes labour, do not add those hours again as a separate saving. If recovering capacity is the main benefit, identify what would use that capacity. Freed time is not automatically a cash saving, and a theoretical increase in output has little value if the next operation remains the bottleneck.

The business question becomes concrete: which decision creates enough avoidable work or loss to justify a test? That may be a recurring false rejection, a defect discovered too late, or a queue of parts waiting for expert review. Choose one before expanding the scope to every quality problem in the factory.

05 Check the records before adding equipment

The webinar asks whether new sensors or cameras are necessary. The practical first step is an inventory, not an assumption that every factory already has adequate data. Existing machine parameters, inspection images, quality histories and drawings may be useful. Their suitability depends on coverage, traceability and whether they represent the decision being studied.

Take a handful of recent parts and try to reconstruct their journey. Can the team connect a serial or batch identifier to the process settings, inspection result and eventual disposition? Are timestamps consistent? Can it distinguish the original inspection from a reinspection? A spreadsheet export can be a useful starting point, but mismatched identifiers can undermine even a sophisticated model.

For visual inspection, retain examples of acceptable variation as well as defects. Lighting, camera position, material finish and part orientation can change what an image contains. If the current images do not show the relevant feature, additional imaging may be necessary. That conclusion should follow the evidence review rather than precede it.

Label quality deserves its own workstream. Have inspectors define defect categories and record uncertain cases instead of forcing every borderline example into an immediate yes or no. Where reviewers disagree, resolve the specification or create an escalation rule. Teaching a model inconsistent labels can reproduce disagreement at greater speed.

06 Choose the decision AI will support

There are several distinct opportunities inside the phrase AI quality control. A visual model can help flag a suspected surface defect. A model built from production records can help prioritise parts for further inspection. Analysis of process history can identify patterns associated with later failures. These are different tasks, with different inputs and validation requirements.

The webinar’s process example is particularly useful: an individual setting can be within its permitted range while the combined production history still deserves investigation. A predictive association may point engineers toward a relationship they have not examined. It does not establish the physical cause or authorise changing a production setting without engineering review.

Describe the intended workflow in one sentence. For example: after the existing inspection, flag a subset of uncertain parts for a quality engineer to review with the original image and process record. That sentence defines when the model runs, what it produces and who acts. It is much more testable than an ambition to automate quality.

Our separate Siemens predictive-quality article examines a specific manufacturing example. Here, the broader decision is whether your process needs better detection, better prioritisation or better investigation. Select the task that matches the evidence you can collect and the action the team can actually take.

07 Give inspectors evidence they can challenge

An unexplained red flag leaves an inspector with little basis for agreeing or disagreeing. A more useful review screen connects the flag to the original evidence: the relevant image region, the part identity, the specification and any process information used in the assessment. It should also allow uncertainty and disagreement to be recorded.

A Scientific Reports study by Senoner and colleagues examined explainable AI in visual inspection, including an experiment with 48 factory workers at Siemens in Zug. Its findings support evaluating explanations as part of human–AI collaboration. Its scope was a particular inspection setting using visual heatmaps, not a guarantee that any explanation will improve any factory process.

For a pilot, assess the combined workflow rather than the model alone. Do inspectors understand what the flag refers to? Can they recognise when the evidence is irrelevant? Does checking it take longer than the existing inspection? Are overrides recorded with reasons that can later be reviewed? The interface and escalation path are part of the operational test.

A highlight is a prompt to investigate, not proof of a defect. Keep access to the original image and the applicable criterion. Avoid designing the screen so that accepting the model is effortless while challenging it is difficult. The quality owner should be able to review both model errors and situations where a correct suggestion was overruled.

08 Run a bounded test alongside the current process

Start with one part family, one clearly defined defect or review decision, and a named quality owner. Agree on the reference method and the decision criteria before inspecting the results. A pilot is easier to interpret when the scope is narrow enough to explain why an error happened and what the team would do about it.

Use historical records to check feasibility, then evaluate on examples the model has not learned from. Where possible, separate evaluation by time or production batch rather than randomly splitting near-identical images of the same part. Otherwise, a test can reward familiarity with the dataset instead of performance on genuinely new production.

A shadow test lets the model make recommendations while the authorised inspection process continues to control release. Compare the recommendations with verified outcomes and log disagreements. Do not silently route production through the new model because its early examples look persuasive. Changes to inspection or release require the organisation’s normal quality approval.

A calendar deadline can keep discovery focused, but it cannot create missing evidence. A short trial may be enough to discover broken identifiers or unusable images. It may be far too short to observe a rare, important defect. Set the evidence requirement as well as the review date, and be prepared to extend collection rather than declare success from too few cases.

At the review, report escapes, false rejections, review time and the number of unresolved examples. Include the cost of data preparation, integration and ongoing supervision. Show where performance differs by part variant or operating condition. A single overall score can hide the exact situation that matters most to the quality team.

09 Decide what happens after the pilot

There are three useful outcomes: proceed with a controlled next step, improve the evidence and retest, or stop because the opportunity is not strong enough. A stopped pilot can still be valuable if it prevents a larger investment built on unreliable labels or a benefit that cannot be realised in production.

Before expanding, define who monitors performance, what changes trigger revalidation and how the team returns to the established process if the new workflow becomes unreliable. New materials, equipment changes or a revised specification can alter the conditions under which the model was evaluated. Record the model version alongside the decisions it supports.

Keep learning deliberate. An inspector’s correction is evidence to review, not a reason to retrain automatically without checking labels and re-evaluating performance. The same discipline applies to a changed threshold: moving it can alter both escapes and false rejections, so its effect needs to be visible to the quality owner.

The next conversation with your team can be simple: which error do we understand least, what evidence would let us verify it, and which decision could we test safely alongside today’s process? Those answers give AI a specific job. They also give operations a way to judge whether it has done that job well.

Identify one quality-control decision worth testing. Discuss your workflow with Embiggen X.