Reliability of an AI prediction: your model is a GPS that never displays its accuracy circle

The reliability of an AI prediction cannot be read in its confidence score. It is measured at the moment of inference, prediction by prediction, just as a GPS displays a radius of uncertainty around the blue dot. Your model, on the other hand, only shows the dot.

You know the scene. You are driving, the blue dot sticks to the road, the circle is tight. You enter a tunnel and the screen changes its behavior: the circle widens, the dot becomes an area. The system does not apologize. It does not go silent either. It tells you that it is less sure, instead of pretending otherwise.

This is what almost all industrial AI systems fail to do. They output a bounding box, a class, a forecast. A blue dot, always sharp. Hence the question of this article: why is the circle missing, and what would it take to display it?

Your GPS circle is not an image, it is an API value

First shift, and it often surprises technical teams. The circle is not a graphic convention. It is a calculated, numerical value transmitted at every position.

In Android, the Location.getAccuracy() method returns a horizontal accuracy in meters, associated with an explicit confidence level: the 68th percentile. The documentation states it directly. If a circle of this radius is drawn around the reported position, "there is a 68% chance that the true location falls within this circle" (Android Developers).

Three details matter in this sentence.

• The radius applies to one measurement. It is not aggregated over the day, nor over the journey.

• It carries a declared confidence level. About once in three times, the actual error therefore exceeds the announced radius. This is accepted, documented, and actionable.

• The hasAccuracy() method exists. The system knows how to say that it has no radius to provide. A vision model, however, always outputs a number.

The service itself reasons the same way. GPS.gov does not announce a dry accuracy: the average daily signal error is kept below a given threshold with a 95% probability, and the accuracy obtained then depends on satellite geometry and signal blockages (GPS.gov).

No one in navigation announces accuracy without stating the corresponding probability. Yet that is exactly what industrial AI does when it announces "94% mAP" and stops there.

Why the circle widens, and why ground truth has nothing to do with it

The tunnel makes a good opening image, but the mechanics are easier to read in an urban canyon. There, the signal is not lost, it is degraded. Visible satellites become scarce and poorly distributed, building facades reflect multiple paths, and the geometry of the problem deteriorates.

The receiver has an indicator for this: the DOP, or dilution of precision, available in GDOP, PDOP, HDOP, VDOP, and TDOP. DOP is a factor that links the precision of raw measurements to that of the final positioning. Here is the decisive point, documented by the ESA: the geometry matrix from which it derives does not depend on the measurements, only on the geometric configuration, and is calculated from the almanac (ESA Navipedia).

In other words, the receiver does not need to know where it actually is to know that it is poorly positioned to find out. No ground truth is involved.

Yet this is the operational condition of a production line, an instrumented intersection, or a moving vehicle: no one has access to ground truth at the moment the decision is made. The ITS PoC of TrustalAI V-Tracking was validated this way, on 100 one-minute videos and without ground truth. MOTA raised from 42% to 72%, precision from 0.65 to 0.91, recall from 0.51 to 0.83.

The parallel with DOP is not rhetorical, it is structural. In both cases, we estimate the quality of an output based on what is observable at the moment of inference.

What aviation calls integrity, and what industrial AI lacks

Second shift. The distinction we apply to AI was standardized thirty years ago in another field, for good reasons.

Integrity, as defined by the ESA, is the measure of trust that can be placed in the correctness of the information supplied by a navigation system. It includes the ability of the system to provide timely warnings when its use becomes hazardous (ESA Navipedia).

Four parameters frame it. They form, without intending to, the specifications for a reliability layer.

The rule connecting them fits in one line: the Protection Level must remain below the Alert Limit. Otherwise, the system declares itself unavailable rather than continuing to display a dot. RAIM is the embedded implementation of this (Inside GNSS). The parallel stops at the concept, and to be perfectly honest: TrustalAI claims no aerospace certification, nor ICAO compliance. Aviation provides a vocabulary and a precedent, not a label. What it demonstrates is that an industry faced with irreversible errors ended up separating two metrics that AI still confuses: the overall performance of the system, and the reliability of the information it delivers right now.

No, your model's confidence score is not a circle

This is the most frequent and most legitimate objection. "My detector already outputs a confidence score, isn't that enough?" No. Five results from literature explain why, each from a different angle.

It talks about the class, not the position, and the correlation is 0.217

A detector produces two distinct things: a class label with an associated probability, and a bounding box resulting from a regression. The score applies to the former.

Jiang and co-authors measured the gap. The Pearson correlation between the classification confidence score and localization quality, measured in IoU, is 0.217, which is virtually zero (Jiang et al., Acquisition of Localization Confidence for Accurate Object Detection, ECCV 2018, figure 2). Their dedicated localization branch rises to 0.617, which confirms that the missing metric is indeed a separate metric entirely.

In short: the score says how likely the model thinks it is a pedestrian. It does not say where that pedestrian is, nor to within how many centimeters. Yet it is this second piece of information that controls braking.

On the class itself, it is poorly calibrated

Even if we only care about the class, the score remains a poor candidate. Guo and co-authors showed that modern networks are poorly calibrated and systematically overconfident, unlike older architectures (Guo et al., On Calibration of Modern Neural Networks, ICML 2017).

The metric they introduce, the ECE (Expected Calibration Error), has become the industry standard. Its practical conclusion fits in one line: a score of 0.95 does not mean "correct 95 times out of 100".

It remains high on things the model has never seen

The most unsettling result, and the one most directly related to our tunnel. Nguyen, Yosinski, and Clune fed trained networks images that were completely unrecognizable to a human—structured noise. Classification with 99.99% confidence (Nguyen et al., Deep Neural Networks are Easily Fooled, CVPR 2015).

The score does not collapse when the model goes outside its domain. It remains high. This is the opposite behavior of a GNSS receiver, and it is the heart of the problem: an out-of-distribution system continues to display a sharp dot. We covered the industrial case in detecting when a part is out of the training distribution.

It ignores context, even though calibration error depends on it

Küppers and co-authors showed that the calibration error of a detector depends on the position of the object in the image and increases near the edges. They propose D-ECE, a calibration conditioned on position and scale (CVPR Workshops 2020).

We are back to the urban canyon. A small, off-center, partially obscured object is not in the same situation as a centered and sharp object. The geometry of the problem has changed; the score, however, does not budge.

It confuses two uncertainties of different natures

Kendall and Gal distinguish two families. Aleatoric uncertainty is the noise inherent in observations, sensor, lighting, occlusion; more data does not reduce it. Epistemic uncertainty is the ignorance of the model itself—what it has not learned. The authors point out that the latter is crucial for safety-critical applications (NeurIPS 2017).

The highway falls under aleatoric: a little noise, tight circle. The tunnel falls under epistemic: the system is out of its validity domain. A real circle separates the two. A confidence score aggregates them into a single number and loses the information that matters.

The automotive industry, moreover, has a name for this territory. The ISO 21448 (SOTIF) standard deals not with failures, but with the inadequacies of the intended function: cases where the system works as specified yet still causes a dangerous situation. It distinguishes unsafe-known scenarios, which are outside the operational domain, from unsafe-unknown scenarios, which are outside the training distribution.

Reliability per prediction or aggregate monitoring: they are not the same measurement

Many teams address this problem with monitoring tools. This is useful, but it is not the same thing. Monitoring is reviewing the journey in the evening. The circle is seeing it at the intersection. We explained the distinction in detail in AI monitoring and reliability per prediction and in why monitoring is no longer enough in production.

A clarification, because this confusion often recurs. Even real-time monitoring remains aggregate monitoring: it narrows the window, but does not drill down to the prediction level. It will tell you that the rate of false positives has increased over the last hour. Not which of the 180,000 detections in that hour was doubtful.

The two approaches are not mutually exclusive. Model drift and data drift remain core challenges, and the reliability layer feeds into them rather than replacing them. Simply put, monitoring answers "is my model degrading?", while prediction reliability answers "can I trust this specific prediction, right now?". This is the same shift described in reliability per prediction and overall performance.

What a circle looks like in computer vision: the Reliability Bubble

In 2D vision, the circle has a precise shape. TrustalAI Vision attaches four uncertainty values to each detection: σx and σy for position, σw and σh for box dimensions. This is what we call the Reliability Bubble. The tighter the bubble fits the bounding box, the more reliable the detection.

The textbook example fits in two lines, and it flips the intuition of perception teams.

Look at the distance alone: the object at 6 m is the closest, so a priori the most urgent. Look at the radius: it is the most risky. The same detections as before, prioritized by what matters, and this separation is what makes a prediction actionable.

In the vision family, this measurement comes at a low cost: -80% in position and orientation errors, 92% fewer false positives, and a latency of 20 ms on CPU and 8 ms on NVIDIA Jetson Orin (source: Client Deck TrustalAI V10). All results measured in real operating conditions against each product's baseline model.

When multiple sources observe the same scene, this uncertainty serves as a fusion weight. This is TrustalAI Vision and its multi-source version V-Fusion, whose PoC conducted with the VEDECOM Institute measured -65% in position errors, -63% in orientation errors, and -83% in false positives against seven probabilistic fusion methods. A fusion approach that knows which source to trust rather than blindly averaging, a topic detailed in our article on cooperative perception.

The same circle, in predictive modeling: the interval

The transposition to predictive models is immediate. A point forecast, "failure in 18 days", is a blue dot. The usable version is "failure in 18 days ± 6 days, 95% CI". This is what TrustalAI Predictive produces on time series and scoring models: -81% in errors, -84% in false positives, and 20 ms latency.

The change in decision-making is clear. With a single point, a planner guesses. With an interval, they decide based on a window: intervene early if the downtime cost is high, or wait if the upper bound allows margin. We developed this reasoning in the confidence interval as a decision metric and in why a point forecast is a bet.

What a real circle costs, and why most methods remain in the lab

Drawing a circle around a prediction has been a solved problem in research for a long time. Drawing it in 20 ms on the production line, without retraining, is not. This is where the gap between a paper and a production line lies.

The KITTI benchmark, on the same fine-tuned YOLOv8 baseline (1,493 frames, 10,049 labeled data points, times measured on NVIDIA L4), sets the scale.

Two readings, and the second matters as much as the first.

• TrustalAI achieves the best uncertainty estimation quality on AUPR, AUROC, and AURC among the tested methods, with an inference time approximately 13 times faster than MC-Dropout, 24 times faster than a Bayesian network, and 5 times faster than an ensemble.

• MC-Dropout achieves a better NLL, 11.85 versus 13.78. Stating this is more honest than omitting it, and the point stands: it pays 191 ms for this NLL. On edge devices, a method running at 191 ms is not a slow method; it is an unusable method.

However, this approach does not invent a new category. It occupies a recognized slot in the reference taxonomy of uncertainty quantification methods, the Single Network Deterministic Methods → External Methods branch (Gawlikowski et al., A survey of uncertainty in deep neural networks, Artificial Intelligence Review 2023). It is the only one compatible with both real-time demands and plug-and-play integration: no retraining, no multi-sampling, no access to model weights, meaning strict black-box compatibility.

What regulation already requires, and what it does not yet require

The EU AI Act, Regulation (EU) 2024/1689, sets out a fundamental requirement in Article 15. High-risk AI systems must achieve an appropriate level of accuracy, robustness, and cybersecurity, and behave consistently in this regard throughout their lifecycle. Article 15(3) adds that these levels of accuracy and the corresponding metrics are declared in the instructions for use.

Notice the nuance; it is structurally key. The instructions cover aggregate accuracy, established once, upstream. It says nothing about the prediction at 2:32 PM. The operator, however, never decides "on average": they decide one prediction at a time.

Neighboring articles point in the same direction. Article 9 organizes risk management, Article 12 mandates logging, and Article 13 regulates the information provided to the user. Article 14 requires effective human oversight: to intervene usefully, an operator must know when to intervene, and an average accuracy will never tell them that.

A reliability layer contributes to this compliance by producing, on a per-prediction basis, the missing link between the instructions and the decision. It does not guarantee it: compliance remains a comprehensive process driven by both the provider and the operator. Our deep dives detail this topic in what the law imposes on industrial AI systems and in risk classification.

What it changes downstream: three action regimes

A circle is only useful if something does something with it. A programmable logic controller cannot resolve a doubt that is not communicated to it. Communicate it, and three regimes emerge.

• Tight circle: the prediction is followed automatically. The robot maintains its pace, the part moves along the line.

• Intermediate circle: the system slows down, takes another image, or queries a second source. In industrial robotics, this is the difference between a two-second slowdown and a complete line stoppage.

• Wide circle: human escalation or refusal to act. We are in the tunnel: the system knows it no longer knows, and says so before making a mistake.

This routing transforms an uncertainty measurement into an operational gain: fewer unnecessary stops on vision lines, fewer false rejects in quality control, fewer perception incidents in autonomous driving, and fewer false alarms in urban video surveillance.

On the integration side, the constraints are deliberately low.

The layer plugs in parallel to the existing machine learning model, without touching the decision pipeline, both at the edge and in the cloud. The details are described in making a vision model reliable without retraining and in edge reliability at 20 ms. For system integrators, the argument is even more direct: you deliver a machine that knows when it does not know, and you can prove it upon acceptance.

It is not the blue dot that gets you to your destination

Four things have shifted along the way.

• The circle already exists as a documented value, per measurement, with an explicit confidence level.

• It is calculated without ground truth, from the structure of the problem, at the moment the decision is made.

• Aviation separated accuracy and integrity thirty years ago, and certifies the latter.

• The native confidence score is not this circle: its correlation with localization quality is 0.217, and it remains high on what the model has never seen.

Uncertainty is often presented as an admission of weakness. A model that doubts is supposedly a worse model. It is the opposite. A system that always outputs a sharp dot is not safer; it is simply less informative. The circle is not an admission of weakness: it is what makes the dot usable.

It is not the blue dot that gets you to your destination. It is the circle.

FAQ: Reliability of an AI Prediction

How do you know if an AI prediction is reliable?

By attaching an uncertainty measurement calculated at the moment of inference, independently of the model's confidence score. Concretely, we produce a boundary around the output: a radius in 2D vision, an interval in predictive. A prediction is reliable when this boundary remains below the threshold tolerated by the action it triggers. The same principle as the GPS circle around the blue dot.

Isn't my model's confidence score enough?

No, for three measured reasons. It relates to the class, not the position: the Pearson correlation with localization quality is 0.217 (Jiang et al., ECCV 2018). It is poorly calibrated and overconfident in modern architectures (Guo et al., ICML 2017). And it remains very high on out-of-distribution inputs, up to 99.99% on unrecognizable images (Nguyen et al., CVPR 2015).

What is the difference between aggregate monitoring and per-prediction reliability?

Aggregate monitoring measures after execution, over time windows or batches: it detects data drift or performance degradation. Per-prediction reliability measures before the decision, output by output. The former tells you that the model is degrading; the latter tells you whether you can trust this specific prediction, right now. The two are complementary and address different questions.

What is a Reliability Bubble?

It is the visual representation of the uncertainty attached to a detection in 2D vision. TrustalAI Vision produces four values per detection: σx and σy for position, σw and σh for the bounding box dimensions. The tighter the bubble fits the box, the more reliable the detection. Downstream modules use it to prioritize objects by risk rather than distance or score.

Do you need to retrain your model to get this measurement?

No. The reliability layer runs in parallel with the existing model and does not access its weights or the client's proprietary data: compatibility is black-box. This is what distinguishes external methods from Bayesian or ensemble approaches, which require modifying or duplicating the model, costing between 70 and 360 ms per image on a datacenter GPU.