On March 31, 2026, about a hundred Apollo Go robotaxis froze in the middle of traffic in Wuhan. A "system failure," according to local police. Passengers were stranded for nearly two hours, and a few vehicles were rear-ended.
No one was injured. But the episode says something specific about autonomous mobility: the average performance of a perception system is not enough. What matters is what the system decides, frame by frame, when the situation is out of the ordinary.
This is exactly where cooperative perception was supposed to help. And it is also where it disappoints, due to the lack of one ingredient: uncertainty weighting.
The offboard sensor: a promise still under-exploited
The idea of cooperative perception is simple. A vehicle only sees what its on-board sensors perceive. A camera mounted on a pole at an intersection, an offboard sensor, referred to as "infrastructure," sees what the vehicle cannot see: the blind spot, the pedestrian behind the truck, the two-wheeler coming from the perpendicular.
Merging these two sources, on-board and offboard, should therefore reduce perception errors. The smart intersection promises a more complete view, earlier.
In practice, this promise remains largely theoretical. Infrastructure sensors are often treated as an auxiliary signal, not as a source of decision-making. We install them, connect them, and then hesitate to really use them.
Why? Because a detection coming from the infrastructure arrives without any confidence indication. The vehicle receives a position, an object class, a bounding box. It does not receive the only information that interests it at the time of the decision: does this detection deserve to be listened to?
The real problem is not the sensor, it is the trust we place in it
Take an equipped intersection. The infrastructure camera signals a pedestrian. Should the vehicle brake?
If the detection is reliable, braking is the right decision. If it is a false positive, a shadow, a reflection, a misclassified object, braking is useless. Repeated, it degrades comfort, wears out components and, above all, undermines the confidence of validation teams in the entire offboard system.
The system therefore makes the most cautious choice: weight the infrastructure weakly, or ignore it. The data exists, but it weighs almost nothing in the fusion. The sensor is installed and amortized on paper, under-exploited in practice.
The limitation appears clearly. A multi-sensor fusion that treats all sources with the same implicit confidence cannot arbitrate between a solid detection and a doubtful detection. It lacks a confidence-per-prediction metric, attached to each detection, in real time.
Probabilistic fusion: weighting each detection by its uncertainty
This is the principle of the TrustalAI reliability layer. Instead of fusing raw detections, we fuse detections accompanied by their uncertainty.
TrustalAI Vision produces, for each detection of a computer vision model, a "Reliability Bubble": an estimation of the uncertainty on the position and size of the object (σx, σy, σw, σh). The tighter the bubble is around the bounding box, the more reliable the detection is.
TrustalAI V-Tracking extends this logic over time: robust, non-deterministic, learning-free multi-object tracking, capable of keeping the trajectory of an object even in dense scenes or noisy environments.
These uncertainties become weights. A highly confident offboard detection weighs heavily in the fusion. An uncertain detection weighs less, without being discarded. The intersection no longer "votes" equally with the vehicle: it votes up to the level of what it actually knows, prediction by prediction.
An important point for an R&D team: none of this touches the customer's model. The layer is plug-and-play, black-box compatible, and works on the existing detection flow, without retraining or accessing the model's IP. It runs at the edge, with a latency of around 20 ms for Vision and a processing budget of <100ms compatible with the decision loop of a vehicle.
Reliability per prediction vs. aggregated monitoring
This is the distinction that structures our work, and it is worth dwelling on.
Most AI supervision approaches observe the behavior of a system after execution. We aggregate metrics, average precision, error rate on a test set, drift observed over a sliding window. This is aggregated monitoring. Useful for a report, an audit, a retraining decision. Useless at the precise moment when the vehicle must brake or not.
Two reasons for this.
First, the aggregate masks the local. A model that is 97% reliable on average can systematically make mistakes on the 3% of cases that matter: night, rain, partial occlusion. The average reassures; the specific case causes accidents.
Second, aggregated monitoring is post-mortem. When the metric goes up, the decision has already been made and the risk has already circulated in the operation. We document a failure; we do not prevent it.
Reliability per prediction reverses the point of view. It attaches a confidence measure to each detection, before the fusion decision is made. It is no longer "how does this model perform in general?", but "can we trust this precise detection, here, now?". For a smart intersection, this is the difference between a sensor we monitor and a sensor we exploit.
What the VEDECOM PoC measures
We tested this approach with the VEDECOM Institute, the national institute for sustainable mobility, on a case of cooperative perception: multi-sensor multi-agent fusion between a smart intersection and an autonomous vehicle. The probabilistic fusion used TrustalAI uncertainties as weights. All benchmarked against 7 fusion methods.
The results, obtained without retraining the customer's AI model and in real time (<80ms):
Indicator | Baseline | With TrustalAI | Reduction |
|---|---|---|---|
False positives | — | — | -83% |
Position errors | 1.44 m | 0.51 m | -65% |
Orientation errors | 6.28° | 2.35° | -63% |
A position error reduced from 1.44 m to 0.51 m is not just a slide figure. At an intersection, half a meter decides whether a pedestrian is perceived in the trajectory or next to it. One eliminated false positive is one less useless emergency braking.
What this changes from an operational perspective:
Offboard sensors become a source of decision-making, not just an auxiliary signal.
Fewer false positives mean fewer emergency stops, hence a smoother drive and better acceptability.
Higher perception confidence paves the way for a gradual reduction of human supervision.
The validation cost of the perception stack drops because we can qualify the contribution of each sensor instead of just undergoing it.
Scope note: these figures come from the VEDECOM PoC in cooperative perception and autonomous driving. They qualify this specific use case. They cannot be transposed as is to other domains without a new benchmark.
Integration and compliance: why it matters now
The EU AI Act classifies ADAS perception and autonomous vehicles as high-risk systems. The consequence is concrete: it is no longer enough to deploy a high-performance AI, we must be able to document the reliability of its decisions before deployment.
A system integrator delivering a cooperative perception system carries this burden of proof. They often work under an obligation of result, and the AI module remains a black box for them, for which they nevertheless assume responsibility. A reliability-per-prediction layer produces precisely the exploitable trace that is missing: a measured confidence, per detection, time-stamped.
In other words, the reliability brick is not just for driving better. It is also for being able to prove it.
FAQ
What is an offboard sensor in cooperative perception?
An offboard sensor is an infrastructure sensor, a camera or lidar installed on a pole, traffic light, or gantry, outside the vehicle. In a smart intersection, it perceives blind spots and objects that on-board sensors cannot see yet. Fused with the vehicle's perception, it completes the scene earlier.
Why do infrastructure sensors remain under-exploited?
Because their detections arrive without any indication of reliability. A fusion system cannot distinguish a solid offboard detection from a false positive. Out of caution, it weights the infrastructure weakly, or even ignores it. The sensor is installed but its contribution remains marginal, due to the lack of a confidence-per-prediction metric.
How does reliability per prediction improve multi-sensor fusion?
It attaches an uncertainty measure to each detection, in real time, before the decision. This uncertainty serves as a weight in the probabilistic fusion: a confident detection weighs heavily, while a doubtful detection weighs less. In the VEDECOM PoC, this weighting reduced false positives by 83% and position errors by 65%, without retraining.
Does the TrustalAI layer modify the existing perception model?
No. TrustalAI Vision and TrustalAI V-Tracking are plug-and-play and black-box compatible. They run on the existing detection flow of the customer's model, without retraining or accessing its IP, at the edge, with a latency compatible with the decision loop of a vehicle (<100ms).
Share
Related articles
