A network of traffic cameras films day and night, and no one annotates what they film. Yet MOTA, precision, and recall require a ground truth. Reliability without ground truth consists of qualifying each prediction of the tracker from the stream, at the moment it is used.
The problem: a tracker can only be measured with labels
Take, for example, an intersection equipped with four cameras. The multi-object tracking system assigns an ID to each pedestrian, bicycle, and vehicle, then tracks it from frame to frame. The supervision center's alerts rely on these tracks. The operator has a simple question: is this tracking good right now? Classical metrics cannot answer this, and the reason is written in their definition.
What MOTA measures
MOTA (Multiple Object Tracking Accuracy) refers to the tracking accuracy metric defined within the CLEAR MOT framework (Bernardin and Stiefelhagen, 2008). It counts three types of errors across a sequence:
false negatives: objects present in the scene that the tracker did not follow;
false positives: tracks that do not correspond to any real object;
identity switches: objects that change their identifier along the way.
The calculation adds up these errors frame by frame, then divides the total by the number of objects present in the ground truth. The MOTA is 1 minus this ratio.
An identity switch refers to the case where a single object changes from one identifier to another over the course of the sequence. The cost of an identity switch in ADAS and video surveillance is the subject of a dedicated article.
Look closely at the formula. The denominator is the number of annotated objects. The numerator also depends on the annotation, since a false negative is an annotated object that the tracker missed. By construction, without annotation, MOTA has no denominator. We deduce this from the formula itself: no study is needed to see it.
Precision and recall follow the same logic. Precision relates correct detections to all detections produced. Recall relates retrieved objects to all annotated objects. To know that a detection is correct, one must know what was actually in the scene.
Huang et al. (2020) recall this for CLEAR MOT and IDF1: these metrics require ground truth annotations. Tuning a tracker therefore remains confined to training data, and its parameters may prove suboptimal on test scenes.
Even with a ground truth, what to measure is still debated. According to Luiten et al. (2020), historical tracking metrics over-weight either detection or the association of detections between frames. Their HOTA metric seeks to balance detection, association, and localization. The debate is over the right way to count errors. No one in this debate proposes to do without the reference.
Why the field has no labels
Ground truth refers to the human annotation of what is actually in each frame. For tracking, it is demanding. The MOTChallenge benchmark provides, for each object and at each frame, a bounding box, an identity, and a visibility level (Dendorfer et al., 2020). The MOT17 version updates the MOT16 sequences with more precise labels, and subsequent versions significantly increase the number of annotated boxes.
A camera at 25 frames per second produces 1,500 frames per minute. Each one would have to carry all its boxes and all its identities. On a continuous stream, annotation cannot keep up. It comes after the fact, on a sample, or never.
In operations, the situation gets even more complicated. A tracker validated on an annotated dataset discovers new intersections, new camera angles, rain, night. Its performance can slip without anyone measuring it, due to a lack of labels on these scenes. Here we find model drift in production, applied to tracking.
MOTA, precision, and recall are therefore evaluation measures. They tell you how good the tracking was on an annotated sequence. On the track that the downstream system is about to use right now, they say nothing.
Research is working on reference-free evaluation, but the field remains young. Huang et al. (2020) propose a self-evaluation score based on internal properties of the tracks, such as their number and average length. The authors themselves list the limitations of their method. And to validate it, they compare it to IDF1, a metric calculated with a ground truth. Even a method designed to do without a reference needs a reference to prove itself.
Estimating the reliability of a track without ground truth
Our response starts from this separation between operations and proof. In operations, the tracker consumes no ground truth. For proof, any evaluation metric is calculated against a reference. The two moments are different, and they call for different tools.
What is measured
TrustalAI V-Tracking provides real-time object tracking for computer vision. It is robust, deterministic, and learning-free. It does not need ground truth to function: it relies on the existing detection stream, without retraining (TrustalAI Client Deck V10, 2026). We have explained elsewhere what learning-free tracking implies. Here, remember one thing: no site annotation is required to run it.
V-Tracking estimates reliability without available ground truth, which is the real condition of a production stream. This estimation falls under reliability by prediction. Reliability by prediction refers to a measure calculated for each individual prediction, at the moment it is produced and before the decision, by a reliability layer independent of the model. For an operator, this amounts to knowing how much confidence to place in each track, at the very moment the downstream system uses it.
The product plugs-and-plays onto the existing detection model. This model remains a black-box: V-Tracking works on its outputs, without accessing your intellectual property.
Careful not to confuse this with the confidence score. The confidence score is the number that the detector itself attaches to each box. Reliability metrics, on the other hand, come from a layer placed alongside the model.
Deterministic: same stream, same result
Deterministic means that the same stream always produces the same result. A tracking decision can therefore be replayed identically, then audited. Suppose an alert is falsely triggered at an intersection. The team takes the video, runs the processing again, and finds the exact same tracks, allowing them to search for the cause in the right place.
This is not a given for all deep learning pipelines. The PyTorch documentation is explicit:
“Completely reproducible results are not guaranteed across PyTorch releases, individual commits, or different platforms.” (PyTorch, 2026).
The same note specifies that results may differ between CPU and GPU, even with the same random seed. Some cuDNN algorithms and some CUDA operations are not deterministic. One can force deterministic variants when they exist, but they are often slower. A deep learning model is not random for all that. Its reproducibility is tuned and paid for in performance, and it is not guaranteed across platforms.
For a system integrator bound by an obligation of results, this property carries weight. Replaying the same stream and finding the same output is the starting point for any incident analysis.
Results of the ITS PoC
Here are the details of the results of the ITS PoC, conducted in the field of Intelligent Transport Systems. It covered 100 videos of one minute of traffic. Reliability was estimated without ground truth available in operations, just like on a real camera network (TrustalAI Client Deck V10, 2026).
Indicator | Baseline | TrustalAI V-Tracking |
|---|---|---|
MOTA | 42 % | 72 % |
Precision | 0.65 | 0.91 |
Recall | 0.51 | 0.83 |
Results of the ITS PoC: MOTA of 72% versus 42% for the baseline, precision of 0.91 versus 0.65, recall of 0.83 versus 0.51 (TrustalAI Client Deck V10, 2026).
Read simply, these figures say two things. On the recall side, the baseline found about one in two objects, and V-Tracking finds more than four out of five. On the precision side, about one in three detections was false with the baseline, compared to less than one in ten with V-Tracking.
These three indicators remain evaluation measures. Like MOTA, precision and recall are calculated by definition against a reference. They serve to prove the value of the tracking. In operations, it is reliability by prediction that is used to decide.
The figures that follow do not come from the ITS PoC. They are the metrics from the product card.
On its product card, TrustalAI V-Tracking displays x2.1 objects tracked, -96.7% false detections, and -46% identity switches (TrustalAI Client Deck V10, 2026).
All results measured in real operating conditions against each product's baseline model.
The proof remains on your side. A TrustalAI PoC is successful when your team reproduces our accuracy figures on your own operational data, with your own ground truth (TrustalAI Client Deck V10, 2026). In production, ground truth does not enter the loop. It serves as a tool for proof.
The reference comes later, the decision cannot wait
The difference between the two approaches lies primarily in the reference. What does each need to exist?
Evaluation against ground truth | Reliability in operations | |
|---|---|---|
What is needed | Annotations, frame by frame | The detection stream alone |
When | After annotation, often never | At the moment of prediction, before the decision |
Granularity | An aggregated score over the sequence | Each prediction, thus each track |
Usage | Proof, PoC, benchmark | Decision of the downstream system |
The two columns are not mutually exclusive. The first proves, the second allows action. The pitfall, in our view, consists in asking the first to do the job of the second.
Monitoring tools remain in the logic of the first column. They wait for numerous predictions, calculate metrics, then report after execution. It is a post-mortem analysis: useful for understanding, too late for deciding. The table reflects the logic of the reliability granularity slide of the Client Deck V10.
Others analyze after execution: the decision has already been made, the action has already been taken. TrustalAI brings reliability at prediction time, before the decision is made.
For a supervision center, the difference is very concrete. A weekly MOTA report, when it exists, tells you that the tracking dropped on Tuesday. Reliability by prediction tells you, at the moment the alert is triggered, if the track initiating it is trustworthy. We compare the two approaches in AI monitoring and reliability by prediction, then from the perspective of overall model performance.
What it changes for an urban or embedded stream
The same constraint is found everywhere a camera runs without an annotator behind it.
On urban streams of Smart Cities and ITS, tracking becomes unstable and no ground truth is available. The challenge is to obtain actionable alerts for traffic management and public safety.
In ADAS, phantom detections and missed obstacles trigger unnecessary emergency braking. A track qualified before the decision helps the downstream system sort things out.
In drones, trains, and planes, scenes are dense and calculation is often done at the edge, closest to the sensor. Tracking more objects in a complex environment supports safer navigation.
In mobile robotics, the question is posed in the same terms, as the scene changes faster than the annotations. In all these cases, no one annotates the production stream. Ground truth is used to validate the system, before and during a PoC. Then, each decision relies on a reliability measurement available in real time.
Frequently Asked Questions
How to evaluate a tracking algorithm without ground truth?
In operations, we do not calculate MOTA: without annotation, this metric has no denominator. Instead, we qualify each prediction at the moment it is produced. TrustalAI V-Tracking operates this way: it relies on the existing detection stream, without annotation or retraining. Ground truth retains a role, that of proof, during a PoC or a benchmark on your own data.
How is MOTA calculated in multi-object tracking?
Over all frames, false negatives, false positives, and identity switches are summed. This total is divided by the number of objects present in the ground truth, and this ratio is then subtracted from 1 (Huang et al., 2020). The metric comes from the CLEAR MOT framework (Bernardin and Stiefelhagen, 2008). A MOTA of 70% means that cumulative errors represent 30% of the annotated objects.
Why is MOTA not enough to judge a tracker in production?
MOTA requires a ground truth that production does not provide: it comes after annotation, and often never. It judges a past sequence, not the track that a system is about to use. Even with annotations, the measurement is debated: according to Luiten et al. (2020), historical metrics over-weight either detection or association. MOTA has its full place for proving, much less for deciding.
What does a deterministic tracker change for auditing?
A deterministic tracker always produces the same output for the same stream. A tracking decision can therefore be replayed identically, then analyzed after an incident. This is not automatic in deep learning: PyTorch does not guarantee perfectly reproducible results between versions or platforms (PyTorch, 2026). TrustalAI V-Tracking is deterministic, which makes every tracking decision replayable.
Do we need to retrain V-Tracking for a new site or a new camera?
No. TrustalAI V-Tracking is learning-free: it requires no retraining and operates on the existing detection stream. It plugs-and-plays onto your detection model, which remains a black-box. To judge its behavior on a new site, the right tool remains a PoC on your own data, with your own ground truth.
Share
Related articles
