Cross-Camera Tracking: From a Detection to an Identity

An investigation often starts with footage of a person nobody can name. The view is clear enough to work from. No photograph of them sits on file, no report carries one, and nobody reviewing the footage recognizes the face. None of that stops you following them across the rest of your cameras. Following somebody does not require a name. It requires something that tells this person apart from everyone else the cameras saw, and that holds when they are seen again somewhere else.

Producing that, and then acting on it, runs in four layers. A camera detects that something is there. A tracker holds that detection together for as long as the object stays in view. A model turns what the camera saw into an identity that can travel beyond the camera that produced it. A comparison then looks for that identity on the cameras around the one that saw it. Each layer does one job and each can fail on its own. A failure in one looks nothing like a failure in another, which is why they are worth taking one at a time. This sits under our guide to multi-camera triangulation, which covers connecting separate sightings of the same thing into a route.

What a camera is actually doing

A camera by itself produces video and nothing else. Detection is the layer that turns that video into statements about what is in it. Every frame of the stream is checked rather than one frame in every few. Frames from all the cameras are batched into a single pass on the graphics hardware, which is what makes that affordable. The cost is one pass over many cameras rather than one pass per camera. On each frame it identifies what is present and where in the frame it sits, and it attaches a score saying how sure it is.

What it looks for is your decision, made camera by camera. A camera over a loading bay can watch for vehicles and people, while one in a corridor watches for people alone. The classes available go well beyond people, and cover vehicles, weapons, protective equipment, text such as a license plate or a street sign, and hazards such as fire.

Each class on each camera carries two numbers. Confidence sets the score below which a detection is thrown away. Raise it and fewer detections survive, including some that were real, while lowering it lets more through including things that were never there. Severity sets how much an alert from that class matters, and every alert from that class carries it.

So this layer produces, for every frame, a set of labeled things with scores, a camera and a time against them. That is a record of what was in view. It says nothing about which person or which car.

Holding one object together inside one stream

That record has a problem the moment you look at it. A person walking across a lobby appears in every frame while they are in view, so one crossing produces a long run of separate detections. Treated individually they would raise hundreds of alerts for a single event, and a day of that would bury whoever is on the desk.

Tracking is what resolves it. Rather than handling each frame on its own, the system follows an object from one frame to the next. It recognizes that the person in this frame is the person who was in the last. The long run of detections collapses into one event with a start, an end and a duration between them.

That continuity survives more than a clean walk across a room. Someone who passes behind a pillar and reappears on the other side is held as one object rather than two. A few frames of absence do not break the track.

All of this happens inside a single stream. A tracker follows an object through the frames of the camera that saw it, and its job ends at the edge of that camera's view. Connecting what one camera saw to what another saw is a separate layer, and it works on a different principle entirely.

Turning a detection into an identity

A tracked object has an identity, but only within the stream it appeared in. Carrying it any further means describing the object in a form another camera can compare against. That is the job of a model, which runs over what the camera saw and produces a vector embedding.

An embedding represents appearance as a position rather than as a description. The model takes the pixels and outputs a set of coordinates, and where those coordinates land is decided entirely by what the thing looked like. Two objects that look alike land close to one another in that space, while two that look different land far apart.

The property that makes this useful is that no person designed what the coordinates mean. The model learned the representation from data, so no individual coordinate corresponds to a feature you could name. There is no axis for jacket color and no axis for height. The information is spread across all of them at once, and only the position as a whole carries any meaning.

That is what separates an embedding from a description. A description is drawn from a vocabulary somebody wrote down, so it can never be finer than that vocabulary allows. A black jacket will always return every black jacket. An embedding is drawn from the image itself, so its resolution is set by the model rather than by the limits of language.

Why an embedding works as a unique identifier

Two detections of the same person are never identical, because the light differs, the angle differs and the person has moved between them. Their embeddings therefore land near each other rather than on one point, which makes matching a question of how near is near enough. What turns that into a workable identity is the size of the space. An embedding space has a large number of dimensions, and each coordinate varies continuously rather than taking one of a few values, so the space is extremely sparse. Two genuinely different people are very unlikely to land close together by accident. Those odds are what let nearness stand as an identity claim rather than a guess.

The practical limit sits in exactly the same place. Somewhere a line has to be drawn deciding what counts as near enough, and wherever it sits it costs something. Draw it tight and the result holds fewer wrong matches while missing more real ones. Draw it loose and the reverse happens. The line is a trade rather than a setting with one correct value.

Matching one camera against another

The embedding a camera's pipeline computes does not stay inside the stream that produced it. It is shared with the pipelines of the cameras next to it, so an object detected on one camera is matched on the ones around it. A match is a comparison against that shared set. That is what carries an identity past the edge of a single view, and it is also what bounds how far the identity travels.

The scoping to neighbors is deliberate. Comparing every new embedding against every embedding an estate of thousands of cameras holds costs in proportion to the estate rather than to the question. It also returns candidates from cameras the object could not physically have reached, which is a longer list and a worse one. Keeping the comparison to neighbors keeps it proportionate and keeps the candidates meaningful.

So a route is assembled hop by hop rather than produced by one lookup. That makes the chain matter in a way an estate-wide comparison would not. A camera not running detection for that class computes no embedding, so an object passes it and the hop does not happen. The trail stops there rather than picking up further along. And a question spanning two distant sites is not one comparison at all.

No comparison can produce a detection that was never made. Where coverage stops, the record stops with it. Two sightings with a gap between them tell you only that the object was in one place and later in another. The route between the two is an inference, and how reasonable an inference it is depends on where those cameras physically sit.

Searching back through what was recorded

Looking back uses the same comparison over a wider range, and the arithmetic behind it does not change. What changes is that every decision affecting the answer was already taken. Across cameras you can still act on what you find. A camera that was not running detection can be switched on today, and tomorrow's route will include it. Looking back offers no such option. Whether a camera was running detection in March is settled, and so is whether its threshold was set high enough, and whether the footage still exists. You are reading a record that an earlier configuration wrote, and the only question is what it happens to contain. What analysis can still do with an incident already recorded covers those decisions and which of them can still be changed.

Three things follow from that. The first is that watching the live cameras and searching what they recorded are not the same piece of work. A deployment set up only to do the first cannot do the second.

The second is whether the embeddings exist yet. Where a period has been processed, the comparison runs against embeddings that are already there. Where it has not, they have to be computed first. That takes time in proportion to the volume of footage, not to the size of your question. Pointing at a month of four hundred cameras is a very different job from a month of four.

The third is easy to miss and worth knowing about. An embedding only means anything relative to the model that produced it, because the coordinate space belongs to that model rather than to the world. Embeddings produced by two different versions of a model are not directly comparable with each other. Over a long look back that can matter, because footage from a year ago may not have been processed by the version running last week.

Retention sits underneath all three, and it has two parts that are often treated as one. Footage that no longer exists cannot be processed, so a period never indexed and since deleted is beyond reach. Identifiers already computed are a separate matter, and they can be kept after the footage they came from is gone. Both numbers are worth checking against how long your cases take to become apparent. A retention window set at installation to fit the storage bought at the time answers a different question entirely.

What you decide

Two settings are yours, and neither should be inherited from a default that was never examined. Both sit in VIDIZMO AI Live Insight, alongside the detection classes each camera watches for. The first is whether identifiers are generated at all. They are by default. Generation can be turned off, or restricted to particular areas, which matters where part of a building is deliberately not watched closely. With it off, a person you cannot already name cannot be followed anywhere.

The second is how long identifiers are kept, which is set separately from how long footage is kept. Holding a record of where something was, with no video left to check it against, is a different proposition from holding both. That belongs with whoever owns policy in your organization rather than with whoever configures the cameras. What is kept is the embedding itself, and no picture is kept with it. A retained identifier is a value derived from appearance rather than a stored image of the person or object it came from. It does not become a gallery subject either, because enrolment is a deliberate act with reference photographs supplied by an administrator. Finding every recording a person appears in covers what enrolment involves and who decides it.

For how a case reaches you in the first place, see how an incident on camera becomes known. For how this sits in a wider camera program, see video intelligence.