Every vendor that sells camera analytics to a city will offer a demonstration, and almost every demonstration uses the vendor's own footage, chosen and tuned to show the product at its best, which tells a city very little about how the same software will behave on its own cameras, at night, in the rain, at the angles its cameras were actually mounted. The only numbers that should decide a purchase are the ones a city measures itself, on its own cameras, over long enough to see ordinary weeks and bad ones. This article explains how to run that kind of video analytics proof of concept: how to scope it, what to decide before it starts, how to count honestly, and what the report to council should say at the end. If the analytics are meant to support a real-time crime center, our guide to real-time crime centers covers the wider decision, and our overview of AI video analytics explains what the analytics themselves do.
Why a city pilot is different from a vendor demonstration
A demonstration is designed to answer whether the software can detect something, and a pilot should answer a harder question, which is whether it detects the things your city cares about, often enough and accurately enough, on your equipment, that the people who receive the alerts can act on them without being buried. Those are different questions because most of what decides the answer belongs to the city rather than the vendor: where cameras point, how high they are mounted, how well they see at night, how busy the scene is, and how many alerts the people on duty can handle. A pilot is also the city's first chance to test the parts of a program that have nothing to do with detection, such as where alerts go, who checks them, how long footage is kept and how the public is told, and a pilot that tests only detection leaves the hardest questions for after the contract is signed.
Scope: length, cameras and departments
A pilot of 60 to 90 days is long enough to cover several weekends, a stretch of bad weather, a monthly cycle and at least one busy event, and short enough to report to council within a single budget cycle. Anything shorter tends to catch only good conditions, and anything much longer turns the pilot into an unfunded deployment that nobody has approved.
The number of cameras should be small enough that every alert can be reviewed, which in practice usually means somewhere between a dozen and a few dozen cameras, chosen deliberately rather than taken from whatever is easiest to connect. Include some of the hard cameras, such as the ones that face into headlights, look across a dark park or cover a busy transit stop, because a pilot run only on well-lit, well-placed cameras will overstate how the system performs once it is deployed citywide.
It is worth involving police and one other department from the start, whether that is traffic, public works or parks, because camera analytics is usually easier to justify to a council when the same cameras and platform serve more than one purpose, and because a second department tests whether the city can actually share the system rather than simply assuming it can. Traffic might test a stopped-vehicle rule on a corridor, public works an after-hours intrusion rule at its yard, and parks a closed-park rule, alongside whatever the police department wants to test.
Success criteria, written down before the pilot starts
The most important decision in a pilot is made before any camera is connected, and it is to write down what success means, because criteria written after the results arrive tend to bend toward whatever the results happen to show. Each rule being tested should have its own criteria, agreed by the department that will use it, and the criteria should be the kind of thing a council member could check against the final report.
| Measure |
What it tells you |
How to record it |
| Alerts per rule per day, by hour |
Whether the volume fits the people who will receive it |
Counted automatically by the system, reported weekly |
| Share of alerts confirmed as real on review |
How much of the alert load is useful |
Every alert marked real or false by the person who reviews it |
| Events the system missed |
Whether it catches what matters |
A sample of footage reviewed by a person each week |
| Time from event to alert |
Whether an alert arrives in time to act on |
Compared against the footage timestamp for a sample of alerts |
| Alerts that reached the right person |
Whether the routing works |
Checked against the routing plan for each alert |
| Staff minutes per alert |
What the program costs in people |
Estimated by the people handling alerts, reviewed at the midpoint |
A threshold for each measure, such as a minimum share of real alerts below which a rule is dropped, makes the final decision much easier, and it also protects the city from a vendor quietly adjusting settings during the pilot to improve a number the city did not know it should be watching.
How to count honestly
Every alert during the pilot should be reviewed by a person and marked as real or false, with a short note saying why, because that record is what the final report will be built on and because a vendor's own summary of how the pilot went is not a substitute for it. Misses matter as much as false alerts, and the only way to find them is to review a sample of footage from cameras and hours where nothing was flagged, looking for the events the rules were supposed to catch.
Results should be broken out by camera, by time of day and by weather, since an average across all cameras can hide a rule that works well in daylight on three cameras and fails badly at night on the rest. Any change to rules, thresholds or camera settings during the pilot should be logged with a date and a reason, so that the report can show results before and after the change rather than blending them together. It is reasonable for the vendor to tune the system in the first week or two, but the measured period should start after that tuning is finished and the settings are frozen.
Governance during the pilot
A pilot is still a real use of cameras on real people, so it should run under the same rules the city would apply to a permanent program, rather than under a temporary arrangement that the public was never told about. That means a written use policy covering what the pilot watches for and what it does not, a retention schedule for alerts and clips, a list of who can see the footage, and a plain statement of whether any features, such as facial recognition, are switched off. In cities with a surveillance technology ordinance, a pilot usually needs the same council approval as a purchase, and even where no ordinance applies, telling the council what the pilot is before it starts is better than explaining it afterward.
The routing of alerts deserves the same care, because the people who receive alerts during the pilot take on real work. Decide in advance which alerts go to whom at each hour, whether that is a patrol supervisor, dispatch, a public works crew or a review queue for the next morning, and agree it with the people who will carry the load, including the 911 director if any alerts go to dispatch.
The report to council
The final report should be short enough that a council member will read it and specific enough that the numbers can be checked, and a structure along these lines works for most cities.
| Section |
What it contains |
| What was tested |
Cameras, rules, departments, dates, and the success criteria set at the start |
| Results by rule |
The measures above, by camera and time of day, against each threshold |
| What worked and what did not |
Rules to keep, rules to change, rules to drop, with reasons |
| What it took in staff time |
Minutes per alert, who carried the work, and what that would mean at full scale |
| Governance |
How footage was handled, who accessed it, any issues raised by the public |
| Cost to continue |
The recurring lines for years one to three, and which fund would carry them |
| Recommendation |
Continue, expand, change scope, or stop |
A report that recommends dropping some rules is usually more credible than one that recommends keeping everything, and councils tend to trust a department that can show it tested something and changed its mind. For an example of the kind of plan a pilot like this would test, see our sample real-time crime center plan for a town of 20,000. The cost section should cover the years after any grant ends, which our guide to budgeting a real-time crime center past the grant period explains line by line.
How VIDIZMO approaches a pilot
AI Live Insight is built to support the kind of measurement this article describes, so a city can judge it on its own cameras rather than on a demonstration. It runs on the cameras a city already has, fetching streams directly from cameras or from any RTSP or ONVIF endpoint, so a pilot does not need new cameras or a new recorder. Rules are set per camera, which makes it straightforward to test the same condition on easy and difficult cameras side by side and to freeze settings once the measured period begins. Every alert moves from New to Acknowledged to Resolved, and resolving an alert or marking it a false positive requires a note, with each action recorded against the person who took it, so the review record the report depends on is kept by the system rather than in a separate spreadsheet. The operator dashboard brings forward the cameras with active alerts, and routing can be configured to the city's own workflow, by alert type and hour, so the pilot tests the arrangement the city would actually use.
If you are planning a pilot and want to see how that record looks on cameras like yours, see how AI video analytics runs on a city's existing cameras.