Ask a veteran shift supervisor how they knew something was wrong, on the night they caught the problem everyone still talks about, and the answer is almost never a rule. It did not look right. The trucks were queuing differently, the crew at bay two was standing instead of moving, there was product where product does not sit. Expert operators run an anomaly detector in their heads, tuned by years of watching normal, and its defining property is that it needs no list of the abnormal: it recognizes departure itself. The industrial-monitoring world has spent a decade trying to reproduce that faculty in software, mostly on sensor streams, and the visual version has always been the hard case, because normal on a production floor is high-dimensional, seasonal, and social. This article is about the approach that has recently become practical, putting written questions to footage instead of training detectors for every contingency, what it genuinely does, and the boundaries a buyer should insist on hearing. It belongs to the operations thread of this series under our pillar guide to AI-powered video analytics for manufacturing.
Two ways to find the abnormal
Everything this series has built so far runs on the first philosophy: enumerate what matters, train a detection type for it, tune thresholds, and alert. It is the right philosophy for the enumerable, and the safety and quality articles show how far it reaches: Personal Protective Equipment (PPE), zones, proximity, presence, defects with taxonomies. Its structural limit is equally plain, and the unsafe-behavior article develops it for the safety domain: a preset list can only contain what someone anticipated, and the incidents that reorganize a plant's understanding of its own risk are, with humbling regularity, the unanticipated ones.
The second philosophy inverts the question the first one puts to every frame. Instead of asking is any of these known patterns present, it asks does what the camera sees depart from what this scene normally is, or from what a described expectation says it should be. The mechanism that makes this practical for video is the one this series introduced earlier: frames cached from the live pipeline are analyzed by a multimodal model against a question written in language. Describe anything unusual in this loading operation. Does the material staging in this aisle match ordinary practice for this shift. Is anything about this queue of trucks atypical. The question defines the analysis, no fixed output vocabulary constrains the answer, and the unit of analysis is a sequence, so the departure can be a pattern over minutes, the queue that stopped rotating, the crew that stopped moving, rather than an object in a frame.
The two philosophies are complements, not rivals, and the architecture that uses them well is the escalation chain our detection-to-escalation article details: cheap, continuous, enumerated detection watches everything and flags the interesting; the open question examines what was flagged, and adds the judgment layer no preset list carries. Running open-ended analysis against every frame of every camera would be economically absurd; running it against the moments the detectors and rules have already surfaced is cheap, and it is also where it is most useful, because the ambiguous residue of a good detection layer is precisely the anomaly question in miniature.
What plants catch with open questions
The catalog of finds from deployments running this pattern reads like the supervisor's instinct, transcribed. Process drift that no threshold owned: material staged in a new place because a door was blocked, a workaround becoming a habit, the slow migration of an informal practice across a shift, each visible as the scene not matching its description of normal. Situations composed of legitimate elements: every object in frame is allowed, a truck, a crew, a pallet, and the configuration is wrong, the truck at the wrong bay for its cargo class, the crew clustered where the procedure wants them distributed. Precursors with no preset type: the machine venting differently, the product surface catching light in a new way, the queue behavior that precedes the yard gridlock by twenty minutes. And the long tail of never-agains, each plant's private list of the one-off events that produced a corrective action and no way to watch for recurrence, which become, for the first time, expressible as standing questions.
What accumulates is also worth naming: every answered question lands as a structured event, searchable and trendable like any detection, so the anomaly layer feeds the same reviews, Key Performance Indicators (KPIs), and correction ledgers the rest of this series builds. An anomaly that recurs stops being an anomaly and becomes a candidate for a trained detection type or a zone rule, which is the healthy lifecycle: the open question discovers, the enumerated layer industrializes, and the question moves on to the next unknown.
Limitations to understand before buying
This capability attracts inflated claims, and the honest boundaries are load-bearing enough that a vendor who omits them is describing a different product.
It is not real-time, and cannot be made so by enthusiasm. Analysis runs on cached frames on a cadence, downstream of the live pipeline; it answers what has been happening, seconds to minutes behind the moment, which suits supervision, drift, and investigation, and disqualifies it from any response that depends on the current frame. The live tier stays with the enumerated detectors and engineered controls, and a deployment should be able to say, condition by condition, which tier owns what.
It carries no accuracy percentage either, and the reason is structural rather than a vendor omission. An open question has no enumerable set of correct answers to score against, so no precision or recall figure exists, and any vendor quoting one for open-ended visual analysis is measuring something else. The workable evaluation is the one this series prescribes for every judgment layer: run it on your footage against your questions during commissioning, review what it surfaces and misses with the people who know the floor, and calibrate trust from that evidence.
The question's wording shapes the answer, which is a property to manage rather than a defect to await fixes for: standing questions live in a governed register with versioned wording, per the discipline the unsafe-behavior article establishes, and wherever an answer will be relied on, the prompt is preserved beside it.
And the same flexibility that makes the method powerful is, unmanaged, its exposure. A system that can be asked anything of footage of a workforce will eventually be asked the wrong thing, so scope is agreed in advance, question authorship is controlled, and the no-discipline, no-identification commitments of the works-council article bind the anomaly layer exactly as they bind every other. The anomaly question is about the plant, not the people, and the register is where that boundary lives in writing.
The question register grows more valuable over time
Something underappreciated happens to a plant that runs this layer for a year: the question register becomes a codification of operational judgment that previously lived only in senior heads. Every entry is a distillation, this is what wrong looks like at the dock, this is what a healthy changeover queue does, authored by the people who knew and now legible to the people who will learn, and the register survives the retirement, the transfer, and the reorganization that have always dissolved that knowledge before. Sites begin to treat it accordingly: new supervisors read the register as onboarding, the quarterly review prunes and sharpens it the way a good team prunes its runbooks, and the never-again list, once a folder of incident reports, becomes a living set of standing watches. The technology made the register possible, but the asset is the judgment it captured, and it belongs to the plant in a way no model ever will.
Keeping the questions current as the plant changes
The operational craft of anomaly detection is mostly the craft of describing normal, and two practices separate the deployments that mature from the ones that flail. The first is seasonality: normal shifts with product mix, weather, staffing, and the calendar, so the standing questions carry context, normal for a changeover day, normal for the summer shutdown, rather than one eternal baseline, and the quarterly retuning pass the alert-fatigue article prescribes reviews the question register alongside the thresholds. The second is adjudication: anomaly findings route to people equipped to judge them, the supervisors whose instinct this layer is imitating, and their dispositions, real finding, known condition, badly phrased question, feed the register the way false-positive reasons feed threshold tuning everywhere in this stack. The supervisors, notably, tend to become the layer's best question authors within a quarter, because they have spent careers knowing what did not look right and lacking anywhere to write it down.
What it costs
Anomaly capability prices differently from detection, and buyers should understand the shape rather than the sticker. The enumerated layer's cost scales with cameras and detection types; the question layer's cost scales with how much footage gets asked about, because each analysis is model inference over a clip. The escalation-chain design is therefore not just an accuracy architecture but a cost architecture: detectors and rules gate what reaches the model tier, standing questions run against flagged moments and scheduled samples rather than continuous streams, and the register's size governs spend directly. A site can run a meaningful anomaly program, the never-again watches, the drift questions on critical zones, at modest inference cost, and can widen it deliberately as findings justify. The failure mode to avoid is the inverted pyramid, open questions run indiscriminately across every camera while the cheap enumerated layer sits under-tuned, which buys maximum cost for minimum precision and is, not coincidentally, the shape some vendors' pricing hopes you choose.
How VIDIZMO fits
The split is the one this series has kept consistent. VIDIZMO AI Live Insight runs the live, enumerated layer on the plant's existing cameras, on the plant's own hardware on site, and caches the frames the open questions draw on. AI Intelligence Hub runs the multimodal analysis and the event-driven workflows that trigger it, an event from the live layer starting the graph that puts the question, with answers returning as structured events, searchable, alertable, trendable, and preserved with their clips in the Nexus portal under its access control and retention. Neither product does this alone, the question register and its governance live in the deployment's written terms, and a site that wants only the enumerated layer runs it standalone until the day the unanticipated argues otherwise.
The adjudicators deserve a week-one expectation too: early findings skew verbose, the model describing three unremarkable things on its way to the one that matters, and dispositions plus sharper question wording tighten the signal within the first review cycles. Teams told this in advance read the early noise as calibration; teams surprised by it read it as failure, and the difference is one paragraph in the kickoff deck.
Timing the pilot also matters more than teams expect. A month containing a changeover, a maintenance window, and ordinary production teaches the layer's questions the full range of legitimate abnormality, while a pilot run across four quiet weeks calibrates normal too narrowly and floods the register with findings the first busy week. Choosing a representative month is free, and it is the difference between a register tuned to the plant and one tuned to a lull.
Expect one cultural effect worth planning for: the first month's findings will include conditions that turn out to be known, tolerated, and quietly relied upon, the unofficial staging area, the sanctioned-by-silence shortcut, and the review that receives them is making policy whether it means to or not. Deciding in advance who owns those calls, and that the default response is regularize-or-remove rather than prosecute, keeps the layer's discoveries from souring the floor on the whole program.
The pilot design follows from everything above: take the five entries from the plant's never-again list, phrase them as questions, and run them against the zones where they happened, silent, for a month. The review at the end asks the only question that matters here: what did the layer see that the preset list never would have, and every deployment so far has ended that meeting with a longer question register than it started.