Integrations, Manufacturing, AI Intelligence Hub, AI Live Insight

Combining Sensor Data With Video for Maintenance: What It Takes

Warehouse staff checking data on a laptop and a tablet between shelves

A vibration alarm on a pump tells a reliability engineer that something about the bearing has changed, and nothing about what the pump looks like at that moment. Footage of the same pump shows the water on the floor, the guard someone left open or the carton jammed against the coupling, and nothing about the reading that made the moment matter. Each record on its own leaves the technician guessing, and joining them by hand means a historian screen, a camera system and two clocks open side by side, which is why most plants only do it after a failure serious enough to investigate.

Internet of Things (IoT) predictive maintenance puts sensors on the assets that matter and raises an alarm when a reading drifts, and this article explains why bringing video to those alarms is worth doing, describes a design for doing it, and is plain about its status. Today the analysis pipeline in AI Live Insight reads video, and bringing a sensor alarm into it is possible with tailoring, built for each plant as an engagement rather than delivered as a shipped connector. It is the sensor part of our guide to predictive maintenance with video AI, which covers what cameras add to a maintenance program as a whole.

What IoT predictive maintenance records, and what a camera records

The condition monitoring system or historian knows the measured side of a failure in detail, including the reading, the threshold it crossed, the asset tag, the time and usually the trend over the weeks before. What it cannot know is anything outside its sensors, such as a seal weeping onto the floor, a jam building at the infeed, or a contractor who opened a guard an hour earlier to clear something. The cameras hold that visible side for whatever they happen to cover, and they know nothing about vibration velocity or winding temperature.

An investigation needs both, and in most plants the two meet only in a person's head. A reliability engineer reads the alarm time from one screen, scrubs the camera recording in another to the same minute, allows for a clock offset nobody has measured, and types what they found into the work order. That effort gets spent after a failure worth investigating, and almost never on the routine alarm a technician clears on a walk-round, which is exactly where a clip would have saved the walk.

In research and vendor writing, sensor fusion for predictive maintenance usually means feeding several sensor streams into one model that predicts failure, and that is not what this article describes. The join here is narrower and easier to put into service, because it takes an alarm the sensor system has already raised and puts the footage of that asset, at that moment, in front of the person who has to act on it.

A design for joining sensor alarms and footage

The design starts on the sensor side, since the condition monitoring system already decides when something is wrong and there is no reason to decide it twice. Each step below is built from a piece of the platform that exists today, and the engagement connects them for a particular plant.

  1. The condition monitoring system or historian raises an alarm on an asset and calls a workflow in AI Intelligence Hub over that workflow's Hypertext Transfer Protocol (HTTP) trigger, passing the asset identifier, the alarm type and the time.
  2. The workflow looks up which cameras cover that asset in a map the plant keeps.
  3. It retrieves the footage from those cameras around the alarm time, drawn from continuous recording.
  4. Where the plant wants more than the clip, it puts a written question to frames from those cameras, such as whether there is liquid on the floor around the pump or whether the guard is open, and keeps the answer as a note for the technician to check against the footage.
  5. It raises the work order, with the alarm details, the camera, the time and a link to the footage, through a connector authored against the maintenance system's Application Programming Interface (API), or it calls a system the plant already uses for intake over HTTP.

Two properties of the platform make this buildable without new product code, and both ship today. A workflow can be called over HTTP by another system, which can then poll for its status and result, and the same workflow can call out to other systems over HTTP in turn, so it acts as a small service the sensor side invokes. The footage exists because AI Live Insight records every camera it reads continuously, for a retention window of 30 days by default, so the minutes before the alarm are on disk whether or not any camera alert fired at the time.

The written question in step four is the optional part, and it carries the same conditions it carries anywhere else. It is an add-on that needs both AI Live Insight and AI Intelligence Hub, it runs on cached frames after the fact rather than in real time, each run costs model usage, and the wording of the question shapes the answer, so the question belongs in the work order beside what it returned. For many alarms the clip alone is what the technician needs.

What each plant builds

Four pieces are particular to each plant, and naming them at the start keeps the engagement honest about its size and about who owns what afterwards.

The first is the asset-to-camera map, a table saying which cameras see which assets and from where. It is easy to build and easy to let rot, because cameras get moved, lines get rearranged and new assets arrive, so the map needs an owner and a line in the plant's change process, or a pump whose camera moved last month will return footage of an empty wall.

The second piece is the sensor system's outbound call when an alarm fires, which sits on the sensor side of the join. Many condition monitoring systems and historians can call a web address when an alarm fires, but the exact mechanism, what the call can carry and whether it can reach the workflow across the plant's network segments are questions for the plant's own controls engineers, and their answer decides how much of the build sits on the sensor side.

The third is the work order mapping, which decides how an alarm type becomes a work order type, priority and craft in the Computerized Maintenance Management System (CMMS), and which fields carry the reading and the footage link. The maintenance system remains the authority on maintenance work, so the mapping is agreed with the maintenance planner rather than imposed by the integration. When the maintenance system runs on premises, the connector can be allowed to reach a private network address, accept a self-signed certificate or use plain HTTP, each of which is a separate switch on that integration step and off by default.

The fourth piece is agreement on time across the sensors, historians, cameras and the servers between them, because each keeps its own clock and a join at the level of a minute forgives small drift while a join at the level of seconds does not. Our article on correlating a line stop with what the cameras saw covers clock synchronization in more detail. Once the join is running, each of these seams needs a named owner, typically a controls engineer for the outbound call, the maintenance planner for the mapping, and the IT and Operational Technology (OT) team for the map, the credentials and the network path.

What the join does not do

The design moves footage to an alarm and leaves the alarm itself entirely to the sensor system. The platform does not analyze vibration, temperature or current signals, does not trend them and does not predict failures or remaining useful life from them, so the condition monitoring program keeps every job it does today. Nothing in the design writes to a control system either, because the alarm arrives from the sensor side, the work order lands in the maintenance system under its own rules, and the programmable controllers stay exactly as reachable as the plant's OT architecture already decides.

Footage is only as reachable as the retention window allows, so an alarm investigated after 30 days on the default setting finds no continuous recording behind it, only whatever clips were cut at the time. A camera that does not see the asset adds nothing, and a camera the platform does not read is not recorded by it, whatever the plant's own recorder is doing with that stream. Alarm floods are the practical risk, since a historian that raises hundreds of nuisance alarms a shift would raise hundreds of clips and work orders as well, so the plant has to decide which alarms deserve footage, and which deserve a person, before the first call is wired.

Scoping the engagement

A first engagement goes best on one asset class whose failures are both instrumented and visible, such as pumps under vibration monitoring that also leak, or conveyors with motor current monitoring whose jams a camera can see. Starting with a handful of alarm types rather than the whole alarm list keeps the first month readable, and the measure of success should be agreed before go-live, which in practice means comparing work orders raised with the clip against those raised without it, on diagnosis time and on how often the first person to arrive brought the right tools.

Someone also has to own the review of the first month, because the early alarms will show where the asset map is wrong, which alarm types produce useless footage and where the clocks disagree. The same workflow pattern runs in the other direction too, since an AI Live Insight alert can itself trigger a workflow that gathers related records and passes the result to another system, and our article on agentic AI for manufacturing investigations covers that general pattern. When the camera sees the problem before any sensor does, our article on visual equipment monitoring on plant cameras is the better starting point, and our article on troubleshooting a machine fault with recorded repairs and an AI assistant covers what the technician does with the clip once the work order is open. The outbound mechanics, and the OT boundary in general, are set out in our article on connecting video AI to plant and maintenance systems.

How VIDIZMO fits

AI Intelligence Hub runs the workflow at the center of this design, which the condition monitoring system calls over HTTP and which calls other systems over HTTP in turn, and it reaches the maintenance system through a connector authored against that system's API. VIDIZMO AI Live Insight records the plant's cameras continuously, cuts clips from that recording, and, together with AI Intelligence Hub, answers written questions put to cached frames as a licensed add-on. Both products, along with the language and vision models behind the written questions, can run on the plant's own servers with Graphics Processing Unit (GPU) capacity, so the footage and the readings stay inside the plant.

The join itself is tailored for each plant as an engagement built on these products, and it is not a shipped connector for any historian or condition monitoring system. Talk to a specialist about your maintenance data.

FAQ

Frequently Asked Questions

What is sensor fusion in predictive maintenance?

In research and vendor writing, sensor fusion usually means combining several sensor streams, such as vibration, temperature and motor current, into one model that predicts failure. Combining sensor data with video is a narrower join, in which an alarm the sensor system has already raised brings the footage of that asset, at that moment, into the work order.

Can a historian alarm pull up camera footage automatically?

It can, with tailoring, built as an engagement for each plant rather than as a shipped connector. The sensor system calls an AI Intelligence Hub workflow over Hypertext Transfer Protocol (HTTP) with the asset and the alarm time, the workflow finds the cameras covering that asset in a map the plant keeps, retrieves the footage around that time from continuous recording, and raises the work order with a link to it.

Does the platform analyze vibration or temperature data?

The condition monitoring system keeps the job of detecting faults from vibration, temperature and current, trending the readings and predicting failures. The join only brings footage to the alarms that system raises, so the sensor program stays exactly as it is.

What does each plant have to build to join sensor alarms and video?

Each plant builds an asset-to-camera map, the sensor system's outbound call when an alarm fires, a mapping from alarm types to work order types in the Computerized Maintenance Management System (CMMS), and agreement on time across sensors, cameras and servers. Each of those seams needs a named owner once it is running.

Does joining sensor alarms to video write to control systems?

Nothing in the design writes to a Programmable Logic Controller (PLC) or any other control system. The alarm comes from the sensor side, the work order goes to the maintenance system under its own rules, and the control network stays as reachable as the plant's Operational Technology (OT) architecture already decides.

TopicsIntegrationsManufacturingAI Intelligence HubAI Live Insight

You may also like

Troubleshooting a Machine Fault With Recorded Repairs and an AI Assistant

Troubleshooting a Machine Fault With Recorded Repairs and an AI Assistant

A fault code appears on the infeed servo at 2 a.m., and the technician who fixed the same fault last spring is not on ...

Visual Equipment Monitoring: Leaks, Fire and Changes a Camera Can See

Visual Equipment Monitoring: Leaks, Fire and Changes a Camera Can See

Many machine failures show themselves well before they stop anything, as oil pooling under a gearbox, a drip from a ...

Investigating a Quality Escape With Plant Footage: From Complaint to Root Cause

Investigating a Quality Escape With Plant Footage: From Complaint to Root Cause

A quality escape is usually found by someone outside the plant. A distributor opens a case and sees the wrong label, or ...

See all posts

See it on your own content

Tell us what you are trying to solve and we will show you how it works on your infrastructure.