Running Agentic AI Workflows on Infrastructure You Control

Agentic AI runs on premise, and the interesting question is not whether the model can be served locally. That part is solved, and it sits inside the wider sovereignty picture covered in the on-premises AI guide. The interesting question is whether every other step in the workflow can run locally too, because a workflow has many more steps than the model call and each one is a place data can leave.

Strip the marketing vocabulary and an agentic workflow is a graph. The nodes are model calls, retrievals against an index, tool invocations against systems that hold state, data transformations, and points where a person inspects an intermediate result and decides whether it proceeds. The edges are control flow. Unlike a fixed pipeline, those edges branch on the content of an intermediate result and can loop back to an earlier node when a check fails. That property is what separates an agent from a script. A script's data flow is knowable before it runs. An agent's is only partly knowable, because the agent chooses some of it at runtime.

An agent is a data-governance problem before it is an AI problem

An agent that decides its own next step decides its own next data flow. That sentence is the whole governance argument, and it is why a workflow diagram is a more useful artefact for a security review than an architecture diagram of the model serving layer.

Take a request-handling workflow of the kind that runs in records offices, claims teams, legal operations and anywhere else inbound requests meet a system of record. A request arrives, the agent works out what is being asked for, retrieves candidate material, reads the matching record, drafts a response, checks the draft for sensitive content, routes it to a person for approval, writes the outcome back, and notifies the requester. Nine steps, none of them exotic. Here is where each one can put data outside your boundary.

Two of those exposures are the ones teams miss, because neither looks like part of the AI system when somebody draws the architecture. Notification previews routinely carry the case number or the summary line that the rest of the architecture was built to protect, and they travel through a service nobody put in scope. The tracing layer gives away more than the notification does, because recording what every other node did is its entire purpose. An observability layer captures each node's inputs and outputs so you can debug a run, which means the trace store holds full prompts, retrieved passages and generated drafts, ranked by relevance to real questions people actually asked. If that store is a hosted service, it is a copy of your most sensitive material sitting outside the boundary, indexed for convenience. Put it under the same controls as the source documents, wherever it lives.

What changes when the whole graph runs inside your environment

Moving the graph inside is not only a change of address for the model, and the model is the smallest part of what has to move. What makes a model act rather than only produce text is the harness around it, the scaffolding that supplies the tools it can call, the memory it carries between steps, the control flow that decides which node runs next, and the loop it runs in. Once that harness sits inside your environment, model calls hit an endpoint on your network and retrieval hits an index on your storage, while tool calls reach internal systems rather than the public internet. Human review stops being a separate product with its own login and its own record, and becomes a node in the same graph, writing to the same audit trail as everything around it. Logging and tracing terminate inside the boundary, so the record of what the agent read and wrote stays under the same controls as the material it read.

The useful consequence is that the compliance review becomes a job with an end to it, because the deployment settles most of the question before a reviewer opens a single node. When any node can call out, reviewing a workflow means auditing every node's configuration and re-auditing it after every edit. When the deployment has no route out, you audit the small set of node types capable of egress at all, confirm none are configured, and the rest of the graph is settled by the deployment rather than by inspection. That is what makes agent workflows approvable in environments where every individual outbound call would otherwise need its own justification.

VIDIZMO AI Intelligence Hub is one implementation of that shape. Workflows are cyclic directed graphs designed in a no-code visual editor, and they are versioned, clonable and deployable across portals, so the graph that was reviewed is an artefact you can point at rather than a description of one. Node types cover AI processing, flow control including conditional IF, loop, switch, parallel processing and split-in-batches, triggers for user prompt and HTTP, data transformation, human review and approval, integration through an HTTP request node and an MCP node, and notification. Python code nodes run in isolated containers. Because human review and approval are node types rather than an adjacent process, the reviewer's decision lands in the same run record as the model output it approved, which is what an output relied on in a proceeding needs behind it. What that record has to contain is set out in chain of custody for AI outputs.

Event-driven execution turns this from a chat feature into automation. An external system fires an HTTP trigger or a webhook and the graph runs end to end with its human review and approval gates in place, so the workflow is reachable from the systems that already hold the work rather than requiring people to come to a new interface and start it by hand.

Permissions are the hard part

Most writing on agentic AI stops at the boundary and never reaches authorization, which is where these systems actually fail in regulated environments. The security community has already named the failure. OWASP's Top 10 for LLM Applications lists Excessive Agency as LLM06:2025, attributing it to "excessive functionality; excessive permissions; excessive autonomy", and Vector and Embedding Weaknesses as LLM08:2025, which warns that "in multi-tenant environments where multiple classes of users or applications share the same vector database, there's a risk of context leakage between users or queries". Both entries appear in a list of the ten most consequential risks, which tells you how routinely they go wrong.

There are two distinct failure shapes and they need different fixes.

An agent reaches further than the person asking

An agent running under its own service account holds a union of access, because breadth is what made it useful during the build. Anyone who can talk to the agent now has an interface to that union. The usual mitigation is instructional, a system prompt telling the agent not to discuss material the user should not see, and instructions are not access control. They are a request made to a probabilistic system that is being actively steered by the person on the other side.

The alternative is to have retrieval execute on behalf of the asking user, so the authorization decision is made by the system that owns the record, using that system's current rules, at query time, and recorded in that system's own audit log. Under the service-identity design your access model has been forked, and the fork will drift. Under the delegated design there is nothing to drift, because you never made a second copy of the policy.

The index flattens permissions unless the design carries them

The second problem comes from the way retrieval stores what it has read, because building a search index makes a second copy of your documents in a form that no longer remembers who was allowed to open the originals. Ingestion turns documents into chunks and embeddings, and a chunk carries no owner unless something deliberately attaches one. An index built without permission metadata will return a passage from a restricted file to a user the source system would have refused, and the citation makes it worse by confirming the document exists and naming it.

The problem gets sharper the longer that copy sits there, because permissions in the source systems keep changing after it was taken. Someone leaves a case team, a matter is restricted, a record is sealed by an order, a retention rule expires. An index built once and never resynchronised is a snapshot of who could see what on the day it was built, and it will answer questions about a sealed record with complete confidence. Any design that treats ingestion as a one-time event has this bug and will not notice it until someone gets an answer they should not have had.

AI Intelligence Hub handles the two retrieval paths separately, which is the right shape. Search is permission-aware with instant updates and inheritance. For live queries against connected systems, delegated-permission passthrough means retrieval executes on behalf of the asking user, so the source system enforces its own access and sealing rules rather than a copy of them. Content ingested into the RAG index is mapped to the owning system's access and sealing status at ingestion and kept in sync, so the indexed path is governed by a mapped permission model rather than by nothing at all. When you evaluate any platform, ask which of your retrieval paths is delegated and which is mapped, then ask what keeps the mapping current. A vendor who cannot answer the second question has the snapshot bug.

Scoping helps as well, and it is cheaper than it looks. Agents take system prompts, agent rules, conversation limits and knowledge bases scoped to specific content, and a multi-agent hierarchy lets a master bot route intent to specialised child bots. A child bot whose knowledge base holds one department's material is easier to reason about, and to defend to an assessor, than a single agent wired to everything. Source citations close the loop, because a citation is also an audit artefact. You can check after the fact that the retrieval behind an answer was one the asking user was entitled to.

What is genuinely harder disconnected

Being honest about this is more useful than pretending the tradeoff is free.

Model updates become a supply chain rather than a background task, with artefacts assembled outside, verified against a manifest, staged, tested and approved before promotion. The full mechanics of that, along with everything else that changes when the network route disappears, are in air-gapped AI and what still works with no internet at all.

Evaluation is harder in a way specific to agents, and it is the part teams underestimate. Evaluating a single model call means comparing an output to an expectation. Evaluating a graph means comparing a path. The same question can be answered correctly by a run that took four steps and by a run that took eleven, looped twice and called a write tool it should not have touched.

Benchmark results are worth carrying into a design review, because they set expectations that a demo will not. τ-bench evaluates agents against domain-specific API tools and written policy rules, comparing the final database state against an annotated goal state, which is a much harder test than judging the text of an answer. Its authors report that "even state-of-the-art function calling agents (like gpt-4o) succeed on <50% of the tasks, and are quite inconsistent (pass^8 <25% in retail)". That second figure is the one to sit with. pass^k measures whether the agent succeeds on all k independent trials of the same task, so a pass^8 below 25% means that on more than three quarters of tasks, at least one of eight attempts went wrong. An agent that is right most of the time and inconsistent about which times is exactly the profile that a human gate, an idempotent write and a bounded loop are there to contain.

The same benchmark isolates how much of that success comes from following your rules rather than from general competence, and the answer should change how you write a system prompt. Removing the domain policy from the prompt cost the retail domain only a few points but took the airline domain from 33.2 to 10.8. Where the correct action is determined by written policy rather than common sense, almost all of the agent's usefulness is coming from rule-following, and rule-following is the part that degrades first. A regression set for an agent is therefore a set of whole runs with fixtured tool responses, asserting on the sequence of nodes visited and the tool calls attempted as well as on the final text. Build that regression set before you need it, because the moment you want it is immediately after a model swap and it is not something you can assemble under pressure. How much of your system moves with you when the model changes is worked through in model portability and avoiding AI vendor lock-in.

Web search stops working altogether, which catches teams out because it is the one capability everybody assumes will survive the disconnection. AI Intelligence Hub includes a web search node and a web search tool backed by a general web index, and in a disconnected deployment neither has anywhere to go. Anything the agent needs to know must already be inside the boundary, either as content in the retrieval index or as an internal system it can reach through the HTTP request or MCP nodes. The same applies to every external API tool, so a workflow prototyped against public services needs its substitutions made before it is tested rather than after. The rest of the harness is unaffected by the disconnection, because nothing else in it needed the outside world to begin with. The workflow designer operates identically in on-premises and air-gapped deployments with no dependency on external APIs, and generation runs on self-hosted models on the customer's own hardware.

Debugging becomes your own responsibility, because the tool that would normally show you what happened inside a run is part of somebody else's hosting. Managed agent platforms hand you a trace viewer as part of the service, and a disconnected deployment has no such thing, so tracing has to be built into the harness you run. Emit a structured record for every node execution carrying a run identifier, the node identifier, inputs, outputs, the model and version used, and elapsed time, and terminate it in a store inside the boundary. Without those records you are trying to explain a system that took many steps and does not repeat itself exactly, working backwards from its final output alone.

Failure modes that appear at scale

A workflow that behaves in a demo can fail in five recognisable ways once real volume hits it.

The first is a loop that never finishes, which happens because the architecture deliberately allows work to go round again and leaves the decision to stop with the model. A cycle whose exit condition is decided by a model can fail to exit, particularly when the model keeps judging its own output insufficient. Bound every loop with a maximum iteration count and make that bound a configuration value rather than a constant buried inside a node. Raise an alarm when a run hits it, because a run that terminates on the bound is a bug report rather than a success.

The second is consumption growth, which surprises people who costed the system from single-call arithmetic. One user question becomes dozens of model calls once planning, retrieval, drafting and checking each take a turn, and a split-in-batches node fanning out across a thousand records multiplies that by a thousand. Running the models yourself does not make this free. It converts a per-call charge into GPU hours and capacity planning, and platform licensing can still meter AI usage, so budget consumption on the same footing either way and measure it per run rather than per call.

The third is error propagation, where one wrong value early in a run is carried forward by every step after it rather than being caught. A wrong value extracted at step two is treated as established fact by step five, summarised confidently at step seven, and reaches the reviewer wearing the same tone as a correct one. Most people assume each step is as reliable as the last, which would make a long run's reliability that per-step figure multiplied by itself once for every step, and even that is bad enough. Research isolating execution from reasoning found something sharper, reporting that "the per-step accuracy of models degrades as the number of steps increases", and not only because the context grows longer. The authors identify a self-conditioning effect where "models become more likely to make mistakes when the context contains their errors from prior turns", which does not go away by using a bigger model. An agent that has erred is measurably more likely to err again, because its own mistake is now part of what it is reading.

That is the argument for validating between steps rather than only at the end, and for clearing or flagging a failed step's output rather than letting it sit in context. Prefer checks that fail loudly, such as asserting an extracted identifier matches a known format or exists in the source system.

The fourth is a retry that repeats work the first attempt already finished, because a node can time out after its write has landed and the retry has no way of knowing that. Make every tool call that changes state idempotent, meaning that running it twice leaves the same result as running it once, usually by carrying a client-supplied key the target system can deduplicate on, and set retry policy per node rather than globally.

The fifth is an unreviewed write to a system of record, and that is the one that turns an internal incident into a headline. Any step that writes to a system of record needs either a human gate or a reversible design with a recorded actor. Agents are useful precisely because they act, and acting on a record other people rely on is where a plausible wrong answer stops being an inconvenience.

Deciding which steps need a human

Reviewer attention is scarce, and a workflow that asks for it everywhere gets it nowhere. Gates that fire on every run are approved without reading within about a week, which is worse than having no gate, because the record now shows a human approved something nobody looked at.

A workable rule puts four questions to each step. Does the output leave your organization. Does it change a record other people will rely on. Can the workflow itself undo it. Would a mistake there be caught by anyone downstream before it mattered. A step that fails any of those needs a person. A step that is internal, reversible and inspectable later does not, and gating it buys nothing.

OWASP arrives at the same place from the security side, prescribing that you "limit the permissions that LLM extensions are granted to other systems to the minimum necessary in order to limit the scope of undesirable actions" and "utilise human-in-the-loop control to require a human to approve high-impact actions before they are taken". High-impact is doing the work in that sentence, and the four questions above are how you decide what counts.

Confidence scores are worth adding on top of those four questions rather than in place of them, because a score tells you how sure the system is while the questions tell you what happens when it is wrong. Fixed gates belong on irreversible steps regardless of how certain the system is, because confidence is an estimate and irreversibility is a fact. Threshold-based escalation belongs on judgement steps, where it concentrates review on the runs that need it. AI Intelligence Hub supports both, with human review and approval nodes placed anywhere in the graph and confidence scoring whose configurable thresholds trigger automatic escalation to a reviewer. Set the threshold from measured behaviour on your own material rather than a default, and revisit it after any model change, since a new model shifts the distribution of confidence scores underneath a threshold tuned for the old one.

Whether a gate is real comes down to what the reviewer can see at the moment of deciding. The reviewer needs the draft, the sources behind it, what the agent did to produce it, and what it is about to do next, all in one place. A gate that offers a paragraph and an approve button is theatre, and it will be described that way by whoever examines the process after something goes wrong.