VIDIZMO integration brief · vLLM · vidizmo.ai/integrations/catalog/vllm

Integrations / vLLM

vLLM

Serve open models at scale with vLLM. Visit website

vLLM is a high-throughput inference server for open-weight models, built for concurrent production load.

It is the self-hosted path where a private or air-gapped deployment needs the throughput of a hosted service without leaving its own boundary.

Self-Hosted Models at Production Scale, Inside Your Boundary

Open as a two-page brief

vLLM is a high-throughput inference server for open-weight models, built for many concurrent requests. Connected to AI Intelligence Hub, it serves the language model that writes answers and the embedding model behind retrieval on GPU servers you own. A large user population gets the responsiveness of a hosted service without a prompt leaving your network. The platform's own transcription, vision, OCR and PII models run locally beside it. Content stays in Nexus.

Nexus your video, audio, images and documents, indexed under your access rules content the asking user may see AI Intelligence Hub agents, chat, workflows, retrieval with citations; model chosen per agent prompts and embeddings over API vLLM model endpoint, hosted by the provider or on your own servers answers Users chat, search, workflow output vLLM VIDIZMO

How it connects

AI Intelligence Hub calls the vLLM server over its REST API on your network, at the address supplied in configuration. The model provider is configuration, set per deployment or per agent. Pointing an agent at vLLM rather than a hosted endpoint changes where inference happens without changing what the workflows do.

Two things flow across your own network. At indexing time the text of transcripts, OCR output, visual descriptions and documents goes to the embedding model on vLLM and vectors return to the index. At question time the question and the passages retrieved for the asking user go to the language model and the answer returns, cited to the passages it drew on.

Nothing flows outside. Licensing verification is the one outbound call the platform makes, independent of AI processing, and it is disabled for a disconnected deployment. Model quality follows the models you host, and the servers need the GPU capacity for them.

What you can do together

  • Ask AI Intelligence Hub a question across policies, procedure videos and scanned forms and get a cited answer, with every request served on your own hardware.
  • Serve a whole workforce at once, because vLLM is built for concurrent production load rather than one request at a time.
  • Run the whole AI stack disconnected: vLLM for language and embedding models, with local Whisper transcription, vision, OCR and PII detection alongside.
  • Choose the model per agent, so one department's agent runs a different model from another's, both on the same servers.
  • Keep every answer inside what the asking user is entitled to open, with each agent scoped further to its own folders.

A scenario

  1. SetupIT points AI Intelligence Hub at the vLLM servers for the language model and the embedding model. The Policy Desk agent is scoped to the policies and procedures folders; a second agent is scoped to one department's recorded teaching sessions.
  2. IngestProcedure videos and recorded sessions are transcribed by the local Whisper model, scanned forms are read by local OCR, and all of that text is embedded through vLLM. Everything stays in Nexus.
  3. 06:50At shift change a charge nurse asks, "What is the current escalation path for a suspected sepsis patient on the step-down unit?" The agent retrieves the policy page and the procedure video she is permitted to open and sends them to vLLM with the question.
  4. Seconds laterThe answer cites the policy page and the moment in the video. She opens a card and the video plays from that point.
  5. Same hourHundreds of staff across the system ask their own questions, and the vLLM servers answer them concurrently.
  6. 14:00A pharmacist attaches a scanned order in the conversation and asks whether it matches the protocol. The answer cites both. No recording, form or prompt left the hospital network at any point.

What stays where

vLLM runs on your servers

You choose the models, load them and size the GPUs behind them for the load you expect.

AI Intelligence Hub sends prompts and text across your own network

The question, the retrieved passages and the text to embed go to vLLM, and answers and vectors come back. Nothing goes to a hosted API.

Nothing is replaced

VLLM is a provider setting. An agent moves between vLLM and a hosted provider by configuration, and the workflows behind it do not change.

Where processing runs

Every inference runs on the hardware you pointed the platform at, in your data center or your own cloud subscription. Air-gapped, the same setup works with licensing verification disabled.

Products and solutions

Next step

See it on your own vLLM instance. We will show the connection made, the data moving and the output, then size it for your deployment.

Request a demonstration  or write to sales@vidizmo.ai

sales@vidizmo.ai  ·  vidizmo.ai/integrations/catalog/vllm

Not what you need?

We build it

Send us your API documentation and we build, test and maintain the connector, at no development cost to you.

Request this integration

Bring an MCP server

If the system publishes a Model Context Protocol server, AI Intelligence Hub connects to it as a client with configuration alone.

Define a REST endpoint

Describe your endpoint and it becomes a node in an agent workflow, without waiting on our roadmap.

Product names, logos and trademarks shown here are the property of their respective owners. Their use does not imply any affiliation with or endorsement by them.