Model Portability: Keeping the Ability to Change Your Mind About AI

Ask whether you could move off your current AI provider and the answer is more encouraging than the question expects. Swapping which model answers a request is a configuration change. Point the client at a different endpoint, change a model identifier, and you have a working system by lunchtime. That is not where the difficulty sits. Model weights are the most interchangeable component in an AI stack, and very few organizations are held in place by the model itself.

What holds them is everything that accumulated around it. The prompt that was rewritten eleven times until it stopped producing the wrong format. The evaluation suite designed around the failure modes of one particular model. The vector index that took days of GPU time to build. The orchestration code that absorbed one provider's SDK conventions because that was the fastest way to ship.

This article sits under the guide to running AI inside infrastructure you control and takes portability as its own subject. It is written to be useful whether or not you ever switch. Almost every measure that makes a model replaceable also makes the system easier to debug, easier to hand to a new team, and easier to reason about when output quality shifts and nobody can say why.

Where lock-in actually forms

AI model lock-in is not one condition but a stack of separate layers with very different switching costs, and the layers that cost the most to move are rarely the ones discussed in a vendor selection meeting.

Three of those deserve more than a table row.

Prompt engineering is empirical work, and that is exactly what makes it non-portable. Models differ in how literally they follow instructions, how much weight they give a system message against a user turn, how readily they add commentary around a requested output, and how they behave when an instruction is ambiguous. A prompt that survives production has been co-adapted to one model's specific habits. Move the model and every prompt is untested again. This is not a catastrophe, and it is real work proportional to how many prompts you run.

Fine-tuning is the layer that produces the sharpest surprise, because the artefact you paid to create is often the one you cannot take. When tuning happens inside a hosted service, the resulting weights typically remain in that service and the thing you own is the training data and the results. Fine-tuned deployments also carry their own retirement clock, which is easy to miss. Microsoft's Foundry Models lifecycle policy documents fine-tuned models retiring in two phases, training first and deployment later, and publishes deployment retirement dates roughly six months after the training retirement for each base model. So a tuned model has a stated end-of-life you inherit from a base model somebody else chose.

There is a defensive habit that makes that inherited end-of-life survivable, and it costs very little if you adopt it early. Keep the training set and the evaluation set under your own version control and treat every tuning run as something you could repeat from data you hold, so a provider change costs you a tuning run rather than a data collection project. Open-weight models change this shape entirely, because tuned weights are files sitting on your storage, and where those weights carry a permissive licence such as Apache 2.0 the obligations on redistributing your derivative amount to preserving notices rather than asking permission. Check that before assuming it, because open weights and open source are not the same thing, and several widely used model licences fail the Open Source Definition's sixth criterion, that a licence "must not restrict anyone from making use of the program in a specific field of endeavor", by attaching an acceptable-use policy naming prohibited applications.

Operational knowledge is the least discussed of the layers and one of the most expensive, because it sits in people rather than in anything you can export. A team that has spent a year with one model knows which tasks it quietly gets wrong and when its confident tone is unreliable. None of that transfers, so budget for a stretch where your engineers' estimates are worse than usual.

Why embeddings are the hardest thing to port

An embedding model maps text into a vector space that the model itself defines. The coordinates carry meaning only relative to other vectors produced by that same model. Two models can both emit 1,024 numbers per chunk and those numbers still describe different geometries. Cosine similarity between a vector from one model and a vector from another is arithmetic without meaning, and there is no conversion function that fixes it, because the spaces were never aligned to begin with.

How many numbers you keep for each chunk is a decision with consequences beyond how much disk the index takes, because it also decides how much of the original text can be reconstructed from the vectors. Research on reconstructing text from embeddings found that dropping a retrieval model from 768 dimensions to 256 cut exact-match reconstruction of the source text from 43.0% to 5.9%, and that product quantization took it to zero while shrinking the index from 61GB to 16GB, with no statistically significant difference in retrieval effectiveness in either case. If your reason for holding an index inside your own boundary is exposure rather than latency, that is a lever worth knowing about before you commit to a geometry.

The consequence is unavoidable. Changing the embedding model means regenerating every vector in the corpus and rebuilding the index. Not the changed documents. All of them.

What a reindex actually costs

The compute line is the one people estimate and usually the smallest of the three. Embedding is far cheaper per token than generation, but the job runs over the entire corpus rather than over a single query, so the arithmetic is total documents times chunks per document times tokens per chunk. For a large document estate this is a batch job measured in hours on dedicated GPUs and considerably longer when it shares hardware with live traffic. It also scales with a corpus that only grows, which means the reindex you can afford today is the cheapest one you will ever run. If you are sizing the hardware for that job, the memory and throughput mechanics in running LLMs on-premises and the realities nobody advertises apply to embedding servers as much as to generation.

The availability line shapes the plan. While regeneration runs you either take retrieval offline or run two indexes at once. Most teams that do this properly build the new index alongside the old one, validate against it, then cut over, which carries storage for two full copies plus a routing decision about which index answers during the overlap. Where retrieval is a user-facing feature rather than a batch process, the dual-run is not optional.

The validation line is the one teams skip, and skipping it is what turns a reindex into a drop in answer quality that nobody can account for. Retrieval quality does not carry across an embedding change by itself, because a different model has different strengths on your domain vocabulary, different sensitivity to chunk boundaries, and different behaviour on very short queries. Teams frequently re-tune chunking at the same time because the new model has a different context window, which compounds the number of variables changing at once. You need a retrieval evaluation set, a list of real queries paired with the passages that should come back, and you need it built before the migration, because assembled afterwards it can only describe the new system rather than compare it to the old one.

The generation model and the embedding model are separate decisions

Nothing requires the model that writes the answers and the model that indexes your corpus to come from the same provider, and there is no technical reason to buy them as a pair. A retrieval system has two model dependencies with switching costs an order of magnitude apart, and treating them as one purchase is the decision that makes portability expensive later.

Once the two are handled separately, the operating rule follows straight from what each one costs to change. Change the generation model freely, since a generation swap costs an evaluation run and some prompt revision. Change the embedding model deliberately and rarely, on a planned schedule, when a genuinely better model justifies the price of a full reindex. Two things you would otherwise be forced to do together become independent, and the expensive one stops being triggered by the cheap one.

There is a second reason to separate them, and this one is about exposure rather than cost. Embedding is the stage where every document you hold passes through a model, while generation only ever sees the few passages retrieval selected. If any part of your stack has a data exposure argument attached to it, the embedding stage is usually the stronger candidate for running inside your own boundary, and doing so does not commit you to self-hosting generation as well.

What an abstraction layer does and does not buy

The reflex response to lock-in worry is to write a wrapper. It is worth doing and worth being precise about what it earns.

An abstraction normalises the call shape and the auth. Message formatting, streaming, retries, credential handling, rate limit backoff, error taxonomy. That work is genuinely useful and it is the easy part. It converts a provider change from a search-and-replace across an entire codebase into an edit inside one module, and it makes running two providers side by side possible at all.

What it does not normalise is behaviour. Two models behind one interface still answer differently, and no amount of interface design changes that. A prompt that reliably returns clean JSON from one model may need an explicit re-instruction on another, or a repair step, or a schema-enforcement mode that only one of them offers. So the abstraction removes the plumbing cost and leaves the evaluation cost untouched, and the evaluation cost is the larger of the two. Anyone selling a migration as trivial because the code is abstracted has counted the smaller number.

There is an honest tradeoff in the design as well. A thin abstraction over several providers drifts toward the lowest common denominator of features. If one provider enforces structured output and another does not, the interface either exposes the capability, in which case the abstraction leaks and provider-specific code returns, or it drops the capability, in which case you gave up a real feature to preserve an option you may never exercise. The better designs pick a side deliberately. They expose a common core and let provider-specific capability through an explicit, clearly named path, so the code that depends on something unusual is visibly the code you will have to revisit.

How to structure a deployment so changing models is a project rather than a rebuild

None of what follows requires a portability initiative or a budget line of its own, because each measure pays for itself in ordinary operations and is worth doing in a shop that never intends to switch anything.

Keep prompts as versioned assets that live outside the application code, because the moment output quality shifts you will need to know exactly which wording was in use. A prompt buried in a source file is invisible to everyone who is not a developer, and a prompt with no history cannot be compared against its replacement when quality moves. Store prompts as data, version them, and record which model and model version each one was validated against.

Treat the evaluation set as an asset with a named owner rather than as a folder of old test cases, because it is the only instrument that tells you whether a swap is safe and it goes stale when nobody curates it. How to build one that actually discriminates between models, including sampling so the hard cases survive and an honest account of the labelling burden, is covered in choosing an on-prem model.

Record the embedding model and its full configuration alongside the index itself, so the index can always account for how it was built. That means the model name, the model version, the chunk size, the overlap, the preprocessing steps and the date of the build. Teams routinely discover months later that nobody can say exactly how the current index was produced, which turns a routine reindex into archaeology. Then plan reindexing as a normal operation rather than an emergency, and run it occasionally on purpose so the pipeline is known to work.

Isolate provider-specific code behind an internal interface you own, which does not have to be a third-party framework and does have to be a real boundary. The test of whether that boundary exists is whether someone can list every file that would change if the provider changed. If nobody can produce that list, the boundary is aspirational.

Log which model and which version produced each output, next to the prompt version that generated it. This is the measure with the broadest payoff. It answers the portability question of whether a quality change came from a model change, and it answers the accountability question of what produced a given result, which in regulated work is asked later by someone with more authority than an engineering manager.

These measures can be properties of the platform rather than disciplines you impose on it. VIDIZMO AI Intelligence Hub is model-agnostic by design, and the model is a setting on a node inside a workflow graph rather than an assumption compiled into the application. Hosted provider APIs and self-hosted open-weight models of the customer's choosing are both selectable, and multiple models can operate concurrently, so different tasks within one deployment can run against different models. Workflow graphs are versioned and clonable, which makes a model change an edit to a tracked artefact instead of a code release. Embedding providers are configured independently of generation, which is the separation the previous section argued for, built in rather than assembled. The default posture is that everything ships self-hosted, with external provider APIs available as tenant-configurable alternatives. The strongest form of portability is the one that crosses the hosted and self-hosted line, where a workload running against a hosted model can be moved onto a self-hosted open-weight model without the system around it being rebuilt.

The procurement angle

Portability that lives only in the architecture erodes at renewal, when the leverage sits on the other side of the table. A few clauses and evaluation questions keep it intact.

• Ask for export of your own artefacts in a usable form, meaning prompts, evaluation data, conversation and output logs, and vector indexes where the vendor holds them. Usable means a documented format readable without the vendor's software, not an opaque dump that technically satisfies the word export.
• Ask for a defined notice period on model deprecation, long enough to run a full evaluation and a migration rather than long enough to change a configuration value. Hold a smaller vendor to what the large platforms already publish. Microsoft's Foundry Models lifecycle policy sets a generally available model's retirement date at 18 months from launch, blocks new customers at 12 months, gives at least 60 days notice before retirement, and declares the official replacement only 90 to 120 days out. After retirement, inference returns 410 Gone. The policy also states plainly that retirement dates are not extendable, and that models from several partner labs run a 12-month lifecycle rather than 18. Notice how little of that window is usable: a replacement you cannot name until 90 days before the deadline is a replacement you cannot evaluate a year in advance.
• Ask for version pinning, so you stay on a specific model version instead of being moved automatically when the provider updates the model behind an alias. Silent version changes are among the most common causes of a system that got worse with no deployment on your side. This one is concrete rather than aspirational. The same Microsoft policy exposes a versionUpgradeOption setting with values that either upgrade when a new default appears, upgrade only at retirement, or never upgrade, in which case the deployment simply stops working on the retirement date. Ask which of those three your vendor supports, and what the equivalent is called.
• Ask for the right to run a second model in parallel during a migration without a licensing penalty. Migrations are validated by running both against real traffic and comparing, and an architecture or a contract that permits only one configured provider turns a comparison into a leap of faith.
• Ask what happens to fine-tuned artefacts and derived data at termination, and get the answer in the agreement rather than in a support conversation two years later.

When lock-in is an acceptable trade

Portability is not free, and an article that pretends otherwise is selling something. The abstraction is code somebody has to maintain, the evaluation set needs an owner and continuous curation, and holding the embedding model stable means occasionally turning down a better one. Most of all, a team that keeps its options open is a team that has not gone deep on any single provider's advanced tooling, and depth is worth real money in some contexts.

For some organizations that trade is worth taking, and taking it on purpose is a different position from drifting into it. If your differentiation comes from doing something unusual with one provider's capabilities, if the workload carries no regulatory constraint on where it runs, and if the application would be rewritten within a few years anyway, then buy the depth. Fine-tune aggressively, use the provider-specific caching and structured output modes, build on their agent framework. You will be locked in, and you will have received something concrete in exchange, which is a better position than being locked in by accident.

The calculation reverses when the workload is regulated, when the data cannot cross a boundary, when the deployment has to run in an environment the provider does not serve, or when the system's expected life is longer than the procurement cycle that chose it. Public sector, defence and clinical deployments tend to land there by default, not by preference.

For everyone else, the useful goal is smaller than full portability, because most organizations do not need to switch next quarter and only need to know in advance what switching would cost them. Knowing which layers would move, which would have to be rebuilt, and roughly what each would cost is most of the value on offer here, and that assessment can be produced in a week without changing a line of code. Do that much, keep the embedding model and the generation model as separate decisions, and hold your prompts and evaluation data as assets you own. The rest is a project, which is exactly what it should be.