The promise of an OpenAI-compatible API is simple: change the base_url, change the model name, and your application talks to a different model. That is true, for the first call. The API shape has become the standard that nearly every open model sits behind, including the models on our inference platform. The misconception is that compatible at the API layer means interchangeable in behaviour. The first is a matter of two lines; the second is work. This article is about where the line runs.
What actually changes in the code
The official client library and most frameworks built on it accept a custom endpoint. In Python a switch comes down to this:
import os
from openai import OpenAI
client = OpenAI(
base_url=os.environ["INFERENCE_BASE_URL"], # previously OpenAI's default
api_key=os.environ["INFERENCE_API_KEY"],
)
response = client.chat.completions.create(
model=os.environ.get("INFERENCE_MODEL", "mistral-small-3.2-24b"),
messages=[
{"role": "system", "content": "Answer briefly, in Dutch."},
{"role": "user", "content": "Summarise this incident report: ..."},
],
temperature=0.2,
)
print(response.choices[0].message.content)
Endpoint, key and model name live in configuration here, not in the code. That is the first thing you arrange in a switch, because it is also what you will need next time. Anyone with base_url spelled out inside a service turns every model change into a release.
/v1/chat/completions is the endpoint the serving stack of nearly every open model offers. Streaming, a system prompt, temperature and max_tokens work the way you are used to. So far the two lines.
Where compatibility ends
Compatible means: the same shape of request and response. It does not mean that every parameter does something, or does the same thing.
- Model-specific parameters — options tied to one provider, such as a setting for how much a model may "think", certain output formats or fields for stored conversations, are ignored or rejected by a different model. An ignored parameter is the more annoying of the two: you get an answer, just not the answer you thought you had configured.
- Context window — every model has its own maximum number of tokens per conversation, and the serving stack can cap it further to keep memory free for more concurrent users. A pipeline that drops a whole document into one prompt needs to be measured again against the new maximum.
- Tokeniser — every model counts differently. Dutch costs roughly 1.3 to 1.6 tokens per word, depending on the tokeniser. A limit you set in tokens means a different number of words on the new model.
- Tool calling — the format is standardised; the quality is not. Whether a model picks the right function, fills the arguments correctly and does not invent a function of its own differs per model and per wording of your tool definitions. This is where switchers most often run aground.
- Structured output — JSON mode exists on most open models, but how strictly a schema is enforced depends on the serving stack. Validate the output yourself, even if you did not have to before.
None of these points is a reason not to switch. They are the points you put on your list before you do.
Recalibrating your prompts
A system prompt that has been polished for months on one model is tuned to that model, not to the task. A different model reads the same instructions differently: it is terser or more verbose, sticks more tightly or more loosely to the requested format, and responds differently to the same temperature.
The only way to make that visible is an evaluation set: a few dozen real prompts from your application with the answer you consider good. Run those against the old and the new model, side by side, before anything goes to production. Without that set, "it works" is a feeling; with it, it is a number. You use the same set at every subsequent model change, and at every new version of the same model too.
This is also the moment to look at your prompts in Dutch. How well a model handles Dutch varies a lot; which open models excel at it is in choosing open models.
Build the way back in from the start
The reason to switch is often that you want to get away from something: a provider outside the EU, a pricing model, an API that changes without notice. After the switch, that reason applies to the new provider just as much. A switch that moves your lock-in instead of removing it is half a job.
Three things keep the way back short:
- Configuration outside the code — endpoint, key and model name as environment variables or secrets, never in the source.
- Open weights under an open-source licence — a model you run with us under Apache 2.0 you can also run somewhere else, including on your own hardware. That is the difference from a closed model behind an API.
- Keeping your evaluation set — it is your proof that the next environment does the job too.
The legislator lends a hand. Under the Data Act, switching charges and egress fees for cloud services must be zero from 12 January 2027. That settles the bill on leaving; the technical portability you arrange yourself, with the three points above.
Switching to Xyphen IT
Our inference API is OpenAI-compatible and runs open models with an origin label; the endpoints, supported parameters and model list are on the technical page. If you want to put your application next to the model rather than across the internet from it, you can with GPU colocation or regular colocation in the same data centre, connected over a private VRF on our EVPN fabric. If we build the application for you, the configuration discipline above is in it from the first commit; that is how we work in software development.
Want to know how much work your switch is? Get in touch or request a proposal; send your evaluation set along and we will run it against the models we offer.