You have decided your AI must run inside your network. The next question is not which model to use, but which architecture you need today, and how to grow tomorrow without throwing away what you built.
At SISCON we work with three architectures. All three give the same privacy guarantee: no prompt, document or answer leaves your infrastructure. What changes is scale, availability and governance.
Three questions before choosing
- How many use cases and how many teams? A well-scoped case does not need the same as a platform several areas use at once.
- What happens if the AI stops for an hour? If it is an annoyance, one server with a backup is enough. If it stops operations, you need high availability.
- Will someone outside ask you for evidence? If a regulator or an audit will ask which model answered, with what data and who approved it, AI governance is no longer optional.
Option 1 · Local Node
Who it is for: a company with a clear first use case that wants results in weeks with a contained investment.
It is one server with open-source components: vLLM or Ollama to serve the model on Docker Compose, LangGraph for agents, pgvector for document search, LibreChat as the interface and Langfuse for traces and evals. Each component has its own license (Apache-2.0, MIT or the PostgreSQL License), and Langfuse keeps some enterprise modules under a commercial license, so they are reviewed before installing.
Its limit: it is a single server with a cold backup. It works very well for one use case, but not for several teams to depend on it in production.
Option 2 · Private Platform
Who it is for: a mid-to-large company where several teams already ask for AI and need a stable internal service.
It runs on Red Hat OpenShift AI 3 in high availability. Models are offered as an internal service (Models-as-a-Service) with metered usage per team, and the platform adds evaluation and guardrails. It is the option when the question stops being “does it work?” and becomes “who uses it, how much and with what quality?”.
Option 3 · Governed Sovereign AI
Who it is for: banking, insurance, healthcare, government and any organization that must be accountable for its models.
It adds IBM watsonx (watsonx.ai, watsonx.data and watsonx.governance) on OpenShift AI, across two sites with disaster recovery and an environment disconnected from the internet. It adds a model inventory, risk control and audit evidence. Disconnected operation is confirmed per product and version during design.
An important note: the platform provides evidence, but it does not replace your compliance. Compliance depends on your processes and your regulatory framework, and is reviewed with your legal team.
Summary
| Criterion | Local Node | Private Platform | Governed Sovereign AI |
|---|---|---|---|
| Who it is for | First use case | Several teams | Regulated sectors |
| Foundation | Open source | Red Hat OpenShift AI 3 | IBM watsonx on OpenShift AI |
| Availability | One server, cold backup | High-availability cluster | Two sites, disconnected environment |
| Governance | Traces and evals | + metered usage, guardrails | + inventory, risk, evidence |
| Time to launch | 4 to 6 weeks | 8 to 12 weeks | 12 to 20 weeks |
Timelines are reference ranges, confirmed during discovery with your use cases, your users and your real volumes.
Four common mistakes
- Buying hardware before measuring. GPU sizing depends on the model, the number of users and the volume. It is sized after testing models with your data, not before.
- Choosing the biggest model. For many cases, a well-evaluated mid-size open model with good context performs just as well and costs far less to run.
- Starting with option 3 without a use case. AI governance protects something that already works. Without a validated case, it only adds months.
- Not measuring quality. Without a reference set and evals, there is no way to know whether the local model is good enough or whether an update made it worse.
It is not a decision forever
The agents, document search and evals you build on the Local Node move to the Private Platform, and from there to Governed Sovereign AI, because everything runs in containers and uses OpenAI-compatible APIs. You can start with one server and grow without rebuilding.
How to decide with evidence
In the first two phases of our method (discover and design) we evaluate open models with your data, size GPUs, storage and network, and the recommended option comes out with its scope. Evals are the exit criterion: nothing goes to production if it degrades measured quality. AI Evaluation provides those evals.
And if what you want is to develop software with agents without code leaving your network, IA Factory runs on any of the three options.