Self-Hosted Open WebUI

Open WebUI is a ChatGPT-style interface you run yourself and point at any model provider. Structurally it is the self-hosted tool architecture with one component added, and that component is the reason this page exists. An API key has to reach a running container without ever being written into the file that describes it, and the way that split works is worth understanding before you need it.

The Architecture

{
  "version": "2.0",
  "platform": "aws",
  "components": [
    { "type": "Network", "name": "network" },
    {
      "type": "Container",
      "name": "chat",
      "source": { "image": "ghcr.io/open-webui/open-webui:main" },
      "ports": [{ "name": "http", "port": 8080 }],
      "cpu": 1,
      "memory": "2Gi"
    },
    { "type": "Volume", "name": "data" },
    {
      "type": "Secret",
      "name": "llm-keys",
      "keys": ["OPENAI_API_KEY"]
    },
    { "type": "LoadBalancer", "name": "edge" },
    { "type": "ImportedDomain", "name": "domain",
      "zone_name": "example.com", "zone_id": "Z0EXAMPLE123456" }
  ],
  "connections": [
    { "from": "domain", "to": "edge", "host": "chat.example.com" },
    { "from": "network", "to": "chat" },
    { "from": "network", "to": "data" },
    { "from": "network", "to": "edge" },
    { "from": "edge", "to": "chat", "health_check_path": "/health" },
    {
      "from": "chat", "to": "data",
      "mount_path": "/app/backend/data"
    },
    { "from": "chat", "to": "llm-keys" }
  ]
}

The Wiring

The Secret component carries a name and a list of key names, and no values. That is the entire idea: the schema describes which keys exist so the infrastructure around them can be built, while the contents live somewhere the schema cannot see. Applying it creates the secret in your cloud’s managed secret store with the keys declared and nothing in them, so the first apply deliberately produces an empty secret. You fill them in afterwards, from the component in the dashboard as key/value pairs or a pasted .env. Because the container resolves its keys as it starts, it cannot start before the values exist: a first apply that fails here is the expected state, not a fault.

chat → llm-keys is what turns a declared key into a usable variable. It projects each key onto the container as an environment variable of the same name, and what is stored alongside the container is a reference rather than the secret itself; the value is fetched and placed in the environment as the container starts. Open WebUI finds OPENAI_API_KEY waiting for it and has no idea it came from anywhere unusual. When two secrets would collide on a variable name, the connection can project a chosen subset of keys, or prefix them so both can be consumed at once.

The rest of the file is the self-hosted tool pattern. The volume holds everything the application accumulates, which for this image means chat history, accounts, and uploaded documents under /app/backend/data. The cpu and memory are larger than a typical tool of this size because Open WebUI runs an embedding model locally when you give it documents to search.

domain → edge puts the interface on chat.example.com, writing the record into the hosted zone the ImportedDomain names and attaching its certificate to the load balancer. The zone has to be verified before this applies; without one, drop domain and its connection and the project keeps its generated address.

The Decisions Behind It

Keys in the file, values in the store. The alternative most people reach for first is an environment variable in env, which puts a live credential in the repository and in every diff. Splitting the declaration from the value keeps the file safe to commit and share, and it means the set of secrets an application needs is reviewable infrastructure while the secrets themselves are not.

A container and a function get different treatment, deliberately. A container is handed the resolved value before its process starts. A function connected to the same secret is handed a reference and fetches the value itself when it runs, because a serverless runtime has nowhere safe to hold it in advance. One component, two mechanisms, each chosen by what the runtime can do without exposing the value.

One secret, several consumers. Keys tend to be shared across services, and a secret can be connected to as many components as need it. Duplicating a secret per service is the version of this that seems simpler and creates several places to rotate.

The Limits

A secret’s contents are outside the schema, which is the point and also the cost: your infra.json no longer fully describes a working system. A fresh apply into a new project produces infrastructure that stands up and does not work until someone fills the values in, so that step belongs in whatever runbook you hand to the next person.

The rest of the ceiling is the self-hosted tool ceiling. State on a volume means a single writer, so this architecture grows by giving the task more cpu and memory rather than by running more of them. Local document retrieval is what consumes that headroom first, and pointing the tool at a hosted embedding provider instead moves the load off your infrastructure and adds one more key to the secret.