# Self-hosted AI workers

## The promise

Your Mac or Linux box runs the model. It opens outbound HTTPS/SSE to Interlock, claims only work
for its project, and calls a model server bound to loopback. Interlock never opens a connection to
your hardware or model port. Interlock does receive the ordinary prompt and result because it is
the durable control plane; “self-hosted” means you own compute and uptime, not control-plane-blind
end-to-end encryption.

Qwen3.6-35B-A3B Q8_0 is the first supported profile. Expect about 38 GB for model weights and a
64 GB Apple Silicon Mac or a Linux GPU host with comparable usable memory. Model source is
`unsloth/Qwen3.6-35B-A3B-GGUF`; review its Apache-2.0 license and the upstream Qwen license before
installing. Quantisation changes quality and memory use, so the deployment alias includes the exact
profile rather than the vague word “Qwen”.

## Install

In Project settings, open **Your hardware** and choose **Create install command**. Run the one-time
command on the machine. The key is shown once and has only `tasks:register`, `tasks:claim`, and
`tasks:report`; it cannot submit application work or read another project.

The installer:

- supports macOS with a system LaunchDaemon and Linux with a systemd service, both running as the
  unprivileged installing user and starting without login;
- verifies a pinned public-key signature on both the release manifest and jar, then verifies the
  manifest SHA-256 and byte count before an atomic upgrade;
- stores the key in an owner-only environment file, never in a service definition or command line;
- binds/adopts llama.cpp on `127.0.0.1`; non-loopback configuration is refused unless the operator
  deliberately sets `LLM_ALLOW_REMOTE=1`;
- keeps the previous jar for rollback and preserves secrets, logs, and models on uninstall.

Lifecycle commands use the same downloaded installer:

```bash
bash setup.sh status
bash setup.sh doctor
bash setup.sh update
bash setup.sh restart
bash setup.sh uninstall   # service/app only; env, logs and model cache remain
```

The host needs Java 21, `curl`, `openssl`, Python 3, and llama.cpp's `llama-server`. It needs
outbound TCP 443 to `api.interlock.sh` and the asset host, plus outbound model download access on
first boot. It needs no inbound firewall rule, public IP, tunnel, SSH exposure, port 8080, or port
11434.

## Route policy

Placement is explicit:

| Policy | Meaning |
|---|---|
| `managed-only` | Never uses customer hardware. A self-hosted model name is a configuration error. |
| `self-hosted-only` | Never creates managed-provider spend. No eligible worker is a typed, unbilled refusal. |
| `self-hosted-then-managed` | Uses the project worker when healthy, otherwise the managed default. This fallback is opt-in. |

Java:

```java
var options = Ai.ChatOptions.withManagedFallback(Models.QWEN_3_6_35B_A3B);
String answer = il.ai().chatWithOptions(options, "Summarise this session");

AiRun run = il.ai().submit(Ai.ChatOptions.selfHosted(Models.QWEN_3_6_35B_A3B), prompt);
AiRun.Snapshot done = run.await(Duration.ofMinutes(10));
```

Node:

```js
const run = await client.aiRunSubmit(prompt, {
  model: 'qwen3-6-35b-a3b', routePolicy: 'self-hosted-only', timeoutMs: 600000,
});
const done = await client.aiRunAwait(run.id, { timeoutMs: 660000 });
```

HTTP uses `POST /api/ai/runs`, `GET /api/ai/runs/{id}`, and
`POST /api/ai/runs/{id}/cancel`. `/api/ai/runs/{id}/stream` immediately sends an SSE hello,
continues with durable task-event deltas and 25-second heartbeats, and may be disconnected at any
time: `GET` by run ID remains canonical. A long inference is not coupled to a Cloudflare request
deadline.

## Operate it

“Ready” means the worker registered `ai/chat@1`, was seen within 90 seconds, is not draining, and
has capacity. “Full” is healthy—all slots are busy. “Drained” renews current leases and claims no
new work. “Offline” means Interlock has not seen it recently; do not confuse that with a model that
is still loading locally.

Before maintenance, drain and wait for running work to reach zero. Update, run `doctor`, then
restart. For key rotation, create a new Task Worker key, replace it in the owner-only env, restart,
prove the worker ready, then revoke the old key. For a model replacement, use a new deployment
alias until readiness and a real completion pass; do not relabel different weights as the old
deployment.

The billed route is `ai.self-hosted.qwen3.6-35b-a3b`. Evidence includes the task/run, worker,
deployment, attempt, token counts, duration, terminal state, and route policy. An eligibility or
admission refusal costs zero. One inference produces one AI charge; the underlying Task does not
produce a second generic charge.

## Diagnose and roll back

1. Run `setup.sh doctor`: release signature/checksum, environment permissions, loopback model
   boundary, and boot service must all pass.
2. In **Your hardware**, distinguish offline, drained, full, and ready. Open `/tasks` for the exact
   `ai/chat@1` attempt and event timeline.
3. Check local logs under `~/.interlock/task-worker/logs`. Prompts/results are customer data; do not
   paste them into tickets. Worker logs redact them by default.
4. If the model is unavailable, `self-hosted-only` should return
   `NO_ELIGIBLE_INFERENCE_WORKER`, not silently spend money. Use managed fallback only when its
   cost and privacy boundary are acceptable.
5. Roll back an upgrade by stopping the service, replacing `app/worker.jar` with
   `app/worker.prev`, and starting it. `uninstall` removes the service/application without deleting
   the environment, logs, or model cache.