LLM Proxy

LLM Proxy¶

The LLM proxy provides a unified inference API across multiple backends --- MLX (Apple Silicon), GGUF (llama.cpp), vLLM (NVIDIA), Transformers (HuggingFace), and REST (remote APIs). It runs a 2-model system (live for real-time voice commands, background for async tasks), manages LoRA adapter training and loading, and processes async jobs via a Redis queue.

Quick Reference¶


Ports	7704 (API server), 7705 (model service)
Health endpoint	`GET /health` — API (:7704) status code is what the docker healthcheck consumes; see Health & Supervision
Source	`jarvis-llm-proxy-api/`
Framework	FastAPI + Uvicorn
Tier	2 --- Command Processing

Architecture¶

The service runs as three processes, started by run.sh (or scripts/serve.sh, see below):

graph LR
    CC["Command Center<br/>:7703"] -->|"/v1/chat/completions"| API["API Server<br/>:7704"]
    API -->|"/internal/model/chat"| MS["Model Service<br/>:7705"]
    API -->|"enqueue job"| Redis["Redis Queue"]
    Redis --> QW["Queue Worker"]
    QW -->|"/internal/model/chat"| MS
    MS --> Backend["LLM Backend<br/>(MLX / GGUF / vLLM / Transformers / REST)"]

    subgraph "Process 1"
        API
    end
    subgraph "Process 2"
        MS
        Backend
    end
    subgraph "Process 3"
        QW
    end

Process	Port	Purpose
API Server	7704	Public-facing FastAPI app. Proxies chat requests to the model service, serves settings/training/pipeline endpoints. Does not load models.
Model Service	7705	Internal FastAPI app. Owns `ModelManager`, loads backends, runs inference. Protected by `X-Internal-Token`.
Queue Worker	---	RQ (Redis Queue) worker. Processes async jobs: background chat, adapter training, vision inference. Always uses the `background` model.

macOS exception

On macOS, the model service is disabled (RUN_MODEL_SERVICE=false). The API server loads models in-process to access Metal/MLX directly. The jarvis CLI handles this automatically.

The native macOS launchd plist also sets JARVIS_CONFIG_URL_STYLE=external so the API server can resolve app-to-app auth calls to Dockerized peers (like jarvis-auth) via their host-published localhost ports — host.docker.internal doesn't resolve from a native, non-Docker process. Requires jarvis-config-client >= 0.2.1. See Service Discovery: URL Resolution.

Health & Supervision¶

Model loading is fault-isolated per slot and does not block process startup:

The model service (:7705) binds immediately and loads models in a background thread; a failed load records that slot as failed instead of crashing the process, and a retry loop re-attempts with 60s→600s exponential cooldown. Its GET /health returns 200 whenever the process is alive, with per-slot state (slots, uptime_s) in the body.
The API server's GET /health (:7704) is the one the docker healthcheck actually watches (it only sees the status code, not the body), so it returns honest codes: 503 when the model service is unreachable, the live-model load failed, or loading has exceeded a 900s grace window; 200 with initializing inside the grace window; 200 with busy on a health-probe read-timeout (a long completion blocks the model service's single-worker event loop and must not flap the container unhealthy). The grace-window clock lives in the API process so a serve.sh respawn of the model service can't hide behind a fresh uptime counter.
scripts/serve.sh is the new default container CMD: execs the API server as PID 1 (migrations run first) and supervises the model service as a child, respawning it with capped backoff to catch native llama.cpp crashes that Python-level fault isolation can't. Compose files that still override the CMD directly should migrate to bash scripts/serve.sh to get supervision.

Model service network binding

Since jarvis-llm-proxy-api#26, the model service (:7705) is published on loopback only by default — both in the compose files (${MODEL_SERVICE_BIND_HOST:-127.0.0.1}:${MODEL_SERVICE_PORT:-7705}:7705) and on bare-metal runs via run.sh (--host ${MODEL_SERVICE_HOST:-127.0.0.1}). The API server and queue worker still reach it over the compose network (llm-proxy-model:7705) or 127.0.0.1 on bare metal — this only closes off-box access to the internal, token-protected /internal/model/* surface. Set MODEL_SERVICE_BIND_HOST (compose) or MODEL_SERVICE_HOST (bare metal) to 0.0.0.0 if a deployment genuinely needs off-box access to port 7705.

Since jarvis-llm-proxy-api#29, native macOS runs (run-prod.sh) also auto-set JARVIS_CONFIG_URL=http://localhost:${CONFIG_SERVICE_PORT:-7700} when unset — without it the service can't discover jarvis-auth via config-service and refuses to start.

Since jarvis-llm-proxy-api#30, if jarvis-auth or jarvis-logs can't be resolved via JARVIS_CONFIG_URL at import time (native macOS, before init() runs), the service falls back to hardcoded localhost defaults (http://localhost:7701 for auth, http://localhost:7702 for logs) — never reached in Docker, where compose sets these env vars explicitly.

2-Model System¶

The service maintains two model slots to balance latency and capability:

Slot	Purpose	Used By	Example
live	Real-time voice commands. Optimized for low latency.	Chat endpoint (default)	Qwen3-14B-Q6_K.gguf
background	Heavier async tasks. Can be a larger model.	Queue worker (always)	Qwen3-32B-Q4_K_M.gguf

Memory optimization: If both slots resolve to the same model path and backend type, ModelManager creates only one backend instance and shares it. This saves ~50% memory on constrained hardware.

Configuration Cascade¶

Each model slot checks settings in order: DB setting -> environment variable -> legacy fallback -> default.

Live model:
  Backend: model.live.backend → JARVIS_LIVE_MODEL_BACKEND → JARVIS_MODEL_BACKEND → "GGUF"
  Path:    model.live.name    → JARVIS_LIVE_MODEL_NAME    → JARVIS_MODEL_NAME

Background model:
  Backend: model.background.backend → JARVIS_BACKGROUND_MODEL_BACKEND → (falls back to live)
  Path:    model.background.name    → JARVIS_BACKGROUND_MODEL_NAME    → (falls back to live)

Hot Swap¶

Models can be swapped at runtime without restarting the service:

POST /internal/model/unload --- unloads all models (used before adapter training to free GPU VRAM)
POST /internal/model/reload --- reloads models from current settings

Inference Backends¶

All backends extend LLMBackendBase and implement generate_text_chat() and unload(). Optional methods include generate_vision_chat(), generate_text_chat_stream(), load_adapter(), and remove_adapter().

GGUF (llama.cpp)¶

The primary local backend. Works on all platforms.


Library	`llama-cpp-python` (pinned: `==0.3.23` for CUDA builds, `==0.3.16` for CPU/ROCm/Vulkan — unpinned installs let every CI rebuild silently pick up a new llama.cpp)
GPU	Metal (macOS), CUDA (Linux)
Adapter support	Constructor-based reload (destroys + recreates model with `lora_path`)
Multi-GPU	Default `JARVIS_GGUF_SPLIT_MODE=-1` (auto): resolves to layer split (`1`) when 2+ CUDA GPUs are visible and the topology is split-worthy — identical GPU names, or mixed cards that all clear an 8GB VRAM floor. A compute card next to a small display/spare card stays single-GPU. Set `JARVIS_GGUF_SPLIT_MODE=0` to force single-GPU, `1` (layer split) or `2` (row split) to force multi-GPU, + `JARVIS_GGUF_TENSOR_SPLIT` (e.g., `"0.5,0.5"` for 2 GPUs). An explicit `0`/`1`/`2` always overrides auto. Auto is NVIDIA-only (gated on `nvidia-smi`); AMD boxes are already pinned to a single discrete GPU by `select_discrete_gpu()` below.

Features:

Thread-safe inference via threading.Lock
Context caching (hash-based prefix matching)
Flash attention support (JARVIS_FLASH_ATTN=true)
Mirostat sampling
Warmup inference on load

Key env vars: JARVIS_N_GPU_LAYERS (-1 for all), JARVIS_N_THREADS, JARVIS_N_BATCH (512), JARVIS_FLASH_ATTN (true), JARVIS_GGUF_SPLIT_MODE (-1 = auto; 0/1/2 force single-GPU/layer/row split), JARVIS_GGUF_MAIN_GPU (0), JARVIS_GGUF_TENSOR_SPLIT.

Why auto-split defaults on

A prior single-GPU default (0) confined both model slots to GPU0 on identical dual-GPU boxes (e.g. dual RTX 3090), OOMing the background model at boot while the second GPU sat idle. Auto (-1) is the new default and only layer-splits when the topology genuinely supports it — see gpu_select.auto_gguf_split_mode() in jarvis-llm-proxy-api.

Discrete-GPU Auto-Select (Vulkan / ROCm)¶

Since jarvis-llm-proxy-api#18, gpu_select.select_discrete_gpu() runs at model-service startup — before ModelManager() triggers llama.cpp's Vulkan/HIP init — and pins the backend to the discrete GPU chosen by device type, not enumeration index:

Vulkan -- vulkaninfo --summary (or a stdlib ctypes libvulkan fallback) locates the DISCRETE_GPU device and sets GGML_VK_VISIBLE_DEVICES.
ROCm -- hipGetDevicePropertiesR0600().integrated (or a rocminfo fallback) locates the non-integrated device and sets HIP_VISIBLE_DEVICES.

This addresses the dGPU+iGPU footgun called out in the Multi-GPU row above: on a box with both a discrete GPU and an integrated GPU (e.g. an RX 9070 next to a Ryzen iGPU), the iGPU can enumerate as device 0, so hardcoding the device index binds the wrong adapter. Auto-select is stdlib-only, always respects an operator-set GGML_VK_VISIBLE_DEVICES / HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES, and is a no-op (leaves the backend default in place) if no discrete GPU is detected or on CPU/CUDA/Metal images, where the required tooling isn't present.

MLX (Apple Silicon)¶

Native Apple Silicon backend using the MLX framework.


Library	`mlx-lm`
GPU	Metal (unified memory)
Adapter support	In-place weight swap --- no model reload needed
Platform	macOS only

Features:

Sophisticated KV cache prefix matching --- finds common prefix between cached and new tokens, trims cache, only processes the suffix. This dramatically speeds up repeated system prompts.
Dynamic LoRA adapter swap via mlx_lm.tuner.utils.load_adapters() / remove_lora_layers() (modifies weights in place)
Vision support via PIL image handling

vLLM (High-Throughput GPU)¶

High-throughput backend for Linux with NVIDIA GPUs.


Library	`vllm`
GPU	CUDA (NVIDIA)
Adapter support	Per-request LoRA via `LoRARequest` --- no reload, concurrent adapters
Multi-GPU	Yes, via `tensor_parallel_size`

Features:

Native prefix caching
Per-request LoRA selection (multiple adapters served concurrently)
JSON structured output via StructuredOutputsParams
Manual chat template formatting (ChatML, Llama3, Mistral)
Supports GGUF files with explicit tokenizer override

Key env vars: JARVIS_VLLM_TENSOR_PARALLEL_SIZE, JARVIS_VLLM_GPU_MEMORY_UTILIZATION (0.9), JARVIS_VLLM_MAX_LORAS (1), JARVIS_VLLM_MAX_LORA_RANK (64).

Transformers (HuggingFace)¶

General-purpose backend using the HuggingFace ecosystem.


Library	`transformers`, `torch`
GPU	CUDA, MPS, CPU (auto-detected)
Adapter support	PEFT LoRA via `PeftModel.from_pretrained()`
Quantization	BitsAndBytes 4-bit / 8-bit

Key env vars: JARVIS_DEVICE, JARVIS_TORCH_DTYPE, JARVIS_USE_QUANTIZATION, JARVIS_QUANTIZATION_TYPE.

REST (Remote API Proxy)¶

Proxies inference to remote APIs --- useful for cloud LLMs, hosted inference servers, or distributed setups. This is how you connect Jarvis to OpenAI, Anthropic, Ollama on another machine, or your own hosted GPU server.


Library	`httpx` (async)
Providers	OpenAI, Anthropic, Ollama, LM Studio, generic
Auth	Bearer token, API key, or custom headers
Vision	Yes (converts images to data URLs)

The REST backend maintains a persistent httpx.AsyncClient and routes all async inference calls through a single dedicated background event loop. This keeps the connection pool bound to one stable loop for the lifetime of the backend instance, making the backend reliable under concurrent multi-request load --- for example, when the model service handles simultaneous voice commands.

Quick Setup¶

Set the backend to REST and point it at your provider:

OpenAIAnthropicOllama (remote)Self-hosted GPU server

# In .env or via settings DB
JARVIS_LIVE_MODEL_BACKEND=REST
JARVIS_LIVE_REST_MODEL_URL=https://api.openai.com
JARVIS_REST_PROVIDER=openai
JARVIS_REST_MODEL_NAME=gpt-4o
JARVIS_REST_AUTH_TYPE=bearer
JARVIS_REST_AUTH_TOKEN=sk-your-api-key

JARVIS_LIVE_MODEL_BACKEND=REST
JARVIS_LIVE_REST_MODEL_URL=https://api.anthropic.com
JARVIS_REST_PROVIDER=anthropic
JARVIS_REST_MODEL_NAME=claude-sonnet-4-20250514
JARVIS_REST_AUTH_TYPE=api_key
JARVIS_REST_AUTH_TOKEN=sk-ant-your-api-key

JARVIS_LIVE_MODEL_BACKEND=REST
JARVIS_LIVE_REST_MODEL_URL=http://192.168.1.50:11434
JARVIS_REST_PROVIDER=ollama
JARVIS_REST_MODEL_NAME=qwen2.5:14b
JARVIS_REST_AUTH_TYPE=none

JARVIS_LIVE_MODEL_BACKEND=REST
JARVIS_LIVE_REST_MODEL_URL=https://llm.yourdomain.com
JARVIS_REST_PROVIDER=openai
JARVIS_REST_MODEL_NAME=your-model
JARVIS_REST_AUTH_TYPE=bearer
JARVIS_REST_AUTH_TOKEN=your-server-token
JARVIS_REST_REQUEST_FORMAT=openai

Hybrid Setup (Local Live + Cloud Background)¶

You can mix local and remote backends --- e.g., fast local model for real-time voice, larger cloud model for deep research:

# Live: local GGUF for low-latency voice commands
JARVIS_LIVE_MODEL_BACKEND=GGUF
JARVIS_LIVE_MODEL_NAME=.models/Qwen3-14B-Q6_K.gguf

# Background: cloud model for async tasks (deep research, summarization)
JARVIS_BACKGROUND_MODEL_BACKEND=REST
JARVIS_BACKGROUND_REST_MODEL_URL=https://api.openai.com
JARVIS_REST_BACKGROUND_MODEL_NAME=gpt-4o
JARVIS_REST_AUTH_TYPE=bearer
JARVIS_REST_AUTH_TOKEN=sk-your-api-key

Full REST Configuration¶

Variable	Default	Description
`JARVIS_LIVE_REST_MODEL_URL`	---	Base URL for live model API
`JARVIS_BACKGROUND_REST_MODEL_URL`	---	Base URL for background model API (falls back to live)
`JARVIS_REST_PROVIDER`	`generic`	Provider type: `openai`, `anthropic`, `ollama`, `lmstudio`, `generic`
`JARVIS_REST_MODEL_NAME`	---	Model name for live requests
`JARVIS_REST_BACKGROUND_MODEL_NAME`	---	Model name for background requests
`JARVIS_REST_AUTH_TYPE`	`none`	Auth type: `bearer`, `api_key`, `custom`, `none`
`JARVIS_REST_AUTH_TOKEN`	---	Auth token or API key. Also configurable as DB setting `rest.auth_token` via Admin → Settings (DB value takes precedence; env var is the fallback; masked in UI; requires reload)
`JARVIS_REST_AUTH_HEADER`	`Authorization`	Custom auth header name (for `custom` auth type)
`JARVIS_REST_REQUEST_FORMAT`	`openai`	Request format: `openai`, `ollama`, `chatml`, `generic`
`JARVIS_REST_TIMEOUT`	`60`	Request timeout in seconds

Configuring the auth token via the admin UI

JARVIS_REST_AUTH_TOKEN is now also available as the rest.auth_token DB-backed setting. Set it in Admin → Settings → REST — the value is masked in the UI and takes precedence over the environment variable. Existing env-var deployments continue to work without any change; the env var remains as a fallback.

The provider setting controls response parsing (different APIs return results in different shapes). The request format controls how messages are serialized. For most OpenAI-compatible APIs (vLLM, LM Studio, text-generation-webui), use provider=openai + request_format=openai.

Native Tool Calling¶

Since jarvis-llm-proxy-api#4, the REST backend supports forwarding structured tool calls to OpenAI-compatible endpoints. This enables the ChatGPTOpenAI prompt provider in command-center to use native tool routing rather than text <tool_call> tag parsing.

When GenerationParams.tools is set, generate_text_chat routes through _chat_completion_with_tools() instead of the plain content path:

tools (OpenAI function definitions) and tool_choice are forwarded in the request body.
Structured tool_calls are parsed from message.tool_calls in the response and returned in ChatResult.tool_calls (previously always None).
The real finish_reason ("tool_calls" or "stop") is returned rather than always returning "stop".

When no tools are passed, the original content-only path runs unchanged — this is fully backward compatible.

Live behavior lane (manual / on-demand only): tests/manual/test_behavior_tool_routing.py routes a 10-utterance corpus (timers, weather, device control, music, news, list-adds) through a real gpt-4.1-nano to verify correct tool selection end-to-end. Excluded from normal CI; requires an OpenAI key:

source ~/.jarvis/secrets/openai.env
JARVIS_REST_PROVIDER=openai JARVIS_REST_AUTH_TYPE=bearer \
  .venv/bin/python -m pytest tests/manual/test_behavior_tool_routing.py -v

Mock (Testing)¶

Returns [mock-text:model] <input>. Used in tests.

Backend Comparison¶

Backend	Platform	Adapter Loading	Multi-GPU	Streaming	Vision
GGUF	All	Model reload	Tensor split	Yes	Separate backend
MLX	macOS	In-place swap	N/A (unified)	Yes	Yes
vLLM	Linux+CUDA	Per-request	Tensor parallel	Yes	Separate backend
Transformers	All	PEFT merge	device_map	No	Separate backend
REST	All (network)	N/A	N/A	No	Yes

Redis Queue¶

Async jobs are processed via RQ (Redis Queue). The queue worker runs as a separate process and always uses the background model.

Job Types¶

Type	Description	Callback
`chat`	Background LLM inference	Posts result to callback URL
`adapter_train`	LoRA adapter training	Posts job status to callback URL
`vision`	Vision inference (image + text)	Posts result to callback URL

Submitting Jobs¶

POST /internal/queue/enqueue

{
  "job_id": "unique-id",
  "job_type": "chat",
  "request": { "messages": [...], "model": "background" },
  "callback_url": "http://jarvis-command-center:7703/api/v0/callback",
  "ttl_seconds": 300,
  "idempotency_key": "optional-dedup-key"
}

Deduplication¶

Jobs with the same job_id + idempotency_key are deduplicated via Redis SET NX with a TTL. This prevents duplicate training jobs or repeated inference requests.

Configuration¶

Variable	Default	Description
`REDIS_URL`	---	Full Redis connection URL
`REDIS_HOST`	localhost	Redis host (if `REDIS_URL` not set)
`REDIS_PORT`	6379	Redis port
`REDIS_DB`	0	Redis database number
`REDIS_PASSWORD`	---	Redis password
`LLM_PROXY_QUEUE_NAME`	`llm_proxy_jobs`	Queue name
`RUN_QUEUE_WORKER`	true	Whether to start the worker process

LoRA Adapter Training¶

The service manages the full adapter lifecycle: training, storage, caching, and inference-time loading.

Training Flow¶

sequenceDiagram
    participant Client
    participant API as API Server :7704
    participant DB as PostgreSQL
    participant Redis
    participant Worker as Queue Worker
    participant MS as Model Service :7705
    participant GPU

    Client->>API: POST /internal/queue/enqueue (adapter_train)
    API->>DB: Create TrainingJob (QUEUED)
    API->>Redis: Enqueue job
    Redis->>Worker: Dequeue
    Worker->>MS: POST /internal/model/unload (free GPU VRAM)
    Worker->>GPU: Run training subprocess
    Note over GPU: train_adapter_mlx.py (macOS)<br/>train_adapter.py (Linux)
    GPU-->>Worker: Training complete
    Worker->>Worker: Zip adapter artifacts
    Worker->>DB: Update job (COMPLETE)
    Worker->>MS: POST /internal/model/reload
    Worker->>Client: POST callback_url (result)

Training Parameters¶

Variable	Default	Description
`JARVIS_ADAPTER_LORA_R`	16	LoRA rank
`JARVIS_ADAPTER_LORA_ALPHA`	32	LoRA alpha (scaling)
`JARVIS_ADAPTER_LORA_DROPOUT`	---	Dropout rate
`JARVIS_ADAPTER_LEARNING_RATE`	---	Learning rate
`JARVIS_ADAPTER_EPOCHS`	---	Number of training epochs
`JARVIS_ADAPTER_BATCH_SIZE`	---	Training batch size
`JARVIS_ADAPTER_MAX_SEQ_LEN`	---	Max sequence length
`JARVIS_ADAPTER_TRAIN_DTYPE`	---	Training dtype
`JARVIS_ADAPTER_TRAIN_LOAD_IN_4BIT`	---	4-bit quantized training

Adapter Storage¶

Adapters are stored locally and optionally synced to S3/MinIO:

Local cache: LLM_PROXY_ADAPTER_DIR (default: /tmp/jarvis-adapters)
S3 storage: s3://{bucket}/{prefix}/{dataset_hash}/adapter.zip
LRU cache: In-memory adapter path cache (max 10 entries, configurable) with optional disk eviction

Resolution order: local cache -> local zip extraction -> S3 download.

Adapter Loading at Inference¶

Adapters are loaded per-request via the adapter_settings field in chat requests:

{
  "messages": [...],
  "adapter_settings": {
    "hash": "abc123",
    "scale": 1.0,
    "enabled": true
  }
}

How each backend handles this:

Backend	Mechanism	Reload Required
GGUF	Destroys model, recreates with `lora_path`	Yes (full reload)
MLX	`load_adapters()` / `remove_lora_layers()`	No (in-place swap)
vLLM	`LoRARequest` per request	No (concurrent)
Transformers	`PeftModel.from_pretrained()`	Partial (merge/unmerge)

Pipeline System¶

The pipeline system orchestrates multi-step model builds: generate training data -> train adapter -> validate -> merge -> convert to GGUF/MLX.

Method	Path	Description
`POST`	`/v1/pipeline/build`	Start pipeline build
`GET`	`/v1/pipeline/status`	Current pipeline status
`POST`	`/v1/pipeline/cancel`	Cancel running pipeline
`GET`	`/v1/pipeline/logs`	SSE stream of build logs
`GET`	`/v1/pipeline/artifacts`	List models/adapters/GGUF/MLX on disk

Pipeline endpoints require superuser JWT authentication.

Embeddings¶

A separate EmbeddingManager provides text embeddings via sentence-transformers:

POST /v1/embeddings

Variable	Default	Description
`JARVIS_EMBEDDING_MODEL`	`all-MiniLM-L6-v2`	Embedding model name

Runs on CPU independently of the LLM backends (384 dimensions by default).

Full API Reference¶

Public API (Port 7704)¶

Method	Path	Auth	Description
`POST`	`/v1/chat/completions`	App auth	OpenAI-compatible chat completions
`GET`	`/v1/models`	None	List loaded models
`GET`	`/v1/engine`	None	Inference engine info
`POST`	`/v1/embeddings`	App auth	Text embeddings
`GET`	`/v1/training/status/{job_id}`	None	Training job status
`GET`	`/v1/adapters/date-keys`	None	Date key vocabulary
`POST`	`/v1/pipeline/build`	Superuser JWT	Start pipeline build
`GET`	`/v1/pipeline/status`	Superuser JWT	Pipeline status
`POST`	`/v1/pipeline/cancel`	Superuser JWT	Cancel pipeline
`GET`	`/v1/pipeline/logs`	Superuser JWT	SSE pipeline logs
`GET`	`/v1/pipeline/artifacts`	Superuser JWT	List on-disk artifacts
`GET`	`/settings/`	Combined auth	List all settings
`PUT`	`/settings/{key}`	Combined auth	Update setting
`POST`	`/internal/queue/enqueue`	App auth	Submit async job
`GET`	`/health`	None	Health check — 503/200 status codes reflect model load state; see Health & Supervision

Internal API (Port 7705)¶

Method	Path	Auth	Description
`POST`	`/internal/model/chat`	Internal token	Run chat inference
`POST`	`/internal/model/chat/stream`	Internal token	Streaming chat (SSE)
`GET`	`/internal/model/models`	Internal token	List loaded models
`POST`	`/internal/model/unload`	Internal token	Unload all models
`POST`	`/internal/model/reload`	Internal token	Reload models
`GET`	`/health`	None	Model service health — always 200 while the process is alive; body reports per-slot state (`slots`, `uptime_s`)

Dependencies¶

PostgreSQL --- training jobs table, settings table
Redis --- async job queue
MinIO/S3 (optional) --- adapter artifact storage

Dependents¶

jarvis-command-center --- primary consumer for intent classification and response generation
jarvis-tts --- LLM-generated wake word responses

Impact if Down¶

No LLM-based command parsing or response generation. Voice commands requiring intent classification will fail. Commands with pre_route() fast-path matching may still work.