Configuration

hedos reads one settings file and keeps its state in one data directory. Both follow the XDG conventions, and both can be moved with environment variables. Neither holds your model weights: hedos only records where they already sit.

The settings file#

The file is optional and hand-editable. hedos never needs it to exist: anything you leave out uses its default.

Loading is tolerant. A missing file or a file that is not valid TOML falls back to the defaults. A table that fails to decode (a wrong type, or a value hedos does not know) falls back to its defaults on its own, so a typo in [pull] never resets [models]. One thing to know: if hedos later saves the file (hedos runtimes approve does), it writes the defaults it fell back to, which drops the other keys of that broken table.

Writes go through a temporary file and a rename, guarded by a short-held lock on a hedos.toml.lock file next to it, so the shelf, the CLI and a running gateway never clobber each other's changes.

Settings are read when a command starts, so a running hedos serve or hedos shelf keeps the values it started with. Restart it after an edit.

Settings#

[models]#

[chat]#

[gateway]#

The gateway always binds 127.0.0.1.

[pull]#

Values outside a range are clamped to it.

[advanced]#

Keys hedos writes but does not read#

When hedos saves the file it writes every key it knows, so a saved file also carries [models] approved_network_runtimes and approved_network_runtime_hashes, [chat] default_model_id, show_stats, send_with_enter and default_bench, a [voice] table (default_voice, speed, auto_speak), and [gateway] enabled, host and max_connections. The current build does not act on any of them. In particular, changing host does not move the gateway off 127.0.0.1.

An example hedos.toml#

[models]
# Folders to scan for loose GGUF and safetensors files. Absolute paths.
watched_folders = ["/Volumes/models/gguf"]
# Another Hugging Face cache to sweep, besides the machine's own.
hf_cache_roots = ["~/work/hf-cache"]
# Keep a model loaded for an hour after its last request.
keep_warm = "one_hour"
# Let several models stay resident, up to 24 GiB between them.
eviction = "budgeted"
ram_budget_mb = 24576

[chat]
default_system_prompt = "Answer briefly."

[gateway]
port = 43367
max_concurrent_inference = 4

[pull]
max_concurrent = 2
auto_resume = true
partial_age_hours = 24
retry_window_minutes = 120
register_timeout_seconds = 120
keep_ended = 20

[advanced]
job_history_limit = 50
check_for_updates = true

Every value here is optional. A file with only [models] keep_warm = "one_hour" is complete.

The data directory#

State lives under ~/.local/share/hedos, or $XDG_DATA_HOME/hedos when that variable is set to an absolute path. If neither XDG_DATA_HOME nor HOME is set, hedos falls back to a hedos-data directory in the current directory. hedos creates what it needs on first use.

None of this holds model weights. Removing the whole directory loses the registry, generated artifacts, history and pull records, but no model: the next hedos scan finds them all again.

Environment variables#

Where hedos keeps things#

Where your models are#

hedos sees exactly one Hugging Face cache from the environment, the same rule the Hugging Face tools follow. Add other caches with models.hf_cache_roots. LM Studio's library is read from ~/.lmstudio/models and ~/.cache/lm-studio/models.

Runtimes#

hedos looks for ollama in /usr/local/bin, /opt/homebrew/bin and ~/.local/bin before PATH, and for uv in ~/.local/bin, /opt/homebrew/bin and /usr/local/bin before PATH.

The Python sidecars do not inherit your whole environment. They get an allowlist (PATH, HOME, USER, LOGNAME, SHELL, the locale and time zone, the temp and proxy variables, and the platform's library path), plus what the runtime itself declares. Credentials in your shell are not passed on.

The shelf screen#

The Ollama daemon and the image daemons (ComfyUI, AUTOMATIC1111) are reached over HTTP on their standard local ports. If Ollama is installed but not running, hedos starts it.

New releases#

The install script#

curl -fsSL https://hedos.ai/install | bash installs the binary into ~/.local/bin. Set HEDOS_BIN_DIR to install somewhere else. The hedos binary itself does not read this variable.

Manifest runtimes and consent#

Besides its built-in runtimes, hedos can serve a model through a runtime described by a manifest: a TOML file that says which models it handles, what it can do, and how to run it. Some ship inside the binary (the decision runtimes python:laya and python:zerank, and the extractor cli:tessera). You can add your own under runtimes.d/ in the data directory.

A manifest runtime runs code on your machine, as you, unsandboxed. So it stays inert until you approve it.

hedos runtimes                         # every manifest runtime and its approval
hedos runtimes approve python:laya     # show what it runs, then ask to confirm
hedos runtimes revoke python:laya      # take the approval back

RUNTIME        SERVES      CONSENT         MODELS
python:laya    chat,judge  needs approval  laya
python:zerank  chat,judge  needs approval  -

hedos ls also points out a model that waits on an approval:

laya can run on python:laya, which needs your approval: `hedos runtimes approve python:laya`

approve shows the runtime's files, what it runs and installs, the release it downloads and that release's sha256 when it pins one, the paths and network access it declares, the models it would serve, and its hash, then asks you to confirm (-y skips the question). A pinned release is downloaded and checked before the approval is recorded, so a failed download approves nothing; running approve again retries it. It records the id in models.approved_host_runtimes and the hash in models.approved_host_runtime_hashes. What a manifest declares is its own account and is not enforced.

The hash covers the manifest and every file beside it. If any of them changes, the approval stops counting and hedos runtimes shows the runtime as changed since approval until you approve it again.

Approvals are read when a process starts. After approving or revoking, restart a running hedos serve (the command reminds you when one is running).

Writing a manifest#

An entry in runtimes.d/ is either a bare *.toml file or a directory holding a manifest.toml beside its files. A runtime with an [env] (a Python environment) or a [serve] (a long-running process) must be a directory. A one-shot command can be a bare file:

id           = "my-tool"
capabilities = ["chat"]
execution    = "sync"
detect       = { extension = "gguf" }

[invoke]
command = "my-tool --model {model} --prompt {prompt}"

A few rules hedos checks when it loads them:

• An id may use letters, digits, dots, underscores, colons and hyphens. Ids of built-in and shipped runtimes are reserved, and a duplicate id keeps the first one.

• A manifest declares exactly one of [invoke] or [serve].

• A manifest needs a detect rule to ever match a model, and the rule uses exactly one form:

  • extension = "gguf": the model's weight file ends in it.
  • file = "bundle.json", optionally with contains = "...": a marker file beside the model, holding that text when contains is given.
  • model_type = "yue2": the model_type the model's config.json names, read as JSON.
  • pipeline = "AceStepPipeline": the diffusers pipeline class its model_index.json names, read as JSON.

  A manifest runtime bids last, so it only serves models no built-in runtime takes.

• execution = "job" is for exactly the runtimes that generate (image or audio). A manifest cannot declare both speak and audio: a model is either a voice reading text or a generator making audio from a description.

• A manifest with a [vm] section is reported as an issue and not run: this build cannot start a VM.

An [invoke] command gets its model's snapshot directory as {model}, and the prompt as {prompt}. A command that takes a structured request can read it on standard input instead, which keeps a document out of the process list:

[invoke]
command = "my-tool json --bundle {model}"
stdin   = "{payload}"

{payload} is the only value stdin takes: the request as one line of JSON. What the command writes to standard output is the answer, exactly as written. Exit status 0 is success, 2 means the request itself is wrong (the gateway answers 400 with the command's last stderr line), and any other status is a failure (500).

A command that is not on the machine can be downloaded instead. An [install] section pins a release archive per platform, by the Rust target triple, and the binary inside it; the command runs it as {bin}:

[invoke]
command = "{bin} json --bundle {model}"
stdin   = "{payload}"

[install]
version = "1.2.0"

[install.targets.aarch64-apple-darwin]
url    = "https://example.com/my-tool-1.2.0-aarch64-apple-darwin.tar.gz"
sha256 = "<the archive's sha256, 64 lowercase hex digits>"
binary = "my-tool-1.2.0-aarch64-apple-darwin/my-tool"

The archive is a .tar.gz fetched over https, and binary is a relative path inside it. hedos runtimes approve downloads this machine's archive, refuses it unless its sha256 matches the pin, and unpacks the one binary into commands/ in the data directory. A command that runs {bin} must declare [install], and one that declares [install] must run {bin}. Until the binary is in place, the runtime answers that it is not installed yet and names the command that fetches it.

A runtime can say what a request may carry. Each [[params]] entry has a lowercase key and a type (int, float, bool, string or enum). An enum must list its choices in a non-empty values, which no other type may carry. Any entry may give a default of its type, and a number a range = [min, max] with the default inside it. A model the runtime serves takes these as its parameters when identification found none of its own, and gives them back when the runtime is revoked:

[[params]]
key     = "cot"
type    = "enum"
values  = ["full", "melody", "off"]
default = "full"

A model that needs another repository beside its own to run (a decoder, a codec) names it under [companions], by role, at most four:

[companions]
vae = { repo = "m-a-p/YuE2-Vae" }

hedos looks for each companion among the Hugging Face repos on the shelf and passes it to a [serve] runtime as --companion vae=<snapshot directory>. Its size counts toward the model's fit. A model whose companion is not on the shelf is refused with the hedos pull that fetches it; hedos ls says the same, and hedos runtimes approve offers to pull it (-y accepts). Nothing is downloaded without that yes.

Bringing an audio model#

A model that makes audio with its own code (HeartMuLa, DiffRhythm, Stable Audio 3, or one hedos has never heard of) is served by a runtime folder in runtimes.d, with no change to hedos: python:yue2 is built this way. hedos has to see that the model makes audio before a runtime can lend it the audio capability: its config.json names a model type hedos knows as an audio generator, or its model card (README.md) is filed under text-to-audio or text-to-music, or tagged music with no chat template beside it. A model with none of these whose architecture ends in ForCausalLM reads as a chat model and keeps chat; give it a card that says what it is. The folder holds a manifest.toml, a requirements.lock (uv pip compile --generate-hashes), and a main.py:

id           = "python:my-music"
modalities   = ["audio"]
capabilities = ["audio"]
execution    = "job"
detect       = { model_type = "my_music" }

[[params]]
key  = "lyrics"
type = "string"

[[params]]
key   = "duration"
type  = "float"
range = [5, 240]

[companions]
codec = { repo = "someone/my-music-codec" }

[env]
lockfile = "requirements.lock"

[serve]
entrypoint = "main.py"

hedos starts main.py --model <snapshot> --workdir <dir>, plus --companion <role>=<snapshot> for each companion, and speaks to it in frames on stdin and stdout: a little-endian u32 length (of what follows), a type byte (1 for JSON, 2 for bytes), then the payload. Anything the model's libraries print belongs on stderr, so main.py moves stdout aside first (see a shipped runtime's opening lines).

1. Load the model, then send {"event": "ready"}.
2. Each job arrives as {"op": "audio", "prompt": "...", "format": "flac", ...} with the fields the request carried, already checked against [[params]]. {"op": "shutdown"} asks the process to exit.
3. While it works, send {"event": "begin"}, then any of {"event": "status", "message": "..."} and {"event": "step", "n": 3, "total": 32}. hedos checks for silence every 600 seconds and stops a job that has sent nothing for at least that long, so a long stage with no progress of its own should send a status now and then.
4. Between steps, read stdin without blocking for {"op": "cancel"} (or shutdown); on one, stop and send {"event": "cancelled"}.
5. Send the file as {"event": "result", "format": "flac", "bytes": N}, then binary frames of at most 16 MiB each that add up to exactly N bytes, then {"event": "done"}.
6. A request the model cannot take is {"event": "error", "message": "...", "fault": "request"} (a 400 at the gateway, or the final error event of a streamed answer); any other failure leaves fault out (a 500, likewise).

The model folder it serves is found by detect; its parameters are what hedos audio and /v1/audio/generations accept for it; and nothing runs until hedos runtimes approve python:my-music.

Problems show up as issue: lines in hedos scan.