An ancient oak alone in a misty meadow at blue hour, its exposed roots threaded with softly glowing cyan fibers
Fringe Tech 22 Oct 2026 12 min read

Self-Hosting Honcho as Long-Term Memory for a Home AI Agent, with Local Models

Self-hosted Honcho gives my home AI agent long-term memory: llama.cpp on a GPU box does the reasoning, Ollama on a Mac mini does the embeddings, and the stack runs on my own machines. Compose changes, per-feature model config, the embedding bootstrap order and a day-one test.

My home AI agent runs Hermes Agent (open source) on a small Linux server. It now has automatic, searchable memory across sessions from Honcho, the open-source memory service from Plastic Labs (AGPL-3.0, version 3.3.0). I tell the agent something in one chat, and a new chat later can recall it.

Everything runs on my own machines:

How it works: Honcho stores the conversation messages. A background worker called the deriver reads them and writes observations about each participant, which Honcho calls a peer. The agent gets that context injected into each turn and has tools to search it and to ask questions about the person.

This does not replace the agent's built-in memory, the small hand-curated MEMORY.md and USER.md files. The two run side by side, and Honcho imported the existing USER.md on its first run.

It is early. I installed and tested this on day one: a fact was extracted within a minute and recalled in a new session. How well it holds up over weeks is still open.

What you need

Footprint: the stack uses about 610 MB of RAM at idle. That is api 290 MB, deriver 274 MB, Postgres 43 MB and Redis 5 MB.

The config uses Honcho's terms, so here they are:

Self-hosting is free. Plastic Labs also runs Honcho Cloud on the same open-source server, with a free tier; this post covers self-hosting.

Setup choices and why

Honcho on the agent server, API on 127.0.0.1, auth off. The agent is Honcho's one client. Since the API is not exposed on the LAN, I skip JWT and token setup. Postgres and Redis are not published to the host.

One local model per feature, set in config.toml. The defaults point at OpenAI's gpt-5.4-mini, so a feature left unset calls OpenAI. The features to override are deriver, summary, the five dialectic levels, dream deduction, dream induction and embedding.

Thinking off for fast work, on for deep work. Deriver, summary and dialectic minimal, low and medium run with thinking off, because extraction does not need it and speed matters. Dialectic high and max and both dream features run with thinking on, because they run rarely and go deeper. The switch goes through Honcho's documented provider_params.extra_body escape hatch, which passes chat_template_kwargs straight to llama.cpp.

1024-dimension embeddings, set before the first data. The dimension stays fixed for the life of the deployment, and the API does not start on a mismatch. The model can change later, as long as it can return the same dimension (step 7).

qwen3-embedding:4b for embeddings. It beat the 0.6B model on my retrieval test for about 2 GB more RAM on the Mac mini. The numbers are in step 7.

Three llama.cpp slots sharing one KV pool. The deriver makes background calls after each message. On a one-slot server those calls queue behind live chats.

One Honcho peer per person. More than one person messages the agent, and the local CLI user is a separate identity. Each person gets their own peer, and my own identities are aliased to one peer.

1. Adjust the compose file

Clone the Honcho repo and start from its compose file. Make these changes:

The changed keys look like this. The rest stays as upstream ships it.

services:
  api:
    ports: ["127.0.0.1:8000:8000"]
    volumes: ["./config.toml:/app/config.toml:ro"]
    restart: unless-stopped
  deriver:
    volumes: ["./config.toml:/app/config.toml:ro"]
    restart: unless-stopped
  database:
    image: docker.io/pgvector/pgvector:pg15
    restart: unless-stopped
  redis:
    image: docker.io/library/redis:8.2
    restart: unless-stopped

/app is the image's working directory, so Honcho picks up config.toml there. Keep the volumes the example file already lists and add this one.

Podman tip: run the standalone docker-compose binary against the user Podman socket with DOCKER_HOST=unix:///run/user/$UID/podman/podman.sock.

Podman tip: enable podman.socket and podman-restart.service as user units and turn on lingering, so unless-stopped containers come back after a reboot:

systemctl --user enable --now podman.socket podman-restart.service
loginctl enable-linger $USER

Podman tip: short image names fail to resolve non-interactively. The fully qualified names above work on both runtimes.

Podman tip: on an SELinux host, mount config.toml with a lowercase :ro,z. An uppercase Z gives the file a label private to one container, so whichever of api and deriver starts second cannot read it. Honcho then starts on its OpenAI defaults without an error. I found this when the re-embed in step 7 got a 401 from OpenAI.

Hold off on starting the stack until the embedding dimension is set in step 3.

2. config.toml: one local model per feature

Each feature gets its own model block. Here is the deriver:

[deriver.model_config]
transport = "openai"
model = "qwen3.8"
[deriver.model_config.overrides]
base_url = "http://<gpu-box>:8080/v1"
api_key_env = "LLM_API_KEY"
[deriver.model_config.overrides.provider_params]
timeout = 600
[deriver.model_config.overrides.provider_params.extra_body.chat_template_kwargs]
enable_thinking = false

What each line does:

Repeat the same block for the remaining LLM features, changing the section names:

A feature without a block falls back to OpenAI, so each one gets a block.

Tip: set a dummy LLM_OPENAI_API_KEY in Honcho's environment. The server expects a provider key at startup, and the local endpoint does not receive it.

Tip: some OpenAI-compatible servers lack json_schema support. For those, DERIVER_MODEL_CONFIG__STRUCTURED_OUTPUT_MODE=json_object exists. I did not need it with llama.cpp.

An orderly greenhouse at night, each growing bed under its own color of grow light: warm amber, cool white, soft violet
Each feature gets its own model settings, and the models run locally. Illustration.

3. Embeddings, then the bootstrap order

The embedding block goes in the same config.toml:

[embedding]
VECTOR_DIMENSIONS = 1024
[embedding.model_config]
transport = "openai"
model = "qwen3-embedding:4b"
max_batch_size = 32
dimensions_mode = "always"
[embedding.model_config.overrides]
base_url = "http://<embedding-box>:11434/v1"
api_key = "ollama"

The 4B model's full output is 2560 dimensions. Qwen3-Embedding models are trained so a vector can be cut short and still work, and Ollama honors the OpenAI dimensions parameter. dimensions_mode = "always" makes Honcho send dimensions=1024 on each call, so the vectors match VECTOR_DIMENSIONS. Honcho's default auto mode also sends it once VECTOR_DIMENSIONS is set; I set it explicitly so the intent is visible in the file. api_key is a placeholder; Ollama does not check it. The Mac mini already serves embeddings to other services in the house, so the only new piece there was the model (ollama pull qwen3-embedding:4b).

Honcho's database columns default to vector(1536). The order below comes from the docstring of Honcho's own embedding script:

  1. Start the database and Redis.
  2. Run the migrations with scripts/provision_db.py.
  3. Run scripts/configure_embeddings.py --yes. It alters the columns from vector(1536) to vector(1024) and rebuilds the HNSW indexes.
  4. Start the api and the deriver.
docker compose up -d database redis
docker compose run --rm --no-deps --entrypoint /app/.venv/bin/python api scripts/provision_db.py
docker compose run --rm --no-deps --entrypoint /app/.venv/bin/python api scripts/configure_embeddings.py --yes
docker compose up -d api deriver

The order matters because the script stops if embeddings already exist, and the API does not start on a dimension mismatch. I chose the dimension once, before the first message.

4. llama.cpp: three slots so memory work does not block chats

On the GPU box, add two flags to llama-server:

llama-server -m <model>.gguf --port 8080 --parallel 3 --kv-unified

--parallel 3 gives the server three slots. --kv-unified makes those slots share one context pool, so a single long chat can still use the full window.

Measured on a 5090-class GPU:

For the agent, this means a reply and two deriver extractions can run at the same time. The reply no longer waits behind the extractions.

Three wooden canoes with small lanterns converging on one lit dock across a still, misty lake at dusk
Three slots heading into one shared pool of context. Illustration.

5. Hermes plugin with one peer per person

  1. Install the plugin:

``bash hermes plugins install honcho ``

  1. Select it as the memory provider in the agent's config.yaml:

``yaml memory: provider: honcho ``

  1. Write ~/.hermes/honcho.json:

``json {"baseUrl": "http://127.0.0.1:8000", "hosts": {"hermes": {"enabled": true, "workspace": "hermes", "aiPeer": "assistant", "peerName": "me", "pinUserPeer": false, "userPeerAliases": {"<messaging-account-id>": "me", "uid:<n>": "me"}, "recallMode": "hybrid", "writeFrequency": "async"}}} ``

What each key does:

Tip: the CLI user appears as uid:<n>, where <n> is the numeric user id of the account running the CLI. Without an alias its messages land on a separate peer, and the log says "session user peer 'uid-<n>' is not the declared owner".

Restart the agent, then check the resolved config and the connection:

hermes honcho status

Trimmed to the useful lines, with my peer names replaced:

Honcho status
  Host:           hermes
  Enabled:        True
  Auth:           API key (not set)
  Workspace:      hermes
  AI peer:        assistant
  User peer:      me
  Recall mode:    hybrid
  Dialectic cad:  every 1 turn
  Reasoning:      base=low, cap=high, heuristic=on
  Write freq:     async

  Connection...
  OK

Auth: API key (not set) is expected: a local baseUrl skips key auth.

6. Test it

  1. In one session, tell the agent a harmless fact. Mine was "my favorite tree is the big oak by the pond."
  2. Within a minute, check the database. In Honcho's schema, documents holds the observations:

``bash docker compose exec -T database psql -U postgres -tAc \ "select count() from messages; select count() from documents;" ``

The messages were stored, their embeddings were present, and the deriver had written one explicit observation that contained the fact.

  1. Open a new session and ask the agent to use honcho_search to look up your favorite tree. Mine answered correctly. The context injected at the start of the session carried the fact as well, so the agent had it before it searched.

Counts at the end of day one, a handful of peers in the workspace:

 count
-------
    18
(1 row)

 count
-------
    35
(1 row)

That is 18 messages and 35 observations. The observations include what Honcho imported from the existing USER.md.

A hand placing a smooth stone on top of a trail cairn above a sea of clouds at dawn, a small device with a soft blue light at its base
Stored in one session, found again in the next. Illustration.

7. Changing the embedding model after data exists

I started on qwen3-embedding:0.6b and moved to qwen3-embedding:4b on day two. Both return 1024 dimensions on request, so the database columns stayed as they were.

The test: 25 short facts of the kind the agent remembers, and 24 questions that ask for them indirectly, such as "how is the house heated" for a fact about the heat pump. Each model embedded the facts and questions the way Honcho calls it, at 1024 dimensions, and I ranked the facts by cosine similarity.

ModelTop-1 hitsMRRTime per queryMemory when loaded
qwen3-embedding:0.6b17/240.82540 ms2.4 GB
qwen3-embedding:4b19/240.860150 ms4.4 GB

The 4B model found two more facts on the first try, and 150 ms per query is small next to a local LLM call. Public MTEB tables put the 8B model about a point above the 4B for roughly double the memory again. On a 16 GB Mac mini that runs other services, the 4B is the better trade.

Vectors from two different models are not comparable, so each stored row needs a new embedding from the new model. Honcho 3.3.0 ships a script that fills in messages with no embedding, but not one that redoes existing ones. This short script runs inside the api container and uses Honcho's own embedding client and config:

"""Re-embed all Honcho documents/message_embeddings row with the currently configured embedding model.

Run inside the api container from /app.
"""
import asyncio
import sys

sys.path.insert(0, "/app")
from sqlalchemy import text  # noqa: E402

from src.config import settings  # noqa: E402
from src.db import engine  # noqa: E402
from src.embedding_client import embedding_client  # noqa: E402


async def main():
    print("model", settings.EMBEDDING.MODEL_CONFIG.model, "dims", settings.EMBEDDING.VECTOR_DIMENSIONS)
    async with engine.begin() as conn:
        for table in ("documents", "message_embeddings"):
            rows = (await conn.execute(text(f"SELECT id, content FROM {table}"))).all()
            vecs = await embedding_client.simple_batch_embed([r.content for r in rows], on_oversize="truncate")
            for r, v in zip(rows, vecs):
                assert len(v) == settings.EMBEDDING.VECTOR_DIMENSIONS
                await conn.execute(text(f"UPDATE {table} SET embedding = CAST(:v AS vector) WHERE id = :id"),
                                   {"v": "[" + ",".join(map(str, v)) + "]", "id": r.id})
            print(table, "re-embedded", len(rows))


asyncio.run(main())

The steps:

# 1. Back up the database
docker compose exec -T database pg_dump -U postgres -Fc postgres > honcho-before-reembed.dump
# 2. Pull the new model on the embedding box, change `model` in config.toml, recreate api and deriver
docker compose up -d --force-recreate api deriver
# 3. Re-embed all rows
docker compose cp reembed.py api:/tmp/reembed.py
docker compose exec -w /app api .venv/bin/python /tmp/reembed.py
model qwen3-embedding:4b dims 1024
documents re-embedded 35
message_embeddings re-embedded 18

With 53 vectors it took a few seconds. The job grows with each message, so choosing the model early keeps it cheap.

Then I re-ran the test from step 6. A new test note came back first in search for a reworded question, and in a fresh session the agent recalled both the new note and the facts from day one. I deleted the test sessions afterwards.

Results and what's next

What is verified:

This is two days of use. Still open: how well the observations hold up over weeks, how dream consolidation behaves, and whether dialectic at the high and max levels is worth its cost on a local model.

Next:

Sources

Fringe Tech Self-Hosting AI

Comments

// Comments are reviewed before appearing. No spam. No noise.