{"key":"brain_project_implementation_handoff_2026_03_22","title":"Brain Project Implementation Handoff (2026-03-22)","content":"Project summary\n- On 2026-03-22, a new self-hosted memory system project named `brain` was created and deployed on `svc-dev` at `/home/svc-admin/projects/brain`.\n- The design goal was a shared core memory backend for multiple AI clients, with one common memory store and first-class source metadata so different clients can write into separate lanes without requiring separate installs.\n- The initial supported sources are intended to be `codex`, `claude`, `gemini`, `manual`, and imported `contextkeep` data.\n\nCore architectural decisions\n- Use one shared backend instead of one database per AI client.\n- Make `source` a first-class memory field rather than relying only on tags.\n- Keep the app compatible with both a bundled local Postgres container and an external Postgres instance later.\n- App code always reads a single `DATABASE_URL`.\n- Default deployment uses a bundled Postgres service in `docker-compose.yml`.\n- Switching to an external DB later only requires changing `.env`, not changing app code or adding a second compose file.\n- Keep REST API and MCP server as separate services in the same stack.\n- Use FastAPI for the REST API and Python MCP server implementation.\n- Use Alembic for schema migrations rather than ad hoc table creation.\n\nChosen deployment details\n- Project directory: `/home/svc-admin/projects/brain`\n- Host: `svc-dev` (`192.168.4.123`)\n- API port: `8010`\n- MCP port: `8011`\n- Database name: `ai-brain`\n- Database user: `ai-brain`\n- Local default database password: `120dMOtVxpoHrwdT`\n\nBundled stack layout\n- `brain-api`: FastAPI REST service\n- `brain-mcp`: MCP server exposed over HTTP\n- `brain-db`: Postgres 16\n\nImportant project files\n- `/home/svc-admin/projects/brain/docker-compose.yml`\n- `/home/svc-admin/projects/brain/.env`\n- `/home/svc-admin/projects/brain/.env.example`\n- `/home/svc-admin/projects/brain/README.md`\n- `/home/svc-admin/projects/brain/backend/Dockerfile`\n- `/home/svc-admin/projects/brain/backend/requirements.txt`\n- `/home/svc-admin/projects/brain/backend/alembic.ini`\n- `/home/svc-admin/projects/brain/backend/alembic/env.py`\n- `/home/svc-admin/projects/brain/backend/alembic/versions/0001_initial.py`\n- `/home/svc-admin/projects/brain/backend/alembic/versions/0002_add_memory_source.py`\n- `/home/svc-admin/projects/brain/backend/app/main.py`\n- `/home/svc-admin/projects/brain/backend/app/db.py`\n- `/home/svc-admin/projects/brain/backend/app/models.py`\n- `/home/svc-admin/projects/brain/backend/app/schemas.py`\n- `/home/svc-admin/projects/brain/backend/app/crud.py`\n- `/home/svc-admin/projects/brain/backend/app/serializers.py`\n- `/home/svc-admin/projects/brain/backend/app/api/health.py`\n- `/home/svc-admin/projects/brain/backend/app/api/memories.py`\n- `/home/svc-admin/projects/brain/backend/app/api/tags.py`\n- `/home/svc-admin/projects/brain/backend/app/mcp_server.py`\n\nEnvironment and configuration\n- `.env` defines `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `DATABASE_URL`, `API_PORT`, and `MCP_PORT`.\n- Local/default `DATABASE_URL` points to the bundled Postgres service hostname `db`.\n- External Postgres support is implemented by overriding `DATABASE_URL` in `.env`, for example pointing it to a remote DB host instead of `db`.\n\nData model decisions\n- `Memory` table stores the current canonical record.\n- Fields include `key`, `title`, `content`, `source`, `status`, `created_at`, and `updated_at`.\n- `source` is indexed and defaults to `manual`.\n- `status` is indexed and defaults to `active`.\n- Tags are normalized via a dedicated `Tag` table and `memory_tags` join table.\n- Revision history is stored in a `Revision` table.\n- Deletes are implemented as archive semantics rather than hard delete at the API layer.\n\nSchema and migrations\n- Alembic migration `0001_initial.py` created the base schema for memories, tags, memory_tags, and revisions.\n- Alembic migration `0002_add_memory_source.py` added first-class `source` support to the `memories` table.\n- Container startup runs `alembic upgrade head` before launching the API process.\n- Early in development, the first prototype created tables before Alembic existed. To move to migration-owned schema, the project Postgres volume had to be reset once with `docker compose down -v`. After that reset, migrations became the source of truth and the schema stabilized.\n\nREST API implemented\n- `GET /health`\n- `GET /memories`\n- `GET /memories?status=all`\n- `GET /memories?q=<text>`\n- `GET /memories?tag=<tag>`\n- `GET /memories?source=<source>`\n- `GET /memories/{key}`\n- `POST /memories`\n- `PUT /memories/{key}`\n- `DELETE /memories/{key}` for archive behavior\n- `GET /tags`\n\nMCP server implementation\n- A dedicated MCP server service named `brain-mcp` was added to the compose stack.\n- The MCP endpoint is exposed at `http://192.168.4.123:8011/mcp`.\n- Implemented MCP tools:\n  - `list_memories`\n  - `retrieve_memory`\n  - `search_memories`\n  - `store_memory`\n  - `archive_memory`\n  - `list_tags`\n- The MCP layer uses the official Python `mcp` package.\n\nMCP host-header decision and fix\n- During early testing, the MCP endpoint returned host-header rejection / `421 Misdirected Request` behavior for requests using `192.168.4.123`.\n- The fix was to update `backend/app/mcp_server.py` so the FastMCP/transport security settings explicitly allow expected hosts.\n- Allowed host patterns were expanded to include loopback and LAN-reachable forms, including `127.0.0.1`, `localhost`, `192.168.4.123`, and `svc-dev` with port patterns.\n- After that fix, the endpoint behaved correctly for external access and clients could connect successfully.\n\nShared-brain design decision\n- The preferred architecture is one shared brain rather than one per AI client.\n- The separation model is implemented through `source` rather than separate deployments.\n- Example intended source values: `codex`, `claude`, `gemini`, `manual`, `contextkeep`.\n- This allows one source of truth while preserving per-client filtering and future namespace expansion.\n\nCurrent imported legacy data\n- Existing ContextKeep memories were imported into `brain`.\n- Import scope: all 54 existing ContextKeep memory records.\n- Imported records retained:\n  - original `key`\n  - original `title`\n  - full `content`\n  - tags\n  - original `created_at` and `updated_at` timestamps\n- Imported records were assigned `source=contextkeep`.\n- Imported records were loaded as current-state records only; synthetic revision history was not backfilled.\n\nHow ContextKeep import was implemented\n- The ContextKeep MCP transport became unreliable during bulk retrieval and began failing with transport-closed errors.\n- Instead of continuing through the MCP tool layer, the underlying ContextKeep storage implementation was inspected locally.\n- ContextKeep stores memories as JSON files under: `/home/svc-admin/ai-projects/projects/homelab/contextkeep/data/memories`\n- Each file contains fields like `key`, `title`, `content`, `tags`, `created_at`, and `updated_at`.\n- A direct export file was generated from those JSON documents.\n- The export file was copied to `svc-dev`.\n- A one-off Python import script was copied into the `brain-api` container and executed with `PYTHONPATH=/app` so it could import `app.db` and `app.models`.\n- The importer created missing tags, inserted memories, set `source=contextkeep`, and preserved the original timestamps.\n\nCommands and operational steps used during implementation\n- SSH / deployment inspection on `svc-dev`:\n  - `ssh svc-dev ...`\n- Compose / stack verification:\n  - `docker compose up -d`\n  - `docker compose down -v` (used once during early schema reset before Alembic became canonical)\n  - `docker ps`\n  - `docker exec ...`\n  - `docker cp ...`\n- API verification:\n  - `curl http://127.0.0.1:8010/health`\n  - `curl http://127.0.0.1:8010/memories`\n  - `curl http://127.0.0.1:8010/openapi.json`\n  - `curl http://127.0.0.1:8010/tags`\n- Database verification:\n  - `docker exec brain-db psql -U ai-brain -d ai-brain -c '\\dt'`\n  - `docker exec brain-db psql -U ai-brain -d ai-brain -c '\\d memories'`\n- MCP verification:\n  - MCP client tool listing against loopback inside the container\n  - direct HTTP checks against `http://192.168.4.123:8011/mcp`\n- Codex MCP registration:\n  - `codex mcp add brain --url http://192.168.4.123:8011/mcp`\n  - `codex mcp list`\n- Claude MCP registration:\n  - `claude mcp add --transport http brain http://192.168.4.123:8011/mcp`\n  - `claude mcp list`\n- Gemini MCP registration:\n  - `gemini mcp add --scope user --transport http brain http://192.168.4.123:8011/mcp`\n  - `gemini mcp list`\n- ContextKeep import prep:\n  - inspection of `/home/svc-admin/ai-projects/projects/homelab/contextkeep/server.py`\n  - inspection of `/home/svc-admin/ai-projects/projects/homelab/contextkeep/core/memory_manager.py`\n  - export generation from JSON files under `data/memories`\n  - `scp` to move import artifacts to `svc-dev`\n  - `docker cp` into `brain-api`\n  - `docker exec -e PYTHONPATH=/app brain-api python /tmp/contextkeep_import.py`\n\nClient integration status\n- Codex is configured to use the `brain` MCP server.\n- Claude is configured to use the `brain` MCP server and reported connected health.\n- Gemini is configured to use the `brain` MCP server and reported connected health.\n- All three are intended to use the same backend rather than separate installs.\n\nImportant implementation notes\n- Gemini initially accepted an MCP add command without explicit transport/scope, but that defaulted to an unusable project/stdio shape. The fix was to re-add it explicitly as `--scope user --transport http`.\n- Claude stored the server in its local project config under `/home/svc-admin/.claude.json`.\n- Gemini stored the server under `/home/svc-admin/.gemini/settings.json`.\n- Codex stored the MCP server registration in `/home/svc-admin/.codex/config.toml`.\n\nValidation completed\n- API health endpoint returned OK.\n- Memory CRUD and archive flow worked before import.\n- Search and filtering by text, tag, and source worked.\n- Tag listing worked.\n- MCP server tool listing worked.\n- Host-header restriction was fixed for the MCP endpoint.\n- After ContextKeep import, `GET /memories?status=all` returned 54 records.\n- After import, `GET /memories?source=contextkeep&status=all` returned 54 records.\n- Spot checks confirmed the expected tags and preserved timestamps on imported memories.\n\nCurrent state after implementation\n- `brain` is live on `svc-dev`.\n- REST API base URL: `http://192.168.4.123:8010`\n- MCP endpoint: `http://192.168.4.123:8011/mcp`\n- ContextKeep memories have been imported.\n- Shared-brain design with first-class `source` is in place.\n- Multiple AI clients are configured to use the same MCP endpoint.\n\nKnown limitations / future work\n- No web UI exists yet. Current interfaces are REST and MCP only.\n- `namespace` is not implemented yet; `source` is the current separation layer.\n- Imported ContextKeep records do not have reconstructed revision history.\n- Direct import tooling used for migration was ad hoc and not yet committed as a reusable migration command inside the project.\n- Prompt/template records and filtered views are still future work.\n\nRecommended next steps\n- Add `namespace` as a first-class field if a second layer of separation becomes necessary.\n- Build a web UI for browsing, filtering, and editing memories.\n- Turn the ContextKeep import path into a reusable project command or script if future migrations are expected.\n- Decide whether to disable or retire ContextKeep from daily use now that `brain` is active.\n- Add docs/examples showing how Codex, Claude, and Gemini should store memories using `source` consistently.\n\nThis memory is intended to function as the repo-adjacent implementation handoff for the initial `brain` build and migration work.","summary":"Project summary\n- On 2026-03-22, a new self-hosted memory system project named `brain` was created and deployed on `svc-dev` at `/home/svc-admin/projects/brain`.\n- The design goal was a shared core memory backend for multiple AI clients, with one common memory store and first-class source metadata so different clients can write into separate lanes without requiring separate installs.\n- The initial supported sources are intended to be `codex`, `claude`, `gemini`, `manual`, and imported `contextkeep` data.\n\nCore architectural decisions\n- Use one shared backend instead of one database per AI client.\n- Make `source` a first-class memory field rather than relying only on tags.\n- Keep the app compatible with both a bundled local Postgres container and an external Postgres instance later.\n- App code always reads a single `DATABASE_URL`.\n- Default deployment uses a bundled Postgres service in `docker-compose.yml`.\n- Switching to an external DB later only requires changing `.env`, not changing app code or adding a second compose file.\n- Keep REST API and MCP server as separate services in the same stack.\n- Use FastAPI for the REST API and Python MCP server implementation.\n- Use Alembic for schema migrations rather than ad hoc table creation.\n\nChosen deployment details\n- Project directory: `/home/svc-admin/projects/brain`\n- Host: `svc-dev` (`192.168.4.123`)\n- API port: `8010`\n- MCP port: `8011`\n- Database name: `ai-brain`\n- Database user: `ai-brain`\n- Local default database password: `120dMOtVxpoHrwdT`\n\nBundled stack layout\n- `brain-api`: FastAPI REST service\n- `brain-mcp`: MCP server exposed over HTTP\n- `brain-db`: Postgres 16\n\nImportant project files\n- `/home/svc-admin/projects/brain/docker-compose.yml`\n- `/home/svc-admin/projects/brain/.env`\n- `/home/svc-admin/projects/brain/.env.example`\n- `/home/svc-admin/projects/brain/README.md`\n- `/home/svc-admin/projects/brain/backend/Dockerfile`\n- `/home/svc-admin/projects/brain/backend/requirements.txt`\n- `/home/svc-admin/projects/brain/backend/alembic.ini`\n- `/home/svc-admin/projects/brain/backend/alembic/env.py`\n- `/home/svc-admin/projects/brain/backend/alembic/versions/0001_initial.py`\n- `/home/svc-admin/projects/brain/backend/alembic/versions/0002_add_memory_source.py`\n- `/home/svc-admin/projects/brain/backend/app/main.py`\n- `/home/svc-admin/projects/brain/backend/app/db.py`\n- `/home/svc-admin/projects/brain/backend/app/models.py`\n- `/home/svc-admin/projects/brain/backend/app/schemas.py`\n- `/home/svc-admin/projects/brain/backend/app/crud.py`\n- `/home/svc-admin/projects/brain/backend/app/serializers.py`\n- `/home/svc-admin/projects/brain/backend/app/api/health.py`\n- `/home/svc-admin/projects/brain/backend/app/api/memories.py`\n- `/home/svc-admin/projects/brain/backend/app/api/tags.py`\n- `/home/svc-admin/projects/brain/backend/app/mcp_server.py`\n\nEnvironment and configuration\n- `.env` defines `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `DATABASE_URL`, `API_PORT`, and `MCP_PORT`.\n- Local/default `DATABASE_URL` points to the bundled Postgres service hostname `db`.\n- External Postgres support is implemented by overriding `DATABASE_URL` in `.env`, for example pointing it to a remote DB host instead of `db`.\n\nData model decisions\n- `Memory` table stores the current canonical record.\n- Fields include `key`, `title`, `content`, `source`, `status`, `created_at`, and `updated_at`.\n- `source` is indexed and defaults to `manual`.\n- `status` is indexed and defaults to `active`.\n- Tags are normalized via a dedicated `Tag` table and `memory_tags` join table.\n- Revision history is stored in a `Revision` table.\n- Deletes are implemented as archive semantics rather than hard delete at the API layer.\n\nSchema and migrations\n- Alembic migration `0001_initial.py` created the base schema for memories, tags, memory_tags, and revisions.\n- Alembic migration `0002_add_memory_source.py` added first-class `source` support to the `memories` table.\n- Container startup runs `alembic upgrade head` before launching the API process.\n- Early in development, the first prototype created tables before Alembic existed. To move to migration-owned schema, the project Postgres volume had to be reset once with `docker compose down -v`. After that reset, migrations became the source of truth and the schema stabilized.\n\nREST API implemented\n- `GET /health`\n- `GET /memories`\n- `GET /memories?status=all`\n- `GET /memories?q=<text>`\n- `GET /memories?tag=<tag>`\n- `GET /memories?source=<source>`\n- `GET /memories/{key}`\n- `POST /memories`\n- `PUT /memories/{key}`\n- `DELETE /memories/{key}` for archive behavior\n- `GET /tags`\n\nMCP server implementation\n- A dedicated MCP server service named `brain-mcp` was added to the compose stack.\n- The MCP endpoint is exposed at `http://192.168.4.123:8011/mcp`.\n- Implemented MCP tools:\n  - `list_memories`\n  - `retrieve_memory`\n  - `search_memories`\n  - `store_memory`\n  - `archive_memory`\n  - `list_tags`\n- The MCP layer uses the official Python `mcp` package.\n\nMCP host-header decision and fix\n- During early testing, the MCP endpoint returned host-header rejection / `421 Misdirected Request` behavior for requests using `192.168.4.123`.\n- The fix was to update `backend/app/mcp_server.py` so the FastMCP/transport security settings explicitly allow expected hosts.\n- Allowed host patterns were expanded to include loopback and LAN-reachable forms, including `127.0.0.1`, `localhost`, `192.168.4.123`, and `svc-dev` with port patterns.\n- After that fix, the endpoint behaved correctly for external access and clients could connect successfully.\n\nShared-brain design decision\n- The preferred architecture is one shared brain rather than one per AI client.\n- The separation model is implemented through `source` rather than separate deployments.\n- Example intended source values: `codex`, `claude`, `gemini`, `manual`, `contextkeep`.\n- This allows one source of truth while preserving per-client filtering and future namespace expansion.\n\nCurrent imported legacy data\n- Existing ContextKeep memories were imported into `brain`.\n- Import scope: all 54 existing ContextKeep memory records.\n- Imported records retained:\n  - original `key`\n  - original `title`\n  - full `content`\n  - tags\n  - original `created_at` and `updated_at` timestamps\n- Imported records were assigned `source=contextkeep`.\n- Imported records were loaded as current-state records only; synthetic revision history was not backfilled.\n\nHow ContextKeep import was implemented\n- The ContextKeep MCP transport became unreliable during bulk retrieval and began failing with transport-closed errors.\n- Instead of continuing through the MCP tool layer, the underlying ContextKeep storage implementation was inspected locally.\n- ContextKeep stores memories as JSON files under: `/home/svc-admin/ai-projects/projects/homelab/contextkeep/data/memories`\n- Each file contains fields like `key`, `title`, `content`, `tags`, `created_at`, and `updated_at`.\n- A direct export file was generated from those JSON documents.\n- The export file was copied to `svc-dev`.\n- A one-off Python import script was copied into the `brain-api` container and executed with `PYTHONPATH=/app` so it could import `app.db` and `app.models`.\n- The importer created missing tags, inserted memories, set `source=contextkeep`, and preserved the original timestamps.\n\nCommands and operational steps used during implementation\n- SSH / deployment inspection on `svc-dev`:\n  - `ssh svc-dev ...`\n- Compose / stack verification:\n  - `docker compose up -d`\n  - `docker compose down -v` (used once during early schema reset before Alembic became canonical)\n  - `docker ps`\n  - `docker exec ...`\n  - `docker cp ...`\n- API verification:\n  - `curl http://127.0.0.1:8010/health`\n  - `curl http://127.0.0.1:8010/memories`\n  - `curl http://127.0.0.1:8010/openapi.json`\n  - `curl http://127.0.0.1:8010/tags`\n- Database verification:\n  - `docker exec brain-db psql -U ai-brain -d ai-brain -c '\\dt'`\n  - `docker exec brain-db psql -U ai-brain -d ai-brain -c '\\d memories'`\n- MCP verification:\n  - MCP client tool listing against loopback inside the container\n  - direct HTTP checks against `http://192.168.4.123:8011/mcp`\n- Codex MCP registration:\n  - `codex mcp add brain --url http://192.168.4.123:8011/mcp`\n  - `codex mcp list`\n- Claude MCP registration:\n  - `claude mcp add --transport http brain http://192.168.4.123:8011/mcp`\n  - `claude mcp list`\n- Gemini MCP registration:\n  - `gemini mcp add --scope user --transport http brain http://192.168.4.123:8011/mcp`\n  - `gemini mcp list`\n- ContextKeep import prep:\n  - inspection of `/home/svc-admin/ai-projects/projects/homelab/contextkeep/server.py`\n  - inspection of `/home/svc-admin/ai-projects/projects/homelab/contextkeep/core/memory_manager.py`\n  - export generation from JSON files under `data/memories`\n  - `scp` to move import artifacts to `svc-dev`\n  - `docker cp` into `brain-api`\n  - `docker exec -e PYTHONPATH=/app brain-api python /tmp/contextkeep_import.py`\n\nClient integration status\n- Codex is configured to use the `brain` MCP server.\n- Claude is configured to use the `brain` MCP server and reported connected health.\n- Gemini is configured to use the `brain` MCP server and reported connected health.\n- All three are intended to use the same backend rather than separate installs.\n\nImportant implementation notes\n- Gemini initially accepted an MCP add command without explicit transport/scope, but that defaulted to an unusable project/stdio shape. The fix was to re-add it explicitly as `--scope user --transport http`.\n- Claude stored the server in its local project config under `/home/svc-admin/.claude.json`.\n- Gemini stored the server under `/home/svc-admin/.gemini/settings.json`.\n- Codex stored the MCP server registration in `/home/svc-admin/.codex/config.toml`.\n\nValidation completed\n- API health endpoint returned OK.\n- Memory CRUD and archive flow worked before import.\n- Search and filtering by text, tag, and source worked.\n- Tag listing worked.\n- MCP server tool listing worked.\n- Host-header restriction was fixed for the MCP endpoint.\n- After ContextKeep import, `GET /memories?status=all` returned 54 records.\n- After import, `GET /memories?source=contextkeep&status=all` returned 54 records.\n- Spot checks confirmed the expected tags and preserved timestamps on imported memories.\n\nCurrent state after implementation\n- `brain` is live on `svc-dev`.\n- REST API base URL: `http://192.168.4.123:8010`\n- MCP endpoint: `http://192.168.4.123:8011/mcp`\n- ContextKeep memories have been imported.\n- Shared-brain design with first-class `source` is in place.\n- Multiple AI clients are configured to use the same MCP endpoint.\n\nKnown limitations / future work\n- No web UI exists yet. Current interfaces are REST and MCP only.\n- `namespace` is not implemented yet; `source` is the current separation layer.\n- Imported ContextKeep records do not have reconstructed revision history.\n- Direct import tooling used for migration was ad hoc and not yet committed as a reusable migration command inside the project.\n- Prompt/template records and filtered views are still future work.\n\nRecommended next steps\n- Add `namespace` as a first-class field if a second layer of separation becomes necessary.\n- Build a web UI for browsing, filtering, and editing memories.\n- Turn the ContextKeep import path into a reusable project command or script if future migrations are expected.\n- Decide whether to disable or retire ContextKeep from daily use now that `brain` is active.\n- Add docs/examples showing how Codex, Claude, and Gemini should store memories using `source` consistently.\n\nThis memory is intended to function as the repo-adjacent implementation handoff for the initial `brain` build and migration work.","status":"active","namespace":"general","namespace_name":"general","namespace_tier":"shared","tags":[]}