replica-omnisciente/.aurelio/plans/2026-07-30_aurelio-consolidation-audit.md
Raphael Cautus (Maestro) 749432fefc feat(brain): garden, mirrors, vault-sync, provider registry consolidation
- .aurelio/garden/: model + agent garden (Google Cloud entries)
- .aurelio/mirrors/: sync-mirrors.yaml + state tracking
- .aurelio/skills/gcp/: Google Cloud skill
- Consolidation audit + execution plan (2026-07-30)
- vault-sync.py: Obsidian → GBrain MCP ingestion daemon
- brain-to-gbrain.py: brain → GBrain migration tool
- Provider registry + dist mirrors updated
- .gitignore: exclude .runner, .mimocode/.cron-lock, drift/target

Co-authored-by: Álvaro de Campos <campos@portugalfuturista.org>
2026-07-31 14:57:24 +01:00

12 KiB
Raw Blame History

Aurelio Consolidation Audit — 30/07/2026

Pre-Hetzner migration. Full state analysis of replica-omnisciente + service topology truth.


1. REPO: replica-omnisciente

5.7GB on disk | .git 618MB (533MB packs) | 7,368 tracked files

1.1 Real Code Projects (keep, maintain)

Project Lang Disk (src) Purpose
dirac/ TS 21M VS Code coding agent (esbuild)
tilth/ Rust 768K Code intelligence MCP (tree-sitter)
toon/ TS 2.1M Toon format library (pnpm monorepo)
shared/ TS 12M Protobuf shared lib (@aurelio/shared)
aurelio-theia/ TS 3.6M IDE monorepo [submodule → Forgejo]
aurelio-gbrain/ TS/Py ~3M GBrain knowledge brain [submodule → Forgejo]
mycelium-rd/ Rust ~1M XSD→codegen research
telminal/ Py 376K Terminal-in-Telegram bot
infra/cdp-bridge/ TS 164K CDP bridge
infra/lab-gateway/ Py ~1M Hardware lab gateway (ESP32/PPK2/Icicle)
infrastructure/fabric/drift/ Rust ~100K Drift agent (SQLite store, plan parser)
dashboards/media-explorer/ TS 952K Media explorer dashboard

1.2 Git Bloat — BINARY BLOBS TRACKED IN HISTORY (618MB .git)

These are tracked files bloating the git pack (533MB):

Path Size What
infra/lab-gateway/MPFS_ICICLE_KIT_2025_07/ ~100M 27 PolarFire SoC FPGA job/bitstream files
realms/tear-de-silicio/standard-app/models/ ~80M FPGA bitstreams + VNNX ML models
.aurelio/knowledge/*.pdf ~32M 4 research PDFs (airships, final reports)
.aurelio/knowledge/session*/ ~16M Session artifacts with screenshots/binary

Fix: git filter-repo to purge binaries → move to LFS or external artifact store. Expected .git reduction: ~250-350MB.

1.3 Local-Only Bloat (gitignored, not in repo but wasting disk)

Path Size Fix
tilth/target/ 3.4G cargo clean — rebuild on demand
mycelium-rd/target/ 345M cargo clean
infrastructure/fabric/drift/target/ 570M cargo clean
aurelio-gbrain/src/assets/ 61M submodule assets, not tracked (ok)
.venv-telminal/ 53M gitignored (ok)

1.4 Brain Sessions — 1,090 dirs, 73MB, uncontrolled growth

Source Count Status
kimi 601 Synchronized from ~/.kimi/sessions/
opencode 178 Synchronized
claude-code 129 Synchronized (+ subagents)
mimo (mo/mp/mq) 125 Synchronized
antigravity 13 Synchronized
hermes 12 Synchronized
uuid (no prefix) 25 Legacy/old format
pi/mimocode 7 Synchronized

268 are untracked (never committed to git). The brain is growing ~50-100 sessions/week.

Fix: Brain sessions should NOT be in git at all. They're runtime data. Add .aurelio/brain/session-*/ to .gitignore. The GBrain PostgreSQL instance on CT223 is the real persistent store — file-based sessions are redundant once imported.

1.5 Stale/Dead Documentation

File Dated Problem
DIRECTORY_GUIDE.md 2026-04-16 Describes root-level memory/, identity/, swarm/ — all moved to .aurelio/. Lists wrong realms.
AGENT_INDEX.md 2025-07-07 Says CT208 is hub (stopped). 63 projects listed, half don't exist. Wrong IPs.
README.md 2026-07-06 Generic, doesn't match actual structure
DESIGN.md 2026-07-06 648 bytes — likely empty stub
SETUP_API_KEYS.md 2026-07-06 Probably stale API key instructions
task.md 2026-07-06 Stale task file
vscode_extension_migration_guide.md 2026-07-06 One-off migration doc, no longer needed
IMPLEMENTATION_SUMMARY.md 2026-07-11 Portal consolidation notes
PORTAL_CONSOLIDATION_SUMMARY.md 2026-07-11 Same topic
NEXT_MONTH_ROADMAP.md 2026-07-11 August 2026 roadmap — expired

1.6 Empty/Stub Realms (23 dirs, most 8KB empty)

Real content (3): nervura-electrica (672K), universalisos (308K), silent-shared-office (32K)

Empty stubs (20): aprendiz-de-sensacoes, matriz-da-maquina, ode-electromagnetica, orquestra-de-hardware, alquimista, etc.

Fix: Delete empty stubs. Move realms/ with real content to top-level project docs or archive.


2. SERVICE TOPOLOGY — TRUTH vs DOCUMENTATION

2.1 What the 07-16 Audit Says (WRONG)

CT208 (192.168.0.125) = ALL Aurelio services
  gbrain :18001, gabinete :18000, bot-rs, 7 MCP servers, 6 kimi swarm agents
ASUS = fabricd :8081, orchestrator :8200

2.2 What Memory Says (CURRENT REALITY)

CT208 = STOPPED (old hub, decommissioned)
CT223 (Gigabyte, dir-rootfs /mnt/disks/usb-sdf) = NEW HUB
  ├── Gabinete :18000
  ├── Bot-rs (Telegram @Aurélio Omnisciente)
  ├── GBrain :18001
  ├── fabricd :8081
  └── Engineering MCPs
CT217 = Hermes gateway 24/7 (Signal/Telegram/Matrix, E2EE)
CT235 = Orchestrator :8200
CT237 (Gigabyte) = Forgejo :3001 (code.portugalfuturista.org), Woodpecker :8001

2.3 MCP Config — 16 configured servers, all npx (wrong for production)

.aurelio/mcp_config.json lists 16 MCP servers using npx commands. But AGENTS.md says they're "remote (mcp-remote to *.portugalfuturista.org)". The config doesn't match reality — these are either:

  • Run as systemd services on CT223 (sourcing :8081, eda :8082, knowledge :8083, codebase-memory :8084)
  • Or dead/never-deployed (civil, artist, mechanic, StitchMCP, dart-mcp-server)

2.4 Local MCP servers on disk (.aurelio/mcp/)

Server Size Status
savearth-mcp/ 84K Python — active (Savearth fleet)
savearth-workspace/ 60K Python — active
electrical-mcp/ 720K TS — active
engineering-mcp/ 340K TS — active
flint-chart-mcp/ 156K TS — purpose unclear
image-pipe-mcp/ 100K TS — purpose unclear
aurelio-tools-mcp/ 92K TS — purpose unclear

2.5 Infrastructure Topology (3 PVE hosts)

PROXMOX CLUSTER "portugalfuturi"
│
├── ASUS (192.168.0.38) — 31Gi, PVE 9.2.2
│   ├── CT215: pf-matrix (Matrix Synapse :matrix.portugalfuturista.org)
│   ├── CT208: OLD HUB — STOPPED
│   ├── Cloudflare tunnel pf-tunnel-v2
│   └── Other CTs (202-220)
│
├── GIGABYTE (192.168.0.104) — 46GB, 2x RTX 3070
│   ├── CT223: NEW HUB (Gabinete, Bot-rs, GBrain, fabricd, MCPs)
│   ├── CT237: Forgejo :3001 + Woodpecker :8001
│   ├── CT216: multi-scrobbler (Subsonic)
│   ├── Docker media stack (Jellyfin, Sonarr, Radarr, etc.)
│   ├── signal-api container :18088
│   └── Monitoring (Prometheus + Grafana + Loki + Blackbox)
│
└── LATTEPANDA (192.168.0.40)
    ├── MinIO :9000/:9001 (S3 / Terraform state)
    ├── LocalStack v3 :4566 (AWS dev)
    └── Azurite :10000-10002 (Azure dev)

OFF-LAN:
├── CT217 (pf-hermes): Hermes gateway 24/7
├── CT235: Orchestrator :8200
├── VPS pf-edge (94.130.24.119): WG endpoint
└── CTs 213/231/234/235/236: Savearth fleet

3. HETZNER MIGRATION — TARGET ARCHITECTURE

3.1 What Should Move to Hetzner (cloud-facing services)

These services benefit from public IP, low latency to external APIs, and 24/7 uptime without LAN dependency:

Service Current Hetzner Target Why
Hermes gateway CT217 Hetzner VM 24/7 messaging, no LAN dependency
GBrain (PostgreSQL) CT223 Hetzner VM (with pg backup) Central knowledge store, API-facing
Gabinete API CT223 Hetzner VM Central API, bot backend
Bot-rs (Telegram) CT223 Hetzner VM Telegram webhook needs public IP
Forgejo + CI CT237 Hetzner VM Git push/pull latency, public access
Cloudflare tunnel endpoint ASUS Cloudflare→Hetzner Remove single LAN point of failure

3.2 What Should Stay on LAN (hardware-dependent, latency-sensitive)

Service Host Why
Hardware lab gateway ASUS/CT206 Physical ESP32/PPK2/Icicle connected
Media stack Gigabyte Local storage, GPU transcoding
Signal-API Gigabyte Phone registration tied to LAN
Cloud emulators Lattepanda Dev-only, LAN-local
Savearth fleet CTs Various Hardware-attached (ESP32/PPK2)
Proxmox cluster itself All 3 hosts Bare metal

3.3 Service Separation Principle

Current problem: CT223 is a god-container running everything. Hetzner migration should split:

HETZNER (public cloud)
├── VM-1: aurelio-core (Gabinete + Bot-rs + GBrain)
├── VM-2: aurelio-forge (Forgejo + Woodpecker CI)
├── VM-3: aurelio-gateway (Hermes + Cloudflare tunnel endpoint)
└── VM-4: aurelio-monitoring (Grafana/Prometheus mirror)

LAN (hardware-locked)
├── ASUS: lab hardware, local CTs
├── Gigabyte: media, signal, CT223→demoted
├── Lattepanda: dev emulators
└── Savearth fleet: hardware testing

VPN: WireGuard mesh Hetzner ↔ LAN (existing pf-edge WG)

4. REPO CLEANUP ACTIONS

4.1 Immediate (low risk)

  1. cargo clean all Rust projects — frees 4.3GB local disk

    cd tilth && cargo clean
    cd mycelium-rd && cargo clean
    cd infrastructure/fabric/drift && cargo clean
    
  2. Delete empty realm stubs (20 dirs × 8KB)

    realms/aprendiz-de-sensacoes, matriz-da-maquina, ode-electromagnetica, ...
    
  3. Delete stale root docs

    DESIGN.md, task.md, SETUP_API_KEYS.md, vscode_extension_migration_guide.md,
    IMPLEMENTATION_SUMMARY.md, PORTAL_CONSOLIDATION_SUMMARY.md, NEXT_MONTH_ROADMAP.md
    
  4. Add brain sessions to .gitignore

    .aurelio/brain/session-*/
    

4.2 Medium (requires git history rewrite)

  1. Purge binary blobs from git — saves ~250MB from .git
    git filter-repo --path infra/lab-gateway/MPFS_ICICLE_KIT_2025_07 --invert-paths
    git filter-repo --path realms/tear-de-silicio/standard-app/models --invert-paths
    git filter-repo --path-glob '.aurelio/knowledge/*.pdf' --invert-paths
    
    Move to: MinIO bucket or Git LFS (if Forgejo LFS is enabled — currently NOT per AGENTS.md)

4.3 Documentation Rewrite

  1. Rewrite DIRECTORY_GUIDE.md to reflect .aurelio/ structure (not root-level)
  2. Rewrite AGENT_INDEX.md with real CT topology (CT223 hub, not CT208)
  3. Update AGENTS.md infra targets table to match memory reality
  4. Delete or archive the 07-16 audit (superseded by this document)

5. SERVICE CONSOLIDATION ACTIONS

5.1 MCP Server Cleanup

16 MCP servers configured but reality is unclear. Need to:

  1. SSH to CT223, enumerate actual systemd services
  2. Compare against .aurelio/mcp_config.json
  3. Remove dead MCP entries (civil, artist, mechanic, StitchMCP, dart-mcp-server)
  4. Consolidate overlapping: electrical-mcp vs engineering-mcp vs electrical-eda-mcp vs electrical-sourcing-mcp — 4 servers for electrical?

5.2 Kimi Swarm Decommission?

6 Kimi swarm agents on old CT208 (orchestrator, log-processor, ota-validator, regression-detector, report-synthesizer, dashboard). CT208 is stopped. Are these:

  • Still needed?
  • Migrated to CT223?
  • Dead and should be removed from repo (.aurelio/swarm/)?

5.3 Lifestream Pipeline Verification

Saved Messages → Lifestream userbot → SQLite → Cron 03:00 → GBrain

This pipeline depends on CT223. Verify it still works after migration.


6. OPEN QUESTIONS (need your input)

  1. Hetzner specs — What VM(s) did you provision? CPU/RAM/disk? Dedicated or shared?
  2. CT223 future — After Hetzner migration, does CT223 stay as LAN fallback or get repurposed?
  3. Git remote strategy — Keep GitHub (origin) + Forgejo? Or consolidate to Forgejo-only now that Hetzner gives public access?
  4. Brain persistence — GBrain PostgreSQL on Hetzner means migrating 1362+ pages. Acceptable? Or keep DB on LAN with Hetzner as read-replica?
  5. Kimi swarm — Dead or alive?