feat(realm): onboard iot-backend realm from savearth-iot-infrastructure

2026-07-04 18:22:30 +01:00 · 2026-07-04 18:22:30 +01:00 · c4e293d874
commit c4e293d874
parent 82c0132807
3 changed files with 339 additions and 8 deletions
--- a/realms/iot-backend/.aurelio/config.toml
+++ b/realms/iot-backend/.aurelio/config.toml
@ -0,0 +1,8 @@
+[identity]
+name = "savearth IoT Backend"
+realm = "iot-backend"
+
+[sync]
+endpoint = "https://mcp.portugalfuturista.org"
+interval_seconds = 300
+auto_push = true
--- a/realms/iot-backend/.aurelio/memory/index.md
+++ b/realms/iot-backend/.aurelio/memory/index.md
@ -0,0 +1,158 @@
+# Realm: IoT Backend (Ode Invisível)
+
+## Identity
+
+**IoT Backend** (repository: `savearth-iot-infrastructure`) is the AWS serverless infrastructure that receives, routes, stores, and provisions savearth shower-monitoring devices. It is the cloud counterpart to the `smart-device-firmware` realm.
+
+This realm is the canonical central-brain index for the `savearth-iot-infrastructure` workspace.
+
+---
+
+## Technology Stack
+
+| Component | Technology |
+|-----------|------------|
+| Cloud Provider | AWS (`eu-north-1`) |
+| IaC | CloudFormation (nested stacks, YAML) |
+| Compute | AWS Lambda (Python 3.12) |
+| Messaging | AWS IoT Core MQTT + IoT Rules |
+| Time-Series | InfluxDB (self-hosted on EC2) |
+| Config/State | DynamoDB |
+| Object Storage | S3 (firmware binaries + manifests) |
+| Secrets | AWS Secrets Manager |
+| Scheduling | Amazon EventBridge |
+| API | API Gateway v2 (HTTP) + Lambda Function URLs |
+| Access | SSM Session Manager (EC2), IAM roles |
+
+---
+
+## Architecture
+
+### Root Stack
+
+`infrastructure/main.yaml` orchestrates 11 nested stacks:
+
+| Nested Stack | Purpose |
+|--------------|---------|
+| `iam` | Cross-service IAM roles and policies |
+| `s3` | Firmware artifact bucket |
+| `dynamodb` | Device config, logs, OTA, allowlist tables |
+| `ota` | OTA job resources and tracking |
+| `ec2` | Self-hosted InfluxDB instance |
+| `lambda` | All business-logic Lambda functions |
+| `iot` | IoT Rules, policies, Thing Groups |
+| `event-bridge` | Scheduled and event-driven triggers |
+| `fleet-provisioning` | Fleet provisioning template + hook |
+| `monitoring` | CloudWatch alarms, SNS notifications |
+| `vpc-endpoints` | VPC endpoints for cost-optimized profile |
+
+### Active Lambda Functions
+
+| Function | Trigger | Role |
+|----------|---------|------|
+| `ProvisioningFunction` | IoT Rule `device/+/status` | Registers Thing, attaches cert, checks OTA, publishes config |
+| `TelemetryFunction` | IoT Rule `device/+/data` | Legacy telemetry processor |
+| `InfluxDBWriterFunction` | IoT Rule `device/+/data` | Primary InfluxDB line-protocol writer |
+| `LogIngestionFunction` | IoT Rule `device/+/logs` | Writes batched logs to DynamoDB `DeviceLogs-v3` |
+| `LogQueryFunction` | Lambda Function URL | Web dashboard log queries |
+| `BackendLogQueryFunction` | API Gateway `GET /logs` | Backend-team log API |
+| `ConfigManagerFunction` | IoT Rule `device/+/config/request` + API Gateway | Reads/writes device config in DynamoDB |
+| `SyncDeviceConfigsFunction` | EventBridge every 5 min | Polls Railway backend, syncs config to DynamoDB |
+| `OTAMonitorFunction` | EventBridge IoT job events | Tracks OTA job status |
+| `ShadowSyncFunction` | (Defined) | Syncs device shadow state |
+| `FleetProvisioningHookFunction` | Fleet provisioning template | Validates device against allowlist |
+| `CertGenFunction` | CloudFormation custom resource | Generates claim certificate |
+
+### Active IoT Rules
+
+| Rule | Topic | Target | Status |
+|------|-------|--------|--------|
+| `ProvisioningRule_puv2lKAwQoRg` | `device/+/status` | `ProvisioningFunction` | ✅ ACTIVE |
+| `ProvisioningRule` | `device/+/status` | `ProvisioningFunction` | ✅ ACTIVE (passes `certificateArn`) |
+| `TelemetryRule_PA4Xgg86MIky` | `device/+/data` | `TelemetryFunction` | ✅ ACTIVE |
+| `InfluxDBVolumeRule_nPLeCdRUlofY` | `device/+/data` | `InfluxDBWriterFunction` | ✅ ACTIVE (primary writer) |
+| `LogIngestionRule_jJ0FeyefFhRd` | `device/+/logs` | `LogIngestionFunction` | ✅ ACTIVE → `DeviceLogs-v3` |
+| `ConfigRequestRule` | `device/+/config/request` | `ConfigManagerFunction` | ✅ ACTIVE |
+
+### DynamoDB Tables
+
+| Table | Key Schema | Purpose |
+|-------|-----------|---------|
+| `DeviceConfiguration-${Environment}` | `device_id` (HASH) | Device settings, thresholds, WiFi, ML params |
+| `FirmwareVersions-${Environment}` | `device_id` (HASH), `timestamp` (RANGE) | Firmware version history |
+| `OTAJobs-${Environment}` | `job_id` (HASH), GSI `device_id` | OTA job tracking |
+| `DeviceAllowlist-${Environment}` | `serial_number` (HASH) | Fleet provisioning validation |
+| `DeviceLogs-v3` | `device_id` (HASH), `timestamp` (RANGE) | Active device log table (24h TTL) |
+
+> **Always use `DeviceLogs-v3`.** `DeviceLogs` is legacy/stale and pending deletion.
+
+### Authentication Model
+
+- Devices use a **shared claim certificate** embedded in firmware (`components/certs/certs/certificate.pem.crt`).
+- MQTT client ID = device MAC address (e.g. `dc:b4:d9:01:58:38`).
+- `IoTClaimPolicy` scopes topics via `${iot:ClientId}`.
+- No per-device certificates in firmware; provisioning attaches the shared cert to a Thing on first status publish.
+
+---
+
+## Key Operational Behaviors
+
+1. **Device Lifecycle**  
+   Device boots → connects with claim cert → publishes `device/{mac}/status` → `ProvisioningFunction` creates Thing, attaches cert, checks pending OTA, publishes config on `device/{mac}/config`.
+
+2. **Telemetry Ingestion**  
+   `device/{mac}/data` messages are processed by both `TelemetryFunction` and `InfluxDBWriterFunction`; the latter is the primary InfluxDB writer.
+
+3. **Log Batching**  
+   Firmware batches ~256 log entries in NVS and sends them to `device/{mac}/logs`. `LogIngestionFunction` writes to `DeviceLogs-v3`. There can be a 5–30 minute delay before logs appear in DynamoDB.
+
+4. **Configuration Sync**  
+   `SyncDeviceConfigsFunction` polls the Railway backend every 5 minutes and writes effective config into `DeviceConfiguration-${Environment}`. Devices request config via `device/{mac}/config/request`.
+
+5. **OTA**  
+   Firmware binaries and manifests live in S3. OTA jobs are tracked in `OTAJobs-${Environment}`. `OTAMonitorFunction` listens to EventBridge job events.
+
+---
+
+## Development Commands
+
+```bash
+# Deploy full stack
+cd infrastructure
+./deploy.sh <artifact-bucket> [keypair] [vpc] [subnet] [email]
+
+# Package only
+aws cloudformation package \
+  --template-file main.yaml \
+  --s3-bucket <artifact-bucket> \
+  --output-template-file packaged.yaml
+
+# Local SAM testing
+sam local invoke <FunctionName> -e events/<event>.json
+
+# Fetch device logs
+python3 tools/fetch_dynamo_logs.py <mac> --time 1h --filter FSM
+
+# Fleet health check
+python3 tools/device_diagnostics.py --summary-only
+```
+
+---
+
+## Deployment Profiles
+
+| Profile | DB | Network | Best For |
+|---------|----|---------|----------|
+| A1 | InfluxDB Cloud | Public | Rapid prototyping |
+| A2 | Amazon Timestream for InfluxDB | Public | AWS-native / compliance |
+| A3 | InfluxDB on EC2 | NAT Gateway | Flexibility |
+| B | InfluxDB on EC2 | VPC Endpoints only | Cost-optimized production |
+
+---
+
+## Inter-Realm Dependencies
+
+- **`smart-device-firmware`** — ESP32-S3 device firmware; source of MQTT messages and OTA consumers.
+- **`flow-meter-pcb`** — Hardware designs whose sensor and peripheral choices drive telemetry fields and config schema.
+- **`guarda-livros-da-ode`** — Backend business API that owns device/hotel configuration rules.
+- **`nervura-electrica`** — Hosts the Proxmox HIL lab, MQTT broker, and InfluxDB infrastructure.
--- a/realms/iot-backend/AGENTS.md
+++ b/realms/iot-backend/AGENTS.md
@ -1,11 +1,176 @@
-# AGENTS.md — IoT Backend
+<!-- This realm corresponds to the `savearth-iot-infrastructure` repository. -->
+<!-- The canonical central brain project/realm name is `iot-backend`. -->

-## Identity
-You are operating inside the **IoT Backend** realm. This is currently a **placeholder realm** in the Hybrid Memory Engine, awaiting formal onboarding and index generation.
+# Agent Guide: savearth — IoT Backend Repository

-## Status
- No `memory/index.md` has been synthesized yet.
- All existing files in this realm are empty placeholders.
+## Current Status

-## Directive
-If you are asked to operate on a project that maps to this realm, execute the **Auto-Study Imperative** from the root `.agent/AGENTS.md` immediately. Generate the missing `memory/index.md` and update this `AGENTS.md` with stack-specific guidance before proceeding with any code changes.
+**Last Updated:** 2026-07-04  
+**Status:** Realm formally onboarded into the Hybrid Memory Engine. Active production stack is `savearth-iot-minimal-v3` in `eu-north-1`.  
+**Repository:** `SavearthTech/savearth-iot-infrastructure`
+
+**Recent additions (last 8 weeks):**
+- `BackendLogQueryFunction` + API Gateway `GET /logs` for backend-team log retrieval.
+- `ConfigManagerFunction` updated to accept `device_id` query string for configuration updates.
+- Cloudflare Tunnel setup guide for secure remote access.
+- `.agent/` converted to `replica-omnisciente` submodule.
+
+---
+
+## Architecture Overview
+
+### Purpose
+This repository houses the **AWS serverless backend** for savearth shower-monitoring devices: device provisioning, telemetry ingestion, log storage, configuration management, and OTA firmware distribution.
+
+### Region & Stack
+- **AWS Region:** `eu-north-1` (Stockholm)
+- **Active production stack:** `savearth-iot-minimal-v3`
+- **Legacy stacks:** `savearth-iot-minimal`, `savearth-log-ingestion` (pending deletion)
+- **Deployment model:** CloudFormation nested stacks (root `infrastructure/main.yaml`)
+
+### Core Services
+| Service | AWS Product | Role |
+|---------|-------------|------|
+| MQTT Broker | AWS IoT Core | Device connectivity and topic routing |
+| Business Logic | AWS Lambda (Python 3.12) | Provisioning, telemetry, logs, config, OTA |
+| Time-Series | InfluxDB on EC2 | Primary telemetry store |
+| Config/State | DynamoDB | Device configuration, OTA jobs, allowlist, logs |
+| Artifacts | S3 | Firmware binaries and manifests |
+| Secrets | Secrets Manager | Claim cert, InfluxDB credentials |
+| Scheduling | EventBridge | Config sync, OTA monitoring |
+| API | API Gateway v2 / Lambda Function URLs | Backend and dashboard APIs |
+
+### Authentication Model
+- Devices authenticate with a **shared claim certificate** embedded in firmware.
+- MQTT client ID = device MAC address (e.g. `dc:b4:d9:01:58:38`).
+- `IoTClaimPolicy` uses `${iot:ClientId}` for topic scoping.
+- **No per-device certificates** in firmware; the provisioning Lambda attaches the shared cert to a Thing on first `device/+/status` publish.
+
+---
+
+## Active IoT Rules
+
+| Rule | Topic | Lambda | Notes |
+|------|-------|--------|-------|
+| `ProvisioningRule_puv2lKAwQoRg` | `device/+/status` | `ProvisioningFunction` | ✅ ACTIVE |
+| `ProvisioningRule` | `device/+/status` | `ProvisioningFunction` | ✅ ACTIVE (passes `certificateArn`) |
+| `TelemetryRule_PA4Xgg86MIky` | `device/+/data` | `TelemetryFunction` | ✅ ACTIVE (legacy) |
+| `InfluxDBVolumeRule_nPLeCdRUlofY` | `device/+/data` | `InfluxDBWriterFunction` | ✅ ACTIVE (primary writer) |
+| `LogIngestionRule_jJ0FeyefFhRd` | `device/+/logs` | `LogIngestionFunction` | ✅ ACTIVE → `DeviceLogs-v3` |
+| `ConfigRequestRule` | `device/+/config/request` | `ConfigManagerFunction` | ✅ ACTIVE |
+
+Disabled rules: `InfluxDBTelemetryRule`, `TelemetryRule` (old Lambda targets), `DeviceLogIngestionRule` (duplicate — consolidated to DeviceLogs-v3).
+
+---
+
+## DynamoDB Tables
+
+- **`DeviceLogs-v3`** — Active log table (written by v3 LogIngestion Lambda). **Always use this table.**
+- **`DeviceLogs`** — Legacy duplicate. Rule disabled. Pending deletion.
+- **`DeviceConfiguration-${Environment}`** — Device settings, thresholds, WiFi, ML params.
+- **`FirmwareVersions-${Environment}`** / `FirmwareVersions` — Firmware version history.
+- **`OTAJobs-${Environment}`** / `OTAJobs` — OTA job tracking.
+- **`DeviceAllowlist-${Environment}`** / `DeviceAllowlist` — Fleet provisioning validation.
+
+---
+
+## Lambda Functions
+
+| Function | Code Path | Trigger |
+|----------|-----------|---------|
+| `ProvisioningFunction` | `functions/provisioning/` | IoT Rule `device/+/status` |
+| `TelemetryFunction` | `functions/telemetry/` | IoT Rule `device/+/data` |
+| `InfluxDBWriterFunction` | `functions/influxdb_writer/` | IoT Rule `device/+/data` |
+| `LogIngestionFunction` | `functions/log_ingestion/` | IoT Rule `device/+/logs` |
+| `LogQueryFunction` | `functions/log_query/` | Lambda Function URL |
+| `BackendLogQueryFunction` | `functions/backend_log_query/` | API Gateway `GET /logs` |
+| `ConfigManagerFunction` | `functions/config_manager/` | IoT Rule + API Gateway |
+| `SyncDeviceConfigsFunction` | `functions/sync_device_configs/` | EventBridge every 5 min |
+| `OTAMonitorFunction` | `functions/ota_monitor/` | EventBridge IoT job events |
+| `ShadowSyncFunction` | `functions/shadow_sync/` | (Defined) |
+| `FleetProvisioningHookFunction` | `functions/fleet_provisioning_hook/` | Fleet provisioning template |
+| `CertGenFunction` | `functions/cert_gen/` | CloudFormation custom resource |
+
+---
+
+## Agent Operational Directives
+
+### File Handling
+- CloudFormation templates are YAML; use care with indentation and intrinsic functions (`!Ref`, `!Sub`, `!GetAtt`).
+- Lambda code is Python 3.12. Follow PEP 8 and conventional commits.
+- `infrastructure/packaged.yaml` is generated by `aws cloudformation package` — do not hand-edit.
+
+### Code Changes
+- When changing topic rules, update both `infrastructure/services/iot/template.yaml` and the firmware topic constants.
+- When adding DynamoDB tables, update all Lambda environment variables and IAM policies in the nested stacks.
+- Keep timestamp normalization logic consistent with `normalize_timestamp_ms()` in the telemetry functions.
+
+### Deployment
+- Use `infrastructure/deploy.sh` for full deployments.
+- Test Lambda changes locally with `sam local invoke` using events from `infrastructure/events/`.
+- The VPC Lambda profile (Telemetry/InfluxDBWriter) cannot reach Secrets Manager without NAT Gateway; credentials must come from environment variables in that profile.
+
+### Cross-Repository Coordination
+- Topic schema changes must be mirrored in `aws-iot-core-poc` firmware (`main/tasks/aws_iot_task.c`).
+- Config schema changes affect both `ConfigManagerFunction` and firmware config parsing.
+- OTA manifests must align with firmware partition tables and version reporting.
+
+---
+
+## Diagnostic Commands
+
+```bash
+# Browse DynamoDB logs (uses DeviceLogs-v3)
+python3 tools/fetch_dynamo_logs.py dc:b4:d9:01:58:38 --time 1h --filter FSM
+
+# Fleet health check
+python3 tools/device_diagnostics.py --summary-only
+
+# Check device certificate status
+aws iot list-thing-principals --thing-name "dc:b4:d9:00:f6:90"
+
+# InfluxDB access (requires SSM tunnel first)
+aws ssm start-session --target i-029a65da01d618f5b --region eu-north-1 \
+    --document-name AWS-StartPortForwardingSession \
+    --parameters '{"portNumber":["8086"],"localPortNumber":["8086"]}'
+```
+
+---
+
+## Important Notes
+
+- **Log batching delay:** Firmware batches logs in NVS flash (~256 entries/batch) and sends asynchronously. There can be a 5-30 minute delay between device activity and DynamoDB log availability.
+- **DynamoDB table:** Always use `DeviceLogs-v3`, never `DeviceLogs` (legacy/stale).
+- **Active stack:** `savearth-iot-minimal-v3`.
+- **Region:** All resources are in `eu-north-1` unless explicitly noted.
+
+---
+
+## Terminal Output Capture Rule
+
+**MANDATORY:** Follow the same `.agentlog` pattern as other savearth repositories.
+
+```bash
+# CORRECT:
+git status --short > /tmp/iot_backend_status.agentlog 2>&1
+# Then read with view_file
+```
+
+---
+
+## Git Commit Rules
+
+Use conventional commit format with infrastructure scope:
+```
+feat(lambda): add shadow sync function for device state
+fix(iot): correct SQL query for certificateArn
+chore(deploy): update nested stack outputs
+docs(infra): add Cloudflare tunnel setup guide
+```
+
+---
+
+## Related Realms
+
+- **`smart-device-firmware`** — ESP32-S3 firmware that consumes/produces this backend's topics.
+- **`flow-meter-pcb`** — Hardware design; sensor and peripheral choices drive telemetry and config schema.