c5753e1c57
critic PR review 揭示已 push commits 的 7 個 blocker，本 commit 全部修復。
## C1 + C2 + M1 + M2 + M3 — KMWriter 真正統一契約（critic 最嚴重 5 條）
### C1 km_writer.py:194 — backfill 自打臉修
- 裸 asyncio.create_task(_backfill_path_a_approval) → await _backfill_path_a_approval_safe()
- 同步 await + 獨立 DLQ km:backfill:dlq + try/except 不阻塞主寫入
- 新增 km_backfill_reconciler_job.py（每 5 分鐘掃 DLQ）+ ENABLE_KM_BACKFILL_RECONCILER flag
- 防 Path B 比 Path A 先完成 → related_approval_id 永遠 NULL 的 race
### C2 km_writer.py:391 — KM_WRITE_AWAIT=false 路徑收緊
- 從 ensure_future（fire-and-forget 比舊版同步寫更糟）
- 改 await writer.write(retry=1, timeout=2.0)（仍 await 但只試一次、超時短）
- docstring 明確標註「緊急回滾用，不保證可靠性」
### M1 decision_manager.py:2178/2203 — 移除 _fire_and_forget 旁路
- 兩處 _fire_and_forget(executor.write_execution_result_to_km(...))
- 改 await asyncio.shield(...) + BaseException 保護（防上層 cancel 中斷）
- KM_WRITE_AWAIT=true 在這條路徑終於真正 await
### M2 incident_service.py:1099 — 自製 path 加 retry+DLQ
- 原本 if settings.KM_WRITE_AWAIT: await asyncio.wait_for else create_task
- 改 3 次指數退避 retry + DLQ 保護（呼叫 km_writer 私有 helper）
### M3 km_writer.py:166 — 冪等聲明對齊實作
- knowledge_repository.create() 加 UPSERT 路徑（pg_insert ON CONFLICT DO UPDATE）
- KnowledgeEntryCreate / KnowledgeEntryRecord 加 path_type 欄位
- migration: ADD COLUMN path_type + partial unique index uix_knowledge_incident_path
## M4 alertmanager.yml — equal: [] 收緊（critic 防爆炸抑制）
- OllamaInstanceDown / KMConverterDown 抑制加 equal: ['cluster'] 約束
- 防多 cluster 場景下任一 Ollama down 誤抑全 AI/SLO 告警
## M5 Alertmanager 版本驗證（已確認 v0.31.1，遠超 v0.22+）
## M6 governance_agent.py — health score 區分 skipped vs ok vs violated
- check_slo_compliance 加 _meta {violated_count, skipped_count, ok_count, all_skipped, status}
- run_self_check: SLO 全 skipped 時獨立發 governance_slo_data_gap 告警
（不污染 self_failure 計數，因為 no_data 是 emitter 未實作不是治理機制故障）
## M7 scripts/check_config_drift.py — 改 AST 解析
- regex 改 ast.parse 找 Settings ClassDef AnnAssign Field(default=...)
- 避免多行 list / default_factory= / 含跳行字串的 false negative
- 4 欄位（AI_FALLBACK_ORDER / ARGOCD_URL / PROMETHEUS_URL / OLLAMA_URL）全對齊
## 新增測試
- test_km_writer_backfill_reconciler.py: 7 cases（C1 reconciler + safe helper）
- test_km_writer_idempotent.py: 5 cases（M3 path_type 注入 + UPSERT 分支）
## 驗證
- 1585 unit tests 全綠（+13 從 1572）
- amtool check-config SUCCESS（8 inhibit_rules / 2 receivers）
- drift checker AST-based 4 欄位全對齊
- Alertmanager v0.31.1 確認支援新語法
## 期望影響
- KMWriter 名實統一：飛輪閉環 KM 寫入路徑 100% 可靠
- M4 抑制爆炸風險解除
- 治理層不再對 SLO no_data 靜默
- drift checker false negative 風險解除
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
█████╗ ██╗ ██╗ ██████╗ ██████╗ ██████╗ ██╗
██╔══██╗██║ ██║██╔═══██╗██╔═══██╗██╔═══██╗██║
███████║██║ █╗ ██║██║ ██║██║ ██║██║ ██║██║
██╔══██║██║███╗██║██║ ██║██║ ██║██║ ██║██║
██║ ██║╚███╔███╔╝╚██████╔╝╚██████╔╝╚██████╔╝██║
╚═╝ ╚═╝ ╚══╝╚══╝ ╚═════╝ ╚═════╝ ╚═════╝ ╚═╝
Zero-Touch Ops. Human-Centric Decisions.
AI-Powered Intelligent Operations Platform
The Future of Operations is Here
When your system breaks at 3 AM, AWOOOI doesn't just alert you—it analyzes the blast radius, calculates how much money you're burning, and presents a one-click fix. You approve. It executes. You go back to sleep.
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
│ ALERT: frontend 5xx rate > 15% │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ GraphRAG │ ──▶ │ Dry-Run │ ──▶ │ Multi-Sig │ │
│ │ Analysis │ │ Simulation │ │ Approval │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ Root Cause: Blast Radius: [x] devops-alice │
│ postgres-db 1 pod, 0 data loss [x] sre-bob │
│ │
│ Monthly Savings: $523.60 if fixed │
│ │
│ [ APPROVE & EXECUTE ] │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
AWOOOI (AI + WOOO Intelligent Operations) transforms reactive firefighting into proactive, AI-assisted decision-making—while keeping humans firmly in control of critical actions.
Enterprise Moats
Four pillars that make AWOOOI enterprise-ready from Day 1:
Privacy Shield
Your PII never leaves your premises. Period.
# Before: Raw sensitive data
"User 192.168.1.100 with email admin@company.com triggered alert"
# After: Consistent pseudonymization
"User [IP_1] with email [EMAIL_1] triggered alert"
# Same value → Same label (AI maintains context without seeing real data)
- Regex-based detection: IP, Email, UUID, API Keys, JWT
- Consistent hashing:
[IP_1]always maps to the same IP within a session - Rehydration Engine: Labels restored only at MCP execution boundary
- Zero PII in logs, zero PII to cloud LLMs
GraphRAG: Topology-Aware Intelligence
AI that understands your microservices like a senior SRE.
┌─────────────────────────────────────┐
│ BLAST RADIUS ANALYSIS │
│ (Upstream Impact) │
└─────────────────────────────────────┘
┌─────────────┐
│ ingress │ ← Will be affected
└──────┬──────┘
│ depends on
▼
┌─────────────┐
│ frontend │ ← Target service
└──────┬──────┘
│ calls
▼
┌───────────────────────┼───────────────────────┐
│ │ │
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ auth-service │ │ product-api │ │ order-api │
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
└─────────────────────┼─────────────────────┘
▼
┌──────────────┐
│ postgres-db │ X ROOT CAUSE
└──────────────┘
- BFS-based traversal with configurable
max_depth(default: 3) - Dual-direction analysis: Upstream (blast radius) + Downstream (root cause)
- Priority ranking: DATABASE > CACHE > QUEUE for root cause identification
- Multiple root causes: No single-point assumptions—collect ALL unhealthy dependencies
Multi-Sig & Dry-Run: Defense in Depth
Every critical action is simulated, validated, and co-signed.
┌────────────────────────────────────────────────────────────────┐
│ RISK MATRIX │
├────────────┬─────────────┬─────────────────────────────────────┤
│ Risk Level │ Signatures │ Required Roles │
├────────────┼─────────────┼─────────────────────────────────────┤
│ LOW │ 0 (auto) │ — │
│ MEDIUM │ 1 │ admin, devops, sre │
│ HIGH │ 2 │ admin, devops, sre │
│ CRITICAL │ 2 │ CTO + CISO (mandatory) │
└────────────┴─────────────┴─────────────────────────────────────┘
TOCTOU Protection (Time-of-Check to Time-of-Use):
1. User clicks "Approve"
2. System re-runs Dry-Run immediately before execution
3. If state changed → Status = VOIDED (not cleared!)
4. Full audit trail preserved for compliance
Dry-Run Checks:
- RBAC Permission validation
- Syntax & parameter validation
- Resource existence verification
- PodDisruptionBudget compliance
- Blast radius calculation
Progressive Autonomy: Trust That Evolves
The more you approve, the less you need to.
┌─────────────────────────────────────────────────────────────────┐
│ TRUST SCORE PROGRESSION │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Score: 0 ──────────────────────────────────────────────▶ 10+ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ HIGH │ ──▶ │ MEDIUM │ ──▶ │ LOW │ │
│ │ 2-sig │ @10 │ 1-sig │ @5 │ auto │ │
│ └─────────┘ └─────────┘ └─────────┘ │
│ │
│ ⚠️ CRITICAL operations NEVER auto-downgrade (enterprise law) │
│ │
│ Single REJECT → Trust score resets to 0 (instant collapse) │
│ │
└─────────────────────────────────────────────────────────────────┘
- Approve → +1 trust score
- Reject → Score resets to 0 (trust collapses instantly)
- Pattern-based:
restart_pod:nginx-*builds trust separately fromdelete_pvc:* - CRITICAL operations (DROP TABLE, DELETE NAMESPACE) → Always requires human dual-signature
leWOOOgo Engine Architecture
AWOOOI is built on the leWOOOgo Engine—a modular, plugin-based architecture inspired by LEGO blocks:
┌─────────────────────────────────────────────────────────────────────────────┐
│ leWOOOgo Engine │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ INPUT │ │ BRAIN │ │ OUTPUT │ │ ACTION │ │ DATA │ │
│ │ ─────── │ │ ─────── │ │ ─────── │ │ ─────── │ │ ─────── │ │
│ │Webhooks │ │ Ollama │ │ Slack │ │ K8s │ │ Postgres│ │
│ │ Kafka │ │ OpenAI │ │ Discord │ │ Shell │ │ Redis │ │
│ │Prometheus│ │ Claude │ │ Email │ │ MCP │ │ S3 │ │
│ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ │
│ │ │ │ │ │ │
│ └─────────────┴─────────────┴─────────────┴─────────────┘ │
│ │ │
│ ┌───────┴───────┐ │
│ │ UI │ │
│ │ ───────────── │ │
│ │ Next.js │ │
│ │ ApprovalCard │ │
│ │ThinkingStream │ │
│ └───────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Module Overview
| Module | Purpose | Key Components |
|---|---|---|
| INPUT | Event ingestion | Prometheus AlertManager, Kafka, Webhooks |
| BRAIN | AI reasoning | Ollama (local), OpenAI, Claude, GraphRAG |
| OUTPUT | Notifications | Slack, Discord, Email, Custom webhooks |
| ACTION | Execution | K8s API, Shell, MCP Bridge, Ansible |
| DATA | Persistence | PostgreSQL, Redis, S3, Vector DB |
| UI | Human interface | Next.js 14, ApprovalCard, ThinkingTerminal |
MCP (Model Context Protocol) Support
// MCP enables AI to safely interact with external tools
await mcpBridge.callTool("kubernetes", "restart_pod", {
pod_name: "[POD_1]", // Redacted in logs
namespace: "production",
graceful: true,
});
// Rehydration happens at execution boundary only
FinOps: Day-1 ROI
Every wasted resource has a dollar sign. AWOOOI shows you exactly how much.
┌─────────────────────────────────────────────────────────────────┐
│ FINOPS COST ANALYSIS │
├─────────────────────────────────────────────────────────────────┤
│ │
│ MONTHLY WASTE DETECTED: $523.60 │
│ │
│ ┌──────────────────┬──────────────────┬──────────────────┐ │
│ │ REALIZABLE │ FREED │ ANNUAL │ │
│ │ $480.00/mo │ $43.60/mo │ $5,760/yr │ │
│ │ ──────────── │ ──────────── │ ──────────── │ │
│ │ PVC deletion │ Pod cleanup │ if all fixed │ │
│ │ Node resize │ (needs scale) │ │ │
│ └──────────────────┴──────────────────┴──────────────────┘ │
│ │
│ TOP RECOMMENDATIONS: │
│ ├─ Delete orphaned PVC 'data-postgres-backup' -$40.00 LOW │
│ ├─ Resize node 'worker-large-01' -$340.00 HIGH│
│ └─ Delete zombie Pod 'legacy-api-5d7b8' -$76.00 MED │
│ │
└─────────────────────────────────────────────────────────────────┘
Scan Types:
- Orphaned PVCs: Storage not mounted by any Pod
- Zombie Pods: CPU < 1% for 7+ consecutive days
- Over-provisioned Nodes: High request, low actual usage
Safety Buffer: wasted = requested - (actual × 1.2) prevents OOM from aggressive recommendations.
Quick Start
Prerequisites
- Python 3.11+
- Node.js 18+
- pnpm 8+
- Docker (optional, for local Ollama)
Installation
# Clone the repository
git clone https://github.com/anthropics/awoooi.git
cd awoooi
# Install dependencies
pnpm install
# Setup Python environment
cd apps/api
python -m venv venv
source venv/bin/activate # or `venv\Scripts\activate` on Windows
pip install -r requirements.txt
Run Tracer Bullet 2.0 (E2E Demo)
Experience the full AWOOOI loop in 30 seconds:
cd apps/api
python scripts/tracer_bullet_2.py
Expected Output:
============================================================
TRACER BULLET 2.0 - FULL LOOP TEST
Test ID: tb2-20260319143052
============================================================
[x] [trigger_alert] PASS
[x] [graphrag_analysis] PASS
[x] [generate_approval] PASS
[x] [multisig_approval] PASS
[x] [mcp_execution] PASS
============================================================
TEST SUMMARY
============================================================
Total Steps: 5
Passed: 5
Failed: 0
Status: ALL PASSED
Start Development Servers
# Terminal 1: API Server
cd apps/api
uvicorn src.main:app --reload --port 8000
# Terminal 2: Web Server
cd apps/web
pnpm dev
Open http://localhost:3000 to see the AWOOOI dashboard.
Project Structure
awoooi/
├── apps/
│ ├── api/ # FastAPI Backend
│ │ ├── src/
│ │ │ ├── services/ # Core services
│ │ │ │ ├── approval.py # Multi-Sig engine
│ │ │ │ ├── dry_run.py # Dry-Run engine
│ │ │ │ ├── trust_engine.py # Progressive autonomy
│ │ │ │ └── graph_rag.py # Topology analysis
│ │ │ └── plugins/
│ │ │ ├── security/ # Privacy Shield
│ │ │ ├── mcp/ # MCP Bridge
│ │ │ └── finops/ # Cost analyzer
│ │ └── scripts/
│ │ └── tracer_bullet_2.py # E2E test
│ │
│ └── web/ # Next.js Frontend
│ └── src/
│ ├── components/
│ │ └── agent/
│ │ ├── approval-card.tsx
│ │ └── thinking-terminal.tsx
│ └── stores/
│ └── agent.store.ts
│
├── packages/
│ └── lewooogo-core/ # Shared types & contracts
│
└── docs/
└── adr/ # Architecture Decision Records
Roadmap
| Phase | Status | Description |
|---|---|---|
| Phase 0 | Complete | Contracts & Scaffolding |
| Phase 1 | Complete | Core Integration (Monorepo, SSE, Ollama) |
| Phase 2 | Complete | HITL (ApprovalCard, Dry-Run, Multi-Sig) |
| Phase 3 | Complete | Enterprise (Privacy Shield, GraphRAG, FinOps) |
| Phase 4 | In Progress | Production Hardening & GA Release |
| Phase 5 | Planned | Multi-cluster, Federation, SaaS |
Contributing
We welcome contributions! Please see our Contributing Guide for details.
# Run tests
pnpm test
# Run linting
pnpm lint
# Format code
pnpm format
License
MIT License - see LICENSE for details.
Built with love by 岑洋國際行銷有限公司
Turning 3 AM pages into peaceful nights since 2026
"The best incident is the one you never have to wake up for."
— AWOOOI Philosophy