wooo/awoooi
1
1
Fork 0
Code Issues 2 Pull Requests 1 Actions 5 Packages Projects Releases Wiki Activity
1,767 Commits 194 Branches 2 Tags
c3fa03fc19cd85749fdce65237fb99323692203c
Commit Graph

79 Commits

Author SHA1 Message Date
Your Name
9908fdf50d feat(p3.1-t2-patha): DiagnosisAggregator 路徑 A + Solver F4 critical reject + 對齊測試
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 1m59s
Details 
Wave 8 P3.1-T2 PathA 啟用 + Solver F4 安全強化 + test 對齊：

PathA — DiagnosisAggregator 信號分類層補 PDI:
- ENABLE_DIAGNOSIS_AGGREGATOR default=False → True
  · PathA 純信號分類層（OOMKilled/CrashLoop 等業務邏輯）
  · 不重複呼叫 K8s/SignOz API（只取 PDI 已收集的 raw 資料）
  · 安全 default on — 純邏輯處理，無外部依賴重疊
- diagnosis_aggregator.py +155 行（PathA 實作）
- pre_decision_investigator.py 已接 (commit 3a2cd151)

F4 — Solver critical risk reject:
- solver_agent.py: _validate_recommended_action 拒絕 risk=critical
  · 鐵律：critical 動作必須走人工審批，不可變 Telegram 按鈕
  · log warning + return None（被 _extract 過濾掉）
- _extract_recommended_actions 改返回 (list, status_str) tuple
  · status="ok"/"empty"/"all_invalid" 供呼叫端決策
- protocol.py +16 / metrics.py +9 / ai_router.py +18 — 配套 metric + protocol field

測試對齊:
- test_solver_recommended_actions.py 拆 test_all_valid → low/medium/high accepted +
  test_critical_rejected
- result tuple unpack: result, _ = _extract_recommended_actions(...)
- test_diagnosis_aggregator_stub.py: feature flag default 改 True 對齊 PathA

Tests: 51 passed (solver 28 + aggregator 16 + router fallback 8)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Multiple Engineers (Wave 8 P3.1-T2 PathA + F4) <noreply@anthropic.com>
 2026-04-27 14:42:29 +08:00
Your Name
fb130c9a28 feat(p3.1-t2): DiagnosisAggregator stub tests + sanitization 補強 + metrics 補欄
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 2m16s
Details 
Wave 8 P3.1-T2 後續補測 + 配套：

新增測試:
- test_diagnosis_aggregator_stub.py (238 行) — 15 tests
  · stub fixture 驗證 _collect_diagnosis_aggregator 接線
  · feature flag default off 不呼叫
  · timeout 邊界 / exception fail-soft

修改:
- core/metrics.py +23 — 新增 DiagnosisAggregator 相關 Prometheus 指標
- sanitization_service.py +24 — 補強 prompt sanitize 邊界（vuln #4 配套）
- RUNBOOK-AGENT-STEP-LATENCY.md / agent_step_latency_rules.yaml — 微調

Tests: 15 passed

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-27 08:30:26 +08:00
Your Name
3a2cd15144 feat(p3.1-t2): Tier-2 三服務感知強化 — Sentry 簽章 + DiagnosisAggregator + Solver actions test
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
Wave 8 P3.1-T2 三項感知強化（多 engineer 補完）：

Sentry Webhook 簽章驗證:
- sentry_webhook.py: 接入 SentryWebhookService.verify_sentry_signature()
- 拒絕無效 sentry-hook-signature → 401 → 防偽造攻擊

DiagnosisAggregator Pod 深診斷整合:
- pre_decision_investigator.py: 新增 _collect_diagnosis_aggregator()
- ENABLE_DIAGNOSIS_AGGREGATOR feature flag 守衛（default=False）
- evidence_snapshot.py: extra_diagnosis 欄位 + build_summary 顯示
- timeout=3.0s + try/except 隔離（fail-soft）
- Conservative 策略：待重疊分析確認 vs PreDecisionInvestigator 不重複

config.py:
- 新增 ENABLE_DIAGNOSIS_AGGREGATOR Field（default=False，K8s ConfigMap 動態啟用）

Solver B1 補測（commit 7c726ebc 對應）:
- test_solver_recommended_actions.py — 20 tests + 3 skipped
- 驗證結構化 recommended_actions（北極星 §1.1 修復多樣性 ≥ 40%）
- LLM 失敗 graceful degraded（candidates=[], degraded=True）

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Multiple Engineers (Wave 8 P3.1-T2) <noreply@anthropic.com>
 2026-04-27 08:24:15 +08:00
Your Name
fefe4c21cd fix(inc-20260425): A1+A2 後續 — Solver/Critic timeout + auto_repair 接線 + Runbook + Grafana
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
延續 595629c0 INC-20260425 修復，補三段 Agent + 全鏈路觀測：

A1 後續 — Solver/Critic 三段 timeout 接線:
- solver_agent.py: AGENT_SOLVER_TIMEOUT_SEC=20.0（env override）
- critic_agent.py: AGENT_CRITIC_TIMEOUT_SEC=15.0（env override）
- protocol.py: 三 Agent 共用 observe_agent_step() 包裹呼叫
  · success/timeout/error outcome label
  · histogram 寫入 aiops_agent_step_duration_seconds

A2 後續 — auto_repair_service 改用 _diagnose_fallback_chain:
- auto_repair_service.py +46 行 — 切換 DIAGNOSE 路由到新 chain（NEMO→GEMINI→CLAUDE）
- 完全避開 Ollama CPU 238s 二次 timeout

新增 metrics:
- core/metrics.py +59 行 — 配合 observe_agent_step 的 histogram bucket + label cardinality

新增測試 (862 行):
- test_agent_step_timeouts.py (475) — 三 Agent 各 timeout 邊界 + outcome label
- test_ai_router_diagnose_fallback.py (387) — _diagnose_fallback_chain 正確序

新增配套:
- docs/runbooks/RUNBOOK-AGENT-STEP-LATENCY.md (350) — INC 故障排查 + 觀測指引
- ops/monitoring/grafana/agent_step_latency_rules.yaml (160)
  · 三 Agent histogram alert rules（p99 > timeout 80% → warning）

驗收: 33 tests pass (test_agent_step_timeouts 22 + test_ai_router_diagnose_fallback 11)

INC-20260425 雙修總工作量（595629c0 + 此 commit）:
  · 5 個 service/agent 檔修改
  · 1 個新 observability 模組
  · 4 個新測試/配套檔
  · 1372+187 = 1559 行新增

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (INC-20260425 後續) <noreply@anthropic.com>
 2026-04-27 08:15:53 +08:00
Your Name
cc547736ab feat(wave6-8): P2.1 fusion + P2.2 governance + P2.4 consensus + Wave 7/8 BLOCKER 修復 
承接 Wave 6/7/8 多 engineer 在 agent 限額前完成的代碼，補 commit 解 production
HEAD 隱性 import error（decision_fusion 已被 decision_manager 引用但檔案 untracked）。

新增（後端核心）:
- decision_fusion.py (562 行) — P2.1 方法 III（OpenClaw + Hermes + Elephant 三 LLM 融合）
- aiops_timeline.py + aiops_timeline_service.py — critic B4 修復
  /api/v1/aiops/timeline endpoint，DB 存取抽到 service 層遵守 leWOOOgo 積木化
- migrations/p2_decision_fusion_columns.sql + rollback — approval_records fusion 欄位

修改（後端整合）:
- decision_manager.py — fusion 三斷鏈修補（critic B1+B2+B3）：
  · B1: 寫 _evidence_snapshot_ref 到 token.proposal_data
  · B2: fusion 前計算 complexity_score 並寫 token
  · B3: fusion composite 寫 token.proposal_data["decision_fusion"]
- auto_approve.py — fusion + consensus 認識（critic B3+B5）：
  · composite > 0.7 → auto_execute_eligible bypass min_confidence
  · source=consensus_engine + score>=0.6 → 規則可信路徑
- consensus_engine.py — db-fix _save_consensus 重用 agent_sessions
- governance_agent.py — db-fix _alert PG 寫入 ai_governance_events
- approval_db.py — fusion 3 欄位 + 2 partial index + CheckConstraint
- db/models.py — schema 對齊 migration
- core/config.py — vuln #1 修復：OLLAMA_URL/_FALLBACK_URL field_validator
  拒絕公網 IP + 外部域名，僅允許私網/loopback/K8s SVC 白名單
- core/feature_flags.py — P2 fusion + consensus flags
- main.py — governance_agent lifespan 啟動
- failover_alerter.py — Wave8-X2: in-memory dedup fallback（Redis 拒絕後不 fail-open）
- ollama_*.py — metrics 整合 + recovery 改善
- auto_repair_service.py — verifier 接線

新增（測試 2438 行）:
- test_decision_fusion.py / test_governance_agent.py / test_consensus_integration.py
- test_p2_db_fixes.py / test_wave8_fusion_fixes.py
- test_config_url_validation.py（vuln #1 12 tests）
- test_failover_alerter.py +Wave8-X2 in-memory dedup 補測

驗收: 116 tests pass (decision_fusion + wave8_fusion + config_url + consensus +
                      governance + p2_db_fixes + failover_alerter)

Conflict resolution:
- 3 檔（config.py + auto_approve.py + decision_manager.py）git stash pop 衝突
  保留 stashed (engineer 最終版)，補回 ValueError 「公網 IP」字樣對齊 test

Note: 此 commit 解 production HEAD 隱性 import error
仍未修: vuln #4 prompt injection / debugger B14 quota fail-closed
       / B25-B26 drain_pending_tasks / B8 governance fail alert

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Multiple Engineers (Wave 6/7/8) <noreply@anthropic.com>
 2026-04-27 08:11:40 +08:00
Your Name
2c57b71db9 feat(wave5-p2): GovernanceAgent 4 項自檢 + Ollama 健康告警規則 + Prometheus metrics 整合
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 10m45s
Details 
MASTER plan_complete_v3.md Wave 5 P2.2 + P2.3 完成（multiple engineers 在限額前完成代碼，補 commit）：

P2.2 — GovernanceAgent 4 項自檢:
- governance_agent.py (342 行) — 每 1 小時自檢循環:
  · trust_drift（信任度漂移檢測）
  · knowledge_degradation（知識退化檢測）
  · llm_hallucination（LLM 幻覺檢測）
  · execution_blast_radius（執行爆炸半徑檢測）
- main.py lifespan: asyncio.create_task(run_governance_loop()) 啟動
  try/except 包裹，schedule 失敗不阻斷主流程
- failover_alerter.py: alert_governance(event_type, payload) 1h dedup
  四類事件 → Telegram MarkdownV2 告警

P2.3 — Ollama 健康規則 + Prometheus Metrics:
- ops/monitoring/ollama_health_rules.yaml (148 行):
  · OllamaHealthDegraded / OllamaPrimaryDown
  · OllamaFailoverTriggered / GeminiQuotaExceeded
  · 補 Prometheus 取資料的 alert rules
- core/metrics.py (57 行):
  · GEMINI_DAILY_CALL_COUNT / GEMINI_DAILY_QUOTA Gauge
  · OLLAMA_FAILOVER_TRIGGERED_TOTAL Counter
  · OLLAMA_CURRENT_PRIMARY_IS_OLLAMA Gauge
- ollama_failover_manager.py:
  · _check_gemini_quota: 每次 check 同步更新 Gauge（讓 Prometheus 取最新值）
  · select_provider: failover 時 inc Counter + 切 Primary Gauge
  · try/except 包裹，metric 失敗不阻斷主路由

E2E 測試:
- test_failover_e2e_dispatch.py (365 行)
  完整 dispatch 路徑：health check → failover decide → alerter → metrics

Tests: 54 passed (e2e_dispatch + failover_manager + failover_alerter)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Multiple Engineers (上 session Wave 5) <noreply@anthropic.com>
 2026-04-26 20:56:19 +08:00
Your Name
55c6b4e2d9 feat(p1): Ollama 多層容災系統 — P1.1 健康檢測 + P1.2 ai_router 整合 + P1.5 容災告警 
ADR-092 P1 飛輪閉環的 Ollama 失敗轉移子系統，全部 Engineer-A2/C/C2 補上。

新服務 (1581 行)：
- ollama_health_monitor.py (356)：3 層健康檢測（TCP/HTTP/推理）
- ollama_failover_manager.py (571)：111→188 自動切換 + Redis 持久化 + recovery callback
- ollama_auto_recovery.py (436)：30s 背景監控 + 連續 3 次 HEALTHY → 切回 + clear_cache
- failover_alerter.py (218)：P1.5 Telegram 容災告警

服務整合：
- ai_router.py: AIProviderEnum.OLLAMA_188 + 120s budget + failover fallback chain
- main.py lifespan: 啟動時 wire callback + start recovery，關閉時優雅 stop
- config.py: OLLAMA_FALLBACK_URL / OLLAMA_HEALTH_CHECK_MODEL / GEMINI_DAILY_QUOTA（帳單熔斷）

K8s 配置：
- 04-configmap.yaml.patch-188-fallback：注入 OLLAMA_FALLBACK_URL=http://192.168.0.188:11434

測試 (2082 行)：
- test_ollama_health_monitor.py (402)
- test_ollama_failover_manager.py (707)
- test_ollama_auto_recovery.py (580)
- test_ai_router_failover_integration.py (257)
- test_lifespan_failover_wiring.py (136)

依賴鏈：service 三件套 + ai_router + main.py 一起 commit，缺一就 ImportError。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-26 20:18:33 +08:00
Your Name
86ee013cdf feat(hermes-complete): Hermes NL 三項補強 + ConsensusEngine + ADR 收尾
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 9m32s
Details 
## Hermes NL 補強（nl_gateway.py）
- T1 hermes_dispatch_log DB 寫入（asyncio.create_task 非阻擋）
- T2 Redis 速率限制：per-chat_id 20 req/min，fail-open
- T3 Multi-turn session：hermes:session:{chat_id}:{user_id} TTL=300s，最近 3 輪

## ConsensusEngine（ADR-095 宣告式設計）
- consensus_engine.py: CONSENSUS_WEIGHTS class 屬性
  security=0.4 鎖定，9 個 Claude Code agent 分配 0.6
- config.py: ENABLE_12AGENT_CONSENSUS=False feature flag

## ADR 狀態
- ADR-093/094/095: Proposed → 🟡 批准實作中
- 各 ADR 加 v1.1 變更紀錄

## K8s ConfigMap
- prod 04-configmap.yaml: 加 3 個 feature flags（均 false）
- dev 02-configmap.yaml: 同步加入

## LOGBOOK
- 記錄 WS0–WS6 + 補強完成，feature flags 啟用指引

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-25 02:22:40 +08:00
Your Name
2572ec46d2 feat(ws4): Hermes NL 自然語言介面 — 12-Agent Claude SDK 接入（ADR-094/095） 
## hermes/ 套件（5 個新模組）

### display_names.py
- 12 agent 視覺識別表（emoji + hashtag + handle + short_name）
- format_response_header() 產生 Telegram 前綴

### agent_loader.py
- 解析 .claude/agents/*.md frontmatter → system prompt
- lru_cache 避免重複讀檔

### safety_hooks.py
- 移植 awoooi-guard.js 20 條 HARD BLOCK 規則（DENY_PATTERNS）
- 5 條 MUTATE_PATTERNS → 須走審批流

### nl_gateway.py
- Layer 1: 關鍵字正則路由（12 條規則，<10ms）
- Layer 3: DEFAULT_AGENT = "debugger"
- Claude Agent SDK query() 非同步串流，取 ResultMessage.result
- 安全降級：SDK error → 友好錯誤訊息

### telegram_webhook.py
- WS4 Hermes NL 接入（@tsenyangbot mention 或私訊觸發）
- HERMES_NL_ENABLED=False（feature flag 保護，預設關閉）

## telegram_gateway.py
- send_hermes_reply(text, chat_id, reply_to_message_id)
  無 500 字截斷，支援 Agent 長回覆

## config.py
- HERMES_NL_ENABLED: bool = False
- TELEGRAM_BOT_USERNAME: str = "tsenyangbot"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-25 02:10:06 +08:00
Your Name
294e0e3387 feat(ws3): ADR-093 Callback User-ID Binding + ADR-094 Webhook 入口 
## T3.1/T3.2 Bound User Check（security_interceptor.py）
- verify_callback() Step 0: 檢查 Redis cb_bind:{nonce}
  → 若有 binding 且 caller != bound_user_id → UserNotWhitelistedError
  → 若 key 不存在（舊格式）→ 降級走 whitelist（向後相容）
  → 若 Redis unavailable → 降級繼續（安全降級）
- bind_callback_user(nonce, user_id): async 方法，TTL=48h

## T3.3 Telegram Webhook 入口（ADR-094）
- apps/api/src/api/v1/telegram_webhook.py（新建）
  POST /api/v1/telegram/webhook
  - X-Telegram-Bot-Api-Secret-Token header 驗證
  - TELEGRAM_WEBHOOK_SECRET="" → dev 跳過（不 break 現有測試）
  - WS4 Hermes NL 接入預留佔位

## T3.4 config.py
- 新增 TELEGRAM_WEBHOOK_SECRET field（預設空字串）

## main.py
- 掛載 telegram_webhook_v1.router 到 /api/v1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-25 02:10:06 +08:00
Your Name
6d5fd3c124 feat(ws2): ADR-093 路由統一 — BIGINT + NotificationMatrix + feature flag 
## 修復

### T2.1 BigInteger overflow 修復
- `db/models.py`: telegram_chat_id Integer → BigInteger
  （原 int32 無法容納群組 ID -1003711974679）

### T2.2 移除 CAST workaround
- `approval_db.py:739`: 移除 CAST(:telegram_chat_id AS BIGINT)
  ORM 已正確使用 BigInteger，workaround 可退役

### T2.3 Redis key 一致性修復
- `heartbeat_report_service.py:575`: telegram:polling_leader → telegram:polling:leader
  （telegram_gateway.py 使用冒號分隔，heartbeat 用底線是 bug）

## 新增

### T2.4 notification_matrix.py
- `services/notification_matrix.py`: ADR-093 路由矩陣
  - Destination(DM/GROUP/BOTH) + RoutingRule dataclass
  - NOTIFICATION_ROUTING dict（TYPE-1 ~ TYPE-8M 完整映射）
  - resolve_chat_ids(type, dm, group, *, tg_group_cutover=False) 灰階切流 API

### T2.5 telegram_gateway.py feature flag 保護
- line 43: 加 notification_matrix import
- line 1827-1834: TG_GROUP_CUTOVER=False 時維持舊行為
  TG_GROUP_CUTOVER=True 時解除 _interactive_types 黑名單，由矩陣控制

### T2.6 Migration SQL
- `migrations/adr093_notification_routing.sql`:
  - CREATE TABLE approval_records (telegram_chat_id BIGINT)
  - CREATE ROLE awoooi_migrator (IF NOT EXISTS)
  - 含舊環境 ALTER COLUMN int→bigint 保護

## 測試同步
- `tests/integration/setup_test_schema.sql`: telegram_chat_id BIGINT

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-25 02:10:06 +08:00
Your Name
bb5f16f8ef fix(aiops-p2): P2.1 LLM品質三修 — Evidence-First + consensus confidence + raw_evidence注入
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
根因:
- consensus_engine 四 ExpertAgent confidence=0.0 → 加權投票 total=0 → 永遠返回 NO_ACTION
- prompts.py 無 Evidence-First 指令 → LLM 靠記憶推理，無真實環境約束
- openclaw.py analyze_alert 建 prompt 未注入 MCP evidence (diagnosis_context)

修復:
- consensus_engine: SRE/Security/Cost/Performance 依訊號強度設 0.45~0.80 confidence
- consensus_engine: _normalize_action 加「重新啟動」別名 → RESTART
- consensus_engine: SecurityAgent 移除未使用的 _target 變數
- prompts.py: 加 Evidence-First Protocol + Skepticism Rules 區塊
- openclaw.py: analyze_alert 提取 diagnosis_context → <raw_evidence> 注入 full_prompt

驗證: consensus score 從 0.0 → 0.744（CrashLoop 測試案例）

P2.1 fix 2026-04-24 ogt + Claude Sonnet 4.6

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-24 15:52:25 +08:00
Your Name
d0591c54b0 fix(security): 體健修復 — 7項 Critical/Major 安全問題全修
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 35s
Details 
## Critical 修復 (C1-C5)
- C1: git rm --cached 03-secrets.yaml（CHANGE_ME 模板不再追蹤）
- C2: git rm --cached awoooi.db + .gitignore 加 *.db（SQLite HARD_RULES 違規）
- C3: sentry-tunnel SENTRY_HOST 改為 process.env fallback
- C4: config.py DATABASE_URL 移除 changeme default，改為必填
- C5: run_migration.py 改為 os.environ["DATABASE_URL"]

## Major 修復 (M1-M4)
- M1: auto_repair /execute 加 CSRF 保護 + AutoRepairPanel.tsx 同步
- M2: drift /rollback /adopt 加 CSRF 保護（/internal/scan 保持無 CSRF）
- M3: terminal /intent 加 CSRF 保護 + terminal.store.ts 同步
- M4: live-dashboard HOST_IPS + host-grid VIP 改為 env var

## 其他
- 新增 apps/web/.env.example（6 個 env var 說明）
- K8s deployment-web 補入 3 個新 env var
- 整合測試：新增 aider_event_repository + ai_router_feedback 真實 DB 測試
- test_terminal.py CSRF dependency override 修復

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-22 01:27:39 +08:00
Your Name
e1539a813e feat(config+main): aider-watch v2 settings + router + lifespan register 
- Add 4 settings to config.py: AIDER_WEBHOOK_SECRET, AIDER_EVENTS_STREAM_KEY, AIDER_PATTERN_EXTRACT_INTERVAL_HOURS, USE_AIDER_FEEDBACK (ADR-091)
- Import aider_events_v1 router in main.py imports (alphabetical after ai_slo_v1)
- Register aider_events_v1.router in include_router block (after alert_operation_logs_v1)
- Register run_aider_event_processor_loop() in lifespan (after compliance_scanner_loop)
- All 65 tests pass (24 action_parsing + 41 aider-watch tests)

Co-Authored-By: Claude Haiku 4.5 (1M context) <noreply@anthropic.com>
 2026-04-20 19:40:02 +08:00
OG T
7e9448f6d0 fix(openclaw): 幻覺 deployment 名雙層防禦 — Prompt + Python validator
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
2026-04-18 晚（台北時區）— ogt + Claude Opus 4.7 (1M)

生產事件 (approval f763bedf, 22:58):
- Alert: KubePodCrashLooping, labels.deployment="awoooi-api"
- NEMOTRON 雖收 inventory "awoooi-api, awoooi-web, awoooi-worker"
  仍輸出 kubectl_command="kubectl rollout restart deployment/awoooi-prod"
  (把 namespace 誤當 deployment 名)
- 執行結果: "Deployment 'awoooi-prod' not found in namespace 'awoooi-prod'"

## Layer 1: NEMOTRON_SYSTEM_PROMPT 強化 (prompts.py)
新增「🔒 DEPLOYMENT NAME RULE (STRICTLY ENFORCED)」區塊:
- namespace NEVER is a deployment name
- "awoooi-prod" 是 NAMESPACE,不可寫 deployment/awoooi-prod
- 若有 inventory,deployment 必須 exact match
- 優先用 labels.deployment,unknown → NO_ACTION

## Layer 2: Python 後驗證 (openclaw.py:1322+)
LLM 回應解析後 regex 抽出 deployment 名,對照 _k8s_inventory:
- 在清單內 → 通過
- 不在清單內 → 降級:
    * kubectl_command → "kubectl get deploy -n {ns}"(純調查)
    * suggested_action → NO_ACTION
    * target_resource → "unknown(hallucinated)"
    * confidence → 0.0
    * description 加註 [安全降級] 並列出合法 inventory
- log 'openclaw_deployment_hallucination_detected' 記錄

效果: 就算 LLM 無視 prompt,Python 層也會擋下。
破壞性 kubectl 絕不執行於不存在的 deployment。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-18 23:26:09 +08:00
OG T
604d8eea37 fix(schema-drift): 補齊 prompts.py + Claude API schema enum 同步 (ADR-090)
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 12m27s
Details 
問題: fe77e6d 擴充了 models/ai.py enum 至 8 值，但兩個地方未同步：
  1. core/prompts.py L77: 缺 INVESTIGATE、OBSERVE
  2. core/prompts.py L176 (NEMOTRON_SYSTEM_PROMPT): 缺 APPLY_HPA、INVESTIGATE、OBSERVE
  3. openclaw.py L564 (_call_claude tools schema): 舊 4 值 enum 約束

影響: LLM 不知道可以輸出 INVESTIGATE/OBSERVE，只能選舊 4 值

修復: 三處統一對齊 8 個 suggested_action 值
  RESTART_DEPLOYMENT|DELETE_POD|SCALE_DEPLOYMENT|APPLY_HPA|TUNE_RESOURCES|INVESTIGATE|OBSERVE|NO_ACTION

Closes: ADR-090 Prompt-Model 三層同步鐵律

2026-04-17 ogt + Claude Sonnet 4.6

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-17 22:10:18 +08:00
OG T
a258d87767 fix(webhooks+prompts): 修復 LLM 對所有告警一律輸出「重啟 AWOOOI 服務」的根本問題
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
根因 (INC-20260416-C365D0 postgres 磁碟告警事故):
1. alert_context 中 alertname 埋在 labels 深處，LLM 看到 alert_type="custom" → 不知道是什麼告警
2. 快取鍵用 alert_type:target_resource → 不同 alertname 共用同一快取 → 全部回傳第一個 LLM 結果
3. 系統 Prompt 無 alert-category 指導 → LLM 永遠輸出 kubectl rollout restart

修復:
- webhooks.py: alert_context 置頂加入 alertname + alert_category + annotations
- openclaw.py: 快取鍵改用 alertname:target_resource（告警名稱才是主要識別符）
- prompts.py: OPENCLAW_SYSTEM_PROMPT + NEMOTRON_SYSTEM_PROMPT 加入 Alert-Specific Analysis Rules
  database/storage 告警 → NO_ACTION + 調查指令；K8s 告警 → 對應重啟指令
  禁止對非 K8s 告警輸出 kubectl rollout restart deployment/awoooi-prod

2026-04-16 ogt + Claude Sonnet 4.6（亞太）

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-16 19:56:13 +08:00
OG T
76558a3cd9 feat(AIOps): 全開 P1-P6 feature flags + Nemotron + offline replay loop
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
- configmap: 啟用 AIOPS_P1~P6 全部總開關與子開關
- configmap: ENABLE_NEMOTRON_COLLABORATION=true（回歸 120s timeout）
- feature_flags.py: 補齊 AIOPS_P6_GOVERNANCE_ENABLED 缺失欄位
- main.py: 掛載 run_offline_replay_loop（ADR-087 Phase 6）

2026-04-15 ogt + Claude Sonnet 4.6（亞太）

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-15 21:59:51 +08:00
OG T
bf45b80bd2 feat(Phase 3.5 + Phase 4): AI 學習成果持久化到 PostgreSQL — 修正「AI 失憶」架構缺陷
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
ADR-085: AI 學習成果不可存在 Cache

架構鐵律確立：
- PostgreSQL = System of Record（AI 的永久記憶）
- Redis = Warm Cache（加速讀取，TTL 到期從 PG 復原）

核心變更：
1. models.py: 新增 PlaybookRecord / DynamicBaselineRecord / LogClusterRecord ORM
2. base.py: ALTER TABLE playbooks 補加 trust_score / requires_approval_level 等欄位
3. playbook_repository.py: 完整雙寫實作（PG upsert + Redis cache）
4. dynamic_baseline_service.py: Holt-Winters 訓練結果寫入 PG，Redis 只作 24h warm cache
5. log_anomaly_detector.py: Drain3 cluster template 寫入 PG（UPSERT on cluster_id）
6. main.py: 啟動時執行 backfill_redis_to_pg()（Redis → PG 冪等補救）

修正的問題：
- Playbook 7天 Redis TTL 到期 → AI 失去所有修復知識
- trust_score EWMA 隨 Redis TTL 歸零 → AI 重新回到初始信任度 0.3
- Holt-Winters 基線 24h TTL → AI 每天重新學習「正常」的定義
- Drain3 cluster 沒有持久化 → AI 把已知 log pattern 反覆當新 pattern

Phase 4 新服務（requirements.txt 已加入 statsmodels + drain3 + numpy）

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-15 15:34:04 +08:00
OG T
f1cbf6db7d feat(adr-081): Phase 1 感官縱深 — 8D 情報蒐集 + 執行後驗證 
成品：
- IncidentEvidence DB model（8D 感官 + pre/post 執行狀態）
- EvidenceSnapshot dataclass（build_summary → LLM 上下文）
- SanitizationService（Prompt Injection 0-tolerance，12 pattern）
- MCPToolRegistry（動態工具登記，suggest_tools 不寫死告警類型）
- PreDecisionInvestigator（8D 並行感官，P99 < 8s，Redis 30s 快取）
- PostExecutionVerifier（warmup 10s → 後狀態評估 success/degraded/failed）
- decision_manager + approval_execution 接線（feature flag 守衛）

Gate 1 修復：D4/D5/D7/D8 補 sanitize_dict_values；移除裸 "error" failure
signal 防 error_rate key 誤判；evidence_snapshot rowcount 零行警告。

測試：130 passed（+111 新增）

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-15 13:08:38 +08:00
OG T
db9e304a14 feat(adr-080): Phase 0 防護欄建立 — AI 自主化飛輪啟動 
- docs/superpowers/specs/2026-04-15-MASTER-ai-autonomous-flywheel-v2.md
  (1456 行，§0-§8 全填完：42-cell 戰術矩陣、7 Phase 計畫、7 ADR 摘要、
   15 KPI、21 Feature Flags、10 風險場景)

- docs/adr/ADR-080-ai-autonomy-flywheel-overview.md
  (7 Phase 結構 + 4 北極星 + 7 架構師 Review Gates + Phase 退出條件)

- apps/api/src/core/feature_flags.py
  (AIOpsFeatureFlags: P1~P6 總開關全 False + 15 細粒度子開關
   is_phase_enabled() / is_sub_flag_enabled() + bool cast 安全)

- apps/api/src/jobs/__init__.py + baseline_snapshot.py
  (Phase 0 基線快照 Job：MCP calls / Playbook confidence / general 比例
   / learning loop rate / auto_repair — 寫入 aiops:baseline:latest)

- apps/api/tests/test_feature_flags.py  (21 tests — 全綠)

- docs/HARD_RULES.md → v1.9
  (新增 Phase 退出條件鐵律：禁止未過 exit conditions 宣告 Phase 完成)

- CLAUDE.md 防失憶閘門 1：強制讀 MASTER §0 Session Resume Protocol

Gate 0 Pass — 21/21 tests green

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-15 12:44:53 +08:00
OG T
8b7e9cbfb8 fix(BLOCKER): LLM 連續失敗 — 4 個違反設計處全部修復
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 14m21s
Details 
統帥盤點發現飛輪沉默真因：4 個違反既定架構設計的 bug 同時撞車。

P0a — Ollama timeout 違反 GAP-B4 設計
  config.py:OPENCLAW_TIMEOUT 從 120s 改 30s
  原 120s 違反 ADR-052 GAP-B4 (LLM 25s hard timeout) 設計
  致 Ollama 過載時 thread 飢餓 120s 才降級

P0b — AI Router silent skip 觀測性修復
  ai_router.py: not_registered/circuit_open/rate_limit/privacy_skip
  全部累積到 errors 陣列，log all_providers_failed 時可知為何 skip
  原本 errors=["ollama: Timeout"] 但 tried=4 個，無法診斷

P1a — send_text 方法不存在 bug
  ai_router.py:1005 tg.send_text() → tg.send_notification(parse_mode=HTML)
  TelegramGateway 只有 send_notification 沒 send_text
  致 fallback 失敗通知本身失敗（雙重靜默）

P1b — resend_stale_ready_tokens 並發爆炸
  decision_manager.py: 加 asyncio.Semaphore(5) + 200ms throttle
  原本 fire_and_forget N 個 task 同時跑，N=108 時 Ollama embedding
  全部 timeout，包括我打的 live-fire 也被擠爆
  改：max 5 並發 + 每完成喘 200ms

CD 流程審查 (Blocker 1): 完全符合 ADR-039 設計，10-15 min 是預期
不需修，是設計就需要這時間。

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-04-14 19:37:03 +08:00
OG T
00a31abb85 feat(heartbeat): ADR-073 P2 心跳整合重構 — HeartbeatReportService + RedisLock
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
- 新增 HeartbeatReportService：11 個並行探針（Ollama/Nemotron/Gemini/Claude/MCP×4/ArgoCD/Velero）
- 重寫 send_heartbeat()：RedisLock 防重發 + 統一發送 SRE_GROUP_CHAT_ID
- 簡化 _heartbeat_loop()：移除散落的 silence 多次發送
- config.py：新增 OLLAMA_REQUIRED_MODELS 欄位
- 03-secrets.example.yaml：補 SRE_GROUP_CHAT_ID 確保 CD Inject 不遺漏

2026-04-12 ogt (ADR-073 Phase 2-3/4)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-12 15:35:13 +08:00
OG T
184b37a8b1 refactor(decision_manager): I2 DI 化 MCP Providers + fix config list type bug 
- DecisionManager.__init__ 注入 SSHProvider/K8sProvider，移除函數內 import+實例化
- config.get_tg_user_whitelist() 支援 list 輸入（monkeypatch/直接傳入），修復 AttributeError
- LOGBOOK 更新（test fix 6e0ee8b）

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-12 13:04:46 +08:00
OG T
5d78c5492b feat(argocd-mcp): 啟用 ArgoCD MCP Provider + token 注入流程
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
- config.py: ARGOCD_URL → https://192.168.0.125:30443（實際 HTTPS NodePort）
- config.py: ARGOCD_MCP_ENABLED=True + SENTRY_MCP_ENABLED=True（預設啟用）
- cd.yaml: 新增 ARGOCD_API_TOKEN Gitea Secret → K8s Secret 注入步驟
- K8s: ARGOCD_API_TOKEN 已手動注入 awoooi-secrets + API pods 已 rollout restart
- ArgoCD: 已開啟 admin account apiKey capability

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-11 09:32:28 +08:00
OG T
a2cc985f60 feat(mcp-phase3): ArgoCD MCP + Sentry MCP + 完整 Provider 註冊
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
ArgoCDProvider (3 工具):
  - argocd_list_apps: 列出所有 App + sync/health 狀態
  - argocd_get_app_status: 詳細狀態 + 問題資源清單
  - argocd_get_sync_history: 最近 N 筆部署記錄
  - 輸入驗證: app_name 白名單 regex
  - 需 ARGOCD_API_TOKEN + ARGOCD_MCP_ENABLED=true

SentryProvider (3 工具):
  - sentry_list_issues: 列出最近 Issues（狀態過濾）
  - sentry_get_issue: 詳情 + stacktrace 最後 5 frames
  - sentry_search_issues: PromQL 風格搜尋
  - issue_id 白名單驗證（只允許純數字）
  - 需 SENTRY_AUTH_TOKEN + SENTRY_MCP_ENABLED=true

providers/__init__.py: 補上 Prometheus + SSH + ArgoCD + Sentry 全部 10 個 providers
config.py: 新增 ARGOCD_URL / ARGOCD_API_TOKEN / ARGOCD_MCP_ENABLED / SENTRY_MCP_ENABLED

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-11 09:11:53 +08:00
OG T
6351e9a0e9 feat(mcp-phase2): MCP Phase 2 — Prometheus MCP + SSH MCP + alert labels
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 13m37s
Details
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 35s
Details 
MCP-2b: prometheus_provider.py
  - prometheus_query (PromQL 即時查詢)
  - prometheus_query_range (歷史趨勢，預設 15 分鐘)
  - prometheus_get_alert_history (告警觸發歷史)
  - config: PROMETHEUS_URL + PROMETHEUS_MCP_ENABLED

MCP-2a: ssh_provider.py
  - 群組A 9 個只讀診斷工具 (top/disk/memory/logs/status/port/nginx/swap)
  - 群組B 6 個安全操作工具 (restart/compose/systemctl/clear-log/ssl/nginx-reload)
  - 四層安全守衛 (白名單/allowed_hosts/forbidden_patterns/trust_score)
  - config: SSH_MCP_ENABLED + SSH_MCP_ALLOWED_HOSTS

K8s: 04-ssh-mcp-secret.example.yaml (ssh-mcp-key Secret 範本 + 建立步驟)

Alert labels: alerts-unified.yml 補充 mcp_provider/host_type/alert_category
  覆蓋: HostHighCpuLoad/HostOutOfMemory/HostOutOfDiskSpace/DockerContainer*
        SignOzDown/SentryDown/HarborDown/GiteaDown

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-11 02:35:35 +08:00
OG T
7768924fea fix(flywheel): 自動修復後移除 Telegram 按鈕 + 心跳告警排除飛輪
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 6m56s
Details 
問題: 自動修復成功後 Telegram 卡片仍顯示批准/拒絕/靜默按鈕

Fix 1 — Telegram 卡片回饋閉環 (積木化合規):
- telegram_gateway.send_approval_card: 發送後自動存 tg_approval:{id} 到 Redis
- telegram_gateway.mark_auto_repaired(): 新方法 — 移除按鈕 + reply 結果
- _try_auto_repair_background: 改呼叫 gateway.mark_auto_repaired() (Service 層)

Fix 2 — 心跳/看門狗告警排除飛輪:
- constants.py: is_heartbeat_alertname() + HEARTBEAT_ALERT_NAMES
- NoAlertsReceived2Hours 等不觸發 _try_auto_repair_background

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-10 11:52:04 +08:00
OG T
8c2983b70a fix(api+web): CORS 補 K3s NodePort origins + sign 補 signer_id/name
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
CORS (config.py):
- 補 http://192.168.0.125:32335 (K3s VIP NodePort)
- 補 http://192.168.0.120:32335 + 121:32335 (K3s nodes)
- 修前: 內網瀏覽器開 :32335 打 API 全 CORS blocked
  (incidents Failed to fetch / monitoring 無法連線根因)

sign body (pending-approvals-card.tsx):
- signer: 'web-ui' → signer_id: CURRENT_USER.id + signer_name: CURRENT_USER.name
- 修前: POST /approvals/{id}/sign 回 403 (缺必填欄位 422 誤報為 403)
  — 實際是 422 Field required signer_id + signer_name

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-09 19:50:48 +08:00
OG T
7857c25677 feat: Ollama 本機 Tool Calling 取代 NVIDIA 雲端 (44s→~5s)
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
- nvidia_provider.py: 新增 OllamaToolProvider
  - 實作 INvidiaProvider protocol，打 Ollama /v1/chat/completions
  - 模型: llama3.1:8b (tool calling 最穩定的 8B)
  - 延遲: 44s → ~5s（本機 M1 Pro 192.168.0.111）
  - get_nvidia_provider() 根據 USE_OLLAMA_TOOL_CALLING 切換
- config.py: USE_OLLAMA_TOOL_CALLING=True (預設開啟), OLLAMA_TOOL_MODEL=llama3.1:8b
- 回退: USE_OLLAMA_TOOL_CALLING=False → 恢復 NvidiaProvider 雲端

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-09 14:55:04 +08:00
OG T
8b5db2f58e feat(infra): 切換 Ollama 到 M1 Pro 192.168.0.111 + NetworkPolicy 更新
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
- OLLAMA_URL: 188 → 111 (M1 Pro, 40+ tok/s vs 0.45 tok/s)
- OPENCLAW_DEFAULT_MODEL: qwen2.5:7b-instruct → deepseek-r1:14b (SRE最強推理)
- OPENCLAW_TIMEOUT: 90s → 120s (deepseek-r1:14b 實測最慢 54s)
- NetworkPolicy v1.3: 新增 192.168.0.111:11434 egress，移除 188 的 Ollama port

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-08 22:05:14 +08:00
OG T
84f1f9f021 refactor(config): GITHUB_WEBHOOK_SECRET → GITEA_WEBHOOK_SECRET (ADR-059) 2026-04-05 14:25:47 +08:00
OG T
5ad403b287 fix(p0): v4.3 — 實測確認 Ollama CPU-only 不可用，DIAGNOSE 統一走 NIM 
實測依據 (2026-04-05):
- Ollama llama3.2:3b CPU-only: 238s 回 {"ok":true}，生產不可用
- Nemotron NIM: 2.2s~27.3s，avg 10.6s，一直是主力（Phase 22 起）
- NIM 從未有隱私問題，Incident 資料一直送雲端 GPU

變更:
- ai_router.py: _local_fallback_chain 廢棄（空 list）
- ai_router.py: DIAGNOSE route/route_sync 改回 _full_fallback_chain
- config.py: 更新 timeout 說明反映實測結果
- test_p0_diagnose_routing.py: 更新 docstring

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-05 01:49:06 +08:00
OG T
a81bf50537 feat(drift): ADR-057 adopt() Gitea PR API 實作 
- DriftAdoptService: 透過 Gitea REST API 建立 branch + commit + PR
  不在 API Pod 內執行 git（修復 C2 安全漏洞）
- adopt() 端點: 501 → 真實實作（呼叫 DriftAdoptService）
- config.py: 新增 GITEA_API_URL / GITEA_API_TOKEN / GITEA_REPO_OWNER / GITEA_REPO_NAME
- K8s secret awoooi-secrets 已注入 GITEA_API_TOKEN
- drift.py: 移除 trigger_drift_scan 中未使用的 interpreter 變數

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-05 00:39:29 +08:00
OG T
96d5e18924 fix(p0): 實測修正 — timeout 依 benchmark 調整，_local_fallback_chain 移除雲端 Nemotron 
- config.py: NEMOTRON_DIAGNOSE_TIMEOUT_SECONDS=60s (NIM 實測 11-45s + 15s buffer)
- config.py: OLLAMA_DIAGNOSE_TIMEOUT_SECONDS=200s (Ollama 實測 ~173s + 27s buffer)
- ollama.py: 新增 per-task timeout (diagnose/force_local 用 200s)
- ai_router.py: _local_fallback_chain 移除 Nemotron (NIM=雲端，不可進 local chain)
- ai_router.py: v4.2 — Option C 分情境路由正式確立

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-05 00:29:09 +08:00
OG T
3455044457 feat(phase25): Nemotron 主動防禦三方向 P0+P1+P2 完整實作
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 38s
Details
Type Sync Check / check-type-sync (push) Failing after 35s
Details 
P0 - DIAGNOSE Privacy-First Routing:
- ai_router.py: _local_fallback_chain [NEMOTRON→OLLAMA→REJECT]
- DIAGNOSE 意圖 override 改為 NEMOTRON (原 OLLAMA)
- DIAGNOSE fallback 使用 local-only 鏈，不觸碰雲端
- 全部失敗時 REJECT + Telegram 通知
- config.py: NEMOTRON_DIAGNOSE_TIMEOUT_SECONDS=30, OLLAMA_DIAGNOSE_TIMEOUT_SECONDS=60
- nemotron.py: 根據 context[task_type] 選擇 timeout

P1 - Knowledge Auto-Harvesting:
- models/knowledge.py: EntryType.AUTO_RUNBOOK + ANTI_PATTERN + symptoms_hash
- EntryStatus.PUBLISHED (ANTI_PATTERN 直接發布，無需審核)
- models/playbook.py: SymptomPattern.compute_hash() (16字元確定性 hash)
- services/runbook_generator.py: NemotronRunbookGenerator (v1.1)
  - generate_runbook() → AUTO_RUNBOOK (DRAFT) + Telegram 審核 card
  - generate_anti_pattern() → ANTI_PATTERN (PUBLISHED) + Telegram 通知
  - 使用 nvidia.chat() (正確介面)，Nemotron 超時時 Minimal fallback
- knowledge_service.py: check_anti_pattern(symptoms_hash, days=7)
- db/models.py: symptoms_hash VARCHAR(16) + ix_knowledge_symptoms_hash
- repositories/knowledge_repository.py: create() 支援 symptoms_hash + status
- auto_repair_service.py: anti_pattern_gate 在 decide() + runbook hook 在 execute()
- migrations/phase8_symptoms_hash.sql: ALTER TABLE + partial index + PUBLISHED constraint

P2 - Config Drift Detection:
- models/drift.py: DriftItem/DriftReport/DriftLevel/DriftIntent/DriftStatus
- services/drift_detector.py: GitStateReader + K8sStateReader + DriftDetector
- services/drift_analyzer.py: 白名單過濾 + DriftLevel 分級
- services/drift_interpreter.py: NemotronDriftInterpreter（意圖分析，不生成修復指令）
- services/drift_remediator.py: rollback(kubectl apply) + adopt(git push gitea)
- api/v1/drift.py: POST /scan, GET /reports, POST /rollback, POST /adopt
- migrations/phase9_drift_reports.sql: drift_reports 表
- k8s/drift-cronjob.yaml: 每小時自動掃描 CronJob

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-04 12:35:05 +08:00
OG T
c65ed5b1c9 feat(telegram): SRE 戰情室群組三頭政治 Triumvirate (ADR-053)
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 7m6s
Details 
- config.py: 新增 OPENCLAW_BOT_TOKEN / NEMOTRON_BOT_TOKEN / SRE_GROUP_CHAT_ID
- telegram_gateway.py: send_to_group / send_as_openclaw / send_as_nemotron / trigger_group_ai_discussion / _send_approval_card_to_group
- send_approval_card 告警發送後非同步觸發群組 AI 雙向討論
- configmap: SRE_GROUP_CHAT_ID=-1003711974679
- secrets: OPENCLAW_BOT_TOKEN / NEMOTRON_BOT_TOKEN CHANGE_ME 佔位

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-03 17:16:05 +08:00
OG T
73e8f8ab77 feat(ai): Phase 24-A+B1 — AI Provider Registry + 絞殺者包裝 (ADR-052)
Some checks failed
E2E Health Check / e2e-health (push) Successful in 16s
Details
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
Brain Layer 雙軌 Registry 架構:
- 新建 src/services/ai_providers/ 目錄 (interfaces + 4 providers)
  - OllamaProvider (local, rca/chat/code_review)
  - GeminiProvider (cloud, rca/chat)
  - ClaudeProvider (cloud, rca/chat/code_review)
  - OpenClawNemoProvider (cloud, rca — 委派 188→NIM)
- 擴展 ai_router.py 加入:
  - AIProviderRegistry (動態註冊/啟停)
  - AIRouterExecutor (Cache + 閘門 CB/RL/Sem + 執行)
- openclaw.py 絞殺者包裝: USE_AI_ROUTER=true 走新路徑
- config.py + ConfigMap 加入 USE_AI_ROUTER=false (安全預設)
- ADR-052 正式文件 (14 項決策 D1-D14)
- HARD_RULES v1.7 加入 AI Router 規範

安全: USE_AI_ROUTER=false 預設不啟用，需手動開啟觀察
回滾: kubectl set env deployment/awoooi-api USE_AI_ROUTER=false

2026-04-02 ogt: Phase 24 首批實作

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-02 13:16:09 +08:00
OG T
c9c60c3a61 feat(mcp-integrations): Phase S 架構修復 + MCP 整合基礎建設
Some checks failed
E2E Health Check / e2e-health (push) Has been cancelled
Details
CD Pipeline / build-and-deploy (push) Has been cancelled
Details
Type Sync Check / check-type-sync (push) Failing after 22s
Details 
Phase S 技術債修復 (首席架構師審查 82→完整):
- S-01: generate_alert_fingerprint 移至 AlertAnalyzer.generate_fingerprint() staticmethod
- S-04: 移除 Pydantic v2 deprecated json_encoders (直接用原生 datetime 序列化)

Sentry MCP 整合 (Phase 23):
- ADR-048: Sentry→OpenClaw AI Triage 架構決策
- sentry_webhook_service.py: parse/analyze/create_incident/build_message Service 層
- config.py: SENTRY_WEBHOOK_SECRET (Fail-Closed HMAC-SHA256)

Playwright MCP 整合 (短期):
- smoke.spec.ts: 5 頁面 E2E smoke test (home/dashboard/incidents/approvals/terminal)
- cd.yaml: E2E Smoke Test 步驟 + Telegram 🎭 Smoke 狀態通知

長期規劃 ADR:
- ADR-049: Figma Code Connect 設計系統同步
- ADR-050: Telegram 互動式 Incident 2.0 (6鍵 Inline Keyboard)
- ADR-051: Context7 依賴升級顧問 (Next.js 14→15, FastAPI 0.115→0.128)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-01 16:20:57 +08:00
OG T
394f85954e fix(api): 修復 Y/n 404 + 停用 Multi-Sig
Some checks failed
E2E Health Check / e2e-health (push) Has been cancelled
Details
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
1. proposal_service._load_incident() 改用 incident_service.get_from_working_memory()
   - brain engine 使用 awoooi:incidents: prefix，資料實際在 incident: prefix
   - 兩個 prefix 不符導致永遠 404 (Y/n 按鈕全部失敗)
   - 2026-04-02 ogt

2. trust_engine CRITICAL required_signatures 2→1
   - 統帥決策: 所有審核只需 1 層簽核
   - 2026-04-02 ogt

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-01 16:16:28 +08:00
OG T
eccf61fbc9 fix(ai): 修復假信心度 + 解除 Shadow Mode (Phase 22 P1)
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details
E2E Health Check / e2e-health (push) Has been cancelled
Details 
1. openclaw.py: LLM 截斷時 confidence 0.82→0.0 (禁止偽造信心度)
2. prompts.py: NEMOTRON schema 範例值改用佔位符，防模型照抄 0.75
3. configmap: SHADOW_MODE_ENABLED=false，開放 low 風險自動執行
   條件門檻: confidence≥90% + trust_score≥5 + playbook_success≥95%

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-01 15:59:42 +08:00
OG T
0fd53422c6 fix(openclaw): NEMOTRON_SYSTEM_PROMPT confidence/reasoning 移至最前
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 5m36s
Details
E2E Health Check / e2e-health (push) Successful in 17s
Details 
Nemo-4B 4B 參數模型輸出長度有限，confidence/reasoning 排在 schema 末尾
時常被截斷，導致 openclaw.py:1045 fallback 補 0.82 假數據。

修復：將 confidence 和 reasoning 移至 schema 最前兩個欄位，確保模型
輸出截斷時仍包含最關鍵欄位。同時明確禁止模型抄範例值。

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-01 13:19:18 +08:00
OG T
22de22c989 refactor(phase-s): Phase S 技術債清理 - 五項架構改善 
S-01: generate_alert_fingerprint() 移至 alert_analyzer_service (Router→Service)
S-02: 移除廢棄 USE_NEW_ENGINE config (Phase R 已完成歷史使命)
S-03: github_webhook.py linter 清理 (Field unused + delivery_id noqa)
S-04: Pydantic v2 遷移 - approval/incident models (class Config → ConfigDict)
S-05: Skill 09 v1.1 更新 (USE_NEW_ENGINE 廢棄說明)

測試: 393 passed, 零失敗

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-01 13:12:02 +08:00
OG T
2ba61acf72 fix(api): Phase R-R2.2 首席架構師 72/100 P2 修復 
P2-01 signal_worker.py: persisted_to_pg 改用 getattr 防 BrainIncident AttributeError
P2-02 IIncidentEngine Protocol: update_incident_status → update_status 對齊 brain 實作
P2-03 config.py USE_NEW_ENGINE: 標記失效 + 回滾路徑更正 (git revert 而非 kubectl)
ADR-046: Option B (IncidentConverter) 決策完成，待實作清單更新
ADR-024: 審查結論 + 正式回滾指令更新
Skill 02: v2.5 版本記錄

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-31 22:33:08 +08:00
OG T
dd526684ab feat(ai): Phase 22 OpenClaw + Nemotron 協作架構 (ADR-044)
All checks were successful
E2E Health Check / e2e-health (push) Successful in 17s
Details 
統帥批准實作「仲裁-執行分工」架構:
- OpenClaw = 仲裁者 (Why + Risk Level)
- Nemotron = 執行者 (How + kubectl Command)

新增功能:
- config.py: ENABLE_NEMOTRON_COLLABORATION Feature Flag
- openclaw.py: generate_incident_proposal_with_tools()
- openclaw.py: _call_nemotron_tools() Nemotron 呼叫
- telegram_gateway.py: TelegramMessage Nemotron 欄位
- telegram_gateway.py: format_with_nemotron() 雙區塊格式
- decision_manager.py: 整合協作方法
- proposal_service.py: 整合協作方法

觸發條件:
- LOW 風險 → 僅 OpenClaw
- MEDIUM/HIGH/CRITICAL → OpenClaw + Nemotron 雙軌

首席架構師審查: 83/100 條件通過

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-03-31 18:52:53 +08:00
OG T
d03668669b fix(openclaw): optimize for Nemo-4B with lightweight prompt and resilient parsing
All checks were successful
E2E Health Check / e2e-health (push) Successful in 26s
Details
2026-03-31 15:59:58 +08:00
OG T
fb0ddf305c fix(api): fix dockerfile to include models.json, remove huge prompt example to fit 4K limit
All checks were successful
E2E Health Check / e2e-health (push) Successful in 17s
Details
2026-03-31 14:03:34 +08:00
OG T
46843c8e19 fix(nvidia): revert to nemotron-mini, truncate context for 4K limit, enforce precise confidence
All checks were successful
E2E Health Check / e2e-health (push) Successful in 17s
Details
2026-03-31 13:57:10 +08:00
OG T
bb85d89874 refactor(api): Phase A P1 快速勝利 (3 項)
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details
E2E Health Check / e2e-health (push) Has been cancelled
Details 
1. 常數提取: SSE_DELAY_SECONDS, MAX_APPROVAL_DISPLAY
2. 錯誤訊息安全化: sanitize_error_message() 移除敏感資訊
3. CI/CD alertname 配置化: is_cicd_alertname() 函數

首席架構師審查 P1 改進 (非阻塞)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-03-30 01:44:42 +08:00
OG T
27509db212 feat(api): Wave 1 安全網 - Circuit Breaker + Global Repair Cooldown 
ADR-038: OpenClaw 雙層保護
- Layer 1: Circuit Breaker (5 failures → 60s cooldown)
- Layer 2: Concurrency Semaphore (max 3 concurrent)
- 新增 src/core/circuit_breaker.py

ADR-039: 全域修復熔斷
- Global Cooldown: 5 repairs/15min → freeze
- StatefulSet Blacklist: postgres/redis/clickhouse 禁止自動重啟
- 新增 src/services/global_repair_cooldown.py
- 整合到 auto_repair_service.py

測試:
- test_circuit_breaker.py (狀態轉換 + Semaphore)
- test_global_repair_cooldown.py (黑名單 + 計數閾值)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-03-29 15:48:03 +08:00