docs(session6): Telegram 全面審計 + ADR-072 Bug 清單 + 規格整合
- LOGBOOK: Session 6 Redis DB10 審計結果(8個系統性問題,P0-P2分級) - ADR-072: AIOps 閉環 Bug 修復清單(drift_interpreter/deployment_name/KM vectorization等) - 規格文件 v2.2: 確認 Sprint A/B/C + MCP 1-4 + ADR-071 全部完成,標記下一步為 ADR-072 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -6,6 +6,34 @@
|
||||
|
||||
---
|
||||
|
||||
## 📍 當前狀態 (2026-04-11 深夜 — Session 6 Telegram 全面審計 + 規格整合)
|
||||
|
||||
### Session 6 Redis DB10 全面審計發現 (2026-04-11 深夜)
|
||||
|
||||
**審計範圍**:62 個 DecisionToken、112 個 Incidents、Telegram 告警統計
|
||||
|
||||
| 發現 | 數字 | 根因 |
|
||||
|------|------|------|
|
||||
| DecisionToken error | 24/62 | deployment_name = "unknown"(無 component label)→ safety guard 攔截 |
|
||||
| DecisionToken ready(無人審核)| 15/62 | approval flow UUID 問題 |
|
||||
| 所有告警 = P3/High | 100% | Prometheus rules 未設 severity label |
|
||||
| incident type = "custom" | 69/112 | alertname_to_type 只有 9 筆 |
|
||||
| KM vectorized = False | 108/112 | RAG 知識庫未累積 |
|
||||
| drift_interpreter 失敗 | 100% | nvidia.chat() 返回 NvidiaProviderResult 非 4-tuple |
|
||||
| outcome/verification_result = null | 全部 | post-repair 驗證未寫入 |
|
||||
|
||||
**ADR-072 Bug 清單建立**:8 個問題,P0/P1/P2 分級
|
||||
|
||||
**規格文件整合確認**:
|
||||
- `docs/superpowers/specs/2026-04-10-infra-rebuild-sprint-abc-design.md` v2.1
|
||||
- Sprint A/B/C + MCP Phase 1-4 + ADR-071 A-J 全部 ✅(與 LOGBOOK 吻合)
|
||||
- `project_master_workplan.md` 已更新(原來錯誤顯示 🔲)
|
||||
- 執行順序確認無邏輯衝突(Section 7 依賴鏈正確)
|
||||
|
||||
**下一步**:ADR-072 P0 Bug 修復(drift_interpreter + deployment_name resolution)
|
||||
|
||||
---
|
||||
|
||||
## 📍 當前狀態 (2026-04-11 — MCP 全驗收完成 + Bug 修復)
|
||||
|
||||
### MCP 全驗收 + Bug 修復 (2026-04-11 深夜)
|
||||
|
||||
102
docs/adr/ADR-072-aiops-bug-fixes.md
Normal file
102
docs/adr/ADR-072-aiops-bug-fixes.md
Normal file
@@ -0,0 +1,102 @@
|
||||
# ADR-072: AIOps 閉環 Bug 修復清單
|
||||
|
||||
> **狀態**: 進行中
|
||||
> **建立**: 2026-04-11 (台北時間)
|
||||
> **背景**: Session 6 對 Redis DB10 + 程式碼全面審計發現 8 個系統性問題
|
||||
|
||||
---
|
||||
|
||||
## 背景
|
||||
|
||||
ADR-070/071 實作完成後,Session 6 對 62 個 DecisionToken、112 個 Incidents 及 Telegram 告警進行全面審計,
|
||||
發現 MCP 雖已實作但閉環仍未真正生效。本 ADR 記錄所有發現與修復方案。
|
||||
|
||||
---
|
||||
|
||||
## Bug 清單
|
||||
|
||||
### P0 — 立即修復(閉環核心失效)
|
||||
|
||||
#### BUG-001: drift_interpreter 永久失敗
|
||||
- **位置**: `apps/api/src/services/drift_interpreter.py:112`
|
||||
- **症狀**: 所有 K8s 漂移解析失敗,`too many values to unpack (expected 4)`
|
||||
- **根因**: `response_text, success, _tokens, _cost = await nvidia.chat(...)` — nvidia_provider 已重構為返回 `NvidiaProviderResult` 物件,非 4-tuple
|
||||
- **修復方案**: 改用 `drift_narrator_service`(ADR-067 Phase 30,qwen2.5:7b-instruct,已存在)做漂移摘要,完全繞過 nvidia_provider
|
||||
- **影響**: 所有 K8s config drift 告警無法生成可讀摘要
|
||||
|
||||
#### BUG-002: 自動修復 deployment name 全為 "unknown"
|
||||
- **位置**: `apps/api/src/services/decision_manager.py:_auto_execute()` + `apps/api/src/services/incident_service.py:extract_affected_services()`
|
||||
- **症狀**: 24/62 DecisionToken = error,safety guard 攔截(`"unknown" in action`)
|
||||
- **根因**: HostHighCpuLoad / DockerContainerUnhealthy 等主機層告警只有 `{alertname, host}` label,無 `component`/`job`/`pod` → `extract_affected_services()` 返回 `[]` → target = "unknown"
|
||||
- **修復方案**: 在 `_auto_execute()` 執行前,呼叫 K8s MCP `k8s_describe_pod` / `kubectl get pods` 根據 alert 類型動態查詢受影響 Pod,填入 target
|
||||
- **影響**: 所有非 K8s 源頭告警的自動修復全部失敗
|
||||
|
||||
#### BUG-003: nemotron_validation 接受無效 deployment name
|
||||
- **位置**: `apps/api/src/services/decision_manager.py:_nemoclaw_second_opinion()`
|
||||
- **症狀**: `<deployment_name>`、`HostHighCpuLoad`、`unknown` 等無效值通過 schema 驗證
|
||||
- **根因**: 只做 JSON schema 結構驗證,不驗證 deployment name 是否真實存在於 K8s
|
||||
- **修復方案**: 加入 K8s MCP `k8s_describe_pod` 存在性檢查,若 deployment 不存在則拒絕並重新分析
|
||||
- **影響**: 無效的修復指令被誤判為合法
|
||||
|
||||
---
|
||||
|
||||
### P1 — 本 Sprint 修復(功能性問題)
|
||||
|
||||
#### BUG-004: KM vectorization 108/112 = False
|
||||
- **位置**: `apps/api/src/services/km_conversion_service.py`
|
||||
- **症狀**: RAG 知識庫未累積,飛輪自癒閉環(ADR-068)學習效果為零
|
||||
- **根因**: KM 轉換服務呼叫後未確保 embedding 寫入 pgvector(Session 6 確認 `vectorized=False`,`persisted_to_pg=False`)
|
||||
- **修復方案**: 修復 `convert()` 後的 embedding 寫入流程,確保 `nomic-embed-text` 768d 向量化並持久化
|
||||
|
||||
#### BUG-005: 15 ready decisions 無人審核
|
||||
- **位置**: incident_id → approval UUID 對應流程
|
||||
- **症狀**: 15 個 `status=ready` 的 DecisionToken 長期無審核,Telegram 未發出審核卡片
|
||||
- **根因**: approval flow 的 `incident_id` UUID 對應問題
|
||||
- **修復方案**: 診斷 `incident_id` 查詢鏈,修復 Telegram 審核卡片未發送的問題
|
||||
|
||||
#### BUG-006: outcome/verification_result 全 null
|
||||
- **位置**: `apps/api/src/services/decision_manager.py:_push_auto_repair_result()`
|
||||
- **症狀**: 所有已完成修復無 outcome 記錄
|
||||
- **根因**: post-repair 驗證邏輯未寫入 DB outcome 欄位
|
||||
- **修復方案**: 修復 `_push_auto_repair_result()` 確保 outcome + verification_result 寫入
|
||||
|
||||
---
|
||||
|
||||
### P2 — 下 Sprint 修復(數據品質問題)
|
||||
|
||||
#### BUG-007: 所有告警 severity = P3/High(label 缺失)
|
||||
- **位置**: `k8s/monitoring/alerts-unified.yml`
|
||||
- **症狀**: 所有 62 個 DecisionToken 對應告警都是 P3(mapping: no label → P3)
|
||||
- **根因**: Prometheus 告警規則大部分未設 `severity` label
|
||||
- **修復方案**: 按告警類型補充 `severity: critical|high|warning|info` label
|
||||
|
||||
#### BUG-008: 69/112 incidents type = "custom"
|
||||
- **位置**: `apps/api/src/api/v1/webhooks.py:1103`(`alertname_to_type` 只有 9 筆)
|
||||
- **症狀**: 大部分告警無法觸發對應 Playbook
|
||||
- **根因**: `alertname_to_type` mapping 只覆蓋 9 種 K8s 告警,其他全部 → "custom"
|
||||
- **修復方案**: 整合 ADR-064 Rule Engine,從 YAML 規則動態推斷告警類型,取代靜態 9 筆 mapping
|
||||
|
||||
---
|
||||
|
||||
## 修復順序
|
||||
|
||||
```
|
||||
BUG-001 (drift_interpreter) → BUG-002 (deployment_name) → BUG-003 (nemotron_validation)
|
||||
→ BUG-004 (KM vectorization) → BUG-005 (approval flow) → BUG-006 (outcome)
|
||||
→ BUG-007 (severity labels) → BUG-008 (alertname_to_type)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 驗收標準
|
||||
|
||||
| Bug | 驗收 |
|
||||
|-----|------|
|
||||
| BUG-001 | K8s config drift 告警 → Telegram 出現可讀摘要(非錯誤訊息)|
|
||||
| BUG-002 | HostHighCpuLoad 告警 → DecisionToken status = completed(非 error)|
|
||||
| BUG-003 | 無效 deployment name → nemotron 拒絕並重新分析 |
|
||||
| BUG-004 | 新 Incident 完成後 vectorized = True + persisted_to_pg = True |
|
||||
| BUG-005 | ready decisions → Telegram 發出審核卡片 |
|
||||
| BUG-006 | 修復完成後 outcome + verification_result 不為 null |
|
||||
| BUG-007 | 新告警 severity label 正確 → P0/P1/P2 分級出現 |
|
||||
| BUG-008 | HostHighCpuLoad → incident_type = "host_cpu"(非 "custom")|
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user