Your Name
c22e5f334e
feat(km): P1-1 KMWriter 統一契約 + 5 caller 切換 + M4 反查鏈補齊
...
12-Agent 全景診斷揪出 KM 寫入鏈路 5 條入口無統一契約,fire-and-forget
在 Pod recycle 時會丟失條目。本次抽 KMWriter 強制 7 條契約。
## 7 條契約強制
1. 同步底線:強制 await asyncio.wait_for(timeout)
2. 重試:3 次指數退避 1s/2s/4s(OperationalError / 網路類例外)
3. 失敗回收:3 次後寫 Redis DLQ km:dlq + log
4. 觀測:structlog event + 預留 metric hook(P1-3 補 emitter)
5. 冪等:incident_id + path_type 為 unique key
6. 禁止吞例外:except 必須 log + raise/DLQ
7. M4 反查鏈:payload 含 approval_id 時自動填 related_approval_id 並回填 Path A
## Caller 切換(5 條入口統一介面)
- incident_service.py:1086 Path A(KB extractor + km_conversion)
- approval_execution.py:771 Path B-人工
- decision_manager.py:2178 Path B-自動成功(消除跨類私有方法調用 M1)
- decision_manager.py:2200 Path B-自動失敗(修 B2 早期吞例外)
- playbook_service.py:210 PlaybookKM(兩份 T0 報告都漏的第三條)
## M4 反查鏈補齊
- knowledge.py + models.py: 補 related_approval_id ORM 欄位
- 對齊 phase26_incident_km_integration.sql:20 schema(partial index 已存在)
- approval↔KM 雙向反查鏈完整(dual-path 縫合線)
## Feature Flag (rollback 保險)
- KM_WRITE_AWAIT=true (default): await + timeout + DLQ 強制
- KM_WRITE_AWAIT=false: fire-and-forget(舊行為)
## 測試
- apps/api/tests/test_km_writer.py: 18 測試全綠
覆蓋 success / timeout / retry / DLQ / 冪等 / KMWriteError /
on_failure=raise / 反查鏈回填
- 1552 unit tests 全綠(無回歸)
## 驗收
飛輪閉環核心 — KM 寫入不再靜默丟失,AI 學習鏈不斷裂。
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-04-29 10:44:39 +08:00
Your Name
025a493f06
feat(p3.2+adr-100): Model Version Tracker + SLO 自治 + KB rot cleaner
...
run-migration / migrate (push) Failing after 12s
CD Pipeline / build-and-deploy (push) Has been cancelled
Wave 8 P3.2 模型版本追蹤 + ADR-100 SLO 自我治理 + 配套:
P3.2 — Model Version Tracking:
- model_version_probe.py (268 行) — 探測 Ollama / OpenRouter 等 provider 的 model version
- model_version_tracker.py (101 行) — 對齊 PG provider_version_history 表
- migrations/p3_2_provider_version_history.sql + rollback — 25 行 schema
- db/models.py +32 行 — ProviderVersionHistory ORM
ADR-100 — AI 自主化 SLO:
- docs/adr/ADR-100-ai-autonomous-slo.md (167 行) — 飛輪 SLO 設計與閾值
- ops/monitoring/slo-rules.yml (254 行) — Prometheus SLO recording rules + alerts
- ops/monitoring/tests/test_slo_rules.yaml (242 行) — promtool unit tests
整合修改:
- main.py +72 行 — Lifespan 啟動 model_version_probe + KB rot cleaner schedule
- gitea_webhook.py +45 行 — webhook 接收 model 版本變化通知
- ci_auto_repair.py / evidence_snapshot.py / pre_decision_investigator.py — 配合接線
新測試:
- test_kb_rot_cleaner_schedule.py (120 行) — 9 tests pass
- test_slo_rules.yaml — promtool 驗收
Tests: 9 passed (test_kb_rot_cleaner_schedule)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-Authored-By: Multiple Engineers (P3.2 + ADR-100) <noreply@anthropic.com >
2026-04-27 14:54:19 +08:00
Your Name
cc547736ab
feat(wave6-8): P2.1 fusion + P2.2 governance + P2.4 consensus + Wave 7/8 BLOCKER 修復
...
承接 Wave 6/7/8 多 engineer 在 agent 限額前完成的代碼,補 commit 解 production
HEAD 隱性 import error(decision_fusion 已被 decision_manager 引用但檔案 untracked)。
新增(後端核心):
- decision_fusion.py (562 行) — P2.1 方法 III(OpenClaw + Hermes + Elephant 三 LLM 融合)
- aiops_timeline.py + aiops_timeline_service.py — critic B4 修復
/api/v1/aiops/timeline endpoint,DB 存取抽到 service 層遵守 leWOOOgo 積木化
- migrations/p2_decision_fusion_columns.sql + rollback — approval_records fusion 欄位
修改(後端整合):
- decision_manager.py — fusion 三斷鏈修補(critic B1+B2+B3):
· B1: 寫 _evidence_snapshot_ref 到 token.proposal_data
· B2: fusion 前計算 complexity_score 並寫 token
· B3: fusion composite 寫 token.proposal_data["decision_fusion"]
- auto_approve.py — fusion + consensus 認識(critic B3+B5):
· composite > 0.7 → auto_execute_eligible bypass min_confidence
· source=consensus_engine + score>=0.6 → 規則可信路徑
- consensus_engine.py — db-fix _save_consensus 重用 agent_sessions
- governance_agent.py — db-fix _alert PG 寫入 ai_governance_events
- approval_db.py — fusion 3 欄位 + 2 partial index + CheckConstraint
- db/models.py — schema 對齊 migration
- core/config.py — vuln #1 修復:OLLAMA_URL/_FALLBACK_URL field_validator
拒絕公網 IP + 外部域名,僅允許私網/loopback/K8s SVC 白名單
- core/feature_flags.py — P2 fusion + consensus flags
- main.py — governance_agent lifespan 啟動
- failover_alerter.py — Wave8-X2: in-memory dedup fallback(Redis 拒絕後不 fail-open)
- ollama_*.py — metrics 整合 + recovery 改善
- auto_repair_service.py — verifier 接線
新增(測試 2438 行):
- test_decision_fusion.py / test_governance_agent.py / test_consensus_integration.py
- test_p2_db_fixes.py / test_wave8_fusion_fixes.py
- test_config_url_validation.py(vuln #1 12 tests)
- test_failover_alerter.py +Wave8-X2 in-memory dedup 補測
驗收: 116 tests pass (decision_fusion + wave8_fusion + config_url + consensus +
governance + p2_db_fixes + failover_alerter)
Conflict resolution:
- 3 檔(config.py + auto_approve.py + decision_manager.py)git stash pop 衝突
保留 stashed (engineer 最終版),補回 ValueError 「公網 IP」字樣對齊 test
Note: 此 commit 解 production HEAD 隱性 import error
仍未修: vuln #4 prompt injection / debugger B14 quota fail-closed
/ B25-B26 drain_pending_tasks / B8 governance fail alert
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-Authored-By: Multiple Engineers (Wave 6/7/8) <noreply@anthropic.com >
2026-04-27 08:11:40 +08:00
Your Name
e96055eef9
fix(p0.4): Playbook 學習鏈三道修復 — partial index + race防護 + 手動路徑接線
...
ADR-092 P0.4 Playbook EWMA 學習閉環的 DB / Repository / Service 三層修補。
DB 層 (db-expert-fix by Engineer-B):
- ApprovalRecord.matched_playbook_id 移除 index=True,改 __table_args__ partial index
(WHERE matched_playbook_id IS NOT NULL) — 多數列 NULL,full index 浪費空間
- adr092_p1_learning_chain_rollback.sql: 純 ROLLBACK SQL(DBA 手動執行)
Repository 層:
- playbook_repository.py: SELECT FOR UPDATE 防 lost update
避免並發 EWMA 更新覆蓋彼此
Service 層 (P0.4 修復):
- proposal_service.py: 手動審核路徑補 _try_playbook_match_id 呼叫
decision_manager auto_execute 路徑已有此邏輯(行 2035),
此處補手動路徑缺口,使 matched_playbook_id 可寫入 DB → EWMA 才能演化
測試:
- test_playbook_repository_race_condition.py: 3 cases SELECT FOR UPDATE 防 race
正確阻擋並發 EWMA 更新(pass)
Note: migration SQL 待 DBA 手動執行(feedback_dev_prod_separation.md),
不執行 alembic upgrade(statu 文件禁忌條款)。
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-04-26 20:19:46 +08:00
Your Name
6d5fd3c124
feat(ws2): ADR-093 路由統一 — BIGINT + NotificationMatrix + feature flag
...
## 修復
### T2.1 BigInteger overflow 修復
- `db/models.py`: telegram_chat_id Integer → BigInteger
(原 int32 無法容納群組 ID -1003711974679)
### T2.2 移除 CAST workaround
- `approval_db.py:739`: 移除 CAST(:telegram_chat_id AS BIGINT)
ORM 已正確使用 BigInteger,workaround 可退役
### T2.3 Redis key 一致性修復
- `heartbeat_report_service.py:575`: telegram:polling_leader → telegram:polling:leader
(telegram_gateway.py 使用冒號分隔,heartbeat 用底線是 bug)
## 新增
### T2.4 notification_matrix.py
- `services/notification_matrix.py`: ADR-093 路由矩陣
- Destination(DM/GROUP/BOTH) + RoutingRule dataclass
- NOTIFICATION_ROUTING dict(TYPE-1 ~ TYPE-8M 完整映射)
- resolve_chat_ids(type, dm, group, *, tg_group_cutover=False) 灰階切流 API
### T2.5 telegram_gateway.py feature flag 保護
- line 43: 加 notification_matrix import
- line 1827-1834: TG_GROUP_CUTOVER=False 時維持舊行為
TG_GROUP_CUTOVER=True 時解除 _interactive_types 黑名單,由矩陣控制
### T2.6 Migration SQL
- `migrations/adr093_notification_routing.sql`:
- CREATE TABLE approval_records (telegram_chat_id BIGINT)
- CREATE ROLE awoooi_migrator (IF NOT EXISTS)
- 含舊環境 ALTER COLUMN int→bigint 保護
## 測試同步
- `tests/integration/setup_test_schema.sql`: telegram_chat_id BIGINT
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-25 02:10:06 +08:00
Your Name
04ff22563e
fix(aiops-p1): Playbook 學習閉環 5斷點全修 + DB Migration(ADR-092 B4)
...
run-migration / migrate (push) Failing after 14s
CD Pipeline / build-and-deploy (push) Failing after 2m7s
【P0.4 補丁】pre_decision_investigator Prometheus query 欄位缺失
- _build_tool_params() 補 "query" 欄位(prometheus_query tool 必要參數)
- 新增 _build_prometheus_query() — 依告警類型生成 PromQL(CPU/Memory/Crash/Disk/HTTP/Pod/fallback)
- 修復後 D3_METRICS 感官維度實際取得資料(原本 100% 回 missing_query_parameter)
【P1 Playbook 學習閉環 B1-B5 全修】
- B2 db/models.py: ApprovalRecord 新增 matched_playbook_id 欄位 + ix_approval_matched_playbook index
- B2 db/models.py: TimelineEvent 新增 incident_id 欄位(MCP 稽核用)+ index
- B3 approval_db.py: record→ApprovalRequest 補回 incident_id + matched_playbook_id
- B4 approval_repository.py: 同 B3(兩個轉換函式必須同步)
- B5 approval_db.py: approval_request_to_record_data 補 matched_playbook_id → DB 才能存值
【P1.5 KM 寫入】approval_execution.py: fire-and-forget → await wait_for(30s)
- 根因:asyncio.create_task 在 Pod recycle 時被殺,KM 寫入靜默遺失
- 修復:await asyncio.wait_for(..., timeout=30.0) + TimeoutError log
【Migration 文件】adr092_p1_learning_chain_fix.sql
- ALTER TABLE approval_records ADD COLUMN matched_playbook_id VARCHAR(36)
- ALTER TABLE timeline_events ADD COLUMN incident_id VARCHAR(64)
- 執行:psql $DATABASE_URL -f apps/api/migrations/adr092_p1_learning_chain_fix.sql
【附帶 Agent 改動】
- decision_manager: Phase 2 YAML NO_ACTION 優先門(主機層/外部服務跳過 Agent Debate)
- alert_rules.yaml: Sentry/ClickHouse + HostDiskUsageHigh/Critical 新規則
- solver_agent: action_title 語意合成兜底(取代靜默丟棄)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-24 15:41:35 +08:00
OG T
9d6aa7ea45
feat(trust): ADR-088 Trust Score 持久化 — L4 自動放行核心
...
CD Pipeline / build-and-deploy (push) Successful in 10m40s
TrustScoreManager 從記憶體升級為 PostgreSQL 持久化,
Pod 重啟後信任分數不再歸零,AI 能真正累積到 L4 自動放行門檻。
變更:
- migrations/adr088_trust_score_persistence.sql: trust_records 表
- db/models.py: TrustRecordDB ORM model
- repositories/interfaces.py: ITrustRepository Protocol
- repositories/trust_repository.py: PG upsert ON CONFLICT DO UPDATE
- services/trust_engine.py: bulk_load() 啟動 warm-up
- services/learning_service.py: _persist_trust() + 2 call sites
- main.py: 啟動時 load_all() → bulk_load()
流程: 批准 5 次 → score=5 寫入 DB → Pod 重啟 → warm-up 讀回
→ evaluate_adjusted_risk MEDIUM→LOW → 自動執行
2026-04-17 ogt + Claude Sonnet 4.6(亞太): ADR-088
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-17 16:14:44 +08:00
OG T
f9ba200638
fix(db): Phase 6 migration 三條 CREATE INDEX 拆開各自 execute
...
CD Pipeline / build-and-deploy (push) Has been cancelled
asyncpg 不支援 prepared statement 內多條 SQL 指令,
原本一個 text("""...""") 包含三條 CREATE INDEX 導致 CrashLoopBackOff。
拆成三個獨立 conn.execute() 呼叫。
2026-04-15 ogt + Claude Sonnet 4.6(亞太)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 19:37:58 +08:00
OG T
fab65e7d7a
fix(alerts): PENDING 收斂無 TTL → 老記錄永久封鎖 Telegram 告警
...
CD Pipeline / build-and-deploy (push) Has started running
根因:find_by_fingerprint 的 PENDING 匹配條件無時間上限,
2026-04-12 建立的 3 筆 PENDING approval records(hit=77/30/17)
持續吃掉所有同指紋告警,造成 2+ 小時 Telegram 靜音。
修正(approval_db.py):
- PENDING_TTL_HOURS = 24:PENDING 記錄逾 24h 不再收斂新告警
- 原本:OR(status=PENDING, created_at>=30min前)
- 修正:OR(PENDING AND created_at>=24h前, created_at>=30min前)
緊急修復:kubectl exec 直接將 7 筆過期 PENDING 記錄設為 expired,
即時恢復 Telegram 告警流(不等部署)。
Phase 6 AI 自我治理閉環(ADR-087):
- feat(db): 新增 ai_governance_events 表 + 3 個 index(base.py + models.py)
- feat(svc): ai_slo_calculator.py — 7d 滾動 SLO(success/override/false_neg)
- feat(svc): trust_drift_detector.py — Playbook 信任度極端偏態偵測
- feat(job): kb_rot_cleaner.py — K8s API/Prom metric/老舊 incident_case 腐爛清理
- feat(svc): decision_manager.py — 自我降級守衛(SLO 違反 → 提高門檻/保守模式)
2026-04-15 ogt + Claude Sonnet 4.6(亞太)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 18:56:26 +08:00
OG T
4e2e6652e3
fix(db): 移除 IncidentEvidence.incident_id 的重複 index 定義
...
CD Pipeline / build-and-deploy (push) Successful in 14m50s
根本原因:incident_id 同時設定 index=True(mapped_column)
與 __table_args__ 中的 Index("ix_incident_evidence_incident_id"),
導致 table.create 生成重複的 CREATE INDEX,
觸發 "already exists" 被靜默捕捉,整個 CREATE TABLE transaction 回滾。
直接效果:Pod 啟動時 incident_evidence 表永遠不會被建立,
導致後續 ALTER TABLE 失敗 → CrashLoopBackOff。
修法:移除 mapped_column 中的 index=True,
索引由 __table_args__ 統一管理。
注意:已在 PostgreSQL 直接建立 incident_evidence 表解除 CrashLoop。
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 16:13:18 +08:00
OG T
14a02263ae
feat(Phase 4): 主動巡檢 + 趨勢預測 + 8D 感官升級 全部完成
...
CD Pipeline / build-and-deploy (push) Failing after 12m32s
## Phase 4 完整交付(ADR-084)
### 新增服務
- trend_predictor.py: numpy 線性回歸,4h 閾值突破預警,R² 信心評分
- proactive_inspector.py: 每 5 分鐘主動巡檢協調器
- DynamicBaselineService(3σ 偏離)
- LogAnomalyDetector(新 Drain3 pattern)
- TrendPredictor(斜率外推 4h 預測)
- Shadow Mode + 30 分鐘去重 + Holt-Winters 背景重訓
### 8D 感官升級(EvidenceSnapshot Phase 4 增強)
- PreDecisionInvestigator._collect_phase4_anomalies(): 決策前讀取
ProactiveInspector 最近巡檢快取 + LogAnomalyDetector 新 pattern
- EvidenceSnapshot.anomaly_context: 新欄位,Phase 4 動態異常上下文
- DiagnosticianAgent._build_prompt(): prompt 包含 anomaly_context,
LLM RCA 可參考動態基線偏差與趨勢預警
### 資料庫遷移
- incident_evidence: ADD COLUMN anomaly_context JSONB(冪等)
### main.py
- 啟動 run_proactive_inspector_loop() asyncio task
2026-04-15 ogt + Claude Sonnet 4.6(亞太): Phase 4 全部完成
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 15:47:05 +08:00
OG T
952c10955b
fix(db): 多 replica 並行啟動競爭 — 每 table 獨立 tx + DROP INDEX IF EXISTS
...
CD Pipeline / build-and-deploy (push) Has been cancelled
根因:單一大 transaction 內兩個 pod 同時建同一個 table,
其中一個 CREATE INDEX 失敗 → 整個 transaction ROLLBACK
→ table 也消失 → 下次重啟同樣情況 → 無限 CrashLoop。
修法三層:
1. 每個 table 用獨立 transaction 建立(失敗不影響其他)
2. 建 table 前先 DROP INDEX IF EXISTS 清殘留孤兒 index
3. 捕捉 "already exists" 讓並行 pod 優雅跳過(不 crash)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 15:38:43 +08:00
OG T
bf45b80bd2
feat(Phase 3.5 + Phase 4): AI 學習成果持久化到 PostgreSQL — 修正「AI 失憶」架構缺陷
...
CD Pipeline / build-and-deploy (push) Has been cancelled
ADR-085: AI 學習成果不可存在 Cache
架構鐵律確立:
- PostgreSQL = System of Record(AI 的永久記憶)
- Redis = Warm Cache(加速讀取,TTL 到期從 PG 復原)
核心變更:
1. models.py: 新增 PlaybookRecord / DynamicBaselineRecord / LogClusterRecord ORM
2. base.py: ALTER TABLE playbooks 補加 trust_score / requires_approval_level 等欄位
3. playbook_repository.py: 完整雙寫實作(PG upsert + Redis cache)
4. dynamic_baseline_service.py: Holt-Winters 訓練結果寫入 PG,Redis 只作 24h warm cache
5. log_anomaly_detector.py: Drain3 cluster template 寫入 PG(UPSERT on cluster_id)
6. main.py: 啟動時執行 backfill_redis_to_pg()(Redis → PG 冪等補救)
修正的問題:
- Playbook 7天 Redis TTL 到期 → AI 失去所有修復知識
- trust_score EWMA 隨 Redis TTL 歸零 → AI 重新回到初始信任度 0.3
- Holt-Winters 基線 24h TTL → AI 每天重新學習「正常」的定義
- Drain3 cluster 沒有持久化 → AI 把已知 log pattern 反覆當新 pattern
Phase 4 新服務(requirements.txt 已加入 statsmodels + drain3 + numpy)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 15:34:04 +08:00
OG T
0f2ec7987c
fix(db): 改用 inspect 跳過現有 table,根治 CrashLoopBackOff
...
CD Pipeline / build-and-deploy (push) Failing after 14m42s
checkfirst=True 只跳過 CREATE TABLE,SQLAlchemy 2.0 仍對
__table_args__ Index 物件發出獨立 CREATE INDEX → duplicate error。
改法:先 inspect 取得現有 tables,只對不存在的 table 呼叫
table.create(),index 永遠只隨新 table 建立,不再 duplicate。
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 15:18:25 +08:00
OG T
a142e6e937
fix(db): create_all checkfirst=True 修復 CrashLoopBackOff
...
CD Pipeline / build-and-deploy (push) Failing after 12m19s
rolling update 時 create_all 嘗試重建既有 index 導致
"ix_incident_evidence_incident_id already exists" 啟動失敗。
checkfirst=True 讓 SQLAlchemy 跳過已存在的 table/index,
init_db() 從此冪等,不再造成 CrashLoopBackOff。
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 15:00:49 +08:00
OG T
5ddba6d6e0
feat(adr-082): Phase 2 多 Agent 協作 — 5 角色辯證系統骨架上線
...
新增 5 個 Agent + Orchestrator + DecisionManager 接線:
- protocol.py: DiagnosisReport / ActionPlan / ReviewVerdict / CriticReport / DecisionPackage 型別系統
- DiagnosticianAgent: RCA 根因分析,confidence < 0.4 → ABSTAIN
- SolverAgent: 修復方案軍師,blast_radius 評分 + 降級 rule-based mock
- ReviewerAgent: 安全審查,HARD_RULES 靜態 pattern + blast_radius 閾值 (>50 revision, >80 reject)
- CriticAgent: 刻意唱反調,強制 3 問批判性思維,critical challenge → REJECT
- CoordinatorAgent: 純規則聚合,6 級決策閘,REQUEST_REVISION → 強制人工
- AgentOrchestrator: 30s 全局超時,Reviewer ‖ Critic 並行,DB Immutable Event Sourcing + Redis Streams
- DecisionManager: AIOPS_P2_ENABLED gate + _package_to_proposal_data 橋接既有 proposal_data 格式
- AgentSession DB table + 4 個複合 index
- ADR-082 決策記錄
Gate 2 修復(7 項):
- CRITICAL: DELETE FROM regex lookahead 位置錯誤(移至 FROM 後)
- CRITICAL: REQUEST_REVISION 可抵達 auto-execute 路徑(改回 requires_human_approval=True)
- IMPORTANT: _extract_json flat regex 不支援巢狀 JSON(改 find/rfind 邊界提取)
- IMPORTANT: all_degraded 遺漏 verdict.degraded(補全 4 個 Agent)
- IMPORTANT: Solver ABSTAIN guard 放行降級假設(改為無論 hypotheses 有無均跳過)
- IMPORTANT: dataclasses.asdict() Enum 未序列化導致 DB 寫入靜默失敗(加 json.dumps default handler)
- IMPORTANT: P2 gate 直讀屬性繞過父 Phase 守衛(改用 is_phase_enabled(2))
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 13:48:55 +08:00
OG T
f1cbf6db7d
feat(adr-081): Phase 1 感官縱深 — 8D 情報蒐集 + 執行後驗證
...
成品:
- IncidentEvidence DB model(8D 感官 + pre/post 執行狀態)
- EvidenceSnapshot dataclass(build_summary → LLM 上下文)
- SanitizationService(Prompt Injection 0-tolerance,12 pattern)
- MCPToolRegistry(動態工具登記,suggest_tools 不寫死告警類型)
- PreDecisionInvestigator(8D 並行感官,P99 < 8s,Redis 30s 快取)
- PostExecutionVerifier(warmup 10s → 後狀態評估 success/degraded/failed)
- decision_manager + approval_execution 接線(feature flag 守衛)
Gate 1 修復:D4/D5/D7/D8 補 sanitize_dict_values;移除裸 "error" failure
signal 防 error_rate key 誤判;evidence_snapshot rowcount 零行警告。
測試:130 passed(+111 新增)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-15 13:08:38 +08:00
OG T
d4b8b1588b
feat(flywheel): Phase 2-2/2-4 — classify_alert_early + alertname/notification_type/alert_category 寫入
...
ADR-073 Phase 2-2: 早期分診,在 LLM 分析前決定 alert_category + notification_type
- webhooks.py: 新增 classify_alert_early() — 6 條規則覆蓋 config_drift/info/backup/infra/k8s/db/general
- webhooks.py: alertmanager_webhook 呼叫 classify_alert_early() 並傳入兩個 create_incident_for_approval() 呼叫點
- incident_service.py: create_incident_for_approval() 新增 notification_type/alert_category 參數,寫入 Incident model
- incident_repository.py: _incident_to_record_data() 新增 alertname/notification_type/alert_category 序列化
- db/models.py: IncidentRecord ORM 新增 alertname/notification_type/alert_category 三個 mapped_column
防止 HostBackupFailed 等告警被誤路由到 K8s executor (ADR-073 Phase 2-4 同步完成)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-12 14:33:11 +08:00
OG T
9846a6cc93
feat(incident): Phase 27 frequency_snapshot DB 持久化 — incidents 表新增 JSONB 欄位
...
CD Pipeline / build-and-deploy (push) Has been cancelled
frequency_stats 原僅存 Redis(TTL 35天),Pod 重啟或超期即失
- incidents.frequency_snapshot JSONB:建立 incident 時寫入快照,永久保存
- incident_repository: _record_to_incident 還原 IncidentFrequencyStats
- _incident_to_record_data 序列化 frequency_stats 快照到 DB
- Migration: phase27_incident_frequency_snapshot.sql 已執行完成
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-10 01:05:41 +08:00
OG T
896bef94ee
fix(web): pending-approvals-card 加防重複點擊 + loading 狀態
...
linter 自動強化: actioningId state 防止同一張卡重複操作
- disabled + opacity 0.6 + cursor not-allowed
- loading 時按鈕顯示 '...'
- finally() 確保 actioningId 清除
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-09 18:38:08 +08:00
OG T
1483218bab
feat(approval): 批准/拒絕後立即回應 Telegram + 持久化 message_id 到 DB
...
CD Pipeline / build-and-deploy (push) Successful in 13m9s
問題:按下 TG 批准/拒絕按鈕後完全沒有任何回應,使用者不知道是否成功。
Telegram message_id 只存 Redis 24h TTL,過期後無法追蹤。
修正:
- approval_records 加 telegram_message_id / telegram_chat_id 欄位(已 ALTER TABLE)
- approval_db.update_telegram_message() — 持久化 message_id 到 DB
- decision_manager: 發送告警卡片後同時寫 Redis + DB
- telegram_gateway._notify_approval_result() — 批准/拒絕後:
1. editMessageReplyMarkup 移除批准/拒絕按鈕,保留資訊按鈕
2. sendMessage reply_to 在原訊息下回覆狀態行
3. fallback: send_notification 發新訊息
- _handle_group_command: chat_id 改為 _chat_id 消除 IDE lint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-09 18:19:31 +08:00
OG T
6f475000f6
fix(db): alert_operation_log.event_type String→PgEnum (create_type=False)
...
CD Pipeline / build-and-deploy (push) Has been cancelled
修正 DatatypeMismatchError: DB 欄位為 native enum alert_event_type,
SQLAlchemy model 誤用 String(50),導致 alert_operation_log 寫入失敗。
使用 PgEnum(create_type=False) 讓 SQLAlchemy 映射已存在的 DB enum,
不重建型別。18 個 event_type 值與 M-003 migration 一致。
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-08 22:42:36 +08:00
OG T
88696dba9b
feat(sprint5.1): Data Safety Guardrails 全鏈路整合 (L1-L5)
...
CD Pipeline / build-and-deploy (push) Failing after 1m33s
Type Sync Check / check-type-sync (push) Failing after 58s
Layer 0 - K8s RBAC:
- k8s/rbac/api-velero-reader.yaml: awoooi-executor SA Velero backup reader
Layer 1 - DB Migration (已在 188 執行):
- M-002: approval_records 新增 approval_level/votes/required_votes
- M-003: alert_event_type ENUM 新增 8 個值
Layer 2 - IaC:
- ops/config/service-registry.yaml: 全服務 Stateful 分級清單 (BLOCK/CRITICAL_HITL/STANDARD_HITL/AUTO)
Layer 3 - Python Services:
- service_registry.py: 讀取 YAML,提供 is_blocked/requires_multisig/get_required_votes
- velero_client.py: kubectl 查詢 Velero 備份年齡,失敗 fallback 999h
- preflight_service.py: Pre-flight 安全檢查 (Q2/Q4 決策)
Layer 1-M001 - Playbook model:
- playbook.py: 新增 requires_approval_level/stateful_targets/requires_pre_backup
Layer 4 - 業務邏輯:
- alert_operation_log_repository.py: 新增 8 個 event_type (Guardrail/Pre-flight/MultiSig/備份)
- auto_repair_service.py: 注入 Service Registry Guardrail 檢查 (BLOCK → 直接拒絕)
- webhooks.py: ALERT_RECEIVED 溯源記錄 + auto_repair flag Q9 + Langfuse trace_id Q10
- db/models.py: ApprovalRecord 同步 approval_level/votes/required_votes 欄位
- docker-health-monitor.sh: 純感知層改造(移除所有 docker restart 邏輯)
Layer 5 - Telegram 通知:
- telegram_gateway.py: T1-T6 六個新通知方法 (Guardrail/Pre-flight/Backup/MultiSig/ChangeApplied)
參考: ADR-062 Data Safety Guardrails, ADR-063 Service Registry IaC
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-08 16:24:09 +08:00
OG T
f20121ad41
feat(audit): Phase 11 告警操作完整溯源 — alert_operation_log + 歷史回填
...
CD Pipeline / build-and-deploy (push) Failing after 1m29s
統帥指令「所有告警訊息通通寫入資料庫,並記錄相關操作」
變更:
- phase11_alert_operation_log.sql: 新表 (Event Sourcing,不可變)
- phase11b_backfill_alert_operation_log.sql: 歷史回填 654 筆
- 14 筆 ALERT_RECEIVED (incidents)
- 265 筆 TELEGRAM_SENT (approval_records)
- 265 筆 USER_ACTION (approval_records)
- 110 筆 EXECUTION_COMPLETED (audit_logs)
- db/models.py: AlertOperationLog SQLAlchemy model
- repositories/alert_operation_log_repository.py: append/list_by_incident/get_stats
- webhooks.py: _try_auto_repair_background 寫入 AUTO_REPAIR_TRIGGERED + EXECUTION_COMPLETED + TELEGRAM_RESULT_SENT
- webhooks.py: _push_to_telegram_background 寫入 TELEGRAM_SENT
- telegram.py: handle_callback 寫入 USER_ACTION (approve/reject)
已執行 migration: awoooi_prod@192.168 .0.188 ✅
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-08 11:22:03 +08:00
OG T
eee6f06215
feat(auto-repair): 所有操作強制寫入 DB — auto_repair_executions 表
...
CD Pipeline / build-and-deploy (push) Failing after 1m32s
統帥指令: 所有自動修復操作(成功/失敗)必須持久化
變更:
- migrations/phase10_auto_repair_executions.sql: 新增表 + 4 個索引
- db/models.py: 新增 AutoRepairExecution SQLAlchemy model
- repositories/audit_log_repository.py: 新增 AutoRepairExecutionRepository (create/list_by_incident/get_stats)
- auto_repair_service.py: execute_auto_repair 成功/失敗分支都寫入 DB
- 新增 similarity_score 參數傳遞
- AutoRepairDecision 新增 similarity_score 欄位
- webhooks.py: 傳入 similarity_score 到 execute_auto_repair
已執行 migration: awoooi_prod@192.168 .0.188:5432 ✅
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-08 11:16:37 +08:00
OG T
658337ec18
fix(phase26): 打通 Incident→DB→KM 完整鏈路 + namespace 修正
...
CD Pipeline / build-and-deploy (push) Failing after 1m29s
Type Sync Check / check-type-sync (push) Failing after 52s
問題根因:
1. create_incident_for_approval 只存 Redis,不存 PostgreSQL
→ TTL 7天後消失,Playbook 萃取永遠找不到 Incident
2. ApprovalRecord 無 incident_id 欄位
→ _trigger_playbook_extraction 靠 regex 掃中文文字找 INC-,永遠失敗
3. operation_parser namespace fallback 是 "default"
→ 所有 deployment 在 awoooi-prod,203 次執行全失敗
修復:
- Incident 同時寫入 Redis + PostgreSQL (save_to_episodic_memory)
- ApprovalRecord 加入 incident_id 欄位 (model + ORM + migration)
- alertmanager_webhook 建立 Approval 後回寫 incident_id
- _trigger_playbook_extraction 直接用 approval.incident_id
- operation_parser DEFAULT_NAMESPACE = "awoooi-prod"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-06 11:46:05 +08:00
OG T
3455044457
feat(phase25): Nemotron 主動防禦三方向 P0+P1+P2 完整實作
...
CD Pipeline / build-and-deploy (push) Failing after 38s
Type Sync Check / check-type-sync (push) Failing after 35s
P0 - DIAGNOSE Privacy-First Routing:
- ai_router.py: _local_fallback_chain [NEMOTRON→OLLAMA→REJECT]
- DIAGNOSE 意圖 override 改為 NEMOTRON (原 OLLAMA)
- DIAGNOSE fallback 使用 local-only 鏈,不觸碰雲端
- 全部失敗時 REJECT + Telegram 通知
- config.py: NEMOTRON_DIAGNOSE_TIMEOUT_SECONDS=30, OLLAMA_DIAGNOSE_TIMEOUT_SECONDS=60
- nemotron.py: 根據 context[task_type] 選擇 timeout
P1 - Knowledge Auto-Harvesting:
- models/knowledge.py: EntryType.AUTO_RUNBOOK + ANTI_PATTERN + symptoms_hash
- EntryStatus.PUBLISHED (ANTI_PATTERN 直接發布,無需審核)
- models/playbook.py: SymptomPattern.compute_hash() (16字元確定性 hash)
- services/runbook_generator.py: NemotronRunbookGenerator (v1.1)
- generate_runbook() → AUTO_RUNBOOK (DRAFT) + Telegram 審核 card
- generate_anti_pattern() → ANTI_PATTERN (PUBLISHED) + Telegram 通知
- 使用 nvidia.chat() (正確介面),Nemotron 超時時 Minimal fallback
- knowledge_service.py: check_anti_pattern(symptoms_hash, days=7)
- db/models.py: symptoms_hash VARCHAR(16) + ix_knowledge_symptoms_hash
- repositories/knowledge_repository.py: create() 支援 symptoms_hash + status
- auto_repair_service.py: anti_pattern_gate 在 decide() + runbook hook 在 execute()
- migrations/phase8_symptoms_hash.sql: ALTER TABLE + partial index + PUBLISHED constraint
P2 - Config Drift Detection:
- models/drift.py: DriftItem/DriftReport/DriftLevel/DriftIntent/DriftStatus
- services/drift_detector.py: GitStateReader + K8sStateReader + DriftDetector
- services/drift_analyzer.py: 白名單過濾 + DriftLevel 分級
- services/drift_interpreter.py: NemotronDriftInterpreter(意圖分析,不生成修復指令)
- services/drift_remediator.py: rollback(kubectl apply) + adopt(git push gitea)
- api/v1/drift.py: POST /scan, GET /reports, POST /rollback, POST /adopt
- migrations/phase9_drift_reports.sql: drift_reports 表
- k8s/drift-cronjob.yaml: 每小時自動掃描 CronJob
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-04 12:35:05 +08:00
OG T
db1aed81d9
fix(db): C1 時區統一遷移 — utc_now → taipei_now (全 5 table)
...
E2E Health Check / e2e-health (push) Successful in 18s
CD Pipeline / build-and-deploy (push) Has been cancelled
🔴 首席架構師審查 C1: 全系統禁止 UTC,必須台北時區 +8
- utc_now() → taipei_now() (調用 src.utils.timezone.now_taipei)
- 影響: ApprovalRecord, TimelineEvent, AuditLog, IncidentRecord, KnowledgeEntryRecord
- 13 處 default/onupdate 全部替換
- 移除 datetime.UTC import
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-02 09:13:36 +08:00
OG T
628387de8c
fix: risklevel migration 自動化 + Telegram Whitelist 注入
...
E2E Health Check / e2e-health (push) Successful in 17s
CD Pipeline / build-and-deploy (push) Has been cancelled
1. init_db() 啟動時自動確保 risklevel enum 包含 'high' 值
(Phase 23 新增,避免舊 DB 缺值導致 InvalidTextRepresentation)
2. CD Pipeline 新增 OPENCLAW_TG_USER_WHITELIST 自動注入
(之前為 CHANGE_ME,已更新為實際 user ID 5619078117)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-02 09:13:13 +08:00
OG T
48a0bc66f7
fix(api): KB 首席架構師審查修復 (I1 tags filter + I2 type annotation)
...
E2E Health Check / e2e-health (push) Successful in 16s
CD Pipeline / build-and-deploy (push) Has been cancelled
- I1: Repository list_entries 實作 tags JSONB @> 篩選 (之前聲明未實作)
- I2: ORM tags 型別從 list[dict[str, Any]] 修正為 list[str]
首席架構師審查: 87/100
C1 時區(UTC→Taipei) 為既有系統性問題,另開 task 統一遷移
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-02 09:04:41 +08:00
OG T
d8be78b135
feat(api): Knowledge Base Phase 1 後端四層架構
...
CD Pipeline / build-and-deploy (push) Successful in 7m0s
E2E Health Check / e2e-health (push) Successful in 17s
Type Sync Check / check-type-sync (push) Failing after 30s
- models/knowledge.py: Pydantic Schema (EntryType/Source/Status/CRUD)
- db/models.py: KnowledgeEntryRecord ORM (PostgreSQL)
- repositories/interfaces.py: IKnowledgeRepository Protocol
- repositories/knowledge_repository.py: PostgreSQL CRUD 實作
- services/knowledge_service.py: 業務邏輯 (get_db_context 內部管理 session)
- api/v1/knowledge.py: REST Router (get_knowledge_service,無直接 DB 存取)
- main.py: 掛載 Knowledge Base Router
- k8s/jobs/migrate-knowledge-entries.yaml: DB Migration Job
API 端點: GET/POST / | GET/PATCH/DELETE /{id} | POST /{id}/approve
GET /search | GET /categories
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-02 00:55:56 +08:00
OG T
30153496d1
fix(api): 修復全部 lint 錯誤 (ruff --fix)
...
- Import sorting (I001)
- Unused imports (F401)
- f-string without placeholders (F541)
- Loop variable unused (B007)
- zip() strict parameter (B905)
- Exception chaining (B904)
- collections.abc imports (UP035)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-03-26 16:06:20 +08:00
OG T
9bff46a1b0
feat: integrate Sentry + fix CI/CD issues
...
Sentry Integration (補強 SignOz):
- Add @sentry/nextjs for frontend error tracking + session replay
- Add sentry-sdk[fastapi] for backend error tracking
- Create sentry.client/server/edge.config.ts
- Integrate with next.config.js + instrumentation.ts
- Add Sentry exception capture in FastAPI error handler
- Create deployment scripts for Self-Hosted @ 192.168.0.110
CI/CD Fixes:
- Fix F821 Undefined name 'Field' in incidents.py
- Add NEXT_PUBLIC_API_URL env var to CI build step
- Add build-arg to Docker build verification
E2E Test Improvements:
- Fix strict mode violations in dashboard-acceptance tests
- Add timeout increase for Phase 4 demo tests
- Make tests more resilient to UI variations
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-03-24 15:19:52 +08:00
OG T
8159d22db9
refactor: ClawBot → OpenClaw 全域更名
...
- 刪除舊版 clawbot.py (已有新版 openclaw.py)
- 更新 models/ai.py 類型定義 (ClawBotAnalysisRequest/Response)
- 更新 api/v1/ai.py import 與註解
- 更新 Discord username
- 更新所有註解與文檔
依據: feedback_openclaw_naming.md (統帥 2026-03-20 正式命名決議)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-03-24 12:57:36 +08:00
OG T
6f049877fc
fix(lint): ruff auto-fix + lewooogo-core src 加入 git
...
- Python: ruff --fix 修復 280 個 lint 錯誤
- lewooogo-core: src/ 目錄未追蹤,導致 CI eslint 失敗
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-03-23 23:51:37 +08:00
OG T
7478dc0254
feat(phase6-9): Complete modular architecture and Agent Teams
...
Phase 6.4 - Modular Architecture:
- Add lewooogo-brain adapters for LLM providers
- Add lewooogo-data dual memory (Redis + PostgreSQL)
- Implement consensus engine for multi-agent decisions
- Add incident memory service for historical context
Phase 9 - Agent Teams (Claude Agent SDK):
- Add base agent class with Claude Sonnet 4 integration
- Implement action planner, blast radius, and security agents
- Add agent API endpoints and proposal workflow
- Integrate ADR-009 OpenClaw Agent Teams architecture
DevOps & CI/CD:
- Add GitHub Actions CI/CD workflows (ci.yaml, cd.yaml)
- Add pre-commit hooks and secrets baseline
- Add docker-compose for local development
- Update Kubernetes network policies
Frontend Improvements:
- Add auto-healing error boundary component
- Update i18n messages for agent features
- Enhance dual-state incident card with execution feedback
Documentation:
- Add 7 ADRs covering MCP, design system, architecture decisions
- Update ARCHITECTURE_MEMORY.md with modular design
- Add GLOBAL_RULES.md and SOUL.md for project identity
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-03-23 18:40:36 +08:00
OG T
1576f2ab20
fix(db): eliminate SQLite brain-split, force PostgreSQL
...
Root cause: Worker used SQLITE_DATABASE_URL causing "no such table: incidents"
because each Pod had isolated SQLite file, not shared PostgreSQL.
Fixes:
- db/base.py: Use DATABASE_URL (PostgreSQL) instead of SQLITE_DATABASE_URL
- Added SQLite prohibition guard with logging
- Added pool_size and pool_pre_ping for production stability
New: packages/lewooogo-data PgMemoryProvider (Phase 6.4d)
- Episodic Memory implementation for PostgreSQL
- init_pg_engine() with auto table creation
- SQLite forbidden by Commander's decree
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-03-23 10:02:43 +08:00
OG T
196d269b92
feat: add all application source code
...
- apps/api: FastAPI backend with Dockerfile
- apps/web: Next.js frontend with Dockerfile
- apps/sensor: Signal collection agent
- packages: shared packages
Co-Authored-By: Claude <noreply@anthropic.com >
2026-03-22 18:57:44 +08:00