Your Name 80c36ba801 fix(incident): F2 NO_ACTION 觸發 resolve_incident + 冪等 guard ...

【根因】INC-20260507-99ADF2 飛輪斷流，566+ stuck incidents（30秒漲 1）核心
原因：NO_ACTION 路徑 (approval_execution.py:251) 提前 return True，跳過
line 482-495 已有的 resolve_incident 呼叫，incident 永遠卡 INVESTIGATING。

【修法】
- approval_execution.py NO_ACTION 分支補 resolve_incident 呼叫 + 成功/失敗
  log，背景 log 加 path="no_action" 用於 prod 量化修法生效率（debugger
  全鏈分析 + critic 1st/2nd 審查必修 #1）。
- incident_service.py resolve_incident 在 line 1106 加 RESOLVED 冪等 guard，
  早於所有副作用（status mutation / Redis / DB / postmortem / KB / KM /
  disposition），順帶修 success path line 482-495 重觸 postmortem 的潛在
  老風險（critic 必修 #2）。

【遵守 Codex 5/6 設計（feedback_respect_codex_design_intent.md）】
- 不動 flywheel_stats_service.py / heartbeat_report_service.py /
  auto_repair_service.record_auto_repair() / metrics_repository UPPER(status)。
- resolve_incident 不寫 auto_repair_executions 表（Codex 5/6 source of
  truth），不污染 24h KPI 計算。

【Test 覆蓋】
- test_approval_execution_no_action.py：NO_ACTION → resolve 被呼叫一次 +
  resolve raise 時仍 return True（NO_ACTION 不能因 resolve 失敗退化成 False，
  否則污染 auto_execute KPI line 207-208 註解契約）。
- test_incident_service_resolve_idempotency.py：RESOLVED → return existing +
  save_to_working_memory 不被呼叫；not_found → return None。

【驗收條件（部署後 24h）】
1. grep `path="no_action"` 中 incident_resolved_after_no_action_execution
   數量 vs background_execution_noop 數量，1:1 才算修復成功。
2. awoooi_flywheel_incidents_stuck 從每 30 秒漲 1 變平緩。
3. SRE 群 24h 內若湧入 >20 份 NO_ACTION postmortem 觸發 follow-up 評估
   resolution_type="no_action" 跳過 postmortem（critic Minor #3 方案 B）。

Refs: INC-20260507-99ADF2, debugger root cause #1 (鏈 A), critic 1st 必修
#1 #2, critic 2nd 必修 #1 #2 #3

Co-Authored-By: Codex (aider) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-05-07 18:55:58 +08:00

..

agents

fix(execution-feedback): 修復系統自動化反饋完全斷鏈的三層 P0 故障

2026-04-25 03:29:38 +08:00

integration

fix(incidents): keep list endpoint pure read

2026-05-06 21:17:25 +08:00

llm_testing

fix(api): 修復全部 lint 錯誤 (ruff --fix)

2026-03-26 16:06:20 +08:00

__init__.py

fix(hermes): ANTHROPIC_API_KEY 注入 + solver 信心度修法 A + 12-Agent 治理文件

2026-04-25 02:33:43 +08:00

conftest.py

docs(logbook): ADR-059 Gitea Webhook 遷移完成記錄

2026-04-05 14:45:02 +08:00

e2e_network_test.py

refactor(api): Phase 22 P1 模組化修復 - Router→Service 封裝

2026-03-31 16:25:00 +08:00

factories.py

feat(test): B5 整合測試框架 — 真實 DB, 5/5 通過

2026-04-10 11:22:57 +08:00

test_action_parser_safety.py

fix(aiops): close ssh and telegram remediation gaps

2026-05-01 16:53:02 +08:00

test_action_parsing.py

fix(aiops): close ssh and telegram remediation gaps

2026-05-01 16:53:02 +08:00

test_agent_loop_foundation.py

feat(aiops): structure agent loop shadow output

2026-05-01 15:09:57 +08:00

test_agent_step_timeouts.py

fix(alerts): bypass proxy timeout for GCP Ollama

2026-05-06 08:55:14 +08:00

test_ai_governance_endpoints.py

feat(governance): AI 治理事件處理鏈四軌交付（C/D/B/A）

2026-05-03 12:42:40 +08:00

test_ai_router_cache_provider_policy.py

fix(ai): require 111 before alert cloud fallback

2026-05-06 00:05:51 +08:00

test_ai_router_diagnose_fallback.py

fix(ai): remove 188 ollama provider

2026-05-06 14:34:48 +08:00

test_ai_router_failover_integration.py

fix(ai): remove 188 ollama provider

2026-05-06 14:34:48 +08:00

test_ai_router_feedback.py

refactor(tests): 技術債清零 — 移除 FakeRepo/FakeSession Mock DB 違規

2026-04-22 01:33:30 +08:00

test_ai_rule_catalog_write.py

feat(flywheel): W1 PR-P1 + ADR-091 T1 — 飛輪 80→90 第一波

2026-04-29 10:44:39 +08:00

test_aider_event_models.py

fix(imports): CI 環境 import path 統一為 src.*（移除 apps.api.src.* PEP 420 假依賴）

2026-04-20 04:13:02 +08:00

test_aider_event_processor.py

fix(tests): 移 DB 測試到 integration 層修復 CI asyncpg 密碼錯誤

2026-04-22 01:41:34 +08:00

test_aider_event_service.py

feat(service): aider_event_service — classify + signal_data builder (uses existing debounce)

2026-04-20 19:40:01 +08:00

test_aider_events_api.py

feat(api): POST /api/v1/aider/events HMAC webhook + Redis stream push

2026-04-20 19:40:01 +08:00

test_alert_approval_guard.py

fix(alerts): guard approval actions and wire playbook learning

2026-05-06 03:34:24 +08:00

test_alert_grouping_service.py

fix(telegram): digest grouped alert storms

2026-05-07 01:51:31 +08:00

test_alert_rule_engine_validation.py

fix(auto): use action parser for repair gates

2026-04-30 14:06:09 +08:00

test_alertmanager_rule_bypass.py

fix(aiops): suppress repeated llm alert loops

2026-05-01 13:02:07 +08:00

test_anomaly_counter.py

fix(api): 修復 34 個 Ruff lint 錯誤

2026-03-29 15:27:49 +08:00

test_aol_to_catalog_writeback_job.py

feat(flywheel): W2 三件 + KMWriter critic 修法（1635 tests 全綠）

2026-04-29 19:44:04 +08:00

test_approval_execution_mcp_audit.py

fix(mcp): audit approved ssh execution path

2026-05-06 16:34:39 +08:00

test_approval_execution_no_action.py

fix(incident): F2 NO_ACTION 觸發 resolve_incident + 冪等 guard

2026-05-07 18:55:58 +08:00

test_approval_execution_retry.py

feat(adr-076): 戰術 B 四大 Task 全部完成 — 告警聚合+重試+自動報告

2026-04-14 14:39:14 +08:00

test_approval_field_alignment.py

fix(api): 修復 34 個 Ruff lint 錯誤

2026-03-29 15:27:49 +08:00

test_auto_repair_service.py

fix(aiops): escalate failed host verification

2026-05-01 10:47:42 +08:00

test_awooop_approval_token.py

fix(awooop): use shared redis for approval gates

2026-05-06 13:18:43 +08:00

test_awooop_operator_auth.py

fix(awooop): authenticate approval decisions

2026-05-06 13:05:51 +08:00

test_awooop_operator_timeline_labels.py

fix(awooop): label outbound timeline events

2026-05-07 10:40:14 +08:00

test_callback_dispatcher_llm.py

feat(dispatch): B2 LLM 動態 MCP 派發安全閘 + telegram_gateway LLM 按鈕流程

2026-04-27 15:22:31 +08:00

test_callback_dispatcher.py

fix(mcp): tag legacy provider calls with audit context

2026-05-06 17:18:52 +08:00

test_channel_hub_grouped_alert_events.py

fix(awooop): anchor legacy channel events to shadow runs

2026-05-07 10:12:52 +08:00

test_chat_manager_ollama_routing.py

fix(chat): route OpenClaw chat through Ollama lane

2026-05-05 15:57:26 +08:00

test_check_trust_drift_commit_outside_context_poc.py

fix(governance): 修治理告警 4 個 silent failure + Prom sentinel 連鎖

2026-05-03 00:18:57 +08:00

test_circuit_breaker.py

feat(api): Wave 1 安全網 - Circuit Breaker + Global Repair Cooldown

2026-03-29 15:48:03 +08:00

test_classify_alert_early.py

fix(ops): classify systemd runner alerts as host resources

2026-05-05 14:28:18 +08:00

test_config_url_validation.py

feat(ollama): ADR-110 GCP 三層容災架構（GCP-A → GCP-B → Local → Gemini）

2026-05-03 22:49:23 +08:00

test_consensus_integration.py

feat(wave6-8): P2.1 fusion + P2.2 governance + P2.4 consensus + Wave 7/8 BLOCKER 修復

2026-04-27 08:11:40 +08:00

test_cs1_auto_execute.py

fix(auto): use action parser for repair gates

2026-04-30 14:06:09 +08:00

test_cs3_auto_execute.py

fix(auto): use action parser for repair gates

2026-04-30 14:06:09 +08:00

test_decision_fusion.py

feat(wave6-8): P2.1 fusion + P2.2 governance + P2.4 consensus + Wave 7/8 BLOCKER 修復

2026-04-27 08:11:40 +08:00

test_decision_manager_bare_metal_kubectl_guard.py

fix(decision): block kubectl actions on bare_metal host alerts

2026-05-02 17:41:28 +08:00

test_decision_manager_docker_prune_routing.py

fix(telegram): separate ssh diagnosis from repair failures

2026-05-06 22:03:19 +08:00

test_decision_manager_telegram_dedup.py

fix(decision): derive telegram dedup from incident signals

2026-05-06 00:19:35 +08:00

test_destructive_patterns.py

fix(auto): use action parser for repair gates

2026-04-30 14:06:09 +08:00

test_diagnosis_aggregator_stub.py

feat(p3.1-t2-patha): DiagnosisAggregator 路徑 A + Solver F4 critical reject + 對齊測試

2026-04-27 14:42:29 +08:00

test_drift_interpreter_ollama_first.py

fix(drift): parse ollama json wrapped responses

2026-05-06 19:39:01 +08:00

test_dynamic_baseline_service.py

fix(ai): restore dynamic baseline holt winters fit

2026-05-06 15:30:31 +08:00

test_emergency_escalation_service.py

fix(aiops): persist emergency intervention traces

2026-05-01 20:34:33 +08:00

test_failover_alerter.py

fix(telegram): format governance and runbook alerts

2026-05-07 00:58:20 +08:00

test_failover_e2e_dispatch.py

fix(ai): remove 188 ollama provider

2026-05-06 14:34:48 +08:00

test_failure_watcher.py

feat(api): Phase 18.6 E2E 測試驗證 (40 tests)

2026-03-31 12:16:54 +08:00

test_feature_flags.py

feat(adr-080): Phase 0 防護欄建立 — AI 自主化飛輪啟動

2026-04-15 12:44:53 +08:00

test_gap_a4_placeholder_resolution.py

fix(ops): classify systemd runner alerts as host resources

2026-05-05 14:28:18 +08:00

test_gemini_provider_security.py

fix(security): keep Gemini key out of request URLs

2026-04-29 22:56:12 +08:00

test_get_incident_type.py

fix(review): 首席架構師 Code Review 修補 — I1 get_incident_type 邏輯修正 + 測試補全

2026-04-11 21:33:19 +08:00

test_gitea_webhook.py

feat(t0): Task A 按鈕一致性測試 + Task C Gitea→Telegram 通知收尾

2026-04-26 20:17:17 +08:00

test_global_repair_cooldown.py

fix(tests): 首席架構師審查修復 - 測試套件 + DI 強化 (96/100 OUTSTANDING)

2026-04-01 11:11:50 +08:00

test_golden_regression.py

fix(ops): classify systemd runner alerts as host resources

2026-05-05 14:28:18 +08:00

test_governance_agent.py

fix(test): align governance self_failure assertions with nested payload schema

2026-05-03 00:05:04 +08:00

test_governance_dispatcher.py

feat(governance): AI 治理事件處理鏈四軌交付（C/D/B/A）

2026-05-03 12:42:40 +08:00

test_governance_remediation_dispatch.py

feat(governance): AI 治理事件處理鏈四軌交付（C/D/B/A）

2026-05-03 12:42:40 +08:00

test_heartbeat_dedup_p0_4.py

feat(heartbeat): noise reduction — silent 6h + warnings hash dedup

2026-05-03 01:48:57 +08:00

test_heartbeat_ollama_endpoints.py

feat(heartbeat): show ollama endpoint topology

2026-05-06 17:58:56 +08:00

test_heartbeat_pod_state_machine.py

feat(heartbeat): full K8s pod lifecycle state machine + regression tests

2026-05-03 01:44:58 +08:00

test_host_repair_agent.py

refactor(api): Re-Review S1/S2/S3 改善 — 消除重複+防禦性驗證+測試隔離

2026-04-07 11:17:40 +08:00

test_incident_service_resolve_idempotency.py

fix(incident): F2 NO_ACTION 觸發 resolve_incident + 冪等 guard

2026-05-07 18:55:58 +08:00

test_incident_timeline_service.py

feat(flywheel): expose incident processing timeline

2026-04-29 23:38:30 +08:00

test_incidents_list_pure_read.py

fix(incidents): batch decision token lookup

2026-05-06 21:27:46 +08:00

test_intent_classifier.py

fix(approval): 修復 Y/n 簽核按鈕無動作問題 (Phase 22 P1)

2026-03-31 16:16:16 +08:00

test_kb_rot_cleaner_schedule.py

feat(p3.2+adr-100): Model Version Tracker + SLO 自治 + KB rot cleaner

2026-04-27 14:54:19 +08:00

test_km_playbook_feedback_loop.py

feat(flywheel): W2 三件 + KMWriter critic 修法（1635 tests 全綠）

2026-04-29 19:44:04 +08:00

test_km_writer_backfill_reconciler.py

fix(km): keep backfill reconciler loop alive

2026-05-06 17:03:22 +08:00

test_km_writer_idempotent.py

fix(critic-review): KMWriter 名實統一 + Alertmanager 修抑制 + drift checker AST 化

2026-04-29 10:44:39 +08:00

test_km_writer.py

fix(critic-review): KMWriter 名實統一 + Alertmanager 修抑制 + drift checker AST 化

2026-04-29 10:44:39 +08:00

test_learning_chain_e2e.py

feat(wave4-5): P1.3+P1.4 真接線 + Ollama_188 provider 註冊 + quota atomic 修復

2026-04-26 20:44:19 +08:00

test_learning_service.py

fix(tests): 移除 Mock 違規 - test_learning_service.py

2026-03-31 12:20:29 +08:00

test_lifespan_failover_wiring.py

feat(p1): Ollama 多層容災系統 — P1.1 健康檢測 + P1.2 ai_router 整合 + P1.5 容災告警

2026-04-26 20:18:33 +08:00

test_llm_tier1_schema.py

fix(api): 修復全部 lint 錯誤 (ruff --fix)

2026-03-26 16:06:20 +08:00

test_local_code_review_cloud_fallback.py

fix(ai): isolate heavy Ollama workloads from GCP alert lane

2026-05-05 23:06:07 +08:00

test_logging_redaction.py

fix(logging): redact telegram bot urls

2026-05-06 16:54:14 +08:00

test_matched_playbook_id_e2e.py

feat(flywheel): W1 PR-P1 + ADR-091 T1 — 飛輪 80→90 第一波

2026-04-29 10:44:39 +08:00

test_mcp_audit_context.py

fix(mcp): normalize audit session ids

2026-05-06 17:40:42 +08:00

test_mcp_audit_service.py

fix(mcp): normalize audit session ids

2026-05-06 17:40:42 +08:00

test_mcp_credential_isolation.py

feat(awooop): Phase 1-8 完整實作 — AwoooP Agent Platform 六平面架構

2026-05-04 19:31:53 +08:00

test_mcp_gateway_audit.py

fix(mcp): persist blocked gateway audit rows

2026-05-06 16:21:43 +08:00

test_mcp_gateway_gate5.py

fix(awooop): use shared redis for approval gates

2026-05-06 13:18:43 +08:00

test_mcp_tool_registry.py

feat(adr-081): Phase 1 感官縱深 — 8D 情報蒐集 + 執行後驗證

2026-04-15 13:08:38 +08:00

test_mcp_tool_result_compat.py

fix(mcp): tolerate legacy tool DTO fields

2026-05-06 16:11:26 +08:00

test_model_regression.py

fix(ai): remove 188 ollama provider

2026-05-06 14:34:48 +08:00

test_model_version_probe.py

fix(ai): remove 188 ollama provider

2026-05-06 14:34:48 +08:00

test_model_version_tracker.py

fix(ai): remove 188 ollama provider

2026-05-06 14:34:48 +08:00

test_notification_matrix_group_cutover.py

fix(telegram): close remaining DM alert routes

2026-04-30 23:02:17 +08:00

test_nvidia_chat.py

refactor(ai): 模組化重構 - NVIDIA chat 移至 NvidiaProvider

2026-03-29 20:49:23 +08:00

test_nvidia_provider.py

fix(ai): keep GCP Ollama lane on safe models

2026-05-05 23:37:33 +08:00

test_ollama_auto_recovery.py

fix(ollama): treat legacy primary as ollama

2026-05-05 13:25:27 +08:00

test_ollama_endpoint_resolver.py

fix(alerts): let GCP Ollama finish before cloud fallback

2026-05-06 05:27:55 +08:00

test_ollama_failover_manager.py

fix(ai): remove 188 ollama provider

2026-05-06 14:34:48 +08:00

test_ollama_health_monitor.py

fix(ai): remove 188 ollama provider

2026-05-06 14:34:48 +08:00

test_ollama_provider_endpoints.py

fix(ai): require 111 before alert cloud fallback

2026-05-06 00:05:51 +08:00

test_openclaw_alert_cloud_fallback_gate.py

fix(ai): keep openclaw before gemini in alert fallback

2026-05-06 20:20:58 +08:00

test_openclaw_cache_key.py

fix(aiops): suppress repeated llm alert loops

2026-05-01 13:02:07 +08:00

test_openclaw_legacy_ollama_failover.py

fix(openclaw): route legacy ollama through failover endpoints

2026-05-05 13:55:52 +08:00

test_openclaw_nemo_provider.py

fix(flywheel): fallback on OpenClaw degraded responses

2026-04-29 22:38:57 +08:00

test_operation_parser_ssh.py

fix(approval): route SSH actions through SSHProvider on manual approve

2026-05-02 12:31:37 +08:00

test_p0_diagnose_routing.py

feat(ai-router): 推翻 A2 鐵律 — DIAGNOSE primary 改 Ollama 本地優先

2026-04-29 11:39:36 +08:00

test_p2_db_fixes.py

test(consensus): expect redis namespace dual write

2026-05-05 13:41:41 +08:00

test_p3_tier1_integrations.py

test(p3.1-t1): test_p3_tier1_integrations 對應 model_rollback + resource_resolver 整合

2026-04-27 08:17:59 +08:00

test_phase2_fallback.py

fix(alerts): let GCP Ollama finish before cloud fallback

2026-05-06 05:27:55 +08:00

test_phase22_nemotron_collab.py

feat(web): OpenClaw 風格龍蝦 SVG + 三色狀態燈號 + 測試修正

2026-04-09 08:55:21 +08:00

test_phase25_auto_harvesting.py

fix(telegram): format governance and runbook alerts

2026-05-07 00:58:20 +08:00

test_phase25_drift_detection.py

test(phase25): Phase 25 P1/P2 source code inspection tests (36 tests)

2026-04-05 00:14:50 +08:00

test_platform_router_order.py

feat(awooop): add run detail timeline

2026-05-07 03:55:01 +08:00

test_playbook_generator.py

feat(playbook): version generated playbooks

2026-04-30 23:59:39 +08:00

test_playbook_service.py

feat(playbook): version generated playbooks

2026-04-30 23:59:39 +08:00

test_playbook_ssh_extraction.py

feat(adr-076): Task 3.3 — SSH 修復 KM 萃取（補齊飛輪雙手）

2026-04-14 15:19:54 +08:00

test_post_execution_verifier.py

fix(mcp): tag legacy provider calls with audit context

2026-05-06 17:18:52 +08:00

test_pre_decision_investigator.py

fix(mcp): tag legacy provider calls with audit context

2026-05-06 17:18:52 +08:00

test_prompt_validation.py

fix(ai): remove 188 ollama provider

2026-05-06 14:34:48 +08:00

test_provider_version_alerter.py

test(p3.2): provider_version_alerter 單元測試（6 passed）

2026-04-27 19:56:51 +08:00

test_rag_provider.py

fix(mcp): P1 修復 - DI 一致性 + 測試補充 + 配置優化

2026-03-29 16:23:30 +08:00

test_redis_multisig.py

fix(lint): ruff auto-fix + lewooogo-core src 加入 git

2026-03-23 23:51:37 +08:00

test_report_generation_service.py

feat(adr-076): 戰術 B 四大 Task 全部完成 — 告警聚合+重試+自動報告

2026-04-14 14:39:14 +08:00

test_rule_engine_auto_execute.py

fix(auto): use action parser for repair gates

2026-04-30 14:06:09 +08:00

test_rule_to_playbook_migrator.py

feat(flywheel): W1 PR-R1 規則→Playbook 遷移 + PR-K1 timeline 防禦 ALTER

2026-04-29 10:49:25 +08:00

test_sanitization_service.py

feat(adr-081): Phase 1 感官縱深 — 8D 情報蒐集 + 執行後驗證

2026-04-15 13:08:38 +08:00

test_secret_redactor.py

security(secrets): 替換 test fixture 真 TG bot token 為假值

2026-04-20 04:23:09 +08:00

test_self_healing_validator_integration.py

feat(flywheel): W2 三件 + KMWriter critic 修法（1635 tests 全綠）

2026-04-29 19:44:04 +08:00

test_sentry_webhook_signature.py

test(p3.1-t2): Sentry Webhook 簽章驗證 dedicated tests

2026-04-27 08:24:59 +08:00

test_shadow_auto_approve.py

feat(webhook): shadow-run auto_approve.evaluate + 補 metadata kwarg

2026-04-27 16:00:00 +08:00

test_smart_router.py

fix(test): ComplexityScorer 模型名稱更新 llama3.2:3b → gemma3:4b

2026-04-09 09:01:59 +08:00

test_solver_recommended_actions.py

feat(p3.1-t2-patha): DiagnosisAggregator 路徑 A + Solver F4 critical reject + 對齊測試

2026-04-27 14:42:29 +08:00

test_ssh_provider_docker_prune.py

feat(aiops): add ssh_docker_prune to auto-repair flywheel for disk-full alerts

2026-05-02 12:31:37 +08:00

test_ssh_provider_tools.py

fix(ssh): suppress asyncssh info log formatting noise

2026-05-07 04:20:26 +08:00

test_telegram_adr050.py

fix(test): 更新 TestHistoryMessageFormat 適配 Phase 27 雙層策略

2026-04-10 01:12:00 +08:00

test_telegram_ai_automation_block.py

feat(flywheel): surface ai automation and code review

2026-04-30 00:09:25 +08:00

test_telegram_button_consistency.py

fix(telegram): route alerts to SRE war room

2026-04-30 15:01:23 +08:00

test_telegram_gateway_error_sanitizer.py

fix(telegram): sanitize failover alert errors

2026-05-06 16:45:47 +08:00

test_telegram_gateway_llm_buttons.py

fix(telegram): P0 async race + P1 short_id 碰撞 + P0 incident_id 修復

2026-04-27 19:56:51 +08:00

test_telegram_integration.py

test(telegram): 修正測試斷言符合 ADR-075 中文風險等級

2026-04-12 21:08:48 +08:00

test_telegram_message_templates.py

fix(telegram): clarify auto repair handoff cards

2026-05-07 02:07:43 +08:00

test_terminal_service.py

fix(approval): 修復 Y/n 簽核按鈕無動作問題 (Phase 22 P1)

2026-03-31 16:16:16 +08:00

test_terminal.py

fix(security): 體健修復 — 7項 Critical/Major 安全問題全修

2026-04-22 01:27:39 +08:00

test_trust_drift_watchdog.py

feat(governance): AI 治理事件處理鏈四軌交付（C/D/B/A）

2026-05-03 12:42:40 +08:00

test_wave8_fusion_fixes.py

feat(wave6-8): P2.1 fusion + P2.2 governance + P2.4 consensus + Wave 7/8 BLOCKER 修復

2026-04-27 08:11:40 +08:00

test_wave8_remaining_blockers.py

fix(test): align governance self_failure assertions with nested payload schema

2026-05-03 00:05:04 +08:00