docs(awooop): record t13 quality classification deployment
This commit is contained in:
@@ -7591,3 +7591,86 @@ HTML contains: AwoooP 治理總覽 / 自動化品質 / 不可宣稱完整閉環
|
||||
- 產線資料仍顯示 `verified_auto_repair_total=0`、`production_claim=false`,因此目前正確說法是「真相可見度已補上」,不是「完整 AI 自動修復閉環已完成」。
|
||||
- 下一步要進 T13:收斂 `execution_unverified`,把 post-execution verification / auto_repair durable record / learning writeback 的缺口從可見化推進到真正閉環。
|
||||
- 目前整體進度更新:約 76%。
|
||||
|
||||
### 2026-05-13 — AwoooP truth-chain T13:NO_ACTION / audit-only 不再誤算成自動修復執行(production verified)
|
||||
|
||||
**目的**:
|
||||
|
||||
- T12d production quality summary 顯示不少 `execution_unverified`,但 live trace 發現其中多數其實是 `NO_ACTION` 或 `ansible_candidate_matched/dry_run` audit row。
|
||||
- 這些 row 是「純觀察 / 候選稽核」,不是 AI 自動修復執行;若算成 execution,Operator 會誤以為「AI 修了但沒驗證」。
|
||||
|
||||
**變更**:
|
||||
|
||||
- truth-chain 新增 effective execution 判定:
|
||||
- `playbook_executed` 且 `output.reason=NO_ACTION` / action 含 `NO_ACTION` / `OBSERVE` / `INVESTIGATE` → `noop_operation_records`
|
||||
- `status=dry_run`、`ansible_candidate_matched`、`ansible_execution_skipped` → audit-only,不算有效修復執行
|
||||
- `automation_quality.facts` 新增:
|
||||
- `effective_execution_records`
|
||||
- `noop_operation_records`
|
||||
- `audit_only_operation_records`
|
||||
- `_truth_status()` / `build_automation_quality()` 都改用 effective execution,避免 NO_ACTION 把 stage 推成 `execution_succeeded` 或 verdict 推成 `execution_unverified`。
|
||||
|
||||
**local verification**:
|
||||
|
||||
```text
|
||||
DATABASE_URL=postgresql+asyncpg://u:p@localhost:5432/db pytest tests/test_awooop_truth_chain_service.py tests/test_platform_router_order.py -q
|
||||
21 passed
|
||||
|
||||
ruff check --select F821 src/services/awooop_truth_chain_service.py tests/test_awooop_truth_chain_service.py
|
||||
OK
|
||||
|
||||
python3 -m py_compile src/services/awooop_truth_chain_service.py tests/test_awooop_truth_chain_service.py
|
||||
OK
|
||||
```
|
||||
|
||||
**production deploy / smoke(完成)**:
|
||||
|
||||
```text
|
||||
Gitea:
|
||||
2054 ai-code-review cecadb33 -> success
|
||||
2053 CD Pipeline cecadb33 -> success
|
||||
tests -> success
|
||||
build-and-deploy -> success
|
||||
post-deploy-checks -> success
|
||||
Deploy marker: 2314bade chore(cd): deploy cecadb3 [skip ci]
|
||||
|
||||
K8s image:
|
||||
awoooi-api 192.168.0.110:5000/awoooi/api:cecadb331badac7aa4fb07922b892875c28a891a
|
||||
awoooi-worker 192.168.0.110:5000/awoooi/api:cecadb331badac7aa4fb07922b892875c28a891a
|
||||
awoooi-web 192.168.0.110:5000/awoooi/web:cecadb331badac7aa4fb07922b892875c28a891a
|
||||
|
||||
health:
|
||||
https://awoooi.wooo.work/api/v1/health -> 200
|
||||
|
||||
summary endpoint smoke, hours=24, limit=30:
|
||||
schema_version=automation_quality_summary_v1
|
||||
incident_total=30
|
||||
evaluated_total=30
|
||||
verified_auto_repair_total=0
|
||||
production_claim.can_claim_full_auto_repair=false
|
||||
score_buckets={green: 0, yellow: 0, red: 30}
|
||||
by_verdict:
|
||||
manual_required_no_action=18
|
||||
received_only=12
|
||||
execution_unverified=0
|
||||
top gate_failures:
|
||||
auto_repair_recorded=30
|
||||
execution_recorded=30
|
||||
evidence_collected=22
|
||||
approval_state=18
|
||||
mcp_gateway_observed=12
|
||||
outbound_recorded=12
|
||||
timeline_recorded=12
|
||||
|
||||
detail smoke:
|
||||
INC-20260513-CF5DCE -> stage=manual_required / verdict=manual_required_no_action / effective_execution_records=0 / noop_operation_records=1
|
||||
INC-20260513-553113 -> stage=manual_required / verdict=manual_required_no_action / effective_execution_records=0 / noop_operation_records=1 / audit_only_operation_records=1
|
||||
INC-20260513-42FCEC -> stage=manual_required / verdict=manual_required_no_action / effective_execution_records=0 / noop_operation_records=1
|
||||
```
|
||||
|
||||
判讀:
|
||||
|
||||
- T13 已完成並推版:現在 Operator Console / truth-chain 不會再把 NO_ACTION 或 Ansible candidate audit 誤認為真正修復執行。
|
||||
- 產線結果更誠實:目前不是「修了但未驗證」,而是「18 筆需人工判斷、12 筆只收到告警、0 筆可宣稱已驗證自動修復」。
|
||||
- 下一步 T14 應從「分類校正」進到真正閉環:讓可安全處理的低風險事件產生 durable `auto_repair_executions`、post-execution `verification_result`、KM / learning writeback;不能再用 NO_ACTION 假裝自動修復。
|
||||
- 目前整體進度更新:約 78%。
|
||||
|
||||
@@ -2025,6 +2025,13 @@ Phase 6 完成後
|
||||
- T12d:Operator Console `/awooop` 首頁新增「自動化品質」面板;summary endpoint 改為 public aggregate 且清空 `examples`,source-level `/truth-chain/{source_id}` 仍需 operator auth。production image `356bfce2c8663c46933df4a9050dfaa9f594436a`、Gitea runs `2050/2049` success、health 200、頁面含 `自動化品質` / `不可宣稱完整閉環`。
|
||||
- 目前總體判讀:真相鏈可見度已從 Telegram 卡片補到 DB / truth-chain API / Run Detail / Operator Console,但 latest production aggregate 仍是 `verified_auto_repair_total=0`。下一步 T13 必須收斂 `execution_unverified`:post-execution verification、auto-repair durable record、learning / KM writeback 缺口要從「可見」推到「閉環」。
|
||||
|
||||
**T13 NO_ACTION / audit-only quality classification production verified(2026-05-13 台北)**:
|
||||
- 觸發:T12d production summary 顯示 `execution_unverified`,但 live trace 證實多數其實是 `NO_ACTION` 或 `ansible_candidate_matched/dry_run` audit row,被 truth-chain 誤算成有效修復執行。
|
||||
- 修正:truth-chain 新增 effective execution 判定;NO_ACTION / OBSERVE / INVESTIGATE row 計入 `noop_operation_records`,`status=dry_run`、`ansible_candidate_matched`、`ansible_execution_skipped` 計為 audit-only,不再推動 `execution_succeeded` / `execution_unverified`。
|
||||
- Production:`cecadb33 fix(awooop): exclude audit-only ops from repair quality` 已部署,Gitea runs `2054/2053` success,deploy marker `2314bade`,API/Worker/Web image 均為 `cecadb331badac7aa4fb07922b892875c28a891a`,health 200。
|
||||
- Smoke:quality summary `hours=24&limit=30` 由舊的 `execution_unverified=11` 校正為 `manual_required_no_action=18`、`received_only=12`、`execution_unverified=0`、`verified_auto_repair_total=0`、`production_claim=false`。
|
||||
- 判讀:T13 完成的是「真相分類校正」,不是自動修復閉環。下一步 T14 必須讓可安全處理的低風險事件產生 durable `auto_repair_executions`、post-execution `verification_result`、KM / learning writeback;禁止再用 NO_ACTION 或 dry-run audit 假裝自動修復。
|
||||
|
||||
---
|
||||
|
||||
### 2026-04-20 晚 (台北) — C1-C4 全流程串接 — Playbook 鏈路保護(commit de2d34d)
|
||||
|
||||
Reference in New Issue
Block a user