Compare commits

...

1004 Commits

Author SHA1 Message Date
Your Name
d40cab8a8f chore(cd): retry post-deploy smoke 2026-06-01 11:44:28 +08:00
AWOOOI CD
0fad4c426c chore(cd): deploy 28395d5 [skip ci] 2026-06-01 11:40:29 +08:00
Your Name
28395d5a6f feat(web): add IwoooS path explorer
Some checks failed
CD Pipeline / tests (push) Successful in 1m22s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / build-and-deploy (push) Successful in 3m43s
CD Pipeline / post-deploy-checks (push) Failing after 23s
2026-06-01 11:34:50 +08:00
Your Name
a0284113de docs: record awooop automation flow gates 2026-06-01 11:25:50 +08:00
AWOOOI CD
1233cb3738 chore(cd): deploy fbcef59 [skip ci] 2026-06-01 11:17:44 +08:00
Your Name
fbcef599f9 feat(awooop): surface automation flow gates
All checks were successful
CD Pipeline / tests (push) Successful in 1m26s
Code Review / ai-code-review (push) Successful in 26s
CD Pipeline / build-and-deploy (push) Successful in 4m12s
CD Pipeline / post-deploy-checks (push) Successful in 2m37s
2026-06-01 11:10:01 +08:00
AWOOOI CD
61675911f7 chore(cd): deploy f9b3585 [skip ci] 2026-06-01 10:57:58 +08:00
Your Name
f9b3585a00 feat(web): add IwoooS topology drilldown
All checks were successful
CD Pipeline / tests (push) Successful in 1m23s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 4m40s
CD Pipeline / post-deploy-checks (push) Successful in 1m53s
2026-06-01 10:51:37 +08:00
Your Name
39569cc72b docs(logbook): record operator summary cache closure 2026-06-01 10:33:56 +08:00
AWOOOI CD
c54a276f13 chore(cd): deploy 74fc19a [skip ci] 2026-06-01 10:20:07 +08:00
Your Name
74fc19ac50 fix(api): keep callback summary cache key stable
All checks were successful
CD Pipeline / tests (push) Successful in 1m20s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 4m15s
CD Pipeline / post-deploy-checks (push) Successful in 1m56s
2026-06-01 10:14:28 +08:00
AWOOOI CD
6fad6de75e chore(cd): deploy 86fe36d [skip ci] 2026-06-01 10:10:40 +08:00
Your Name
86fe36dc55 feat(web): add IwoooS topology atlas
All checks were successful
CD Pipeline / tests (push) Successful in 1m21s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m38s
CD Pipeline / post-deploy-checks (push) Successful in 2m28s
2026-06-01 10:02:56 +08:00
AWOOOI CD
07000d532c chore(cd): deploy 0826037 [skip ci] 2026-06-01 10:02:09 +08:00
Your Name
08260372a9 fix(api): initialize redis for operator summary cache
All checks were successful
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m38s
CD Pipeline / post-deploy-checks (push) Successful in 1m39s
2026-06-01 09:56:37 +08:00
Your Name
ece378515f fix(ci): use public api for post deploy smoke
All checks were successful
Code Review / ai-code-review (push) Successful in 25s
2026-06-01 09:48:31 +08:00
AWOOOI CD
2cfa165b35 chore(cd): deploy d4483e7 [skip ci] 2026-06-01 09:43:22 +08:00
Your Name
d4483e730e fix(api): share operator summary cache through redis
Some checks failed
CD Pipeline / tests (push) Successful in 1m20s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m35s
CD Pipeline / post-deploy-checks (push) Failing after 20s
2026-06-01 09:38:16 +08:00
AWOOOI CD
8938706062 chore(cd): deploy d84ccb6 [skip ci] 2026-06-01 09:30:59 +08:00
Your Name
d84ccb630a feat(web): add IwoooS gate radar
All checks were successful
CD Pipeline / tests (push) Successful in 1m18s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m26s
CD Pipeline / post-deploy-checks (push) Successful in 2m41s
2026-06-01 09:23:10 +08:00
Your Name
159f514f55 fix(awooop): cache heavy operator summaries
Some checks failed
CD Pipeline / tests (push) Successful in 1m28s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-06-01 09:20:18 +08:00
Your Name
0e30171858 docs: record telegram callback truth-chain rollout 2026-06-01 02:20:34 +08:00
AWOOOI CD
14a31974af chore(cd): deploy 1afd7e9 [skip ci] 2026-06-01 02:12:54 +08:00
Your Name
1afd7e9e9f feat(web): add IwoooS visual mesh
All checks were successful
CD Pipeline / tests (push) Successful in 1m43s
Code Review / ai-code-review (push) Successful in 15s
CD Pipeline / build-and-deploy (push) Successful in 4m12s
CD Pipeline / post-deploy-checks (push) Successful in 2m43s
2026-06-01 02:03:11 +08:00
AWOOOI CD
68c8bb9e5c chore(cd): deploy 6061b5c [skip ci] 2026-06-01 01:57:51 +08:00
Your Name
6061b5cd54 feat(telegram): mirror callback click truth chain
All checks were successful
CD Pipeline / tests (push) Successful in 1m35s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m49s
CD Pipeline / post-deploy-checks (push) Successful in 2m8s
2026-06-01 01:52:01 +08:00
Your Name
5b6b9ced79 docs: record homepage ansible runtime rollout 2026-06-01 01:38:30 +08:00
AWOOOI CD
fc06da44df chore(cd): deploy a9db3d0 [skip ci] 2026-06-01 01:33:03 +08:00
Your Name
a9db3d0e7f fix(web): reflect live ansible runtime readiness
All checks were successful
CD Pipeline / tests (push) Successful in 1m32s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m32s
CD Pipeline / post-deploy-checks (push) Successful in 2m2s
2026-06-01 01:27:40 +08:00
Your Name
115030b35f docs: record ssh mcp adapter rollout 2026-06-01 01:24:17 +08:00
AWOOOI CD
e6f2d1d07c chore(cd): deploy 87378b4 [skip ci] 2026-06-01 01:18:25 +08:00
Your Name
87378b452d fix(api): normalize ssh mcp evidence inputs
All checks were successful
CD Pipeline / tests (push) Successful in 1m29s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m40s
2026-06-01 01:11:26 +08:00
Your Name
b83f9c5a52 fix(web): make IwoooS focus deck responsive
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / tests (push) Has been cancelled
Code Review / ai-code-review (push) Has been cancelled
2026-06-01 01:09:41 +08:00
Your Name
8a3ddb8249 docs: record mcp evidence matrix rollout 2026-06-01 01:06:41 +08:00
AWOOOI CD
5077d4d02e chore(cd): deploy 21f5142 [skip ci] 2026-06-01 01:02:59 +08:00
Your Name
21f5142d08 feat(web): add IwoooS focus deck
All checks were successful
CD Pipeline / tests (push) Successful in 1m32s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m51s
CD Pipeline / post-deploy-checks (push) Successful in 2m11s
2026-06-01 00:54:58 +08:00
Your Name
ba22e70266 fix(web): expose mcp evidence on run detail
Some checks failed
CD Pipeline / tests (push) Successful in 1m36s
Code Review / ai-code-review (push) Successful in 18s
CD Pipeline / build-and-deploy (push) Has started running
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-06-01 00:52:19 +08:00
Your Name
9ccc447f81 docs: record alerts handoff e2e verification 2026-06-01 00:42:36 +08:00
AWOOOI CD
722875135b chore(cd): deploy 6474717 [skip ci] 2026-06-01 00:28:44 +08:00
Your Name
64747170f1 fix(web): unify IwoooS security entry
All checks were successful
CD Pipeline / tests (push) Successful in 1m35s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 4m2s
CD Pipeline / post-deploy-checks (push) Successful in 2m29s
2026-06-01 00:21:11 +08:00
AWOOOI CD
58c009c2c7 chore(cd): deploy 607fc29 [skip ci] 2026-06-01 00:20:07 +08:00
Your Name
607fc291e9 fix(web): clarify alert operator handoff
Some checks failed
CD Pipeline / tests (push) Successful in 1m33s
Code Review / ai-code-review (push) Successful in 16s
CD Pipeline / build-and-deploy (push) Successful in 4m6s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-06-01 00:14:43 +08:00
Your Name
2860bd2b4b docs(logbook): record alerts operator flow rollout [skip ci] 2026-06-01 00:02:06 +08:00
AWOOOI CD
c80aae3461 chore(cd): deploy d40c4a9 [skip ci] 2026-05-31 23:55:52 +08:00
Your Name
d40c4a9fdb feat(web): add IwoooS command map
All checks were successful
CD Pipeline / tests (push) Successful in 1m33s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m35s
CD Pipeline / post-deploy-checks (push) Successful in 2m23s
2026-05-31 23:48:09 +08:00
Your Name
a73ccffb84 fix(web): surface alert operator flow state
Some checks failed
CD Pipeline / tests (push) Successful in 1m41s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 23:41:43 +08:00
Your Name
bc505cc35e docs(logbook): record telegram truth chain rollout [skip ci] 2026-05-31 23:26:21 +08:00
AWOOOI CD
151cb88c15 chore(cd): deploy dc2679e [skip ci] 2026-05-31 23:21:33 +08:00
Your Name
dc2679ea75 feat(web): promote IwoooS unlock path
All checks were successful
CD Pipeline / tests (push) Successful in 1m33s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m45s
CD Pipeline / post-deploy-checks (push) Successful in 2m15s
2026-05-31 23:15:51 +08:00
AWOOOI CD
4f053d97f8 chore(cd): deploy 356e4d4 [skip ci] 2026-05-31 23:14:29 +08:00
Your Name
356e4d41cc fix(telegram): link incident truth chain from alerts
Some checks failed
CD Pipeline / tests (push) Successful in 1m35s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m37s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-31 23:08:01 +08:00
Your Name
920488c5ff docs(logbook): record alerts evidence chain rollout [skip ci] 2026-05-31 22:54:32 +08:00
AWOOOI CD
d41194683b chore(cd): deploy 7d30b03 [skip ci] 2026-05-31 22:49:47 +08:00
Your Name
7d30b0342c fix(web): connect alerts to incident evidence chain
All checks were successful
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 6m49s
CD Pipeline / post-deploy-checks (push) Successful in 1m49s
2026-05-31 22:41:43 +08:00
Your Name
3c7a469ae4 feat(web): add IwoooS host tool evidence chain
Some checks failed
CD Pipeline / tests (push) Successful in 1m38s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 22:38:54 +08:00
Your Name
ce5da0bfb4 docs(logbook): record monitoring evidence chain rollout [skip ci] 2026-05-31 21:54:28 +08:00
AWOOOI CD
2b7768639f chore(cd): deploy 5a23dec [skip ci] 2026-05-31 21:48:59 +08:00
Your Name
5a23dec72e fix(web): connect monitoring to incident evidence chain
All checks were successful
CD Pipeline / tests (push) Successful in 1m36s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 5m4s
CD Pipeline / post-deploy-checks (push) Successful in 2m16s
2026-05-31 21:42:10 +08:00
AWOOOI CD
54a93d29ba chore(cd): deploy 70dfb2e [skip ci] 2026-05-31 21:34:18 +08:00
Your Name
70dfb2eec3 feat(web): add IwoooS security mesh matrix
All checks were successful
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / build-and-deploy (push) Successful in 5m1s
CD Pipeline / post-deploy-checks (push) Successful in 2m2s
2026-05-31 21:27:59 +08:00
Your Name
537faf6427 docs(logbook): record authorizations truth chain rollout [skip ci] 2026-05-31 21:17:19 +08:00
AWOOOI CD
25d42f1bf8 chore(cd): deploy 6add97b [skip ci] 2026-05-31 21:11:03 +08:00
Your Name
6add97b9d7 fix(web): connect authorizations to incident truth chain
All checks were successful
CD Pipeline / tests (push) Successful in 1m40s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m45s
CD Pipeline / post-deploy-checks (push) Successful in 2m12s
2026-05-31 21:03:56 +08:00
Your Name
5d49719bd4 feat(web): add VibeWork security onboarding card
Some checks failed
CD Pipeline / tests (push) Successful in 1m38s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 20:58:54 +08:00
Your Name
27d2740f29 docs(logbook): record approvals truth chain rollout [skip ci] 2026-05-31 20:41:05 +08:00
AWOOOI CD
636970a21e chore(cd): deploy ff6a7c1 [skip ci] 2026-05-31 20:32:24 +08:00
Your Name
ff6a7c1611 fix(web): surface incident truth chain in approvals
All checks were successful
CD Pipeline / tests (push) Successful in 1m36s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 4m0s
CD Pipeline / post-deploy-checks (push) Successful in 1m59s
2026-05-31 20:26:25 +08:00
Your Name
07764ce13f feat(web): add VibeWork to IwoooS security scope
Some checks failed
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 20:22:11 +08:00
Your Name
364551218d docs(logbook): record tickets truth chain rollout [skip ci] 2026-05-31 20:09:03 +08:00
AWOOOI CD
9e4c4c955a chore(cd): deploy e9977f3 [skip ci] 2026-05-31 20:05:40 +08:00
Your Name
e9977f39c1 fix(web): connect tickets to incident truth chain
All checks were successful
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m36s
CD Pipeline / post-deploy-checks (push) Successful in 1m27s
2026-05-31 19:58:47 +08:00
AWOOOI CD
33601f7b1c chore(cd): deploy 4938747 [skip ci] 2026-05-31 19:56:06 +08:00
Your Name
49387477d2 feat(web): surface IwoooS work radar
All checks were successful
CD Pipeline / tests (push) Successful in 1m29s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m37s
CD Pipeline / post-deploy-checks (push) Successful in 1m51s
2026-05-31 19:50:40 +08:00
AWOOOI CD
b07debf84d chore(cd): deploy c017fcf [skip ci] 2026-05-31 19:36:21 +08:00
Your Name
c017fcf954 feat(web): add interactive IwoooS security visuals
All checks were successful
CD Pipeline / tests (push) Successful in 1m29s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m28s
CD Pipeline / post-deploy-checks (push) Successful in 1m56s
2026-05-31 19:31:01 +08:00
Your Name
6737a3d48b docs(logbook): record web health probe rollout [skip ci] 2026-05-31 19:20:57 +08:00
AWOOOI CD
7461d4de0e chore(cd): deploy 56c8a41 [skip ci] 2026-05-31 19:16:44 +08:00
Your Name
56c8a41e5b fix(web): add cheap health probe endpoint
All checks were successful
CD Pipeline / tests (push) Successful in 1m30s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m34s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-31 19:10:31 +08:00
Your Name
fb9e8bffa6 fix(web): 延遲渲染 IwoooS drilldown 區塊
Some checks failed
CD Pipeline / tests (push) Successful in 1m30s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Has been cancelled
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-31 19:05:33 +08:00
Your Name
aee3a91f6c docs(logbook): record work items incident audit production verification [skip ci] 2026-05-31 18:59:16 +08:00
AWOOOI CD
af70ce8e4f chore(cd): deploy 59b4943 [skip ci] 2026-05-31 18:52:02 +08:00
Your Name
59b4943bf9 feat(web): 視覺化 IwoooS 資安指揮板
All checks were successful
CD Pipeline / tests (push) Successful in 1m35s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m43s
CD Pipeline / post-deploy-checks (push) Successful in 3m10s
2026-05-31 18:46:13 +08:00
AWOOOI CD
ab780892b6 chore(cd): deploy 7987da7 [skip ci] 2026-05-31 18:45:47 +08:00
Your Name
7987da7f3f fix(health): surface ollama endpoint diagnosis
Some checks failed
CD Pipeline / tests (push) Successful in 1m30s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m44s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-31 18:39:18 +08:00
Your Name
e6a433da22 fix(web): surface incident audit chain in work items
Some checks failed
CD Pipeline / tests (push) Successful in 1m31s
CD Pipeline / build-and-deploy (push) Has been cancelled
CD Pipeline / post-deploy-checks (push) Has been cancelled
Code Review / ai-code-review (push) Has been cancelled
2026-05-31 18:38:07 +08:00
Your Name
d996426337 docs(logbook): record ollama local fallback recovery [skip ci] 2026-05-31 18:31:53 +08:00
Your Name
3e964ee4c1 docs(logbook): clarify ollama local fallback boundary [skip ci] 2026-05-31 18:20:09 +08:00
Your Name
c03a57a184 docs(logbook): record run incident audit closure [skip ci] 2026-05-31 18:18:21 +08:00
Your Name
337378e55b docs(logbook): record iwooos production verification [skip ci] 2026-05-31 18:16:15 +08:00
AWOOOI CD
3c1f94a20a chore(cd): deploy 8699fe0 [skip ci] 2026-05-31 18:12:18 +08:00
Your Name
8699fe0c7f fix(api): align kb extractor ollama model
All checks were successful
CD Pipeline / tests (push) Successful in 1m23s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / build-and-deploy (push) Successful in 3m35s
CD Pipeline / post-deploy-checks (push) Successful in 3m23s
2026-05-31 18:07:03 +08:00
AWOOOI CD
8f73058b93 chore(cd): deploy bdcb059 [skip ci] 2026-05-31 18:05:14 +08:00
Your Name
165abaeae7 docs(logbook): record momo backup verification closure [skip ci] 2026-05-31 17:58:55 +08:00
Your Name
bdcb059444 fix(web): add incident audit timeline to run detail
Some checks failed
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 5m44s
CD Pipeline / post-deploy-checks (push) Failing after 30s
2026-05-31 17:57:47 +08:00
Your Name
716ed5a77c fix(web): 收斂 IwoooS 單一資安入口
Some checks failed
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 22s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 17:55:02 +08:00
Your Name
af46941ca5 docs(logbook): record awooop run drilldown evidence [skip ci] 2026-05-31 17:43:27 +08:00
AWOOOI CD
ff4a379192 chore(cd): deploy 86b6481 [skip ci] 2026-05-31 17:41:06 +08:00
Your Name
86b6481009 fix(web): 接入 Kali 112 只讀快照
All checks were successful
CD Pipeline / tests (push) Successful in 1m21s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m28s
2026-05-31 17:35:43 +08:00
AWOOOI CD
a8f6a85002 chore(cd): deploy a21f94c [skip ci] 2026-05-31 17:34:10 +08:00
Your Name
a21f94ced1 fix(alerts): clarify execution result verdict
Some checks failed
CD Pipeline / tests (push) Successful in 1m17s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m11s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-31 17:28:55 +08:00
AWOOOI CD
c6d1106cfd chore(cd): deploy 88f196a [skip ci] 2026-05-31 17:28:29 +08:00
Your Name
88f196a040 fix(web): add incident drilldown flow to status chain
Some checks failed
CD Pipeline / tests (push) Successful in 1m16s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m18s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-31 17:23:41 +08:00
Your Name
ccea510e87 docs(logbook): record source mismatch visibility [skip ci] 2026-05-31 17:14:41 +08:00
AWOOOI CD
8043eefffa chore(cd): deploy f1e4e39 [skip ci] 2026-05-31 17:11:25 +08:00
Your Name
f1e4e3949e fix(web): show source mismatch reason in status chain
All checks were successful
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m23s
CD Pipeline / post-deploy-checks (push) Successful in 1m51s
2026-05-31 17:06:26 +08:00
Your Name
79c34c4cf9 docs(logbook): record awooop truth chain drilldown [skip ci] 2026-05-31 16:55:48 +08:00
AWOOOI CD
7894156ded chore(cd): deploy aee92bc [skip ci] 2026-05-31 16:52:58 +08:00
Your Name
752de4e1b3 docs(logbook): record telegram result backfill [skip ci] 2026-05-31 16:49:24 +08:00
Your Name
aee92bc7a3 fix(awooop): chunk run context lookups
All checks were successful
CD Pipeline / tests (push) Successful in 1m20s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m55s
CD Pipeline / post-deploy-checks (push) Successful in 1m47s
2026-05-31 16:47:15 +08:00
AWOOOI CD
b92025a829 chore(cd): deploy dc4ef7e [skip ci] 2026-05-31 16:34:40 +08:00
Your Name
dc4ef7ed34 fix(web): 加速 IwoooS 資安進度可視化
All checks were successful
CD Pipeline / tests (push) Successful in 1m20s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m33s
CD Pipeline / post-deploy-checks (push) Successful in 2m1s
2026-05-31 16:28:45 +08:00
Your Name
f877e707ce fix(alerts): 收斂拒絕審批結果原因
Some checks failed
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 16:23:48 +08:00
Your Name
497e36ba9d fix(awooop): surface ansible apply proof
Some checks failed
CD Pipeline / tests (push) Successful in 1m28s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 16:19:07 +08:00
AWOOOI CD
2022eaa9e8 chore(cd): deploy 921af1c [skip ci] 2026-05-31 16:18:48 +08:00
Your Name
921af1c4c2 fix(alerts): 補齊審批終局處置結論
Some checks failed
CD Pipeline / tests (push) Successful in 1m27s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 16:13:35 +08:00
Your Name
ff9c939278 docs(logbook): 記錄 IwoooS 繁中收斂部署 [skip ci] 2026-05-31 16:04:58 +08:00
Your Name
aa47f4bc31 docs(logbook): 記錄處置結果契約部署 [skip ci] 2026-05-31 16:02:47 +08:00
AWOOOI CD
a28f84722b chore(cd): deploy e9a8a2b [skip ci] 2026-05-31 15:58:18 +08:00
Your Name
e9a8a2b3e9 test(alerts): 對齊 no-action 修復語意測試
All checks were successful
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m47s
2026-05-31 15:53:14 +08:00
Your Name
8d9525fb3b docs(logbook): record momo backup ansible apply proof [skip ci] 2026-05-31 15:52:40 +08:00
Your Name
5ed5022cd7 fix(web): 收斂 IwoooS 英文內容為繁中
Some checks failed
Ansible Lint / lint (push) Successful in 30s
CD Pipeline / tests (push) Failing after 46s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
Code Review / ai-code-review (push) Successful in 12s
2026-05-31 15:50:39 +08:00
Your Name
3d8b395032 fix(alerts): 補齊處置結果與人工通知契約
Some checks failed
CD Pipeline / tests (push) Failing after 45s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
Code Review / ai-code-review (push) Successful in 12s
2026-05-31 15:46:07 +08:00
AWOOOI CD
03f2abf576 chore(cd): deploy ebd9ca8 [skip ci] 2026-05-31 15:44:54 +08:00
Your Name
ebd9ca865f fix(api): include momo backup script in runtime image
Some checks failed
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 24s
CD Pipeline / build-and-deploy (push) Successful in 3m59s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-31 15:39:29 +08:00
AWOOOI CD
5bd5e7e49f chore(cd): deploy 75f6929 [skip ci] 2026-05-31 15:35:51 +08:00
Your Name
a169669559 fix(ansible): satisfy momo backup playbook lint
All checks were successful
Ansible Lint / lint (push) Successful in 36s
2026-05-31 15:30:32 +08:00
Your Name
75f6929bad fix(awooop): add momo backup user ansible repair
Some checks failed
Ansible Lint / lint (push) Failing after 32s
CD Pipeline / tests (push) Successful in 1m18s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 5m55s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-31 15:28:15 +08:00
Your Name
12a3be5f2d fix(web): 側邊欄 nav 全語系繁中收斂
Some checks failed
CD Pipeline / tests (push) Successful in 1m20s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 15:25:03 +08:00
Your Name
eedc69909e docs(logbook): record 188 readonly ansible proof [skip ci] 2026-05-31 15:18:49 +08:00
Your Name
05e87fa91f docs(logbook): 記錄 IwoooS 菜單整合部署 [skip ci] 2026-05-31 15:15:46 +08:00
AWOOOI CD
f9a62206ed chore(cd): deploy 50c9d51 [skip ci] 2026-05-31 15:10:29 +08:00
Your Name
50c9d51df9 feat(web): 整合 IwoooS 安全合規菜單
All checks were successful
Ansible Lint / lint (push) Successful in 30s
CD Pipeline / tests (push) Successful in 1m18s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 5m15s
CD Pipeline / post-deploy-checks (push) Successful in 2m43s
2026-05-31 15:03:32 +08:00
Your Name
872d1aa5e4 fix(awooop): honor approval repair metadata
Some checks failed
CD Pipeline / tests (push) Successful in 1m17s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 15:02:22 +08:00
Your Name
f615ac506e fix(awooop): add read-only 188 ansible check-mode
Some checks failed
Ansible Lint / lint (push) Successful in 32s
CD Pipeline / tests (push) Successful in 1m16s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / build-and-deploy (push) Has been cancelled
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-31 14:59:37 +08:00
AWOOOI CD
e8bf5ba55c chore(cd): deploy 697fff9 [skip ci] 2026-05-31 14:55:14 +08:00
Your Name
697fff96d8 fix(awooop): show diagnostic ops as non repair
All checks were successful
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m38s
CD Pipeline / post-deploy-checks (push) Successful in 2m5s
2026-05-31 14:50:01 +08:00
Your Name
0db345418f docs(logbook): 記錄 IwoooS 全產品快照部署 [skip ci] 2026-05-31 14:47:39 +08:00
Your Name
42fd9827f5 docs(logbook): update ansible check-mode production counts [skip ci] 2026-05-31 14:47:07 +08:00
Your Name
a3479b3254 docs(logbook): record ansible check-mode ssh mcp proof [skip ci] 2026-05-31 14:43:22 +08:00
AWOOOI CD
a183dc9b8f chore(cd): deploy 8b8773a [skip ci] 2026-05-31 14:43:14 +08:00
Your Name
8b8773ab7b feat(web): 新增 IwoooS 全產品只讀套用快照
All checks were successful
CD Pipeline / tests (push) Successful in 1m21s
Code Review / ai-code-review (push) Successful in 15s
CD Pipeline / build-and-deploy (push) Successful in 4m19s
CD Pipeline / post-deploy-checks (push) Successful in 1m54s
2026-05-31 14:37:24 +08:00
AWOOOI CD
4744670e4e chore(cd): deploy 8c40621 [skip ci] 2026-05-31 14:36:58 +08:00
Your Name
8c40621d42 fix(alerts): distinguish diagnostic ops from repair
Some checks failed
CD Pipeline / tests (push) Successful in 1m22s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m14s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-31 14:31:07 +08:00
Your Name
273071b654 fix(awooop): keep external incident ids out of aol bigint
Some checks failed
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 14:26:24 +08:00
AWOOOI CD
1697d91a68 chore(cd): deploy 1a72a2f [skip ci] 2026-05-31 14:20:36 +08:00
Your Name
1a72a2f664 fix(awooop): use ssh mcp transport for ansible check-mode
All checks were successful
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m45s
CD Pipeline / post-deploy-checks (push) Successful in 1m53s
2026-05-31 14:15:11 +08:00
AWOOOI CD
db48ad8678 chore(cd): deploy c50da9a [skip ci] 2026-05-31 14:08:17 +08:00
Your Name
c50da9a2b3 fix(alerts): preserve bare metal domain guard
All checks were successful
CD Pipeline / tests (push) Successful in 1m18s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m55s
CD Pipeline / post-deploy-checks (push) Successful in 2m7s
2026-05-31 14:02:46 +08:00
Your Name
e2ab879636 fix(alerts): correct telegram execution truth
Some checks failed
CD Pipeline / tests (push) Failing after 52s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
Code Review / ai-code-review (push) Successful in 11s
2026-05-31 13:58:39 +08:00
Your Name
943a6feacf docs(logbook): record ansible check-mode truth chain blocker [skip ci] 2026-05-31 13:58:15 +08:00
AWOOOI CD
7b2efc14c4 chore(cd): deploy 126316a [skip ci] 2026-05-31 13:53:33 +08:00
Your Name
126316a414 fix(awooop): make ansible cooldown query asyncpg safe
All checks were successful
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m16s
CD Pipeline / post-deploy-checks (push) Successful in 1m26s
2026-05-31 13:48:04 +08:00
AWOOOI CD
e1355c8e04 chore(cd): deploy dad8c0f [skip ci] 2026-05-31 13:42:51 +08:00
Your Name
dad8c0fbfc fix(awooop): link ansible evidence to incidents
All checks were successful
CD Pipeline / tests (push) Successful in 1m21s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m42s
CD Pipeline / post-deploy-checks (push) Successful in 1m53s
2026-05-31 13:37:12 +08:00
AWOOOI CD
28cd4b01fe chore(cd): deploy 57b21a4 [skip ci] 2026-05-31 13:33:59 +08:00
Your Name
57b21a4399 feat(web): compact iwooos security compliance entry
All checks were successful
CD Pipeline / tests (push) Successful in 1m23s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 4m51s
CD Pipeline / post-deploy-checks (push) Successful in 2m4s
2026-05-31 13:28:06 +08:00
AWOOOI CD
8ba6a1c08e chore(cd): deploy cd17a67 [skip ci] 2026-05-31 13:23:40 +08:00
Your Name
d6a6519594 chore(types): sync approval response types
All checks were successful
Type Sync Check / check-type-sync (push) Successful in 33s
2026-05-31 13:22:07 +08:00
Your Name
cd17a67774 fix(alerts): surface legacy hitl backlog
Some checks failed
CD Pipeline / tests (push) Successful in 1m21s
Code Review / ai-code-review (push) Successful in 13s
Type Sync Check / check-type-sync (push) Failing after 40s
CD Pipeline / build-and-deploy (push) Successful in 5m22s
CD Pipeline / post-deploy-checks (push) Successful in 2m19s
2026-05-31 13:16:22 +08:00
AWOOOI CD
656c90e01d chore(cd): deploy e45e52e [skip ci] 2026-05-31 13:14:33 +08:00
Your Name
e45e52e526 fix(awooop): cooldown ansible check-mode transport blockers
Some checks failed
CD Pipeline / tests (push) Successful in 1m25s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m56s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-31 13:08:45 +08:00
AWOOOI CD
46cc56c3ce chore(cd): deploy 9080ba3 [skip ci] 2026-05-31 13:00:29 +08:00
Your Name
9080ba3670 feat(awooop): run ansible check-mode evidence worker
All checks were successful
CD Pipeline / tests (push) Successful in 1m28s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 5m9s
CD Pipeline / post-deploy-checks (push) Successful in 1m30s
2026-05-31 12:53:22 +08:00
Your Name
742980f398 fix(cd): export source link gate env
Some checks failed
CD Pipeline / tests (push) Successful in 1m18s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-31 12:50:25 +08:00
AWOOOI CD
3fc9460eef chore(cd): deploy 83e27fa [skip ci] 2026-05-31 12:48:11 +08:00
Your Name
b7b4eb53b5 docs(logbook): record ansible runtime readiness deploy [skip ci] 2026-05-31 12:44:12 +08:00
Your Name
83e27fa2b2 fix(cd): harden source link post-deploy gate
Some checks failed
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 8s
CD Pipeline / build-and-deploy (push) Successful in 4m8s
CD Pipeline / post-deploy-checks (push) Failing after 11s
2026-05-31 12:43:19 +08:00
AWOOOI CD
ca2d95e9f2 chore(cd): deploy 514c201 [skip ci] 2026-05-31 12:38:07 +08:00
Your Name
514c201ff4 fix(api-tests): use asyncio run in cs1 tests
All checks were successful
CD Pipeline / tests (push) Successful in 1m23s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 7m29s
CD Pipeline / post-deploy-checks (push) Successful in 2m34s
2026-05-31 12:30:09 +08:00
Your Name
a192e5f56b fix(web): avoid stale iwooos deploy evidence
Some checks failed
CD Pipeline / tests (push) Failing after 48s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
Code Review / ai-code-review (push) Successful in 13s
2026-05-31 12:26:07 +08:00
Your Name
da519423e1 fix(api): install ansible runtime for truth chain
Some checks failed
CD Pipeline / tests (push) Failing after 1m39s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
Code Review / ai-code-review (push) Successful in 11s
2026-05-31 12:20:41 +08:00
AWOOOI CD
04ac5085cd chore(cd): deploy 4808995 [skip ci] 2026-05-29 12:45:09 +08:00
Your Name
4ea6fb98a6 fix(ops): harden reboot recovery and backup alerts 2026-05-29 12:45:09 +08:00
Your Name
ae7b39d96a fix(ops): harden reboot recovery and backup alerts 2026-05-29 12:41:34 +08:00
AWOOOI CD
70637ec871 chore(cd): deploy 9e093a9 [skip ci] 2026-05-29 11:48:32 +08:00
Your Name
9e093a9525 fix(api): reconcile inactive stale incidents
All checks were successful
CD Pipeline / tests (push) Successful in 1m26s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m23s
CD Pipeline / post-deploy-checks (push) Successful in 2m17s
2026-05-29 11:43:19 +08:00
AWOOOI CD
f0a77d79f4 chore(cd): deploy d7db0fa [skip ci] 2026-05-29 11:38:39 +08:00
Your Name
d7db0faa4d fix(api): stabilize flywheel success rate window
All checks were successful
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m5s
CD Pipeline / post-deploy-checks (push) Successful in 1m59s
2026-05-29 11:33:29 +08:00
Your Name
2828865699 docs(logbook): record provider source evidence deploy [skip ci] 2026-05-29 11:28:19 +08:00
AWOOOI CD
0836066265 chore(cd): deploy 92316dd [skip ci] 2026-05-29 11:22:38 +08:00
Your Name
92316dda04 fix(api): resolve db-only stale incidents
All checks were successful
CD Pipeline / tests (push) Successful in 1m33s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m54s
CD Pipeline / post-deploy-checks (push) Successful in 2m8s
2026-05-29 11:15:46 +08:00
Your Name
aeaa77bbe1 fix(web): show provider source evidence on homepage
Some checks failed
CD Pipeline / tests (push) Has started running
CD Pipeline / build-and-deploy (push) Has been cancelled
CD Pipeline / post-deploy-checks (push) Has been cancelled
Code Review / ai-code-review (push) Has been cancelled
2026-05-29 11:14:28 +08:00
Your Name
d6d2719e02 fix(alerts): deploy drift guard with canonical rules
Some checks failed
Code Review / ai-code-review (push) Has been cancelled
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 29s
2026-05-29 11:14:12 +08:00
Your Name
badff58cc3 feat(web): add iwooos stage completion report
Some checks failed
Code Review / ai-code-review (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / tests (push) Has been cancelled
2026-05-29 11:13:50 +08:00
Your Name
7d2128b53c fix(alerts): keep prometheus canonical rules in sync
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 32s
2026-05-29 11:09:33 +08:00
Your Name
aebd1b5b4f docs(logbook): record homepage fast evidence deploy [skip ci] 2026-05-29 10:39:35 +08:00
AWOOOI CD
845e14b8b0 chore(cd): deploy 1b28dcf [skip ci] 2026-05-29 10:35:19 +08:00
Your Name
1b28dcf3f9 fix(web): speed up homepage live evidence loading
All checks were successful
CD Pipeline / tests (push) Successful in 1m39s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m52s
CD Pipeline / post-deploy-checks (push) Successful in 3m18s
2026-05-29 10:28:37 +08:00
Your Name
5f69416eec feat(web): show iwooos next security tasks
Some checks failed
CD Pipeline / tests (push) Successful in 1m50s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / build-and-deploy (push) Has started running
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-29 10:24:22 +08:00
Your Name
a842e53332 docs(logbook): record homepage live evidence deploy [skip ci] 2026-05-26 11:59:12 +08:00
AWOOOI CD
b39fded8c7 chore(cd): deploy 01c6cb2 [skip ci] 2026-05-26 11:50:58 +08:00
Your Name
01c6cb2941 fix(web): stream homepage evidence sources independently
All checks were successful
CD Pipeline / tests (push) Successful in 1m30s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m38s
CD Pipeline / post-deploy-checks (push) Successful in 1m51s
2026-05-26 11:45:54 +08:00
AWOOOI CD
5cfee5cf1b chore(cd): deploy 320718a [skip ci] 2026-05-26 11:30:22 +08:00
Your Name
320718aa36 feat(web): bind homepage blueprint to live evidence
All checks were successful
CD Pipeline / tests (push) Successful in 1m36s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m23s
CD Pipeline / post-deploy-checks (push) Successful in 1m24s
2026-05-26 11:25:14 +08:00
Your Name
8305454f37 docs(logbook): record homepage drilldown deploy [skip ci] 2026-05-26 11:12:05 +08:00
AWOOOI CD
81f4751cee chore(cd): deploy 15f9d3a [skip ci] 2026-05-26 11:06:10 +08:00
Your Name
15f9d3aff5 fix(web): wrap incident flow evidence on mobile
All checks were successful
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m6s
CD Pipeline / post-deploy-checks (push) Successful in 1m27s
2026-05-26 11:01:18 +08:00
AWOOOI CD
63d0fc6333 chore(cd): deploy 6aec948 [skip ci] 2026-05-26 10:49:52 +08:00
Your Name
6aec9489d4 feat(web): add homepage blueprint drilldown
All checks were successful
CD Pipeline / tests (push) Successful in 1m19s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m5s
CD Pipeline / post-deploy-checks (push) Successful in 1m33s
2026-05-26 10:44:45 +08:00
Your Name
87545bc7dd docs(logbook): record homepage blueprint deploy [skip ci] 2026-05-26 10:26:43 +08:00
AWOOOI CD
bda2f7a0ca chore(cd): deploy 55d1df2 [skip ci] 2026-05-26 10:20:28 +08:00
Your Name
55d1df24e7 feat(web): render automation blueprint diagrams
All checks were successful
CD Pipeline / tests (push) Successful in 1m20s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / build-and-deploy (push) Successful in 3m44s
CD Pipeline / post-deploy-checks (push) Successful in 2m1s
2026-05-26 10:15:07 +08:00
Your Name
a03c5541a4 docs(logbook): record homepage scroll fix [skip ci] 2026-05-26 07:39:30 +08:00
AWOOOI CD
68d01d147b chore(cd): deploy f0f4ac2 [skip ci] 2026-05-26 05:51:48 +08:00
Your Name
f0f4ac2a43 fix(web): restore homepage vertical scroll
All checks were successful
CD Pipeline / tests (push) Successful in 1m27s
Code Review / ai-code-review (push) Successful in 17s
CD Pipeline / build-and-deploy (push) Successful in 3m52s
CD Pipeline / post-deploy-checks (push) Successful in 2m18s
2026-05-26 05:45:56 +08:00
AWOOOI CD
8a71934e47 chore(cd): deploy 7870489 [skip ci] 2026-05-26 01:51:25 +08:00
Your Name
dcd8e71a0f docs(logbook): record homepage automation map deploy [skip ci] 2026-05-26 01:50:37 +08:00
Your Name
7870489b08 fix(web): add awooop approval legacy hitl copy
All checks were successful
CD Pipeline / tests (push) Successful in 1m22s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m56s
CD Pipeline / post-deploy-checks (push) Successful in 1m38s
2026-05-26 01:46:31 +08:00
AWOOOI CD
0a2abe81c0 chore(cd): deploy 5009148 [skip ci] 2026-05-26 00:38:10 +08:00
Your Name
50091485a9 feat(web): surface iwooos progress and compact ux
All checks were successful
CD Pipeline / tests (push) Successful in 1m23s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m36s
CD Pipeline / post-deploy-checks (push) Successful in 1m56s
2026-05-26 00:32:16 +08:00
AWOOOI CD
e28079109c chore(cd): deploy 480292b [skip ci] 2026-05-26 00:31:21 +08:00
Your Name
480292b04d fix(approval): map rejected incidents to escalated
Some checks failed
CD Pipeline / tests (push) Successful in 1m38s
Code Review / ai-code-review (push) Successful in 33s
CD Pipeline / build-and-deploy (push) Successful in 4m42s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-26 00:25:01 +08:00
AWOOOI CD
b019a982d8 chore(cd): deploy 7cfe623 [skip ci] 2026-05-26 00:21:22 +08:00
Your Name
7cfe62313d fix(approval): sync incidents by incident_id
All checks were successful
CD Pipeline / tests (push) Successful in 1m34s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m40s
CD Pipeline / post-deploy-checks (push) Successful in 1m36s
2026-05-26 00:14:47 +08:00
AWOOOI CD
c7cd307422 chore(cd): deploy 0a981a5 [skip ci] 2026-05-26 00:13:28 +08:00
Your Name
0a981a5990 feat(web): show automation product work map
Some checks failed
CD Pipeline / tests (push) Successful in 1m29s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-26 00:07:51 +08:00
AWOOOI CD
eb6308f7b5 chore(cd): deploy 88b1925 [skip ci] 2026-05-25 23:52:09 +08:00
Your Name
88b19259c5 fix(awooop): surface legacy HITL backlog
All checks were successful
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m40s
CD Pipeline / post-deploy-checks (push) Successful in 1m38s
2026-05-25 23:46:50 +08:00
AWOOOI CD
a21cb05af3 chore(cd): deploy 3953ef6 [skip ci] 2026-05-25 23:27:03 +08:00
Your Name
3953ef6d57 fix(ollama): disable thinking for deepseek call sites
All checks were successful
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 26s
CD Pipeline / build-and-deploy (push) Successful in 5m27s
CD Pipeline / post-deploy-checks (push) Successful in 1m40s
2026-05-25 23:19:31 +08:00
Your Name
6112fd07ae feat(web): deep link callback trace evidence
Some checks failed
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-25 23:16:42 +08:00
Your Name
48a7228fff docs(logbook): record callback trace action lens deploy [skip ci] 2026-05-25 23:13:01 +08:00
AWOOOI CD
f6b8a91cd0 chore(cd): deploy fd253bc [skip ci] 2026-05-25 23:05:59 +08:00
Your Name
fd253bc93c feat(web): explain callback trace backlog handling
All checks were successful
CD Pipeline / tests (push) Successful in 1m33s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m23s
2026-05-25 22:59:43 +08:00
Your Name
b691367d40 docs(logbook): record callback trace backlog deploy [skip ci] 2026-05-25 22:18:17 +08:00
AWOOOI CD
c7e26d698c chore(cd): deploy 5845fa8 [skip ci] 2026-05-25 22:14:40 +08:00
Your Name
5845fa80a4 fix(web): add callback trace work item titles
All checks were successful
CD Pipeline / tests (push) Successful in 1m28s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m43s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-25 22:09:19 +08:00
AWOOOI CD
704ed5e0ba chore(cd): deploy 44f48b6 [skip ci] 2026-05-25 22:05:48 +08:00
Your Name
44f48b68fe feat(web): surface callback trace backlog work item
All checks were successful
CD Pipeline / tests (push) Successful in 1m27s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m28s
CD Pipeline / post-deploy-checks (push) Successful in 1m41s
2026-05-25 22:00:42 +08:00
Your Name
2c058e5adf docs(logbook): record trace recovery deploy [skip ci] 2026-05-25 21:55:31 +08:00
AWOOOI CD
5f783d5a58 chore(cd): deploy b2fc03d [skip ci] 2026-05-25 21:52:58 +08:00
Your Name
b2fc03d09f feat(awooop): show callback trace recovery
All checks were successful
CD Pipeline / tests (push) Successful in 1m26s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m41s
CD Pipeline / post-deploy-checks (push) Successful in 1m41s
2026-05-25 21:47:40 +08:00
Your Name
6a379862e7 docs(logbook): record trace gap decision deploy [skip ci] 2026-05-25 21:39:58 +08:00
AWOOOI CD
bb1a0722b3 chore(cd): deploy 32e172e [skip ci] 2026-05-25 21:37:52 +08:00
Your Name
32e172ed8b feat(awooop): classify callback trace gaps
All checks were successful
CD Pipeline / tests (push) Successful in 1m33s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m32s
CD Pipeline / post-deploy-checks (push) Successful in 1m33s
2026-05-25 21:32:37 +08:00
Your Name
f52fdebe0a docs(logbook): record callback freshness deploy [skip ci] 2026-05-25 21:27:35 +08:00
AWOOOI CD
14b617e242 chore(cd): deploy dcde86c [skip ci] 2026-05-25 21:25:21 +08:00
Your Name
dcde86c7f9 feat(awooop): show callback gap freshness
All checks were successful
CD Pipeline / tests (push) Successful in 1m32s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
2026-05-25 21:20:10 +08:00
Your Name
101b08946a docs(logbook): record trace gap prefix deploy [skip ci] 2026-05-25 21:11:47 +08:00
AWOOOI CD
5d22f59dde chore(cd): deploy 345c678 [skip ci] 2026-05-25 21:09:22 +08:00
Your Name
345c6781b8 feat(awooop): show trace ref gap prefixes
All checks were successful
CD Pipeline / tests (push) Successful in 1m33s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m46s
CD Pipeline / post-deploy-checks (push) Successful in 1m43s
2026-05-25 21:03:53 +08:00
Your Name
900fee47c9 docs(logbook): record action card trace refs deploy [skip ci] 2026-05-25 20:50:25 +08:00
AWOOOI CD
1396f1da56 chore(cd): deploy 9e15fd0 [skip ci] 2026-05-25 20:45:29 +08:00
Your Name
9e15fd08b3 feat(web): land iwooos security posture surfaces
All checks were successful
CD Pipeline / tests (push) Successful in 1m39s
Code Review / ai-code-review (push) Successful in 15s
CD Pipeline / build-and-deploy (push) Successful in 5m19s
CD Pipeline / post-deploy-checks (push) Successful in 2m11s
2026-05-25 20:35:52 +08:00
Your Name
9ec584943a feat(awooop): trace non-incident action cards
Some checks failed
CD Pipeline / tests (push) Successful in 1m32s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-25 20:33:22 +08:00
Your Name
0778a448d8 docs(logbook): record source ref recency deploy [skip ci] 2026-05-25 20:21:42 +08:00
AWOOOI CD
d50de0fa6e chore(cd): deploy a8b7299 [skip ci] 2026-05-25 20:17:52 +08:00
Your Name
a8b7299d1c feat(awooop): show source ref gap recency
All checks were successful
CD Pipeline / tests (push) Successful in 1m32s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m41s
CD Pipeline / post-deploy-checks (push) Successful in 2m13s
2026-05-25 20:12:19 +08:00
Your Name
f30405997d docs(logbook): record source ref prefix deploy [skip ci] 2026-05-25 20:01:38 +08:00
AWOOOI CD
f743321ba8 chore(cd): deploy c644cfe [skip ci] 2026-05-25 19:56:14 +08:00
Your Name
c644cfe993 feat(awooop): show source ref gap prefixes
All checks were successful
CD Pipeline / tests (push) Successful in 1m31s
Code Review / ai-code-review (push) Successful in 25s
CD Pipeline / build-and-deploy (push) Successful in 4m13s
CD Pipeline / post-deploy-checks (push) Successful in 1m34s
2026-05-25 19:49:48 +08:00
Your Name
640e35977f docs(logbook): record source ref gap deploy [skip ci] 2026-05-25 19:12:55 +08:00
AWOOOI CD
d004561617 chore(cd): deploy 9b802aa [skip ci] 2026-05-25 19:11:06 +08:00
Your Name
9b802aa7c6 feat(awooop): surface telegram source ref gaps
All checks were successful
CD Pipeline / tests (push) Successful in 1m20s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m13s
CD Pipeline / post-deploy-checks (push) Successful in 1m47s
2026-05-25 19:06:10 +08:00
Your Name
d0084a5f44 docs(logbook): record telegram source refs deploy [skip ci] 2026-05-25 19:02:09 +08:00
AWOOOI CD
0172d3cfa6 chore(cd): deploy 23fc499 [skip ci] 2026-05-25 19:00:33 +08:00
Your Name
23fc499b97 feat(telegram): extract incident refs from callback buttons
All checks were successful
CD Pipeline / tests (push) Successful in 1m21s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m35s
CD Pipeline / post-deploy-checks (push) Successful in 1m19s
2026-05-25 18:55:28 +08:00
Your Name
c792f37440 docs(logbook): record legacy callback gap deploy [skip ci] 2026-05-25 17:59:51 +08:00
AWOOOI CD
ea151ea54f chore(cd): deploy 411c0b2 [skip ci] 2026-05-25 17:58:00 +08:00
Your Name
411c0b2bc0 fix(awooop): clarify legacy callback snapshot gaps
All checks were successful
CD Pipeline / tests (push) Successful in 1m18s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m22s
2026-05-25 17:53:01 +08:00
Your Name
41856b2e9b docs(logbook): record callback snapshot verification [skip ci] 2026-05-25 17:49:36 +08:00
AWOOOI CD
5f1c33d73a chore(cd): deploy 5d05aa3 [skip ci] 2026-05-25 17:46:59 +08:00
Your Name
5d05aa38c5 fix(awooop): mark mixed callback snapshots partial
All checks were successful
CD Pipeline / tests (push) Successful in 1m15s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m37s
CD Pipeline / post-deploy-checks (push) Successful in 1m43s
2026-05-25 17:41:57 +08:00
Your Name
72c4ccbf86 docs(logbook): record callback coverage deploy [skip ci] 2026-05-25 17:20:35 +08:00
AWOOOI CD
6e122f0b58 chore(cd): deploy 44d24b1 [skip ci] 2026-05-25 17:13:37 +08:00
Your Name
44d24b1858 fix(awooop): keep callback audit summary stable
All checks were successful
CD Pipeline / tests (push) Successful in 1m15s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m51s
CD Pipeline / post-deploy-checks (push) Successful in 1m30s
2026-05-25 17:08:14 +08:00
AWOOOI CD
0c1f9a1e37 chore(cd): deploy 449c4ac [skip ci] 2026-05-25 17:02:03 +08:00
Your Name
449c4ac807 feat(awooop): surface telegram callback coverage
All checks were successful
CD Pipeline / tests (push) Successful in 1m21s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m59s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
2026-05-25 16:56:28 +08:00
Your Name
b7ee1f47ff docs(logbook): record telegram evidence chain deploy [skip ci] 2026-05-25 16:44:49 +08:00
AWOOOI CD
6116498a32 chore(cd): deploy f844822 [skip ci] 2026-05-25 16:40:36 +08:00
Your Name
f84482299b feat(telegram): surface awooop agent evidence chain
All checks were successful
CD Pipeline / tests (push) Successful in 1m15s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m28s
CD Pipeline / post-deploy-checks (push) Successful in 1m38s
2026-05-25 16:35:27 +08:00
Your Name
2e0d7f65c1 docs(logbook): record agent evidence chain deploy [skip ci] 2026-05-25 16:21:34 +08:00
AWOOOI CD
3fa628417e chore(cd): deploy b30005f [skip ci] 2026-05-25 16:16:07 +08:00
Your Name
b30005f4c1 fix(web): use run detail i18n namespace
All checks were successful
CD Pipeline / tests (push) Successful in 1m18s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m14s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-25 16:11:21 +08:00
AWOOOI CD
c38a3a9794 chore(cd): deploy 48a31ea [skip ci] 2026-05-25 16:03:58 +08:00
Your Name
48a31ea2b9 feat(web): surface awooop agent evidence chain
All checks were successful
CD Pipeline / tests (push) Successful in 1m22s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m28s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-25 15:57:56 +08:00
Your Name
683984dc47 docs(logbook): record homepage truth metrics deploy [skip ci] 2026-05-25 15:42:27 +08:00
AWOOOI CD
a64145fddf chore(cd): deploy ffe479d [skip ci] 2026-05-25 15:35:03 +08:00
Your Name
ffe479dbcc fix(web): align homepage automation truth metrics
All checks were successful
CD Pipeline / tests (push) Successful in 1m18s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m26s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
2026-05-25 15:30:00 +08:00
Your Name
d6d7c27152 docs(logbook): record work item i18n deploy [skip ci] 2026-05-25 15:18:26 +08:00
AWOOOI CD
a8c0ee2af1 chore(cd): deploy cd5cabd [skip ci] 2026-05-25 15:15:17 +08:00
Your Name
cd5cabd952 fix(web): repair awooop work item i18n namespace
All checks were successful
CD Pipeline / tests (push) Successful in 1m20s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m14s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
2026-05-25 15:10:24 +08:00
Your Name
6b28e1ecc1 docs(logbook): record ai route work item deploy [skip ci] 2026-05-25 14:52:15 +08:00
AWOOOI CD
bd5340cfe1 chore(cd): deploy 63b4c34 [skip ci] 2026-05-25 14:48:06 +08:00
Your Name
63b4c3453f feat(awooop): project ai route repair work item
All checks were successful
CD Pipeline / tests (push) Successful in 1m22s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m30s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
2026-05-25 14:42:57 +08:00
Your Name
e5cd01c9cb docs(logbook): record ai route evidence deploy [skip ci] 2026-05-25 14:32:18 +08:00
AWOOOI CD
24d9f25fe7 chore(cd): deploy 6729674 [skip ci] 2026-05-25 14:26:55 +08:00
Your Name
67296746c0 feat(awooop): surface ai route repair evidence
All checks were successful
CD Pipeline / tests (push) Successful in 1m24s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m53s
CD Pipeline / post-deploy-checks (push) Successful in 1m48s
2026-05-25 14:21:25 +08:00
Your Name
e570d9f6a9 docs(logbook): record gcp-a repair evidence [skip ci] 2026-05-25 14:06:18 +08:00
Your Name
62b07a95ff docs(logbook): record ai route lane deploy [skip ci] 2026-05-25 13:34:19 +08:00
AWOOOI CD
463229848c chore(cd): deploy ed3e658 [skip ci] 2026-05-25 13:30:10 +08:00
Your Name
ed3e658578 feat(awooop): surface degraded ai route lanes
All checks were successful
CD Pipeline / tests (push) Successful in 1m25s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m37s
CD Pipeline / post-deploy-checks (push) Successful in 1m44s
2026-05-25 13:24:53 +08:00
Your Name
19d306c720 docs(logbook): record ollama policy order deploy [skip ci] 2026-05-25 12:47:47 +08:00
AWOOOI CD
1cb480427e chore(cd): deploy b9fc874 [skip ci] 2026-05-25 12:43:21 +08:00
Your Name
b9fc8748a5 fix(ollama): enforce prod provider order
All checks were successful
CD Pipeline / tests (push) Successful in 1m21s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 5m15s
CD Pipeline / post-deploy-checks (push) Successful in 1m44s
2026-05-25 12:35:17 +08:00
Your Name
fe3f1e39fc fix(ollama): route prod primary to repaired gcp-b
Some checks failed
CD Pipeline / tests (push) Successful in 1m22s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-25 12:31:50 +08:00
AWOOOI CD
58909a5c31 chore(cd): deploy 9ccf230 [skip ci] 2026-05-25 12:30:59 +08:00
Your Name
9ccf230a5f fix(ollama): cooldown provider health probes
Some checks failed
CD Pipeline / tests (push) Successful in 1m24s
Code Review / ai-code-review (push) Successful in 17s
CD Pipeline / build-and-deploy (push) Successful in 3m37s
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-25 12:25:32 +08:00
AWOOOI CD
b9356ba1f4 chore(cd): deploy 2dcd214 [skip ci] 2026-05-25 12:16:44 +08:00
Your Name
2dcd214156 fix(ollama): cooldown noisy failed endpoints
All checks were successful
CD Pipeline / tests (push) Successful in 58s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m45s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-25 12:11:48 +08:00
AWOOOI CD
8a78344bcc chore(cd): deploy 6f1e788 [skip ci] 2026-05-25 12:07:55 +08:00
Your Name
6f1e788b67 fix(ollama): fail over prod to local 111 while GCP endpoints are down
All checks were successful
CD Pipeline / tests (push) Successful in 57s
Code Review / ai-code-review (push) Successful in 30s
CD Pipeline / build-and-deploy (push) Successful in 3m25s
CD Pipeline / post-deploy-checks (push) Successful in 1m46s
2026-05-25 12:03:02 +08:00
Your Name
3aed1f3123 docs(logbook): record ollama fallback deploy success [skip ci] 2026-05-25 11:59:05 +08:00
AWOOOI CD
979eb0fdd0 chore(cd): deploy 5298786 [skip ci] 2026-05-25 11:54:10 +08:00
Your Name
a909bc2ce9 fix(ansible): satisfy ollama fallback lint
All checks were successful
Ansible Lint / lint (push) Successful in 32s
2026-05-25 11:50:40 +08:00
Your Name
5298786180 fix(ollama): restore 111 fallback before gemini
Some checks failed
Ansible Lint / lint (push) Failing after 39s
CD Pipeline / tests (push) Successful in 56s
Code Review / ai-code-review (push) Successful in 7s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m36s
2026-05-25 11:48:53 +08:00
Your Name
46292459b7 docs(logbook): record callback capture list summary [skip ci] 2026-05-25 11:28:39 +08:00
Your Name
f169085cd3 chore(cd): deploy e1e640f [skip ci] 2026-05-25 11:26:46 +08:00
AWOOOI CD
4edcb5b586 chore(cd): deploy e1e640f [skip ci] 2026-05-25 11:21:22 +08:00
Your Name
e1e640f5d5 feat(awooop): summarize callback capture in runs list
All checks were successful
CD Pipeline / tests (push) Successful in 38s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m49s
CD Pipeline / post-deploy-checks (push) Successful in 1m43s
2026-05-25 11:16:27 +08:00
Your Name
814a44d539 docs(logbook): record callback capture status [skip ci] 2026-05-25 11:06:12 +08:00
AWOOOI CD
3ca834c31d chore(cd): deploy 04684ee [skip ci] 2026-05-25 11:00:06 +08:00
Your Name
04684eef5f feat(awooop): show callback evidence capture status
All checks were successful
CD Pipeline / tests (push) Successful in 1m5s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / build-and-deploy (push) Successful in 4m30s
CD Pipeline / post-deploy-checks (push) Successful in 1m55s
2026-05-25 10:54:39 +08:00
Your Name
1c8ebdf283 docs(logbook): record callback source snapshots [skip ci] 2026-05-25 10:38:43 +08:00
AWOOOI CD
c573fd42dd chore(cd): deploy dd1c513 [skip ci] 2026-05-25 10:34:14 +08:00
Your Name
dd1c513841 feat(telegram): persist callback evidence source snapshots
All checks were successful
CD Pipeline / tests (push) Successful in 1m3s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 3m41s
CD Pipeline / post-deploy-checks (push) Successful in 1m54s
2026-05-25 10:28:43 +08:00
AWOOOI CD
0a845498ff chore(cd): deploy ca0045e [skip ci] 2026-05-25 10:18:11 +08:00
Your Name
753879b45f docs(logbook): record GCP-A Ollama failover 2026-05-25 10:16:04 +08:00
Your Name
ca0045eeeb fix(ollama): fail over primary to GCP-B while GCP-A is unreachable
All checks were successful
CD Pipeline / tests (push) Successful in 1m8s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Successful in 3m46s
CD Pipeline / post-deploy-checks (push) Successful in 1m19s
2026-05-25 10:13:29 +08:00
Your Name
01284d1e4f docs(logbook): record callback status chain snapshots [skip ci] 2026-05-25 10:10:38 +08:00
AWOOOI CD
9aba9974e6 chore(cd): deploy daf9d4b [skip ci] 2026-05-25 10:04:51 +08:00
Your Name
daf9d4b00b feat(telegram): persist callback status chain snapshots
All checks were successful
CD Pipeline / tests (push) Successful in 1m8s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m23s
CD Pipeline / post-deploy-checks (push) Successful in 1m31s
2026-05-25 09:58:42 +08:00
Your Name
4818ba45c0 docs(logbook): record callback evidence snapshots [skip ci] 2026-05-25 09:34:27 +08:00
AWOOOI CD
1bee07e765 chore(cd): deploy 263d752 [skip ci] 2026-05-25 09:28:40 +08:00
Your Name
263d752367 feat(telegram): persist callback owner review snapshots
All checks were successful
CD Pipeline / tests (push) Successful in 1m10s
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / build-and-deploy (push) Successful in 4m23s
CD Pipeline / post-deploy-checks (push) Successful in 1m28s
2026-05-25 09:23:35 +08:00
Your Name
862f35fee7 docs(logbook): record telegram owner review triage [skip ci] 2026-05-25 09:12:28 +08:00
AWOOOI CD
42efb2fbe8 chore(cd): deploy eeece58 [skip ci] 2026-05-25 09:07:40 +08:00
Your Name
eeece58c0d feat(telegram): show callback owner review triage
All checks were successful
CD Pipeline / tests (push) Successful in 1m11s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m11s
CD Pipeline / post-deploy-checks (push) Successful in 1m29s
2026-05-25 09:01:50 +08:00
Your Name
b466674621 docs(logbook): record callback owner review triage [skip ci] 2026-05-25 08:57:18 +08:00
AWOOOI CD
386468305e chore(cd): deploy 383a29a [skip ci] 2026-05-25 08:51:07 +08:00
Your Name
383a29a139 feat(governance): show callback owner review triage
All checks were successful
CD Pipeline / tests (push) Successful in 1m9s
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / build-and-deploy (push) Successful in 4m10s
CD Pipeline / post-deploy-checks (push) Successful in 1m27s
2026-05-25 08:46:21 +08:00
Your Name
b184a09086 docs(logbook): record callback owner review work items [skip ci] 2026-05-25 08:40:30 +08:00
AWOOOI CD
ea75ea4633 chore(cd): deploy 73aad41 [skip ci] 2026-05-25 08:33:54 +08:00
Your Name
73aad41359 fix(governance): link callback work item back to queue
All checks were successful
CD Pipeline / tests (push) Successful in 1m8s
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / build-and-deploy (push) Successful in 4m26s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-25 08:27:34 +08:00
AWOOOI CD
390b13e873 chore(cd): deploy 1566609 [skip ci] 2026-05-25 08:19:25 +08:00
Your Name
156660929e feat(governance): surface callback owner review work items
All checks were successful
CD Pipeline / tests (push) Successful in 1m18s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m36s
CD Pipeline / post-deploy-checks (push) Successful in 2m9s
2026-05-25 08:14:01 +08:00
Your Name
2c2446e56e docs(logbook): record km completion callback evidence [skip ci] 2026-05-25 01:21:27 +08:00
AWOOOI CD
fcaaad8708 chore(cd): deploy 760d674 [skip ci] 2026-05-25 00:01:18 +08:00
Your Name
760d6745a5 feat(governance): surface km completion callback evidence
Some checks failed
CD Pipeline / tests (push) Successful in 1m10s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m19s
E2E Health Check / e2e-health (push) Failing after 34s
CD Pipeline / post-deploy-checks (push) Successful in 1m46s
2026-05-24 23:55:16 +08:00
Your Name
318ca645d0 docs(logbook): record km completion detail visibility [skip ci] 2026-05-24 23:42:32 +08:00
AWOOOI CD
a76c5e0801 chore(cd): deploy ac46866 [skip ci] 2026-05-24 23:36:29 +08:00
Your Name
ac4686615f feat(governance): surface km completion state in details
All checks were successful
CD Pipeline / tests (push) Successful in 1m10s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m12s
CD Pipeline / post-deploy-checks (push) Successful in 1m49s
2026-05-24 23:31:16 +08:00
Your Name
ede2b3752b docs(logbook): record stale km completion preview rollout [skip ci] 2026-05-24 23:26:36 +08:00
AWOOOI CD
825de2ef58 chore(cd): deploy 4cfc6a4 [skip ci] 2026-05-24 23:20:20 +08:00
Your Name
4cfc6a4c79 feat(governance): preview stale km completion batches
All checks were successful
CD Pipeline / tests (push) Successful in 1m8s
Code Review / ai-code-review (push) Successful in 11s
Type Sync Check / check-type-sync (push) Successful in 26s
CD Pipeline / build-and-deploy (push) Successful in 4m9s
CD Pipeline / post-deploy-checks (push) Successful in 1m33s
2026-05-24 23:15:03 +08:00
Your Name
1a4ac330b1 docs(logbook): record stale km completion queue rollout [skip ci] 2026-05-24 23:04:25 +08:00
AWOOOI CD
c16b2931e8 chore(cd): deploy 0e447bb [skip ci] 2026-05-24 22:58:54 +08:00
Your Name
0e447bbe47 test(gitea): skip review background tasks in mock mode
All checks were successful
CD Pipeline / tests (push) Successful in 1m8s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 5m57s
CD Pipeline / post-deploy-checks (push) Successful in 1m30s
2026-05-24 22:52:10 +08:00
Your Name
0a8a15075a feat(governance): surface stale km completion queue
Some checks failed
CD Pipeline / tests (push) Successful in 5m28s
Code Review / ai-code-review (push) Successful in 11s
Type Sync Check / check-type-sync (push) Successful in 25s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-24 22:42:59 +08:00
Your Name
bd2762e76c docs(logbook): record stale km burndown rollout [skip ci] 2026-05-24 22:27:43 +08:00
AWOOOI CD
a68bc7f024 chore(cd): deploy ded2223 [skip ci] 2026-05-24 22:22:07 +08:00
Your Name
ded2223d14 feat(governance): surface stale km burndown
All checks were successful
CD Pipeline / tests (push) Successful in 5m28s
Code Review / ai-code-review (push) Successful in 12s
Type Sync Check / check-type-sync (push) Successful in 25s
CD Pipeline / build-and-deploy (push) Successful in 4m6s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-24 22:11:33 +08:00
Your Name
f4253f22f8 docs(logbook): record stale km owner review inbox rollout [skip ci] 2026-05-24 21:48:52 +08:00
AWOOOI CD
63be59ef8a chore(cd): deploy 0c447ac [skip ci] 2026-05-24 21:43:02 +08:00
Your Name
0c447acb19 feat(governance): surface stale km owner review inbox
All checks were successful
CD Pipeline / tests (push) Successful in 5m29s
Code Review / ai-code-review (push) Successful in 16s
Type Sync Check / check-type-sync (push) Successful in 28s
CD Pipeline / build-and-deploy (push) Successful in 4m12s
CD Pipeline / post-deploy-checks (push) Successful in 1m30s
2026-05-24 21:32:29 +08:00
Your Name
d04377dd20 docs(logbook): add stale km batch browser smoke [skip ci] 2026-05-24 21:07:44 +08:00
Your Name
beb1c9006b docs(logbook): record stale km batch queue rollout [skip ci] 2026-05-24 21:05:16 +08:00
AWOOOI CD
a0ac6c090a chore(cd): deploy 943093a [skip ci] 2026-05-24 20:57:35 +08:00
Your Name
943093a49b feat(governance): batch queue stale km reviews
All checks were successful
CD Pipeline / tests (push) Successful in 5m47s
Code Review / ai-code-review (push) Successful in 11s
Type Sync Check / check-type-sync (push) Successful in 27s
CD Pipeline / build-and-deploy (push) Successful in 4m13s
CD Pipeline / post-deploy-checks (push) Successful in 2m11s
2026-05-24 20:47:31 +08:00
Your Name
fb40b8f469 docs(logbook): record stale km completion rollout [skip ci] 2026-05-24 18:46:27 +08:00
AWOOOI CD
63642f3dcb chore(cd): deploy 630cd53 [skip ci] 2026-05-24 18:38:40 +08:00
Your Name
630cd5381c feat(governance): complete stale km owner review
All checks were successful
CD Pipeline / tests (push) Successful in 5m28s
Code Review / ai-code-review (push) Successful in 12s
Type Sync Check / check-type-sync (push) Successful in 26s
CD Pipeline / build-and-deploy (push) Successful in 5m12s
CD Pipeline / post-deploy-checks (push) Successful in 1m31s
2026-05-24 18:28:10 +08:00
Your Name
00cf6f009d docs(logbook): record km owner review queue rollout [skip ci] 2026-05-24 18:07:57 +08:00
AWOOOI CD
cda1f86633 chore(cd): deploy 9bdeebe [skip ci] 2026-05-24 18:00:47 +08:00
Your Name
9bdeebeb1e feat(governance): queue stale km owner review
All checks were successful
CD Pipeline / tests (push) Successful in 5m28s
Code Review / ai-code-review (push) Successful in 14s
Type Sync Check / check-type-sync (push) Successful in 27s
CD Pipeline / build-and-deploy (push) Successful in 4m19s
CD Pipeline / post-deploy-checks (push) Successful in 1m39s
2026-05-24 17:40:42 +08:00
Your Name
7bb03652f2 docs(logbook): record km stale queue rollout [skip ci] 2026-05-24 17:26:13 +08:00
AWOOOI CD
96d812b7cc chore(cd): deploy 9b01f1f [skip ci] 2026-05-24 17:19:33 +08:00
Your Name
9b01f1fa46 fix(api): serialize startup bootstrap ddl
All checks were successful
CD Pipeline / tests (push) Successful in 5m29s
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / build-and-deploy (push) Successful in 4m9s
CD Pipeline / post-deploy-checks (push) Successful in 1m57s
2026-05-24 17:10:26 +08:00
AWOOOI CD
5b8f14e32e chore(cd): deploy 841b057 [skip ci] 2026-05-24 16:56:55 +08:00
Your Name
841b057ada feat(governance): surface stale km priority queue
Some checks failed
CD Pipeline / tests (push) Successful in 5m29s
Code Review / ai-code-review (push) Successful in 11s
Type Sync Check / check-type-sync (push) Successful in 32s
CD Pipeline / build-and-deploy (push) Failing after 5m43s
CD Pipeline / post-deploy-checks (push) Has been skipped
2026-05-24 16:46:14 +08:00
Your Name
b87090be01 docs(governance): record t153 km degradation rollout [skip ci] 2026-05-24 16:30:12 +08:00
AWOOOI CD
c9b2e763f5 chore(cd): deploy de68514 [skip ci] 2026-05-24 16:24:48 +08:00
Your Name
de68514283 fix(governance): dedupe km degradation owner review
All checks were successful
CD Pipeline / tests (push) Successful in 5m4s
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / build-and-deploy (push) Successful in 4m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m38s
2026-05-24 16:14:51 +08:00
Your Name
7fd52d26b5 docs(awooop): record t152 ansible runtime readiness [skip ci] 2026-05-24 16:00:55 +08:00
AWOOOI CD
9d89cdddea chore(cd): deploy 5dacdb4 [skip ci] 2026-05-24 15:48:03 +08:00
Your Name
5dacdb4738 fix(awooop): resolve ansible runtime path in container
All checks were successful
CD Pipeline / tests (push) Successful in 5m46s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m53s
CD Pipeline / post-deploy-checks (push) Successful in 1m51s
2026-05-24 15:36:32 +08:00
AWOOOI CD
1a6ce1bcd4 chore(cd): deploy 0423c43 [skip ci] 2026-05-24 15:30:17 +08:00
Your Name
0423c43b84 fix(web): repair automation evidence runtime detail jsx
Some checks failed
CD Pipeline / tests (push) Failing after 3m58s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
Code Review / ai-code-review (push) Successful in 12s
2026-05-24 15:16:46 +08:00
Your Name
0b2657e546 fix(awooop): locate ansible catalog from api cwd
Some checks failed
CD Pipeline / tests (push) Successful in 5m49s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Failing after 1m25s
CD Pipeline / post-deploy-checks (push) Has been skipped
2026-05-24 15:06:13 +08:00
Your Name
1322216f73 feat(awooop): expose ansible runtime readiness
Some checks failed
CD Pipeline / tests (push) Failing after 51s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
Code Review / ai-code-review (push) Successful in 12s
2026-05-24 15:01:51 +08:00
Your Name
4874f2b649 docs(awooop): record t151 execution evidence [skip ci] 2026-05-24 14:55:05 +08:00
AWOOOI CD
cd81d604d9 chore(cd): deploy dc09dac [skip ci] 2026-05-24 14:45:10 +08:00
Your Name
dc09dac4d4 feat(awooop): surface execution backend evidence
All checks were successful
CD Pipeline / tests (push) Successful in 5m49s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m5s
CD Pipeline / post-deploy-checks (push) Successful in 1m46s
2026-05-24 14:35:42 +08:00
Your Name
17b62da59a docs(awooop): record t150 rollout evidence [skip ci] 2026-05-24 14:28:43 +08:00
Your Name
b98f93a62f fix(ci): include argocd resource evidence in rollout risk
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-24 14:26:53 +08:00
Your Name
a282eb8c97 docs(awooop): record t149 argocd cleanup [skip ci] 2026-05-24 14:23:19 +08:00
AWOOOI CD
6a41f1c22f chore(cd): deploy 4d622f1 [skip ci] 2026-05-24 14:10:33 +08:00
Your Name
4d622f184d fix(k8s): stop retaining failed cronjob noise
All checks were successful
CD Pipeline / tests (push) Successful in 5m54s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 4m32s
CD Pipeline / post-deploy-checks (push) Successful in 1m44s
2026-05-24 13:59:39 +08:00
Your Name
9281c11eea docs(awooop): record t148 route fallback [skip ci] 2026-05-24 13:56:02 +08:00
AWOOOI CD
6428a15a11 chore(cd): deploy 478e25b [skip ci] 2026-05-24 13:51:09 +08:00
Your Name
478e25b6a2 fix(api): fallback ai route status to connectivity
All checks were successful
CD Pipeline / tests (push) Successful in 5m59s
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / build-and-deploy (push) Successful in 4m17s
CD Pipeline / post-deploy-checks (push) Successful in 1m36s
2026-05-24 13:39:20 +08:00
Your Name
82e471a7f2 docs(awooop): record t147 evidence fallback [skip ci] 2026-05-24 13:34:30 +08:00
AWOOOI CD
bca493e83c chore(cd): deploy df922e8 [skip ci] 2026-05-24 13:27:18 +08:00
Your Name
df922e8c67 fix(web): keep evidence visible when quality fails
All checks were successful
CD Pipeline / tests (push) Successful in 4m56s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m57s
CD Pipeline / post-deploy-checks (push) Successful in 1m46s
2026-05-24 13:18:57 +08:00
AWOOOI CD
05dd8450a8 chore(cd): deploy 54f227c [skip ci] 2026-05-24 13:12:17 +08:00
Your Name
54f227c597 fix(web): render evidence card before quality summary
All checks were successful
CD Pipeline / tests (push) Successful in 5m57s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m57s
CD Pipeline / post-deploy-checks (push) Successful in 1m46s
2026-05-24 13:02:44 +08:00
Your Name
12c39a17a8 docs(awooop): record t145 route evidence [skip ci] 2026-05-24 12:54:25 +08:00
AWOOOI CD
80ccf8c16f chore(cd): deploy bdccb80 [skip ci] 2026-05-24 12:48:13 +08:00
Your Name
bdccb80ed7 fix(api): bound ai route status checks
All checks were successful
CD Pipeline / tests (push) Successful in 5m39s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m10s
CD Pipeline / post-deploy-checks (push) Successful in 1m34s
2026-05-24 12:38:51 +08:00
AWOOOI CD
b17acbb043 chore(cd): deploy df06c02 [skip ci] 2026-05-24 12:26:22 +08:00
Your Name
df06c025ff fix(web): show ai route fallback evidence
All checks were successful
CD Pipeline / tests (push) Successful in 5m57s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m56s
CD Pipeline / post-deploy-checks (push) Successful in 1m34s
2026-05-24 12:15:02 +08:00
Your Name
b20daeabd8 docs(awooop): record t144 provider chain evidence [skip ci] 2026-05-24 12:01:50 +08:00
AWOOOI CD
c932635057 chore(cd): deploy 9bac571 [skip ci] 2026-05-24 11:54:52 +08:00
Your Name
9bac5718da feat(health): expose ollama provider chain
All checks were successful
CD Pipeline / tests (push) Successful in 6m8s
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / build-and-deploy (push) Successful in 4m38s
CD Pipeline / post-deploy-checks (push) Successful in 1m42s
2026-05-24 11:44:37 +08:00
Your Name
06dfdf7ead docs(awooop): record t143 probe and cd evidence repair [skip ci] 2026-05-24 11:25:19 +08:00
AWOOOI CD
7211d0b7f2 chore(cd): deploy 22a4b44 [skip ci] 2026-05-24 11:14:22 +08:00
Your Name
22a4b44aef fix(ci): report provider degradation as warning
All checks were successful
CD Pipeline / tests (push) Successful in 5m55s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 3m59s
CD Pipeline / post-deploy-checks (push) Successful in 1m48s
2026-05-24 10:59:21 +08:00
AWOOOI CD
f3b85cda4f chore(cd): deploy 19de834 [skip ci] 2026-05-24 10:53:44 +08:00
Your Name
19de834557 fix(cd): gate deploy on synced revision
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
2026-05-24 10:43:05 +08:00
AWOOOI CD
a6328c3864 chore(cd): deploy abcca65 [skip ci] 2026-05-24 10:38:07 +08:00
Your Name
abcca6521c fix(cd): use ready k8s control plane
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-24 10:27:36 +08:00
Your Name
8558ac2d20 fix(k8s): use lightweight api probes
Some checks failed
CD Pipeline / tests (push) Successful in 6m51s
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / build-and-deploy (push) Failing after 4m11s
CD Pipeline / post-deploy-checks (push) Has been skipped
2026-05-24 10:11:20 +08:00
Your Name
6d2b0ed4cd ops(runner): add isolation readiness gate [skip ci] 2026-05-24 09:56:47 +08:00
Your Name
4407b46bb6 ops(runner): inventory workflow labels [skip ci] 2026-05-24 09:52:04 +08:00
Your Name
22b45006b7 ops(runner): add pool inventory audit [skip ci] 2026-05-24 09:47:02 +08:00
Your Name
8ddc783af5 docs(awooop): record t139 stage evidence [skip ci] 2026-05-21 20:56:40 +08:00
AWOOOI CD
5ed577481f chore(cd): deploy f322781 [skip ci] 2026-05-21 20:49:48 +08:00
Your Name
f322781798 ci(cd): expose build and post-deploy stages
All checks were successful
CD Pipeline / tests (push) Successful in 9m16s
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / build-and-deploy (push) Successful in 4m47s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
2026-05-21 20:35:09 +08:00
Your Name
f5f3a10bf6 docs(awooop): record t138 cicd evidence surface [skip ci] 2026-05-21 20:30:35 +08:00
AWOOOI CD
a5ed12937c chore(cd): deploy 4bdb012 [skip ci] 2026-05-21 20:16:38 +08:00
Your Name
4bdb012caa feat(awooop): surface cicd rollout evidence
All checks were successful
CD Pipeline / tests (push) Successful in 4m1s
Code Review / ai-code-review (push) Successful in 17s
CD Pipeline / build-and-deploy (push) Successful in 3m27s
CD Pipeline / post-deploy-checks (push) Successful in 1m49s
2026-05-21 20:06:26 +08:00
Your Name
0c59a1aafd docs(awooop): record t137 rollout risk evidence [skip ci] 2026-05-21 19:53:00 +08:00
AWOOOI CD
77e443a681 chore(cd): deploy 8e68dc1 [skip ci] 2026-05-21 19:45:00 +08:00
Your Name
8e68dc1e35 ci(cd): surface recovered rollout risk evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
2026-05-21 19:37:30 +08:00
Your Name
4887708717 docs(awooop): record t136 api image layering evidence [skip ci] 2026-05-21 19:32:51 +08:00
AWOOOI CD
460cc19e76 chore(cd): deploy 4d6f722 [skip ci] 2026-05-21 19:21:03 +08:00
Your Name
4d6f7225d9 ci(api): avoid runtime image chown rebuilds
All checks were successful
CD Pipeline / tests (push) Successful in 3m57s
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / build-and-deploy (push) Successful in 11m51s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-21 19:12:30 +08:00
Your Name
da8456cf07 docs(awooop): record t135 runner ownership evidence [skip ci] 2026-05-21 19:10:21 +08:00
AWOOOI CD
5aa46bc95e chore(cd): deploy 9b465ee [skip ci] 2026-05-21 19:02:08 +08:00
Your Name
9b465ee140 ci(runner): drain legacy docker act runner safely
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-21 18:53:45 +08:00
Your Name
19739339e7 docs(awooop): record t134 runner cleanup evidence [skip ci] 2026-05-21 18:43:08 +08:00
AWOOOI CD
7ed4b19b0c chore(cd): deploy d3d1c2c [skip ci] 2026-05-21 18:35:50 +08:00
AWOOOI CD
d3d1c2c27a chore(cd): deploy 75f1ef0 [skip ci] 2026-05-21 18:05:05 +08:00
Your Name
7cc898caf1 ci(cd): include api bytecode in runner cleanup
All checks were successful
Code Review / ai-code-review (push) Successful in 14s
2026-05-21 18:02:23 +08:00
Your Name
75f1ef0ca1 ci(cd): clean host runner workspace artifacts
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
2026-05-21 17:55:55 +08:00
Your Name
e4c3662814 docs(awooop): record t133 dockerfile cleanup [skip ci] 2026-05-21 16:27:18 +08:00
AWOOOI CD
918e918641 chore(cd): deploy 2603e43 [skip ci] 2026-05-21 08:21:56 +00:00
Your Name
2603e43bf2 chore(web): normalize docker env syntax
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 4m0s
CD Pipeline / build-and-deploy (push) Successful in 5m24s
CD Pipeline / post-deploy-checks (push) Successful in 2m3s
2026-05-21 16:13:08 +08:00
Your Name
12adc1e364 docs(awooop): record t132 dispatch evidence [skip ci] 2026-05-21 16:11:21 +08:00
AWOOOI CD
c44188b8ba chore(cd): deploy 251f5ad [skip ci] 2026-05-21 16:04:45 +08:00
Your Name
251f5ad658 docs(awooop): record t132 runner pressure gate [skip ci] 2026-05-21 15:53:43 +08:00
Your Name
b3ab4da03b ci(cd): wait for host web build pressure
All checks were successful
Code Review / ai-code-review (push) Successful in 17s
2026-05-21 15:51:36 +08:00
Your Name
8164121870 docs(awooop): record t131 snapshot hydration [skip ci] 2026-05-21 15:42:06 +08:00
AWOOOI CD
290f409d80 chore(cd): deploy b63c829 [skip ci] 2026-05-21 07:36:52 +00:00
Your Name
b63c829f9a fix(web): stabilize dashboard snapshot hydration
All checks were successful
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / tests (push) Successful in 4m9s
CD Pipeline / build-and-deploy (push) Successful in 4m21s
CD Pipeline / post-deploy-checks (push) Successful in 2m23s
2026-05-21 15:28:21 +08:00
Your Name
efc454a346 docs(awooop): record t130 overview actions [skip ci] 2026-05-21 15:22:35 +08:00
AWOOOI CD
6725aaae5b chore(cd): deploy d94f427 [skip ci] 2026-05-21 15:16:06 +08:00
Your Name
d94f427a09 feat(awooop): add source flow action links
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m59s
CD Pipeline / build-and-deploy (push) Successful in 4m42s
CD Pipeline / post-deploy-checks (push) Successful in 2m36s
2026-05-21 15:08:09 +08:00
Your Name
0fc66370c7 docs(awooop): record t129 overview source flow [skip ci] 2026-05-21 14:59:08 +08:00
AWOOOI CD
59d1708034 chore(cd): deploy ce3f2fe [skip ci] 2026-05-21 06:53:10 +00:00
Your Name
ce3f2fed36 feat(awooop): surface source flow on overview
All checks were successful
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / tests (push) Successful in 5m52s
CD Pipeline / build-and-deploy (push) Successful in 4m45s
CD Pipeline / post-deploy-checks (push) Successful in 1m44s
2026-05-21 14:43:12 +08:00
Your Name
be585c4071 docs(awooop): record t128 approvals source flow [skip ci] 2026-05-21 14:32:42 +08:00
AWOOOI CD
992bb05e6b chore(cd): deploy 140c9cd [skip ci] 2026-05-21 06:27:53 +00:00
Your Name
140c9cdaef feat(awooop): show source flow in approvals
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m52s
CD Pipeline / build-and-deploy (push) Successful in 3m40s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-21 14:20:13 +08:00
Your Name
e89bb267ea docs(awooop): record t127 production readback [skip ci] 2026-05-21 14:12:03 +08:00
AWOOOI CD
39f0f7655c chore(cd): deploy ebb73af [skip ci] 2026-05-21 14:06:21 +08:00
Your Name
ebb73af16b feat(awooop): show source flow in work items
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 4m0s
CD Pipeline / build-and-deploy (push) Successful in 3m54s
CD Pipeline / post-deploy-checks (push) Successful in 2m2s
2026-05-21 13:58:17 +08:00
Your Name
2380d6f555 docs(awooop): record t126 production readback [skip ci] 2026-05-21 13:37:56 +08:00
AWOOOI CD
9206e27103 chore(cd): deploy 9c96669 [skip ci] 2026-05-21 13:32:39 +08:00
Your Name
9c966699f0 feat(awooop): show source flow in runs list
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 3m55s
CD Pipeline / build-and-deploy (push) Successful in 3m36s
CD Pipeline / post-deploy-checks (push) Successful in 1m39s
2026-05-21 13:24:53 +08:00
Your Name
3d1315e103 docs(awooop): record t125 frontend readback [skip ci] 2026-05-21 13:13:49 +08:00
AWOOOI CD
b0f9ab70d2 chore(cd): deploy 53a3c84 [skip ci] 2026-05-21 13:08:43 +08:00
Your Name
53a3c846e5 feat(awooop): surface source evidence flow
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m54s
CD Pipeline / build-and-deploy (push) Successful in 4m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m50s
2026-05-21 13:00:59 +08:00
Your Name
1ae8f0d179 docs(awooop): record t124 source link canary [skip ci] 2026-05-21 12:48:52 +08:00
AWOOOI CD
7ae59c1cb0 chore(cd): deploy 867e0e7 [skip ci] 2026-05-21 12:42:49 +08:00
Your Name
867e0e73df ci(awooop): add dedicated source link canary
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 4m8s
CD Pipeline / build-and-deploy (push) Successful in 4m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m58s
2026-05-21 12:34:51 +08:00
Your Name
89a5a2ea85 docs(awooop): record t123 refresh candidate gate [skip ci] 2026-05-21 12:27:39 +08:00
Your Name
4b6c9b9554 ci(awooop): verify source link refresh candidate
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-21 12:25:19 +08:00
Your Name
7f91159a1c docs(awooop): record t122 rolling canary verification [skip ci] 2026-05-21 12:20:12 +08:00
Your Name
31b95449ff ci(awooop): align source canary work item id
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-21 12:17:44 +08:00
Your Name
bbe081fc57 ci(awooop): refresh source correlation canary
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-21 12:13:07 +08:00
Your Name
8adae4788c docs(awooop): record t121 cd gate verification [skip ci] 2026-05-21 11:59:28 +08:00
AWOOOI CD
7b36864cca chore(cd): deploy 3f5fb9d [skip ci] 2026-05-21 03:55:42 +00:00
Your Name
3f5fb9d8b2 ci(awooop): gate source correlation applied link
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-21 11:45:39 +08:00
Your Name
b15b61d90b test(awooop): add source correlation apply smoke
All checks were successful
Code Review / ai-code-review (push) Successful in 15s
2026-05-21 11:26:54 +08:00
Your Name
50993a4566 docs(awooop): record t119 production verification [skip ci] 2026-05-21 11:05:53 +08:00
AWOOOI CD
5aaf4f4148 chore(cd): deploy efb38cf [skip ci] 2026-05-21 11:01:23 +08:00
Your Name
efb38cf6af feat(awooop): verify source correlation links in status chain
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 6m9s
CD Pipeline / build-and-deploy (push) Successful in 4m39s
CD Pipeline / post-deploy-checks (push) Successful in 1m55s
2026-05-21 10:51:20 +08:00
Your Name
ac7f642e41 docs(awooop): record t118 production verification [skip ci] 2026-05-21 10:36:49 +08:00
AWOOOI CD
593d928dea chore(cd): deploy fe3bf5d [skip ci] 2026-05-21 02:31:46 +00:00
Your Name
fe3bf5dc18 feat(awooop): apply source correlation links
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 4m1s
CD Pipeline / build-and-deploy (push) Successful in 4m8s
CD Pipeline / post-deploy-checks (push) Successful in 2m2s
2026-05-21 10:23:29 +08:00
Your Name
d25237a31f docs(awooop): record t117 production verification [skip ci] 2026-05-21 10:06:01 +08:00
AWOOOI CD
242b2f415d chore(cd): deploy 88e7477 [skip ci] 2026-05-21 10:01:32 +08:00
Your Name
88e7477a7c feat(awooop): record source correlation review decisions
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 3m59s
CD Pipeline / build-and-deploy (push) Successful in 3m48s
CD Pipeline / post-deploy-checks (push) Successful in 1m55s
2026-05-21 09:53:36 +08:00
Your Name
ee5a54ecba docs(awooop): record t116 source review rollout [skip ci] 2026-05-21 09:34:46 +08:00
AWOOOI CD
1c5781018c chore(cd): deploy f671637 [skip ci] 2026-05-21 09:28:04 +08:00
Your Name
f671637e23 fix(awooop): json-safe recurrence audit context
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 4m13s
CD Pipeline / build-and-deploy (push) Successful in 4m26s
CD Pipeline / post-deploy-checks (push) Successful in 1m57s
2026-05-21 09:20:00 +08:00
AWOOOI CD
72043adac1 chore(cd): deploy b5deca9 [skip ci] 2026-05-21 09:17:00 +08:00
Your Name
b5deca91df fix(awooop): record source review dry-run audit
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 4m4s
CD Pipeline / build-and-deploy (push) Successful in 3m34s
CD Pipeline / post-deploy-checks (push) Successful in 1m49s
2026-05-21 09:09:14 +08:00
AWOOOI CD
2e54b803f0 chore(cd): deploy cf8bb36 [skip ci] 2026-05-21 09:03:12 +08:00
Your Name
cf8bb364a3 feat(awooop): surface source evidence review work items
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 4m6s
CD Pipeline / build-and-deploy (push) Successful in 4m8s
CD Pipeline / post-deploy-checks (push) Successful in 1m57s
2026-05-21 08:54:45 +08:00
Your Name
a2cbf9e328 docs(awooop): record t115 provider canary rollout [skip ci] 2026-05-20 21:02:42 +08:00
AWOOOI CD
508df4c732 chore(cd): deploy f3fbd39 [skip ci] 2026-05-20 12:58:32 +00:00
Your Name
f3fbd39898 feat(awooop): add provider upstream canary
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 5m50s
CD Pipeline / build-and-deploy (push) Successful in 3m58s
CD Pipeline / post-deploy-checks (push) Successful in 1m48s
2026-05-20 20:48:36 +08:00
Your Name
e6cc008b87 docs(awooop): record t114 source correlation rollout [skip ci] 2026-05-20 20:33:59 +08:00
AWOOOI CD
b7aa90ae33 chore(cd): deploy ef95d1e [skip ci] 2026-05-20 20:27:43 +08:00
Your Name
ef95d1ef6b feat(awooop): show incident source correlation evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 4m4s
CD Pipeline / build-and-deploy (push) Successful in 3m58s
CD Pipeline / post-deploy-checks (push) Successful in 1m55s
2026-05-20 20:19:36 +08:00
Your Name
26cab7a324 docs(awooop): record t113 provider freshness heartbeat [skip ci] 2026-05-20 20:04:37 +08:00
AWOOOI CD
deccae937d chore(cd): deploy 017d57c [skip ci] 2026-05-20 19:58:58 +08:00
Your Name
017d57c96a fix(ci): use internal metrics for provider freshness smoke
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 4m0s
CD Pipeline / build-and-deploy (push) Successful in 3m30s
CD Pipeline / post-deploy-checks (push) Successful in 2m9s
2026-05-20 19:51:28 +08:00
AWOOOI CD
6003fd03ec chore(cd): deploy 31cae35 [skip ci] 2026-05-20 19:45:44 +08:00
Your Name
31cae35edd chore(cd): trigger source provider heartbeat deploy
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 4m7s
CD Pipeline / build-and-deploy (push) Successful in 3m52s
CD Pipeline / post-deploy-checks (push) Successful in 1m44s
2026-05-20 19:37:44 +08:00
Your Name
71380224b6 fix(ci): keep provider smoke secret out of step env
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-20 19:37:14 +08:00
Your Name
ced36f2521 feat(awooop): add source provider freshness heartbeat
Some checks failed
CD Pipeline / tests (push) Failing after 6s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
Code Review / ai-code-review (push) Failing after 8s
2026-05-20 19:32:22 +08:00
AWOOOI CD
b1f666826f chore(cd): deploy ae9d0b7 [skip ci] 2026-05-20 11:26:26 +00:00
Your Name
4ee9689483 docs(awooop): record t112 source provider freshness alert [skip ci] 2026-05-20 19:22:16 +08:00
Your Name
ae9d0b7385 feat(monitoring): alert on stale source provider ingestion
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 25s
CD Pipeline / tests (push) Successful in 3m26s
CD Pipeline / build-and-deploy (push) Successful in 3m38s
CD Pipeline / post-deploy-checks (push) Successful in 1m25s
2026-05-20 19:19:21 +08:00
Your Name
4a9d76d29e docs(awooop): record t111 source freshness rollout [skip ci] 2026-05-20 16:37:04 +08:00
AWOOOI CD
b7bab4abcc chore(cd): deploy c2bf579 [skip ci] 2026-05-20 08:33:10 +00:00
Your Name
c2bf579a99 feat(web): show source provider freshness on alerts
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 3m55s
CD Pipeline / build-and-deploy (push) Successful in 3m45s
CD Pipeline / post-deploy-checks (push) Successful in 2m25s
2026-05-20 16:25:26 +08:00
Your Name
d84bae95cf docs(awooop): record t110 source coverage rollout [skip ci] 2026-05-20 16:18:52 +08:00
AWOOOI CD
eea9c82f91 chore(cd): deploy 49ad1cf [skip ci] 2026-05-20 16:12:27 +08:00
Your Name
49ad1cfb1a feat(web): show source dossier coverage on alerts
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m51s
CD Pipeline / build-and-deploy (push) Successful in 3m32s
CD Pipeline / post-deploy-checks (push) Successful in 1m40s
2026-05-20 16:05:01 +08:00
Your Name
31a49c72de docs(awooop): record t109 source refs rollout [skip ci] 2026-05-20 15:46:37 +08:00
AWOOOI CD
2d37149eaf chore(cd): deploy 3aa90b8 [skip ci] 2026-05-20 15:42:49 +08:00
Your Name
3aa90b8ecf feat(awooop): expose source refs on incidents
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m58s
CD Pipeline / build-and-deploy (push) Successful in 3m36s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
2026-05-20 15:35:13 +08:00
Your Name
a60896bd78 docs(awooop): record t108 execution evidence rollout [skip ci] 2026-05-20 15:31:06 +08:00
AWOOOI CD
f79e671819 chore(cd): deploy d4573cd [skip ci] 2026-05-20 15:27:23 +08:00
Your Name
d4573cd00a feat(awooop): expose execution evidence on incidents
All checks were successful
Code Review / ai-code-review (push) Successful in 17s
CD Pipeline / tests (push) Successful in 3m27s
CD Pipeline / build-and-deploy (push) Successful in 4m6s
CD Pipeline / post-deploy-checks (push) Successful in 1m33s
2026-05-20 15:19:48 +08:00
Your Name
312042ae6d docs(awooop): record t107 mcp evidence rollout [skip ci] 2026-05-20 15:13:42 +08:00
AWOOOI CD
fb9c7d930c chore(cd): deploy c426b1c [skip ci] 2026-05-20 07:09:35 +00:00
Your Name
c426b1ce7b feat(awooop): expose mcp evidence details on incidents
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m31s
CD Pipeline / build-and-deploy (push) Successful in 4m12s
CD Pipeline / post-deploy-checks (push) Successful in 2m1s
2026-05-20 15:01:52 +08:00
Your Name
f85a876868 docs(web): record t106 incident evidence rollout [skip ci] 2026-05-20 14:53:39 +08:00
AWOOOI CD
543c938956 chore(cd): deploy 2eaffe0 [skip ci] 2026-05-20 14:48:19 +08:00
Your Name
2eaffe07aa feat(web): surface incident automation evidence counts
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m52s
CD Pipeline / build-and-deploy (push) Successful in 3m26s
CD Pipeline / post-deploy-checks (push) Successful in 1m53s
2026-05-20 14:40:53 +08:00
Your Name
b9a0f289b2 docs(web): record t105 alerts status-chain rollout [skip ci] 2026-05-20 14:37:16 +08:00
AWOOOI CD
5b699ec312 chore(cd): deploy 0870cdf [skip ci] 2026-05-20 14:33:16 +08:00
Your Name
0870cdf789 fix(web): show status chain evidence on alerts
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 3m55s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m21s
2026-05-20 14:25:45 +08:00
Your Name
076946412e docs(web): record t104 homepage live data rollout [skip ci] 2026-05-20 14:13:15 +08:00
AWOOOI CD
ed3a16468a chore(cd): deploy 72af10b [skip ci] 2026-05-20 06:08:49 +00:00
Your Name
72af10b43b fix(web): align homepage evidence with live data
All checks were successful
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / tests (push) Successful in 3m58s
CD Pipeline / build-and-deploy (push) Successful in 3m47s
CD Pipeline / post-deploy-checks (push) Successful in 1m53s
2026-05-20 14:00:55 +08:00
Your Name
ef811c979b docs(monitoring): record t103 alert chain evidence rollout [skip ci] 2026-05-20 13:29:37 +08:00
Your Name
4956fbb849 fix(monitoring): verify alert rule deploy content
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 23s
2026-05-20 13:26:24 +08:00
AWOOOI CD
1b525b7c18 chore(cd): deploy 598f33a [skip ci] 2026-05-20 13:19:12 +08:00
Your Name
598f33ae8b fix(monitoring): clarify alert chain smoke evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 22s
CD Pipeline / tests (push) Successful in 3m55s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m33s
2026-05-20 13:11:44 +08:00
Your Name
ce0d6a75c4 docs(monitoring): record t102 target freshness rollout [skip ci] 2026-05-20 13:01:49 +08:00
Your Name
cbb0221f0f docs(monitoring): record t102 target coverage cleanup [skip ci] 2026-05-20 12:59:57 +08:00
AWOOOI CD
f542aa52f0 chore(cd): deploy 6e5d68e [skip ci] 2026-05-20 12:56:00 +08:00
Your Name
89f397594e ci: clean b5 test bytecode artifacts
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
2026-05-20 12:55:28 +08:00
Your Name
6e5d68eebc test(monitoring): avoid script bytecode cleanup noise
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 3m54s
CD Pipeline / build-and-deploy (push) Successful in 3m32s
CD Pipeline / post-deploy-checks (push) Successful in 1m47s
2026-05-20 12:48:30 +08:00
Your Name
8fa8d690a2 fix(monitoring): stabilize post-deploy target coverage
Some checks failed
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 4m7s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-20 12:41:09 +08:00
Your Name
60f7dc23d3 docs(web): record t101 status-chain dashboard rollout [skip ci] 2026-05-20 12:27:06 +08:00
AWOOOI CD
426f0dedad chore(cd): deploy 5bc346b [skip ci] 2026-05-20 12:19:49 +08:00
Your Name
5bc346b97e feat(web): drive incident flow summaries from status chain
All checks were successful
Code Review / ai-code-review (push) Successful in 16s
CD Pipeline / tests (push) Successful in 4m12s
CD Pipeline / build-and-deploy (push) Successful in 4m34s
CD Pipeline / post-deploy-checks (push) Successful in 1m48s
2026-05-20 12:11:41 +08:00
Your Name
1d6636cd0d docs(web): record t100 dashboard flow rollout [skip ci] 2026-05-20 11:55:45 +08:00
AWOOOI CD
20026d4671 chore(cd): deploy 0c1f126 [skip ci] 2026-05-20 03:51:18 +00:00
Your Name
0c1f126479 fix(web): clarify incident flow stage on dashboard
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m57s
CD Pipeline / build-and-deploy (push) Successful in 3m42s
CD Pipeline / post-deploy-checks (push) Successful in 1m49s
2026-05-20 11:43:23 +08:00
Your Name
1faaaf8fbc docs(governance): record t99 event history rollout [skip ci] 2026-05-20 11:35:15 +08:00
AWOOOI CD
a0e56bbaad chore(cd): deploy 9307060 [skip ci] 2026-05-20 03:31:15 +00:00
Your Name
93070600b4 fix(governance): keep event history filter responses ordered
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m56s
CD Pipeline / build-and-deploy (push) Successful in 4m35s
CD Pipeline / post-deploy-checks (push) Successful in 1m46s
2026-05-20 11:23:21 +08:00
AWOOOI CD
55e642eeaf chore(cd): deploy 739a8e0 [skip ci] 2026-05-20 11:11:25 +08:00
Your Name
739a8e0f78 feat(governance): link work items to event history
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m35s
CD Pipeline / build-and-deploy (push) Successful in 3m50s
CD Pipeline / post-deploy-checks (push) Successful in 1m42s
2026-05-20 11:03:52 +08:00
Your Name
4a24d3e4fc docs(governance): record t98 archive history rollout [skip ci] 2026-05-20 10:38:02 +08:00
AWOOOI CD
e7691a1f15 chore(cd): deploy edb6dae [skip ci] 2026-05-20 02:31:46 +00:00
Your Name
edb6daef88 feat(governance): attach km archive history to dedupe groups
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 6m56s
Type Sync Check / check-type-sync (push) Successful in 6m51s
CD Pipeline / build-and-deploy (push) Successful in 4m43s
CD Pipeline / post-deploy-checks (push) Successful in 1m47s
2026-05-20 10:20:01 +08:00
Your Name
9b0f68f6c4 docs(governance): record t97 deploy marker [skip ci] 2026-05-20 10:07:21 +08:00
Your Name
d19f6ad7a9 docs(governance): record km archive history rollout [skip ci] 2026-05-20 10:06:21 +08:00
AWOOOI CD
8a3069755d chore(cd): deploy 14697ba [skip ci] 2026-05-20 10:00:42 +08:00
Your Name
14697ba20e feat(governance): surface km archive audit history
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
Type Sync Check / check-type-sync (push) Successful in 27s
CD Pipeline / tests (push) Successful in 4m9s
CD Pipeline / build-and-deploy (push) Successful in 3m57s
CD Pipeline / post-deploy-checks (push) Successful in 1m45s
2026-05-20 09:52:30 +08:00
Your Name
967d4b77b6 docs(governance): record km archive fingerprint rollout [skip ci] 2026-05-20 09:33:00 +08:00
AWOOOI CD
5fe9f725aa chore(cd): deploy 584d2a7 [skip ci] 2026-05-20 01:27:41 +00:00
Your Name
584d2a77ff feat(governance): bind km archive confirm to dry-run fingerprint
All checks were successful
Code Review / ai-code-review (push) Successful in 14s
Type Sync Check / check-type-sync (push) Successful in 31s
CD Pipeline / tests (push) Successful in 4m8s
CD Pipeline / build-and-deploy (push) Successful in 4m48s
CD Pipeline / post-deploy-checks (push) Successful in 2m13s
2026-05-20 09:19:32 +08:00
Your Name
83ca72e989 docs(governance): record km archive preview rollout [skip ci] 2026-05-20 01:58:16 +08:00
AWOOOI CD
42b668bbff chore(cd): deploy ba904ec [skip ci] 2026-05-19 17:43:31 +00:00
Your Name
ba904ec4a1 feat(governance): require dry-run preview before km archive
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 3m34s
CD Pipeline / build-and-deploy (push) Successful in 4m5s
CD Pipeline / post-deploy-checks (push) Successful in 1m41s
2026-05-20 01:35:43 +08:00
Your Name
839b3ea960 docs(governance): record km stale ratio recheck rollout [skip ci] 2026-05-20 01:07:52 +08:00
AWOOOI CD
b7eb3f7da2 chore(cd): deploy d283e65 [skip ci] 2026-05-20 00:59:50 +08:00
Your Name
d283e65340 feat(governance): trace km stale ratio rechecks
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
Type Sync Check / check-type-sync (push) Successful in 26s
CD Pipeline / tests (push) Successful in 3m34s
CD Pipeline / build-and-deploy (push) Successful in 4m2s
CD Pipeline / post-deploy-checks (push) Successful in 1m48s
2026-05-20 00:52:14 +08:00
Your Name
5ac315c119 docs(governance): record km archive rollout [skip ci] 2026-05-20 00:42:30 +08:00
AWOOOI CD
3c9404d241 chore(cd): deploy c8a995a [skip ci] 2026-05-19 16:37:41 +00:00
Your Name
c8a995aff2 feat(governance): archive duplicate km review drafts
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
Type Sync Check / check-type-sync (push) Successful in 33s
CD Pipeline / tests (push) Successful in 3m31s
CD Pipeline / build-and-deploy (push) Successful in 4m41s
CD Pipeline / post-deploy-checks (push) Successful in 1m53s
2026-05-20 00:30:17 +08:00
Your Name
101cd42974 docs(awooop): record km dedupe smoke [skip ci] 2026-05-20 00:10:48 +08:00
AWOOOI CD
7569cff19e chore(cd): deploy 0cd6301 [skip ci] 2026-05-19 16:04:08 +00:00
Your Name
0cd6301d0e feat(governance): expose km draft dedupe plan
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
Type Sync Check / check-type-sync (push) Successful in 33s
CD Pipeline / tests (push) Successful in 4m3s
E2E Health Check / e2e-health (push) Successful in 23s
CD Pipeline / build-and-deploy (push) Successful in 4m54s
CD Pipeline / post-deploy-checks (push) Successful in 2m9s
2026-05-19 23:56:03 +08:00
Your Name
65badab6fd docs(awooop): refresh km draft smoke totals [skip ci] 2026-05-19 23:43:37 +08:00
Your Name
d4e94e88c4 docs(awooop): record km worker followup smoke [skip ci] 2026-05-19 23:43:01 +08:00
Your Name
04ab2901cc docs(awooop): record km draft dedupe rollout [skip ci] 2026-05-19 23:42:13 +08:00
AWOOOI CD
3ea90aa331 chore(cd): deploy 855716b [skip ci] 2026-05-19 23:35:31 +08:00
Your Name
855716b5b8 feat(awooop): surface km review draft dedupe
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
Type Sync Check / check-type-sync (push) Successful in 33s
CD Pipeline / tests (push) Successful in 3m57s
CD Pipeline / build-and-deploy (push) Successful in 4m47s
CD Pipeline / post-deploy-checks (push) Successful in 2m3s
2026-05-19 23:27:33 +08:00
Your Name
9c122a4a37 docs(governance): record hermes km healthcheck rollout [skip ci] 2026-05-19 23:16:45 +08:00
AWOOOI CD
07744bf83d chore(cd): deploy 8342cfa [skip ci] 2026-05-19 15:06:47 +00:00
Your Name
8342cfa460 fix(governance): stop km healthcheck requeue
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 2m1s
CD Pipeline / build-and-deploy (push) Successful in 4m45s
CD Pipeline / post-deploy-checks (push) Successful in 1m46s
2026-05-19 23:01:03 +08:00
AWOOOI CD
ac0d2329f7 chore(cd): deploy de6dbe0 [skip ci] 2026-05-19 22:53:48 +08:00
Your Name
de6dbe07c9 fix(knowledge): query tags on json columns
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m55s
CD Pipeline / build-and-deploy (push) Successful in 3m54s
CD Pipeline / post-deploy-checks (push) Successful in 1m41s
2026-05-19 22:47:57 +08:00
AWOOOI CD
53f8737546 chore(cd): deploy edf97ad [skip ci] 2026-05-19 14:39:28 +00:00
Your Name
edf97ad8ca feat(governance): process hermes km healthchecks
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 2m13s
CD Pipeline / build-and-deploy (push) Successful in 5m14s
CD Pipeline / post-deploy-checks (push) Successful in 1m55s
2026-05-19 22:32:55 +08:00
Your Name
bda857a8f3 docs(governance): record dispatch history linkage [skip ci] 2026-05-19 22:19:06 +08:00
AWOOOI CD
ac91ba3e17 chore(cd): deploy e2a2e03 [skip ci] 2026-05-19 22:14:12 +08:00
Your Name
e2a2e03c79 fix(governance): link events to dispatch history
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 5m55s
CD Pipeline / build-and-deploy (push) Successful in 3m38s
CD Pipeline / post-deploy-checks (push) Successful in 1m46s
2026-05-19 22:04:31 +08:00
Your Name
955dbce670 docs(governance): record km healthcheck backlog rollout [skip ci] 2026-05-19 21:58:36 +08:00
AWOOOI CD
9e9b30689f chore(cd): deploy 2f68b3f [skip ci] 2026-05-19 21:52:56 +08:00
Your Name
2f68b3f472 fix(governance): drain km healthcheck backlog
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 6m2s
CD Pipeline / build-and-deploy (push) Successful in 4m26s
CD Pipeline / post-deploy-checks (push) Successful in 1m37s
2026-05-19 21:43:19 +08:00
AWOOOI CD
271aadcefe chore(cd): deploy b85ab70 [skip ci] 2026-05-19 21:37:10 +08:00
Your Name
b85ab70c45 fix(governance): intake km healthcheck dispatches
All checks were successful
Code Review / ai-code-review (push) Successful in 22s
CD Pipeline / tests (push) Successful in 6m3s
CD Pipeline / build-and-deploy (push) Successful in 4m26s
CD Pipeline / post-deploy-checks (push) Successful in 1m21s
2026-05-19 21:27:30 +08:00
AWOOOI CD
aee0a70021 chore(cd): deploy c99be25 [skip ci] 2026-05-19 21:17:24 +08:00
Your Name
c99be252d3 feat(governance): surface km healthcheck dispatch
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
Type Sync Check / check-type-sync (push) Successful in 38s
CD Pipeline / tests (push) Successful in 5m51s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
2026-05-19 21:07:55 +08:00
Your Name
3b50ff3cc3 docs(governance): record knowledge ownership rollout [skip ci] 2026-05-19 20:54:24 +08:00
AWOOOI CD
17fbd1a567 chore(cd): deploy 4452a00 [skip ci] 2026-05-19 20:48:40 +08:00
Your Name
4452a006bf feat(governance): show knowledge degradation ownership
All checks were successful
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / tests (push) Successful in 6m20s
CD Pipeline / build-and-deploy (push) Successful in 4m54s
CD Pipeline / post-deploy-checks (push) Successful in 1m48s
2026-05-19 20:38:29 +08:00
Your Name
7dc724c9d4 docs(web): record homepage automation evidence rollout [skip ci] 2026-05-19 18:38:47 +08:00
AWOOOI CD
a4fe31218b chore(cd): deploy 61d82b3 [skip ci] 2026-05-19 18:32:15 +08:00
Your Name
61d82b3ad3 feat(web): surface automation evidence on homepage
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 5m54s
CD Pipeline / build-and-deploy (push) Successful in 4m37s
CD Pipeline / post-deploy-checks (push) Successful in 1m28s
2026-05-19 18:22:37 +08:00
Your Name
6ea041d463 docs(metrics): record alert chain durable evidence rollout [skip ci] 2026-05-19 18:09:47 +08:00
AWOOOI CD
6f6cf90a17 chore(cd): deploy c516f9f [skip ci] 2026-05-19 10:05:22 +00:00
Your Name
c516f9fc71 fix(metrics): refresh alert chain timestamp from durable evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
CD Pipeline / tests (push) Successful in 5m53s
CD Pipeline / build-and-deploy (push) Successful in 4m13s
CD Pipeline / post-deploy-checks (push) Successful in 1m29s
2026-05-19 17:55:47 +08:00
Your Name
f0a9b1e00a docs(governance): record knowledge alert clarity rollout [skip ci] 2026-05-19 15:50:20 +08:00
AWOOOI CD
477a7d46a8 chore(cd): deploy bf8974b [skip ci] 2026-05-19 15:45:38 +08:00
Your Name
bf8974be03 fix(governance): normalize knowledge degradation payloads
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 5m55s
CD Pipeline / build-and-deploy (push) Successful in 4m22s
CD Pipeline / post-deploy-checks (push) Successful in 1m39s
2026-05-19 15:35:59 +08:00
AWOOOI CD
81ac1f0f55 chore(cd): deploy 795c9a4 [skip ci] 2026-05-19 07:24:34 +00:00
Your Name
795c9a4e93 fix(governance): clarify knowledge degradation alerts
All checks were successful
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / tests (push) Successful in 6m1s
CD Pipeline / build-and-deploy (push) Successful in 4m32s
CD Pipeline / post-deploy-checks (push) Successful in 1m34s
2026-05-19 15:14:47 +08:00
AWOOOI CD
038f1a0d6d chore(cd): deploy d6c941e [skip ci] 2026-05-19 15:08:39 +08:00
Your Name
d6c941ea39 fix(ci): feed observability pod status into alert smoke
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-19 14:58:34 +08:00
Your Name
842069a1fd docs(ci): record e2e smoke cleanup rollout [skip ci] 2026-05-19 14:54:01 +08:00
AWOOOI CD
3be2c9695a chore(cd): deploy 8272047 [skip ci] 2026-05-19 14:50:58 +08:00
Your Name
8272047371 fix(ci): clean e2e smoke workspace artifacts
All checks were successful
Code Review / ai-code-review (push) Successful in 33s
2026-05-19 14:40:56 +08:00
Your Name
0adebd1add docs(ci): record runner cache cleanup rollout [skip ci] 2026-05-19 14:39:43 +08:00
AWOOOI CD
169e828ebb chore(cd): deploy 947a84e [skip ci] 2026-05-19 06:35:32 +00:00
Your Name
947a84e6c1 fix(ci): clean root-owned pytest cache artifacts
All checks were successful
Code Review / ai-code-review (push) Successful in 31s
2026-05-19 14:25:19 +08:00
Your Name
dc34e81224 docs(awooop): record ai route visibility rollout [skip ci] 2026-05-19 14:19:34 +08:00
AWOOOI CD
815dcf370f chore(cd): deploy 170f927 [skip ci] 2026-05-19 06:14:55 +00:00
Your Name
170f927bc6 fix(ci): build cicd notification payload without python
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-19 14:03:23 +08:00
AWOOOI CD
570b99e9fd chore(cd): deploy 56a8085 [skip ci] 2026-05-19 13:54:50 +08:00
Your Name
56a8085dcf feat(awooop): surface ai provider route status
All checks were successful
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / tests (push) Successful in 6m2s
CD Pipeline / build-and-deploy (push) Successful in 4m21s
CD Pipeline / post-deploy-checks (push) Successful in 1m21s
2026-05-19 13:45:04 +08:00
Your Name
3477c7569a docs(api): record decision manager ollama fallback rollout [skip ci] 2026-05-19 13:28:51 +08:00
AWOOOI CD
11842170df chore(cd): deploy a379a80 [skip ci] 2026-05-19 13:25:02 +08:00
Your Name
a379a80ce1 fix(api): route decision manager ollama calls through fallback
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 5m59s
CD Pipeline / build-and-deploy (push) Successful in 3m30s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-19 13:15:21 +08:00
Your Name
a0ca2ccb7f docs(api): record direct ollama fallback rollout [skip ci] 2026-05-19 13:10:40 +08:00
AWOOOI CD
4de626fcd5 chore(cd): deploy 35fe37c [skip ci] 2026-05-19 13:05:43 +08:00
Your Name
35fe37c82a fix(api): route direct ollama callers through ordered fallback
All checks were successful
Code Review / ai-code-review (push) Successful in 23s
CD Pipeline / tests (push) Successful in 5m51s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m14s
2026-05-19 12:56:13 +08:00
Your Name
8a0a3f89aa docs(api): record ollama route order rollout [skip ci] 2026-05-19 12:44:02 +08:00
AWOOOI CD
1b09a64e01 chore(cd): deploy 45cd55b [skip ci] 2026-05-19 12:41:11 +08:00
Your Name
45cd55b2da fix(api): enforce global ollama endpoint order
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 5m13s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-19 12:32:19 +08:00
AWOOOI CD
5fa0e1452c chore(cd): deploy 36aeea8 [skip ci] 2026-05-19 12:28:37 +08:00
Your Name
36aeea80a3 fix(api): avoid local ollama health blocking gcp route
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m27s
CD Pipeline / build-and-deploy (push) Successful in 4m22s
CD Pipeline / post-deploy-checks (push) Successful in 2m0s
2026-05-19 12:22:46 +08:00
Your Name
1d285dd9d4 fix(api): suppress batch reconcile postmortems
Some checks failed
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m18s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-19 12:18:17 +08:00
AWOOOI CD
f9d53469f9 chore(cd): deploy db4fa42 [skip ci] 2026-05-19 04:13:48 +00:00
Your Name
db4fa420ea fix(api): tolerate legacy incident outcomes
All checks were successful
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / tests (push) Successful in 1m16s
CD Pipeline / build-and-deploy (push) Successful in 4m35s
CD Pipeline / post-deploy-checks (push) Successful in 1m47s
2026-05-19 12:07:54 +08:00
AWOOOI CD
3514ff38fe chore(cd): deploy 6da0c39 [skip ci] 2026-05-19 12:00:44 +08:00
Your Name
6da0c3969b fix(api): tolerate legacy incident decision chains
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m23s
CD Pipeline / build-and-deploy (push) Successful in 3m37s
CD Pipeline / post-deploy-checks (push) Successful in 1m34s
2026-05-19 11:55:42 +08:00
AWOOOI CD
ab2862a214 chore(cd): deploy d0835a7 [skip ci] 2026-05-19 11:49:59 +08:00
Your Name
d0835a7be1 fix(api): reconcile completed stuck incidents
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m2s
CD Pipeline / build-and-deploy (push) Successful in 3m34s
CD Pipeline / post-deploy-checks (push) Successful in 1m35s
2026-05-19 11:45:15 +08:00
Your Name
50833a0efb docs(web): record t72 homepage live status rollout [skip ci] 2026-05-19 11:20:57 +08:00
AWOOOI CD
8234a3ee5b chore(cd): deploy 10f2f1a [skip ci] 2026-05-19 11:16:56 +08:00
Your Name
10f2f1abaf fix(web): stabilize homepage live status
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m8s
CD Pipeline / build-and-deploy (push) Successful in 3m30s
CD Pipeline / post-deploy-checks (push) Successful in 1m22s
2026-05-19 11:12:09 +08:00
Your Name
504d038a9e docs(awooop): record t71 work queue status chain rollout [skip ci] 2026-05-19 10:55:50 +08:00
AWOOOI CD
1333d24040 chore(cd): deploy aa33033 [skip ci] 2026-05-19 10:48:02 +08:00
Your Name
aa330339b8 feat(awooop): surface status chain on work queues
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m30s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m15s
2026-05-19 10:42:44 +08:00
Your Name
a0f41658db docs(awooop): record t70 operator status chain rollout [skip ci] 2026-05-19 10:25:43 +08:00
AWOOOI CD
4f151f5da5 chore(cd): deploy 784ebf4 [skip ci] 2026-05-19 10:18:38 +08:00
Your Name
784ebf49ef feat(awooop): surface status chain in operator console
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m16s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-19 10:13:33 +08:00
Your Name
30b2f5bd6e docs(telegram): record t69 status chain rollout [skip ci] 2026-05-19 09:50:34 +08:00
AWOOOI CD
383cc6ab2a chore(cd): deploy 109f55a [skip ci] 2026-05-19 09:45:46 +08:00
Your Name
109f55a12b feat(telegram): surface awooop status chain
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m15s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m16s
2026-05-19 09:40:43 +08:00
Your Name
c06d518254 docs(awooop): record t68 drift remediation evidence [skip ci] 2026-05-19 09:25:33 +08:00
AWOOOI CD
3e94fba7e8 chore(cd): deploy 64b3482 [skip ci] 2026-05-19 09:19:06 +08:00
Your Name
64b34828a7 feat(drift): record remediation evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m10s
CD Pipeline / build-and-deploy (push) Successful in 3m42s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-19 09:13:58 +08:00
Your Name
5bf49f81be docs(awooop): record t67 drift rollback evidence [skip ci] 2026-05-19 02:28:59 +08:00
Your Name
cc4b16c027 docs(awooop): record t66 drift cleanup evidence [skip ci] 2026-05-19 02:24:43 +08:00
AWOOOI CD
a9e7b5f656 chore(cd): deploy 01ba1e6 [skip ci] 2026-05-19 02:19:43 +08:00
Your Name
01ba1e6f13 fix(drift): read git state from gitea main
All checks were successful
Code Review / ai-code-review (push) Successful in 19s
CD Pipeline / tests (push) Successful in 1m29s
CD Pipeline / build-and-deploy (push) Successful in 3m40s
CD Pipeline / post-deploy-checks (push) Successful in 2m5s
2026-05-19 02:14:26 +08:00
AWOOOI CD
2c4e8bb666 chore(cd): deploy 107c4f1 [skip ci] 2026-05-19 02:08:59 +08:00
Your Name
107c4f11cc fix(drift): normalize kustomize runtime defaults
All checks were successful
Code Review / ai-code-review (push) Successful in 21s
CD Pipeline / tests (push) Successful in 2m31s
CD Pipeline / build-and-deploy (push) Successful in 3m38s
CD Pipeline / post-deploy-checks (push) Successful in 1m53s
2026-05-19 02:02:43 +08:00
Your Name
9cfae83da3 docs(awooop): record t64 drift fingerprint dedupe [skip ci] 2026-05-19 01:22:51 +08:00
AWOOOI CD
77d85b33c6 chore(cd): deploy 9843c59 [skip ci] 2026-05-19 01:18:01 +08:00
Your Name
9843c59450 fix(drift): dedupe semantic fingerprint repeats
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m15s
CD Pipeline / build-and-deploy (push) Successful in 3m26s
CD Pipeline / post-deploy-checks (push) Successful in 1m34s
2026-05-19 01:12:55 +08:00
AWOOOI CD
1ca4912270 chore(cd): deploy 69ed35f [skip ci] 2026-05-19 01:01:15 +08:00
Your Name
69ed35fb5e fix(drift): render interpretation objects safely
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m13s
CD Pipeline / build-and-deploy (push) Successful in 3m37s
CD Pipeline / post-deploy-checks (push) Successful in 1m28s
2026-05-19 00:56:16 +08:00
AWOOOI CD
fa9d2a5d5f chore(cd): deploy 0b5268a [skip ci] 2026-05-19 00:44:59 +08:00
Your Name
0b5268a666 feat(drift): surface fingerprint state handoff
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m14s
CD Pipeline / build-and-deploy (push) Successful in 3m46s
CD Pipeline / post-deploy-checks (push) Successful in 1m35s
2026-05-19 00:39:49 +08:00
Your Name
55ab8732c5 docs(awooop): record t63 handoff and drift dedup [skip ci] 2026-05-19 00:23:22 +08:00
AWOOOI CD
12fa97759b chore(cd): deploy 0367dde [skip ci] 2026-05-19 00:18:34 +08:00
Your Name
0367dde686 fix(drift): dedupe blocked auto-adopt escalations
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m4s
CD Pipeline / build-and-deploy (push) Successful in 3m41s
CD Pipeline / post-deploy-checks (push) Successful in 1m39s
2026-05-19 00:13:41 +08:00
Your Name
fb9b0b3b7c feat(awooop): record recurrence handoff proposals 2026-05-19 00:13:40 +08:00
Your Name
0028993851 docs(awooop): record t62 recurrence dry run [skip ci] 2026-05-18 21:51:48 +08:00
AWOOOI CD
5c934de83d chore(cd): deploy d1ebcda [skip ci] 2026-05-18 21:47:10 +08:00
Your Name
d1ebcdac10 feat(awooop): preview recurrence repair work items
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m11s
CD Pipeline / build-and-deploy (push) Successful in 3m33s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-18 21:42:20 +08:00
Your Name
51660ecbb1 docs(awooop): record t61 recurrence work items [skip ci] 2026-05-18 20:41:18 +08:00
AWOOOI CD
bc99683432 chore(cd): deploy b506145 [skip ci] 2026-05-18 20:35:43 +08:00
Your Name
b50614528e feat(awooop): surface recurrence repair work items
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m20s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-18 20:30:43 +08:00
Your Name
bbf5105fb4 docs(awooop): record t60 recurrence repair evidence [skip ci] 2026-05-18 20:17:20 +08:00
AWOOOI CD
d321f44e49 chore(cd): deploy 4b8f946 [skip ci] 2026-05-18 20:11:19 +08:00
Your Name
4b8f946699 fix(awooop): preserve recurrence repair fields
All checks were successful
Code Review / ai-code-review (push) Successful in 21s
CD Pipeline / tests (push) Successful in 1m20s
CD Pipeline / build-and-deploy (push) Successful in 3m37s
CD Pipeline / post-deploy-checks (push) Successful in 1m38s
2026-05-18 20:06:20 +08:00
AWOOOI CD
e36c9b1800 chore(cd): deploy 7fa0673 [skip ci] 2026-05-18 19:55:42 +08:00
Your Name
7fa06731da feat(awooop): link recurring alerts to repair work
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m21s
CD Pipeline / build-and-deploy (push) Successful in 4m2s
CD Pipeline / post-deploy-checks (push) Successful in 1m45s
2026-05-18 19:50:12 +08:00
Your Name
4ec116c012 docs(awooop): record t59 recurring alert links [skip ci] 2026-05-18 19:33:18 +08:00
AWOOOI CD
41ed3c0421 chore(cd): deploy 94f8c68 [skip ci] 2026-05-18 11:28:56 +00:00
Your Name
94f8c68b77 feat(awooop): show recurring alert links
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m6s
CD Pipeline / build-and-deploy (push) Successful in 3m55s
CD Pipeline / post-deploy-checks (push) Successful in 1m57s
2026-05-18 19:23:37 +08:00
Your Name
d709e25d69 docs(awooop): record t58 source dossier coverage [skip ci] 2026-05-18 19:13:05 +08:00
AWOOOI CD
ba1e7997ad chore(cd): deploy 213523c [skip ci] 2026-05-18 11:06:39 +00:00
Your Name
213523c77d feat(awooop): surface source dossier coverage
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m8s
CD Pipeline / build-and-deploy (push) Successful in 3m52s
CD Pipeline / post-deploy-checks (push) Successful in 1m34s
2026-05-18 19:01:28 +08:00
Your Name
fbde48438b docs(awooop): record t57 callback evidence search [skip ci] 2026-05-18 16:36:40 +08:00
AWOOOI CD
17d3c161e4 chore(cd): deploy 28c2b36 [skip ci] 2026-05-18 16:30:31 +08:00
Your Name
28c2b365b3 fix(awooop): type callback reply project filter
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m7s
CD Pipeline / build-and-deploy (push) Successful in 3m28s
CD Pipeline / post-deploy-checks (push) Successful in 1m37s
2026-05-18 16:25:45 +08:00
AWOOOI CD
31f778d60b chore(cd): deploy 08a75f4 [skip ci] 2026-05-18 16:22:07 +08:00
Your Name
08a75f4b5a feat(awooop): search callback reply evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m8s
CD Pipeline / build-and-deploy (push) Successful in 3m44s
CD Pipeline / post-deploy-checks (push) Successful in 1m31s
2026-05-18 16:17:05 +08:00
Your Name
e4e1244c0f docs(awooop): record t56 callback filter rollout [skip ci] 2026-05-18 16:04:11 +08:00
AWOOOI CD
aff2a57db7 chore(cd): deploy f3494e0 [skip ci] 2026-05-18 07:59:24 +00:00
Your Name
f3494e0bfb feat(awooop): filter runs by callback reply state
All checks were successful
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / tests (push) Successful in 1m9s
CD Pipeline / build-and-deploy (push) Successful in 3m53s
CD Pipeline / post-deploy-checks (push) Successful in 1m40s
2026-05-18 15:54:21 +08:00
Your Name
e81e3f7b8a docs(awooop): record t55 callback list evidence [skip ci] 2026-05-18 15:43:21 +08:00
AWOOOI CD
32d4d1ea8b chore(cd): deploy 0e3c63e [skip ci] 2026-05-18 15:38:35 +08:00
Your Name
0e3c63ec15 fix(awooop): preserve callback summary in run list response
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m8s
CD Pipeline / build-and-deploy (push) Successful in 3m44s
CD Pipeline / post-deploy-checks (push) Successful in 1m24s
2026-05-18 15:33:40 +08:00
AWOOOI CD
be551ac761 chore(cd): deploy 20d62ee [skip ci] 2026-05-18 15:29:42 +08:00
Your Name
20d62ee0cf feat(awooop): surface callback replies on run list
All checks were successful
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / tests (push) Successful in 1m25s
CD Pipeline / build-and-deploy (push) Successful in 3m35s
CD Pipeline / post-deploy-checks (push) Successful in 1m50s
2026-05-18 15:24:39 +08:00
Your Name
584bd4b31b docs(awooop): record t54 callback timeline evidence [skip ci] 2026-05-18 15:03:55 +08:00
AWOOOI CD
f35527c7ed chore(cd): deploy 1a16e08 [skip ci] 2026-05-18 14:59:23 +08:00
Your Name
1a16e083e7 feat(awooop): show callback reply states in timeline
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m8s
CD Pipeline / build-and-deploy (push) Successful in 3m24s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-18 14:54:49 +08:00
Your Name
ed37000eba docs(awooop): record t53 callback reply evidence [skip ci] 2026-05-18 14:48:54 +08:00
AWOOOI CD
82e33f6a17 chore(cd): deploy c972302 [skip ci] 2026-05-18 06:45:36 +00:00
Your Name
c97230252a feat(telegram): record callback reply evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m13s
CD Pipeline / build-and-deploy (push) Successful in 3m21s
CD Pipeline / post-deploy-checks (push) Successful in 1m19s
2026-05-18 14:40:47 +08:00
Your Name
e9e6cda06e docs(awooop): record t51 t52 evidence [skip ci] 2026-05-18 14:31:57 +08:00
AWOOOI CD
10965af845 chore(cd): deploy 8ca875e [skip ci] 2026-05-18 06:28:11 +00:00
Your Name
8ca875e6ad fix(web): keep navigation shell before hydration
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m17s
CD Pipeline / build-and-deploy (push) Successful in 3m53s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-18 14:22:55 +08:00
AWOOOI CD
ea96bb0971 chore(cd): deploy 1ee0740 [skip ci] 2026-05-18 14:17:08 +08:00
Your Name
1ee0740b13 fix(telegram): harden detail history html fallback
All checks were successful
Code Review / ai-code-review (push) Successful in 26s
CD Pipeline / tests (push) Successful in 1m15s
CD Pipeline / build-and-deploy (push) Successful in 3m43s
CD Pipeline / post-deploy-checks (push) Successful in 2m1s
2026-05-18 14:12:08 +08:00
Your Name
79038a6efb docs(awooop): record t50 mcp run evidence [skip ci] 2026-05-18 14:04:30 +08:00
AWOOOI CD
5d36638c79 chore(cd): deploy 9d02ab8 [skip ci] 2026-05-18 14:00:09 +08:00
Your Name
9d02ab8080 feat(awooop): surface mcp investigation evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m6s
CD Pipeline / build-and-deploy (push) Successful in 3m30s
CD Pipeline / post-deploy-checks (push) Successful in 2m12s
2026-05-18 13:55:27 +08:00
Your Name
b9597d8d70 docs(awooop): record t49 host mcp evidence [skip ci] 2026-05-18 12:36:09 +08:00
AWOOOI CD
749b210997 chore(cd): deploy 5cb10a6 [skip ci] 2026-05-18 12:29:41 +08:00
Your Name
5cb10a6d2d fix(mcp): enrich host log evidence params
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 2m29s
CD Pipeline / build-and-deploy (push) Successful in 4m15s
CD Pipeline / post-deploy-checks (push) Successful in 1m41s
2026-05-18 12:23:39 +08:00
AWOOOI CD
0e7fe211de chore(cd): deploy 64c7044 [skip ci] 2026-05-18 04:19:18 +00:00
Your Name
64c7044282 fix(mcp): balance host alert tool suggestions
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m17s
CD Pipeline / build-and-deploy (push) Successful in 3m41s
CD Pipeline / post-deploy-checks (push) Successful in 1m43s
2026-05-18 12:14:21 +08:00
AWOOOI CD
989390f7ce chore(cd): deploy 98a10cb [skip ci] 2026-05-18 12:08:19 +08:00
Your Name
98a10cbc7b fix(awooop): initialize mcp runtime for signal worker
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m14s
CD Pipeline / build-and-deploy (push) Successful in 3m24s
CD Pipeline / post-deploy-checks (push) Successful in 1m30s
2026-05-18 12:03:35 +08:00
AWOOOI CD
df7d957310 chore(cd): deploy a023c53 [skip ci] 2026-05-18 11:54:16 +08:00
Your Name
a023c535db fix(awooop): bridge signal worker observations
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 1m4s
CD Pipeline / build-and-deploy (push) Successful in 3m23s
CD Pipeline / post-deploy-checks (push) Successful in 1m22s
2026-05-18 11:49:33 +08:00
Your Name
161e337e77 docs(awooop): record t48 verified auto-repair gate 2026-05-18 11:24:00 +08:00
AWOOOI CD
c4c1e22587 chore(cd): deploy 3f7bf24 [skip ci] 2026-05-18 11:12:06 +08:00
Your Name
3f7bf24b23 fix(ci): make secret base64 helper runner-portable
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-18 11:06:37 +08:00
Your Name
1a2b04f5cf fix(awooop): persist signal metadata and auto-repair prestate
Some checks failed
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m14s
CD Pipeline / build-and-deploy (push) Failing after 3m33s
CD Pipeline / post-deploy-checks (push) Has been skipped
2026-05-18 10:59:54 +08:00
Your Name
5c240744eb docs(awooop): record t47 production verification 2026-05-18 10:34:59 +08:00
AWOOOI CD
9f64739544 chore(cd): deploy 5d10c8f [skip ci] 2026-05-18 10:32:01 +08:00
Your Name
5d10c8fbfe fix(awooop): parallelize quality summary truth-chain fetch
All checks were successful
Code Review / ai-code-review (push) Successful in 22s
CD Pipeline / tests (push) Successful in 1m7s
CD Pipeline / build-and-deploy (push) Successful in 3m20s
CD Pipeline / post-deploy-checks (push) Successful in 1m14s
2026-05-18 10:27:32 +08:00
Your Name
168241e3c5 docs(awooop): record t46 production verification 2026-05-18 10:23:41 +08:00
AWOOOI CD
fd0888b092 chore(cd): deploy daf672a [skip ci] 2026-05-18 10:17:19 +08:00
Your Name
daf672aa1e feat(awooop): show automation claim on work items
Some checks failed
Code Review / ai-code-review (push) Failing after 1s
CD Pipeline / tests (push) Successful in 1m2s
CD Pipeline / build-and-deploy (push) Successful in 3m25s
CD Pipeline / post-deploy-checks (push) Successful in 1m14s
2026-05-18 10:12:51 +08:00
Your Name
fd5ea0cf94 docs(telegram): record t45 production verification 2026-05-18 09:57:35 +08:00
AWOOOI CD
8bacb65a75 chore(cd): deploy 0dd4b48 [skip ci] 2026-05-18 09:52:39 +08:00
Your Name
0dd4b486c5 fix(telegram): keep info callbacks nonfatal
All checks were successful
Code Review / ai-code-review (push) Successful in 19s
CD Pipeline / tests (push) Successful in 1m10s
CD Pipeline / build-and-deploy (push) Successful in 3m41s
CD Pipeline / post-deploy-checks (push) Successful in 1m21s
2026-05-18 09:47:40 +08:00
Your Name
ae18751d17 docs(ci): record secret guard verification 2026-05-18 09:42:09 +08:00
Your Name
986d1a937d fix(ci): run secret surface guard with node
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
2026-05-18 09:41:09 +08:00
Your Name
9f2974f4c5 fix(ci): guard gitea workflow secret surfaces
Some checks failed
Code Review / ai-code-review (push) Failing after 10s
2026-05-18 09:39:13 +08:00
Your Name
e8b507be54 docs(awooop): record legacy mcp production verification 2026-05-18 09:26:59 +08:00
AWOOOI CD
13d6aa41d8 chore(cd): deploy 902593f [skip ci] 2026-05-18 01:22:24 +00:00
Your Name
902593f775 feat(awooop): surface legacy mcp evidence in run detail
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m10s
CD Pipeline / build-and-deploy (push) Successful in 4m7s
CD Pipeline / post-deploy-checks (push) Successful in 1m57s
2026-05-18 09:16:59 +08:00
Your Name
bc701b8fd3 docs(ops): record momo telegram log hygiene 2026-05-18 09:06:14 +08:00
Your Name
756fe92601 fix(ops): converge openclaw compose project
All checks were successful
Ansible Lint / lint (push) Successful in 35s
2026-05-18 08:53:55 +08:00
Your Name
41a7ec93d6 docs(ci): record ansible lint recovery 2026-05-18 04:44:46 +08:00
Your Name
dca1eb642f fix(ansible): clear lint baseline debt
All checks were successful
Ansible Lint / lint (push) Successful in 28s
2026-05-18 04:17:39 +08:00
Your Name
ec18dec0d3 chore(ci): trigger ansible lint with runner label fix
Some checks failed
Ansible Lint / lint (push) Failing after 38s
2026-05-18 03:57:06 +08:00
Your Name
8a7a332190 fix(ci): align ansible lint runner label
All checks were successful
Code Review / ai-code-review (push) Successful in 15s
2026-05-18 03:29:59 +08:00
Your Name
24f4324ae9 fix(ops): align openclaw systemd project and redact token
Some checks failed
Ansible Lint / lint (push) Has been cancelled
2026-05-18 02:48:33 +08:00
Your Name
6b60f6b086 docs(awooop): record t38 production verification 2026-05-18 00:43:26 +08:00
AWOOOI CD
a42e40a68c chore(cd): deploy f0bb303 [skip ci] 2026-05-18 00:37:50 +08:00
Your Name
f0bb303655 fix(awooop): surface auto repair verification state
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m7s
CD Pipeline / build-and-deploy (push) Successful in 3m38s
CD Pipeline / post-deploy-checks (push) Successful in 1m29s
2026-05-18 00:32:50 +08:00
Your Name
40ec5055e1 docs(awooop): record t37 telegram callback closure 2026-05-18 00:12:08 +08:00
AWOOOI CD
68b20be2b4 chore(cd): deploy 9e1b15d [skip ci] 2026-05-18 00:09:05 +08:00
Your Name
9e1b15dabf fix(telegram): sync rejected polling callbacks
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m40s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m36s
2026-05-18 00:03:52 +08:00
AWOOOI CD
06f64c6ddd chore(cd): deploy 913e1ab [skip ci] 2026-05-17 23:59:40 +08:00
Your Name
913e1abcfa fix(telegram): execute approved callbacks
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 1m10s
CD Pipeline / build-and-deploy (push) Successful in 3m39s
CD Pipeline / post-deploy-checks (push) Successful in 1m31s
2026-05-17 23:54:50 +08:00
Your Name
ba971e7a29 docs(awooop): record t36 incident header rollout 2026-05-17 23:46:36 +08:00
AWOOOI CD
bb4041579c chore(cd): deploy 69f2ec5 [skip ci] 2026-05-17 15:42:44 +00:00
Your Name
69f2ec5ec9 feat(awooop): add incident evidence headers
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m7s
CD Pipeline / build-and-deploy (push) Successful in 3m40s
CD Pipeline / post-deploy-checks (push) Successful in 1m28s
2026-05-17 23:37:53 +08:00
Your Name
a6699c41f8 docs(awooop): record t35 incident evidence rollout 2026-05-17 22:58:39 +08:00
AWOOOI CD
d4b2cf003f chore(cd): deploy 76c302a [skip ci] 2026-05-17 22:54:32 +08:00
Your Name
76c302ab5f feat(awooop): expose incident evidence links
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m1s
CD Pipeline / build-and-deploy (push) Successful in 3m26s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
2026-05-17 22:49:55 +08:00
Your Name
2d579cdf1e docs(awooop): record t34 incident deep link rollout 2026-05-17 22:39:42 +08:00
AWOOOI CD
6e9029273b chore(cd): deploy ef1e28b [skip ci] 2026-05-17 22:31:57 +08:00
Your Name
ef1e28b73a fix(telegram): keep url buttons out of callback assertions
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m5s
CD Pipeline / build-and-deploy (push) Successful in 3m45s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-17 22:26:51 +08:00
Your Name
6868a9a93d feat(awooop): link telegram alerts to incident runs
Some checks failed
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Failing after 1m58s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
2026-05-17 22:17:21 +08:00
Your Name
3aabceb234 docs(awooop): record t33 evidence filter rollout 2026-05-17 21:38:38 +08:00
AWOOOI CD
0d9cde51aa chore(cd): deploy a3f2b01 [skip ci] 2026-05-17 13:28:05 +00:00
Your Name
a3f2b010f8 fix(awooop): widen remediation filter context
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m16s
CD Pipeline / build-and-deploy (push) Successful in 3m45s
CD Pipeline / post-deploy-checks (push) Successful in 1m26s
2026-05-17 21:22:56 +08:00
AWOOOI CD
e6a62bb13b chore(cd): deploy 665e72b [skip ci] 2026-05-17 13:19:13 +00:00
Your Name
665e72ba33 feat(awooop): filter runs by remediation evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m4s
CD Pipeline / build-and-deploy (push) Successful in 4m9s
CD Pipeline / post-deploy-checks (push) Successful in 1m26s
2026-05-17 21:13:54 +08:00
Your Name
171443ee94 docs(awooop): record t32 telegram evidence rollout 2026-05-17 21:07:10 +08:00
AWOOOI CD
5b8f324523 chore(cd): deploy cfaa4d0 [skip ci] 2026-05-17 21:04:11 +08:00
Your Name
cfaa4d0a4a feat(telegram): surface remediation evidence on alert cards
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m5s
CD Pipeline / build-and-deploy (push) Successful in 3m23s
CD Pipeline / post-deploy-checks (push) Successful in 1m23s
2026-05-17 20:59:32 +08:00
Your Name
f02923b24a docs(awooop): record t31 list evidence rollout 2026-05-17 20:48:24 +08:00
AWOOOI CD
06489ef844 chore(cd): deploy 64fc19b [skip ci] 2026-05-17 20:40:49 +08:00
Your Name
64fc19b4d5 fix(awooop): align run list evidence table columns
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m18s
CD Pipeline / build-and-deploy (push) Successful in 3m21s
CD Pipeline / post-deploy-checks (push) Successful in 1m19s
2026-05-17 20:36:05 +08:00
AWOOOI CD
5f3f8fc253 chore(cd): deploy 0592402 [skip ci] 2026-05-17 20:31:24 +08:00
Your Name
0592402779 feat(awooop): surface remediation evidence in run lists
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 1m9s
CD Pipeline / build-and-deploy (push) Successful in 4m1s
CD Pipeline / post-deploy-checks (push) Successful in 1m54s
2026-05-17 20:26:03 +08:00
Your Name
27c2a3d980 docs(awooop): record t30 run timeline rollout
All checks were successful
E2E Health Check / e2e-health (push) Successful in 22s
2026-05-15 05:28:51 +08:00
AWOOOI CD
3ca3502147 chore(cd): deploy 5af7108 [skip ci] 2026-05-15 05:13:42 +08:00
Your Name
5af7108b18 fix(awooop): avoid run timeline hydration mismatch
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 1m13s
CD Pipeline / build-and-deploy (push) Successful in 3m20s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-15 05:09:08 +08:00
AWOOOI CD
befe503aa4 chore(cd): deploy 226f551 [skip ci] 2026-05-15 04:06:46 +08:00
Your Name
226f551e77 fix(awooop): sort mixed run timeline timestamps
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m2s
CD Pipeline / build-and-deploy (push) Successful in 3m25s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-15 03:30:48 +08:00
AWOOOI CD
1db4ef093c chore(cd): deploy bc89940 [skip ci] 2026-05-15 02:37:24 +08:00
Your Name
bc89940564 feat(awooop): link remediation evidence to run timeline
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 1m12s
CD Pipeline / build-and-deploy (push) Successful in 4m14s
CD Pipeline / post-deploy-checks (push) Successful in 1m54s
2026-05-15 02:31:46 +08:00
Your Name
6ec424b15c docs(awooop): record t29 telegram history rollout
All checks were successful
E2E Health Check / e2e-health (push) Successful in 22s
2026-05-14 23:43:12 +08:00
AWOOOI CD
615fa23390 chore(cd): deploy 65001da [skip ci] 2026-05-14 23:38:26 +08:00
Your Name
65001da0d8 fix(telegram): preserve incident history html output
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m1s
CD Pipeline / build-and-deploy (push) Successful in 3m27s
CD Pipeline / post-deploy-checks (push) Successful in 1m29s
2026-05-14 23:33:43 +08:00
Your Name
f4a8390dc0 docs(frontend): record t28 incident timeline rollout 2026-05-14 23:18:20 +08:00
AWOOOI CD
7257aa3a9f chore(cd): deploy 475f2e4 [skip ci] 2026-05-14 23:14:01 +08:00
Your Name
475f2e452d feat(frontend): expand incident timeline event details
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 1m7s
CD Pipeline / build-and-deploy (push) Successful in 3m37s
CD Pipeline / post-deploy-checks (push) Successful in 1m23s
2026-05-14 23:09:12 +08:00
Your Name
d9d119ede2 docs(governance): record t27 remediation history rollout 2026-05-14 23:05:26 +08:00
AWOOOI CD
8d098f564d chore(cd): deploy 392cfb9 [skip ci] 2026-05-14 15:01:41 +00:00
Your Name
392cfb9025 feat(governance): surface remediation dry run history
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m1s
CD Pipeline / build-and-deploy (push) Successful in 3m45s
CD Pipeline / post-deploy-checks (push) Successful in 1m39s
2026-05-14 22:56:51 +08:00
Your Name
53cd7f9d66 docs(governance): record t26 dry run history rollout 2026-05-14 22:47:42 +08:00
AWOOOI CD
9870ed5e30 chore(cd): deploy 6aaaf87 [skip ci] 2026-05-14 14:43:33 +00:00
Your Name
6aaaf87ade feat(governance): persist remediation dry run history
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 1m4s
CD Pipeline / build-and-deploy (push) Successful in 3m44s
CD Pipeline / post-deploy-checks (push) Successful in 1m24s
2026-05-14 22:38:42 +08:00
Your Name
36cb9d6aeb docs(governance): record t25 remediation dry run rollout 2026-05-14 22:32:22 +08:00
AWOOOI CD
3749cc2ab5 chore(cd): deploy 04fdaee [skip ci] 2026-05-14 22:25:30 +08:00
Your Name
04fdaee83a feat(governance): add remediation dry run entrypoint
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m5s
CD Pipeline / build-and-deploy (push) Successful in 3m43s
CD Pipeline / post-deploy-checks (push) Successful in 1m33s
2026-05-14 22:20:34 +08:00
Your Name
102f92dfc3 docs(governance): record t24 remediation queue rollout 2026-05-14 22:06:36 +08:00
AWOOOI CD
cf173c49d8 chore(cd): deploy 44f7471 [skip ci] 2026-05-14 22:01:20 +08:00
Your Name
44f7471b21 fix(awooop): keep work items telemetry from blocking
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m3s
CD Pipeline / build-and-deploy (push) Successful in 3m21s
CD Pipeline / post-deploy-checks (push) Successful in 1m28s
2026-05-14 21:56:54 +08:00
AWOOOI CD
224ae9e202 chore(cd): deploy aa63ae5 [skip ci] 2026-05-14 21:50:04 +08:00
Your Name
aa63ae5eca feat(governance): surface verification remediation queue
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m0s
CD Pipeline / build-and-deploy (push) Successful in 3m25s
CD Pipeline / post-deploy-checks (push) Successful in 1m16s
2026-05-14 21:45:33 +08:00
Your Name
f97127f704 docs(governance): record t23 auto repair gateway rollout 2026-05-14 21:24:55 +08:00
AWOOOI CD
33e4c9231e chore(cd): deploy 813d088 [skip ci] 2026-05-14 21:17:50 +08:00
Your Name
813d088339 feat(auto-repair): route ssh diagnostics through mcp gateway
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
run-migration / migrate (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m11s
CD Pipeline / build-and-deploy (push) Successful in 3m17s
CD Pipeline / post-deploy-checks (push) Successful in 1m16s
2026-05-14 21:13:05 +08:00
Your Name
0567135647 docs(governance): record t22 verifier breakdown rollout 2026-05-14 20:59:54 +08:00
AWOOOI CD
2582ad9425 chore(cd): deploy bad48de [skip ci] 2026-05-14 20:54:13 +08:00
Your Name
bad48dee04 feat(governance): explain verifier failures
All checks were successful
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / tests (push) Successful in 1m21s
CD Pipeline / build-and-deploy (push) Successful in 3m23s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-14 20:49:20 +08:00
Your Name
dd269b195c docs(governance): record t21 verifier coverage rollout 2026-05-14 20:40:01 +08:00
AWOOOI CD
b1893395f0 chore(cd): deploy 485c58d [skip ci] 2026-05-14 20:33:59 +08:00
Your Name
485c58d085 feat(governance): surface verification coverage
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m9s
CD Pipeline / build-and-deploy (push) Successful in 3m49s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-14 20:28:53 +08:00
Your Name
bc1a11e373 docs(governance): record t20 slo state rollout 2026-05-14 20:07:05 +08:00
AWOOOI CD
e37cbe1910 chore(cd): deploy 809bc96 [skip ci] 2026-05-14 12:02:33 +00:00
Your Name
809bc9670b feat(governance): surface adr100 slo states
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m0s
CD Pipeline / build-and-deploy (push) Successful in 4m0s
CD Pipeline / post-deploy-checks (push) Successful in 1m55s
2026-05-14 19:57:32 +08:00
Your Name
6c16a7b162 docs(governance): record t19 km slo rollout 2026-05-14 19:48:37 +08:00
AWOOOI CD
7d3685ef58 chore(cd): deploy 21dcfbd [skip ci] 2026-05-14 19:43:39 +08:00
Your Name
21dcfbd991 fix(governance): collapse km slo fallback series
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 22s
CD Pipeline / tests (push) Successful in 1m6s
CD Pipeline / build-and-deploy (push) Successful in 5m17s
CD Pipeline / post-deploy-checks (push) Successful in 1m38s
2026-05-14 19:37:15 +08:00
Your Name
d2a4a17969 fix(governance): stabilize adr100 km growth slo
Some checks failed
Code Review / ai-code-review (push) Successful in 22s
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 25s
CD Pipeline / tests (push) Successful in 1m11s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-14 19:33:52 +08:00
Your Name
cdb8bf6802 docs(governance): record adr100 slo emitter rollout 2026-05-14 19:22:39 +08:00
AWOOOI CD
80a056539c chore(cd): deploy b92c9e2 [skip ci] 2026-05-14 19:18:22 +08:00
Your Name
b92c9e285f fix(governance): scope adr100 automation metrics
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m16s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m28s
2026-05-14 19:13:33 +08:00
AWOOOI CD
b677cb11de chore(cd): deploy 368386a [skip ci] 2026-05-14 19:09:38 +08:00
Your Name
368386abc0 fix(governance): skip non-finite slo values
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m4s
CD Pipeline / build-and-deploy (push) Successful in 3m18s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-14 19:05:16 +08:00
AWOOOI CD
d1b0ee7e96 chore(cd): deploy 13cf02b [skip ci] 2026-05-14 19:01:24 +08:00
Your Name
13cf02b740 feat(governance): emit adr100 slo metrics
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m0s
CD Pipeline / build-and-deploy (push) Successful in 3m21s
CD Pipeline / post-deploy-checks (push) Successful in 1m16s
2026-05-14 18:57:03 +08:00
Your Name
1670ff1960 docs(awooop): record t17b governance rollout 2026-05-14 18:47:39 +08:00
AWOOOI CD
9b32d3a9e7 chore(cd): deploy 6220f52 [skip ci] 2026-05-14 10:44:25 +00:00
Your Name
6220f52266 fix(governance): cast dispatch status filter
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m25s
CD Pipeline / build-and-deploy (push) Successful in 3m46s
CD Pipeline / post-deploy-checks (push) Successful in 1m16s
2026-05-14 18:39:11 +08:00
AWOOOI CD
5ef9240583 chore(cd): deploy 08d28dc [skip ci] 2026-05-14 18:35:32 +08:00
Your Name
08d28dc44b fix(governance): normalize event and dispatch queries
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m0s
CD Pipeline / build-and-deploy (push) Successful in 3m18s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-14 18:31:11 +08:00
Your Name
6571260dd2 docs(awooop): record t17 production rollout 2026-05-14 18:17:45 +08:00
AWOOOI CD
687f37d837 chore(cd): deploy e8c4512 [skip ci] 2026-05-14 18:14:01 +08:00
Your Name
e8c4512a40 feat(awooop): surface automation work chain
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m33s
CD Pipeline / build-and-deploy (push) Successful in 4m3s
CD Pipeline / post-deploy-checks (push) Successful in 1m51s
2026-05-14 18:08:13 +08:00
Your Name
aa8b72043b docs(awooop): record t16 automation boundary 2026-05-14 01:15:45 +08:00
Your Name
b5288d4b7d docs(logbook): record t16 auto repair live fire 2026-05-14 01:14:12 +08:00
AWOOOI CD
a9b846c82a chore(cd): deploy 5604dd0 [skip ci] 2026-05-14 01:05:29 +08:00
Your Name
5604dd0256 fix(auto-repair): mark approval execution status
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m10s
CD Pipeline / build-and-deploy (push) Successful in 3m27s
CD Pipeline / post-deploy-checks (push) Successful in 1m30s
2026-05-14 01:00:49 +08:00
AWOOOI CD
5361ad8f7e chore(cd): deploy 6f6d032 [skip ci] 2026-05-14 00:53:13 +08:00
Your Name
6f6d032ca9 fix(mcp): grant rollout verifier read tool
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
run-migration / migrate (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m14s
CD Pipeline / build-and-deploy (push) Successful in 3m35s
CD Pipeline / post-deploy-checks (push) Successful in 1m37s
2026-05-14 00:48:23 +08:00
AWOOOI CD
a91c38675a chore(cd): deploy 5fb73a5 [skip ci] 2026-05-13 16:42:16 +00:00
Your Name
5fb73a5612 fix(verifier): recognize rollout success evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m3s
CD Pipeline / build-and-deploy (push) Successful in 3m38s
CD Pipeline / post-deploy-checks (push) Successful in 1m30s
2026-05-14 00:37:32 +08:00
AWOOOI CD
c42b2dfe06 chore(cd): deploy b1ecb55 [skip ci] 2026-05-14 00:26:17 +08:00
Your Name
b1ecb55bd6 fix(verification): align playbook and mcp evidence for canary alerts
All checks were successful
Code Review / ai-code-review (push) Successful in 18s
CD Pipeline / tests (push) Successful in 1m2s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m39s
2026-05-14 00:21:44 +08:00
AWOOOI CD
42d0d076d6 chore(cd): deploy d835b66 [skip ci] 2026-05-14 00:11:33 +08:00
Your Name
d835b666cf fix(alertmanager): keep auto repair moving on ai fallback
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m10s
CD Pipeline / build-and-deploy (push) Successful in 3m25s
CD Pipeline / post-deploy-checks (push) Successful in 1m30s
2026-05-14 00:06:49 +08:00
AWOOOI CD
39581ab824 chore(cd): deploy a0a0731 [skip ci] 2026-05-13 15:48:16 +00:00
Your Name
a0a0731cd6 fix(auto-repair): preserve exact playbook candidates
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 5m46s
CD Pipeline / build-and-deploy (push) Successful in 4m6s
CD Pipeline / post-deploy-checks (push) Successful in 1m28s
2026-05-13 23:38:19 +08:00
AWOOOI CD
5161a9dfd6 chore(cd): deploy 7a8cbb3 [skip ci] 2026-05-13 23:25:53 +08:00
Your Name
7a8cbb3241 fix(auto-repair): prefer exact playbooks and fail failed steps
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m3s
CD Pipeline / build-and-deploy (push) Successful in 3m31s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-13 23:21:17 +08:00
AWOOOI CD
ae643552e9 chore(cd): deploy 8885c1b [skip ci] 2026-05-13 23:10:18 +08:00
Your Name
8885c1b49d fix(cd): rebuild API image when T16 seed script changes
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
2026-05-13 23:05:00 +08:00
Your Name
4ee57b710d fix(ops): support API image path for T16 seed script
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
2026-05-13 23:03:40 +08:00
AWOOOI CD
5a31702885 chore(cd): deploy dcaf16c [skip ci] 2026-05-13 23:01:16 +08:00
Your Name
dcaf16cecc fix(docker): preserve nested T16 ops script in build context
All checks were successful
CD Pipeline / tests (push) Successful in 1m12s
CD Pipeline / build-and-deploy (push) Successful in 3m24s
CD Pipeline / post-deploy-checks (push) Successful in 1m23s
2026-05-13 22:56:35 +08:00
AWOOOI CD
07ed014a83 chore(cd): deploy c5f4baf [skip ci] 2026-05-13 22:54:32 +08:00
Your Name
c5f4bafcaf fix(docker): include T16 seed script in API image
All checks were successful
CD Pipeline / tests (push) Successful in 1m18s
CD Pipeline / build-and-deploy (push) Successful in 3m19s
CD Pipeline / post-deploy-checks (push) Successful in 1m35s
2026-05-13 22:49:52 +08:00
AWOOOI CD
1277865343 chore(cd): deploy 7df94e9 [skip ci] 2026-05-13 22:44:15 +08:00
Your Name
7df94e9bef fix(k8s): fit auto repair canary resource floor
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m29s
CD Pipeline / build-and-deploy (push) Successful in 3m22s
CD Pipeline / post-deploy-checks (push) Successful in 1m59s
2026-05-13 22:39:09 +08:00
AWOOOI CD
8bb601eecd chore(cd): deploy 1778a69 [skip ci] 2026-05-13 22:35:22 +08:00
Your Name
1778a692e0 feat(awooop): add auto repair canary live-fire target
Some checks failed
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m11s
CD Pipeline / build-and-deploy (push) Failing after 6m52s
CD Pipeline / post-deploy-checks (push) Has been skipped
2026-05-13 22:30:20 +08:00
Your Name
0337b62349 docs(awooop): record event dossier rollout [skip ci] 2026-05-13 22:16:50 +08:00
AWOOOI CD
39e6ce747d chore(cd): deploy e947e60 [skip ci] 2026-05-13 22:12:55 +08:00
Your Name
e947e60d11 fix(awooop): type dossier run filter
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m3s
CD Pipeline / build-and-deploy (push) Successful in 3m47s
CD Pipeline / post-deploy-checks (push) Successful in 1m43s
2026-05-13 22:08:00 +08:00
AWOOOI CD
a21fc0f35a chore(cd): deploy 77aace7 [skip ci] 2026-05-13 22:04:10 +08:00
Your Name
77aace7515 feat(awooop): show inbound event dossiers
All checks were successful
Code Review / ai-code-review (push) Successful in 17s
CD Pipeline / tests (push) Successful in 1m19s
CD Pipeline / build-and-deploy (push) Successful in 3m30s
CD Pipeline / post-deploy-checks (push) Successful in 1m34s
2026-05-13 21:59:16 +08:00
Your Name
eb73591286 docs(awooop): record inbound envelope and agent boundary 2026-05-13 21:49:14 +08:00
AWOOOI CD
011085ce3d chore(cd): deploy a524e46 [skip ci] 2026-05-13 21:43:35 +08:00
Your Name
a524e468e4 fix(awooop): mark inbound-only truth chains received
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m19s
CD Pipeline / build-and-deploy (push) Successful in 3m24s
CD Pipeline / post-deploy-checks (push) Successful in 1m25s
2026-05-13 21:38:47 +08:00
AWOOOI CD
365d93f07e chore(cd): deploy 7950851 [skip ci] 2026-05-13 21:34:15 +08:00
Your Name
795085170a feat(awooop): persist inbound source envelopes
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m23s
CD Pipeline / build-and-deploy (push) Successful in 3m37s
CD Pipeline / post-deploy-checks (push) Successful in 1m34s
2026-05-13 21:29:04 +08:00
AWOOOI CD
c888444287 chore(cd): deploy ea320a2 [skip ci] 2026-05-13 21:19:22 +08:00
Your Name
ea320a2087 db(awooop): add inbound truth-chain envelope columns
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
run-migration / migrate (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m1s
CD Pipeline / build-and-deploy (push) Successful in 3m36s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-13 21:14:43 +08:00
Your Name
ebf0f57272 docs(awooop): record alertmanager truth-chain mirror 2026-05-13 20:47:31 +08:00
AWOOOI CD
dc865cf53d chore(cd): deploy 8d7b938 [skip ci] 2026-05-13 20:41:39 +08:00
Your Name
8d7b938f78 fix(awooop): surface alert inbound by provider event
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m5s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m19s
2026-05-13 20:37:02 +08:00
AWOOOI CD
453e22f80d chore(cd): deploy c6e4752 [skip ci] 2026-05-13 20:33:27 +08:00
Your Name
c6e47526a7 fix(awooop): use db-safe timestamps for alert mirrors
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m15s
CD Pipeline / build-and-deploy (push) Successful in 3m19s
CD Pipeline / post-deploy-checks (push) Successful in 1m26s
2026-05-13 20:28:49 +08:00
AWOOOI CD
9b7a91d828 chore(cd): deploy c2d01eb [skip ci] 2026-05-13 20:22:21 +08:00
Your Name
c2d01eb6f1 feat(awooop): mirror alertmanager events into truth chain
All checks were successful
Code Review / ai-code-review (push) Successful in 19s
CD Pipeline / tests (push) Successful in 2m10s
CD Pipeline / build-and-deploy (push) Successful in 3m22s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-13 20:16:42 +08:00
Your Name
21042ad0e7 docs(awooop): record 188 key rotation verification 2026-05-13 20:05:41 +08:00
AWOOOI CD
bcf2ed7841 chore(cd): deploy 6064e6d [skip ci] 2026-05-13 20:02:11 +08:00
Your Name
6064e6d03f fix(cd): disable unsafe 188 secret sync path
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
2026-05-13 19:57:13 +08:00
Your Name
830dc0dcd0 fix(cd): keep 188 deploy key out of step env
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-13 19:41:12 +08:00
Your Name
88dbcd912e docs(awooop): record t14c telegram flow progress 2026-05-13 19:38:55 +08:00
AWOOOI CD
2f5d812608 chore(cd): deploy 74c4767 [skip ci] 2026-05-13 11:34:47 +00:00
Your Name
74c47672da feat(telegram): show automation flow progress
All checks were successful
Code Review / ai-code-review (push) Successful in 22s
CD Pipeline / tests (push) Successful in 1m13s
CD Pipeline / build-and-deploy (push) Successful in 3m39s
CD Pipeline / post-deploy-checks (push) Successful in 1m31s
2026-05-13 19:29:51 +08:00
Your Name
872abea008 docs(awooop): record t14b auto approved evidence link 2026-05-13 19:24:01 +08:00
AWOOOI CD
edba52f401 chore(cd): deploy 596f2f6 [skip ci] 2026-05-13 19:19:24 +08:00
Your Name
596f2f6820 fix(awooop): link auto approved execution evidence
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m17s
CD Pipeline / build-and-deploy (push) Successful in 3m42s
CD Pipeline / post-deploy-checks (push) Successful in 1m21s
2026-05-13 19:14:17 +08:00
Your Name
c68cbd3139 docs(awooop): record t14a verification deployment 2026-05-13 19:05:57 +08:00
AWOOOI CD
9c9cf68063 chore(cd): deploy 3bad354 [skip ci] 2026-05-13 19:00:59 +08:00
Your Name
3bad354414 fix(cd): include ed25519 deploy host keyscan
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
2026-05-13 18:55:49 +08:00
Your Name
518a16e895 fix(awooop): persist auto repair verification fallback
Some checks failed
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m10s
CD Pipeline / build-and-deploy (push) Failing after 3m16s
CD Pipeline / post-deploy-checks (push) Has been skipped
2026-05-13 18:47:46 +08:00
Your Name
a28baa6197 docs(awooop): record t13 quality classification deployment 2026-05-13 17:34:46 +08:00
AWOOOI CD
2314badec5 chore(cd): deploy cecadb3 [skip ci] 2026-05-13 17:29:11 +08:00
Your Name
cecadb331b fix(awooop): exclude audit-only ops from repair quality
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m1s
CD Pipeline / build-and-deploy (push) Successful in 3m41s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-13 17:24:29 +08:00
AWOOOI CD
55b28336e5 chore(cd): deploy 22beddc [skip ci] 2026-05-13 09:17:44 +00:00
Your Name
22beddc8a8 fix(awooop): classify no action audits correctly
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m11s
CD Pipeline / build-and-deploy (push) Successful in 3m39s
CD Pipeline / post-deploy-checks (push) Successful in 1m37s
2026-05-13 17:12:44 +08:00
Your Name
c1e2567b15 docs(awooop): record t12d quality overview deployment 2026-05-13 16:49:29 +08:00
AWOOOI CD
90156a7c1a chore(cd): deploy 356bfce [skip ci] 2026-05-13 16:38:53 +08:00
Your Name
356bfce2c8 fix(awooop): expose quality summary aggregate
All checks were successful
Code Review / ai-code-review (push) Successful in 16s
CD Pipeline / tests (push) Successful in 1m7s
CD Pipeline / build-and-deploy (push) Successful in 3m27s
CD Pipeline / post-deploy-checks (push) Successful in 1m32s
2026-05-13 16:34:20 +08:00
AWOOOI CD
94fc25dc39 chore(cd): deploy e420306 [skip ci] 2026-05-13 16:28:36 +08:00
Your Name
e4203060f3 feat(awooop): surface automation quality overview
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m15s
CD Pipeline / build-and-deploy (push) Successful in 3m25s
CD Pipeline / post-deploy-checks (push) Successful in 1m15s
2026-05-13 16:23:47 +08:00
Your Name
aafe7273e3 docs(awooop): record t12 quality summary deployment 2026-05-13 16:06:30 +08:00
AWOOOI CD
d339e3ebad chore(cd): deploy ae7c7cb [skip ci] 2026-05-13 16:01:50 +08:00
Your Name
ae7c7cbd23 feat(awooop): summarize automation quality
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m14s
CD Pipeline / build-and-deploy (push) Successful in 3m43s
CD Pipeline / post-deploy-checks (push) Successful in 1m29s
2026-05-13 15:56:42 +08:00
Your Name
c00e911b28 docs(awooop): record t12 automation quality deployment 2026-05-13 12:59:34 +08:00
AWOOOI CD
15ff939b1f chore(cd): deploy 0f08024 [skip ci] 2026-05-13 04:56:44 +00:00
Your Name
0f080240c6 feat(awooop): expose automation quality gate
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m13s
CD Pipeline / build-and-deploy (push) Successful in 3m33s
CD Pipeline / post-deploy-checks (push) Successful in 1m26s
2026-05-13 12:51:52 +08:00
Your Name
d886526f23 docs(awooop): record t12 outbound truth deployment 2026-05-13 12:35:54 +08:00
AWOOOI CD
d33856f874 chore(cd): deploy 04c7bb1 [skip ci] 2026-05-13 12:33:11 +08:00
Your Name
04c7bb1c97 fix(awooop): store outbound sent timestamp as naive utc
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m17s
CD Pipeline / build-and-deploy (push) Successful in 3m51s
CD Pipeline / post-deploy-checks (push) Successful in 1m25s
2026-05-13 12:28:04 +08:00
AWOOOI CD
3a1cedc90d chore(cd): deploy d449ba4 [skip ci] 2026-05-13 04:25:23 +00:00
Your Name
d449ba4720 fix(awooop): write outbound sent timestamp as parameter
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m18s
CD Pipeline / build-and-deploy (push) Successful in 3m48s
CD Pipeline / post-deploy-checks (push) Successful in 1m24s
2026-05-13 12:20:20 +08:00
AWOOOI CD
e2785899a2 chore(cd): deploy e57474a [skip ci] 2026-05-13 12:17:10 +08:00
Your Name
e57474adfb fix(awooop): cast outbound sent status timestamp gate
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m22s
CD Pipeline / build-and-deploy (push) Successful in 3m30s
CD Pipeline / post-deploy-checks (push) Successful in 1m24s
2026-05-13 12:12:16 +08:00
AWOOOI CD
971afafc01 chore(cd): deploy 7fa9f74 [skip ci] 2026-05-13 12:09:18 +08:00
Your Name
7fa9f743dd fix(awooop): strengthen outbound truth references
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m12s
CD Pipeline / build-and-deploy (push) Successful in 3m33s
CD Pipeline / post-deploy-checks (push) Successful in 1m15s
2026-05-13 12:04:26 +08:00
Your Name
7d506b785d docs(awooop): record t11 gateway detail deployment 2026-05-13 11:57:54 +08:00
AWOOOI CD
8e14f1bf3e chore(cd): deploy c486087 [skip ci] 2026-05-13 03:54:32 +00:00
Your Name
c486087294 feat(awooop): surface gateway summary in details
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 1m4s
CD Pipeline / build-and-deploy (push) Successful in 3m47s
CD Pipeline / post-deploy-checks (push) Successful in 1m16s
2026-05-13 11:49:37 +08:00
Your Name
51528b2cf9 docs(awooop): record t10 gateway truth chain deployment 2026-05-13 11:38:56 +08:00
AWOOOI CD
5daa005c1b chore(cd): deploy a99dccf [skip ci] 2026-05-13 03:35:06 +00:00
Your Name
a99dccfc73 feat(awooop): summarize gateway usage in truth chain
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m8s
CD Pipeline / build-and-deploy (push) Successful in 3m45s
CD Pipeline / post-deploy-checks (push) Successful in 1m14s
2026-05-13 11:30:08 +08:00
Your Name
90603ad9bb docs(awooop): record t9 approval gateway deployment 2026-05-13 11:27:19 +08:00
AWOOOI CD
77877dd501 chore(cd): deploy 34bfe56 [skip ci] 2026-05-13 11:23:43 +08:00
Your Name
34bfe56f53 fix(awooop): persist approved ssh gateway blocks
All checks were successful
Code Review / ai-code-review (push) Successful in 20s
CD Pipeline / tests (push) Successful in 3m58s
CD Pipeline / build-and-deploy (push) Successful in 3m47s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-13 11:15:54 +08:00
AWOOOI CD
ce83e8dc00 chore(cd): deploy a0a2a5b [skip ci] 2026-05-13 11:10:27 +08:00
Your Name
a0a2a5b1f0 feat(awooop): gate approved ssh execution
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
run-migration / migrate (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m22s
CD Pipeline / build-and-deploy (push) Successful in 6m36s
CD Pipeline / post-deploy-checks (push) Successful in 1m42s
2026-05-13 11:02:24 +08:00
Your Name
85a1bcef52 docs(awooop): record t8 post verify gateway deployment 2026-05-13 10:46:25 +08:00
AWOOOI CD
f19fe4aa90 chore(cd): deploy 1a03bce [skip ci] 2026-05-13 10:41:33 +08:00
Your Name
1a03bceb5c feat(awooop): route post verify mcp through gateway
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m7s
CD Pipeline / build-and-deploy (push) Successful in 10m15s
CD Pipeline / post-deploy-checks (push) Successful in 1m54s
2026-05-13 10:30:03 +08:00
Your Name
15873b9e0c docs(awooop): record t7 mcp gateway deployment 2026-05-13 10:25:47 +08:00
AWOOOI CD
8ac4ba24f7 chore(cd): deploy 42789db [skip ci] 2026-05-13 10:22:15 +08:00
Your Name
42789dbe9e fix(awooop): enable awoooi mcp gateway shadow
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
run-migration / migrate (push) Successful in 9s
CD Pipeline / tests (push) Successful in 2m32s
CD Pipeline / build-and-deploy (push) Successful in 12m19s
CD Pipeline / post-deploy-checks (push) Successful in 1m26s
2026-05-13 10:07:20 +08:00
AWOOOI CD
7ed9859260 chore(cd): deploy 0b70749 [skip ci] 2026-05-13 10:01:23 +08:00
Your Name
0b707495a1 fix(migrations): retrigger mcp gateway seed
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
run-migration / migrate (push) Successful in 9s
CD Pipeline / tests (push) Successful in 1m1s
CD Pipeline / build-and-deploy (push) Successful in 6m59s
CD Pipeline / post-deploy-checks (push) Successful in 1m27s
2026-05-13 09:53:15 +08:00
Your Name
e177eca25d fix(migrations): set tenant context for mcp seed
Some checks failed
Code Review / ai-code-review (push) Successful in 10s
run-migration / migrate (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m10s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-13 09:51:13 +08:00
Your Name
146cf411ae fix(ci): retry migrations on permission denied
Some checks failed
Code Review / ai-code-review (push) Successful in 10s
run-migration / migrate (push) Failing after 9s
CD Pipeline / tests (push) Successful in 1m21s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-13 09:48:56 +08:00
Your Name
57ed07d1d0 feat(awooop): route sense mcp through gateway
Some checks failed
Code Review / ai-code-review (push) Successful in 10s
run-migration / migrate (push) Failing after 8s
CD Pipeline / tests (push) Successful in 1m14s
CD Pipeline / build-and-deploy (push) Has been cancelled
CD Pipeline / post-deploy-checks (push) Has been cancelled
2026-05-13 09:46:12 +08:00
Your Name
5ecd21e664 docs(awooop): record t6 incident visibility deployment 2026-05-13 09:33:17 +08:00
AWOOOI CD
c01012d767 chore(cd): deploy af9798a [skip ci] 2026-05-13 09:29:04 +08:00
Your Name
af9798a62e feat(awooop): surface reconciliation in incident timeline
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m9s
CD Pipeline / build-and-deploy (push) Successful in 5m4s
CD Pipeline / post-deploy-checks (push) Successful in 1m15s
2026-05-13 09:22:51 +08:00
Your Name
5294f0712f docs(awooop): record t5 reconciliation deployment 2026-05-13 09:14:15 +08:00
AWOOOI CD
631fc22090 chore(cd): deploy 1003fa4 [skip ci] 2026-05-13 09:10:20 +08:00
Your Name
1003fa4246 feat(awooop): expose incident reconciliation state
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 59s
CD Pipeline / build-and-deploy (push) Successful in 7m3s
CD Pipeline / post-deploy-checks (push) Successful in 1m15s
2026-05-13 09:02:16 +08:00
Your Name
54814bc65e docs(awooop): record t4 drift fingerprint deployment 2026-05-13 07:52:42 +08:00
AWOOOI CD
3d38039b86 chore(cd): deploy 5b34877 [skip ci] 2026-05-13 07:40:58 +08:00
Your Name
5b34877429 feat(awooop): expose drift repeat fingerprint
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m17s
CD Pipeline / build-and-deploy (push) Successful in 3m20s
CD Pipeline / post-deploy-checks (push) Successful in 1m13s
2026-05-13 07:36:21 +08:00
Your Name
b0a8302dd7 docs(awooop): record t3 decision audit deployment 2026-05-13 04:17:04 +08:00
AWOOOI CD
90b9ddb7a5 chore(cd): deploy 3799e0d [skip ci] 2026-05-12 20:12:20 +00:00
Your Name
3799e0db0d feat(awooop): audit ansible decision candidates
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m1s
CD Pipeline / build-and-deploy (push) Successful in 3m33s
CD Pipeline / post-deploy-checks (push) Successful in 1m17s
2026-05-13 04:07:23 +08:00
Your Name
f61747aeac docs(awooop): record t3 ansible deployment 2026-05-13 04:03:48 +08:00
AWOOOI CD
07000dae3a chore(cd): deploy ca80972 [skip ci] 2026-05-12 19:59:30 +00:00
Your Name
49ffb5bb19 fix(ci): repair migration audit json literal
All checks were successful
Code Review / ai-code-review (push) Successful in 9s
2026-05-13 03:59:22 +08:00
Your Name
ca80972dc7 feat(awooop): expose ansible audit truth surface
Some checks failed
Code Review / ai-code-review (push) Successful in 10s
run-migration / migrate (push) Failing after 9s
CD Pipeline / tests (push) Successful in 2m21s
CD Pipeline / build-and-deploy (push) Successful in 3m50s
CD Pipeline / post-deploy-checks (push) Successful in 1m19s
2026-05-13 03:53:13 +08:00
Your Name
feda8a0b4b fix(ci): harden migration audit seed
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
2026-05-13 03:40:41 +08:00
Your Name
124c3c545b docs(awooop): record t2 truth-chain deployment 2026-05-13 03:31:35 +08:00
AWOOOI CD
dba3e405f4 chore(cd): deploy b4d367e [skip ci] 2026-05-13 03:26:51 +08:00
Your Name
b4d367eeb4 feat(awooop): expose mcp bridge truth chain
All checks were successful
Code Review / ai-code-review (push) Successful in 13s
CD Pipeline / tests (push) Successful in 1m17s
CD Pipeline / build-and-deploy (push) Successful in 3m55s
CD Pipeline / post-deploy-checks (push) Successful in 1m45s
2026-05-13 03:21:31 +08:00
Your Name
b81cb28615 docs(awooop): record t2 mcp bridge smoke 2026-05-13 00:33:03 +08:00
AWOOOI CD
c18c6f6fe2 chore(cd): deploy 94d006e [skip ci] 2026-05-12 23:48:50 +08:00
Your Name
94d006eac8 feat(awooop): bridge legacy mcp audit into gateway timeline
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m4s
CD Pipeline / build-and-deploy (push) Successful in 3m22s
CD Pipeline / post-deploy-checks (push) Successful in 1m14s
2026-05-12 23:44:19 +08:00
Your Name
96a8cf3ad5 docs(awooop): record t1 truth-chain smoke 2026-05-12 23:36:51 +08:00
Your Name
f318fd3a89 fix(ci): harden migration workflow audit
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-12 23:29:54 +08:00
AWOOOI CD
1a62c322bc chore(cd): deploy 24b15f4 [skip ci] 2026-05-12 15:26:34 +00:00
Your Name
24b15f4ad2 feat(awooop): harden outbound truth chain mirror
Some checks failed
Code Review / ai-code-review (push) Successful in 10s
run-migration / migrate (push) Failing after 8s
CD Pipeline / tests (push) Successful in 1m4s
CD Pipeline / build-and-deploy (push) Successful in 3m27s
CD Pipeline / post-deploy-checks (push) Successful in 1m18s
2026-05-12 23:21:45 +08:00
Your Name
c652f37b69 docs(awooop): 記錄 truth-chain production smoke 2026-05-12 23:05:16 +08:00
AWOOOI CD
c523a22d89 chore(cd): deploy f7c8453 [skip ci] 2026-05-12 15:00:31 +00:00
Your Name
f7c84530d6 feat(awooop): 新增 truth-chain 查詢 API
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m16s
CD Pipeline / build-and-deploy (push) Successful in 3m29s
CD Pipeline / post-deploy-checks (push) Successful in 1m19s
2026-05-12 22:55:36 +08:00
Your Name
56228dbb79 docs(awooop): 盤點 Telegram 自動化真相鏈缺口 2026-05-12 22:41:05 +08:00
Your Name
de16c88418 chore(rls): 套用 outbound message canary
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-12 21:55:23 +08:00
Your Name
edd06485e0 docs(rls): 記錄 projects canary 套用 2026-05-12 21:41:14 +08:00
AWOOOI CD
7f94bc5776 chore(cd): deploy 7d92f0a [skip ci] 2026-05-12 13:30:31 +00:00
Your Name
7d92f0acd7 chore(rls): stage projects canary path
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m8s
CD Pipeline / build-and-deploy (push) Successful in 3m49s
CD Pipeline / post-deploy-checks (push) Successful in 1m25s
2026-05-12 21:25:24 +08:00
Your Name
b7af597459 chore(rls): 套用 tool registry canary wave1.1
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
2026-05-12 21:15:14 +08:00
Your Name
1617b73a9d docs(rls): 記錄 canary wave1 production apply 2026-05-12 20:55:40 +08:00
Your Name
8c4dc7a5a8 chore(rls): 新增 manual script gate 與 canary wave1
Some checks failed
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m5s
CD Pipeline / build-and-deploy (push) Failing after 10m6s
CD Pipeline / post-deploy-checks (push) Has been skipped
2026-05-12 20:23:27 +08:00
AWOOOI CD
be8ddf4599 chore(cd): deploy ff30c61 [skip ci] 2026-05-12 20:01:07 +08:00
Your Name
ff30c61c4c fix(rls): 收斂 API DB access context
All checks were successful
Code Review / ai-code-review (push) Successful in 21s
CD Pipeline / tests (push) Successful in 1m20s
CD Pipeline / build-and-deploy (push) Successful in 4m15s
CD Pipeline / post-deploy-checks (push) Successful in 1m58s
2026-05-12 19:55:13 +08:00
Your Name
33c0577e93 docs(ops): 記錄 RLS role bootstrap 套用 2026-05-12 19:35:28 +08:00
Your Name
f0255e0300 chore(ops): 補強 RLS role bootstrap gate
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
2026-05-12 18:36:35 +08:00
Your Name
0bc1878778 chore(ops): 新增 RLS preflight 與 registry certbot 修復包
All checks were successful
Code Review / ai-code-review (push) Successful in 13s
2026-05-12 18:25:53 +08:00
Your Name
a18e2f9c3f fix(security): 停用 GitHub production deploy 2026-05-12 16:22:16 +08:00
Your Name
6b02f49fc6 docs(backup): 校正 MOMO 備份驗證紀錄 2026-05-12 15:53:20 +08:00
Your Name
216b7d78e2 fix(backup): 接入 MOMO PG 備份失敗通知
Some checks failed
Code Review / ai-code-review (push) Successful in 11s
Ansible Lint / lint (push) Has been cancelled
2026-05-12 15:50:44 +08:00
Your Name
abdab85362 docs(awooop): record host backup notification deploy 2026-05-12 14:59:17 +08:00
Your Name
116fdbb33f docs(awooop): record ops notification deployment 2026-05-12 14:55:48 +08:00
AWOOOI CD
9db1e9b7a5 chore(cd): deploy 1a74286 [skip ci] 2026-05-12 14:48:50 +08:00
Your Name
1a74286dfa fix(awooop): mirror ops notifications through api
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
2026-05-12 14:43:09 +08:00
AWOOOI CD
b437a33043 chore(cd): deploy 03ba967 [skip ci] 2026-05-12 14:31:32 +08:00
Your Name
03ba9678d5 fix(awooop): label cicd outbound timeline
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
CD Pipeline / tests (push) Successful in 1m3s
CD Pipeline / build-and-deploy (push) Successful in 4m1s
CD Pipeline / post-deploy-checks (push) Successful in 1m26s
2026-05-12 14:26:29 +08:00
Your Name
d74beb2176 fix(ci): prevent docker lock self match
All checks were successful
Code Review / ai-code-review (push) Successful in 11s
2026-05-12 14:21:57 +08:00
AWOOOI CD
f824308b6a chore(cd): deploy cb7151c [skip ci] 2026-05-12 06:12:20 +00:00
Your Name
cb7151cc27 fix(awooop): set shadow run defaults for mirrors
All checks were successful
Code Review / ai-code-review (push) Successful in 10s
CD Pipeline / tests (push) Successful in 1m5s
CD Pipeline / build-and-deploy (push) Successful in 10m20s
CD Pipeline / post-deploy-checks (push) Successful in 2m33s
2026-05-12 14:01:03 +08:00
Your Name
ad8ead2546 fix(awooop): route ci notifications through event mirror
Some checks failed
Code Review / ai-code-review (push) Successful in 14s
CD Pipeline / tests (push) Successful in 1m18s
CD Pipeline / post-deploy-checks (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-05-12 13:58:08 +08:00
547 changed files with 218736 additions and 6637 deletions

View File

@@ -1,832 +0,0 @@
{
"permissions": {
"allow": [
"Read(**)",
"Glob(**)",
"Grep(**)",
"Bash(curl *)",
"Bash(kubectl get *)",
"Bash(kubectl describe *)",
"Bash(kubectl logs *)",
"Bash(kubectl rollout status *)",
"Bash(docker ps *)",
"Bash(docker logs *)",
"Bash(ls *)",
"Bash(cat *)",
"Bash(head *)",
"Bash(tail *)",
"Bash(grep *)",
"Bash(find *)",
"Bash(pwd)",
"Bash(which *)",
"Bash(echo *)",
"Bash(git status *)",
"Bash(git log *)",
"Bash(git diff *)",
"Bash(git branch *)",
"Bash(git remote *)",
"Edit(**)",
"Write(apps/**)",
"Write(packages/**)",
"Write(docs/**)",
"Write(.agents/**)",
"Write(k8s/**)",
"Write(scripts/**)",
"Bash(pnpm *)",
"Bash(npm *)",
"Bash(npx *)",
"Bash(node *)",
"Bash(python *)",
"Bash(python3 *)",
"Bash(pip *)",
"Bash(cd *)",
"Bash(mkdir *)",
"Bash(touch *)",
"Bash(cp *)",
"Bash(mv *)",
"Bash(chmod *)",
"Bash(pytest *)",
"Bash(playwright *)",
"Bash(git add *)",
"Bash(git commit *)",
"Bash(git stash *)",
"Bash(ssh *)",
"Bash(scp *)",
"Bash(export KUBECONFIG=*)",
"Bash(git push:*)",
"Bash(claude --version)",
"Bash(git check-ignore:*)",
"WebSearch",
"Bash(claude plugin:*)",
"Bash(claude --channels)",
"Bash(claude --channels plugin:telegram@claude-plugins-official --help)",
"Bash(bash)",
"Bash(source ~/.zshrc)",
"Bash(~/.bun/bin/bun --version)",
"Bash(env)",
"Bash(claude upgrade:*)",
"Bash(/Users/ogt/.local/bin/claude --help)",
"Bash(CLAUDE_CODE_EXPERIMENTAL_CHANNELS=1 claude --help)",
"Bash(claude --channels plugin:telegram@claude-plugins-official --print \"hello\")",
"Bash(mkdir -p ~/.claude/channels/telegram)",
"Bash(~/.claude/channels/telegram/.env)",
"Bash(~/.bun/bin/bun run:*)",
"Bash(sudo ln:*)",
"Bash(ln -sf ~/.bun/bin/bun /opt/homebrew/bin/bun)",
"Bash(xargs python:*)",
"Bash(uv --version)",
"Bash(pip3 install:*)",
"Bash(pip3 show:*)",
"Bash(ruff *)",
"Bash(mypy *)",
"Bash(black *)",
"Bash(isort *)",
"Bash(timeout *)",
"Bash(wc *)",
"Bash(sort *)",
"Bash(uniq *)",
"Bash(awk *)",
"Bash(sed *)",
"Bash(tr *)",
"Bash(tee *)",
"Bash(xargs *)",
"Bash(test *)",
"Bash([ *)",
"Bash(true)",
"Bash(false)",
"Bash(date *)",
"Bash(sleep *)",
"Bash(kill *)",
"Bash(pkill *)",
"Bash(ps *)",
"Bash(top *)",
"Bash(htop *)",
"Bash(df *)",
"Bash(du *)",
"Bash(free *)",
"Bash(uname *)",
"Bash(hostname *)",
"Bash(whoami)",
"Bash(id *)",
"Bash(groups *)",
"Bash(stat *)",
"Bash(file *)",
"Bash(realpath *)",
"Bash(dirname *)",
"Bash(basename *)",
"Bash(type *)",
"Bash(command *)",
"Bash(hash *)",
"Bash(alias *)",
"Bash(set *)",
"Bash(unset *)",
"Bash(printenv *)",
"Bash(diff *)",
"Bash(cmp *)",
"Bash(comm *)",
"Bash(join *)",
"Bash(paste *)",
"Bash(cut *)",
"Bash(rev *)",
"Bash(nl *)",
"Bash(fmt *)",
"Bash(fold *)",
"Bash(pr *)",
"Bash(expand *)",
"Bash(unexpand *)",
"Bash(od *)",
"Bash(xxd *)",
"Bash(hexdump *)",
"Bash(strings *)",
"Bash(base64 *)",
"Bash(md5sum *)",
"Bash(sha256sum *)",
"Bash(jq *)",
"Bash(yq *)",
"Bash(gh *)",
"Bash(docker build *)",
"Bash(docker run *)",
"Bash(docker exec *)",
"Bash(docker compose *)",
"Bash(docker-compose *)",
"Bash(docker images *)",
"Bash(docker inspect *)",
"Bash(docker network *)",
"Bash(docker volume *)",
"Bash(kubectl apply *)",
"Bash(kubectl create *)",
"Bash(kubectl exec *)",
"Bash(kubectl port-forward *)",
"Bash(kubectl config *)",
"Bash(helm *)",
"Bash(terraform *)",
"Bash(ansible *)",
"Bash(bun *)",
"Bash(deno *)",
"Bash(cargo *)",
"Bash(rustc *)",
"Bash(go *)",
"Bash(java *)",
"Bash(javac *)",
"Bash(gradle *)",
"Bash(mvn *)",
"Bash(make *)",
"Bash(cmake *)",
"Bash(ninja *)",
"Bash(uv *)",
"Bash(poetry *)",
"Bash(pipx *)",
"Bash(virtualenv *)",
"Bash(venv *)",
"Bash(conda *)",
"Bash(brew *)",
"Bash(apt *)",
"Bash(apt-get *)",
"Bash(yum *)",
"Bash(dnf *)",
"Bash(pacman *)",
"Bash(snap *)",
"Bash(flatpak *)",
"Bash(systemctl status *)",
"Bash(journalctl *)",
"Bash(service * status)",
"Bash(nc *)",
"Bash(netstat *)",
"Bash(ss *)",
"Bash(lsof *)",
"Bash(nmap *)",
"Bash(dig *)",
"Bash(nslookup *)",
"Bash(host *)",
"Bash(ping *)",
"Bash(traceroute *)",
"Bash(mtr *)",
"Bash(wget *)",
"Bash(http *)",
"Bash(httpie *)",
"Bash(hadolint apps/api/Dockerfile)",
"Bash(docker info:*)",
"Bash(kubectl cluster-info:*)",
"Read(//var/run/**)",
"Bash(open -a Docker)",
"Bash(git rm:*)",
"Bash(git reset:*)",
"Bash(kubectl --kubeconfig ~/.kube/config get pods -n awoooi -o wide)",
"Bash(kubectl scale:*)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollam@192.168.0.188 \"docker ps -a | grep -i claw\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps -a | grep -i claw\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker start clawbot && sleep 3 && docker logs clawbot --tail=10\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps | grep clawbot && docker port clawbot\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot --tail=30\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cat /home/ollama/clawbot/.env | grep -E ''\\(TG_|TELEGRAM\\)'' | head -5\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker inspect clawbot --format=''{{range .Mounts}}{{.Source}}:{{.Destination}} {{end}}''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker inspect clawbot --format=''{{range .Config.Env}}{{println .}}{{end}}'' | grep -E ''\\(TG_|TELEGRAM|ENABLED\\)''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot 2>&1 | grep -i ''logout\\\\|log.out\\\\|shutdown\\\\|stop'' | tail -20\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot 2>&1 | grep -E ''\\(getMe|getUpdates|sendMessage\\).*200'' | tail -5\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot 2>&1 | grep -i ''success\\\\|started\\\\|初始化'' | head -20\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot 2>&1 | grep -E ''2026-03-\\(19|20|21\\)'' | grep -i ''error\\\\|fail\\\\|logout\\\\|400\\\\|401'' | head -20\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker stop clawbot && docker rm clawbot && echo ''✅ OpenClaw 已永久停用''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd /home/ollama/clawbot-v5 && docker-compose ps 2>/dev/null || ls -la docker-compose.yml 2>/dev/null || find /home/ollama -name ''docker-compose*.yml'' -type f 2>/dev/null | head -5\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd /home/ollama/clawbot-v5 && docker-compose up -d && sleep 3 && docker-compose ps\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd /home/ollama/clawbot-v5 && docker compose up -d 2>&1 || docker run -d --name clawbot --restart unless-stopped -p 8088:8088 -v /var/run/docker.sock:/var/run/docker.sock 192.168.0.110:5000/library/clawbot:stable-v6 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot --tail=15 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps --format ''table {{.Names}}\\\\t{{.Status}}'' | grep -E ''clawbot|litellm''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd /home/ollama/clawbot-v5 && sed -i ''s|TELEGRAM_BOT_TOKEN=.*|TELEGRAM_BOT_TOKEN=8569720657:AAHrJ5CMOb4rP0IYJrCUiDViLsnpK69uEUI|'' .env && grep TELEGRAM_BOT_TOKEN .env\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd /home/ollama/clawbot-v5 && docker compose down && docker compose up -d && sleep 5 && docker logs clawbot --tail=10\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps --format ''{{.Names}}'' | grep -i alert\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker stop alertmanager && docker rm alertmanager && echo ''✅ 舊 AIOPS Alertmanager 已停用''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps --format ''table {{.Names}}\\\\t{{.Image}}\\\\t{{.Status}}''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cat /home/ollama/momo-pro/monitoring/prometheus/alert_rules.yml 2>/dev/null | grep -A5 ''ClawbotDown\\\\|telegram\\\\|AIOPS'' | head -30\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"find /home/ollama -name ''*.yml'' -type f 2>/dev/null | xargs grep -l ''ClawbotDown\\\\|telegram'' 2>/dev/null | head -5\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec clawbot grep -r ''協同警報\\\\|ClawbotDown'' /app 2>/dev/null | head -5\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec prometheus cat /etc/prometheus/prometheus.yml 2>/dev/null | grep -A10 ''alerting\\\\|alertmanager''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps | grep -i alert || echo ''✅ 沒有 alertmanager 在運行''\")",
"Bash(jq -r '.status, .components | to_entries[] | \"\"\"\"\\\\\\(.key\\): \\\\\\(.value.status\\)\"\"\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps --format ''table {{.Names}}\\\\t{{.Status}}'' | grep clawbot && docker logs clawbot --tail=15\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker inspect clawbot --format=''{{range .Config.Env}}{{println .}}{{end}}'' | grep TELEGRAM\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd /home/ollama/clawbot-v5 && sed -i ''s|TELEGRAM_BOT_TOKEN=.*|TELEGRAM_BOT_TOKEN=8569720657:AAFjDyjAN94QQrjn1gBnFXAyS20EUyozH8c|'' .env && docker compose down && docker compose up -d && sleep 5 && docker logs clawbot --tail=10\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec clawbot grep -r ''ClawBotDown\\\\|ClawbotDown'' /app 2>/dev/null | head -5 || echo ''在程式碼中找不到''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec prometheus cat /etc/prometheus/alerts.yml 2>/dev/null | grep -A10 ''ClawBot\\\\|clawbot'' | head -30\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec prometheus cat /etc/prometheus/alerts.yml 2>/dev/null | grep -i ''clawbot\\\\|claw'' -A5 -B5\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot --since=5m 2>&1 | grep -i ''clawbot\\\\|incident\\\\|alert'' | tail -20\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot --tail 50 2>&1\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot 2>&1 | grep -i ''telegram\\\\|polling\\\\|bot'' | tail -20\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps --format ''table {{.Names}}\\\\t{{.Status}}\\\\t{{.Ports}}'' | grep -E ''claw|NAME''\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot 2>&1 | grep -E ''telegram|Telegram|error|Error'' | tail -20\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps | grep ollama\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps -a --format ''table {{.Names}}\\\\t{{.Status}}'' | head -20\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"sed -i ''s|host.docker.internal|172.17.0.1|g'' /home/ollama/clawbot-v5/.env && cat /home/ollama/clawbot-v5/.env | grep OLLAMA\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd /home/ollama/clawbot-v5 && docker-compose restart clawbot && sleep 3 && docker logs clawbot --tail 30 2>&1\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd /home/ollama/clawbot-v5 && docker compose restart clawbot && sleep 5 && docker logs clawbot --tail 30 2>&1\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec clawbot curl -s http://172.17.0.1:11434/api/tags | head -c 200\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot 2>&1 | tail -10\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot 2>&1 | grep -iE ''error|telegram|polling|alert|send'' | tail -30\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cat /home/ollama/clawbot-v5/.env | grep OLLAMA\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd /home/ollama/clawbot-v5 && docker compose up -d --force-recreate clawbot && sleep 5 && docker logs clawbot 2>&1 | tail -20\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec clawbot curl -s http://172.17.0.1:11434/api/tags | head -c 100\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot --since 5m 2>&1 | tail -30\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec momo-db psql -U postgres -d clawbot -c \"\"SELECT enum_range\\(NULL::approvalstatus\\);\"\"\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec -e PGPASSWORD=clawbot123 momo-db psql -U clawbot -d clawbot -c \"\"SELECT enum_range\\(NULL::approvalstatus\\);\"\"\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps | grep -E ''postgres|db''\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker exec momo-db env | grep -i postgres\")",
"Bash(sshpass -p \"0936223270\" ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"PGPASSWORD=AwoooiProd2026 psql -h localhost -U awoooi -d awoooi_prod -c \"\"SELECT enum_range\\(NULL::approvalstatus\\);\"\"\")",
"Bash(KUBECONFIG=~/.kube/config kubectl config get-contexts)",
"Bash(docker tag:*)",
"Bash(docker push:*)",
"Bash(ssh ollama@192.168.0.188 \"cd ~/awoooi-build && find apps/web/src -name ''''*.ts'''' -o -name ''''*.tsx'''' | head -30 | xargs md5sum\")",
"Bash(rsync -avz --exclude 'node_modules' --exclude '.next' --exclude '.turbo' --exclude '*.log' /Users/ogt/awoooi/ ollama@192.168.0.188:~/awoooi-build/)",
"Bash(gh run:*)",
"Bash(APPROVAL_ID=\"ea43578e-17cd-40b9-b4c3-8fe8e92f225c\" __NEW_LINE_76dc92b2699cd7d5__ echo \"=== 檢查 Approval Metadata ===\" curl -s \"https://awoooi.wooo.work/api/v1/approvals/pending\")",
"Bash(APPROVAL_ID=\"865ab726-c3b9-447e-86a9-65a6227516e6\" __NEW_LINE_db14ef76ca26af32__ echo \"=== 簽核 ===\" curl -s -X POST \"https://awoooi.wooo.work/api/v1/approvals/$APPROVAL_ID/sign\" -H \"Content-Type: application/json\" -d '{\"\"\"\"signer_id\"\"\"\":\"\"\"\"commander\"\"\"\",\"\"\"\"signer_name\"\"\"\":\"\"\"\"Commander\"\"\"\",\"\"\"\"comment\"\"\"\":\"\"\"\"Test resolution\"\"\"\"}')",
"Read(//Users/ogt/awoooi/**)",
"Bash(APPROVAL_ID=\"e9445e68-6c3e-4899-b507-3b9b7bcaf0a7\" __NEW_LINE_680ad94d4896e58a__ echo \"=== 簽核 ===\" curl -s -X POST \"https://awoooi.wooo.work/api/v1/approvals/$APPROVAL_ID/sign\" -H \"Content-Type: application/json\" -d '{\"\"\"\"signer_id\"\"\"\":\"\"\"\"commander\"\"\"\",\"\"\"\"signer_name\"\"\"\":\"\"\"\"Commander\"\"\"\",\"\"\"\"comment\"\"\"\":\"\"\"\"Final test\"\"\"\"}')",
"Bash(APPROVAL_ID=\"eb0afb4e-834b-4af7-9ae0-3c58232fdd99\" INCIDENT=\"INC-20260323-F05CD6\" __NEW_LINE_47f1c3803a64b43c__ echo \"=== 簽核前 Incident 狀態 ===\" curl -s \"https://awoooi.wooo.work/api/v1/incidents/$INCIDENT\")",
"Bash(mkdir -p /Users/ogt/awoooi/.claude/hooks)",
"Bash(/Users/ogt/awoooi/.claude/hooks/pre-commit-check.sh:*)",
"Bash(git -C /Users/ogt/awoooi status packages/lewooogo-core/)",
"Bash(git -C /Users/ogt/awoooi ls-files packages/lewooogo-core/src/)",
"Bash(git -C /Users/ogt/awoooi status --short)",
"Bash(git -C /Users/ogt/awoooi add apps/api/pyproject.toml apps/api/scripts/ apps/api/src/ apps/web/.eslintrc.js apps/web/src/ packages/lewooogo-core/.eslintrc.js)",
"Bash(git -C /Users/ogt/awoooi diff --cached --stat)",
"Bash(git -C:*)",
"Bash(for wf:*)",
"Bash(do)",
"Bash(done)",
"Bash(jq 'if type == \"\"\"\"array\"\"\"\" then .[0] | {incident_id, status, decision} else . end')",
"Bash(PYTHONPATH=. python -c \"from src.api.v1.stats import router; print\\(''✅ stats.py 載入成功,路由數:'', len\\(router.routes\\)\\)\")",
"Bash(PYTHONPATH=. pytest tests/ -v --tb=short)",
"Bash(PYTHONPATH=. pytest tests/test_stats_api.py -v --tb=short)",
"Bash(PYTHONPATH=. pytest tests/test_webhook_telegram_integration.py::TestNewAlertTelegramPush -v --tb=long)",
"Bash(PYTHONPATH=. pytest tests/test_webhook_telegram_integration.py::TestNewAlertTelegramPush -v --tb=short)",
"Bash(PYTHONPATH=. pytest tests/test_webhook_telegram_integration.py -v --tb=short)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get pods -n awoooi')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get ns awoooi && kubectl get all -n awoooi')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get ns | head -20')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get pods -n awoooi-prod')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl logs awoooi-worker-bb89b5ffc-bpf45 -n awoooi-prod --tail=50')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl logs awoooi-worker-bb89b5ffc-bpf45 -n awoooi-prod --tail=100 | grep -i telegram')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl logs awoooi-api-8c9489b6c-cm8g5 -n awoooi-prod --tail=50 | grep -i webhook')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl logs awoooi-api-8c9489b6c-cm8g5 -n awoooi-prod --tail=30')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get pods -n monitoring | grep alertmanager')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"kubectl get configmap alertmanager-config -n monitoring -o jsonpath=''{.data.alertmanager\\\\.yml}'' | head -50\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get svc -n awoooi-prod')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"kubectl patch configmap alertmanager-config -n monitoring --type merge -p ''{\"\"data\"\":{\"\"alertmanager.yml\"\":\"\"global:\\\\n resolve_timeout: 5m\\\\n\\\\nroute:\\\\n group_by: [\\\\\"\"alertname\\\\\"\", \\\\\"\"severity\\\\\"\"]\\\\n group_wait: 30s\\\\n group_interval: 5m\\\\n repeat_interval: 4h\\\\n receiver: \\\\\"\"awoooi-webhook\\\\\"\"\\\\n routes:\\\\n - match:\\\\n severity: critical\\\\n receiver: \\\\\"\"awoooi-webhook\\\\\"\"\\\\n group_wait: 10s\\\\n repeat_interval: 1h\\\\n - match:\\\\n severity: warning\\\\n receiver: \\\\\"\"awoooi-webhook\\\\\"\"\\\\n group_wait: 1m\\\\n repeat_interval: 4h\\\\n\\\\nreceivers:\\\\n - name: \\\\\"\"awoooi-webhook\\\\\"\"\\\\n webhook_configs:\\\\n - url: \\\\\"\"http://192.168.0.120:32334/api/v1/webhook/alertmanager\\\\\"\"\\\\n send_resolved: true\\\\n\\\\ninhibit_rules:\\\\n - source_match:\\\\n severity: \\\\\"\"critical\\\\\"\"\\\\n target_match:\\\\n severity: \\\\\"\"warning\\\\\"\"\\\\n equal: [\\\\\"\"alertname\\\\\"\", \\\\\"\"instance\\\\\"\"]\\\\n\"\"}}''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl rollout restart deployment/alertmanager -n monitoring && kubectl rollout status deployment/alertmanager -n monitoring')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"kubectl get configmap alertmanager-config -n monitoring -o jsonpath=''{.data.alertmanager\\\\.yml}'' | grep -A 3 ''url:''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get pods -n awoooi-prod -o jsonpath=\"\"{range .items[*]}{.metadata.name}{\\\\\"\" \\\\\"\"}{.spec.containers[*].image}{\\\\\"\"\\\\\\\\n\\\\\"\"}{end}\"\"')",
"Bash(git mv:*)",
"Bash(for file:*)",
"Bash(do echo:*)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10 wooo@192.168.0.120 \"echo ''Connected''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"kubectl get deployment -n awoooi-prod -o jsonpath=''{range .items[*]}{.metadata.name}{\"\" selector: \"\"}{.spec.selector.matchLabels}{\"\"\\\\n\"\"}{end}''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"kubectl delete deployment awoooi-api awoooi-web awoooi-worker -n awoooi-prod\")",
"WebFetch(domain:awoooi.wooo.work)",
"WebFetch(domain:api.awoooi.wooo.work)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get pods -n awoooi-prod -o wide')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get svc,ingress -n awoooi-prod')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl exec -n awoooi-prod deploy/awoooi-api -- curl -sf http://localhost:8000/api/v1/health 2>&1')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'curl -sf http://10.43.125.201:8000/api/v1/health 2>&1 || echo \"\"FAILED\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'sudo nginx -t 2>&1 && sudo cat /etc/nginx/sites-enabled/awoooi* 2>/dev/null || sudo cat /etc/nginx/conf.d/awoooi* 2>/dev/null || echo \"\"No awoooi nginx config found\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'cat /etc/nginx/sites-enabled/* 2>/dev/null | grep -A5 awoooi || cat /etc/nginx/conf.d/* 2>/dev/null | grep -A5 awoooi || ls -la /etc/nginx/ 2>/dev/null || echo \"\"No nginx on this host\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'ls /etc/nginx/sites-enabled/ 2>/dev/null && cat /etc/nginx/sites-enabled/*awoooi* 2>/dev/null || echo \"\"Checking conf.d...\"\" && ls /etc/nginx/conf.d/ 2>/dev/null')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'grep -l awoooi /etc/nginx/sites-enabled/* 2>/dev/null || grep -r \"\"awoooi\"\" /etc/nginx/sites-enabled/ 2>/dev/null | head -20')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'grep -r \"\"awoooi\\\\|32334\\\\|32335\"\" /etc/nginx/ 2>/dev/null | head -20')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S cp /tmp/awoooi-prod.conf /etc/nginx/conf.d/ && echo \"\"Config copied\"\" && sudo nginx -t 2>&1')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S ls -la /etc/nginx/ssl/ 2>/dev/null || echo \"\"No ssl dir\"\" && sudo ls -la /etc/letsencrypt/live/ 2>/dev/null | head -10')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S sed -i \"\"s|/etc/nginx/ssl/awoooi.crt|/etc/letsencrypt/live/awoooi.wooo.work/fullchain.pem|g\"\" /etc/nginx/conf.d/awoooi-prod.conf && sudo sed -i \"\"s|/etc/nginx/ssl/awoooi.key|/etc/letsencrypt/live/awoooi.wooo.work/privkey.pem|g\"\" /etc/nginx/conf.d/awoooi-prod.conf && echo \"\"Paths fixed\"\" && sudo nginx -t 2>&1')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S nginx -s reload && echo \"\"Nginx reloaded!\"\" && sleep 2')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'grep -r \"\"awoooi\"\" /etc/nginx/sites-enabled/ 2>/dev/null | head -5')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S grep -rl \"\"awoooi.wooo.work\"\" /etc/nginx/ 2>/dev/null')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'curl -sf http://192.168.0.121:32334/api/v1/health 2>&1 || echo \"\"FAILED to reach 121\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S rm /etc/nginx/conf.d/awoooi-prod.conf && sudo nginx -t && sudo nginx -s reload && echo \"\"Cleaned up duplicate config\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S tail -30 /var/log/nginx/error.log 2>/dev/null')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'grep -r \"\"api.awoooi\"\" /etc/nginx/ 2>/dev/null || echo \"\"No api.awoooi config found\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get configmap awoooi-config -n awoooi-prod -o yaml | grep -A5 NEXT_PUBLIC')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl get deployment awoooi-web -n awoooi-prod -o yaml | grep -A20 \"\"env:\"\" | head -25')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S tail -10 /var/log/nginx/access.log 2>/dev/null | grep awoooi')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S tail -5 /var/log/nginx/error.log 2>/dev/null')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S stat /etc/nginx/sites-available/awoooi.wooo.work.conf 2>/dev/null | grep -E \"\"Modify|Change|Birth\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl logs -n awoooi-prod -l app=awoooi-web --tail=30 2>/dev/null | grep -i \"\"api\\\\|error\\\\|fetch\"\" | head -20')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S tail -20 /var/log/nginx/access.log 2>/dev/null | grep -E \"\"awoooi.*api\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S tail -20 /var/log/nginx/awoooi-prod-access.log 2>/dev/null')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl exec -n awoooi-prod deploy/awoooi-web -- env | grep -i api')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl exec -n awoooi-prod deploy/awoooi-web -- sh -c \"\"grep -r \\\\\"\"NEXT_PUBLIC_API_URL\\\\|api.awoooi\\\\\"\" /app/.next/static/chunks/*.js 2>/dev/null | head -5 || grep -r \\\\\"\"awoooi.wooo.work\\\\\"\" /app/.next/static/chunks/*.js 2>/dev/null | head -3\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'kubectl exec -n awoooi-prod deploy/awoooi-web -- sh -c \"\"find /app/.next -name \\\\\"\"*.js\\\\\"\" -exec grep -l \\\\\"\"awoooi\\\\\"\" {} \\\\; 2>/dev/null | head -3\"\"')",
"Bash(./scripts/qa-zero-touch.sh)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S cat /etc/nginx/sites-available/awoooi.wooo.work.conf')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S cp /tmp/awoooi.wooo.work.conf /etc/nginx/sites-available/awoooi.wooo.work.conf && sudo nginx -t 2>&1')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'echo \"\"0936223270\"\" | sudo -S nginx -s reload && echo \"\"✅ Nginx reloaded with load balancing!\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'cd /opt && sudo ls -la sentry 2>/dev/null || echo \"\"Sentry 目錄不存在,需要建立\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'sudo mkdir -p /opt/sentry && sudo chown wooo:wooo /opt/sentry && cd /opt/sentry && git clone https://github.com/getsentry/self-hosted.git . 2>&1 | tail -5')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'echo \"\"0936223270\"\" | sudo -S mkdir -p /opt/sentry && echo \"\"0936223270\"\" | sudo -S chown wooo:wooo /opt/sentry && cd /opt/sentry && git clone https://github.com/getsentry/self-hosted.git . 2>&1 | tail -10')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'cd /opt/sentry && ls -la 2>&1 | head -20')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'cd /opt/sentry && git describe --tags 2>/dev/null || git rev-parse --short HEAD')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'cd /opt/sentry && ./install.sh --help 2>&1 | head -30 || echo \"\"No help available, checking script...\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'cd /opt/sentry && nohup ./install.sh --skip-user-creation --no-report-self-hosted-issues > /tmp/sentry-install.log 2>&1 &')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'tail -30 /tmp/sentry-install.log 2>/dev/null || echo \"\"日誌檔案尚未建立,等待中...\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'grep -E \"\"^\\\\▶|^Creating|^Starting|^Error|^✓|Pulling\"\" /tmp/sentry-install.log 2>/dev/null | tail -40')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'echo \"\"=== 日誌行數 ===\"\" && wc -l /tmp/sentry-install.log && echo \"\"\"\" && echo \"\"=== 最近進度 ===\"\" && tail -10 /tmp/sentry-install.log')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'echo \"\"=== 日誌行數 ===\"\" && wc -l /tmp/sentry-install.log && echo \"\"\"\" && echo \"\"=== 關鍵階段 ===\"\" && grep -E \"\"^▶|✓|Error|Creating|Starting\"\" /tmp/sentry-install.log | tail -20')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'echo \"\"=== 日誌行數 ===\"\" && wc -l /tmp/sentry-install.log && echo \"\"\"\" && echo \"\"=== 最近 20 行 ===\"\" && tail -20 /tmp/sentry-install.log')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'echo \"\"=== 日誌行數 ===\"\" && wc -l /tmp/sentry-install.log && echo \"\"\"\" && echo \"\"=== 關鍵階段 ===\"\" && grep -E \"\"^▶|✓|Error|Creating|Starting|Building|DONE\"\" /tmp/sentry-install.log | tail -30')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'echo \"\"=== 日誌行數 ===\"\" && wc -l /tmp/sentry-install.log && echo \"\"\"\" && echo \"\"=== 最近關鍵階段 ===\"\" && grep -E \"\"^▶|✓|Error|Creating|Starting|DONE|Completed|success\"\" /tmp/sentry-install.log | tail -25')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.110 'grep -E \"\"^▶|✓|Error|Completed|success|fail\"\" /tmp/sentry-install.log | tail -15')",
"Bash(redis-cli -h 192.168.0.188 -p 6380 KEYS incident:*)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cat /home/ollama/momo-pro/monitoring/alertmanager.yml 2>/dev/null || cat /etc/alertmanager/alertmanager.yml 2>/dev/null || echo ''Config not found''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot --tail 30 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker logs clawbot --tail 20 2>&1 | grep -iE ''telegram|send|alert|incident|error''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cat /home/ollama/clawbot-v5/.env | grep -E ''TELEGRAM|TG_'' | head -5\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cat /home/ollama/clawbot-v5/.env | grep -E ''REDIS|POSTGRES|DATABASE'' | head -5\")",
"Bash(ssh ollama@192.168.0.188 'curl -s \"\"http://localhost:9093/api/v2/alerts?active=true\"\" | python3 -c \"\"import sys,json; alerts=json.load\\(sys.stdin\\); print\\(f\\\\\"\"Active alerts: {len\\(alerts\\)}\\\\\"\"\\)\"\"')",
"Bash(ssh ollama@192.168.0.188 'curl -s \"\"http://localhost:9093/api/v2/alerts\"\" | python3 -c \"\"import sys,json; alerts=json.load\\(sys.stdin\\); print\\(f\\\\\"\"Total alerts: {len\\(alerts\\)}\\\\\"\"\\); [print\\(a[\\\\\"\"labels\\\\\"\"][\\\\\"\"alertname\\\\\"\"]\\) for a in alerts[:5]]\"\"')",
"Bash(ssh ollama@192.168.0.188 'redis-cli -p 6380 -n 0 GET incident:INC-20260324-36AF55 | python3 -c \"\"import sys,json; d=json.load\\(sys.stdin\\); print\\(f\\\\\"\"Status: {d.get\\(\\\\\"\"status\\\\\"\"\\)}\\\\\"\"\\); print\\(f\\\\\"\"message_id: {d.get\\(\\\\\"\"message_id\\\\\"\", \\\\\"\"NONE\\\\\"\"\\)}\\\\\"\"\\); print\\(f\\\\\"\"chat_id: {d.get\\(\\\\\"\"chat_id\\\\\"\", \\\\\"\"NONE\\\\\"\"\\)}\\\\\"\"\\)\"\"')",
"Bash(ssh ollama@192.168.0.188 'redis-cli -p 6380 -n 0 GET incident:INC-20260324-36AF55 | python3 -c \"\"import sys,json; d=json.load\\(sys.stdin\\); print\\(f\\\\\"\"status: {d.get\\('status'\\)}\\\\\"\"\\); print\\(f\\\\\"\"message_id: {d.get\\('message_id'\\)}\\\\\"\"\\); print\\(f\\\\\"\"created_at: {d.get\\('created_at'\\)}\\\\\"\"\\)\"\"')",
"Bash(redis-cli -h 192.168.0.188 -p 6380 -n 0 KEYS *approval*)",
"Bash(redis-cli -h 192.168.0.188 -p 6380 -n 0 KEYS *incident*)",
"Bash(redis-cli -h 192.168.0.188 -p 6380 -n 0 KEYS *pending*)",
"Bash(redis-cli -h 192.168.0.188 -p 6380 -n 0 KEYS *)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/k3s-prod.yaml kubectl get pods -n awoooi-prod -o wide)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/k3s-prod.yaml kubectl get deployment awoooi-api -n awoooi-prod -o jsonpath='{.spec.template.spec.containers[0].image}')",
"Bash(kubectl --kubeconfig=/Users/ogt/awoooi/k3s-prod.yaml get deployment awoooi-api -n awoooi-prod -o jsonpath='{.spec.template.spec.containers[0].image}')",
"Bash(python3 -c \":*)",
"Bash(/tmp/awoooi-tg-secret.yaml:*)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/k3s-prod.yaml kubectl apply -f /tmp/awoooi-tg-secret.yaml)",
"Bash(for pod:*)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.188 \"curl -fsSL https://ollama.com/install.sh | sh\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no -o PreferredAuthentications=password wooo@192.168.0.188 \"echo connected && ollama --version\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no -o PreferredAuthentications=password ollama@192.168.0.188 \"curl -fsSL https://ollama.com/install.sh | sh\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S curl -fsSL https://ollama.com/install.sh | sudo -S sh\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"ollama --version\")",
"Bash(__NEW_LINE_95e9df111552805b__ echo:*)",
"Bash(sshpass -p '0936223270' scp /Users/ogt/awoooi/k8s/nginx/awoooi-prod.conf ollama@192.168.0.188:/tmp/awoooi-prod.conf)",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S cp /tmp/awoooi-prod.conf /etc/nginx/conf.d/awoooi-prod.conf && echo ''0936223270'' | sudo -S nginx -t 2>&1\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S ls -la /etc/nginx/ssl/ 2>/dev/null || echo ''No ssl dir''; echo ''0936223270'' | sudo -S ls -la /etc/nginx/conf.d/ 2>/dev/null | head -10\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S grep -r ''ssl_certificate'' /etc/nginx/ 2>/dev/null | head -5\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S grep -A 20 ''server_name awoooi'' /etc/nginx/sites-enabled/all-sites.conf 2>/dev/null | head -30\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S ls -la /etc/nginx/sites-enabled/ 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S cat /etc/nginx/sites-available/awoooi.wooo.work.conf 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S rm /etc/nginx/conf.d/awoooi-prod.conf && echo ''0936223270'' | sudo -S nginx -t 2>&1\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S nginx -s reload 2>&1\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S systemctl reload nginx 2>&1\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker logs openclaw 2>&1 | tail -30\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker ps -a --format ''table {{.Names}}\\\\t{{.Status}}\\\\t{{.Image}}'' 2>&1 | head -15\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker logs clawbot 2>&1 | grep -i telegram | tail -20\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker logs clawbot 2>&1 | tail -30\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker exec alertmanager cat /etc/alertmanager/alertmanager.yml 2>&1 | head -30\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"curl -sf ''http://localhost:9093/api/v1/alerts'' | jq ''.data | length'' 2>/dev/null || curl -sf ''http://localhost:9093/api/v2/alerts'' | jq ''length'' 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker exec alertmanager wget -qO- ''http://localhost:9093/api/v2/alerts'' 2>&1 | head -100\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n awoooi-prod logs -l app=awoooi-worker --tail=50 2>&1\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"cat /home/ollama/alertmanager/alertmanager.yml 2>/dev/null || docker exec alertmanager cat /etc/alertmanager/alertmanager.yml\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker cp /tmp/alertmanager.yml alertmanager:/etc/alertmanager/alertmanager.yml && docker exec alertmanager amtool check-config /etc/alertmanager/alertmanager.yml && docker kill -s SIGHUP alertmanager\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker inspect alertmanager --format ''{{range .Mounts}}{{.Source}} -> {{.Destination}}{{println}}{{end}}''\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker exec alertmanager cat /etc/alertmanager/alertmanager.yml\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker restart alertmanager && sleep 3 && docker exec alertmanager cat /etc/alertmanager/alertmanager.yml\")",
"Bash(sshpass -p '0936223270' ssh ollama@192.168.0.188 \"docker logs clawbot 2>&1 | grep -i ''telegram\\\\|webhook\\\\|alert'' | tail -10\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=30 2>/dev/null | grep -E ''''POST|webhook|alertmanager|ManualTest''''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=30 2>/dev/null | grep -iE ''''POST|webhook''''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=50 2>/dev/null | grep -iE ''''POST.*webhook|alertmanager_webhook|NewFingerprint''''\")",
"Bash(kustomize build:*)",
"Bash(KUBECONFIG=~/.kube/config kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath='{.data}')",
"Bash(KUBECONFIG=/Users/ogt/.kube/config kubectl exec deploy/awoooi-api -n awoooi-prod -- env)",
"Bash(git checkout:*)",
"Bash(jq -r '.status // \"\"\"\"failed\"\"\"\"')",
"Bash(jq -r '.total // \"\"\"\"error\"\"\"\"')",
"Bash(redis-cli -h 192.168.0.188 -p 6380 -n 10 XLEN awoooi:signals)",
"Bash(redis-cli -h 192.168.0.188 -p 6380 -n 10 XRANGE awoooi:signals - + COUNT 5)",
"Bash(SENTRY_TOKEN=\"2b73050606d2b32f54095b4e177f4842f2bfe69d4b17da25f6daa4739148a972\" curl -s \"http://192.168.0.110:9000/api/0/organizations/\" -H \"Authorization: Bearer $SENTRY_TOKEN\")",
"Bash(SENTRY_TOKEN=\"2b73050606d2b32f54095b4e177f4842f2bfe69d4b17da25f6daa4739148a972\" curl -s \"http://192.168.0.110:9000/api/0/organizations/sentry/projects/\" -H \"Authorization: Bearer $SENTRY_TOKEN\")",
"Bash(SENTRY_TOKEN=\"2b73050606d2b32f54095b4e177f4842f2bfe69d4b17da25f6daa4739148a972\" curl -s \"http://192.168.0.110:9000/api/0/projects/sentry/awoooi-api/rules/\" -H \"Authorization: Bearer $SENTRY_TOKEN\")",
"Bash(SENTRY_TOKEN=\"2b73050606d2b32f54095b4e177f4842f2bfe69d4b17da25f6daa4739148a972\" __NEW_LINE_583db0bbb6875db0__ echo \"=== Alert Rules ===\" curl -s \"http://192.168.0.110:9000/api/0/projects/sentry/awoooi-api/rules/\" -H \"Authorization: Bearer $SENTRY_TOKEN\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get nodes -o wide && echo ''---'' && kubectl top nodes 2>/dev/null || echo ''metrics-server not installed''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod -o wide && echo ''---'' && kubectl get pvc -n awoooi-prod 2>/dev/null && echo ''---'' && kubectl get sc 2>/dev/null && echo ''---'' && kubectl get deploy -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get ns && echo ''---'' && kubectl get svc -A | grep -E ''prometheus|grafana|metrics|signoz|longhorn|argocd'' || echo ''No monitoring/gitops services found''\")",
"Bash(ssh wooo@192.168.0.120 \"cat /etc/rancher/k3s/config.yaml 2>/dev/null || echo ''--- K3s default config \\(no custom config.yaml\\) ---'' && echo ''---'' && sudo k3s check-config 2>/dev/null | head -30 || echo ''check-config not available''\")",
"Bash(ssh wooo@192.168.0.120 \"free -h && echo ''---'' && swapon --show && echo ''---'' && df -h /var/lib/rancher/k3s\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n cnpg-system && echo ''---'' && kubectl get svc -n monitoring\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get all -n awoooi-prod -o wide 2>/dev/null && echo ''---QUOTA---'' && kubectl describe quota -n awoooi-prod 2>/dev/null && echo ''---EVENTS---'' && kubectl get events -n awoooi-prod --sort-by=''.lastTimestamp'' 2>/dev/null | tail -20\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get helmcharts -A 2>/dev/null || echo ''No HelmCharts'' && echo ''---'' && kubectl get helmreleases -A 2>/dev/null || echo ''No HelmReleases'' && echo ''---'' && kubectl api-resources | grep -E ''argo|flux|velero|longhorn'' || echo ''No GitOps/Backup CRDs''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get ds -A && echo ''---'' && kubectl get cm -n kube-system | grep -E ''traefik|coredns'' && echo ''---REGISTRIES---'' && sudo cat /etc/rancher/k3s/registries.yaml 2>/dev/null || echo ''No registries.yaml''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get ingress -A 2>/dev/null || echo ''No Ingress'' && echo ''---HPA---'' && kubectl get hpa -A 2>/dev/null || echo ''No HPA'' && echo ''---PDB---'' && kubectl get pdb -A 2>/dev/null || echo ''No PDB'' && echo ''---SYSCTL---'' && cat /proc/sys/net/core/somaxconn && cat /proc/sys/fs/file-max\")",
"Bash(ssh wooo@192.168.0.120 \"systemctl status k3s | head -20 && echo ''---K3S-VERSION---'' && k3s --version && echo ''---ETCD-STATUS---'' && sudo k3s etcd-snapshot list 2>/dev/null | head -5 || echo ''No etcd snapshots''\")",
"Bash(ssh wooo@192.168.0.121 \"free -h && swapon --show && echo ''---DISK---'' && df -h /var/lib/rancher/k3s 2>/dev/null\")",
"Bash(ssh wooo@192.168.0.120 \"sudo ls -la /var/lib/rancher/k3s/server/db/ 2>/dev/null && echo ''---TOKEN---'' && sudo cat /var/lib/rancher/k3s/server/token 2>/dev/null | head -1 | cut -c1-20\")",
"Bash(ssh -o ConnectTimeout=10 wooo@192.168.0.120 \"ps aux | grep k3s | grep -v grep | head -3 && echo ''---'' && sudo cat /etc/systemd/system/k3s.service 2>/dev/null | grep -E ''ExecStart|datastore''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"echo ''0936223270'' | sudo -S mkdir -p /backup/k3s_etcd 2>/dev/null && echo ''0936223270'' | sudo -S chown ollama:ollama /backup/k3s_etcd 2>/dev/null && echo ''=== 188 備份目錄 ==='' && ls -la /backup/\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"mkdir -p ~/.ssh && chmod 700 ~/.ssh && echo ''ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnTnbjtSPwrI/pN6DByDxsFDOR4+sVnk7hb+eOr+Pb4e7o7QGbyKaJC2eKP7uRBilPqeScuvNKZhwmY8ZOuhjId+ZyLK0jZXHdq3a6tjsQ4MwPGyT2aMaD7x2jKzPbFojR0P5lmQWH2zjxeVuB7UeBIejaYk3gQEMFVES8Xh84yxFvy9jlwKmZFAI0gIhx0nPOTPB7onTyb8L5snUbwQQntoHWYFbb83+wui/kM15aLT5r8uvS2yZdsWWrDvAyuIShde1ceTBevwwqxezH1egXGoGkvZYYF7vHFu3X6jF7Nfp4qVfo0EfFV3omy90HzoFvoEXCC+jIWU0TjUqdEgGIEj2b+YXw3bIs+k+g/0/iJzA5LLUNb2vHVHoUmah4ZNlfiGU7e6hTYXjLjoXJlz9gfv6LYywhgktdThi9sUCn6rzbatlMrY0HNUE6uOwRTugMq1YUEJCvRqeFmtX5yF6xGp+FbOjIr1kMmplbRQRqKIrpQoqEn0+UBXC7OwJNCk8= wooo@mon'' >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys && echo ''SSH key 已加入''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"echo ''ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCs3nQ11B+V/VEchNR9Uzj57JoKXOJ8S1UVjCTHkUDL8FnrbdPFr0zvpYgX0a/Ipj9wHkqU6z6Ho6MQj3X2+HaK5fC0fZ3aZE1QT2df/x0xXdyka9XSaTFaymKzNTvfmum40koBkNccKyO5SLSjTcoTZCDHP4RqHHu/MYjQMejG7yeyCFmgumrHh5T/0DXPf5zl0Ff1C5U3VCLPxz5vq63JB2dTfrjQLg3sO0ZI3KTZE8aFj3txKz5snDZX3nE1tHZMKLecwwEqi130BtVZcm8zXDqX83gtUDp/WLfPyKCmzZzGf6YgEofIsyrVup8XnD9xNoFmbEeBdFocGWeoIVIn+faOpU22fvQ34L57GHhNQwygZOPKsZa9XNKjayKdKQl3gcAA2wnkZgN0cyIEYvTd3O+Z5Xvff2dat+0sDMK571V+0JEdAMOpQjFO7DkwjKHn/gHLmvRjYLiUOItX9JysFgYuHs8omad2LmeUIkQrBD2I2hyvY49HaJKWctk4Jm0= root@mon'' >> ~/.ssh/authorized_keys && echo ''Root SSH key added''\")",
"Bash(grep -r \"\"\"zod\"\"\" /Users/ogt/awoooi/package.json /Users/ogt/awoooi/apps/*/package.json /Users/ogt/awoooi/packages/*/package.json)",
"Bash(__NEW_LINE_144503b060dfd3dd__ echo:*)",
"Bash(__NEW_LINE_ae2a22b14586d7aa__ echo:*)",
"Bash(__NEW_LINE_e17561a4e55f74d4__ echo:*)",
"Bash(ssh wooo@192.168.0.120 \"echo ''''0936223270'''' | sudo -S cat /etc/rancher/k3s/k3s.yaml 2>/dev/null | sed ''''s|https://127.0.0.1:6443|https://192.168.0.125:6443|g''''\")",
"Bash(KUBECONFIG=/tmp/kubeconfig-vip.yaml kubectl get nodes)",
"Bash(kubectl --kubeconfig=/tmp/kubeconfig-vip.yaml get rs -n awoooi-prod)",
"Bash(kubectl --kubeconfig=/tmp/kubeconfig-vip.yaml get pods -A --no-headers)",
"Bash(kubectl --kubeconfig=/tmp/kubeconfig-vip.yaml get jobs -A --no-headers)",
"Bash(kubectl --kubeconfig=/tmp/kubeconfig-vip.yaml get rs -n awoooi-prod --no-headers)",
"Bash(kubectl --kubeconfig=/tmp/kubeconfig-vip.yaml delete job api-watchdog-29556380 -n wooo-aiops-uat)",
"Bash(kubectl --kubeconfig=/tmp/kubeconfig-vip.yaml get pods -n awoooi-prod)",
"Bash(kubectl --kubeconfig=/tmp/kubeconfig-vip.yaml get pods -A)",
"Bash(kubectl --kubeconfig=/tmp/kubeconfig-vip.yaml get svc -A)",
"Bash(PGPASSWORD=changeme psql -h 192.168.0.188 -U awoooi -d awoooi_prod -f /Users/ogt/awoooi/apps/api/scripts/migrate_phase18_audit_logs.sql)",
"Bash(PLAYWRIGHT_BASE_URL=http://192.168.0.125:32335 npx playwright test phase11-conversational.spec.ts --reporter=list)",
"Bash(PLAYWRIGHT_BASE_URL=http://192.168.0.125:32335 npx playwright test phase11-conversational.spec.ts --reporter=list --workers=1)",
"Bash(KUBECONFIG=~/.kube/config kubectl get nodes --server=https://192.168.0.125:6443 --insecure-skip-tls-verify)",
"Bash(source .venv/bin/activate)",
"Read(//etc/postgresql/14/main/**)",
"Bash(for port:*)",
"Bash(kubectl top:*)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl top pods -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get pods -n awoooi-prod -o wide)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get svc -n awoooi-prod)",
"Bash(jq -r '.components | to_entries[] | \"\"\"\"\\\\\\(.key\\): \\\\\\(.value.status\\)\"\"\"\"')",
"Bash(tar -xzf velero-v1.13.0-darwin-arm64.tar.gz)",
"Bash(sudo mv:*)",
"Bash(velero version:*)",
"Bash(mkdir -p ~/bin)",
"Bash(mv velero-v1.13.0-darwin-arm64/velero ~/bin/)",
"Bash(~/bin/velero version:*)",
"Bash(k8s/velero/00-namespace.yaml:*)",
"Bash(k8s/velero/01-credentials.yaml:*)",
"Bash(k8s/velero/02-velero-install.yaml:*)",
"Bash(tar -xzf velero.tar.gz)",
"Bash(/tmp/velero-credentials:*)",
"Bash(__NEW_LINE_e85d95513fc16492__ ~/bin/velero install --provider aws --plugins velero/velero-plugin-for-aws:v1.9.0 --bucket velero-backups --secret-file /tmp/velero-credentials --backup-location-config region=minio,s3ForcePathStyle=true,s3Url=http://192.168.0.188:9000 --use-volume-snapshots=false --dry-run -o yaml)",
"Bash(__NEW_LINE_e85d95513fc16492__ head:*)",
"Bash(k8s/velero/README.md:*)",
"Bash(KUBECONFIG=/Users/ogt/.kube/config kubectl apply -f /Users/ogt/awoooi/k8s/velero/velero-install-full.yaml)",
"Bash(sshpass -p '09362233270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"whoami && hostname && cat /etc/sudoers.d/* 2>/dev/null | head -5 || echo ''no sudoers.d files''\")",
"Bash(sshpass -p '09362233270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"kubectl get nodes 2>&1 || echo ''kubectl failed, checking k3s kubeconfig...'' && ls -la /etc/rancher/k3s/k3s.yaml 2>&1\")",
"Bash(sshpass -p '09362233270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"sudo -l 2>&1 | head -20\")",
"Bash(sshpass -p '09362233270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''09362233270'' | sudo -S -l 2>&1\")",
"Bash(sshpass -p '09362233270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get nodes 2>&1\")",
"Bash(sshpass -p '0936223270' scp /Users/ogt/awoooi/k8s/velero/velero-install-full.yaml wooo@192.168.0.120:/tmp/velero-install-full.yaml)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''''0936223270'''' | sudo -S kubectl apply -f /tmp/velero-install-full.yaml 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get pods -n velero 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get backupstoragelocation -n velero 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl logs -n velero deploy/velero --tail=30 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl logs -n velero deploy/velero --tail=10 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get secret cloud-credentials -n velero -o jsonpath=''{.data.cloud}'' 2>&1 | base64 -d\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S curl -s http://192.168.0.188:9000/velero-backups/ 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl rollout restart deployment/velero -n velero 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get backups -n velero 2>&1\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl describe backup test-backup-20260328-2114 -n velero 2>&1 | tail -30\")",
"Bash(sshpass -p:*)",
"Read(//Users/ogt/awoooi/=== 測試 /approvals/**)",
"Bash(kubectl --kubeconfig=/Users/ogt/.kube/config get svc -n velero -o wide)",
"Bash(kubectl --kubeconfig=/Users/ogt/.kube/config get pods -n velero -o wide)",
"Bash(KUBECONFIG=/Users/ogt/.kube/config kubectl get svc -n velero)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'echo \"\"0936223270\"\" | sudo -S sh -c \"\"kubectl get pods -A | grep -E \\\\\"\"kube-state|state-metrics\\\\\"\"\"\"')",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'echo \"\"0936223270\"\" | sudo -S sh -c \"\"kubectl get ns | grep -E \\\\\"\"wooo|aiops|legacy|old\\\\\"\"\"\"')",
"Bash(KUBECONFIG=~/.kube/config kubectl get ns --no-headers)",
"WebFetch(domain:build.nvidia.com)",
"WebFetch(domain:ollama.com)",
"WebFetch(domain:docs.api.nvidia.com)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"curl -s ''http://admin:admin@localhost:3002/api/search?type=dash-db'' | python3 -c \"\"import sys,json; d=json.load\\(sys.stdin\\); print\\(f''Dashboard 數量: {len\\(d\\)}''\\); [print\\(f\\\\\"\" - {i[''title'']}\\\\\"\"\\) for i in d[:10]]\"\"\")",
"Bash(jq '.ai_provider // .data.ai_provider // \"\"\"\"not found\"\"\"\"')",
"Bash(KUBECONFIG=~/.kube/config kubectl logs -n awoooi-prod deployment/awoooi-api --tail=50)",
"Bash(export NVIDIA_API_KEY=\"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\")",
"Bash(curl -s -X POST \"https://integrate.api.nvidia.com/v1/chat/completions\" -H \"Content-Type: application/json\" -H \"Authorization: Bearer $NVIDIA_API_KEY\" -d '{:*)",
"Bash(/tmp/fix-network-policy.yaml:*)",
"Bash(__NEW_LINE_acde7a92ceae01f6__ scp:*)",
"Bash(curl -s -X POST https://awoooi.wooo.work/api/v1/webhooks/alertmanager -H 'Content-Type: application/json' -d '{:*)",
"Bash(ssh ollama@192.168.0.188 'curl -s \"\"http://localhost:9090/api/v1/targets\"\" 2>/dev/null | grep -o \"\"\\\\\"\"health\\\\\"\":\\\\\"\"[^\\\\\"\"]*\\\\\"\"\"\" | sort | uniq -c')",
"Bash(ssh ollama@192.168.0.188 'curl -s \"\"http://localhost:9090/api/v1/rules\"\" 2>/dev/null | grep -o \"\"\\\\\"\"name\\\\\"\":\\\\\"\"[^\\\\\"\"]*\\\\\"\"\"\" | sort | uniq')",
"Bash(ssh ollama@192.168.0.188 'curl -s \"\"http://localhost:9090/api/v1/targets\"\" 2>/dev/null | grep -o \"\"\\\\\"\"job\\\\\"\":\\\\\"\"[^\\\\\"\"]*\\\\\"\"\"\" | sort | uniq -c | sort -rn')",
"Bash(ssh ollama@192.168.0.188 'curl -s \"\"http://localhost:9090/api/v1/query?query=up\"\" 2>/dev/null | grep -o \"\"\\\\\"\"instance\\\\\"\":\\\\\"\"[^\\\\\"\"]*\\\\\"\"\"\" | sort | uniq')",
"Bash(for i:*)",
"Bash(do sleep:*)",
"Bash(kubectl patch:*)",
"Bash(ssh wooo@192.168.0.110 \"cat /tmp/runner_clean.log 2>/dev/null; echo ''---''; ps aux | grep ''Runner.Listener'' | grep -v grep | wc -l\")",
"Bash(KUBECONFIG=~/.kube/config kubectl logs -n awoooi-prod -l app=awoooi-api --tail=200)",
"Bash(/Users/ogt/awoooi/ops/monitoring/deploy-exporters.sh:*)",
"WebFetch(domain:github.com)",
"WebFetch(domain:docs.ollama.com)",
"Skill(telegram:configure)",
"Skill(telegram:configure:*)",
"Bash(USE_NEW_ENGINE=true pytest tests/test_incident*.py -v --tb=short -x)",
"Bash(USE_NEW_ENGINE=true pytest tests/test_approval_field_alignment.py tests/test_learning_service.py -v --tb=short)",
"Bash(/tmp/debug_approval.py:*)",
"Bash(/tmp/debug_approval2.py:*)",
"Bash(/tmp/bulk_sign.sh:*)",
"Bash(bash /tmp/bulk_sign.sh)",
"Bash(/tmp/check_deploy.py:*)",
"Bash(/tmp/check_buttons.py:*)",
"Bash(ssh ollama@192.168.0.188 \"docker logs openclaw --since=10s 2>&1 | grep -Ev ''\\(GET|POST\\) /health'' | tail -10 && echo ''---'' && docker exec openclaw env | grep OPENAI_API_KEY | cut -c1-30\")",
"Read(//Users/ogt/awoooi/https:/awoooi.wooo.work/_next/static/chunks/app/%5Blocale%5D/**)",
"Bash(find /Users/ogt/awoooi/apps/web -type f \\\\\\(-name *.spec.ts -o -name *.spec.tsx \\\\\\))",
"Bash(kubectl -n awoooi-prod get pods)",
"Bash(kubectl -n production get pods)",
"Bash(ssh -o StrictHostKeyChecking=no wooo@192.168.0.121 \"export KUBECONFIG=/etc/rancher/k3s/k3s.yaml && sudo kubectl get deployment awoooi-web -n awoooi-prod -o jsonpath=''{.spec.template.spec.containers[0].image}'' && echo '''' && sudo kubectl get pods -n awoooi-prod -l app=awoooi-web --no-headers\")",
"Bash(KUBECONFIG=/Users/ogt/.kube/config kubectl get pods -n awoooi-prod)",
"Bash(for run_id in 166 165)",
"mcp__plugin_playwright_playwright__browser_navigate",
"mcp__plugin_playwright_playwright__browser_take_screenshot",
"Bash(open \"http://192.168.0.110:3001/wooo/awoooi/actions\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runs?limit=5\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runs/166/jobs\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runs?limit=10\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runners\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/admin/runners\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runs?limit=3\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runs/169/jobs\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/jobs/179/logs\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" JOB_ID=180 curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/jobs/$JOB_ID/logs\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runs?limit=2\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" JOB_ID=181 curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/jobs/$JOB_ID/logs\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runs/172/jobs\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/jobs/182/logs\" -H \"Authorization: token $TOKEN\")",
"Bash(TOKEN=\"REDACTED_GITEA_TOKEN\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runs/178\" -H \"Authorization: token $TOKEN\")",
"mcp__plugin_playwright_playwright__browser_snapshot",
"mcp__plugin_playwright_playwright__browser_fill_form",
"mcp__plugin_playwright_playwright__browser_click",
"Bash(GITEA_TOKEN=\"e6c9fecb1f0148939493ae0fa30407d28c91279d\" curl -s \"http://192.168.0.110:3001/api/v1/repos/wooo/awoooi/actions/runs?limit=5\" -H \"Authorization: token $GITEA_TOKEN\")",
<<<<<<< Updated upstream
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 /tmp/a4_smoke.py)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -c \"from src.repositories.aider_event_repository import AiderEventRepository; print\\('import OK'\\)\")",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_service.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_service.py -v --tb=short)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -c \"from src.services.aider_event_service import classify_severity, should_create_incident, build_signal_data; print\\('✓ All imports successful'\\)\")",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_service.py::test_build_signal_data_redacts_secrets_in_annotations -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_events_api.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_service.py tests/test_aider_events_api.py tests/test_aider_event_models.py tests/test_secret_redactor.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_processor.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_processor.py tests/test_aider_event_service.py tests/test_aider_events_api.py tests/test_aider_event_models.py tests/test_secret_redactor.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -c \"from src.workers.aider_event_processor import AiderEventProcessor, get_aider_event_processor, run_aider_event_processor_loop; print\\('✓ All imports successful'\\)\")",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_processor.py -v --tb=short)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_processor.py tests/test_aider_event_service.py tests/test_aider_events_api.py tests/test_aider_event_models.py tests/test_secret_redactor.py --tb=short)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_ai_router_feedback.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_service.py tests/test_aider_events_api.py tests/test_aider_event_models.py tests/test_secret_redactor.py tests/test_aider_event_processor.py tests/test_ai_router_feedback.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -c \"from src.services.ai_router import AIRouter; from src.db.base import get_session_factory; print\\('✓ Imports successful, no circular imports'\\)\")",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_ai_router_feedback.py tests/test_aider_event_service.py -v --tb=short)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -c \"from src.api.v1 import aider_events; from src.workers.aider_event_processor import run_aider_event_processor_loop; from src.core.config import settings; print\\('AIDER_WEBHOOK_SECRET' in settings.__fields__, 'USE_AIDER_FEEDBACK' in settings.__fields__\\)\")",
"Bash(AIDER_WEBHOOK_SECRET=testsecret /Users/ogt/.pyenv/versions/3.11.7/bin/python3 -c \"from src.main import app; print\\('app OK; title:', app.title\\)\")",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_action_parsing.py tests/test_aider_event_service.py tests/test_aider_events_api.py tests/test_aider_event_models.py tests/test_secret_redactor.py tests/test_aider_event_processor.py tests/test_ai_router_feedback.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_action_parsing.py tests/test_aider_event_service.py tests/test_aider_events_api.py tests/test_aider_event_models.py tests/test_secret_redactor.py tests/test_aider_event_processor.py tests/test_ai_router_feedback.py -q)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pip install -e .[dev] --quiet)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pip install -e '.[dev]' --quiet)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/ -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -c \"from aider_watch_client.aiderw import main as awmain; from aider_watch_client.cli import main as climain; print\\('✓ imports ok'\\)\")",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pip show aider-watch-client)",
"Bash(tailscale status *)",
"Bash(kubectl rollout *)",
"Bash(bash /Users/ogt/awoooi/scripts/aider_watch_client/scripts/install.sh)",
"Bash(git rebase *)",
"Bash(/opt/homebrew/bin/aiderw --message \"add docstring to hello function\" --exit)",
"Bash(kubectl -n awoooi-prod get pod -l app=awoooi-api -o jsonpath='{.items[0].metadata.name}')",
"Bash(kubectl -n awoooi-prod exec awoooi-api-7b9464c969-8ml88 -- python -c ' *)",
"Bash(kubectl -n awoooi-prod rollout restart deployment/awoooi-api)",
"Bash(kubectl -n awoooi-prod get pod -l app=awoooi-api --no-headers)",
"Bash(kubectl -n awoooi-prod rollout status deployment/awoooi-api --timeout=120s)",
"Bash(/opt/homebrew/bin/aider-watch flush *)",
"Bash(kubectl -n awoooi-prod get pod -l app=awoooi-api -o wide)",
"Bash(kubectl -n awoooi-prod rollout status deployment/awoooi-api --timeout=30s)",
"Bash(kubectl -n awoooi-prod exec awoooi-api-6657fb9cf7-47lcg -- python -c \"import src.services.telegram_gateway as tg; import inspect; lines = inspect.getsource\\(tg\\); idx = lines.find\\('response_body=e.response.text'\\); print\\('FOUND' if idx >= 0 else 'NOT FOUND'\\)\")",
"Read(//opt/gitea/**)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/ -q)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/unit/test_aider_event_service.py tests/unit/test_aider_model.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_events_api.py tests/test_aider_event_models.py tests/test_aider_event_service.py tests/test_aider_event_processor.py -v)",
"Bash(kubectl -n awoooi-prod get svc)",
"Bash(kubectl -n openclaw get pod)",
"Bash(kubectl -n awoooi-prod exec awoooi-api-7cd784c875-r4qkz -- python -c ' *)",
"Bash(kubectl -n awoooi-prod logs awoooi-api-7cd784c875-qt6j2 --since=10m)",
"Bash(kubectl -n awoooi-prod logs awoooi-api-7cd784c875-qt6j2 --since=15m)",
"Bash(kubectl -n awoooi-prod logs awoooi-api-7cd784c875-qt6j2 --since=20m)",
"Bash(kubectl -n awoooi-prod get secret awoooi-secrets -o yaml)",
"Bash(kubectl -n awoooi-prod logs awoooi-api-7cd784c875-qt6j2 --since=30m)",
"Bash(kubectl -n awoooi-prod logs awoooi-api-7cd784c875-qt6j2 --since=2h)",
"Bash(kubectl -n awoooi-prod logs awoooi-api-7cd784c875-qt6j2)",
"Bash(kubectl -n awoooi-prod get pod -l app=awoooi-api -o jsonpath='{range .items[*]}{.metadata.name} {.status.containerStatuses[0].imageID}{\"\\\\n\"}{end}')",
"Bash(kubectl -n awoooi-prod get ingress)",
"Bash(kubectl -n awoooi-prod get svc awoooi-api-svc)",
"Bash(kubectl -n awoooi-prod logs -l app=awoooi-api --since=60s --prefix)",
"Bash(kubectl -n awoooi-prod logs -l app=awoooi-api --since=5m --prefix)",
"Bash(kubectl -n awoooi-prod logs pod/awoooi-api-86bc79766d-dn5ll --since=5m)",
"Bash(kubectl -n awoooi-prod logs pod/awoooi-api-86bc79766d-dn5ll --since=10m)",
"Bash(kubectl -n awoooi-prod logs pod/awoooi-api-86bc79766d-dn5ll)",
"Bash(kubectl -n awoooi-prod logs -l app=awoooi-api --since=90s --prefix)",
"Bash(kubectl -n awoooi-prod logs pod/awoooi-api-86bc79766d-4x69p --since=5m)",
"Bash(redis-cli -h 192.168.0.188 -p 6380 -n 10 SCAN 0 MATCH \"playbook:PB-*\" COUNT 500)",
"Bash(redis-cli -h 192.168.0.188 -p 6380 -n 10 DBSIZE)",
"Bash(wait)",
"Read(//Users/**)",
"Read(//Users/ooo/.claude/**)",
"Bash(mkdir -p /Users/ogt/awoooi/.claude/agents)",
"Bash(cp /Users/ogt/.claude/agents/*.md /Users/ogt/awoooi/.claude/agents/)",
"Bash(kubectl -n awoooi-prod logs --tail=400 -l app=awoooi-api --prefix=true)",
"Bash(kubectl -n awoooi-prod logs --tail=300 awoooi-api-65c69fd649-bxbwp)",
"Bash(kubectl -n awoooi-prod logs --tail=20000 -l app=awoooi-api --prefix=false --since=24h)",
"Bash(kubectl -n awoooi-prod logs --since=24h awoooi-api-65c69fd649-bxbwp)",
"Bash(kubectl -n awoooi-prod logs --since=24h -l app=awoooi-api --prefix=false)",
"Bash(kubectl -n awoooi-prod logs --since=24h awoooi-api-65c69fd649-fmbxd)",
"Bash(kubectl -n awoooi-prod logs --since=3h awoooi-api-65c69fd649-fmbxd)",
"Bash(kubectl -n awoooi-prod logs --since=3h awoooi-api-65c69fd649-bxbwp)",
"Bash(kubectl -n awoooi-prod logs -l app=awoooi-api --tail=30 --since=30m)",
"Bash(kubectl -n awoooi-prod get pods -o wide)",
"Bash(kubectl -n awoooi-prod get pods -l app=awoooi-api -o jsonpath='{.items[0].metadata.creationTimestamp}')",
"Bash(kubectl -n awoooi-prod logs -l app=awoooi-api --tail=5 --since=5m)",
"Bash(kubectl -n awoooi-prod describe pod -l app=awoooi-api)",
"Bash(kubectl -n awoooi-prod logs -l app=awoooi-api --tail=20 --since=10m)",
"Bash(kubectl -n awoooi-prod exec deployment/awoooi-api -- python3 -c ' *)",
"Bash(PGPASSWORD=\"\" psql -h 188.188.188.188 -U aiops -d aiops -c \"\\\\d timeline_events\")",
"Bash(kubectl -n awoooi-prod get deploy awoooi-api -o yaml)",
"Bash(PGPASSWORD=\"\" psql --version)",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- env)",
"Bash(kubectl -n awoooi-prod logs --tail=500 deploy/awoooi-api)",
"Bash(kubectl cp *)",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- sh -c 'curl -sG \"$PROMETHEUS_URL/api/v1/query\" --data-urlencode \"query=up\" 2>&1 | head -c 400')",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- sh -c 'for q in \"sum\\(rate\\(http_requests_total{status=~\\\\\"5..\\\\\"}[5m]\\)\\) / sum\\(rate\\(http_requests_total[5m]\\)\\)\" \"avg\\(rate\\(container_cpu_usage_seconds_total{namespace=\\\\\"awoooi-prod\\\\\",container=\\\\\"awoooi-api\\\\\"}[5m]\\)\\)\" \"pg_stat_activity_count{datname=\\\\\"awoooi\\\\\"}\" \"increase\\(kube_pod_container_status_restarts_total{namespace=\\\\\"awoooi-prod\\\\\"}[15m]\\)\"; do echo \"---- $q\"; curl -sG \"$PROMETHEUS_URL/api/v1/query\" --data-urlencode \"query=$q\" 2>&1 | head -c 250; echo; done')",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- sh -c 'PGPASSWORD=as0V1mohktaFbGIx3R0iCatbMJ6XxFDL psql -h 192.168.0.188 -U awoooi -d awoooi_prod -c \"SELECT metric_name, count\\(*\\), max\\(trained_at\\) FROM dynamic_baseline_record GROUP BY metric_name;\" 2>&1 | head -20')",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- sh -c 'PGPASSWORD=as0V1mohktaFbGIx3R0iCatbMJ6XxFDL psql -h 192.168.0.188 -U awoooi -d awoooi_prod -c \"SELECT count\\(*\\) as asset_count FROM asset_inventory; SELECT count\\(*\\) as coverage_count FROM asset_coverage_snapshot; SELECT count\\(*\\) as host_cap_count FROM host_capacity_snapshot; SELECT count\\(*\\) as compl_count FROM asset_compliance_snapshot; SELECT count\\(*\\) as rule_cat FROM alert_rule_catalog; SELECT count\\(*\\) as log_cluster FROM log_cluster_record;\" 2>&1')",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- sh -c 'python3 -c \" *)",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- python3 -c ' *)",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- sh -c 'for q in \"http_requests_total\" \"container_cpu_usage_seconds_total\" \"container_memory_usage_bytes\" \"kube_pod_container_status_restarts_total\" \"pg_stat_activity_count\" \"node_cpu_seconds_total\" \"node_load1\"; do echo -n \"$q => \"; curl -sG \"$PROMETHEUS_URL/api/v1/query\" --data-urlencode \"query=count\\($q\\)\" 2>&1 | head -c 180; echo; done')",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- sh -c 'curl -sG \"$PROMETHEUS_URL/api/v1/query\" --data-urlencode \"query=container_cpu_usage_seconds_total\" 2>&1 | python3 -c \"import json,sys; d=json.load\\(sys.stdin\\); rs=d[\\\\\"data\\\\\"][\\\\\"result\\\\\"][:3]; [print\\(r[\\\\\"metric\\\\\"]\\) for r in rs]; print\\(\\\\\"total series:\\\\\", len\\(d[\\\\\"data\\\\\"][\\\\\"result\\\\\"]\\)\\)\"')",
"Bash(kubectl -n awoooi-prod exec deploy/awoooi-api -- sh -c 'which kubectl 2>&1; kubectl version --client 2>&1 | head -3; kubectl -n awoooi-prod get deploy awoooi-api 2>&1 | head -3')",
"Bash(kubectl -n awoooi-prod logs --tail=2000 deploy/awoooi-api)",
"Bash(psql --version)",
"WebFetch(domain:core.telegram.org)",
"mcp__plugin_context7_context7__resolve-library-id",
"mcp__plugin_context7_context7__query-docs",
"WebFetch(domain:docs.claude.com)",
"Bash(git tag *)",
"Read(//usr/**)",
"Bash(psql -h 192.168.0.110 -U awoooi_user -d awoooi -c \"SELECT id, alertname, status, confidence, description, created_at FROM approval_records WHERE status='PENDING' AND DATE\\(created_at AT TIME ZONE 'Asia/Taipei'\\) = CURRENT_DATE AT TIME ZONE 'Asia/Taipei' ORDER BY created_at DESC LIMIT 10;\")",
"Bash(kubectl -n awoooi-prod get deployment awoooi-api -o jsonpath='{.spec.template.spec.containers[0].image}')",
"Bash(kubectl -n awoooi-prod get deployment awoooi-api -o jsonpath='{.spec.template.spec.containers[0].imagePullPolicy}{\"\\\\n\"}{.spec.template.metadata.labels}{\"\\\\n\"}')",
"Bash(kubectl kustomize *)",
"Bash(kubectl -n awoooi-prod rollout status deployment/awoooi-api --timeout=60s)",
"Bash(kubectl -n awoooi-prod get pods -l app=awoooi-api --no-headers)",
"Bash(kubectl -n awoooi-prod patch deployment awoooi-api -p '{\"spec\":{\"template\":{\"spec\":{\"containers\":[{\"name\":\"api\",\"image\":\"192.168.0.110:5000/awoooi/api:cbd28e29a08435deb8c66af51654d8fa65120a14\"}]}}}}')",
"Bash(kubectl -n awoooi-prod get deployment awoooi-api -o jsonpath='{.spec.template.spec.containers[0].image}{\"\\\\n\"}')",
"Bash(kubectl -n awoooi-prod get pods -l app=awoooi-api -o jsonpath='{range .items[*]}{.metadata.name}{\"\\\\t\"}{.spec.containers[0].image}{\"\\\\n\"}{end}')",
"Bash(kubectl -n awoooi-prod get pdb awoooi-api-pdb -o jsonpath='{.spec.minAvailable}')",
"Bash(kubectl -n awoooi-prod get pods -l app=awoooi-api -o wide)",
"Bash(kubectl -n awoooi-prod describe rs -l app=awoooi-api)",
"Bash(kubectl -n awoooi-prod get events --sort-by='.lastTimestamp')",
"Bash(kubectl -n awoooi-prod get deployment awoooi-api -o jsonpath='{.spec.replicas}{\"\\\\n\"}{.status.replicas}{\"\\\\n\"}{.status.readyReplicas}{\"\\\\n\"}{.status.updatedReplicas}{\"\\\\n\"}')",
"Bash(kubectl -n awoooi-prod get pods -l app=awoooi-api --sort-by=.metadata.creationTimestamp -o jsonpath='{range .items[*]}{.metadata.name}{\":\"}{.metadata.creationTimestamp}{\"\\\\n\"}{end}')",
"Bash(kubectl -n awoooi-prod get deployment awoooi-api -o jsonpath='{.status.conditions[*]}')",
"Bash(kubectl -n awoooi-prod describe deployment awoooi-api)",
"Bash(kubectl -n awoooi-prod get rs -l app=awoooi-api -o jsonpath='{range .items[*]}{.metadata.name}{\":\"}{.spec.template.spec.containers[0].image}{\"\\\\n\"}{end}')",
"Bash(kubectl -n awoooi-prod get deployment awoooi-api -o yaml)",
"Bash(kubectl -n awoooi-prod rollout status deployment/awoooi-api --timeout=180s)",
"Bash(kubectl -n awoooi-prod set image deployment/awoooi-api api=192.168.0.110:5000/awoooi/api:cbd28e29a08435deb8c66af51654d8fa65120a14 --record=false)",
"Bash(kubectl -n awoooi-prod get pods -l app=awoooi-api -o jsonpath='{range .items[*]}{.metadata.name}{\"\\\\t\"}{.spec.containers[0].image}{\"\\\\t\"}{.status.phase}{\"\\\\n\"}{end}')",
"Bash(kubectl -n awoooi-prod get deployment awoooi-api -o jsonpath='{.status.replicas}{\"\\\\t\"}{.status.readyReplicas}{\"\\\\t\"}{.status.updatedReplicas}')",
"Bash(bash /tmp/diagnostic.sh)",
"WebFetch(domain:docs.github.com)",
"WebFetch(domain:docs.sonarsource.com)",
"WebFetch(domain:gitea.com)",
"WebFetch(domain:docs.gitea.com)",
"WebFetch(domain:www.sonarsource.com)",
"WebFetch(domain:golangci-lint.run)",
"WebFetch(domain:www.uber.com)",
"Bash(bash scripts/ops/deploy-alerts.sh --dry-run)",
"Bash(bash scripts/ops/deploy-alerts.sh)",
"Bash(promtool check *)",
"WebFetch(domain:openrouter.ai)",
"WebFetch(domain:qwenlm.github.io)",
"WebFetch(domain:aclanthology.org)",
"WebFetch(domain:datanorth.ai)",
"WebFetch(domain:www.infoq.com)",
"WebFetch(domain:aws.amazon.com)",
"WebFetch(domain:artificialanalysis.ai)",
"WebFetch(domain:www.alibabacloud.com)",
"WebFetch(domain:docs.langchain.com)",
"WebFetch(domain:arxiv.org)",
"WebFetch(domain:blog.kilo.ai)",
"WebFetch(domain:www.siliconflow.com)",
"WebFetch(domain:aicompetence.org)",
"Bash(redis-cli -h 192.168.0.188 -p 6380 ping)",
"Bash(redis-cli ping *)"
=======
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest apps/api/tests/test_aider_event_models.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_action_parsing.py -v --collect-only)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_action_parsing.py --collect-only)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -m pytest tests/test_aider_event_models.py tests/test_secret_redactor.py -v)",
"Bash(/Users/ogt/.pyenv/versions/3.11.7/bin/python3 -c \"from src.repositories.aider_event_repository import AiderEventRepository; print\\('import OK'\\)\")"
>>>>>>> Stashed changes
],
"deny": [
"Bash(rm -rf *)",
"Bash(git push --force *)",
"Bash(git reset --hard *)",
"Bash(kubectl delete *)",
"Bash(docker rm -f *)"
],
"additionalDirectories": [
"/Users/ogt/.claude/projects/-Users-ogt-awoooi/memory",
"/Users/ogt/awoooi/.claude/hooks",
"/Users/ogt/.claude/channels/telegram",
<<<<<<< Updated upstream
"/Users/ogt",
"/Users/ogt/.claude",
"/Users/ogt/awoooi/apps/web/src/app/[locale]/aiops"
]
},
"hooks": {
"PreToolUse": [
{
"matcher": "",
"hooks": [
{
"type": "command",
"command": "node $CLAUDE_PROJECT_DIR/.claude/hooks/awoooi-guard.js 2>/dev/null || true"
},
{
"type": "command",
"command": "node /Users/ogt/.claude/hooks/branch-protection.js"
},
{
"type": "command",
"command": "node /Users/ogt/.claude/hooks/commit-quality.js"
},
{
"type": "command",
"command": "node /Users/ogt/.claude/hooks/large-file-warner.js"
},
{
"type": "command",
"command": "node /Users/ogt/.claude/hooks/mcp-health.js"
}
]
}
],
"PostToolUse": [
{
"matcher": "",
"hooks": [
{
"type": "command",
"command": "node /Users/ogt/.claude/hooks/audit-log.js"
},
{
"type": "command",
"command": "node /Users/ogt/.claude/hooks/suggest-compact.js"
}
]
}
],
"Stop": [
{
"matcher": "",
"hooks": [
{
"type": "command",
"command": "node /Users/ogt/.claude/hooks/cost-tracker.js"
},
{
"type": "command",
"command": "node /Users/ogt/.claude/hooks/session-summary.js"
}
]
}
=======
"/Users/ogt/aider-watch"
>>>>>>> Stashed changes
]
}
}

View File

@@ -1,827 +0,0 @@
{
"permissions": {
"allow": [
"Bash(pnpm install:*)",
"Bash(npm --version)",
"Bash(npm install:*)",
"Bash(pnpm --version)",
"Bash(pnpm dev:*)",
"Bash(pnpm add:*)",
"Bash(ls -la /Users/ogt/awoooi/apps/web/next.config.*)",
"Bash(pkill -f \"next dev\")",
"Bash(curl -sL http://localhost:3000/zh-TW)",
"Bash(curl -s http://localhost:3000/zh-TW)",
"Bash(pnpm --filter web build)",
"Bash(curl -s http://localhost:3001/zh-TW)",
"Bash(curl -s -o /dev/null -w \"%{http_code}\" http://localhost:3000/zh-TW)",
"Bash(kubectl apply:*)",
"Bash(chmod +x /Users/ogt/awoooi/deploy-infra.sh)",
"Bash(./deploy-infra.sh)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"mkdir -p /tmp/awoooi-k8s\")",
"Bash(sshpass -p '0936223270' scp -o StrictHostKeyChecking=no /Users/ogt/awoooi/k8s/awoooi-prod/01-namespace-quota.yaml /Users/ogt/awoooi/k8s/awoooi-prod/02-network-policy.yaml /Users/ogt/awoooi/k8s/awoooi-prod/04-configmap.yaml wooo@192.168.0.120:/tmp/awoooi-k8s/)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"sudo kubectl apply -f /tmp/awoooi-k8s/01-namespace-quota.yaml\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl apply -f /tmp/awoooi-k8s/01-namespace-quota.yaml 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl apply -f /tmp/awoooi-k8s/02-network-policy.yaml 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl apply -f /tmp/awoooi-k8s/04-configmap.yaml 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get ns awoooi-prod -o wide 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get networkpolicy -n awoooi-prod 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get resourcequota,limitrange,configmap -n awoooi-prod 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"rm -rf /tmp/awoooi-k8s\")",
"Bash(PYTHONPATH=. python -c \"from src.main import app; print\\(''Import OK''\\)\")",
"Bash(curl -s http://localhost:8000/api/v1/health/ready)",
"Bash(curl -s http://localhost:8000/api/v1/health/live)",
"Bash(curl -s http://localhost:8000/)",
"Bash(pkill -f \"uvicorn src.main:app\")",
"Bash(pkill -f \"node.*next\")",
"Bash(curl -s http://localhost:8000/api/v1/health)",
"Read(//Users/ogt/awoooi/apps/api/**)",
"Bash(pnpm typecheck:*)",
"Read(//Users/ogt/awoooi/apps/web/**)",
"Bash(curl -s -X POST http://localhost:8000/api/v1/dashboard/demo/spike/clear)",
"Read(//Users/ogt/awoooi/=== 驗證英文頁面 \\(/en/**)",
"Bash(jq \".devDependencies | keys | map\\(select\\(startswith\\(\"\"@playwright\"\"\\) or startswith\\(\"\"playwright\"\"\\)\\)\\)\")",
"Bash(npx playwright:*)",
"Bash(curl -s http://localhost:3000/zh-TW/demo -o /dev/null -w \"Frontend: HTTP %{http_code}\\\\n\")",
"Bash(__NEW_LINE_ef548029029cdfac__ echo:*)",
"Bash(curl -s http://localhost:8000/api/v1/health -o /dev/null -w \"Backend: HTTP %{http_code}\\\\n\")",
"Bash(echo '=== 已產出的截圖 ===' find /Users/ogt/awoooi/apps/web/test-results -name *.png)",
"Bash(echo '=== Playwright E2E 測試結果 ===' echo echo '📸 截圖證據 \\(test-results/screenshots/\\):' ls -la /Users/ogt/awoooi/apps/web/test-results/screenshots/ __NEW_LINE_db74e5f56e34db17__ echo echo '🎬 錄影證據 \\(.webm\\):' find /Users/ogt/awoooi/apps/web/test-results -name *.webm -exec ls -la {})",
"Bash(__NEW_LINE_db74e5f56e34db17__ echo:*)",
"Bash(source .venv/bin/activate)",
"Bash(python scripts/demo_multisig.py)",
"Bash(python -c \"from src.api.v1.approvals import router; print\\(''✅ Approvals router loaded:'', len\\(router.routes\\), ''routes''\\)\")",
"Bash(npx tsc:*)",
"Bash(chmod +x /Users/ogt/awoooi/scripts/demo-multisig-flow.sh)",
"Bash(python -c \"from src.main import app; print\\(''✅ API loads successfully''\\)\")",
"Bash(jq)",
"Bash(/Users/ogt/awoooi/scripts/demo-multisig-flow.sh)",
"Bash(curl -s -X POST \"http://localhost:8000/api/v1/approvals\" -H \"Content-Type: application/json\" -d '{:*)",
"Bash(curl -s http://localhost:8000/api/v1/openapi.json)",
"Bash(python -c \":*)",
"Bash(curl -s http://localhost:3000 -o /dev/null -w \"%{http_code}\")",
"Bash(lsof -ti:3000,3001,8000)",
"Bash(curl -s http://localhost:8000/health)",
"Bash(curl -s http://localhost:8000/api/v1/approvals/pending)",
"Bash(curl -s -o /dev/null -w \"%{http_code}\" http://localhost:3001/zh-TW/demo)",
"Bash(ls -la test-results/*.png)",
"Bash(cp test-results/cpo102-*.png /Users/ogt/awoooi/docs/screenshots/)",
"Bash(ssh ogt@192.168.0.120 'cat /etc/rancher/k3s/k3s.yaml')",
"Bash(python -c \"from src.main import app; print\\(''✅ main.py imports OK''\\)\")",
"Bash(curl -s http://localhost:8000/api/v1/approvals/k8s-test)",
"Bash(sqlite3 awoooi.db \".tables\")",
"Bash(sshpass -p 0936223270 ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 'sudo cat /etc/rancher/k3s/k3s.yaml')",
"Bash(kubectl --kubeconfig=/Users/ogt/awoooi/apps/api/k3s-prod.yaml get deployments -n awoooi-prod)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get deployments -n awoooi-prod 2>/dev/null\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get deployments -A 2>/dev/null\")",
"Bash(curl -s -X POST http://localhost:8000/api/v1/approvals -H \"Content-Type: application/json\" -d '{:*)",
"Bash(APPROVAL_ID=\"b58a0d86-fa4e-43ca-881c-02e978cd7943\")",
"Bash(curl -s -X POST \"http://localhost:8000/api/v1/approvals/$APPROVAL_ID/sign\" -H \"Content-Type: application/json\" -d '{:*)",
"Bash(sqlite3 /Users/ogt/awoooi/apps/api/awoooi.db \"SELECT operation_type, target_resource, namespace, success, dry_run_passed, dry_run_message, error_message, execution_duration_ms, created_at FROM audit_logs ORDER BY created_at DESC LIMIT 1;\" -header -column)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S kubectl get pods -n monitoring -l app=grafana 2>/dev/null\")",
"Bash(curl -s http://192.168.0.188:11434/api/tags)",
"Bash(python -c \"from src.main import app; print\\(''✅ Compile OK''\\)\")",
"Bash(curl -s http://localhost:8000/api/v1/ai/status)",
"Bash(curl -s -X POST http://localhost:8000/api/v1/ai/analyze-and-propose -H \"Content-Type: application/json\" -d '{}')",
"Bash(curl -s -X POST http://192.168.0.188:11434/api/generate -H \"Content-Type: application/json\" -d '{\"\"\"\"model\"\"\"\":\"\"\"\"llama3.2:1b\"\"\"\",\"\"\"\"prompt\"\"\"\":\"\"\"\"Output only JSON: {\\\\\"\"\"\"action\\\\\"\"\"\":\\\\\"\"\"\"test\\\\\"\"\"\"}\"\"\"\",\"\"\"\"stream\"\"\"\":false,\"\"\"\"format\"\"\"\":\"\"\"\"json\"\"\"\"}' --max-time 30)",
"Bash(curl -s -X POST http://localhost:8000/api/v1/ai/analyze-and-propose -H \"Content-Type: application/json\" -d '{}' --max-time 60)",
"Bash(PROMPT='你是 ClawBot AI。分析以下監控數據輸出純 JSON無其他文字。:*)",
"Bash(curl -s -X POST http://192.168.0.188:11434/api/generate -H \"Content-Type: application/json\" -d \"{\"\"model\"\":\"\"llama3.2:1b\"\",\"\"prompt\"\":\"\"$PROMPT\"\",\"\"stream\"\":false,\"\"format\"\":\"\"json\"\",\"\"options\"\":{\"\"num_predict\"\":256,\"\"temperature\"\":0.1}}\" --max-time 60)",
"Bash(curl -s -X POST http://192.168.0.188:11434/api/generate -H \"Content-Type: application/json\" -d '{\"\"\"\"model\"\"\"\":\"\"\"\"llama3.2:1b\"\"\"\",\"\"\"\"prompt\"\"\"\":\"\"\"\"Harbor service returning 404. Output JSON: {\\\\\"\"\"\"suggested_action\\\\\"\"\"\":\\\\\"\"\"\"RESTART_DEPLOYMENT\\\\\"\"\"\",\\\\\"\"\"\"target_resource\\\\\"\"\"\":\\\\\"\"\"\"harbor\\\\\"\"\"\",\\\\\"\"\"\"namespace\\\\\"\"\"\":\\\\\"\"\"\"default\\\\\"\"\"\",\\\\\"\"\"\"risk_level\\\\\"\"\"\":\\\\\"\"\"\"medium\\\\\"\"\"\",\\\\\"\"\"\"reasoning\\\\\"\"\"\":\\\\\"\"\"\"Service down\\\\\"\"\"\",\\\\\"\"\"\"confidence\\\\\"\"\"\":0.8,\\\\\"\"\"\"affected_services\\\\\"\"\"\":[]}\"\"\"\",\"\"\"\"stream\"\"\"\":false,\"\"\"\"format\"\"\"\":\"\"\"\"json\"\"\"\",\"\"\"\"options\"\"\"\":{\"\"\"\"num_predict\"\"\"\":128,\"\"\"\"temperature\"\"\"\":0.1}}' --max-time 30)",
"Bash(curl -v -X POST http://192.168.0.188:11434/api/generate -H \"Content-Type: application/json\" -d '{\"\"\"\"model\"\"\"\":\"\"\"\"llama3.2:1b\"\"\"\",\"\"\"\"prompt\"\"\"\":\"\"\"\"Say hello\"\"\"\",\"\"\"\"stream\"\"\"\":false}' --max-time 30)",
"Bash(curl -s -X POST http://localhost:8000/api/v1/ai/analyze-and-propose -H \"Content-Type: application/json\" -d '{}' --max-time 120)",
"Bash(curl -s http://localhost:8000/api/v1/ai/analyze-and-propose -X POST -H \"Content-Type: application/json\")",
"Bash(curl -s http://localhost:8000/api/v1/dashboard)",
"Bash(ls -la ~/Downloads/image*.png)",
"Bash(ls -la ~/Desktop/image*.png)",
"Bash(ls -la /Users/ogt/awoooi/apps/web/public/*.png)",
"WebFetch(domain:openclaw.ai)",
"Bash(ls -la /Users/ogt/Downloads/*.png)",
"Bash(ls -la /Users/ogt/.gemini/antigravity/brain/*/image*.png)",
"Bash(ls -lat /Users/ogt/Downloads/*.png)",
"Bash(curl -s http://localhost:8000/api/v1/approvals)",
"Bash(curl -s -X GET http://localhost:8000/api/v1/approvals/)",
"Bash(APPROVAL_ID=\"4989729e-e518-4e7e-8dff-5c3269e0c82b\")",
"Bash(curl -s -X POST \"http://localhost:8000/api/v1/approvals/$APPROVAL_ID/sign\" -H \"Content-Type: application/json\" -d '{\"\"\"\"signer_id\"\"\"\": \"\"\"\"ciso-001\"\"\"\", \"\"\"\"signer_name\"\"\"\": \"\"\"\"Demo CISO\"\"\"\", \"\"\"\"comment\"\"\"\": \"\"\"\"資安確認,核准執行\"\"\"\"}')",
"Bash(curl -s http://localhost:8000/api/v1/webhooks/health)",
"Bash(curl -s -X POST http://localhost:8000/api/v1/webhooks/alerts -H \"Content-Type: application/json\" -d '{:*)",
"Bash(curl -s http://localhost:3000)",
"Bash(ls -la apps/web/test-results/*.png)",
"Bash(curl -s http://localhost:3000/zh-TW/demo)",
"Bash(curl -s -o /dev/null -w \"%{http_code}\" http://localhost:3333/zh-TW/demo)",
"Bash(curl -s http://localhost:8001/api/v1/approvals/pending)",
"Bash(curl -s -X POST http://localhost:8001/api/v1/approvals -H \"Content-Type: application/json\" -d '{:*)",
"Bash(curl -s http://localhost:8001/openapi.json)",
"Bash(curl -s http://localhost:8001/docs)",
"Bash(curl -s http://localhost:8001/api/v1/webhooks/grafana -X OPTIONS)",
"Bash(pnpm run:*)",
"Bash(node scripts/screenshot-rbac.mjs)",
"Bash(pnpm exec:*)",
"Bash(curl -s http://localhost:3333 -o /dev/null -w \"%{http_code}\")",
"Bash(curl -s http://localhost:3333/zh-TW/demo -o /dev/null -w \"%{http_code}\")",
"Bash(python3 -c \"import sys,json; d=json.load\\(sys.stdin\\); print\\(f''''Count: {d[count]}''''\\); [print\\(f''''- {a[id][:8]}... risk={a[risk_level]}''''\\) for a in d[''''approvals''''][:3]]\")",
"Bash(curl -s http://localhost:3000/zh-TW/demo -o /dev/null -w \"%{http_code}\")",
"Bash(python -c \"import sys,json; d=json.load\\(sys.stdin\\); print\\(f'''' Connected: {d[\"\"success\"\"]}''''\\); print\\(f'''' Namespaces: {d[\"\"namespaces\"\"][:3]}...''''\\)\" __NEW_LINE_57ae1c1c812968e7__ echo \"\" echo \"3. 資料庫持久化:\" sqlite3 /Users/ogt/awoooi/apps/api/awoooi.db \"SELECT COUNT\\(*\\) as approvals FROM approval_records;\" sqlite3 /Users/ogt/awoooi/apps/api/awoooi.db \"SELECT COUNT\\(*\\) as timeline FROM timeline_events;\" sqlite3 /Users/ogt/awoooi/apps/api/awoooi.db \"SELECT COUNT\\(*\\) as audits FROM audit_logs;\")",
"Bash(head -2 __NEW_LINE_9bf9481fbdf30d4e__ echo \"\" echo \"2. 告警收斂跳過 LLM 日誌 \\(應該有 4 次\\):\" grep -c \"alert_converged_skip_llm\" /tmp/api-server.log)",
"Bash(python -m json.tool)",
"Bash(__NEW_LINE_7463bff94cecc20f__ echo:*)",
"Bash(__NEW_LINE_13846c8488c5fa9a__ echo:*)",
"Bash(__NEW_LINE_13846c8488c5fa9a__ ls:*)",
"Bash(python -c \"import sys,json; d=json.load\\(sys.stdin\\); print\\(f'''' Status: {d[\"\"status\"\"]}''''\\)\" __NEW_LINE_32366ca1bb050259__ echo \"\" echo \"2. 待簽核記錄 \\(含 hit_count\\):\" curl -s http://localhost:8000/api/v1/approvals/pending)",
"Read(//Users/ogt/awoooi/**)",
"Bash(curl -s http://localhost:8000/api/v1/timeline/events?limit=10)",
"Bash(curl -s http://localhost:8000/api/v1/timeline/events?limit=5)",
"Bash(ls -la /Users/ogt/awoooi/apps/api/*.txt /Users/ogt/awoooi/apps/api/*.toml)",
"Bash(ls -la /Users/ogt/awoooi/docker-compose*.yml)",
"Bash(ls /Users/ogt/awoooi/k8s/awoooi-prod/*rbac* /Users/ogt/awoooi/k8s/awoooi-prod/*service-account*)",
"Bash(kubectl kustomize:*)",
"Bash(docker compose:*)",
"Bash(docker info:*)",
"Bash(python3 -c \"import sys,json; d=json.load\\(sys.stdin\\); print\\(''''API Status:'''', d.get\\(''''status'''', ''''unknown''''\\)\\)\")",
"Bash(pkill -9 -f uvicorn)",
"Bash(lsof -ti:8000)",
"Bash(open -a Docker)",
"Bash(docker stop:*)",
"Bash(lsof -ti:3000)",
"Bash(docker start:*)",
"Bash(docker ps:*)",
"Bash(curl -s http://localhost:3000 -o /dev/null -w 'HTTP Status: %{http_code}\\\\n')",
"Bash(curl -I http://localhost:8000/api/v1/dashboard/stream)",
"Bash(curl -s http://localhost:8000/openapi.json)",
"Bash(curl -s http://localhost:8000/api/v1/dashboard/stream --max-time 3 -w \"\\\\n--- HTTP Status: %{http_code} ---\\\\n\")",
"Bash(curl -s http://localhost:8000/api/v1/dashboard/stream --max-time 3)",
"Bash(curl -s http://localhost:3000/zh-TW -o /dev/null -w \"HTTP Status: %{http_code}\\\\n\")",
"Bash(curl -s -D - http://localhost:8000/api/v1/dashboard/stream --max-time 2)",
"Bash(chmod +x /Users/ogt/awoooi/scripts/deploy-infra.sh)",
"Bash(./scripts/deploy-infra.sh)",
"Bash(pnpm --filter @awoooi/web build)",
"Bash(timeout 10 env MOCK_MODE=true OTEL_ENABLED=false uvicorn src.main:app --host 0.0.0.0 --port 8099)",
"Bash(timeout 8 pnpm --filter @awoooi/web dev)",
"Bash(git diff:*)",
"Bash(curl -s -I http://localhost:8000/api/v1/dashboard/stream)",
"Bash(timeout 3 curl -s -N http://localhost:8000/api/v1/dashboard/stream)",
"Bash(grep -n \"NEXT_PUBLIC\\\\|API_URL\\\\|localhost\" /Users/ogt/awoooi/apps/web/.env*)",
"Bash(timeout 2 curl -s -D - -N http://localhost:8000/api/v1/dashboard/stream)",
"Bash(curl -s http://localhost:3000/)",
"Bash(python -m py_compile scripts/fire_test_alert.py)",
"Bash(python -m scripts.fire_test_alert --help)",
"Bash(python -m scripts.fire_test_alert)",
"Bash(python -m scripts.fire_test_alert --type k8s_pod_crash)",
"Bash(timeout 3 curl -s -N -H \"Origin: http://localhost:3000\" http://localhost:8000/api/v1/dashboard/stream)",
"Bash(python -m scripts.fire_test_alert --type disk_full)",
"Bash(docker restart:*)",
"Bash(curl -s -w \"\\\\nHTTP_CODE: %{http_code}\\\\n\" http://localhost:3000)",
"Bash(docker exec:*)",
"Bash(docker rmi:*)",
"Bash(timeout 5 curl -s -N http://localhost:8000/api/v1/dashboard/stream)",
"Bash(curl -s http://localhost:3000 -w \"\\\\nHTTP: %{http_code}\\\\n\")",
"Bash(timeout 120 docker logs awoooi-api -f --since 1s)",
"Bash(curl -s -I -H \"Origin: http://localhost:3000\" http://localhost:8000/api/v1/dashboard/stream)",
"Bash(curl -s -X OPTIONS -H \"Origin: http://localhost:3000\" -H \"Access-Control-Request-Method: GET\" http://localhost:8000/api/v1/dashboard/stream -I)",
"Bash(node /Users/ogt/awoooi/scripts/verify-sse.js)",
"Bash(python -m scripts.fire_test_alert --type db_connection_timeout)",
"Bash(npm run:*)",
"Bash(docker-compose down:*)",
"Bash(docker-compose build:*)",
"Bash(docker-compose up:*)",
"Bash(pkill -f 'next dev')",
"Bash(node /Users/ogt/awoooi/scripts/test-approval-flow.js)",
"Bash(python -m scripts.fire_test_alert --type pod_crash)",
"Bash(node /Users/ogt/awoooi/scripts/test-k8s-executor.js)",
"Bash(kubectl cluster-info:*)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl cluster-info)",
"Bash(ls -la /Users/ogt/awoooi/apps/web/src/app/[locale]/)",
"Bash(python -c \"from src.api.v1 import audit_logs; print\\(''API module loads OK''\\)\")",
"Bash(curl -s http://localhost:3000/zh-TW/action-logs)",
"Bash(pnpm build:*)",
"Bash(curl -s http://localhost:8000/api/v1/audit-logs)",
"Bash(xargs -r kill -9 2)",
"Bash(/dev/null source:*)",
"Bash(python -c \"from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor; print\\(''''httpx ok''''\\)\")",
"Bash(sqlite3 /Users/ogt/awoooi/apps/api/awoooi.db \"SELECT * FROM audit_logs ORDER BY created_at DESC LIMIT 5;\")",
"Bash(sqlite3 /Users/ogt/awoooi/apps/api/awoooi.db \"SELECT name FROM sqlite_master WHERE type=''table'';\")",
"Bash(sqlite3 /Users/ogt/awoooi/apps/api/awoooi.db \"SELECT id, event_type, status, title, created_at FROM timeline_events ORDER BY created_at DESC LIMIT 5;\")",
"Bash(curl -s http://localhost:8000/api/v1/audit-logs/stats)",
"Bash(curl -s http://localhost:8000/api/v1/timeline?limit=10)",
"Bash(curl -s \"http://localhost:8000/api/v1/timeline\")",
"Bash(curl -s http://localhost:8000/api/v1/docs)",
"Bash(chmod +x /Users/ogt/awoooi/scripts/setup-guardrails.sh /Users/ogt/awoooi/scripts/ai_code_reviewer.py)",
"Bash(ls -la /Users/ogt/awoooi/apps/web/.eslintrc*)",
"Bash(ls -la scripts/*.py scripts/*.sh .pre-commit-config.yaml .secrets.baseline apps/web/.eslintrc.js)",
"Bash(python -m src.services.test_context_gatherer)",
"Bash(python -m pytest src/services/test_context_gatherer.py -v)",
"Bash(grep -r \"ClawBot\\\\|clawbot\\\\|CLAWBOT\" --include=*.py --include=*.ts --include=*.tsx apps/)",
"Bash(python scripts/e2e_openclaw_test.py)",
"Bash(python -m pytest tests/e2e_network_test.py -v --tb=short)",
"Bash(chmod +x /Users/ogt/awoooi/apps/api/scripts/apply_prometheus_config.sh /Users/ogt/awoooi/apps/api/scripts/fire_live_alert.py)",
"Bash(./scripts/apply_prometheus_config.sh)",
"Bash(python scripts/fire_live_alert.py oomkilled)",
"Bash(python scripts/fire_live_alert.py oomkilled --api-url http://localhost:8000)",
"Bash(python scripts/fire_live_alert.py highcpu --api-url http://localhost:8000)",
"Bash(python scripts/fire_live_alert.py podcrash --api-url http://localhost:8000)",
"Bash(python -m pytest tests/test_webhook_telegram_integration.py -v)",
"Bash(ls -la /Users/ogt/awoooi/apps/api/.env*)",
"Bash(ls -la /Users/ogt/wooo-aiops/.env*)",
"Bash(ls -la /Users/ogt/AIOps/.env*)",
"Bash(/Users/ogt/awoooi/apps/api/.env:*)",
"Bash(/tmp/deploy-188-home.sh:*)",
"Bash(chmod +x /tmp/deploy-188-home.sh)",
"Bash(scp /tmp/awoooi-api-deploy.tar.gz /tmp/deploy-188-home.sh ollama@192.168.0.188:/tmp/)",
"Bash(ssh ollama@192.168.0.188 \"bash /tmp/deploy-188-home.sh\")",
"Bash(ssh ollama@192.168.0.188 \"curl -s http://localhost:8000/api/v1/webhooks/health\")",
"Bash(ssh ollama@192.168.0.188 \"tail -50 /tmp/openclaw.log\")",
"Bash(ssh ollama@192.168.0.188 \"cd /home/ollama/awoooi-api && source .venv/bin/activate && pip install sqlalchemy aiosqlite -q && pip install httpx python-dotenv pydantic-settings -q\")",
"Bash(ssh ollama@192.168.0.188 \"cd /home/ollama/awoooi-api && pkill -f ''uvicorn src.main:app'' 2>/dev/null; sleep 1; source .venv/bin/activate && nohup uvicorn src.main:app --host 0.0.0.0 --port 8000 > /tmp/openclaw.log 2>&1 & sleep 3 && curl -s http://localhost:8000/api/v1/webhooks/health\")",
"Bash(ssh ollama@192.168.0.188:*)",
"Bash(pkill -f ngrok)",
"Bash(pkill -f \"ssh -fN.*8001\")",
"Bash(ssh -fN -L 8001:localhost:8000 ollama@192.168.0.188)",
"Bash(curl -s http://localhost:8001/api/v1/webhooks/health)",
"Bash(BOT_TOKEN=\"8569720657:AAHdvKf_P2ms-QKFTyqTLtLiqEggz8cpjMk\" curl -s \"https://api.telegram.org/bot$BOT_TOKEN/getWebhookInfo\")",
"Bash(curl -s https://api.telegram.org/bot$BOT_TOKEN/getWebhookInfo)",
"Bash(curl -s http://localhost:8001/api/v1/webhooks/)",
"Bash(curl -s http://localhost:8001/)",
"Bash(curl -s http://localhost:8001/api/v1/health)",
"Bash(scp /tmp/awoooi-api-v7.tar.gz ollama@192.168.0.188:/tmp/)",
"Bash(tar -czvf /tmp/awoooi-api-v7.1.tar.gz src/ requirements.txt pyproject.toml)",
"Bash(scp /tmp/awoooi-api-v7.1.tar.gz ollama@192.168.0.188:/tmp/)",
"Bash(ssh ollama@192.168.0.188 \"tail -10 /tmp/openclaw.log | grep -E ''''clickhouse|signoz_gold''''\")",
"Bash(ssh ogt@192.168.0.188 \"cd /home/ollama/awoooi-api && tail -50 nohup.out 2>/dev/null || journalctl -u awoooi-api --no-pager -n 50 2>/dev/null || echo ''請手動檢查日誌''\")",
"Bash(curl -s --connect-timeout 5 http://192.168.0.188:8123/ -d \"SELECT 1 FORMAT JSONEachRow\")",
"Bash(curl -s --connect-timeout 5 http://192.168.0.188:11434/api/tags)",
"Bash(ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o BatchMode=yes -o ConnectTimeout=5 ollama@192.168.0.188 \"echo ok\")",
"Bash(ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o BatchMode=yes -o ConnectTimeout=5 wooo@192.168.0.188 \"echo ok\")",
"Bash(ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o BatchMode=yes -o ConnectTimeout=5 root@192.168.0.188 \"echo ok\")",
"Bash(curl -s --connect-timeout 5 http://192.168.0.188:8001/health)",
"Bash(ssh root@192.168.0.188 \"cat /tmp/openclaw.log 2>/dev/null | tail -100 || echo ''Log file not found''\")",
"Bash(ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=5 ollama@192.168.0.188 \"echo ok\")",
"Bash(ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=5 wooo@192.168.0.188 \"echo ok\")",
"Bash(scp /Users/ogt/awoooi/apps/api/src/services/signoz_client.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/services/)",
"Bash(scp /Users/ogt/awoooi/apps/api/src/services/openclaw.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/services/)",
"Bash(scp /Users/ogt/awoooi/apps/api/src/services/telegram_gateway.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/services/)",
"Bash(scp /Users/ogt/awoooi/apps/api/src/api/v1/webhooks.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/api/v1/)",
"Bash(scp /Users/ogt/awoooi/apps/api/src/models/ai.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/models/)",
"Bash(ssh ollama@192.168.0.188 \"cd /home/ollama/awoooi-api && pkill -f ''''uvicorn src.main:app'''' && sleep 2 && nohup .venv/bin/python3 -m uvicorn src.main:app --host 0.0.0.0 --port 8000 > nohup.out 2>&1 &\")",
"Bash(curl -s --connect-timeout 5 http://192.168.0.188:8000/health)",
"Bash(curl -s --connect-timeout 10 http://192.168.0.188:8000/health)",
"Bash(curl -s -X POST http://192.168.0.188:8000/api/v1/webhooks/alerts -H \"Content-Type: application/json\" -d '{:*)",
"Bash(curl -s -X POST http://192.168.0.188:8000/api/v1/webhooks/alerts -H \"Content-Type: application/json\" -d '{\"\"alert_type\"\":\"\"high_cpu\"\",\"\"severity\"\":\"\"critical\"\",\"\"source\"\":\"\"signoz\"\",\"\"target_resource\"\":\"\"api-gateway\"\",\"\"namespace\"\":\"\"awoooi-prod\"\",\"\"message\"\":\"\"CPU 92% test\"\"}')",
"Bash(curl -s --connect-timeout 5 http://192.168.0.188:8000/api/v1/webhooks/alerts -X POST -H \"Content-Type: application/json\" -d '{\"\"alert_type\"\":\"\"high_cpu\"\",\"\"severity\"\":\"\"critical\"\",\"\"source\"\":\"\"signoz\"\",\"\"target_resource\"\":\"\"api-gateway\"\",\"\"namespace\"\":\"\"awoooi-prod\"\",\"\"message\"\":\"\"CPU 92% - 統帥全自主驗收 v2\"\"}')",
"Bash(curl -s --connect-timeout 30 --max-time 120 -X POST http://192.168.0.188:8000/api/v1/webhooks/alerts -H \"Content-Type: application/json\" -d '{:*)",
"Bash(curl -s --connect-timeout 30 --max-time 180 -X POST http://192.168.0.188:8000/api/v1/webhooks/alerts -H \"Content-Type: application/json\" -d '{:*)",
"Bash(curl -s http://192.168.0.188:8000/api/v1/webhooks/alerts -X POST -H \"Content-Type: application/json\" -d '{\"\"alert_type\"\":\"\"k8s_pod_crash\"\",\"\"severity\"\":\"\"critical\"\",\"\"source\"\":\"\"signoz\"\",\"\"target_resource\"\":\"\"inventory-api\"\",\"\"namespace\"\":\"\"commerce\"\",\"\"message\"\":\"\"Pod crash - 統帥終極驗收\"\"}' --connect-timeout 30 --max-time 180)",
"Bash(ssh -o ConnectTimeout=10 ollama@192.168.0.188 \"echo OK && ps aux | grep uvicorn | grep -v grep | head -2\")",
"Bash(curl -s http://192.168.0.188:8000/api/v1/webhooks/alerts -X POST -H \"Content-Type: application/json\" -d '{\"\"alert_type\"\":\"\"ssl_expiry\"\",\"\"severity\"\":\"\"critical\"\",\"\"source\"\":\"\"signoz\"\",\"\"target_resource\"\":\"\"nginx-ingress\"\",\"\"namespace\"\":\"\"ingress\"\",\"\"message\"\":\"\"SSL 即將過期 - 終極驗收\"\"}' --connect-timeout 30 --max-time 180)",
"Bash(curl -s http://192.168.0.188:8000/api/v1/webhooks/alerts -X POST -H \"Content-Type: application/json\" -d '{\"\"alert_type\"\":\"\"db_connection_timeout\"\",\"\"severity\"\":\"\"critical\"\",\"\"source\"\":\"\"signoz\"\",\"\"target_resource\"\":\"\"postgres-primary\"\",\"\"namespace\"\":\"\"database\"\",\"\"message\"\":\"\"DB 連線逾時 - SignOz 整合終極測試\"\"}' --connect-timeout 30 --max-time 180)",
"Bash(curl -s http://192.168.0.188:8000/api/v1/webhooks/alerts -X POST -H \"Content-Type: application/json\" -d '{\"\"alert_type\"\":\"\"service_404\"\",\"\"severity\"\":\"\"critical\"\",\"\"source\"\":\"\"signoz\"\",\"\"target_resource\"\":\"\"auth-service\"\",\"\"namespace\"\":\"\"identity\"\",\"\"message\"\":\"\"Service 404 - SignOz + Ollama 整合終極測試\"\"}' --connect-timeout 30 --max-time 180)",
"Bash(curl -s http://192.168.0.188:8000/api/v1/webhooks/alerts -X POST -H \"Content-Type: application/json\" -d '{\"\"alert_type\"\":\"\"high_cpu\"\",\"\"severity\"\":\"\"warning\"\",\"\"source\"\":\"\"signoz\"\",\"\"target_resource\"\":\"\"recommendation-engine\"\",\"\"namespace\"\":\"\"ml\"\",\"\"message\"\":\"\"CPU 78% - Ollama 最終測試\"\"}' --connect-timeout 30 --max-time 200)",
"Bash(scp apps/api/src/services/openclaw.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/services/openclaw.py)",
"Bash(scp /Users/ogt/awoooi/apps/api/src/core/http_client.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/core/)",
"Bash(scp /Users/ogt/awoooi/apps/api/src/main.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/)",
"Bash(scp /Users/ogt/awoooi/apps/api/src/core/config.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/core/)",
"Bash(scp /Users/ogt/awoooi/apps/api/src/api/v1/health.py ollama@192.168.0.188:/home/ollama/awoooi-api/src/api/v1/)",
"Bash(ssh -o ConnectTimeout=5 ollama@192.168.0.188 \"ps aux | grep uvicorn | grep -v grep\")",
"Bash(curl -s -H \"Origin: http://localhost:3000\" -H \"Access-Control-Request-Method: GET\" -X OPTIONS http://192.168.0.188:8000/api/v1/health -v)",
"Bash(curl -s http://192.168.0.188:8000/api/v1/health)",
"Bash(curl -s -N --max-time 3 http://192.168.0.188:8000/api/v1/dashboard/stream)",
"Bash(curl -s http://localhost:3000/zh-TW -o /dev/null -w \"%{http_code}\")",
"Bash(open http://localhost:3000/zh-TW)",
"Bash(open http://localhost:3001/zh-TW)",
"Bash(curl -s -H \"Origin: http://localhost:3001\" http://192.168.0.188:8000/api/v1/dashboard/stream --max-time 3)",
"Bash(curl -s -I -H \"Origin: http://localhost:3001\" http://192.168.0.188:8000/api/v1/health)",
"Bash(curl -s http://192.168.0.188:8000/api/v1/approvals/pending)",
"Bash(curl -s http://192.168.0.188:8000/api/v1/approvals)",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/approvals?status=pending_approval\")",
"Bash(xargs sed:*)",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/approvals/history?limit=5\")",
"Bash(curl -s http://192.168.0.188:8000/api/v1/approvals/approved)",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/timeline?limit=10\")",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/action-logs\")",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/timeline/events?limit=10\")",
"Bash(ssh ogt@192.168.0.188 \"kubectl get nodes\")",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/approvals/k8s-test\")",
"Bash(scp /Users/ogt/awoooi/apps/api/k3s-prod.yaml ogt@192.168.0.188:~/awoooi-api/k3s-prod.yaml)",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/timeline/events?limit=5\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.120 \"cat /etc/rancher/k3s/k3s.yaml\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no wooo@192.168.0.188 \"echo ''SSH OK'' && pwd\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"echo ''SSH OK'' && pwd && ls -la ~/awoooi-api/ 2>/dev/null || echo ''Directory not found''\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"sshpass -p ''0936223270'' scp -o StrictHostKeyChecking=no wooo@192.168.0.120:/etc/rancher/k3s/k3s.yaml ~/awoooi-api/k3s-prod.yaml && sed -i ''s/127.0.0.1/192.168.0.120/g'' ~/awoooi-api/k3s-prod.yaml && echo ''Kubeconfig deployed!'' && head -10 ~/awoooi-api/k3s-prod.yaml\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd ~/awoooi-api && pkill -f ''uvicorn'' 2>/dev/null; sleep 1; nohup .venv/bin/uvicorn src.main:app --host 0.0.0.0 --port 8000 --reload > nohup.out 2>&1 & sleep 3; echo ''=== API Restarted ==='' && tail -20 nohup.out\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 \"cd ~/awoooi-api && pkill -f ''uvicorn src.main'' || true\")",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/health\" --connect-timeout 5)",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10 ollama@192.168.0.188 \"cd ~/awoooi-api && source .venv/bin/activate && nohup uvicorn src.main:app --host 0.0.0.0 --port 8000 > nohup.out 2>&1 &\")",
"Bash(sshpass -p:*)",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/health\" --connect-timeout 10)",
"Bash(curl -s \"http://192.168.0.188:8000/api/v1/timeline/events?limit=8\")",
"Bash(curl -s http://localhost:3000/zh-TW -o /dev/null -w \"Frontend: HTTP %{http_code}\\\\n\")",
"Bash(sshpass -p '0936223270' ssh -o StrictHostKeyChecking=no ollama@192.168.0.188 'curl -s http://localhost:8000/api/v1/approvals/pending | jq -r \"\".approvals[] | \\\\\"\"ID: \\\\\\(.id\\) | Action: \\\\\\(.action\\)\\\\\"\"\"\"')",
"Bash(curl -s --connect-timeout 5 https://awoooi.wooo.tw/api/v1/health)",
"Bash(curl -s --connect-timeout 5 https://awoooi.wooo.tw/api/v1/approvals/pending)",
"Bash(ssh ollama@192.168.70.188 \"ps aux | grep uvicorn | grep -v grep | head -3\")",
"Bash(ssh -o ConnectTimeout=10 ollama@192.168.70.188 \"echo ''SSH Connected''\")",
"Bash(ping -c 2 -t 5 192.168.70.188)",
"Bash(curl -s --connect-timeout 10 https://awoooi.wooo.tw/api/v1/health)",
"Bash(ssh -o ConnectTimeout=10 ollama@192.168.0.188 \"echo ''SSH Connected to 188 Base''\")",
"Bash(grep -B 5 -A 30 \"async def add_signature\" /Users/ogt/awoooi/apps/api/src/services/*.py)",
"Bash(ssh ogt@192.168.0.188 \"cd /home/ogt/awoooi && docker compose ps\")",
"Bash(ls -la .env*)",
"Bash(.env:*)",
"Bash(timeout 15 python -m uvicorn src.main:app --host 0.0.0.0 --port 8001)",
"Bash(timeout 20 python -m uvicorn src.main:app --host 0.0.0.0 --port 8001)",
"Bash(timeout 25 python -m uvicorn src.main:app --host 0.0.0.0 --port 8001)",
"Bash(ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no ogt@192.168.0.188 \"cd /home/ogt/wooo-aiops && docker compose ps clawbot 2>/dev/null || docker ps | grep -i claw\")",
"Bash(ls -la ~/.ssh/*.pub)",
"Bash(ssh -i ~/.ssh/id_rsa -o ConnectTimeout=5 -o StrictHostKeyChecking=no -o PasswordAuthentication=no ogt@192.168.0.188 \"echo connected\")",
"Bash(curl -s \"https://api.telegram.org/bot8569720657:AAHdvKf_P2ms-QKFTyqTLtLiqEggz8cpjMk/logOut\")",
"Bash(curl -s \"https://api.telegram.org/bot8569720657:AAHdvKf_P2ms-QKFTyqTLtLiqEggz8cpjMk/close\")",
"Bash(curl -s \"https://api.telegram.org/bot8569720657:AAHdvKf_P2ms-QKFTyqTLtLiqEggz8cpjMk/getUpdates?timeout=3&limit=1\")",
"Bash(ping -c 1 192.168.0.188)",
"Bash(python -m tests.test_redis_multisig)",
"Bash(curl -v -X POST http://localhost:8000/api/v1/webhooks/signals -H \"Content-Type: application/json\" -d '{:*)",
"Bash(python3 -c \":*)",
"Bash(echo ' 無法連線' __NEW_LINE_8fc87454f9798a7d__ echo echo [結論]: echo ' /signals 端點尚未部署到 .188' echo ' 程式碼已完成,需要執行:' echo \" cd apps/api && docker build -t awoooi-api . && docker-compose up -d\")",
"Bash(__NEW_LINE_dc88f37970737861__ cd:*)",
"Bash(__NEW_LINE_dc88f37970737861__ echo:*)",
"Read(//Users/**)",
"Bash(tail -20 __NEW_LINE_8b049957a9782734__ echo \"\" echo \"[Step 2] 等待容器啟動 \\(10 秒\\)...\" sleep 10 __NEW_LINE_8b049957a9782734__ echo \"\" echo \"[Step 3] 檢查容器狀態...\" docker compose ps)",
"Bash(tail -5 __NEW_LINE_275e0094e9dcb44a__ echo \"\" echo \"[1.2] 重建 API 容器 \\(含 Signal Worker\\)...\" docker compose build api)",
"Bash(1 __NEW_LINE_275e0094e9dcb44a__ echo \"\" echo \"[1.4] 等待服務就緒 \\(15 秒\\)...\" sleep 15 __NEW_LINE_275e0094e9dcb44a__ echo \"\" echo \"[1.5] 檢查容器狀態...\" docker compose ps)",
"Bash(__NEW_LINE_f4c8301ec5249760__ echo:*)",
"Bash(__NEW_LINE_21ba3cf3700d942d__ cd:*)",
"Bash(1 __NEW_LINE_9a14b79fc58c11ba__ echo \"\" echo \"[1.3] 等待服務就緒 \\(15 秒\\)...\" sleep 15 __NEW_LINE_9a14b79fc58c11ba__ echo \"\" echo \"[1.4] 檢查容器狀態...\" docker compose ps api)",
"Bash(1 __NEW_LINE_6b654ca5be87c137__ echo \"\" echo \"[2] 等待服務就緒 \\(15 秒\\)...\" sleep 15 __NEW_LINE_6b654ca5be87c137__ echo \"\" echo \"[3] 發送測試 Signal...\" curl -s -X POST http://localhost:8000/api/v1/webhooks/signals -H \"Content-Type: application/json\" -d '{:*)",
"Bash(__NEW_LINE_564908ddf866c081__ echo:*)",
"Bash(chmod +x /Users/ogt/awoooi/apps/api/scripts/test_phase63_aggregation.py)",
"Bash(python scripts/test_phase63_aggregation.py)",
"Bash(xargs -r docker exec -i awoooi-redis redis-cli DEL)",
"Bash(chmod +x /Users/ogt/awoooi/apps/api/scripts/test_race_condition.py)",
"Bash(python scripts/test_race_condition.py)",
"Bash(chmod +x /Users/ogt/awoooi/apps/api/scripts/test_phase64_proposal.py)",
"Bash(python scripts/test_phase64_proposal.py)",
"Bash(python agent.py --alert FINAL_PHASE_6_TEST)",
"Bash(AWOOOI_REDIS_URL=\"redis://localhost:6379/0\" python agent.py --alert FINAL_PHASE_6_TEST)",
"Bash(curl -s http://localhost:8000/api/v1/incidents)",
"Bash(curl -s -X POST http://localhost:8000/api/v1/incidents/INC-20260322-06085B/proposal)",
"Bash(grep -r \"mock\\\\|Mock\\\\|MOCK\\\\|fake\\\\|Fake\\\\|dummy\\\\|hardcode\" /Users/ogt/awoooi/apps/web/src --include=*.tsx --include=*.ts -l)",
"Bash(NEXT_PUBLIC_API_URL=http://localhost:8000 pnpm next build --no-lint)",
"Bash(grep -v \"Traceback\\\\|File \"\"/usr\\\\|^\\\\s*$\")",
"Bash(python -c \"import sys,json; d=json.load\\(sys.stdin\\); print\\(f''''Signal Count: {len\\(d[\"\"signals\"\"]\\)}''''\\); [print\\(f'''' - {s[\"\"alert_name\"\"]} \\({s[\"\"signal_id\"\"]}\\)''''\\) for s in d[''''signals'''']]\")",
"Bash(curl -s -o /dev/null -w \"%{http_code}\" http://localhost:3003/zh-TW)",
"Bash(curl -s -X GET \"http://localhost:8000/api/v1/incidents\" -H \"Origin: http://localhost:3003\" -H \"Access-Control-Request-Method: GET\" -v)",
"Bash(grep -r TELEGRAM /Users/ogt/awoooi/apps/api/.env*)",
"Bash(grep -r TELEGRAM_BOT_TOKEN /Users/ogt/awoooi --include=*.env* --include=*.yaml --include=*.yml)",
"Bash(curl -s -I -X OPTIONS \"http://localhost:8000/api/v1/incidents\" -H \"Origin: http://localhost:3000\" -H \"Access-Control-Request-Method: GET\")",
"Bash(curl -s \"http://localhost:8000/api/v1/incidents\" -H \"Origin: http://localhost:3000\")",
"Bash(python /tmp/e2e_drill.py)",
"Bash(python -c \"import sys,json; d=json.load\\(sys.stdin\\); i=[x for x in d[''''incidents''''] if x[''''incident_id'''']==''''INC-20260322-06085B''''][0]; print\\(f\"\"Incident: {i[''''incident_id'''']}\"\"\\); print\\(f\"\"Signals: {i[''''signal_count'''']}\"\"\\); print\\(f\"\"Updated: {i[''''updated_at'''']}\"\"\\)\")",
"Bash(curl -s -X POST \"http://localhost:8000/api/v1/telegram/test\")",
"Bash(curl -s -X POST \"http://localhost:8000/api/v1/telegram/test-push\" -H \"Content-Type: application/json\" -d '{\"\"\"\"approval_id\"\"\"\": \"\"\"\"15ab6844-ca4e-4a13-aead-dc71cd342445\"\"\"\", \"\"\"\"risk_level\"\"\"\": \"\"\"\"critical\"\"\"\", \"\"\"\"resource_name\"\"\"\": \"\"\"\"api-gateway\"\"\"\", \"\"\"\"root_cause\"\"\"\": \"\"\"\"E2E DRILL - PodCrashLoopBackOff\"\"\"\", \"\"\"\"suggested_action\"\"\"\": \"\"\"\"RESTART_DEPLOYMENT\"\"\"\", \"\"\"\"estimated_downtime\"\"\"\": \"\"\"\"5-15 min\"\"\"\"}')",
"Bash(curl -s -o /dev/null -w \"HTTP Status: %{http_code}\\\\n\" http://localhost:3000/zh-TW)",
"Bash(curl -s -I \"http://localhost:8000/api/v1/incidents\" -H \"Origin: http://localhost:3000\")",
"Bash(curl -s -X POST http://localhost:8000/api/v1/incidents/INC-20260322-19DF60/proposal)",
"Bash(curl -s -X POST \"http://localhost:8000/api/v1/telegram/test-push\" -H \"Content-Type: application/json\" -d '{\"\"\"\"approval_id\"\"\"\": \"\"\"\"942e762e-fb97-480f-b21a-d3be67fa70b1\"\"\"\", \"\"\"\"risk_level\"\"\"\": \"\"\"\"critical\"\"\"\", \"\"\"\"resource_name\"\"\"\": \"\"\"\"core-system\"\"\"\", \"\"\"\"root_cause\"\"\"\": \"\"\"\"E2E DRILL TAKE 2 - 二次實彈演習\"\"\"\", \"\"\"\"suggested_action\"\"\"\": \"\"\"\"INVESTIGATE_SERVICE\"\"\"\", \"\"\"\"estimated_downtime\"\"\"\": \"\"\"\"5-15 min\"\"\"\"}')",
"Bash(curl -s \"http://localhost:8000/api/v1/incidents\" -H \"Origin: http://localhost:3000\" -H \"Accept: application/json\")",
"Bash(python -c \"import sys,json; d=json.load\\(sys.stdin\\); print\\(f''''Incidents: {d[\"\"count\"\"]}''''\\); [print\\(f'''' - {i[\"\"incident_id\"\"]} | {i[\"\"severity\"\"]} | {i[\"\"signal_count\"\"]} signals | {i[\"\"affected_services\"\"]}''''\\) for i in d[''''incidents'''']]\")",
"Bash(curl -s \"http://localhost:8000/api/v1/approvals/pending\" -H \"Origin: http://localhost:3000\")",
"Bash(python -c \"import sys,json; d=json.load\\(sys.stdin\\); print\\(f''''Pending: {d[\"\"count\"\"]} approvals''''\\); [print\\(f'''' - {a[\"\"id\"\"][:8]}... | {a[\"\"risk_level\"\"]} | {a[\"\"action\"\"][:30]}...''''\\) for a in d[''''approvals''''][:3]]\")",
"Bash(mkdir -p /Users/ogt/awoooi/apps/web/public/fonts)",
"Bash(curl -sL -o DSEG7Classic-Bold.woff2 \"https://cdn.jsdelivr.net/npm/dseg@0.46.0/fonts/DSEG7-Classic/DSEG7Classic-Bold.woff2\")",
"Bash(curl -sL -o DSEG7Classic-Bold.woff \"https://cdn.jsdelivr.net/npm/dseg@0.46.0/fonts/DSEG7-Classic/DSEG7Classic-Bold.woff\")",
"Bash(curl -sL -o DSEG7Classic-Regular.woff2 \"https://cdn.jsdelivr.net/npm/dseg@0.46.0/fonts/DSEG7-Classic/DSEG7Classic-Regular.woff2\")",
"Bash(curl -sL -o DSEG7Classic-Regular.woff \"https://cdn.jsdelivr.net/npm/dseg@0.46.0/fonts/DSEG7-Classic/DSEG7Classic-Regular.woff\")",
"Bash(pnpm next:*)",
"Bash(chmod +x /Users/ogt/awoooi/scripts/bootstrap_prod.sh)",
"Bash(/Users/ogt/awoooi/.env:*)",
"Bash(grep -E \"^\\\\.env$|03-secrets\\\\.yaml\" .gitignore)",
"Bash(echo 'Adding to .gitignore...' if ! grep -q ^.env$ .gitignore)",
"Bash(then echo:*)",
"Bash(git add:*)",
"Bash(git commit:*)",
"Bash(git push:*)",
"Bash(git remote:*)",
"Bash(gh repo:*)",
"Bash(gh api:*)",
"Bash(gh run:*)",
"Bash(ls -la pnpm-*.yaml package.json turbo.json)",
"Bash(git status:*)",
"Bash(gh workflow:*)",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod -o wide\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-api-77545758fc-xnncc -n awoooi-prod --tail=50\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-api-77545758fc-xnncc -n awoooi-prod 2>&1 | grep -i ''cors'' -A 5 -B 5\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-api-79948cbbbf-b8cgj -n awoooi-prod --tail=100\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod -l app=awoooi-api --sort-by=.metadata.creationTimestamp -o name | tail -1 | xargs kubectl logs -n awoooi-prod --tail=50\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath=''{.data.OPENCLAW_TG_USER_WHITELIST}'' | base64 -d\")",
"Bash(ssh wooo@192.168.0.120 'kubectl patch secret awoooi-secrets -n awoooi-prod --type='\"''\"'json'\"''\"' -p='\"''\"'[:*)",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout restart deployment/awoooi-api -n awoooi-prod && kubectl rollout status deployment/awoooi-api -n awoooi-prod --timeout=120s\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout restart deployment/awoooi-worker -n awoooi-prod && kubectl rollout status deployment/awoooi-worker -n awoooi-prod --timeout=120s\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-worker-747967b787-fcx2r -n awoooi-prod --tail=30\")",
"Bash(ssh wooo@192.168.0.110 \"ps aux | grep -E ''actions-runner|Runner'' | grep -v grep\")",
"Bash(curl -sf http://192.168.0.120:32334/api/v1/health)",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-api-fd795cd87-rdpgn -n awoooi-prod --tail=30\")",
"Bash(ssh wooo@192.168.0.110 \"curl -sf http://192.168.0.120:32334/api/v1/health | jq .status\")",
"Bash(ssh wooo@192.168.0.110 \"curl -sf http://192.168.0.120:32334/api/v1/health\")",
"Bash(ssh wooo@192.168.0.120 \"curl -sf http://localhost:32334/api/v1/health\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get svc -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"curl -sf http://10.43.125.201:8000/api/v1/health\")",
"Bash(ssh wooo@192.168.0.120 \"curl -sf http://10.43.105.105:3000/ -o /dev/null && echo ''Web OK''\")",
"Bash(ssh ogt@192.168.0.188 \"ls -la /etc/nginx/sites-available/\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=50\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-api-795c95ff76-wch2p -n awoooi-prod --tail=30\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod && ss -tlnp | grep 32334\")",
"Bash(ssh wooo@192.168.0.120 \"curl -sf http://127.0.0.1:32334/api/v1/health | head -c 200\")",
"Bash(ssh wooo@192.168.0.120 \"sudo ufw status 2>/dev/null || sudo iptables -L INPUT -n | head -20\")",
"Bash(ssh wooo@192.168.0.110 \"curl -sf --connect-timeout 5 http://192.168.0.120:32334/api/v1/health | head -c 100\")",
"Bash(ssh wooo@192.168.0.110 \"curl -v --connect-timeout 5 http://192.168.0.120:32334/api/v1/health 2>&1 | head -30\")",
"Bash(ssh wooo@192.168.0.120 \"cat /etc/systemd/system/k3s.service 2>/dev/null | grep -i exec || ps aux | grep k3s | head -3\")",
"Bash(ssh wooo@192.168.0.120 \"cat /etc/systemd/system/k3s.service\")",
"Bash(ssh wooo@192.168.0.120 \"netstat -tlnp 2>/dev/null | grep 32334 || ss -tlnp | grep 32334\")",
"Bash(ssh wooo@192.168.0.110 \"curl -sf --connect-timeout 5 http://192.168.0.120:31234/health 2>&1 | head -c 100\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get networkpolicy -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get networkpolicy allow-nginx-ingress -n awoooi-prod -o yaml\")",
"Bash(curl -sk https://awoooi.wooo.work/api/v1/health)",
"Bash(curl -sk -I -X OPTIONS https://awoooi.wooo.work/api/v1/health -H \"Origin: https://awoooi.wooo.work\" -H \"Access-Control-Request-Method: GET\")",
"Bash(ssh wooo@192.168.0.120 \"curl -sI --connect-timeout 3 http://127.0.0.1:32334/api/v1/health 2>&1 | head -5\")",
"Bash(ssh wooo@192.168.0.120 \"curl -sI --connect-timeout 3 http://127.0.0.1:32335/ 2>&1 | head -5\")",
"Bash(ssh wooo@192.168.0.121 \"curl -sI --connect-timeout 3 http://127.0.0.1:32334/api/v1/health 2>&1 | head -5\")",
"Bash(ssh wooo@192.168.0.121 \"curl -sI --connect-timeout 3 http://127.0.0.1:32335/ 2>&1 | head -5\")",
"Bash(ssh wooo@192.168.0.120 \"sudo iptables -t nat -L KUBE-NODEPORTS -n 2>/dev/null | head -20\")",
"Bash(ssh wooo@192.168.0.120 \"sudo netstat -tlnp | grep -E ''32334|32335''\")",
"Bash(ssh wooo@192.168.0.120 \"ss -tlnp 2>/dev/null | grep -E ''32334|32335'' || netstat -tln | grep -E ''32334|32335''\")",
"Bash(ssh wooo@192.168.0.120 \"ss -tln | grep -E ''32334|32335|:323''\")",
"Bash(ssh wooo@192.168.0.120 \"ss -tln\")",
"Bash(ssh wooo@192.168.0.120 \"export KUBECONFIG=/home/wooo/.kube/config-120; /home/wooo/bin/kubectl get svc -n awoooi-prod -o wide\")",
"Bash(ssh wooo@192.168.0.120 \"which kubectl || find /usr -name kubectl 2>/dev/null | head -1\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get svc -n awoooi-prod && kubectl get pods -n awoooi-prod -o wide\")",
"Bash(ssh wooo@192.168.0.120 \"export KUBECONFIG=/home/wooo/.kube/config-120 && kubectl logs awoooi-api-546b88465d-lb8zm -n awoooi-prod --tail 80\")",
"Bash(ssh wooo@192.168.0.120 \"KUBECONFIG=/home/wooo/.kube/config-120 kubectl logs awoooi-api-546b88465d-lb8zm -n awoooi-prod --tail 80 2>&1\")",
"Bash(ssh wooo@192.168.0.120 \"ls -la /home/wooo/.kube/ && cat /home/wooo/.kube/config-120 2>/dev/null | head -20 || cat /etc/rancher/k3s/k3s.yaml 2>/dev/null | head -20\")",
"Bash(ssh wooo@192.168.0.120 \"sudo cat /etc/rancher/k3s/k3s.yaml | head -20\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && kubectl logs awoooi-api-546b88465d-lb8zm -n awoooi-prod --tail 100 2>&1\")",
"Bash(ssh wooo@192.168.0.110 \"which kubectl 2>/dev/null || find /home/wooo -name kubectl 2>/dev/null | head -1 || ls -la /home/wooo/bin/\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl logs awoooi-api-546b88465d-lb8zm -n awoooi-prod --tail 100 2>&1\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl describe pod awoooi-api-546b88465d-lb8zm -n awoooi-prod | tail -40\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl get svc -n awoooi-prod -o wide\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl exec -n awoooi-prod deploy/awoooi-api -- curl -sf http://localhost:8000/api/v1/health 2>&1\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl exec -n awoooi-prod deploy/awoooi-api -- wget -qO- http://localhost:8000/api/v1/health 2>&1\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl logs deployment/awoooi-api -n awoooi-prod --tail 20 2>&1\")",
"Bash(ssh wooo@192.168.0.110 \"curl -sf http://192.168.0.120:32334/api/v1/health 2>&1 || echo ''FAILED to connect to 120:32334''\")",
"Bash(ssh wooo@192.168.0.110 \"curl -sf http://192.168.0.121:32334/api/v1/health 2>&1 || echo ''FAILED to connect to 121:32334''\")",
"Bash(ssh wooo@192.168.0.110 \"ssh wooo@192.168.0.120 ''cat /etc/rancher/k3s/k3s.yaml 2>/dev/null || echo No k3s.yaml''\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl get pods -n awoooi-prod -o wide | grep Running\")",
"Bash(ssh -o ConnectTimeout=5 wooo@192.168.0.120 \"ufw status 2>/dev/null || firewall-cmd --state 2>/dev/null || echo ''No firewall command found''\")",
"Bash(ssh -o ConnectTimeout=5 wooo@192.168.0.121 \"ufw status 2>/dev/null || firewall-cmd --state 2>/dev/null || echo ''No firewall command found''\")",
"Bash(pip3 show:*)",
"Bash(docker build:*)",
"Bash(docker version:*)",
"Bash(docker run:*)",
"Bash(curl -vI -H \"Origin: https://awoooi.wooo.work\" http://localhost:8889/api/v1/health)",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl get endpoints awoooi-api-svc -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl get pods -n awoooi-prod -o wide\")",
"Bash(ssh wooo@192.168.0.120 \"sudo -n ufw status 2>/dev/null || sudo -n iptables -L INPUT -n 2>/dev/null | head -20 || echo ''Need sudo for firewall check''\")",
"Bash(ssh wooo@192.168.0.120 \"ss -tln | grep -E ''32334|32335|:323'' || echo ''No NodePort listeners found''\")",
"Bash(ssh wooo@192.168.0.121 \"ss -tln | grep -E ''32334|32335|:323'' || echo ''No NodePort listeners found''\")",
"Bash(ssh wooo@192.168.0.120 \"ps aux | grep -E ''kube-proxy|k3s'' | grep -v grep | head -5\")",
"Bash(ssh wooo@192.168.0.120 \"cat /proc/sys/net/ipv4/ip_forward\")",
"Bash(ssh wooo@192.168.0.120 \"systemctl status k3s 2>/dev/null | head -15 || ps aux | grep ''k3s server'' | grep -v grep\")",
"Bash(ssh wooo@192.168.0.120 \"curl -sf --connect-timeout 5 http://127.0.0.1:32334/api/v1/health 2>&1 || echo ''LOCALHOST NodePort FAILED''\")",
"Bash(ssh wooo@192.168.0.120 \"curl -sf --connect-timeout 5 http://192.168.0.120:32334/api/v1/health 2>&1 || echo ''EXTERNAL IP NodePort FAILED''\")",
"Bash(ssh wooo@192.168.0.120 \"cat /etc/iptables/rules.v4 2>/dev/null || iptables-save 2>/dev/null | grep -E ''DROP|REJECT|32334|32335'' | head -10 || echo ''Cannot read iptables without sudo''\")",
"Bash(ssh wooo@192.168.0.121 \"curl -sf --connect-timeout 5 http://192.168.0.120:32334/api/v1/health 2>&1 || echo ''Worker->Master NodePort FAILED''\")",
"Bash(ssh wooo@192.168.0.120 \"cat /etc/rancher/k3s/config.yaml 2>/dev/null || ls -la /etc/rancher/k3s/ 2>/dev/null || echo ''No K3s config found''\")",
"Bash(ssh wooo@192.168.0.120 \"netstat -an 2>/dev/null | grep 32334 || ss -an | grep 32334 || echo ''No socket found for 32334''\")",
"Bash(ssh wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S iptables -L INPUT -n 2>&1 | head -20\")",
"Bash(ssh wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S iptables -t nat -L KUBE-NODEPORTS -n 2>&1 | head -20\")",
"Bash(ssh wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S iptables -L KUBE-ROUTER-INPUT -n 2>&1 | head -30\")",
"Bash(ssh wooo@192.168.0.120 \"echo ''0936223270'' | sudo -S iptables -t nat -L KUBE-NODEPORTS -n 2>&1 | grep -i awoooi || echo ''NO AWOOOI RULES FOUND''\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl get svc awoooi-api-svc -n awoooi-prod -o yaml | grep -A5 ''spec:''\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl get networkpolicy -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl apply -f - 2>&1\")",
"Bash(curl -sf --connect-timeout 10 https://awoooi.wooo.work/api/v1/health)",
"Bash(curl -skf --connect-timeout 10 https://awoooi.wooo.work/api/v1/health)",
"Bash(curl -sI https://awoooi.wooo.work/)",
"Bash(curl -skI https://awoooi.wooo.work/)",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl logs deployment/awoooi-api -n awoooi-prod --tail 50 2>&1\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl rollout restart deployment/awoooi-api -n awoooi-prod && /home/wooo/kubectl rollout status deployment/awoooi-api -n awoooi-prod --timeout=120s\")",
"Bash(curl -sf https://awoooi.wooo.work/api/v1/health)",
"Bash(curl -skf https://awoooi.wooo.work/api/v1/health)",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl logs deployment/awoooi-api -n awoooi-prod --tail 40 2>&1\")",
"Bash(for i:*)",
"Bash(do curl:*)",
"Bash(echo \"Request $i sent\")",
"Bash(done)",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl logs deployment/awoooi-api -n awoooi-prod --tail 100 2>&1\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl logs deployment/awoooi-api -n awoooi-prod --tail 30 2>&1\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl get configmap awoooi-config -n awoooi-prod -o yaml | grep OTEL\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl exec deployment/awoooi-api -n awoooi-prod -- env | grep OTEL\")",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl exec deployment/awoooi-api -n awoooi-prod -- python -c \"\"import socket; s=socket.socket\\(\\); s.settimeout\\(5\\); s.connect\\(\\(''192.168.0.188'', 24317\\)\\); print\\(''✅ Connection to 24317 OK''\\); s.close\\(\\)\"\" 2>&1\")",
"Bash(curl -vI https://awoooi.wooo.work)",
"Bash(curl -vI https://awoooi.wooo.work/api/v1/health)",
"Bash(curl -sf -X POST https://awoooi.wooo.work/api/v1/webhooks/signals -H \"Content-Type: application/json\" -d '{:*)",
"Bash(curl -s -X POST https://awoooi.wooo.work/api/v1/webhooks/signals -H \"Content-Type: application/json\" -d '{\"\"source\"\": \"\"prometheus\"\", \"\"severity\"\": \"\"P1\"\", \"\"message\"\": \"\"Test alert from CLI\"\"}')",
"Bash(curl -s -X POST https://awoooi.wooo.work/api/v1/webhooks/signals -H \"Content-Type: application/json\" -d '{:*)",
"Bash(ssh wooo@192.168.0.110 \"export KUBECONFIG=/home/wooo/.kube/config-120 && /home/wooo/kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath=''''{.data.WEBHOOK_HMAC_SECRET}'''' 2>/dev/null\")",
"Bash(timeout 15 curl -N -s https://awoooi.wooo.work/api/v1/dashboard/stream)",
"Bash(bash:*)",
"Bash(curl -s https://awoooi.wooo.work/api/v1/metrics/gold)",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT DISTINCT metric_name FROM signoz_metrics.distributed_samples_v4 WHERE unix_milli > \\(toUnixTimestamp\\(now\\(\\)\\) - 1800\\) * 1000 LIMIT 20 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT count\\(\\) as trace_count FROM signoz_traces.distributed_signoz_index_v2 WHERE timestamp > now\\(\\) - INTERVAL 30 MINUTE FORMAT TabSeparated\")",
"Bash(ssh wooo@192.168.0.120 \"KUBECONFIG=/home/wooo/.kube/config-120 /home/wooo/bin/kubectl get configmap awoooi-config -n awoooi-prod -o jsonpath=''{.data}'' | python3 -m json.tool 2>/dev/null | head -30\")",
"Bash(ssh wooo@192.168.0.120 \"KUBECONFIG=/home/wooo/.kube/config-120 /home/wooo/bin/kubectl logs deployment/awoooi-api -n awoooi-prod --tail 50 2>&1\")",
"Bash(ssh wooo@192.168.0.120 \"which kubectl || ls -la ~/bin/kubectl 2>/dev/null || ls -la /usr/local/bin/kubectl 2>/dev/null || echo ''kubectl not found''\")",
"Bash(ssh wooo@192.168.0.120 \"export KUBECONFIG=/home/wooo/.kube/config-120 && kubectl get configmap awoooi-config -n awoooi-prod -o jsonpath=''{.data}'' 2>&1\")",
"Bash(ssh wooo@192.168.0.120 \"ls -la ~/.kube/ 2>/dev/null; cat ~/.kube/config 2>/dev/null | head -20 || echo ''checking k3s default...''; sudo cat /etc/rancher/k3s/k3s.yaml 2>/dev/null | head -5 || echo ''no k3s config''\")",
"Bash(ssh wooo@192.168.0.120 \"sudo k3s kubectl get configmap awoooi-config -n awoooi-prod -o yaml 2>&1\")",
"Bash(ssh wooo@192.168.0.120 \"sudo k3s kubectl logs deployment/awoooi-api -n awoooi-prod --tail 100 2>&1\")",
"Bash(nc -zv 192.168.0.188 24317)",
"Bash(curl -s http://192.168.0.188:24318/v1/traces -X POST -H \"Content-Type: application/json\" -d '{}')",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT DISTINCT serviceName, count\\(\\) as cnt FROM signoz_traces.distributed_signoz_index_v2 WHERE timestamp > now\\(\\) - INTERVAL 24 HOUR GROUP BY serviceName ORDER BY cnt DESC LIMIT 20 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"DESCRIBE TABLE signoz_traces.distributed_signoz_index_v2 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT serviceName, count\\(\\) as cnt FROM signoz_traces.distributed_signoz_index_v2 WHERE timestamp > now\\(\\) - INTERVAL 5 MINUTE GROUP BY serviceName ORDER BY cnt DESC LIMIT 10 FORMAT TabSeparated\")",
"Bash(curl -s https://awoooi.wooo.work/api/v1/health)",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT serviceName, count\\(\\) as cnt FROM signoz_traces.distributed_signoz_index_v2 WHERE timestamp > now\\(\\) - INTERVAL 10 MINUTE GROUP BY serviceName ORDER BY cnt DESC LIMIT 10 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT service_name, count\\(\\) as cnt FROM signoz_logs.distributed_logs WHERE timestamp > now\\(\\) - INTERVAL 30 MINUTE GROUP BY service_name ORDER BY cnt DESC LIMIT 10 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SHOW TABLES FROM signoz_logs FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT count\\(\\) as total FROM signoz_logs.distributed_logs_v2 WHERE timestamp > now\\(\\) - INTERVAL 30 MINUTE FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT JSONExtractString\\(resources_string, ''service.name''\\) as svc, count\\(\\) as cnt FROM signoz_logs.distributed_logs_v2 WHERE timestamp > now\\(\\) - INTERVAL 5 MINUTE GROUP BY svc ORDER BY cnt DESC LIMIT 10 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"DESCRIBE TABLE signoz_logs.distributed_logs_v2 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT resources_string[''service.name''] as svc, count\\(\\) as cnt FROM signoz_logs.distributed_logs_v2 WHERE timestamp > \\(toUnixTimestamp64Nano\\(now64\\(\\)\\) - 300000000000\\) GROUP BY svc ORDER BY cnt DESC LIMIT 10 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT body, resources_string FROM signoz_logs.distributed_logs_v2 WHERE timestamp > \\(toUnixTimestamp64Nano\\(now64\\(\\)\\) - 60000000000\\) LIMIT 1 FORMAT JSONEachRow\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT serviceName, count\\(\\) as cnt FROM signoz_traces.distributed_signoz_index_v2 WHERE timestamp > now\\(\\) - INTERVAL 2 MINUTE GROUP BY serviceName ORDER BY cnt DESC LIMIT 10 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT serviceName, name, timestamp FROM signoz_traces.distributed_signoz_index_v2 WHERE timestamp > now\\(\\) - INTERVAL 5 MINUTE ORDER BY timestamp DESC LIMIT 5 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT serviceName, name, formatDateTime\\(timestamp, ''%Y-%m-%d %H:%M:%S''\\) as ts FROM signoz_traces.distributed_signoz_index_v2 ORDER BY timestamp DESC LIMIT 10 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT count\\(\\) FROM signoz_traces.distributed_signoz_index_v2 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT count\\(\\) FROM signoz_traces.distributed_signoz_spans FORMAT TabSeparated\")",
"Bash(ssh wooo@192.168.0.188 \"docker ps | grep -E ''otel|signoz''\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT metric_name, sum\\(value\\) as total FROM signoz_metrics.distributed_samples_v4 WHERE metric_name LIKE ''otelcol%span%'' AND unix_milli > \\(toUnixTimestamp\\(now\\(\\)\\) - 300\\) * 1000 GROUP BY metric_name FORMAT TabSeparated\")",
"Bash(for t:*)",
"Bash(do)",
"Bash(echo -n \"$t: \")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT count\\(\\) FROM signoz_traces.$t FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"SELECT serviceName, count\\(\\) as cnt FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp > now\\(\\) - INTERVAL 10 MINUTE GROUP BY serviceName ORDER BY cnt DESC LIMIT 10 FORMAT TabSeparated\")",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \":*)",
"Bash(curl -s 'http://192.168.0.188:8123/' --data \"DESCRIBE TABLE signoz_traces.distributed_signoz_index_v3 FORMAT TabSeparated\")",
"Bash(AWOOOI_API_URL=https://awoooi.wooo.work WEBHOOK_HMAC_SECRET=\"CHANGE_ME_TO_RANDOM_64_CHARS\" python scripts/fire_live_alert.py oomkilled)",
"Bash(timeout 10 curl -sN https://awoooi.wooo.work/api/v1/dashboard/stream)",
"Bash(curl -s https://awoooi.wooo.work/api/v1/dashboard)",
"Bash(npm list:*)",
"Bash(node scripts/verify-frontend.js)",
"Bash(node /Users/ogt/awoooi/scripts/verify-frontend.js)",
"Bash(python -c \"from src.services.proposal_service import ProposalService; print\\(''''✅ ProposalService OK''''\\)\")",
"Bash(python -c \"from src.services.openclaw import OpenClawService; print\\(''''✅ OpenClawService OK''''\\)\")",
"Bash(curl -s http://192.168.0.120:32334/api/v1/incidents)",
"Bash(jq -r \".incidents[:2] | .[] | \"\"\\\\\\(.incident_id\\) - \\\\\\(.status\\) - \\\\\\(.severity\\)\"\"\")",
"Bash(curl -s -X POST \"http://192.168.0.120:32334/api/v1/incidents/INC-20260322-4B3152/propose\" -H \"Content-Type: application/json\")",
"Bash(kubectl logs:*)",
"Bash(ssh ogt@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail 30\")",
"Bash(curl -sv -X POST \"http://192.168.0.120:32334/api/v1/incidents/INC-20260322-4B3152/propose\" -H \"Content-Type: application/json\")",
"Bash(curl -s http://192.168.0.120:32334/api/v1/health)",
"Bash(curl -s \"http://192.168.0.120:32334/api/v1/incidents/INC-20260322-4B3152\")",
"Bash(curl -sv \"http://192.168.0.120:32334/api/v1/incidents\")",
"Bash(curl -s --retry 3 --retry-delay 2 \"http://192.168.0.120:32334/api/v1/health\")",
"Bash(curl -s --retry 3 --retry-delay 2 http://192.168.0.120:32334/api/v1/health)",
"Bash(do echo:*)",
"Bash(curl -s -X POST \"https://awoooi.wooo.work/api/v1/incidents/INC-20260322-4B3152/propose\" -H \"Content-Type: application/json\")",
"Bash(curl -s -X POST \"https://awoooi.wooo.work/api/v1/incidents/INC-20260322-4B3152/proposal\" -H \"Content-Type: application/json\")",
"Bash(curl -s -X POST \"https://awoooi.wooo.work/api/v1/incidents/INC-20260322-D6C6A0/proposal\" -H \"Content-Type: application/json\")",
"Bash(curl -s http://192.168.0.120:32334/api/v1/approvals/pending)",
"Bash(kubectl get:*)",
"Bash(curl -s -w \"\\\\nHTTP_CODE: %{http_code}\\\\n\" http://192.168.0.120:32334/api/v1/health)",
"Bash(curl -s http://awoooi.wooo.work/api/v1/health)",
"Bash(curl -s http://awoooi.wooo.work/api/v1/approvals/pending)",
"Bash(curl -sL https://awoooi.wooo.work/api/v1/approvals/pending -k)",
"Bash(ssh root@192.168.0.120 \"kubectl get pods -n awoooi-prod -o wide\")",
"Bash(ssh root@192.168.0.120 \"kubectl logs -n awoooi-prod -l app=awoooi-api --tail=30\")",
"Bash(curl -sL https://awoooi.wooo.work/api/v1/timeline -k)",
"Bash(curl -sL https://awoooi.wooo.work/api/v1/incidents -k)",
"Bash(curl -sL \"https://awoooi.wooo.work/api/v1/approvals?include_history=true\" -k)",
"Bash(curl -sL \"https://awoooi.wooo.work/api/v1/incidents/INC-20260322-4B3152\" -k)",
"Bash(curl -sL \"https://awoooi.wooo.work/api/v1/audit-logs?limit=10\" -k)",
"Bash(curl -sL https://awoooi.wooo.work/api/v1/audit-logs?limit=10 -k)",
"Bash(ssh ogt@192.168.0.120 \"kubectl logs -n awoooi-prod -l app=awoooi-api --tail=100\")",
"Bash(ssh ogt@192.168.0.120 \"kubectl logs -n awoooi-prod -l app=awoooi-web --tail=50\")",
"Bash(ssh ogt@192.168.0.188 \"kubectl --kubeconfig=/etc/rancher/k3s/k3s.yaml logs -n awoooi-prod -l app=awoooi-api --tail=100 2>/dev/null || docker logs awoooi-api --tail=100 2>/dev/null\")",
"Bash(curl -sL \"https://awoooi.wooo.work/api/v1/approvals/pending\" -k -w \"\\\\n\\\\nHTTP: %{http_code}\\\\nTime: %{time_total}s\\\\n\")",
"Bash(curl -sL -X POST https://awoooi.wooo.work/api/v1/approvals/182e07c1-118a-49d7-b71c-7d33c5484d9b/sign -H 'Content-Type: application/json' -d '{\"\"\"\"signer_id\"\"\"\": \"\"\"\"test-debug\"\"\"\", \"\"\"\"signer_name\"\"\"\": \"\"\"\"Debug Test\"\"\"\", \"\"\"\"comment\"\"\"\": \"\"\"\"Testing\"\"\"\"}' -k)",
"Bash(curl -s https://wwooo.aiops.tw/api/v1/health)",
"Bash(curl -s https://wwooo.aiops.tw/api/v1/incidents?limit=5)",
"Bash(curl -s https://wwooo.aiops.tw/api/v1/approvals/pending)",
"Bash(curl -v -s \"https://wwooo.aiops.tw/api/v1/health\")",
"Bash(curl -s \"https://wwooo.aiops.tw/\")",
"Bash(curl -s --connect-timeout 5 \"http://192.168.0.120:32334/api/v1/health\")",
"Bash(curl -s --connect-timeout 5 \"http://192.168.0.120:32334/api/v1/incidents?limit=5\")",
"Bash(ssh -o ConnectTimeout=5 wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-worker-867f67f55d-kvdl2 -n awoooi-prod --tail=50\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod | grep -E ''NAME|worker''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod | grep worker\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-worker-5bdc5699bb-kcv9q -n awoooi-prod --tail=30\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get networkpolicy -n awoooi-prod -o wide\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod --show-labels | grep worker\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get networkpolicy allow-required-egress -n awoooi-prod -o yaml\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl patch networkpolicy allow-required-egress -n awoooi-prod --type=''json'' -p=''[{\"\"op\"\": \"\"replace\"\", \"\"path\"\": \"\"/spec/podSelector/matchLabels\"\", \"\"value\"\": {\"\"system\"\": \"\"awoooi\"\"}}]''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout restart deployment/awoooi-worker -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-worker-5bdc5699bb-kcv9q -n awoooi-prod --tail=15\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod --tail=40\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod 2>&1 | grep -E ''signal_worker|redis_pool|INFO'' | tail -10\")",
"Bash(ssh wooo@192.168.0.120 \"curl -s http://localhost:32334/api/v1/health\")",
"Bash(ssh wooo@192.168.0.120 'curl -s -X POST \"\"http://localhost:32334/api/v1/webhooks/signals\"\" -H \"\"Content-Type: application/json\"\" -d \"\"{:*)",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod | grep -E ''NAME|worker|api''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod && echo ''==='' && kubectl logs deployment/awoooi-worker -n awoooi-prod --tail=30\")",
"Bash(ssh wooo@192.168.0.120 \"curl -s http://localhost:32334/api/v1/incidents?limit=5\")",
"Bash(ssh wooo@192.168.0.120 \"curl -s http://localhost:32334/api/v1/approvals/pending\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod 2>&1 | head -50\")",
"Bash(ssh wooo@192.168.0.120 \"curl -s http://localhost:32334/api/v1/health | jq ''.components''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get secret -n awoooi-prod -o name\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath=''{.data.WEBHOOK_HMAC_SECRET}'' | base64 -d\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod --tail=20 2>&1 | grep -E ''signal|incident|telegram|INFO''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod --tail=30\")",
"Bash(ssh wooo@192.168.0.120 \"curl -s ''http://localhost:32334/api/v1/incidents?limit=5''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod 2>&1 | grep -iE ''telegram|notification|send'' | tail -10\")",
"Bash(ssh wooo@192.168.0.120 \"curl -s ''http://localhost:32334/api/v1/approvals/pending''\")",
"Bash(ssh wooo@192.168.0.120 \"curl -s ''http://localhost:32334/api/v1/incidents?limit=2'' && echo ''---'' && curl -s ''http://localhost:32334/api/v1/approvals/pending''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod | grep worker && echo ''---'' && kubectl logs deployment/awoooi-worker -n awoooi-prod --tail=30\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-worker-6b8cc94d9c-xjdwr -n awoooi-prod --tail=40\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get networkpolicy allow-required-egress -n awoooi-prod -o jsonpath=''{.spec.podSelector}''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl patch networkpolicy allow-required-egress -n awoooi-prod --type=''json'' -p=''[{\"\"op\"\": \"\"replace\"\", \"\"path\"\": \"\"/spec/podSelector\"\", \"\"value\"\": {\"\"matchLabels\"\": {\"\"system\"\": \"\"awoooi\"\"}}}]''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl delete pod awoooi-worker-6b8cc94d9c-xjdwr -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-worker-6b8cc94d9c-pmzj7 -n awoooi-prod --tail=30\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-worker-6b8cc94d9c-pmzj7 -n awoooi-prod --tail=20\")",
"Bash(ls -la /Users/ogt/awoooi/apps/api/scripts/fire*.py)",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod --tail=50\")",
"Bash(ssh wooo@192.168.0.120 \"curl -s ''http://localhost:32334/api/v1/incidents?limit=3''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod 2>&1 | grep -iE ''proposal|approval|llm|ai|ollama|generate'' | tail -20\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get deployment awoooi-worker -n awoooi-prod -o jsonpath=''{.spec.template.spec.containers[0].envFrom}''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get deployment awoooi-api -n awoooi-prod -o jsonpath=''{.spec.template.spec.containers[0].envFrom}''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get configmap awoooi-config -n awoooi-prod -o jsonpath=''''{.data}''''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath=''{.data}'' | tr '','' ''\\\\n''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl exec deployment/awoooi-api -n awoooi-prod -- python -c ''import os; print\\(os.getenv\\(\"\"DATABASE_URL\"\", \"\"NOT SET\"\"\\)[:50]\\)''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-api-75ffbfb88b-2htfh -n awoooi-prod --tail=50\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl exec awoooi-api-6687db5564-rv755 -n awoooi-prod -- env | grep DATABASE\")",
"Bash(ssh wooo@192.168.0.120 \"PGPASSWORD=''CHANGE_ME'' psql -h 192.168.0.188 -U awoooi -d awoooi_prod -c ''SELECT 1'' 2>&1 || echo ''Connection failed''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod\")",
"Bash(curl -sv http://192.168.0.120:32334/api/v1/health)",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-api-75ffbfb88b-2htfh -n awoooi-prod --tail=20 2>&1\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-worker-7fb7d5b55f-n48gk -n awoooi-prod --tail=20 2>&1\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get rs -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl scale rs awoooi-api-75ffbfb88b -n awoooi-prod --replicas=0\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl scale rs awoooi-worker-7fb7d5b55f -n awoooi-prod --replicas=0\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod --tail=10\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get deploy -n awoooi-prod -o wide\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get deploy awoooi-api -n awoooi-prod -o jsonpath=''{.spec.replicas}''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get deploy awoooi-worker -n awoooi-prod -o jsonpath=''{.spec.replicas}''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout status deployment/awoooi-api -n awoooi-prod --timeout=5s\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout history deployment/awoooi-api -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout undo deployment/awoooi-api -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout undo deployment/awoooi-worker -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout status deployment/awoooi-api -n awoooi-prod --timeout=30s\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get rs awoooi-api-6687db5564 -n awoooi-prod -o jsonpath=''{.metadata.annotations.deployment\\\\.kubernetes\\\\.io/revision}''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl delete pod awoooi-api-7f487f7cbb-5f88g -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout undo deployment/awoooi-api -n awoooi-prod --to-revision=46\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod --tail=15\")",
"Bash(curl -s http://192.168.0.120:32334/api/v1/incidents?limit=3)",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod --since=2m\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --since=2m | grep -i webhook\")",
"Bash(curl -sv -X POST http://192.168.0.120:32334/api/v1/webhooks/alertmanager -H \"Content-Type: application/json\" -d '{:*)",
"Bash(ssh wooo@192.168.0.120 \"kubectl get endpoints -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"curl -s http://localhost:32334/api/v1/health | jq ''{status}''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-worker -n awoooi-prod --since=30s\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-api-fc4744758-7wfv5 -n awoooi-prod --tail=30 2>&1\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-worker-6fc548887b-b9mtf -n awoooi-prod --tail=30 2>&1\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get configmap awoooi-config -n awoooi-prod -o yaml\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath=''''{.data}''''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pod awoooi-worker-6fc548887b-b9mtf -n awoooi-prod -o jsonpath=''{.metadata.labels}''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get networkpolicy -n awoooi-prod -o yaml\")",
"Bash(ssh wooo@192.168.0.120 'kubectl patch networkpolicy allow-required-egress -n awoooi-prod --type=json -p=\"\"[{\\\\\"\"op\\\\\"\": \\\\\"\"replace\\\\\"\", \\\\\"\"path\\\\\"\": \\\\\"\"/spec/podSelector/matchLabels\\\\\"\", \\\\\"\"value\\\\\"\": {\\\\\"\"system\\\\\"\": \\\\\"\"awoooi\\\\\"\"}}]\"\"')",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout restart deployment/awoooi-api deployment/awoooi-worker -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs awoooi-api-6c69b77894-d6jqq -n awoooi-prod --tail=20\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl run nc-test --rm -it --restart=Never --image=busybox -- nc -zv 192.168.0.188 5432\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get pods -n awoooi-prod -o=custom-columns=''NAME:.metadata.name,IMAGE:.spec.containers[0].image''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl exec awoooi-api-6687db5564-rv755 -n awoooi-prod -- ls -la *.db 2>/dev/null || echo ''No SQLite files''\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl exec awoooi-api-6687db5564-rv755 -n awoooi-prod -- env | grep -E ''MOCK|DATABASE|SQLITE''\")",
"Bash(curl -s \"http://192.168.0.120:32334/api/v1/approvals\")",
"Bash(python -m py_compile src/lewooogo_brain/engines/incident_engine.py src/lewooogo_brain/engines/proposal_engine.py src/lewooogo_brain/skills/loader.py)",
"Bash(python packages/lewooogo-brain/tests/test_skill_loader.py)",
"Bash(python packages/lewooogo-brain/tests/test_incident_engine.py)",
"Bash(python packages/lewooogo-brain/tests/test_guardrails.py)",
"Bash(python -m py_compile src/lewooogo_brain/engines/proposal_engine.py src/lewooogo_brain/engines/incident_engine.py src/lewooogo_brain/skills/loader.py)",
"Bash(PYTHONPATH=/Users/ogt/awoooi/packages/lewooogo-brain/src python -c \":*)",
"Bash(curl -s --connect-timeout 5 http://192.168.0.188:8000/api/v1/health)",
"Bash(curl -s \"https://awoooi.wooo.work/api/v1/approvals/pending\")",
"Bash(curl -s \"https://awoooi.wooo.work/api/v1/approvals?status=pending\")",
"Bash(curl -s \"https://awoooi.wooo.work/api/v1/incidents\")",
"Bash(uv sync:*)",
"Bash(python -c \"from src.routers.proposals import router; print\\(''✅ Router 語法驗證通過''\\)\")",
"Bash(curl -s -X GET \"https://awoooi.wooo.work/api/v1/health\" --connect-timeout 10)",
"Bash(curl -s -X GET \"https://awoooi.wooo.work/api/v1/incidents\" --connect-timeout 10)",
"Bash(curl -s -o /dev/null -w \"%{http_code}\" \"https://awoooi.wooo.work\" --connect-timeout 10)",
"Bash(curl -s -o /dev/null -w \"%{http_code}\" -L \"https://awoooi.wooo.work\" --connect-timeout 10)",
"Bash(curl -s -X POST \"https://awoooi.wooo.work/api/v1/incidents/test-123/propose\" -H \"Content-Type: application/json\" -d '{\"\"require_dry_run\"\": true}' --connect-timeout 10)",
"Bash(ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no ollama@192.168.0.120 \"kubectl get pods -n awoooi-prod -o wide\")",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get pods -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs awoooi-api-64c8659cff-grslz -n awoooi-prod --tail=50)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath='{.data.DATABASE_URL}')",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl rollout restart deployment/awoooi-api -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get pods -n awoooi-prod -l app=awoooi-api)",
"Bash(curl -s \"https://awoooi.wooo.work/api/v1/health\" --connect-timeout 10)",
"Bash(curl -s -o /dev/null -w \"%{http_code}\" -L \"https://awoooi.wooo.work/zh-TW\" --connect-timeout 10)",
"Bash(python -c \"from src.routers.proposals import router; print\\(''✅ Router import successful''\\)\")",
"Bash(PGPASSWORD=postgres psql -h 192.168.0.188 -U awoooi -d awoooi_dev -c \"SELECT incident_id, status, severity FROM incidents LIMIT 5;\")",
"Bash(PGPASSWORD=AwoooiProd2026 psql -h 192.168.0.188 -U awoooi -d awoooi_prod -c \"SELECT incident_id, status, severity FROM incidents LIMIT 5;\")",
"Bash(curl -sf http://192.168.0.120:32334/api/v1/incidents)",
"Bash(curl -v \"http://192.168.0.120:32334/api/v1/incidents\")",
"Bash(export KUBECONFIG=/Users/ogt/.kube/config-120)",
"Bash(curl -sI \"http://awoooi.wooo.work/\")",
"Bash(openssl s_client -servername awoooi.wooo.work -connect awoooi.wooo.work:443)",
"Bash(openssl x509:*)",
"Bash(curl -s -X POST \"http://192.168.0.120:32334/api/v1/incidents/INC-20260323-7DE10B/propose\" -H \"Content-Type: application/json\" -d '{\"\"\"\"require_dry_run\"\"\"\": true}')",
"Bash(python -c \"from src.services.executor import execute_approved_proposal, get_executor, ActionExecutor; print\\(''✅ Import successful''\\)\")",
"Bash(curl -s https://awoooi.woooo.cc/api/v1/incidents)",
"Bash(curl -s https://awoooi.woooo.cc/api/v1/health)",
"Bash(curl -s --connect-timeout 10 https://awoooi.woooo.cc/api/v1/health)",
"Bash(ssh ogt@192.168.70.202 \"sudo kubectl get pods -n awoooi 2>/dev/null\")",
"Bash(curl -s --connect-timeout 5 http://192.168.70.200:8000/api/v1/health)",
"Bash(ssh ogt@192.168.70.202 \"sudo kubectl get pods -n awoooi-prod\")",
"Bash(ssh -o StrictHostKeyChecking=no ogt@192.168.70.202 \"sudo kubectl get pods -n awoooi-prod\")",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get pods -A)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs -n awoooi-prod awoooi-worker-7479556d76-jbbps --tail 30)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs -n awoooi-prod -l app=awoooi-api --tail 20)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl exec -n awoooi-prod deployment/awoooi-api -- curl -s http://localhost:8000/api/v1/incidents)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl exec -n awoooi-prod deployment/awoooi-api -- python -c \"import httpx; r = httpx.get\\(''http://localhost:8000/api/v1/incidents''\\); print\\(r.text\\)\")",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get ingress -n awoooi-prod -o wide)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get svc -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get deployment awoooi-worker -n awoooi-prod -o jsonpath='{.spec.template.spec.containers[0].env}')",
"Bash(curl -s --connect-timeout 5 http://192.168.70.202:32334/api/v1/health)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl describe deployment awoooi-worker -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get configmap -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl describe deployment awoooi-api -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get configmap awoooi-config -n awoooi-prod -o yaml)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get secrets -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath='{.data}')",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath='{.data.REDIS_URL}')",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl rollout restart deployment/awoooi-worker -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get pods -n awoooi-prod -l app=awoooi-worker)",
"Bash(curl -s --connect-timeout 5 https://awoooi.wooo.work/api/v1/health)",
"Bash(curl -s https://awoooi.wooo.work/api/v1/incidents)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs -n awoooi-prod -l app=awoooi-worker --tail 10)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get svc -n wooo-aiops-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get svc -A)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs -n awoooi-prod awoooi-worker-76bdf9786d-rvtmz --tail 15)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl exec -n awoooi-prod deployment/awoooi-api -- python -c \"import os; print\\(os.getenv\\(''REDIS_URL'', ''NOT_SET''\\)\\)\")",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get deployment awoooi-api -n awoooi-prod -o yaml)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl rollout restart deployment/awoooi-api deployment/awoooi-worker -n awoooi-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs -n awoooi-prod awoooi-api-865cdc97db-6mpzz --tail 20)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get pods -n wooo-aiops-prod -l app=redis)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get pods -n wooo-aiops-prod)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl exec -n wooo-aiops-prod redis-6c6fcd64b8-8wznx -- redis-cli ping)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl exec -n awoooi-prod awoooi-api-6445c76797-mrl7p -- python -c \"import redis; r=redis.Redis\\(host=''10.43.239.47'', port=6379, db=10\\); print\\(r.ping\\(\\)\\)\")",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get networkpolicy -A)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get networkpolicy allow-required-egress -n awoooi-prod -o yaml)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl patch networkpolicy allow-required-egress -n awoooi-prod --type='json' -p='[{\"\"op\"\": \"\"add\"\", \"\"path\"\": \"\"/spec/egress/0/ports/-\"\", \"\"value\"\": {\"\"port\"\": 6379, \"\"protocol\"\": \"\"TCP\"\"}}]')",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs -n awoooi-prod awoooi-api-5fcc484b85-qpwt6 --tail 15)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl exec -n awoooi-prod awoooi-api-6445c76797-mrl7p -- python -c \"import os; print\\(''REDIS_URL:'', os.getenv\\(''REDIS_URL''\\)\\); import redis; r=redis.Redis.from_url\\(os.getenv\\(''REDIS_URL''\\)\\); print\\(''PING:'', r.ping\\(\\)\\)\")",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs -n awoooi-prod awoooi-worker-59d7588d75-p5tht --tail 20)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs -n awoooi-prod -l app=awoooi-worker --tail 30)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get deployment awoooi-worker -n awoooi-prod -o yaml)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get networkpolicy -n awoooi-prod -o wide)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl apply -f -)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs -n awoooi-prod awoooi-worker-6cd7dcbc9-5mtfq --tail 15)",
"Bash(jq .incidents[0])",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl get configmap awoooi-config -n awoooi-prod -o jsonpath='{.data.OPENCLAW_URL}')",
"Bash(curl -s --connect-timeout 5 http://192.168.0.188:8088/health)",
"Bash(curl -s --connect-timeout 5 http://192.168.0.188:8088/)",
"Bash(nc -zv 192.168.0.188 8088 -w 5)",
"Bash(ping -c 2 192.168.0.188)",
"Bash(ping -c 2 192.168.70.202)",
"Bash(grep -n \"mapToDualState\" /Users/ogt/awoooi/apps/web/src/app/[locale]/page.tsx -A 30)",
"Bash(head -40 /Users/ogt/awoooi/apps/web/src/app/[locale]/page.tsx)",
"Bash(ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no ollama@192.168.0.188 \"docker ps -a | grep -i claw; docker start openclaw 2>/dev/null || docker start clawbot 2>/dev/null || echo ''Container not found, listing all:'' && docker ps -a --format ''table {{.Names}}\\\\t{{.Status}}'' | head -10\")",
"Bash(curl -s --connect-timeout 5 http://192.168.0.188:8089/health)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl rollout status deployment/awoooi-web -n awoooi-prod --timeout=60s)",
"Bash(grep -rn \"clawbot\\\\|ClawBot\" /Users/ogt/awoooi/ --include=*.yaml --include=*.yml --include=*.json)",
"Bash(grep -rn \"ClawBot\\\\|clawbot\" /Users/ogt/awoooi/apps/ --include=*.py --include=*.ts --include=*.tsx)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs deployment/awoooi-api -n awoooi-prod --tail=100)",
"Bash(KUBECONFIG=/Users/ogt/awoooi/apps/api/k3s-prod.yaml kubectl logs deployment/awoooi-api -n awoooi-prod --tail=200)",
"Bash(export KUBECONFIG=/Users/ogt/awoooi/k3s-prod.yaml)",
"Bash(ssh root@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=200 2>&1 | grep -iE ''error|fail|exception|execute|background|parse'' | tail -40\")",
"Bash(curl -s https://awoooi.wooo.work/api/v1/approvals)",
"Bash(ssh k3s@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=200 2>&1 | grep -iE ''error|fail|execute|background|parse'' | tail -40\")",
"Bash(ssh ubuntu@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=200 2>&1 | grep -iE ''error|fail|execute|background|parse'' | tail -40\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=200 2>&1 | grep -iE ''error|fail|execute|background|parse|skip'' | tail -50\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=500 2>&1 | grep -iE ''background_execution|approve_action|reject|k8s_executor'' | tail -30\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl get deploy,sts -n awoooi-prod\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl rollout status deployment/awoooi-api -n awoooi-prod --timeout=120s 2>&1\")",
"Bash(ssh wooo@192.168.0.120 \"kubectl logs deployment/awoooi-api -n awoooi-prod --tail=50 2>&1 | grep -iE ''background_execution|k8s_executor|parse'' | tail -10\")"
],
"additionalDirectories": [
"/Users/ogt/awoooi/docs",
"/Users/ogt/.claude/projects/-Users-ogt-awoooi/memory",
"/Users/ogt/awoooi/apps/web/src/app",
"/Users/ogt/awoooi/apps/api",
"/Users/ogt/awoooi/apps/api/http:/localhost:8000/api/v1",
"/Users/ogt/awoooi/apps/web/public",
"/Users/ogt/Downloads",
"/Users/ogt/awoooi/apps/web/test-results",
"/Users/ogt/awoooi",
"/Users/ogt/awoooi/apps/web/src/app/[locale]",
"/tmp"
]
}
}

View File

@@ -19,10 +19,18 @@
# 文件與腳本(不需要進 image
# 注意: docs/runbooks/, docs/adr/, .agents/skills/ 供 RAG 索引 (ADR-067 Phase 33)
# scripts/ 大部分不需要進 image但 CronJob 腳本需要
# scripts/ 大部分不需要進 image僅白名單 production runtime/ops 種子腳本
# 2026-04-12 ogt (ADR-073 P2-1): 白名單允許 cron_km_vectorize.py
scripts
# 2026-05-13 codex: 白名單 T16 auto-repair canary PlayBook seed script
# 2026-05-31 codex: MOMO backup Ansible playbook copies the backup script from
# the controller image; keep only this backup script in the runtime context.
scripts/**
!scripts/
!scripts/cron_km_vectorize.py
!scripts/backup/
!scripts/backup/backup-momo-188-pg.sh
!scripts/ops/
!scripts/ops/awooop-seed-auto-repair-canary-playbook.py
# Node 快取monorepo 根目錄)
node_modules

View File

@@ -10,7 +10,7 @@ on:
jobs:
lint:
runs-on: self-hosted
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

View File

@@ -43,10 +43,19 @@ jobs:
├ 📝 ${{ steps.commit.outputs.message }}
├ 🔖 <code>${{ steps.commit.outputs.short_sha }}</code>
└ 🌿 dev branch"
printf '%b' "$MSG" | curl -fS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
-d "chat_id=${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
-d "parse_mode=HTML" \
--data-urlencode "text@-"
if AWOOI_CICD_STATUS=running \
AWOOI_CICD_STAGE=dev-deploy \
AWOOI_CICD_JOB_NAME="[DEV] 部署開始" \
AWOOI_CICD_COMMIT_SHA="${GITHUB_SHA}" \
AWOOI_CICD_SUMMARY="${{ steps.commit.outputs.message }}" \
scripts/ci/notify-awoooi-cicd.sh; then
echo "Dev deploy start notification mirrored through AWOOI API"
else
printf '%b' "$MSG" | curl -fS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
-d "chat_id=${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
-d "parse_mode=HTML" \
--data-urlencode "text@-"
fi
# API 測試 (同 prod CI確保 dev 也通過)
- name: Run API Tests
@@ -78,11 +87,18 @@ jobs:
echo "✅ API 測試通過"
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: ${{ env.HARBOR }}
username: ${{ secrets.HARBOR_USERNAME }}
password: ${{ secrets.HARBOR_PASSWORD }}
run: |
HARBOR_USERNAME="$(cat <<'AWOOOI_SECRET_HARBOR_USERNAME'
${{ secrets.HARBOR_USERNAME }}
AWOOOI_SECRET_HARBOR_USERNAME
)"
HARBOR_PASSWORD="$(cat <<'AWOOOI_SECRET_HARBOR_PASSWORD'
${{ secrets.HARBOR_PASSWORD }}
AWOOOI_SECRET_HARBOR_PASSWORD
)"
printf '%s' "$HARBOR_PASSWORD" | docker login "${{ env.HARBOR }}" \
-u "$HARBOR_USERNAME" \
--password-stdin
# Dev API 鏡像:強制重建,不用 cache確保 models.json 等配置文件更新)
- name: Build and Push API (Dev)
@@ -98,16 +114,37 @@ jobs:
# 注入 Dev K8s Secrets
- name: Inject Dev K8s Secrets
env:
SSH_PRIVATE_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
TG_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
TG_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }}
NVIDIA_API_KEY: ${{ secrets.NVIDIA_API_KEY }}
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
run: |
secret_b64() {
python3 -c 'import base64, sys; data=sys.stdin.buffer.read(); data=data[:-1] if data.endswith(b"\n") else data; sys.stdout.write(base64.b64encode(data).decode())'
}
write_deploy_key() {
mkdir -p ~/.ssh
umask 077
cat > ~/.ssh/deploy_key <<'AWOOOI_DEPLOY_KEY'
${{ secrets.DEPLOY_SSH_KEY }}
AWOOOI_DEPLOY_KEY
chmod 600 ~/.ssh/deploy_key
}
TG_BOT_TOKEN_B64="$(secret_b64 <<'AWOOOI_SECRET_TG_BOT_TOKEN'
${{ secrets.TELEGRAM_BOT_TOKEN }}
AWOOOI_SECRET_TG_BOT_TOKEN
)"
TG_CHAT_ID_B64="$(secret_b64 <<'AWOOOI_SECRET_TG_CHAT_ID'
${{ secrets.TELEGRAM_CHAT_ID }}
AWOOOI_SECRET_TG_CHAT_ID
)"
NVIDIA_API_KEY_B64="$(secret_b64 <<'AWOOOI_SECRET_NVIDIA_API_KEY'
${{ secrets.NVIDIA_API_KEY }}
AWOOOI_SECRET_NVIDIA_API_KEY
)"
GEMINI_API_KEY_B64="$(secret_b64 <<'AWOOOI_SECRET_GEMINI_API_KEY'
${{ secrets.GEMINI_API_KEY }}
AWOOOI_SECRET_GEMINI_API_KEY
)"
mkdir -p ~/.ssh
echo "$SSH_PRIVATE_KEY" > ~/.ssh/deploy_key
chmod 600 ~/.ssh/deploy_key
write_deploy_key
# 2026-05-05 Codex: kubectl runs on 120 control-plane. 121 is a
# worker and its local kubeconfig points at 127.0.0.1:6443.
ssh -o StrictHostKeyChecking=no -i ~/.ssh/deploy_key wooo@192.168.0.120 << SECRETS
@@ -115,19 +152,19 @@ jobs:
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
sudo kubectl patch secret awoooi-secrets -n awoooi-dev --type='json' -p='[
{"op":"replace","path":"/data/OPENCLAW_TG_BOT_TOKEN","value":"'"$(echo -n "${TG_BOT_TOKEN}" | base64 -w 0)"'"},
{"op":"replace","path":"/data/OPENCLAW_TG_CHAT_ID","value":"'"$(echo -n "${TG_CHAT_ID}" | base64 -w 0)"'"}
{"op":"replace","path":"/data/OPENCLAW_TG_BOT_TOKEN","value":"${TG_BOT_TOKEN_B64}"},
{"op":"replace","path":"/data/OPENCLAW_TG_CHAT_ID","value":"${TG_CHAT_ID_B64}"}
]' || echo "⚠️ Telegram Secrets patch 跳過"
if [ -n "${NVIDIA_API_KEY}" ]; then
if [ -n "${NVIDIA_API_KEY_B64}" ]; then
sudo kubectl patch secret awoooi-secrets -n awoooi-dev --type='json' -p='[
{"op":"replace","path":"/data/NVIDIA_API_KEY","value":"'"$(echo -n "${NVIDIA_API_KEY}" | base64 -w 0)"'"}
{"op":"replace","path":"/data/NVIDIA_API_KEY","value":"${NVIDIA_API_KEY_B64}"}
]' && echo "✅ NVIDIA_API_KEY 已注入 dev"
fi
if [ -n "${GEMINI_API_KEY}" ]; then
if [ -n "${GEMINI_API_KEY_B64}" ]; then
sudo kubectl patch secret awoooi-secrets -n awoooi-dev --type='json' -p='[
{"op":"replace","path":"/data/GEMINI_API_KEY","value":"'"$(echo -n "${GEMINI_API_KEY}" | base64 -w 0)"'"}
{"op":"replace","path":"/data/GEMINI_API_KEY","value":"${GEMINI_API_KEY_B64}"}
]' && echo "✅ GEMINI_API_KEY 已注入 dev"
fi
@@ -136,8 +173,6 @@ jobs:
# 部署到 awoooi-dev
- name: Deploy to Dev K8s
env:
SSH_PRIVATE_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
run: |
cat k8s/awoooi-dev/02-configmap.yaml | \
ssh -o StrictHostKeyChecking=no -i ~/.ssh/deploy_key wooo@192.168.0.120 \
@@ -184,10 +219,20 @@ jobs:
├ 🔖 <code>${{ steps.commit.outputs.short_sha }}</code>
├ ⏱️ 耗時: ${MINUTES}m ${SECONDS}s
└ 🩺 http://192.168.0.125:32344/api/v1/health"
printf '%b' "$MSG" | curl -fS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
-d "chat_id=${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
-d "parse_mode=HTML" \
--data-urlencode "text@-"
if AWOOI_CICD_STATUS=success \
AWOOI_CICD_STAGE=dev-deploy \
AWOOI_CICD_JOB_NAME="[DEV] 部署完成" \
AWOOI_CICD_COMMIT_SHA="${GITHUB_SHA}" \
AWOOI_CICD_DURATION_SECONDS="${DURATION}" \
AWOOI_CICD_SUMMARY="${{ steps.commit.outputs.message }}" \
scripts/ci/notify-awoooi-cicd.sh; then
echo "Dev deploy success notification mirrored through AWOOI API"
else
printf '%b' "$MSG" | curl -fS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
-d "chat_id=${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
-d "parse_mode=HTML" \
--data-urlencode "text@-"
fi
- name: Notify Dev Deploy Failure
if: failure()
@@ -196,7 +241,16 @@ jobs:
├ 📝 ${{ steps.commit.outputs.message }}
├ 🔖 <code>${{ steps.commit.outputs.short_sha }}</code>
└ 🔗 <a href=\"http://192.168.0.110:3001/wooo/awoooi/actions\">查看日誌</a>"
printf '%b' "$MSG" | curl -fS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
-d "chat_id=${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
-d "parse_mode=HTML" \
--data-urlencode "text@-"
if AWOOI_CICD_STATUS=failed \
AWOOI_CICD_STAGE=dev-deploy \
AWOOI_CICD_JOB_NAME="[DEV] 部署失敗" \
AWOOI_CICD_COMMIT_SHA="${GITHUB_SHA}" \
AWOOI_CICD_SUMMARY="${{ steps.commit.outputs.message }}" \
scripts/ci/notify-awoooi-cicd.sh; then
echo "Dev deploy failure notification mirrored through AWOOI API"
else
printf '%b' "$MSG" | curl -fS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
-d "chat_id=${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
-d "parse_mode=HTML" \
--data-urlencode "text@-"
fi

File diff suppressed because it is too large Load Diff

View File

@@ -30,6 +30,9 @@ jobs:
with:
fetch-depth: 50
- name: Guard Workflow Secret Surfaces
run: node scripts/ci/check-gitea-step-env-secrets.js
- name: Skip Stale Main Push
id: stale
run: |
@@ -102,7 +105,6 @@ jobs:
- name: Notify Code Review Start
if: steps.stale.outputs.skip != 'true'
env:
TG_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
TG_CHAT_ID: ${{ env.TELEGRAM_ALERT_CHAT_ID }}
SHORT_SHA: ${{ steps.ctx.outputs.short_sha }}
BRANCH: ${{ steps.ctx.outputs.branch }}
@@ -110,18 +112,33 @@ jobs:
FILES_DISPLAY: ${{ steps.ctx.outputs.files_display }}
run: |
set -euo pipefail
if [ -z "${TG_BOT_TOKEN:-}" ] || [ -z "${TG_CHAT_ID:-}" ]; then
echo "Telegram secret missing; skip start notification"
exit 0
fi
TG_BOT_TOKEN="$(cat <<'AWOOOI_SECRET_TG_BOT_TOKEN'
${{ secrets.TELEGRAM_BOT_TOKEN }}
AWOOOI_SECRET_TG_BOT_TOKEN
)"
html_escape() { sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g'; }
COMMIT_ESC="$(printf '%s' "$COMMIT_MSG" | html_escape)"
FILES_ESC="$(printf '%s\n' "$FILES_DISPLAY" | html_escape)"
MSG="$(printf '🔍 <b>Code Review 啟動</b>\n──────────────────────\n📦 Commit <code>%s</code> 🌿 <code>%s</code>\n📝 <code>%s</code>\n📁 <b>變更檔案:</b>\n%s\n──────────────────────\n🤖 <b>Hermes → OpenClaw → Elephant Alpha → NemoTron</b>\n📊 即時進度:<a href=\"%s\">%s</a>' "$SHORT_SHA" "$BRANCH" "$COMMIT_ESC" "$FILES_ESC" "$REPORT_URL" "$REPORT_URL")"
curl -fsS -X POST "https://api.telegram.org/bot${TG_BOT_TOKEN}/sendMessage" \
-H "Content-Type: application/json" \
-d "$(jq -n --arg c "$TG_CHAT_ID" --arg t "$MSG" '{chat_id:$c,text:$t,parse_mode:"HTML",disable_web_page_preview:true}')" \
>/dev/null
if AWOOI_CICD_STATUS=running \
AWOOI_CICD_STAGE=code-review \
AWOOI_CICD_JOB_NAME="Code Review 啟動" \
AWOOI_CICD_COMMIT_SHA="${GITHUB_SHA}" \
AWOOI_CICD_TRIGGERED_BY="${GITHUB_ACTOR:-CI}" \
AWOOI_CICD_SUMMARY="${COMMIT_MSG}" \
AWOOI_CICD_WORKFLOW_URL="${REPORT_URL}" \
scripts/ci/notify-awoooi-cicd.sh; then
echo "Code review start notification mirrored through AWOOI API"
else
if [ -z "${TG_BOT_TOKEN:-}" ] || [ -z "${TG_CHAT_ID:-}" ]; then
echo "Telegram secret missing and AWOOI API notify failed; skip start notification"
exit 0
fi
curl -fsS -X POST "https://api.telegram.org/bot${TG_BOT_TOKEN}/sendMessage" \
-H "Content-Type: application/json" \
-d "$(jq -n --arg c "$TG_CHAT_ID" --arg t "$MSG" '{chat_id:$c,text:$t,parse_mode:"HTML",disable_web_page_preview:true}')" \
>/dev/null
fi
- name: Run Deterministic Review
if: steps.stale.outputs.skip != 'true'
@@ -139,15 +156,14 @@ jobs:
- name: Notify Code Review Completion
if: always() && steps.stale.outputs.skip != 'true'
env:
TG_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
TG_CHAT_ID: ${{ env.TELEGRAM_ALERT_CHAT_ID }}
SHORT_SHA: ${{ steps.ctx.outputs.short_sha }}
run: |
set -euo pipefail
if [ -z "${TG_BOT_TOKEN:-}" ] || [ -z "${TG_CHAT_ID:-}" ]; then
echo "Telegram secret missing; skip completion notification"
exit 0
fi
TG_BOT_TOKEN="$(cat <<'AWOOOI_SECRET_TG_BOT_TOKEN'
${{ secrets.TELEGRAM_BOT_TOKEN }}
AWOOOI_SECRET_TG_BOT_TOKEN
)"
REPORT=/tmp/code-review-report.json
if [ ! -s "$REPORT" ]; then
cat > "$REPORT" <<'JSON'
@@ -180,7 +196,25 @@ jobs:
TOP_ESC="$(printf '%s' "$TOP_ISSUE" | html_escape)"
MSG="$(printf '%s <b>Code Review 完成・%s</b>\n──────────────────────\n🔴 CRITICAL <code>%s</code> 🟠 HIGH <code>%s</code> 🟡 MEDIUM <code>%s</code> 🟢 LOW <code>%s</code>\n──────────────────────\n⚠ <b>主要問題</b>\n%s\n\n🔍 <b>整體風險等級</b>\n%s%s\n\n⚠ <b>最高關注問題</b>\n1. %s\n──────────────────────\n🤖 Elephant Alpha<b>%s</b> ✅ %s\n📊 完整報告:<a href=\"%s\">%s</a>' "$STATUS" "$SHORT_SHA" "$CRITICAL" "$HIGH" "$MEDIUM" "$LOW" "$ISSUE_LINE" "$RISK" "$SUMMARY_ESC" "$TOP_ESC" "$RISK" "$ACTION_ESC" "$REPORT_URL" "$REPORT_URL")"
curl -fsS -X POST "https://api.telegram.org/bot${TG_BOT_TOKEN}/sendMessage" \
-H "Content-Type: application/json" \
-d "$(jq -n --arg c "$TG_CHAT_ID" --arg t "$MSG" '{chat_id:$c,text:$t,parse_mode:"HTML",disable_web_page_preview:true}')" \
>/dev/null
CICD_STATUS=success
if [ "$RISK" = "MEDIUM" ]; then CICD_STATUS=pending; fi
if [ "$RISK" = "HIGH" ] || [ "$RISK" = "CRITICAL" ]; then CICD_STATUS=failed; fi
if AWOOI_CICD_STATUS="${CICD_STATUS}" \
AWOOI_CICD_STAGE=code-review \
AWOOI_CICD_JOB_NAME="Code Review 完成・${RISK}" \
AWOOI_CICD_COMMIT_SHA="${GITHUB_SHA}" \
AWOOI_CICD_TRIGGERED_BY="${GITHUB_ACTOR:-CI}" \
AWOOI_CICD_SUMMARY="CRITICAL=${CRITICAL}; HIGH=${HIGH}; MEDIUM=${MEDIUM}; LOW=${LOW}; ${SUMMARY}" \
AWOOI_CICD_WORKFLOW_URL="${REPORT_URL}" \
scripts/ci/notify-awoooi-cicd.sh; then
echo "Code review completion notification mirrored through AWOOI API"
else
if [ -z "${TG_BOT_TOKEN:-}" ] || [ -z "${TG_CHAT_ID:-}" ]; then
echo "Telegram secret missing and AWOOI API notify failed; skip completion notification"
exit 0
fi
curl -fsS -X POST "https://api.telegram.org/bot${TG_BOT_TOKEN}/sendMessage" \
-H "Content-Type: application/json" \
-d "$(jq -n --arg c "$TG_CHAT_ID" --arg t "$MSG" '{chat_id:$c,text:$t,parse_mode:"HTML",disable_web_page_preview:true}')" \
>/dev/null
fi

View File

@@ -1,7 +1,7 @@
# =============================================================================
# Deploy Prometheus Alert Rules (獨立 workflow)
# 2026-04-05 Claude Code (ADR-039 I3): 從 cd.yaml 分離
# 觸發條件: ops/monitoring/alerts-unified.yml 有變更 或 workflow_dispatch
# 觸發條件: ops/monitoring/alerts-unified.yml / slo-rules.yml 有變更 或 workflow_dispatch
# 說明: 告警規則部署不依賴應用構建,獨立觸發以加快響應速度
# =============================================================================
@@ -12,6 +12,8 @@ on:
branches: [main]
paths:
- 'ops/monitoring/alerts-unified.yml'
- 'ops/monitoring/slo-rules.yml'
- 'scripts/ops/deploy-alerts.sh'
workflow_dispatch:
env:
@@ -30,11 +32,15 @@ jobs:
run: |
pip3 install -q pyyaml 2>/dev/null || pip install -q pyyaml
python3 -c "import yaml; yaml.safe_load(open('ops/monitoring/alerts-unified.yml')); print('YAML OK')"
python3 -c "import yaml; yaml.safe_load(open('ops/monitoring/slo-rules.yml')); print('SLO YAML OK')"
- name: Setup SSH key
run: |
mkdir -p ~/.ssh
echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/id_ed25519
umask 077
cat > ~/.ssh/id_ed25519 <<'AWOOOI_DEPLOY_KEY'
${{ secrets.DEPLOY_SSH_KEY }}
AWOOOI_DEPLOY_KEY
chmod 600 ~/.ssh/id_ed25519
ssh-keyscan 192.168.0.110 >> ~/.ssh/known_hosts
@@ -50,6 +56,17 @@ jobs:
SHORT_SHA="${{ github.sha }}"
SHORT_SHA="${SHORT_SHA:0:7}"
MSG="${EMOJI} Prometheus 告警規則部署 ${STATUS} (${SHORT_SHA})"
curl -fS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
-d "chat_id=${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
--data-urlencode "text=${MSG}" || true
CICD_STATUS="success"
[ "$STATUS" != "success" ] && CICD_STATUS="failed"
if AWOOI_CICD_STATUS="${CICD_STATUS}" \
AWOOI_CICD_STAGE=deploy-alerts \
AWOOI_CICD_JOB_NAME="Prometheus 告警規則部署" \
AWOOI_CICD_COMMIT_SHA="${{ github.sha }}" \
AWOOI_CICD_SUMMARY="${MSG}" \
scripts/ci/notify-awoooi-cicd.sh; then
echo "Alert rule deploy notification mirrored through AWOOI API"
else
curl -fS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
-d "chat_id=${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
--data-urlencode "text=${MSG}" || true
fi

View File

@@ -51,10 +51,52 @@ jobs:
echo "status=failed" >> $GITHUB_OUTPUT
exit 1
- name: Source Provider Freshness Smoke
run: |
SOURCE_CANARY_RUN_REF="gitea-e2e-${GITHUB_RUN_ID:-manual}-${GITHUB_RUN_ATTEMPT:-1}"
echo "SOURCE_CANARY_RUN_REF=${SOURCE_CANARY_RUN_REF}" >> "$GITHUB_ENV"
echo "SOURCE_LINK_CANARY_WORK_ITEM_ID=source-evidence:sentry:upstream_canary:awoooi-source-link-canary-${SOURCE_CANARY_RUN_REF}" >> "$GITHUB_ENV"
OPERATOR_KEY="$(cat <<'AWOOOI_SECRET_AWOOOP_OPERATOR_API_KEY'
${{ secrets.AWOOOP_OPERATOR_API_KEY }}
AWOOOI_SECRET_AWOOOP_OPERATOR_API_KEY
)"
AWOOOP_OPERATOR_API_KEY="${OPERATOR_KEY}" \
AWOOOP_OPERATOR_ID=gitea-e2e-health \
python3 scripts/alert_chain_smoke_test.py \
--api-url https://awoooi.wooo.work \
--metrics-api-url http://192.168.0.125:32334 \
--source-provider-heartbeat \
--source-provider-upstream-canary \
--run-ref "${SOURCE_CANARY_RUN_REF}" \
--source-link-canary-target-incident-id INC-20260505-25E744 \
--json
- name: Source Correlation Applied-Link Smoke
run: |
python3 scripts/awooop_source_correlation_apply_smoke.py \
--api-url https://awoooi.wooo.work \
--target-incident-id INC-20260505-25E744 \
--allow-existing-apply \
--refresh-if-stale-days 6 \
--refresh-work-item-id "${SOURCE_LINK_CANARY_WORK_ITEM_ID}" \
--verify-refresh-candidate \
--reviewer-id gitea_e2e_source_link_canary \
--operator-note "T124 dedicated source-link canary refresh; append-only status-chain proof"
- name: Notify Telegram on Failure
if: failure()
run: |
curl -s -X POST "https://api.telegram.org/bot${{ secrets.OPENCLAW_TG_BOT_TOKEN }}/sendMessage" \
-d chat_id="${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
-d parse_mode="HTML" \
-d text="🔴 <b>[E2E Health Check]</b> 失敗%0A%0A📅 $(TZ=Asia/Taipei date '+%Y-%m-%d %H:%M')%0A🔗 API 健康檢查未通過%0A%0A請檢查 K3s 叢集狀態"
MSG="E2E Health Check 失敗API 健康檢查未通過"
if AWOOI_CICD_STATUS=failed \
AWOOI_CICD_STAGE=e2e-health \
AWOOI_CICD_JOB_NAME="E2E Health Check" \
AWOOI_CICD_COMMIT_SHA="${{ github.sha }}" \
AWOOI_CICD_SUMMARY="${MSG}" \
scripts/ci/notify-awoooi-cicd.sh; then
echo "E2E failure notification mirrored through AWOOI API"
else
curl -s -X POST "https://api.telegram.org/bot${{ secrets.OPENCLAW_TG_BOT_TOKEN }}/sendMessage" \
-d chat_id="${{ env.TELEGRAM_ALERT_CHAT_ID }}" \
-d parse_mode="HTML" \
-d text="🔴 <b>[E2E Health Check]</b> 失敗%0A%0A📅 $(TZ=Asia/Taipei date '+%Y-%m-%d %H:%M')%0A🔗 API 健康檢查未通過%0A%0A請檢查 K3s 叢集狀態"
fi

View File

@@ -57,23 +57,34 @@ jobs:
- name: Identify new migrations
id: diff
run: |
NEW_FILES=$(git diff --name-only --diff-filter=A HEAD~1 HEAD -- 'apps/api/migrations/*.sql' || true)
ALL_NEW_FILES=$(git diff --no-renames --name-only --diff-filter=A HEAD~1 HEAD -- 'apps/api/migrations/*.sql' || true)
NEW_FILES=$(echo "$ALL_NEW_FILES" | grep -Ev '(_down|rollback)\.sql$' || true)
SKIPPED_ROLLBACK_FILES=$(echo "$ALL_NEW_FILES" | grep -E '(_down|rollback)\.sql$' || true)
echo "new_files<<EOF" >> $GITHUB_OUTPUT
echo "$NEW_FILES" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
echo "=== New migration files ==="
echo "$NEW_FILES"
if [ -n "$SKIPPED_ROLLBACK_FILES" ]; then
echo "=== Rollback/down migrations skipped by design ==="
echo "$SKIPPED_ROLLBACK_FILES"
fi
- name: Apply new migrations
if: steps.diff.outputs.new_files != ''
env:
# 從 Gitea secrets 取,不直接明碼輸出。
# MIGRATION_DATABASE_URL 是限權帳號DATABASE_URL 只在 PostgreSQL
# 明確回報「必須是 table owner」時作為受控 fallback。
PGURL: ${{ secrets.MIGRATION_DATABASE_URL }}
OWNER_PGURL: ${{ secrets.DATABASE_URL }}
run: |
set -euo pipefail
# 從 Gitea secrets 取,不放 step-level env避免 runner log 展開。
# MIGRATION_DATABASE_URL 是限權帳號DATABASE_URL 只在 PostgreSQL
# 明確回報「必須是 table owner」時作為受控 fallback。
PGURL="$(cat <<'AWOOOI_SECRET_MIGRATION_DATABASE_URL'
${{ secrets.MIGRATION_DATABASE_URL }}
AWOOOI_SECRET_MIGRATION_DATABASE_URL
)"
OWNER_PGURL="$(cat <<'AWOOOI_SECRET_DATABASE_URL'
${{ secrets.DATABASE_URL }}
AWOOOI_SECRET_DATABASE_URL
)"
if [ -z "$PGURL" ]; then
echo "::error::MIGRATION_DATABASE_URL secret not set in Gitea"
exit 1
@@ -96,7 +107,7 @@ jobs:
echo "=== Applying: $file ==="
migration_err="$(mktemp)"
if ! apply_migration "$PGURL_PSQL" "$file" 2>"$migration_err"; then
if grep -q "must be owner of table" "$migration_err"; then
if grep -Eq "(must be owner of table|permission denied for table)" "$migration_err"; then
if [ -z "$OWNER_PGURL_PSQL" ]; then
cat "$migration_err" >&2
echo "::error::migration requires table owner but DATABASE_URL secret is not set"
@@ -115,11 +126,16 @@ jobs:
- name: Seed asset_discovery_run (audit)
if: steps.diff.outputs.new_files != ''
env:
PGURL: ${{ secrets.MIGRATION_DATABASE_URL }}
OWNER_PGURL: ${{ secrets.DATABASE_URL }}
run: |
set -euo pipefail
PGURL="$(cat <<'AWOOOI_SECRET_MIGRATION_DATABASE_URL'
${{ secrets.MIGRATION_DATABASE_URL }}
AWOOOI_SECRET_MIGRATION_DATABASE_URL
)"
OWNER_PGURL="$(cat <<'AWOOOI_SECRET_DATABASE_URL'
${{ secrets.DATABASE_URL }}
AWOOOI_SECRET_DATABASE_URL
)"
if [ -z "$PGURL" ]; then
echo "::error::MIGRATION_DATABASE_URL secret not set in Gitea"
exit 1
@@ -127,14 +143,15 @@ jobs:
PGURL_PSQL="${PGURL/postgresql+asyncpg:\/\//postgresql:\/\/}"
OWNER_PGURL_PSQL="${OWNER_PGURL/postgresql+asyncpg:\/\//postgresql:\/\/}"
FILES_JSON=$(echo "${{ steps.diff.outputs.new_files }}" | jq -Rn '[inputs | select(length > 0)]')
SUMMARY_JSON=$(jq -cn \
--arg commit_sha "${{ github.sha }}" \
--argjson files "$FILES_JSON" \
'{type: "ci_migration", commit_sha: $commit_sha, files: $files}')
SUMMARY_JSON_SQL=${SUMMARY_JSON//\'/\'\'}
seed_audit() {
local url="$1"
psql "$url" \
-v ON_ERROR_STOP=1 \
-v commit_sha="${{ github.sha }}" \
-v files_json="$FILES_JSON" \
-c "
psql "$url" -v ON_ERROR_STOP=1 <<SQL
INSERT INTO asset_discovery_run (
run_id, triggered_by, scope, scan_depth, status,
started_at, ended_at, tools_used, summary
@@ -146,14 +163,10 @@ jobs:
'success',
NOW(),
NOW(),
'{\"psql\": 1, \"gitea_ci\": 1}'::jsonb,
jsonb_build_object(
'type', 'ci_migration',
'commit_sha', :'commit_sha',
'files', :'files_json'::jsonb
)
'{"psql": 1, "gitea_ci": 1}'::jsonb,
'${SUMMARY_JSON_SQL}'::jsonb
);
"
SQL
}
audit_err="$(mktemp)"
@@ -176,11 +189,25 @@ jobs:
- name: Notify Telegram (if configured)
if: always()
env:
TG_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
TG_CHAT: ${{ env.TELEGRAM_ALERT_CHAT_ID }}
run: |
TG_TOKEN="$(cat <<'AWOOOI_SECRET_TG_TOKEN'
${{ secrets.TELEGRAM_BOT_TOKEN }}
AWOOOI_SECRET_TG_TOKEN
)"
STATUS="${{ job.status }}"
CICD_STATUS="success"
[ "$STATUS" != "success" ] && CICD_STATUS="failed"
if AWOOI_CICD_STATUS="${CICD_STATUS}" \
AWOOI_CICD_STAGE=run-migration \
AWOOI_CICD_JOB_NAME="Migration CI" \
AWOOI_CICD_COMMIT_SHA="${{ github.sha }}" \
AWOOI_CICD_SUMMARY="Migration CI: ${STATUS}" \
scripts/ci/notify-awoooi-cicd.sh; then
echo "Migration notification mirrored through AWOOI API"
exit 0
fi
if [ -n "$TG_TOKEN" ] && [ -n "$TG_CHAT" ]; then
STATUS="${{ job.status }}"
MSG="🗄️ Migration CI: \`${STATUS}\` — commit ${{ github.sha }}"
curl -s -X POST "https://api.telegram.org/bot${TG_TOKEN}/sendMessage" \
-d chat_id="${TG_CHAT}" \

View File

@@ -13,12 +13,10 @@
name: CD
# 2026-05-12 Codex: GitHub 僅保留唯讀備份;生產 CI/CD 只能從 Gitea 執行。
# 本 workflow 曾可 push / workflow_dispatch 後 build、patch secret、kubectl apply
# 會和 `.gitea/workflows/cd.yaml` 競爭 K3s production 狀態,因此硬停用。
on:
push:
branches: [main]
paths-ignore:
- 'docs/**'
- '*.md'
workflow_dispatch:
inputs:
force_deploy:
@@ -60,6 +58,7 @@ jobs:
# ==================== Pre-flight Check (10s Fail-Fast) ====================
pre-flight-check:
name: "Pre-flight Check"
if: ${{ false }}
runs-on: [self-hosted, harbor, k8s]
timeout-minutes: 1
steps:
@@ -133,6 +132,7 @@ jobs:
# 2026-03-29 Claude Code: 確保監控覆蓋率 >= 90%
monitoring-coverage:
name: "Monitoring Coverage"
if: ${{ false }}
runs-on: [self-hosted, harbor, k8s]
needs: pre-flight-check
timeout-minutes: 2
@@ -152,6 +152,7 @@ jobs:
# ==================== 路徑偵測 (使用 dorny/paths-filter) ====================
detect-changes:
name: Detect Changes
if: ${{ false }}
runs-on: [self-hosted, harbor, k8s]
needs: [pre-flight-check, monitoring-coverage]
timeout-minutes: 1
@@ -197,11 +198,7 @@ jobs:
runs-on: [self-hosted, harbor, k8s]
needs: [detect-changes, build-web]
timeout-minutes: 20
if: |
!inputs.skip_api && (
needs.detect-changes.outputs.api == 'true' ||
(needs.detect-changes.outputs.api == 'false' && needs.detect-changes.outputs.web == 'false')
)
if: ${{ false }}
outputs:
image_tag: ${{ steps.tag.outputs.tag }}
steps:
@@ -238,11 +235,7 @@ jobs:
runs-on: [self-hosted, harbor, k8s]
needs: detect-changes
timeout-minutes: 20
if: |
!inputs.skip_web && (
needs.detect-changes.outputs.web == 'true' ||
(needs.detect-changes.outputs.api == 'false' && needs.detect-changes.outputs.web == 'false')
)
if: ${{ false }}
outputs:
image_tag: ${{ steps.tag.outputs.tag }}
steps:
@@ -293,7 +286,7 @@ jobs:
concurrency:
group: runner-awoooi-cd-mutex
cancel-in-progress: false
if: always() && (needs.build-api.result == 'success' || needs.build-api.result == 'skipped') && (needs.build-web.result == 'success' || needs.build-web.result == 'skipped')
if: ${{ false }}
environment: production
steps:
# 2026-03-29: Runner 診斷檔案清理 (防止並行衝突)

View File

@@ -14,15 +14,10 @@
name: Deploy to Production
# 2026-05-12 Codex: GitHub 是唯讀備份production deploy 只能從 Gitea 進入。
# 這份歷史 workflow 仍含 Harbor build/push 與 kubectl apply/rollout會和 Gitea CD 競爭。
# 保留檔案供稽核,但停用所有 job。
on:
push:
branches:
- main
paths:
- 'apps/api/**'
- 'apps/web/**'
- 'k8s/awoooi-prod/**'
- '.github/workflows/deploy-prod.yml'
workflow_dispatch:
inputs:
deploy_api:
@@ -70,6 +65,7 @@ jobs:
# ===========================================================================
build:
name: "Build Images"
if: ${{ false }}
runs-on: [self-hosted, harbor, k8s]
outputs:
image_tag: ${{ steps.meta.outputs.tag }}
@@ -138,6 +134,7 @@ jobs:
deploy:
name: "Deploy to K3s"
needs: build
if: ${{ false }}
runs-on: [self-hosted, harbor, k8s]
steps:
@@ -210,7 +207,7 @@ jobs:
smoke-test:
name: "Smoke Tests"
needs: deploy
if: ${{ !inputs.skip_tests }}
if: ${{ false }}
runs-on: [self-hosted, harbor, k8s]
steps:
@@ -248,7 +245,7 @@ jobs:
notify:
name: "Send Notification"
needs: [build, deploy, smoke-test]
if: always()
if: ${{ false }}
runs-on: [self-hosted, harbor, k8s]
steps:

1
.gitignore vendored
View File

@@ -93,3 +93,4 @@ tsconfig.tsbuildinfo
!.aiderignore
.claude/settings.local.json
.claude/settings.json
.claude/settings.json.bak*

View File

@@ -1 +1 @@
# 2026-04-05 warm-up deploy triggered
# 2026-05-20 source-provider-heartbeat deploy trigger

View File

@@ -44,28 +44,6 @@ FROM python:3.11-slim
WORKDIR /app
# Copy installed packages from builder
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
# 2026-04-01 ogt: CACHE_BUST 強制失效 src/ 和 models.json 層
# deps 層 (pip install) 仍可 cache代碼/配置變更必須重建
ARG CACHE_BUST=none
COPY apps/api/src/ ./src/
COPY apps/api/models.json ./models.json
# 2026-04-09 ogt: 規則引擎配置 — alert_rule_engine.py 從此檔載入規則
COPY apps/api/alert_rules.yaml ./alert_rules.yaml
# 2026-04-10 Claude Sonnet 4.6: drift_detector 需要 k8s/ YAML 做 Git state 比對
COPY k8s/ ./k8s/
# 2026-04-10 Claude Sonnet 4.6: RAG 知識庫索引來源 (ADR-067 Phase 33)
COPY docs/ ./docs/
COPY .agents/skills/ ./.agents/skills/
# 2026-05-04 Claude Sonnet 4.6 (Task 1.2): hermes agent_loader 的 system prompt 來源
# agent_loader.py 預設讀 /app/.claude/agents/,對應 K8s AGENTS_DIR 環境變數
COPY .claude/agents/ ./.claude/agents/
# 2026-04-12 ogt (ADR-073 P2-1): CronJob 腳本 — 獨立腳本取代 inline Python
COPY scripts/ ./scripts/
# Install openssh-client + curl — SSH_COMMAND Playbook + healthcheck
# Install kubectl — drift_detector 需要 kubectl 讀取 K8s 實際狀態
# (2026-04-09 Claude Sonnet 4.6 Asia/Taipei, Bug #6 修正 — python:3.11-slim 無 openssh-client)
@@ -75,8 +53,38 @@ RUN apt-get update && apt-get install -y --no-install-recommends openssh-client
chmod +x kubectl && mv kubectl /usr/local/bin/kubectl && \
rm -rf /var/lib/apt/lists/*
# Create non-root user
RUN useradd -m -u 1000 appuser && chown -R appuser:appuser /app
# Create non-root user before copying app artifacts so COPY --chown can avoid
# an expensive full-tree chown layer on every source-only rebuild.
RUN useradd -m -u 1000 appuser
# Copy installed packages from builder
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
# 2026-04-01 ogt: CACHE_BUST 強制失效 src/ 和 models.json 層
# deps 層 (pip install) 仍可 cache代碼/配置變更必須重建
ARG CACHE_BUST=none
COPY --chown=appuser:appuser apps/api/src/ ./src/
# 2026-04-09 ogt: 規則引擎配置 — alert_rule_engine.py 從此檔載入規則
COPY --chown=appuser:appuser apps/api/models.json ./models.json
COPY --chown=appuser:appuser apps/api/alert_rules.yaml ./alert_rules.yaml
# 2026-04-10 Claude Sonnet 4.6: drift_detector 需要 k8s/ YAML 做 Git state 比對
COPY --chown=appuser:appuser k8s/ ./k8s/
# 2026-05-24 Codex: truth-chain / Ansible readiness needs the repo-known
# playbook catalog in the API image.
# 2026-05-31 Codex: ansible-core is now installed through pyproject.toml so
# this catalog can graduate from visibility-only to check-mode runtime-ready
# once repair SSH material is mounted and readable. This still does not enable
# automatic apply; approval/execution code remains the gate.
COPY --chown=appuser:appuser infra/ansible/ ./infra/ansible/
# 2026-04-10 Claude Sonnet 4.6: RAG 知識庫索引來源 (ADR-067 Phase 33)
COPY --chown=appuser:appuser docs/ ./docs/
COPY --chown=appuser:appuser .agents/skills/ ./.agents/skills/
# 2026-05-04 Claude Sonnet 4.6 (Task 1.2): hermes agent_loader 的 system prompt 來源
# agent_loader.py 預設讀 /app/.claude/agents/,對應 K8s AGENTS_DIR 環境變數
COPY --chown=appuser:appuser .claude/agents/ ./.claude/agents/
# 2026-04-12 ogt (ADR-073 P2-1): CronJob 腳本 — 獨立腳本取代 inline Python
COPY --chown=appuser:appuser scripts/ ./scripts/
USER appuser
# Expose port

View File

@@ -0,0 +1,36 @@
-- ADR-090-D: automation_operation_log.operation_type adds Ansible executor audit states
-- Created: 2026-05-12 Taipei
--
-- Purpose:
-- T3 Ansible declarative executor visibility. These operation types allow
-- the AI automation truth chain to record that Ansible was matched,
-- check-mode executed, applied, rolled back, or explicitly skipped.
--
-- Safety:
-- This migration only expands the CHECK allowlist. It does not execute
-- Ansible, change approval behavior, or create auto-remediation rows.
ALTER TABLE automation_operation_log
DROP CONSTRAINT IF EXISTS automation_operation_log_type_valid;
ALTER TABLE automation_operation_log
ADD CONSTRAINT automation_operation_log_type_valid CHECK (operation_type IN (
'monitor_configured','monitor_removed',
'alert_fired','alert_suppressed','alert_routed',
'rule_created','rule_updated','rule_matched','rule_rejected','rule_deprecated',
'playbook_generated','playbook_updated','playbook_executed',
'remediation_executed','remediation_verified','remediation_rolled_back',
'self_correction_attempted',
'km_created','km_updated','km_linked',
'asset_discovered','coverage_recalculated',
'capacity_recommendation','quota_enforced',
'notification_formatted',
'ansible_candidate_matched',
'ansible_check_mode_executed',
'ansible_apply_executed',
'ansible_rollback_executed',
'ansible_execution_skipped'
));
COMMENT ON CONSTRAINT automation_operation_log_type_valid ON automation_operation_log IS
'ADR-090-D: allow first-class Ansible executor audit states for AwoooP truth-chain visibility.';

View File

@@ -0,0 +1,19 @@
-- ADR-090-D rollback: remove Ansible executor audit states from operation_type allowlist.
-- Only apply after confirming no automation_operation_log rows use ansible_* operation types.
ALTER TABLE automation_operation_log
DROP CONSTRAINT IF EXISTS automation_operation_log_type_valid;
ALTER TABLE automation_operation_log
ADD CONSTRAINT automation_operation_log_type_valid CHECK (operation_type IN (
'monitor_configured','monitor_removed',
'alert_fired','alert_suppressed','alert_routed',
'rule_created','rule_updated','rule_matched','rule_rejected','rule_deprecated',
'playbook_generated','playbook_updated','playbook_executed',
'remediation_executed','remediation_verified','remediation_rolled_back',
'self_correction_attempted',
'km_created','km_updated','km_linked',
'asset_discovered','coverage_recalculated',
'capacity_recommendation','quota_enforced',
'notification_formatted'
));

View File

@@ -0,0 +1,164 @@
-- T9: approved SSH execution MCP Gateway seed
-- 目的:讓 Telegram/Approval 已批准的 SSH 修復動作通過 AwoooP Gateway 五閘門。
-- 邊界:只授權 approval_executorwrite/admin 仍需 Gate 5 短效 approval key。
SELECT set_config('app.project_id', 'awoooi', FALSE);
WITH agent_body AS (
SELECT jsonb_build_object(
'schema_version', 'awooop_agent_contract_v1',
'agent_id', 'approval_executor',
'display_name', 'Approval Executor',
'project_id', 'awoooi',
'purpose', 'Approved SSH execution through AwoooP MCP Gateway',
'allowed_scopes', jsonb_build_array('read', 'write', 'admin'),
'requires_gate5_for_scopes', jsonb_build_array('write', 'admin'),
'stage', 't9_ssh_approval_gateway'
) AS body_json
),
inserted_revision AS (
INSERT INTO awooop_contract_revisions (
project_id,
contract_family,
contract_id,
version_major,
version_minor,
lifecycle_status,
body_json,
body_hash,
body_schema_version,
publisher_id,
published_at
)
SELECT
'awoooi',
'agent',
'approval_executor',
1,
0,
'active',
body_json,
encode(digest(body_json::text, 'sha256'), 'hex'),
'v1.0',
'migration:t9_ssh_approval_gateway',
NOW()
FROM agent_body
ON CONFLICT (project_id, contract_family, contract_id, version_major, version_minor)
DO NOTHING
RETURNING revision_id, project_id, contract_family, contract_id
),
chosen_revision AS (
SELECT revision_id, project_id, contract_family, contract_id
FROM inserted_revision
UNION ALL
SELECT revision_id, project_id, contract_family, contract_id
FROM awooop_contract_revisions
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id = 'approval_executor'
AND version_major = 1
AND version_minor = 0
AND lifecycle_status = 'active'
),
upsert_pointer AS (
INSERT INTO awooop_active_revisions (
project_id,
contract_family,
contract_id,
active_revision_id,
updated_at
)
SELECT DISTINCT ON (project_id, contract_family, contract_id)
project_id,
contract_family,
contract_id,
revision_id,
NOW()
FROM chosen_revision
ORDER BY project_id, contract_family, contract_id, revision_id
ON CONFLICT (project_id, contract_family, contract_id)
DO UPDATE SET
active_revision_id = EXCLUDED.active_revision_id,
updated_at = NOW()
RETURNING contract_id
)
SELECT 'approval_executor_active_contracts', count(*) FROM upsert_pointer;
WITH gateway_tools(tool_name, description, required_scope) AS (
VALUES
('ssh_diagnose', 'SSH host diagnosis read', 'read'),
('ssh_docker_restart', 'Approved Docker container restart over SSH', 'write'),
('ssh_docker_compose_restart', 'Approved Docker Compose service restart over SSH', 'write'),
('ssh_systemctl_restart', 'Approved systemd service restart over SSH', 'write'),
('ssh_clear_docker_logs', 'Approved Docker log truncation over SSH', 'write'),
('ssh_renew_ssl', 'Approved certbot renewal over SSH', 'write'),
('ssh_reload_nginx', 'Approved nginx config test and reload over SSH', 'write'),
('ssh_docker_prune', 'Approved Docker prune over SSH with provider disk guard', 'admin')
),
upsert_tools AS (
INSERT INTO awooop_mcp_tool_registry (
project_id,
tool_name,
tool_type,
description,
allowed_scopes,
environment_tags,
is_active,
updated_at
)
SELECT
'awoooi',
tool_name,
'mcp_server',
description,
jsonb_build_array(required_scope),
'{"env": "prod"}'::jsonb,
TRUE,
NOW()
FROM gateway_tools
ON CONFLICT (project_id, tool_name)
DO UPDATE SET
description = EXCLUDED.description,
allowed_scopes = EXCLUDED.allowed_scopes,
environment_tags = EXCLUDED.environment_tags,
is_active = TRUE,
updated_at = NOW()
RETURNING tool_id, tool_name, allowed_scopes
),
upsert_grants AS (
INSERT INTO awooop_mcp_grants (
project_id,
agent_id,
tool_id,
granted_by,
granted_scopes,
expires_at,
is_revoked,
revoked_at,
revoked_by
)
SELECT
'awoooi',
'approval_executor',
tool_id,
'migration:t9_ssh_approval_gateway',
allowed_scopes,
NULL,
FALSE,
NULL,
NULL
FROM upsert_tools
ON CONFLICT (project_id, agent_id, tool_id)
DO UPDATE SET
granted_by = EXCLUDED.granted_by,
granted_scopes = EXCLUDED.granted_scopes,
expires_at = NULL,
is_revoked = FALSE,
revoked_at = NULL,
revoked_by = NULL
RETURNING grant_id
)
SELECT
'approval_executor_ssh_gateway',
(SELECT count(*) FROM upsert_tools) AS tool_rows,
(SELECT count(*) FROM upsert_grants) AS grant_rows;

View File

@@ -0,0 +1,43 @@
-- Rollback for T9 approved SSH execution MCP Gateway seed.
-- Contract revisions are append-only; rollback revokes approval_executor grants
-- and deactivates only the write/admin tools introduced here.
SELECT set_config('app.project_id', 'awoooi', FALSE);
UPDATE awooop_mcp_grants
SET
is_revoked = TRUE,
revoked_at = NOW(),
revoked_by = 'rollback:t9_ssh_approval_gateway'
WHERE project_id = 'awoooi'
AND agent_id = 'approval_executor'
AND granted_by = 'migration:t9_ssh_approval_gateway'
AND is_revoked = FALSE;
UPDATE awooop_mcp_tool_registry
SET
is_active = FALSE,
updated_at = NOW()
WHERE project_id = 'awoooi'
AND tool_name IN (
'ssh_docker_restart',
'ssh_docker_compose_restart',
'ssh_systemctl_restart',
'ssh_clear_docker_logs',
'ssh_renew_ssl',
'ssh_reload_nginx',
'ssh_docker_prune'
);
DELETE FROM awooop_active_revisions
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id = 'approval_executor';
UPDATE awooop_contract_revisions
SET lifecycle_status = 'revoked'
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id = 'approval_executor'
AND publisher_id = 'migration:t9_ssh_approval_gateway'
AND lifecycle_status = 'active';

View File

@@ -0,0 +1,166 @@
-- T23: auto-repair executor read-only MCP Gateway seed
-- 目的:讓 YAML_RULE/PlayBook 的只讀 SSH 診斷步驟經過 AwoooP MCP Gateway。
-- 邊界:只授權 read scopewrite/admin SSH 工具仍必須走 approval_executor + Gate 5。
SELECT set_config('app.project_id', 'awoooi', FALSE);
WITH agent_body AS (
SELECT jsonb_build_object(
'schema_version', 'awooop_agent_contract_v1',
'agent_id', 'auto_repair_executor',
'display_name', 'Auto Repair Executor',
'project_id', 'awoooi',
'purpose', 'Read-only auto-repair diagnostics through AwoooP MCP Gateway',
'allowed_scopes', jsonb_build_array('read'),
'forbidden_scopes', jsonb_build_array('write', 'admin'),
'stage', 't23_auto_repair_diagnostic_gateway'
) AS body_json
),
inserted_revision AS (
INSERT INTO awooop_contract_revisions (
project_id,
contract_family,
contract_id,
version_major,
version_minor,
lifecycle_status,
body_json,
body_hash,
body_schema_version,
publisher_id,
published_at
)
SELECT
'awoooi',
'agent',
'auto_repair_executor',
1,
0,
'active',
body_json,
encode(digest(body_json::text, 'sha256'), 'hex'),
'v1.0',
'migration:t23_auto_repair_executor_read_gateway',
NOW()
FROM agent_body
ON CONFLICT (project_id, contract_family, contract_id, version_major, version_minor)
DO NOTHING
RETURNING revision_id, project_id, contract_family, contract_id
),
chosen_revision AS (
SELECT revision_id, project_id, contract_family, contract_id
FROM inserted_revision
UNION ALL
SELECT revision_id, project_id, contract_family, contract_id
FROM awooop_contract_revisions
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id = 'auto_repair_executor'
AND version_major = 1
AND version_minor = 0
AND lifecycle_status = 'active'
),
upsert_pointer AS (
INSERT INTO awooop_active_revisions (
project_id,
contract_family,
contract_id,
active_revision_id,
updated_at
)
SELECT DISTINCT ON (project_id, contract_family, contract_id)
project_id,
contract_family,
contract_id,
revision_id,
NOW()
FROM chosen_revision
ORDER BY project_id, contract_family, contract_id, revision_id
ON CONFLICT (project_id, contract_family, contract_id)
DO UPDATE SET
active_revision_id = EXCLUDED.active_revision_id,
updated_at = NOW()
RETURNING contract_id
)
SELECT 'auto_repair_executor_active_contracts', count(*) FROM upsert_pointer;
WITH read_tools(tool_name, description) AS (
VALUES
('ssh_diagnose', 'SSH host/container diagnosis read'),
('ssh_get_top_processes', 'SSH top processes read'),
('ssh_get_disk_usage', 'SSH disk usage read'),
('ssh_get_memory_info', 'SSH memory info read'),
('ssh_get_container_logs', 'SSH container logs read'),
('ssh_get_container_status', 'SSH container status read'),
('ssh_get_service_status', 'SSH service status read'),
('ssh_check_port', 'SSH port check read'),
('ssh_get_nginx_error_log', 'SSH nginx error log read'),
('ssh_get_swap_info', 'SSH swap info read')
),
upsert_tools AS (
INSERT INTO awooop_mcp_tool_registry (
project_id,
tool_name,
tool_type,
description,
allowed_scopes,
environment_tags,
is_active,
updated_at
)
SELECT
'awoooi',
tool_name,
'mcp_server',
description,
'["read"]'::jsonb,
'{"env": "prod"}'::jsonb,
TRUE,
NOW()
FROM read_tools
ON CONFLICT (project_id, tool_name)
DO UPDATE SET
description = EXCLUDED.description,
allowed_scopes = EXCLUDED.allowed_scopes,
environment_tags = EXCLUDED.environment_tags,
is_active = TRUE,
updated_at = NOW()
RETURNING tool_id, tool_name, allowed_scopes
),
upsert_grants AS (
INSERT INTO awooop_mcp_grants (
project_id,
agent_id,
tool_id,
granted_by,
granted_scopes,
expires_at,
is_revoked,
revoked_at,
revoked_by
)
SELECT
'awoooi',
'auto_repair_executor',
tool_id,
'migration:t23_auto_repair_executor_read_gateway',
allowed_scopes,
NULL,
FALSE,
NULL,
NULL
FROM upsert_tools
ON CONFLICT (project_id, agent_id, tool_id)
DO UPDATE SET
granted_by = EXCLUDED.granted_by,
granted_scopes = EXCLUDED.granted_scopes,
expires_at = NULL,
is_revoked = FALSE,
revoked_at = NULL,
revoked_by = NULL
RETURNING grant_id
)
SELECT
'auto_repair_executor_read_gateway',
(SELECT count(*) FROM upsert_tools) AS tool_rows,
(SELECT count(*) FROM upsert_grants) AS grant_rows;

View File

@@ -0,0 +1,24 @@
-- Rollback T23 auto-repair executor read-only MCP Gateway grant.
SELECT set_config('app.project_id', 'awoooi', FALSE);
UPDATE awooop_mcp_grants
SET is_revoked = TRUE,
revoked_at = NOW(),
revoked_by = 'rollback:t23_auto_repair_executor_read_gateway'
WHERE project_id = 'awoooi'
AND agent_id = 'auto_repair_executor'
AND granted_by = 'migration:t23_auto_repair_executor_read_gateway';
DELETE FROM awooop_active_revisions
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id = 'auto_repair_executor';
UPDATE awooop_contract_revisions
SET lifecycle_status = 'retired'
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id = 'auto_repair_executor'
AND publisher_id = 'migration:t23_auto_repair_executor_read_gateway'
AND lifecycle_status = 'active';

View File

@@ -0,0 +1,25 @@
-- =============================================================================
-- AwoooP / AWOOOI MCP Gateway Shadow Onboarding
-- 2026-05-13 Codex + ogt
--
-- 背景:
-- AWOOOI 已完成 read-only MCP tool registry / grants seed但 project 本身仍停在
-- legacy_awoooi_default會被 MCP Gateway Gate 1 正確攔截。
--
-- 邊界:
-- 只把 AWOOOI 租戶升到 shadow讓既有 Gate 1 生效。
-- write/admin tool 仍未授權;自動修復/破壞性動作不因本 migration 開放。
-- =============================================================================
BEGIN;
SELECT set_config('app.project_id', 'awoooi', FALSE);
UPDATE awooop_projects
SET
migration_mode = 'shadow',
updated_at = NOW()
WHERE project_id = 'awoooi'
AND migration_mode = 'legacy_awoooi_default';
COMMIT;

View File

@@ -0,0 +1,20 @@
-- =============================================================================
-- Rollback: AwoooP / AWOOOI MCP Gateway Shadow Onboarding
-- 2026-05-13 Codex + ogt
--
-- 只回退仍停在 shadow 的 AWOOOI若已由人工/後續 migration 推進到 canary/active
-- 不自動降級。
-- =============================================================================
BEGIN;
SELECT set_config('app.project_id', 'awoooi', FALSE);
UPDATE awooop_projects
SET
migration_mode = 'legacy_awoooi_default',
updated_at = NOW()
WHERE project_id = 'awoooi'
AND migration_mode = 'shadow';
COMMIT;

View File

@@ -0,0 +1,211 @@
-- T7: awoooi read-only MCP Gateway seed
-- 目的:讓決策前感官 MCP 能通過 AwoooP Gateway Gate 2/3產生 first-class audit。
-- 邊界:只授權 read scope不授權 restart/delete/scale/apply/rollback 等 write/admin 工具。
SELECT set_config('app.project_id', 'awoooi', FALSE);
WITH agent_seed(agent_id, display_name) AS (
VALUES
('pre_decision_investigator', 'Pre-decision Investigator'),
('post_execution_verifier', 'Post-execution Verifier')
),
agent_body AS (
SELECT
agent_id,
jsonb_build_object(
'schema_version', 'awooop_agent_contract_v1',
'agent_id', agent_id,
'display_name', display_name,
'project_id', 'awoooi',
'purpose', 'Read-only MCP sensing through AwoooP Gateway',
'allowed_scopes', jsonb_build_array('read'),
'forbidden_scopes', jsonb_build_array('write', 'admin'),
'stage', 't7_mcp_gateway_read_sense'
) AS body_json
FROM agent_seed
),
inserted_revision AS (
INSERT INTO awooop_contract_revisions (
project_id,
contract_family,
contract_id,
version_major,
version_minor,
lifecycle_status,
body_json,
body_hash,
body_schema_version,
publisher_id,
published_at
)
SELECT
'awoooi',
'agent',
agent_id,
1,
0,
'active',
body_json,
encode(digest(body_json::text, 'sha256'), 'hex'),
'v1.0',
'migration:t7_mcp_gateway_read_seed',
NOW()
FROM agent_body
ON CONFLICT (project_id, contract_family, contract_id, version_major, version_minor)
DO NOTHING
RETURNING revision_id, project_id, contract_family, contract_id
),
chosen_revision AS (
SELECT revision_id, project_id, contract_family, contract_id
FROM inserted_revision
UNION ALL
SELECT revision_id, project_id, contract_family, contract_id
FROM awooop_contract_revisions
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id IN (SELECT agent_id FROM agent_seed)
AND version_major = 1
AND version_minor = 0
AND lifecycle_status = 'active'
),
upsert_pointer AS (
INSERT INTO awooop_active_revisions (
project_id,
contract_family,
contract_id,
active_revision_id,
updated_at
)
SELECT DISTINCT ON (project_id, contract_family, contract_id)
project_id,
contract_family,
contract_id,
revision_id,
NOW()
FROM chosen_revision
ORDER BY project_id, contract_family, contract_id, revision_id
ON CONFLICT (project_id, contract_family, contract_id)
DO UPDATE SET
active_revision_id = EXCLUDED.active_revision_id,
updated_at = NOW()
RETURNING contract_id
)
SELECT 'active_agent_contracts', count(*) FROM upsert_pointer;
WITH read_tools(tool_name, description) AS (
VALUES
('k8s_get_pod_logs', 'Kubernetes pod logs read'),
('k8s_get_events', 'Kubernetes events read'),
('k8s_describe_pod', 'Kubernetes pod describe read'),
('k8s_get_hpa_status', 'Kubernetes HPA status read'),
('k8s_get_node_conditions', 'Kubernetes node conditions read'),
('ssh_diagnose', 'SSH host diagnosis read'),
('ssh_get_top_processes', 'SSH top processes read'),
('ssh_get_disk_usage', 'SSH disk usage read'),
('ssh_get_memory_info', 'SSH memory info read'),
('ssh_get_container_logs', 'SSH container logs read'),
('ssh_get_container_status', 'SSH container status read'),
('ssh_get_service_status', 'SSH service status read'),
('ssh_check_port', 'SSH port check read'),
('ssh_get_nginx_error_log', 'SSH nginx error log read'),
('ssh_get_swap_info', 'SSH swap info read'),
('prometheus_query', 'Prometheus instant query read'),
('prometheus_query_range', 'Prometheus range query read'),
('prometheus_get_alert_history', 'Prometheus alert history read'),
('gold_metrics', 'SigNoz gold metrics read'),
('trace_url', 'SigNoz trace URL read'),
('system_metrics', 'SigNoz system metrics read'),
('query_logs', 'SigNoz logs read'),
('error_logs_summary', 'SigNoz error logs summary read'),
('list_approvals', 'Approval records read'),
('get_approval', 'Approval detail read'),
('list_incidents', 'Incident records read'),
('list_timeline', 'Timeline records read'),
('read_file', 'Filesystem allowlisted file read'),
('list_directory', 'Filesystem allowlisted directory read'),
('search_in_file', 'Filesystem allowlisted file search'),
('list_dashboards', 'Grafana dashboards read'),
('get_dashboard', 'Grafana dashboard read'),
('get_panel_data', 'Grafana panel data read'),
('generate_dashboard_url', 'Grafana dashboard URL read'),
('search_runbook', 'Runbook semantic search read'),
('get_index_stats', 'Runbook index stats read'),
('argocd_list_apps', 'ArgoCD apps read'),
('argocd_get_app_status', 'ArgoCD app status read'),
('argocd_get_sync_history', 'ArgoCD sync history read'),
('sentry_list_issues', 'Sentry issues read'),
('sentry_get_issue', 'Sentry issue detail read'),
('sentry_search_issues', 'Sentry issue search read')
),
upsert_tools AS (
INSERT INTO awooop_mcp_tool_registry (
project_id,
tool_name,
tool_type,
description,
allowed_scopes,
environment_tags,
is_active,
updated_at
)
SELECT
'awoooi',
tool_name,
'mcp_server',
description,
'["read"]'::jsonb,
'{"env": "prod"}'::jsonb,
TRUE,
NOW()
FROM read_tools
ON CONFLICT (project_id, tool_name)
DO UPDATE SET
description = EXCLUDED.description,
allowed_scopes = EXCLUDED.allowed_scopes,
environment_tags = EXCLUDED.environment_tags,
is_active = TRUE,
updated_at = NOW()
RETURNING tool_id
),
grant_agents(agent_id) AS (
VALUES
('pre_decision_investigator'),
('post_execution_verifier')
),
upsert_grants AS (
INSERT INTO awooop_mcp_grants (
project_id,
agent_id,
tool_id,
granted_by,
granted_scopes,
expires_at,
is_revoked,
revoked_at,
revoked_by
)
SELECT
'awoooi',
grant_agents.agent_id,
upsert_tools.tool_id,
'migration:t7_mcp_gateway_read_seed',
'["read"]'::jsonb,
NULL,
FALSE,
NULL,
NULL
FROM upsert_tools
CROSS JOIN grant_agents
ON CONFLICT (project_id, agent_id, tool_id)
DO UPDATE SET
granted_scopes = EXCLUDED.granted_scopes,
expires_at = NULL,
is_revoked = FALSE,
revoked_at = NULL,
revoked_by = NULL
RETURNING grant_id
)
SELECT
'awoooi_read_tools',
(SELECT count(*) FROM upsert_tools) AS tool_rows,
(SELECT count(*) FROM upsert_grants) AS grant_rows;

View File

@@ -0,0 +1,77 @@
-- Rollback for T7 awoooi read-only MCP Gateway seed.
-- Contract revisions are append-only; rollback revokes grants and deactivates the seeded read tools.
SELECT set_config('app.project_id', 'awoooi', FALSE);
UPDATE awooop_mcp_grants
SET
is_revoked = TRUE,
revoked_at = NOW(),
revoked_by = 'rollback:t7_mcp_gateway_read_seed'
WHERE project_id = 'awoooi'
AND agent_id IN ('pre_decision_investigator', 'post_execution_verifier')
AND granted_by = 'migration:t7_mcp_gateway_read_seed'
AND is_revoked = FALSE;
UPDATE awooop_mcp_tool_registry
SET
is_active = FALSE,
updated_at = NOW()
WHERE project_id = 'awoooi'
AND tool_name IN (
'k8s_get_pod_logs',
'k8s_get_events',
'k8s_describe_pod',
'k8s_get_hpa_status',
'k8s_get_node_conditions',
'ssh_diagnose',
'ssh_get_top_processes',
'ssh_get_disk_usage',
'ssh_get_memory_info',
'ssh_get_container_logs',
'ssh_get_container_status',
'ssh_get_service_status',
'ssh_check_port',
'ssh_get_nginx_error_log',
'ssh_get_swap_info',
'prometheus_query',
'prometheus_query_range',
'prometheus_get_alert_history',
'gold_metrics',
'trace_url',
'system_metrics',
'query_logs',
'error_logs_summary',
'list_approvals',
'get_approval',
'list_incidents',
'list_timeline',
'read_file',
'list_directory',
'search_in_file',
'list_dashboards',
'get_dashboard',
'get_panel_data',
'generate_dashboard_url',
'search_runbook',
'get_index_stats',
'argocd_list_apps',
'argocd_get_app_status',
'argocd_get_sync_history',
'sentry_list_issues',
'sentry_get_issue',
'sentry_search_issues'
);
DELETE FROM awooop_active_revisions
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id IN ('pre_decision_investigator', 'post_execution_verifier');
UPDATE awooop_contract_revisions
SET lifecycle_status = 'revoked'
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id IN ('pre_decision_investigator', 'post_execution_verifier')
AND publisher_id = 'migration:t7_mcp_gateway_read_seed'
AND lifecycle_status = 'active';

View File

@@ -0,0 +1,213 @@
-- T7: awoooi read-only MCP Gateway seed
-- 目的:讓決策前感官 MCP 能通過 AwoooP Gateway Gate 2/3產生 first-class audit。
-- 邊界:只授權 read scope不授權 restart/delete/scale/apply/rollback 等 write/admin 工具。
SELECT set_config('app.project_id', 'awoooi', FALSE);
WITH agent_seed(agent_id, display_name) AS (
VALUES
('pre_decision_investigator', 'Pre-decision Investigator'),
('post_execution_verifier', 'Post-execution Verifier')
),
agent_body AS (
SELECT
agent_id,
jsonb_build_object(
'schema_version', 'awooop_agent_contract_v1',
'agent_id', agent_id,
'display_name', display_name,
'project_id', 'awoooi',
'purpose', 'Read-only MCP sensing through AwoooP Gateway',
'allowed_scopes', jsonb_build_array('read'),
'forbidden_scopes', jsonb_build_array('write', 'admin'),
'stage', 't7_mcp_gateway_read_sense'
) AS body_json
FROM agent_seed
),
inserted_revision AS (
INSERT INTO awooop_contract_revisions (
project_id,
contract_family,
contract_id,
version_major,
version_minor,
lifecycle_status,
body_json,
body_hash,
body_schema_version,
publisher_id,
published_at
)
SELECT
'awoooi',
'agent',
agent_id,
1,
0,
'active',
body_json,
encode(digest(body_json::text, 'sha256'), 'hex'),
'v1.0',
'migration:t7_mcp_gateway_read_seed',
NOW()
FROM agent_body
ON CONFLICT (project_id, contract_family, contract_id, version_major, version_minor)
DO NOTHING
RETURNING revision_id, project_id, contract_family, contract_id
),
chosen_revision AS (
SELECT revision_id, project_id, contract_family, contract_id
FROM inserted_revision
UNION ALL
SELECT revision_id, project_id, contract_family, contract_id
FROM awooop_contract_revisions
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id IN (SELECT agent_id FROM agent_seed)
AND version_major = 1
AND version_minor = 0
AND lifecycle_status = 'active'
),
upsert_pointer AS (
INSERT INTO awooop_active_revisions (
project_id,
contract_family,
contract_id,
active_revision_id,
updated_at
)
SELECT DISTINCT ON (project_id, contract_family, contract_id)
project_id,
contract_family,
contract_id,
revision_id,
NOW()
FROM chosen_revision
ORDER BY project_id, contract_family, contract_id, revision_id
ON CONFLICT (project_id, contract_family, contract_id)
DO UPDATE SET
active_revision_id = EXCLUDED.active_revision_id,
updated_at = NOW()
RETURNING contract_id
)
SELECT 'active_agent_contracts', count(*) FROM upsert_pointer;
WITH read_tools(tool_name, description) AS (
VALUES
('k8s_get_pod_logs', 'Kubernetes pod logs read'),
('k8s_get_events', 'Kubernetes events read'),
('k8s_describe_pod', 'Kubernetes pod describe read'),
('k8s_get_hpa_status', 'Kubernetes HPA status read'),
('k8s_get_node_conditions', 'Kubernetes node conditions read'),
('ssh_diagnose', 'SSH host diagnosis read'),
('ssh_get_top_processes', 'SSH top processes read'),
('ssh_get_disk_usage', 'SSH disk usage read'),
('ssh_get_memory_info', 'SSH memory info read'),
('ssh_get_container_logs', 'SSH container logs read'),
('ssh_get_container_status', 'SSH container status read'),
('ssh_get_service_status', 'SSH service status read'),
('ssh_check_port', 'SSH port check read'),
('ssh_get_nginx_error_log', 'SSH nginx error log read'),
('ssh_get_swap_info', 'SSH swap info read'),
('prometheus_query', 'Prometheus instant query read'),
('prometheus_query_range', 'Prometheus range query read'),
('prometheus_get_alert_history', 'Prometheus alert history read'),
('gold_metrics', 'SigNoz gold metrics read'),
('trace_url', 'SigNoz trace URL read'),
('system_metrics', 'SigNoz system metrics read'),
('query_logs', 'SigNoz logs read'),
('error_logs_summary', 'SigNoz error logs summary read'),
('list_approvals', 'Approval records read'),
('get_approval', 'Approval detail read'),
('list_incidents', 'Incident records read'),
('list_timeline', 'Timeline records read'),
('read_file', 'Filesystem allowlisted file read'),
('list_directory', 'Filesystem allowlisted directory read'),
('search_in_file', 'Filesystem allowlisted file search'),
('list_dashboards', 'Grafana dashboards read'),
('get_dashboard', 'Grafana dashboard read'),
('get_panel_data', 'Grafana panel data read'),
('generate_dashboard_url', 'Grafana dashboard URL read'),
('search_runbook', 'Runbook semantic search read'),
('get_index_stats', 'Runbook index stats read'),
('argocd_list_apps', 'ArgoCD apps read'),
('argocd_get_app_status', 'ArgoCD app status read'),
('argocd_get_sync_history', 'ArgoCD sync history read'),
('sentry_list_issues', 'Sentry issues read'),
('sentry_get_issue', 'Sentry issue detail read'),
('sentry_search_issues', 'Sentry issue search read')
),
upsert_tools AS (
INSERT INTO awooop_mcp_tool_registry (
project_id,
tool_name,
tool_type,
description,
allowed_scopes,
environment_tags,
is_active,
updated_at
)
SELECT
'awoooi',
tool_name,
'mcp_server',
description,
'["read"]'::jsonb,
'{"env": "prod"}'::jsonb,
TRUE,
NOW()
FROM read_tools
ON CONFLICT (project_id, tool_name)
DO UPDATE SET
description = EXCLUDED.description,
allowed_scopes = EXCLUDED.allowed_scopes,
environment_tags = EXCLUDED.environment_tags,
is_active = TRUE,
updated_at = NOW()
RETURNING tool_id
),
grant_agents(agent_id) AS (
VALUES
('pre_decision_investigator'),
('post_execution_verifier')
),
upsert_grants AS (
INSERT INTO awooop_mcp_grants (
project_id,
agent_id,
tool_id,
granted_by,
granted_scopes,
expires_at,
is_revoked,
revoked_at,
revoked_by
)
SELECT
'awoooi',
grant_agents.agent_id,
upsert_tools.tool_id,
'migration:t7_mcp_gateway_read_seed',
'["read"]'::jsonb,
NULL,
FALSE,
NULL,
NULL
FROM upsert_tools
CROSS JOIN grant_agents
ON CONFLICT (project_id, agent_id, tool_id)
DO UPDATE SET
granted_scopes = EXCLUDED.granted_scopes,
expires_at = NULL,
is_revoked = FALSE,
revoked_at = NULL,
revoked_by = NULL
RETURNING grant_id
)
SELECT
'awoooi_read_tools',
(SELECT count(*) FROM upsert_tools) AS tool_rows,
(SELECT count(*) FROM upsert_grants) AS grant_rows;
-- v4 exists only to retrigger run-migration after Gitea skipped the v2->v3 rename-only push.

View File

@@ -0,0 +1,79 @@
-- Rollback for T7 awoooi read-only MCP Gateway seed.
-- Contract revisions are append-only; rollback revokes grants and deactivates the seeded read tools.
SELECT set_config('app.project_id', 'awoooi', FALSE);
UPDATE awooop_mcp_grants
SET
is_revoked = TRUE,
revoked_at = NOW(),
revoked_by = 'rollback:t7_mcp_gateway_read_seed'
WHERE project_id = 'awoooi'
AND agent_id IN ('pre_decision_investigator', 'post_execution_verifier')
AND granted_by = 'migration:t7_mcp_gateway_read_seed'
AND is_revoked = FALSE;
UPDATE awooop_mcp_tool_registry
SET
is_active = FALSE,
updated_at = NOW()
WHERE project_id = 'awoooi'
AND tool_name IN (
'k8s_get_pod_logs',
'k8s_get_events',
'k8s_describe_pod',
'k8s_get_hpa_status',
'k8s_get_node_conditions',
'ssh_diagnose',
'ssh_get_top_processes',
'ssh_get_disk_usage',
'ssh_get_memory_info',
'ssh_get_container_logs',
'ssh_get_container_status',
'ssh_get_service_status',
'ssh_check_port',
'ssh_get_nginx_error_log',
'ssh_get_swap_info',
'prometheus_query',
'prometheus_query_range',
'prometheus_get_alert_history',
'gold_metrics',
'trace_url',
'system_metrics',
'query_logs',
'error_logs_summary',
'list_approvals',
'get_approval',
'list_incidents',
'list_timeline',
'read_file',
'list_directory',
'search_in_file',
'list_dashboards',
'get_dashboard',
'get_panel_data',
'generate_dashboard_url',
'search_runbook',
'get_index_stats',
'argocd_list_apps',
'argocd_get_app_status',
'argocd_get_sync_history',
'sentry_list_issues',
'sentry_get_issue',
'sentry_search_issues'
);
DELETE FROM awooop_active_revisions
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id IN ('pre_decision_investigator', 'post_execution_verifier');
UPDATE awooop_contract_revisions
SET lifecycle_status = 'revoked'
WHERE project_id = 'awoooi'
AND contract_family = 'agent'
AND contract_id IN ('pre_decision_investigator', 'post_execution_verifier')
AND publisher_id = 'migration:t7_mcp_gateway_read_seed'
AND lifecycle_status = 'active';
-- v4 rollback companion for the retrigger migration.

View File

@@ -0,0 +1,77 @@
-- T16 verifier gap: allow rollout status evidence through AwoooP MCP Gateway.
-- Boundary: read-only scope only; no restart/delete/scale grant is added here.
SELECT set_config('app.project_id', 'awoooi', FALSE);
WITH upsert_tool AS (
INSERT INTO awooop_mcp_tool_registry (
project_id,
tool_name,
tool_type,
description,
allowed_scopes,
environment_tags,
is_active,
updated_at
)
VALUES (
'awoooi',
'k8s_watch_rollout',
'mcp_server',
'Kubernetes deployment rollout status read',
'["read"]'::jsonb,
'{"env": "prod"}'::jsonb,
TRUE,
NOW()
)
ON CONFLICT (project_id, tool_name)
DO UPDATE SET
description = EXCLUDED.description,
allowed_scopes = EXCLUDED.allowed_scopes,
environment_tags = EXCLUDED.environment_tags,
is_active = TRUE,
updated_at = NOW()
RETURNING tool_id
),
grant_agents(agent_id) AS (
VALUES
('pre_decision_investigator'),
('post_execution_verifier')
),
upsert_grants AS (
INSERT INTO awooop_mcp_grants (
project_id,
agent_id,
tool_id,
granted_by,
granted_scopes,
expires_at,
is_revoked,
revoked_at,
revoked_by
)
SELECT
'awoooi',
grant_agents.agent_id,
upsert_tool.tool_id,
'migration:t16_rollout_verifier_seed',
'["read"]'::jsonb,
NULL,
FALSE,
NULL,
NULL
FROM upsert_tool
CROSS JOIN grant_agents
ON CONFLICT (project_id, agent_id, tool_id)
DO UPDATE SET
granted_scopes = EXCLUDED.granted_scopes,
expires_at = NULL,
is_revoked = FALSE,
revoked_at = NULL,
revoked_by = NULL
RETURNING grant_id
)
SELECT
'k8s_watch_rollout_read_grants' AS seed,
(SELECT count(*) FROM upsert_tool) AS tool_rows,
(SELECT count(*) FROM upsert_grants) AS grant_rows;

View File

@@ -0,0 +1,24 @@
-- Roll back T16 rollout verifier read grant seed.
SELECT set_config('app.project_id', 'awoooi', FALSE);
UPDATE awooop_mcp_grants
SET
is_revoked = TRUE,
revoked_at = NOW(),
revoked_by = 'migration:t16_rollout_verifier_seed_down'
WHERE project_id = 'awoooi'
AND agent_id IN ('pre_decision_investigator', 'post_execution_verifier')
AND tool_id IN (
SELECT tool_id
FROM awooop_mcp_tool_registry
WHERE project_id = 'awoooi'
AND tool_name = 'k8s_watch_rollout'
);
UPDATE awooop_mcp_tool_registry
SET
is_active = FALSE,
updated_at = NOW()
WHERE project_id = 'awoooi'
AND tool_name = 'k8s_watch_rollout';

View File

@@ -0,0 +1,21 @@
-- AwoooP Phase 7 T15b: inbound event truth-chain columns
--
-- Purpose:
-- Telegram cards are only the notification surface. Operators need a
-- redacted replay envelope for inbound alerts so Alertmanager, Sentry, and
-- SignOz events can be correlated with incidents, approvals, logs, and
-- automation decisions without storing raw secrets or PII.
ALTER TABLE awooop_conversation_event
ADD COLUMN IF NOT EXISTS content_redacted TEXT,
ADD COLUMN IF NOT EXISTS redaction_version VARCHAR(32) NOT NULL DEFAULT 'audit_sink_v1',
ADD COLUMN IF NOT EXISTS source_envelope JSONB NOT NULL DEFAULT '{}'::jsonb;
COMMENT ON COLUMN awooop_conversation_event.content_redacted IS
'Full inbound event content after audit_sink redaction; raw unredacted payload text is not stored.';
COMMENT ON COLUMN awooop_conversation_event.redaction_version IS
'Redaction algorithm/version used for content_redacted and source_envelope.';
COMMENT ON COLUMN awooop_conversation_event.source_envelope IS
'Redacted source metadata for inbound replay/audit, including payload hash, provider, source refs, and log correlation hints.';

View File

@@ -0,0 +1,6 @@
-- Rollback for AwoooP Phase 7 T15b inbound truth-chain columns.
-- Safe only if no consumers depend on the redacted replay fields.
ALTER TABLE awooop_conversation_event DROP COLUMN IF EXISTS source_envelope;
ALTER TABLE awooop_conversation_event DROP COLUMN IF EXISTS redaction_version;
ALTER TABLE awooop_conversation_event DROP COLUMN IF EXISTS content_redacted;

View File

@@ -0,0 +1,21 @@
-- AwoooP Phase 7 T1: outbound message truth-chain columns
--
-- Purpose:
-- Telegram must remain a summary channel, but the operator console needs a
-- complete redacted replay of the rendered card and the source envelope that
-- produced it. Store redacted content only; raw unredacted Telegram text stays
-- out of PostgreSQL.
ALTER TABLE awooop_outbound_message
ADD COLUMN IF NOT EXISTS content_redacted TEXT,
ADD COLUMN IF NOT EXISTS redaction_version VARCHAR(32) NOT NULL DEFAULT 'audit_sink_v1',
ADD COLUMN IF NOT EXISTS source_envelope JSONB NOT NULL DEFAULT '{}'::jsonb;
COMMENT ON COLUMN awooop_outbound_message.content_redacted IS
'Full rendered outbound content after audit_sink redaction; raw unredacted text is not stored.';
COMMENT ON COLUMN awooop_outbound_message.redaction_version IS
'Redaction algorithm/version used for content_redacted and source_envelope.';
COMMENT ON COLUMN awooop_outbound_message.source_envelope IS
'Redacted source metadata for replay/audit, including payload hash and adapter context.';

View File

@@ -0,0 +1,6 @@
-- Rollback for AwoooP Phase 7 T1 outbound truth-chain columns.
-- Safe only if no consumers depend on the redacted replay fields.
ALTER TABLE awooop_outbound_message DROP COLUMN IF EXISTS source_envelope;
ALTER TABLE awooop_outbound_message DROP COLUMN IF EXISTS redaction_version;
ALTER TABLE awooop_outbound_message DROP COLUMN IF EXISTS content_redacted;

View File

@@ -46,6 +46,10 @@ dependencies = [
# 2026-04-16 ogt + Claude Sonnet 4.6: SSH MCP sensor 修復 — asyncssh 缺失導致 sensors_succeeded=0
# 根因: ssh_provider.py 中 import asyncssh 在 try/except 外,所有 15 個 SSH tool 直接 ImportError
"asyncssh>=2.14.0",
# 2026-05-31 Codex: AwoooP truth-chain Ansible runtime gate 需要
# production API image 內真的存在 ansible-playbook否則只能顯示
# candidate audit無法進入 check-mode executor readiness。
"ansible-core>=2.16.0,<2.18.0",
]
# [tool.uv.sources]

View File

@@ -58,3 +58,8 @@ pytest>=7.4.0
pytest-asyncio>=0.23.0
ruff>=0.1.0
sentry-sdk[fastapi]>=2.0.0
# AwoooP Ansible runtime readiness
# 2026-05-31 Codex: production API image must include ansible-playbook before
# truth-chain can honestly mark check-mode executor readiness as available.
ansible-core>=2.16.0,<2.18.0

View File

@@ -9,7 +9,7 @@ AwoooP Phase 1 Batch 1 回填腳本
awooop_phase1_batch1_rls_2026-05-04.sql Step AADD COLUMN nullable已執行
執行方式:
export DATABASE_URL="postgresql+asyncpg://awoooi:<password>@192.168.0.188:5432/awoooi_prod"
從 secret manager / operator vault 設定 DATABASE_URL禁止在指令或檔案中寫入 URL。
cd apps/api && python scripts/awooop_phase1_batch1_backfill.py
2026-05-04 ogt + Claude Sonnet 4.6ADR-118 Batch 1 C-3 修正)

View File

@@ -37,6 +37,7 @@ logging = structlog.get_logger(__name__)
OLLAMA_URL = os.getenv("OLLAMA_URL", "http://34.143.170.20:11434")
EMBEDDING_MODEL = "bge-m3:latest"
EXPECTED_DIM = 1024
PROJECT_ID = os.getenv("AWOOOP_PROJECT_ID", "awoooi")
async def embed_text(client: httpx.AsyncClient, text: str) -> list[float]:
@@ -162,6 +163,7 @@ async def main(dry_run: bool, batch_size: int) -> None:
conn = await asyncpg.connect(database_url)
try:
await conn.execute("SELECT set_config('app.project_id', $1, FALSE)", PROJECT_ID)
# 統計待嵌入筆數
rag_null = await conn.fetchval("SELECT COUNT(*) FROM rag_chunks WHERE embedding IS NULL")
pb_null = await conn.fetchval("SELECT COUNT(*) FROM playbook_embeddings WHERE embedding IS NULL")

View File

@@ -15,7 +15,7 @@ from sqlalchemy import text
from sqlalchemy.ext.asyncio import create_async_engine
# 2026-04-22 ogt: 移除硬碼 changeme改為讀取環境變數強制要求設定
# 執行前: export DATABASE_URL="postgresql+asyncpg://awoooi:<password>@192.168.0.188:5432/awoooi_prod"
# 執行前: 從 secret manager / operator vault 設定 DATABASE_URL禁止在指令或檔案中寫入 URL。
DATABASE_URL = os.environ["DATABASE_URL"]
MIGRATION_SQLS = [

View File

@@ -22,17 +22,48 @@ from datetime import datetime
from typing import Annotated
import structlog
from fastapi import APIRouter, Query
from fastapi import APIRouter, HTTPException, Query
from src.models.governance import (
GovernanceEventsResponse,
GovernanceQueueResponse,
GovernanceSummaryResponse,
KnowledgeReviewDraftArchiveRequest,
KnowledgeReviewDraftArchiveResponse,
KnowledgeReviewDraftDedupeResponse,
KnowledgeStaleCandidatesResponse,
KnowledgeStaleOwnerReviewBatchQueueRequest,
KnowledgeStaleOwnerReviewBatchQueueResponse,
KnowledgeStaleOwnerReviewBurnDownResponse,
KnowledgeStaleOwnerReviewCompleteRequest,
KnowledgeStaleOwnerReviewCompleteResponse,
KnowledgeStaleOwnerReviewCompletionBatchPreviewRequest,
KnowledgeStaleOwnerReviewCompletionBatchPreviewResponse,
KnowledgeStaleOwnerReviewCompletionQueueResponse,
KnowledgeStaleOwnerReviewInboxResponse,
KnowledgeStaleOwnerReviewRequest,
KnowledgeStaleOwnerReviewResponse,
)
from src.services.governance_km_review_service import (
KmReviewDraftArchiveError,
archive_km_review_draft_duplicates,
)
from src.services.governance_km_stale_review_service import (
KmStaleOwnerReviewError,
batch_queue_km_stale_owner_reviews,
complete_km_stale_owner_review,
preview_km_stale_owner_review_completion_batch,
query_km_stale_owner_review_burndown,
query_km_stale_owner_review_completion_queue,
query_km_stale_owner_review_inbox,
queue_km_stale_owner_review,
)
from src.services.governance_query_service import (
query_governance_events,
query_governance_queue,
query_governance_summary,
query_km_review_draft_dedupe,
query_km_stale_candidates,
)
logger = structlog.get_logger(__name__)
@@ -46,6 +77,7 @@ router = APIRouter()
@router.get("/ai/governance/events", response_model=GovernanceEventsResponse)
async def get_governance_events(
event_id: Annotated[list[str] | None, Query(alias="event_id")] = None,
event_type: Annotated[list[str] | None, Query(alias="event_type")] = None,
from_: Annotated[datetime | None, Query(alias="from")] = None,
to: Annotated[datetime | None, Query(alias="to")] = None,
@@ -58,6 +90,7 @@ async def get_governance_events(
查詢 AI 治理事件列表(分頁)。
- event_type: 多值過濾(可重複傳)
- event_id: 多值精準過濾(可重複傳),供 Telegram 詳情 / 歷史與 Work Items 錨點回看
- from / to: ISO 8601 時間範圍URL 傳 from 參數)
- status: resolved / unresolved
- severity: critical / warning / info由 event_type 映射決定)
@@ -66,6 +99,7 @@ async def get_governance_events(
"""
logger.debug(
"governance_events_request",
event_ids=event_id,
event_types=event_type,
from_=from_,
to=to,
@@ -75,6 +109,7 @@ async def get_governance_events(
size=size,
)
return await query_governance_events(
event_ids=event_id,
event_types=event_type,
from_dt=from_,
to_dt=to,
@@ -93,8 +128,9 @@ async def get_governance_events(
async def get_governance_queue(
dispatch_status: Annotated[
str,
Query(pattern="^(pending|dispatched|succeeded|failed)$"),
Query(pattern="^(all|pending|dispatched|executing|succeeded|failed|skipped|cancelled)$"),
] = "pending",
event_type: Annotated[list[str] | None, Query(alias="event_type")] = None,
page: Annotated[int, Query(ge=1)] = 1,
size: Annotated[int, Query(ge=10, le=100)] = 20,
) -> GovernanceQueueResponse:
@@ -104,22 +140,360 @@ async def get_governance_queue(
governance_remediation_dispatch 表由 Track D 建立,尚未完成時
本 endpoint 回傳 { table_pending: true, items: [], total: 0 },不拋 500。
- dispatch_status: pendingdefault/ dispatched / succeeded / failed
- dispatch_status: pendingdefault/ dispatched / executing / succeeded / failed / skipped / cancelled / all
- event_type: 多值過濾(可重複傳)
- page / size: 分頁
"""
logger.debug(
"governance_queue_request",
dispatch_status=dispatch_status,
event_type=event_type,
page=page,
size=size,
)
return await query_governance_queue(
dispatch_status=dispatch_status,
event_types=event_type,
page=page,
size=size,
)
# =============================================================================
# GET /api/v1/ai/governance/km-review-drafts/dedupe
# =============================================================================
@router.get(
"/ai/governance/km-review-drafts/dedupe",
response_model=KnowledgeReviewDraftDedupeResponse,
)
async def get_km_review_draft_dedupe(
limit: Annotated[int, Query(ge=10, le=200)] = 100,
) -> KnowledgeReviewDraftDedupeResponse:
"""
查詢 Hermes KM healthcheck review drafts 的去重 read model。
這是 read-only owner review surface只回傳 canonical / duplicate /
owner_action不自動 archive、不自動 approve/publish KM。
"""
logger.debug("km_review_draft_dedupe_request", limit=limit)
return await query_km_review_draft_dedupe(limit=limit)
# =============================================================================
# POST /api/v1/ai/governance/km-review-drafts/dedupe/{event_id}/archive-duplicates
# =============================================================================
@router.post(
"/ai/governance/km-review-drafts/dedupe/{governance_event_id}/archive-duplicates",
response_model=KnowledgeReviewDraftArchiveResponse,
)
async def post_km_review_draft_archive_duplicates(
governance_event_id: str,
request: KnowledgeReviewDraftArchiveRequest,
) -> KnowledgeReviewDraftArchiveResponse:
"""
Owner 審核後封存 Hermes KM healthcheck duplicate review drafts。
這不是 read endpoint必須明確傳 owner_approved=true且後端會重新比對
最新 dedupe plan。封存為 KnowledgeEntry.status=archived不刪除資料。
"""
logger.info(
"km_review_draft_archive_request",
governance_event_id=governance_event_id,
canonical_entry_id=request.canonical_entry_id,
duplicate_count=len(request.duplicate_entry_ids),
owner=request.owner,
dry_run=request.dry_run,
owner_approved=request.owner_approved,
)
try:
return await archive_km_review_draft_duplicates(
governance_event_id=governance_event_id,
request=request,
)
except KmReviewDraftArchiveError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
# =============================================================================
# GET /api/v1/ai/governance/km-stale-candidates
# =============================================================================
@router.get(
"/ai/governance/km-stale-candidates",
response_model=KnowledgeStaleCandidatesResponse,
)
async def get_km_stale_candidates(
project_id: Annotated[str, Query(min_length=1, max_length=64)] = "awoooi",
limit: Annotated[int, Query(ge=5, le=100)] = 20,
) -> KnowledgeStaleCandidatesResponse:
"""
查詢 stale KM 的 read-only 優先處理清單。
Hermes 可以用這個 read model 產生 KM 更新草稿owner console 則能先看
哪些條目有 Incident / Sentry / SigNoz / PlayBook 脈絡,避免只看到總數。
"""
logger.debug(
"km_stale_candidates_request",
project_id=project_id,
limit=limit,
)
return await query_km_stale_candidates(project_id=project_id, limit=limit)
# =============================================================================
# GET /api/v1/ai/governance/km-stale-owner-reviews
# =============================================================================
@router.get(
"/ai/governance/km-stale-owner-reviews",
response_model=KnowledgeStaleOwnerReviewInboxResponse,
)
async def get_km_stale_owner_reviews(
project_id: Annotated[str, Query(min_length=1, max_length=64)] = "awoooi",
dispatch_status: Annotated[
str,
Query(pattern="^(all|pending|dispatched|executing|succeeded|failed|skipped|cancelled)$"),
] = "pending",
limit: Annotated[int, Query(ge=5, le=100)] = 20,
) -> KnowledgeStaleOwnerReviewInboxResponse:
"""
查詢 stale KM owner-review 工作台。
這是 read-only inbox把 dispatch trail 與 KM priority context 合併,
讓 operator 可以依 P0/P1、score、batch 來源與流程階段逐筆 completion。
"""
logger.debug(
"km_stale_owner_reviews_request",
project_id=project_id,
dispatch_status=dispatch_status,
limit=limit,
)
try:
return await query_km_stale_owner_review_inbox(
project_id=project_id,
dispatch_status=dispatch_status,
limit=limit,
)
except KmStaleOwnerReviewError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
# =============================================================================
# GET /api/v1/ai/governance/km-stale-owner-review-burndown
# =============================================================================
@router.get(
"/ai/governance/km-stale-owner-review-burndown",
response_model=KnowledgeStaleOwnerReviewBurnDownResponse,
)
async def get_km_stale_owner_review_burndown(
project_id: Annotated[str, Query(min_length=1, max_length=64)] = "awoooi",
limit: Annotated[int, Query(ge=1, le=100)] = 20,
) -> KnowledgeStaleOwnerReviewBurnDownResponse:
"""
查詢 stale KM owner-review 完成與 stale ratio burn-down 狀態。
這是 read-only dashboard把 pending review、completion audit、recheck
snapshot 與距離治理門檻的剩餘筆數放在同一個前端面板。
"""
logger.debug(
"km_stale_owner_review_burndown_request",
project_id=project_id,
limit=limit,
)
return await query_km_stale_owner_review_burndown(
project_id=project_id,
limit=limit,
)
# =============================================================================
# GET /api/v1/ai/governance/km-stale-owner-review-completion-queue
# =============================================================================
@router.get(
"/ai/governance/km-stale-owner-review-completion-queue",
response_model=KnowledgeStaleOwnerReviewCompletionQueueResponse,
)
async def get_km_stale_owner_review_completion_queue(
project_id: Annotated[str, Query(min_length=1, max_length=64)] = "awoooi",
status_bucket: Annotated[
str,
Query(pattern="^(all|ready|blocked|completed|failed|pending)$"),
] = "all",
priority_tier: Annotated[list[str] | None, Query(alias="priority_tier")] = None,
recommended_completion_outcome: Annotated[
str,
Query(pattern="^(all|refresh_with_evidence|archive|supersede)$"),
] = "all",
batch_governance_event_id: Annotated[str | None, Query(max_length=120)] = None,
can_preview: bool | None = None,
limit: Annotated[int, Query(ge=1, le=100)] = 20,
) -> KnowledgeStaleOwnerReviewCompletionQueueResponse:
"""
查詢 stale KM owner-review completion 分流。
這是 read-only queue把 active / completed / failed dispatch 拆成
ready、blocked、completed、failed讓前端呈現下一步卡點打開頁面不寫 KM。
"""
logger.debug(
"km_stale_owner_review_completion_queue_request",
project_id=project_id,
status_bucket=status_bucket,
priority_tiers=priority_tier,
recommended_completion_outcome=recommended_completion_outcome,
batch_governance_event_id=batch_governance_event_id,
can_preview=can_preview,
limit=limit,
)
try:
return await query_km_stale_owner_review_completion_queue(
project_id=project_id,
status_bucket=status_bucket,
priority_tiers=priority_tier,
recommended_completion_outcome=recommended_completion_outcome,
batch_governance_event_id=batch_governance_event_id,
can_preview=can_preview,
limit=limit,
)
except KmStaleOwnerReviewError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
# =============================================================================
# POST /api/v1/ai/governance/km-stale-owner-review-completion-queue/batch-preview
# =============================================================================
@router.post(
"/ai/governance/km-stale-owner-review-completion-queue/batch-preview",
response_model=KnowledgeStaleOwnerReviewCompletionBatchPreviewResponse,
)
async def post_km_stale_owner_review_completion_batch_preview(
request: KnowledgeStaleOwnerReviewCompletionBatchPreviewRequest,
) -> KnowledgeStaleOwnerReviewCompletionBatchPreviewResponse:
"""
Preview a bounded set of owner-review completion candidates.
This endpoint is intentionally dry-run only: it does not write KM, does not
enqueue a batch executor, and does not create governance audit rows. Each
item must still be completed through the single-item dry-run + owner confirm
endpoint.
"""
logger.info(
"km_stale_owner_review_completion_batch_preview_request",
project_id=request.project_id,
status_bucket=request.status_bucket,
priority_tiers=request.priority_tiers,
recommended_completion_outcome=request.recommended_completion_outcome,
batch_governance_event_id=request.batch_governance_event_id,
limit=request.limit,
owner=request.owner,
)
try:
return await preview_km_stale_owner_review_completion_batch(request=request)
except KmStaleOwnerReviewError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
# =============================================================================
# POST /api/v1/ai/governance/km-stale-candidates/batch-queue-review
# =============================================================================
@router.post(
"/ai/governance/km-stale-candidates/batch-queue-review",
response_model=KnowledgeStaleOwnerReviewBatchQueueResponse,
)
async def post_km_stale_candidate_batch_queue_review(
request: KnowledgeStaleOwnerReviewBatchQueueRequest,
) -> KnowledgeStaleOwnerReviewBatchQueueResponse:
"""
將 P0/P1 stale KM 批次排入 owner review。
這個 endpoint 只建立 batch audit 與逐筆 owner-review dispatch不改寫 KM。
真正 refresh / archive / supersede 仍需單筆 dry-run fingerprint + owner approval。
"""
logger.info(
"km_stale_candidate_batch_queue_review_request",
project_id=request.project_id,
priority_tiers=request.priority_tiers,
limit=request.limit,
owner=request.owner,
dry_run=request.dry_run,
)
try:
return await batch_queue_km_stale_owner_reviews(request=request)
except KmStaleOwnerReviewError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
# =============================================================================
# POST /api/v1/ai/governance/km-stale-candidates/{entry_id}/queue-review
# =============================================================================
@router.post(
"/ai/governance/km-stale-candidates/{entry_id}/queue-review",
response_model=KnowledgeStaleOwnerReviewResponse,
)
async def post_km_stale_candidate_queue_review(
entry_id: str,
request: KnowledgeStaleOwnerReviewRequest,
) -> KnowledgeStaleOwnerReviewResponse:
"""
將單筆 stale KM candidate 排入 owner review。
這個 endpoint 只建立治理事件與 dispatch work item不修改 KM 內容。
實際 refresh / archive / supersede 仍需 owner 在後續流程確認。
"""
logger.info(
"km_stale_candidate_queue_review_request",
entry_id=entry_id,
owner=request.owner,
dry_run=request.dry_run,
)
try:
return await queue_km_stale_owner_review(entry_id=entry_id, request=request)
except KmStaleOwnerReviewError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
# =============================================================================
# POST /api/v1/ai/governance/km-stale-candidates/{entry_id}/complete-review
# =============================================================================
@router.post(
"/ai/governance/km-stale-candidates/{entry_id}/complete-review",
response_model=KnowledgeStaleOwnerReviewCompleteResponse,
)
async def post_km_stale_candidate_complete_review(
entry_id: str,
request: KnowledgeStaleOwnerReviewCompleteRequest,
) -> KnowledgeStaleOwnerReviewCompleteResponse:
"""
Owner 審核後完成 stale KM 的 refresh / archive / supersede 流程。
必須先 dry-run 取得 fingerprint真正寫入時需 owner_approved=true。
後端會寫 KM、terminal audit dispatch 與 stale ratio recheck dispatch。
"""
logger.info(
"km_stale_candidate_complete_review_request",
entry_id=entry_id,
dispatch_id=request.dispatch_id,
owner=request.owner,
review_outcome=request.review_outcome,
dry_run=request.dry_run,
owner_approved=request.owner_approved,
)
try:
return await complete_km_stale_owner_review(
entry_id=entry_id,
request=request,
)
except KmStaleOwnerReviewError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
# =============================================================================
# GET /api/v1/ai/governance/summary
# =============================================================================

View File

@@ -18,8 +18,15 @@ Endpoints:
from __future__ import annotations
import structlog
from fastapi import APIRouter, Query
from fastapi import APIRouter, HTTPException, Query
from pydantic import BaseModel, Field
from src.services.adr100_remediation_service import (
RemediationMode,
RemediationNotFoundError,
get_adr100_remediation_service,
)
from src.services.adr100_slo_status_service import get_adr100_slo_status_service
from src.services.ai_slo_calculator import AiSloCalculator
logger = structlog.get_logger(__name__)
@@ -27,6 +34,20 @@ logger = structlog.get_logger(__name__)
router = APIRouter()
class RemediationPreviewRequest(BaseModel):
"""ADR-100 remediation preview request."""
work_item_id: str = Field(min_length=1)
mode: RemediationMode = "auto"
class RemediationDryRunRequest(BaseModel):
"""ADR-100 remediation dry-run request."""
work_item_id: str = Field(min_length=1)
mode: RemediationMode = "auto"
@router.get("/ai/slo")
async def get_ai_slo(
force_refresh: bool = Query(False, description="忽略快取,強制重算"),
@@ -50,9 +71,65 @@ async def get_ai_slo(
if cached:
data = cached.to_dict()
data["cache_hit"] = True
data["adr100"] = await get_adr100_slo_status_service().fetch_report()
return data
report = await calc.run()
data = report.to_dict()
data["cache_hit"] = False
data["adr100"] = await get_adr100_slo_status_service().fetch_report()
return data
@router.get("/ai/slo/remediation/preview")
async def preview_ai_slo_remediation(
work_item_id: str = Query(..., min_length=1),
mode: RemediationMode = Query("auto"),
) -> dict:
"""Preview the safe remediation plan for one ADR-100 queue item."""
try:
return await get_adr100_remediation_service().preview(work_item_id, mode)
except RemediationNotFoundError as exc:
raise HTTPException(status_code=404, detail="remediation_work_item_not_found") from exc
@router.post("/ai/slo/remediation/preview")
async def preview_ai_slo_remediation_post(request: RemediationPreviewRequest) -> dict:
"""POST variant for clients that prefer JSON bodies."""
try:
return await get_adr100_remediation_service().preview(
request.work_item_id,
request.mode,
)
except RemediationNotFoundError as exc:
raise HTTPException(status_code=404, detail="remediation_work_item_not_found") from exc
@router.post("/ai/slo/remediation/dry-run")
async def dry_run_ai_slo_remediation(request: RemediationDryRunRequest) -> dict:
"""Run a read-only ADR-100 remediation dry-run."""
try:
return await get_adr100_remediation_service().dry_run(
request.work_item_id,
request.mode,
)
except RemediationNotFoundError as exc:
raise HTTPException(status_code=404, detail="remediation_work_item_not_found") from exc
@router.get("/ai/slo/remediation/history")
async def list_ai_slo_remediation_history(
limit: int = Query(50, ge=1, le=200),
incident_id: str | None = Query(default=None, min_length=1),
work_item_id: str | None = Query(default=None, min_length=1),
) -> dict:
"""List durable ADR-100 remediation dry-run history from alert_operation_log."""
return await get_adr100_remediation_service().history(
limit=limit,
incident_id=incident_id,
work_item_id=work_item_id,
)

View File

@@ -20,6 +20,7 @@ from pydantic import BaseModel
from src.core.config import settings
from src.core.logging import get_logger
from src.core.sse import EventPublisher, EventType, SSEEvent, get_publisher
from src.services.dashboard_metrics_service import fetch_pending_approval_count
from src.services.host_aggregator import AggregatedStatus, HostAggregator
router = APIRouter()
@@ -141,12 +142,14 @@ async def dashboard_update_loop(publisher: EventPublisher) -> None:
try:
# Fetch aggregated status
status = await HostAggregator.fetch_all()
pending_approvals = await fetch_pending_approval_count()
# Publish to all connected clients
event = SSEEvent(
type=EventType.HOST_UPDATE,
data={
"overall_status": status.overall_status,
"pending_approvals": pending_approvals,
"hosts": [
{
"ip": h.ip,
@@ -206,7 +209,9 @@ async def get_dashboard() -> DashboardResponse:
logger.info("dashboard_fetch")
status = await HostAggregator.fetch_all()
return aggregated_to_response(status)
response = aggregated_to_response(status)
response.pending_approvals = await fetch_pending_approval_count()
return response
@router.get("/dashboard/stream")

View File

@@ -13,10 +13,12 @@ leWOOOgo 積木化原則:
建立者: Claude Code (Phase 25 P2)
"""
from typing import Literal
from fastapi import APIRouter, BackgroundTasks, HTTPException
from pydantic import BaseModel, Field
from src.core.csrf import CSRFToken # Phase 20: CSRF Protection
from src.models.drift import (
DriftListResponse,
DriftReport,
@@ -28,6 +30,10 @@ from src.repositories.drift_repository import get_drift_repository
from src.services.drift_adopt_service import get_drift_adopt_service
from src.services.drift_analyzer import get_drift_analyzer
from src.services.drift_detector import get_drift_detector
from src.services.drift_fingerprint_state_service import (
DriftFingerprintStateNotFoundError,
get_drift_fingerprint_state_service,
)
from src.services.drift_interpreter import get_drift_interpreter
from src.services.drift_remediator import get_drift_remediator
from src.utils.timezone import now_taipei
@@ -37,6 +43,42 @@ router = APIRouter(prefix="/drift", tags=["drift"])
# 2026-04-09 Claude Sonnet 4.6: B4 drift_reports 持久化 — 改用 DB repository
class DriftFingerprintHandoffRequest(BaseModel):
"""Record-only handoff request for a stable drift fingerprint."""
report_id: str | None = Field(default=None, min_length=1)
namespace: str | None = Field(default="awoooi-prod", min_length=1)
handoff_kind: Literal[
"open_pr_review",
"manual_investigation",
"zero_diff_pr_cleanup",
] = "open_pr_review"
pr_url: str | None = Field(default=None, min_length=1)
note: str | None = Field(default=None, max_length=500)
class DriftFingerprintRemediationRequest(BaseModel):
"""Record-only remediation request for a stable drift fingerprint."""
report_id: str | None = Field(default=None, min_length=1)
namespace: str | None = Field(default="awoooi-prod", min_length=1)
remediation_kind: Literal[
"live_env_rollback",
"git_adopted",
"git_rollback",
"zero_diff_pr_cleanup",
"manual_noop",
] = "live_env_rollback"
remediation_status: Literal[
"executed_unverified",
"verified_no_drift",
"verification_failed",
] | None = None
verification_report_id: str | None = Field(default=None, min_length=1)
note: str | None = Field(default=None, max_length=1000)
commands_summary: list[str] = Field(default_factory=list, max_length=12)
@router.post("/scan", response_model=DriftScanResponse, summary="觸發漂移掃描")
async def trigger_drift_scan(
request: DriftScanRequest,
@@ -99,6 +141,72 @@ async def list_drift_reports() -> DriftListResponse:
return DriftListResponse(items=items, total=len(items))
@router.get("/fingerprints/state", summary="查詢 Config Drift fingerprint 狀態")
async def get_drift_fingerprint_state(
report_id: str | None = None,
namespace: str | None = "awoooi-prod",
) -> dict:
"""
以 stable fingerprint 聚合漂移狀態。
此 endpoint 只建立 read model重複次數、PR 狀態、是否零 diff、
人工交接歷史與下一步。它不修改 drift / incident / auto-repair 狀態。
"""
svc = get_drift_fingerprint_state_service()
try:
return await svc.get_state(report_id=report_id, namespace=namespace)
except DriftFingerprintStateNotFoundError as exc:
raise HTTPException(status_code=404, detail="drift_report_not_found") from exc
@router.post("/fingerprints/handoff", summary="記錄 Config Drift fingerprint 交接")
async def record_drift_fingerprint_handoff(
request: DriftFingerprintHandoffRequest,
) -> dict:
"""
記錄 stable fingerprint 已轉人工 / PR review 的歷史證據。
安全邊界:只寫 alert_operation_log / timeline_events不修改 drift 狀態、
incident 狀態、自動修復結果,不建立外部 ticket也不 merge PR。
"""
svc = get_drift_fingerprint_state_service()
try:
return await svc.record_handoff(
report_id=request.report_id,
namespace=request.namespace,
handoff_kind=request.handoff_kind,
pr_url=request.pr_url,
note=request.note,
)
except DriftFingerprintStateNotFoundError as exc:
raise HTTPException(status_code=404, detail="drift_report_not_found") from exc
@router.post("/fingerprints/remediation", summary="記錄 Config Drift fingerprint 修復")
async def record_drift_fingerprint_remediation(
request: DriftFingerprintRemediationRequest,
) -> dict:
"""
記錄 stable fingerprint 已完成的修復 / 驗證證據。
安全邊界:只寫 alert_operation_log / timeline_events不修改 drift 狀態、
incident 狀態、自動修復結果,不建立外部 ticket也不執行 kubectl。
"""
svc = get_drift_fingerprint_state_service()
try:
return await svc.record_remediation(
report_id=request.report_id,
namespace=request.namespace,
remediation_kind=request.remediation_kind,
remediation_status=request.remediation_status,
verification_report_id=request.verification_report_id,
note=request.note,
commands_summary=request.commands_summary,
)
except DriftFingerprintStateNotFoundError as exc:
raise HTTPException(status_code=404, detail="drift_report_not_found") from exc
@router.post("/reports/{report_id}/rollback", summary="覆蓋回 Git 狀態")
async def rollback_drift(report_id: str, _csrf_token: CSRFToken) -> dict: # Phase 20: CSRF Protection (驗證用,不需要使用值)
"""

View File

@@ -418,7 +418,9 @@ async def _send_gitea_notification(
logger.debug("gitea_tg_skipped", reason="Bot token not configured")
return
from src.services.telegram_gateway import get_telegram_gateway # type: ignore[import]
from src.services.telegram_gateway import (
get_telegram_gateway, # type: ignore[import]
)
gateway = get_telegram_gateway()
await gateway.initialize()
await gateway.send_alert_notification(message)
@@ -502,15 +504,22 @@ async def handle_pull_request(
review_id = f"gitea-pr-{payload.repository.id}-{pr.number}-{uuid.uuid4().hex[:8]}"
# 背景執行審查 (委派給 Service)
service = get_gitea_webhook_service()
background_tasks.add_task(
service.review_pull_request,
repo=payload.repository,
pr=pr,
sender=payload.sender,
review_id=review_id,
action=payload.action,
)
if settings.MOCK_MODE:
logger.info(
"gitea_pr_review_background_skipped_mock_mode",
review_id=review_id,
repo=payload.repository.full_name,
)
else:
service = get_gitea_webhook_service()
background_tasks.add_task(
service.review_pull_request,
repo=payload.repository,
pr=pr,
sender=payload.sender,
review_id=review_id,
action=payload.action,
)
logger.info(
"gitea_pr_review_scheduled",
@@ -561,17 +570,24 @@ async def handle_push(
review_id = f"gitea-push-{payload.repository.id}-{payload.after[:8]}-{uuid.uuid4().hex[:8]}"
# 背景執行審查 (委派給 Service)
service = get_gitea_webhook_service()
background_tasks.add_task(
service.review_push,
repo=payload.repository,
commits=commits,
sender=payload.sender,
review_id=review_id,
ref=ref,
before_sha=payload.before,
after_sha=payload.after,
)
if settings.MOCK_MODE:
logger.info(
"gitea_push_review_background_skipped_mock_mode",
review_id=review_id,
repo=payload.repository.full_name,
)
else:
service = get_gitea_webhook_service()
background_tasks.add_task(
service.review_push,
repo=payload.repository,
commits=commits,
sender=payload.sender,
review_id=review_id,
ref=ref,
before_sha=payload.before,
after_sha=payload.after,
)
logger.info(
"gitea_push_review_scheduled",

View File

@@ -11,7 +11,7 @@ Endpoints:
Components Checked:
- PostgreSQL (192.168.0.188:5432)
- Redis (192.168.0.188:6380)
- Ollama (settings.OLLAMA_URL / ADR-110 provider pool)
- Ollama ADR-110 provider pool (GCP-A -> GCP-B -> 111)
- OpenClaw (192.168.0.188:8089)
- SigNoz (192.168.0.188:3301)
"""
@@ -26,9 +26,16 @@ from pydantic import BaseModel
from src.core.config import settings
from src.core.logging import get_logger
from src.services.health_check_service import get_health_check_service
from src.services.ollama_endpoint_circuit_breaker import (
get_ollama_endpoint_cooldown_remaining_seconds,
record_ollama_endpoint_failure,
record_ollama_endpoint_success,
)
from src.services.ollama_endpoint_resolver import resolve_ollama_order
router = APIRouter()
logger = get_logger("awoooi.health")
CORE_COMPONENTS = ("api", "postgresql", "redis", "ollama", "openclaw", "signoz")
# =============================================================================
@@ -40,6 +47,11 @@ class ComponentHealth(BaseModel):
status: Literal["up", "down", "degraded"]
latency_ms: float | None = None
error: str | None = None
provider_name: str | None = None
diagnosis_code: str | None = None
retry_after_seconds: float | None = None
cooldown_remaining_seconds: float | None = None
is_cooldown: bool = False
class HealthResponse(BaseModel):
@@ -50,6 +62,7 @@ class HealthResponse(BaseModel):
mock_mode: bool
timestamp: datetime
components: dict[str, ComponentHealth]
ollama_route_order: list[str] = []
# =============================================================================
@@ -106,8 +119,125 @@ async def check_redis() -> ComponentHealth:
async def check_ollama() -> ComponentHealth:
"""Async Ollama health check via /api/tags"""
return await _http_health_check("ollama", settings.OLLAMA_URL, "/api/tags")
"""Async aggregate Ollama health check via ADR-110 provider chain."""
aggregate, _details = await check_ollama_provider_chain()
return aggregate
async def check_ollama_provider_chain() -> tuple[ComponentHealth, dict[str, ComponentHealth]]:
"""
Check the full Ollama provider chain.
The aggregate ``ollama`` component represents route availability:
- up: GCP-A is reachable
- degraded: GCP-A is unavailable but GCP-B or 111 is reachable
- down: no configured Ollama endpoint is reachable
"""
selections = tuple(
selection
for selection in resolve_ollama_order("healthcheck")
if selection.url and selection.provider_name != "ollama_unconfigured"
)
if not selections:
aggregate = ComponentHealth(
status="down",
error="no Ollama endpoints configured",
)
return aggregate, {}
checked = await asyncio.gather(
*(
_ollama_endpoint_health_check(selection.provider_name, selection.url)
for selection in selections
)
)
details = {
selection.provider_name: result
for selection, result in zip(selections, checked, strict=False)
}
primary = selections[0]
primary_status = details[primary.provider_name].status
if primary.provider_name == "ollama_gcp_a" and primary_status == "up":
return details[primary.provider_name], details
first_available = next(
(
selection
for selection in selections
if details[selection.provider_name].status == "up"
),
None,
)
if first_available:
fallback = details[first_available.provider_name]
return (
ComponentHealth(
status="degraded",
latency_ms=fallback.latency_ms,
error=f"primary unavailable; fallback active: {first_available.provider_name}",
),
details,
)
errors = ", ".join(
f"{provider}={health.error or health.status}"
for provider, health in details.items()
)
return (
ComponentHealth(
status="down",
error=f"all Ollama endpoints unavailable: {errors}",
),
details,
)
async def _ollama_endpoint_health_check(name: str, url: str) -> ComponentHealth:
cooldown_remaining = get_ollama_endpoint_cooldown_remaining_seconds(url)
if cooldown_remaining > 0:
return ComponentHealth(
status="down",
error=f"recent endpoint failure cooldown: {cooldown_remaining:.0f}s",
provider_name=name,
diagnosis_code="endpoint_cooldown",
retry_after_seconds=round(cooldown_remaining, 1),
cooldown_remaining_seconds=round(cooldown_remaining, 1),
is_cooldown=True,
)
result = await _http_health_check(name, url, "/api/tags")
result.provider_name = name
if result.status == "up":
result.diagnosis_code = "endpoint_reachable"
record_ollama_endpoint_success(url)
else:
result.diagnosis_code = _classify_ollama_endpoint_failure(name, result.error)
record_ollama_endpoint_failure(url)
return result
def _classify_ollama_endpoint_failure(
provider_name: str,
error: str | None,
) -> str:
"""Return a stable diagnosis code for UI/alert rendering."""
normalized_error = (error or "").lower()
if "cooldown" in normalized_error:
return "endpoint_cooldown"
if "502" in normalized_error or "bad gateway" in normalized_error:
return (
"local_proxy_upstream_unreachable"
if provider_name == "ollama_local"
else "proxy_upstream_unreachable"
)
if "timeout" in normalized_error:
return "endpoint_timeout"
if "connection refused" in normalized_error:
return "endpoint_connection_refused"
if "no route to host" in normalized_error or "network is unreachable" in normalized_error:
return "endpoint_network_unreachable"
return "endpoint_unreachable"
async def check_openclaw() -> ComponentHealth:
@@ -120,6 +250,30 @@ async def check_signoz() -> ComponentHealth:
return await _http_health_check("signoz", settings.SIGNOZ_URL, "/api/v1/health")
def _determine_overall_status(
components: dict[str, ComponentHealth],
) -> Literal["healthy", "degraded", "unhealthy"]:
"""Determine overall health from core aggregate components only."""
statuses = [
components[name].status
for name in CORE_COMPONENTS
if name in components
]
down_count = statuses.count("down")
degraded_count = statuses.count("degraded")
critical_down = (
components.get("postgresql", ComponentHealth(status="down")).status == "down"
or components.get("redis", ComponentHealth(status="down")).status == "down"
)
if critical_down or down_count >= 3:
return "unhealthy"
if down_count >= 1 or degraded_count > 0:
return "degraded"
return "healthy"
# =============================================================================
# Endpoints
# =============================================================================
@@ -142,34 +296,28 @@ async def get_health() -> HealthResponse:
results = await asyncio.gather(
check_postgresql(),
check_redis(),
check_ollama(),
check_ollama_provider_chain(),
check_openclaw(),
check_signoz(),
)
ollama_aggregate, ollama_details = results[2]
components = {
"api": ComponentHealth(status="up", latency_ms=0.0),
"postgresql": results[0],
"redis": results[1],
"ollama": results[2],
"ollama": ollama_aggregate,
"openclaw": results[3],
"signoz": results[4],
}
components.update(ollama_details)
# Determine overall status
statuses = [c.status for c in components.values()]
down_count = statuses.count("down")
degraded_count = statuses.count("degraded")
# Critical services: postgresql, redis
critical_down = components["postgresql"].status == "down" or components["redis"].status == "down"
if critical_down or down_count >= 3:
overall_status: Literal["healthy", "degraded", "unhealthy"] = "unhealthy"
elif down_count >= 1 or degraded_count > 0:
overall_status = "degraded"
else:
overall_status = "healthy"
overall_status = _determine_overall_status(components)
ollama_route_order = [
selection.provider_name
for selection in resolve_ollama_order("healthcheck")
if selection.url and selection.provider_name != "ollama_unconfigured"
]
logger.info(
"health_check_complete",
@@ -185,6 +333,7 @@ async def get_health() -> HealthResponse:
mock_mode=settings.MOCK_MODE,
timestamp=datetime.now(UTC),
components=components,
ollama_route_order=ollama_route_order,
)

View File

@@ -134,6 +134,7 @@ class IncidentTimelineResponse(BaseModel):
timeline: list[IncidentTimelineStage] = Field(default_factory=list)
events: list[IncidentTimelineEvent] = Field(default_factory=list)
ascii_timeline: str
reconciliation: dict[str, Any] = Field(default_factory=dict)
# =============================================================================

View File

@@ -13,9 +13,11 @@ from src.api.v1.platform.events import router as events_router
from src.api.v1.platform.operator_runs import router as operator_runs_router
from src.api.v1.platform.runs import router as runs_router
from src.api.v1.platform.tenants import router as tenants_router
from src.api.v1.platform.truth_chain import router as truth_chain_router
router = APIRouter()
router.include_router(events_router)
router.include_router(truth_chain_router)
# 2026-05-06 Codex: FastAPI 依註冊順序比對路由。Operator Console 的
# `/runs/list` 必須排在 `/runs/{run_id}` 前面,否則 `list` 會被當成
# run_id造成前端 Run 監控頁 HTTP 422。

View File

@@ -6,13 +6,32 @@ AwoooP Operator Console — Channel Events API
from __future__ import annotations
from datetime import datetime
from typing import Any
from datetime import UTC, datetime
from typing import Annotated, Any, Literal
from uuid import UUID
from fastapi import APIRouter, Query
from pydantic import BaseModel
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel, Field
from src.core.awooop_operator_auth import (
AwoooPOperatorPrincipal,
verify_awooop_operator,
)
from src.services.channel_event_dossier_service import (
RecurrenceWorkItemHandoffKind,
RecurrenceWorkItemMode,
RecurrenceWorkItemNotFoundError,
SourceCorrelationReviewDecision,
fetch_channel_event_dossier,
fetch_channel_event_dossier_coverage,
fetch_channel_event_dossier_recurrence,
fetch_recurrence_work_item_dry_run,
fetch_recurrence_work_item_handoff,
fetch_recurrence_work_item_preview,
fetch_source_correlation_apply,
fetch_source_correlation_review_decision,
)
from src.services.channel_hub import record_external_alert_event
from src.services.platform_operator_service import list_recent_channel_events
router = APIRouter()
@@ -35,6 +54,513 @@ class RecentEventsResponse(BaseModel):
limit: int
class ChannelEventDossierItem(BaseModel):
event_id: UUID
project_id: str
channel_type: str
provider: str | None
stage: str
provider_event_id: str
content_preview: str | None
content_redacted: str | None
has_redacted_content: bool
redaction_version: str | None
source_url: str | None
content_sha256: str | None
content_length: int | None
source_refs: dict[str, Any]
source_ref_count: int
log_correlation: dict[str, Any]
alertname: str | None
severity: str | None
namespace: str | None
target_resource: str | None
fingerprint: str | None
is_duplicate: bool
provider_ts: datetime | None
received_at: datetime
class ChannelEventDossierSummary(BaseModel):
source_count: int
duplicate_total: int
redacted_total: int
source_ref_total: int
class ChannelEventDossierResponse(BaseModel):
events: list[ChannelEventDossierItem]
total: int
limit: int
summary: ChannelEventDossierSummary
class ChannelEventProviderCoverage(BaseModel):
provider: str
total: int
duplicate_total: int
redacted_total: int
source_ref_total: int
missing_source_refs_total: int
sentry_ref_total: int
signoz_ref_total: int
alert_ref_total: int
latest_received_at: datetime | None
class ChannelEventDossierCoverageSummary(BaseModel):
source_count: int
source_envelope_total: int
missing_source_envelope_total: int
with_source_refs_total: int
missing_source_refs_total: int
duplicate_total: int
redacted_total: int
source_ref_total: int
sentry_ref_total: int
signoz_ref_total: int
alert_ref_total: int
latest_received_at: datetime | None
class ChannelEventDossierCoverageResponse(BaseModel):
project_id: str
limit: int
summary: ChannelEventDossierCoverageSummary
providers: list[ChannelEventProviderCoverage]
SourceProviderName = Literal["sentry", "signoz"]
class SourceProviderHeartbeatRequest(BaseModel):
"""Low-noise freshness heartbeat for external source-provider mirrors."""
project_id: str = Field(default="awoooi", min_length=1, max_length=64)
providers: list[SourceProviderName] = Field(
default_factory=lambda: ["sentry", "signoz"],
min_length=1,
max_length=2,
)
reason: str = Field(
default="scheduled_provider_freshness_smoke",
min_length=1,
max_length=120,
)
run_ref: str | None = Field(default=None, max_length=120)
class SourceProviderHeartbeatItem(BaseModel):
provider: SourceProviderName
event_id: str
conversation_event_id: UUID
class SourceProviderHeartbeatResponse(BaseModel):
status: str
project_id: str
items: list[SourceProviderHeartbeatItem]
class ChannelEventRecurrenceSummary(BaseModel):
source_event_total: int
recurrence_group_total: int
recurrent_group_total: int
duplicate_event_total: int
linked_run_total: int
unlinked_event_total: int
auto_repair_linked_total: int = 0
verified_repair_group_total: int = 0
open_work_item_group_total: int = 0
manual_gate_group_total: int = 0
automation_gap_group_total: int = 0
failed_repair_group_total: int = 0
source_correlation_review_group_total: int = 0
source_correlation_decision_recorded_group_total: int = 0
source_correlation_applied_group_total: int = 0
latest_received_at: datetime | None
class ChannelEventRecurrenceItem(BaseModel):
recurrence_key: str
provider: str | None
alertname: str | None
severity: str | None
namespace: str | None
target_resource: str | None
fingerprint: str | None
latest_stage: str | None = None
latest_event_id: UUID | None
latest_provider_event_id: str | None
latest_content_preview: str | None
latest_run_id: UUID | None
latest_run_state: str | None
latest_agent_id: str | None
latest_incident_id: str | None = None
incident_ids: list[str] = Field(default_factory=list)
repair_summary: dict[str, Any] | None = None
work_item: dict[str, Any] | None = None
source_correlation_review: dict[str, Any] | None = None
source_correlation_apply: dict[str, Any] | None = None
occurrence_total: int
duplicate_total: int
linked_run_total: int
source_ref_total: int
missing_source_refs_total: int
sentry_ref_total: int
signoz_ref_total: int
alert_ref_total: int
stage_counts: dict[str, int] = Field(default_factory=dict)
run_state_counts: dict[str, int]
first_received_at: datetime | None
latest_received_at: datetime | None
class ChannelEventRecurrenceResponse(BaseModel):
project_id: str
limit: int
summary: ChannelEventRecurrenceSummary
items: list[ChannelEventRecurrenceItem]
class RecurrenceWorkItemDryRunRequest(BaseModel):
"""AwoooP recurrence work item dry-run request."""
project_id: str | None = Field(default=None, min_length=1)
work_item_id: str = Field(min_length=1)
mode: RecurrenceWorkItemMode = "auto"
provider: str | None = Field(default=None, min_length=1)
limit: int = Field(default=300, ge=1, le=300)
class RecurrenceWorkItemHandoffRequest(BaseModel):
"""AwoooP recurrence work item handoff request."""
project_id: str | None = Field(default=None, min_length=1)
work_item_id: str = Field(min_length=1)
mode: RecurrenceWorkItemMode = "auto"
handoff_kind: RecurrenceWorkItemHandoffKind = "ticket_proposal"
provider: str | None = Field(default=None, min_length=1)
limit: int = Field(default=300, ge=1, le=300)
class SourceCorrelationReviewDecisionRequest(BaseModel):
"""Record-only source evidence review decision."""
project_id: str | None = Field(default=None, min_length=1)
work_item_id: str = Field(min_length=1)
decision: SourceCorrelationReviewDecision
target_incident_id: str | None = Field(default=None, min_length=1, max_length=30)
reviewer_id: str = Field(default="operator_console", min_length=1, max_length=100)
operator_note: str | None = Field(default=None, max_length=500)
provider: str | None = Field(default=None, min_length=1)
limit: int = Field(default=300, ge=1, le=300)
class SourceCorrelationApplyRequest(BaseModel):
"""Append-only source evidence link apply request."""
project_id: str | None = Field(default=None, min_length=1)
work_item_id: str = Field(min_length=1)
reviewer_id: str = Field(default="operator_console", min_length=1, max_length=100)
operator_note: str | None = Field(default=None, max_length=500)
provider: str | None = Field(default=None, min_length=1)
limit: int = Field(default=300, ge=1, le=300)
@router.get(
"/events/dossier",
response_model=ChannelEventDossierResponse,
summary="查詢 Channel Event 來源卷宗",
description=(
"返回 redacted inbound source envelope供 AwoooP Run Detail 顯示"
"告警來源、source refs、Sentry / SignOz / Alertmanager 關聯與去重狀態。"
),
)
async def get_event_dossier(
project_id: str | None = Query(None, description="租戶 ID可選"),
run_id: UUID | None = Query(None, description="Run ID可選"),
provider_event_id: str | None = Query(
None, description="provider_event_id可選"
),
limit: int = Query(20, ge=1, le=50, description="最多返回筆數"),
) -> dict[str, Any]:
return await fetch_channel_event_dossier(
project_id=project_id,
run_id=run_id,
provider_event_id=provider_event_id,
limit=limit,
)
@router.get(
"/events/dossier/coverage",
response_model=ChannelEventDossierCoverageResponse,
summary="查詢 Channel Event 來源卷宗覆蓋率",
description=(
"返回近期 inbound event 的 source_envelope / source_refs / 去重 / "
"Sentry / SignOz 關聯覆蓋率,供 AwoooP Run List 顯示告警是否已入庫。"
),
)
async def get_event_dossier_coverage(
project_id: str | None = Query(None, description="租戶 ID可選"),
provider: str | None = Query(
None, description="provider可選如 sentry / signoz"
),
limit: int = Query(100, ge=1, le=200, description="最多納入統計筆數"),
) -> dict[str, Any]:
return await fetch_channel_event_dossier_coverage(
project_id=project_id,
provider=provider,
limit=limit,
)
@router.post(
"/events/dossier/provider-heartbeat",
response_model=SourceProviderHeartbeatResponse,
summary="寫入 Sentry / SignOz 來源卷宗 freshness heartbeat",
description=(
"受 AwoooP operator key 保護的低噪音 smoke。只寫入來源卷宗與"
"completed shadow run不建立 Incident、不送 Telegram、不宣稱真實上游告警。"
),
)
async def create_source_provider_heartbeat(
payload: SourceProviderHeartbeatRequest,
operator: Annotated[
AwoooPOperatorPrincipal,
Depends(verify_awooop_operator),
],
) -> dict[str, Any]:
timestamp = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
items: list[dict[str, Any]] = []
for provider in payload.providers:
event_id = f"heartbeat-{timestamp}"
event_uuid = await record_external_alert_event(
project_id=payload.project_id,
provider=provider,
event_id=event_id,
stage="heartbeat",
title="SourceProviderHeartbeat",
severity="info",
namespace="awoooi-prod",
target_resource="source-provider-ingestion",
fingerprint=f"source-provider-heartbeat:{provider}",
labels={
"provider": provider,
"synthetic": "true",
"alert_category": "alertchain_provider_freshness",
"telegram": "not_sent",
"incident": "not_created",
},
annotations={
"summary": (
"Low-noise provider freshness smoke; verifies AwoooP "
"source dossier ingestion without creating an incident."
),
"reason": payload.reason,
},
payload={
"reason": payload.reason,
"run_ref": payload.run_ref,
"operator_id": operator.operator_id,
"auth_method": operator.auth_method,
"synthetic": True,
"side_effects": {
"incident_created": False,
"telegram_sent": False,
"approval_created": False,
},
},
)
if event_uuid is None:
raise HTTPException(
status_code=500,
detail=f"{provider} provider heartbeat was not recorded",
)
items.append(
{
"provider": provider,
"event_id": event_id,
"conversation_event_id": event_uuid,
}
)
return {
"status": "recorded",
"project_id": payload.project_id,
"items": items,
}
@router.get(
"/events/dossier/recurrence",
response_model=ChannelEventRecurrenceResponse,
summary="查詢 Channel Event 重複發生與關聯 Run 狀態",
description=(
"將近期 inbound source events 依 fingerprint / alertname / namespace / target 分組,"
"顯示重複發生次數、去重數、source refs 與最新 linked run 狀態。"
),
)
async def get_event_dossier_recurrence(
project_id: str | None = Query(None, description="租戶 ID可選"),
provider: str | None = Query(
None, description="provider可選如 alertmanager / sentry / signoz"
),
limit: int = Query(100, ge=1, le=300, description="最多納入統計筆數"),
) -> dict[str, Any]:
return await fetch_channel_event_dossier_recurrence(
project_id=project_id,
provider=provider,
limit=limit,
)
@router.get(
"/events/dossier/recurrence/work-item/preview",
summary="預覽重複告警工作項的安全處理計畫",
description=(
"依 recurrence read model 找出指定 work_item返回下一步、pre-flight checks "
"與 read-only / no-write 保證;不修改 incident、auto-repair 或 ticket 狀態。"
),
)
async def preview_event_recurrence_work_item(
work_item_id: str = Query(..., min_length=1, description="recurrence work_item_id"),
project_id: str | None = Query(None, description="租戶 ID可選"),
provider: str | None = Query(
None, description="provider可選如 alertmanager / sentry / signoz"
),
mode: RecurrenceWorkItemMode = Query("auto", description="預覽模式"),
limit: int = Query(300, ge=1, le=300, description="最多納入統計筆數"),
) -> dict[str, Any]:
try:
return await fetch_recurrence_work_item_preview(
project_id=project_id,
work_item_id=work_item_id,
mode=mode,
provider=provider,
limit=limit,
)
except RecurrenceWorkItemNotFoundError as exc:
raise HTTPException(
status_code=404,
detail="recurrence_work_item_not_found",
) from exc
@router.post(
"/events/dossier/recurrence/work-item/dry-run",
summary="乾跑重複告警工作項的安全處理流程",
description=(
"依 recurrence read model 產生 dry-run 結果並寫入 pre-flight history"
"但不修改 incident、auto-repair 或 ticket 狀態。"
),
)
async def dry_run_event_recurrence_work_item(
request: RecurrenceWorkItemDryRunRequest,
) -> dict[str, Any]:
try:
return await fetch_recurrence_work_item_dry_run(
project_id=request.project_id,
work_item_id=request.work_item_id,
mode=request.mode,
provider=request.provider,
limit=request.limit,
)
except RecurrenceWorkItemNotFoundError as exc:
raise HTTPException(
status_code=404,
detail="recurrence_work_item_not_found",
) from exc
@router.post(
"/events/dossier/recurrence/work-item/handoff",
summary="記錄重複告警工作項的交接提案",
description=(
"依 recurrence read model 與 dry-run 結果記錄 ticket proposal / 人工接手歷史,"
"但不修改 incident、auto-repair 或外部 ticket 狀態。"
),
)
async def handoff_event_recurrence_work_item(
request: RecurrenceWorkItemHandoffRequest,
) -> dict[str, Any]:
try:
return await fetch_recurrence_work_item_handoff(
project_id=request.project_id,
work_item_id=request.work_item_id,
mode=request.mode,
handoff_kind=request.handoff_kind,
provider=request.provider,
limit=request.limit,
)
except RecurrenceWorkItemNotFoundError as exc:
raise HTTPException(
status_code=404,
detail="recurrence_work_item_not_found",
) from exc
@router.post(
"/events/dossier/recurrence/source-correlation/review",
summary="記錄來源證據與 Incident 配對審核結果",
description=(
"針對 source_correlation_review work item 記錄 operator 審核決定。"
"本 API 僅寫入 alert_operation_log / 可選 timeline_events"
"不修改 Incident 狀態、不回寫 source event、不建立外部 ticket。"
),
)
async def review_source_correlation_work_item(
request: SourceCorrelationReviewDecisionRequest,
) -> dict[str, Any]:
try:
return await fetch_source_correlation_review_decision(
project_id=request.project_id,
work_item_id=request.work_item_id,
decision=request.decision,
target_incident_id=request.target_incident_id,
reviewer_id=request.reviewer_id,
operator_note=request.operator_note,
provider=request.provider,
limit=request.limit,
)
except RecurrenceWorkItemNotFoundError as exc:
raise HTTPException(
status_code=404,
detail="recurrence_work_item_not_found",
) from exc
@router.post(
"/events/dossier/recurrence/source-correlation/apply",
summary="套用已確認的來源證據與 Incident 配對",
description=(
"只接受已寫入 accepted review 的 source_correlation_review work item。"
"成功時以 append-only 方式新增 source_correlation_linked 來源事件,"
"並寫入 alert_operation_log / timeline_events。"
"不修改 Incident 狀態、不修改 auto-repair 結果、不建立外部 ticket。"
),
)
async def apply_source_correlation_work_item(
request: SourceCorrelationApplyRequest,
) -> dict[str, Any]:
try:
return await fetch_source_correlation_apply(
project_id=request.project_id,
work_item_id=request.work_item_id,
reviewer_id=request.reviewer_id,
operator_note=request.operator_note,
provider=request.provider,
limit=request.limit,
)
except RecurrenceWorkItemNotFoundError as exc:
raise HTTPException(
status_code=404,
detail="recurrence_work_item_not_found",
) from exc
@router.get(
"/events/recent",
response_model=RecentEventsResponse,
@@ -47,7 +573,9 @@ class RecentEventsResponse(BaseModel):
async def list_recent_events(
project_id: str | None = Query(None, description="租戶 ID可選"),
channel_type: str | None = Query(None, description="通道類型(可選)"),
provider_prefix: str | None = Query(None, description="provider_event_id 前綴(可選)"),
provider_prefix: str | None = Query(
None, description="provider_event_id 前綴(可選)"
),
limit: int = Query(20, ge=1, le=100, description="最多返回筆數"),
) -> dict[str, Any]:
return await list_recent_channel_events(

View File

@@ -25,12 +25,24 @@ from src.core.awooop_operator_auth import (
from src.services.platform_operator_service import (
decide_approval as decide_approval_svc,
)
from src.services.platform_operator_service import (
get_ai_route_status as get_ai_route_status_svc,
)
from src.services.platform_operator_service import (
get_awooop_status_chain as get_awooop_status_chain_svc,
)
from src.services.platform_operator_service import (
get_run_detail as get_run_detail_svc,
)
from src.services.platform_operator_service import (
list_cicd_events as list_cicd_events_svc,
)
from src.services.platform_operator_service import (
list_approvals as list_approvals_svc,
)
from src.services.platform_operator_service import (
list_callback_replies as list_callback_replies_svc,
)
from src.services.platform_operator_service import (
list_runs as list_runs_svc,
)
@@ -51,6 +63,8 @@ class RunItem(BaseModel):
step_count: int
created_at: datetime
timeout_at: datetime | None
remediation_summary: dict[str, Any] | None = None
callback_reply_summary: dict[str, Any] | None = None
class ListRunsResponse(BaseModel):
@@ -60,12 +74,167 @@ class ListRunsResponse(BaseModel):
per_page: int
class OperatorSummaryCacheInfo(BaseModel):
schema_version: str = "operator_summary_cache_v1"
status: str
source: str
ttl_seconds: int
age_seconds: float = 0.0
stored_at: datetime
expires_at: datetime
class CallbackReplyItem(BaseModel):
message_id: UUID
run_id: UUID
project_id: str
status: str
needs_human: bool
action: str | None = None
incident_id: str | None = None
event_at: datetime | None = None
channel_type: str
message_type: str
send_status: str
send_error: str | None = None
provider_message_id: str | None = None
triggered_by_state: str | None = None
content_preview: str | None = None
run_state: str | None = None
agent_id: str | None = None
run_created_at: datetime | None = None
callback_reply: dict[str, Any]
awooop_status_chain: dict[str, Any] | None = None
persisted_awooop_status_chain: dict[str, Any] | None = None
km_stale_completion_summary: dict[str, Any] | None = None
persisted_km_stale_completion_summary: dict[str, Any] | None = None
evidence_capture_status: dict[str, Any] | None = None
run_detail_href: str | None = None
class OutboundReplyMarkupGapPrefix(BaseModel):
prefix: str
total: int
recent_24h_total: int = 0
first_sent_at: datetime | None = None
last_sent_at: datetime | None = None
class CallbackReplyAuditSummary(BaseModel):
schema_version: str
project_id: str
outbound_total: int
outbound_source_envelope_total: int
outbound_source_refs_total: int
outbound_trace_ref_total: int = 0
outbound_incident_ref_total: int
outbound_reply_markup_total: int = 0
outbound_reply_markup_missing_incident_ref_total: int = 0
outbound_reply_markup_missing_incident_ref_recent_1h_total: int = 0
outbound_reply_markup_missing_incident_ref_recent_24h_total: int = 0
outbound_reply_markup_missing_incident_ref_latest_sent_at: datetime | None = None
outbound_reply_markup_missing_trace_ref_total: int = 0
outbound_reply_markup_missing_trace_ref_recent_1h_total: int = 0
outbound_reply_markup_missing_trace_ref_recent_24h_total: int = 0
outbound_reply_markup_missing_trace_ref_latest_sent_at: datetime | None = None
outbound_reply_markup_trace_ref_gap_status: str = "clean"
outbound_reply_markup_trace_ref_gap_next_action: str = "none"
outbound_reply_markup_trace_ref_after_gap_total: int = 0
outbound_reply_markup_trace_ref_after_gap_first_sent_at: datetime | None = None
outbound_reply_markup_trace_ref_after_gap_latest_sent_at: datetime | None = None
outbound_reply_markup_trace_ref_gap_recovery_status: str = "not_needed"
outbound_reply_markup_missing_incident_ref_top_prefixes: list[
OutboundReplyMarkupGapPrefix
] = Field(default_factory=list)
outbound_reply_markup_missing_trace_ref_top_prefixes: list[
OutboundReplyMarkupGapPrefix
] = Field(default_factory=list)
outbound_failed_total: int
callback_total: int
callback_sent_total: int
callback_fallback_total: int
callback_rescue_total: int
callback_failed_total: int
callback_detail_total: int
callback_history_total: int
callback_snapshot_captured_total: int
callback_snapshot_partial_total: int
callback_snapshot_missing_total: int
callback_incident_total: int
inbound_callback_total: int = 0
inbound_callback_recent_24h_total: int = 0
inbound_callback_latest_at: datetime | None = None
inbound_callback_mirror_status: str = "no_callback_observed"
inbound_callback_next_action: str = "press_any_telegram_callback_after_rollout"
snapshot_status: str
next_action: str
latest_outbound_at: datetime | None = None
latest_callback_at: datetime | None = None
class ListCallbackRepliesResponse(BaseModel):
items: list[CallbackReplyItem]
total: int
page: int
per_page: int
summary: CallbackReplyAuditSummary | None = None
cache: OperatorSummaryCacheInfo | None = None
class CicdEventItem(BaseModel):
id: str
project_id: str
alertname: str
stage: str | None = None
status: str | None = None
severity: str | None = None
commit_sha: str | None = None
triggered_by: str | None = None
duration_seconds: int = 0
summary: str | None = None
description: str | None = None
workflow_url: str | None = None
alert_id: str | None = None
source: str | None = None
action_detail: str | None = None
needs_attention: bool = False
created_at: datetime
class ListCicdEventsResponse(BaseModel):
items: list[CicdEventItem]
total: int
limit: int
class AiRouteStatusResponse(BaseModel):
schema_version: str
workload_type: str
policy_order: list[dict[str, Any]]
selected_provider: str | None = None
selected_url: str | None = None
selected_model: str | None = None
fallback_chain: list[dict[str, Any]]
route_reason: str
route_source: str
route_error: str | None = None
health: dict[str, dict[str, Any]]
lane_mode: str | None = None
active_lane: dict[str, Any] | None = None
skipped_lanes: list[dict[str, Any]] = Field(default_factory=list)
operator_action: dict[str, Any] | None = None
repair_evidence: dict[str, Any] | None = None
checked_at: datetime
class ApprovalItem(BaseModel):
run_id: UUID
project_id: str
agent_id: str
created_at: datetime
timeout_at: datetime | None
remediation_summary: dict[str, Any] | None = None
awooop_status_chain: dict[str, Any] | None = None
class ListApprovalsResponse(BaseModel):
@@ -95,7 +264,8 @@ class DecideApprovalResponse(BaseModel):
response_model=ListRunsResponse,
summary="列出 Runs",
description=(
"返回 awooop_run_state 記錄,支援 project_id / state filter 與分頁。\n\n"
"返回 awooop_run_state 記錄,支援 project_id / state / remediation_status / "
"callback_reply_status / incident_id filter 與分頁。\n\n"
"- 按 created_at DESC 排序\n"
"- 注意:此路徑為 /runs/list 以避免與 runs.py 的 /runs/{run_id} 衝突"
),
@@ -103,14 +273,102 @@ class DecideApprovalResponse(BaseModel):
async def list_runs(
project_id: str | None = Query(None, description="租戶 ID可選"),
state: str | None = Query(None, description="Run 狀態 filter可選"),
remediation_status: str | None = Query(
None,
description="AI 證據狀態 filterno_evidence/mcp_observed/read_only_dry_run/write_observed/blocked/observed",
),
callback_reply_status: str | None = Query(
None,
description="Telegram callback reply 狀態 filterno_callback/sent/fallback_sent/rescue_sent/failed/observed",
),
incident_id: str | None = Query(None, description="關聯 Incident ID filter可選"),
page: int = Query(1, ge=1, description="頁碼,從 1 開始"),
per_page: int = Query(_DEFAULT_PER_PAGE, ge=1, le=_MAX_PER_PAGE, description="每頁筆數"),
) -> dict[str, Any]:
return await list_runs_svc(
project_id=project_id, state=state, page=page, per_page=per_page
project_id=project_id,
state=state,
remediation_status=remediation_status,
callback_reply_status=callback_reply_status,
incident_id=incident_id,
page=page,
per_page=per_page,
)
@router.get(
"/runs/callback-replies",
response_model=ListCallbackRepliesResponse,
summary="列出 Telegram Callback Reply Evidence",
description=(
"從 AwoooP outbound mirror 查詢 Telegram 詳情 / 歷史 callback reply 的"
"送達、fallback、救援與失敗證據只讀不修改 incident、run 或 Telegram 狀態。"
),
)
async def list_callback_replies(
project_id: str | None = Query(None, description="租戶 ID可選"),
callback_reply_status: str | None = Query(
None,
description="Telegram callback reply 狀態 filtersent/fallback_sent/rescue_sent/failed/observed/no_callback",
),
action: str | None = Query(None, description="Callback action filter例如 detail/history"),
incident_id: str | None = Query(None, description="關聯 Incident ID filter可選"),
page: int = Query(1, ge=1, description="頁碼,從 1 開始"),
per_page: int = Query(20, ge=1, le=_MAX_PER_PAGE, description="每頁筆數"),
refresh: bool = Query(False, description="略過短 TTL 快取並重新聚合"),
) -> dict[str, Any]:
return await list_callback_replies_svc(
project_id=project_id,
callback_reply_status=callback_reply_status,
action=action,
incident_id=incident_id,
page=page,
per_page=per_page,
refresh=refresh,
)
@router.get(
"/cicd/events",
response_model=ListCicdEventsResponse,
summary="列出 CI/CD evidence events",
description=(
"從 alert_operation_log 讀取 CI/CD notification evidence供 AwoooP "
"Deployments / Run Console 顯示 rollout-risk、success、failed 等階段狀態。"
),
)
async def list_cicd_events(
project_id: str | None = Query(None, description="租戶 ID目前支援 awoooi"),
stage: str | None = Query(None, description="CI/CD stage filter可選"),
status: str | None = Query(None, description="CI/CD status filterrunning/success/failed/pending"),
limit: int = Query(12, ge=1, le=50, description="最多返回筆數"),
) -> dict[str, Any]:
return await list_cicd_events_svc(
project_id=project_id,
stage=stage,
status_filter=status,
limit=limit,
)
@router.get(
"/ai-route-status",
response_model=AiRouteStatusResponse,
summary="查詢 AI Provider 路由狀態",
description=(
"回傳目前 Ollama/Gemini 路由策略、即時 primary、fallback chain 與健康狀態;"
"只讀,不觸發推理或自動修復。"
),
)
async def get_ai_route_status(
workload_type: str | None = Query(
"deep_rca",
description="工作負載類型,例如 deep_rca/hermes/interactive/embedding/rag/code_review/image_analysis",
),
) -> dict[str, Any]:
return await get_ai_route_status_svc(workload_type=workload_type)
@router.get(
"/runs/{run_id}/detail",
summary="查詢 Run 詳細時間線",
@@ -126,6 +384,27 @@ async def get_run_detail(
return await get_run_detail_svc(run_id=run_id, project_id=project_id)
@router.get(
"/status-chain",
summary="查詢 AwoooP 狀態鏈",
description=(
"依 incident_id 查詢 truth-chain + ADR-100 history 合併後的只讀狀態鏈,"
"供 Work Items、Approvals、Monitoring 等操作頁面共用。"
),
)
async def get_awooop_status_chain(
project_id: str | None = Query(None, description="租戶 ID可選"),
incident_id: list[str] | None = Query(
None,
description="Incident ID可重複傳入以合併同一工作項的多個事件",
),
) -> dict[str, Any]:
return await get_awooop_status_chain_svc(
project_id=project_id,
incident_ids=incident_id or [],
)
@router.get(
"/approvals",
response_model=ListApprovalsResponse,
@@ -138,8 +417,16 @@ async def get_run_detail(
async def list_approvals(
project_id: str | None = Query(None, description="租戶 ID可選"),
run_id: str | None = Query(None, description="Run ID可選M8 詳情頁查單筆)"),
remediation_status: str | None = Query(
None,
description="AI 證據狀態 filterno_evidence/mcp_observed/read_only_dry_run/write_observed/blocked/observed",
),
) -> dict[str, Any]:
return await list_approvals_svc(project_id=project_id, run_id=run_id)
return await list_approvals_svc(
project_id=project_id,
run_id=run_id,
remediation_status=remediation_status,
)
@router.post(

View File

@@ -0,0 +1,66 @@
"""AwoooP Operator Console — truth-chain read API."""
from __future__ import annotations
from typing import Any
from fastapi import APIRouter, Depends, Query
from src.core.awooop_operator_auth import (
AwoooPOperatorPrincipal,
verify_awooop_operator,
)
from src.services.awooop_truth_chain_service import (
fetch_automation_quality_summary,
fetch_truth_chain,
)
router = APIRouter()
@router.get(
"/truth-chain/quality/summary",
summary="查詢 AI 自動化品質總覽",
description=(
"T12c read-only aggregate endpoint. 聚合最近 incident 的 automation quality gate"
"讓 Operator 不必逐張 Telegram 卡片判斷是否真正完成 AI 自動修復。"
"此總覽不回傳逐筆 examplessource-level truth-chain 詳情仍需 operator auth。"
),
)
async def get_automation_quality_summary(
project_id: str = Query("awoooi", description="租戶 ID"),
hours: int = Query(24, ge=1, le=168, description="回看小時數"),
limit: int = Query(200, ge=1, le=500, description="最多評估 incident 數"),
refresh: bool = Query(False, description="略過短 TTL 快取並重新聚合"),
) -> dict[str, Any]:
summary = await fetch_automation_quality_summary(
project_id=project_id,
hours=hours,
limit=limit,
refresh=refresh,
)
summary["examples"] = []
summary["visibility_note"] = (
"Aggregate only. Use /truth-chain/{source_id} with operator auth for source-level details."
)
return summary
@router.get(
"/truth-chain/{source_id}",
summary="查詢 Telegram / Incident / Drift 真相鏈",
description=(
"T0 read-only endpoint. 聚合 incident、approval、evidence、MCP、"
"automation_operation_log、drift repeat state 與 outbound mirror"
"讓 Operator Console 能判斷 Telegram 卡片目前卡在哪個流程節點。"
),
)
async def get_truth_chain(
source_id: str,
project_id: str = Query("awoooi", description="租戶 ID"),
operator: AwoooPOperatorPrincipal = Depends(verify_awooop_operator),
) -> dict[str, Any]:
# operator dependency intentionally gates this read API even though the
# principal is not otherwise needed by the aggregation query.
_ = operator
return await fetch_truth_chain(source_id=source_id, project_id=project_id)

View File

@@ -64,6 +64,7 @@ async def rag_debug() -> dict:
"""診斷用:確認容器內 docs 路徑 + Ollama 連線"""
import os
from pathlib import Path
import httpx
paths_check = {}
@@ -78,12 +79,23 @@ async def rag_debug() -> dict:
try:
async with httpx.AsyncClient(timeout=10.0) as c:
from src.core.config import get_settings as _gs
from src.services.ollama_endpoint_resolver import resolve_ollama_order
settings = _gs()
r = await c.post(
f"{settings.OLLAMA_URL}/api/embeddings",
json={"model": settings.OLLAMA_EMBEDDING_MODEL, "prompt": "test"},
)
ollama_ok = r.status_code == 200 if r.status_code == 200 else f"http_{r.status_code}"
statuses: list[str] = []
for endpoint in resolve_ollama_order("embedding"):
if not endpoint.url:
continue
r = await c.post(
f"{endpoint.url}/api/embeddings",
json={"model": settings.OLLAMA_EMBEDDING_MODEL, "prompt": "test"},
)
if r.status_code == 200:
ollama_ok = True
break
statuses.append(f"{endpoint.provider_name}=http_{r.status_code}")
if ollama_ok is not True:
ollama_ok = ", ".join(statuses) or "no_endpoint"
except Exception as e:
ollama_ok = f"error: {type(e).__name__}: {e}"

View File

@@ -14,12 +14,15 @@ AWOOOI API - Sentry Webhook Handler
🔴 HARD RULE: 時間顯示使用 Asia/Taipei (UTC+8)
"""
import json
import uuid
from typing import Any
import structlog
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request
from pydantic import BaseModel
from src.core.awooop_operator_auth import authenticate_awooop_operator_headers
from src.core.circuit_breaker import get_openclaw_guard
from src.core.metrics import (
record_alert_chain_failure,
@@ -35,8 +38,10 @@ from src.models.approval import (
)
from src.services.anomaly_counter import get_anomaly_counter
from src.services.approval_db import get_approval_service
from src.services.channel_hub import record_external_alert_event
from src.services.openclaw_http_service import get_openclaw_http_service
from src.services.sentry_service import get_sentry_service
# 2026-04-27 P3.1-T2 by Claude — Tier-2 三服務感知強化:補 SentryWebhookService 簽章驗證
from src.services.sentry_webhook_service import (
SentrySignatureError,
@@ -87,6 +92,114 @@ async def sentry_webhook_health() -> dict:
return {"status": "ok", "webhook": "sentry"}
def _sentry_event_tag(event_data: dict[str, Any], key: str) -> str | None:
tags = event_data.get("tags") or []
for tag in tags:
if isinstance(tag, list | tuple) and len(tag) >= 2 and str(tag[0]) == key:
return str(tag[1])
if isinstance(tag, dict) and str(tag.get("key")) == key:
value = tag.get("value")
return str(value) if value is not None else None
return None
def _is_sentry_upstream_canary(payload: dict[str, Any]) -> bool:
data = payload.get("data") if isinstance(payload, dict) else None
if not isinstance(data, dict) or payload.get("action") != "triggered":
return False
issue_data = data.get("issue") if isinstance(data.get("issue"), dict) else {}
event_data = data.get("event") if isinstance(data.get("event"), dict) else {}
issue_id = str(issue_data.get("id") or "")
short_id = str(issue_data.get("shortId") or "")
title = str(issue_data.get("title") or "")
return (
issue_id.startswith("awoooi-canary-")
or short_id.upper().startswith("AWOOOI-CANARY")
or title == "AwoooPSourceProviderCanary"
or (_sentry_event_tag(event_data, "awoooi_canary") or "").lower() == "true"
)
async def _record_sentry_upstream_canary(
payload: dict[str, Any],
request: Request,
) -> dict[str, Any]:
operator = authenticate_awooop_operator_headers(
request.headers.get("x-awooop-operator-id"),
request.headers.get("x-awooop-operator-key"),
)
data = payload.get("data") if isinstance(payload.get("data"), dict) else {}
issue_data = data.get("issue") if isinstance(data.get("issue"), dict) else {}
event_data = data.get("event") if isinstance(data.get("event"), dict) else {}
issue_id = str(
issue_data.get("id")
or issue_data.get("shortId")
or _sentry_event_tag(event_data, "run_ref")
or "awoooi-canary-unknown"
)
source_url = (
issue_data.get("permalink")
or issue_data.get("web_url")
or issue_data.get("url")
)
event_uuid = await record_external_alert_event(
project_id="awoooi",
provider="sentry",
event_id=issue_id,
stage="upstream_canary",
title=str(issue_data.get("title") or "AwoooPSourceProviderCanary"),
severity=str(issue_data.get("level") or "info"),
namespace="awoooi-prod",
target_resource=str(issue_data.get("culprit") or "source-provider-ingestion"),
fingerprint=f"source-provider-canary:sentry:{issue_id}",
source_url=source_url,
labels={
"project": issue_data.get("project", {}),
"level": issue_data.get("level", "info"),
"awoooi_canary": "true",
"operator_id": operator.operator_id,
"telegram": "not_sent",
"incident": "not_created",
"approval": "not_created",
},
annotations={
"message": event_data.get("message"),
"summary": (
"Operator-signed Sentry webhook canary; records upstream "
"source evidence without creating incident, approval, or Telegram."
),
},
payload={
"raw_canary": payload,
"operator_id": operator.operator_id,
"auth_method": operator.auth_method,
"side_effects": {
"incident_created": False,
"approval_created": False,
"telegram_sent": False,
"openclaw_called": False,
},
},
)
if event_uuid is None:
raise HTTPException(
status_code=500,
detail="sentry upstream canary was not recorded",
)
return {
"status": "canary_recorded",
"provider": "sentry",
"event_id": issue_id,
"conversation_event_id": str(event_uuid),
"side_effects": {
"incident_created": False,
"approval_created": False,
"telegram_sent": False,
"openclaw_called": False,
},
}
@router.post("/error")
async def handle_sentry_error(
request: Request,
@@ -108,6 +221,14 @@ async def handle_sentry_error(
try:
# 2026-04-27 P3.1-T2 by Claude — Tier-2 三服務感知強化:接入 SentryWebhookService 簽章驗證
body = await request.body()
try:
payload_from_body = json.loads(body.decode("utf-8") or "{}")
except json.JSONDecodeError:
payload_from_body = {}
if isinstance(payload_from_body, dict) and _is_sentry_upstream_canary(payload_from_body):
return await _record_sentry_upstream_canary(payload_from_body, request)
sig_header = request.headers.get("sentry-hook-signature", "")
try:
verify_sentry_signature(body, sig_header)
@@ -124,16 +245,60 @@ async def handle_sentry_error(
# 提取錯誤資訊
issue_data = payload.get("data", {}).get("issue", {})
event_data = payload.get("data", {}).get("event", {})
issue_id = issue_data.get("id")
source_url = (
issue_data.get("permalink")
or issue_data.get("web_url")
or issue_data.get("url")
)
background_tasks.add_task(
record_external_alert_event,
project_id="awoooi",
provider="sentry",
event_id=str(issue_id or issue_data.get("shortId") or "unknown"),
stage="received",
title=str(issue_data.get("title") or "Sentry issue"),
severity=str(issue_data.get("level") or "error"),
namespace="sentry",
target_resource=str(issue_data.get("culprit") or issue_data.get("project", {}).get("slug") or "unknown"),
fingerprint=f"sentry-{issue_id or issue_data.get('shortId') or 'unknown'}",
source_url=source_url,
labels={
"project": issue_data.get("project", {}),
"level": issue_data.get("level"),
"culprit": issue_data.get("culprit"),
},
annotations={"message": event_data.get("message")},
payload=payload,
)
# Phase 10.2.1: 去重檢查 (10 分鐘內不重複發送)
issue_id = issue_data.get("id")
sentry_service = get_sentry_service()
if not await sentry_service.check_dedup(issue_id, ttl=SENTRY_DEDUP_TTL):
background_tasks.add_task(
record_external_alert_event,
project_id="awoooi",
provider="sentry",
event_id=str(issue_id or issue_data.get("shortId") or "unknown"),
stage="deduplicated",
title=str(issue_data.get("title") or "Sentry issue"),
severity=str(issue_data.get("level") or "error"),
namespace="sentry",
target_resource=str(issue_data.get("culprit") or issue_data.get("project", {}).get("slug") or "unknown"),
fingerprint=f"sentry-{issue_id or issue_data.get('shortId') or 'unknown'}",
source_url=source_url,
labels={"project": issue_data.get("project", {}), "level": issue_data.get("level")},
annotations={"message": event_data.get("message")},
payload={"dedup_ttl": SENTRY_DEDUP_TTL},
is_duplicate=True,
)
return {"status": "deduplicated", "issue_id": issue_id, "ttl": SENTRY_DEDUP_TTL}
event_data = payload.get("data", {}).get("event", {})
error_context = {
"issue_id": issue_data.get("id"),
"source_url": source_url,
"title": issue_data.get("title"),
"culprit": issue_data.get("culprit"),
"level": issue_data.get("level"),
@@ -169,6 +334,8 @@ async def handle_sentry_error(
"message": "Analysis scheduled"
}
except HTTPException:
raise
except Exception as e:
logger.exception("Sentry webhook processing failed")
raise HTTPException(status_code=500, detail=str(e)) from e
@@ -256,6 +423,29 @@ async def analyze_and_comment(
analysis=analysis,
anomaly_frequency=frequency_dict,
)
await record_external_alert_event(
project_id="awoooi",
provider="sentry",
event_id=str(issue_id or error_context.get("issue_id") or "unknown"),
stage="approval_linked",
title=str(error_context.get("title") or "Sentry issue"),
severity=str(error_context.get("level") or "error"),
namespace="sentry",
target_resource=str(error_context.get("culprit") or error_context.get("project") or "unknown"),
fingerprint=f"sentry-{issue_id or error_context.get('issue_id') or 'unknown'}",
approval_id=approval_id,
source_url=error_context.get("source_url"),
labels={
"project": error_context.get("project"),
"level": error_context.get("level"),
},
annotations={"message": error_context.get("message")},
payload={
"anomaly_frequency": frequency_dict,
"ai_analyzed": analysis is not None,
"ai_provider": analysis.analyzed_by if analysis else None,
},
)
# 4. 發送 Telegram 告警 (含頻率資訊)
await send_sentry_telegram_alert(

View File

@@ -1,7 +1,3 @@
from __future__ import annotations
import asyncio
"""
AWOOOI API - SignOz Webhook Handler
====================================
@@ -17,12 +13,17 @@ AWOOOI API - SignOz Webhook Handler
🔴 HARD RULE: 時間顯示使用 Asia/Taipei (UTC+8)
"""
from __future__ import annotations
import asyncio
import uuid
from typing import TYPE_CHECKING
import structlog
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request
from pydantic import BaseModel
from src.core.awooop_operator_auth import authenticate_awooop_operator_headers
from src.core.metrics import (
record_alert_chain_failure,
record_alert_chain_success,
@@ -37,10 +38,14 @@ from src.models.approval import (
)
from src.services.anomaly_counter import get_anomaly_counter
from src.services.approval_db import get_approval_service
from src.services.channel_hub import record_external_alert_event
from src.services.incident_service import get_incident_service
from src.services.telegram_gateway import get_telegram_gateway
from src.utils.timezone import now_taipei_iso
if TYPE_CHECKING:
from src.services.openclaw import LLMAnalysisResult
logger = structlog.get_logger(__name__)
router = APIRouter(prefix="/webhooks/signoz", tags=["SignOz Webhook"])
@@ -67,6 +72,101 @@ class SignOzAlertPayload(BaseModel):
generatorURL: str | None = None
def _is_signoz_upstream_canary(alert: dict) -> bool:
labels = alert.get("labels", {}) if isinstance(alert.get("labels"), dict) else {}
annotations = (
alert.get("annotations", {})
if isinstance(alert.get("annotations"), dict)
else {}
)
alert_name = str(alert.get("alertname") or labels.get("alertname") or "")
return (
str(labels.get("awoooi_canary", "")).lower() == "true"
or alert_name == "AwoooPSourceProviderCanary"
or str(annotations.get("awooop_canary", "")).lower() == "true"
)
async def _record_signoz_upstream_canary(
alert: dict,
request: Request,
) -> dict:
operator = authenticate_awooop_operator_headers(
request.headers.get("x-awooop-operator-id"),
request.headers.get("x-awooop-operator-key"),
)
labels = alert.get("labels", {}) if isinstance(alert.get("labels"), dict) else {}
annotations = (
alert.get("annotations", {})
if isinstance(alert.get("annotations"), dict)
else {}
)
alert_name = str(alert.get("alertname") or labels.get("alertname") or "AwoooPSourceProviderCanary")
run_ref = str(labels.get("run_ref") or labels.get("fingerprint") or "unknown")
event_id = f"awooop-canary-{run_ref}"
severity = str(labels.get("severity") or "info")
service_name = str(labels.get("service_name") or labels.get("service") or "source-provider-ingestion")
namespace = str(labels.get("namespace") or "awoooi-prod")
fingerprint = str(labels.get("fingerprint") or f"source-provider-canary:signoz:{run_ref}")
event_uuid = await record_external_alert_event(
project_id="awoooi",
provider="signoz",
event_id=event_id,
stage="upstream_canary",
title=alert_name,
severity=severity,
namespace=namespace,
target_resource=service_name,
fingerprint=fingerprint,
source_url=alert.get("generatorURL"),
labels={
**labels,
"awoooi_canary": "true",
"operator_id": operator.operator_id,
"telegram": "not_sent",
"incident": "not_created",
"approval": "not_created",
},
annotations={
**annotations,
"summary": annotations.get("summary")
or (
"Operator-signed SignOz webhook canary; records upstream "
"source evidence without creating incident, approval, or Telegram."
),
},
payload={
"raw_canary": alert,
"operator_id": operator.operator_id,
"auth_method": operator.auth_method,
"side_effects": {
"incident_created": False,
"approval_created": False,
"telegram_sent": False,
"openclaw_called": False,
},
},
)
if event_uuid is None:
raise HTTPException(
status_code=500,
detail="signoz upstream canary was not recorded",
)
return {
"status": "canary_recorded",
"provider": "signoz",
"event_id": event_id,
"alert_name": alert_name,
"conversation_event_id": str(event_uuid),
"side_effects": {
"incident_created": False,
"approval_created": False,
"telegram_sent": False,
"openclaw_called": False,
},
}
@router.post("/alert")
async def handle_signoz_alert(
request: Request,
@@ -99,11 +199,35 @@ async def handle_signoz_alert(
results.append({"status": "ignored", "reason": "not firing"})
continue
if _is_signoz_upstream_canary(alert):
results.append(await _record_signoz_upstream_canary(alert, request))
continue
# 提取告警資訊
alert_name = alert.get("alertname", alert.get("labels", {}).get("alertname", "unknown"))
labels = alert.get("labels", {})
annotations = alert.get("annotations", {})
severity = labels.get("severity", "warning")
source_url = alert.get("generatorURL")
service_name = labels.get("service_name", labels.get("service", "unknown"))
fingerprint = labels.get("fingerprint") or f"signoz-{alert_name}-{service_name}"
background_tasks.add_task(
record_external_alert_event,
project_id="awoooi",
provider="signoz",
event_id=str(fingerprint),
stage="received",
title=str(alert_name),
severity=str(severity),
namespace=str(labels.get("namespace", "signoz")),
target_resource=str(service_name),
fingerprint=str(fingerprint),
source_url=source_url,
labels=labels,
annotations=annotations,
payload=alert,
)
# 背景處理
background_tasks.add_task(
@@ -113,6 +237,8 @@ async def handle_signoz_alert(
annotations=annotations,
severity=severity,
starts_at=alert.get("startsAt"),
source_url=source_url,
raw_payload=alert,
)
results.append({
@@ -122,6 +248,8 @@ async def handle_signoz_alert(
return {"status": "ok", "processed": len(results), "results": results}
except HTTPException:
raise
except Exception as e:
logger.exception("signoz_webhook_error", error=str(e))
raise HTTPException(status_code=500, detail=str(e)) from e
@@ -133,6 +261,8 @@ async def process_signoz_alert(
annotations: dict,
severity: str,
starts_at: str | None,
source_url: str | None = None,
raw_payload: dict | None = None,
):
"""
背景處理 SignOz 告警
@@ -190,6 +320,7 @@ async def process_signoz_alert(
"annotations": annotations,
"fingerprint": f"signoz-{alert_name}-{labels.get('service_name', 'unknown')}",
}
fingerprint = signal_data["fingerprint"]
# ADR-037: 傳遞頻率統計到 Incident
incident = await incident_service.create_incident_from_signal(
signal_data, frequency_stats=anomaly_frequency
@@ -229,6 +360,30 @@ async def process_signoz_alert(
anomaly_frequency=anomaly_frequency,
analysis_result=analysis_result, # 帶入 AI 結果
)
await record_external_alert_event(
project_id="awoooi",
provider="signoz",
event_id=str(fingerprint),
stage="incident_linked",
title=str(alert_name),
severity=str(severity),
namespace=str(labels.get("namespace", "signoz")),
target_resource=str(labels.get("service_name", labels.get("service", "unknown"))),
fingerprint=str(fingerprint),
incident_id=str(incident.incident_id),
approval_id=str(approval_id),
source_url=source_url or trace_url,
labels=labels,
annotations=annotations,
payload={
"raw_alert": raw_payload or {},
"trace_url": trace_url,
"has_signoz_metrics": bool(signoz_metrics),
"ai_provider": ai_provider,
"tokens": tokens,
"cost": cost,
},
)
# =================================================================
# Step 5: 發送 Telegram 告警
@@ -282,7 +437,7 @@ async def create_signoz_approval(
severity: str,
incident_id: str,
anomaly_frequency: dict | None = None,
analysis_result: "LLMAnalysisResult" | None = None,
analysis_result: LLMAnalysisResult | None = None,
) -> str:
"""
為 SignOz 告警建立 Approval 記錄
@@ -379,7 +534,7 @@ async def send_signoz_telegram(
annotations: dict,
severity: str,
anomaly_frequency: dict | None = None,
analysis_result: "LLMAnalysisResult" | None = None,
analysis_result: LLMAnalysisResult | None = None,
ai_provider: str = "none",
):
"""
@@ -442,6 +597,7 @@ async def _send_log_summary_notification(
帶 5s 軟超時:超時後摘要繼續生成並存 Redis不阻塞告警主流程
"""
import html as _html
from src.services.log_summary_service import get_log_summary_service
from src.services.telegram_gateway import get_telegram_gateway

View File

@@ -19,6 +19,7 @@ Endpoints:
- 每個 Nonce 只能使用一次
"""
import asyncio
from uuid import UUID
from fastapi import APIRouter, HTTPException, status
@@ -27,6 +28,8 @@ from pydantic import BaseModel
from src.core.config import settings
from src.core.logging import get_logger
from src.services.approval_db import get_approval_service
from src.services.approval_execution import get_execution_service
from src.services.incident_approval_service import get_incident_approval_service
from src.services.security_interceptor import (
NonceReplayError,
UserNotWhitelistedError,
@@ -64,6 +67,80 @@ class TestPushRequest(BaseModel):
incident_id: str = ""
async def _run_telegram_approved_execution(approval) -> None:
"""Run the approved action that originated from a Telegram callback."""
approval_id = str(getattr(approval, "id", ""))
incident_id = getattr(approval, "incident_id", None)
try:
result = await get_execution_service().execute_approved_action(approval)
logger.info(
"telegram_approval_execution_completed",
approval_id=approval_id,
incident_id=incident_id,
success=bool(result),
)
except Exception as exc:
logger.error(
"telegram_approval_execution_failed",
approval_id=approval_id,
incident_id=incident_id,
error=str(exc),
)
def _schedule_telegram_approved_execution(approval) -> bool:
"""Schedule execution after Telegram approval reaches required signatures."""
try:
asyncio.create_task(_run_telegram_approved_execution(approval))
logger.info(
"telegram_approval_execution_scheduled",
approval_id=str(getattr(approval, "id", "")),
incident_id=getattr(approval, "incident_id", None),
)
return True
except Exception as exc:
logger.error(
"telegram_approval_execution_schedule_failed",
approval_id=str(getattr(approval, "id", "")),
incident_id=getattr(approval, "incident_id", None),
error=str(exc),
)
return False
async def _finalize_telegram_approval(approval, execution_triggered: bool) -> bool:
"""Complete the execution handoff for Telegram approvals.
ApprovalDBService only records the signature/status transition. The actual
executor scheduling lives in API callers, so Telegram must mirror the REST
approval endpoint instead of stopping at a visual approval stamp.
"""
if not execution_triggered:
return False
return _schedule_telegram_approved_execution(approval)
async def _sync_telegram_rejection(approval_id: str) -> bool:
"""Keep Incident state aligned when an approval is rejected from Telegram."""
try:
await get_incident_approval_service().on_approval_status_change(
approval_id=approval_id,
new_status="rejected",
)
logger.info(
"telegram_rejection_incident_synced",
approval_id=approval_id,
)
return True
except Exception as exc:
logger.error(
"telegram_rejection_incident_sync_failed",
approval_id=approval_id,
error=str(exc),
)
return False
# =============================================================================
# Endpoints
# =============================================================================
@@ -139,6 +216,17 @@ async def telegram_webhook(
# =========================================================================
try:
gateway = get_telegram_gateway()
mirror_callback = getattr(gateway, "mirror_callback_query_received", None)
if callable(mirror_callback):
await mirror_callback(
update_id=update.update_id,
callback_query_id=callback_query_id,
callback_data=callback_data,
user_id=user_id,
username=username,
message_id=message_id,
chat_id=message.get("chat", {}).get("id"),
)
result = await gateway.handle_callback(
callback_query_id=callback_query_id,
callback_data=callback_data,
@@ -198,21 +286,50 @@ async def telegram_webhook(
)
if approval:
status_value = approval.status.value if hasattr(approval.status, "value") else str(approval.status)
if (
"Cannot sign" in msg
or "already signed" in msg
or "Concurrent modification" in msg
):
logger.info(
"telegram_approval_ignored_already_processed",
approval_id=approval_id,
user_id=user_id,
status=status_value,
message=msg,
)
await _log_user_action("approve_duplicate", False, getattr(approval, "incident_id", None))
return {
"ok": True,
"message": "Already processed",
"approval_id": approval_id,
"status": status_value,
"execution_triggered": False,
"execution_scheduled": False,
}
execution_scheduled = await _finalize_telegram_approval(
approval=approval,
execution_triggered=execution_triggered,
)
logger.info(
"telegram_approval_signed",
approval_id=approval_id,
user_id=user_id,
status=approval.status.value,
status=status_value,
execution_triggered=execution_triggered,
execution_scheduled=execution_scheduled,
)
await _log_user_action("approve", True, getattr(approval, "incident_id", None))
return {
"ok": True,
"message": "Approved",
"message": "Approved" if execution_triggered else "Signed",
"approval_id": approval_id,
"status": approval.status.value,
"status": status_value,
"execution_triggered": execution_triggered,
"execution_scheduled": execution_scheduled,
}
elif action == "reject":
@@ -224,10 +341,12 @@ async def telegram_webhook(
)
if approval:
incident_synced = await _sync_telegram_rejection(approval_id)
logger.info(
"telegram_approval_rejected",
approval_id=approval_id,
user_id=user_id,
incident_synced=incident_synced,
)
await _log_user_action("reject", False, getattr(approval, "incident_id", None))
@@ -236,6 +355,7 @@ async def telegram_webhook(
"message": "Rejected",
"approval_id": approval_id,
"status": approval.status.value,
"incident_synced": incident_synced,
}
return {"ok": False, "message": "Unknown action"}

View File

@@ -71,6 +71,29 @@ async def telegram_webhook(request: Request) -> dict:
update_id=body.get("update_id"),
)
if update_type == "callback_query":
callback = body.get("callback_query", {}) or {}
message = callback.get("message", {}) or {}
user = callback.get("from", {}) or {}
callback_query_id = callback.get("id")
callback_data = callback.get("data")
user_id = user.get("id")
if callback_query_id and callback_data and user_id:
from src.services.telegram_gateway import get_telegram_gateway
gateway = get_telegram_gateway()
mirror_callback = getattr(gateway, "mirror_callback_query_received", None)
if callable(mirror_callback):
await mirror_callback(
update_id=body.get("update_id"),
callback_query_id=callback_query_id,
callback_data=callback_data,
user_id=user_id,
username=user.get("username") or user.get("first_name") or str(user_id),
message_id=message.get("message_id"),
chat_id=(message.get("chat") or {}).get("id"),
)
# WS5: chat_member 同步 Approvers 白名單ADR-093
if update_type in ("chat_member", "my_chat_member") or (
"chat_member" in body or "my_chat_member" in body

View File

@@ -55,7 +55,10 @@ from src.services.alertmanager_llm_guard import (
from src.services.approval_db import get_approval_service
from src.services.auto_approve import get_auto_approve_policy
from src.services.auto_repair_service import AutoRepairService
from src.services.channel_hub import record_grouped_alert_event
from src.services.channel_hub import (
record_alertmanager_event,
record_grouped_alert_event,
)
# Phase 15.2: Trace Context (moved to SignalProducerService)
# get_trace_context 已移至 Service 層
@@ -136,6 +139,38 @@ def _should_use_alertmanager_rule_first(
)
async def _analyze_alertmanager_with_timeout(
openclaw,
alert_context: dict,
*,
alert_id: str,
alertname: str,
) -> tuple:
"""Run Alertmanager AI analysis without letting it block the workflow forever."""
try:
return await asyncio.wait_for(
openclaw.analyze_alert(alert_context),
timeout=ALERTMANAGER_BACKGROUND_AI_TIMEOUT_SECONDS,
)
except TimeoutError:
logger.warning(
"alertmanager_openclaw_timeout_fallback",
alert_id=alert_id,
alertname=alertname,
timeout_sec=ALERTMANAGER_BACKGROUND_AI_TIMEOUT_SECONDS,
)
return None, "fallback_timeout", "", None, "", 0, 0.0
except Exception as exc:
logger.warning(
"alertmanager_openclaw_failed_fallback",
alert_id=alert_id,
alertname=alertname,
error=str(exc),
)
return None, "fallback_error", "", None, "", 0, 0.0
async def _escalate_auto_repair_unavailable(
*,
incident_id: str,
@@ -163,6 +198,19 @@ async def _escalate_auto_repair_unavailable(
)
def _auto_repair_action_label(result, fallback_target: str) -> str:
"""Build a verifier label that includes the actual playbook steps."""
playbook_id = getattr(result, "playbook_id", None) or "unknown"
steps = getattr(result, "executed_steps", None) or []
step_text = " | ".join(str(step) for step in steps).strip()
if not step_text:
step_text = fallback_target
step_text = " ".join(step_text.split())
if len(step_text) > 240:
step_text = f"{step_text[:237]}..."
return f"auto_repair_playbook:{playbook_id} {step_text}".strip()
async def _try_auto_repair_background(
incident_id: str,
approval_id: str,
@@ -252,6 +300,46 @@ async def _try_auto_repair_background(
},
)
_pre_execution_snapshot = None
try:
from src.core.feature_flags import aiops_flags
if aiops_flags.is_sub_flag_enabled("AIOPS_P1_PRE_DECISION_INVESTIGATOR"):
from src.services.evidence_snapshot import get_latest_snapshot
from src.services.post_execution_verifier import get_post_execution_verifier
_pre_execution_snapshot = await get_latest_snapshot(incident_id)
if _pre_execution_snapshot is None:
from src.services.pre_decision_investigator import (
get_pre_decision_investigator,
)
_pre_execution_snapshot = await asyncio.wait_for(
get_pre_decision_investigator().investigate(incident),
timeout=60.0,
)
if _pre_execution_snapshot is not None:
await asyncio.wait_for(
get_post_execution_verifier().capture_pre_execution_state(
incident,
_pre_execution_snapshot,
),
timeout=30.0,
)
except asyncio.TimeoutError:
logger.warning(
"auto_repair_pre_state_capture_timeout",
incident_id=incident_id,
approval_id=approval_id,
)
except Exception as _pre_state_err:
logger.warning(
"auto_repair_pre_state_capture_failed",
incident_id=incident_id,
approval_id=approval_id,
error=str(_pre_state_err),
)
# 執行自動修復
logger.info(
"auto_repair_executing",
@@ -263,6 +351,7 @@ async def _try_auto_repair_background(
playbook=decision.playbook,
is_cold_start=decision.is_cold_start,
similarity_score=decision.similarity_score,
run_post_verification=False,
)
logger.info(
@@ -273,6 +362,20 @@ async def _try_auto_repair_background(
# 記錄執行結果
if result:
try:
await get_approval_service().update_execution_status(
approval_id=approval_id,
success=result.success,
error_message=result.error,
)
except Exception as _approval_status_err:
logger.warning(
"auto_repair_approval_status_update_failed",
approval_id=approval_id,
incident_id=incident_id,
error=str(_approval_status_err),
)
await op_log.append(
"EXECUTION_COMPLETED",
incident_id=incident_id,
@@ -336,11 +439,10 @@ async def _try_auto_repair_background(
from src.services.evidence_snapshot import get_latest_snapshot
from src.services.learning_service import get_learning_service
_snapshot = await get_latest_snapshot(incident_id)
_action_label = (
f"{target_resource}:{namespace}"
if not result.success
else f"auto_repair_playbook:{result.playbook_id}"
_snapshot = _pre_execution_snapshot or await get_latest_snapshot(incident_id)
_action_label = _auto_repair_action_label(
result,
fallback_target=f"{target_resource}:{namespace}",
)
_verifier = get_post_execution_verifier()
_verify_result = await asyncio.wait_for(
@@ -792,6 +894,7 @@ async def verify_webhook_signature(
# 戰略 B: 滑動時間窗 (ADR-073: 5 分鐘改 30 分鐘,防同一問題反覆重建 Incident2026-04-12 ogt)
DEBOUNCE_WINDOW_MINUTES = 30
ALERTMANAGER_BACKGROUND_AI_TIMEOUT_SECONDS = 90.0
# =============================================================================
@@ -1105,7 +1208,12 @@ async def receive_alert(
# 呼叫 OpenClaw LLM 分析 (v7.0 含 SignOz 整合)
# 2026-03-29 ogt: 加入 Token/Cost 追蹤
openclaw = get_openclaw()
analysis_result, ai_provider, raw_response, signoz_metrics, signoz_trace_url, ai_tokens, ai_cost = await openclaw.analyze_alert(alert_context)
analysis_result, ai_provider, raw_response, signoz_metrics, signoz_trace_url, ai_tokens, ai_cost = await _analyze_alertmanager_with_timeout(
openclaw,
alert_context,
alert_id=alert_id,
alertname=alert.alert_type,
)
if analysis_result:
# LLM 分析成功
@@ -1439,6 +1547,39 @@ class AlertmanagerPayload(BaseModel):
alerts: list[AlertmanagerAlert]
_CICD_JOB_STATUSES = frozenset({"running", "success", "failed", "pending"})
def _cicd_job_status_from_alert(alert: AlertmanagerAlert) -> str:
"""將 CI/CD Alertmanager label 轉成 TelegramGateway 支援的狀態。
2026-05-12 Codex: Gitea workflow 先送進 AWOOI API不能只靠
severity=info 推 success否則 failed/pending 事件進 AwoooP 後語義會失真。
"""
labels = alert.labels or {}
for key in ("status", "job_status", "ci_status"):
value = str(labels.get(key) or "").strip().lower()
if value in _CICD_JOB_STATUSES:
return value
severity = str(labels.get("severity") or "").strip().lower()
if severity == "info":
return "success"
if severity in {"critical", "error"}:
return "failed"
return "running"
def _cicd_duration_seconds_from_alert(alert: AlertmanagerAlert) -> int:
labels = alert.labels or {}
raw = labels.get("duration_seconds") or labels.get("duration") or 0
try:
value = int(str(raw).strip())
except (TypeError, ValueError):
return 0
return max(value, 0)
def is_internal_ip(client_ip: str) -> bool:
"""檢查是否為內網 IP"""
import ipaddress
@@ -1475,6 +1616,11 @@ async def _process_new_alert_background(
try:
service = get_approval_service()
openclaw = get_openclaw()
traced_alert_labels = {
**(alert_labels or {}),
"fingerprint": fingerprint,
"alert_id": alert_id,
}
rule_response = match_rule(alert_context)
should_bypass_llm = _should_use_alertmanager_rule_first(rule_response, alert_category)
@@ -1602,6 +1748,10 @@ async def _process_new_alert_background(
# 2026-04-27 ogt + Claude Sonnet 4.6: CS2 規則引擎自動執行
# 設計is_rule_based=True 確定性高,滿足條件直接執行,不等人工審核
# 安全防線CRITICAL / destructive patterns / NO_ACTION / 空 kubectl → 全部降級 PENDING
_cs2_auto_approval = None
_cs2_executor = None
_cs2_exec_success: bool | None = None
_cs2_exec_error: str | None = None
try:
from src.models.approval import ApprovalRequest, ApprovalStatus
from src.services.approval_execution import ApprovalExecutionService
@@ -1625,6 +1775,7 @@ async def _process_new_alert_background(
)
# 使用 DB 中剛建立的 approval.id 讓 executor 可回寫
_auto_approval.id = approval.id
_cs2_auto_approval = _auto_approval
_cs2_executor = ApprovalExecutionService()
_cs2_exec_success = await _cs2_executor.execute_approved_action(_auto_approval)
@@ -1647,6 +1798,8 @@ async def _process_new_alert_background(
exec_success=_cs2_exec_success,
)
except Exception as _auto_err:
_cs2_exec_success = False if _cs2_auto_approval is not None else None
_cs2_exec_error = str(_auto_err)
logger.warning(
"cs2_auto_execute_failed_degraded_to_pending",
approval_id=str(approval.id),
@@ -1662,7 +1815,7 @@ async def _process_new_alert_background(
message=message,
source="alertmanager",
alertname=alertname,
alert_labels=alert_labels,
alert_labels=traced_alert_labels,
notification_type=notification_type,
alert_category=alert_category,
)
@@ -1678,6 +1831,41 @@ async def _process_new_alert_background(
error=str(_meta_err),
)
await record_alertmanager_event(
project_id="awoooi",
alert_id=alert_id,
alertname=alertname,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
stage="incident_linked",
notification_type=notification_type,
alert_category=alert_category,
incident_id=incident_id,
approval_id=str(approval.id),
repeat_count=1,
labels=traced_alert_labels,
annotations=alert_context.get("annotations", {}),
)
if _cs2_auto_approval is not None and _cs2_exec_success is not None:
try:
_cs2_auto_approval.incident_id = incident_id
_cs2_executor = _cs2_executor or ApprovalExecutionService()
await _cs2_executor.finalize_auto_approved_execution(
_cs2_auto_approval,
success=_cs2_exec_success,
error_message=_cs2_exec_error,
)
except Exception as _cs2_finalize_err:
logger.warning(
"cs2_auto_execute_finalize_failed",
approval_id=str(approval.id),
incident_id=incident_id,
error=str(_cs2_finalize_err),
)
_is_heartbeat = is_heartbeat_alertname(alertname)
if can_auto_repair and not _is_heartbeat:
await _try_auto_repair_background(
@@ -1731,7 +1919,12 @@ async def _process_new_alert_background(
record_alert_chain_success("alertmanager")
return
analysis_result, ai_provider, raw_response, signoz_metrics, signoz_trace_url, ai_tokens, ai_cost = await openclaw.analyze_alert(alert_context)
analysis_result, ai_provider, raw_response, signoz_metrics, signoz_trace_url, ai_tokens, ai_cost = await _analyze_alertmanager_with_timeout(
openclaw,
alert_context,
alert_id=alert_id,
alertname=alertname,
)
if analysis_result:
risk_mapping = {
@@ -1841,8 +2034,15 @@ async def _process_new_alert_background(
and "NO_ACTION" not in (analysis_result.action_title or "")
and is_safe_kubectl_action(_cs3_kubectl)
)
_cs3_auto_approval = None
_cs3_executor = None
_cs3_exec_success: bool | None = None
_cs3_exec_error: str | None = None
if _cs3_can_auto:
try:
from src.models.approval import ApprovalRequest, ApprovalStatus
from src.services.approval_execution import ApprovalExecutionService
_cs3_auto_approval = ApprovalRequest(
action=approval_create.action,
description=approval_create.description,
@@ -1859,8 +2059,17 @@ async def _process_new_alert_background(
else "cs3_auto_confident_execution",
},
)
_cs3_auto_approval.id = approval.id
_cs3_executor = ApprovalExecutionService()
_cs3_exec_success = await _cs3_executor.execute_approved_action(_cs3_auto_approval)
try:
await service.update_execution_status(approval.id, _cs3_exec_success)
except Exception as _cs3_upd_err:
logger.warning(
"cs3_auto_execute_status_update_failed",
approval_id=str(approval.id),
error=str(_cs3_upd_err),
)
logger.info(
"cs3_llm_auto_executed",
approval_id=str(approval.id),
@@ -1876,6 +2085,8 @@ async def _process_new_alert_background(
),
)
except Exception as _cs3_exec_err:
_cs3_exec_success = False if _cs3_auto_approval is not None else None
_cs3_exec_error = str(_cs3_exec_err)
logger.warning("cs3_llm_auto_execute_failed", error=str(_cs3_exec_err))
incident_id = await create_incident_for_approval(
@@ -1887,7 +2098,7 @@ async def _process_new_alert_background(
message=message,
source="alertmanager",
alertname=alertname,
alert_labels=alert_labels,
alert_labels=traced_alert_labels,
notification_type=notification_type,
alert_category=alert_category,
)
@@ -1903,6 +2114,41 @@ async def _process_new_alert_background(
error=str(_meta_err),
)
await record_alertmanager_event(
project_id="awoooi",
alert_id=alert_id,
alertname=alertname,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
stage="incident_linked",
notification_type=notification_type,
alert_category=alert_category,
incident_id=incident_id,
approval_id=str(approval.id),
repeat_count=1,
labels=traced_alert_labels,
annotations=alert_context.get("annotations", {}),
)
if _cs3_auto_approval is not None and _cs3_exec_success is not None:
try:
_cs3_auto_approval.incident_id = incident_id
_cs3_executor = _cs3_executor or ApprovalExecutionService()
await _cs3_executor.finalize_auto_approved_execution(
_cs3_auto_approval,
success=_cs3_exec_success,
error_message=_cs3_exec_error,
)
except Exception as _cs3_finalize_err:
logger.warning(
"cs3_auto_execute_finalize_failed",
approval_id=str(approval.id),
incident_id=incident_id,
error=str(_cs3_finalize_err),
)
root_cause = analysis_result.description or message
estimated_downtime = blast.estimated_downtime if blast else "~30s"
primary_responsibility = analysis_result.primary_responsibility or "COLLAB"
@@ -1978,11 +2224,17 @@ async def _process_new_alert_background(
else:
# LLM 失敗 - 使用預設值
# 2026-04-27 Claude Sonnet 4.6: shadow-run Step1 — 補 metadata kwarg讓 extra_metadata 可觀測
_matched_playbook_id_cs4 = await resolve_playbook_id_for_alert(
rule_id=str(rule_response.get("rule_id", "")),
alertname=alertname,
affected_services=[target_resource] if target_resource else [],
severity="medium",
)
_approval_metadata_cs4 = {
"source": "fallback",
"confidence_score": None,
"is_rule_based": False,
"playbook_id": None,
"playbook_id": _matched_playbook_id_cs4,
}
fallback_create = ApprovalRequestCreate(
action="OBSERVE",
@@ -1997,6 +2249,7 @@ async def _process_new_alert_background(
dry_run_checks=[],
requested_by="OpenClaw (fallback)",
metadata=_approval_metadata_cs4,
matched_playbook_id=_matched_playbook_id_cs4,
)
approval = await service.create_approval_with_fingerprint(
@@ -2034,7 +2287,7 @@ async def _process_new_alert_background(
message=message,
source="alertmanager",
alertname=alertname,
alert_labels=alert_labels,
alert_labels=traced_alert_labels,
notification_type=notification_type,
alert_category=alert_category,
)
@@ -2050,6 +2303,55 @@ async def _process_new_alert_background(
error=str(_meta_err),
)
await record_alertmanager_event(
project_id="awoooi",
alert_id=alert_id,
alertname=alertname,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
stage="incident_linked",
notification_type=notification_type,
alert_category=alert_category,
incident_id=fallback_incident_id,
approval_id=str(approval.id),
repeat_count=1,
labels=traced_alert_labels,
annotations=alert_context.get("annotations", {}),
)
_is_heartbeat = is_heartbeat_alertname(alertname)
if can_auto_repair and not _is_heartbeat:
await _try_auto_repair_background(
incident_id=fallback_incident_id,
approval_id=str(approval.id),
alert_type=alert_type,
target_resource=target_resource,
namespace=namespace,
)
elif not can_auto_repair and not _is_heartbeat:
from src.repositories.alert_operation_log_repository import get_alert_operation_log_repository
_op_log_fallback = get_alert_operation_log_repository()
await _op_log_fallback.append(
"GUARDRAIL_BLOCKED",
incident_id=fallback_incident_id,
approval_id=str(approval.id),
actor="prometheus-rule",
action_detail=f"Prometheus rule 設定 auto_repair=falsefallback 轉人工: {alertname}",
success=False,
context={"alertname": alertname, "auto_repair_flag": False},
)
await _escalate_auto_repair_unavailable(
incident_id=fallback_incident_id,
approval_id=str(approval.id),
alert_type=alert_type,
target_resource=target_resource,
namespace=namespace,
failure_reason="Prometheus rule auto_repair=falsefallback 未進入自動修復評估",
attempted_actions="llm_fallback -> guardrail:auto_repair_false -> emergency_intervention",
)
await _push_to_telegram_background(
approval_id=str(approval.id),
risk_level="medium",
@@ -2139,6 +2441,7 @@ async def alertmanager_webhook(
# (2026-04-08 Claude Sonnet 4.6 Asia/TaipeiADR-062 Q9)
# ==========================================================================
_alert_labels = alert.labels or {}
_alert_annotations = alert.annotations or {}
_alertname_for_log = _alert_labels.get("alertname", "UnknownAlert")
# Q9: auto_repair flag — Rule=false 強制 HITL不觸發自動修復背景任務
_can_auto_repair_by_rule = _alert_labels.get("auto_repair", "true").lower() == "true"
@@ -2154,6 +2457,7 @@ async def alertmanager_webhook(
"alert_id": alert_id,
"alertname": _alertname_for_log,
"labels": _alert_labels,
"annotations": _alert_annotations,
"auto_repair_flag": _can_auto_repair_by_rule,
},
)
@@ -2182,11 +2486,12 @@ async def alertmanager_webhook(
telegram = get_telegram_gateway()
# 解析 CI/CD 狀態
stage = alert.labels.get("stage", "")
job_status = "success" if alert.labels.get("severity") == "info" else "running"
job_status = _cicd_job_status_from_alert(alert)
commit_sha = alert.labels.get("commit", "")
triggered_by = alert.labels.get("triggered_by", "CI")
workflow_url = alert.annotations.get("workflow_url", "")
summary = alert.annotations.get("summary", alertname)
detail_message = alert.annotations.get("description", "")
await telegram.send_cicd_progress(
job_name=summary,
@@ -2194,6 +2499,8 @@ async def alertmanager_webhook(
stage=stage,
commit_sha=commit_sha,
triggered_by=triggered_by,
duration_seconds=_cicd_duration_seconds_from_alert(alert),
message=detail_message,
workflow_url=workflow_url,
)
@@ -2293,6 +2600,22 @@ async def alertmanager_webhook(
target=target_resource,
fingerprint=fingerprint,
)
background_tasks.add_task(
record_alertmanager_event,
project_id="awoooi",
alert_id=alert_id,
alertname=alertname,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
stage="received",
notification_type=notification_type,
alert_category=alert_category,
source_url=alert.generatorURL,
labels=dict(alert.labels) if alert.labels else {},
annotations=dict(alert.annotations) if alert.annotations else {},
)
# ==========================================================================
# ADR-076: 告警聚合引擎 — 5 分鐘滑動視窗,防止告警風暴
@@ -2375,6 +2698,26 @@ async def alertmanager_webhook(
hit_count=updated_approval.hit_count,
reason="Converged alert - Telegram already sent for this fingerprint",
)
background_tasks.add_task(
record_alertmanager_event,
project_id="awoooi",
alert_id=alert_id,
alertname=alertname,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
stage="converged",
notification_type=notification_type,
alert_category=alert_category,
incident_id=getattr(updated_approval, "incident_id", None),
approval_id=str(updated_approval.id),
repeat_count=updated_approval.hit_count,
is_duplicate=True,
source_url=alert.generatorURL,
labels=dict(alert.labels) if alert.labels else {},
annotations=dict(alert.annotations) if alert.annotations else {},
)
return AlertResponse(
success=True,
@@ -2402,10 +2745,27 @@ async def alertmanager_webhook(
message=message,
source="alertmanager",
alertname=alertname,
alert_labels=alert.labels,
alert_labels={**alert.labels, "fingerprint": fingerprint, "alert_id": alert_id},
notification_type="TYPE-1",
alert_category=alert_category,
)
background_tasks.add_task(
record_alertmanager_event,
project_id="awoooi",
alert_id=alert_id,
alertname=alertname,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
stage="incident_linked",
notification_type="TYPE-1",
alert_category=alert_category,
incident_id=_info_incident_id,
source_url=alert.generatorURL,
labels={**alert.labels, "fingerprint": fingerprint, "alert_id": alert_id},
annotations=dict(alert.annotations) if alert.annotations else {},
)
# 2026-04-15 ogt: TYPE-1 純資訊告警建立後立即關閉
# 設計原則: backup/heartbeat/info 告警無需追蹤狀態,通知即完成
# 防止 incidents 表無限累積 INVESTIGATING 記錄ADR-073 漏洞修補)
@@ -2425,7 +2785,7 @@ async def alertmanager_webhook(
record_alert_chain_success("alertmanager")
return AlertResponse(
success=True,
message=f"✅ TYPE-1 純資訊告警已通知 (no LLM)",
message="✅ TYPE-1 純資訊告警已通知 (no LLM)",
alert_id=alert_id,
approval_created=False,
)
@@ -2437,6 +2797,23 @@ async def alertmanager_webhook(
fingerprint=fingerprint,
ttl_seconds=ALERTMANAGER_LLM_INFLIGHT_LOCK_TTL_SECONDS,
)
background_tasks.add_task(
record_alertmanager_event,
project_id="awoooi",
alert_id=alert_id,
alertname=alertname,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
stage="llm_inflight_suppressed",
notification_type=notification_type,
alert_category=alert_category,
is_duplicate=True,
source_url=alert.generatorURL,
labels=dict(alert.labels) if alert.labels else {},
annotations=dict(alert.annotations) if alert.annotations else {},
)
return AlertResponse(
success=True,
message="🛡️ 告警已由同指紋背景 AI 分析處理中,跳過重複 LLM 呼叫",

View File

@@ -609,6 +609,70 @@ class Settings(BaseSettings):
"(X-AwoooP-Operator-Key header)"
),
)
ENABLE_AWOOOP_ANSIBLE_CHECK_MODE_WORKER: bool = Field(
default=False,
description=(
"True=consume ansible_candidate_matched AOL rows and run "
"ansible-playbook --check --diff only. Apply remains disabled."
),
)
AWOOOP_ANSIBLE_CHECK_MODE_INTERVAL_SECONDS: int = Field(
default=300,
ge=60,
description="AwoooP Ansible check-mode worker polling interval.",
)
AWOOOP_ANSIBLE_CHECK_MODE_BATCH_LIMIT: int = Field(
default=1,
ge=1,
le=5,
description="Maximum Ansible check-mode candidates claimed per worker tick.",
)
AWOOOP_ANSIBLE_CHECK_MODE_TIMEOUT_SECONDS: int = Field(
default=180,
ge=30,
le=600,
description="Timeout for one ansible-playbook --check --diff execution.",
)
AWOOOP_ANSIBLE_CHECK_MODE_STARTUP_SLEEP_SECONDS: int = Field(
default=120,
ge=0,
le=900,
description="Delay before the check-mode worker first tick after API startup.",
)
AWOOOP_ANSIBLE_CHECK_MODE_TRANSPORT_PROFILE: str = Field(
default="ssh_mcp",
description=(
"SSH transport profile used by Ansible check-mode. Production uses "
"the existing ssh-mcp key so repair-bot forced-command remains reserved "
"for whitelist repairs."
),
)
AWOOOP_ANSIBLE_CHECK_MODE_SSH_KEY_PATH: str = Field(
default="/run/secrets/ssh_mcp_key",
description="Private key path for Ansible check-mode SSH transport.",
)
AWOOOP_ANSIBLE_CHECK_MODE_KNOWN_HOSTS_PATH: str = Field(
default="/etc/ssh-mcp/known_hosts",
description="known_hosts path for Ansible check-mode SSH transport.",
)
AWOOOP_ANSIBLE_CHECK_MODE_CANDIDATE_MAX_AGE_HOURS: int = Field(
default=24,
ge=1,
le=168,
description=(
"Only recent Ansible candidate audit rows are eligible for automatic "
"check-mode claims; older backlog remains visible but is not drained as noise."
),
)
AWOOOP_ANSIBLE_CHECK_MODE_TRANSPORT_COOLDOWN_SECONDS: int = Field(
default=21_600,
ge=300,
le=86_400,
description=(
"Cooldown after transport-level check-mode blockers such as "
"forced-command repair SSH denial."
),
)
# ==========================================================================
# 統帥鐵律:禁止 SQLite (AWOOOI 憲法)

View File

@@ -37,8 +37,8 @@ REDIS_KEY_DECISION = "decision:"
APPROVAL_TO_INCIDENT_STATUS = {
"pending": "investigating",
"approved": "resolved",
"rejected": "rejected",
"expired": "expired",
"rejected": "escalated",
"expired": "escalated",
}
# Incident 狀態 → 是否活躍

View File

@@ -17,6 +17,7 @@ PostgreSQL 事務管理器,確保多表操作原子性。
from typing import Any
import structlog
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
logger = structlog.get_logger(__name__)
@@ -49,14 +50,20 @@ class UnitOfWork:
- Redis 操作失敗時必須手動呼叫 rollback()
"""
def __init__(self, session_factory: async_sessionmaker[AsyncSession]):
def __init__(
self,
session_factory: async_sessionmaker[AsyncSession],
project_id: str | None = None,
):
"""
初始化 UnitOfWork
Args:
session_factory: SQLAlchemy async session factory
project_id: RLS project context. None means contextvar/default awoooi.
"""
self._session_factory = session_factory
self._project_id = project_id
self._session: AsyncSession | None = None
self._committed = False
@@ -74,9 +81,18 @@ class UnitOfWork:
async def __aenter__(self) -> "UnitOfWork":
"""進入事務"""
from src.core.context import get_current_project_id
self._session = self._session_factory()
effective_pid = (
self._project_id if self._project_id is not None else get_current_project_id()
)
await self._session.execute(
text("SELECT set_config('app.project_id', :pid, TRUE)"),
{"pid": effective_pid},
)
self._committed = False
logger.debug("uow_started")
logger.debug("uow_started", project_id=effective_pid)
return self
async def __aexit__(

View File

@@ -635,6 +635,13 @@ class AwoooPConversationEvent(Base):
content_type: Mapped[str] = mapped_column(String(32), nullable=False, default="text")
content_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
content_preview: Mapped[str | None] = mapped_column(String(256), nullable=True)
content_redacted: Mapped[str | None] = mapped_column(Text, nullable=True)
redaction_version: Mapped[str] = mapped_column(
String(32), nullable=False, server_default=text("'audit_sink_v1'")
)
source_envelope: Mapped[dict[str, Any]] = mapped_column(
JSONB, nullable=False, server_default=text("'{}'::jsonb")
)
attachment_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
is_duplicate: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
provider_ts: Mapped[datetime | None] = mapped_column(nullable=True)
@@ -680,6 +687,13 @@ class AwoooPOutboundMessage(Base):
message_type: Mapped[str] = mapped_column(String(32), nullable=False)
content_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
content_preview: Mapped[str | None] = mapped_column(String(256), nullable=True)
content_redacted: Mapped[str | None] = mapped_column(Text, nullable=True)
redaction_version: Mapped[str] = mapped_column(
String(32), nullable=False, server_default=text("'audit_sink_v1'")
)
source_envelope: Mapped[dict[str, Any]] = mapped_column(
JSONB, nullable=False, server_default=text("'{}'::jsonb")
)
provider_message_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
send_status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending")
send_error: Mapped[str | None] = mapped_column(Text, nullable=True)

View File

@@ -106,10 +106,13 @@ async def get_db() -> AsyncGenerator[AsyncSession, None]:
factory = get_session_factory()
async with factory() as session:
try:
from src.core.context import get_current_project_id
# AwoooP Phase 2.3 (2026-05-04 ogt): SET LOCAL app.project_id 讓 RLS Policy 生效
# 預設 'awoooi',多租戶路由將在 middleware 注入實際 project_id
# 預設 'awoooi',多租戶路由將透過 contextvar 注入實際 project_id
await session.execute(
text("SELECT set_config('app.project_id', 'awoooi', TRUE)")
text("SELECT set_config('app.project_id', :pid, TRUE)"),
{"pid": get_current_project_id()},
)
yield session
await session.commit()
@@ -154,6 +157,9 @@ async def get_db_context(project_id: str | None = None) -> AsyncGenerator[AsyncS
# Initialization
# =============================================================================
_DB_BOOTSTRAP_LOCK_NAME = "awoooi:init_db:ddl"
async def init_db() -> None:
"""
Initialize database tables
@@ -162,6 +168,28 @@ async def init_db() -> None:
"""
engine = get_engine()
async with engine.connect() as lock_conn:
# 2026-05-24 ogt + Codex: 兩個 API replica 同時啟動時PostgreSQL 會在
# ALTER TABLE ... IF NOT EXISTS 上互相等待並 deadlock。整段 bootstrap
# DDL 必須序列化,避免 rollout 因一個 pod CrashLoop 變成 1/2 ready。
await lock_conn.execute(
text("SELECT pg_advisory_lock(hashtext(:lock_name))"),
{"lock_name": _DB_BOOTSTRAP_LOCK_NAME},
)
try:
await _run_init_db_ddl(engine)
finally:
await lock_conn.execute(
text("SELECT pg_advisory_unlock(hashtext(:lock_name))"),
{"lock_name": _DB_BOOTSTRAP_LOCK_NAME},
)
async def _run_init_db_ddl(engine: AsyncEngine) -> None:
"""
Run idempotent DB bootstrap DDL while caller holds the bootstrap advisory lock.
"""
# 2026-04-15 ogt: 多 replica 並行啟動競爭修復
# 問題:單一大 transaction 裡兩個 pod 同時建 table → 其中一個 CREATE INDEX 失敗
# PostgreSQL 中 transaction 內任何錯誤導致整個 transaction ROLLBACK

View File

@@ -633,6 +633,8 @@ class AlertOperationLog(Base):
"RESOLVED", "SILENCED", "ESCALATED", "GUARDRAIL_BLOCKED",
"PRE_FLIGHT_PASSED", "PRE_FLIGHT_FAILED", "BACKUP_TRIGGERED",
"BACKUP_COMPLETED", "BACKUP_FAILED", "APPROVAL_ESCALATED", "CHANGE_APPLIED",
"NOTIFICATION_CLASSIFIED", "MANUAL_FIX_RECORDED", "KM_CONVERTED",
"PLAYBOOK_DRAFT_CREATED", "STATE_GUARD_BLOCKED",
name="alert_event_type", create_type=False,
),
nullable=False, index=True,

View File

@@ -23,6 +23,7 @@ from src.db.base import get_db_context
from src.hermes.agent_loader import get_agent_system_prompt
from src.hermes.display_names import DEFAULT_AGENT, format_response_header
from src.hermes.safety_hooks import is_dangerous_input, is_mutate_intent
from src.services.ollama_endpoint_resolver import resolve_ollama_order
logger = structlog.get_logger(__name__)
@@ -261,44 +262,48 @@ async def process_nl_message(
t0 = time.monotonic()
# 呼叫 Ollama 本地模型111零費用按 agent 選模型)
# 呼叫 Ollama 模型(GCP-A → GCP-B → 111零費用按 agent 選模型)
model = _pick_model(agent_name)
success = False
error_type: str | None = None
try:
from src.services.ollama_endpoint_resolver import resolve_ollama_endpoint
ollama_base = resolve_ollama_endpoint("hermes")
async with httpx.AsyncClient(timeout=_OLLAMA_TIMEOUT) as _hc:
resp = await _hc.post(
f"{ollama_base}/api/chat",
json={
"model": model,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": prompt_with_ctx},
],
"stream": False,
"options": {"num_predict": 1500, "temperature": 0.3},
},
)
resp.raise_for_status()
result_text = resp.json().get("message", {}).get("content", "")
result_text = _strip_think_tags(result_text)
if not result_text:
result_text = "_Agent 回應為空請稍後再試。_"
success = True
except Exception as exc:
error_type = type(exc).__name__
logger.error(
"hermes_nl_ollama_error",
error=str(exc),
agent=agent_name,
model=model,
exc_type=error_type,
)
result_text = ""
async with httpx.AsyncClient(timeout=_OLLAMA_TIMEOUT) as _hc:
for endpoint in resolve_ollama_order("hermes"):
if not endpoint.url:
continue
try:
resp = await _hc.post(
f"{endpoint.url}/api/chat",
json={
"model": model,
# Keep Hermes responses in message.content across Ollama 0.24+.
"think": False,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": prompt_with_ctx},
],
"stream": False,
"options": {"num_predict": 1500, "temperature": 0.3},
},
)
resp.raise_for_status()
result_text = resp.json().get("message", {}).get("content", "")
result_text = _strip_think_tags(result_text)
if not result_text:
result_text = "_Agent 回應為空請稍後再試。_"
success = True
break
except Exception as exc:
error_type = type(exc).__name__
logger.error(
"hermes_nl_ollama_error",
error=str(exc),
agent=agent_name,
model=model,
provider=endpoint.provider_name,
exc_type=error_type,
)
if not success:
result_text = f"_Hermes 暫時無法連線({error_type}請稍後再試。_"
latency_ms = int((time.monotonic() - t0) * 1000)

View File

@@ -0,0 +1,44 @@
"""AwoooP Ansible check-mode worker loop.
Runs only when explicitly enabled by settings. The worker consumes pending
``ansible_candidate_matched`` rows and records check-mode evidence; it never
executes Ansible apply.
"""
from __future__ import annotations
import asyncio
import structlog
from src.core.config import settings
from src.services.awooop_ansible_check_mode_service import run_pending_check_modes_once
logger = structlog.get_logger(__name__)
async def run_awooop_ansible_check_mode_loop() -> None:
if not settings.ENABLE_AWOOOP_ANSIBLE_CHECK_MODE_WORKER:
logger.info("awooop_ansible_check_mode_worker_disabled")
return
logger.info(
"awooop_ansible_check_mode_worker_started",
interval_seconds=settings.AWOOOP_ANSIBLE_CHECK_MODE_INTERVAL_SECONDS,
batch_limit=settings.AWOOOP_ANSIBLE_CHECK_MODE_BATCH_LIMIT,
timeout_seconds=settings.AWOOOP_ANSIBLE_CHECK_MODE_TIMEOUT_SECONDS,
)
await asyncio.sleep(settings.AWOOOP_ANSIBLE_CHECK_MODE_STARTUP_SLEEP_SECONDS)
while True:
try:
result = await run_pending_check_modes_once(
limit=settings.AWOOOP_ANSIBLE_CHECK_MODE_BATCH_LIMIT,
timeout_seconds=settings.AWOOOP_ANSIBLE_CHECK_MODE_TIMEOUT_SECONDS,
)
if result.get("claimed") or result.get("blockers"):
logger.info("awooop_ansible_check_mode_worker_tick", **result)
except Exception as exc:
logger.warning("awooop_ansible_check_mode_worker_failed", error=str(exc))
await asyncio.sleep(settings.AWOOOP_ANSIBLE_CHECK_MODE_INTERVAL_SECONDS)

View File

@@ -0,0 +1,308 @@
"""
Hermes KB Growth Worker
=======================
消費 governance_remediation_dispatch 中的 hermes_kb_growth_healthcheck work item
把 knowledge_degradation 告警推進成可審核的 KM 草稿。
邊界:
- 可以建立 REVIEW 狀態的 auto_runbook 草稿,讓 owner 在前端審核。
- 不可以直接把 KM 標成 APPROVED / PUBLISHED。
- 不修改 immutable ai_governance_events流程進度寫回 dispatch.decision_context。
2026-05-19 ogt + Codex: T90 Hermes KB growth healthcheck worker。
"""
from __future__ import annotations
import asyncio
from copy import deepcopy
from typing import Any
import structlog
from src.db.base import get_db_context
from src.db.models import GovernanceRemediationDispatch
from src.models.knowledge import (
EntrySource,
EntryStatus,
EntryType,
KnowledgeEntry,
KnowledgeEntryCreate,
)
from src.repositories.governance_remediation_dispatch_repo import (
InvalidStatusTransition,
list_pending_by_executor,
transition_status,
update_decision_context,
)
from src.repositories.knowledge_repository import KnowledgeDBRepository
logger = structlog.get_logger(__name__)
EXECUTOR_TYPE = "hermes_kb_growth_healthcheck"
DEFAULT_INTERVAL_SECONDS = 300
DEFAULT_LIMIT = 20
async def run_hermes_kb_growth_once(limit: int = DEFAULT_LIMIT) -> dict[str, int]:
"""執行一輪 Hermes KB growth healthcheck。
Returns:
統計資訊,供 log / smoke test 判讀。
"""
rows = await list_pending_by_executor(EXECUTOR_TYPE, limit=limit)
result = {
"scanned": len(rows),
"processed": 0,
"skipped": 0,
"failed": 0,
}
for row in rows:
try:
await _process_dispatch(row)
result["processed"] += 1
except InvalidStatusTransition as exc:
result["skipped"] += 1
logger.info(
"hermes_kb_growth_dispatch_skipped",
dispatch_id=row.id,
event_id=row.governance_event_id,
reason=str(exc),
)
except Exception as exc:
result["failed"] += 1
logger.exception(
"hermes_kb_growth_dispatch_failed",
dispatch_id=row.id,
event_id=row.governance_event_id,
error=str(exc),
)
await _mark_failed_if_started(row.id, str(exc))
if any(result.values()):
logger.info("hermes_kb_growth_once_completed", **result)
return result
async def run_hermes_kb_growth_loop(
interval_seconds: int = DEFAULT_INTERVAL_SECONDS,
limit: int = DEFAULT_LIMIT,
) -> None:
"""背景 loop定期消費 Hermes KB growth dispatch。"""
logger.info(
"hermes_kb_growth_loop_started",
interval_seconds=interval_seconds,
limit=limit,
)
while True:
try:
await run_hermes_kb_growth_once(limit=limit)
except asyncio.CancelledError:
raise
except Exception as exc:
logger.exception("hermes_kb_growth_loop_error", error=str(exc))
await asyncio.sleep(interval_seconds)
async def _process_dispatch(row: GovernanceRemediationDispatch) -> None:
"""處理單筆 pending dispatch最後停在 waiting_owner_review。"""
dispatched = await transition_status(row.id, "pending", "dispatched")
executing = await transition_status(dispatched.id, "dispatched", "executing")
km_entry = await _create_or_get_km_review_draft(executing)
updated_context = _build_review_context(
executing.decision_context or {},
dispatch_id=executing.id,
governance_event_id=executing.governance_event_id,
km_entry_id=km_entry.id,
)
await update_decision_context(executing.id, updated_context)
await transition_status(executing.id, "executing", "succeeded")
logger.info(
"hermes_kb_growth_review_draft_ready",
dispatch_id=executing.id,
event_id=executing.governance_event_id,
km_entry_id=km_entry.id,
workflow_stage="waiting_owner_review",
)
async def _create_or_get_km_review_draft(
dispatch: GovernanceRemediationDispatch,
) -> KnowledgeEntry:
"""以 governance event tag 做冪等,建立或取得 REVIEW 狀態 KM 草稿。"""
dispatch_tag = f"dispatch:{dispatch.id}"
event_tag = f"governance_event:{dispatch.governance_event_id}"
payload = _build_km_review_entry_payload(dispatch)
async with get_db_context() as db:
repo = KnowledgeDBRepository(db)
existing, _ = await repo.list_entries(tags=[event_tag], limit=1)
if existing:
return existing[0]
existing, _ = await repo.list_entries(tags=[dispatch_tag], limit=1)
if existing:
return existing[0]
return await repo.create(payload)
def _build_km_review_entry_payload(
dispatch: GovernanceRemediationDispatch,
) -> KnowledgeEntryCreate:
"""把 governance dispatch 轉成待審核的 KM 草稿 payload。"""
context = dispatch.decision_context or {}
workflow = context.get("workflow") if isinstance(context.get("workflow"), dict) else {}
impact = workflow.get("impact") if isinstance(workflow.get("impact"), dict) else {}
extra = context.get("extra") if isinstance(context.get("extra"), dict) else {}
ownership = context.get("ownership") if isinstance(context.get("ownership"), dict) else {}
if not ownership and isinstance(extra.get("ownership"), dict):
ownership = extra["ownership"]
stale_count = _pick_first(impact, extra, key="stale_count")
total_count = _pick_first(impact, extra, key="total_count")
stale_ratio = _pick_first(impact, context, key="stale_ratio")
threshold = _pick_first(impact, context, key="threshold")
stale_days = _pick_first(impact, extra, key="stale_days")
lead_agent = ownership.get("lead_agent") or "Hermes"
human_owner = ownership.get("human_owner") or "KM owner / SRE owner"
content = "\n".join([
"# KM 健康檢查草稿",
"",
"## 來源",
f"- governance_event_id: {dispatch.governance_event_id}",
f"- dispatch_id: {dispatch.id}",
f"- executor_type: {dispatch.executor_type}",
"",
"## 影響摘要",
f"- stale_count: {_format_unknown(stale_count)}",
f"- total_count: {_format_unknown(total_count)}",
f"- stale_ratio: {_format_ratio(stale_ratio)}",
f"- threshold: {_format_ratio(threshold)}",
f"- stale_days: {_format_unknown(stale_days)}",
"",
"## AI 已完成",
"- Hermes 已接手 knowledge_degradation dispatch。",
"- 已產生 KM 更新草稿與 owner review work item。",
"- 尚未把任何條目標成 approved / published。",
"",
"## Owner 審核重點",
"- 優先反查最近被 Incident、Sentry、SigNoz、PlayBook 引用的 KM。",
"- 確認草稿內容沒有把過期處置方式寫回正式知識庫。",
"- 審核通過後再進入 km_writeback_after_approval。",
"",
"## 安全邊界",
"- writes_km_without_approval=false",
f"- lead_agent={lead_agent}",
f"- human_owner={human_owner}",
])
return KnowledgeEntryCreate(
title=f"KM healthcheck review draft - {dispatch.governance_event_id[:8]}",
content=content,
entry_type=EntryType.AUTO_RUNBOOK,
category="AI治理",
tags=[
"governance:knowledge_degradation",
"workflow:kb_growth_healthcheck",
"stage:waiting_owner_review",
"agent:Hermes",
"needs_owner_review",
f"dispatch:{dispatch.id}",
f"governance_event:{dispatch.governance_event_id}",
],
source=EntrySource.AI_EXTRACTED,
status=EntryStatus.REVIEW,
path_type="hermes_kb_growth_healthcheck",
created_by="hermes_kb_growth_worker",
)
def _build_review_context(
context: dict[str, Any],
*,
dispatch_id: str,
governance_event_id: str,
km_entry_id: str,
) -> dict[str, Any]:
"""更新 dispatch read model讓 Work Items/Telegram 可見目前停在 owner review。"""
updated = deepcopy(context)
workflow = updated.setdefault("workflow", {})
if not isinstance(workflow, dict):
workflow = {}
updated["workflow"] = workflow
stages = workflow.setdefault("stage_by_dispatch_status", {})
if not isinstance(stages, dict):
stages = {}
workflow["stage_by_dispatch_status"] = stages
stages.update({
"executing": "draft_km_updates",
"succeeded": "waiting_owner_review",
"failed": "needs_manual_km_triage",
})
workflow["current_stage"] = "waiting_owner_review"
workflow["next_action"] = "owner_review_km_draft"
workflow["needs_human_review"] = True
workflow["writes_km_without_approval"] = False
workflow["kb_draft_entry_id"] = km_entry_id
updated["next_action"] = "owner_review_km_draft"
updated["decision_path"] = "draft_created_waiting_owner_review"
updated["proposed_action"] = "Hermes 已建立 KM 更新草稿,等待 owner 審核"
updated["worker_result"] = {
"worker": "Hermes",
"executor_type": EXECUTOR_TYPE,
"dispatch_id": dispatch_id,
"governance_event_id": governance_event_id,
"km_draft_entry_id": km_entry_id,
"stage": "waiting_owner_review",
"status": "draft_created",
"writes_km_without_approval": False,
}
return updated
async def _mark_failed_if_started(dispatch_id: str, error: str) -> None:
"""若 worker 已取得 dispatch將它收斂到 failed保留錯誤。"""
for from_status in ("executing", "dispatched"):
try:
await transition_status(
dispatch_id,
from_status,
"failed",
last_error=error[:500],
)
return
except InvalidStatusTransition:
continue
except Exception as exc:
logger.warning(
"hermes_kb_growth_mark_failed_failed",
dispatch_id=dispatch_id,
from_status=from_status,
error=str(exc),
)
return
def _pick_first(*sources: dict[str, Any], key: str) -> Any:
for source in sources:
if key in source:
return source[key]
return None
def _format_unknown(value: Any) -> str:
return "unknown" if value is None else str(value)
def _format_ratio(value: Any) -> str:
try:
return f"{float(value) * 100:.1f}%"
except (TypeError, ValueError):
return "unknown"

View File

@@ -0,0 +1,289 @@
"""
Incident Lifecycle Reconciler
=============================
把已有強證據的舊 stuck incident 收斂回 RESOLVED。
範圍刻意保守:
- auto_repair_executions.success = true
- approval_records.status = EXECUTION_SUCCESS
- approval_records.status = EXPIRED
不處理單純 APPROVED / NO_ACTION / manual_required避免把仍需人工的事件
誤當作自動修復完成。
"""
from __future__ import annotations
import asyncio
from dataclasses import dataclass
import httpx
import structlog
from sqlalchemy import text
from src.core.config import settings
from src.db.base import get_db_context
from src.utils.timezone import now_taipei
logger = structlog.get_logger(__name__)
BATCH_LIMIT = 100
INTERVAL_SECONDS = 1800
_PROMETHEUS_TIMEOUT_SECONDS = 5.0
@dataclass(frozen=True)
class LifecycleCandidate:
incident_id: str
resolution_type: str
reason: str
direct_db_only: bool = False
async def run_incident_lifecycle_reconciler_loop() -> None:
"""每 30 分鐘收斂一小批已有完成證據的 stuck incident。"""
while True:
try:
resolved, errors = await reconcile_stuck_incidents()
if resolved > 0 or errors > 0:
logger.info(
"incident_lifecycle_reconciler_done",
resolved=resolved,
errors=errors,
batch_limit=BATCH_LIMIT,
)
except Exception as exc:
logger.warning("incident_lifecycle_reconciler_loop_failed", error=str(exc))
await asyncio.sleep(INTERVAL_SECONDS)
async def reconcile_stuck_incidents(limit: int = BATCH_LIMIT) -> tuple[int, int]:
"""
找出已完成但仍卡在 INVESTIGATING 的 incident透過 IncidentService 統一路徑結案。
Returns:
(resolved_count, error_count)
"""
candidates = await _fetch_candidates(limit)
remaining = max(0, limit - len(candidates))
if remaining > 0:
active_alertnames = await _fetch_active_alertnames()
if active_alertnames is not None:
candidates.extend(
await _fetch_inactive_or_duplicate_alert_candidates(
limit=remaining,
active_alertnames=active_alertnames,
exclude_incident_ids={c.incident_id for c in candidates},
)
)
if not candidates:
return 0, 0
from src.services.incident_service import get_incident_service
incident_service = get_incident_service()
resolved = 0
errors = 0
for candidate in candidates:
try:
if candidate.direct_db_only:
result = await _resolve_db_only(candidate.incident_id)
else:
result = await incident_service.resolve_incident(
candidate.incident_id,
resolution_type=candidate.resolution_type,
emit_postmortem=False,
)
if not result:
continue
resolved += 1
logger.info(
"incident_lifecycle_reconciled",
incident_id=candidate.incident_id,
reason=candidate.reason,
resolution_type=candidate.resolution_type,
direct_db_only=candidate.direct_db_only,
)
except Exception as exc:
errors += 1
logger.warning(
"incident_lifecycle_reconcile_failed",
incident_id=candidate.incident_id,
reason=candidate.reason,
error=str(exc),
)
return resolved, errors
async def _fetch_active_alertnames() -> set[str] | None:
"""Read current firing alertnames from Prometheus. None means fail-closed."""
try:
async with httpx.AsyncClient(timeout=_PROMETHEUS_TIMEOUT_SECONDS) as client:
response = await client.get(
f"{settings.PROMETHEUS_URL.rstrip('/')}/api/v1/query",
params={"query": 'ALERTS{alertstate="firing"}'},
)
response.raise_for_status()
payload = response.json()
except Exception as exc:
logger.warning("incident_lifecycle_active_alerts_fetch_failed", error=str(exc))
return None
result = payload.get("data", {}).get("result", [])
active_alertnames = {
item.get("metric", {}).get("alertname")
for item in result
if item.get("metric", {}).get("alertname")
}
logger.info(
"incident_lifecycle_active_alerts_loaded",
active_alert_count=len(active_alertnames),
)
return active_alertnames
async def _resolve_db_only(incident_id: str) -> bool:
from src.repositories.incident_repository import get_incident_repository
now = now_taipei()
return await get_incident_repository().update_status(
incident_id=incident_id,
status="resolved",
updated_at=now,
resolved_at=now,
)
async def _fetch_candidates(limit: int) -> list[LifecycleCandidate]:
async with get_db_context() as db:
result = await db.execute(
text(
"""
WITH stale AS (
SELECT
i.incident_id,
i.created_at,
EXISTS (
SELECT 1
FROM auto_repair_executions are
WHERE are.incident_id = i.incident_id
AND are.success IS TRUE
) AS has_success_auto_repair,
EXISTS (
SELECT 1
FROM approval_records ar
WHERE ar.incident_id = i.incident_id
AND ar.status::text = 'EXECUTION_SUCCESS'
) AS has_execution_success,
EXISTS (
SELECT 1
FROM approval_records ar
WHERE ar.incident_id = i.incident_id
AND ar.status::text = 'EXPIRED'
) AS has_expired_approval
FROM incidents i
WHERE i.status = 'INVESTIGATING'
AND i.created_at <= now() - interval '24 hours'
)
SELECT
incident_id,
CASE
WHEN has_success_auto_repair THEN 'auto_repair'
WHEN has_execution_success THEN 'auto_repair'
ELSE 'timeout'
END AS resolution_type,
CASE
WHEN has_success_auto_repair THEN 'auto_repair_execution_success'
WHEN has_execution_success THEN 'approval_execution_success'
ELSE 'approval_expired'
END AS reason
FROM stale
WHERE has_success_auto_repair
OR has_execution_success
OR has_expired_approval
ORDER BY created_at DESC
LIMIT :limit
"""
),
{
"limit": limit,
},
)
rows = result.mappings().all()
return [
LifecycleCandidate(
incident_id=str(row["incident_id"]),
resolution_type=str(row["resolution_type"]),
reason=str(row["reason"]),
)
for row in rows
]
async def _fetch_inactive_or_duplicate_alert_candidates(
*,
limit: int,
active_alertnames: set[str],
exclude_incident_ids: set[str],
) -> list[LifecycleCandidate]:
"""
收斂 Alertmanager 已不再 firing 的舊 incident以及同一 active alertname 的舊重複案。
若 Prometheus/Alertmanager 讀不到 active alertnames上層會 fail-closed 不呼叫本函式。
"""
active_list = list(active_alertnames) or ["__no_active_alertnames__"]
exclude_list = list(exclude_incident_ids) or ["__no_excluded_incidents__"]
async with get_db_context() as db:
result = await db.execute(
text(
"""
WITH ranked AS (
SELECT
i.incident_id,
i.alertname,
i.created_at,
row_number() OVER (
PARTITION BY i.alertname
ORDER BY i.created_at DESC, i.incident_id DESC
) AS rn
FROM incidents i
WHERE i.status = 'INVESTIGATING'
AND i.created_at <= now() - interval '24 hours'
AND NOT (i.incident_id = ANY(:exclude_incident_ids))
)
SELECT
incident_id,
CASE
WHEN alertname = ANY(:active_alertnames)
THEN 'active_duplicate_stale'
ELSE 'inactive_alert_stale'
END AS reason
FROM ranked
WHERE NOT (alertname = ANY(:active_alertnames) AND rn = 1)
ORDER BY created_at ASC
LIMIT :limit
"""
),
{
"active_alertnames": active_list,
"exclude_incident_ids": exclude_list,
"limit": limit,
},
)
rows = result.mappings().all()
return [
LifecycleCandidate(
incident_id=str(row["incident_id"]),
resolution_type="timeout",
reason=str(row["reason"]),
direct_db_only=True,
)
for row in rows
]

View File

@@ -28,7 +28,7 @@ from datetime import timedelta
import structlog
from sqlalchemy import select, update
from src.db.base import get_session_factory
from src.db.base import get_db_context
from src.db.models import AiGovernanceEvent, KnowledgeEntryRecord
from src.utils.timezone import now_taipei
@@ -129,7 +129,7 @@ class KbRotCleaner:
rot_reasons: dict[str, list[str]] = {}
total = 0
async with get_session_factory()() as session:
async with get_db_context() as session:
# 只掃 active 狀態(非 archived
q = await session.execute(
select(KnowledgeEntryRecord).where(
@@ -193,7 +193,7 @@ class KbRotCleaner:
if not result.stale_ids:
return
async with get_session_factory()() as session:
async with get_db_context() as session:
# 逐條更新(避免 bulk update 覆蓋 tags JSONB
q = await session.execute(
select(KnowledgeEntryRecord).where(
@@ -220,7 +220,7 @@ class KbRotCleaner:
async def _save_event(self, result: RotScanResult) -> None:
"""寫 kb_stale 事件到 ai_governance_events。"""
try:
async with get_session_factory()() as session:
async with get_db_context() as session:
event = AiGovernanceEvent(
event_type="kb_stale",
details=result.to_dict(),

View File

@@ -33,7 +33,7 @@ from datetime import timedelta
import structlog
from sqlalchemy import and_, select, update
from src.db.base import get_session_factory
from src.db.base import get_db_context
from src.db.models import KnowledgeEntryRecord
from src.models.knowledge import EntryStatus
from src.utils.timezone import now_taipei
@@ -112,8 +112,7 @@ class KnowledgeDecayJob:
cutoff = now_taipei() - timedelta(days=DECAY_AGE_DAYS)
decayable_statuses = [EntryStatus.DRAFT.value, EntryStatus.REVIEW.value]
session_factory = get_session_factory()
async with session_factory() as db:
async with get_db_context() as db:
# 查30 天未引用view_count=0且 updated_at < cutoff 的 draft/review 條目
stmt = select(KnowledgeEntryRecord).where(
and_(

View File

@@ -29,7 +29,7 @@ from datetime import timedelta
import structlog
from sqlalchemy import and_, select
from src.db.base import get_session_factory
from src.db.base import get_db_context
from src.db.models import AgentSession, AiGovernanceEvent, AutoRepairExecution, IncidentEvidence
from src.utils.timezone import now_taipei
@@ -109,9 +109,7 @@ class OfflineReplayService:
async def _run_replay(self) -> OfflineReplayReport:
cutoff = now_taipei() - timedelta(days=REPLAY_LOOKBACK_DAYS)
session_factory = get_session_factory()
async with session_factory() as db:
async with get_db_context() as db:
# 1. 取最近 N 個有 AgentSession(coordinator) 的 Incident
stmt = (
select(AgentSession.incident_id)
@@ -137,7 +135,7 @@ class OfflineReplayService:
)
results: list[IncidentReplayResult] = []
async with session_factory() as db:
async with get_db_context() as db:
for incident_id in incident_ids:
r = await self._replay_one(db, incident_id)
results.append(r)

View File

@@ -31,20 +31,26 @@ from fastapi.responses import JSONResponse, Response
from prometheus_client import CONTENT_TYPE_LATEST, generate_latest
from sentry_sdk.integrations.fastapi import FastApiIntegration
from sentry_sdk.integrations.starlette import StarletteIntegration
from uvicorn.middleware.proxy_headers import ProxyHeadersMiddleware
from src.api.v1 import agents as agents_v1 # Phase 9.5: Agent Teams API
from src.api.v1 import ai as ai_v1
from src.api.v1 import aider_events as aider_events_v1 # aider-watch v2 ADR-091
from src.api.v1 import ai_governance as ai_governance_v1 # 2026-05-02: /governance 頁面 3 endpoints
from src.api.v1 import (
ai_governance as ai_governance_v1, # 2026-05-02: /governance 頁面 3 endpoints
)
from src.api.v1 import ai_slo as ai_slo_v1 # Phase 6 ADR-087: AI SLO 自我治理
from src.api.v1 import aider_events as aider_events_v1 # aider-watch v2 ADR-091
from src.api.v1 import aiops_kpi as aiops_kpi_v1 # ADR-090 § Phase 7 KPI Dashboard
from src.api.v1 import aiops_timeline as aiops_timeline_v1 # 2026-04-27 Wave8-X3 B4 timeline endpoint
from src.api.v1 import approvals as approvals_v1
from src.api.v1 import (
aiops_timeline as aiops_timeline_v1, # 2026-04-27 Wave8-X3 B4 timeline endpoint
)
from src.api.v1 import alert_operation_logs as alert_operation_logs_v1
from src.api.v1 import approvals as approvals_v1
from src.api.v1 import audit_logs as audit_logs_v1
from src.api.v1 import auto_repair as auto_repair_v1 # #8: 自動升級決策
from src.api.v1 import csrf as csrf_v1 # Phase 20: CSRF Protection
from src.api.v1 import dashboard as dashboard_v1
from src.api.v1 import drift as drift_v1 # Phase 25 P2: Config Drift Detection
from src.api.v1 import errors as errors_v1 # #40: Sentry 錯誤 BFF API
from src.api.v1 import (
gitea_webhook as gitea_webhook_v1, # ADR-059: Gitea → OpenClaw (GitHub → Gitea 遷移)
@@ -56,19 +62,20 @@ from src.api.v1 import incidents as incidents_v1 # Phase 6.4: Decision Proposal
from src.api.v1 import knowledge as knowledge_v1 # KB Phase 1: Knowledge Base
from src.api.v1 import learning as learning_v1 # Phase D-G P0: Learning API
from src.api.v1 import metrics as metrics_v1 # Phase 7: Gold Metrics (真實血脈)
from src.api.v1 import monitoring as monitoring_v1 # 2026-04-03: 監控工具狀態
from src.api.v1 import notifications as notifications_v1 # 2026-04-10: 通知頻道狀態
from src.api.v1 import (
platform as platform_v1, # AwoooP Phase 4: Platform ShellShadow Mode
)
from src.api.v1 import playbooks as playbooks_v1 # #7: Playbook 萃取
from src.api.v1 import proposals as proposals_v1 # Phase 6.4h: Proposals CRUD API
from src.api.v1 import rag as rag_v1 # Phase 33 ADR-067: RAG 知識庫
from src.api.v1 import (
sentry_webhook as sentry_webhook_v1, # Phase 10.2.1: Sentry → Telegram
)
from src.api.v1 import (
signoz_webhook as signoz_webhook_v1, # Phase 21: SignOz → Telegram (ADR-037)
)
from src.api.v1 import drift as drift_v1 # Phase 25 P2: Config Drift Detection
from src.api.v1 import platform as platform_v1 # AwoooP Phase 4: Platform ShellShadow Mode
from src.api.v1 import rag as rag_v1 # Phase 33 ADR-067: RAG 知識庫
from src.api.v1 import monitoring as monitoring_v1 # 2026-04-03: 監控工具狀態
from src.api.v1 import notifications as notifications_v1 # 2026-04-10: 通知頻道狀態
from src.api.v1 import stats as stats_v1 # Phase 6.5: Statistics Analytics
from src.api.v1 import telegram as telegram_v1 # Phase 5.4: Telegram Gateway
from src.api.v1 import telegram_webhook as telegram_webhook_v1 # ADR-094: Webhook入口
@@ -76,11 +83,13 @@ from src.api.v1 import terminal as terminal_v1 # Phase 19.1: Omni-Terminal SSE
from src.api.v1 import timeline as timeline_v1
from src.api.v1 import webhooks as webhooks_v1
from src.core.config import settings
from src.core.feature_flags import aiops_flags # ADR-080: AI 自主化飛輪 feature flags 啟動驗證
from src.core.http_client import close_all_http_clients, init_all_http_clients
from src.core.logging import get_logger, setup_logging
from src.core.redis_client import close_redis_pool, init_redis_pool
from src.services.flywheel_stats_service import get_flywheel_stats_service
from src.core.redis_client import (
close_redis_pool,
close_worker_redis_pool,
init_redis_pool,
)
from src.core.sse import get_publisher
from src.core.telemetry import setup_telemetry, shutdown_telemetry
@@ -92,7 +101,10 @@ from src.routers import proposals as proposals_router
# Legacy route imports (to be migrated)
from src.routes import agent, notifications, pipelines, plugins
from src.services.adr100_slo_metrics_service import get_adr100_slo_metrics_service
from src.services.alert_chain_metrics_service import get_alert_chain_metrics_service
from src.services.executor import close_executor
from src.services.flywheel_stats_service import get_flywheel_stats_service
# Phase 5: OpenClaw AI Engine
from src.services.openclaw import close_openclaw
@@ -267,16 +279,21 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
# 2026-04-05 ogt: 重開機後 Redis 清空,從 DB restore 未解決的 incidents
# 統帥批准: 數據必須長久記錄,重開機後自動恢復 Working Memory
try:
from src.services.incident_service import get_incident_service
from sqlalchemy import select
from src.db.base import get_db_context
from src.db.models import IncidentRecord
from sqlalchemy import select
from src.models.incident import IncidentStatus
from src.services.incident_service import get_incident_service
incident_service = get_incident_service()
async with get_db_context() as db:
result = await db.execute(
select(IncidentRecord).where(
IncidentRecord.status.in_(["investigating", "mitigating"])
IncidentRecord.status.in_([
IncidentStatus.INVESTIGATING,
IncidentStatus.MITIGATING,
])
)
)
records = result.scalars().all()
@@ -284,31 +301,16 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
restored = 0
for record in records:
try:
from src.models.incident import Incident
incident = Incident(
incident_id=record.incident_id,
status=record.status,
severity=record.severity,
signals=record.signals or [],
affected_services=record.affected_services or [],
decision_chain=record.decision_chain,
proposal_ids=record.proposal_ids or [],
outcome=record.outcome,
created_at=record.created_at,
updated_at=record.updated_at,
resolved_at=record.resolved_at,
closed_at=record.closed_at,
ttl_days=record.ttl_days,
vectorized=record.vectorized,
# ADR-073: 分類欄位必須還原,否則 KM 寫入時全為 "unknown"
notification_type=record.notification_type,
alert_category=record.alert_category,
)
incident = incident_service._record_to_incident(record)
if await incident_service.save_to_working_memory(incident):
restored += 1
except Exception:
except Exception as record_error:
# 舊資料 source 值不合法node-exporter 等)→ 跳過
pass
logger.warning(
"working_memory_warmup_record_skipped",
incident_id=getattr(record, "incident_id", None),
error=str(record_error),
)
logger.info("working_memory_warmed_up", restored=restored, total=len(records))
except Exception as e:
@@ -351,7 +353,9 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
logger.warning("playbook_pg_backfill_schedule_failed", error=str(e))
try:
from src.services.playbook_embedding_service import ensure_playbook_embeddings_indexed
from src.services.playbook_embedding_service import (
ensure_playbook_embeddings_indexed,
)
asyncio.create_task(ensure_playbook_embeddings_indexed())
logger.info("playbook_embedding_indexing_scheduled")
except Exception as e:
@@ -499,6 +503,40 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
except Exception as e:
logger.warning("approval_timeout_resolver_schedule_failed", error=str(e))
# T73: 已有完成證據但仍卡在 INVESTIGATING 的舊 incident 小批次收斂。
# 僅處理 auto-repair success / approval EXECUTION_SUCCESS / approval EXPIRED
# 不自動關閉 manual_required 或單純 APPROVED 事件。
try:
from src.jobs.incident_lifecycle_reconciler import (
INTERVAL_SECONDS as INCIDENT_LIFECYCLE_RECONCILER_INTERVAL,
)
from src.jobs.incident_lifecycle_reconciler import (
run_incident_lifecycle_reconciler_loop,
)
asyncio.create_task(run_incident_lifecycle_reconciler_loop())
logger.info(
"incident_lifecycle_reconciler_scheduled",
interval_sec=INCIDENT_LIFECYCLE_RECONCILER_INTERVAL,
)
except Exception as e:
logger.warning("incident_lifecycle_reconciler_schedule_failed", error=str(e))
# AwoooP Ansible check-mode worker.
# 只執行 ansible-playbook --check --diff 並回寫 automation_operation_log
# apply 仍必須走 approval gate本 worker 不寫 auto_repair_executions。
try:
from src.jobs.awooop_ansible_check_mode_job import (
run_awooop_ansible_check_mode_loop,
)
asyncio.create_task(run_awooop_ansible_check_mode_loop())
logger.info(
"awooop_ansible_check_mode_worker_scheduled",
enabled=settings.ENABLE_AWOOOP_ANSIBLE_CHECK_MODE_WORKER,
interval_seconds=settings.AWOOOP_ANSIBLE_CHECK_MODE_INTERVAL_SECONDS,
)
except Exception as e:
logger.warning("awooop_ansible_check_mode_worker_schedule_failed", error=str(e))
# ADR-083 Phase 3: Evolver Agent每日— Playbook 自動合併 + 低信任封存
# 2026-04-15 ogt + Claude Sonnet 4.6(亞太): Phase 3 初始建立
try:
@@ -510,7 +548,9 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
# ADR-104 T2: LLM Playbook DRAFT governance每小時
try:
from src.jobs.playbook_generation_governance_job import run_playbook_generation_governance_loop
from src.jobs.playbook_generation_governance_job import (
run_playbook_generation_governance_loop,
)
asyncio.create_task(run_playbook_generation_governance_loop())
logger.info(
"playbook_generation_governance_loop_scheduled",
@@ -554,11 +594,11 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
# 2026-04-27 P3.1-T3 by Claude
try:
from src.utils.timezone import now_taipei
from datetime import datetime as _dt
async def _run_kb_rot_cleaner_loop() -> None:
from src.jobs.kb_rot_cleaner import get_kb_rot_cleaner
import asyncio as _asyncio
from src.jobs.kb_rot_cleaner import get_kb_rot_cleaner
while True:
try:
now = now_taipei()
@@ -649,14 +689,24 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
except Exception as e:
logger.warning("governance_dispatcher_schedule_failed", error=str(e))
# T90 2026-05-19 ogt + Codex: Hermes KB growth worker每 5 分鐘)
# 消費 knowledge_degradation 的 hermes_kb_growth_healthcheck dispatch
# 只產生 REVIEW 草稿並停在 owner review不直接批准或發布 KM。
try:
from src.jobs.hermes_kb_growth_worker import run_hermes_kb_growth_loop
asyncio.create_task(run_hermes_kb_growth_loop())
logger.info("hermes_kb_growth_worker_scheduled", interval_sec=300)
except Exception as e:
logger.warning("hermes_kb_growth_worker_schedule_failed", error=str(e))
# 2026-04-25 P1.2 by Claude Engineer-A2 — failover 整合到 ai_router + lifespan
# OllamaFailoverManager + OllamaAutoRecoveryService 飛輪接線:
# failover 切換時 → recovery_callback → set_current_primary → Redis 持久化
# recovery service 每 30s 檢查 → 111 連續 3 次 HEALTHY → 自動切回 → clear_cache
# 順序:先取 singleton → wire callback → 啟動 recovery service才能接收 callback
try:
from src.services.ollama_failover_manager import get_ollama_failover_manager
from src.services.ollama_auto_recovery import get_ollama_auto_recovery_service
from src.services.ollama_failover_manager import get_ollama_failover_manager
_failover_mgr = get_ollama_failover_manager()
_recovery_svc = get_ollama_auto_recovery_service()
@@ -669,8 +719,8 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
# alerter 還沒注入 Redis → dedup fail-open告警會送出且無 dedup 保護(重複告警風險)
# 修法configure_alerter() 提前到 start() 之前Redis pool 在 lifespan 早期已就緒
try:
from src.services.failover_alerter import configure_alerter
from src.core.redis_client import get_redis
from src.services.failover_alerter import configure_alerter
configure_alerter(get_redis())
logger.info("failover_alerter_configured")
except Exception as _alerter_err:
@@ -754,6 +804,7 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
# Phase 6.1: 關閉 Signal Worker (先關閉 Consumer)
await close_signal_worker()
await close_worker_redis_pool()
await publisher.stop()
await close_executor()
await close_openclaw()
@@ -806,11 +857,8 @@ else:
# Middleware
# =============================================================================
# 2026-04-03 ogt: Nginx 反向代理修正 — 讓 FastAPI 信任 X-Forwarded-Proto
# 解決問題: /api/v1/knowledge (無結尾斜線) 307 redirect 產生 http:// Location
# 原因: FastAPI 不知道自己在 HTTPS 後面redirect 回 http://
# 效果: 有了此中間件307 Location 會是 https://
from uvicorn.middleware.proxy_headers import ProxyHeadersMiddleware
# 2026-04-03 ogt: Nginx 反向代理修正 — 讓 FastAPI 信任 X-Forwarded-Proto
# 避免 /api/v1/knowledge 等 redirect 在 HTTPS 反向代理後產生 http:// Location
app.add_middleware(ProxyHeadersMiddleware, trusted_hosts="*")
# CORS - Strict Whitelist (Iron Law #2)
@@ -1006,6 +1054,15 @@ app.include_router(platform_v1.router, prefix="/api/v1/platform", tags=["AwoooP
@app.get("/metrics", include_in_schema=False)
async def prometheus_metrics() -> Response:
"""Prometheus metrics endpoint for alerting"""
# 2026-05-19 Codex — T85 Alert Chain DB evidence refresh.
# record_alert_chain_success() 是 process-local gauge部署後第一個 scrape
# 可能尚未收到新 webhook導致 smoke test 誤判 metric 不存在。
# 先用 AwoooP inbound / alert_operation_log 的 durable evidence 回填 last_success。
try:
await get_alert_chain_metrics_service().refresh_last_success_gauge()
except Exception as exc:
logger.warning("prometheus_metrics_alert_chain_evidence_error", error=str(exc))
content = generate_latest().decode("utf-8")
# 2026-05-07 ogt + Claude Sonnet 4.6 — INC-20260507-99ADF2 修復
# 飛輪指標awoooi_flywheel_*)原本只在 /api/v1/stats/flywheel/metrics 暴露,
@@ -1016,6 +1073,13 @@ async def prometheus_metrics() -> Response:
content += flywheel_metrics.to_prometheus_lines()
except Exception:
logger.warning("prometheus_metrics_flywheel_error")
# 2026-05-14 Codex — T18 ADR-100 SLO emitter
# GovernanceAgent 讀 Prometheus recording rules若 /metrics 不吐底層 DB totals
# sli:* rules 會全空並每小時重複發 governance_slo_data_gap。
try:
content += await get_adr100_slo_metrics_service().to_prometheus_lines()
except Exception as exc:
logger.warning("prometheus_metrics_adr100_slo_error", error=str(exc))
return Response(content=content, media_type=CONTENT_TYPE_LATEST)

View File

@@ -167,6 +167,8 @@ class ApprovalRequest(ApprovalRequestBase):
fingerprint: str | None = Field(default=None, description="告警指紋 Hash")
hit_count: int = Field(default=1, description="聚合觸發次數")
last_seen_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc), description="最後觸發時間")
telegram_message_id: int | None = Field(default=None, description="Telegram approval card message ID")
telegram_chat_id: int | None = Field(default=None, description="Telegram chat ID for the approval card")
# 2026-04-14 Claude Sonnet 4.6: incident_id 已移至 Base避免 ApprovalRequestCreate 缺欄位)
@property
@@ -216,6 +218,10 @@ class ApprovalRequestResponse(BaseModel):
hit_count: int = 1
last_seen_at: datetime | None = None
# Phase 6.5: Incident 關聯 (用於簽核後更新 Incident 狀態)
incident_id: str | None = None
matched_playbook_id: str | None = None
telegram_message_id: int | None = None
telegram_chat_id: int | None = None
metadata: dict | None = None
@classmethod
@@ -241,6 +247,10 @@ class ApprovalRequestResponse(BaseModel):
hit_count=approval.hit_count,
last_seen_at=approval.last_seen_at,
# Phase 6.5
incident_id=approval.incident_id,
matched_playbook_id=approval.matched_playbook_id,
telegram_message_id=approval.telegram_message_id,
telegram_chat_id=approval.telegram_chat_id,
metadata=approval.metadata,
)

View File

@@ -87,13 +87,27 @@ class DispatchItem(BaseModel):
governance_event_id: str
event_type: str
dispatch_status: str
executor_type: str | None = None
proposed_action: str = Field(description="≤120 字動作摘要")
playbook_id: str | None = None
playbook_trust: float | None = Field(default=None, ge=0.0, le=1.0)
created_at: datetime
dispatched_at: datetime | None = None
started_at: datetime | None = None
completed_at: datetime | None = None
operator_note: str | None = None
decision_path: str | None = None
workflow_stage: str | None = None
workflow_steps: list[str] = Field(default_factory=list)
next_action: str | None = None
lead_agent: str | None = None
support_agents: list[str] = Field(default_factory=list)
human_owner: str | None = None
kb_draft_entry_id: str | None = None
worker_status: str | None = None
dry_run_plan_fingerprint: str | None = None
archived_count: int | None = None
stale_ratio_snapshot: dict | None = None
class GovernanceQueueResponse(BaseModel):
@@ -107,6 +121,475 @@ class GovernanceQueueResponse(BaseModel):
)
# =============================================================================
# Endpoint 2B: KM review draft dedupe
# =============================================================================
class KnowledgeReviewDraftDedupeGroup(BaseModel):
governance_event_id: str
canonical_entry_id: str
canonical_title: str
canonical_updated_at: datetime | None = None
preferred_source: Literal["dispatch_context", "latest_review_draft"]
duplicate_entry_ids: list[str] = Field(default_factory=list)
duplicate_count: int
total_entries: int
suggested_action: str
owner_action: str
writes_on_read: bool = False
can_archive_without_owner_approval: bool = False
archive_history: list[DispatchItem] = Field(default_factory=list)
class KnowledgeReviewDraftDedupeResponse(BaseModel):
schema_version: str = "km_review_draft_dedupe_v1"
total_review_drafts: int
event_group_total: int
duplicate_draft_total: int
groups: list[KnowledgeReviewDraftDedupeGroup]
generated_at: datetime
class KnowledgeReviewDraftArchiveRequest(BaseModel):
canonical_entry_id: str = Field(min_length=1, max_length=120)
duplicate_entry_ids: list[str] = Field(min_length=1, max_length=100)
owner: str = Field(default="operator_console", min_length=1, max_length=100)
owner_approved: bool = False
dry_run: bool = False
dry_run_plan_fingerprint: str | None = Field(
default=None,
max_length=80,
description="Dry-run response fingerprint that must be echoed before a write.",
)
class KnowledgeReviewDraftStaleRatioSnapshot(BaseModel):
stale_count: int
total_count: int
stale_ratio: float
threshold: float
stale_days: int
class KnowledgeReviewDraftArchiveResponse(BaseModel):
schema_version: str = "km_review_draft_archive_v1"
governance_event_id: str
canonical_entry_id: str
requested_duplicate_entry_ids: list[str]
archived_entry_ids: list[str] = Field(default_factory=list)
skipped_entry_ids: list[str] = Field(default_factory=list)
would_archive_entry_ids: list[str] = Field(default_factory=list)
status: Literal["dry_run", "archived", "noop_already_archived"]
owner: str
owner_approved: bool
dry_run: bool
writes_km: bool
writes_governance_audit: bool
audit_dispatch_id: str | None = None
stale_ratio_snapshot: KnowledgeReviewDraftStaleRatioSnapshot | None = None
stale_ratio_recheck_status: Literal[
"dry_run",
"completed",
"already_active",
"not_requested",
] = "not_requested"
stale_ratio_recheck_dispatch_id: str | None = None
dry_run_plan_fingerprint: str | None = None
next_action: str = "stale_ratio_recheck"
generated_at: datetime
# =============================================================================
# Endpoint 2C: KM stale candidates
# =============================================================================
class KnowledgeStaleCandidate(BaseModel):
entry_id: str
project_id: str
title: str
entry_type: str
category: str | None = None
status: str
source: str | None = None
updated_at: datetime | None = None
stale_days: int
view_count: int
priority_score: int
priority_tier: Literal["P0", "P1", "P2"]
recommended_action: Literal[
"refresh_with_evidence",
"owner_review",
"archive_or_supersede",
]
reasons: list[str] = Field(default_factory=list)
correlation_sources: list[str] = Field(default_factory=list)
related_incident_id: str | None = None
related_playbook_id: str | None = None
related_approval_id: str | None = None
tags: list[str] = Field(default_factory=list)
owner_review_dispatch_id: str | None = None
owner_review_status: str | None = None
owner_review_stage: str | None = None
owner_review_next_action: str | None = None
class KnowledgeStaleCandidatesResponse(BaseModel):
schema_version: str = "km_stale_candidates_v1"
project_id: str
total_stale: int
returned: int
threshold_days: int
writes_on_read: bool = False
manual_review_required: bool = True
items: list[KnowledgeStaleCandidate]
generated_at: datetime
class KnowledgeStaleOwnerReviewRequest(BaseModel):
owner: str = Field(default="operator_console", min_length=1, max_length=100)
owner_note: str | None = Field(default=None, max_length=240)
dry_run: bool = False
class KnowledgeStaleOwnerReviewResponse(BaseModel):
schema_version: str = "km_stale_owner_review_v1"
entry_id: str
project_id: str
status: Literal["dry_run", "queued", "already_queued"]
governance_event_id: str | None = None
dispatch_id: str | None = None
workflow_stage: str
recommended_action: Literal[
"refresh_with_evidence",
"owner_review",
"archive_or_supersede",
]
owner: str
owner_note: str | None = None
writes_km: bool = False
writes_governance_audit: bool
next_action: str = "owner_review_stale_km_candidate"
generated_at: datetime
class KnowledgeStaleOwnerReviewBatchQueueRequest(BaseModel):
project_id: str = Field(default="awoooi", min_length=1, max_length=64)
priority_tiers: list[Literal["P0", "P1", "P2"]] = Field(
default_factory=lambda: ["P0", "P1"],
min_length=1,
max_length=3,
)
limit: int = Field(default=10, ge=1, le=50)
owner: str = Field(default="operator_console", min_length=1, max_length=100)
owner_note: str | None = Field(default=None, max_length=240)
dry_run: bool = False
dry_run_plan_fingerprint: str | None = Field(
default=None,
max_length=80,
description="Dry-run response fingerprint that must be echoed before queueing a batch.",
)
class KnowledgeStaleOwnerReviewBatchItem(BaseModel):
entry_id: str
title: str
priority_tier: Literal["P0", "P1", "P2"]
recommended_action: Literal[
"refresh_with_evidence",
"owner_review",
"archive_or_supersede",
]
status: Literal["would_queue", "queued", "already_queued", "skipped"]
reason: str | None = None
governance_event_id: str | None = None
dispatch_id: str | None = None
workflow_stage: str
class KnowledgeStaleOwnerReviewBatchQueueResponse(BaseModel):
schema_version: str = "km_stale_owner_review_batch_v1"
project_id: str
status: Literal["dry_run", "queued", "noop_already_queued"]
owner: str
owner_note: str | None = None
dry_run: bool
priority_tiers: list[str]
requested_limit: int
candidate_count: int
queued_count: int
already_queued_count: int
skipped_count: int
batch_governance_event_id: str | None = None
batch_dispatch_id: str | None = None
workflow_stage: str
writes_km: bool = False
writes_governance_audit: bool
stale_ratio_snapshot: KnowledgeReviewDraftStaleRatioSnapshot | None = None
dry_run_plan_fingerprint: str | None = None
items: list[KnowledgeStaleOwnerReviewBatchItem] = Field(default_factory=list)
next_action: str = "owner_review_stale_km_batch"
generated_at: datetime
class KnowledgeStaleOwnerReviewInboxItem(BaseModel):
dispatch_id: str
governance_event_id: str
entry_id: str
project_id: str
title: str
dispatch_status: str
workflow_stage: str
next_action: str | None = None
owner: str | None = None
owner_note: str | None = None
batch_governance_event_id: str | None = None
batch_dispatch_id: str | None = None
priority_tier: Literal["P0", "P1", "P2"]
priority_score: int
recommended_action: Literal[
"refresh_with_evidence",
"owner_review",
"archive_or_supersede",
]
stale_days: int
view_count: int
correlation_sources: list[str] = Field(default_factory=list)
reasons: list[str] = Field(default_factory=list)
related_incident_id: str | None = None
related_playbook_id: str | None = None
related_approval_id: str | None = None
dry_run_plan_fingerprint: str | None = None
queued_at: datetime | None = None
started_at: datetime | None = None
completed_at: datetime | None = None
class KnowledgeStaleOwnerReviewInboxResponse(BaseModel):
schema_version: str = "km_stale_owner_review_inbox_v1"
project_id: str
dispatch_status: str
total: int
returned: int
writes_on_read: bool = False
manual_review_required: bool = True
items: list[KnowledgeStaleOwnerReviewInboxItem] = Field(default_factory=list)
generated_at: datetime
class KnowledgeStaleOwnerReviewBurnDownItem(BaseModel):
completion_dispatch_id: str
governance_event_id: str
source_dispatch_id: str | None = None
recheck_dispatch_id: str | None = None
entry_id: str | None = None
project_id: str
dispatch_status: str
workflow_stage: str
review_outcome: Literal[
"refresh_with_evidence",
"archive",
"supersede",
] | None = None
owner: str | None = None
completed_at: datetime | None = None
stale_ratio_snapshot: KnowledgeReviewDraftStaleRatioSnapshot | None = None
stale_count_delta: int | None = None
stale_ratio_delta: float | None = None
above_threshold: bool | None = None
class KnowledgeStaleOwnerReviewBurnDownResponse(BaseModel):
schema_version: str = "km_stale_owner_review_burndown_v1"
project_id: str
burn_down_status: Literal["above_threshold", "at_or_below_threshold", "no_data"]
current_snapshot: KnowledgeReviewDraftStaleRatioSnapshot | None = None
entries_to_threshold: int
pending_owner_reviews: int
completed_owner_reviews: int
completion_audit_total: int
stale_ratio_recheck_total: int
latest_stale_count_delta: int | None = None
latest_stale_ratio_delta: float | None = None
writes_on_read: bool = False
manual_review_required: bool = True
returned: int
items: list[KnowledgeStaleOwnerReviewBurnDownItem] = Field(default_factory=list)
generated_at: datetime
class KnowledgeStaleOwnerReviewCompletionQueueItem(BaseModel):
dispatch_id: str
governance_event_id: str
entry_id: str
project_id: str
title: str
dispatch_status: str
workflow_stage: str
readiness: Literal["ready", "blocked", "completed", "failed"]
recommended_completion_outcome: Literal[
"refresh_with_evidence",
"archive",
"supersede",
]
next_action: str
blockers: list[str] = Field(default_factory=list)
required_owner_fields: list[str] = Field(default_factory=list)
can_preview: bool
can_confirm_after_preview: bool
writes_km_on_confirm: bool
owner: str | None = None
owner_note: str | None = None
batch_governance_event_id: str | None = None
batch_dispatch_id: str | None = None
priority_tier: Literal["P0", "P1", "P2"]
priority_score: int
recommended_action: Literal[
"refresh_with_evidence",
"owner_review",
"archive_or_supersede",
]
stale_days: int
view_count: int
correlation_sources: list[str] = Field(default_factory=list)
reasons: list[str] = Field(default_factory=list)
related_incident_id: str | None = None
related_playbook_id: str | None = None
related_approval_id: str | None = None
dry_run_plan_fingerprint: str | None = None
queued_at: datetime | None = None
started_at: datetime | None = None
completed_at: datetime | None = None
class KnowledgeStaleOwnerReviewCompletionQueueResponse(BaseModel):
schema_version: str = "km_stale_owner_review_completion_queue_v1"
project_id: str
status_bucket: Literal["all", "ready", "blocked", "completed", "failed", "pending"]
priority_tiers: list[str] = Field(default_factory=list)
recommended_completion_outcome: Literal[
"all",
"refresh_with_evidence",
"archive",
"supersede",
] = "all"
batch_governance_event_id: str | None = None
can_preview: bool | None = None
total: int
returned: int
pending_count: int
ready_count: int
blocked_count: int
completed_count: int
failed_count: int
writes_on_read: bool = False
manual_review_required: bool = True
batch_writes_allowed: bool = False
items: list[KnowledgeStaleOwnerReviewCompletionQueueItem] = Field(default_factory=list)
generated_at: datetime
class KnowledgeStaleOwnerReviewCompletionBatchPreviewRequest(BaseModel):
project_id: str = Field(default="awoooi", min_length=1, max_length=64)
status_bucket: Literal["all", "ready", "blocked", "completed", "failed", "pending"] = "ready"
priority_tiers: list[Literal["P0", "P1", "P2"]] = Field(
default_factory=lambda: ["P0", "P1", "P2"],
min_length=1,
max_length=3,
)
recommended_completion_outcome: Literal[
"all",
"refresh_with_evidence",
"archive",
"supersede",
] = "all"
batch_governance_event_id: str | None = Field(default=None, max_length=120)
limit: int = Field(default=10, ge=1, le=30)
owner: str = Field(default="operator_console", min_length=1, max_length=100)
owner_note: str | None = Field(default=None, max_length=240)
class KnowledgeStaleOwnerReviewCompletionBatchPreviewResponse(BaseModel):
schema_version: str = "km_stale_owner_review_completion_batch_preview_v1"
project_id: str
status: Literal["dry_run"] = "dry_run"
owner: str
owner_note: str | None = None
status_bucket: Literal["all", "ready", "blocked", "completed", "failed", "pending"]
priority_tiers: list[str]
recommended_completion_outcome: Literal[
"all",
"refresh_with_evidence",
"archive",
"supersede",
]
batch_governance_event_id: str | None = None
requested_limit: int
candidate_count: int
previewable_count: int
blocked_count: int
completed_count: int
failed_count: int
writes_km: bool = False
writes_governance_audit: bool = False
batch_writes_allowed: bool = False
manual_review_required: bool = True
dry_run_plan_fingerprint: str
next_action: str = "preview_each_ready_item_then_confirm_single_item"
items: list[KnowledgeStaleOwnerReviewCompletionQueueItem] = Field(default_factory=list)
generated_at: datetime
class KnowledgeStaleOwnerReviewCompleteRequest(BaseModel):
dispatch_id: str | None = Field(
default=None,
max_length=120,
description="Owner-review dispatch id. Optional when the backend can resolve the active item by entry id.",
)
owner: str = Field(default="operator_console", min_length=1, max_length=100)
owner_approved: bool = False
dry_run: bool = False
review_outcome: Literal[
"refresh_with_evidence",
"archive",
"supersede",
]
owner_note: str | None = Field(default=None, max_length=500)
updated_title: str | None = Field(default=None, min_length=1, max_length=255)
updated_content: str | None = Field(default=None, min_length=1)
superseded_by_entry_id: str | None = Field(default=None, max_length=120)
dry_run_plan_fingerprint: str | None = Field(
default=None,
max_length=80,
description="Dry-run response fingerprint that must be echoed before a write.",
)
class KnowledgeStaleOwnerReviewCompleteResponse(BaseModel):
schema_version: str = "km_stale_owner_review_complete_v1"
entry_id: str
project_id: str
status: Literal["dry_run", "completed", "already_completed"]
review_outcome: Literal[
"refresh_with_evidence",
"archive",
"supersede",
]
governance_event_id: str
dispatch_id: str
audit_dispatch_id: str | None = None
stale_ratio_recheck_dispatch_id: str | None = None
workflow_stage: str
owner: str
owner_approved: bool
dry_run: bool
writes_km: bool
writes_governance_audit: bool
stale_ratio_snapshot: KnowledgeReviewDraftStaleRatioSnapshot | None = None
dry_run_plan_fingerprint: str | None = None
next_action: str = "stale_ratio_recheck"
generated_at: datetime
# =============================================================================
# Endpoint 3: summary
# =============================================================================

View File

@@ -388,10 +388,7 @@ class McpGateway:
parameters: dict[str, Any],
) -> MCPToolResult:
"""呼叫底層 MCP provider 執行工具"""
registry = get_provider_registry()
provider = registry.get(ctx.tool_name) or registry.get(
tool_row.tool_name if tool_row else ctx.tool_name
)
provider = await self._resolve_provider(ctx, tool_row)
# 找不到 provider → 回傳 shadow no-op
if provider is None:
@@ -407,14 +404,57 @@ class McpGateway:
)
audit_params = dict(parameters)
existing_audit = (
parameters.get("_mcp_audit")
if isinstance(parameters, dict) and isinstance(parameters.get("_mcp_audit"), dict)
else {}
)
audit_params["_mcp_audit"] = {
"project_id": ctx.project_id,
"agent_id": ctx.agent_id,
"run_id": str(ctx.run_id) if ctx.run_id else None,
"trace_id": ctx.trace_id,
"incident_id": existing_audit.get("incident_id") or ctx.trace_id,
"session_id": existing_audit.get("session_id"),
"flywheel_node": existing_audit.get("flywheel_node"),
"agent_role": existing_audit.get("agent_role") or ctx.agent_id,
"gateway_path": "awooop_mcp_gateway",
}
return await provider.execute(ctx.tool_name, audit_params)
async def _resolve_provider(
self,
ctx: GatewayContext,
tool_row: AwoooPMcpToolRegistry | None,
):
"""Find the provider that owns ctx.tool_name.
ProviderRegistry is keyed by provider name (`kubernetes`, `ssh_host`, ...),
while GatewayContext intentionally uses the governed tool name
(`kubectl_get`, `ssh_diagnose`, ...). Scan provider tool manifests as the
compatibility bridge until registry exposes a first-class tool index.
"""
registry = get_provider_registry()
direct = registry.get(ctx.tool_name)
if direct is not None:
return direct
lookup_name = tool_row.tool_name if tool_row else ctx.tool_name
for provider in registry.all():
try:
tools = await provider.list_tools()
except Exception as exc:
logger.debug(
"mcp_gateway_provider_manifest_skipped",
provider=getattr(provider, "name", None),
tool_name=lookup_name,
error=str(exc),
)
continue
if any(tool.name == lookup_name for tool in tools):
return provider
return None
# ── Audit log ─────────────────────────────────────────────────────────────
async def _write_audit(
@@ -442,6 +482,15 @@ class McpGateway:
json.dumps(result.output, sort_keys=True, default=str).encode()
).hexdigest()
gate_payload = {
**gate_result.as_dict(),
"schema_version": "awooop_mcp_gateway_audit_v1",
"gateway_path": "awooop_mcp_gateway",
"policy_enforced": True,
"is_shadow": ctx.is_shadow,
"required_scope": ctx.required_scope,
}
audit = AwoooPMcpGatewayAudit(
project_id=ctx.project_id,
run_id=ctx.run_id,
@@ -451,7 +500,7 @@ class McpGateway:
tool_name=ctx.tool_name,
input_hash=input_hash,
output_hash=output_hash,
gate_result=gate_result.as_dict(),
gate_result=gate_payload,
result_status=result_status,
block_gate=block_gate,
block_reason=block_reason,

View File

@@ -65,6 +65,13 @@ DEFAULT_HOST_USERS = {
# AI/Web host is operated by the ollama account in the current topology.
"192.168.0.188": "ollama",
}
SHORT_HOST_MAP = {
"110": "192.168.0.110",
"120": "192.168.0.120",
"121": "192.168.0.121",
"188": "192.168.0.188",
"wooo": "192.168.0.110",
}
DIAG_TIMEOUT = 10 # 診斷類超時(秒)
OP_TIMEOUT = 60 # 操作類超時(秒)
@@ -127,7 +134,9 @@ def _normalize_ssh_host(value: str) -> str:
if host.count(":") == 1:
maybe_host, maybe_port = host.rsplit(":", 1)
if maybe_port.isdigit():
return maybe_host
host = maybe_host
if host in SHORT_HOST_MAP:
return SHORT_HOST_MAP[host]
return host
@@ -240,6 +249,10 @@ class SSHProvider(MCPToolProvider):
),
input_schema={"type": "object", "properties": {
"host": {"type": "string", "description": "Target host IP"},
"container_name": {
"type": "string",
"description": "Optional Docker container name for container-focused diagnostics",
},
}, "required": ["host"]},
server_name=self.name,
),
@@ -542,12 +555,23 @@ class SSHProvider(MCPToolProvider):
# 所有接受用戶字串的工具,必須先通過 _validate_param() 白名單驗證
if tool_name == "ssh_diagnose":
# 2026-04-27 Claude Sonnet 4.6: 主機告警自動診斷 — 只讀,不修改任何狀態
return (
command = (
"echo '=== CPU TOP ===' && ps aux --sort=-%cpu | head -15 && "
"echo '=== MEMORY ===' && free -h && "
"echo '=== DISK ===' && df -h && "
"echo '=== LOAD ===' && uptime"
)
container_name = params.get("container_name")
if container_name:
name = _validate_param("container_name", str(container_name))
command = (
f"{command} && "
f"echo '=== DOCKER STATS {name} ===' && "
f"docker stats --no-stream {name} 2>&1 && "
f"echo '=== DOCKER INSPECT {name} ===' && "
f"docker inspect {name} 2>&1 | head -80"
)
return command
if tool_name == "ssh_get_top_processes":
return "ps aux --sort=-%cpu | head -15"
@@ -564,7 +588,10 @@ class SSHProvider(MCPToolProvider):
return f"docker logs {name} --tail {tail} 2>&1"
if tool_name == "ssh_get_container_status":
name = _validate_param("filter_name", params["filter_name"])
raw_name = params.get("filter_name") or params.get("container_name") or params.get("name")
if not raw_name:
raise ValueError("Missing filter_name for ssh_get_container_status")
name = _validate_param("filter_name", str(raw_name))
return f"docker ps -a --filter name={name}"
if tool_name == "ssh_get_service_status":

View File

@@ -106,6 +106,8 @@ def _record_to_request(record: ApprovalRecord) -> ApprovalRequest:
# B4 fix 2026-04-24 ogt + Claude Sonnet 4.6: 補回 DB 欄位(人工審核路徑讀回必要)
incident_id=getattr(record, "incident_id", None),
matched_playbook_id=getattr(record, "matched_playbook_id", None),
telegram_message_id=getattr(record, "telegram_message_id", None),
telegram_chat_id=getattr(record, "telegram_chat_id", None),
)

View File

@@ -18,7 +18,14 @@ import structlog
from sqlalchemy import text
from src.db.base import get_db_context
from src.models.drift import DriftInterpretation, DriftIntent, DriftItem, DriftLevel, DriftReport, DriftStatus
from src.models.drift import (
DriftIntent,
DriftInterpretation,
DriftItem,
DriftLevel,
DriftReport,
DriftStatus,
)
logger = structlog.get_logger(__name__)
@@ -167,6 +174,40 @@ class DriftReportRepository:
{"report_id": report_id, "narrative": narrative},
)
async def get_repeat_state(
self,
report: DriftReport,
*,
include_values: bool = True,
) -> dict:
"""Return stable fingerprint repeat state for a drift report."""
from src.services.drift_repeat_state import build_drift_repeat_state
async with get_db_context() as db:
result = await db.execute(
text("""
SELECT
report_id,
namespace,
status,
scanned_at,
created_at,
items
FROM drift_reports
WHERE namespace = :namespace
AND created_at > now() - interval '24 hours'
ORDER BY scanned_at DESC
LIMIT 200
"""),
{"namespace": report.namespace},
)
rows = [dict(row) for row in result.mappings().all()]
return build_drift_repeat_state(
report,
rows,
include_values=include_values,
)
_drift_repo: DriftReportRepository | None = None

View File

@@ -356,6 +356,75 @@ async def list_pending(
return list(result.scalars().all())
async def list_pending_by_executor(
executor_type: str,
*,
limit: int = 50,
) -> list[GovernanceRemediationDispatch]:
"""列出指定 executor 的 pending dispatch按 dispatched_at ASC
用於 Hermes / 其他 worker 消費自己的 work item。由 repository 層集中查詢,
避免 job 直接散落表名與狀態條件。
Args:
executor_type: dispatch.executor_type例如 hermes_kb_growth_healthcheck
limit: 本輪最多取幾筆,避免 backlog 一次拖垮 worker
Returns:
最舊優先的 pending dispatch 列表。
"""
async with get_db_context() as db:
result = await db.execute(
select(GovernanceRemediationDispatch)
.where(GovernanceRemediationDispatch.dispatch_status == "pending")
.where(GovernanceRemediationDispatch.executor_type == executor_type)
.order_by(GovernanceRemediationDispatch.dispatched_at.asc())
.limit(limit)
)
return list(result.scalars().all())
async def update_decision_context(
dispatch_id: str,
decision_context: dict[str, Any],
) -> GovernanceRemediationDispatch:
"""更新 dispatch 的 decision_context保留同一 row 的 audit trail。
這只更新 dispatch work item 的讀模型上下文,不修改 immutable
ai_governance_events也不代表治理事件已被解決。
Args:
dispatch_id: governance_remediation_dispatch.id
decision_context: 新的 JSONB context
Returns:
更新後的 GovernanceRemediationDispatch ORM 物件
Raises:
DispatchNotFound: 找不到 dispatch_id
"""
async with get_db_context() as db:
result = await db.execute(
select(GovernanceRemediationDispatch)
.where(GovernanceRemediationDispatch.id == dispatch_id)
)
row = result.scalar_one_or_none()
if row is None:
raise DispatchNotFound(f"dispatch_id={dispatch_id!r} 不存在")
row.decision_context = decision_context
await db.flush()
await db.refresh(row)
logger.info(
"dispatch_decision_context_updated",
dispatch_id=dispatch_id,
event_id=row.governance_event_id,
executor_type=row.executor_type,
)
return row
async def list_by_event(
event_id: str,
) -> list[GovernanceRemediationDispatch]:

View File

@@ -19,7 +19,12 @@ from sqlalchemy import select
from src.db.base import get_db_context
from src.db.models import IncidentRecord
from src.models.incident import Incident, IncidentFrequencyStats, IncidentStatus, Severity
from src.models.incident import (
Incident,
IncidentFrequencyStats,
IncidentStatus,
Severity,
)
from src.repositories.interfaces import IIncidentRepository
logger = structlog.get_logger(__name__)
@@ -41,8 +46,8 @@ def _record_to_incident(record: IncidentRecord) -> Incident:
return Incident(
incident_id=record.incident_id,
status=IncidentStatus(record.status),
severity=Severity(record.severity),
status=IncidentStatus(_normalize_status(record.status)),
severity=Severity(_normalize_severity(record.severity)),
signals=record.signals or [],
affected_services=record.affected_services or [],
proposal_ids=record.proposal_ids or [],
@@ -93,6 +98,36 @@ def _incident_to_record_data(incident: Incident) -> dict[str, Any]:
}
def _normalize_status(value: str | IncidentStatus) -> str:
if isinstance(value, IncidentStatus):
return value.value
raw = str(value)
if raw in IncidentStatus.__members__:
return IncidentStatus[raw].value
normalized = raw.strip().lower()
if normalized == "open":
return IncidentStatus.INVESTIGATING.value
return normalized
def _normalize_severity(value: str | Severity) -> str:
if isinstance(value, Severity):
return value.value
raw = str(value)
if raw in Severity.__members__:
return Severity[raw].value
legacy_map = {
"critical": Severity.P0.value,
"high": Severity.P1.value,
"warning": Severity.P2.value,
"medium": Severity.P2.value,
"info": Severity.P3.value,
"low": Severity.P3.value,
"none": Severity.P3.value,
}
return legacy_map.get(raw.strip().lower(), raw)
# =============================================================================
# IncidentDBRepository
# =============================================================================
@@ -136,8 +171,8 @@ class IncidentDBRepository(IIncidentRepository):
async def get_active(self) -> list[Incident]:
"""取得所有活躍的 Incident"""
active_statuses = [
IncidentStatus.INVESTIGATING.value,
IncidentStatus.MITIGATING.value,
IncidentStatus.INVESTIGATING,
IncidentStatus.MITIGATING,
]
async with get_db_context() as db:
result = await db.execute(

View File

@@ -190,7 +190,7 @@ class KnowledgeDBRepository:
count_query = count_query.where(KnowledgeEntryRecord.status == status)
if tags:
for tag in tags:
tag_filter = KnowledgeEntryRecord.tags.op('@>')(f'["{tag}"]')
tag_filter = _json_string_array_has_tag(tag)
query = query.where(tag_filter)
count_query = count_query.where(tag_filter)
if q:
@@ -347,3 +347,18 @@ class KnowledgeDBRepository:
created_at=record.created_at,
updated_at=record.updated_at,
)
def _json_string_array_has_tag(tag: str):
"""建立 JSON/JSONB 皆相容的 tag filter。
production 的 knowledge_entries.tags 目前是 JSON 欄位,不支援 json @> text。
這裡改用帶引號的字串比對,避免把 tag 片段誤判成完整 tag。
"""
escaped = (
tag
.replace("\\", "\\\\")
.replace("%", "\\%")
.replace("_", "\\_")
)
return KnowledgeEntryRecord.tags.cast(String).ilike(f'%"{escaped}"%', escape="\\")

View File

@@ -19,10 +19,11 @@ router = APIRouter()
logger = logging.getLogger(__name__)
# ==================== Ollama Config ====================
# 2026-05-03 ogt: ADR-110 GCP-A Primary — 改從 settings 讀取,不再硬編碼 111
def _get_ollama_base_url() -> str:
from src.core.config import get_settings
return get_settings().OLLAMA_URL
# 2026-05-19 Codex: agent thinking stream follows GCP-A → GCP-B → 111.
def _get_ollama_endpoints():
from src.services.ollama_endpoint_resolver import resolve_ollama_order
return resolve_ollama_order("interactive")
OLLAMA_MODEL = "llama3.2:latest" # 可根據實際部署調整
OLLAMA_TIMEOUT = 120.0 # 串流超時
@@ -112,66 +113,82 @@ async def get_agent_thinking(
# 1. 開始思考
yield f"data: {json.dumps({'type': 'thinking', 'content': '正在連接 AI 模型...'}, ensure_ascii=False)}\n\n"
try:
async with httpx.AsyncClient(timeout=OLLAMA_TIMEOUT) as client:
# 2. 發送請求到 Ollama
yield f"data: {json.dumps({'type': 'thinking', 'content': f'模型: {model}'}, ensure_ascii=False)}\n\n"
last_error = ""
async with httpx.AsyncClient(timeout=OLLAMA_TIMEOUT) as client:
# 2. 發送請求到 Ollama
yield f"data: {json.dumps({'type': 'thinking', 'content': f'模型: {model}'}, ensure_ascii=False)}\n\n"
async with client.stream(
"POST",
f"{_get_ollama_base_url()}/api/generate",
json={
"model": model,
"prompt": prompt,
"stream": True,
},
) as response:
if response.status_code != 200:
yield f"data: {json.dumps({'type': 'error', 'content': f'Ollama 錯誤: HTTP {response.status_code}'}, ensure_ascii=False)}\n\n"
yield "data: [DONE]\n\n"
return
yield f"data: {json.dumps({'type': 'thinking', 'content': '開始接收 AI 回應...'}, ensure_ascii=False)}\n\n"
# 3. 串流讀取 Ollama 回應
buffer = ""
async for line in response.aiter_lines():
if not line:
for endpoint in _get_ollama_endpoints():
if not endpoint.url:
continue
try:
async with client.stream(
"POST",
f"{endpoint.url}/api/generate",
json={
"model": model,
"prompt": prompt,
"stream": True,
},
) as response:
if response.status_code != 200:
last_error = f"HTTP {response.status_code}"
logger.warning(
"agent_thinking_ollama_http_error",
provider=endpoint.provider_name,
status=response.status_code,
)
continue
try:
chunk = json.loads(line)
token = chunk.get("response", "")
done = chunk.get("done", False)
yield f"data: {json.dumps({'type': 'thinking', 'content': '開始接收 AI 回應...'}, ensure_ascii=False)}\n\n"
if token:
# 累積 token每 10 字符或遇到標點符號時發送
buffer += token
if len(buffer) >= 10 or any(p in buffer for p in "。!?,、\n"):
yield f"data: {json.dumps({'type': 'thinking', 'content': buffer}, ensure_ascii=False)}\n\n"
buffer = ""
# 3. 串流讀取 Ollama 回應
buffer = ""
async for line in response.aiter_lines():
if not line:
continue
if done:
# 發送剩餘 buffer
if buffer:
yield f"data: {json.dumps({'type': 'thinking', 'content': buffer}, ensure_ascii=False)}\n\n"
# 發送完成訊息
yield f"data: {json.dumps({'type': 'result', 'content': '分析完成'}, ensure_ascii=False)}\n\n"
break
try:
chunk = json.loads(line)
token = chunk.get("response", "")
done = chunk.get("done", False)
except json.JSONDecodeError as e:
logger.warning(f"JSON 解析失敗: {line[:100]}... - {e}")
continue
if token:
# 累積 token每 10 字符或遇到標點符號時發送
buffer += token
if len(buffer) >= 10 or any(p in buffer for p in "。!?,、\n"):
yield f"data: {json.dumps({'type': 'thinking', 'content': buffer}, ensure_ascii=False)}\n\n"
buffer = ""
except httpx.ConnectError as e:
logger.error(f"無法連接 Ollama: {e}")
yield f"data: {json.dumps({'type': 'error', 'content': f'無法連接 Ollama ({_get_ollama_base_url()})'}, ensure_ascii=False)}\n\n"
except httpx.TimeoutException as e:
logger.error(f"Ollama 超時: {e}")
yield f"data: {json.dumps({'type': 'error', 'content': '請求超時'}, ensure_ascii=False)}\n\n"
except Exception as e:
logger.error(f"未知錯誤: {e}")
yield f"data: {json.dumps({'type': 'error', 'content': f'未知錯誤: {str(e)}'}, ensure_ascii=False)}\n\n"
if done:
# 發送剩餘 buffer
if buffer:
yield f"data: {json.dumps({'type': 'thinking', 'content': buffer}, ensure_ascii=False)}\n\n"
# 發送完成訊息
yield f"data: {json.dumps({'type': 'result', 'content': '分析完成'}, ensure_ascii=False)}\n\n"
yield "data: [DONE]\n\n"
return
except json.JSONDecodeError as e:
logger.warning(f"JSON 解析失敗: {line[:100]}... - {e}")
continue
except (httpx.ConnectError, httpx.TimeoutException) as e:
last_error = type(e).__name__
logger.error(
"agent_thinking_ollama_endpoint_failed",
provider=endpoint.provider_name,
error=str(e),
)
except Exception as e:
last_error = str(e)
logger.error(
"agent_thinking_unknown_error",
provider=endpoint.provider_name,
error=str(e),
)
error_content = f"Ollama 全端點不可用: {last_error or 'unknown'}"
yield f"data: {json.dumps({'type': 'error', 'content': error_content}, ensure_ascii=False)}\n\n"
# 4. 結束標記
yield "data: [DONE]\n\n"

View File

@@ -0,0 +1,606 @@
"""
ADR-100 Remediation Service
===========================
Safe operator entrypoints for verification remediation work items.
T25: remediation queue items are now actionable without mutating incident state:
- preview: show the selected guardrail path
- dry-run: collect read-only current state and validate supported executor routing
"""
from __future__ import annotations
import asyncio
from typing import Any, Literal, Protocol
import structlog
from src.models.incident import Incident
from src.repositories.incident_repository import IncidentDBRepository
from src.services.adr100_slo_status_service import (
Adr100SloStatusService,
get_adr100_slo_status_service,
)
from src.services.auto_repair_service import AutoRepairService
from src.services.post_execution_verifier import (
PostExecutionVerifier,
_assess_recovery,
_build_prometheus_query,
get_post_execution_verifier,
)
logger = structlog.get_logger(__name__)
RemediationMode = Literal["auto", "reverify", "replay"]
_READY_STATUSES = {"ready_for_replay", "ready_for_reverify"}
class RemediationNotFoundError(LookupError):
"""Requested ADR-100 remediation work item is not in the current read model."""
class _IncidentRepository(Protocol):
async def get_by_id(self, incident_id: str) -> Incident | None:
...
class Adr100RemediationService:
"""Read-only remediation preview and dry-run service."""
def __init__(
self,
*,
slo_service: Adr100SloStatusService | None = None,
incident_repository: _IncidentRepository | None = None,
auto_repair_service: AutoRepairService | None = None,
verifier: PostExecutionVerifier | None = None,
timeline_service: Any | None = None,
alert_operation_log_repository: Any | None = None,
record_history: bool = True,
) -> None:
self._slo_service = slo_service or get_adr100_slo_status_service()
self._incident_repository = incident_repository or IncidentDBRepository()
self._auto_repair_service = auto_repair_service or AutoRepairService()
self._verifier = verifier or get_post_execution_verifier()
self._timeline_service = timeline_service
self._alert_operation_log_repository = alert_operation_log_repository
self._record_history_enabled = record_history
async def preview(self, work_item_id: str, mode: RemediationMode = "auto") -> dict[str, Any]:
"""Return the safe execution plan for a remediation queue item."""
item = await self._find_work_item(work_item_id)
selected_mode = _select_mode(item, mode)
checks = _base_checks(item)
allowed = all(check["passed"] for check in checks)
return {
"schema_version": "adr100_remediation_preview_v1",
"work_item_id": item.get("work_item_id"),
"incident_id": item.get("incident_id"),
"auto_repair_id": item.get("auto_repair_id"),
"mode": selected_mode,
"allowed": allowed,
"safety_level": "read_only",
"writes_incident_state": False,
"writes_auto_repair_result": False,
"checks": checks,
"plan": _plan_for_item(item, selected_mode),
"source": "adr100.verification_coverage.remediation_queue",
}
async def dry_run(self, work_item_id: str, mode: RemediationMode = "auto") -> dict[str, Any]:
"""Run a safe, read-only remediation dry-run for one queue item."""
item = await self._find_work_item(work_item_id)
selected_mode = _select_mode(item, mode)
checks = _base_checks(item)
incident = await self._load_incident(item)
checks.append({
"name": "incident_loaded",
"passed": incident is not None,
"detail": item.get("incident_id") or "missing incident_id",
})
if incident is None or not all(check["passed"] for check in checks):
payload = _dry_run_blocked_payload(item, selected_mode, checks)
payload["history"] = await self._record_dry_run_history(item, payload)
return payload
if selected_mode == "replay":
return await self._dry_run_replay(item, incident, checks)
return await self._dry_run_reverify(item, incident, checks)
async def history(
self,
*,
limit: int = 50,
incident_id: str | None = None,
work_item_id: str | None = None,
) -> dict[str, Any]:
"""Return durable dry-run history written by this remediation service."""
safe_limit = max(1, min(limit, 200))
fetch_limit = min(max(safe_limit * 4, 50), 200)
rows: list[Any] = []
repo = self._alert_operation_log_repository
if repo is None:
from src.repositories.alert_operation_log_repository import (
get_alert_operation_log_repository,
)
repo = get_alert_operation_log_repository()
for event_type in ("PRE_FLIGHT_PASSED", "PRE_FLIGHT_FAILED"):
try:
batch, _total = await repo.list_recent(
limit=fetch_limit,
event_type=event_type,
incident_id=incident_id,
)
rows.extend(batch)
except Exception as exc:
logger.warning(
"adr100_remediation_history_fetch_failed",
event_type=event_type,
incident_id=incident_id,
error=str(exc),
)
rows.sort(key=_record_created_at, reverse=True)
items: list[dict[str, Any]] = []
for row in rows:
context = getattr(row, "context", None) or {}
if context.get("schema_version") != "adr100_remediation_dry_run_history_v1":
continue
if work_item_id and context.get("work_item_id") != work_item_id:
continue
items.append(_history_item(row, context))
if len(items) >= safe_limit:
break
return {
"schema_version": "adr100_remediation_history_v1",
"total": len(items),
"limit": safe_limit,
"filters": {
"incident_id": incident_id,
"work_item_id": work_item_id,
},
"items": items,
"by_work_item": _summarize_history_by_work_item(items),
}
async def _find_work_item(self, work_item_id: str) -> dict[str, Any]:
report = await self._slo_service.fetch_report()
coverage = report.get("verification_coverage") or {}
queue = coverage.get("remediation_queue") or {}
for item in queue.get("items") or []:
if item.get("work_item_id") == work_item_id:
return dict(item)
raise RemediationNotFoundError(work_item_id)
async def _load_incident(self, item: dict[str, Any]) -> Incident | None:
incident_id = str(item.get("incident_id") or "")
if not incident_id:
return None
return await self._incident_repository.get_by_id(incident_id)
async def _dry_run_reverify(
self,
item: dict[str, Any],
incident: Incident,
checks: list[dict[str, Any]],
) -> dict[str, Any]:
post_state = await self._collect_current_state(incident)
action_taken = f"dry_run_reverify:{item.get('playbook_id') or 'unknown'}"
result = _assess_recovery(None, post_state, action_taken)
payload = _dry_run_result_payload(
item=item,
mode="reverify",
checks=checks,
post_state=post_state,
verification_result_preview=result,
extra={
"promql": _promql_for_incident(incident),
"mcp_route": {
"agent_id": "post_execution_verifier",
"required_scope": "read",
"is_shadow": True,
"flywheel_node": "verify",
},
},
)
payload["history"] = await self._record_dry_run_history(item, payload)
return payload
async def _dry_run_replay(
self,
item: dict[str, Any],
incident: Incident,
checks: list[dict[str, Any]],
) -> dict[str, Any]:
diagnostic_command = _diagnostic_command_for_incident(incident)
route = self._auto_repair_service.preview_read_only_ssh_mcp_route(
incident,
diagnostic_command,
)
checks.append({
"name": "supported_executor_route",
"passed": route is not None,
"detail": "mcp:ssh_diagnose" if route else "missing host/container route",
})
post_state = await self._collect_current_state(incident)
action_taken = f"dry_run_replay:{item.get('playbook_id') or 'unknown'}"
result = _assess_recovery(None, post_state, action_taken)
payload = _dry_run_result_payload(
item=item,
mode="replay",
checks=checks,
post_state=post_state,
verification_result_preview=result,
extra={
"diagnostic_command_preview": diagnostic_command,
"mcp_route": route,
"promql": _promql_for_incident(incident),
},
)
payload["history"] = await self._record_dry_run_history(item, payload)
return payload
async def _collect_current_state(self, incident: Incident) -> dict[str, Any]:
try:
return await asyncio.wait_for(
self._verifier._collect_post_state(incident),
timeout=12.0,
)
except asyncio.TimeoutError:
logger.warning(
"adr100_remediation_dry_run_timeout",
incident_id=incident.incident_id,
)
return {}
except Exception as exc:
logger.warning(
"adr100_remediation_dry_run_collect_failed",
incident_id=incident.incident_id,
error=str(exc),
)
return {}
async def _record_dry_run_history(
self,
item: dict[str, Any],
payload: dict[str, Any],
) -> dict[str, Any]:
if not self._record_history_enabled:
return {"recorded": False, "reason": "disabled"}
incident_id = str(item.get("incident_id") or "")
if not incident_id:
return {"recorded": False, "reason": "missing_incident_id"}
history: dict[str, Any] = {
"recorded": False,
"alert_operation_id": None,
"timeline_event_id": None,
}
context = _history_context(item, payload)
allowed = bool(payload.get("allowed"))
try:
repo = self._alert_operation_log_repository
if repo is None:
from src.repositories.alert_operation_log_repository import (
get_alert_operation_log_repository,
)
repo = get_alert_operation_log_repository()
record = await repo.append(
"PRE_FLIGHT_PASSED" if allowed else "PRE_FLIGHT_FAILED",
incident_id=incident_id,
auto_repair_id=str(item.get("auto_repair_id") or "") or None,
actor="adr100_remediation_service",
action_detail=f"adr100_remediation_dry_run:{payload.get('mode')}"[:200],
success=allowed,
context=context,
)
if record is not None:
history["alert_operation_id"] = getattr(record, "id", None)
except Exception as exc:
logger.warning(
"adr100_remediation_alert_operation_history_failed",
incident_id=incident_id,
error=str(exc),
)
try:
timeline = self._timeline_service
if timeline is None:
from src.services.approval_db import get_timeline_service
timeline = get_timeline_service()
event = await timeline.add_event(
event_type="verifier",
status=_timeline_status(payload),
title="ADR-100 remediation dry-run",
description=_history_description(context),
actor="adr100_remediation_service",
actor_role=str(payload.get("mode") or "dry_run"),
incident_id=incident_id,
)
if event:
history["timeline_event_id"] = event.get("id")
except Exception as exc:
logger.warning(
"adr100_remediation_timeline_history_failed",
incident_id=incident_id,
error=str(exc),
)
history["recorded"] = bool(
history.get("alert_operation_id") or history.get("timeline_event_id")
)
return history
def _select_mode(item: dict[str, Any], requested: RemediationMode) -> Literal["reverify", "replay"]:
if requested in ("reverify", "replay"):
return requested
if item.get("remediation_status") == "ready_for_reverify":
return "reverify"
if item.get("remediation_action") == "reverify_with_promql_template":
return "reverify"
return "replay"
def _base_checks(item: dict[str, Any]) -> list[dict[str, Any]]:
status = str(item.get("remediation_status") or "unknown")
action = str(item.get("remediation_action") or "unknown")
return [
{
"name": "queue_item_ready",
"passed": status in _READY_STATUSES,
"detail": status,
},
{
"name": "read_only_guardrail",
"passed": action in {
"replay_with_supported_executor",
"reverify_with_promql_template",
},
"detail": action,
},
{
"name": "no_state_mutation",
"passed": True,
"detail": "dry_run_does_not_update_incident_or_auto_repair_rows",
},
]
def _plan_for_item(item: dict[str, Any], mode: str) -> dict[str, Any]:
if mode == "reverify":
return {
"step": "collect_current_state_and_assess",
"agent_id": "post_execution_verifier",
"required_scope": "read",
"writes": [],
}
return {
"step": "validate_supported_executor_route_then_collect_current_state",
"agent_id": "auto_repair_executor",
"required_scope": "read",
"writes": [],
"target_action": item.get("remediation_action"),
}
def _dry_run_blocked_payload(
item: dict[str, Any],
mode: str,
checks: list[dict[str, Any]],
) -> dict[str, Any]:
return {
"schema_version": "adr100_remediation_dry_run_v1",
"work_item_id": item.get("work_item_id"),
"incident_id": item.get("incident_id"),
"auto_repair_id": item.get("auto_repair_id"),
"mode": mode,
"allowed": False,
"executed": False,
"safety_level": "read_only",
"writes_incident_state": False,
"writes_auto_repair_result": False,
"checks": checks,
"verification_result_preview": "blocked",
"post_state_summary": {},
}
def _dry_run_result_payload(
*,
item: dict[str, Any],
mode: str,
checks: list[dict[str, Any]],
post_state: dict[str, Any],
verification_result_preview: str,
extra: dict[str, Any],
) -> dict[str, Any]:
return {
"schema_version": "adr100_remediation_dry_run_v1",
"work_item_id": item.get("work_item_id"),
"incident_id": item.get("incident_id"),
"auto_repair_id": item.get("auto_repair_id"),
"mode": mode,
"allowed": all(check["passed"] for check in checks),
"executed": True,
"safety_level": "read_only",
"writes_incident_state": False,
"writes_auto_repair_result": False,
"checks": checks,
"verification_result_preview": verification_result_preview,
"post_state_summary": _summarize_post_state(post_state),
**extra,
}
def _summarize_post_state(post_state: dict[str, Any]) -> dict[str, Any]:
keys = sorted(post_state.keys())
return {
"tool_count": len(keys),
"tools": keys[:8],
"has_state": bool(post_state),
}
def _history_context(item: dict[str, Any], payload: dict[str, Any]) -> dict[str, Any]:
return {
"schema_version": "adr100_remediation_dry_run_history_v1",
"work_item_id": item.get("work_item_id"),
"auto_repair_id": item.get("auto_repair_id"),
"playbook_id": item.get("playbook_id"),
"alertname": item.get("alertname"),
"mode": payload.get("mode"),
"allowed": payload.get("allowed"),
"executed": payload.get("executed"),
"safety_level": payload.get("safety_level"),
"writes_incident_state": payload.get("writes_incident_state"),
"writes_auto_repair_result": payload.get("writes_auto_repair_result"),
"verification_result_preview": payload.get("verification_result_preview"),
"post_state_summary": payload.get("post_state_summary"),
"mcp_route": payload.get("mcp_route"),
"checks": payload.get("checks"),
}
def _timeline_status(payload: dict[str, Any]) -> str:
if not payload.get("allowed"):
return "warning"
if payload.get("verification_result_preview") == "success":
return "success"
return "warning"
def _history_description(context: dict[str, Any]) -> str:
tool_count = (context.get("post_state_summary") or {}).get("tool_count", 0)
route = context.get("mcp_route") or {}
agent = route.get("agent_id") or "unknown_agent"
tool = route.get("tool_name") or "current_state"
return (
f"mode={context.get('mode')} "
f"preview={context.get('verification_result_preview')} "
f"tools={tool_count} route={agent}/{tool} "
f"writes_incident={context.get('writes_incident_state')} "
f"writes_auto_repair={context.get('writes_auto_repair_result')}"
)[:500]
def _record_created_at(record: Any) -> str:
value = getattr(record, "created_at", None)
if hasattr(value, "isoformat"):
return value.isoformat()
return str(value or "")
def _history_item(record: Any, context: dict[str, Any]) -> dict[str, Any]:
route = context.get("mcp_route") or {}
post_state = context.get("post_state_summary") or {}
return {
"id": str(getattr(record, "id", "")),
"incident_id": getattr(record, "incident_id", None),
"auto_repair_id": getattr(record, "auto_repair_id", None)
or context.get("auto_repair_id"),
"event_type": str(getattr(record, "event_type", "")),
"actor": getattr(record, "actor", None),
"success": getattr(record, "success", None),
"created_at": _record_created_at(record),
"work_item_id": context.get("work_item_id"),
"playbook_id": context.get("playbook_id"),
"alertname": context.get("alertname"),
"mode": context.get("mode"),
"allowed": context.get("allowed"),
"executed": context.get("executed"),
"safety_level": context.get("safety_level"),
"verification_result_preview": context.get("verification_result_preview"),
"tool_count": post_state.get("tool_count", 0),
"tools": post_state.get("tools") or [],
"agent_id": route.get("agent_id"),
"tool_name": route.get("tool_name") or "current_state",
"required_scope": route.get("required_scope"),
"writes_incident_state": context.get("writes_incident_state"),
"writes_auto_repair_result": context.get("writes_auto_repair_result"),
"checks": context.get("checks") or [],
}
def _summarize_history_by_work_item(items: list[dict[str, Any]]) -> list[dict[str, Any]]:
summary: dict[str, dict[str, Any]] = {}
for item in items:
key = str(item.get("work_item_id") or item.get("incident_id") or item.get("id"))
if key not in summary:
summary[key] = {
"work_item_id": item.get("work_item_id"),
"incident_id": item.get("incident_id"),
"count": 0,
"latest_at": item.get("created_at"),
"latest_event_type": item.get("event_type"),
"latest_success": item.get("success"),
"latest_preview": item.get("verification_result_preview"),
"latest_mode": item.get("mode"),
"latest_agent_id": item.get("agent_id"),
"latest_tool_name": item.get("tool_name"),
"required_scope": item.get("required_scope"),
}
summary[key]["count"] += 1
return list(summary.values())
def _diagnostic_command_for_incident(incident: Incident) -> str:
labels = _labels_for_incident(incident)
host = str(labels.get("host") or labels.get("instance") or "{host}")
container = str(labels.get("container_name") or labels.get("container") or "")
if container:
return f"ssh {host} 'uptime; docker stats --no-stream {container}'"
return f"ssh {host} 'uptime; docker stats --no-stream'"
def _promql_for_incident(incident: Incident) -> str:
labels = _labels_for_incident(incident)
alertname = ""
if incident.signals:
signal = incident.signals[0]
alertname = labels.get("alertname") or getattr(signal, "alert_name", "")
return _build_prometheus_query(alertname, labels)
def _labels_for_incident(incident: Incident) -> dict[str, Any]:
if incident.signals:
return incident.signals[0].labels or {}
return {}
_service: Adr100RemediationService | None = None
def get_adr100_remediation_service() -> Adr100RemediationService:
"""Return singleton ADR-100 remediation service."""
global _service
if _service is None:
_service = Adr100RemediationService()
return _service
def set_adr100_remediation_service(service: Adr100RemediationService | None) -> None:
"""Inject ADR-100 remediation service for tests."""
global _service
_service = service

View File

@@ -0,0 +1,354 @@
"""
ADR-100 SLO metrics emitter.
Prometheus recording rules for the AI flywheel SLOs expect a small set of
counter-like metrics. The source of truth already lives in PostgreSQL, so this
read-side emitter exposes DB totals on /metrics without changing runtime write
paths or introducing another state store.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from time import time
from sqlalchemy import text
from src.db.base import get_db_context
@dataclass(frozen=True)
class AutomationOperationSample:
outcome: str
operation_type: str
count: int
@dataclass(frozen=True)
class VerificationSample:
outcome: str
count: int
@dataclass(frozen=True)
class Adr100SloMetricsSnapshot:
automation_operations: list[AutomationOperationSample] = field(default_factory=list)
automation_operations_24h: list[AutomationOperationSample] = field(default_factory=list)
post_execution_verifications: list[VerificationSample] = field(default_factory=list)
post_execution_verifications_24h: list[VerificationSample] = field(default_factory=list)
knowledge_entries_total: int = 0
knowledge_entries_created_24h: int = 0
high_confidence_total: int = 0
high_confidence_success_total: int = 0
emitted_at: float = field(default_factory=time)
class Adr100SloMetricsService:
"""Build ADR-100 Prometheus samples from production DB state."""
async def to_prometheus_lines(self) -> str:
snapshot = await self.fetch_snapshot()
return render_adr100_slo_metrics(snapshot)
async def fetch_snapshot(self) -> Adr100SloMetricsSnapshot:
async with get_db_context() as db:
automation_rows = (
await db.execute(text(_AUTOMATION_OPERATION_SQL))
).fetchall()
automation_24h_rows = (
await db.execute(text(_AUTOMATION_OPERATION_24H_SQL))
).fetchall()
verification_rows = (
await db.execute(text(_POST_EXECUTION_VERIFICATION_SQL))
).fetchall()
verification_24h_rows = (
await db.execute(text(_POST_EXECUTION_VERIFICATION_24H_SQL))
).fetchall()
knowledge_total = int(
(await db.execute(text("SELECT count(*) FROM knowledge_entries"))).scalar()
or 0
)
knowledge_created_24h = int(
(
await db.execute(
text(
"""
SELECT count(*)
FROM knowledge_entries
WHERE created_at >= NOW() - INTERVAL '24 hours'
"""
)
)
).scalar()
or 0
)
confidence_row = (
await db.execute(text(_HIGH_CONFIDENCE_APPROVAL_SQL))
).one()
return Adr100SloMetricsSnapshot(
automation_operations=[
AutomationOperationSample(
outcome=str(row.outcome),
operation_type=str(row.operation_type),
count=int(row.count or 0),
)
for row in automation_rows
],
automation_operations_24h=[
AutomationOperationSample(
outcome=str(row.outcome),
operation_type=str(row.operation_type),
count=int(row.count or 0),
)
for row in automation_24h_rows
],
post_execution_verifications=[
VerificationSample(
outcome=str(row.outcome),
count=int(row.count or 0),
)
for row in verification_rows
],
post_execution_verifications_24h=[
VerificationSample(
outcome=str(row.outcome),
count=int(row.count or 0),
)
for row in verification_24h_rows
],
knowledge_entries_total=knowledge_total,
knowledge_entries_created_24h=knowledge_created_24h,
high_confidence_total=int(confidence_row.high_confidence_total or 0),
high_confidence_success_total=int(
confidence_row.high_confidence_success_total or 0
),
)
def render_adr100_slo_metrics(snapshot: Adr100SloMetricsSnapshot) -> str:
"""Render ADR-100 SLO metrics in Prometheus text exposition format."""
lines: list[str] = [
"",
"# HELP automation_operation_log_total DB-derived AI automation operation count for ADR-100 SLOs",
"# TYPE automation_operation_log_total counter",
]
if snapshot.automation_operations:
for sample in snapshot.automation_operations:
lines.append(
"automation_operation_log_total"
f'{{outcome="{_escape_label(sample.outcome)}",'
f'operation_type="{_escape_label(sample.operation_type)}"}} '
f"{sample.count}"
)
else:
lines.append(
'automation_operation_log_total{outcome="none",operation_type="none"} 0'
)
lines.extend([
"# HELP automation_operation_created_24h DB-derived AI automation operation count created in the last 24 hours for ADR-100 SLO dashboards",
"# TYPE automation_operation_created_24h gauge",
])
if snapshot.automation_operations_24h:
for sample in snapshot.automation_operations_24h:
lines.append(
"automation_operation_created_24h"
f'{{outcome="{_escape_label(sample.outcome)}",'
f'operation_type="{_escape_label(sample.operation_type)}"}} '
f"{sample.count}"
)
else:
lines.append(
'automation_operation_created_24h{outcome="none",operation_type="none"} 0'
)
lines.extend([
"# HELP post_execution_verification_total DB-derived post execution verification result count for ADR-100 SLOs",
"# TYPE post_execution_verification_total counter",
])
if snapshot.post_execution_verifications:
for sample in snapshot.post_execution_verifications:
lines.append(
"post_execution_verification_total"
f'{{outcome="{_escape_label(sample.outcome)}"}} {sample.count}'
)
else:
lines.append('post_execution_verification_total{outcome="none"} 0')
lines.extend([
"# HELP post_execution_verification_created_24h DB-derived post execution verification result count created in the last 24 hours for ADR-100 SLO dashboards",
"# TYPE post_execution_verification_created_24h gauge",
])
if snapshot.post_execution_verifications_24h:
for sample in snapshot.post_execution_verifications_24h:
lines.append(
"post_execution_verification_created_24h"
f'{{outcome="{_escape_label(sample.outcome)}"}} {sample.count}'
)
else:
lines.append('post_execution_verification_created_24h{outcome="none"} 0')
lines.extend([
"# HELP knowledge_entries_total DB-derived knowledge entry count for ADR-100 SLOs",
"# TYPE knowledge_entries_total counter",
f"knowledge_entries_total {snapshot.knowledge_entries_total}",
"# HELP knowledge_entries_created_24h DB-derived knowledge entries created in the last 24 hours for ADR-100 SLOs",
"# TYPE knowledge_entries_created_24h gauge",
f"knowledge_entries_created_24h {snapshot.knowledge_entries_created_24h}",
"# HELP approval_records_high_confidence_total DB-derived high confidence approval decisions for ADR-100 SLOs",
"# TYPE approval_records_high_confidence_total counter",
f"approval_records_high_confidence_total {snapshot.high_confidence_total}",
"# HELP approval_records_high_confidence_success_total DB-derived high confidence approval decisions with successful verification for ADR-100 SLOs",
"# TYPE approval_records_high_confidence_success_total counter",
(
"approval_records_high_confidence_success_total "
f"{snapshot.high_confidence_success_total}"
),
"# HELP adr100_slo_emitter_last_success_timestamp Last successful ADR-100 DB metrics emission timestamp",
"# TYPE adr100_slo_emitter_last_success_timestamp gauge",
f"adr100_slo_emitter_last_success_timestamp {snapshot.emitted_at:.0f}",
"",
])
return "\n".join(lines)
def _escape_label(value: str) -> str:
return value.replace("\\", "\\\\").replace("\n", "\\n").replace('"', '\\"')
_AUTOMATION_OPERATION_SQL = """
WITH automation_scope AS (
SELECT
CASE
WHEN status <> 'success' THEN status
WHEN actor = 'approval_execution'
AND COALESCE(input->>'requested_by', '') NOT ILIKE 'auto%%'
THEN 'human_required'
ELSE 'auto_executed'
END AS outcome,
operation_type
FROM automation_operation_log
WHERE operation_type IN (
'playbook_executed',
'remediation_executed',
'remediation_verified',
'remediation_rolled_back',
'self_correction_attempted'
)
UNION ALL
SELECT
CASE WHEN success THEN 'auto_executed' ELSE 'failed' END AS outcome,
'auto_repair_executed' AS operation_type
FROM auto_repair_executions
)
SELECT
outcome,
operation_type,
count(*) AS count
FROM automation_scope
GROUP BY outcome, operation_type
ORDER BY outcome, operation_type
"""
_AUTOMATION_OPERATION_24H_SQL = """
WITH automation_scope AS (
SELECT
CASE
WHEN status <> 'success' THEN status
WHEN actor = 'approval_execution'
AND COALESCE(input->>'requested_by', '') NOT ILIKE 'auto%%'
THEN 'human_required'
ELSE 'auto_executed'
END AS outcome,
operation_type
FROM automation_operation_log
WHERE operation_type IN (
'playbook_executed',
'remediation_executed',
'remediation_verified',
'remediation_rolled_back',
'self_correction_attempted'
)
AND created_at >= NOW() - INTERVAL '24 hours'
UNION ALL
SELECT
CASE WHEN success THEN 'auto_executed' ELSE 'failed' END AS outcome,
'auto_repair_executed' AS operation_type
FROM auto_repair_executions
WHERE created_at >= NOW() - INTERVAL '24 hours'
)
SELECT
outcome,
operation_type,
count(*) AS count
FROM automation_scope
GROUP BY outcome, operation_type
ORDER BY outcome, operation_type
"""
_POST_EXECUTION_VERIFICATION_SQL = """
SELECT verification_result AS outcome, count(*) AS count
FROM incident_evidence
WHERE verification_result IS NOT NULL
GROUP BY verification_result
ORDER BY verification_result
"""
_POST_EXECUTION_VERIFICATION_24H_SQL = """
SELECT verification_result AS outcome, count(*) AS count
FROM incident_evidence
WHERE verification_result IS NOT NULL
AND collected_at >= NOW() - INTERVAL '24 hours'
GROUP BY verification_result
ORDER BY verification_result
"""
_HIGH_CONFIDENCE_APPROVAL_SQL = """
WITH approval_confidence AS (
SELECT
id,
incident_id,
COALESCE(
CASE
WHEN extra_metadata->>'confidence_score' ~ '^[0-9]+(\\.[0-9]+)?$'
THEN (extra_metadata->>'confidence_score')::numeric
ELSE NULL
END,
CASE
WHEN extra_metadata->>'confidence' ~ '^[0-9]+(\\.[0-9]+)?$'
THEN (extra_metadata->>'confidence')::numeric
ELSE NULL
END,
composite_score,
0
) AS confidence
FROM approval_records
)
SELECT
count(*) FILTER (WHERE confidence >= 0.8) AS high_confidence_total,
count(*) FILTER (
WHERE confidence >= 0.8
AND EXISTS (
SELECT 1
FROM incident_evidence ev
WHERE ev.incident_id = approval_confidence.incident_id
AND ev.verification_result = 'success'
)
) AS high_confidence_success_total
FROM approval_confidence
"""
_adr100_slo_metrics_service: Adr100SloMetricsService | None = None
def get_adr100_slo_metrics_service() -> Adr100SloMetricsService:
global _adr100_slo_metrics_service
if _adr100_slo_metrics_service is None:
_adr100_slo_metrics_service = Adr100SloMetricsService()
return _adr100_slo_metrics_service

View File

@@ -0,0 +1,743 @@
"""
Read-only ADR-100 SLO status snapshot.
GovernanceAgent.check_slo_compliance() can emit governance alerts when an SLO is
violated. This service is intentionally read-only so dashboards can show the
same Prometheus-backed state without producing Telegram/DB side effects.
"""
from __future__ import annotations
import math
from dataclasses import dataclass
from typing import Any
import httpx
import structlog
from sqlalchemy import text
from src.core.config import settings
from src.db.base import get_db_context
from src.utils.timezone import now_taipei_iso
logger = structlog.get_logger(__name__)
@dataclass(frozen=True)
class Adr100SloDefinition:
name: str
query: str
target: float
hard_red_line: float
direction: str
unit: str
window: str
denominator_query: str | None = None
denominator_window_seconds: int = 0
minimum_events: float = 1.0
ADR100_SLO_DEFINITIONS: tuple[Adr100SloDefinition, ...] = (
Adr100SloDefinition(
name="autonomy_rate",
query="sli:autonomy_rate:5m",
target=0.80,
hard_red_line=0.70,
direction="above",
unit="percent",
window="5m",
denominator_query="sum(rate(automation_operation_log_total[5m]))",
denominator_window_seconds=300,
),
Adr100SloDefinition(
name="decision_accuracy",
query="sli:decision_accuracy:5m",
target=0.90,
hard_red_line=0.85,
direction="above",
unit="percent",
window="5m",
denominator_query='sum(rate(automation_operation_log_total{outcome="auto_executed"}[5m]))',
denominator_window_seconds=300,
),
Adr100SloDefinition(
name="confidence_calibration",
query="sli:confidence_calibration:1h",
target=0.80,
hard_red_line=0.70,
direction="above",
unit="percent",
window="1h",
denominator_query="sum(rate(approval_records_high_confidence_total[1h]))",
denominator_window_seconds=3600,
),
Adr100SloDefinition(
name="km_growth_rate",
query="max(knowledge_entries_created_24h) or max(sli:km_growth_rate:24h)",
target=20.0,
hard_red_line=5.0,
direction="above",
unit="count",
window="24h",
),
)
class Adr100SloStatusService:
"""Fetch ADR-100 SLO status from Prometheus without writing governance events."""
async def fetch_report(self) -> dict[str, Any]:
prom_url = getattr(
settings,
"PROMETHEUS_URL",
"http://prometheus.observability.svc:9090",
).rstrip("/")
metrics: list[dict[str, Any]] = []
async with httpx.AsyncClient(timeout=5.0) as client:
for definition in ADR100_SLO_DEFINITIONS:
metrics.append(await self._fetch_metric(client, prom_url, definition))
evaluable = [metric for metric in metrics if metric.get("evaluable")]
ok_count = sum(1 for metric in evaluable if metric.get("status") == "ok")
overall_compliance = (ok_count / len(evaluable)) if evaluable else None
verification_coverage = await self._fetch_verification_coverage()
overall_status = _overall_status(metrics, evaluable, verification_coverage)
return {
"schema_version": "adr100_slo_status_v1",
"source": "prometheus+postgresql",
"evaluated_at": now_taipei_iso(),
"overall_status": overall_status,
"overall_compliance": overall_compliance,
"evaluable_count": len(evaluable),
"metric_count": len(metrics),
"metrics": metrics,
"verification_coverage": verification_coverage,
}
async def _fetch_metric(
self,
client: httpx.AsyncClient,
prom_url: str,
definition: Adr100SloDefinition,
) -> dict[str, Any]:
denominator_value: float | None = None
sample_count: float | None = None
if definition.denominator_query:
denominator_result = await _query_prometheus_value(
client,
prom_url,
definition.denominator_query,
)
if denominator_result["status"] != "ok":
return _metric_payload(
definition,
value=None,
status="no_data",
reason=denominator_result["reason"],
denominator_value=None,
sample_count=None,
)
denominator_value = float(denominator_result["value"])
sample_count = denominator_value * definition.denominator_window_seconds
if sample_count < definition.minimum_events:
return _metric_payload(
definition,
value=None,
status="skipped_low_volume",
reason="denominator_below_minimum_events",
denominator_value=denominator_value,
sample_count=sample_count,
)
value_result = await _query_prometheus_value(client, prom_url, definition.query)
if value_result["status"] != "ok":
status = (
"skipped_low_volume"
if value_result["reason"] == "prometheus_nan_or_inf"
else "no_data"
)
return _metric_payload(
definition,
value=None,
status=status,
reason=value_result["reason"],
denominator_value=denominator_value,
sample_count=sample_count,
)
value = float(value_result["value"])
status = _classify_status(value, definition)
return _metric_payload(
definition,
value=value,
status=status,
reason=None,
denominator_value=denominator_value,
sample_count=sample_count if sample_count is not None else value,
)
async def _fetch_verification_coverage(self) -> dict[str, Any]:
"""Summarize whether recent auto-repair executions have verifier evidence."""
try:
async with get_db_context() as db:
summary_row = (
await db.execute(text(_VERIFICATION_COVERAGE_SQL))
).mappings().one()
recent_rows = (
await db.execute(text(_VERIFICATION_COVERAGE_RECENT_SQL))
).mappings().all()
recent_non_success_rows = (
await db.execute(text(_VERIFICATION_COVERAGE_NON_SUCCESS_SQL))
).mappings().all()
except Exception as exc:
logger.warning("adr100_verification_coverage_query_error", error=str(exc))
return {
"schema_version": "adr100_verification_coverage_v1",
"source": "postgresql",
"window": "24h",
"status": "error",
"reason": "postgresql_query_error",
"evaluable": False,
"total_auto": 0,
"successful_auto": 0,
"verified_auto": 0,
"verified_success": 0,
"verified_non_success": 0,
"unverified_auto": 0,
"coverage_rate": None,
"verification_success_rate": None,
"last_auto_at": None,
"last_verified_auto_at": None,
"last_verification_evidence_at": None,
"latest_auto_age_seconds": None,
"last_verified_auto_age_seconds": None,
"recent_unverified": [],
"recent_non_success": [],
"non_success_breakdown": {
"by_verification_result": [],
"by_failure_class": [],
},
"remediation_queue": _remediation_queue_payload([]),
}
return _build_verification_coverage_payload(
summary_row,
recent_rows,
recent_non_success_rows,
)
_VERIFICATION_COVERAGE_SQL = """
WITH recent_auto AS (
SELECT id, incident_id, success, created_at
FROM auto_repair_executions
WHERE created_at >= NOW() - INTERVAL '24 hours'
),
per_auto AS (
SELECT
are.id,
are.incident_id,
are.success,
are.created_at,
latest.verification_result,
latest.collected_at AS verification_collected_at,
latest.self_healing_score
FROM recent_auto are
LEFT JOIN LATERAL (
SELECT ev.verification_result, ev.collected_at, ev.self_healing_score
FROM incident_evidence ev
WHERE ev.incident_id = are.incident_id
AND ev.verification_result IS NOT NULL
ORDER BY ev.collected_at DESC
LIMIT 1
) latest ON TRUE
)
SELECT
count(*)::int AS total_auto,
count(*) FILTER (WHERE success)::int AS successful_auto,
count(*) FILTER (WHERE verification_result IS NOT NULL)::int AS verified_auto,
count(*) FILTER (WHERE verification_result = 'success')::int AS verified_success,
count(*) FILTER (WHERE verification_result IN ('degraded','failed','timeout'))::int AS verified_non_success,
count(*) FILTER (WHERE verification_result IS NULL)::int AS unverified_auto,
max(created_at) AS last_auto_at,
max(created_at) FILTER (WHERE verification_result IS NOT NULL) AS last_verified_auto_at,
max(verification_collected_at) AS last_verification_evidence_at,
EXTRACT(EPOCH FROM (NOW() - max(created_at)))::int AS latest_auto_age_seconds,
EXTRACT(EPOCH FROM (NOW() - (max(created_at) FILTER (WHERE verification_result IS NOT NULL))))::int
AS last_verified_auto_age_seconds
FROM per_auto
"""
_VERIFICATION_COVERAGE_RECENT_SQL = """
WITH recent_auto AS (
SELECT id, incident_id, success, created_at
FROM auto_repair_executions
WHERE created_at >= NOW() - INTERVAL '24 hours'
),
per_auto AS (
SELECT
are.id,
are.incident_id,
are.success,
are.created_at,
latest.verification_result
FROM recent_auto are
LEFT JOIN LATERAL (
SELECT ev.verification_result
FROM incident_evidence ev
WHERE ev.incident_id = are.incident_id
AND ev.verification_result IS NOT NULL
ORDER BY ev.collected_at DESC
LIMIT 1
) latest ON TRUE
)
SELECT id, incident_id, success, created_at
FROM per_auto
WHERE verification_result IS NULL
ORDER BY created_at DESC
LIMIT 5
"""
_VERIFICATION_COVERAGE_NON_SUCCESS_SQL = """
WITH recent_auto AS (
SELECT
id,
incident_id,
success,
playbook_id,
playbook_name,
triggered_by,
risk_level,
error_message,
created_at
FROM auto_repair_executions
WHERE created_at >= NOW() - INTERVAL '24 hours'
),
per_auto AS (
SELECT
are.id AS auto_repair_id,
are.incident_id,
are.success AS auto_success,
are.playbook_id,
are.playbook_name,
are.triggered_by,
are.risk_level,
left(coalesce(are.error_message, ''), 240) AS auto_error,
are.created_at AS auto_created_at,
latest.verification_result,
latest.collected_at AS verification_collected_at,
left(coalesce(latest.post_execution_state::text, ''), 700) AS post_state_text,
left(coalesce(latest.evidence_summary, ''), 300) AS evidence_summary
FROM recent_auto are
LEFT JOIN LATERAL (
SELECT
ev.verification_result,
ev.collected_at,
ev.post_execution_state,
ev.evidence_summary
FROM incident_evidence ev
WHERE ev.incident_id = are.incident_id
AND ev.verification_result IS NOT NULL
ORDER BY ev.collected_at DESC
LIMIT 1
) latest ON TRUE
)
SELECT
p.*,
i.status::text AS incident_status,
i.severity::text AS incident_severity,
i.alert_category,
i.alertname
FROM per_auto p
LEFT JOIN incidents i ON i.incident_id = p.incident_id
WHERE p.verification_result IS NOT NULL
AND p.verification_result <> 'success'
ORDER BY p.auto_created_at DESC
LIMIT 8
"""
async def _query_prometheus_value(
client: httpx.AsyncClient,
prom_url: str,
query: str,
) -> dict[str, Any]:
try:
response = await client.get(
f"{prom_url}/api/v1/query",
params={"query": query},
)
data = response.json()
if data.get("status") != "success":
return {"status": "error", "reason": "prometheus_query_failed"}
results = data.get("data", {}).get("result", [])
if not results:
return {
"status": "no_data",
"reason": "prometheus_empty_result_metric_not_emitted",
}
raw_value = results[0]["value"][1]
value = float(raw_value)
if not math.isfinite(value):
return {
"status": "skipped",
"reason": "prometheus_nan_or_inf",
"raw_value": raw_value,
}
return {"status": "ok", "value": value}
except Exception as exc:
logger.warning("adr100_slo_prometheus_query_error", query=query, error=str(exc))
return {"status": "error", "reason": "prometheus_query_error"}
def _metric_payload(
definition: Adr100SloDefinition,
*,
value: float | None,
status: str,
reason: str | None,
denominator_value: float | None,
sample_count: float | None,
) -> dict[str, Any]:
return {
"name": definition.name,
"query": definition.query,
"value": value,
"target": definition.target,
"hard_red_line": definition.hard_red_line,
"direction": definition.direction,
"unit": definition.unit,
"window": definition.window,
"status": status,
"evaluable": status in {"ok", "warning", "violated"},
"reason": reason,
"denominator_query": definition.denominator_query,
"denominator_value": denominator_value,
"sample_count": sample_count,
}
def _classify_status(value: float, definition: Adr100SloDefinition) -> str:
if definition.direction == "above":
if value < definition.hard_red_line:
return "violated"
if value < definition.target:
return "warning"
return "ok"
if value > definition.hard_red_line:
return "violated"
if value > definition.target:
return "warning"
return "ok"
def _build_verification_coverage_payload(
summary_row: Any,
recent_unverified_rows: Any,
recent_non_success_rows: Any = (),
) -> dict[str, Any]:
row = dict(summary_row)
total_auto = int(row.get("total_auto") or 0)
verified_auto = int(row.get("verified_auto") or 0)
verified_success = int(row.get("verified_success") or 0)
verified_non_success = int(row.get("verified_non_success") or 0)
unverified_auto = int(row.get("unverified_auto") or 0)
if total_auto == 0:
status = "skipped_low_volume"
reason = "no_auto_repair_executions_24h"
evaluable = False
elif unverified_auto > 0:
status = "warning"
reason = "verification_backlog_present"
evaluable = True
elif verified_non_success > 0:
status = "warning"
reason = "non_success_verification_present"
evaluable = True
else:
status = "ok"
reason = None
evaluable = True
coverage_rate = (verified_auto / total_auto) if total_auto else None
verification_success_rate = (verified_success / verified_auto) if verified_auto else None
recent_non_success = [
_non_success_finding_payload(dict(raw))
for raw in recent_non_success_rows
]
remediation_queue = _remediation_queue_payload(recent_non_success)
return {
"schema_version": "adr100_verification_coverage_v1",
"source": "postgresql",
"window": "24h",
"status": status,
"reason": reason,
"evaluable": evaluable,
"total_auto": total_auto,
"successful_auto": int(row.get("successful_auto") or 0),
"verified_auto": verified_auto,
"verified_success": verified_success,
"verified_non_success": verified_non_success,
"unverified_auto": unverified_auto,
"coverage_rate": coverage_rate,
"verification_success_rate": verification_success_rate,
"last_auto_at": _iso(row.get("last_auto_at")),
"last_verified_auto_at": _iso(row.get("last_verified_auto_at")),
"last_verification_evidence_at": _iso(row.get("last_verification_evidence_at")),
"latest_auto_age_seconds": _int_or_none(row.get("latest_auto_age_seconds")),
"last_verified_auto_age_seconds": _int_or_none(row.get("last_verified_auto_age_seconds")),
"recent_unverified": [
{
"id": str(item.get("id")),
"incident_id": str(item.get("incident_id")),
"success": bool(item.get("success")),
"created_at": _iso(item.get("created_at")),
}
for item in (dict(raw) for raw in recent_unverified_rows)
],
"recent_non_success": recent_non_success,
"non_success_breakdown": {
"by_verification_result": _count_breakdown(
item["verification_result"] for item in recent_non_success
),
"by_failure_class": _count_breakdown(
item["failure_class"] for item in recent_non_success
),
"by_remediation_status": _count_breakdown(
item["remediation_status"] for item in remediation_queue["items"]
),
},
"remediation_queue": remediation_queue,
}
def _non_success_finding_payload(row: dict[str, Any]) -> dict[str, Any]:
failure_class = _classify_non_success_failure(row)
remediation = _remediation_for_failure_class(failure_class)
return {
"auto_repair_id": str(row.get("auto_repair_id")),
"incident_id": str(row.get("incident_id")),
"incident_status": str(row.get("incident_status") or "unknown"),
"incident_severity": str(row.get("incident_severity") or "unknown"),
"alert_category": row.get("alert_category"),
"alertname": row.get("alertname"),
"auto_success": bool(row.get("auto_success")),
"playbook_id": row.get("playbook_id"),
"playbook_name": row.get("playbook_name"),
"triggered_by": row.get("triggered_by"),
"risk_level": row.get("risk_level"),
"verification_result": str(row.get("verification_result") or "unknown"),
"failure_class": failure_class,
"next_step": _next_step_for_failure_class(failure_class),
"remediation_status": remediation["status"],
"remediation_action": remediation["action"],
"remediation_owner": remediation["owner"],
"remediation_reason": remediation["reason"],
"auto_error_excerpt": _short_text(row.get("auto_error"), 180),
"evidence_excerpt": _short_text(row.get("evidence_summary"), 180),
"auto_created_at": _iso(row.get("auto_created_at")),
"verification_collected_at": _iso(row.get("verification_collected_at")),
}
def _classify_non_success_failure(row: dict[str, Any]) -> str:
combined = " ".join(
str(row.get(key) or "")
for key in ("auto_error", "post_state_text", "evidence_summary")
).lower()
if "unsupported scheme" in combined:
return "unsupported_action_scheme"
if "missing_query_parameter" in combined:
return "verifier_missing_promql"
if "empty_pod_name" in combined:
return "verifier_target_missing_pod"
if not bool(row.get("auto_success")):
return "auto_repair_execution_failed"
result = str(row.get("verification_result") or "").lower()
if result in {"failed", "timeout"}:
return f"verification_{result}"
return "verification_degraded"
def _remediation_for_failure_class(failure_class: str) -> dict[str, str]:
"""Map a non-success verification class to a read-only remediation work item.
This is dashboard triage metadata only. It does not auto-close incidents,
replay repairs, or approve write actions.
"""
if failure_class == "unsupported_action_scheme":
return {
"status": "ready_for_replay",
"action": "replay_with_supported_executor",
"owner": "auto_repair_executor",
"reason": "executor_gateway_available_after_t23",
}
if failure_class == "verifier_missing_promql":
return {
"status": "ready_for_reverify",
"action": "reverify_with_promql_template",
"owner": "post_execution_verifier",
"reason": "promql_template_available_after_t23",
}
if failure_class == "verifier_target_missing_pod":
return {
"status": "needs_target_mapping",
"action": "map_target_and_reverify",
"owner": "post_execution_verifier",
"reason": "verifier_target_missing",
}
if failure_class == "auto_repair_execution_failed":
return {
"status": "needs_playbook_ticket",
"action": "create_playbook_ticket",
"owner": "solver_or_operator",
"reason": "execution_failed_after_route_normalization",
}
if failure_class in {"verification_failed", "verification_timeout"}:
return {
"status": "manual_review",
"action": "escalate_verification_failure",
"owner": "sre_operator",
"reason": "verifier_returned_hard_failure",
}
return {
"status": "manual_review",
"action": "inspect_degraded_evidence",
"owner": "sre_operator",
"reason": "degraded_evidence_requires_human_context",
}
def _next_step_for_failure_class(failure_class: str) -> str:
if failure_class == "unsupported_action_scheme":
return "normalize_playbook_executor"
if failure_class == "verifier_missing_promql":
return "add_verifier_query_template"
if failure_class == "verifier_target_missing_pod":
return "map_verifier_target"
if failure_class == "auto_repair_execution_failed":
return "review_auto_repair_execution"
if failure_class in {"verification_failed", "verification_timeout"}:
return "escalate_verification_failure"
return "review_degraded_verification"
def _remediation_queue_payload(recent_non_success: list[dict[str, Any]]) -> dict[str, Any]:
items: list[dict[str, Any]] = []
for item in recent_non_success:
items.append({
"work_item_id": (
f"verification:{item.get('incident_id')}:{item.get('auto_repair_id')}"
),
"incident_id": item.get("incident_id"),
"auto_repair_id": item.get("auto_repair_id"),
"alertname": item.get("alertname"),
"playbook_id": item.get("playbook_id"),
"failure_class": item.get("failure_class"),
"verification_result": item.get("verification_result"),
"remediation_status": item.get("remediation_status"),
"remediation_action": item.get("remediation_action"),
"remediation_owner": item.get("remediation_owner"),
"remediation_reason": item.get("remediation_reason"),
"source": "adr100_verification_coverage",
"auto_created_at": item.get("auto_created_at"),
"verification_collected_at": item.get("verification_collected_at"),
})
ready_for_ai = sum(
1 for item in items
if item.get("remediation_status") in {"ready_for_replay", "ready_for_reverify"}
)
needs_human = sum(
1 for item in items
if item.get("remediation_status") in {
"needs_target_mapping",
"needs_playbook_ticket",
"manual_review",
}
)
return {
"schema_version": "adr100_remediation_queue_v1",
"source": "recent_non_success_read_model",
"total": len(items),
"ready_for_ai": ready_for_ai,
"needs_human": needs_human,
"items": items,
"by_status": _count_breakdown(
item.get("remediation_status") for item in items
),
"by_action": _count_breakdown(
item.get("remediation_action") for item in items
),
}
def _count_breakdown(values: Any) -> list[dict[str, Any]]:
counts: dict[str, int] = {}
for value in values:
key = str(value or "unknown")
counts[key] = counts.get(key, 0) + 1
return [
{"name": name, "count": count}
for name, count in sorted(counts.items(), key=lambda item: (-item[1], item[0]))
]
def _short_text(value: Any, limit: int) -> str | None:
if value is None:
return None
text = " ".join(str(value).split())
if not text:
return None
return text[:limit]
def _iso(value: Any) -> str | None:
return value.isoformat() if hasattr(value, "isoformat") else None
def _int_or_none(value: Any) -> int | None:
return int(value) if value is not None else None
def _overall_status(
metrics: list[dict[str, Any]],
evaluable: list[dict[str, Any]],
verification_coverage: dict[str, Any] | None = None,
) -> str:
if any(metric.get("status") == "violated" for metric in metrics):
return "violated"
if verification_coverage and verification_coverage.get("status") in {"violated", "warning"}:
return str(verification_coverage["status"])
if any(metric.get("status") == "warning" for metric in metrics):
return "warning"
if evaluable and any(metric.get("status") == "skipped_low_volume" for metric in metrics):
return "partial"
if evaluable:
return "ok"
if any(metric.get("status") == "no_data" for metric in metrics):
return "no_data"
return "skipped_low_volume"
_adr100_slo_status_service: Adr100SloStatusService | None = None
def get_adr100_slo_status_service() -> Adr100SloStatusService:
global _adr100_slo_status_service
if _adr100_slo_status_service is None:
_adr100_slo_status_service = Adr100SloStatusService()
return _adr100_slo_status_service

View File

@@ -842,14 +842,13 @@ class AIRouter:
空 dict 代表無資料或查詢失敗caller 應降級為忽略)。
"""
try:
from src.db.base import get_session_factory
from src.db.base import get_db_context
from src.repositories.aider_event_repository import AiderEventRepository
except ImportError:
return {}
try:
sf = get_session_factory()
async with sf() as sess:
async with get_db_context() as sess:
repo_obj = AiderEventRepository(sess)
stats = await repo_obj.model_stats_since(days=days)
except Exception:

View File

@@ -28,7 +28,7 @@ from datetime import timedelta
import structlog
from sqlalchemy import func, select, text
from src.db.base import get_session_factory
from src.db.base import get_db_context
from src.db.models import AiGovernanceEvent, AutoRepairExecution, ApprovalRecord
from src.utils.timezone import now_taipei
@@ -127,7 +127,7 @@ class AiSloCalculator:
try:
since = now_taipei() - timedelta(days=SLO_WINDOW_DAYS)
async with get_session_factory()() as session:
async with get_db_context() as session:
slo1 = await self._calc_auto_success_rate(session, since)
slo2 = await self._calc_human_override_rate(session, since)
slo3 = await self._calc_false_neg_rate(session, since)
@@ -210,7 +210,7 @@ class AiSloCalculator:
只在 any_violated=True 時呼叫。不管舊違反是否解決。
"""
try:
async with get_session_factory()() as session:
async with get_db_context() as session:
event = AiGovernanceEvent(
event_type="slo_violation",
details=report.to_dict(),

View File

@@ -0,0 +1,160 @@
"""DB-backed Alert Chain metric refresh.
`record_alert_chain_success()` updates Prometheus gauges in process memory. A
deployment or Pod restart clears that memory, so the first scrape after deploy
can report `awoooi_alert_chain_last_success_timestamp` as missing even though
Alertmanager / Sentry / SigNoz evidence is already in DB.
"""
from __future__ import annotations
from dataclasses import dataclass
from time import time
from typing import Any, Mapping
import structlog
from sqlalchemy import text
from src.core.metrics import ALERT_CHAIN_LAST_SUCCESS
from src.db.base import get_db_context
logger = structlog.get_logger(__name__)
_REFRESH_INTERVAL_SECONDS = 15
@dataclass(frozen=True)
class AlertChainEvidenceSample:
source: str
last_success_timestamp: float
evidence_source: str
class AlertChainMetricsService:
"""Refresh process-local alert-chain gauges from durable DB evidence."""
def __init__(self) -> None:
self._last_refresh_at = 0.0
self._last_samples: list[AlertChainEvidenceSample] = []
async def refresh_last_success_gauge(
self,
*,
project_id: str = "awoooi",
force: bool = False,
) -> list[AlertChainEvidenceSample]:
now_ts = time()
if (
not force
and self._last_samples
and now_ts - self._last_refresh_at < _REFRESH_INTERVAL_SECONDS
):
apply_alert_chain_last_success_samples(self._last_samples)
return self._last_samples
samples = await self.fetch_latest_success_samples(project_id=project_id)
apply_alert_chain_last_success_samples(samples)
self._last_samples = samples
self._last_refresh_at = now_ts
return samples
async def fetch_latest_success_samples(
self,
*,
project_id: str = "awoooi",
) -> list[AlertChainEvidenceSample]:
async with get_db_context(project_id) as db:
result = await db.execute(text(_ALERT_CHAIN_EVIDENCE_SQL), {"project_id": project_id})
return merge_alert_chain_evidence_rows(result.mappings().all())
def apply_alert_chain_last_success_samples(samples: list[AlertChainEvidenceSample]) -> None:
"""Write durable last-success timestamps into the existing Prometheus gauge.
Do not write `awoooi_alert_chain_healthy` here: that gauge represents an
explicit runtime failure signal. Durable success evidence should not mask a
later in-memory failure from the same process.
"""
for sample in samples:
ALERT_CHAIN_LAST_SUCCESS.labels(source=sample.source).set(
sample.last_success_timestamp
)
def merge_alert_chain_evidence_rows(
rows: list[Mapping[str, Any]] | list[Any],
) -> list[AlertChainEvidenceSample]:
"""Normalize SQL rows and keep the newest evidence per source."""
latest_by_source: dict[str, AlertChainEvidenceSample] = {}
for row in rows:
source = str(_row_value(row, "source") or "").strip().lower()
evidence_source = str(_row_value(row, "evidence_source") or "db").strip()
raw_timestamp = _row_value(row, "last_success_timestamp")
if not source or raw_timestamp is None:
continue
try:
timestamp = float(raw_timestamp)
except (TypeError, ValueError):
continue
if timestamp <= 0:
continue
sample = AlertChainEvidenceSample(
source=source,
last_success_timestamp=timestamp,
evidence_source=evidence_source,
)
previous = latest_by_source.get(source)
if previous is None or sample.last_success_timestamp > previous.last_success_timestamp:
latest_by_source[source] = sample
return sorted(latest_by_source.values(), key=lambda item: item.source)
def _row_value(row: Mapping[str, Any] | Any, key: str) -> Any:
if isinstance(row, Mapping):
return row.get(key)
return getattr(row, key, None)
_ALERT_CHAIN_EVIDENCE_SQL = """
WITH conversation_evidence AS (
SELECT
lower(platform_subject_id) AS source,
EXTRACT(EPOCH FROM MAX(COALESCE(provider_ts, received_at))) AS last_success_timestamp,
'awooop_conversation_event' AS evidence_source
FROM awooop_conversation_event
WHERE project_id = :project_id
AND channel_type = 'internal'
AND lower(COALESCE(platform_subject_id, '')) IN ('alertmanager', 'sentry', 'signoz')
GROUP BY lower(platform_subject_id)
),
alertmanager_log_evidence AS (
SELECT
'alertmanager' AS source,
EXTRACT(EPOCH FROM MAX(created_at)) AS last_success_timestamp,
'alert_operation_log' AS evidence_source
FROM alert_operation_log
WHERE event_type = 'ALERT_RECEIVED'
AND actor = 'alertmanager'
)
SELECT source, last_success_timestamp, evidence_source
FROM conversation_evidence
WHERE last_success_timestamp IS NOT NULL
UNION ALL
SELECT source, last_success_timestamp, evidence_source
FROM alertmanager_log_evidence
WHERE last_success_timestamp IS NOT NULL
"""
_alert_chain_metrics_service: AlertChainMetricsService | None = None
def get_alert_chain_metrics_service() -> AlertChainMetricsService:
global _alert_chain_metrics_service
if _alert_chain_metrics_service is None:
_alert_chain_metrics_service = AlertChainMetricsService()
return _alert_chain_metrics_service

View File

@@ -33,6 +33,7 @@ import yaml
from src.constants.alert_types import ALERTNAME_TO_TYPE
from src.services.action_parser import parse_kubectl_action
from src.services.ollama_endpoint_resolver import resolve_ollama_order
logger = structlog.get_logger(__name__)
@@ -297,6 +298,12 @@ def _matches(rule: dict, alertname: str, alert_type: str, message: str, instance
if alertnames and alertname in alertnames:
return True
# 2026-05-31 ogt + Codex: 有明確 alertname 的規則不得只靠寬鬆 message
# keyword 命中,否則 HostPreviousBootStorageErrorsDetected 這類主機 storage
# 告警會誤配到 minio_disk_high。
if alertnames and alertname and alertname != "custom":
return False
# alert_type 部分匹配
for kw in match.get("alert_type", []):
if kw.lower() in alert_type.lower():
@@ -634,10 +641,12 @@ async def _insert_catalog_ai_generated(
)
return
from sqlalchemy import text as _sql
import src.db.base as _db_base
import json as _json
from sqlalchemy import text as _sql
import src.db.base as _db_base
# 從 rule_dict 提取欄位
# 'expr' 在 OpenClaw YAML 規則中不存在(非 PromQL
# 使用 alertname 作為語意佔位(與 yaml_hardcoded 同等策略)
@@ -704,17 +713,37 @@ async def _insert_catalog_ai_generated(
async def _call_ollama(prompt: str, ollama_url: str, model: str) -> str | None:
"""呼叫 Ollama 生成規則 YAML"""
try:
async with httpx.AsyncClient(timeout=60) as client:
resp = await client.post(
f"{ollama_url}/api/generate",
json={"model": model, "prompt": prompt, "stream": False, "options": {"temperature": 0.1}},
)
resp.raise_for_status()
return resp.json().get("response", "")
except Exception as e:
logger.warning("auto_rule_ollama_failed", error=str(e))
return None
endpoints = list(resolve_ollama_order("deep_rca"))
if ollama_url and all(endpoint.url != ollama_url for endpoint in endpoints):
from types import SimpleNamespace
endpoints.insert(0, SimpleNamespace(url=ollama_url, provider_name="custom"))
async with httpx.AsyncClient(timeout=60) as client:
for endpoint in endpoints:
if not endpoint.url:
continue
try:
resp = await client.post(
f"{endpoint.url}/api/generate",
json={
"model": model,
"prompt": prompt,
"stream": False,
# Preserve response-body compatibility with Ollama 0.24 thinking models.
"think": False,
"options": {"temperature": 0.1},
},
)
resp.raise_for_status()
return resp.json().get("response", "")
except Exception as e:
logger.warning(
"auto_rule_ollama_failed",
provider=endpoint.provider_name,
error=str(e),
)
return None
async def _call_gemini(prompt: str, api_key: str) -> str | None:

View File

@@ -0,0 +1,26 @@
"""
Approval action classifier
==========================
2026-05-31 ogt + Codex: Telegram 告警鏈路一致性修復。
將 OBSERVE / INVESTIGATE / NO_ACTION 這類「純觀察、未執行修復」的
判斷集中,避免 execution、Telegram、統計各自用不同語意。
"""
from __future__ import annotations
def is_no_action_approval_action(action: str | None) -> bool:
"""Return True when an approval action records observation instead of repair."""
text = (action or "").strip()
upper = text.upper()
if not text:
return True
return (
"NO_ACTION" in upper
or "NO-ACTION" in upper
or "NOACTION" in upper
or "(未設)" in text
or upper.startswith("OBSERVE")
or upper.startswith("INVESTIGATE")
)

View File

@@ -110,10 +110,10 @@ def approval_record_to_request(record: ApprovalRecord) -> ApprovalRequest:
hit_count=record.hit_count,
last_seen_at=record.last_seen_at,
# B3 fix 2026-04-24 ogt + Claude Sonnet 4.6: 補回 DB 欄位(人工審核路徑讀回必要)
# incident_id / matched_playbook_id 在 ApprovalRequest 基礎模型中有定義
# telegram_message_id / telegram_chat_id 只在 DB model不在 Pydantic ApprovalRequest
incident_id=getattr(record, "incident_id", None),
matched_playbook_id=getattr(record, "matched_playbook_id", None),
telegram_message_id=getattr(record, "telegram_message_id", None),
telegram_chat_id=getattr(record, "telegram_chat_id", None),
)
@@ -659,6 +659,9 @@ class ApprovalDBService:
approval_id: UUID,
success: bool,
error_message: str | None = None,
execution_kind: str | None = None,
repair_executed: bool | None = None,
repair_attempted: bool | None = None,
) -> None:
"""
更新執行狀態
@@ -669,21 +672,47 @@ class ApprovalDBService:
"""
async with get_db_context() as db:
status = ApprovalStatus.EXECUTION_SUCCESS if success else ApprovalStatus.EXECUTION_FAILED
values: dict = {"status": status}
result = await db.execute(
select(ApprovalRecord).where(ApprovalRecord.id == str(approval_id))
)
record = result.scalar_one_or_none()
if record is None:
logger.warning(
"approval_execution_status_update_missing",
id=str(approval_id),
success=success,
)
return
record.status = status
if not success and error_message:
# 截斷至合理長度,避免爆欄位
values["rejection_reason"] = str(error_message)[:2000]
await db.execute(
update(ApprovalRecord)
.where(ApprovalRecord.id == str(approval_id))
.values(**values)
)
record.rejection_reason = str(error_message)[:2000]
if execution_kind:
# 2026-05-31 ogt + Codex: OBSERVE/NO_ACTION 仍需 terminal 狀態,
# 但前台/報表必須能分辨「未執行修復」而非真正 execution success。
metadata = dict(record.extra_metadata or {})
metadata["execution_kind"] = execution_kind
metadata["repair_executed"] = (
repair_executed
if repair_executed is not None
else execution_kind not in {
"no_action",
"diagnostic",
"parse_failed",
"unsupported_action",
}
)
if repair_attempted is not None:
metadata["repair_attempted"] = repair_attempted
record.extra_metadata = metadata
logger.info(
"approval_execution_status_updated",
id=str(approval_id),
success=success,
has_error=bool(error_message),
execution_kind=execution_kind,
)
async def update_incident_id(self, approval_id: UUID, incident_id: str) -> None:

File diff suppressed because it is too large Load Diff

View File

@@ -22,9 +22,10 @@ Phase 8: 自動化層實作
- P0/P1 嚴重度 Incident 需要人工確認
"""
from dataclasses import dataclass
from collections.abc import Callable
from typing import Protocol
from dataclasses import dataclass
import re
from typing import Any, Protocol
import structlog
@@ -81,6 +82,55 @@ class AutoRepairResult:
execution_time_ms: int = 0
@dataclass(frozen=True)
class _SshMcpRoute:
"""Route a legacy SSH playbook command to a governed MCP tool."""
tool_name: str
params: dict[str, Any]
_SHORT_HOST_MAP: dict[str, str] = {
"110": "192.168.0.110",
"120": "192.168.0.120",
"121": "192.168.0.121",
"188": "192.168.0.188",
}
_SSH_DIAGNOSTIC_KEYWORDS = (
"ps aux",
"docker stats",
"docker inspect",
"docker logs",
"docker ps",
"docker top",
"df -h",
"du -",
"free -h",
"journalctl",
"systemctl show",
"tail ",
"top ",
"uptime",
)
_SSH_WRITE_KEYWORDS = (
"docker restart",
"docker start",
"docker stop",
"docker rm",
"docker prune",
"systemctl restart",
"systemctl stop",
"systemctl start",
"truncate ",
" rm ",
"rm -",
"certbot renew",
"bash ",
)
# =============================================================================
# Auto Repair Service Interface
# =============================================================================
@@ -108,6 +158,7 @@ class IAutoRepairService(Protocol):
self,
incident: Incident,
playbook: Playbook,
run_post_verification: bool = True,
) -> AutoRepairResult:
"""
執行自動修復
@@ -320,7 +371,16 @@ class AutoRepairService:
)
# 4. 檢查最佳匹配
best_match = recommendations[0]
best_match = self._select_best_recommendation(recommendations, symptoms)
if best_match is not recommendations[0]:
logger.warning(
"auto_repair_exact_match_prioritized",
incident_id=incident.incident_id,
selected_playbook_id=best_match.playbook.playbook_id,
original_playbook_id=recommendations[0].playbook.playbook_id,
selected_similarity=best_match.similarity_score,
original_similarity=recommendations[0].similarity_score,
)
# 2026-04-07 Claude Code: 統帥指令「直接全部跳成自動修復」
# 移除: 相似度門檻、is_high_quality 門檻、冷啟動機制、風險等級門檻
@@ -378,6 +438,7 @@ class AutoRepairService:
playbook: Playbook,
is_cold_start: bool = False,
similarity_score: float | None = None,
run_post_verification: bool = True,
) -> AutoRepairResult:
"""
執行自動修復
@@ -414,6 +475,8 @@ class AutoRepairService:
executed_steps.append(
f"Step {step.step_number}: {step.command[:50]}... -> {step_result}"
)
if self._is_step_failure_result(step_result):
raise RuntimeError(f"Step {step.step_number} failed: {step_result}")
# 更新 Playbook 統計
await self._playbook_service.record_execution(
@@ -579,10 +642,17 @@ class AutoRepairService:
error=str(_inner_e),
)
_vl_task = _asyncio.create_task(_verify_and_learn())
if hasattr(self, "_pending_tasks"):
self._pending_tasks.add(_vl_task)
_vl_task.add_done_callback(self._pending_tasks.discard)
if run_post_verification:
_vl_task = _asyncio.create_task(_verify_and_learn())
if hasattr(self, "_pending_tasks"):
self._pending_tasks.add(_vl_task)
_vl_task.add_done_callback(self._pending_tasks.discard)
else:
logger.info(
"auto_repair_service_post_verify_delegated",
incident_id=incident.incident_id,
playbook_id=playbook.playbook_id,
)
except Exception as _vl_e:
logger.warning("auto_repair_verifier_setup_failed", error=str(_vl_e))
@@ -688,6 +758,44 @@ class AutoRepairService:
keywords=keywords[:10],
)
def _select_best_recommendation(
self,
recommendations,
symptoms: SymptomPattern,
):
"""Prefer deterministic alert/service matches over fuzzy similarity only.
A higher fuzzy score must not outrank a playbook that explicitly names the
firing alert or affected service. Live-fire T16 proved that this can route
a safe K8s canary into an unrelated host diagnostic playbook.
"""
symptom_alerts = {str(name) for name in (symptoms.alert_names or []) if name}
symptom_services = {
str(service) for service in (symptoms.affected_services or []) if service
}
def _priority(recommendation) -> tuple[int, int, float]:
pattern = recommendation.playbook.symptom_pattern
playbook_alerts = {
str(name) for name in (pattern.alert_names or []) if name
}
playbook_services = {
str(service) for service in (pattern.affected_services or []) if service
}
alert_exact = int(bool(symptom_alerts & playbook_alerts))
service_exact = int(bool(symptom_services & playbook_services))
return (alert_exact, service_exact, float(recommendation.similarity_score or 0.0))
return max(recommendations, key=_priority)
@staticmethod
def _is_step_failure_result(step_result: str) -> bool:
"""Treat executor-declared failures as failed auto-repair executions."""
normalized = (step_result or "").strip().upper()
return normalized.startswith("FAILED:") or normalized == "UNKNOWN_ACTION_TYPE"
def _get_max_risk_level(self, playbook: Playbook) -> RiskLevel:
"""取得 Playbook 中最高的風險等級"""
risk_order = {
@@ -860,6 +968,175 @@ class AutoRepairService:
# 安全降級:檢查失敗 → 保守拒絕
return False
def _route_legacy_ssh_command_to_mcp(
self,
incident: Incident,
command: str,
) -> _SshMcpRoute | None:
"""Map read-only legacy ``ssh {host} '...'`` steps to MCP Gateway.
YAML_RULE playbooks predate the URI executor and can contain compound
shell diagnostics. Those commands should not bypass the newer
scheme-based HostRepairAgent or loosen its shell safety guard; read-only
diagnostics are instead routed to the governed SSH MCP provider.
"""
raw_command = (command or "").strip()
lowered = raw_command.lower()
if not lowered.startswith("ssh "):
return None
if any(token in lowered for token in _SSH_WRITE_KEYWORDS):
return None
if not any(token in lowered for token in _SSH_DIAGNOSTIC_KEYWORDS):
return None
host = self._resolve_ssh_host_for_incident(incident, raw_command)
if not host:
return None
params: dict[str, Any] = {"host": host}
container_name = self._resolve_container_name_for_incident(incident, raw_command)
if container_name:
params["container_name"] = container_name
return _SshMcpRoute(tool_name="ssh_diagnose", params=params)
def preview_read_only_ssh_mcp_route(
self,
incident: Incident,
command: str,
) -> dict[str, Any] | None:
"""Preview whether a legacy SSH diagnostic can use the MCP Gateway.
This is used by remediation dry-runs to prove the supported executor
path without running the original PlayBook step or writing an execution
result.
"""
route = self._route_legacy_ssh_command_to_mcp(incident, command)
if route is None:
return None
return {
"tool_name": route.tool_name,
"params": route.params,
"agent_id": "auto_repair_executor",
"required_scope": "read",
"flywheel_node": "execute",
}
def _resolve_ssh_host_for_incident(self, incident: Incident, command: str) -> str:
"""Resolve ``{host}``, short host labels, and exporter instance ports."""
labels = self._incident_labels(incident)
raw_host = ""
match = re.match(r"ssh\s+([^\s'\"]+)", command.strip(), flags=re.IGNORECASE)
if match:
raw_host = match.group(1)
if not raw_host or "{" in raw_host or "}" in raw_host:
raw_host = (
str(labels.get("host") or "")
or str(labels.get("instance") or "")
or str(labels.get("node") or "")
or str(labels.get("exported_instance") or "")
)
return self._normalize_ssh_host(raw_host)
@staticmethod
def _normalize_ssh_host(raw_host: str) -> str:
host = (raw_host or "").strip()
if host.startswith("ssh://"):
host = host.removeprefix("ssh://")
if "@" in host:
host = host.rsplit("@", 1)[1]
if host.startswith("[") and "]" in host:
host = host[1:host.index("]")]
if host.count(":") == 1:
maybe_host, maybe_port = host.rsplit(":", 1)
if maybe_port.isdigit():
host = maybe_host
if host in _SHORT_HOST_MAP:
return _SHORT_HOST_MAP[host]
match = re.fullmatch(r"(?:node-exporter-|host-)?(110|120|121|188)", host)
if match:
return _SHORT_HOST_MAP[match.group(1)]
return host
def _resolve_container_name_for_incident(
self,
incident: Incident,
command: str,
) -> str:
labels = self._incident_labels(incident)
for key in ("container_name", "container", "name"):
value = str(labels.get(key) or "").strip()
if value and "{" not in value and "}" not in value:
return value
match = re.search(
r"docker\s+(?:stats\s+--no-stream|inspect|logs|top|ps\s+-a\s+--filter\s+name=)\s+([a-zA-Z0-9._-]+)",
command,
)
return match.group(1) if match else ""
@staticmethod
def _incident_labels(incident: Incident) -> dict[str, Any]:
for signal in incident.signals or []:
labels = getattr(signal, "labels", None)
if labels:
return labels
return {}
async def _execute_ssh_mcp_route(
self,
incident: Incident,
route: _SshMcpRoute,
) -> str:
"""Execute a routed SSH diagnostic through AwoooP MCP Gateway."""
try:
from src.db.base import get_db_context
from src.plugins.mcp.gateway import GatewayContext, McpGateway, McpGatewayError
from src.services.mcp_audit_context import with_mcp_audit_context
incident_id = incident.incident_id
params = with_mcp_audit_context(
route.params,
session_id=f"incident:{incident_id}:auto_repair_execute",
incident_id=incident_id,
flywheel_node="execute",
agent_role="auto_repair_executor",
)
async with get_db_context("awoooi") as db:
ctx = GatewayContext(
project_id="awoooi",
agent_id="auto_repair_executor",
tool_name=route.tool_name,
trace_id=incident_id,
is_shadow=False,
environment={"env": "prod"},
required_scope="read",
)
result = await McpGateway(db).call(ctx, params)
except McpGatewayError as exc:
return f"FAILED: mcp:{route.tool_name} {exc.error_code}: {exc}"
except Exception as exc:
logger.warning(
"auto_repair_ssh_mcp_route_failed",
incident_id=incident.incident_id,
tool=route.tool_name,
error=str(exc),
)
return f"FAILED: mcp:{route.tool_name} {exc}"
if result.success:
preview = str(result.output or "")[:500]
return f"SUCCESS: mcp:{route.tool_name} {preview}".strip()
return f"FAILED: mcp:{route.tool_name} {result.error or 'execution failed'}"
async def _execute_step(self, incident: Incident, step) -> str:
"""
執行單一修復步驟
@@ -891,6 +1168,10 @@ class AutoRepairService:
# 2026-04-06 Claude Code: Sprint 3 — repair_by_uri (URI scheme 路由)
if step.action_type == ActionType.SSH_COMMAND:
route = self._route_legacy_ssh_command_to_mcp(incident, step.command)
if route is not None:
return await self._execute_ssh_mcp_route(incident, route)
from src.services.host_repair_agent import HostRepairAgent
agent = HostRepairAgent()
approved = not getattr(step, "requires_approval", False)

View File

@@ -0,0 +1,633 @@
"""AwoooP Ansible audit helpers.
This module is intentionally non-executing. It exposes the Ansible audit
contract and repo-known playbook catalog so the truth chain can say whether
Ansible was actually considered or executed, without pretending that catalog
hints are runtime remediation.
"""
from __future__ import annotations
import json
from typing import Any
import structlog
from sqlalchemy import text
from src.db.base import get_db_context
logger = structlog.get_logger(__name__)
def _automation_operation_log_incident_id(value: str) -> int | None:
# ADR-090 keeps external INC-* ids in input JSON; the DB column is BIGINT.
normalized = str(value or "").strip()
return int(normalized) if normalized.isdigit() else None
ANSIBLE_OPERATION_TYPES = frozenset({
"ansible_candidate_matched",
"ansible_check_mode_executed",
"ansible_apply_executed",
"ansible_rollback_executed",
"ansible_execution_skipped",
})
_CATALOG: tuple[dict[str, Any], ...] = (
{
"catalog_id": "ansible:110-devops",
"playbook_path": "infra/ansible/playbooks/110-devops.yml",
"inventory_hosts": ["host_110"],
"domains": ["swap", "harbor", "sentry", "gitea", "langfuse", "bitan", "runner", "keepalived", "nginx"],
"keywords": [
"110",
"docker",
"container",
"dockercontainerunhealthy",
"swap",
"harbor",
"sentry",
"gitea",
"langfuse",
"bitan",
"runner",
"github-runner",
"keepalived",
],
"supports_check_mode": True,
"auto_apply_enabled": False,
"approval_required": True,
"risk_level": "medium",
},
{
"catalog_id": "ansible:188-momo-backup-user",
"playbook_path": "infra/ansible/playbooks/188-momo-backup-user.yml",
"check_mode_playbook_path": "infra/ansible/playbooks/188-momo-backup-user.yml",
"inventory_hosts": ["host_188"],
"domains": ["momo_backup", "postgresql", "cron", "awooop_notification"],
"keywords": [
"188",
"momo",
"momopostgresbackupfailed",
"postgres",
"postgresql",
"backup",
"momo pg_backup",
"momo postgres backup",
"pg_backup",
"momo-pg-backup",
"cron",
"crontab",
],
"supports_check_mode": True,
"auto_apply_enabled": False,
"approval_required": True,
"risk_level": "low",
},
{
"catalog_id": "ansible:188-ai-web",
"playbook_path": "infra/ansible/playbooks/188-ai-web.yml",
"check_mode_playbook_path": "infra/ansible/playbooks/188-ai-web-readonly.yml",
"inventory_hosts": ["host_188"],
"domains": ["docker", "momo_backup", "signoz", "minio", "litellm", "n8n", "open_webui", "nginx"],
"keywords": [
"188",
"docker",
"container",
"dockercontainerunhealthy",
"momo",
"backup",
"postgresql",
"pg_backup",
"signoz",
"minio",
"litellm",
"n8n",
"open-webui",
"openwebui",
"docker-registry",
],
"supports_check_mode": True,
"auto_apply_enabled": False,
"approval_required": True,
"risk_level": "medium",
},
{
"catalog_id": "ansible:nginx-sync",
"playbook_path": "infra/ansible/playbooks/nginx-sync.yml",
"inventory_hosts": ["host_110", "host_188"],
"domains": ["nginx", "proxy", "ollama_proxy", "tls"],
"keywords": ["nginx", "proxy", "ollama", "gcp", "tls", "cert", "502", "upstream"],
"supports_check_mode": True,
"auto_apply_enabled": False,
"approval_required": True,
"risk_level": "medium",
},
{
"catalog_id": "ansible:restore-password-auth",
"playbook_path": "infra/ansible/playbooks/restore-password-auth.yml",
"inventory_hosts": ["host_110", "host_120", "host_121", "host_188"],
"domains": ["ssh", "password_auth"],
"keywords": ["ssh", "passwordauthentication", "password auth", "login", "auth"],
"supports_check_mode": False,
"auto_apply_enabled": False,
"approval_required": True,
"risk_level": "high",
},
)
def get_ansible_catalog_item(catalog_id: str) -> dict[str, Any] | None:
"""Return one repo-known Ansible catalog item without exposing mutability."""
for item in _CATALOG:
if item["catalog_id"] == catalog_id:
return dict(item)
return None
def _get(row: dict[str, Any], key: str) -> Any:
return row.get(key)
def _tags(row: dict[str, Any]) -> list[str]:
raw = _get(row, "tags")
if isinstance(raw, list):
return [str(item).lower() for item in raw]
if isinstance(raw, str):
return [part.strip().lower() for part in raw.split(",") if part.strip()]
return []
def _first_present(row: dict[str, Any], keys: tuple[str, ...]) -> Any:
for key in keys:
value = _get(row, key)
if value not in (None, ""):
return value
return None
def _json_object(row: dict[str, Any], key: str) -> dict[str, Any]:
raw = _get(row, key)
if isinstance(raw, dict):
return raw
if isinstance(raw, str) and raw.strip():
try:
parsed = json.loads(raw)
except json.JSONDecodeError:
return {}
return parsed if isinstance(parsed, dict) else {}
return {}
def _json_value(row: dict[str, Any], container_key: str, value_key: str) -> Any:
return _json_object(row, container_key).get(value_key)
def _first_present_with_json(row: dict[str, Any], keys: tuple[str, ...], json_keys: tuple[tuple[str, str], ...] = ()) -> Any:
value = _first_present(row, keys)
if value not in (None, ""):
return value
for container_key, value_key in json_keys:
value = _json_value(row, container_key, value_key)
if value not in (None, ""):
return value
return None
def _bool_or_none(value: Any) -> bool | None:
if isinstance(value, bool):
return value
if value in (None, ""):
return None
normalized = str(value).strip().lower()
if normalized in {"1", "true", "yes", "y", "on"}:
return True
if normalized in {"0", "false", "no", "n", "off"}:
return False
return None
def _int_or_none(value: Any) -> int | None:
if value in (None, ""):
return None
try:
return int(value)
except (TypeError, ValueError):
return None
def _is_ansible_operation(row: dict[str, Any]) -> bool:
operation_type = str(_get(row, "operation_type") or "").lower()
if operation_type in ANSIBLE_OPERATION_TYPES:
return True
if "ansible" in _tags(row):
return True
executor = str(
_first_present(
row,
(
"input_executor",
"input_execution_backend",
"output_executor",
"output_execution_backend",
),
)
or ""
).lower()
if executor == "ansible":
return True
playbook_path = str(
_first_present(row, ("input_playbook_path", "output_playbook_path", "input_ansible_playbook_path", "output_ansible_playbook_path"))
or ""
).lower()
return "infra/ansible/" in playbook_path or playbook_path.endswith(".yml") and "ansible" in playbook_path
def _ansible_record(row: dict[str, Any]) -> dict[str, Any]:
return {
"op_id": _get(row, "op_id"),
"parent_op_id": _get(row, "parent_op_id"),
"operation_type": _get(row, "operation_type"),
"status": _get(row, "status"),
"actor": _get(row, "actor"),
"catalog_id": _first_present(row, ("input_catalog_id", "output_catalog_id")),
"playbook_id": _first_present(row, ("input_playbook_id", "output_playbook_id")),
"playbook_path": _first_present(
row,
("input_playbook_path", "output_playbook_path", "input_ansible_playbook_path", "output_ansible_playbook_path"),
),
"execution_mode": _first_present(row, ("input_execution_mode", "output_execution_mode")),
"check_mode": _first_present(row, ("input_check_mode", "output_check_mode")),
"check_mode_executed": _first_present_with_json(
row,
("input_check_mode_executed", "output_check_mode_executed", "dry_run_check_mode_executed"),
(("dry_run_result", "check_mode_executed"),),
),
"apply_enabled": _first_present(row, ("input_apply_enabled", "output_apply_enabled")),
"apply_executed": _first_present_with_json(
row,
("input_apply_executed", "output_apply_executed", "dry_run_apply_executed"),
(("dry_run_result", "apply_executed"),),
),
"approval_source": _first_present(row, ("input_approval_source", "output_approval_source")),
"returncode": _first_present_with_json(
row,
("output_returncode", "input_returncode", "dry_run_returncode"),
(("dry_run_result", "returncode"),),
),
"not_used_reason": _first_present(row, ("input_not_used_reason", "output_not_used_reason")),
"dry_run_result": _get(row, "dry_run_result"),
"error": _get(row, "error"),
"duration_ms": _get(row, "duration_ms"),
"tags": _get(row, "tags"),
"created_at": _get(row, "created_at"),
}
def summarize_ansible_execution(records: list[dict[str, Any]]) -> dict[str, Any]:
"""Summarize durable Ansible audit rows for Telegram and Operator UI."""
typed_records = [row for row in records if isinstance(row, dict)]
check_mode_total = 0
apply_total = 0
rollback_total = 0
pending_check_mode_total = 0
applied_success_total = 0
terminal_check_mode_parent_ids = {
str(row.get("parent_op_id"))
for row in typed_records
if str(row.get("operation_type") or "") in {
"ansible_check_mode_executed",
"ansible_execution_skipped",
}
and row.get("parent_op_id")
}
for row in typed_records:
operation_type = str(row.get("operation_type") or "")
status = str(row.get("status") or "").lower()
if operation_type == "ansible_check_mode_executed" and status != "pending":
check_mode_total += 1
elif operation_type == "ansible_apply_executed":
apply_total += 1
if status in {"success", "completed", "executed", "applied"}:
applied_success_total += 1
elif operation_type == "ansible_rollback_executed":
rollback_total += 1
elif (
operation_type == "ansible_candidate_matched"
and str(row.get("op_id")) not in terminal_check_mode_parent_ids
):
pending_check_mode_total += 1
latest_record = typed_records[0] if typed_records else {}
latest_apply = next(
(row for row in typed_records if row.get("operation_type") == "ansible_apply_executed"),
{},
)
latest_check = next(
(row for row in typed_records if row.get("operation_type") == "ansible_check_mode_executed"),
{},
)
focused = latest_apply or latest_check or latest_record
returncode = _int_or_none(focused.get("returncode"))
approval_source = focused.get("approval_source")
return {
"check_mode_total": check_mode_total,
"apply_total": apply_total,
"rollback_total": rollback_total,
"pending_check_mode_total": pending_check_mode_total,
"applied_success_total": applied_success_total,
"applied": applied_success_total > 0,
"controlled_apply": bool(latest_apply) and bool(approval_source),
"latest_operation_type": focused.get("operation_type"),
"latest_status": focused.get("status"),
"latest_catalog_id": focused.get("catalog_id"),
"latest_playbook_path": focused.get("playbook_path"),
"latest_execution_mode": focused.get("execution_mode"),
"latest_check_mode": _bool_or_none(focused.get("check_mode")),
"latest_check_mode_executed": _bool_or_none(focused.get("check_mode_executed")),
"latest_apply_enabled": _bool_or_none(focused.get("apply_enabled")),
"latest_apply_executed": _bool_or_none(focused.get("apply_executed")),
"latest_returncode": returncode if returncode is not None else focused.get("returncode"),
"approval_source": approval_source,
"latest_actor": focused.get("actor"),
"latest_op_id": focused.get("op_id"),
"latest_parent_op_id": focused.get("parent_op_id"),
}
def _flatten_text(value: Any, pieces: list[str], remaining: int = 80) -> int:
if remaining <= 0 or value is None:
return remaining
if isinstance(value, dict):
for key, item in value.items():
remaining = _flatten_text(key, pieces, remaining)
remaining = _flatten_text(item, pieces, remaining)
if remaining <= 0:
break
return remaining
if isinstance(value, list):
for item in value:
remaining = _flatten_text(item, pieces, remaining)
if remaining <= 0:
break
return remaining
pieces.append(str(value).lower())
return remaining - 1
def _source_haystack(incident: dict[str, Any] | None, drift: dict[str, Any] | None) -> str:
pieces: list[str] = []
_flatten_text(incident, pieces)
_flatten_text(drift, pieces)
return " ".join(pieces)
def _catalog_hints(incident: dict[str, Any] | None, drift: dict[str, Any] | None) -> dict[str, Any]:
haystack = _source_haystack(incident, drift)
candidates: list[dict[str, Any]] = []
unmatched: list[str] = []
for item in _CATALOG:
matched = [keyword for keyword in item["keywords"] if keyword in haystack]
public_item = {
key: value
for key, value in item.items()
if key
in {
"catalog_id",
"playbook_path",
"inventory_hosts",
"domains",
"supports_check_mode",
"check_mode_playbook_path",
"auto_apply_enabled",
"approval_required",
"risk_level",
}
}
if matched:
candidates.append({
**public_item,
"match_score": len(matched),
"matched_keywords": matched,
})
else:
unmatched.append(item["catalog_id"])
candidates.sort(key=lambda row: (-int(row["match_score"]), str(row["catalog_id"])))
return {
"match_mode": "static_catalog_keyword_hint_v1",
"decision_effect": "none",
"available_count": len(_CATALOG),
"candidates": candidates,
"unmatched_catalog_ids": unmatched,
}
def build_ansible_truth(
automation_ops: list[dict[str, Any]],
*,
incident: dict[str, Any] | None,
drift: dict[str, Any] | None,
) -> dict[str, Any]:
"""Build the truth-chain Ansible section from audited facts and catalog hints."""
records = [_ansible_record(row) for row in automation_ops if _is_ansible_operation(row)]
summary = summarize_ansible_execution(records)
return {
"considered": bool(records),
"records": records,
"summary": summary,
"audit_contract": {
"schema_version": "ansible_executor_audit_v1",
"operation_types": sorted(ANSIBLE_OPERATION_TYPES),
"required_audit_fields": [
"incident_id",
"operation_type",
"status",
"actor",
"input.executor",
"input.catalog_id",
"input.execution_mode",
"input.playbook_path",
"input.check_mode",
"input.approval_source",
"output.returncode",
"output.not_used_reason",
"dry_run_result",
],
"default_execution_mode": "catalog/dry-run audit only until approval execution is explicitly wired",
},
"candidate_catalog": _catalog_hints(incident, drift),
"not_used_reason": (
None
if records
else "no automation_operation_log row with Ansible operation type, tag, or executor backend for this source"
),
}
def _incident_public_dict(incident: Any) -> dict[str, Any]:
if incident is None:
return {}
if isinstance(incident, dict):
return incident
severity = getattr(incident, "severity", None)
signals_payload: list[dict[str, Any]] = []
for signal in getattr(incident, "signals", None) or []:
signals_payload.append({
"alert_name": getattr(signal, "alert_name", None),
"labels": getattr(signal, "labels", None) or {},
"annotations": getattr(signal, "annotations", None) or {},
})
return {
"incident_id": getattr(incident, "incident_id", None),
"project_id": getattr(incident, "project_id", None),
"alertname": getattr(incident, "alertname", None),
"alert_category": getattr(incident, "alert_category", None),
"notification_type": getattr(incident, "notification_type", None),
"severity": getattr(severity, "value", severity),
"affected_services": getattr(incident, "affected_services", None) or [],
"signals": signals_payload,
}
def build_ansible_decision_audit_payload(
*,
incident: Any,
proposal_data: dict[str, Any],
decision_path: str,
not_used_reason: str,
) -> dict[str, Any] | None:
"""Return an AOL payload when Ansible has catalog candidates for a decision."""
incident_payload = _incident_public_dict(incident)
hints = _catalog_hints(incident_payload, None)
candidates = hints.get("candidates") or []
if not candidates:
return None
incident_id = str(incident_payload.get("incident_id") or "")
input_payload = {
"incident_id": incident_id,
"executor": "ansible",
"execution_backend": "ansible",
"decision_path": decision_path,
"check_mode": True,
"apply_enabled": False,
"approval_required": True,
"candidate_catalog_schema": hints["match_mode"],
"executor_candidates": [
{
"catalog_id": row["catalog_id"],
"playbook_path": row["playbook_path"],
"check_mode_playbook_path": row.get("check_mode_playbook_path"),
"inventory_hosts": row["inventory_hosts"],
"supports_check_mode": row["supports_check_mode"],
"auto_apply_enabled": row["auto_apply_enabled"],
"approval_required": row["approval_required"],
"risk_level": row["risk_level"],
"match_score": row["match_score"],
"matched_keywords": row["matched_keywords"],
}
for row in candidates[:5]
],
"proposal_source": proposal_data.get("source", ""),
"proposal_risk_level": proposal_data.get("risk_level", ""),
"proposal_action_preview": str(
proposal_data.get("action")
or proposal_data.get("kubectl_command")
or ""
)[:240],
}
output_payload = {
"not_used_reason": not_used_reason,
"decision_effect": "audit_only",
"next_required_step": "wire approval_execution to Ansible check-mode before apply",
}
return {
"operation_type": "ansible_candidate_matched",
"status": "dry_run",
"input": input_payload,
"output": output_payload,
"dry_run_result": {
"check_mode_executed": False,
"candidate_count": len(candidates),
"reason": not_used_reason,
},
"tags": ["ansible", "decision", "candidate", "check_mode_pending"],
}
async def record_ansible_decision_audit(
*,
incident: Any,
proposal_data: dict[str, Any],
decision_path: str,
not_used_reason: str,
) -> bool:
"""Write a best-effort Ansible candidate audit row for one decision."""
payload = build_ansible_decision_audit_payload(
incident=incident,
proposal_data=proposal_data,
decision_path=decision_path,
not_used_reason=not_used_reason,
)
if payload is None:
return False
incident_id = payload["input"]["incident_id"]
project_id = getattr(incident, "project_id", None) or "awoooi"
try:
async with get_db_context(str(project_id)) as db:
existing = await db.execute(
text("""
SELECT op_id
FROM automation_operation_log
WHERE operation_type = 'ansible_candidate_matched'
AND coalesce(incident_id::text, input ->> 'incident_id') = :incident_id
AND input ->> 'executor' = 'ansible'
LIMIT 1
"""),
{"incident_id": incident_id},
)
if existing.scalar() is not None:
return False
await db.execute(
text("""
INSERT INTO automation_operation_log (
operation_type, actor, status, incident_id,
input, output, dry_run_result, tags
) VALUES (
:operation_type,
'decision_manager',
:status,
:incident_db_id,
CAST(:input AS jsonb),
CAST(:output AS jsonb),
CAST(:dry_run_result AS jsonb),
:tags
)
"""),
{
"operation_type": payload["operation_type"],
"status": payload["status"],
"incident_id": incident_id,
"incident_db_id": _automation_operation_log_incident_id(incident_id),
"input": json.dumps(payload["input"], ensure_ascii=False),
"output": json.dumps(payload["output"], ensure_ascii=False),
"dry_run_result": json.dumps(payload["dry_run_result"], ensure_ascii=False),
"tags": payload["tags"],
},
)
return True
except Exception as exc:
logger.warning(
"ansible_decision_audit_write_failed",
incident_id=incident_id,
error=str(exc),
)
return False

View File

@@ -0,0 +1,661 @@
"""Safe Ansible check-mode executor for AwoooP truth-chain evidence.
This service is deliberately dry-run only. It claims pending
``ansible_candidate_matched`` AOL rows, runs ``ansible-playbook --check --diff``,
and writes the result back as ``ansible_check_mode_executed``. It never enables
apply and never writes auto_repair_executions.
"""
from __future__ import annotations
import asyncio
import json
import os
import re
import shutil
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Any
import structlog
from sqlalchemy import text
from src.core.config import settings
from src.db.base import get_db_context
from src.services.awooop_ansible_audit_service import get_ansible_catalog_item
logger = structlog.get_logger(__name__)
_SAFE_HOST_RE = re.compile(r"^[A-Za-z0-9_.-]+$")
_PLAYBOOK_PREFIX = Path("infra/ansible/playbooks")
_STDOUT_LIMIT = 20_000
_STDERR_LIMIT = 12_000
FORCED_COMMAND_BLOCKER = "ansible_repair_ssh_forced_command_denies_ansible_bootstrap"
REPAIR_FORCED_COMMAND_KEY_PATH = Path("/etc/repair-ssh/id_ed25519")
REPAIR_FORCED_COMMAND_KNOWN_HOSTS_PATH = Path("/etc/repair-known-hosts/known_hosts")
@dataclass(frozen=True)
class AnsibleCheckModeClaim:
op_id: str
source_candidate_op_id: str
incident_id: str
catalog_id: str
playbook_path: str
inventory_hosts: tuple[str, ...]
input_payload: dict[str, Any]
@dataclass(frozen=True)
class AnsibleCommandSpec:
command: list[str]
cwd: Path
env: dict[str, str]
playbook_abs_path: Path
inventory_abs_path: Path
@dataclass(frozen=True)
class AnsibleRunResult:
returncode: int
stdout: str
stderr: str
duration_ms: int
timed_out: bool = False
def _tail(text_value: str, limit: int) -> str:
if len(text_value) <= limit:
return text_value
return text_value[-limit:]
def _json_loads(value: Any) -> dict[str, Any]:
if isinstance(value, dict):
return value
if isinstance(value, str):
try:
parsed = json.loads(value)
except json.JSONDecodeError:
return {}
return parsed if isinstance(parsed, dict) else {}
return {}
def _incident_id_from_payload(payload: dict[str, Any]) -> str:
return str(payload.get("incident_id") or "").strip()
def _automation_operation_log_incident_id(value: str) -> int | None:
# ADR-090 keeps external INC-* ids in input JSON; the DB column is BIGINT.
normalized = str(value or "").strip()
return int(normalized) if normalized.isdigit() else None
def _check_mode_ssh_key_path() -> Path:
return Path(settings.AWOOOP_ANSIBLE_CHECK_MODE_SSH_KEY_PATH)
def _check_mode_known_hosts_path() -> Path:
return Path(settings.AWOOOP_ANSIBLE_CHECK_MODE_KNOWN_HOSTS_PATH)
def _uses_repair_forced_command_transport(key_path: Path | None = None) -> bool:
return (key_path or _check_mode_ssh_key_path()) == REPAIR_FORCED_COMMAND_KEY_PATH
def detect_ansible_transport_blockers(*values: Any) -> list[str]:
combined = " ".join(str(value or "") for value in values)
blockers: list[str] = []
if "REPAIR_DENIED:invalid_command" in combined:
blockers.append(FORCED_COMMAND_BLOCKER)
return blockers
def _playbook_roots(module_path: Path | None = None) -> list[Path]:
resolved_module_path = (module_path or Path(__file__)).resolve()
return [
Path("/app/infra/ansible"),
Path.cwd() / "infra" / "ansible",
*(parent / "infra" / "ansible" for parent in resolved_module_path.parents),
]
def _runtime_blockers(
*,
playbook_root: Path | None = None,
check_mode_ssh_key_path: Path | None = None,
check_mode_known_hosts_path: Path | None = None,
) -> list[str]:
root = playbook_root or next((path for path in _playbook_roots() if path.exists()), None)
ssh_key_path = check_mode_ssh_key_path or _check_mode_ssh_key_path()
known_hosts_path = check_mode_known_hosts_path or _check_mode_known_hosts_path()
blockers: list[str] = []
if shutil.which("ansible-playbook") is None:
blockers.append("ansible_playbook_binary_missing")
if root is None:
blockers.append("ansible_playbook_catalog_missing")
elif not (root / "inventory" / "hosts.yml").exists():
blockers.append("ansible_inventory_missing")
if not ssh_key_path.is_file() or not os.access(ssh_key_path, os.R_OK):
blockers.append("ansible_check_mode_ssh_key_missing")
if not known_hosts_path.is_file() or not os.access(known_hosts_path, os.R_OK):
blockers.append("ansible_check_mode_known_hosts_missing")
return blockers
def _safe_candidate(input_payload: dict[str, Any]) -> dict[str, Any]:
candidates = input_payload.get("executor_candidates")
if not isinstance(candidates, list) or not candidates:
raise ValueError("missing_executor_candidates")
for candidate in candidates:
if not isinstance(candidate, dict):
continue
catalog_id = str(candidate.get("catalog_id") or "")
catalog_item = get_ansible_catalog_item(catalog_id)
if not catalog_item:
continue
if catalog_item.get("supports_check_mode") is not True:
continue
if catalog_item.get("auto_apply_enabled") is True:
continue
catalog_playbook_path = str(catalog_item.get("playbook_path") or "")
candidate_playbook_path = str(candidate.get("playbook_path") or catalog_playbook_path)
check_mode_playbook_path = str(
candidate.get("check_mode_playbook_path")
or catalog_item.get("check_mode_playbook_path")
or catalog_playbook_path
)
if candidate_playbook_path not in {catalog_playbook_path, check_mode_playbook_path}:
continue
inventory_hosts = candidate.get("inventory_hosts") or catalog_item.get("inventory_hosts") or []
if (
isinstance(inventory_hosts, list)
and inventory_hosts
and all(isinstance(host, str) and _SAFE_HOST_RE.fullmatch(host) for host in inventory_hosts)
):
return {
"catalog_id": catalog_id,
"playbook_path": check_mode_playbook_path,
"catalog_playbook_path": catalog_playbook_path,
"source_candidate_playbook_path": candidate_playbook_path,
"check_mode_playbook_path": check_mode_playbook_path,
"inventory_hosts": tuple(inventory_hosts),
"risk_level": str(candidate.get("risk_level") or catalog_item.get("risk_level") or ""),
}
raise ValueError("no_safe_check_mode_candidate")
def build_ansible_check_mode_claim_input(
*,
source_candidate_op_id: str,
candidate_input: dict[str, Any],
) -> dict[str, Any]:
safe = _safe_candidate(candidate_input)
incident_id = _incident_id_from_payload(candidate_input)
return {
"incident_id": incident_id,
"executor": "ansible",
"execution_backend": "ansible",
"execution_mode": "check_mode",
"transport_profile": settings.AWOOOP_ANSIBLE_CHECK_MODE_TRANSPORT_PROFILE,
"check_mode": True,
"diff": True,
"apply_enabled": False,
"approval_required_before_apply": True,
"source_candidate_op_id": source_candidate_op_id,
"catalog_id": safe["catalog_id"],
"playbook_path": safe["playbook_path"],
"catalog_playbook_path": safe["catalog_playbook_path"],
"source_candidate_playbook_path": safe["source_candidate_playbook_path"],
"check_mode_playbook_path": safe["check_mode_playbook_path"],
"inventory_hosts": list(safe["inventory_hosts"]),
"risk_level": safe["risk_level"],
}
def _resolve_playbook_path(playbook_root: Path, playbook_path: str) -> Path:
relative = Path(playbook_path)
if relative.is_absolute() or not str(relative).startswith(str(_PLAYBOOK_PREFIX)):
raise ValueError("unsafe_playbook_path")
repo_root = playbook_root.parent.parent
resolved = (repo_root / relative).resolve()
allowed_root = (repo_root / _PLAYBOOK_PREFIX).resolve()
if allowed_root not in resolved.parents:
raise ValueError("playbook_outside_catalog")
if resolved.suffix not in {".yml", ".yaml"} or not resolved.exists():
raise ValueError("playbook_not_found")
return resolved
def build_ansible_check_mode_command(
*,
playbook_path: str,
inventory_hosts: tuple[str, ...],
playbook_root: Path | None = None,
check_mode_ssh_key_path: Path | None = None,
check_mode_known_hosts_path: Path | None = None,
) -> AnsibleCommandSpec:
root = playbook_root or next((path for path in _playbook_roots() if path.exists()), None)
if root is None:
raise ValueError("ansible_playbook_catalog_missing")
inventory_path = (root / "inventory" / "hosts.yml").resolve()
if not inventory_path.exists():
raise ValueError("ansible_inventory_missing")
if not inventory_hosts or not all(_SAFE_HOST_RE.fullmatch(host) for host in inventory_hosts):
raise ValueError("unsafe_inventory_hosts")
playbook_abs = _resolve_playbook_path(root, playbook_path)
ssh_key_path = check_mode_ssh_key_path or _check_mode_ssh_key_path()
known_hosts_path = check_mode_known_hosts_path or _check_mode_known_hosts_path()
ssh_common_args = (
f"-o UserKnownHostsFile={known_hosts_path} "
"-o IdentitiesOnly=yes -o BatchMode=yes"
)
extra_vars = {
"ansible_ssh_private_key_file": str(ssh_key_path),
"ansible_ssh_common_args": ssh_common_args,
}
command = [
"ansible-playbook",
"-i",
str(inventory_path),
str(playbook_abs),
"--check",
"--diff",
"--limit",
",".join(inventory_hosts),
"--extra-vars",
json.dumps(extra_vars, ensure_ascii=False, separators=(",", ":")),
]
env = {
**os.environ,
"ANSIBLE_HOST_KEY_CHECKING": "true",
"ANSIBLE_RETRY_FILES_ENABLED": "false",
}
return AnsibleCommandSpec(
command=command,
cwd=root,
env=env,
playbook_abs_path=playbook_abs,
inventory_abs_path=inventory_path,
)
async def _run_ansible_command(spec: AnsibleCommandSpec, *, timeout_seconds: int) -> AnsibleRunResult:
started = time.monotonic()
process = await asyncio.create_subprocess_exec(
*spec.command,
cwd=str(spec.cwd),
env=spec.env,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
timed_out = False
try:
stdout_bytes, stderr_bytes = await asyncio.wait_for(
process.communicate(),
timeout=timeout_seconds,
)
except TimeoutError:
timed_out = True
process.kill()
stdout_bytes, stderr_bytes = await process.communicate()
duration_ms = int((time.monotonic() - started) * 1000)
return AnsibleRunResult(
returncode=124 if timed_out else int(process.returncode or 0),
stdout=stdout_bytes.decode("utf-8", "replace"),
stderr=stderr_bytes.decode("utf-8", "replace"),
duration_ms=duration_ms,
timed_out=timed_out,
)
def _build_result_payload(result: AnsibleRunResult) -> tuple[str, dict[str, Any], dict[str, Any], str | None]:
status = "success" if result.returncode == 0 else "failed"
stdout_tail = _tail(result.stdout, _STDOUT_LIMIT)
stderr_tail = _tail(result.stderr, _STDERR_LIMIT)
output = {
"executor": "ansible",
"execution_mode": "check_mode",
"transport_profile": settings.AWOOOP_ANSIBLE_CHECK_MODE_TRANSPORT_PROFILE,
"ssh_key_path": settings.AWOOOP_ANSIBLE_CHECK_MODE_SSH_KEY_PATH,
"known_hosts_path": settings.AWOOOP_ANSIBLE_CHECK_MODE_KNOWN_HOSTS_PATH,
"check_mode": True,
"apply_enabled": False,
"approval_required_before_apply": True,
"returncode": result.returncode,
"timed_out": result.timed_out,
"stdout_tail": stdout_tail,
"stderr_tail": stderr_tail,
"next_required_step": "approval_required_before_ansible_apply",
}
dry_run_result = {
"check_mode_executed": True,
"apply_executed": False,
"safe_to_apply_without_approval": False,
"transport_profile": settings.AWOOOP_ANSIBLE_CHECK_MODE_TRANSPORT_PROFILE,
"ssh_key_path": settings.AWOOOP_ANSIBLE_CHECK_MODE_SSH_KEY_PATH,
"known_hosts_path": settings.AWOOOP_ANSIBLE_CHECK_MODE_KNOWN_HOSTS_PATH,
"returncode": result.returncode,
"timed_out": result.timed_out,
"stdout_tail": stdout_tail,
"stderr_tail": stderr_tail,
}
error = None if result.returncode == 0 else (stderr_tail or f"ansible_check_mode_failed_rc_{result.returncode}")
return status, output, dry_run_result, error
async def claim_pending_check_modes(
*,
project_id: str = "awoooi",
limit: int = 1,
candidate_max_age_hours: int | None = None,
) -> list[AnsibleCheckModeClaim]:
"""Claim pending Ansible candidates by inserting pending check-mode rows."""
claims: list[AnsibleCheckModeClaim] = []
max_age_hours = candidate_max_age_hours or settings.AWOOOP_ANSIBLE_CHECK_MODE_CANDIDATE_MAX_AGE_HOURS
async with get_db_context(project_id) as db:
result = await db.execute(
text("""
SELECT
candidate.op_id,
candidate.input
FROM automation_operation_log candidate
WHERE candidate.operation_type = 'ansible_candidate_matched'
AND candidate.status = 'dry_run'
AND candidate.input ->> 'executor' = 'ansible'
AND candidate.created_at >= NOW() - (:candidate_max_age_hours * INTERVAL '1 hour')
AND COALESCE((candidate.dry_run_result ->> 'check_mode_executed')::boolean, false) = false
AND NOT EXISTS (
SELECT 1
FROM automation_operation_log existing
WHERE existing.parent_op_id = candidate.op_id
AND existing.operation_type IN (
'ansible_check_mode_executed',
'ansible_execution_skipped'
)
)
ORDER BY candidate.created_at ASC
LIMIT :limit
FOR UPDATE SKIP LOCKED
"""),
{
"limit": max(1, limit),
"candidate_max_age_hours": max(1, max_age_hours),
},
)
rows = result.mappings().all()
for row in rows:
source_op_id = str(row["op_id"])
candidate_input = _json_loads(row["input"])
try:
claim_input = build_ansible_check_mode_claim_input(
source_candidate_op_id=source_op_id,
candidate_input=candidate_input,
)
except ValueError as exc:
await _insert_skipped_candidate(
db,
source_candidate_op_id=source_op_id,
candidate_input=candidate_input,
reason=str(exc),
)
continue
inserted = await db.execute(
text("""
INSERT INTO automation_operation_log (
operation_type, actor, status, incident_id,
input, output, dry_run_result,
parent_op_id, tags
) VALUES (
'ansible_check_mode_executed',
'ansible_check_mode_worker',
'pending',
:incident_db_id,
CAST(:input AS jsonb),
'{}'::jsonb,
CAST(:dry_run_result AS jsonb),
CAST(:parent_op_id AS uuid),
:tags
)
RETURNING op_id
"""),
{
"incident_id": _incident_id_from_payload(claim_input),
"incident_db_id": _automation_operation_log_incident_id(
_incident_id_from_payload(claim_input)
),
"input": json.dumps(claim_input, ensure_ascii=False),
"dry_run_result": json.dumps({
"check_mode_executed": False,
"apply_executed": False,
"claim_state": "claimed",
}, ensure_ascii=False),
"parent_op_id": source_op_id,
"tags": ["ansible", "check_mode", "pending", "apply_locked"],
},
)
op_id = str(inserted.scalar_one())
claims.append(
AnsibleCheckModeClaim(
op_id=op_id,
source_candidate_op_id=source_op_id,
incident_id=str(claim_input.get("incident_id") or ""),
catalog_id=str(claim_input["catalog_id"]),
playbook_path=str(claim_input["playbook_path"]),
inventory_hosts=tuple(str(host) for host in claim_input["inventory_hosts"]),
input_payload=claim_input,
)
)
return claims
async def recent_ansible_transport_blockers(
*,
project_id: str = "awoooi",
cooldown_seconds: int | None = None,
include_repair_forced_command_blocker: bool | None = None,
) -> list[str]:
"""Return transport blockers observed from recent failed check-mode rows."""
cooldown = cooldown_seconds or settings.AWOOOP_ANSIBLE_CHECK_MODE_TRANSPORT_COOLDOWN_SECONDS
include_forced_blocker = (
_uses_repair_forced_command_transport()
if include_repair_forced_command_blocker is None
else include_repair_forced_command_blocker
)
async with get_db_context(project_id) as db:
result = await db.execute(
text("""
SELECT
coalesce(output::text, '') AS output_text,
coalesce(dry_run_result::text, '') AS dry_run_text,
coalesce(error, '') AS error_text,
coalesce(stderr_feed_back, '') AS stderr_text
FROM automation_operation_log
WHERE operation_type = 'ansible_check_mode_executed'
AND status = 'failed'
AND created_at >= NOW() - (:cooldown_seconds * INTERVAL '1 second')
ORDER BY created_at DESC
LIMIT 20
"""),
{"cooldown_seconds": max(60, cooldown)},
)
blockers: set[str] = set()
for row in result.mappings().all():
detected = detect_ansible_transport_blockers(
row.get("output_text"),
row.get("dry_run_text"),
row.get("error_text"),
row.get("stderr_text"),
)
blockers.update(
blocker
for blocker in detected
if blocker != FORCED_COMMAND_BLOCKER or include_forced_blocker
)
return sorted(blockers)
async def _insert_skipped_candidate(
db: Any,
*,
source_candidate_op_id: str,
candidate_input: dict[str, Any],
reason: str,
) -> None:
input_payload = {
"incident_id": _incident_id_from_payload(candidate_input),
"executor": "ansible",
"execution_backend": "ansible",
"execution_mode": "check_mode",
"transport_profile": settings.AWOOOP_ANSIBLE_CHECK_MODE_TRANSPORT_PROFILE,
"check_mode": True,
"apply_enabled": False,
"source_candidate_op_id": source_candidate_op_id,
"not_used_reason": reason,
}
await db.execute(
text("""
INSERT INTO automation_operation_log (
operation_type, actor, status, incident_id,
input, output, dry_run_result,
parent_op_id, tags
) VALUES (
'ansible_execution_skipped',
'ansible_check_mode_worker',
'dry_run',
:incident_db_id,
CAST(:input AS jsonb),
CAST(:output AS jsonb),
CAST(:dry_run_result AS jsonb),
CAST(:parent_op_id AS uuid),
:tags
)
"""),
{
"incident_id": _incident_id_from_payload(input_payload),
"incident_db_id": _automation_operation_log_incident_id(
_incident_id_from_payload(input_payload)
),
"input": json.dumps(input_payload, ensure_ascii=False),
"output": json.dumps({
"not_used_reason": reason,
"decision_effect": "skipped_before_runtime",
}, ensure_ascii=False),
"dry_run_result": json.dumps({
"check_mode_executed": False,
"apply_executed": False,
"skipped": True,
"reason": reason,
}, ensure_ascii=False),
"parent_op_id": source_candidate_op_id,
"tags": ["ansible", "check_mode", "skipped", "apply_locked"],
},
)
async def finalize_check_mode_claim(
claim: AnsibleCheckModeClaim,
result: AnsibleRunResult,
*,
project_id: str = "awoooi",
) -> None:
status, output, dry_run_result, error = _build_result_payload(result)
async with get_db_context(project_id) as db:
await db.execute(
text("""
UPDATE automation_operation_log
SET status = :status,
output = CAST(:output AS jsonb),
dry_run_result = CAST(:dry_run_result AS jsonb),
error = :error,
duration_ms = :duration_ms,
stderr_feed_back = :stderr
WHERE op_id = CAST(:op_id AS uuid)
"""),
{
"status": status,
"output": json.dumps(output, ensure_ascii=False),
"dry_run_result": json.dumps(dry_run_result, ensure_ascii=False),
"error": _tail(error or "", 2000) or None,
"duration_ms": result.duration_ms,
"stderr": _tail(result.stderr, _STDERR_LIMIT),
"op_id": claim.op_id,
},
)
async def run_claimed_check_mode(
claim: AnsibleCheckModeClaim,
*,
timeout_seconds: int,
project_id: str = "awoooi",
) -> AnsibleRunResult:
try:
spec = build_ansible_check_mode_command(
playbook_path=claim.playbook_path,
inventory_hosts=claim.inventory_hosts,
)
result = await _run_ansible_command(spec, timeout_seconds=timeout_seconds)
except Exception as exc:
result = AnsibleRunResult(
returncode=1,
stdout="",
stderr=f"ansible_check_mode_runtime_error: {exc}",
duration_ms=0,
)
await finalize_check_mode_claim(claim, result, project_id=project_id)
logger.info(
"ansible_check_mode_claim_completed",
op_id=claim.op_id,
source_candidate_op_id=claim.source_candidate_op_id,
incident_id=claim.incident_id,
catalog_id=claim.catalog_id,
returncode=result.returncode,
timed_out=result.timed_out,
)
return result
async def run_pending_check_modes_once(
*,
project_id: str = "awoooi",
limit: int = 1,
timeout_seconds: int | None = None,
) -> dict[str, Any]:
blockers = _runtime_blockers()
if blockers:
logger.warning("ansible_check_mode_runtime_blocked", blockers=blockers)
return {"claimed": 0, "completed": 0, "failed": 0, "blockers": blockers}
transport_blockers = await recent_ansible_transport_blockers(project_id=project_id)
if transport_blockers:
logger.warning("ansible_check_mode_transport_blocked", blockers=transport_blockers)
return {"claimed": 0, "completed": 0, "failed": 0, "blockers": transport_blockers}
claims = await claim_pending_check_modes(
project_id=project_id,
limit=limit,
candidate_max_age_hours=settings.AWOOOP_ANSIBLE_CHECK_MODE_CANDIDATE_MAX_AGE_HOURS,
)
completed = 0
failed = 0
for claim in claims:
result = await run_claimed_check_mode(
claim,
timeout_seconds=timeout_seconds or settings.AWOOOP_ANSIBLE_CHECK_MODE_TIMEOUT_SECONDS,
project_id=project_id,
)
completed += 1
if result.returncode != 0:
failed += 1
return {
"claimed": len(claims),
"completed": completed,
"failed": failed,
"blockers": [],
}

View File

@@ -0,0 +1,63 @@
"""Shared AwoooP operator deeplinks used by Telegram-facing services."""
from urllib.parse import quote
AWOOOP_WEB_BASE_URL = "https://awoooi.wooo.work"
def incident_runs_url(
incident_id: str,
*,
project_id: str = "awoooi",
locale: str = "zh-TW",
) -> str:
safe_project_id = quote(str(project_id or "awoooi"), safe="")
safe_incident_id = quote(str(incident_id or ""), safe="")
return (
f"{AWOOOP_WEB_BASE_URL}/{locale}/awooop/runs"
f"?project_id={safe_project_id}&incident_id={safe_incident_id}"
)
def incident_alerts_url(
incident_id: str,
*,
project_id: str = "awoooi",
locale: str = "zh-TW",
) -> str:
safe_project_id = quote(str(project_id or "awoooi"), safe="")
safe_incident_id = quote(str(incident_id or ""), safe="")
return (
f"{AWOOOP_WEB_BASE_URL}/{locale}/alerts"
f"?project_id={safe_project_id}&incident_id={safe_incident_id}"
)
def incident_truth_chain_button_row(
incident_id: str,
*,
project_id: str = "awoooi",
) -> list[dict[str, str]]:
if not incident_id:
return []
return [
{
"text": "🔎 真相鏈",
"url": incident_alerts_url(incident_id, project_id=project_id),
},
{
"text": "🧭 Runs",
"url": incident_runs_url(incident_id, project_id=project_id),
},
]
def incident_truth_chain_reply_markup(
incident_id: str,
*,
project_id: str = "awoooi",
) -> dict | None:
row = incident_truth_chain_button_row(incident_id, project_id=project_id)
if not row:
return None
return {"inline_keyboard": [row]}

File diff suppressed because it is too large Load Diff

View File

@@ -336,7 +336,7 @@ async def _get_tenant_budget_limit(project_id: str) -> Decimal | None:
try:
from sqlalchemy import text
from src.db.base import get_db_context
async with get_db_context() as db:
async with get_db_context(project_id) as db:
row = await db.execute(
text("SELECT budget_limit_usd FROM awooop_projects WHERE project_id = :pid"),
{"pid": project_id},

File diff suppressed because it is too large Load Diff

View File

@@ -30,6 +30,7 @@ import asyncio
import hashlib
import html
import json
import re
from datetime import UTC, datetime
from typing import Any
from uuid import NAMESPACE_URL, UUID, uuid5
@@ -39,13 +40,95 @@ from sqlalchemy import select, text
from sqlalchemy.ext.asyncio import AsyncSession
from src.db.awooop_models import AwoooPRunState
from src.services.audit_sink import _redact_string
from src.services.audit_sink import _redact_string, sanitize
from src.services.platform_runtime import create_run
logger = structlog.get_logger(__name__)
# Progressive Feedback Policy等待超過此秒數才發 interim 訊息
_INTERIM_WAIT_SECONDS = 30
_INBOUND_REDACTION_VERSION = "audit_sink_v1"
_OUTBOUND_REDACTION_VERSION = "audit_sink_v1"
_INCIDENT_ID_RE = re.compile(r"\bINC-\d{8}-[A-Z0-9]{6}\b")
def _db_timestamp_now() -> datetime:
"""Return UTC now in the timestamp shape accepted by the production DB path."""
return datetime.now(UTC).replace(tzinfo=None)
def _compact_unique(values: list[str | None], *, limit: int = 20) -> list[str]:
"""Return stable non-empty values without leaking duplicate source refs."""
return sorted({str(value).strip() for value in values if str(value or "").strip()})[:limit]
def build_inbound_source_envelope(
*,
provider: str,
stage: str,
provider_event_id: str,
raw_event_id: str | None = None,
raw_content: str | None = None,
alertname: str | None = None,
severity: str | None = None,
namespace: str | None = None,
target_resource: str | None = None,
fingerprint: str | None = None,
incident_id: str | None = None,
approval_id: str | None = None,
source_url: str | None = None,
labels: dict[str, Any] | None = None,
annotations: dict[str, Any] | None = None,
extra: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Build a redaction-friendly inbound replay envelope for truth-chain use."""
content_sha256 = hashlib.sha256(raw_content.encode()).hexdigest() if raw_content else None
text_refs = _INCIDENT_ID_RE.findall(raw_content or "")
provider_name = str(provider or "unknown").strip().lower() or "unknown"
is_provider_heartbeat = str(stage or "").strip().lower() == "heartbeat"
source_refs = {
"event_ids": _compact_unique([raw_event_id]),
"incident_ids": _compact_unique([incident_id, *text_refs]),
"approval_ids": _compact_unique([approval_id]),
"alert_ids": _compact_unique([provider_event_id, raw_event_id]),
"fingerprints": _compact_unique([fingerprint]),
"sentry_issue_ids": _compact_unique(
[raw_event_id, provider_event_id]
if provider_name == "sentry" and not is_provider_heartbeat
else []
),
"signoz_alerts": _compact_unique(
[raw_event_id, alertname]
if provider_name == "signoz" and not is_provider_heartbeat
else []
),
}
envelope: dict[str, Any] = {
"schema_version": "inbound_source_envelope_v1",
"redaction_version": _INBOUND_REDACTION_VERSION,
"adapter": f"{provider_name}_webhook",
"provider": provider_name,
"stage": stage,
"provider_event_id": provider_event_id,
"source_url": source_url,
"content_sha256": content_sha256,
"content_length": len(raw_content) if raw_content is not None else 0,
"source_refs": source_refs,
"log_correlation": {
"alertname": alertname,
"severity": severity,
"namespace": namespace,
"target_resource": target_resource,
"fingerprint": fingerprint,
},
"labels": labels or {},
"annotations": annotations or {},
}
if extra:
envelope["extra"] = extra
sanitized = sanitize(envelope)
sanitized["content_sha256"] = content_sha256
return sanitized
def _input_sha256(input_payload: dict[str, Any] | None) -> str | None:
@@ -82,11 +165,15 @@ async def ensure_completed_shadow_run(
INSERT INTO awooop_run_state (
run_id, project_id, agent_id, state,
trigger_type, trigger_ref, is_shadow,
input_sha256, created_at, completed_at, timeout_at
input_sha256,
attempt_count, max_attempts, cost_usd, step_count,
created_at, completed_at, timeout_at
) VALUES (
:run_id, :project_id, :agent_id, 'completed',
:trigger_type, :trigger_ref, TRUE,
:input_sha256, NOW(), NOW(), NOW()
:input_sha256,
0, 3, 0.0000, 0,
NOW(), NOW(), NOW()
)
ON CONFLICT (run_id) DO NOTHING
RETURNING run_id
@@ -117,6 +204,32 @@ def build_grouped_alert_run_id(project_id: str, provider_event_id: str) -> UUID:
return uuid5(NAMESPACE_URL, f"awooop:grouped-alert:{project_id}:{provider_event_id}")
def build_alertmanager_provider_event_id(alert_id: str, fingerprint: str, stage: str) -> str:
"""建立 Alertmanager inbound event 的冪等 provider_event_id。"""
safe_alert_id = str(alert_id).strip() or "unknown"
safe_fingerprint = str(fingerprint).strip()[:32] or "no-fingerprint"
safe_stage = str(stage).strip()[:32] or "received"
return f"alertmanager:{safe_stage}:{safe_alert_id}:{safe_fingerprint}"
def build_alertmanager_run_id(project_id: str, provider_event_id: str) -> UUID:
"""為 Alertmanager inbound mirror 建立穩定 shadow run_id。"""
return uuid5(NAMESPACE_URL, f"awooop:alertmanager:{project_id}:{provider_event_id}")
def build_external_alert_provider_event_id(provider: str, event_id: str, stage: str) -> str:
"""建立 Sentry/SignOz 等外部告警 inbound event 的冪等 provider_event_id。"""
safe_provider = str(provider).strip().lower()[:32] or "external"
safe_event_id = str(event_id).strip()[:96] or "unknown"
safe_stage = str(stage).strip()[:32] or "received"
return f"{safe_provider}:{safe_stage}:{safe_event_id}"
def build_external_alert_run_id(project_id: str, provider_event_id: str) -> UUID:
"""為外部告警 inbound mirror 建立穩定 shadow run_id。"""
return uuid5(NAMESPACE_URL, f"awooop:external-alert:{project_id}:{provider_event_id}")
# ─────────────────────────────────────────────────────────────────────────────
# 入站事件記錄
# ─────────────────────────────────────────────────────────────────────────────
@@ -132,6 +245,7 @@ async def mirror_inbound_event(
channel_chat_id: str | None = None,
content_type: str = "text",
raw_content: str | None = None,
source_envelope: dict[str, Any] | None = None,
attachment_sha256: str | None = None,
provider_ts: datetime | None = None,
run_id: UUID | None = None,
@@ -145,12 +259,32 @@ async def mirror_inbound_event(
"""
content_hash: str | None = None
content_preview: str | None = None
content_redacted: str | None = None
if raw_content is not None:
content_hash = hashlib.sha256(raw_content.encode()).hexdigest()
# previewredact 後截取前 256 字元
redacted = _redact_string(raw_content)
content_preview = redacted[:256] if len(redacted) > 256 else redacted
content_redacted = _redact_string(raw_content)
content_preview = (
content_redacted[:256] if len(content_redacted) > 256 else content_redacted
)
if source_envelope and source_envelope.get("schema_version") == "inbound_source_envelope_v1":
original_content_sha256 = source_envelope.get("content_sha256")
envelope = sanitize(source_envelope)
envelope.setdefault("redaction_version", _INBOUND_REDACTION_VERSION)
envelope["content_sha256"] = content_hash or original_content_sha256
envelope.setdefault("content_length", len(raw_content) if raw_content is not None else 0)
else:
envelope = build_inbound_source_envelope(
provider=channel_type,
stage="received",
provider_event_id=provider_event_id,
raw_event_id=provider_event_id,
raw_content=raw_content,
extra=source_envelope,
)
source_envelope_json = json.dumps(envelope, ensure_ascii=False, default=str)
result = await db.execute(
text("""
@@ -158,16 +292,28 @@ async def mirror_inbound_event(
project_id, channel_type, provider_event_id,
platform_subject_id, channel_user_id, channel_chat_id,
run_id, content_type, content_hash, content_preview,
content_redacted, redaction_version, source_envelope,
attachment_sha256, is_duplicate, provider_ts, received_at
) VALUES (
:project_id, :channel_type, :provider_event_id,
:platform_subject_id, :channel_user_id, :channel_chat_id,
:run_id, :content_type, :content_hash, :content_preview,
:content_redacted, :redaction_version, CAST(:source_envelope AS jsonb),
:attachment_sha256, :is_duplicate, :provider_ts, NOW()
)
ON CONFLICT (project_id, channel_type, provider_event_id) DO UPDATE SET
is_duplicate = TRUE,
run_id = COALESCE(EXCLUDED.run_id, awooop_conversation_event.run_id)
run_id = COALESCE(EXCLUDED.run_id, awooop_conversation_event.run_id),
content_redacted = COALESCE(
awooop_conversation_event.content_redacted,
EXCLUDED.content_redacted
),
redaction_version = EXCLUDED.redaction_version,
source_envelope = CASE
WHEN awooop_conversation_event.source_envelope = '{}'::jsonb
THEN EXCLUDED.source_envelope
ELSE awooop_conversation_event.source_envelope
END
RETURNING event_id
"""),
{
@@ -181,6 +327,9 @@ async def mirror_inbound_event(
"content_type": content_type,
"content_hash": content_hash,
"content_preview": content_preview,
"content_redacted": content_redacted,
"redaction_version": _INBOUND_REDACTION_VERSION,
"source_envelope": source_envelope_json,
"attachment_sha256": attachment_sha256,
"is_duplicate": is_duplicate,
"provider_ts": provider_ts,
@@ -205,6 +354,41 @@ def build_grouped_alert_provider_event_id(alert_id: str, fingerprint: str) -> st
return f"alert-group:{safe_alert_id}:{safe_fingerprint}"
def format_alertmanager_event_content(
*,
stage: str,
alert_id: str,
alertname: str,
severity: str,
namespace: str,
target_resource: str,
fingerprint: str,
notification_type: str | None = None,
alert_category: str | None = None,
incident_id: str | None = None,
approval_id: str | None = None,
repeat_count: int | None = None,
) -> str:
"""格式化 Alertmanager inbound mirror 摘要,讓 truth-chain 可回查。"""
head = f"Incident: {incident_id}" if incident_id else f"Fingerprint: {fingerprint}"
return "\n".join(
[
f"Alertmanager inbound {stage}",
head,
f"Alert ID: {alert_id}",
f"Approval: {approval_id or '-'}",
f"Alert: {alertname}",
f"Severity: {severity}",
f"Namespace: {namespace or 'default'}",
f"Target: {target_resource or '-'}",
f"Fingerprint: {fingerprint}",
f"Notification Type: {notification_type or '-'}",
f"Alert Category: {alert_category or '-'}",
f"Repeat Count: {repeat_count if repeat_count is not None else '-'}",
]
)
def format_grouped_alert_event_content(
*,
alert_id: str,
@@ -406,7 +590,7 @@ async def record_grouped_alert_event(
channel_chat_id=f"alert-group:{group_key}",
content_type="text",
raw_content=content,
provider_ts=datetime.now(UTC),
provider_ts=_db_timestamp_now(),
run_id=run_id,
)
@@ -440,6 +624,267 @@ async def record_grouped_alert_event(
return None
async def record_alertmanager_event(
*,
project_id: str,
alert_id: str,
alertname: str,
severity: str,
namespace: str,
target_resource: str,
fingerprint: str,
stage: str,
notification_type: str | None = None,
alert_category: str | None = None,
incident_id: str | None = None,
approval_id: str | None = None,
repeat_count: int | None = None,
is_duplicate: bool = False,
source_url: str | None = None,
labels: dict[str, Any] | None = None,
annotations: dict[str, Any] | None = None,
source_extra: dict[str, Any] | None = None,
) -> UUID | None:
"""
將 Alertmanager inbound alert 鏡像到 AwoooP conversation_event。
Telegram 不應是唯一事實來源;每個 firing alert 至少要有 received
event建立 incident/approval 後再補 incident_linked event 供 truth-chain
依 incident_id 回查。DB 失敗 fail-open不影響 Alertmanager ACK。
"""
try:
from src.db.base import get_db_context
incident_ref = str(incident_id) if incident_id else None
approval_ref = str(approval_id) if approval_id else None
provider_event_id = build_alertmanager_provider_event_id(
alert_id=alert_id,
fingerprint=fingerprint,
stage=stage,
)
content = format_alertmanager_event_content(
stage=stage,
alert_id=alert_id,
alertname=alertname,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
notification_type=notification_type,
alert_category=alert_category,
incident_id=incident_ref,
approval_id=approval_ref,
repeat_count=repeat_count,
)
source_envelope = build_inbound_source_envelope(
provider="alertmanager",
stage=stage,
provider_event_id=provider_event_id,
raw_event_id=alert_id,
raw_content=content,
alertname=alertname,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
incident_id=incident_ref,
approval_id=approval_ref,
source_url=source_url,
labels=labels,
annotations=annotations,
extra={
"notification_type": notification_type,
"alert_category": alert_category,
"repeat_count": repeat_count,
**(source_extra or {}),
},
)
async with get_db_context(project_id) as db:
run_id = build_alertmanager_run_id(project_id, provider_event_id)
await ensure_completed_shadow_run(
db,
project_id=project_id,
run_id=run_id,
agent_id="legacy-alertmanager-webhook",
trigger_type="alertmanager_inbound",
trigger_ref=provider_event_id,
input_payload={
"stage": stage,
"alert_id": alert_id,
"alertname": alertname,
"severity": severity,
"namespace": namespace,
"target_resource": target_resource,
"fingerprint": fingerprint,
"notification_type": notification_type,
"alert_category": alert_category,
"incident_id": incident_ref,
"approval_id": approval_ref,
"repeat_count": repeat_count,
},
)
event_id = await mirror_inbound_event(
db,
project_id=project_id,
channel_type="internal",
provider_event_id=provider_event_id,
platform_subject_id="alertmanager",
channel_user_id="alertmanager",
channel_chat_id=f"alertmanager:{namespace or 'default'}",
content_type="text",
raw_content=content,
source_envelope=source_envelope,
provider_ts=_db_timestamp_now(),
run_id=run_id,
is_duplicate=is_duplicate,
)
logger.info(
"alertmanager_event_recorded",
project_id=project_id,
alert_id=alert_id,
event_id=str(event_id),
stage=stage,
incident_id=incident_ref,
fingerprint=fingerprint,
)
return event_id
except Exception as exc:
logger.warning(
"alertmanager_event_record_failed",
project_id=project_id,
alert_id=alert_id,
stage=stage,
fingerprint=fingerprint,
error=str(exc),
)
return None
async def record_external_alert_event(
*,
project_id: str,
provider: str,
event_id: str,
stage: str,
title: str,
severity: str,
namespace: str | None = None,
target_resource: str | None = None,
fingerprint: str | None = None,
incident_id: str | None = None,
approval_id: str | None = None,
source_url: str | None = None,
labels: dict[str, Any] | None = None,
annotations: dict[str, Any] | None = None,
payload: dict[str, Any] | None = None,
is_duplicate: bool = False,
) -> UUID | None:
"""
將 Sentry / SignOz 等非 Alertmanager 告警鏡像到 conversation_event。
這是 truth-chain 的最低共用入口:只寫 redacted content + source_envelope
不改變原本 webhook 的通知、審批或自動化行為。
"""
provider_name = str(provider or "external").strip().lower() or "external"
provider_event_id = build_external_alert_provider_event_id(provider_name, event_id, stage)
content = "\n".join([
f"{provider_name} inbound {stage}",
f"Event ID: {event_id}",
f"Title: {title}",
f"Severity: {severity}",
f"Namespace: {namespace or '-'}",
f"Target: {target_resource or '-'}",
f"Fingerprint: {fingerprint or '-'}",
f"Incident: {incident_id or '-'}",
f"Approval: {approval_id or '-'}",
f"Source URL: {source_url or '-'}",
])
source_envelope = build_inbound_source_envelope(
provider=provider_name,
stage=stage,
provider_event_id=provider_event_id,
raw_event_id=event_id,
raw_content=content,
alertname=title,
severity=severity,
namespace=namespace,
target_resource=target_resource,
fingerprint=fingerprint,
incident_id=str(incident_id) if incident_id else None,
approval_id=str(approval_id) if approval_id else None,
source_url=source_url,
labels=labels,
annotations=annotations,
extra={
"payload": payload or {},
},
)
try:
from src.db.base import get_db_context
async with get_db_context(project_id) as db:
run_id = build_external_alert_run_id(project_id, provider_event_id)
await ensure_completed_shadow_run(
db,
project_id=project_id,
run_id=run_id,
agent_id=f"legacy-{provider_name}-webhook",
trigger_type=f"{provider_name}_inbound",
trigger_ref=provider_event_id,
input_payload={
"provider": provider_name,
"event_id": event_id,
"stage": stage,
"severity": severity,
"namespace": namespace,
"target_resource": target_resource,
"fingerprint": fingerprint,
"incident_id": str(incident_id) if incident_id else None,
"approval_id": str(approval_id) if approval_id else None,
},
)
event_uuid = await mirror_inbound_event(
db,
project_id=project_id,
channel_type="internal",
provider_event_id=provider_event_id,
platform_subject_id=provider_name,
channel_user_id=provider_name,
channel_chat_id=f"{provider_name}:{namespace or 'default'}",
content_type="text",
raw_content=content,
source_envelope=source_envelope,
provider_ts=_db_timestamp_now(),
run_id=run_id,
is_duplicate=is_duplicate,
)
logger.info(
"external_alert_event_recorded",
project_id=project_id,
provider=provider_name,
event_id=event_id,
stage=stage,
conversation_event_id=str(event_uuid),
incident_id=str(incident_id) if incident_id else None,
approval_id=str(approval_id) if approval_id else None,
)
return event_uuid
except Exception as exc:
logger.warning(
"external_alert_event_record_failed",
project_id=project_id,
provider=provider_name,
event_id=event_id,
stage=stage,
error=str(exc),
)
return None
# ─────────────────────────────────────────────────────────────────────────────
# 出站訊息記錄
# ─────────────────────────────────────────────────────────────────────────────
@@ -453,6 +898,7 @@ async def record_outbound_message(
channel_chat_id: str,
message_type: str, # 'interim' | 'final' | 'error' | 'approval_request'
content: str | None = None,
source_envelope: dict[str, Any] | None = None,
provider_message_id: str | None = None,
send_status: str = "pending",
conversation_event_id: UUID | None = None,
@@ -467,12 +913,27 @@ async def record_outbound_message(
"""
content_hash: str | None = None
content_preview: str | None = None
content_redacted: str | None = None
if content is not None:
content_hash = hashlib.sha256(content.encode()).hexdigest()
redacted = _redact_string(content)
content_preview = redacted[:256]
content_redacted = _redact_string(content)
content_preview = content_redacted[:256]
envelope: dict[str, Any] = sanitize(source_envelope or {})
envelope.update({
"schema_version": "outbound_source_envelope_v1",
"redaction_version": _OUTBOUND_REDACTION_VERSION,
"content_sha256": content_hash,
"content_length": len(content) if content is not None else 0,
})
source_envelope_json = json.dumps(envelope, ensure_ascii=False, default=str)
actual_status = "shadow" if is_shadow else send_status
sent_at = (
_db_timestamp_now()
if actual_status == "sent"
else None
)
await ensure_completed_shadow_run(
db,
@@ -495,14 +956,18 @@ async def record_outbound_message(
INSERT INTO awooop_outbound_message (
project_id, run_id, conversation_event_id,
channel_type, channel_chat_id, message_type,
content_hash, content_preview, provider_message_id,
send_status, queued_at,
content_hash, content_preview, content_redacted,
redaction_version, source_envelope,
provider_message_id,
send_status, queued_at, sent_at,
triggered_by_state, waiting_since
) VALUES (
:project_id, :run_id, :conversation_event_id,
:channel_type, :channel_chat_id, :message_type,
:content_hash, :content_preview, :provider_message_id,
:send_status, NOW(),
:content_hash, :content_preview, :content_redacted,
:redaction_version, CAST(:source_envelope AS jsonb),
:provider_message_id,
:send_status, NOW(), :sent_at,
:triggered_by_state, :waiting_since
)
RETURNING message_id
@@ -516,8 +981,12 @@ async def record_outbound_message(
"message_type": message_type,
"content_hash": content_hash,
"content_preview": content_preview,
"content_redacted": content_redacted,
"redaction_version": _OUTBOUND_REDACTION_VERSION,
"source_envelope": source_envelope_json,
"provider_message_id": provider_message_id,
"send_status": actual_status,
"sent_at": sent_at,
"triggered_by_state": triggered_by_state,
"waiting_since": waiting_since,
},

View File

@@ -25,7 +25,7 @@ import structlog
from src.core.config import get_settings
from src.repositories.incident_repository import get_incident_repository
from src.repositories.k8s_repository import get_k8s_repository
from src.services.ollama_endpoint_resolver import resolve_ollama_endpoint
from src.services.ollama_endpoint_resolver import resolve_ollama_order
from src.utils.timezone import now_taipei
logger = structlog.get_logger(__name__)
@@ -96,42 +96,51 @@ class ChatManager:
settings = get_settings()
model = settings.OPENCLAW_DEFAULT_MODEL
ollama_url = resolve_ollama_endpoint("interactive")
try:
async with httpx.AsyncClient(timeout=40.0) as client:
resp = await client.post(
f"{ollama_url}/api/chat",
json={
"model": model,
"stream": False,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message},
],
"options": {"num_predict": 900, "temperature": 0.2},
},
)
resp.raise_for_status()
data = resp.json()
raw = data.get("message", {}).get("content", "").strip()
text = re.sub(r"<think>.*?</think>", "", raw, flags=re.DOTALL).strip() or raw
async with httpx.AsyncClient(timeout=40.0) as client:
for endpoint in resolve_ollama_order("interactive"):
if not endpoint.url:
continue
try:
resp = await client.post(
f"{endpoint.url}/api/chat",
json={
"model": model,
"stream": False,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message},
],
"options": {"num_predict": 900, "temperature": 0.2},
},
)
resp.raise_for_status()
data = resp.json()
raw = data.get("message", {}).get("content", "").strip()
text = re.sub(r"<think>.*?</think>", "", raw, flags=re.DOTALL).strip() or raw
eval_count = data.get("eval_count", 0)
prompt_eval_count = data.get("prompt_eval_count", 0)
total_tokens = eval_count + prompt_eval_count
eval_count = data.get("eval_count", 0)
prompt_eval_count = data.get("prompt_eval_count", 0)
total_tokens = eval_count + prompt_eval_count
logger.info(
"openclaw_ollama_chat_usage",
model=model,
endpoint=ollama_url,
prompt_tokens=prompt_eval_count,
output_tokens=eval_count,
)
logger.info(
"openclaw_ollama_chat_usage",
model=model,
endpoint=endpoint.url,
provider=endpoint.provider_name,
prompt_tokens=prompt_eval_count,
output_tokens=eval_count,
)
return f"{text}\n\n<i>🦙 {model} | {total_tokens} tokens | 免費</i>"
except Exception as e:
logger.warning("openclaw_chat_failed", error=str(e))
return None
return f"{text}\n\n<i>🦙 {model} | {total_tokens} tokens | 免費</i>"
except Exception as e:
logger.warning(
"openclaw_chat_endpoint_failed",
provider=endpoint.provider_name,
endpoint=endpoint.url,
error=str(e),
)
logger.warning("openclaw_chat_failed_all_endpoints", model=model)
return None
async def _call_nemotron(self, system_prompt: str, user_message: str) -> str | None:
"""
@@ -146,43 +155,60 @@ class ChatManager:
system_prompt = f"{NEMOCLAW_PERSONA}\n{system_prompt}"
# 2026-05-05 Codex: ADR-110 interactive lane由 resolver 管理 GCP-A/GCP-B/111 拓撲
OLLAMA_URL = resolve_ollama_endpoint("interactive")
MODEL = "deepseek-r1:14b"
try:
async with httpx.AsyncClient(timeout=120.0) as client:
resp = await client.post(
f"{OLLAMA_URL}/api/chat",
json={
"model": MODEL,
"stream": False,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message},
],
"options": {"num_predict": 1200},
},
)
resp.raise_for_status()
data = resp.json()
raw = data.get("message", {}).get("content", "").strip()
async with httpx.AsyncClient(timeout=120.0) as client:
for endpoint in resolve_ollama_order("interactive"):
if not endpoint.url:
continue
try:
resp = await client.post(
f"{endpoint.url}/api/chat",
json={
"model": MODEL,
"stream": False,
# Ollama 0.24 separates deepseek-r1 thinking from final text.
# Chat callers expect message.content to contain the answer.
"think": False,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message},
],
"options": {"num_predict": 1200},
},
)
resp.raise_for_status()
data = resp.json()
raw = data.get("message", {}).get("content", "").strip()
# 過濾 deepseek-r1 的 <think>...</think> 推理區塊
text = re.sub(r"<think>.*?</think>", "", raw, flags=re.DOTALL).strip()
if not text:
text = raw # 萬一全是 think block直接回傳原文
# 過濾 deepseek-r1 的 <think>...</think> 推理區塊
text = re.sub(r"<think>.*?</think>", "", raw, flags=re.DOTALL).strip()
if not text:
text = raw # 萬一全是 think block直接回傳原文
eval_count = data.get("eval_count", 0)
prompt_eval_count = data.get("prompt_eval_count", 0)
total_tokens = eval_count + prompt_eval_count
eval_count = data.get("eval_count", 0)
prompt_eval_count = data.get("prompt_eval_count", 0)
total_tokens = eval_count + prompt_eval_count
logger.info("nemotron_ollama_usage", model=MODEL,
prompt_tokens=prompt_eval_count, output_tokens=eval_count)
logger.info(
"nemotron_ollama_usage",
model=MODEL,
provider=endpoint.provider_name,
prompt_tokens=prompt_eval_count,
output_tokens=eval_count,
)
return f"{text}\n\n<i>🦙 {MODEL} | {total_tokens} tokens | 免費</i>"
except Exception as e:
logger.warning("nemotron_chat_failed", model=MODEL, error=str(e))
return None
return f"{text}\n\n<i>🦙 {MODEL} | {total_tokens} tokens | 免費</i>"
except Exception as e:
logger.warning(
"nemotron_chat_endpoint_failed",
model=MODEL,
provider=endpoint.provider_name,
endpoint=endpoint.url,
error=str(e),
)
logger.warning("nemotron_chat_failed_all_endpoints", model=MODEL)
return None
async def generate_response(
self,

View File

@@ -0,0 +1,31 @@
"""
Dashboard Metrics Service
=========================
Small DB-backed counters used by the war-room dashboard.
"""
from sqlalchemy import text
from src.core.logging import get_logger
from src.db.base import get_db_context
logger = get_logger("awoooi.dashboard_metrics")
async def fetch_pending_approval_count() -> int:
"""Read the live HITL backlog from approval_records."""
try:
async with get_db_context() as db:
result = await db.execute(
text(
"""
SELECT count(*)
FROM approval_records
WHERE upper(status::text) = 'PENDING'
"""
)
)
return int(result.scalar_one() or 0)
except Exception as exc:
logger.warning("dashboard_pending_approval_count_failed", error=str(exc))
return 0

View File

@@ -29,6 +29,7 @@ import httpx
import structlog
from src.core.config import get_settings
from src.services.ollama_endpoint_resolver import resolve_ollama_order
if TYPE_CHECKING:
from src.models.incident import Incident
@@ -186,9 +187,41 @@ class DecisionFusionEngine:
# settings 延遲讀取(避免測試環境初始化問題)
self._settings = get_settings()
@property
def _ollama_url(self) -> str:
return getattr(self._settings, "OLLAMA_URL", "http://34.143.170.20:11434") # 2026-05-03 ogt: ADR-110 GCP-A Primary
async def _call_ollama_generate(
self,
*,
prompt: str,
timeout_sec: float,
num_predict: int,
) -> str:
"""Call Ollama in the global order: GCP-A -> GCP-B -> 111."""
last_error: Exception | None = None
async with httpx.AsyncClient(
timeout=httpx.Timeout(timeout_sec, connect=5.0)
) as client:
for endpoint in resolve_ollama_order("deep_rca"):
if not endpoint.url:
continue
try:
resp = await client.post(
f"{endpoint.url}{_OLLAMA_GENERATE_PATH}",
json={
"model": "qwen3:8b",
"prompt": prompt,
"stream": False,
"options": {"num_predict": num_predict, "temperature": 0.1},
},
)
resp.raise_for_status()
return resp.json().get("response", "").strip()
except Exception as exc:
last_error = exc
logger.debug(
"decision_fusion_ollama_endpoint_failed",
provider=endpoint.provider_name,
error=str(exc),
)
raise RuntimeError(str(last_error) if last_error else "no_ollama_endpoint")
# =========================================================================
# Public API
@@ -322,21 +355,12 @@ class DecisionFusionEngine:
)
try:
async with httpx.AsyncClient(
timeout=httpx.Timeout(_HERMES_TIMEOUT_SEC, connect=5.0)
) as client:
resp = await client.post(
f"{self._ollama_url}{_OLLAMA_GENERATE_PATH}",
json={
"model": "qwen3:8b",
"prompt": prompt,
"stream": False,
"options": {"num_predict": 16, "temperature": 0.1},
},
)
if resp.status_code == 200:
text = resp.json().get("response", "").strip()
return self._extract_float(text, default=0.5)
text = await self._call_ollama_generate(
prompt=prompt,
timeout_sec=_HERMES_TIMEOUT_SEC,
num_predict=16,
)
return self._extract_float(text, default=0.5)
except Exception as exc:
logger.debug("hermes_score_failed", error=str(exc))
@@ -451,20 +475,11 @@ class DecisionFusionEngine:
"只回覆一個 0-1 的小數,不要解釋。"
)
async with httpx.AsyncClient(
timeout=httpx.Timeout(_ELEPHANT_TIMEOUT_SEC, connect=5.0)
) as client:
resp = await client.post(
f"{self._ollama_url}{_OLLAMA_GENERATE_PATH}",
json={
"model": "qwen3:8b",
"prompt": prompt,
"stream": False,
"options": {"num_predict": 32, "temperature": 0.1},
},
)
resp.raise_for_status()
raw_text = resp.json().get("response", "").strip()
raw_text = await self._call_ollama_generate(
prompt=prompt,
timeout_sec=_ELEPHANT_TIMEOUT_SEC,
num_predict=32,
)
# 移除 deepseek/qwen3 <think> 標籤
clean = re.sub(r"<think>.*?</think>", "", raw_text, flags=re.DOTALL).strip()

Some files were not shown because too many files have changed in this diff Show More