89e05e6ea2
- ADR-037 監控增強架構 - MONITORING_MASTER_PLAN 主計畫 - MASTER_EXECUTION_SCHEDULE 執行排程 - Phase D/E/Worker HPA Runbooks Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
265 lines
7.0 KiB
Markdown
265 lines
7.0 KiB
Markdown
# RunBook: Phase D — Sentry Comment 回寫啟動指南
|
||
|
||
> **類型**: 操作型 RunBook
|
||
> **優先級**: 🔴 P0（功能框架已建，只缺 Token 配置）
|
||
> **建立**: 2026-03-29 12:35 (台北)
|
||
> **建立者**: Antigravity
|
||
> **工時預估**: 1.5–2 小時
|
||
> **前置條件**: AWOOOI API 正常運行 (`/api/v1/health` 返回 200)
|
||
|
||
---
|
||
|
||
## 背景與現況
|
||
|
||
### 🔍 精確現況診斷
|
||
|
||
`sentry_webhook.py` 的 `post_sentry_comment()` 函式已實作完整邏輯：
|
||
|
||
```
|
||
sentry_webhook.py:251 → 呼叫 post_sentry_comment()
|
||
sentry_service.py:206 → post_issue_comment() 已實作 POST /api/0/issues/{id}/comments/
|
||
sentry_service.py:223 → 若 SENTRY_AUTH_TOKEN 為空，直接 return None 並 warning
|
||
```
|
||
|
||
**唯一阻塞點**：`settings.SENTRY_AUTH_TOKEN` 環境變數未設定，導致 comment 靜默跳過。
|
||
|
||
### 資料流確認
|
||
|
||
```
|
||
Sentry Issue 觸發
|
||
↓
|
||
/api/v1/webhooks/sentry/error (sentry_webhook.py)
|
||
↓
|
||
analyze_and_comment() [Background Task]
|
||
↓
|
||
call_openclaw_analyzer() → OpenClaw AI 分析
|
||
↓
|
||
create_sentry_approval() → 建立 Approval ✅ 已運作
|
||
↓
|
||
send_sentry_telegram_alert()→ Telegram 通知 ✅ 已運作
|
||
↓
|
||
post_sentry_comment() → ❌ SENTRY_AUTH_TOKEN 缺失，靜默跳過
|
||
```
|
||
|
||
---
|
||
|
||
## Step 1: 取得 Sentry API Token
|
||
|
||
### 1.1 登入 Sentry 後台
|
||
|
||
```
|
||
瀏覽器開啟：http://192.168.0.110:9000
|
||
帳號：參見 docs/security/SECRETS_REFERENCE.md
|
||
```
|
||
|
||
### 1.2 建立 API Token
|
||
|
||
```
|
||
路徑：設定 → API → Auth Tokens → Create New Token
|
||
|
||
權限設定：
|
||
☑ project:read
|
||
☑ project:write
|
||
☑ issues:write ← 必須勾選，否則無法回寫 comment
|
||
☑ event:read
|
||
|
||
Token 名稱建議：awoooi-openclaw-comment-writer
|
||
```
|
||
|
||
### 1.3 記錄 Token（請勿存入代碼庫！）
|
||
|
||
```bash
|
||
# 暫存到環境變數（本機測試用）
|
||
export SENTRY_AUTH_TOKEN="sentry_xxx..."
|
||
|
||
# 驗證 Token 有效性
|
||
curl -s http://192.168.0.110:9000/api/0/organizations/ \
|
||
-H "Authorization: Bearer $SENTRY_AUTH_TOKEN" | python3 -m json.tool | head -20
|
||
# 預期看到 organization 列表，無 401
|
||
```
|
||
|
||
---
|
||
|
||
## Step 2: 注入 GitHub Secrets（CD 自動化）
|
||
|
||
### 2.1 加入 GitHub Repository Secrets
|
||
|
||
```
|
||
路徑：GitHub → owenhytsai/awoooi → Settings → Secrets → Actions
|
||
|
||
新增以下 Secrets：
|
||
名稱：SENTRY_AUTH_TOKEN
|
||
值：步驟 1.2 取得的 Token
|
||
```
|
||
|
||
### 2.2 更新 K8s Secret（手動注入生產環境）
|
||
|
||
```bash
|
||
# 在 192.168.0.120（K3s Master）執行
|
||
kubectl patch secret awoooi-secrets -n awoooi-prod \
|
||
--patch="{\"data\":{\"SENTRY_AUTH_TOKEN\":\"$(echo -n 'YOUR_TOKEN' | base64)\"}}"
|
||
|
||
# 驗證
|
||
kubectl get secret awoooi-secrets -n awoooi-prod -o jsonpath='{.data.SENTRY_AUTH_TOKEN}' | base64 -d
|
||
```
|
||
|
||
### 2.3 更新 k8s/awoooi-prod/03-secrets.yaml（模板）
|
||
|
||
```yaml
|
||
# k8s/awoooi-prod/03-secrets.yaml
|
||
# 新增以下欄位（使用 CD 自動注入，非硬編碼）
|
||
stringData:
|
||
# ... 現有欄位 ...
|
||
SENTRY_AUTH_TOKEN: "${SENTRY_AUTH_TOKEN}" # CD 自動注入
|
||
```
|
||
|
||
---
|
||
|
||
## Step 3: 更新 CD Workflow 自動注入
|
||
|
||
```yaml
|
||
# .github/workflows/cd.yaml
|
||
# 在 "Inject K8s Secrets" 步驟中新增 SENTRY_AUTH_TOKEN
|
||
|
||
- name: Inject K8s Secrets
|
||
run: |
|
||
kubectl patch secret awoooi-secrets -n awoooi-prod \
|
||
--patch="{\"data\":{
|
||
\"OPENCLAW_TG_BOT_TOKEN\":\"$(echo -n '${{ secrets.OPENCLAW_TG_BOT_TOKEN }}' | base64)\",
|
||
\"OPENCLAW_TG_CHAT_ID\":\"$(echo -n '${{ secrets.OPENCLAW_TG_CHAT_ID }}' | base64)\",
|
||
\"SENTRY_AUTH_TOKEN\":\"$(echo -n '${{ secrets.SENTRY_AUTH_TOKEN }}' | base64)\"
|
||
}}"
|
||
```
|
||
|
||
---
|
||
|
||
## Step 4: 驗證 Sentry Comment 功能
|
||
|
||
### 4.1 本地單元測試（快速驗證）
|
||
|
||
```bash
|
||
cd /Users/ogt/awoooi
|
||
source .env
|
||
|
||
# 設定測試 Token
|
||
export SENTRY_AUTH_TOKEN="你的真實 Token"
|
||
export SENTRY_SELF_HOSTED_URL="http://192.168.0.110:9000"
|
||
|
||
# 用 Python 直接測試 SentryService
|
||
python3 -c "
|
||
import asyncio
|
||
import sys
|
||
sys.path.insert(0, 'apps/api/src')
|
||
from services.sentry_service import SentryService
|
||
|
||
async def test():
|
||
svc = SentryService(
|
||
base_url='http://192.168.0.110:9000',
|
||
auth_token='$SENTRY_AUTH_TOKEN'
|
||
)
|
||
# 先列出 Issues 找一個真實 ID
|
||
issues = await svc.list_issues(project='awoooi-api', limit=3)
|
||
if issues:
|
||
issue_id = issues[0]['id']
|
||
print(f'找到 Issue: {issue_id}')
|
||
result = await svc.post_issue_comment(
|
||
issue_id=issue_id,
|
||
text='🤖 **AWOOOI 測試** - Sentry Comment 回寫功能正常運作。'
|
||
)
|
||
print(f'Comment 結果: {result}')
|
||
else:
|
||
print('無 Issue 可測試')
|
||
|
||
asyncio.run(test())
|
||
"
|
||
```
|
||
|
||
### 4.2 E2E 端對端驗證（生產環境）
|
||
|
||
```bash
|
||
# 1. 在 Sentry 手動觸發一個測試 Issue
|
||
curl -X POST http://localhost:8000/api/v1/webhooks/sentry/error \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"action": "triggered",
|
||
"data": {
|
||
"issue": {
|
||
"id": "TEST-001",
|
||
"title": "AWOOOI Comment 功能測試",
|
||
"level": "error",
|
||
"culprit": "test.py:1",
|
||
"firstSeen": "2026-03-29T12:00:00Z",
|
||
"count": 1,
|
||
"project": {"slug": "awoooi-api"}
|
||
},
|
||
"event": {
|
||
"message": "這是一個測試錯誤",
|
||
"platform": "python"
|
||
}
|
||
}
|
||
}'
|
||
|
||
# 預期回應
|
||
# {"status": "accepted", "issue_id": "TEST-001", "message": "Analysis scheduled"}
|
||
|
||
# 2. 等待 60 秒後（OpenClaw 分析需時）
|
||
sleep 60
|
||
|
||
# 3. 到 Sentry UI 的 Issue 中確認是否有 AI 分析 Comment
|
||
# http://192.168.0.110:9000/organizations/sentry/issues/TEST-001/
|
||
```
|
||
|
||
### 4.3 確認日誌
|
||
|
||
```bash
|
||
# 查看 API Pod 日誌
|
||
kubectl logs -n awoooi-prod \
|
||
$(kubectl get pod -n awoooi-prod -l app=awoooi-api -o name | head -1) \
|
||
--tail=50 | grep -i sentry_comment
|
||
|
||
# 預期看到
|
||
# sentry_comment_posted issue_id=xxx comment_id=12345
|
||
```
|
||
|
||
---
|
||
|
||
## Step 5: 部署與驗收
|
||
|
||
```bash
|
||
# 觸發 CD
|
||
git add k8s/awoooi-prod/03-secrets.yaml .github/workflows/cd.yaml
|
||
git commit -m "feat(sentry): enable comment write-back via SENTRY_AUTH_TOKEN injection"
|
||
git push origin main
|
||
|
||
# 確認 CD 成功
|
||
gh run list --workflow=cd.yaml --limit 1
|
||
|
||
# 確認 Pod 有新 Token
|
||
kubectl exec -n awoooi-prod \
|
||
$(kubectl get pod -n awoooi-prod -l app=awoooi-api -o name | head -1) \
|
||
-- env | grep SENTRY_AUTH_TOKEN
|
||
```
|
||
|
||
---
|
||
|
||
## 驗收標準
|
||
|
||
| 項目 | 通過條件 |
|
||
|------|---------|
|
||
| K8s Secret 已注入 | `kubectl get secret` 確認 `SENTRY_AUTH_TOKEN` 不為空 |
|
||
| Token 有效 | Sentry API `/api/0/organizations/` 返回 200 |
|
||
| Comment 回寫 | Sentry Issue 中有「AI 錯誤分析」Comment |
|
||
| 日誌正常 | `sentry_comment_posted` 日誌出現，無 `sentry_comment_failed` |
|
||
| 頻率統計 | Comment 含「頻率統計」表格（24h 次數 > 1 時顯示）|
|
||
|
||
---
|
||
|
||
## 常見問題排除
|
||
|
||
| 症狀 | 診斷指令 | 解法 |
|
||
|------|---------|------|
|
||
| `sentry_comment_skipped` 日誌 | `env \| grep SENTRY_AUTH_TOKEN` | Secret 未注入，重跑 Step 3 |
|
||
| `sentry_api_unauthorized` | 手動 curl Sentry API | Token 權限不足，重新建立 |
|
||
| `sentry_api_timeout` | `curl -v http://192.168.0.110:9000/` | Sentry 服務本身異常 |
|
||
| OpenClaw 分析失敗 | `curl http://192.168.0.188:8089/health` | OpenClaw 服務需重啟 |
|