Files
ewoooc/templates/admin/observability_overview.html
OoO e0a8d87c2c
All checks were successful
CD Pipeline / deploy (push) Successful in 2m35s
feat(p51): RAG 召回詳情新頁 + overview 三主機 24h sparkline
新頁 /observability/rag_queries:補完 RAG 觀測深度
之前只看 caller 級命中率,現在能看每筆查詢的真實內容。

O-1: route + template
- 篩選:時段(1/6/24/72/168h)/ caller / saved_only flag
- 整體 KPI 4 卡:總查詢 / 命中率 / saved_call 率 / 反饋平均分
- by caller 表:每個 caller 的查詢/命中/saved/反饋細節
- 最近 50 筆查詢詳情表
- 「查 hits」按鈕 → 彈 modal 載入 ai_insights JOIN 內容預覽
  (新 endpoint /observability/rag_queries/<id>/hits 回傳 JSON)

O-2: 入口
- sidebar AI 觀測 group 加「RAG 召回詳情」(11b)
- /observability/overview 入口卡升級為 9 項

O-3: overview 三主機 24h sparkline
- 每張主機卡片下方加 60px 高 chart.js sparkline
- 折線:每小時 uptime % bucket(0-100% Y 軸隱藏,純視覺)
- routes/admin_observability_routes.py::observability_overview
  新加 host_sparkline 查詢(GROUP BY host_label, hour)
- 三主機卡片視覺化升級:原本只有「100%」字,現在加趨勢線

Phase 38→51 累計 16 commits / 10 觀測頁。
觀測台戰役從「raw stats」到「視覺方格 UI 完整體」。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 20:09:28 +08:00

334 lines
16 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{% extends "ewoooc_base.html" %}
{% block title %}觀測台總覽{% endblock %}
{% block ewooo_content %}
<div class="container-fluid mt-3">
<h2 class="mb-3"><i class="fas fa-satellite-dish me-2"></i>AI 觀測台總覽
<small class="text-muted">{{ today }} · 全景一頁看(資料來源 8 表跨 JOIN</small>
</h2>
<!-- 三主機健康卡片(含 24h sparkline-->
<div class="row g-3 mb-3">
{% if summary.hosts %}
{% for h in summary.hosts %}
<div class="col-lg-4 col-md-6">
<div class="card h-100" style="border-left: 4px solid
{% if h.uptime_pct >= 99 %}#198754
{% elif h.uptime_pct >= 90 %}#ffc107
{% else %}#dc3545{% endif %};">
<div class="card-body">
<div class="d-flex justify-content-between align-items-start">
<div>
<small class="text-muted d-block">{{ h.label }}</small>
<h3 class="mb-0">{{ "%.1f"|format(h.uptime_pct) }}<small class="text-muted">%</small></h3>
<small class="text-muted">24h 在線率({{ h.up }}/{{ h.total }} probe</small>
</div>
<div class="text-end">
<i class="fas fa-server" style="font-size: 1.8em; color: #ddd;"></i>
<div class="mt-2"><small class="text-muted">{{ h.avg_ms }} ms</small></div>
</div>
</div>
{% if host_sparkline.get(h.label) %}
<div class="mt-2" style="height: 60px;">
<canvas data-host-sparkline="{{ h.label }}"></canvas>
</div>
{% endif %}
</div>
</div>
</div>
{% endfor %}
{% else %}
<div class="col-12">
<div class="alert alert-warning">
<i class="fas fa-exclamation-triangle me-1"></i>
host_health_probes 表無資料migration 029 是否已跑scheduler probe job 是否已啟動?)
</div>
</div>
{% endif %}
</div>
<!-- AI 呼叫 + 成本卡片 -->
<div class="row g-3 mb-3">
{% if summary.ai_calls %}
<div class="col-lg-3 col-md-6">
<div class="card h-100">
<div class="card-body">
<small class="text-muted d-block"><i class="fas fa-chart-bar me-1"></i>24h AI 呼叫</small>
<h3 class="mb-0">{{ "{:,}".format(summary.ai_calls.total) }}</h3>
<small class="text-muted">Token{{ "{:,}".format(summary.ai_calls.tokens) }}</small>
</div>
</div>
</div>
<div class="col-lg-3 col-md-6">
<div class="card h-100">
<div class="card-body">
<small class="text-muted d-block"><i class="fas fa-coins me-1"></i>成本</small>
<h3 class="mb-0">${{ "%.2f"|format(summary.ai_calls.cost_24h) }}<small class="text-muted">/24h</small></h3>
<small class="text-muted">當月累計 ${{ "%.2f"|format(summary.month_cost) }}</small>
</div>
</div>
</div>
<div class="col-lg-3 col-md-6">
<div class="card h-100">
<div class="card-body">
<small class="text-muted d-block"><i class="fas fa-exclamation-triangle me-1"></i>錯誤率</small>
<h3 class="mb-0
{% if summary.ai_calls.error_rate >= 15 %}text-danger
{% elif summary.ai_calls.error_rate >= 5 %}text-warning
{% else %}text-success{% endif %}">{{ "%.1f"|format(summary.ai_calls.error_rate) }}<small>%</small></h3>
<small class="text-muted">{{ summary.ai_calls.errors }} 次失敗</small>
</div>
</div>
</div>
<div class="col-lg-3 col-md-6">
<div class="card h-100">
<div class="card-body">
<small class="text-muted d-block"><i class="fas fa-magnifying-glass-chart me-1"></i>RAG 命中率</small>
<h3 class="mb-0 text-success">{{ "%.1f"|format(summary.ai_calls.rag_rate) }}<small>%</small></h3>
<small class="text-muted">{{ summary.ai_calls.rag_hits }} hits · cache {{ "%.0f"|format(summary.ai_calls.cache_rate) }}%</small>
</div>
</div>
</div>
{% endif %}
</div>
<!-- 預算告警 -->
{% if summary.budget_alerts %}
<div class="card mb-3" style="border-left: 4px solid #ffc107;">
<div class="card-header bg-warning bg-opacity-25">
<strong><i class="fas fa-wallet me-1"></i>預算告警 — 共 {{ summary.budget_alerts|length }} 項超出閾值</strong>
</div>
<div class="card-body p-0">
<table class="table table-sm mb-0">
<thead class="table-light">
<tr><th>週期</th><th>供應商</th><th class="text-end">已花費</th><th class="text-end">預算</th><th class="text-end">使用率</th></tr>
</thead>
<tbody>
{% for b in summary.budget_alerts %}
<tr>
<td><span class="badge bg-secondary">{{ b.period }}</span></td>
<td><code>{{ b.provider }}</code></td>
<td class="text-end">${{ "%.2f"|format(b.spent) }}</td>
<td class="text-end">${{ "%.2f"|format(b.budget) }}</td>
<td class="text-end">
<strong class="{% if b.ratio >= 1.0 %}text-danger{% else %}text-warning{% endif %}">
{{ "%.0f"|format(b.ratio * 100) }}%
</strong>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<div class="card-footer text-end">
<a href="/observability/budget" class="btn btn-sm btn-outline-warning">
<i class="fas fa-arrow-right me-1"></i>進預算控管處理
</a>
</div>
</div>
{% endif %}
<!-- AIOps + MCP + RAG 學習 -->
<div class="row g-3 mb-3">
{% if summary.aiops %}
<div class="col-lg-4 col-md-6">
<div class="card h-100">
<div class="card-header"><strong><i class="fas fa-shield-virus me-1"></i>AIOps 自癒 7d</strong></div>
<div class="card-body">
<div class="row g-2">
<div class="col-6"><small class="text-muted d-block">事件總數</small><h4 class="mb-0">{{ summary.aiops.incidents_total }}</h4></div>
<div class="col-6"><small class="text-muted d-block">未解決</small><h4 class="mb-0 {% if summary.aiops.incidents_open > 0 %}text-danger{% endif %}">{{ summary.aiops.incidents_open }}</h4></div>
<div class="col-6"><small class="text-muted d-block">P0/P1</small><h4 class="mb-0 {% if summary.aiops.incidents_p0_p1 > 0 %}text-danger{% endif %}">{{ summary.aiops.incidents_p0_p1 }}</h4></div>
<div class="col-6"><small class="text-muted d-block">自癒成功率</small><h4 class="mb-0
{% if summary.aiops.heal_rate >= 80 %}text-success
{% elif summary.aiops.heal_rate >= 50 %}text-warning
{% else %}text-danger{% endif %}">{{ "%.0f"|format(summary.aiops.heal_rate) }}%</h4></div>
</div>
</div>
<div class="card-footer text-end p-2">
<a href="/observability/host_health" class="btn btn-sm btn-outline-secondary">
<i class="fas fa-arrow-right me-1"></i>主機健康
</a>
</div>
</div>
</div>
{% endif %}
{% if summary.mcp %}
<div class="col-lg-4 col-md-6">
<div class="card h-100">
<div class="card-header"><strong><i class="fas fa-bolt me-1"></i>MCP 24h</strong></div>
<div class="card-body">
<div class="row g-2">
<div class="col-6"><small class="text-muted d-block">tool 呼叫</small><h4 class="mb-0">{{ "{:,}".format(summary.mcp.total) }}</h4></div>
<div class="col-6"><small class="text-muted d-block">使用 server</small><h4 class="mb-0">{{ summary.mcp.servers }}</h4></div>
<div class="col-6"><small class="text-muted d-block">cache 命中</small><h4 class="mb-0">{{ summary.mcp.cache_hits }}</h4></div>
<div class="col-6"><small class="text-muted d-block">成本</small><h4 class="mb-0">${{ "%.4f"|format(summary.mcp.cost) }}</h4></div>
</div>
</div>
<div class="card-footer text-end p-2">
<a href="/observability/host_health" class="btn btn-sm btn-outline-secondary">
<i class="fas fa-arrow-right me-1"></i>查 MCP 健康
</a>
</div>
</div>
</div>
{% endif %}
{% if summary.episodes %}
<div class="col-lg-4 col-md-6">
<div class="card h-100">
<div class="card-header"><strong><i class="fas fa-brain me-1"></i>RAG 學習鏈路 30d</strong></div>
<div class="card-body">
<div class="row g-2">
<div class="col-6"><small class="text-muted d-block">待審核</small><h4 class="mb-0 {% if summary.episodes.pending > 0 %}text-warning{% endif %}">{{ summary.episodes.pending }}</h4></div>
<div class="col-6"><small class="text-muted d-block">總 episodes</small><h4 class="mb-0">{{ summary.episodes.total_30d }}</h4></div>
<div class="col-6"><small class="text-muted d-block">已晉升</small><h4 class="mb-0 text-success">{{ summary.episodes.approved_30d }}</h4></div>
<div class="col-6"><small class="text-muted d-block">晉升率</small><h4 class="mb-0">{{ "%.0f"|format(summary.episodes.approval_rate) }}%</h4></div>
</div>
</div>
<div class="card-footer text-end p-2">
{% if summary.episodes.pending > 0 %}
<a href="/observability/promotion_review" class="btn btn-sm btn-warning">
<i class="fas fa-arrow-right me-1"></i>立即審核 ({{ summary.episodes.pending }})
</a>
{% else %}
<a href="/observability/promotion_review" class="btn btn-sm btn-outline-secondary">
<i class="fas fa-arrow-right me-1"></i>晉升審核
</a>
{% endif %}
</div>
</div>
</div>
{% endif %}
</div>
<!-- PPT 視覺審核 -->
{% if summary.ppt and summary.ppt.total > 0 %}
<div class="card mb-3">
<div class="card-header"><strong><i class="fas fa-search me-1"></i>PPT 視覺審核 7d</strong></div>
<div class="card-body">
<div class="row g-2">
<div class="col-md-3 col-6"><small class="text-muted d-block">總筆數</small><h4 class="mb-0">{{ summary.ppt.total }}</h4></div>
<div class="col-md-3 col-6"><small class="text-muted d-block">通過</small><h4 class="mb-0 text-success">{{ summary.ppt.passed }}</h4></div>
<div class="col-md-3 col-6"><small class="text-muted d-block">失敗</small><h4 class="mb-0 {% if summary.ppt.failed > 0 %}text-warning{% endif %}">{{ summary.ppt.failed }}</h4></div>
<div class="col-md-3 col-6"><small class="text-muted d-block">通過率</small><h4 class="mb-0">{{ "%.0f"|format(summary.ppt.pass_rate) }}%</h4></div>
</div>
</div>
<div class="card-footer text-end p-2">
<a href="/observability/ppt_audit_history" class="btn btn-sm btn-outline-secondary">
<i class="fas fa-arrow-right me-1"></i>PPT 審核歷史
</a>
</div>
</div>
{% endif %}
<!-- 9 大入口 -->
<div class="card">
<div class="card-header"><strong><i class="fas fa-th me-1"></i>9 大子頁入口</strong></div>
<div class="card-body">
<div class="row g-2">
<div class="col-lg-4 col-md-6">
<a href="/observability/agent_orchestration" class="btn btn-outline-info w-100 text-start" style="border-width: 2px;">
<i class="fas fa-network-wired me-2"></i><strong>Agent 編排矩陣</strong>
<small class="d-block text-muted ms-4">4 Agent × Ollama × Gemini × MCP × RAG 全景 + 自動建議</small>
</a>
</div>
<div class="col-lg-4 col-md-6">
<a href="/observability/business_intel" class="btn btn-outline-warning w-100 text-start" style="border-width: 2px;">
<i class="fas fa-briefcase me-2"></i><strong>商業面 × AI 編排</strong>
<small class="d-block text-muted ms-4">AI 價格決策 + 閉環學習 + 競品比對全景</small>
</a>
</div>
<div class="col-lg-4 col-md-6">
<a href="/observability/host_health" class="btn btn-outline-primary w-100 text-start">
<i class="fas fa-heartbeat me-2"></i>主機健康監控
<small class="d-block text-muted ms-4">三主機 + MCP + AIOps + 24h 趨勢 + 一鍵 AutoHeal</small>
</a>
</div>
<div class="col-lg-4 col-md-6">
<a href="/observability/ai_calls" class="btn btn-outline-primary w-100 text-start">
<i class="fas fa-chart-bar me-2"></i>AI 呼叫總覽
<small class="d-block text-muted ms-4">24h 統計 + RAG×MCP 編排矩陣 + 一鍵 Code Review</small>
</a>
</div>
<div class="col-lg-4 col-md-6">
<a href="/observability/budget" class="btn btn-outline-primary w-100 text-start">
<i class="fas fa-wallet me-2"></i>預算控管
<small class="d-block text-muted ms-4">當月支出 + RAG 策略建議 + 一鍵 force-throttle</small>
</a>
</div>
<div class="col-lg-4 col-md-6">
<a href="/observability/promotion_review" class="btn btn-outline-primary w-100 text-start">
<i class="fas fa-brain me-2"></i>RAG 學習晉升審核
<small class="d-block text-muted ms-4">待審 episode + RAG Top 3 相似已晉升輔助</small>
</a>
</div>
<div class="col-lg-4 col-md-6">
<a href="/observability/rag_queries" class="btn btn-outline-primary w-100 text-start">
<i class="fas fa-magnifying-glass-chart me-2"></i>RAG 召回詳情
<small class="d-block text-muted ms-4">每筆 query 的 hits / saved_call / 反饋追蹤</small>
</a>
</div>
<div class="col-lg-4 col-md-6">
<a href="/observability/quality_trend" class="btn btn-outline-primary w-100 text-start">
<i class="fas fa-comments me-2"></i>Caller 反饋趨勢
<small class="d-block text-muted ms-4">蒸餾池分布 + 最差 caller RAG 根因建議</small>
</a>
</div>
<div class="col-lg-4 col-md-6">
<a href="/observability/ppt_audit_history" class="btn btn-outline-primary w-100 text-start">
<i class="fas fa-search me-2"></i>PPT 視覺審核歷史
<small class="d-block text-muted ms-4">7d audit 紀錄 + RAG 修法 + 一鍵 AiderHeal</small>
</a>
</div>
</div>
</div>
</div>
<p class="text-muted mt-3"><small>
<i class="fas fa-robot me-1"></i>Operation Ollama-First v5.0 / Phase 51 — AI 觀測台總覽(含 24h sparkline
資料來源host_health_probes / ai_calls / ai_call_budgets / learning_episodes / ai_insights /
rag_query_log / mcp_calls / incidents / heal_logs / ppt_audit_results
</small></p>
</div>
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.1/dist/chart.umd.min.js"></script>
<script>
// Phase 51 O-3: 三主機 24h sparkline
(function() {
const data = {{ host_sparkline | tojson }};
document.querySelectorAll('canvas[data-host-sparkline]').forEach(el => {
const label = el.getAttribute('data-host-sparkline');
const sp = data[label];
if (!sp || !sp.hours || !sp.hours.length) return;
new Chart(el, {
type: 'line',
data: {
labels: sp.hours,
datasets: [{
data: sp.uptime_pct,
borderColor: '#0d6efd',
backgroundColor: 'rgba(13,110,253,0.15)',
borderWidth: 1.5,
fill: true,
tension: 0.4,
pointRadius: 0,
}]
},
options: {
responsive: true, maintainAspectRatio: false,
plugins: { legend: { display: false }, tooltip: { enabled: true,
callbacks: { label: c => `${c.label}: ${c.parsed.y.toFixed(0)}%` } } },
scales: {
x: { display: false },
y: { display: false, min: 0, max: 100 }
}
}
});
});
})();
</script>
{% endblock %}