V10.522 補 PChome backfill coverage 狀態
Some checks failed
CD Pipeline / deploy (push) Failing after 1m5s
Some checks failed
CD Pipeline / deploy (push) Failing after 1m5s
This commit is contained in:
OoO
@@ -4,6 +4,7 @@
|
||||
================================================================================
|
||||
|
||||
【已完成】
|
||||
- V10.522 將 PChome 補抓狀態 API 接上 read-only coverage snapshot：`/api/ai/pchome-match/backfill/status` 會同步回傳身份覆蓋、新鮮率、待刷新與待補抓數，Dashboard 補抓產線即使沒有最近任務結果，也能直接判讀下一步該刷新過期價格或補抓未搜尋商品。
|
||||
- V10.521 將比價新鮮度 stale 指標上屏：首頁 KPI / PChome 補抓產線 / daily / growth 都顯示價格過期數，讓操作員分清「已確認同款但價格待刷新」與「尚未找到身份配對」；過期 identity refresh 也優先刷新 `total_price / price_alert_exact` 的正式價差配對。
|
||||
- V10.520 拆開過期價格刷新與搜尋救援：`run_expired_identity_refresh()` 只刷新既有 `identity_v2` PChome product_id，不再因少數 product_id 查不到或低分而同步進入慢速 `fresh_search_recovery`；缺失 / 低分候選交給 `run_retryable_candidate_revalidation()` 處理，避免正式刷新 500+ 筆時被外部搜尋拖死，讓價格新鮮度可以穩定批次回升。
|
||||
- V10.519 對齊 Webcrumbs host data metadata 與新版比價覆蓋口徑：`services/webcrumbs_host_data_service.py` 會同時輸出身份覆蓋、價格新鮮、過期配對與待補抓數，讓 shared-ui plugin / 其他專案 proxy 不會把 `coverage_rate` 誤讀成價格可用率。
|
||||
|
||||
@@ -402,7 +402,7 @@ YOUTUBE_API_KEY = os.getenv('YOUTUBE_API_KEY', '')
|
||||
# ==========================================
|
||||
# 系統版本與路徑
|
||||
# ==========================================
|
||||
SYSTEM_VERSION = "V10.521"
|
||||
SYSTEM_VERSION = "V10.522"
|
||||
LOG_FILE_PATH = os.path.join(BASE_DIR, 'logs/system.log')
|
||||
public_url = PUBLIC_URL # 用於模板顯示
|
||||
|
||||
|
||||
@@ -89,6 +89,7 @@ SQL漏斗(~300筆)
|
||||
- 後台入口：`POST /api/ai/product-picks/generate`，`/ai_intelligence` 可手動產生清單。
|
||||
- 配對來源仍以 PChome crawler 真實搜尋結果為準；無競品資料時不生成挑品。
|
||||
- 比對覆蓋率補強入口：`POST /api/ai/pchome-match/backfill`，優先補抓仍無有效 PChome 配對的高價 ACTIVE 商品，完成後自動重算 AI 挑品清單。
|
||||
- 補抓狀態入口：`GET /api/ai/pchome-match/backfill/status` 除背景任務狀態外，必須回傳 read-only coverage snapshot：`active_with_price` / `valid_matches` / `match_rate` / `fresh_matches` / `fresh_match_rate` / `stale_matches` / `pending` / `actionable_review_count`，供 Dashboard 顯示目前該刷新過期價格或補抓未搜尋商品；此端點不寫 DB、不呼叫 LLM、不抓外站。
|
||||
- 排程閉環：`run_pchome_match_backfill_task` 每日 10:30 執行，補抓 PChome 待比對商品、寫入歷史價格，再重算 `strategy='product_pick'` 清單。
|
||||
- PChome / MOMO 競價摘要出口 `services/competitor_intel_repository.py` 使用 30 分鐘共享快取（`COMPETITOR_INTEL_CACHE_TTL_SECONDS` 可調），避免 `/growth_analysis`、`/daily_sales`、PPT/AI 報表每次請求重跑昂貴覆蓋率與價差趨勢查詢；`run_competitor_price_feeder_task` 與 PChome backfill 完成後會主動清除快取。快取只包摘要輸出，不改 matcher 的高信心門檻與 identity_v2 準確性規則。
|
||||
- 商品看板第一屏：`/` 的 V2 看板直接以 `products`、`price_records`、`competitor_prices`、`competitor_match_attempts`、`competitor_match_reviews`、`ai_price_recommendations` 顯示比對覆蓋率、PChome 優勢、MOMO 威脅、AI 挑品、待比對優先清單與 PChome 覆核隊列；`filter=ai_picks` 可查看 50 品 AI 挑品列表，`filter=pchome_review` 可直接查看需人工處理的比價覆核 SKU，並以 DB 分頁支援 search/category/status 後的完整隊列，不得只截前 50 筆。覆核狀態篩選必須至少包含全部、需單位價、已排除、低信心、價格過期、找不到同款與人工閉環，讓人工可依 matcher 診斷類型分批處理。列內顯示候選 PChome 商品、候選價、match score、單位價換算摘要、人工動作與 matcher 診斷原因標籤（品牌不符、商品線不符、容量差異、組合差異、需單位價、價差極端等），不得只顯示籠統「待比對」。`/api/export/excel/pchome-review` 必須匯出同一套覆核隊列、人工處置、候選 PChome、單位價比較與原始診斷，讓人工覆核、簡報與後續 AI 分析共用同一份證據。`/api/pchome-review/<sku>/decision` 是人工閉環入口：`accept_identity` 才可把候選寫入 `competitor_prices` 與 `competitor_price_history` 並打上 `manual_review/manual_accept/identity_v2`；`reject_identity`、`unit_price_required` 與 `needs_research` 只寫 `competitor_match_reviews` 並追加 manual attempt，不得把不同販售組合或否決候選灌入正式價差。PChome feeder 後續搜尋同一候選時必須讀取 `competitor_match_reviews`：已否決候選寫 `manual_rejected` 並跳過正式寫入，且必須繼續評估下一個候選，不能讓已否決候選長期阻塞同 SKU；已標記單位價候選寫 `manual_unit_price_required`；已要求補搜尋候選寫 `manual_needs_research` 並停留在覆核隊列；已採用候選可保守補到最低門檻並保留 `manual_review/manual_accept` 標籤。搜尋候選池只有強同款分數達 `0.90` 才可提前停止，避免 0.76 灰區候選卡掉後續更精準搜尋詞。人工 `reject_identity`、`unit_price_required`、`needs_research` 若命中當前正式候選，必須將同候選 `competitor_prices` 過期，不得繼續顯示正式總價差。商品列表必須將 `manual_rejected`、`manual_unit_price_required`、`manual_needs_research` 顯示為明確人工閉環狀態，不可回落成籠統「待比對」。`fetch_competitor_coverage()` 必須輸出人工採用、人工否決、人工單位價與採用率，daily/growth/PPT 共用 payload 必須顯示人工閉環成效，避免只呈現待審數。商品看板深度快取同時寫入 `data/dashboard_full_cache.pkl`，供多個 Gunicorn worker 共用，避免部署後各 worker 重複重建 7,000+ 商品統計造成開頁變慢；所有資料異動與 AI 挑品重算都透過 `clear_dashboard_cache()` 同步清除記憶體與共享快取，手動重算 API 會立即預熱商品看板快取，避免第一位使用者承擔重建成本。
|
||||
|
||||
@@ -12,6 +12,9 @@
|
||||
|
||||
## 📅 詳細更新日誌 (考古存檔)
|
||||
|
||||
### 2026-06-01：PChome 比價新鮮度操作閉環
|
||||
- **V10.522 PChome backfill status 附帶 coverage snapshot**: `/api/ai/pchome-match/backfill/status` 在背景任務狀態外同步回傳 read-only `fetch_competitor_coverage()` 摘要，包含身份覆蓋、新鮮率、過期價格與待補抓數；Dashboard 補抓產線即使尚無最近 run result，也會顯示「身份覆蓋 / 新鮮 / 待刷新 / 待補抓」，讓操作員能分辨下一步該跑 expired identity refresh 還是 unmatched backfill。此狀態端點不寫 DB、不呼叫 LLM、不抓外站。
|
||||
|
||||
### 2026-05-31：Webcrumbs 共用 UI Runtime 與市場情報 writer approval
|
||||
- **V10.521 比價新鮮度 stale 指標上屏**: 首頁比價監控總覽、PChome 補抓產線、daily 競價覆蓋與 growth 比價資料品質同步顯示 `stale_matches` / 價格過期數，讓操作員能分清「已確認同款但價格待刷新」與「尚未找到身份配對」，不再只看到新鮮率下降。過期 identity refresh 也優先刷新 `total_price / price_alert_exact` 的正式價差配對，讓最能進決策與告警的舊價格先回新鮮。
|
||||
- **V10.520 PChome 過期價格刷新快慢路徑拆分**: `run_expired_identity_refresh()` 改為只刷新已確認 `identity_v2` 的既有 PChome product_id；若 product_id 已查不到或回傳後低分，不再同步跑慢速 fresh search recovery，而是記錄 `refresh_no_result` / low-score 並交給 `run_retryable_candidate_revalidation()` 的近門檻救援路徑。這能避免正式回刷 500+ 筆時被少數缺失 ID 拖到長時間卡住，讓價格新鮮度批次回升更可控。
|
||||
|
||||
@@ -1741,7 +1741,6 @@ def api_pchome_match_backfill():
|
||||
PchomeBackfillAlreadyRunning,
|
||||
fail_pchome_backfill_run,
|
||||
finish_pchome_backfill_run,
|
||||
get_pchome_backfill_status,
|
||||
start_pchome_backfill_run,
|
||||
update_pchome_backfill_run,
|
||||
)
|
||||
@@ -1873,19 +1872,62 @@ def api_pchome_match_backfill():
|
||||
'success': True,
|
||||
'message': f'已啟動 PChome 未搜尋補抓，優先處理 {limit} 筆高價未配對商品；完成後會重算 AI 挑品清單',
|
||||
'limit': limit,
|
||||
'data': get_pchome_backfill_status(),
|
||||
'data': _get_pchome_backfill_status_payload(),
|
||||
}), 202
|
||||
|
||||
|
||||
def _build_pchome_backfill_coverage_payload():
|
||||
"""讀取目前 PChome 身份覆蓋與價格新鮮度，供補抓狀態卡判斷下一步。"""
|
||||
engine = None
|
||||
try:
|
||||
from config import DATABASE_PATH
|
||||
from sqlalchemy import create_engine
|
||||
from services.competitor_intel_repository import fetch_competitor_coverage
|
||||
|
||||
engine = create_engine(DATABASE_PATH)
|
||||
coverage = fetch_competitor_coverage(engine) or {}
|
||||
return {
|
||||
'available': True,
|
||||
'active_with_price': int(coverage.get('active_with_price') or 0),
|
||||
'valid_matches': int(coverage.get('valid_matches') or 0),
|
||||
'match_rate': float(coverage.get('match_rate') or 0),
|
||||
'fresh_matches': int(coverage.get('fresh_matches') or 0),
|
||||
'fresh_match_rate': float(coverage.get('fresh_match_rate') or 0),
|
||||
'stale_matches': int(coverage.get('stale_matches') or 0),
|
||||
'pending': int(coverage.get('pending') or 0),
|
||||
'actionable_review_count': int(coverage.get('actionable_review_count') or 0),
|
||||
'unit_comparable_count': int(coverage.get('unit_comparable_count') or 0),
|
||||
'rescore_accepted_count': int(coverage.get('rescore_accepted_count') or 0),
|
||||
'manual_accept_count': int(coverage.get('manual_accept_count') or 0),
|
||||
'manual_reject_count': int(coverage.get('manual_reject_count') or 0),
|
||||
'manual_unit_price_count': int(coverage.get('manual_unit_price_count') or 0),
|
||||
}
|
||||
except Exception as exc:
|
||||
logger.warning(f"[PChomeBackfill] coverage snapshot unavailable: {exc}")
|
||||
return {
|
||||
'available': False,
|
||||
'error': str(exc),
|
||||
}
|
||||
finally:
|
||||
if engine is not None:
|
||||
engine.dispose()
|
||||
|
||||
|
||||
def _get_pchome_backfill_status_payload():
|
||||
from services.pchome_backfill_status import get_pchome_backfill_status
|
||||
|
||||
status = get_pchome_backfill_status()
|
||||
status['coverage'] = _build_pchome_backfill_coverage_payload()
|
||||
return status
|
||||
|
||||
|
||||
@ai_bp.route('/api/ai/pchome-match/backfill/status', methods=['GET'])
|
||||
@login_required
|
||||
def api_pchome_match_backfill_status():
|
||||
"""取得 PChome 未搜尋補抓的背景執行狀態。"""
|
||||
from services.pchome_backfill_status import get_pchome_backfill_status
|
||||
|
||||
return jsonify({
|
||||
'success': True,
|
||||
'data': get_pchome_backfill_status(),
|
||||
'data': _get_pchome_backfill_status_payload(),
|
||||
})
|
||||
|
||||
|
||||
|
||||
@@ -517,6 +517,9 @@ def test_ai_product_pick_agent_uses_real_competitor_data_and_dashboard_action():
|
||||
assert "generate_product_pick_list(engine" in route_source
|
||||
assert "@ai_bp.route('/api/ai/pchome-match/backfill', methods=['POST'])" in route_source
|
||||
assert "@ai_bp.route('/api/ai/pchome-match/backfill/status', methods=['GET'])" in route_source
|
||||
assert "_build_pchome_backfill_coverage_payload" in route_source
|
||||
assert "fetch_competitor_coverage" in route_source
|
||||
assert "status['coverage'] = _build_pchome_backfill_coverage_payload()" in route_source
|
||||
assert "run_unmatched_priority(limit=unmatched_limit)" in route_source
|
||||
assert "run_retryable_candidate_revalidation" in route_source
|
||||
assert "generate_product_pick_list(engine, limit=50)" in route_source
|
||||
@@ -559,6 +562,10 @@ def test_ai_product_pick_agent_uses_real_competitor_data_and_dashboard_action():
|
||||
dashboard_js = (ROOT / "web/static/js/page-dashboard-v2.js").read_text(encoding="utf-8")
|
||||
assert "loadPchomeBackfillStatus" in dashboard_js
|
||||
assert "window.backfillPchomeMatches" in dashboard_js
|
||||
assert "formatBackfillCoverageSummary" in dashboard_js
|
||||
assert "status.coverage" in dashboard_js
|
||||
assert "待刷新 ${formatBackfillCount(coverage.stale_matches)}" in dashboard_js
|
||||
assert "待補抓 ${formatBackfillCount(coverage.pending)}" in dashboard_js
|
||||
assert "'product_pick':['bg-success'" in template
|
||||
assert "kpiMatchRate" in template
|
||||
|
||||
|
||||
@@ -299,6 +299,22 @@ let priceChartInstance = null;
|
||||
return Number(value || 0).toLocaleString();
|
||||
}
|
||||
|
||||
function formatBackfillRate(value) {
|
||||
const numeric = Number(value || 0);
|
||||
if (!Number.isFinite(numeric)) return '0%';
|
||||
return `${numeric.toFixed(1).replace(/\.0$/, '')}%`;
|
||||
}
|
||||
|
||||
function formatBackfillCoverageSummary(coverage) {
|
||||
if (!coverage || coverage.available === false) return '';
|
||||
return (
|
||||
`身份覆蓋 ${formatBackfillRate(coverage.match_rate)}`
|
||||
+ ` · 新鮮 ${formatBackfillRate(coverage.fresh_match_rate)}`
|
||||
+ ` · 待刷新 ${formatBackfillCount(coverage.stale_matches)}`
|
||||
+ ` · 待補抓 ${formatBackfillCount(coverage.pending)}`
|
||||
);
|
||||
}
|
||||
|
||||
function schedulePchomeBackfillPoll() {
|
||||
if (pchomeBackfillPollTimer) {
|
||||
clearTimeout(pchomeBackfillPollTimer);
|
||||
@@ -314,6 +330,7 @@ let priceChartInstance = null;
|
||||
const currentRun = status.current_run || {};
|
||||
const result = currentRun.result || status.last_result || {};
|
||||
const pickResult = currentRun.pick_result || {};
|
||||
const coverageSummary = formatBackfillCoverageSummary(status.coverage);
|
||||
const running = Boolean(status.running || currentRun.running);
|
||||
const progressPct = Math.max(0, Math.min(Number(status.progress_pct || currentRun.progress_pct || 0), 100));
|
||||
const statusKey = status.status || currentRun.status || 'idle';
|
||||
@@ -337,9 +354,12 @@ let priceChartInstance = null;
|
||||
+ ` · 待覆核 ${formatBackfillCount(result.skipped_low_score)}`
|
||||
+ ` · 無結果 ${formatBackfillCount(result.skipped_no_result)}`
|
||||
+ pickWritten
|
||||
+ (coverageSummary ? ` · ${coverageSummary}` : '')
|
||||
);
|
||||
} else {
|
||||
elements.result.textContent = running ? '正在累積結果' : '尚無最近結果';
|
||||
elements.result.textContent = running
|
||||
? (coverageSummary ? `正在累積結果 · ${coverageSummary}` : '正在累積結果')
|
||||
: (coverageSummary || '尚無最近結果');
|
||||
}
|
||||
}
|
||||
if (elements.trigger) {
|
||||
|
||||
Reference in New Issue
Block a user