V10.524 補 PChome 過期價格刷新入口
Some checks failed
CD Pipeline / deploy (push) Failing after 1m28s

This commit is contained in:
OoO
2026-06-01 00:29:20 +08:00
parent 915b8d061d
commit 87d47f171c
11 changed files with 259 additions and 46 deletions

View File

@@ -4,6 +4,7 @@
================================================================================
【已完成】
- V10.524 將「待刷新」變成可操作入口:商品看板 PChome 補抓產線新增「刷新過期 120 筆」按鈕,呼叫 `/api/ai/pchome-match/refresh-stale` 背景執行 `run_expired_identity_refresh()`,只刷新既有 `identity_v2` 的 PChome product_id不跑 fresh search recovery、不呼叫 LLM完成後重算 AI 挑品並清除 Dashboard / 競價快取。
- V10.523 補一批高分真同款 exact identity 比價規則Beauty Foot 足膜、KAMERIA 積雪草足膜、TS6 蜜愛潤滑液 / 蜜桃煥白凝膠 / 極淨白+煥白組合、Vaseline 嬰兒高純修護凝膠在規格、入數、品牌與品線完全對齊時可進 `exact / total_price / price_alert_exact`,讓可用比價覆蓋增加;同時保留 TS6 香味衣物手洗精等 variant-sensitive 款式在 `manual_review`,不放寬全域門檻。
- V10.522 將 PChome 補抓狀態 API 接上 read-only coverage snapshot`/api/ai/pchome-match/backfill/status` 會同步回傳身份覆蓋、新鮮率、待刷新與待補抓數Dashboard 補抓產線即使沒有最近任務結果,也能直接判讀下一步該刷新過期價格或補抓未搜尋商品。
- V10.521 將比價新鮮度 stale 指標上屏:首頁 KPI / PChome 補抓產線 / daily / growth 都顯示價格過期數,讓操作員分清「已確認同款但價格待刷新」與「尚未找到身份配對」;過期 identity refresh 也優先刷新 `total_price / price_alert_exact` 的正式價差配對。

View File

@@ -402,7 +402,7 @@ YOUTUBE_API_KEY = os.getenv('YOUTUBE_API_KEY', '')
# ==========================================
# 系統版本與路徑
# ==========================================
SYSTEM_VERSION = "V10.523"
SYSTEM_VERSION = "V10.524"
LOG_FILE_PATH = os.path.join(BASE_DIR, 'logs/system.log')
public_url = PUBLIC_URL # 用於模板顯示

View File

@@ -89,6 +89,7 @@ SQL漏斗(~300筆)
- 後台入口:`POST /api/ai/product-picks/generate``/ai_intelligence` 可手動產生清單。
- 配對來源仍以 PChome crawler 真實搜尋結果為準;無競品資料時不生成挑品。
- 比對覆蓋率補強入口:`POST /api/ai/pchome-match/backfill`,優先補抓仍無有效 PChome 配對的高價 ACTIVE 商品,完成後自動重算 AI 挑品清單。
- 過期價格刷新入口:`POST /api/ai/pchome-match/refresh-stale`,只針對已建立 `identity_v2``expires_at` 過期的 PChome product_id 執行 `run_expired_identity_refresh()`;不得跑 fresh search recovery不得呼叫 LLM完成後重算 AI 挑品並清除 Dashboard / competitor intel cache。
- 補抓狀態入口:`GET /api/ai/pchome-match/backfill/status` 除背景任務狀態外,必須回傳 read-only coverage snapshot`active_with_price` / `valid_matches` / `match_rate` / `fresh_matches` / `fresh_match_rate` / `stale_matches` / `pending` / `actionable_review_count`,供 Dashboard 顯示目前該刷新過期價格或補抓未搜尋商品;此端點不寫 DB、不呼叫 LLM、不抓外站。
- 排程閉環:`run_pchome_match_backfill_task` 每日 10:30 執行,補抓 PChome 待比對商品、寫入歷史價格,再重算 `strategy='product_pick'` 清單。
- PChome / MOMO 競價摘要出口 `services/competitor_intel_repository.py` 使用 30 分鐘共享快取(`COMPETITOR_INTEL_CACHE_TTL_SECONDS` 可調),避免 `/growth_analysis``/daily_sales`、PPT/AI 報表每次請求重跑昂貴覆蓋率與價差趨勢查詢;`run_competitor_price_feeder_task` 與 PChome backfill 完成後會主動清除快取。快取只包摘要輸出,不改 matcher 的高信心門檻與 identity_v2 準確性規則。

View File

@@ -13,6 +13,7 @@
## 📅 詳細更新日誌 (考古存檔)
### 2026-06-01PChome 比價新鮮度操作閉環
- **V10.524 PChome 過期價格刷新手動入口**: 商品看板 PChome 補抓產線新增「刷新過期 120 筆」按鈕與 `/api/ai/pchome-match/refresh-stale`,背景執行既有 `run_expired_identity_refresh()`,只刷新已建立 `identity_v2` 的 PChome product_id不跑 fresh search recovery、不呼叫 LLM完成後重算 AI 挑品並清除 Dashboard / competitor intel cache`stale_matches` 從觀測指標變成可直接操作的任務。
- **V10.523 高分真同款 exact identity 比價規則補強**: 針對正式環境反覆出現、分數已達 1.0 但因 `multi_component_pair` 或 variant review gate 被留在人工審核的真同款,補 Beauty Foot 足膜、KAMERIA 積雪草足膜、TS6 蜜愛潤滑液 / 蜜桃煥白凝膠 / 極淨白+煥白組合、Vaseline 嬰兒高純修護凝膠 focused exact identity。這些案例只有在品牌、品線、容量 / 重量與入數完全對齊時才進 `exact / total_price / price_alert_exact`TS6 香味衣物手洗精等款式敏感商品仍維持 `manual_review`,全域 `MIN_MATCH_SCORE` 與 overwrite 保護不變。
- **V10.522 PChome backfill status 附帶 coverage snapshot**: `/api/ai/pchome-match/backfill/status` 在背景任務狀態外同步回傳 read-only `fetch_competitor_coverage()` 摘要包含身份覆蓋、新鮮率、過期價格與待補抓數Dashboard 補抓產線即使尚無最近 run result也會顯示「身份覆蓋 / 新鮮 / 待刷新 / 待補抓」,讓操作員能分辨下一步該跑 expired identity refresh 還是 unmatched backfill。此狀態端點不寫 DB、不呼叫 LLM、不抓外站。

View File

@@ -1725,6 +1725,44 @@ def api_generate_product_picks():
return jsonify({'success': False, 'error': str(e)}), 500
def _feeder_result_payload(result):
return {
'total_skus': int(getattr(result, 'total_skus', 0) or 0),
'matched': int(getattr(result, 'matched', 0) or 0),
'skipped_no_result': int(getattr(result, 'skipped_no_result', 0) or 0),
'skipped_low_score': int(getattr(result, 'skipped_low_score', 0) or 0),
'errors': int(getattr(result, 'errors', 0) or 0),
'history_written': int(getattr(result, 'history_written', 0) or 0),
'attempts_written': int(getattr(result, 'attempts_written', 0) or 0),
'duration_sec': round(float(getattr(result, 'duration_sec', 0) or 0), 2),
}
def _pick_result_payload(result):
return {
'candidates': int(getattr(result, 'candidates', 0) or 0),
'written': int(getattr(result, 'written', 0) or 0),
'generated_at': getattr(result, 'generated_at', None),
}
def _combined_feeder_payload(revalidation_result, feeder_result):
revalidation_payload = _feeder_result_payload(revalidation_result)
feeder_payload = _feeder_result_payload(feeder_result)
return {
'total_skus': revalidation_payload['total_skus'] + feeder_payload['total_skus'],
'matched': revalidation_payload['matched'] + feeder_payload['matched'],
'skipped_no_result': revalidation_payload['skipped_no_result'] + feeder_payload['skipped_no_result'],
'skipped_low_score': revalidation_payload['skipped_low_score'] + feeder_payload['skipped_low_score'],
'errors': revalidation_payload['errors'] + feeder_payload['errors'],
'history_written': revalidation_payload['history_written'] + feeder_payload['history_written'],
'attempts_written': revalidation_payload['attempts_written'] + feeder_payload['attempts_written'],
'duration_sec': round(revalidation_payload['duration_sec'] + feeder_payload['duration_sec'], 2),
'retryable_candidate_revalidation': revalidation_payload,
'unmatched_priority_backfill': feeder_payload,
}
@ai_bp.route('/api/ai/pchome-match/backfill', methods=['POST'])
@login_required
def api_pchome_match_backfill():
@@ -1745,41 +1783,6 @@ def api_pchome_match_backfill():
update_pchome_backfill_run,
)
def _feeder_result_payload(result):
return {
'total_skus': int(getattr(result, 'total_skus', 0) or 0),
'matched': int(getattr(result, 'matched', 0) or 0),
'skipped_no_result': int(getattr(result, 'skipped_no_result', 0) or 0),
'skipped_low_score': int(getattr(result, 'skipped_low_score', 0) or 0),
'errors': int(getattr(result, 'errors', 0) or 0),
'history_written': int(getattr(result, 'history_written', 0) or 0),
'attempts_written': int(getattr(result, 'attempts_written', 0) or 0),
'duration_sec': round(float(getattr(result, 'duration_sec', 0) or 0), 2),
}
def _pick_result_payload(result):
return {
'candidates': int(getattr(result, 'candidates', 0) or 0),
'written': int(getattr(result, 'written', 0) or 0),
'generated_at': getattr(result, 'generated_at', None),
}
def _combined_feeder_payload(revalidation_result, feeder_result):
revalidation_payload = _feeder_result_payload(revalidation_result)
feeder_payload = _feeder_result_payload(feeder_result)
return {
'total_skus': revalidation_payload['total_skus'] + feeder_payload['total_skus'],
'matched': revalidation_payload['matched'] + feeder_payload['matched'],
'skipped_no_result': revalidation_payload['skipped_no_result'] + feeder_payload['skipped_no_result'],
'skipped_low_score': revalidation_payload['skipped_low_score'] + feeder_payload['skipped_low_score'],
'errors': revalidation_payload['errors'] + feeder_payload['errors'],
'history_written': revalidation_payload['history_written'] + feeder_payload['history_written'],
'attempts_written': revalidation_payload['attempts_written'] + feeder_payload['attempts_written'],
'duration_sec': round(revalidation_payload['duration_sec'] + feeder_payload['duration_sec'], 2),
'retryable_candidate_revalidation': revalidation_payload,
'unmatched_priority_backfill': feeder_payload,
}
try:
run = start_pchome_backfill_run(
limit=limit,
@@ -1789,12 +1792,13 @@ def api_pchome_match_backfill():
return jsonify({
'success': False,
'message': 'PChome 補抓已在執行中,請稍後查看進度',
'data': exc.status,
'data': _get_pchome_backfill_status_payload(),
}), 409
run_id = run['run_id']
def _run_backfill():
engine = None
try:
from config import DATABASE_PATH
from sqlalchemy import create_engine
@@ -1864,6 +1868,9 @@ def api_pchome_match_backfill():
except Exception as exc:
fail_pchome_backfill_run(run_id, str(exc))
logger.error(f"[PChomeBackfill] 背景補抓失敗: {exc}", exc_info=True)
finally:
if engine is not None:
engine.dispose()
thread = threading.Thread(target=_run_backfill, daemon=True)
thread.start()
@@ -1876,6 +1883,115 @@ def api_pchome_match_backfill():
}), 202
@ai_bp.route('/api/ai/pchome-match/refresh-stale', methods=['POST'])
@login_required
def api_pchome_match_refresh_stale():
"""背景刷新已建立 identity_v2 但價格過期的 PChome 商品。"""
import threading
payload = request.get_json(silent=True) or {}
try:
limit = max(5, min(int(payload.get('limit', 120)), 300))
except (TypeError, ValueError):
limit = 120
from services.pchome_backfill_status import (
PchomeBackfillAlreadyRunning,
fail_pchome_backfill_run,
finish_pchome_backfill_run,
start_pchome_backfill_run,
update_pchome_backfill_run,
)
try:
run = start_pchome_backfill_run(
limit=limit,
operator=session.get('username') or 'web',
)
except PchomeBackfillAlreadyRunning:
return jsonify({
'success': False,
'message': 'PChome 產線已有任務執行中,請稍後查看進度',
'data': _get_pchome_backfill_status_payload(),
}), 409
run_id = run['run_id']
def _run_refresh_stale():
engine = None
try:
from config import DATABASE_PATH
from sqlalchemy import create_engine
from services.ai_product_pick_agent import generate_product_pick_list
from services.cache_manager import clear_dashboard_cache
from services.competitor_intel_repository import clear_competitor_intel_cache
from services.competitor_price_feeder import CompetitorPriceFeeder
update_pchome_backfill_run(
run_id,
stage='refreshing_stale',
message=f'正在刷新 {limit} 筆過期 identity_v2 PChome 價格',
)
engine = create_engine(DATABASE_PATH)
feeder = CompetitorPriceFeeder(engine=engine)
result = feeder.run_expired_identity_refresh(limit=limit)
result_payload = _feeder_result_payload(result)
update_pchome_backfill_run(
run_id,
stage='generating_picks',
message='過期價格刷新完成,正在重算 AI 挑品清單',
result=result_payload,
)
pick_result = generate_product_pick_list(engine, limit=50)
pick_payload = _pick_result_payload(pick_result)
update_pchome_backfill_run(
run_id,
stage='clearing_cache',
message='AI 挑品已重算,正在清除看板快取',
result=result_payload,
pick_result=pick_payload,
)
clear_dashboard_cache()
clear_competitor_intel_cache()
finish_pchome_backfill_run(
run_id,
result=result_payload,
pick_result=pick_payload,
message=(
f"PChome 過期價格刷新完成:檢查 {result_payload['total_skus']} 筆、"
f"更新 {result_payload['matched']} 筆、"
f"AI 挑品寫入 {pick_payload['written']}"
),
)
logger.info(
"[PChomeRefreshStale] done total=%s matched=%s no=%s low=%s errors=%s history=%s duration=%ss pick_written=%s",
result_payload['total_skus'],
result_payload['matched'],
result_payload['skipped_no_result'],
result_payload['skipped_low_score'],
result_payload['errors'],
result_payload['history_written'],
result_payload['duration_sec'],
pick_result.written,
)
except Exception as exc:
fail_pchome_backfill_run(run_id, str(exc))
logger.error(f"[PChomeRefreshStale] 背景刷新失敗: {exc}", exc_info=True)
finally:
if engine is not None:
engine.dispose()
thread = threading.Thread(target=_run_refresh_stale, daemon=True)
thread.start()
return jsonify({
'success': True,
'message': f'已啟動 PChome 過期價格刷新,優先處理 {limit} 筆已建立 identity_v2 的舊價格',
'limit': limit,
'data': _get_pchome_backfill_status_payload(),
}), 202
def _build_pchome_backfill_coverage_payload():
"""讀取目前 PChome 身份覆蓋與價格新鮮度,供補抓狀態卡判斷下一步。"""
engine = None

View File

@@ -26,6 +26,7 @@ ACTIVE_TTL_SECONDS = int(os.getenv("PCHOME_BACKFILL_ACTIVE_TTL_SECONDS", "7200")
STAGE_ORDER = (
"queued",
"refreshing_stale",
"revalidating",
"matching",
"generating_picks",
@@ -36,12 +37,13 @@ STAGE_ORDER = (
STAGE_LABELS = {
"idle": "尚未執行",
"queued": "已排入背景補抓",
"refreshing_stale": "刷新過期 PChome 價格",
"revalidating": "重新評分近門檻候選",
"matching": "比對高優先未配對商品",
"generating_picks": "重算 AI 挑品清單",
"clearing_cache": "清除看板與競價快取",
"completed": "補抓完成",
"failed": "補抓失敗",
"completed": "產線完成",
"failed": "產線失敗",
"stale": "執行狀態逾時",
}

View File

@@ -64,6 +64,7 @@
<div class="dashboard-backfill-card"
data-pchome-backfill-card
data-backfill-endpoint="/api/ai/pchome-match/backfill"
data-refresh-stale-endpoint="/api/ai/pchome-match/refresh-stale"
data-status-endpoint="/api/ai/pchome-match/backfill/status"
data-pchome-backfill-action="backfillPchomeMatches">
<div class="dashboard-backfill-main">
@@ -80,12 +81,20 @@
<span data-pchome-backfill-status>讀取狀態中</span>
<span data-pchome-backfill-result>--</span>
</div>
<button class="dashboard-action-button is-primary"
type="button"
data-pchome-backfill-trigger
data-limit="60">
<i class="fas fa-magnifying-glass-chart"></i> 補抓 60 筆
</button>
<div class="dashboard-backfill-actions">
<button class="dashboard-action-button"
type="button"
data-pchome-refresh-stale-trigger
data-limit="120">
<i class="fas fa-rotate"></i> 刷新過期 120 筆
</button>
<button class="dashboard-action-button is-primary"
type="button"
data-pchome-backfill-trigger
data-limit="60">
<i class="fas fa-magnifying-glass-chart"></i> 補抓 60 筆
</button>
</div>
</div>
</section>

View File

@@ -516,11 +516,14 @@ def test_ai_product_pick_agent_uses_real_competitor_data_and_dashboard_action():
assert "@ai_bp.route('/api/ai/product-picks/generate', methods=['POST'])" in route_source
assert "generate_product_pick_list(engine" in route_source
assert "@ai_bp.route('/api/ai/pchome-match/backfill', methods=['POST'])" in route_source
assert "@ai_bp.route('/api/ai/pchome-match/refresh-stale', methods=['POST'])" in route_source
assert "@ai_bp.route('/api/ai/pchome-match/backfill/status', methods=['GET'])" in route_source
assert "_build_pchome_backfill_coverage_payload" in route_source
assert "fetch_competitor_coverage" in route_source
assert "status['coverage'] = _build_pchome_backfill_coverage_payload()" in route_source
assert "run_unmatched_priority(limit=unmatched_limit)" in route_source
assert "run_expired_identity_refresh(limit=limit)" in route_source
assert "stage='refreshing_stale'" in route_source
assert "run_retryable_candidate_revalidation" in route_source
assert "generate_product_pick_list(engine, limit=50)" in route_source
assert "start_pchome_backfill_run" in route_source
@@ -556,12 +559,16 @@ def test_ai_product_pick_agent_uses_real_competitor_data_and_dashboard_action():
assert "backfillPchomeMatches" in template
assert "/api/ai/product-picks/generate" in template
assert "/api/ai/pchome-match/backfill" in template
assert "/api/ai/pchome-match/refresh-stale" in dashboard_template
assert "/api/ai/pchome-match/backfill/status" in dashboard_template
assert "PCHOME MATCH BACKFILL" in dashboard_template
assert "data-pchome-backfill-trigger" in dashboard_template
assert "data-pchome-refresh-stale-trigger" in dashboard_template
assert "刷新過期 120 筆" in dashboard_template
dashboard_js = (ROOT / "web/static/js/page-dashboard-v2.js").read_text(encoding="utf-8")
assert "loadPchomeBackfillStatus" in dashboard_js
assert "window.backfillPchomeMatches" in dashboard_js
assert "window.refreshStalePchomeMatches" in dashboard_js
assert "formatBackfillCoverageSummary" in dashboard_js
assert "status.coverage" in dashboard_js
assert "待刷新 ${formatBackfillCount(coverage.stale_matches)}" in dashboard_js

View File

@@ -55,3 +55,18 @@ def test_pchome_backfill_status_records_failure(tmp_path, monkeypatch):
assert failed["status"] == "failed"
assert failed["last_error"] == "crawler timeout"
assert failed["recent_runs"][0]["last_error"] == "crawler timeout"
def test_pchome_backfill_status_supports_stale_refresh_stage(tmp_path, monkeypatch):
monkeypatch.setenv("PCHOME_BACKFILL_STATUS_PATH", str(tmp_path / "status.json"))
run = start_pchome_backfill_run(limit=120, operator="tester")
refreshing = update_pchome_backfill_run(
run["run_id"],
stage="refreshing_stale",
message="正在刷新過期價格",
)
assert refreshing["stage"] == "refreshing_stale"
assert refreshing["stage_label"] == "刷新過期 PChome 價格"
assert refreshing["progress_pct"] > run["progress_pct"]

View File

@@ -244,6 +244,14 @@
gap: 2px;
}
.dashboard-backfill-actions {
display: flex;
justify-content: flex-end;
gap: 8px;
flex-wrap: wrap;
min-width: 0;
}
.dashboard-focus-grid {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
@@ -1276,6 +1284,10 @@
width: 100%;
}
.dashboard-backfill-actions {
width: 100%;
}
.dashboard-search,
.dashboard-select,
.dashboard-segmented {

View File

@@ -287,10 +287,12 @@ let priceChartInstance = null;
return {
card,
trigger: document.querySelector('[data-pchome-backfill-trigger]'),
refreshStaleTrigger: document.querySelector('[data-pchome-refresh-stale-trigger]'),
status: document.querySelector('[data-pchome-backfill-status]'),
result: document.querySelector('[data-pchome-backfill-result]'),
progress: document.querySelector('[data-pchome-backfill-progress]'),
backfillEndpoint: card ? card.dataset.backfillEndpoint : '/api/ai/pchome-match/backfill',
refreshStaleEndpoint: card ? card.dataset.refreshStaleEndpoint : '/api/ai/pchome-match/refresh-stale',
statusEndpoint: card ? card.dataset.statusEndpoint : '/api/ai/pchome-match/backfill/status'
};
}
@@ -366,9 +368,16 @@ let priceChartInstance = null;
elements.trigger.disabled = running;
elements.trigger.classList.toggle('is-loading', running);
elements.trigger.innerHTML = running
? '<i class="fas fa-spinner fa-spin"></i> 補抓中'
? '<i class="fas fa-spinner fa-spin"></i> 執行中'
: '<i class="fas fa-search"></i> 補抓 60 筆';
}
if (elements.refreshStaleTrigger) {
elements.refreshStaleTrigger.disabled = running;
elements.refreshStaleTrigger.classList.toggle('is-loading', running);
elements.refreshStaleTrigger.innerHTML = running
? '<i class="fas fa-spinner fa-spin"></i> 執行中'
: '<i class="fas fa-rotate"></i> 刷新過期 120 筆';
}
if (running) {
schedulePchomeBackfillPoll();
@@ -430,10 +439,50 @@ let priceChartInstance = null;
});
}
function refreshStalePchomeMatches() {
const elements = getPchomeBackfillElements();
if (!elements.card || !elements.refreshStaleTrigger) return;
const limit = Number(elements.refreshStaleTrigger.dataset.limit || 120);
if (!confirm(`啟動 PChome 過期價格刷新 ${limit} 筆?`)) return;
elements.refreshStaleTrigger.disabled = true;
if (elements.status) {
elements.status.textContent = '正在送出過期價格刷新任務';
}
fetch(elements.refreshStaleEndpoint, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-CSRFToken': getCSRFToken()
},
body: JSON.stringify({ limit })
})
.then(response => response.json().then(data => ({ ok: response.ok, status: response.status, data })))
.then(({ ok, status, data }) => {
renderPchomeBackfillStatus(data);
if (!ok && status !== 409) {
throw new Error(data.message || data.error || 'PChome 過期價格刷新啟動失敗');
}
schedulePchomeBackfillPoll();
})
.catch(error => {
if (elements.status) {
elements.status.textContent = error.message || 'PChome 過期價格刷新啟動失敗';
}
if (elements.refreshStaleTrigger) {
elements.refreshStaleTrigger.disabled = false;
}
});
}
window.backfillPchomeMatches = backfillPchomeMatches;
window.refreshStalePchomeMatches = refreshStalePchomeMatches;
document.querySelectorAll('[data-pchome-backfill-trigger]').forEach(button => {
button.addEventListener('click', backfillPchomeMatches);
});
document.querySelectorAll('[data-pchome-refresh-stale-trigger]').forEach(button => {
button.addEventListener('click', refreshStalePchomeMatches);
});
loadPchomeBackfillStatus();
function runPchomeReviewDecision(button) {