新增市場情報人工樣本抓取計畫
All checks were successful
CD Pipeline / deploy (push) Successful in 1m3s

This commit is contained in:
OoO
2026-05-19 00:09:56 +08:00
parent f51dc173f7
commit 774f1b4b45
10 changed files with 440 additions and 31 deletions

View File

@@ -98,6 +98,7 @@
- Phase 27 legacy source bridge preview新增 `services/market_intel/legacy_source_bridge.py` 與 `/api/market_intel/legacy_source_bridge`,只讀盤點既有 `promo_products`、`competitor_prices`、`competitor_price_history`,產生導入 `market_*` 的 mapping / dedupe / blocked operation preview預設 `execute=false` 不連 DB`execute=true` 也只做 read-only query不寫 DB、不建立 ORM session、不連外、不掛 schedulerUI 新增「既有資料橋接預覽」panel版本同步至 V10.182。
- Phase 45 migration live smoke preview新增 `/api/market_intel/migration_live_smoke` 與 UI「正式 DB 只讀 smoke」panel預設 `execute=false` 只回 planned人工 smoke 才可用 `execute=true` 整理 catalog / seed diff 結果,不執行 migration、不寫 DB、不跑 rollback、不掛 scheduler版本同步至 V10.207。
- Phase 46 live DB inventory preview新增 `/api/market_intel/live_db_inventory` 與 UI「正式 DB 庫存總覽」panel預設 `execute=false` 不連 DB人工 smoke 才可用 `execute=true` 對 `market_*` tables 做只讀 count / group by建立平台、活動、商品、比對、告警與 crawler run 基準;版本同步至 V10.209。
- Phase 47 manual sample fetch plan新增 `/api/market_intel/manual_sample_plan` 與 UI「人工樣本 Fetch 計畫」panel預設只輸出平台順序、每平台 1 個公開入口、MCP gate、操作員步驟與備援不抓外站、不寫 DB、不建立 crawler run、不掛 scheduler版本同步至 V10.214。
- Schema smoke`tests/test_market_intel_skeleton.py` 檢查 `Base.metadata` 內含 ADR-035 八張 `market_*` tables。
- Desktop UI QA本機只註冊 `market_intel_bp` 的 Flask harness 載入 `/market_intel`,確認 Phase 15、候選預覽、writer preview、安全 flags、點陣暖紙視覺正常console error 0。
- API QA`/api/market_intel/schema_smoke` 通過 7 張表與 `market_platforms` 必要欄位檢查;`/api/market_intel/platform_seed_writer_plan` 回傳 4 筆 dry-run upsert preview`writes_executed=false`,四平台皆 `blocked_dry_run_only`。

View File

@@ -320,7 +320,7 @@ YOUTUBE_API_KEY = os.getenv('YOUTUBE_API_KEY', '')
# ==========================================
# 系統版本與路徑
# ==========================================
SYSTEM_VERSION = "V10.213"
SYSTEM_VERSION = "V10.214"
LOG_FILE_PATH = os.path.join(BASE_DIR, 'logs/system.log')
public_url = PUBLIC_URL # 用於模板顯示

View File

@@ -172,6 +172,7 @@ EwoooC 目前已有 MOMO EDM / 節慶活動資料、`promo_products`、PChome
- 2026-05-18 追加 migration catalog review preview`services.market_intel.migration_catalog_review``/api/market_intel/migration_catalog_review` 將 schema_db_probe 與 platform_seed_db_diff 的只讀結果歸納為 `planned_no_probe` / `not_applied` / `partial_schema` / `already_applied` / `probe_error`,並輸出 risk level、apply path、finding 與下一步。預設 `execute=false` 不連 DB`execute=true` 仍只查 catalog / seed diff不執行 psql、不寫 DB、不跑 rollback、不掛 scheduler。
- 2026-05-18 追加 migration live smoke preview`services.market_intel.migration_live_smoke``/api/market_intel/migration_live_smoke``execute=true` 的正式 DB 只讀探測整理成 smoke result例如 `passed_not_applied_seed_table_missing``attention_partial_schema``passed_already_applied``failed_catalog_probe_error`。UI 預設仍只呼叫 `execute=false`;人工 smoke 可用 `execute=true` 只讀查 catalog不執行 migration、不寫 DB、不跑 rollback、不掛 scheduler。
- 2026-05-18 追加 live DB inventory preview`services.market_intel.live_db_inventory``/api/market_intel/live_db_inventory` 預設 `execute=false` 只回 planned不連 DB人工 smoke 可用 `execute=true``market_*` tables 執行只讀 count / group by產生平台、活動狀態、商品活躍度、比對狀態、告警審核佇列與 crawler run 基準總覽。此探針不建立 ORM session、不寫 DB、不執行 migration、不連外、不掛 scheduler。
- 2026-05-19 追加 manual sample fetch plan preview`services.market_intel.manual_sample_plan``/api/market_intel/manual_sample_plan` 定義第一次人工 sample fetch 的平台順序、每平台 1 個公開入口、MCP fetch gate、正式 DB 庫存基準、操作員步驟與備援。此階段只產生計畫,不抓外部網站、不寫 DB、不建立 crawler run、不掛 scheduler、不繞反爬。
### Phase 4Coupang / Shopee Adapter

View File

@@ -19,7 +19,7 @@
| `edm_routes.py` | EDM 與節慶儀表板 | `/edm`, `/festival` |
| `monthly_routes.py` | 月結分析 | `/monthly_summary_analysis`, `/api/monthly_summary_data` |
| `daily_sales_routes.py` | 當日業績 | `/daily_sales`, `/daily_sales/export*` |
| `market_intel_routes.py` | 市場情報 Phase 46 live DB inventory preview | `/market_intel`, `/market_intel/*`, `/api/market_intel/status`, `/api/market_intel/schema`, `/api/market_intel/schema_smoke`, `/api/market_intel/schema_db_probe`, `/api/market_intel/platform_seed_db_diff`, `/api/market_intel/legacy_source_bridge`, `/api/market_intel/mcp_readiness`, `/api/market_intel/mcp_tool_contract`, `/api/market_intel/mcp_deploy_preflight`, `/api/market_intel/mcp_activation_runbook`, `/api/market_intel/mcp_fetch_gate`, `/api/market_intel/scheduler_plan`, `/api/market_intel/match_review_plan`, `/api/market_intel/opportunity_plan`, `/api/market_intel/opportunity_scoring_plan`, `/api/market_intel/opportunity_evidence_plan`, `/api/market_intel/opportunity_alert_plan`, `/api/market_intel/adapters`, `/api/market_intel/dry_run_plan`, `/api/market_intel/discovery_plan`, `/api/market_intel/manual_discovery`, `/api/market_intel/candidate_preview`, `/api/market_intel/platform_seed_plan`, `/api/market_intel/platform_seed_write_guard`, `/api/market_intel/platform_seed_writer_plan`, `/api/market_intel/migration_blueprint`, `/api/market_intel/migration_apply_drill`, `/api/market_intel/migration_catalog_review`, `/api/market_intel/migration_live_smoke`, `/api/market_intel/live_db_inventory`, `/api/market_intel/seed_writer_cli_status`, `/api/market_intel/write_approval_runbook`, `/api/market_intel/deployment_readiness` |
| `market_intel_routes.py` | 市場情報 Phase 47 manual sample fetch plan | `/market_intel`, `/market_intel/*`, `/api/market_intel/status`, `/api/market_intel/schema`, `/api/market_intel/schema_smoke`, `/api/market_intel/schema_db_probe`, `/api/market_intel/platform_seed_db_diff`, `/api/market_intel/legacy_source_bridge`, `/api/market_intel/mcp_readiness`, `/api/market_intel/mcp_tool_contract`, `/api/market_intel/mcp_deploy_preflight`, `/api/market_intel/mcp_activation_runbook`, `/api/market_intel/mcp_fetch_gate`, `/api/market_intel/scheduler_plan`, `/api/market_intel/manual_sample_plan`, `/api/market_intel/match_review_plan`, `/api/market_intel/opportunity_plan`, `/api/market_intel/opportunity_scoring_plan`, `/api/market_intel/opportunity_evidence_plan`, `/api/market_intel/opportunity_alert_plan`, `/api/market_intel/adapters`, `/api/market_intel/dry_run_plan`, `/api/market_intel/discovery_plan`, `/api/market_intel/manual_discovery`, `/api/market_intel/candidate_preview`, `/api/market_intel/platform_seed_plan`, `/api/market_intel/platform_seed_write_guard`, `/api/market_intel/platform_seed_writer_plan`, `/api/market_intel/migration_blueprint`, `/api/market_intel/migration_apply_drill`, `/api/market_intel/migration_catalog_review`, `/api/market_intel/migration_live_smoke`, `/api/market_intel/live_db_inventory`, `/api/market_intel/seed_writer_cli_status`, `/api/market_intel/write_approval_runbook`, `/api/market_intel/deployment_readiness` |
| `api_routes.py` | 通用任務與查詢 API | `/api/run_task`, `/api/history/*` |
| `export_routes.py` | 匯出功能 | `/api/export/*` |
| `import_routes.py` | 匯入功能 | `/api/import_excel`, `/api/import/monthly_summary` |

View File

@@ -157,6 +157,12 @@ def market_intel_scheduler_plan():
return jsonify(_service().build_scheduler_plan())
@market_intel_bp.route("/api/market_intel/manual_sample_plan")
@login_required
def market_intel_manual_sample_plan():
return jsonify(_service().build_manual_sample_plan())
@market_intel_bp.route("/api/market_intel/match_review_plan")
@login_required
def market_intel_match_review_plan():

View File

@@ -18,6 +18,7 @@ def build_deployment_readiness_preview(
mcp_activation_runbook = service.build_mcp_activation_runbook()
mcp_fetch_gate = service.build_mcp_fetch_gate()
scheduler_plan = service.build_scheduler_plan()
manual_sample_plan = service.build_manual_sample_plan()
match_review_plan = service.build_match_review_plan()
opportunity_plan = service.build_opportunity_plan()
opportunity_scoring_plan = service.build_opportunity_scoring_plan()
@@ -77,6 +78,14 @@ def build_deployment_readiness_preview(
and not scheduler_plan["crawler_job_started"]
and not scheduler_plan["database_write_executed"]
),
"manual_sample_plan_preview_safe": bool(
manual_sample_plan["mode"] == "manual_sample_fetch_plan_preview"
and not manual_sample_plan["sample_fetch_executed"]
and not manual_sample_plan["external_network_executed"]
and not manual_sample_plan["database_write_executed"]
and not manual_sample_plan["database_commit_executed"]
and not manual_sample_plan["scheduler_attached"]
),
"match_review_plan_preview_safe": bool(
match_review_plan["mode"] == "match_review_plan_preview"
and not match_review_plan["review_queue_created"]
@@ -279,6 +288,7 @@ def build_deployment_readiness_preview(
"/api/market_intel/mcp_activation_runbook",
"/api/market_intel/mcp_fetch_gate",
"/api/market_intel/scheduler_plan",
"/api/market_intel/manual_sample_plan",
"/api/market_intel/match_review_plan",
"/api/market_intel/opportunity_plan",
"/api/market_intel/opportunity_scoring_plan",
@@ -312,6 +322,7 @@ def build_deployment_readiness_preview(
"mcp_activation_runbook": mcp_activation_runbook,
"mcp_fetch_gate": mcp_fetch_gate,
"scheduler_plan": scheduler_plan,
"manual_sample_plan": manual_sample_plan,
"match_review_plan": match_review_plan,
"opportunity_plan": opportunity_plan,
"opportunity_scoring_plan": opportunity_scoring_plan,

View File

@@ -0,0 +1,167 @@
"""市場情報第一次人工 sample fetch 計畫。
本模組只組裝操作員檢查表與樣本範圍,不抓外部網站、不寫 DB、不掛排程。
"""
SAMPLE_PLATFORM_ORDER = ("pchome", "momo", "coupang", "shopee")
MAX_SAMPLE_SOURCES_PER_PLATFORM = 1
def _status_value(runtime_status, name, default=False):
if isinstance(runtime_status, dict):
return runtime_status.get(name, default)
return getattr(runtime_status, name, default)
def _sort_adapters(adapters):
priority = {code: index for index, code in enumerate(SAMPLE_PLATFORM_ORDER)}
return sorted(
adapters,
key=lambda adapter: priority.get(adapter.platform_code, len(priority)),
)
def _build_platform_sample(adapter, priority):
sources = list(adapter.campaign_sources())[:MAX_SAMPLE_SOURCES_PER_PLATFORM]
return {
"priority": priority,
"platform_code": adapter.platform_code,
"platform_name": adapter.platform_name,
"base_url": adapter.base_url,
"sample_source_count": len(sources),
"max_sources_per_manual_sample": MAX_SAMPLE_SOURCES_PER_PLATFORM,
"request_interval_sec": adapter.safety_policy.request_interval_sec,
"timeout_sec": adapter.safety_policy.timeout_sec,
"selected_sources": [source.to_dict() for source in sources],
"expected_diagnostics": [
"status_code",
"content_length",
"page_hash",
"title",
"campaign_link_candidates",
],
"write_status": "blocked_preview_only",
"network_status": "not_executed",
}
def build_manual_sample_fetch_plan_preview(
*,
runtime_status,
adapters,
mcp_fetch_gate,
live_db_inventory,
):
"""建立第一次人工 sample fetch 計畫;不執行 fetch。"""
adapters = _sort_adapters(list(adapters))
sample_platforms = [
_build_platform_sample(adapter, priority=index + 1)
for index, adapter in enumerate(adapters)
]
inventory_safe = bool(
live_db_inventory.get("mode") == "live_db_inventory_planned"
and not live_db_inventory.get("read_only_query_executed")
and not live_db_inventory.get("database_write_executed")
)
gate_checks = {
"platform_adapters_registered": bool(sample_platforms),
"market_intel_enabled": bool(_status_value(runtime_status, "enabled")),
"market_intel_crawler_enabled": bool(
_status_value(runtime_status, "crawler_enabled")
),
"database_write_still_blocked": not bool(
_status_value(runtime_status, "database_write_allowed")
),
"scheduler_detached": not bool(
_status_value(runtime_status, "scheduler_attached")
),
"mcp_fetch_gate_open": bool(mcp_fetch_gate.get("manual_fetch_gate_open")),
"live_inventory_preview_safe": inventory_safe,
"manual_operator_approval": False,
}
blocked_reasons = [
key for key, passed in gate_checks.items()
if not passed
]
blocked_reasons.append("sample_fetch_not_executed_by_api")
return {
"mode": "manual_sample_fetch_plan_preview",
"ready_for_manual_sample_fetch": False,
"sample_fetch_executed": False,
"external_network_executed": False,
"database_connection_opened": False,
"database_session_created": False,
"database_write_executed": False,
"database_commit_executed": False,
"scheduler_attached": False,
"writes_executed": False,
"would_write_database": False,
"platform_count": len(sample_platforms),
"sample_source_total": sum(
item["sample_source_count"] for item in sample_platforms
),
"sample_platform_order": list(SAMPLE_PLATFORM_ORDER),
"sample_platforms": sample_platforms,
"gate_checks": gate_checks,
"blocked_reasons": blocked_reasons,
"sample_policy": {
"max_platforms_per_manual_batch": 1,
"max_sources_per_platform": MAX_SAMPLE_SOURCES_PER_PLATFORM,
"first_batch_platform": sample_platforms[0]["platform_code"]
if sample_platforms
else None,
"write_policy": "no_database_write_until_operator_approves_import",
"network_policy": "public_pages_only_mcp_gate_required",
},
"operator_sequence": [
{
"key": "confirm_live_inventory_baseline",
"label": "先以人工只讀庫存 smoke 確認 market_* 表與 seed 基準",
"status": "required",
},
{
"key": "enable_fetch_flags_temporarily",
"label": "只在操作窗口暫時開啟 MARKET_INTEL_ENABLED 與 MARKET_INTEL_CRAWLER_ENABLED",
"status": "manual_required",
},
{
"key": "verify_mcp_fetch_gate",
"label": "MCP readiness、router、external health 與 tool contract 必須全過",
"status": "required",
},
{
"key": "run_one_platform_one_source",
"label": "第一批只跑 1 個平台的 1 個公開活動入口",
"status": "manual_required",
},
{
"key": "review_parser_diagnostics",
"label": "只審查 title、page_hash、候選活動連結與錯誤不寫 market_* 商品資料",
"status": "required",
},
],
"fallback_plan": [
{
"key": "turn_flags_off",
"label": "關閉 MARKET_INTEL_ENABLED 與 MARKET_INTEL_CRAWLER_ENABLED",
},
{
"key": "stay_preview_only",
"label": "保留所有市場情報 API/UI 預覽,不掛 scheduler、不寫 DB",
},
{
"key": "do_not_retry_fast",
"label": "若平台回應異常,不做密集重試,改回人工檢查 adapter 入口",
},
],
"safe_boundaries": [
"do_not_fetch_from_api_preview",
"do_not_write_market_tables",
"do_not_attach_scheduler",
"do_not_use_login_or_member_pages",
"do_not_bypass_anti_bot",
"do_not_touch_momo_db_lifecycle",
],
}

View File

@@ -30,6 +30,9 @@ from services.market_intel.mcp_contract import build_mcp_tool_contract_preview
from services.market_intel.mcp_deploy_preflight import build_mcp_deploy_preflight_plan
from services.market_intel.mcp_fetch_gate import build_mcp_fetch_gate_preview
from services.market_intel.mcp_readiness import build_mcp_readiness_plan
from services.market_intel.manual_sample_plan import (
build_manual_sample_fetch_plan_preview,
)
from services.market_intel.migration_blueprint import build_migration_blueprint
from services.market_intel.migration_catalog_review import (
build_migration_catalog_review_preview,
@@ -92,7 +95,7 @@ class MarketIntelRuntimeStatus:
class MarketIntelService:
"""市場情報入口服務,先集中 feature gate 與安全狀態。"""
phase = "phase_46_live_db_inventory_preview"
phase = "phase_47_manual_sample_fetch_plan"
def get_runtime_status(self) -> MarketIntelRuntimeStatus:
return MarketIntelRuntimeStatus(
@@ -421,6 +424,17 @@ class MarketIntelService:
plan["phase"] = self.phase
return plan
def build_manual_sample_plan(self):
"""回報第一次人工 sample fetch 計畫;不抓外部頁、不寫 DB。"""
plan = build_manual_sample_fetch_plan_preview(
runtime_status=self.get_runtime_status(),
adapters=get_adapter_registry().values(),
mcp_fetch_gate=self.build_mcp_fetch_gate(),
live_db_inventory=self.build_live_db_inventory(),
)
plan["phase"] = self.phase
return plan
def build_match_review_plan(self):
"""回報商品比對審核計畫;不建立 queue、不自動確認。"""
schema_smoke = self.build_schema_smoke()

View File

@@ -486,6 +486,24 @@
</div>
</div>
<div class="market-intel-panel" data-market-intel-manual-sample>
<div class="market-intel-preview-head">
<div>
<p class="market-intel-muted momo-mono mb-1">MANUAL SAMPLE / FETCH PLAN</p>
<h2 class="market-intel-preview-title">人工樣本 Fetch 計畫</h2>
</div>
<button class="market-intel-icon-button" type="button" title="重新整理人工樣本 Fetch 計畫" data-market-intel-manual-sample-refresh>
<i class="fas fa-rotate-right" aria-hidden="true"></i>
</button>
</div>
<div class="market-intel-preview-meta" data-market-intel-manual-sample-meta>
<span class="market-intel-pill">loading</span>
</div>
<div data-market-intel-manual-sample-body>
<div class="market-intel-empty">讀取人工樣本 Fetch 計畫中...</div>
</div>
</div>
<div class="market-intel-panel" data-market-intel-scheduler>
<div class="market-intel-preview-head">
<div>
@@ -735,6 +753,7 @@
const mcpPreflightRoot = document.querySelector('[data-market-intel-mcp-preflight]');
const mcpActivationRoot = document.querySelector('[data-market-intel-mcp-activation]');
const mcpFetchGateRoot = document.querySelector('[data-market-intel-mcp-fetch-gate]');
const manualSampleRoot = document.querySelector('[data-market-intel-manual-sample]');
const schedulerRoot = document.querySelector('[data-market-intel-scheduler]');
const matchReviewRoot = document.querySelector('[data-market-intel-match-review]');
const opportunityRoot = document.querySelector('[data-market-intel-opportunity]');
@@ -748,7 +767,7 @@
const liveInventoryRoot = document.querySelector('[data-market-intel-live-inventory]');
const approvalRoot = document.querySelector('[data-market-intel-approval]');
const deployRoot = document.querySelector('[data-market-intel-deploy]');
if (!root && !writerRoot && !cliRoot && !dbProbeRoot && !seedDiffRoot && !legacyBridgeRoot && !mcpReadinessRoot && !mcpPreflightRoot && !mcpActivationRoot && !mcpFetchGateRoot && !schedulerRoot && !matchReviewRoot && !opportunityRoot && !opportunityScoringRoot && !opportunityEvidenceRoot && !opportunityAlertRoot && !migrationRoot && !migrationDrillRoot && !catalogReviewRoot && !liveSmokeRoot && !liveInventoryRoot && !approvalRoot && !deployRoot) return;
if (!root && !writerRoot && !cliRoot && !dbProbeRoot && !seedDiffRoot && !legacyBridgeRoot && !mcpReadinessRoot && !mcpPreflightRoot && !mcpActivationRoot && !mcpFetchGateRoot && !manualSampleRoot && !schedulerRoot && !matchReviewRoot && !opportunityRoot && !opportunityScoringRoot && !opportunityEvidenceRoot && !opportunityAlertRoot && !migrationRoot && !migrationDrillRoot && !catalogReviewRoot && !liveSmokeRoot && !liveInventoryRoot && !approvalRoot && !deployRoot) return;
const meta = root ? root.querySelector('[data-market-intel-preview-meta]') : null;
const body = root ? root.querySelector('[data-market-intel-preview-body]') : null;
@@ -790,6 +809,10 @@
const mcpFetchGateBody = mcpFetchGateRoot ? mcpFetchGateRoot.querySelector('[data-market-intel-mcp-fetch-gate-body]') : null;
const mcpFetchGateRefresh = mcpFetchGateRoot ? mcpFetchGateRoot.querySelector('[data-market-intel-mcp-fetch-gate-refresh]') : null;
const mcpFetchGateEndpoint = "{{ url_for('market_intel.market_intel_mcp_fetch_gate') }}?fetch=false&execute=false";
const manualSampleMeta = manualSampleRoot ? manualSampleRoot.querySelector('[data-market-intel-manual-sample-meta]') : null;
const manualSampleBody = manualSampleRoot ? manualSampleRoot.querySelector('[data-market-intel-manual-sample-body]') : null;
const manualSampleRefresh = manualSampleRoot ? manualSampleRoot.querySelector('[data-market-intel-manual-sample-refresh]') : null;
const manualSampleEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_plan') }}";
const schedulerMeta = schedulerRoot ? schedulerRoot.querySelector('[data-market-intel-scheduler-meta]') : null;
const schedulerBody = schedulerRoot ? schedulerRoot.querySelector('[data-market-intel-scheduler-body]') : null;
const schedulerRefresh = schedulerRoot ? schedulerRoot.querySelector('[data-market-intel-scheduler-refresh]') : null;
@@ -1535,6 +1558,117 @@
}
};
const renderManualSampleMeta = data => {
manualSampleMeta.innerHTML = [
`mode=${data.mode || 'unknown'}`,
`ready=${data.ready_for_manual_sample_fetch ? 'yes' : 'no'}`,
`platforms=${data.platform_count || 0}`,
`sources=${data.sample_source_total || 0}`,
`network=${data.external_network_executed ? 'executed' : 'blocked'}`
].map(item => `<span class="market-intel-pill">${escapeHtml(item)}</span>`).join('');
};
const renderManualSampleBody = data => {
const blockers = (data.blocked_reasons || []).join(' / ');
const checks = Object.entries(data.gate_checks || {});
const platforms = data.sample_platforms || [];
const sequence = data.operator_sequence || [];
const fallback = data.fallback_plan || [];
const boundaries = data.safe_boundaries || [];
const renderCheck = ([name, passed]) => `
<div class="market-intel-check">
<div>
<strong>${escapeHtml(name)}</strong>
</div>
<span>${passed ? 'PASS' : 'BLOCK'}</span>
</div>
`;
const renderNamedItem = (item, status) => {
const normalized = typeof item === 'string' ? { key: item, label: item } : (item || {});
return `
<div class="market-intel-check">
<div>
<strong>${escapeHtml(normalized.key || normalized.label || 'item')}</strong>
<small>${escapeHtml(normalized.label || normalized.key || '')}</small>
</div>
<span>${escapeHtml(normalized.status || status || 'required').toUpperCase()}</span>
</div>
`;
};
const renderPlatform = item => {
const firstSource = (item.selected_sources || [])[0] || {};
return `
<div class="market-intel-check">
<div>
<strong>${escapeHtml(item.priority)}. ${escapeHtml(item.platform_code)}</strong>
<small>${escapeHtml(firstSource.name || item.platform_name)} / interval=${escapeHtml(item.request_interval_sec)}s / timeout=${escapeHtml(item.timeout_sec)}s</small>
</div>
<span>${escapeHtml(item.network_status || 'not_executed').toUpperCase()}</span>
</div>
`;
};
manualSampleBody.innerHTML = `
<div class="market-intel-empty mb-3">此卡只規劃第一次人工樣本 fetchAPI/UI 不抓外站、不寫 DB、不建立 crawler run、不掛 scheduler。${blockers ? `阻擋:${escapeHtml(blockers)}` : ''}</div>
<div class="market-intel-deploy-grid">
<div data-market-intel-manual-sample-checks>
<p class="market-intel-deploy-section-title">SAMPLE GATES</p>
<div class="market-intel-check-list">${
checks.length
? checks.map(renderCheck).join('')
: '<div class="market-intel-empty">尚未提供 sample gate。</div>'
}</div>
</div>
<div data-market-intel-manual-sample-platforms>
<p class="market-intel-deploy-section-title">SAMPLE PLATFORMS</p>
<div class="market-intel-check-list">${
platforms.length
? platforms.map(renderPlatform).join('')
: '<div class="market-intel-empty">尚未提供平台樣本。</div>'
}</div>
</div>
<div data-market-intel-manual-sample-sequence>
<p class="market-intel-deploy-section-title">OPERATOR SEQUENCE</p>
<div class="market-intel-check-list">${
sequence.length
? sequence.map(item => renderNamedItem(item, 'required')).join('')
: '<div class="market-intel-empty">尚未提供人工順序。</div>'
}</div>
</div>
<div data-market-intel-manual-sample-fallback>
<p class="market-intel-deploy-section-title">FALLBACK</p>
<div class="market-intel-check-list">${
fallback.length
? fallback.map(item => renderNamedItem(item, 'ready')).join('')
: '<div class="market-intel-empty">尚未提供備援方案。</div>'
}</div>
</div>
<div data-market-intel-manual-sample-boundaries>
<p class="market-intel-deploy-section-title">BOUNDARIES</p>
<div class="market-intel-check-list">${
boundaries.length
? boundaries.map(item => renderNamedItem(item, 'required')).join('')
: '<div class="market-intel-empty">尚未提供安全邊界。</div>'
}</div>
</div>
</div>
`;
};
const loadManualSample = async () => {
if (!manualSampleMeta || !manualSampleBody) return;
manualSampleBody.innerHTML = '<div class="market-intel-empty">讀取人工樣本 Fetch 計畫中...</div>';
try {
const response = await fetch(manualSampleEndpoint, { credentials: 'same-origin' });
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const data = await response.json();
renderManualSampleMeta(data);
renderManualSampleBody(data);
} catch (error) {
manualSampleMeta.innerHTML = '<span class="market-intel-pill">error</span>';
manualSampleBody.innerHTML = `<div class="market-intel-empty">人工樣本 Fetch 計畫讀取失敗:${escapeHtml(error.message)}</div>`;
}
};
const renderSchedulerMeta = data => {
schedulerMeta.innerHTML = [
`mode=${data.mode || 'unknown'}`,
@@ -3017,6 +3151,9 @@
if (mcpFetchGateRefresh) {
mcpFetchGateRefresh.addEventListener('click', loadMcpFetchGate);
}
if (manualSampleRefresh) {
manualSampleRefresh.addEventListener('click', loadManualSample);
}
if (schedulerRefresh) {
schedulerRefresh.addEventListener('click', loadScheduler);
}
@@ -3066,6 +3203,7 @@
loadMcpPreflight();
loadMcpActivation();
loadMcpFetchGate();
loadManualSample();
loadScheduler();
loadMatchReview();
loadOpportunity();

View File

@@ -502,6 +502,10 @@ def test_market_intel_preview_template_uses_safe_fetch_false_endpoint():
assert "data-market-intel-live-inventory-tables" in template
assert "data-market-intel-live-inventory-platforms" in template
assert "data-market-intel-live-inventory-alerts" in template
assert "data-market-intel-manual-sample" in template
assert "data-market-intel-manual-sample-checks" in template
assert "data-market-intel-manual-sample-platforms" in template
assert "data-market-intel-manual-sample-sequence" in template
assert "data-market-intel-approval" in template
assert "data-market-intel-approval-gates" in template
assert "data-market-intel-deploy" in template
@@ -517,6 +521,7 @@ def test_market_intel_preview_template_uses_safe_fetch_false_endpoint():
assert "market_intel.market_intel_mcp_deploy_preflight" in template
assert "market_intel.market_intel_mcp_activation_runbook" in template
assert "market_intel.market_intel_mcp_fetch_gate" in template
assert "market_intel.market_intel_manual_sample_plan" in template
assert "market_intel.market_intel_scheduler_plan" in template
assert "market_intel.market_intel_match_review_plan" in template
assert "market_intel.market_intel_opportunity_plan" in template
@@ -552,7 +557,7 @@ def test_legacy_source_bridge_default_is_planned_only():
bridge = MarketIntelService().build_legacy_source_bridge()
assert bridge["mode"] == "legacy_source_bridge_planned"
assert bridge["phase"] == "phase_46_live_db_inventory_preview"
assert bridge["phase"] == "phase_47_manual_sample_fetch_plan"
assert bridge["execute_requested"] is False
assert bridge["read_only_query_executed"] is False
assert bridge["database_connection_opened"] is False
@@ -710,7 +715,7 @@ def test_mcp_tool_contract_preview_is_read_only_and_whitelisted():
contract = MarketIntelService().build_mcp_tool_contract()
assert contract["mode"] == "mcp_tool_contract_preview"
assert contract["phase"] == "phase_46_live_db_inventory_preview"
assert contract["phase"] == "phase_47_manual_sample_fetch_plan"
assert contract["caller"] == "market_intel"
assert contract["contract_ready"] is True
assert contract["blocked_reasons"] == []
@@ -843,7 +848,7 @@ def test_mcp_activation_runbook_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "mcp_activation_runbook_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["deployment_actions_executed"] is False
assert data["docker_command_executed"] is False
assert data["ssh_command_executed"] is False
@@ -856,7 +861,7 @@ def test_mcp_fetch_gate_default_blocks_external_fetch():
gate = MarketIntelService().build_mcp_fetch_gate(fetch_requested=True)
assert gate["mode"] == "mcp_fetch_gate_planned"
assert gate["phase"] == "phase_46_live_db_inventory_preview"
assert gate["phase"] == "phase_47_manual_sample_fetch_plan"
assert gate["fetch_requested"] is True
assert gate["manual_fetch_gate_open"] is False
assert gate["network_request_allowed"] is False
@@ -926,7 +931,7 @@ def test_mcp_fetch_gate_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "mcp_fetch_gate_planned"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["fetch_requested"] is False
assert data["network_request_allowed"] is False
assert data["external_network_executed"] is False
@@ -934,11 +939,70 @@ def test_mcp_fetch_gate_route_is_preview_only():
assert data["scheduler_attached"] is False
def test_manual_sample_plan_preview_blocks_fetch_and_write():
plan = MarketIntelService().build_manual_sample_plan()
assert plan["mode"] == "manual_sample_fetch_plan_preview"
assert plan["phase"] == "phase_47_manual_sample_fetch_plan"
assert plan["ready_for_manual_sample_fetch"] is False
assert plan["sample_fetch_executed"] is False
assert plan["external_network_executed"] is False
assert plan["database_session_created"] is False
assert plan["database_write_executed"] is False
assert plan["database_commit_executed"] is False
assert plan["scheduler_attached"] is False
assert plan["writes_executed"] is False
assert plan["would_write_database"] is False
assert plan["platform_count"] == 4
assert plan["sample_source_total"] == 4
assert plan["sample_policy"]["max_platforms_per_manual_batch"] == 1
assert plan["sample_policy"]["first_batch_platform"] == "pchome"
assert [item["platform_code"] for item in plan["sample_platforms"]] == [
"pchome",
"momo",
"coupang",
"shopee",
]
assert all(
item["sample_source_count"] == 1
and item["network_status"] == "not_executed"
and item["write_status"] == "blocked_preview_only"
for item in plan["sample_platforms"]
)
assert "market_intel_enabled" in plan["blocked_reasons"]
assert "mcp_fetch_gate_open" in plan["blocked_reasons"]
assert "sample_fetch_not_executed_by_api" in plan["blocked_reasons"]
assert "do_not_fetch_from_api_preview" in plan["safe_boundaries"]
assert "do_not_bypass_anti_bot" in plan["safe_boundaries"]
def test_manual_sample_plan_route_is_preview_only():
from routes.market_intel_routes import market_intel_bp
app = Flask(__name__)
app.secret_key = "test-secret"
app.register_blueprint(market_intel_bp)
client = app.test_client()
with client.session_transaction() as session:
session["logged_in"] = True
response = client.get("/api/market_intel/manual_sample_plan")
data = response.get_json()
assert response.status_code == 200
assert data["mode"] == "manual_sample_fetch_plan_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["sample_fetch_executed"] is False
assert data["external_network_executed"] is False
assert data["database_write_executed"] is False
assert data["scheduler_attached"] is False
def test_scheduler_plan_preview_blocks_job_attachment():
plan = MarketIntelService().build_scheduler_plan()
assert plan["mode"] == "scheduler_attach_plan_preview"
assert plan["phase"] == "phase_46_live_db_inventory_preview"
assert plan["phase"] == "phase_47_manual_sample_fetch_plan"
assert plan["ready_to_attach_scheduler"] is False
assert plan["scheduler_attached"] is False
assert plan["scheduler_registration_executed"] is False
@@ -976,7 +1040,7 @@ def test_scheduler_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "scheduler_attach_plan_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["scheduler_registration_executed"] is False
assert data["crawler_job_started"] is False
assert data["external_network_executed"] is False
@@ -987,7 +1051,7 @@ def test_match_review_plan_preview_blocks_auto_confirm():
plan = MarketIntelService().build_match_review_plan()
assert plan["mode"] == "match_review_plan_preview"
assert plan["phase"] == "phase_46_live_db_inventory_preview"
assert plan["phase"] == "phase_47_manual_sample_fetch_plan"
assert plan["ready_for_review_queue"] is False
assert plan["review_queue_created"] is False
assert plan["auto_match_executed"] is False
@@ -1023,7 +1087,7 @@ def test_match_review_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "match_review_plan_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["review_queue_created"] is False
assert data["auto_confirm_executed"] is False
assert data["external_network_executed"] is False
@@ -1034,7 +1098,7 @@ def test_opportunity_plan_preview_blocks_alerts_and_ai_summary():
plan = MarketIntelService().build_opportunity_plan()
assert plan["mode"] == "opportunity_plan_preview"
assert plan["phase"] == "phase_46_live_db_inventory_preview"
assert plan["phase"] == "phase_47_manual_sample_fetch_plan"
assert plan["ready_for_opportunity_queue"] is False
assert plan["opportunity_queue_created"] is False
assert plan["threat_alert_dispatched"] is False
@@ -1075,7 +1139,7 @@ def test_opportunity_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "opportunity_plan_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["opportunity_queue_created"] is False
assert data["threat_alert_dispatched"] is False
assert data["ai_summary_generated"] is False
@@ -1086,7 +1150,7 @@ def test_opportunity_scoring_plan_preview_blocks_scoring_and_alerts():
plan = MarketIntelService().build_opportunity_scoring_plan()
assert plan["mode"] == "opportunity_scoring_plan_preview"
assert plan["phase"] == "phase_46_live_db_inventory_preview"
assert plan["phase"] == "phase_47_manual_sample_fetch_plan"
assert plan["ready_for_scoring_job"] is False
assert plan["scoring_job_created"] is False
assert plan["score_calculation_executed"] is False
@@ -1134,7 +1198,7 @@ def test_opportunity_scoring_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "opportunity_scoring_plan_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["scoring_job_created"] is False
assert data["score_calculation_executed"] is False
assert data["sample_scores_generated"] is False
@@ -1146,7 +1210,7 @@ def test_opportunity_evidence_plan_preview_blocks_queries_and_alerts():
plan = MarketIntelService().build_opportunity_evidence_plan()
assert plan["mode"] == "opportunity_evidence_plan_preview"
assert plan["phase"] == "phase_46_live_db_inventory_preview"
assert plan["phase"] == "phase_47_manual_sample_fetch_plan"
assert plan["ready_for_evidence_bundle"] is False
assert plan["evidence_bundle_created"] is False
assert plan["evidence_query_executed"] is False
@@ -1192,7 +1256,7 @@ def test_opportunity_evidence_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "opportunity_evidence_plan_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["evidence_bundle_created"] is False
assert data["evidence_query_executed"] is False
assert data["sample_evidence_generated"] is False
@@ -1205,7 +1269,7 @@ def test_opportunity_alert_plan_preview_blocks_dispatch_and_llm_calls():
plan = MarketIntelService().build_opportunity_alert_plan()
assert plan["mode"] == "opportunity_alert_plan_preview"
assert plan["phase"] == "phase_46_live_db_inventory_preview"
assert plan["phase"] == "phase_47_manual_sample_fetch_plan"
assert plan["ready_for_alert_candidates"] is False
assert plan["alert_candidate_created"] is False
assert plan["alert_queue_created"] is False
@@ -1290,7 +1354,7 @@ def test_opportunity_alert_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "opportunity_alert_plan_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["alert_candidate_created"] is False
assert data["alert_queue_created"] is False
assert data["review_queue_created"] is False
@@ -1368,7 +1432,7 @@ def test_mcp_deploy_preflight_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "mcp_external_deploy_preflight_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["deployment_actions_executed"] is False
assert data["docker_command_executed"] is False
assert data["ssh_command_executed"] is False
@@ -1383,7 +1447,7 @@ def test_mcp_readiness_default_is_planned_only(monkeypatch):
readiness = MarketIntelService().build_mcp_readiness()
assert readiness["mode"] == "mcp_readiness_planned"
assert readiness["phase"] == "phase_46_live_db_inventory_preview"
assert readiness["phase"] == "phase_47_manual_sample_fetch_plan"
assert readiness["execute_requested"] is False
assert readiness["router_enabled"] is False
assert readiness["external_mcp_complete"] is False
@@ -1771,6 +1835,7 @@ def test_deployment_readiness_reports_app_only_release_gate():
assert readiness["checks"]["mcp_activation_runbook_preview_safe"] is True
assert readiness["checks"]["mcp_fetch_gate_preview_safe"] is True
assert readiness["checks"]["scheduler_plan_preview_safe"] is True
assert readiness["checks"]["manual_sample_plan_preview_safe"] is True
assert readiness["checks"]["match_review_plan_preview_safe"] is True
assert readiness["checks"]["opportunity_plan_preview_safe"] is True
assert readiness["checks"]["opportunity_scoring_plan_preview_safe"] is True
@@ -1801,6 +1866,7 @@ def test_deployment_readiness_reports_app_only_release_gate():
assert "/api/market_intel/mcp_activation_runbook" in readiness["production_smoke_targets"]
assert "/api/market_intel/mcp_fetch_gate" in readiness["production_smoke_targets"]
assert "/api/market_intel/scheduler_plan" in readiness["production_smoke_targets"]
assert "/api/market_intel/manual_sample_plan" in readiness["production_smoke_targets"]
assert "/api/market_intel/match_review_plan" in readiness["production_smoke_targets"]
assert "/api/market_intel/opportunity_plan" in readiness["production_smoke_targets"]
assert "/api/market_intel/opportunity_scoring_plan" in readiness["production_smoke_targets"]
@@ -1847,6 +1913,11 @@ def test_deployment_readiness_reports_app_only_release_gate():
assert readiness["mcp_activation_runbook"]["docker_command_executed"] is False
assert readiness["mcp_fetch_gate"]["network_request_allowed"] is False
assert readiness["mcp_fetch_gate"]["external_network_executed"] is False
assert readiness["manual_sample_plan"]["mode"] == "manual_sample_fetch_plan_preview"
assert readiness["manual_sample_plan"]["sample_fetch_executed"] is False
assert readiness["manual_sample_plan"]["external_network_executed"] is False
assert readiness["manual_sample_plan"]["database_write_executed"] is False
assert readiness["manual_sample_plan"]["scheduler_attached"] is False
assert readiness["scheduler_plan"]["scheduler_registration_executed"] is False
assert readiness["scheduler_plan"]["crawler_job_started"] is False
assert readiness["scheduler_plan"]["database_write_executed"] is False
@@ -1951,7 +2022,7 @@ def test_migration_apply_drill_planned_is_safe_and_manual_only():
drill = MarketIntelService().build_migration_apply_drill()
assert drill["mode"] == "migration_apply_drill_preview"
assert drill["phase"] == "phase_46_live_db_inventory_preview"
assert drill["phase"] == "phase_47_manual_sample_fetch_plan"
assert drill["execute_requested"] is False
assert drill["schema_state"] == "planned_no_db_probe"
assert drill["drill_ready_for_operator_review"] is True
@@ -2066,7 +2137,7 @@ def test_migration_apply_drill_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "migration_apply_drill_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["execute_requested"] is False
assert data["migration_executed"] is False
assert data["rollback_executed"] is False
@@ -2078,7 +2149,7 @@ def test_migration_catalog_review_planned_is_safe_and_diagnostic():
review = MarketIntelService().build_migration_catalog_review()
assert review["mode"] == "migration_catalog_review_preview"
assert review["phase"] == "phase_46_live_db_inventory_preview"
assert review["phase"] == "phase_47_manual_sample_fetch_plan"
assert review["execute_requested"] is False
assert review["catalog_state"] == "planned_no_probe"
assert review["seed_state"] == "planned_no_probe"
@@ -2193,7 +2264,7 @@ def test_migration_catalog_review_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "migration_catalog_review_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["execute_requested"] is False
assert data["catalog_state"] == "planned_no_probe"
assert data["migration_executed"] is False
@@ -2206,7 +2277,7 @@ def test_migration_live_smoke_planned_is_preview_only():
smoke = MarketIntelService().build_migration_live_smoke()
assert smoke["mode"] == "migration_live_smoke_preview"
assert smoke["phase"] == "phase_46_live_db_inventory_preview"
assert smoke["phase"] == "phase_47_manual_sample_fetch_plan"
assert smoke["execute_requested"] is False
assert smoke["smoke_result"] == "planned_no_execution"
assert smoke["live_smoke_passed"] is False
@@ -2268,7 +2339,7 @@ def test_migration_live_smoke_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "migration_live_smoke_preview"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["execute_requested"] is False
assert data["smoke_result"] == "planned_no_execution"
assert data["migration_executed"] is False
@@ -2281,7 +2352,7 @@ def test_live_db_inventory_planned_is_preview_only():
inventory = MarketIntelService().build_live_db_inventory()
assert inventory["mode"] == "live_db_inventory_planned"
assert inventory["phase"] == "phase_46_live_db_inventory_preview"
assert inventory["phase"] == "phase_47_manual_sample_fetch_plan"
assert inventory["execute_requested"] is False
assert inventory["read_only_query_executed"] is False
assert inventory["database_connection_opened"] is False
@@ -2425,7 +2496,7 @@ def test_live_db_inventory_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "live_db_inventory_planned"
assert data["phase"] == "phase_46_live_db_inventory_preview"
assert data["phase"] == "phase_47_manual_sample_fetch_plan"
assert data["execute_requested"] is False
assert data["read_only_query_executed"] is False
assert data["database_write_executed"] is False