wooo/ewoooc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

新增市場情報候選佇列 writer gate
Some checks failed
CD Pipeline / deploy (push) Has been cancelled
Details

Browse Source
This commit is contained in:
OoO
2026-05-19 09:30:10 +08:00
parent 21cdbdb75c
commit 9ada594197
12 changed files with 702 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
TODO_NEXT_STEPS.txt
View File

@@ -116,6 +116,7 @@
- Phase 54 manual sample candidate queue transaction：新增 `/api/market_intel/manual_sample_review/candidate_queue_transaction` POST 與 UI transaction preview 按鈕，將 queue row preview 轉成 `market_alert_review_queue` idempotent insert statement、payload hash 與 rollback plan；不開 DB connection、不開 transaction、不 commit、不建立 approval record；版本同步至 V10.227。
- V10.228 補 PPT 視覺 QA 背景狀態卡：新增 `/observability/ppt_audit/vision_status` 與頁面 Vision QA 狀態卡，讓立即視覺 QA 排入後可看 queued/running/completed/error 與最近審核摘要，不必刷新猜測。
- V10.229 修正 PPT 視覺 QA 多 worker 狀態漂移：將 queued/running/completed/error 寫入 `/app/data/ppt_vision_audit_status.json` runtime state，所有 Gunicorn worker 共用同一份狀態並阻擋重複排入。
- Phase 55 candidate queue writer CLI gate：新增 `/api/market_intel/manual_sample_review/candidate_queue_writer_status` POST、`scripts/market_intel_candidate_queue_writer.py` 與 UI writer gate 按鈕，定義 `MARKET_INTEL_QUEUE_WRITE_APPROVAL` 一次性 token、execute/apply flags、備份、migration smoke 與 rollback gate；本階段仍不開 DB connection、不寫 `market_alert_review_queue`、不 commit、不掛 scheduler；版本同步至 V10.230。
- Schema smoke：`tests/test_market_intel_skeleton.py` 檢查 `Base.metadata` 內含 ADR-035 八張 `market_*` tables。
- Desktop UI QA：本機只註冊 `market_intel_bp` 的 Flask harness 載入 `/market_intel`，確認 Phase 15、候選預覽、writer preview、安全 flags、點陣暖紙視覺正常，console error 0。
- API QA：`/api/market_intel/schema_smoke` 通過 7 張表與 `market_platforms` 必要欄位檢查；`/api/market_intel/platform_seed_writer_plan` 回傳 4 筆 dry-run upsert preview，`writes_executed=false`，四平台皆 `blocked_dry_run_only`。

2
config.py
View File

@@ -320,7 +320,7 @@ YOUTUBE_API_KEY = os.getenv('YOUTUBE_API_KEY', '')
# ==========================================
# 系統版本與路徑
# ==========================================
SYSTEM_VERSION = "V10.229"
SYSTEM_VERSION = "V10.230"
LOG_FILE_PATH = os.path.join(BASE_DIR, 'logs/system.log')
public_url = PUBLIC_URL # 用於模板顯示

1
docs/adr/ADR-035-cross-platform-market-campaign-intelligence.md
View File

@@ -180,6 +180,7 @@ EwoooC 目前已有 MOMO EDM / 節慶活動資料、`promo_products`、PChome
- 2026-05-19 追加 manual sample candidate queue draft：`services.market_intel.manual_sample_candidate_queue` 與 `/api/market_intel/manual_sample_review/candidate_queue_draft` 將 handoff 候選轉成只讀人工審核 queue draft，包含 queue item key、review priority、review state 與 approval_required。此入口保持 CSRF 保護，不建立正式 queue、不保存草案、不寫 market_*、不自動核准候選、不掛 scheduler。
- 2026-05-19 追加 manual sample candidate queue approval：`/api/market_intel/manual_sample_review/candidate_queue_approval` 將 queue draft 對齊既有 `market_alert_review_queue` 契約，輸出 row preview、必填欄位 gate、寫入 flags、備份與人工批准 gate。此入口保持 CSRF 保護，不建立 approval record、不寫 review queue、不開 DB transaction、不掛 scheduler。
- 2026-05-19 追加 manual sample candidate queue transaction：`/api/market_intel/manual_sample_review/candidate_queue_transaction` 將 queue row preview 轉成 idempotent insert statement preview、payload hash、runtime order 與 rollback plan。此入口保持 CSRF 保護，不開 DB connection、不開 transaction、不 commit、不建立 approval record、不寫 `market_alert_review_queue`。
- 2026-05-19 追加 candidate queue writer CLI gate：`services.market_intel.candidate_queue_writer_cli`、`scripts/market_intel_candidate_queue_writer.py` 與 `/api/market_intel/manual_sample_review/candidate_queue_writer_status` 定義 `MARKET_INTEL_QUEUE_WRITE_APPROVAL` 一次性 token、execute/apply flags、備份、migration smoke 與 rollback gate。此階段只回 writer status，不開 DB connection、不啟用實際 writer、不寫 `market_alert_review_queue`、不掛 scheduler。
### Phase 4：Coupang / Shopee Adapter

4
docs/memory/code_modularization_inventory_20260430.md
View File

@@ -7,11 +7,12 @@
- Python 總量：約 90,293 行（排除 `venv/`、`backups/`、`__pycache__/`、`.claude/worktrees/`）。
- 最大壓力區：`services/` 約 42,364 行、`routes/` 約 29,511 行。
- `app.py` 目前約 1,232 行，功能定位應固定為 bootstrap / Blueprint registration / startup guard，不再承接新 route。
- 目前工作樹仍有 26 個 Python 檔案達到或超過 800 行；這些不是禁止修 bug，而是禁止繼續塞新功能。
- 目前工作樹仍有 27 個 Python 檔案達到或超過 800 行；這些不是禁止修 bug，而是禁止繼續塞新功能。
- 2026-05-05 追記：Phase 38→56 觀測台戰役讓 `routes/admin_observability_routes.py` 與 `run_scheduler.py` 進入大檔治理清單；後續觀測台功能應先抽 query/action service，不再把新 SQL 與 L2 mutation 直接塞回 route。
- 2026-05-06 追記：跨平台市場情報模組啟動前，必須先把新增爬蟲、排程、DB schema、UI route 全部隔離在 `market_*` / `services/market_intel/` / `routes/market_intel_routes.py`，不可塞回既有大檔。
- 2026-05-18 追記：Phase 42 市場情報只在 `app.py` 的 `EXPECTED_METADATA_TABLES` 補上 `market_alert_review_queue` 名稱，未新增 route / bootstrap 邏輯；後續仍應把 metadata verification 抽到 app factory 或 startup guard module，避免 `app.py` 繼續承接功能。
- 2026-05-19 追記：同步治理測試盤點，校正 `routes/admin_observability_routes.py` 行數；此處只更新 inventory，不變更觀測台功能。
- 2026-05-19 追記：V10.229 之後 `services/ppt_vision_service.py` 進入 800 行治理清單；本次只補 inventory 讓守門測試反映現況，不變更 PPT 視覺 QA 功能。
## 達到或超過 800 行檔案清單
@@ -41,6 +42,7 @@
| 844 | `services/ollama_service.py` | P2 Ollama client | host health / request client / fallback policy / response parsing |
| 837 | `services/code_review_pipeline_service.py` | P2 Code review pipeline service | scan orchestration / finding normalization / persistence adapter |
| 832 | `routes/export_routes.py` | P2 Export flow | export command/router glue / file path / download orchestration |
| 816 | `services/ppt_vision_service.py` | P2 PPT vision QA service | runtime state / queue status / model probe / audit execution 分離 |
| 809 | `services/competitor_price_feeder.py` | P2 competitor price feeder | crawler scheduling / price normalization / cache strategy |
| 805 | `routes/bot_api_routes.py` | P2 Bot API Blueprint | route glue / bot action service |

2
routes/README.md
View File

@@ -19,7 +19,7 @@
| `edm_routes.py` | EDM 與節慶儀表板 | `/edm`, `/festival` |
| `monthly_routes.py` | 月結分析 | `/monthly_summary_analysis`, `/api/monthly_summary_data` |
| `daily_sales_routes.py` | 當日業績 | `/daily_sales`, `/daily_sales/export*` |
| `market_intel_routes.py` | 市場情報 Phase 54 manual sample candidate queue transaction | `/market_intel`, `/market_intel/*`, `/api/market_intel/status`, `/api/market_intel/schema`, `/api/market_intel/schema_smoke`, `/api/market_intel/schema_db_probe`, `/api/market_intel/platform_seed_db_diff`, `/api/market_intel/legacy_source_bridge`, `/api/market_intel/mcp_readiness`, `/api/market_intel/mcp_tool_contract`, `/api/market_intel/mcp_deploy_preflight`, `/api/market_intel/mcp_activation_runbook`, `/api/market_intel/mcp_fetch_gate`, `/api/market_intel/scheduler_plan`, `/api/market_intel/manual_sample_plan`, `/api/market_intel/manual_sample_acceptance`, `/api/market_intel/manual_sample_review`, `/api/market_intel/manual_sample_review/evaluate`, `/api/market_intel/manual_sample_review/candidate_handoff`, `/api/market_intel/manual_sample_review/candidate_queue_draft`, `/api/market_intel/manual_sample_review/candidate_queue_approval`, `/api/market_intel/manual_sample_review/candidate_queue_transaction`, `/api/market_intel/match_review_plan`, `/api/market_intel/opportunity_plan`, `/api/market_intel/opportunity_scoring_plan`, `/api/market_intel/opportunity_evidence_plan`, `/api/market_intel/opportunity_alert_plan`, `/api/market_intel/adapters`, `/api/market_intel/dry_run_plan`, `/api/market_intel/discovery_plan`, `/api/market_intel/manual_discovery`, `/api/market_intel/candidate_preview`, `/api/market_intel/platform_seed_plan`, `/api/market_intel/platform_seed_write_guard`, `/api/market_intel/platform_seed_writer_plan`, `/api/market_intel/migration_blueprint`, `/api/market_intel/migration_apply_drill`, `/api/market_intel/migration_catalog_review`, `/api/market_intel/migration_live_smoke`, `/api/market_intel/live_db_inventory`, `/api/market_intel/seed_writer_cli_status`, `/api/market_intel/write_approval_runbook`, `/api/market_intel/deployment_readiness` |
| `market_intel_routes.py` | 市場情報 Phase 55 candidate queue writer CLI gate | `/market_intel`, `/market_intel/*`, `/api/market_intel/status`, `/api/market_intel/schema`, `/api/market_intel/schema_smoke`, `/api/market_intel/schema_db_probe`, `/api/market_intel/platform_seed_db_diff`, `/api/market_intel/legacy_source_bridge`, `/api/market_intel/mcp_readiness`, `/api/market_intel/mcp_tool_contract`, `/api/market_intel/mcp_deploy_preflight`, `/api/market_intel/mcp_activation_runbook`, `/api/market_intel/mcp_fetch_gate`, `/api/market_intel/scheduler_plan`, `/api/market_intel/manual_sample_plan`, `/api/market_intel/manual_sample_acceptance`, `/api/market_intel/manual_sample_review`, `/api/market_intel/manual_sample_review/evaluate`, `/api/market_intel/manual_sample_review/candidate_handoff`, `/api/market_intel/manual_sample_review/candidate_queue_draft`, `/api/market_intel/manual_sample_review/candidate_queue_approval`, `/api/market_intel/manual_sample_review/candidate_queue_transaction`, `/api/market_intel/manual_sample_review/candidate_queue_writer_status`, `/api/market_intel/match_review_plan`, `/api/market_intel/opportunity_plan`, `/api/market_intel/opportunity_scoring_plan`, `/api/market_intel/opportunity_evidence_plan`, `/api/market_intel/opportunity_alert_plan`, `/api/market_intel/adapters`, `/api/market_intel/dry_run_plan`, `/api/market_intel/discovery_plan`, `/api/market_intel/manual_discovery`, `/api/market_intel/candidate_preview`, `/api/market_intel/platform_seed_plan`, `/api/market_intel/platform_seed_write_guard`, `/api/market_intel/platform_seed_writer_plan`, `/api/market_intel/migration_blueprint`, `/api/market_intel/migration_apply_drill`, `/api/market_intel/migration_catalog_review`, `/api/market_intel/migration_live_smoke`, `/api/market_intel/live_db_inventory`, `/api/market_intel/seed_writer_cli_status`, `/api/market_intel/write_approval_runbook`, `/api/market_intel/deployment_readiness` |
| `api_routes.py` | 通用任務與查詢 API | `/api/run_task`, `/api/history/*` |
| `export_routes.py` | 匯出功能 | `/api/export/*` |
| `import_routes.py` | 匯入功能 | `/api/import_excel`, `/api/import/monthly_summary` |

38
routes/market_intel_routes.py
View File

@@ -9,6 +9,9 @@ from flask import Blueprint, jsonify, render_template, request
from auth import login_required
from config import SYSTEM_VERSION
from services.market_intel import MarketIntelService
from services.market_intel.candidate_queue_writer_cli import (
build_candidate_queue_writer_cli_plan,
)
TAIPEI_TZ = timezone(timedelta(hours=8))
@@ -292,6 +295,41 @@ def market_intel_manual_sample_candidate_queue_transaction():
)
@market_intel_bp.route(
"/api/market_intel/manual_sample_review/candidate_queue_writer_status",
methods=["POST"],
)
@login_required
def market_intel_manual_sample_candidate_queue_writer_status():
payload = request.get_json(silent=True)
limit = request.args.get("limit", default=20, type=int)
execute_requested = request.args.get("execute", "false").lower() == "true"
apply_real_write = request.args.get("apply_real_write", "false").lower() == "true"
service = _service()
payload_error = None
if not isinstance(payload, dict):
sample_result = None
payload_error = "invalid_json_object"
else:
sample_result = payload.get("sample_result", payload)
transaction_preview = service.build_manual_sample_candidate_queue_transaction(
sample_result=sample_result,
payload_error=payload_error,
limit=limit,
)
data = build_candidate_queue_writer_cli_plan(
transaction_preview=transaction_preview,
execute_requested=execute_requested,
apply_real_write=apply_real_write,
approval_token=None,
approval_token_secret=None,
)
data["phase"] = service.phase
status_code = 400 if payload_error else 200
return jsonify(data), status_code
@market_intel_bp.route("/api/market_intel/match_review_plan")
@login_required
def market_intel_match_review_plan():

91
scripts/market_intel_candidate_queue_writer.py Executable file
View File

@@ -0,0 +1,91 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Market intelligence candidate queue writer CLI gate.
This script prints a JSON gate status. Real queue writes are intentionally
disabled in this phase even when --execute and --apply-real-write are provided.
"""
import argparse
import contextlib
import json
import os
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
if str(REPO_ROOT) not in sys.path:
sys.path.insert(0, str(REPO_ROOT))
with contextlib.redirect_stdout(sys.stderr):
from services.market_intel import MarketIntelService # noqa: E402
from services.market_intel.candidate_queue_writer_cli import ( # noqa: E402
APPROVAL_ENV_VAR,
build_candidate_queue_writer_cli_plan,
)
def parse_args(argv=None):
parser = argparse.ArgumentParser(
description="Preview candidate queue writer execution gates."
)
parser.add_argument(
"--sample-json",
default=None,
help="Path to one manual sample result JSON file. Default: empty payload.",
)
parser.add_argument(
"--execute",
action="store_true",
help="Request real execution. Still blocked in this phase.",
)
parser.add_argument(
"--apply-real-write",
action="store_true",
help="Request the guarded queue write transaction. Still blocked in this phase.",
)
parser.add_argument(
"--approval-token",
default=None,
help=f"One-time approval token checked against {APPROVAL_ENV_VAR}.",
)
return parser.parse_args(argv)
def _load_sample_result(path):
if not path:
return {}
with open(path, encoding="utf-8") as handle:
payload = json.load(handle)
return payload.get("sample_result", payload) if isinstance(payload, dict) else payload
def main(argv=None):
args = parse_args(argv)
sample_result = _load_sample_result(args.sample_json)
if not isinstance(sample_result, dict):
sample_result = None
payload_error = "invalid_json_object"
else:
payload_error = None
service = MarketIntelService()
transaction_preview = service.build_manual_sample_candidate_queue_transaction(
sample_result=sample_result,
payload_error=payload_error,
)
plan = build_candidate_queue_writer_cli_plan(
transaction_preview=transaction_preview,
execute_requested=args.execute,
apply_real_write=args.apply_real_write,
approval_token=args.approval_token,
approval_token_secret=os.getenv(APPROVAL_ENV_VAR),
)
plan["phase"] = service.phase
print(json.dumps(plan, ensure_ascii=False, indent=2, sort_keys=True))
return int(plan.get("exit_code", 2))
if __name__ == "__main__":
raise SystemExit(main())

176
services/market_intel/candidate_queue_writer_cli.py Normal file
View File

@@ -0,0 +1,176 @@
"""市場情報候選審核 queue writer CLI gate。
本模組只建立正式 queue writer 前的安全 gate 狀態；不開 DB connection、
不開 transaction、不寫 market_alert_review_queue、不掛 scheduler。
"""
import hmac
import os
APPROVAL_ENV_VAR = "MARKET_INTEL_QUEUE_WRITE_APPROVAL"
MIN_APPROVAL_TOKEN_LENGTH = 16
def _approval_token_valid(approval_token, approval_token_secret):
if not approval_token or not approval_token_secret:
return False
if len(str(approval_token_secret)) < MIN_APPROVAL_TOKEN_LENGTH:
return False
return hmac.compare_digest(str(approval_token), str(approval_token_secret))
def build_candidate_queue_writer_cli_plan(
*,
transaction_preview,
execute_requested=False,
approval_token=None,
approval_token_secret=None,
apply_real_write=False,
):
"""建立候選審核 queue writer CLI gate；本階段拒絕實際寫入。"""
approval_token_present = bool(approval_token)
approval_token_secret = approval_token_secret or os.getenv(APPROVAL_ENV_VAR)
approval_token_secret_configured = bool(approval_token_secret)
approval_token_valid = _approval_token_valid(approval_token, approval_token_secret)
summary = transaction_preview.get("transaction_summary", {})
statement_count = int(summary.get("statement_count") or 0)
transaction_preview_created = bool(
transaction_preview.get("transaction_preview_created")
)
writer_enabled = False
gates = [
{
"key": "script_created",
"label": "scripts/market_intel_candidate_queue_writer.py exists",
"passed": True,
},
{
"key": "transaction_preview_created",
"label": "候選 queue transaction preview 已建立",
"passed": transaction_preview_created,
},
{
"key": "transaction_has_statements",
"label": "transaction preview 至少包含一筆 statement",
"passed": statement_count > 0,
},
{
"key": "execute_requested",
"label": "--execute flag was explicitly provided",
"passed": bool(execute_requested),
},
{
"key": "approval_token_present",
"label": f"{APPROVAL_ENV_VAR} approval token was provided",
"passed": approval_token_present,
},
{
"key": "approval_token_secret_configured",
"label": f"{APPROVAL_ENV_VAR} environment token is configured",
"passed": approval_token_secret_configured,
},
{
"key": "approval_token_valid",
"label": "approval token matches the configured environment token",
"passed": approval_token_valid,
},
{
"key": "apply_real_write_requested",
"label": "--apply-real-write flag was explicitly provided",
"passed": bool(apply_real_write),
},
{
"key": "backup_verified",
"label": "正式寫入前必須確認最新備份已完成",
"passed": False,
},
{
"key": "migration_live_smoke_passed",
"label": "正式 schema live smoke 必須通過",
"passed": False,
},
{
"key": "queue_writer_implementation_enabled",
"label": "候選 queue writer 實際寫入實作仍未啟用",
"passed": writer_enabled,
},
{
"key": "manual_operator_approval",
"label": "操作者需在 CLI 明確批准一次性寫入",
"passed": bool(execute_requested and apply_real_write and approval_token_valid),
},
{
"key": "crawler_stays_disabled",
"label": "queue writer 不掛 crawler 或 scheduler",
"passed": True,
},
]
blocked_reasons = [gate["key"] for gate in gates if not gate["passed"]]
blocked_reasons.append("candidate_queue_writer_execution_not_enabled")
return {
"mode": "candidate_queue_writer_cli_blocked",
"target_table": "market_alert_review_queue",
"execute_requested": bool(execute_requested),
"apply_real_write_requested": bool(apply_real_write),
"approval_token_present": approval_token_present,
"approval_token_valid": approval_token_valid,
"approval_env_var": APPROVAL_ENV_VAR,
"approval_token_secret_configured": approval_token_secret_configured,
"queue_writer_implementation_enabled": writer_enabled,
"ready_for_real_write": False,
"writes_executed": False,
"would_write_database": False,
"database_connection_opened": False,
"database_session_created": False,
"explicit_transaction_opened": False,
"database_write_executed": False,
"database_commit_executed": False,
"database_rollback_executed": False,
"external_network_executed": False,
"scheduler_attached": False,
"exit_code": 2 if execute_requested else 0,
"blocked_reasons": blocked_reasons,
"approval_gates": gates,
"transaction_preview_summary": {
"mode": transaction_preview.get("mode"),
"transaction_preview_created": transaction_preview_created,
"transaction_ready": bool(transaction_preview.get("transaction_ready")),
"transaction_opened": bool(transaction_preview.get("transaction_opened")),
"transaction_committed": bool(
transaction_preview.get("transaction_committed")
),
"statement_count": statement_count,
"idempotency_key_count": int(summary.get("idempotency_key_count") or 0),
"conflict_policy": summary.get("conflict_policy"),
},
"transaction_preview": transaction_preview,
"rollback_plan": [
{
"key": "no_write_no_db_rollback_required",
"label": "本階段不寫 DB；若被阻擋，不需要 DB rollback",
},
{
"key": "future_dedupe_key_cleanup_review",
"label": "未來正式寫入若需回退，必須依 dedupe_key 人工審核清理",
},
],
"safety_contract": {
"refuses_api_execution": True,
"refuses_execute_without_apply_flag": True,
"requires_independent_approval_token": True,
"does_not_open_transaction_from_status": True,
"does_not_commit_from_status": True,
"keeps_crawler_disabled_for_queue_write": True,
"target_table": "market_alert_review_queue",
},
"safe_boundaries": [
"do_not_execute_candidate_queue_writer_from_api",
"do_not_open_database_connection_from_queue_writer_status",
"do_not_commit_queue_writer_status",
"do_not_attach_scheduler_from_queue_writer",
"no_remove_orphans",
"no_momo_db_lifecycle_change",
],
}

20
services/market_intel/deployment_readiness.py
View File

@@ -3,6 +3,10 @@
本模組只組裝 preview payload，不執行 git、部署、SSH、migration 或 DB write。
"""
from services.market_intel.candidate_queue_writer_cli import (
build_candidate_queue_writer_cli_plan,
)
def build_deployment_readiness_preview(
*,
@@ -36,6 +40,9 @@ def build_deployment_readiness_preview(
manual_sample_candidate_queue_transaction = (
service.build_manual_sample_candidate_queue_transaction(sample_result={})
)
candidate_queue_writer_cli_status = build_candidate_queue_writer_cli_plan(
transaction_preview=manual_sample_candidate_queue_transaction,
)
match_review_plan = service.build_match_review_plan()
opportunity_plan = service.build_opportunity_plan()
opportunity_scoring_plan = service.build_opportunity_scoring_plan()
@@ -191,6 +198,18 @@ def build_deployment_readiness_preview(
and not manual_sample_candidate_queue_transaction["database_commit_executed"]
and not manual_sample_candidate_queue_transaction["scheduler_attached"]
),
"candidate_queue_writer_cli_status_safe": bool(
candidate_queue_writer_cli_status["mode"]
== "candidate_queue_writer_cli_blocked"
and not candidate_queue_writer_cli_status["ready_for_real_write"]
and not candidate_queue_writer_cli_status["writes_executed"]
and not candidate_queue_writer_cli_status["would_write_database"]
and not candidate_queue_writer_cli_status["database_connection_opened"]
and not candidate_queue_writer_cli_status["explicit_transaction_opened"]
and not candidate_queue_writer_cli_status["database_write_executed"]
and not candidate_queue_writer_cli_status["database_commit_executed"]
and not candidate_queue_writer_cli_status["scheduler_attached"]
),
"match_review_plan_preview_safe": bool(
match_review_plan["mode"] == "match_review_plan_preview"
and not match_review_plan["review_queue_created"]
@@ -441,6 +460,7 @@ def build_deployment_readiness_preview(
"manual_sample_candidate_queue_transaction": (
manual_sample_candidate_queue_transaction
),
"candidate_queue_writer_cli_status": candidate_queue_writer_cli_status,
"match_review_plan": match_review_plan,
"opportunity_plan": opportunity_plan,
"opportunity_scoring_plan": opportunity_scoring_plan,

2
services/market_intel/service.py
View File

@@ -108,7 +108,7 @@ class MarketIntelRuntimeStatus:
class MarketIntelService:
"""市場情報入口服務，先集中 feature gate 與安全狀態。"""
phase = "phase_54_manual_sample_candidate_queue_transaction"
phase = "phase_55_candidate_queue_writer_cli_gate"
def get_runtime_status(self) -> MarketIntelRuntimeStatus:
return MarketIntelRuntimeStatus(

68
templates/market_intel/disabled.html
View File

@@ -251,7 +251,7 @@
align-items: stretch;
display: grid;
gap: 0.75rem;
grid-template-columns: minmax(0, 1fr) auto auto auto auto auto;
grid-template-columns: minmax(0, 1fr) auto auto auto auto auto auto;
min-width: 0;
}
@@ -588,6 +588,9 @@
<button class="market-intel-icon-button" type="button" title="產生 queue transaction preview" data-market-intel-sample-candidate-queue-transaction>
<i class="fas fa-code" aria-hidden="true"></i>
</button>
<button class="market-intel-icon-button" type="button" title="檢查 queue writer CLI gate" data-market-intel-sample-candidate-queue-writer>
<i class="fas fa-terminal" aria-hidden="true"></i>
</button>
</div>
</div>
@@ -916,12 +919,14 @@
const sampleCandidateQueueDraft = sampleReviewRoot ? sampleReviewRoot.querySelector('[data-market-intel-sample-candidate-queue-draft]') : null;
const sampleCandidateQueueApproval = sampleReviewRoot ? sampleReviewRoot.querySelector('[data-market-intel-sample-candidate-queue-approval]') : null;
const sampleCandidateQueueTransaction = sampleReviewRoot ? sampleReviewRoot.querySelector('[data-market-intel-sample-candidate-queue-transaction]') : null;
const sampleCandidateQueueWriter = sampleReviewRoot ? sampleReviewRoot.querySelector('[data-market-intel-sample-candidate-queue-writer]') : null;
const sampleReviewEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_review') }}";
const sampleReviewEvaluateEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_review_evaluate') }}";
const sampleCandidateHandoffEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_handoff') }}";
const sampleCandidateQueueDraftEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_queue_draft') }}";
const sampleCandidateQueueApprovalEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_queue_approval') }}";
const sampleCandidateQueueTransactionEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_queue_transaction') }}";
const sampleCandidateQueueWriterEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_queue_writer_status') }}";
const schedulerMeta = schedulerRoot ? schedulerRoot.querySelector('[data-market-intel-scheduler-meta]') : null;
const schedulerBody = schedulerRoot ? schedulerRoot.querySelector('[data-market-intel-scheduler-body]') : null;
const schedulerRefresh = schedulerRoot ? schedulerRoot.querySelector('[data-market-intel-scheduler-refresh]') : null;
@@ -2260,6 +2265,64 @@
}
};
const renderCandidateQueueWriter = data => {
const blockers = (data.blocked_reasons || []).join(' / ');
const summary = data.transaction_preview_summary || {};
const gates = data.approval_gates || [];
sampleReviewMeta.innerHTML = [
`mode=${data.mode || 'unknown'}`,
`ready=${data.ready_for_real_write ? 'yes' : 'no'}`,
`statements=${summary.statement_count || 0}`,
`execute=${data.execute_requested ? 'yes' : 'no'}`,
`apply=${data.apply_real_write_requested ? 'yes' : 'no'}`
].map(item => `<span class="market-intel-pill">${escapeHtml(item)}</span>`).join('');
sampleReviewBody.innerHTML = `
<div class="market-intel-empty mb-3">此卡只檢查 CLI writer gate；不讀取 token、不開 DB connection、不寫 queue。${blockers ? `阻擋：${escapeHtml(blockers)}` : ''}</div>
<div class="market-intel-check-list">${
gates.map(gate => `
<div class="market-intel-check">
<span>${gate.passed ? 'PASS' : 'BLOCK'}</span>
<div>
<strong>${escapeHtml(gate.key)}</strong>
<small>${escapeHtml(gate.label)}</small>
</div>
</div>
`).join('')
}</div>
`;
};
const loadCandidateQueueWriter = async () => {
if (!sampleReviewMeta || !sampleReviewBody || !sampleReviewInput) return;
let parsed;
try {
parsed = JSON.parse(sampleReviewInput.value || '{}');
} catch (error) {
sampleReviewMeta.innerHTML = '<span class="market-intel-pill">json_error</span>';
sampleReviewBody.innerHTML = `<div class="market-intel-empty">JSON 格式錯誤：${escapeHtml(error.message)}</div>`;
return;
}
sampleReviewBody.innerHTML = '<div class="market-intel-empty">檢查 queue writer CLI gate 中...</div>';
try {
const response = await fetch(sampleCandidateQueueWriterEndpoint, {
method: 'POST',
credentials: 'same-origin',
headers: {
'Content-Type': 'application/json',
'X-CSRFToken': csrfToken
},
body: JSON.stringify({ sample_result: parsed })
});
const data = await response.json();
if (!response.ok && !data.mode) throw new Error(`HTTP ${response.status}`);
renderCandidateQueueWriter(data);
} catch (error) {
sampleReviewMeta.innerHTML = '<span class="market-intel-pill">error</span>';
sampleReviewBody.innerHTML = `<div class="market-intel-empty">queue writer CLI gate 檢查失敗：${escapeHtml(error.message)}</div>`;
}
};
const renderSchedulerMeta = data => {
schedulerMeta.innerHTML = [
`mode=${data.mode || 'unknown'}`,
@@ -3766,6 +3829,9 @@
if (sampleCandidateQueueTransaction) {
sampleCandidateQueueTransaction.addEventListener('click', loadCandidateQueueTransaction);
}
if (sampleCandidateQueueWriter) {
sampleCandidateQueueWriter.addEventListener('click', loadCandidateQueueWriter);
}
if (schedulerRefresh) {
schedulerRefresh.addEventListener('click', loadScheduler);
}

345
tests/test_market_intel_skeleton.py
View File

@@ -522,6 +522,7 @@ def test_market_intel_preview_template_uses_safe_fetch_false_endpoint():
assert "data-market-intel-sample-candidate-queue-draft" in template
assert "data-market-intel-sample-candidate-queue-approval" in template
assert "data-market-intel-sample-candidate-queue-transaction" in template
assert "data-market-intel-sample-candidate-queue-writer" in template
assert "data-market-intel-approval" in template
assert "data-market-intel-approval-gates" in template
assert "data-market-intel-deploy" in template
@@ -545,6 +546,7 @@ def test_market_intel_preview_template_uses_safe_fetch_false_endpoint():
assert "market_intel.market_intel_manual_sample_candidate_queue_draft" in template
assert "market_intel.market_intel_manual_sample_candidate_queue_approval" in template
assert "market_intel.market_intel_manual_sample_candidate_queue_transaction" in template
assert "market_intel.market_intel_manual_sample_candidate_queue_writer_status" in template
assert "X-CSRFToken" in template
assert "market_intel.market_intel_scheduler_plan" in template
assert "market_intel.market_intel_match_review_plan" in template
@@ -581,7 +583,7 @@ def test_legacy_source_bridge_default_is_planned_only():
bridge = MarketIntelService().build_legacy_source_bridge()
assert bridge["mode"] == "legacy_source_bridge_planned"
assert bridge["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert bridge["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert bridge["execute_requested"] is False
assert bridge["read_only_query_executed"] is False
assert bridge["database_connection_opened"] is False
@@ -739,7 +741,7 @@ def test_mcp_tool_contract_preview_is_read_only_and_whitelisted():
contract = MarketIntelService().build_mcp_tool_contract()
assert contract["mode"] == "mcp_tool_contract_preview"
assert contract["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert contract["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert contract["caller"] == "market_intel"
assert contract["contract_ready"] is True
assert contract["blocked_reasons"] == []
@@ -872,7 +874,7 @@ def test_mcp_activation_runbook_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "mcp_activation_runbook_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["deployment_actions_executed"] is False
assert data["docker_command_executed"] is False
assert data["ssh_command_executed"] is False
@@ -885,7 +887,7 @@ def test_mcp_fetch_gate_default_blocks_external_fetch():
gate = MarketIntelService().build_mcp_fetch_gate(fetch_requested=True)
assert gate["mode"] == "mcp_fetch_gate_planned"
assert gate["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert gate["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert gate["fetch_requested"] is True
assert gate["manual_fetch_gate_open"] is False
assert gate["network_request_allowed"] is False
@@ -955,7 +957,7 @@ def test_mcp_fetch_gate_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "mcp_fetch_gate_planned"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["fetch_requested"] is False
assert data["network_request_allowed"] is False
assert data["external_network_executed"] is False
@@ -967,7 +969,7 @@ def test_manual_sample_plan_preview_blocks_fetch_and_write():
plan = MarketIntelService().build_manual_sample_plan()
assert plan["mode"] == "manual_sample_fetch_plan_preview"
assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert plan["ready_for_manual_sample_fetch"] is False
assert plan["sample_fetch_executed"] is False
assert plan["external_network_executed"] is False
@@ -1015,7 +1017,7 @@ def test_manual_sample_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "manual_sample_fetch_plan_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["sample_fetch_executed"] is False
assert data["external_network_executed"] is False
assert data["database_write_executed"] is False
@@ -1026,7 +1028,7 @@ def test_manual_sample_acceptance_preview_blocks_candidate_import():
acceptance = MarketIntelService().build_manual_sample_acceptance()
assert acceptance["mode"] == "manual_sample_acceptance_preview"
assert acceptance["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert acceptance["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert acceptance["contract_ready"] is True
assert acceptance["sample_result_loaded"] is False
assert acceptance["sample_result_accepted"] is False
@@ -1068,7 +1070,7 @@ def test_manual_sample_acceptance_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "manual_sample_acceptance_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["sample_result_loaded"] is False
assert data["candidate_import_allowed"] is False
assert data["external_network_executed"] is False
@@ -1080,7 +1082,7 @@ def test_manual_sample_review_preview_is_planned_until_result_loaded():
review = MarketIntelService().build_manual_sample_review()
assert review["mode"] == "manual_sample_review_preview"
assert review["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert review["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert review["contract_ready"] is True
assert review["sample_result_loaded"] is False
assert review["sample_result_reviewed"] is False
@@ -1191,7 +1193,7 @@ def test_manual_sample_review_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "manual_sample_review_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["sample_result_loaded"] is False
assert data["sample_result_reviewed"] is False
assert data["candidate_import_allowed"] is False
@@ -1230,7 +1232,7 @@ def test_manual_sample_review_evaluation_preview_accepts_payload_without_persist
)
assert review["mode"] == "manual_sample_review_evaluation_preview"
assert review["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert review["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert review["review_request_type"] == "operator_posted_json"
assert review["payload_received"] is True
assert review["payload_valid_json_object"] is True
@@ -1292,7 +1294,7 @@ def test_manual_sample_review_evaluate_route_is_post_only_and_no_write():
assert response.status_code == 200
assert data["mode"] == "manual_sample_review_evaluation_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["payload_received"] is True
assert data["payload_valid_json_object"] is True
assert data["payload_persisted"] is False
@@ -1372,7 +1374,7 @@ def test_manual_sample_candidate_handoff_preview_creates_candidates_without_pers
)
assert handoff["mode"] == "manual_sample_candidate_handoff_preview"
assert handoff["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert handoff["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert handoff["payload_received"] is True
assert handoff["payload_valid_json_object"] is True
assert handoff["payload_persisted"] is False
@@ -1436,7 +1438,7 @@ def test_manual_sample_candidate_handoff_route_is_post_only_and_no_write():
assert response.status_code == 200
assert data["mode"] == "manual_sample_candidate_handoff_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["payload_received"] is True
assert data["handoff_ready"] is True
assert data["candidate_handoff_created"] is True
@@ -1495,7 +1497,7 @@ def test_manual_sample_candidate_queue_draft_preview_builds_review_items_without
)
assert queue_draft["mode"] == "manual_sample_candidate_queue_draft_preview"
assert queue_draft["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert queue_draft["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert queue_draft["payload_received"] is True
assert queue_draft["payload_valid_json_object"] is True
assert queue_draft["payload_persisted"] is False
@@ -1569,7 +1571,7 @@ def test_manual_sample_candidate_queue_draft_route_is_post_only_and_no_write():
assert response.status_code == 200
assert data["mode"] == "manual_sample_candidate_queue_draft_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["payload_received"] is True
assert data["handoff_ready"] is True
assert data["queue_draft_ready"] is True
@@ -1632,7 +1634,7 @@ def test_manual_sample_candidate_queue_approval_preview_blocks_write_and_maps_ro
)
assert approval["mode"] == "manual_sample_candidate_queue_approval_preview"
assert approval["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert approval["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert approval["payload_received"] is True
assert approval["payload_valid_json_object"] is True
assert approval["payload_persisted"] is False
@@ -1710,7 +1712,7 @@ def test_manual_sample_candidate_queue_approval_route_is_post_only_and_no_write(
assert response.status_code == 200
assert data["mode"] == "manual_sample_candidate_queue_approval_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["payload_received"] is True
assert data["approval_preview_created"] is True
assert data["approval_request_created"] is False
@@ -1773,7 +1775,7 @@ def test_manual_sample_candidate_queue_transaction_preview_blocks_execution():
)
assert transaction["mode"] == "manual_sample_candidate_queue_transaction_preview"
assert transaction["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert transaction["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert transaction["payload_received"] is True
assert transaction["payload_valid_json_object"] is True
assert transaction["payload_persisted"] is False
@@ -1851,7 +1853,7 @@ def test_manual_sample_candidate_queue_transaction_route_is_post_only_and_no_wri
assert response.status_code == 200
assert data["mode"] == "manual_sample_candidate_queue_transaction_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["payload_received"] is True
assert data["transaction_preview_created"] is True
assert data["transaction_ready"] is False
@@ -1884,11 +1886,169 @@ def test_manual_sample_candidate_queue_transaction_blocks_invalid_payload():
assert "queue_transaction_preview_not_ready" in transaction["blocked_reasons"]
def test_candidate_queue_writer_cli_gate_blocks_real_write_even_with_token():
from services.market_intel.candidate_queue_writer_cli import (
build_candidate_queue_writer_cli_plan,
)
sample_result = {
"batch_id": "sample-batch-13",
"platform_code": "momo",
"source_key": "homepage",
"source_url": "https://www.momoshop.com.tw/",
"status": "fetched",
"status_code": 200,
"content_length": 1700,
"page_hash": "a" * 64,
"title": "MOMO 活動",
"diagnostics": {
"link_count": 1,
"same_host_link_count": 1,
"campaign_link_candidates": [
{
"confidence_band": "high",
"score": 94,
"url": "https://www.momoshop.com.tw/activity/sample",
"text": "品牌活動",
},
],
},
}
transaction = MarketIntelService().build_manual_sample_candidate_queue_transaction(
sample_result=sample_result
)
status = build_candidate_queue_writer_cli_plan(
transaction_preview=transaction,
execute_requested=True,
apply_real_write=True,
approval_token=TEST_APPROVAL_TOKEN,
approval_token_secret=TEST_APPROVAL_TOKEN,
)
assert status["mode"] == "candidate_queue_writer_cli_blocked"
assert status["execute_requested"] is True
assert status["apply_real_write_requested"] is True
assert status["approval_token_present"] is True
assert status["approval_token_valid"] is True
assert status["queue_writer_implementation_enabled"] is False
assert status["ready_for_real_write"] is False
assert status["writes_executed"] is False
assert status["would_write_database"] is False
assert status["database_connection_opened"] is False
assert status["database_session_created"] is False
assert status["explicit_transaction_opened"] is False
assert status["database_write_executed"] is False
assert status["database_commit_executed"] is False
assert status["scheduler_attached"] is False
assert status["exit_code"] == 2
assert status["transaction_preview_summary"]["statement_count"] == 1
assert "approval_token_valid" not in status["blocked_reasons"]
assert "queue_writer_implementation_enabled" in status["blocked_reasons"]
assert "candidate_queue_writer_execution_not_enabled" in status["blocked_reasons"]
assert status["safety_contract"]["refuses_api_execution"] is True
assert (
"do_not_execute_candidate_queue_writer_from_api"
in status["safe_boundaries"]
)
def test_candidate_queue_writer_status_route_never_leaks_approval_token(monkeypatch):
from routes.market_intel_routes import market_intel_bp
monkeypatch.setenv("MARKET_INTEL_QUEUE_WRITE_APPROVAL", TEST_APPROVAL_TOKEN)
app = Flask(__name__)
app.secret_key = "test-secret"
app.register_blueprint(market_intel_bp)
client = app.test_client()
with client.session_transaction() as session:
session["logged_in"] = True
response = client.post(
"/api/market_intel/manual_sample_review/candidate_queue_writer_status"
"?execute=true&apply_real_write=true",
json={
"sample_result": {
"batch_id": "sample-batch-14",
"platform_code": "pchome",
"source_key": "homepage",
"source_url": "https://24h.pchome.com.tw/",
"status": "fetched",
"status_code": 200,
"content_length": 1800,
"page_hash": "b" * 64,
"title": "PChome 活動",
"diagnostics": {
"link_count": 1,
"same_host_link_count": 1,
"campaign_link_candidates": [
{
"confidence_band": "medium",
"score": 74,
"url": "https://24h.pchome.com.tw/activity/sample",
"text": "活動頁",
},
],
},
}
},
)
data = response.get_json()
payload = json.dumps(data, ensure_ascii=False, sort_keys=True)
assert response.status_code == 200
assert data["mode"] == "candidate_queue_writer_cli_blocked"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["execute_requested"] is True
assert data["apply_real_write_requested"] is True
assert data["approval_token_present"] is False
assert data["approval_token_valid"] is False
assert data["approval_token_secret_configured"] is True
assert data["ready_for_real_write"] is False
assert data["writes_executed"] is False
assert data["would_write_database"] is False
assert data["database_connection_opened"] is False
assert data["database_write_executed"] is False
assert data["database_commit_executed"] is False
assert data["scheduler_attached"] is False
assert data["transaction_preview_summary"]["statement_count"] == 1
assert "approval_token_present" in data["blocked_reasons"]
assert "approval_token_valid" in data["blocked_reasons"]
assert "approval_token_hint" not in payload
assert TEST_APPROVAL_TOKEN not in payload
assert "APPROVED_MARKET_INTEL_QUEUE_WRITE" not in payload
def test_candidate_queue_writer_status_blocks_invalid_payload():
from routes.market_intel_routes import market_intel_bp
app = Flask(__name__)
app.secret_key = "test-secret"
app.register_blueprint(market_intel_bp)
client = app.test_client()
with client.session_transaction() as session:
session["logged_in"] = True
response = client.post(
"/api/market_intel/manual_sample_review/candidate_queue_writer_status",
data="not-json",
content_type="application/json",
)
data = response.get_json()
assert response.status_code == 400
assert data["mode"] == "candidate_queue_writer_cli_blocked"
assert data["ready_for_real_write"] is False
assert data["writes_executed"] is False
assert data["transaction_preview_summary"]["statement_count"] == 0
assert "transaction_preview_created" in data["blocked_reasons"]
def test_scheduler_plan_preview_blocks_job_attachment():
plan = MarketIntelService().build_scheduler_plan()
assert plan["mode"] == "scheduler_attach_plan_preview"
assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert plan["ready_to_attach_scheduler"] is False
assert plan["scheduler_attached"] is False
assert plan["scheduler_registration_executed"] is False
@@ -1926,7 +2086,7 @@ def test_scheduler_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "scheduler_attach_plan_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["scheduler_registration_executed"] is False
assert data["crawler_job_started"] is False
assert data["external_network_executed"] is False
@@ -1937,7 +2097,7 @@ def test_match_review_plan_preview_blocks_auto_confirm():
plan = MarketIntelService().build_match_review_plan()
assert plan["mode"] == "match_review_plan_preview"
assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert plan["ready_for_review_queue"] is False
assert plan["review_queue_created"] is False
assert plan["auto_match_executed"] is False
@@ -1973,7 +2133,7 @@ def test_match_review_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "match_review_plan_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["review_queue_created"] is False
assert data["auto_confirm_executed"] is False
assert data["external_network_executed"] is False
@@ -1984,7 +2144,7 @@ def test_opportunity_plan_preview_blocks_alerts_and_ai_summary():
plan = MarketIntelService().build_opportunity_plan()
assert plan["mode"] == "opportunity_plan_preview"
assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert plan["ready_for_opportunity_queue"] is False
assert plan["opportunity_queue_created"] is False
assert plan["threat_alert_dispatched"] is False
@@ -2025,7 +2185,7 @@ def test_opportunity_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "opportunity_plan_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["opportunity_queue_created"] is False
assert data["threat_alert_dispatched"] is False
assert data["ai_summary_generated"] is False
@@ -2036,7 +2196,7 @@ def test_opportunity_scoring_plan_preview_blocks_scoring_and_alerts():
plan = MarketIntelService().build_opportunity_scoring_plan()
assert plan["mode"] == "opportunity_scoring_plan_preview"
assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert plan["ready_for_scoring_job"] is False
assert plan["scoring_job_created"] is False
assert plan["score_calculation_executed"] is False
@@ -2084,7 +2244,7 @@ def test_opportunity_scoring_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "opportunity_scoring_plan_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["scoring_job_created"] is False
assert data["score_calculation_executed"] is False
assert data["sample_scores_generated"] is False
@@ -2096,7 +2256,7 @@ def test_opportunity_evidence_plan_preview_blocks_queries_and_alerts():
plan = MarketIntelService().build_opportunity_evidence_plan()
assert plan["mode"] == "opportunity_evidence_plan_preview"
assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert plan["ready_for_evidence_bundle"] is False
assert plan["evidence_bundle_created"] is False
assert plan["evidence_query_executed"] is False
@@ -2142,7 +2302,7 @@ def test_opportunity_evidence_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "opportunity_evidence_plan_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["evidence_bundle_created"] is False
assert data["evidence_query_executed"] is False
assert data["sample_evidence_generated"] is False
@@ -2155,7 +2315,7 @@ def test_opportunity_alert_plan_preview_blocks_dispatch_and_llm_calls():
plan = MarketIntelService().build_opportunity_alert_plan()
assert plan["mode"] == "opportunity_alert_plan_preview"
assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert plan["ready_for_alert_candidates"] is False
assert plan["alert_candidate_created"] is False
assert plan["alert_queue_created"] is False
@@ -2240,7 +2400,7 @@ def test_opportunity_alert_plan_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "opportunity_alert_plan_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["alert_candidate_created"] is False
assert data["alert_queue_created"] is False
assert data["review_queue_created"] is False
@@ -2318,7 +2478,7 @@ def test_mcp_deploy_preflight_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "mcp_external_deploy_preflight_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["deployment_actions_executed"] is False
assert data["docker_command_executed"] is False
assert data["ssh_command_executed"] is False
@@ -2333,7 +2493,7 @@ def test_mcp_readiness_default_is_planned_only(monkeypatch):
readiness = MarketIntelService().build_mcp_readiness()
assert readiness["mode"] == "mcp_readiness_planned"
assert readiness["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert readiness["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert readiness["execute_requested"] is False
assert readiness["router_enabled"] is False
assert readiness["external_mcp_complete"] is False
@@ -2729,6 +2889,7 @@ def test_deployment_readiness_reports_app_only_release_gate():
assert readiness["checks"]["manual_sample_candidate_queue_draft_post_safe"] is True
assert readiness["checks"]["manual_sample_candidate_queue_approval_post_safe"] is True
assert readiness["checks"]["manual_sample_candidate_queue_transaction_post_safe"] is True
assert readiness["checks"]["candidate_queue_writer_cli_status_safe"] is True
assert readiness["checks"]["match_review_plan_preview_safe"] is True
assert readiness["checks"]["opportunity_plan_preview_safe"] is True
assert readiness["checks"]["opportunity_scoring_plan_preview_safe"] is True
@@ -3029,6 +3190,39 @@ def test_deployment_readiness_reports_app_only_release_gate():
readiness["manual_sample_candidate_queue_transaction"]["scheduler_attached"]
is False
)
assert (
readiness["candidate_queue_writer_cli_status"]["mode"]
== "candidate_queue_writer_cli_blocked"
)
assert (
readiness["candidate_queue_writer_cli_status"]["ready_for_real_write"]
is False
)
assert readiness["candidate_queue_writer_cli_status"]["writes_executed"] is False
assert (
readiness["candidate_queue_writer_cli_status"]["would_write_database"]
is False
)
assert (
readiness["candidate_queue_writer_cli_status"]["database_connection_opened"]
is False
)
assert (
readiness["candidate_queue_writer_cli_status"]["explicit_transaction_opened"]
is False
)
assert (
readiness["candidate_queue_writer_cli_status"]["database_write_executed"]
is False
)
assert (
readiness["candidate_queue_writer_cli_status"]["database_commit_executed"]
is False
)
assert (
readiness["candidate_queue_writer_cli_status"]["scheduler_attached"]
is False
)
assert readiness["scheduler_plan"]["scheduler_registration_executed"] is False
assert readiness["scheduler_plan"]["crawler_job_started"] is False
assert readiness["scheduler_plan"]["database_write_executed"] is False
@@ -3133,7 +3327,7 @@ def test_migration_apply_drill_planned_is_safe_and_manual_only():
drill = MarketIntelService().build_migration_apply_drill()
assert drill["mode"] == "migration_apply_drill_preview"
assert drill["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert drill["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert drill["execute_requested"] is False
assert drill["schema_state"] == "planned_no_db_probe"
assert drill["drill_ready_for_operator_review"] is True
@@ -3248,7 +3442,7 @@ def test_migration_apply_drill_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "migration_apply_drill_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["execute_requested"] is False
assert data["migration_executed"] is False
assert data["rollback_executed"] is False
@@ -3260,7 +3454,7 @@ def test_migration_catalog_review_planned_is_safe_and_diagnostic():
review = MarketIntelService().build_migration_catalog_review()
assert review["mode"] == "migration_catalog_review_preview"
assert review["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert review["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert review["execute_requested"] is False
assert review["catalog_state"] == "planned_no_probe"
assert review["seed_state"] == "planned_no_probe"
@@ -3375,7 +3569,7 @@ def test_migration_catalog_review_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "migration_catalog_review_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["execute_requested"] is False
assert data["catalog_state"] == "planned_no_probe"
assert data["migration_executed"] is False
@@ -3388,7 +3582,7 @@ def test_migration_live_smoke_planned_is_preview_only():
smoke = MarketIntelService().build_migration_live_smoke()
assert smoke["mode"] == "migration_live_smoke_preview"
assert smoke["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert smoke["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert smoke["execute_requested"] is False
assert smoke["smoke_result"] == "planned_no_execution"
assert smoke["live_smoke_passed"] is False
@@ -3450,7 +3644,7 @@ def test_migration_live_smoke_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "migration_live_smoke_preview"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["execute_requested"] is False
assert data["smoke_result"] == "planned_no_execution"
assert data["migration_executed"] is False
@@ -3463,7 +3657,7 @@ def test_live_db_inventory_planned_is_preview_only():
inventory = MarketIntelService().build_live_db_inventory()
assert inventory["mode"] == "live_db_inventory_planned"
assert inventory["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert inventory["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert inventory["execute_requested"] is False
assert inventory["read_only_query_executed"] is False
assert inventory["database_connection_opened"] is False
@@ -3607,7 +3801,7 @@ def test_live_db_inventory_route_is_preview_only():
assert response.status_code == 200
assert data["mode"] == "live_db_inventory_planned"
assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["execute_requested"] is False
assert data["read_only_query_executed"] is False
assert data["database_write_executed"] is False
@@ -3779,3 +3973,68 @@ def test_seed_writer_cli_script_outputs_blocked_plan():
assert data["exit_code"] == 0
assert data["transaction_preview"]["statement_count"] == 4
assert data["transaction_preview"]["transaction_opened"] is False
def test_candidate_queue_writer_cli_script_outputs_blocked_gate(tmp_path):
sample_file = tmp_path / "sample.json"
sample_file.write_text(
json.dumps(
{
"batch_id": "sample-batch-15",
"platform_code": "momo",
"source_key": "homepage",
"source_url": "https://www.momoshop.com.tw/",
"status": "fetched",
"status_code": 200,
"content_length": 1700,
"page_hash": "c" * 64,
"title": "MOMO 活動",
"diagnostics": {
"link_count": 1,
"same_host_link_count": 1,
"campaign_link_candidates": [
{
"confidence_band": "high",
"score": 94,
"url": "https://www.momoshop.com.tw/activity/sample",
"text": "品牌活動",
},
],
},
},
ensure_ascii=False,
),
encoding="utf-8",
)
env = {
**os.environ,
"MOMO_ALLOW_INSECURE_CONFIG_FOR_TESTS": "true",
"SECRET_KEY": "test",
"LOGIN_PASSWORD": "test",
}
result = subprocess.run(
[
sys.executable,
"scripts/market_intel_candidate_queue_writer.py",
"--sample-json",
str(sample_file),
],
capture_output=True,
check=False,
env=env,
text=True,
)
data = json.loads(result.stdout)
assert result.returncode == 0
assert data["mode"] == "candidate_queue_writer_cli_blocked"
assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate"
assert data["execute_requested"] is False
assert data["apply_real_write_requested"] is False
assert data["writes_executed"] is False
assert data["database_connection_opened"] is False
assert data["database_write_executed"] is False
assert data["database_commit_executed"] is False
assert data["scheduler_attached"] is False
assert data["transaction_preview_summary"]["statement_count"] == 1
assert "execute_requested" in data["blocked_reasons"]