diff --git a/TODO_NEXT_STEPS.txt b/TODO_NEXT_STEPS.txt index 5f29bd6..b99b6b6 100644 --- a/TODO_NEXT_STEPS.txt +++ b/TODO_NEXT_STEPS.txt @@ -116,6 +116,7 @@ - Phase 54 manual sample candidate queue transaction:新增 `/api/market_intel/manual_sample_review/candidate_queue_transaction` POST 與 UI transaction preview 按鈕,將 queue row preview 轉成 `market_alert_review_queue` idempotent insert statement、payload hash 與 rollback plan;不開 DB connection、不開 transaction、不 commit、不建立 approval record;版本同步至 V10.227。 - V10.228 補 PPT 視覺 QA 背景狀態卡:新增 `/observability/ppt_audit/vision_status` 與頁面 Vision QA 狀態卡,讓立即視覺 QA 排入後可看 queued/running/completed/error 與最近審核摘要,不必刷新猜測。 - V10.229 修正 PPT 視覺 QA 多 worker 狀態漂移:將 queued/running/completed/error 寫入 `/app/data/ppt_vision_audit_status.json` runtime state,所有 Gunicorn worker 共用同一份狀態並阻擋重複排入。 + - Phase 55 candidate queue writer CLI gate:新增 `/api/market_intel/manual_sample_review/candidate_queue_writer_status` POST、`scripts/market_intel_candidate_queue_writer.py` 與 UI writer gate 按鈕,定義 `MARKET_INTEL_QUEUE_WRITE_APPROVAL` 一次性 token、execute/apply flags、備份、migration smoke 與 rollback gate;本階段仍不開 DB connection、不寫 `market_alert_review_queue`、不 commit、不掛 scheduler;版本同步至 V10.230。 - Schema smoke:`tests/test_market_intel_skeleton.py` 檢查 `Base.metadata` 內含 ADR-035 八張 `market_*` tables。 - Desktop UI QA:本機只註冊 `market_intel_bp` 的 Flask harness 載入 `/market_intel`,確認 Phase 15、候選預覽、writer preview、安全 flags、點陣暖紙視覺正常,console error 0。 - API QA:`/api/market_intel/schema_smoke` 通過 7 張表與 `market_platforms` 必要欄位檢查;`/api/market_intel/platform_seed_writer_plan` 回傳 4 筆 dry-run upsert preview,`writes_executed=false`,四平台皆 `blocked_dry_run_only`。 diff --git a/config.py b/config.py index 7e003b6..be753ab 100644 --- a/config.py +++ b/config.py @@ -320,7 +320,7 @@ YOUTUBE_API_KEY = os.getenv('YOUTUBE_API_KEY', '') # ========================================== # 系統版本與路徑 # ========================================== -SYSTEM_VERSION = "V10.229" +SYSTEM_VERSION = "V10.230" LOG_FILE_PATH = os.path.join(BASE_DIR, 'logs/system.log') public_url = PUBLIC_URL # 用於模板顯示 diff --git a/docs/adr/ADR-035-cross-platform-market-campaign-intelligence.md b/docs/adr/ADR-035-cross-platform-market-campaign-intelligence.md index 7ac1ab0..939ab5e 100644 --- a/docs/adr/ADR-035-cross-platform-market-campaign-intelligence.md +++ b/docs/adr/ADR-035-cross-platform-market-campaign-intelligence.md @@ -180,6 +180,7 @@ EwoooC 目前已有 MOMO EDM / 節慶活動資料、`promo_products`、PChome - 2026-05-19 追加 manual sample candidate queue draft:`services.market_intel.manual_sample_candidate_queue` 與 `/api/market_intel/manual_sample_review/candidate_queue_draft` 將 handoff 候選轉成只讀人工審核 queue draft,包含 queue item key、review priority、review state 與 approval_required。此入口保持 CSRF 保護,不建立正式 queue、不保存草案、不寫 market_*、不自動核准候選、不掛 scheduler。 - 2026-05-19 追加 manual sample candidate queue approval:`/api/market_intel/manual_sample_review/candidate_queue_approval` 將 queue draft 對齊既有 `market_alert_review_queue` 契約,輸出 row preview、必填欄位 gate、寫入 flags、備份與人工批准 gate。此入口保持 CSRF 保護,不建立 approval record、不寫 review queue、不開 DB transaction、不掛 scheduler。 - 2026-05-19 追加 manual sample candidate queue transaction:`/api/market_intel/manual_sample_review/candidate_queue_transaction` 將 queue row preview 轉成 idempotent insert statement preview、payload hash、runtime order 與 rollback plan。此入口保持 CSRF 保護,不開 DB connection、不開 transaction、不 commit、不建立 approval record、不寫 `market_alert_review_queue`。 +- 2026-05-19 追加 candidate queue writer CLI gate:`services.market_intel.candidate_queue_writer_cli`、`scripts/market_intel_candidate_queue_writer.py` 與 `/api/market_intel/manual_sample_review/candidate_queue_writer_status` 定義 `MARKET_INTEL_QUEUE_WRITE_APPROVAL` 一次性 token、execute/apply flags、備份、migration smoke 與 rollback gate。此階段只回 writer status,不開 DB connection、不啟用實際 writer、不寫 `market_alert_review_queue`、不掛 scheduler。 ### Phase 4:Coupang / Shopee Adapter diff --git a/docs/memory/code_modularization_inventory_20260430.md b/docs/memory/code_modularization_inventory_20260430.md index abf262e..4931037 100644 --- a/docs/memory/code_modularization_inventory_20260430.md +++ b/docs/memory/code_modularization_inventory_20260430.md @@ -7,11 +7,12 @@ - Python 總量:約 90,293 行(排除 `venv/`、`backups/`、`__pycache__/`、`.claude/worktrees/`)。 - 最大壓力區:`services/` 約 42,364 行、`routes/` 約 29,511 行。 - `app.py` 目前約 1,232 行,功能定位應固定為 bootstrap / Blueprint registration / startup guard,不再承接新 route。 -- 目前工作樹仍有 26 個 Python 檔案達到或超過 800 行;這些不是禁止修 bug,而是禁止繼續塞新功能。 +- 目前工作樹仍有 27 個 Python 檔案達到或超過 800 行;這些不是禁止修 bug,而是禁止繼續塞新功能。 - 2026-05-05 追記:Phase 38→56 觀測台戰役讓 `routes/admin_observability_routes.py` 與 `run_scheduler.py` 進入大檔治理清單;後續觀測台功能應先抽 query/action service,不再把新 SQL 與 L2 mutation 直接塞回 route。 - 2026-05-06 追記:跨平台市場情報模組啟動前,必須先把新增爬蟲、排程、DB schema、UI route 全部隔離在 `market_*` / `services/market_intel/` / `routes/market_intel_routes.py`,不可塞回既有大檔。 - 2026-05-18 追記:Phase 42 市場情報只在 `app.py` 的 `EXPECTED_METADATA_TABLES` 補上 `market_alert_review_queue` 名稱,未新增 route / bootstrap 邏輯;後續仍應把 metadata verification 抽到 app factory 或 startup guard module,避免 `app.py` 繼續承接功能。 - 2026-05-19 追記:同步治理測試盤點,校正 `routes/admin_observability_routes.py` 行數;此處只更新 inventory,不變更觀測台功能。 +- 2026-05-19 追記:V10.229 之後 `services/ppt_vision_service.py` 進入 800 行治理清單;本次只補 inventory 讓守門測試反映現況,不變更 PPT 視覺 QA 功能。 ## 達到或超過 800 行檔案清單 @@ -41,6 +42,7 @@ | 844 | `services/ollama_service.py` | P2 Ollama client | host health / request client / fallback policy / response parsing | | 837 | `services/code_review_pipeline_service.py` | P2 Code review pipeline service | scan orchestration / finding normalization / persistence adapter | | 832 | `routes/export_routes.py` | P2 Export flow | export command/router glue / file path / download orchestration | +| 816 | `services/ppt_vision_service.py` | P2 PPT vision QA service | runtime state / queue status / model probe / audit execution 分離 | | 809 | `services/competitor_price_feeder.py` | P2 competitor price feeder | crawler scheduling / price normalization / cache strategy | | 805 | `routes/bot_api_routes.py` | P2 Bot API Blueprint | route glue / bot action service | diff --git a/routes/README.md b/routes/README.md index 5544733..4b565ad 100644 --- a/routes/README.md +++ b/routes/README.md @@ -19,7 +19,7 @@ | `edm_routes.py` | EDM 與節慶儀表板 | `/edm`, `/festival` | | `monthly_routes.py` | 月結分析 | `/monthly_summary_analysis`, `/api/monthly_summary_data` | | `daily_sales_routes.py` | 當日業績 | `/daily_sales`, `/daily_sales/export*` | -| `market_intel_routes.py` | 市場情報 Phase 54 manual sample candidate queue transaction | `/market_intel`, `/market_intel/*`, `/api/market_intel/status`, `/api/market_intel/schema`, `/api/market_intel/schema_smoke`, `/api/market_intel/schema_db_probe`, `/api/market_intel/platform_seed_db_diff`, `/api/market_intel/legacy_source_bridge`, `/api/market_intel/mcp_readiness`, `/api/market_intel/mcp_tool_contract`, `/api/market_intel/mcp_deploy_preflight`, `/api/market_intel/mcp_activation_runbook`, `/api/market_intel/mcp_fetch_gate`, `/api/market_intel/scheduler_plan`, `/api/market_intel/manual_sample_plan`, `/api/market_intel/manual_sample_acceptance`, `/api/market_intel/manual_sample_review`, `/api/market_intel/manual_sample_review/evaluate`, `/api/market_intel/manual_sample_review/candidate_handoff`, `/api/market_intel/manual_sample_review/candidate_queue_draft`, `/api/market_intel/manual_sample_review/candidate_queue_approval`, `/api/market_intel/manual_sample_review/candidate_queue_transaction`, `/api/market_intel/match_review_plan`, `/api/market_intel/opportunity_plan`, `/api/market_intel/opportunity_scoring_plan`, `/api/market_intel/opportunity_evidence_plan`, `/api/market_intel/opportunity_alert_plan`, `/api/market_intel/adapters`, `/api/market_intel/dry_run_plan`, `/api/market_intel/discovery_plan`, `/api/market_intel/manual_discovery`, `/api/market_intel/candidate_preview`, `/api/market_intel/platform_seed_plan`, `/api/market_intel/platform_seed_write_guard`, `/api/market_intel/platform_seed_writer_plan`, `/api/market_intel/migration_blueprint`, `/api/market_intel/migration_apply_drill`, `/api/market_intel/migration_catalog_review`, `/api/market_intel/migration_live_smoke`, `/api/market_intel/live_db_inventory`, `/api/market_intel/seed_writer_cli_status`, `/api/market_intel/write_approval_runbook`, `/api/market_intel/deployment_readiness` | +| `market_intel_routes.py` | 市場情報 Phase 55 candidate queue writer CLI gate | `/market_intel`, `/market_intel/*`, `/api/market_intel/status`, `/api/market_intel/schema`, `/api/market_intel/schema_smoke`, `/api/market_intel/schema_db_probe`, `/api/market_intel/platform_seed_db_diff`, `/api/market_intel/legacy_source_bridge`, `/api/market_intel/mcp_readiness`, `/api/market_intel/mcp_tool_contract`, `/api/market_intel/mcp_deploy_preflight`, `/api/market_intel/mcp_activation_runbook`, `/api/market_intel/mcp_fetch_gate`, `/api/market_intel/scheduler_plan`, `/api/market_intel/manual_sample_plan`, `/api/market_intel/manual_sample_acceptance`, `/api/market_intel/manual_sample_review`, `/api/market_intel/manual_sample_review/evaluate`, `/api/market_intel/manual_sample_review/candidate_handoff`, `/api/market_intel/manual_sample_review/candidate_queue_draft`, `/api/market_intel/manual_sample_review/candidate_queue_approval`, `/api/market_intel/manual_sample_review/candidate_queue_transaction`, `/api/market_intel/manual_sample_review/candidate_queue_writer_status`, `/api/market_intel/match_review_plan`, `/api/market_intel/opportunity_plan`, `/api/market_intel/opportunity_scoring_plan`, `/api/market_intel/opportunity_evidence_plan`, `/api/market_intel/opportunity_alert_plan`, `/api/market_intel/adapters`, `/api/market_intel/dry_run_plan`, `/api/market_intel/discovery_plan`, `/api/market_intel/manual_discovery`, `/api/market_intel/candidate_preview`, `/api/market_intel/platform_seed_plan`, `/api/market_intel/platform_seed_write_guard`, `/api/market_intel/platform_seed_writer_plan`, `/api/market_intel/migration_blueprint`, `/api/market_intel/migration_apply_drill`, `/api/market_intel/migration_catalog_review`, `/api/market_intel/migration_live_smoke`, `/api/market_intel/live_db_inventory`, `/api/market_intel/seed_writer_cli_status`, `/api/market_intel/write_approval_runbook`, `/api/market_intel/deployment_readiness` | | `api_routes.py` | 通用任務與查詢 API | `/api/run_task`, `/api/history/*` | | `export_routes.py` | 匯出功能 | `/api/export/*` | | `import_routes.py` | 匯入功能 | `/api/import_excel`, `/api/import/monthly_summary` | diff --git a/routes/market_intel_routes.py b/routes/market_intel_routes.py index 17d7d2e..0673b5e 100644 --- a/routes/market_intel_routes.py +++ b/routes/market_intel_routes.py @@ -9,6 +9,9 @@ from flask import Blueprint, jsonify, render_template, request from auth import login_required from config import SYSTEM_VERSION from services.market_intel import MarketIntelService +from services.market_intel.candidate_queue_writer_cli import ( + build_candidate_queue_writer_cli_plan, +) TAIPEI_TZ = timezone(timedelta(hours=8)) @@ -292,6 +295,41 @@ def market_intel_manual_sample_candidate_queue_transaction(): ) +@market_intel_bp.route( + "/api/market_intel/manual_sample_review/candidate_queue_writer_status", + methods=["POST"], +) +@login_required +def market_intel_manual_sample_candidate_queue_writer_status(): + payload = request.get_json(silent=True) + limit = request.args.get("limit", default=20, type=int) + execute_requested = request.args.get("execute", "false").lower() == "true" + apply_real_write = request.args.get("apply_real_write", "false").lower() == "true" + service = _service() + payload_error = None + if not isinstance(payload, dict): + sample_result = None + payload_error = "invalid_json_object" + else: + sample_result = payload.get("sample_result", payload) + + transaction_preview = service.build_manual_sample_candidate_queue_transaction( + sample_result=sample_result, + payload_error=payload_error, + limit=limit, + ) + data = build_candidate_queue_writer_cli_plan( + transaction_preview=transaction_preview, + execute_requested=execute_requested, + apply_real_write=apply_real_write, + approval_token=None, + approval_token_secret=None, + ) + data["phase"] = service.phase + status_code = 400 if payload_error else 200 + return jsonify(data), status_code + + @market_intel_bp.route("/api/market_intel/match_review_plan") @login_required def market_intel_match_review_plan(): diff --git a/scripts/market_intel_candidate_queue_writer.py b/scripts/market_intel_candidate_queue_writer.py new file mode 100755 index 0000000..3a3b977 --- /dev/null +++ b/scripts/market_intel_candidate_queue_writer.py @@ -0,0 +1,91 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +"""Market intelligence candidate queue writer CLI gate. + +This script prints a JSON gate status. Real queue writes are intentionally +disabled in this phase even when --execute and --apply-real-write are provided. +""" + +import argparse +import contextlib +import json +import os +import sys +from pathlib import Path + + +REPO_ROOT = Path(__file__).resolve().parents[1] +if str(REPO_ROOT) not in sys.path: + sys.path.insert(0, str(REPO_ROOT)) + +with contextlib.redirect_stdout(sys.stderr): + from services.market_intel import MarketIntelService # noqa: E402 + from services.market_intel.candidate_queue_writer_cli import ( # noqa: E402 + APPROVAL_ENV_VAR, + build_candidate_queue_writer_cli_plan, + ) + + +def parse_args(argv=None): + parser = argparse.ArgumentParser( + description="Preview candidate queue writer execution gates." + ) + parser.add_argument( + "--sample-json", + default=None, + help="Path to one manual sample result JSON file. Default: empty payload.", + ) + parser.add_argument( + "--execute", + action="store_true", + help="Request real execution. Still blocked in this phase.", + ) + parser.add_argument( + "--apply-real-write", + action="store_true", + help="Request the guarded queue write transaction. Still blocked in this phase.", + ) + parser.add_argument( + "--approval-token", + default=None, + help=f"One-time approval token checked against {APPROVAL_ENV_VAR}.", + ) + return parser.parse_args(argv) + + +def _load_sample_result(path): + if not path: + return {} + with open(path, encoding="utf-8") as handle: + payload = json.load(handle) + return payload.get("sample_result", payload) if isinstance(payload, dict) else payload + + +def main(argv=None): + args = parse_args(argv) + sample_result = _load_sample_result(args.sample_json) + if not isinstance(sample_result, dict): + sample_result = None + payload_error = "invalid_json_object" + else: + payload_error = None + + service = MarketIntelService() + transaction_preview = service.build_manual_sample_candidate_queue_transaction( + sample_result=sample_result, + payload_error=payload_error, + ) + plan = build_candidate_queue_writer_cli_plan( + transaction_preview=transaction_preview, + execute_requested=args.execute, + apply_real_write=args.apply_real_write, + approval_token=args.approval_token, + approval_token_secret=os.getenv(APPROVAL_ENV_VAR), + ) + plan["phase"] = service.phase + print(json.dumps(plan, ensure_ascii=False, indent=2, sort_keys=True)) + return int(plan.get("exit_code", 2)) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/services/market_intel/candidate_queue_writer_cli.py b/services/market_intel/candidate_queue_writer_cli.py new file mode 100644 index 0000000..1b9e686 --- /dev/null +++ b/services/market_intel/candidate_queue_writer_cli.py @@ -0,0 +1,176 @@ +"""市場情報候選審核 queue writer CLI gate。 + +本模組只建立正式 queue writer 前的安全 gate 狀態;不開 DB connection、 +不開 transaction、不寫 market_alert_review_queue、不掛 scheduler。 +""" + +import hmac +import os + + +APPROVAL_ENV_VAR = "MARKET_INTEL_QUEUE_WRITE_APPROVAL" +MIN_APPROVAL_TOKEN_LENGTH = 16 + + +def _approval_token_valid(approval_token, approval_token_secret): + if not approval_token or not approval_token_secret: + return False + if len(str(approval_token_secret)) < MIN_APPROVAL_TOKEN_LENGTH: + return False + return hmac.compare_digest(str(approval_token), str(approval_token_secret)) + + +def build_candidate_queue_writer_cli_plan( + *, + transaction_preview, + execute_requested=False, + approval_token=None, + approval_token_secret=None, + apply_real_write=False, +): + """建立候選審核 queue writer CLI gate;本階段拒絕實際寫入。""" + approval_token_present = bool(approval_token) + approval_token_secret = approval_token_secret or os.getenv(APPROVAL_ENV_VAR) + approval_token_secret_configured = bool(approval_token_secret) + approval_token_valid = _approval_token_valid(approval_token, approval_token_secret) + summary = transaction_preview.get("transaction_summary", {}) + statement_count = int(summary.get("statement_count") or 0) + transaction_preview_created = bool( + transaction_preview.get("transaction_preview_created") + ) + writer_enabled = False + gates = [ + { + "key": "script_created", + "label": "scripts/market_intel_candidate_queue_writer.py exists", + "passed": True, + }, + { + "key": "transaction_preview_created", + "label": "候選 queue transaction preview 已建立", + "passed": transaction_preview_created, + }, + { + "key": "transaction_has_statements", + "label": "transaction preview 至少包含一筆 statement", + "passed": statement_count > 0, + }, + { + "key": "execute_requested", + "label": "--execute flag was explicitly provided", + "passed": bool(execute_requested), + }, + { + "key": "approval_token_present", + "label": f"{APPROVAL_ENV_VAR} approval token was provided", + "passed": approval_token_present, + }, + { + "key": "approval_token_secret_configured", + "label": f"{APPROVAL_ENV_VAR} environment token is configured", + "passed": approval_token_secret_configured, + }, + { + "key": "approval_token_valid", + "label": "approval token matches the configured environment token", + "passed": approval_token_valid, + }, + { + "key": "apply_real_write_requested", + "label": "--apply-real-write flag was explicitly provided", + "passed": bool(apply_real_write), + }, + { + "key": "backup_verified", + "label": "正式寫入前必須確認最新備份已完成", + "passed": False, + }, + { + "key": "migration_live_smoke_passed", + "label": "正式 schema live smoke 必須通過", + "passed": False, + }, + { + "key": "queue_writer_implementation_enabled", + "label": "候選 queue writer 實際寫入實作仍未啟用", + "passed": writer_enabled, + }, + { + "key": "manual_operator_approval", + "label": "操作者需在 CLI 明確批准一次性寫入", + "passed": bool(execute_requested and apply_real_write and approval_token_valid), + }, + { + "key": "crawler_stays_disabled", + "label": "queue writer 不掛 crawler 或 scheduler", + "passed": True, + }, + ] + blocked_reasons = [gate["key"] for gate in gates if not gate["passed"]] + blocked_reasons.append("candidate_queue_writer_execution_not_enabled") + + return { + "mode": "candidate_queue_writer_cli_blocked", + "target_table": "market_alert_review_queue", + "execute_requested": bool(execute_requested), + "apply_real_write_requested": bool(apply_real_write), + "approval_token_present": approval_token_present, + "approval_token_valid": approval_token_valid, + "approval_env_var": APPROVAL_ENV_VAR, + "approval_token_secret_configured": approval_token_secret_configured, + "queue_writer_implementation_enabled": writer_enabled, + "ready_for_real_write": False, + "writes_executed": False, + "would_write_database": False, + "database_connection_opened": False, + "database_session_created": False, + "explicit_transaction_opened": False, + "database_write_executed": False, + "database_commit_executed": False, + "database_rollback_executed": False, + "external_network_executed": False, + "scheduler_attached": False, + "exit_code": 2 if execute_requested else 0, + "blocked_reasons": blocked_reasons, + "approval_gates": gates, + "transaction_preview_summary": { + "mode": transaction_preview.get("mode"), + "transaction_preview_created": transaction_preview_created, + "transaction_ready": bool(transaction_preview.get("transaction_ready")), + "transaction_opened": bool(transaction_preview.get("transaction_opened")), + "transaction_committed": bool( + transaction_preview.get("transaction_committed") + ), + "statement_count": statement_count, + "idempotency_key_count": int(summary.get("idempotency_key_count") or 0), + "conflict_policy": summary.get("conflict_policy"), + }, + "transaction_preview": transaction_preview, + "rollback_plan": [ + { + "key": "no_write_no_db_rollback_required", + "label": "本階段不寫 DB;若被阻擋,不需要 DB rollback", + }, + { + "key": "future_dedupe_key_cleanup_review", + "label": "未來正式寫入若需回退,必須依 dedupe_key 人工審核清理", + }, + ], + "safety_contract": { + "refuses_api_execution": True, + "refuses_execute_without_apply_flag": True, + "requires_independent_approval_token": True, + "does_not_open_transaction_from_status": True, + "does_not_commit_from_status": True, + "keeps_crawler_disabled_for_queue_write": True, + "target_table": "market_alert_review_queue", + }, + "safe_boundaries": [ + "do_not_execute_candidate_queue_writer_from_api", + "do_not_open_database_connection_from_queue_writer_status", + "do_not_commit_queue_writer_status", + "do_not_attach_scheduler_from_queue_writer", + "no_remove_orphans", + "no_momo_db_lifecycle_change", + ], + } diff --git a/services/market_intel/deployment_readiness.py b/services/market_intel/deployment_readiness.py index 25c1832..db73144 100644 --- a/services/market_intel/deployment_readiness.py +++ b/services/market_intel/deployment_readiness.py @@ -3,6 +3,10 @@ 本模組只組裝 preview payload,不執行 git、部署、SSH、migration 或 DB write。 """ +from services.market_intel.candidate_queue_writer_cli import ( + build_candidate_queue_writer_cli_plan, +) + def build_deployment_readiness_preview( *, @@ -36,6 +40,9 @@ def build_deployment_readiness_preview( manual_sample_candidate_queue_transaction = ( service.build_manual_sample_candidate_queue_transaction(sample_result={}) ) + candidate_queue_writer_cli_status = build_candidate_queue_writer_cli_plan( + transaction_preview=manual_sample_candidate_queue_transaction, + ) match_review_plan = service.build_match_review_plan() opportunity_plan = service.build_opportunity_plan() opportunity_scoring_plan = service.build_opportunity_scoring_plan() @@ -191,6 +198,18 @@ def build_deployment_readiness_preview( and not manual_sample_candidate_queue_transaction["database_commit_executed"] and not manual_sample_candidate_queue_transaction["scheduler_attached"] ), + "candidate_queue_writer_cli_status_safe": bool( + candidate_queue_writer_cli_status["mode"] + == "candidate_queue_writer_cli_blocked" + and not candidate_queue_writer_cli_status["ready_for_real_write"] + and not candidate_queue_writer_cli_status["writes_executed"] + and not candidate_queue_writer_cli_status["would_write_database"] + and not candidate_queue_writer_cli_status["database_connection_opened"] + and not candidate_queue_writer_cli_status["explicit_transaction_opened"] + and not candidate_queue_writer_cli_status["database_write_executed"] + and not candidate_queue_writer_cli_status["database_commit_executed"] + and not candidate_queue_writer_cli_status["scheduler_attached"] + ), "match_review_plan_preview_safe": bool( match_review_plan["mode"] == "match_review_plan_preview" and not match_review_plan["review_queue_created"] @@ -441,6 +460,7 @@ def build_deployment_readiness_preview( "manual_sample_candidate_queue_transaction": ( manual_sample_candidate_queue_transaction ), + "candidate_queue_writer_cli_status": candidate_queue_writer_cli_status, "match_review_plan": match_review_plan, "opportunity_plan": opportunity_plan, "opportunity_scoring_plan": opportunity_scoring_plan, diff --git a/services/market_intel/service.py b/services/market_intel/service.py index 0a5b10c..800d339 100644 --- a/services/market_intel/service.py +++ b/services/market_intel/service.py @@ -108,7 +108,7 @@ class MarketIntelRuntimeStatus: class MarketIntelService: """市場情報入口服務,先集中 feature gate 與安全狀態。""" - phase = "phase_54_manual_sample_candidate_queue_transaction" + phase = "phase_55_candidate_queue_writer_cli_gate" def get_runtime_status(self) -> MarketIntelRuntimeStatus: return MarketIntelRuntimeStatus( diff --git a/templates/market_intel/disabled.html b/templates/market_intel/disabled.html index 230f0e7..d820f0f 100644 --- a/templates/market_intel/disabled.html +++ b/templates/market_intel/disabled.html @@ -251,7 +251,7 @@ align-items: stretch; display: grid; gap: 0.75rem; - grid-template-columns: minmax(0, 1fr) auto auto auto auto auto; + grid-template-columns: minmax(0, 1fr) auto auto auto auto auto auto; min-width: 0; } @@ -588,6 +588,9 @@ + @@ -916,12 +919,14 @@ const sampleCandidateQueueDraft = sampleReviewRoot ? sampleReviewRoot.querySelector('[data-market-intel-sample-candidate-queue-draft]') : null; const sampleCandidateQueueApproval = sampleReviewRoot ? sampleReviewRoot.querySelector('[data-market-intel-sample-candidate-queue-approval]') : null; const sampleCandidateQueueTransaction = sampleReviewRoot ? sampleReviewRoot.querySelector('[data-market-intel-sample-candidate-queue-transaction]') : null; + const sampleCandidateQueueWriter = sampleReviewRoot ? sampleReviewRoot.querySelector('[data-market-intel-sample-candidate-queue-writer]') : null; const sampleReviewEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_review') }}"; const sampleReviewEvaluateEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_review_evaluate') }}"; const sampleCandidateHandoffEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_handoff') }}"; const sampleCandidateQueueDraftEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_queue_draft') }}"; const sampleCandidateQueueApprovalEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_queue_approval') }}"; const sampleCandidateQueueTransactionEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_queue_transaction') }}"; + const sampleCandidateQueueWriterEndpoint = "{{ url_for('market_intel.market_intel_manual_sample_candidate_queue_writer_status') }}"; const schedulerMeta = schedulerRoot ? schedulerRoot.querySelector('[data-market-intel-scheduler-meta]') : null; const schedulerBody = schedulerRoot ? schedulerRoot.querySelector('[data-market-intel-scheduler-body]') : null; const schedulerRefresh = schedulerRoot ? schedulerRoot.querySelector('[data-market-intel-scheduler-refresh]') : null; @@ -2260,6 +2265,64 @@ } }; + const renderCandidateQueueWriter = data => { + const blockers = (data.blocked_reasons || []).join(' / '); + const summary = data.transaction_preview_summary || {}; + const gates = data.approval_gates || []; + sampleReviewMeta.innerHTML = [ + `mode=${data.mode || 'unknown'}`, + `ready=${data.ready_for_real_write ? 'yes' : 'no'}`, + `statements=${summary.statement_count || 0}`, + `execute=${data.execute_requested ? 'yes' : 'no'}`, + `apply=${data.apply_real_write_requested ? 'yes' : 'no'}` + ].map(item => `${escapeHtml(item)}`).join(''); + sampleReviewBody.innerHTML = ` +
此卡只檢查 CLI writer gate;不讀取 token、不開 DB connection、不寫 queue。${blockers ? `阻擋:${escapeHtml(blockers)}` : ''}
+
${ + gates.map(gate => ` +
+ ${gate.passed ? 'PASS' : 'BLOCK'} +
+ ${escapeHtml(gate.key)} + ${escapeHtml(gate.label)} +
+
+ `).join('') + }
+ `; + }; + + const loadCandidateQueueWriter = async () => { + if (!sampleReviewMeta || !sampleReviewBody || !sampleReviewInput) return; + let parsed; + try { + parsed = JSON.parse(sampleReviewInput.value || '{}'); + } catch (error) { + sampleReviewMeta.innerHTML = 'json_error'; + sampleReviewBody.innerHTML = `
JSON 格式錯誤:${escapeHtml(error.message)}
`; + return; + } + + sampleReviewBody.innerHTML = '
檢查 queue writer CLI gate 中...
'; + try { + const response = await fetch(sampleCandidateQueueWriterEndpoint, { + method: 'POST', + credentials: 'same-origin', + headers: { + 'Content-Type': 'application/json', + 'X-CSRFToken': csrfToken + }, + body: JSON.stringify({ sample_result: parsed }) + }); + const data = await response.json(); + if (!response.ok && !data.mode) throw new Error(`HTTP ${response.status}`); + renderCandidateQueueWriter(data); + } catch (error) { + sampleReviewMeta.innerHTML = 'error'; + sampleReviewBody.innerHTML = `
queue writer CLI gate 檢查失敗:${escapeHtml(error.message)}
`; + } + }; + const renderSchedulerMeta = data => { schedulerMeta.innerHTML = [ `mode=${data.mode || 'unknown'}`, @@ -3766,6 +3829,9 @@ if (sampleCandidateQueueTransaction) { sampleCandidateQueueTransaction.addEventListener('click', loadCandidateQueueTransaction); } + if (sampleCandidateQueueWriter) { + sampleCandidateQueueWriter.addEventListener('click', loadCandidateQueueWriter); + } if (schedulerRefresh) { schedulerRefresh.addEventListener('click', loadScheduler); } diff --git a/tests/test_market_intel_skeleton.py b/tests/test_market_intel_skeleton.py index 77b3266..109caa5 100644 --- a/tests/test_market_intel_skeleton.py +++ b/tests/test_market_intel_skeleton.py @@ -522,6 +522,7 @@ def test_market_intel_preview_template_uses_safe_fetch_false_endpoint(): assert "data-market-intel-sample-candidate-queue-draft" in template assert "data-market-intel-sample-candidate-queue-approval" in template assert "data-market-intel-sample-candidate-queue-transaction" in template + assert "data-market-intel-sample-candidate-queue-writer" in template assert "data-market-intel-approval" in template assert "data-market-intel-approval-gates" in template assert "data-market-intel-deploy" in template @@ -545,6 +546,7 @@ def test_market_intel_preview_template_uses_safe_fetch_false_endpoint(): assert "market_intel.market_intel_manual_sample_candidate_queue_draft" in template assert "market_intel.market_intel_manual_sample_candidate_queue_approval" in template assert "market_intel.market_intel_manual_sample_candidate_queue_transaction" in template + assert "market_intel.market_intel_manual_sample_candidate_queue_writer_status" in template assert "X-CSRFToken" in template assert "market_intel.market_intel_scheduler_plan" in template assert "market_intel.market_intel_match_review_plan" in template @@ -581,7 +583,7 @@ def test_legacy_source_bridge_default_is_planned_only(): bridge = MarketIntelService().build_legacy_source_bridge() assert bridge["mode"] == "legacy_source_bridge_planned" - assert bridge["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert bridge["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert bridge["execute_requested"] is False assert bridge["read_only_query_executed"] is False assert bridge["database_connection_opened"] is False @@ -739,7 +741,7 @@ def test_mcp_tool_contract_preview_is_read_only_and_whitelisted(): contract = MarketIntelService().build_mcp_tool_contract() assert contract["mode"] == "mcp_tool_contract_preview" - assert contract["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert contract["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert contract["caller"] == "market_intel" assert contract["contract_ready"] is True assert contract["blocked_reasons"] == [] @@ -872,7 +874,7 @@ def test_mcp_activation_runbook_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "mcp_activation_runbook_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["deployment_actions_executed"] is False assert data["docker_command_executed"] is False assert data["ssh_command_executed"] is False @@ -885,7 +887,7 @@ def test_mcp_fetch_gate_default_blocks_external_fetch(): gate = MarketIntelService().build_mcp_fetch_gate(fetch_requested=True) assert gate["mode"] == "mcp_fetch_gate_planned" - assert gate["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert gate["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert gate["fetch_requested"] is True assert gate["manual_fetch_gate_open"] is False assert gate["network_request_allowed"] is False @@ -955,7 +957,7 @@ def test_mcp_fetch_gate_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "mcp_fetch_gate_planned" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["fetch_requested"] is False assert data["network_request_allowed"] is False assert data["external_network_executed"] is False @@ -967,7 +969,7 @@ def test_manual_sample_plan_preview_blocks_fetch_and_write(): plan = MarketIntelService().build_manual_sample_plan() assert plan["mode"] == "manual_sample_fetch_plan_preview" - assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert plan["ready_for_manual_sample_fetch"] is False assert plan["sample_fetch_executed"] is False assert plan["external_network_executed"] is False @@ -1015,7 +1017,7 @@ def test_manual_sample_plan_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "manual_sample_fetch_plan_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["sample_fetch_executed"] is False assert data["external_network_executed"] is False assert data["database_write_executed"] is False @@ -1026,7 +1028,7 @@ def test_manual_sample_acceptance_preview_blocks_candidate_import(): acceptance = MarketIntelService().build_manual_sample_acceptance() assert acceptance["mode"] == "manual_sample_acceptance_preview" - assert acceptance["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert acceptance["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert acceptance["contract_ready"] is True assert acceptance["sample_result_loaded"] is False assert acceptance["sample_result_accepted"] is False @@ -1068,7 +1070,7 @@ def test_manual_sample_acceptance_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "manual_sample_acceptance_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["sample_result_loaded"] is False assert data["candidate_import_allowed"] is False assert data["external_network_executed"] is False @@ -1080,7 +1082,7 @@ def test_manual_sample_review_preview_is_planned_until_result_loaded(): review = MarketIntelService().build_manual_sample_review() assert review["mode"] == "manual_sample_review_preview" - assert review["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert review["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert review["contract_ready"] is True assert review["sample_result_loaded"] is False assert review["sample_result_reviewed"] is False @@ -1191,7 +1193,7 @@ def test_manual_sample_review_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "manual_sample_review_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["sample_result_loaded"] is False assert data["sample_result_reviewed"] is False assert data["candidate_import_allowed"] is False @@ -1230,7 +1232,7 @@ def test_manual_sample_review_evaluation_preview_accepts_payload_without_persist ) assert review["mode"] == "manual_sample_review_evaluation_preview" - assert review["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert review["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert review["review_request_type"] == "operator_posted_json" assert review["payload_received"] is True assert review["payload_valid_json_object"] is True @@ -1292,7 +1294,7 @@ def test_manual_sample_review_evaluate_route_is_post_only_and_no_write(): assert response.status_code == 200 assert data["mode"] == "manual_sample_review_evaluation_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["payload_received"] is True assert data["payload_valid_json_object"] is True assert data["payload_persisted"] is False @@ -1372,7 +1374,7 @@ def test_manual_sample_candidate_handoff_preview_creates_candidates_without_pers ) assert handoff["mode"] == "manual_sample_candidate_handoff_preview" - assert handoff["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert handoff["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert handoff["payload_received"] is True assert handoff["payload_valid_json_object"] is True assert handoff["payload_persisted"] is False @@ -1436,7 +1438,7 @@ def test_manual_sample_candidate_handoff_route_is_post_only_and_no_write(): assert response.status_code == 200 assert data["mode"] == "manual_sample_candidate_handoff_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["payload_received"] is True assert data["handoff_ready"] is True assert data["candidate_handoff_created"] is True @@ -1495,7 +1497,7 @@ def test_manual_sample_candidate_queue_draft_preview_builds_review_items_without ) assert queue_draft["mode"] == "manual_sample_candidate_queue_draft_preview" - assert queue_draft["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert queue_draft["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert queue_draft["payload_received"] is True assert queue_draft["payload_valid_json_object"] is True assert queue_draft["payload_persisted"] is False @@ -1569,7 +1571,7 @@ def test_manual_sample_candidate_queue_draft_route_is_post_only_and_no_write(): assert response.status_code == 200 assert data["mode"] == "manual_sample_candidate_queue_draft_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["payload_received"] is True assert data["handoff_ready"] is True assert data["queue_draft_ready"] is True @@ -1632,7 +1634,7 @@ def test_manual_sample_candidate_queue_approval_preview_blocks_write_and_maps_ro ) assert approval["mode"] == "manual_sample_candidate_queue_approval_preview" - assert approval["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert approval["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert approval["payload_received"] is True assert approval["payload_valid_json_object"] is True assert approval["payload_persisted"] is False @@ -1710,7 +1712,7 @@ def test_manual_sample_candidate_queue_approval_route_is_post_only_and_no_write( assert response.status_code == 200 assert data["mode"] == "manual_sample_candidate_queue_approval_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["payload_received"] is True assert data["approval_preview_created"] is True assert data["approval_request_created"] is False @@ -1773,7 +1775,7 @@ def test_manual_sample_candidate_queue_transaction_preview_blocks_execution(): ) assert transaction["mode"] == "manual_sample_candidate_queue_transaction_preview" - assert transaction["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert transaction["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert transaction["payload_received"] is True assert transaction["payload_valid_json_object"] is True assert transaction["payload_persisted"] is False @@ -1851,7 +1853,7 @@ def test_manual_sample_candidate_queue_transaction_route_is_post_only_and_no_wri assert response.status_code == 200 assert data["mode"] == "manual_sample_candidate_queue_transaction_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["payload_received"] is True assert data["transaction_preview_created"] is True assert data["transaction_ready"] is False @@ -1884,11 +1886,169 @@ def test_manual_sample_candidate_queue_transaction_blocks_invalid_payload(): assert "queue_transaction_preview_not_ready" in transaction["blocked_reasons"] +def test_candidate_queue_writer_cli_gate_blocks_real_write_even_with_token(): + from services.market_intel.candidate_queue_writer_cli import ( + build_candidate_queue_writer_cli_plan, + ) + + sample_result = { + "batch_id": "sample-batch-13", + "platform_code": "momo", + "source_key": "homepage", + "source_url": "https://www.momoshop.com.tw/", + "status": "fetched", + "status_code": 200, + "content_length": 1700, + "page_hash": "a" * 64, + "title": "MOMO 活動", + "diagnostics": { + "link_count": 1, + "same_host_link_count": 1, + "campaign_link_candidates": [ + { + "confidence_band": "high", + "score": 94, + "url": "https://www.momoshop.com.tw/activity/sample", + "text": "品牌活動", + }, + ], + }, + } + transaction = MarketIntelService().build_manual_sample_candidate_queue_transaction( + sample_result=sample_result + ) + status = build_candidate_queue_writer_cli_plan( + transaction_preview=transaction, + execute_requested=True, + apply_real_write=True, + approval_token=TEST_APPROVAL_TOKEN, + approval_token_secret=TEST_APPROVAL_TOKEN, + ) + + assert status["mode"] == "candidate_queue_writer_cli_blocked" + assert status["execute_requested"] is True + assert status["apply_real_write_requested"] is True + assert status["approval_token_present"] is True + assert status["approval_token_valid"] is True + assert status["queue_writer_implementation_enabled"] is False + assert status["ready_for_real_write"] is False + assert status["writes_executed"] is False + assert status["would_write_database"] is False + assert status["database_connection_opened"] is False + assert status["database_session_created"] is False + assert status["explicit_transaction_opened"] is False + assert status["database_write_executed"] is False + assert status["database_commit_executed"] is False + assert status["scheduler_attached"] is False + assert status["exit_code"] == 2 + assert status["transaction_preview_summary"]["statement_count"] == 1 + assert "approval_token_valid" not in status["blocked_reasons"] + assert "queue_writer_implementation_enabled" in status["blocked_reasons"] + assert "candidate_queue_writer_execution_not_enabled" in status["blocked_reasons"] + assert status["safety_contract"]["refuses_api_execution"] is True + assert ( + "do_not_execute_candidate_queue_writer_from_api" + in status["safe_boundaries"] + ) + + +def test_candidate_queue_writer_status_route_never_leaks_approval_token(monkeypatch): + from routes.market_intel_routes import market_intel_bp + + monkeypatch.setenv("MARKET_INTEL_QUEUE_WRITE_APPROVAL", TEST_APPROVAL_TOKEN) + + app = Flask(__name__) + app.secret_key = "test-secret" + app.register_blueprint(market_intel_bp) + client = app.test_client() + with client.session_transaction() as session: + session["logged_in"] = True + + response = client.post( + "/api/market_intel/manual_sample_review/candidate_queue_writer_status" + "?execute=true&apply_real_write=true", + json={ + "sample_result": { + "batch_id": "sample-batch-14", + "platform_code": "pchome", + "source_key": "homepage", + "source_url": "https://24h.pchome.com.tw/", + "status": "fetched", + "status_code": 200, + "content_length": 1800, + "page_hash": "b" * 64, + "title": "PChome 活動", + "diagnostics": { + "link_count": 1, + "same_host_link_count": 1, + "campaign_link_candidates": [ + { + "confidence_band": "medium", + "score": 74, + "url": "https://24h.pchome.com.tw/activity/sample", + "text": "活動頁", + }, + ], + }, + } + }, + ) + data = response.get_json() + payload = json.dumps(data, ensure_ascii=False, sort_keys=True) + + assert response.status_code == 200 + assert data["mode"] == "candidate_queue_writer_cli_blocked" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" + assert data["execute_requested"] is True + assert data["apply_real_write_requested"] is True + assert data["approval_token_present"] is False + assert data["approval_token_valid"] is False + assert data["approval_token_secret_configured"] is True + assert data["ready_for_real_write"] is False + assert data["writes_executed"] is False + assert data["would_write_database"] is False + assert data["database_connection_opened"] is False + assert data["database_write_executed"] is False + assert data["database_commit_executed"] is False + assert data["scheduler_attached"] is False + assert data["transaction_preview_summary"]["statement_count"] == 1 + assert "approval_token_present" in data["blocked_reasons"] + assert "approval_token_valid" in data["blocked_reasons"] + assert "approval_token_hint" not in payload + assert TEST_APPROVAL_TOKEN not in payload + assert "APPROVED_MARKET_INTEL_QUEUE_WRITE" not in payload + + +def test_candidate_queue_writer_status_blocks_invalid_payload(): + from routes.market_intel_routes import market_intel_bp + + app = Flask(__name__) + app.secret_key = "test-secret" + app.register_blueprint(market_intel_bp) + client = app.test_client() + with client.session_transaction() as session: + session["logged_in"] = True + + response = client.post( + "/api/market_intel/manual_sample_review/candidate_queue_writer_status", + data="not-json", + content_type="application/json", + ) + data = response.get_json() + + assert response.status_code == 400 + assert data["mode"] == "candidate_queue_writer_cli_blocked" + assert data["ready_for_real_write"] is False + assert data["writes_executed"] is False + assert data["transaction_preview_summary"]["statement_count"] == 0 + assert "transaction_preview_created" in data["blocked_reasons"] + + def test_scheduler_plan_preview_blocks_job_attachment(): plan = MarketIntelService().build_scheduler_plan() assert plan["mode"] == "scheduler_attach_plan_preview" - assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert plan["ready_to_attach_scheduler"] is False assert plan["scheduler_attached"] is False assert plan["scheduler_registration_executed"] is False @@ -1926,7 +2086,7 @@ def test_scheduler_plan_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "scheduler_attach_plan_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["scheduler_registration_executed"] is False assert data["crawler_job_started"] is False assert data["external_network_executed"] is False @@ -1937,7 +2097,7 @@ def test_match_review_plan_preview_blocks_auto_confirm(): plan = MarketIntelService().build_match_review_plan() assert plan["mode"] == "match_review_plan_preview" - assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert plan["ready_for_review_queue"] is False assert plan["review_queue_created"] is False assert plan["auto_match_executed"] is False @@ -1973,7 +2133,7 @@ def test_match_review_plan_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "match_review_plan_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["review_queue_created"] is False assert data["auto_confirm_executed"] is False assert data["external_network_executed"] is False @@ -1984,7 +2144,7 @@ def test_opportunity_plan_preview_blocks_alerts_and_ai_summary(): plan = MarketIntelService().build_opportunity_plan() assert plan["mode"] == "opportunity_plan_preview" - assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert plan["ready_for_opportunity_queue"] is False assert plan["opportunity_queue_created"] is False assert plan["threat_alert_dispatched"] is False @@ -2025,7 +2185,7 @@ def test_opportunity_plan_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "opportunity_plan_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["opportunity_queue_created"] is False assert data["threat_alert_dispatched"] is False assert data["ai_summary_generated"] is False @@ -2036,7 +2196,7 @@ def test_opportunity_scoring_plan_preview_blocks_scoring_and_alerts(): plan = MarketIntelService().build_opportunity_scoring_plan() assert plan["mode"] == "opportunity_scoring_plan_preview" - assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert plan["ready_for_scoring_job"] is False assert plan["scoring_job_created"] is False assert plan["score_calculation_executed"] is False @@ -2084,7 +2244,7 @@ def test_opportunity_scoring_plan_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "opportunity_scoring_plan_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["scoring_job_created"] is False assert data["score_calculation_executed"] is False assert data["sample_scores_generated"] is False @@ -2096,7 +2256,7 @@ def test_opportunity_evidence_plan_preview_blocks_queries_and_alerts(): plan = MarketIntelService().build_opportunity_evidence_plan() assert plan["mode"] == "opportunity_evidence_plan_preview" - assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert plan["ready_for_evidence_bundle"] is False assert plan["evidence_bundle_created"] is False assert plan["evidence_query_executed"] is False @@ -2142,7 +2302,7 @@ def test_opportunity_evidence_plan_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "opportunity_evidence_plan_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["evidence_bundle_created"] is False assert data["evidence_query_executed"] is False assert data["sample_evidence_generated"] is False @@ -2155,7 +2315,7 @@ def test_opportunity_alert_plan_preview_blocks_dispatch_and_llm_calls(): plan = MarketIntelService().build_opportunity_alert_plan() assert plan["mode"] == "opportunity_alert_plan_preview" - assert plan["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert plan["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert plan["ready_for_alert_candidates"] is False assert plan["alert_candidate_created"] is False assert plan["alert_queue_created"] is False @@ -2240,7 +2400,7 @@ def test_opportunity_alert_plan_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "opportunity_alert_plan_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["alert_candidate_created"] is False assert data["alert_queue_created"] is False assert data["review_queue_created"] is False @@ -2318,7 +2478,7 @@ def test_mcp_deploy_preflight_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "mcp_external_deploy_preflight_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["deployment_actions_executed"] is False assert data["docker_command_executed"] is False assert data["ssh_command_executed"] is False @@ -2333,7 +2493,7 @@ def test_mcp_readiness_default_is_planned_only(monkeypatch): readiness = MarketIntelService().build_mcp_readiness() assert readiness["mode"] == "mcp_readiness_planned" - assert readiness["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert readiness["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert readiness["execute_requested"] is False assert readiness["router_enabled"] is False assert readiness["external_mcp_complete"] is False @@ -2729,6 +2889,7 @@ def test_deployment_readiness_reports_app_only_release_gate(): assert readiness["checks"]["manual_sample_candidate_queue_draft_post_safe"] is True assert readiness["checks"]["manual_sample_candidate_queue_approval_post_safe"] is True assert readiness["checks"]["manual_sample_candidate_queue_transaction_post_safe"] is True + assert readiness["checks"]["candidate_queue_writer_cli_status_safe"] is True assert readiness["checks"]["match_review_plan_preview_safe"] is True assert readiness["checks"]["opportunity_plan_preview_safe"] is True assert readiness["checks"]["opportunity_scoring_plan_preview_safe"] is True @@ -3029,6 +3190,39 @@ def test_deployment_readiness_reports_app_only_release_gate(): readiness["manual_sample_candidate_queue_transaction"]["scheduler_attached"] is False ) + assert ( + readiness["candidate_queue_writer_cli_status"]["mode"] + == "candidate_queue_writer_cli_blocked" + ) + assert ( + readiness["candidate_queue_writer_cli_status"]["ready_for_real_write"] + is False + ) + assert readiness["candidate_queue_writer_cli_status"]["writes_executed"] is False + assert ( + readiness["candidate_queue_writer_cli_status"]["would_write_database"] + is False + ) + assert ( + readiness["candidate_queue_writer_cli_status"]["database_connection_opened"] + is False + ) + assert ( + readiness["candidate_queue_writer_cli_status"]["explicit_transaction_opened"] + is False + ) + assert ( + readiness["candidate_queue_writer_cli_status"]["database_write_executed"] + is False + ) + assert ( + readiness["candidate_queue_writer_cli_status"]["database_commit_executed"] + is False + ) + assert ( + readiness["candidate_queue_writer_cli_status"]["scheduler_attached"] + is False + ) assert readiness["scheduler_plan"]["scheduler_registration_executed"] is False assert readiness["scheduler_plan"]["crawler_job_started"] is False assert readiness["scheduler_plan"]["database_write_executed"] is False @@ -3133,7 +3327,7 @@ def test_migration_apply_drill_planned_is_safe_and_manual_only(): drill = MarketIntelService().build_migration_apply_drill() assert drill["mode"] == "migration_apply_drill_preview" - assert drill["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert drill["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert drill["execute_requested"] is False assert drill["schema_state"] == "planned_no_db_probe" assert drill["drill_ready_for_operator_review"] is True @@ -3248,7 +3442,7 @@ def test_migration_apply_drill_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "migration_apply_drill_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["execute_requested"] is False assert data["migration_executed"] is False assert data["rollback_executed"] is False @@ -3260,7 +3454,7 @@ def test_migration_catalog_review_planned_is_safe_and_diagnostic(): review = MarketIntelService().build_migration_catalog_review() assert review["mode"] == "migration_catalog_review_preview" - assert review["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert review["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert review["execute_requested"] is False assert review["catalog_state"] == "planned_no_probe" assert review["seed_state"] == "planned_no_probe" @@ -3375,7 +3569,7 @@ def test_migration_catalog_review_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "migration_catalog_review_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["execute_requested"] is False assert data["catalog_state"] == "planned_no_probe" assert data["migration_executed"] is False @@ -3388,7 +3582,7 @@ def test_migration_live_smoke_planned_is_preview_only(): smoke = MarketIntelService().build_migration_live_smoke() assert smoke["mode"] == "migration_live_smoke_preview" - assert smoke["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert smoke["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert smoke["execute_requested"] is False assert smoke["smoke_result"] == "planned_no_execution" assert smoke["live_smoke_passed"] is False @@ -3450,7 +3644,7 @@ def test_migration_live_smoke_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "migration_live_smoke_preview" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["execute_requested"] is False assert data["smoke_result"] == "planned_no_execution" assert data["migration_executed"] is False @@ -3463,7 +3657,7 @@ def test_live_db_inventory_planned_is_preview_only(): inventory = MarketIntelService().build_live_db_inventory() assert inventory["mode"] == "live_db_inventory_planned" - assert inventory["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert inventory["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert inventory["execute_requested"] is False assert inventory["read_only_query_executed"] is False assert inventory["database_connection_opened"] is False @@ -3607,7 +3801,7 @@ def test_live_db_inventory_route_is_preview_only(): assert response.status_code == 200 assert data["mode"] == "live_db_inventory_planned" - assert data["phase"] == "phase_54_manual_sample_candidate_queue_transaction" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" assert data["execute_requested"] is False assert data["read_only_query_executed"] is False assert data["database_write_executed"] is False @@ -3779,3 +3973,68 @@ def test_seed_writer_cli_script_outputs_blocked_plan(): assert data["exit_code"] == 0 assert data["transaction_preview"]["statement_count"] == 4 assert data["transaction_preview"]["transaction_opened"] is False + + +def test_candidate_queue_writer_cli_script_outputs_blocked_gate(tmp_path): + sample_file = tmp_path / "sample.json" + sample_file.write_text( + json.dumps( + { + "batch_id": "sample-batch-15", + "platform_code": "momo", + "source_key": "homepage", + "source_url": "https://www.momoshop.com.tw/", + "status": "fetched", + "status_code": 200, + "content_length": 1700, + "page_hash": "c" * 64, + "title": "MOMO 活動", + "diagnostics": { + "link_count": 1, + "same_host_link_count": 1, + "campaign_link_candidates": [ + { + "confidence_band": "high", + "score": 94, + "url": "https://www.momoshop.com.tw/activity/sample", + "text": "品牌活動", + }, + ], + }, + }, + ensure_ascii=False, + ), + encoding="utf-8", + ) + env = { + **os.environ, + "MOMO_ALLOW_INSECURE_CONFIG_FOR_TESTS": "true", + "SECRET_KEY": "test", + "LOGIN_PASSWORD": "test", + } + result = subprocess.run( + [ + sys.executable, + "scripts/market_intel_candidate_queue_writer.py", + "--sample-json", + str(sample_file), + ], + capture_output=True, + check=False, + env=env, + text=True, + ) + data = json.loads(result.stdout) + + assert result.returncode == 0 + assert data["mode"] == "candidate_queue_writer_cli_blocked" + assert data["phase"] == "phase_55_candidate_queue_writer_cli_gate" + assert data["execute_requested"] is False + assert data["apply_real_write_requested"] is False + assert data["writes_executed"] is False + assert data["database_connection_opened"] is False + assert data["database_write_executed"] is False + assert data["database_commit_executed"] is False + assert data["scheduler_attached"] is False + assert data["transaction_preview_summary"]["statement_count"] == 1 + assert "execute_requested" in data["blocked_reasons"]