fe7fd7a3e0
問題: LLM 測試因模型波動導致 CI 失敗 解決方案: 三層測試策略 - Tier 1 (CI): Schema 驗證 + Golden Responses - Tier 2 (Nightly): 屬性測試 + Live LLM - Tier 3 (Weekly): 語意相似度測試 新增檔案: - ADR-018-llm-testing-strategy.md - tests/llm_testing/ 框架 - schema_validators.py: Pydantic Schema 驗證 - property_validators.py: kubectl/風險等級驗證 - golden_responses.py: 預錄回應管理 - tests/test_llm_tier1_schema.py: 35 個 Tier 1 測試 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
32 lines
769 B
Python
32 lines
769 B
Python
"""
|
|
LLM Testing Framework - ADR-018 Implementation
|
|
===============================================
|
|
|
|
三層 LLM 測試策略:
|
|
- Tier 1: Schema 驗證 (CI)
|
|
- Tier 2: 屬性測試 (Nightly)
|
|
- Tier 3: 語意品質 (Weekly)
|
|
|
|
版本: v1.0
|
|
建立: 2026-03-26 (台北時區)
|
|
"""
|
|
|
|
from .golden_responses import GoldenResponseManager
|
|
from .property_validators import (
|
|
validate_kubectl_syntax,
|
|
validate_risk_level,
|
|
validate_chinese_ratio,
|
|
validate_response_length,
|
|
)
|
|
from .schema_validators import LLMProposalOutput, validate_proposal_schema
|
|
|
|
__all__ = [
|
|
"GoldenResponseManager",
|
|
"LLMProposalOutput",
|
|
"validate_proposal_schema",
|
|
"validate_kubectl_syntax",
|
|
"validate_risk_level",
|
|
"validate_chinese_ratio",
|
|
"validate_response_length",
|
|
]
|