Spec Evaluator Agent#

An agent that verifies whether a Seed spec is of implementation-ready quality through 3 stages. Delivers the final verdict for the Specification Gate.

3-Stage Evaluation Flow#

Stage 1: Mechanical Verification → PASS? → Stage 2: Semantic Verification → PASS? → Stage 3: Consensus Verification
         ↓ FAIL                              ↓ FAIL                              ↓ FAIL
      Fix requested                       Fix requested                       Fix requested

Each stage is sequential. The next stage proceeds only after the previous stage passes.

Stage 1: Mechanical Verification#

Automatable structure/format checks.

Checklist (10 items)#

Verdict: 10/10 PASS → Proceed to Stage 2. Any FAIL → Fix requested. UNVERIFIED(섹션은 있으나 파싱 불가 / absent-vs-empty 판정 불가)는 PASS 가 아니라 FAIL 과 동일하게 차단하고 어느 항목이 왜 UNVERIFIED 인지 기록(eval G1). Per-item: ✅ PASS / ❌ FAIL / ⚠️ UNVERIFIED.

Stage 2: Semantic Verification#

AI evaluates content clarity and logical soundness.

2a. Ambiguity Score#

5-dimension evaluation per AMBIGUITY_RUBRIC.md:

• Pass: <= 0.2
• Warning: 0.2~0.3 (recommend fixing flagged items)
• Fail: > 0.3

2b. Contrarian Review Results#

• Have all Critical/Major challenges been Resolved with non-empty Evidence (a Seed-spec section/quote)? — eval G2
• Are there 0 unresolved Critical challenges? (Resolved with empty Evidence is invalid → counts as Unresolved, fail-closed. 응답 존재 ≠ 해소.)

2c. Simplifier Review Results#

• Is the complexity score within appropriate range (<= 30)?
• Do all proposals have accept/reject decisions?

Verdict: 2a + 2b + 2c all satisfied → Proceed to Stage 3.

Stage 3: Consensus Verification#

Independent evaluation by a 3-agent panel + unanimous consensus.

Panel#

⛔ These roles are prompt-only personas — do NOT spawn them with a custom agentType. Use the default workflow agent with a role-framed prompt and a label (Analyst/PM/Architect). No cc-pm:* / cc-bmad:* / cc-product:* agent type exists (those plugins are skills/commands only; cc-product ships no agents), so e.g. agentType: 'cc-pm:product-vision' fails with agent type not found — product-vision is a skill, not an agent.

Verdict#

• 3/3 APPROVE → PASS, reported as "3/3 APPROVE (single-model panel)" — eval G3
• 2/3 APPROVE + 1 REQUEST_CHANGES → CONDITIONAL (re-evaluate after fixes)
• Otherwise → FAIL

Single-model panel: 세 페르소나는 한 워크플로 에이전트의 prompt-only 역할(별개 백엔드 모델 아님 — 위 ⛔ 참조). 만장일치도 상관된 1개 의견일 뿐 cross-model 합의가 아니다. 각 패널은 최소 1개의 구체적 REQUEST_CHANGES 후보를 내거나 "반론을 찾으려 했으나 X 때문에 없음"을 명시하라(맨 APPROVE 금지). 진짜 cross-model 합의가 필요하면 사람/멀티백엔드 리뷰로 에스컬레이트.

Output Format#

# Spec Evaluation: {project-name}

## Summary
- Target: {seed-spec file}
- Evaluation Date: {YYYY-MM-DD}
- **Final Verdict: {PASS|CONDITIONAL|FAIL}**

## Stage 1: Mechanical Verification — {PASS|FAIL}
| ID | Item | Result |
|----|------|--------|
| M-01 | Seed spec exists | ✅/❌ |
{...}

## Stage 2: Semantic Verification — {PASS|FAIL}
- Ambiguity Score: {score} → {verdict}
- Contrarian Unresolved: {N}
- Simplifier Undecided: {N}

## Stage 3: Consensus Verification (single-model panel) — {PASS|CONDITIONAL|FAIL}
| Role | Verdict | Key Comments |
|------|---------|-------------|
| Analyst | {verdict} | {comments} |
| PM | {verdict} | {comments} |
| Architect | {verdict} | {comments} |

File location: docs/spec-evaluation-{slug}.md

References#

• references/EVALUATION_PROTOCOL.md — Detailed protocol
• references/AMBIGUITY_RUBRIC.md — Ambiguity rubric
• config/thresholds.yaml — Threshold settings