LogoSkills

evaluate:ambiguity-score

Ambiguity Score Calculation โ€” Measure spec clarity through 5-dimension quantitative evaluation

ํ•ญ๋ชฉ๋‚ด์šฉ
Invoke/evaluate:ambiguity-score
Categoryspec-clarity
Complexitymoderate

/spec:ambiguity โ€” ๋ช…์„ธ์„œ "์• ๋งคํ•จ" ์ ์ˆ˜ ๋งค๊ธฐ๊ธฐ#

ํ•œ๋งˆ๋””๋กœ#

์ž‘์„ฑํ•œ ๋ช…์„ธ์„œ(Seed spec)๊ฐ€ ์–ผ๋งˆ๋‚˜ ๋˜๋ ทํ•œ์ง€, ์•„๋‹ˆ๋ฉด ์• ๋งคํ•œ์ง€๋ฅผ 0~1์ ์งœ๋ฆฌ ์ ์ˆ˜๋กœ ์ฑ„์ ํ•ด ์ฃผ๋Š” ๋„๊ตฌ์ž…๋‹ˆ๋‹ค. ๊ธ€์“ฐ๊ธฐ ์„ ์ƒ๋‹˜์ด ๋นจ๊ฐ„ํŽœ์œผ๋กœ "์ด ๋ถ€๋ถ„ ๋ฌด์Šจ ๋œป์ธ์ง€ ๋ชจ๋ฅด๊ฒ ์–ด์š”"๋ผ๊ณ  ์งš์–ด์ฃผ๊ณ  ์ข…ํ•ฉ ์ ์ˆ˜๊นŒ์ง€ ๋งค๊ฒจ์ฃผ๋Š” ๊ฒƒ๊ณผ ๊ฐ™์•„์š”.

๋ˆ„๊ฐ€ยท์–ธ์ œ ์“ฐ๋‚˜์š”#

  • ๊ธฐํš์„œยท์š”๊ตฌ์‚ฌํ•ญ(Seed spec)์„ ๋‹ค ์“ด ๋’ค, "์ด๋Œ€๋กœ ๊ฐœ๋ฐœ์— ๋„˜๊ฒจ๋„ ๋  ๋งŒํผ ๋ช…ํ™•ํ•œ๊ฐ€?"๋ฅผ ํ™•์ธํ•˜๊ณ  ์‹ถ์„ ๋•Œ
  • ๋ˆ„๊ตฐ๊ฐ€ ์ฝ๊ณ  ์˜คํ•ดํ•  ๋งŒํ•œ ์• ๋งคํ•œ ํ‘œํ˜„์ด ๋‚จ์•„ ์žˆ๋Š”์ง€ ์ ๊ฒ€ํ•˜๊ณ  ์‹ถ์„ ๋•Œ

๋ฌด์—‡์„ ํ•ด์ฃผ๋‚˜์š”#

๋ช…์„ธ์„œ๋ฅผ 5๊ฐ€์ง€ ๊ธฐ์ค€์œผ๋กœ ์‚ดํŽด๋ณธ ๋’ค ์ข…ํ•ฉ ์• ๋งคํ•จ ์ ์ˆ˜์™€ ๋“ฑ๊ธ‰์„ ์•Œ๋ ค์ฃผ๊ณ , ๊ทธ ์ ์ˆ˜๋ฅผ ๋ช…์„ธ์„œ ํŒŒ์ผ ์•ˆ์—๋„ ๊ธฐ๋กํ•ด ์ค๋‹ˆ๋‹ค.

  • 5๊ฐ€์ง€ ์ฑ„์  ๊ธฐ์ค€
    • ๋‹จ์–ด ์ •ํ™•์„ฑ โ€” "๋น ๋ฅด๊ฒŒ", "์ ์ ˆํžˆ", "ํ•„์š”์‹œ" ๊ฐ™์€ ๋‘๋ฃจ๋ญ‰์ˆ ํ•œ ๋ง์„ ์“ฐ์ง€ ์•Š์•˜๋Š”์ง€
    • ์ง€์นญ ๋ช…ํ™•์„ฑ โ€” "์ด๊ฒƒ", "๊ทธ๊ฒƒ" ๊ฐ™์€ ๋ง์ด ๋ฌด์—‡์„ ๊ฐ€๋ฆฌํ‚ค๋Š”์ง€ ๋ถ„๋ช…ํ•œ์ง€
    • ์™„์ „์„ฑ โ€” ๋ชจ๋“  ๊ธฐ๋Šฅ์— ํ•ฉ๊ฒฉ ๊ธฐ์ค€์ด ์žˆ๊ณ , ์˜ˆ์™ธ ์ƒํ™ฉยท์˜ค๋ฅ˜ ์ƒํ™ฉ์ด ๋น ์ง€์ง€ ์•Š์•˜๋Š”์ง€
    • ์ผ๊ด€์„ฑ โ€” ์šฉ์–ด๊ฐ€ ํ†ต์ผ๋๋Š”์ง€, ์•ž๋’ค ์ˆซ์ž๋‚˜ ์šฐ์„ ์ˆœ์œ„๊ฐ€ ์„œ๋กœ ์–ด๊ธ‹๋‚˜์ง€ ์•Š๋Š”์ง€
    • ํ…Œ์ŠคํŠธ ๊ฐ€๋Šฅ์„ฑ โ€” ๊ฐ ์š”๊ตฌ์‚ฌํ•ญ์ด ํ•ฉ๊ฒฉ/๋ถˆํ•ฉ๊ฒฉ์„ ๊ฐ๊ด€์ ์œผ๋กœ ํŒ์ •ํ•  ์ˆ˜ ์žˆ๋Š”์ง€
  • ๊ฒฐ๊ณผ ๋“ฑ๊ธ‰: CLEAR(ํ†ต๊ณผ) / WARNING(์กฐ๊ฑด๋ถ€) / AMBIGUOUS(๋ฏธํก) / UNCLEAR(์žฌ์ž‘์„ฑ ํ•„์š”)

์–ด๋–ป๊ฒŒ ์“ฐ๋‚˜์š”#

/spec:ambiguity                      # ๊ฐ€์žฅ ์ตœ๊ทผ Seed ๋ช…์„ธ์„œ๋ฅผ ์ฑ„์ 
/spec:ambiguity --file {path}        # ํŠน์ • ํŒŒ์ผ์„ ์ง€์ •ํ•ด ์ฑ„์ 
/spec:ambiguity --verbose            # ๊ธฐ์ค€๋ณ„ ์ƒ์„ธ ๋ถ„์„๊นŒ์ง€ ํ•จ๊ป˜ ๋ณด๊ธฐ
  • ๊ทธ๋ƒฅ ์‹คํ–‰ํ•˜๋ฉด ๊ฐ€์žฅ ์ตœ๊ทผ์— ๋งŒ๋“  ๋ช…์„ธ์„œ๋ฅผ ์ž๋™์œผ๋กœ ์ฐพ์•„ ์ฑ„์ ํ•ฉ๋‹ˆ๋‹ค.
  • --file๋กœ ์ฑ„์ ํ•  ํŒŒ์ผ์„ ์ง์ ‘ ์ง€์ •ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • --verbose๋ฅผ ๋ถ™์ด๋ฉด 5๊ฐ€์ง€ ๊ธฐ์ค€๋ณ„๋กœ ์–ด๋””๊ฐ€ ์™œ ์• ๋งคํ•œ์ง€ ์ž์„ธํ•œ ๋ถ„์„๊นŒ์ง€ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค.

์•ˆ์—์„œ ๋ฌด์Šจ ์ผ์ด ๋ฒŒ์–ด์ง€๋‚˜์š”#

  1. ํŒŒ์ผ ๋ถˆ๋Ÿฌ์˜ค๊ธฐ โ€” ๋Œ€์ƒ ๋ช…์„ธ์„œ๋ฅผ ์ฝ๊ณ  ํ•ญ๋ชฉ๋ณ„๋กœ ๋‚˜๋ˆ•๋‹ˆ๋‹ค.
  2. 5๊ฐ€์ง€ ๊ธฐ์ค€ ์ฑ„์  โ€” ๋‹จ์–ด ์ •ํ™•์„ฑยท์ง€์นญ ๋ช…ํ™•์„ฑยท์™„์ „์„ฑยท์ผ๊ด€์„ฑยทํ…Œ์ŠคํŠธ ๊ฐ€๋Šฅ์„ฑ์„ ๊ฐ๊ฐ 0~1์ ์œผ๋กœ ๋งค๊น๋‹ˆ๋‹ค.
  3. ์ข…ํ•ฉ ์ ์ˆ˜ ๊ณ„์‚ฐ โ€” ๊ธฐ์ค€๋งˆ๋‹ค ์ •ํ•ด์ง„ ๊ฐ€์ค‘์น˜๋ฅผ ๊ณฑํ•ด ํ•˜๋‚˜์˜ ์ตœ์ข… ์ ์ˆ˜๋กœ ํ•ฉ์‚ฐํ•ฉ๋‹ˆ๋‹ค.
  4. ๋“ฑ๊ธ‰ ํŒ์ • โ€” ์ ์ˆ˜์— ๋”ฐ๋ผ ํ†ต๊ณผ/์กฐ๊ฑด๋ถ€/๋ฏธํก/์žฌ์ž‘์„ฑ ํ•„์š” ์ค‘ ํ•˜๋‚˜๋กœ ํŒ์ •ํ•ฉ๋‹ˆ๋‹ค.
  5. ๋ช…์„ธ์„œ์— ๊ธฐ๋ก โ€” ๊ณ„์‚ฐ๋œ ์• ๋งคํ•จ ์ ์ˆ˜๋ฅผ ๋ช…์„ธ์„œ ํŒŒ์ผ ์ •๋ณด๋ž€์— ๋‹ค์‹œ ์ ์–ด ๋‘ก๋‹ˆ๋‹ค.

โš™๏ธ ์ƒ์„ธ ์˜ต์…˜ยท์‹คํ–‰ ๋ช…์„ธ (๊ฐœ๋ฐœ์ž / AI ์—์ด์ „ํŠธ์šฉ)

Usage#

/spec:ambiguity                      # Evaluate the latest Seed spec
/spec:ambiguity --file {path}        # Evaluate a specific file
/spec:ambiguity --verbose            # Include detailed per-dimension analysis

Parameters#

ParameterDescriptionDefault
--fileTarget file pathLatest docs/seed-spec-*.md
--verboseDetailed analysis outputfalse

Execution Flow#

1. Load Target File#

  • Read Seed spec file
  • Parse each section

2. 5-Dimension Evaluation#

Evaluate each dimension per AMBIGUITY_RUBRIC.md:

2a. Lexical Precision (weight 0.25)

  • Scan for prohibited words in the spec's own language โ€” English ("fast", "appropriate", "etc.", "if needed" ...) AND Korean ("๋น ๋ฅด๊ฒŒ", "์ ์ ˆํžˆ", "๋“ฑ๋“ฑ", "ํ•„์š”์‹œ" ...). Use the locale-tagged table in AMBIGUITY_RUBRIC.md.
  • Use semantic judgment, not keyword-matching alone โ€” a vague phrase in any language counts even if it is not in the table.
  • If the spec language is not covered by the table, mark this dimension UNVERIFIED (not 0.0) and fall back to semantic judgment โ€” never score an un-scannable language as "0 prohibited words".
  • Check whether specific values/criteria have been substituted
  • Score: 0.0 (0 prohibited words) ~ 1.0 (pervasively ambiguous)

2b. Reference Clarity (weight 0.20)

  • Scan for pronouns/demonstratives โ€” English ("this", "that", "said") AND Korean ("์ด๊ฒƒ", "๊ทธ๊ฒƒ", "ํ•ด๋‹น", "์œ„์˜")
  • Check whether abbreviations are defined
  • Check for implicit subjects ("is processed" โ†’ missing agent)
  • Score: 0.0 (all references clear) ~ 1.0 (key references unclear)

2c. Completeness (weight 0.20)

  • Check whether AC (acceptance criteria) exist for all features
  • Check whether edge cases are identified
  • Check whether error scenarios are defined
  • Score: 0.0 (all complete) ~ 1.0 (most missing)

2d. Consistency (weight 0.20)

  • Check terminology unity
  • Check for numeric/logical contradictions between sections
  • Check priority consistency
  • Score: 0.0 (fully consistent) ~ 1.0 (severe contradictions)

2e. Testability (weight 0.15)

  • Check whether each requirement has Pass/Fail criteria
  • Check automation feasibility
  • Check reproducibility
  • Score: 0.0 (all testable) ~ 1.0 (most require subjective judgment)

3. Final Score Calculation#

score = (lexical ร— 0.25) + (reference ร— 0.20) + (completeness ร— 0.20)
      + (consistency ร— 0.20) + (testability ร— 0.15)

4. Verdict#

GradeScoreResult
CLEAR<= 0.2Pass
WARNING0.2~0.3Conditional โ€” Improvement recommendations provided
AMBIGUOUS0.3~0.5Fail โ€” Required fix items presented
UNCLEAR> 0.5Reject โ€” Rewrite required

5. Seed Spec Update#

  • Update the ambiguity score in metadata

Output#

Evaluation results printed to console. With --verbose, detailed per-dimension analysis is included. Ambiguity score in Seed spec metadata is updated.

  • /spec:seed โ€” Create Seed spec
  • /spec:verify โ€” Verification + lock
  • /spec:evaluate โ€” 3-stage integrated evaluation

References#

  • references/AMBIGUITY_RUBRIC.md โ€” Detailed 5-dimension rubric
  • config/thresholds.yaml โ€” Threshold settings