Stop measuring prompt quality by gut feeling

Pattern: Provocative thesis + Framework (CLEAR)

Keywords: prompt quality scoring, prompt evaluation, CLEAR prompt evaluation

Priority: P1 (month 2-3)

Score: 70

Length: 1,500-2,000 words

Thesis: 5 out of 5 people we interviewed said prompt quality is "vibes-based." Nobody has a systematic way to measure whether a prompt is good. The CLEAR framework gives you 5 measurable dimensions — Clarity (25%), Level of Detail (20%), Effectiveness (25%), Adaptability (15%), Robustness (15%) — turning gut feeling into actionable scores.

Outline:

Opening: "Quality is vibes-based" (cite discovery research — 5/5 interviews)
What happens without metrics: inconsistent outputs, no improvement loop, no team alignment
The CLEAR framework: 5 dimensions explained with examples
Before/after: a prompt scored at 42 vs the same prompt improved to 78 — what changed
How scores appear like a linter, not a modal (Invisible Intelligence)
Two-phase evaluation: scores + suggestions → improved fields with delta
CTA: score your first prompt in prompt-x

Draft

title: Stop measuring prompt quality by gut feeling

slug: prompt-quality-scoring-clear-framework

description: Nobody has a systematic way to measure prompt quality. CLEAR gives you 5 measurable dimensions to score and improve prompts consistently.

keywords: prompt quality scoring, CLEAR prompt evaluation, prompt evaluation, prompt testing tool author: Mariano date: 2026-04-13

We interviewed AI engineers and prompt engineers about how they evaluate prompt quality. The answer, every time: gut feeling.

Not one team had a systematic way to measure whether a prompt is good. Not one. They knew a prompt was working when it produced the right output. They knew it was failing when it didn't. But between those two states, there was nothing but intuition.

The problem with gut-based evaluation is that it doesn't scale. It doesn't compound. It doesn't create feedback loops. You can't improve what you can't measure.

CLEAR is a framework for measuring prompt quality systematically. Five dimensions. Measurable. Actionable. Built into the editor, not bolted on top.

What happens without metrics

Teams that evaluate prompts by intuition hit three concrete problems:

Inconsistency across team members. One engineer thinks a prompt is clear; another thinks it's ambiguous. One thinks it has enough detail; another wants more context. The same prompt passes QA in one session and fails in the next. There's no shared definition of what "good" means.

No improvement loop. You can't optimize what you can't quantify. You change a prompt, deploy it, and something feels different — but you have no way to know if it's actually better. Did output quality improve? Did consistency improve? Did edge case handling improve? You don't know. You're flying blind.

No team alignment on trade-offs. When everyone evaluates quality differently, team members make different trade-off decisions. One person prioritizes conciseness. Another prioritizes completeness. The prompt becomes a compromise between competing intuitions, satisfying no one.

The result: prompts that feel vaguely wrong but you can't articulate why. Prompts that work 85% of the time but you can't diagnose the 15%. Prompts that your team disagrees about.

Introducing CLEAR: Five measurable dimensions

CLEAR is an evaluation framework with five dimensions. Each dimension is scored independently. Each dimension has clear definitions. Scores are combined into a single 0–100 quality metric.

C — Clarity Does the prompt have a unambiguous intent? Does each field have a single, clear purpose? Clarity answers: "Could a different person read this prompt and interpret it the same way I did?"

Low clarity: role and persona are mixed together, instructions are implied rather than explicit, field boundaries are blurry. A "senior engineer" prompt that also smuggles in task instructions without separating them.

High clarity: role is distinct from context, context is distinct from task, instructions are explicit and isolated. Each field contains one semantic unit.

L — Level of Detail Are instructions specific enough for the model to execute? Is context sufficient? Does the prompt give the model what it needs to succeed?

Low detail: "Write a summary" without defining what type of summary, how long, what format, who the audience is. The model has to guess.

High detail: "Write a 150-word executive summary for a VP of Product who needs to make a budget decision. Include: one-sentence key finding, three supporting data points, recommended action. Format as bullet points."

Detail isn't about verbosity. A 20-word prompt can have high detail if those 20 words are specific and targeted. A 500-word prompt can have low detail if it's rambling and vague.

E — Effectiveness Will this prompt produce the desired outcome? Are the right fields active for this task? Is the prompt architected to succeed?

Effectiveness is different from detail. You can have a well-detailed prompt that still fails because it's architecturally wrong. A prompt that asks a model to "creative brainstorm ideas" but then constraints it to "follow the template exactly" is fighting itself.

Low effectiveness: conflicting fields, unclear success criteria, missing critical information, wrong model capabilities matched to the task.

High effectiveness: fields work together, success criteria are measurable, all prerequisites are present, the task is within the model's reliable capability range.

A — Adaptability Can this prompt handle edge cases? Are variables and constraints robust? Does it degrade gracefully when given unusual input?

Adaptability asks: does this prompt work for the 80% case and also handle the 20%?

Low adaptability: the prompt breaks when you feed it an edge case. A "code reviewer" prompt that works fine for Python but fails completely on Rust. Instructions written for 2,000-word documents that collapse when given 200-word input.

High adaptability: variables are flexible, constraints scale, the prompt anticipates edge cases and routes around them gracefully.

R — Robustness Will this prompt work consistently? Is it resilient to input variation?

Robustness is about consistency. Same model, same prompt, different inputs: do you get reliable output quality?

Low robustness: output quality varies wildly depending on input format, input complexity, or minor variations in phrasing. Sometimes the model follows instructions; sometimes it ignores them.

High robustness: output quality is consistent. The prompt works whether input is structured or unstructured, simple or complex, well-formatted or messy.

How scoring works

Each dimension is scored on a 0–10 scale. Scores are weighted by importance for your specific task. Overall score is 0–100.

A typical weighting for a code review prompt might be:

Clarity: 20% (everyone needs to understand it)
Level of Detail: 25% (details drive output quality)
Effectiveness: 30% (does it work?)
Adaptability: 15% (code review patterns are fairly standard)
Robustness: 10% (code is usually well-formatted)

A different prompt — say, a content moderation prompt — would weight differently:

Clarity: 15% (simpler task)
Level of Detail: 20%
Effectiveness: 25%
Adaptability: 25% (needs to handle edge cases, different content types)
Robustness: 15%

You define the weights. The framework provides structure.

In the editor, scoring surfaces inline. You don't open a separate "evaluation" panel. You don't click "analyze." The score appears as you type. Like a linter for code, but for prompts.

Before and after: From 42 to 78

Here's a real example. A prompt for "senior code reviewer" that scores 42/100:

ROLE: Code reviewer
CONTEXT: You review code for an ecommerce platform
TASK: Review the attached code. Look for issues. Provide feedback.
CONSTRAINTS: Be helpful

Breakdown:

Clarity: 3/10 — "Code reviewer" is vague. What kind of code? What level of experience? "Issues" is undefined. What kind of issues? "Feedback" is vague. "Be helpful" is meaningless.
Level of Detail: 4/10 — Almost no specific guidance. No format, no prioritization, no scope definition.
Effectiveness: 5/10 — The prompt might work, but it's fighting a losing battle. The model has to infer everything.
Adaptability: 6/10 — Might work for small pieces of code; breaks for large, complex files.
Robustness: 4/10 — Output is likely inconsistent. Sometimes you get useful feedback; sometimes you get nonsense.

Weighted score: 42/100 — This prompt is barely functional.

Now, after improvement:

ROLE: You are a senior code reviewer with 15 years of experience in Python and distributed systems.

CONTEXT: You're reviewing pull requests for an ecommerce platform handling order processing and inventory. Security and performance are equally critical. Code is written for clarity over brevity.

TASK: Review the attached code for:
1. Security vulnerabilities (highest priority)
2. Performance bottlenecks (medium priority)
3. Code clarity and maintainability (lowest priority)

For each issue found:
- State the issue clearly
- Explain why it matters
- Propose a specific fix
- Rate severity (critical, high, medium, low)

OUTPUT FORMAT: Markdown with ## Issue headers, each containing Description, Impact, and Proposed Fix sections.

CONSTRAINTS:
- Focus on issues that matter. Skip style preferences and minor inconsistencies.
- Explain your reasoning. Don't just list problems.
- If code is unconventional but works, identify it as a pattern choice, not an error.
- Assume code is intentional unless it appears genuinely broken.
- If you can't be certain about something, say so.

EDGE CASES:
- For files over 500 lines, prioritize the most critical systems first
- For multiple vulnerabilities in the same area, consolidate feedback
- If the codebase uses patterns you don't recognize, ask clarifying questions rather than rejecting them

Breakdown:

Clarity: 9/10 — Role, context, task, and constraints are all distinct. Intent is unambiguous.
Level of Detail: 8/10 — Specific guidance on what to review, output format, edge cases. Clear priorities.
Effectiveness: 9/10 — The prompt is architected to succeed. Fields support the task. Model has what it needs.
Adaptability: 8/10 — Handles edge cases explicitly. Scales from small to large files.
Robustness: 7/10 — More consistent. Explicit output format reduces variance.

Weighted score: 78/100 — This prompt is production-ready.

The delta is 36 points. Same task, same model, different prompt quality. The second version will produce measurably better code reviews.

The evaluation loop: Score → Improve → Repeat

Scoring alone doesn't help. Improvement does.

The system doesn't just surface a number. It surfaces field-level suggestions.

When Clarity is low, the system flags which fields are ambiguous. When Level of Detail is low, it suggests specific additions. When Robustness is low, it recommends constraint refinements.

The workflow looks like this:

You write a prompt. The system scores it: 42/100.
You read the suggestions. Clarity is 3/10. Suggestion: "Expand role from 'Code reviewer' to include experience level, specialization, and coding preferences."
You apply the suggestion. Role now reads: "You are a senior code reviewer with 15 years of experience in Python and distributed systems."
You rescore. Clarity is now 7/10. Level of Detail is still 4/10. Suggestion: "Add output format to task section. Add success criteria to constraints section."
You iterate. Apply suggestions. Watch the score improve. Stop when it reaches your target (usually 75+).

This is a feedback loop. Every change shows you exactly what improved and what's still weak.

Why this works like a linter

Traditional prompt evaluation happens in a modal. You click "Evaluate," wait for a response, read through paragraphs of analysis, and try to translate subjective feedback into edits.

CLEAR surfaces scoring inline, in the editor, as you work. Like a code linter that runs in your IDE and flags issues in red as you type.

The scoring appears in the background. Suggestions appear as you focus on each field. You never leave the editing context.

This is Invisible Intelligence: the system helps you improve without interrupting your workflow. You don't see "AI is thinking" spinners. You don't see a separate "AI evaluation" button. The feedback just surfaces, naturally, at the moment you need it.

Building team standards

When everyone on your team uses CLEAR, you build shared standards.

Instead of debating whether a prompt is "good," you align on a score threshold. "We ship prompts that score 75+ on CLEAR." Everyone understands what 75 means. Clarity is at least 7/10. Robustness is at least 7/10.

Changes to a prompt show you exactly what moved. A refactor that improves Robustness from 6 to 8 is visible, measurable, discussable.

New team members learn quality standards by examples, not by intuition. They see what a 75 looks like. They see what a 90 looks like. They build mental models of quality based on concrete metrics.

Over time, your team's prompts get consistently better because you're optimizing toward measurable standards, not debating feelings.

From vibes to metrics

The status quo is prompts evaluated by whoever deploys them last. The outcome is inconsistency, lost improvement cycles, and prompts that feel vaguely wrong but nobody can explain why.

CLEAR shifts evaluation from intuition to metrics. From subjective debate to measurable feedback. From "this feels good" to "this is a 42, here's why, here's how to improve it."

The prompt that scores 90 will outperform the prompt that scores 60. The improvement isn't philosophical. It's measurable.

prompt-x scores prompts across 5 CLEAR dimensions with field-level improvement suggestions. Score your first prompt at prompt-x.io/try.

Related reading