Evaluate
The Evaluate page runs an LLM-as-a-judge pipeline to score a prompt/response pair across five quality dimensions, then lets you auto-optimize the prompt to address weaknesses.
Overview
Evaluating prompt quality subjectively is hard — what "good" means varies by context. The Evaluate tool uses an LLM as a judge: it sends your prompt and the model's response to a second LLM call that analyzes and scores the output across five structured categories.
This is a common pattern in production LLM systems where automated quality assurance is needed at scale.
How to use
- Paste the prompt you used (the input you gave the model) in the left panel
- Paste the model response you received in the lower left panel
- Click Run Judge
- Read the scorecard on the right — each category gets a score from 1–5 with a justification
Score categories
| Category | What it measures |
|---|---|
| Clarity | Is the response easy to understand? Is it well-structured and unambiguous? |
| Relevance | Does the response directly address what was asked? |
| Completeness | Does it cover all aspects of the request, or does it leave things out? |
| Creativity | Is there original thinking, novel framing, or insightful elaboration? |
| Consistency | Is the response internally consistent — no contradictions or logical gaps? |
Each score is 1–5. The Executive Summary at the top gives an overall assessment in plain language.
Auto-Optimize
After receiving scores, click Auto-Optimize. This triggers a second LLM call that reads your original prompt, the response, and the scorecard — then rewrites your prompt to address the weak scores.
The optimized prompt replaces the content in the Prompt Context panel so you can immediately re-evaluate.
Interpreting scores
- 5 / 5 — Excellent. The response fully satisfies this criterion
- 4 / 5 — Good. Minor improvements possible
- 3 / 5 — Acceptable. Noticeable gaps — worth addressing
- 2 / 5 — Weak. The prompt likely needs restructuring
- 1 / 5 — Poor. The response fails this criterion significantly
Focus optimization effort on the lowest-scoring categories. A score of 3 or below in Relevance usually means the prompt is too vague — the model didn't understand what you wanted.