Compare
Run two different prompts against the same model simultaneously and compare results side-by-side — latency, token cost, and quality.
Overview
The Compare tool runs Prompt A and Prompt B simultaneously against the same model and displays the results in a split-screen view. It answers the most common prompt engineering question: "Which version of my prompt is better?"
How to use
- Type your first prompt in the Prompt A panel
- Type a variation in the Prompt B panel
- Click Run Both — both prompts fire simultaneously
- Review the results, metrics, and winner badge
Note
Both prompts run with the same model configuration (model, temperature, max tokens) set in the Config panel. The only variable is the prompt text itself.
Metrics explained
| Metric | What it means |
|---|---|
| Latency | Time in milliseconds from request to first token received |
| Tokens | Total tokens consumed (prompt + completion). Higher = higher cost |
| Cost | Estimated cost in USD based on token count and model pricing |
| Speed winner | Whichever prompt returned first, and by what % margin |
Winner badge
After both runs complete, each result panel shows a winner badge if that version outperformed the other on speed. The badge shows the percentage improvement — e.g., "A was 34% faster."
For quality comparison (not just speed), use the Evaluate tool to run the LLM judge on each output.
Common use cases
- Prompt wording — "Summarize this article" vs "Give me a 3-sentence summary of this article"
- Instruction style — imperative vs conversational phrasing
- Specificity testing — vague prompt vs highly constrained prompt
- System prompt A/B — different persona definitions for the same task
- Token efficiency — shorter prompt vs detailed prompt and their cost tradeoffs
Tip
Keep Prompt A as your current "baseline" and Prompt B as your experimental variation. This way you always know which direction you're iterating.