SparkEval is a lightweight LLM evaluation tool built by MagicPill Labs for small teams and individuals. It compares model providers side by side, supports custom graders, and runs automated evaluations against your own test datasets. The free tier covers 10 custom graders and 50 evaluations a month; paid tiers start at $10 a month.
Get Started on SparkEval ↗What SparkEval Does
Everything you need to evaluate and compare LLM performance without the enterprise complexity.
Compare Providers
Side-by-side comparison of AI model providers. See how GPT-4, Claude, Gemini, and others stack up on your data.
Custom Graders
Define your own evaluation criteria. Score on accuracy, tone, format, or any custom metric that matters.
Automated Evals
Schedule recurring evaluations. Track model performance over time and catch regressions early.
Dataset Testing
Upload your test datasets and run evaluations at scale. Batch testing for systematic quality assurance.
Pricing
Start free, upgrade when you need more. No credit card required.
Free
- Custom Graders 10
- File Uploads 50
- Dataset Test Problems 100
- Automated Evals/mo 50
- Export as JSON —
Basic
- Custom Graders 50
- File Uploads 250
- Dataset Test Problems 500
- Automated Evals/mo 100
- Export as JSON —
Pro
- Custom Graders Unlimited
- File Uploads Unlimited
- Dataset Test Problems Unlimited
- Automated Evals/mo Unlimited
- Export as JSON ✓