LLM-as-a-Judge
In one line
LLM-as-a-judge is the practice of using one LLM to grade or compare the answers of another — a standard way to scale evaluation beyond what human labelling can cover.
Going deeper
LLM-as-a-judge replaces row-by-row human grading with a strong LLM that scores or compares answers under a fixed rubric. It is faster, cheaper and more consistent than human labelling, and the gap widens as case counts grow.
It has well-documented blind spots. Judges tend to prefer longer answers, favour responses that share their own tone, and react more than they should to small wording changes in the rubric. Treating their absolute scores cautiously and using them mainly as 'model A vs model B with a fixed prompt' comparisons is the safer pattern.
It also fits B2B and GEO evaluation. When you want to know whether your content gets cited more often than a competitor's for a given prompt set, a judge model can compare answers with and without your source and turn a fuzzy KPI into something you can track in a tight loop.
Related terms
Instruction Tuning
Instruction tuning is the fine-tuning step that teaches a base LLM to follow instructions in natural language — the stage that turns 'a model that completes text' into 'a model you can actually ask things'.
LLMKnowledge Distillation
Knowledge distillation trains a smaller 'student' model to mimic a larger 'teacher' model — preserving most of the quality while drastically cutting cost and latency.
LLMAI Watermarking
AI watermarking embeds an imperceptible signal in AI-generated text, images or audio so that the content can later be identified as machine-made.
LLMLLM
A large language model (LLM) is a neural network trained on massive text corpora to understand and generate human language — the engine behind ChatGPT, Claude, Gemini and similar products.
GEO·AEOCitation Rate
Citation rate is the share of a defined prompt set in which an AI answer cites your brand or domain — the headline KPI of GEO.
How does your brand show up in AI answers?
Villion measures how your brand appears across ChatGPT, Perplexity and AI Overviews, then automates the work that lifts citation rate and share of voice.
Get a free audit