Prompt Panel Methodology: Sizing & Running One for GEO

Last updated: 6 November 2026

Prompt panel methodology is the structured process of testing a fixed set of questions against AI engines, repeatedly and on a schedule, to measure how often a brand appears in the answers. Industry practice puts panel size between twenty and one hundred prompts, with weekly runs of 100–200 prompts described as the minimum viable system for tracking AI share of voice, according to Digital Applied (2026).

Key Takeaways

Prompt panel size in industry practice ranges from roughly twenty to one hundred prompts, with some practitioners recommending thirty to fifty tested across five or more engines, according to Arfadia (2026).
A weekly run of 100–200 prompts is described as the minimum viable system for tracking AI share of voice, per Digital Applied (2026).
A single prompt run is unreliable on its own: 95% of product titles tracked across 22.5 million ChatGPT Shopping offers appeared in under 30% of repeated runs of the same prompt, according to Nick Lafferty (2026).
A 2026 field study of 30,504 responses recommends at least seven runs per prompt and engine for brand-presence measurement, and eight runs for source-coverage measurement, spread across two to four weeks, per NeuralAdX (2026).
Only 3.8% of sources were cited by all four major AI engines in a July 2026 study of 161,286 prompts across ChatGPT, Gemini, Perplexity and Google AI Overviews, according to NeuralAdX (2026).

What is a prompt panel and what problem does it solve?

A prompt panel is a fixed, repeatable set of questions run against one or more AI engines — such as ChatGPT, Perplexity, Google AI Overviews, Claude, Gemini and Microsoft Copilot — to measure how consistently a brand, product or source is mentioned or cited in the generated answers. Generative engine optimisation (GEO) is the discipline of improving a brand's visibility inside these AI-generated answers, and the prompt panel is its primary measurement instrument, playing a role roughly equivalent to a keyword rank tracker in traditional SEO.

The problem it solves is simple but acute: AI answers are not deterministic. The same question put to the same engine can produce a different answer minutes later, so a single screenshot of ChatGPT naming (or not naming) a brand proves almost nothing. A prompt panel solves this by repeating a defined question set over time, turning a one-off observation into a trend line.

This matters commercially because most UK marketing teams are currently flying blind here. 45% of marketing leaders cannot accurately measure their brand's visibility within AI-generated answers, and only 9% have the tools to track all relevant metrics across platforms, according to Semrush (2026). Semrush's own 2026 AI Visibility Index illustrates the scale needed to do this properly at the market level — it grew from an initial analysis of 2,500 prompts to 126 million US AI search prompts analysed between January and April 2026, per Semrush (2026). A business-level prompt panel is a scaled-down version of that same logic, sized for one brand rather than an entire market.

Aether AI, the self-service GEO platform built by Aether Agency Ltd, exists to automate this process: generating AI-optimised content and tracking citations across six engines — ChatGPT, Perplexity, Google AI Overviews, Claude, Gemini and Copilot — so that teams don't have to run panels manually in a spreadsheet.

How does prompt panel methodology differ from single-prompt testing or A/B testing?

Prompt panel methodology tests a broad, repeated set of real customer questions over time, while single-prompt testing checks one query once and A/B testing compares two content variants against a fixed outcome metric like click-through rate. The three approaches answer different questions and are frequently confused.

Single-prompt testing — typing one query into ChatGPT and noting the result — tells you nothing about consistency, because AI answers vary run to run. It is a spot check, useful for a quick sanity test but worthless as evidence of trend or improvement.

A/B testing, borrowed from conversion rate optimisation, assumes a stable measurement environment: show version A to half of users, version B to the other half, compare a clear metric. AI answer generation doesn't offer that stability — there's no "version B" of ChatGPT's output you can isolate, and the model itself introduces randomness independent of anything you change on your website.

Prompt panel methodology instead treats variance as the thing being measured, not noise to be eliminated. Instead of one query, it uses twenty to one hundred (sometimes thirty to fifty tested across five or more engines, per Arfadia, 2026). Instead of one run, it repeats each prompt — ideally at least seven times per prompt per engine for brand-presence tracking, per NeuralAdX (2026). The output isn't a single pass/fail — it's a mention-rate percentage with a confidence band attached.

Method	What it measures	Typical sample	Reliability
Single-prompt test	One answer, one moment	1 prompt, 1 run	Low — a snapshot, not a trend
A/B test (web content)	Conversion impact of a content change	2 variants, large traffic split	High, but doesn't fit AI answer variance
Prompt panel	Mention/citation consistency over time	20–100+ prompts, 7+ runs each, per NeuralAdX (2026)	Moderate-to-high, scales with runs and panel size

What are the step-by-step stages for setting up a prompt panel from scratch?

Setting up a prompt panel from scratch follows a defined sequence: define the question set, select engines, fix a run schedule, build a scoring rubric, assign reviewers, then analyse and report. Skipping any stage tends to produce numbers that look precise but aren't.

Draft the question set. Pull from real customer enquiries, sales call transcripts, support tickets and Google Search Console query data — not from guesswork. A prompt panel should be "a list of 20 to 50 questions chosen to mirror what real customers ask AI assistants", per WhatsGEO (2026).
Map prompts to funnel stage. Separate awareness-stage questions ("what is [category]") from decision-stage ones ("best [category] provider in [place]") so results can be segmented.
Select the engines. At minimum, cover the engines your customers actually use — commonly ChatGPT, Google AI Overviews and Perplexity for UK B2B audiences, extending to Claude, Gemini and Copilot for fuller coverage.
Fix the run cadence. Run each prompt at least seven times per engine for brand-presence measurement, and eight times for source-coverage measurement, spread across two to four weeks, per NeuralAdX (2026).
Build the scoring rubric (see below) and brief every panellist on it before the first review.
Freeze the panel. Lock the exact wording of every prompt. Changing wording mid-cycle invalidates the comparison to earlier runs.
Run, score, analyse, report. Produce a mention-rate trend, a citation-share breakdown by domain, and a short narrative of what changed and why.

Who should sit on a prompt panel and what expertise do they need?

A prompt panel needs a small, cross-functional group — typically a content or SEO lead, a subject-matter expert from the relevant service line, and someone with basic statistical literacy to interpret the scores. Seniority matters less than domain knowledge and consistency of judgement.

The content or SEO lead owns the rubric and the schedule, and is the point of continuity between review cycles — without this role, panels drift. The subject-matter expert (for example, a facilities manager scoring prompts about security services, or a compliance officer scoring prompts about regulated financial advice) catches factual errors or misleading claims in AI-generated answers that a generalist would miss. The analyst — this doesn't need to be a dedicated data scientist, but someone comfortable with percentages and basic variance — turns raw scores into a trend a board will trust.

For most UK SMEs, three to five panellists is workable. Larger organisations running panels across multiple product lines may need a panellist per line, coordinated by a single GEO or content owner who aggregates results centrally — which is the operating model Aether AI's own citation-tracking dashboard is built to support, replacing a manual spreadsheet exercise with automated scoring across the six tracked engines.

"A keyword is what someone types into a search box; a prompt is what they actually want, in a full sentence. Tracking prompts forces you to confront the real question set in your market — and most brands discover the questions they answer well are not the ones being asked." — Lauren Dawkins, Head of Content, Aether AI

How many panellists and prompt variations are statistically sensible?

The statistically sensible range for panel size sits between twenty and one hundred prompts, with each prompt repeated multiple times per engine rather than run once. Repetition count matters more than most teams assume, because AI output is genuinely inconsistent run to run.

That inconsistency is well documented: in Profound's tracking of 22.5 million ChatGPT Shopping offers, 95% of product titles appeared in under 30% of runs of the same prompt, according to Nick Lafferty (2026). A brand name could appear in one run and vanish in the next ten, purely from model variance — not because anything changed on the brand's website. A single run per prompt would therefore be close to meaningless.

The clearest guidance on repetition comes from a 2026 field study of 30,504 responses, which proposed at least seven runs per prompt and engine for brand-presence measurement, and eight runs for source-coverage measurement, spread across two to four weeks, according to NeuralAdX (2026). A panel of thirty to fifty prompts across five or more engines, as suggested by Arfadia (2026), combined with that repetition guidance, implies a defensible monthly cycle requires somewhere in the region of 1,000–2,000 individual prompt runs. For continuous, always-on tracking rather than a periodic audit, a weekly run of 100–200 prompts is described as the minimum viable system for AI share-of-voice measurement, per Digital Applied (2026).

On panellists specifically — the people scoring outputs, as distinct from the prompts being run — there's no published statistical standard, because this is a qualitative review step, not a sampling exercise. Three independent scorers per batch is a reasonable working minimum, since it allows a majority view when two agree and one differs.

What criteria and scoring rubric should panellists use?

A prompt panel scoring rubric should rate each AI answer on at least four dimensions: brand presence, citation accuracy, sentiment, and source quality. Scoring consistently across panellists is what turns subjective impressions into a usable dataset.

A practical rubric looks like this:

Brand presence — was the brand mentioned at all? (Yes/No, then: named directly, named indirectly, or absent)
Position — where did the mention appear in the answer (first paragraph, mid-answer, footnote-style list)?
Citation accuracy — did the engine link to the correct, current page, or an outdated or wrong one?
Sentiment — neutral, positive, negative or inaccurate/misleading framing
Source overlap — which other domains were cited alongside the brand in the same answer?

That last criterion matters more than it first appears. A July 2026 study of 161,286 prompts across ChatGPT, Gemini, Perplexity and Google AI Overviews found that only 3.8% of sources were shared by all four engines, with pairwise source overlap around 17%, according to NeuralAdX (2026). This means a panel that only checks one engine is structurally blind to most of the citation landscape — a brand cited heavily on Perplexity may be invisible on Google AI Overviews, and a rubric that doesn't record source overlap per engine will miss this entirely.

Score every answer against the same five-point checklist, every cycle, with no ad hoc additions mid-run — consistency across cycles is what makes the trend line meaningful.

How should disagreements between panellists be resolved and documented?

Disagreements between panellists should be resolved by majority scoring with a documented tie-breaker rule, and every disputed score should be logged with the reasoning, not just the final number. Undocumented overrides are the single most common way a panel's credibility quietly erodes.

A workable process: where three panellists score the same answer and two agree, the majority score stands. Where there's a genuine split (common with sentiment, less common with factual presence/absence), the content or SEO lead who owns the rubric makes the final call — but must record a one-line rationale in the panel log. Over several cycles, these logs become a useful record of where the rubric itself is ambiguous and needs tightening.

Common sources of disagreement include: whether an indirect mention (a product described without the brand name) counts as "presence"; whether a citation to an outdated page still counts as "accurate"; and whether a neutral-but-incomplete answer counts as negative sentiment. Resolving these inconsistently across cycles — rather than fixing the rubric wording once — is what causes month-on-month trend lines to look erratic for reasons that have nothing to do with actual AI visibility change.

Prompt panels raise UK GDPR considerations whenever the prompts or outputs contain personal data — for example, a customer's name, complaint detail, or identifiable case history used to construct a realistic test question. The Information Commissioner's Office (ICO) is the UK's data protection regulator, and its guidance on the UK GDPR applies in full to internal measurement exercises, not just customer-facing systems.

Three practical steps keep a panel compliant. First, anonymise or fictionalise any prompt built from a real customer enquiry before it enters the panel — swap names, locations and identifying detail for generic placeholders. Second, treat AI engine outputs with the same caution: if a panellist pastes a real customer's query into a public AI tool to see how it's handled, that query (and any personal data it contains) may be processed and retained by a third party outside the organisation's control, which is a data-sharing decision requiring the same scrutiny as any other third-party processor under UK GDPR. Third, document retention — panel logs, scores and raw outputs should have a defined deletion schedule, consistent with the organisation's existing data retention policy, rather than accumulating indefinitely in a shared spreadsheet.

Regulated sectors — financial services firms under Financial Conduct Authority (FCA) rules, or healthcare organisations handling anything that could intersect with patient data — should also check that prompt panel outputs don't get treated as marketing claims subject to separate advertising or regulatory standards, since an AI engine's phrasing of a regulated product is not something the brand controls directly.

Your prompt panel methodology checklist

Build a question set of 20–50 prompts drawn from real customer enquiries and search queries, per WhatsGEO (2026)
Select at least three AI engines to cover, given low cross-engine source overlap, per NeuralAdX (2026)
Repeat each prompt at least seven times per engine before drawing conclusions, per NeuralAdX (2026)
Freeze prompt wording for the full review cycle — never edit mid-run
Score every answer against a fixed rubric: presence, position, citation accuracy, sentiment, source overlap
Anonymise any prompt built from a real customer record before it enters the panel, consistent with UK GDPR
Log every panellist disagreement and its resolution, not just the final score
Report a trend line, not a single snapshot — one run proves nothing

FAQ

What is a prompt panel methodology?

Prompt panel methodology is a structured, repeatable process for testing a fixed set of questions against AI engines to measure how consistently a brand appears in generated answers. It differs from a one-off prompt check by repeating the same questions over weeks and tracking the mention rate as a trend, rather than a single observation.

How many prompts should a prompt panel contain?

Industry practice places panel size between twenty and one hundred prompts, with thirty to fifty tested across five or more engines suggested by some practitioners, according to Arfadia (2026). For continuous weekly tracking, Digital Applied (2026) describes 100–200 prompts run weekly as the minimum viable system.

Why does the same prompt give different answers each time it's run?

AI engines generate answers probabilistically, so wording, sources cited and even whether a brand is mentioned can vary between identical runs of the same prompt. This variance is substantial: 95% of product titles tracked across 22.5 million ChatGPT Shopping offers appeared in under 30% of repeated runs, according to Nick Lafferty (2026) — which is precisely why a single prompt test proves little on its own.

How often should each prompt be repeated for reliable results?

A 2026 field study based on 30,504 responses recommends at least seven runs per prompt and engine for brand-presence measurement, and eight runs for source-coverage measurement, spread across two to four weeks, according to NeuralAdX (2026).

Do I need to track every AI engine, or is one enough?

Tracking only one engine misses most of the picture. A July 2026 study of 161,286 prompts found only 3.8% of sources were shared across ChatGPT, Gemini, Perplexity and Google AI Overviews, with pairwise overlap around 17%, according to NeuralAdX (2026) — meaning a brand visible on one engine can be entirely absent from another.

Yes, if prompts or outputs contain personal data such as a real customer's name or case detail. The ICO's guidance on the UK GDPR requires that such data be anonymised, minimised and retained only as long as necessary, which applies to internal measurement exercises as much as customer-facing systems.

How is a prompt panel different from checking Google Search Console?

Google Search Console reports on traditional search clicks and impressions for queries Google attributes to a site, but it does not report what AI engines like ChatGPT or Perplexity say about a brand. A prompt panel is the separate, purpose-built method for measuring presence inside AI-generated answers, which Search Console was never designed to capture.

Running your prompt panel with Aether AI

Building and scoring a prompt panel by hand — drafting twenty to fifty questions, running each seven or more times across several engines, and logging every result in a spreadsheet — is exactly the manual workload that makes most UK teams give up on AI-visibility measurement before they start. Aether AI was built to remove that workload: it tracks citations automatically across six AI engines — ChatGPT, Perplexity, Google AI Overviews, Claude, Gemini and Copilot — so a prompt panel runs on a schedule rather than needing a human to re-type questions every week.

Aether AI's own measured baseline shows why this matters: across 700 live AI engine scans of UK brands' target keywords in July–August 2026, those brands were named in just 23 answers — 3.3% — with the longest-established brand appearing in 6.3% of its 270 scans. (Aether AI is built and operated by Aether Agency Ltd. Priority First, Pulse Operations and Aether Agency are businesses connected to the founding team.) That low baseline is typical of what a properly run panel reveals — and it's the gap Aether AI's clients close through ongoing tracking and content optimisation.

If your team wants to see where your own brand currently stands before committing to a full panel build, Aether AI's free AI-visibility audit at /audit gives a starting snapshot, with full pricing available at /pricing.