Learn/What the Council's Agreement Is Worth
Adoption

What the Council's Agreement Is WorthJune 2026

A council of AI advisors earns you real confidence only when the voices are independent — and five personas on one model are not.

Hybrid

Researched and drafted by Urania · Edited and verified by Zach Rossmiller

A prompt that circulated widely in 2026 tells one model to convene a council of five named advisors — a contrarian, a first-principles thinker, an expansionist, an outsider, an executor — let them argue, and return a single verdict. The promise is that the panel thinks better than the model alone: less flattery, fewer blind spots, sharper decisions. This guide takes the promise seriously and asks a narrower question with a clearer answer: what is a council's agreement actually worth? It draws on the research the pattern descends from — role and persona prompting, multi-agent debate, self-consistency, mixture-of-agents — and on the replication studies that tested whether the debate holds up. The throughline is independence: a council helps in proportion to how independent its voices really are.

In Plain Terms

The usual way to ask an AI for help is to ask once and read what comes back. The council prompt changes the shape of the request: instead of one answer, you ask the same model to play several characters — a skeptic, a builder, an outsider — who each weigh in, and then to settle their disagreement into a single recommendation. People who use it say the result is less of a yes-man and better at catching the thing they missed.

Two things are true at once, and holding both is the whole point. The first: asking for several points of view genuinely helps. When you tell a model to answer as a careful skeptic, it answers differently than when you tell it to answer as an eager builder — a role really does change what it notices. Asking for five roles and then a synthesis is a cheap way to make a single model look at a problem from angles it would otherwise skip. That is real, and it is worth doing.

The second: the five advisors are not five minds. They are one model wearing five hats. They were trained on the same data, they share the same instincts, and they fail in the same places. So when all five "agree," that agreement is cheap — it is one voice repeated five times in different costumes. It tells you the model is confident. It does not tell you the model is right. A confident wrong answer will be confidently wrong in all five voices.

This matters because of what people reach for a council to do. If you want to surface considerations, structure a messy decision, or get a first draft with its weak spots already flagged, a one-model council is excellent, and you supply the final judgment. But if you want the agreement itself to mean something — to count as a second opinion, a check, a reason to trust the answer — then you need the voices to actually be independent. That means different models, not one model in costume, and ideally something outside the models entirely — a test, a tool, a person — that can say no.

So this is not a debunking. The council is a real and useful pattern, and there is more than one way to build it. The job of this guide is to lay out the design space, from the cheap single-model version to genuinely independent ones, and to leave you able to build the right one for a given task and to read its agreement for exactly what it is worth.

Chapter One

A popular prompt, a big promise

Reading The pattern is a custom prompt, described here from public posts rather than cited; for the genuinely multi-model version it is often confused with, see Andrej Karpathy, llm-council.

1.1The prompt, and the seats

The shape is always the same. A single instruction tells the model never to answer in one voice; instead it is to spin up a panel of advisors, each assigned a distinct angle, who comment in turn before the model closes with one verdict. The seats vary, but a common roster runs: a contrarian who attacks the weakest link, a first-principles thinker who ignores how the question was framed, an expansionist who hunts for upside, an outsider who notices what familiarity hides, and an executor who names the next concrete step. Often the prompt is saved once and reused — pasted into a project's custom instructions so the council persists across a whole thread of questions.

It is worth being plain about one thing the marketing around this pattern tends to blur: there is no built-in "council" feature. The setup is an ordinary prompt placed in an existing custom-instructions field. Nothing new is switched on; the model is simply told to organize its answer as a panel. That is not a criticism — it is the first clue to what the council can and cannot be, because everything the panel does happens inside one model, in one pass.

1.2What it claims to buy

The claims are reasonable on their face. A council is said to reduce sycophancy, because at least one seat is told to disagree; to surface blind spots, because the outsider and the contrarian are paid to look where you don't; and to stress-test a plan, because the executor has to turn talk into a step. Stripped of the hype, the honest version of the claim is testable: forcing multiple explicit perspectives and then reconciling them produces a better answer than asking once. The rest of this guide tests exactly that — and finds it is true, with a precise and important limit on what the "agreement" at the end is worth.

Check your understanding

Pick a real decision you might run through a council. Before reading on, decide which of two things you'd want from it: a wider set of angles you will judge yourself, or an answer whose agreement you could actually lean on. Hold that distinction — the whole guide turns on it.

Chapter Two

Why it helps at all

Reading Anthropic, Giving Claude a role with a system prompt; Wang et al., Unleashing the Emergent Cognitive Synergy in LLMs (NAACL 2024).

2.1A role changes what the model sees

Start with the part that is well established. Giving a model a role is one of the most reliable ways to steer it; Anthropic calls role prompting the single most powerful use of a system prompt, the thing that turns a general assistant into a domain expert. The detail that matters for councils is in their own example: a data scientist and a marketing strategist, handed the identical dataset, surface different things. The role is not decoration — it changes which features of a problem the model attends to. So asking one model to answer as five deliberately different experts is a cheap way to walk it past five different sets of attention it would otherwise collapse into one.

2.2Cognitive synergy, and its limits

This has been studied directly. In work introducing what its authors call Solo Performance Prompting, a single model is prompted to identify several fine-grained personas, let them collaborate over a few turns, and synthesize — and the result beats both plain prompting and chain-of-thought on tasks that mix knowledge and reasoning, while reducing factual hallucination. That is the research backbone under the council prompt, and it is genuine: multiple personas, well chosen, did better than one.

The same paper draws the limit just as clearly, and the council's promoters rarely mention it. The gains showed up on three narrow, constructed tasks — a trivia-writing task, a word-association game, a logic puzzle — not open-ended life or business decisions. And the effect was emergent only in the strongest model available: the persona collaboration helped GPT-4, and did not help less capable models like GPT-3.5 or a mid-size Llama. So the technique is real but conditional. It rewards a strong model on a suitable task, and it is a way of organizing thinking — not a guarantee, and not free of the failure mode the next chapter is about.

Practice

Take one question you care about. Ask a strong model to answer it plainly. Then ask it to answer as three named experts who disagree, and reconcile them. Compare. You are looking for whether the second answer surfaced a consideration the first one skipped — that is the cognitive-synergy effect, and also its ceiling.

Chapter Three

What its agreement is worth

Reading Synthesis of Chapters 1–2; the claim made here is evidenced directly in Chapter 4. See Du et al., Multiagent Debate and Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet.

3.1One model, one set of blind spots

Here is the catch the council prompt cannot escape. The five advisors are all generated by the same model — the same weights, trained on the same data, carrying the same priors and the same gaps. Their mistakes are not independent; they are correlated. When the contrarian and the executor and the outsider all land in the same place, that convergence is not five witnesses corroborating a fact. It is one witness, repeating itself in five registers.

3.2Agreement is not confirmation

The picture above is the whole argument in one image. Independent estimates that scatter around the truth average toward it — that is why a real second opinion is worth having. Correlated estimates that cluster tightly can sit far from the truth and never reveal it, because nothing in the cluster points elsewhere. A single-model council produces the tight cluster. Its consensus is a measure of the model's confidence, not of the answer's correctness.

So the rule to carry: treat a one-model council's agreement as organized thinking, not as a check. It has done you the service of laying out the considerations and pressuring the weak ones. It has not given you a second opinion, no matter how many voices signed the verdict. To get something that can disagree for real, you have to change the architecture — which is the back half of this guide.

Check your understanding

Recall a time you were confidently wrong about something in your own field. Now ask: would five personas of a single model — all trained on roughly the consensus you shared — have caught it? Usually not. The error you couldn't see is precisely the one a copy of your own priors also can't see.

Chapter Four

The debate, examined

Reading Du et al., Multiagent Debate (ICML 2024); Huang et al., LLMs Cannot Self-Correct Reasoning Yet; Smit et al., Should We Be Going MAD?; If Multi-Agent Debate Is the Answer, What Is the Question?; Wang et al., Self-Consistency.

4.1The debate that started it

The strongest case for the pattern comes from a 2023 result on multi-agent debate. Run several copies of one model, have each propose an answer, then let them read one another's reasoning and revise over a couple of rounds, and the final answer is more accurate on math and reasoning than a single pass — a "society of minds" that talks itself toward the truth. The setup is modest: three agents, two rounds, identical model. It is the most credible reason to believe a council of voices beats one voice.

4.2What replication found

Then people checked it against the right baseline. The natural comparison is not a single greedy answer but self-consistency: sample several independent answers from the model and take the majority vote. When researchers replicated the debate with that comparison held at an equal number of responses, multi-agent debate did not pull ahead — it underperformed plain majority voting. Their conclusion was deflating and precise: what looks like "debate" is better understood as a way of reaching consistency across several generations, not a genuine critique. Broader sweeps reached the same place: across many methods, benchmarks, and models, same-model debate did not reliably beat chain-of-thought or self-consistency even when it spent considerably more compute, and the one factor that helped was using different models rather than copies of one.

4.3The honest read

Put the two findings together. The persona council and the debate council both help a single model, and both help for the same humble reason: they push it to consider more than one line of reasoning and then aggregate. That is roughly what self-consistency already does, more cheaply. The ceiling on a one-model council — whether its voices are called personas or agents — is structured self-consistency. Useful, often worth it, and not what the word "debate" advertises. Nothing here is independent deliberation. To get that, the voices have to come from somewhere the model's own priors don't reach.

Chapter Five

Building it across models

Reading If Multi-Agent Debate Is the Answer, What Is the Question?; Wang et al., Mixture-of-Agents; Andrej Karpathy, llm-council.

5.1The lever is heterogeneity

The same skeptical literature that deflates same-model debate points straight at what does work: use different models. When agents are drawn from genuinely different systems, their errors decorrelate, and the aggregate can beat any one of them. The clearest demonstration is mixture-of-agents, where a layered ensemble of several open-source models — explicitly different from one another — outscored a leading single model on a standard preference benchmark, roughly sixty-five percent to fifty-eight. The authors name the effect the "collaborativeness" of language models: a model given other models' answers tends to produce a better one. The lever is not arguing harder. It is being genuinely different.

5.2A council across models

The cleanest worked example of this is a small open-source project, an "LLM council," that wires the idea into three plain stages. First, the same question goes to several different frontier models in parallel, and each answers on its own. Second, every model is shown all the answers — anonymized, so it can't play favorites — and ranks them. Third, a designated chairman model reads the answers and the rankings and writes one synthesis. It is a single pass, not a debate: answer, rank once, synthesize.

Three lines capture the shape, and one warning lives inside them:

answers  = [ask(m, question)               for m in COUNCIL]   # 1. independent answers
rankings = [rank(m, anonymize(answers))    for m in COUNCIL]   # 2. blind peer review
final    = ask(CHAIRMAN, synth(question, answers, rankings))   # 3. one synthesis
The multi-model council in three stages — the shape of the open-source council.py. COUNCIL is a list of different models; the independence lives entirely in that list.

The warning: heterogeneity here is a configuration choice, not a property of the design. Fill COUNCIL with five different models and you have a real council. Fill it with the same model five times — or run the viral prompt, which is exactly that — and you are back at the left of the spectrum, paying more for the same correlated answer. The architecture only buys independence if you actually put independent things in it.

5.3The check that can say no

Even independent models can agree and still be wrong; convergence raises confidence, it does not guarantee truth. The strongest councils add a voice that is not a model at all — a test that runs, a scanner that flags, a tool that checks, a person who signs off. This is the maker–checker split: the thing producing the answer is not the thing allowed to certify it. A non-model check is the only voice in the room that can say no for a reason the models don't share.

ZR's own AI Production Readiness Framework (AIF) is this pattern in production, and it tunes it tellingly. It runs five independent models over the same code, but it does not have them rank each other; instead it counts convergence directly — when three or more agree, a finding is marked high-confidence — and it backs the models with deterministic scanners that flag issues on their own authority. The models supply independent opinion; the scanners supply the hard no. Agreement is treated as evidence precisely because the voices are independent, and it is never the last word.

Chapter Six

Building your council

Reading Synthesis of Chapters 2–5, with implementation grounded in Andrej Karpathy, llm-council, and the LibreChat agent and endpoint documentation.

6.1Two councils, two machines

The councils worth building are two different kinds of thing, and conflating them is most of the confusion in this whole topic. The single-model council is a prompt: text you give to one model, asking it to play several advisors and settle them into one answer. The multi-model council is a harness: code that sends your question to several genuinely different models at once, collects their answers, and reconciles them. One is something you write. The other is something you run.

Reach for the prompt when you are the one judging the answer and you want the model to widen the field before it commits. Reach for the harness when the agreement itself has to mean something, because only the harness gives you voices that can genuinely disagree. The rest of this chapter builds each, with the evidence for when the harness earns its extra cost.

6.2The single-model council is a prompt

There is no feature to switch on. The single-model council is a system prompt — the five-advisor instruction below, written once and reused. It lives wherever you keep instructions: a saved preset, a project's custom instructions, or, if you run your own stack, a LibreChat preset or a no-code Agent whose instructions are the council and whose model is whatever you point it at. One model call, no infrastructure.

You are a council of five advisors. Answer in two passes.

PASS 1 — each advisor speaks once, in its own voice:
Skeptic — the weakest assumption and the most likely failure.
First-principles — ignore how I framed it; what is the real problem?
Builder — the single concrete next step.
Outsider — what someone from another field would notice.
User's advocate — what the person on the receiving end actually needs.

PASS 2 — weigh the five, drop the weak points, give one verdict.

Then, separately: mark every claim you are not sure of, and name the one
thing here that a test, a second model, or a person should check before I act.
The single-model council. Save it as a preset or an agent's instructions. Excellent for options and structure; its agreement is your thinking organized, not a second opinion.

Use it for what it's good at: generating options, structuring a messy decision, drafting with the weak spots pre-flagged, breaking your own framing — the work where you hold the judgment and want a wider field before you decide. Its ceiling is the one Chapter 4 measured: a single model arguing with itself tops out around structured self-consistency — useful, not independent. And recall the bound from Chapter 2: the persona gains appeared only in the strongest models, on narrow tasks. What it cannot do is hand you a second opinion. Five hats on one model share one set of blind spots, so its agreement is your own reasoning organized — never a check on it.

6.3The multi-model council is a harness

When the agreement has to carry weight, you need voices that can actually disagree — different models, not one model in costume. That is not a prompt; it's a small orchestration, and the cleanest reference is Karpathy's llm-council: about four hundred lines that do three things. It sends your question to several different models in parallel; it shows each model the others' answers, anonymized so none can play favorites, and has them rank; and a chairman model compiles the result into one response.

Two implementation choices carry the whole thing, and both are worth stealing. First, it reaches the models through OpenRouter — a single API that fronts models from OpenAI, Google, Anthropic, xAI, and others — so adding or swapping a council member is a one-line config change, not a new SDK. Treat models as commoditized endpoints behind one gateway; don't hand-write five vendor integrations. Second, the calls run concurrently, so the council's latency is the slowest model, not the sum — the difference between a few seconds and most of a minute.

import asyncio, os
from openai import AsyncOpenAI                  # OpenRouter speaks the OpenAI API
client = AsyncOpenAI(base_url="https://openrouter.ai/api/v1", api_key=os.environ["OPENROUTER_API_KEY"])

COUNCIL  = ["openai/gpt-5.1", "google/gemini-3-pro", "anthropic/claude-sonnet-4.5"]
CHAIRMAN = "google/gemini-3-pro"

async def ask(model, prompt):
    msg  = [{"role": "user", "content": prompt}]
    resp = await client.chat.completions.create(model=model, messages=msg)
    return resp.choices[0].message.content

async def council(question):
    answers = await asyncio.gather(*(ask(m, question) for m in COUNCIL))   # parallel: latency = slowest
    bloc = largest_agreeing(answers)        # peer-rank (Karpathy) or simply count the bloc
    if len(bloc) < QUORUM:
        return escalate(answers)          # a genuine split goes to a human
    return await ask(CHAIRMAN, brief(answers))   # the chairman synthesizes the winners
The multi-model council — the shape of Karpathy's council.py. OpenRouter is one API to many models; asyncio.gather fans them out in parallel. The independence lives entirely in the COUNCIL list: make it genuinely different models.

You can keep Karpathy's anonymized peer-ranking, or do the simpler and often-sufficient thing the skeleton hints at: count the agreement. Cluster the answers and let the size of the largest agreeing bloc be your confidence — a near-unanimous council is real evidence; a genuine split is the finding, and should go to a person rather than be averaged away. Either way, the reason to pay for this is decorrelation: independent models fail in different places, so their convergence carries information a single model's self-agreement never can. That isn't a hunch — a heterogeneous ensemble of open models has beaten a strong single model on a standard benchmark (mixture-of-agents, roughly sixty-five to fifty-eight), and the systematic read of the debate literature is that heterogeneity, not more rounds, is the lever. The cost is real: several model calls per question instead of one, plus the orchestration to keep. Spend it when a wrong answer is expensive enough to be worth catching.

6.4Where it runs: LibreChat, and its edge

If your stack is LibreChat, the dividing line falls somewhere useful. The single-model council fits natively: save it as a preset, or build it as an Agent — pick a model, paste the council instructions, done. No code. LibreChat will also happily talk to every model the harness needs; point a custom endpoint at OpenRouter and the whole roster is available in one place:

endpoints:
  custom:
    - name: "OpenRouter"
      apiKey: "${OPENROUTER_API_KEY}"
      baseURL: "https://openrouter.ai/api/v1"
      models:
        default: ["openai/gpt-5.1", "anthropic/claude-sonnet-4.5", "google/gemini-3-pro"]
An OpenRouter custom endpoint in librechat.yaml — every council member reachable in the UI through one gateway.

But the council logic — fan out, rank or count, synthesize — is not something the UI does for you. LibreChat's Agent Chains run agents in sequence, not the parallel fan-out-and-compare a council needs; parallel multi-agent execution is, as of this writing, a requested feature rather than a shipped one. So the honest pattern is: run the harness (council.py, or your own) as its own service, and if you want it inside LibreChat, wrap it as an MCP server or an Agent Action and let LibreChat be the front door that calls it. The prompt lives in the UI; the council that has to be independent lives in code the UI calls.

6.5The check that lets it ship

Independence raises confidence; it does not guarantee truth. Even genuinely different models can converge on the same wrong answer. When an answer has to be correct rather than merely well-considered, you seat one voice that is not a model at all — a test that runs, a scanner that flags, a person who signs — and give it the veto. This is the maker–checker split made structural: the thing that produces the answer never certifies it, and the flow stops at the check.

This is exactly what AIF does in production: five independent models supply the opinion, their convergence sets a confidence tier, and deterministic scanners — not models — supply the hard no, so nothing ships on consensus alone. In skeleton, every checked council is the same shape: a council proposes, a non-model check disposes.

candidate = council(question)      # independent models converge on an answer
report    = check(candidate)       # a test, a scanner, or a person — not a model

if report.passes:
    ship(candidate)
else:
    revise(candidate, report)    # consensus alone never ships
The checked council. The council proposes; a check that is not a model disposes. Only that check can veto the consensus.

6.6How far to climb

The three councils are a ladder, and the rule is to climb only as far as the stakes require. Start with the prompt: it's free, instant, and right for the large share of work where you are the judge and just want a wider field. Build the harness when a wrong answer is expensive enough to pay for genuine independence — when you need the agreement to be evidence, not atmosphere. Add the check when wrong is not an option, and let nothing ship on agreement alone.

Most real work spans more than one rung — a little exploration, a real decision, a few places where being wrong is costly — so most real systems use more than one council, matched to the part of the task in front of them. The spectrum isn't a menu you pick from once; it's a gauge you read per task against a single question: what would it cost to be confidently wrong here? Build up to that, and no further.

Practice · put it together

Take the decision from Chapter 1 and price the downside: what would it cost to be confidently wrong about it? Let that set your rung. If the cost is low, write the single-model prompt and use it. If it's high, sketch the harness — which three or four different models, ranked or counted, and what would the chairman see? If wrong is unacceptable, name the non-model check that holds the veto. Then build the lowest rung that covers the cost, and write down the one error that could still get through it.

Reference

Glossary

AI council
A prompt or pipeline that gathers several "advisors" on a question and reconciles them into one answer. The voices may be personas of one model or genuinely separate models — a difference this guide argues is decisive.
Role prompting
Assigning a model an expert identity to steer what it attends to. A reliable lever: the same input read "as a data scientist" and "as a strategist" surfaces different things.
Cognitive synergy (SPP)
Solo Performance Prompting: one model simulating multiple personas that collaborate and synthesize. Improves some tasks over single-pass — but only emerged in the strongest models tested.
Multi-agent debate (MAD)
Several model instances propose answers, read each other's, and revise over rounds. Improves on a naive single pass; often does not beat self-consistency at equal compute.
Self-consistency
Sample several independent answers from a model and take the majority vote. The cheap aggregation baseline that same-model debate is really competing with.
Mixture-of-agents (MoA)
A layered ensemble of different models, each refining on the others' outputs. Evidence that heterogeneity — not argument — is what lifts the aggregate.
Model heterogeneity
Using genuinely different models rather than copies of one. The property that lets a council's errors decorrelate and its agreement carry weight.
Error correlation
The tendency of voices drawn from the same model to be wrong in the same way. Why a single-model council's agreement is cheap.
Chairman
A final model that reads all answers (and any rankings) and writes one synthesis. The synthesis step of a multi-model council.
Maker–checker
The discipline of separating the thing that produces an answer from the thing allowed to certify it. The "check that can say no."
Sycophancy
A model's tendency to agree with and flatter the user. Councils aim to reduce it by seating a dissenter — but a dissenter sharing the model's priors only goes so far.
Sources

Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei & Heng Ji, Unleashing the Emergent Cognitive Synergy in Large Language Models, NAACL 2024. · Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum & Igor Mordatch, Improving Factuality and Reasoning in Language Models through Multiagent Debate, ICML 2024. · Jie Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet (2023). · Andries Smit et al., Should We Be Going MAD? A Look at Multi-Agent Debate Strategies for LLMs, ICML 2024. · If Multi-Agent Debate Is the Answer, What Is the Question? (2025). · Xuezhi Wang et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models, ICLR 2023. · Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang & James Zou, Mixture-of-Agents Enhances Large Language Model Capabilities (2024). · Anthropic, Giving Claude a role with a system prompt. · Andrej Karpathy, llm-council.

An unofficial study text. Commentary synthesized in original wording; every figure is conceptual or drawn from the cited research and was verified against it. Don't cite Urania or this guide — cite the primary sources above. Researched and drafted by Urania, an AI research system; edited, verified, and signed by Zach Rossmiller, who is accountable for what's published.