A hundred AI agents that count as three

November 9, 2026

More and more solutions combine several language models and take the majority answer. This works less well than it might seem when all of them get the same questions wrong. A scientific paper by Telefónica Tech and the University of Valladolid explains why, and how to measure it.

______

If you want to form an opinion on a subject and ask ten friends, it seems as though you have ten opinions. But if all ten read the same newspaper, you have effectively asked just one source: they will give you the same version, with the same things right and the same things wrong, and the fact that they agree tells you nothing new.

This situation is more common than it might seem in generative AI. A widely used approach is not to rely on a single model: several models are asked, or the same model is asked several times, and the most common answer is taken as correct. If five agents agree, it seems reasonable to trust the result.

The problem is that many of those models are, so to speak, reading the same newspaper. When their errors are independent, they can offset one another. When they all fail at the same time, the majority does not correct the error: it confirms it.

The majority gets it right when each voter gets things wrong in a different way, not when everyone makes the same mistakes.

The small print of the wisdom of crowds

The mathematical justification for voting comes from Condorcet's Jury Theorem, formulated in 1785. Put simply, it says that if each voter is correct more than half the time and votes independently, the majority is more likely to be correct than any one of them, with its accuracy approaching 100% as more voters are added.

The important condition is the second one: independence. And that is where the problem lies. Two copies of the same model share the same training data, the same fine-tuning and, often, the same instructions or prompt. Even models from different companies have been trained on much of the same content from the internet. Recent studies comparing hundreds of models confirm that they tend to fail on the same questions.

Voting across models only improves reliability if their errors are at least partly independent.

How many votes really count

In the paper, published in the scientific journal Discover Informatics, we measure this dependence using a single number: the correlation between the agents' errors. It is 0 when one AI agent getting a question wrong tells us nothing about whether another will fail, and 1 when they always get the same questions right and wrong together.

From this, we can calculate something very useful for anyone responsible for an AI project: the effective number of voters, meaning how many genuinely independent agents the group is equivalent to. The result is striking: with a correlation of 0.3, one hundred agents are equivalent to around three independent voters. And no matter how many more agents are added, the group will never exceed that figure.

The effective number of independent voters represented by N agents (horizontal axis), depending on the correlation between their errors (ρ). If they are independent (ρ = 0), every agent adds another effective vote; with ρ = 0.3, the curve levels off at around three, no matter how many agents are added. Source: Morante and Romero García-Rubio (2026), Discover Informatics, fig. 1 (CC BY 4.0).

There is another, less intuitive consequence. With correlated errors, adding agents does not push majority-vote accuracy towards 100%. Instead, it plateaus below that level and, in some cases, can remain below the accuracy of a single model. What is more, when the majority is wrong, it is wrong with greater confidence: consensus reinforces the error instead of correcting it.

■ The number of agents alone tells us very little about the reliability of a multi-agent system. What matters is how similar their errors are.

A hundred AI agents that make the same mistakes can be worth no more than three that think independently.

What we found when we measured it using real models

To test whether this could be measured in practice, we evaluated three teams of five agents using 562 MMLU questions, a benchmark covering 57 subjects ranging from law to chemistry:

Five samples from the same model, varying the randomness of the response, known as temperature.
Five “personas” of the same model, using prompts that assigned a different profile to each one.
Five models from five different developers (OpenAI, Mistral AI, DeepSeek, Moonshot AI and xAI).

The first two teams, which appear diverse at first glance, turned out to be almost identical: their error correlation was 0.85 and each team was, in practice, worth little more than a single independent voter. Majority voting did not outperform the best individual member. The five samples from the same model all failed on the same 71 questions, 12.6% of the total.

The team made up of different models behaved differently. Its error correlation fell to 0.38, making it equivalent to around two independent voters, and its majority vote answered 92.3% of the questions correctly, 8.4 percentage points more than the baseline model. All five models failed together on only 14 questions, or 2.5%.

Even so, there was a useful surprise: the best model in the group achieved 94% accuracy on its own, outperforming the majority vote. When models have very different performance levels, giving every vote the same weight can penalise the strongest model.

Left: correlation between the errors of each five-agent team (Temperature: samples from the same model; Persona: different profiles of the same model; Family: models from five different developers). Right: majority-vote accuracy as samples from the same model are added (blue), compared with the vote across five different models (green) and the best individual model (dotted line). Source: Morante and Romero García-Rubio (2026), Discover Informatics, fig. 3 (CC BY 4.0).

■ This is a limited proof of concept: a single benchmark and answers consisting of just one letter. But it shows that the measurement is straightforward and inexpensive using the evaluation sets that any serious project should already have.

Changing the temperature or the “persona” does not create real diversity. Using different models does.

Real diversity versus cosmetic diversity

All of this leads to what we believe is the most useful distinction for anyone designing these systems:

Cosmetic diversity: more samples from the same model, a different temperature or a different tone in the prompt. The answers may look different, but this does not change which questions the system gets wrong.
Structural diversity: different base models, trained with different data and objectives. This can correct errors because different model families tend to fail on different questions.

■ Nor is it enough to replace voting with another model that reads the others' arguments and produces a synthesis. If the arguments it receives share the same bias, the synthesis may turn that bias into a fluent and convincing narrative, making it harder to audit than a simple vote count.

Independence between agents should not be assumed: it needs to be designed and measured.

What to ask before trusting a multi-agent system

The paper proposes a six-point checklist for anyone deploying these systems. In practical terms, these are questions that any company can ask its provider, or its own team:

How similar are the agents' errors? This should be measured using a set of questions with known answers.
How many independent agents is the system actually equivalent to? The relevant figure is not how many agents there are, but how many effective votes they represent.
Does the ensemble outperform the best individual model? If it does not, the additional cost may not be justified.
Where does the diversity come from? Different models or variations of the same one?
How are the answers combined, and is there an audit trail? This is particularly important when one model summarises the others.
Is the risk of a shared error acceptable for the type of decision the system is supporting?

■ When the aim is to detect errors, such as in contract review, risk analysis or support for critical decisions, it is worth paying for independence. When consistency is more important, correlation may be acceptable, but it should be a conscious decision.

An easy consensus between similar models may be redundancy, not truth.

Ten friends who read the same newspaper are not ten sources, and five models that learnt from the same data are not five votes. In a team of language models, diversity of perspective does not come automatically: it has to be built and verified. Adding another model to the table does not always add another vote. Sometimes it only adds an echo.

FURTHER READING
Morante, S. and Romero García-Rubio, M. (2026). Correlation limits the reliability of ensemble decision making with large language models.
Discover Informatics, 1, 35.
Open access: https://doi.org/10.1007/s44564-026-00040-z