Should we be surprised when underdogs win penalty shootouts?

The reality may be closer than the commentary suggests

The World Cup has been captivating, with the first few weeks seeing a Messi hat-trick, an exciting underdog tale by debutants Cabo Verde, the inevitable intersection of flash mobs and rowboats, an unusual intervention and no shortage of penalty shootout surprises.

The large number of penalty shootouts adds to the drama, including Australia’s sad demise against Egypt. We had five shootouts in Qatar 2022, and (at the time of writing) have already had four this World Cup in the rounds of 32 and 16.

One commentary curiosity was the first match to feature a shootout – Germany versus Paraguay in the Round of 32. Many people strongly felt that Germany had the upper hand, given their unbeaten record in World Cups.  After the plucky Paraguayans held their nerve for the win, the shock was palpable. British commentator Guy Mowbray was quoted as saying “Hold the front page. Hold the back page. Germany have lost a shootout at the World Cup!”

This got me thinking (and not for the first time questioning the statistical rigour of sports commentators): was the result actually a surprise? Did Germany actually have the upper hand going into the shootout, and if so, by how much? Fortunately, this is something we can test statistically.

Data was sourced from Wikipedia , plus team strength ELO scores from Kaggle (measured before the respective tournament began). A dataset plus short analysis code is available on GitHub . The table below shows the results by country. There are only 35 penalty shootouts in World Cup history prior to 2026, so it’s a relatively small dataset.

Table 1 – World Cup penalty shootout results by country, excluding 2026

Is it unusual for some countries to be much better at penalty shootouts than others?

We can see from the table that some countries have impressive records, including Germany’s perfect 4-0 record. Others have a less stellar record, including Spain’s single win from five shootouts. But is the range of spreads unusual?

There are ways in the statistical toolkit to test this explicitly. A somewhat obscure one is Tarone’s C(α) test , which asks if the spread (or overdispersion) of the binomial variables is different from a specified rate (here a 50% win rate). Put simply, if everyone had 0% or 100% records and few people had win rates in the middle, this would be statistically unusual. Such overdispersion is actually a common feature of actuarial modelling, where particularly poor drivers seem to crash more than our models predict, leading to a larger spread of outcomes than a typical distribution would predict.

Applying the test gives a p-value of 0.04, which scrapes in as statistically significant under the common 0.05 threshold. So there appear to be countries that are unusually good and bad at shootouts.

Does the stronger team really have the advantage in a shootout?

A second question then - can we predict which countries are more likely to win a shootout? To do this, we test if the team with the higher ELO score (going into the World Cup) is more likely to win a shootout. In this case we find that the stronger-rated team wins 62% of the time, with a slightly-less-authoritative-but-still-suggestive p-value of 0.133. I take this as decent but not definitive evidence that the ‘stronger’ teams do in fact do better, Spain and England’s records notwithstanding.

Unsurprisingly, people have already studied this topic more thoroughly than I have and come to similar(ish) conclusions. For instance, this paper used a dataset of over 400 shootouts and suggests that a stronger team (as measured by market value of its players) might win up to 60% of its penalty shootouts, similar to the rate we observed for the World Cup.

So was Paraguay's win over Germany actually a surprise?

So were the commentators right to be so shocked? There is some room for surprise – it’s probably fair to say that Germany could be slightly favoured based on the evidence. But the real answer is that it’s not particularly surprising that Paraguay won; I reckon a weaker team still has about a 40% chance of winning a shootout, showing that anything can happen in the beautiful game.

And until we can make STATS101 courses compulsory for sports commentators, we can continue to enjoy their ride of emotions. Happy World Cup watching!