"Meaningless"; "Uncompetitive". Wonderful pejoratives, tossed around like confetti to describe potential world cup formats, and fixtures against weak nations over the past week. But can anyone define them?

I have, in the past tried to define both 'meaning', and 'competitiveness', but not applied them in a way to compare formats. This aim of this post is to do so.

Competitiveness is the easier of the two to define. The simplest method is to calculate the expected margin between any two teams. Even teams will be at 0, the rest of the differences can be calculated from global ODI rankings idealised for up to 32 teams. Calculating every permutation of group and competitor is laborous, so I have simplified it slightly (but I think correctly) by calculating the average quality of each seeding tier for groups of different sizes as follows:

	Number of groups
Teams per group	1	2	4	6	8
1	1200	1200	1175	1141.67	1081.25
2	1200	1150	987.5	808.33	706.25
3	1150	1075	762.5	625	512.5
4	1150	900	650	491.67	381.25
5	1100	800	550
6	1050	725	475
7	950	675	412.5
8	850	625	350
9	800
10	800

Thus, in an 32 team cup, the average group will have a team of rating 1081, one of 706, one of 512, and one of 381. We can then calculate the competitiveness of each group stage if the top n seeds make that round. Upsets in earlier rounds or qualifying would mean the actual competitiveness factor will be slightly lower, but it ought to affect each relatively evenly. I've calculated the average difference by using the standard deviation, rather than the average sum of differences for all teams (which is much harder) , but it should be fairly similar. The conversion factor from rating difference to an expected margin is 0.2. It can be seen that the average margin for most combinations sits somewhere between 0.5 and 1.0 of the standard deviation of actual margins (60 runs).

group x teams	std. dev.	exp. margin	avg. quality
1x10	162.02	32.4	1025
1x8	125.18	25.04	1081.25
1x6	58.45	11.69	1141.67
2x7	211.01	42.2	932.14
2x6	194.94	38.99	975
4x5	254.49	50.9	825
8x4	304.75	60.95	670.31
6x4	281.69	56.34	766.67
4x4	234.19	46.84	893.75
2x4	131.3	26.26	1081.25
4x3	206.53	41.31	975
8x2	265.17	53.03	893.75
4x2	132.58	26.52	1081.25
4x2 (5-12)	151.38	30.28	875
2x2	35.36	7.07	1175
1x2	0	0	1200

Twelve formats have been chosen for consideration. By multiplying the number of games played in each combination by the expected margin, and dividing by the total number of games in the tournament, we can calculate an average expected margin and rating quality for each format.

no. teams	format	avg. margin	avg. quality
12 team	(2x6,1x6,2x2,1x2)	28.17	1015.63
12 team	(4x3,2x4,2x2,1x2)	30.55	1045.37
10 team	(1x10,2x2,1x2)	30.67	1034.9
12 team	(4x3,4x2,2x2,1x2)	32.42	1030.26
16 team	(4x4,1x8,2x2,1x2)	33.44	1005
16 team	(4x4,2x4,2x2,1x2)	37.27	973.72
14 team	(2x7,4x2,2x2,1x2)	38.63	959.69
16 team	(4x4,4x2,2x2,1x2)	40.14	945.97
20 team	(4x5,4x2(5-12),4x2,2x2,1x2)	44.65	870.1
24 team	(6x4,4x3,4x2,2x2,1x2)	48.07	857.73
24 team	(6x4,4x3,2x2,1x2)	49.77	840.2
32 team	(8x4,8x2,4x2,2x2,1x2)	55.08	749.21

The format used in the 1999 world cup has the lowest average margin. Sort of anyway, the model doesn't adjust for skipped games in the super-6 stage that push it back to 30.52. The format used in the WT20 is second-best which shows the ICC is doing something right. There is very little difference between most models. Obviously competitiveness gets worse as more teams are added, but the change in expected probability of victory for a 10 team competition (0.70) to a 20 team competition (0.77) is only 7%. Until a tournament reduces to 6 teams or fewer, the probability of an uneven contest is fairly stable.

Meaning is a little more difficult. For this I am going to use a precise mathematical definition:

A game's "meaning" is the change in probability that a team will qualify for the next round based on the result of a single game.

For a knockout game between even sides, the change in probability is 0.5: each team begins with a 50% chance of progressing and ends with either 0% or 100% chance. For a four team group with two qualifiers, the probability of progressing begins at 0.5. A team needs 2 wins to progress so the probability of progressing becomes 0.75 after a win and 0.25 after a loss (again assuming even contests). In practise it is slightly more complex than this as a team might not progress with 2 wins, or progress with 1 win (in 3% of cases). The difference is miniscule and roughly even across group sizes, so for the sake of simplification it is being ignored.

Three adjustments were made however:

• The probability change depends on the teams playing. If a team has a 70% chance of winning, it has a 70% chance of a 0.3 change, and a 30% of a 0.7 change, which gives an average change of 0.42. At 90% that drops to 0.18. Even contests are more "meaningful". To adjust for this, a factor was introduced to the account for larger cups have more uneven groups, in line with the calculations above.
• A factor to account for upsets in secondary group stages, as happened in 1999 (Zimbabwe), 2003 (Zimbabwe, Kenya) and 2007 (Bangladesh, Ireland). This was set at 0.8.
• For the 20 team cup, outlined here, the winner of the group receives a bye through the next stage. To reflect that, top spot has the change in probability for that stage added to it (0.5)

no. teams	format	avg. chg. prob.	no. games	% knockouts
12 team	(4x3,4x2,2x2,1x2)	34.2%	19	36.8%
16 team	(4x4,4x2,2x2,1x2)	25.8%	31	22.6%
12 team	(4x3,2x4,2x2,1x2)	25.6%	27	11.1%
16 team	(4x4,2x4,2x2,1x2)	21.5%	39	17.9%
20 team	(4x5,4x2(5-12),4x2,2x2,1x2)	21.1%	51	21.6%
24 team	(6x4,4x3,4x2,2x2,1x2)	21.0%	55	12.7%
12 team	(2x6,1x6,2x2,1x2)	19.5%	48	6.3%
24 team	(6x4,4x3,2x2,1x2)	18.8%	51	13.7%
14 team	(2x7,4x2,2x2,1x2)	18.6%	49	14.3%
16 team	(4x4,1x8,2x2,1x2)	17.3%	55	5.5%
32 team	(8x4,8x2,4x2,2x2,1x2)	17.2%	63	23.8%
10 team	(1x10,2x2,1x2)	13.4%	48	6.3%

Needless to say there is a strong correlation between meaning and the number of games, particularly the number of knockout matches. The sooner a team faces elimination, the more likely the games are to be meaningful. Far from being the most "meaningful" format, the 10 team world cup fares particularly poorly, each match being roughly the equivalent of a knockout between two teams where one was 97% likely to win.

We can relate these two factors by comparing them to a baseline factor. 60 runs for competitiveness and 12.5% for meaning, then graphing them against each other. Better, obviously, is furthest from the xy-origin of 1.

Previous world cups are marked in green, cups that don't meet the media criteria of 48 games in red. 12 team tournaments with squares, the biggest and smallest with triangles, and serious proposals with larger boxes.

It is worth noting here that how you interpret the graph depends on what you want from a world cup. If competitiveness is the only criteria, then it runs 12,10,16,14,20. But that, perversely, rates the 2007 cup over the 2011, when it clearly wasn't, even though both had a similar number of upsets.

I think competitiveness is a valuable measure, which is why it is included in my interpretation of meaning, via the adjustment from equal, to unequal groups (the graph heavily favours knock-out heavy competitions without the adjustment). But meaning conforms much more closely with what we have seen, the 12 team cup was good, but not a lot better than the 14, or the unfortunate 16 team editions. And a 10-team cup, while competitive, would in reality be a tedious bore, as game after game would be played for small odds, and only a handful of concluding matches having a real effect on qualification.

Obviously the ICC and its media partners have an interest in maintaining the presence of certain sides in the competition. But let's not pretend it adds "meaning". By using a defintion of "meaning" that we can implicitly understand - that a game is meaningful when it affects the expectations we had of it - it can be seen that a 10-team world cup would have been the most meaningless yet contrived.

Cricket - Articles 5th July, 2011 15:17:11   [#] 

Comments

Quantifying World Cup Formats
Going back to the previous article, on the format, I missed it earlier. Having seen it now, I agree with the comments that 20 team event is suitable for T20 wcup

For ODI version, I think we can try to tighten things with a 15 team cup (only 3 groups of 5 in initial league stages). The 3 winners can go direct to semis and the 3 winners of the 2nd round play-offs can play among themselves to identify the 4th semifinalist. Earlier in Roar, I had suggested a 5-team super league with winner going to finals directly and 2-3 fighting out in lone SF. But the SuperLeague concept had no takers.

How would the 2 15-team formats fare on the competitiveness/meaningful scale?
marees  7th July, 2011 05:45:47  

Quantifying World Cup Formats
Marees, the difficulty with the first 15 team cup you describe is the top-3 teams play less games, and have to sit for a week while the play-offs were run. You could get around that by having two play-offs (top-3, 2nd-3) with 1st in the top-3 going direct to the final, and 2nd playing the winner of the run-off group for the other final place. But it seems overly clever to me.

At any rate, with a play-off of 2nd place teams:
15 (3x5,1x3(4-6),2x2,1x2) Comp: 38.06; Mean 20.4%
With a super-group of 6 (1st and 2nd from each group):
15 (3x5,1x6,2x2,1x2) Comp: 31.57; Mean 18.8%

The super-group isn't bad actually, competitiveness is closer to 10-team than 14, but similar level of meaning to the 14 team edition. And 48 games, which matters. Length would be similar to now: 42 days, with a minimum of 34.
Russ  7th July, 2011 09:48:16