DRS controversy has been rumbling along in the background since the ICC introduced it. Whether for umpiring incompetence or bad luck, it has taken on a larger life since the Ashes started. That both teams (and their fans) are committed and experienced in its use has made its errors more obvious and troublesome. Weirdly, although I have quite firm views on the system, and have commented on numerous websites about it, I've only ever written a short piece discussing potential changes; rather than taking the time to write a thorough analysis. Now would seem an opportune time.

The DRS is often conflated with *-eye, or with technology. Properly, it is composed of three parts, and numerous sub-parts: the technology, which comprises *-eye, hotspot, snicko, and naturally, the tv cameras and audio; the player review; and the interface and process by which the review is actually done.

All three are flawed in current incarnations.

Although the technology receives the most criticism, by and large it is the least flawed component. It has limitations, rather than outright problems, and these aren't well dealt with by the system.

• Snicko is not used because it is too slow to match the image with the audio. The interface of image to picture is never quite clear - is the centre the frame, or one of the edges. And it has unknown reliability - it isn't clear if a ball passing the bat can cause a noise. That's a lot of interpretation, even if it picks up sounds reliably.
• Hotspot was sold up the river by over-enthusiastic commentators, because it has been clear for years that it cannot pick up fine nicks - though what is a nick, and at what level must we see one: visual? molecular? It also needs a lot of interpretation, as it must match the ball to various heat spots, some of which can be caused by incidental contact. But while false negatives abound, a clear edge that appears hotspot is often definitive, so it has some value.
• *-eye is probably the most reliable, though Hawkeye is oversold by its proponents. For reasons unknown the exact margin of error is never discussed, nor the circumstances under which it becomes unreliable. The ICC has put forward two interpretations that are both unnecessarily lenient - the 3m rule and the half ball hitting - and inconsistent depending on the decision being made. Interpreting it as "within 95% certainty" would be a big step forward, because the current rules invite ridicule.
• And lest we forget, a very large portion of the review system depends on television technology not far advanced from several decades ago. Its limitations are plainly obvious, and the lack of energy put into improving them is pathetic: the pictures remain of low quality, confused by shadow, with limited frame-rates that confuse runouts and make deflections harder to see, and the problem of foreshortening on catches near the ground is well known, but continues to cause problems.

In short, the technology is incapable, as presently designed, to provide fast, consistent, and objective decisions. You need only listen to the commentary to see that a dependence on televisuals for decisions is prone to interpretation and error; and not necessarily an improvement on the central umpire. Some of these can be fixed: improved frame-rates and higher definition cameras for runouts and stumpings; the introduction of automated no-ball checking, a trivial problem for computer vision; the use of *-eye systems to track deflections with high frame-rates, avoiding the need for hotspot and snicko. The ICC could easily invest in a proper decision system, and it has failed to do so, leaving the technology in the hands of the television studios, and the money in the hands of people whose primary aim is entertainment, not better decisions.

I've never felt comfortable with player reviews, mostly for aesthetic reasons. I like to see a raised finger and the game move on; I don't like to see extended discussions of dismissals, costing many minutes (and therefore overs) each day; I especially hate waiting around for an umpire the check a no-ball. The absence of any sense of what the umpires are looking for compounds the problem.

The players, needless to say, have reacted to the ability to review some decisions as economic theory suggests they would. While the system was sold as removing howlers, the players treat it as a resource. Reviews are spent on key wickets, and at key moments - particularly the final wicket of close matches, both Hobart and Trent Bridge suffering the indignity. They are overly cautious when holding one review, and reckless with their first; and they play the odds, looking for opportunities where the system will help them, rather than working to improve decisions over-all.

We should expect nothing less, and adding players to the decision making process has had entirely predictable results, which, unless you are a keen student of game theory hasn't added much to the game except extended footage of earnest discussions; and the opportunity to swear at deluded batsmen.

The most misunderstood part of the DRS is the process itself. Expectations have been put on it to make decisions, when it has been primarily designed to augment decisions. This is obvious if one reads the DRS protocol, though judging by the number of journalists claiming the third umpire "over-turns" the decision few have.

We can discuss the system as a series of decisions made based on different information, drawn for observation and the technological output. The central umpire has a set of observed occurrences (call it \(\{O_C\}\)); the third umpire has a different set of observations - not necessarily superior, but consistent with what the viewer at home sees - \(\{O_T\}\).

For the original decision, the central umpire makes a decision, \(D\), by determining if there was a wicket (\(W\)) based on the balance of probabilities - usually giving the batsman some benefit of the doubt, but not required by the laws.

\begin{equation}
D_C = Decision_C( P( W | \{O_C\} ))
\end{equation}

It is important to note here that the third umpire does not make the final decision, but rather it rests with the central umpire. There is an expectation from viewers that the third umpire makes the decision themselves, based on what they see and the original decision:

\begin{equation}
D_R = Decision_T( P( W | D_C \wedge \{O_T\} ))
\end{equation}

However, as the central umpire makes the decision, and the third umpire merely conveys the observations to him, it is actually as follows:

\begin{equation}
D_R = Decision_C( P( W | \{O_C\} \wedge \{O_T\} ))
\end{equation}

This would not matter if the observations of the third umpire always led to a certain value for \(W\), as there is no difference between the two equations in that case. But the absence of certainty means the system produces results that aren't always easy to interpret.

The Trott LBW at Trent Bridge was a case in point. The decision rested on whether he had hit the ball, for which there was no conclusive evidence. Hotspot, as discussed above, can only prove an edge if a mark in the right position, (\(H\)) is detected, otherwise the probability of an edge, \(E\) is an unknown (let's say \(a\)).

\begin{eqnarray}
P( E | H ) = 1 \\
P( E | \neg H ) = a
\end{eqnarray}

The central umpire may have based his determination that there was an edge on hearing a noise, \(N\), holding that opinion with probability b. As hotspot came up negative, the probability of there being an edge is reduced, but there is no set amount by which it might be reduced, we only know that:

\begin{eqnarray}
P( E | N ) = b \\
a \leq P( E | N \wedge \neg H ) \leq b
\end{eqnarray}

If potential sources of the noise are found - the ground or pad being hit by the bat - then it is reasonable for the umpire to re-think their determination of an edge, and therefore change their decision as \(P( E )\) declines - remembering that this may have been the key observation in determining \(D\).

The downside to this process is that in general, the central umpire's faith in their own judgement declines. Further observations are as likely to be inconclusive or contradictory as confirmative. The central umpire is forced to make a decision on their own, and then decide if, based on footage they don't see, but only have relayed to them, that original judgement was wrong.

This is further confused by strict process under which Hawkeye is used, whereby the technology does make a decision, and that depends on the original decision.

\begin{equation}
D_R = Decision_H( P( W | D_C \wedge \{O_H\} ))
\end{equation}

This creates its own inconsistency, in that should the central umpire decide that there was an edge, but was satisfied that the ball was hitting the stumps, the absence of an edge does not change the first decision. And hence a ball clipping leg stump is not out (umpire's call), even if the umpire would have given it out, if not for the edge.

The path forward for the DRS is to recognise that the technology has severe limitations, and focus our attention on those aspects of it that work, and work quickly and without intervention. The central umpire is only undermined by making multiple decisions, and given easily accessible technology, there is no reason they could not augment their original decision with help, rather than a complicated process involving player requests and an interpreted but inconclusive technology. Where the technology is conclusive, and can be relayed to the umpire, the right decision will always be made. Where it is not conclusive, then nothing is lost by ignoring it as in these cases:

\begin{equation}
P( W | \{O_C\} \wedge \{O_T\} ) \approx P( W | \{O_C\} )
\end{equation}

That is, there is no change in the probability of a wicket with more, but useless and time consuming observations.

The central umpire, given a hand-held device to show the results - as now, either out, not out, or unpire's call - of automated no-ball, *-eye, and edge detection could make prompt and unambiguous decisions with confidence that the tv replays would struggle to second-guess them. There would be no need for player reviews because a player couldn't reasonably expect the decision to be over-turned when the primary evidence has already been examined. In particular cases (such as whether the ball hit bat or pad first), when necessary, they could instigate discussion with the third umpire to review the footage. But by and large, there is no reason why in today's technological environment, a central umpire should be subject to review by television footage that is very often inconclusive and adding little.

That sort of system is easily achievable, but some way off. Right now, the ICC needs to do two things: firstly, focus on improving the quality of the technology to meet the requirements of a decision system, and not a broadcaster; and secondly, better communicate the process of decision making, by broadcasting the umpire's discussions to allow people to understand the reasoning behind the decision. Despite boundless good-will from people who believe in technological solutions to umpiring ills, trust in the DRS has fallen to levels where it cannot survive.

Cricket - Articles 9th August, 2013 00:30:38   [#] 

Comments

Ball tracking process
Thanks for the clear and careful summary, which makes it easier to express what I would change about it:

The process is not actually meant treat *-eye differently from the other technology, except in the case of the "3m rule", where extra wording was hastily thrown in without any regard for the existing protocol after incidents in the World Cup.

Specifically, the protocol apart from that exception does not ask *-eye to make a decision based on the original decision, and would function in exactly the same way if phrases like "if an 'out' decision is being reviewed" were removed. The situation where the ball is shown hitting stumps (or impacting in line), but without the required (more than) half the ball in line, is labelled "umpire's call" by someone on the broadcasting side of things and commentators have interpreted this as saying the decision stands. None of this comes from the DRS protocol, which simply says that this situation doesn't allow the question to be answered with 'a high degree of confidence' and so the evidence should be reported as 'inconclusive'. In the example you describe, the on-field umpire would be perfectly justified in giving the batsman out.

Jonathan D  9th August, 2013 17:34:04  

Over-turning
Jonathan, that's a fair point, and you are right, the central umpire should be advised on each individual point and assess against his "recollection and opinion of the original incident" to use the quote. I suspect an umpire wouldn't overturn in that instance - perhaps because the umpire's call allows them a get out clause to stand by a not out. But again, without audio we can't know.
Russ  9th August, 2013 19:07:01  

A treatise on DRS
"There would be no need for player reviews because a player couldn't reasonably expect the decision to be over-turned when the primary evidence has already been examined."

This is something which umpire Taufel has commented about. The hand held device could currently be substituted by a umpire armed with two things (1) real time replays (and the ability to toggle the various camera inputs - heat, frame rate etc.) and (2) the ability to initiate reviews.

I think the TV umpire's expertise as an umpire is currently severely underutilized.
Kartikeya  11th August, 2013 14:42:13  

A treatise on DRS
A hand-held device is probably no harder than real-time reviews, although it would help on a limited set of areas for the moment.

I sort of agree that the third umpire is underused. But one could say the same for square-leg (judging height for instance). Review isn't really the word you are looking for there; more, call for assistance. A review implies the third umpire gets involved independently; which is problematic, as seen in Australian domestic cricket. It is hard to initiate a review without knowing what you'll see, and having seen something as probabilistically likely, but still uncertain, to not make a decision.
Russ  12th August, 2013 01:02:22