Hawk Eye Public Understanding

  • Upload
    imfresh

  • View
    225

  • Download
    0

Embed Size (px)

Citation preview

  • 8/6/2019 Hawk Eye Public Understanding

    1/30

    Umpires, Referees and the Public Understanding of Decision Technology:Why cricket uses Hawk-Eye well while tennis uses it badly.

    Introduction1

    Public understanding of the capabilities of new technologies is a pressing problem with

    portentous political consequences. At the time of the first Gulf War the conclusion of a

    publicised debate over the accuracy of the Patriot anti-missile missile had consequences

    for the credibility of all star-wars-type defence systems (Collins and Pinch 1998 ch1).

    With increased computer power and speed it will not be long before it is impossible for

    the ordinary TV viewer to tell the difference between synthetic reality and photography

    with huge consequences for news-making should a government choose to use it the

    power corruptly. These and related technologies need to be understood by the public.

    Sport is at the cutting edge of the introduction of such technologies to a wide audience.

    In an earlier paper in this journal (Collins and Evans 2008) we argued that decision

    support aids such as `Hawk-Eye, which are used to assist lbw decisions in cricket and

    line calls in tennis, are less accurate than they seem. We now take the argument forward

    in the light of new developments in the use of this technology and new information about

    decision aids in general. In particular we argue that, in terms of public understanding, an

    example of correct application of technology to sport is Hawk-Eye as it is currently

    applied to cricket; a case of the incorrect application is Hawk-Eye as it is currently

    applied to tennis. We will show how crickets approach could be applied to tennis. We

    1 Thanks to Josh Nall for many helpful and provoking comments and for drawing our

    attention to certain pieces of important information. We are also grateful to the membersof the KES seminar group whose remarks results in substantial improvements to the final

    draft and to Peter Szirmak and Henk Jonkhoff for advice and information during an

    extended exchange of correspondence.

    1

  • 8/6/2019 Hawk Eye Public Understanding

    2/30

    will also discuss a number of other simple practical changes that can and should be made

    to the use of decision-aid technology.

    In another place (Collins, under submission) we introduce certain technical terms that are

    part of the philosophical approach that frames the argument. We use the term

    `ontological authority to refer to the power of match officials (or machines) to define

    reality (for example, whether the ball was `in or `out); `epistemological privilege

    implies some advantage, due to vantage point or special skills, in respect of knowing

    what happened; `presumptive justice is the assumption that if you were the match

    official (or the judge in an ordinary courtroom at which you were not present) you would

    make the same decisions, or that you could do no better; `transparent justice implies that

    the correct decision is seen to be made directly; `transparent injustice is the complement

    of transparent justice; and `false transparency is the impression that justice is being seen

    to be done whereas it is not (as in, say, the early days of Stalinist show trials). We now

    show how these terms are employed in the case of sports judgements.

    Traditionally, umpiring and refereeing has been the domain ofpresumptive justice:

    spectators were ready to accept the ontological authority of match officials because they

    were seen to have epistemological privilege. With the introduction of new technology

    such as television replays, epistemological privilege has, in the case of many decisions,

    shifted to the television viewer, bringing instances oftransparent injustice. Some sports

    have tried to rectify the situation by introducing `off-field officials who review television

    replays and advise the on-field official, usually replacing transparent injustice with

    transparent justice. Certain sports decision aids, which present a clear and sharp

    2

  • 8/6/2019 Hawk Eye Public Understanding

    3/30

    reconstruction of the path of the ball, its impact footprint, and the line, give the

    impression that they can make exact determinations of ball and line position to a

    millimetre. Given that the admitted average error of the Hawk-Eye device, as used in

    tennis, is 3.6mm, this can give rise tofalse transparency.

    Through the use of examples, and such information we can gather about available

    decision aid technologies, we now try to develop a series of recommendations that will

    maintain the credibility of match officials while allowing the smooth introduction of new

    technologies without misleading the public.

    Failures in the off-field official system and suggested resolution

    The first set of examples of decision-making in sport is concerned with the relationship

    between technology, off-field officials and on-field officials. As intimated above, off-

    field officials when they use technology to advise on-field officials can eliminate cases of

    transparent injustice and maintain transparent justice. In the case of incidents on the 15th

    and 16th of January 2010 during the South Africa-England test cricket match this process

    was seen by the public to fail (the examples are described in detail in Appendix 1). The

    only technology in use in respect of the set of examples described were television replays

    and a `stump microphone a microphone embedded in the wicket to pick up the sound

    caused by any marginal impact. This set of examples and recommendations has more to

    do with the smooth introduction of technology to sport and with public satisfaction with it

    rather than the public understanding of technology.

    Over two days of this test match three decisions that were referred to the off-field umpire

    were considered by commentators and, one must assume, most viewers (they certainly

    3

  • 8/6/2019 Hawk Eye Public Understanding

    4/30

    included this papers authors), to have delivered transparent injustice (one case being

    marginal). A fourth decision could not be referred to the off-field umpire because the

    England team had exhausted its quota of `reviews (at least one of which should have

    been successful rather than unsuccessful and so not deducted from the quota), but

    television replays revealed that the wrong decision had been made and that a review

    should have been successful. That makes a total of four (or at least 3.5) cases of

    transparent injustice over two days.

    An important feature of the review system as implemented in this cricket series was that

    the umpires decision was to be taken as final unless it could be seen to be clearly wrong

    by the off-field umpire using the available technology. In the first case, the off-field

    umpire should probably have over-ruled the on-field umpire because the bowlers foot

    was not clearly behind the relevant line, as could be seen by television viewers, but he did

    not. In the second case the off-field umpire should have over-ruled the on-field umpire

    because the fact that the ball hit the bat was evident from the sound transmitted from the

    stump microphone (the output of which was not available to the on-field umpire) but

    though television viewers and commentators heard the sound the off-field umpire did not

    hear it -- it appears he had the volume turned down. In the third case the off-field umpire

    did over-rule the on-field umpire on the basis of television replays but should not have

    done as the replays were indecisive as everyone could see. In the fourth case replays

    showed the ball clearly deviating as it hit the bat whereas the on-field umpires decision

    was that it had not hit the bat.

    4

  • 8/6/2019 Hawk Eye Public Understanding

    5/30

    The way to avoid these problems and use off-field officials successfully, at least in terms

    of many kinds of decision, is demonstrated by their use in rugby football. Here the on-

    field referee engages in a choreographed and audible (to television viewers) conversation

    with the off-field referee. The tenor of the conversation is set by the question Is there

    anything to stop me awarding a try? The on-field referee then goes chooses to ask a

    series of questions which the off-field referee answers audibly. One may imagine a

    similar procedure applied to cricket which, as far as one can see, would have avoided the

    South-Africa problem. The audible dialogue, accompanied by replays visible to

    everyone at the ground and at home, might go like this (adjusted for different

    circumstances):

    On-field-umpire (ONFU): I have given an `out decision on the grounds of caught

    by the wicket-keeper. Was it a legal delivery or a no-ball?

    Off-field Umpire (OFFU): It was a legal delivery: some part of the bowlers foot

    was definitely behind the line

    ONFU: Do you have sufficient reason to believe it did not touch bat or glove so

    that you can confidently overturn my decision?

    OFFU: Yes

    ONFU: What is the evidence?

    OFFU: There was no sound nor was there a mark on the bat visible with the use

    of infra-red technology (see below). Finally, there was no clear deflection of the

    5

  • 8/6/2019 Hawk Eye Public Understanding

    6/30

    ball visible on television replays. Though I cannot be sure the ball did not touch

    the bat through viewing the television replay the lack of supporting evidence from

    other technology makes me as sure as I can be that it did not touch the bat.

    ONFU: In that case I will change my decision to `not out.

    This procedure restores ontological authority to the on-field umpire who uses the

    technological aids indirectly via the off-field umpire. The audibility of the exchange

    allows for technicians or the on-field umpire to intervene where residual doubts remain

    (as in the case of the inaudible sound); this gives the best chance that the conclusion will

    satisfy everyone except the most partisan.

    It might be still better if the on-field umpire choreographed the replays, giving

    instructions to the third umpire to present different views which were visible on the field

    of play. This would finally recognize the truth of the matter which is that nowadays the

    crowd has epistemological parity in respect of at least some decisions and is in a good

    position to follow the reasoning of the umpires in those cases. Nevertheless, in virtue of

    the fact that the default, in case of any doubt, is the original umpires decision, there must

    be a clear over-rule or no change, the ontological authority stays where it has traditionally

    been with the on-field umpire. In most cases this procedure would lead to transparent

    justice with readily acceptable presumptive justice in marginal cases. In such a case it

    might be that the on-field umpire could call on the use of technology at will (as in the

    case of rugby football) as well as a limited number of reviews to be called for by players.2

    2 To harmonise ontological authority with epistemological privilege in cricket still more

    thoroughly and make transparent injustice still less likely the off-field umpire should

    continually monitor the bowlers front foot, ready to advise in case an `out decision turns

    6

  • 8/6/2019 Hawk Eye Public Understanding

    7/30

    Reconstructed Track Devices (RTDs)

    `Hawk-Eye is a well-known example of what we call a `Reconstructed Track Device or

    RTD. RTDs use visible-light TV cameras to follow the path of the ball and a procedure

    to filter pixels in each frame. Certain pixels are taken to represent an indicator of the

    position of ball and certain others an indicator of the position of the line or of other

    features of the playing arena, between-frame consistency helping to extract significant

    pixels from background. The space and time coordinates of these pixels are represented

    numerically and a statistical algorithm reconstructs the flight and impact point of the ball

    and crucial features of the playing area by combining information about the pixels in the

    different frames with information about the size of the ball, the physics of its distortion

    (in the case of tennis), the width of the line (in tennis), and so forth. The output of the

    calculations can then be used to make an `in/out decision in tennis and/or to construct an

    image of the playing area and the flight of the ball using colours that give an appearance

    approximating to the real setting but with sharpened edges and idealised precision. In the

    case of tennis, information about the likely distortion of the ball upon bouncing can be

    used to estimate the size and shape of the contact footprint and the visually reconstructed

    bounce point can be elongated to represent what is known of the physics. Every such

    system is, of course, subject to statistical uncertainty. However sharp the reconstructed

    images, they represent an `estimate which has errors, whether their distribution is fully

    understood or not. The errors will affect the accuracy of the estimated bounce point, the

    shape of the reconstructed footprint and the reconstruction of the line.

    out to be a `no ball. As things stand, TV viewers occasionally see `out decisions given

    to no-balls which is another easily avoidable case of transparent injustice.

    7

  • 8/6/2019 Hawk Eye Public Understanding

    8/30

    As mentioned, in an earlier paper we discussed the accuracy of Hawk-Eye. In this paper

    we will simply note certain facts:

    1) While it is sometimes claimed that Hawk-Eye has passed its tests for accuracy of

    line-calling in tennis with a 100% record, the International Tennis Federations

    tests of line calling accuracy have a tolerance of 5mm in respect of the edge of the

    line and 10mm in respect of the true position of the ball. This appears to mean

    that a call would be deemed accurate even when a ball that was 5mm `in was

    called 5mm `out.3

    2) Hawk-Eye Innovations website agrees that Hawk-Eye has an average accuracy in

    tennis of 3.6mm4 An average accuracy of 3.6mm implies that there are larger

    errors as well as smaller errors. For example the following `triangular

    distribution of 132 errors has an average accuracy of 3.3mm.5

    3 Though we understand that the criteria used by the ITF varied greatly at the time thatHawk-Eye and other devices were competing for acceptance. Thus, Auto-Ref, an RTD

    developed in Canada, was rejected. The ITF technical report we have been shown

    reveals the results of 120 bounce trials of which 119 are marked as `correct and one ismarked as `wrong. The one that is marked as wrong shows the ITF measuring the ball

    as impacting 2mm one side of a line whereas Auto-Ref marked it as 3mm on the other

    side, a total discrepancy of 5mm with both calls being within 5mm of the line. (Miller,

    2006) Our thanks are due to Peter Szirmak for information about Auto-Ref.

    4 See http://www.hawkeyeinnovations.co.uk/?page_id=1011. The figure comes from a

    test conducted in 2006.

    5 It is likely in any real RTDs distribution of errors that there would be fewer large errors

    and more small ones. The only information we have on the error distributions of RTDs isthat Hawk-Eyes errors do not follow the `normal distribution. We would like to see

    error-distributions published. Here is an error distribution made available to us in respect

    of TEL (Tennis Electronic Lines), which works from the electrical signals produced bythe interaction of a metallised ball and wires buried in the court and about which we have

    learned much from Henk Jonkhoff. This is the kind of data that all such devices should

    make available along with notes concerning the exact conditions under which they were

    8

    http://www.hawkeyeinnovations.co.uk/?page_id=1011http://www.hawkeyeinnovations.co.uk/?page_id=1011
  • 8/6/2019 Hawk Eye Public Understanding

    9/30

    3) While the ITFs testing procedure rules out errors larger than 5mm when they

    affect a line call, the testing procedure means that few impacts within 5mm occur

    in the course of testing (around 10% of around 100 bounces).

    4) It is in the nature of statistical inference that on a very few occasions large errors

    will occur. Such occasions might well not be detected by a limited calibration test

    such as is used by the ITF. We cannot say if this actually happens in the case of

    any particular RTD.

    5) Devices, such as RTDs, that use ambient light as the source of their signal are

    likely to perform less well as conditions become darker. This is simply because

    there are few photons and illuminated pixels on which to base the statistical

    inferences. We cannot say if this problem actually applies to any particular RTD

    generated. In the diagram the horizontal axis shows number of bounces while the vertical

    axis shows deviation from true bounce point as defined by the mark made on alcohol-

    moistened talc-covered perspex

    Number of errors246810121416182022Size of error in

    mms109876543210

    9

  • 8/6/2019 Hawk Eye Public Understanding

    10/30

    and note that the Hawk-Eye website states that testing occurs in a range of

    conditions, including dark or overcast conditions.6

    6) We do not know if the ITFs testing procedure includes conditions as dark as, say,

    those which pertained at the end of the Federer-Nadal Wimbledon final in 2008. 7

    It seems unlikely, however, since the ITF uses high-speed cameras as one method

    of establishing the true bounce point of the ball and these perform badly in the

    dark.

    7) In the case of that part of the lbw rule that involves projecting the flight of the ball

    after it has hit the pad in order to determine whether it would hit the stumps,

    relatively larger errors are likely to be more frequent because, in general, a longer

    flight path has to be estimated than in the case when the ball strikes the ground or

    some other object. In general, the longer the projected flight in relationship to the

    recorded flight the greater the uncertainty is likely to be.

    8) It is believed by some that no projection or estimation is involved in the case of

    tennis.8 Cameras have a finite frame-speed, however. If the frame-speed is, say,

    50 frames per second, and the ball is moving at about 100 mph it will travel about

    3 feet between frames. Therefore, at that ball speed the average distance between

    6 http://www.hawkeyeinnovations.co.uk/?page_id=1011

    7 The BBC website reports: `The final shot was struck in near darkness on Centre Court

    at 2115 BST. See: http://news.bbc.co.uk/sport1/hi/tennis/7490443.stm [accessed 9

    February 2010]

    8 For example, David Gower, ex-captain of the English cricket team, and now a TV

    commentator and newspaper columnist, writes that, in contrast to cricket, Hawk-Eye

    when used in tennis showed `fact in graphic form (Gower, 2009).

    10

    http://news.bbc.co.uk/sport1/hi/tennis/7490443.stmhttp://news.bbc.co.uk/sport1/hi/tennis/7490443.stm
  • 8/6/2019 Hawk Eye Public Understanding

    11/30

    the last frame and the bounce of a tennis ball will be 1.5 feet. Sometimes the

    distance that has to be predicted will be 3 feet, or even more if the ball is

    travelling faster (tennis balls can travel at 150mph and perhaps faster).9 The

    difference between cricket and tennis is, then, a matter of degree. One can

    imagine that a combination of circumstances might mean that the last frame

    capture of a tennis ball is 4 feet from the bounce point and here the prediction

    required might be still greater than that needed for the projected flight path of

    some lbw decisions.

    Hawk-Eye in cricket

    We now compare the use of Hawk-Eye in tennis and in the cricket. We begin with

    cricket and the use of Hawk-Eye in the South Africa-England test match series. In these

    matches the umpires decision was taken to be the default and was only over-ruled if

    Hawk-Eye indicated that a big mistake had been made. How this works is most easily

    exemplified in the case of the `lbw decision, which effectively asks the umpire to decide

    whether or not the ball would have gone on to hit the stumps if it had not hit the

    batsmans pad.10 The decision is notoriously difficult as the umpire has to extrapolate

    from what did happen to what would have happened if the balls flight had continued

    9 We have been unable to obtain any information about the frame-rate of Hawk-Eyes

    cameras though we have made repeated requests for information. We have been told that

    competing RTDs, such as `Auto-Ref, use a frame rate which is much higher than 50fps(though the principle still applies). We use the figure of 50fps as illustrative because that

    is our best estimate of Hawk-Eyes frame rate though other sources suggest 60fps (which

    would not make a big difference to our rough calculation). Perhaps the frame-rate has

    changed over the years it is rumoured that in early years at least, Hawk-Eye, used theoutput from broadcast TV which uses 25 fps cameras and we have no idea what the

    current frame-rates are.

    10 See Collins and Evans (2008) for a full explanation

    11

  • 8/6/2019 Hawk Eye Public Understanding

    12/30

    uninterrupted. Where the umpire cannot be sure, the convention in cricket is that the

    batsman gets the benefit of the doubt and is given not out even if the umpire thinks it is

    possible that the ball might have gone on to hit the outer edge of the stumps.

    Figure 1: The grey box illustrates the zone of uncertain on one edge of the wicketequal to half a stump and half a ball (not to scale)11.

    In the review system, where Hawk-Eye is used to check the umpires decision, the crucial

    element in cricket is what we are going to call `the zone of uncertainty. This is an area

    about 55 millimetres, or a little more than 2 inches wide (half the width of the ball plus

    half the width of one stump) around edge of the stumps (see Figure 1).12 In this area, the

    11 Readers of Collins and Evans (2008) will find that this is very close to what we

    suggested there though we illustrated the point with a shaded rectangle inside the zone ofuncertainty.

    12 The ball can be up to 22.9 cm in circumference = c 7.3cm in diameter. A stump may

    be up to 3.81 cm in width.

    12

  • 8/6/2019 Hawk Eye Public Understanding

    13/30

    RTD does not overrule the on-field umpires decision. Thus, in the case where an umpire

    gives a `not out decision and there is an appeal, the umpire is not over-ruled unless the

    inside edge of the ball is shown to be striking a point c.55mms inside the outer edge of

    the wicket. The ICCs guidelines express this as follows, saying that, when the inside

    edge of the ball is shown in what we have called the zone of uncertainty:

    the third [off-field] umpire should report that he does not have a high degree of

    confidence so the original on-field decision remains.13

    Similarly, if the umpire gives an out decision, this is only overturned if the RTD

    suggests that no part of the ball was inside zone of uncertainty.

    Interestingly, there are also zones of uncertainty in respect of the other two judgments

    that have to be made to apply the lbw rule correctly. These concern the impact point of

    the ball on the ground as it bounces before striking the batsman and the impact point of

    the ball on the pad. With these decisions the zone of uncertainty is half the width of the

    ball (about 36 mm). A further feature recognising the fact that an RTD cannot entirely

    replace human judgement is the following rule used by the ICC:

    if the point of impact[on the pad] is greater than 250cm from the stumps, the third

    umpire will inform his on-field colleague of the exact distance, the approximate distance

    from the point of pitching to the point of impact and where the ball is predicted to hit the

    stumps. The on-field umpire will then apply the normal cricketing principles concerning

    levels of certainty in making his final decision.14

    13 See:http://www.lords.org/data/files/drs-guide-10354.pdf[accessed 9 February 2010]

    14 Collins (though not the commentators or anyone else that he has heard from) remains

    puzzled by one judgement in the South Africa vs. England test again concerning De

    13

    http://www.lords.org/data/files/drs-guide-10354.pdfhttp://www.lords.org/data/files/drs-guide-10354.pdfhttp://www.lords.org/data/files/drs-guide-10354.pdfhttp://www.lords.org/data/files/drs-guide-10354.pdf
  • 8/6/2019 Hawk Eye Public Understanding

    14/30

    In our language, the zones of uncertainty which revert the judgment from the off-field

    umpire to the on-field umpires initial decision, along with the 2.5 metre rule, restore the

    on-field umpires ontological authority except where a big mistake that is unlikely to

    have been caused by an error or estimation is seen to be made.15 In the case of a big

    mistake, viewers have epistemological parity and can see the reason for the over-rule. In

    the case of the application of the zone of uncertainty, viewers, recognising that the

    technology of the RTD cannot give them epistemological parity with the umpire are

    likely to be happy to recognise the umpires epistemological privilege in virtue of their

    proximity to the action and skill at making judgements which has been enhanced by

    training. Presumptive justice holds sway to most viewers satisfaction.

    With the exception of the putative problem discussed in footnote 13, none of the

    umpiring troubles discussed above were caused by Hawk-Eye. Hawk-Eye worked well

    given the zone of uncertainty which resolved any question about its accuracy.

    Villiers first innings. At one point an appeal for lbw was answered by an `out call from

    the on-field umpire. De Villiers asked for a review. Hawk-Eye showed the ball passingover the top of the wicket but also showed that the ball had hit the pad more than 2.5

    metres from the wicket. In this case Collins was expecting the on-field umpires decision

    to be upheld given that Hawk-Eye had to make an inference over such a long distanceand in view of the rule set out above. But the game passed on without remark. Once

    more, to satisfy Collins that justice had been done he would have had to have heard the

    conversation between third umpire and on-field umpire. (It could well have been that the

    on-field umpire changed his mind but was he apprised of the fact that the decision hadbeen largely passed to him in view of the distances involved?)

    15 Russell (1997) argues that where a big mistake is made a decision is not really a

    decision at all.

    14

  • 8/6/2019 Hawk Eye Public Understanding

    15/30

    Hawk-Eye in tennis

    In tennis, at least as it is being played at the time this paper is being written, the use of

    RTDs such as Hawk-Eye is very different. Here there is no zone of uncertainty. What

    happens is that the RTD makes it decision based on the modelled reconstruction and

    announces `in or `out with no reference to the error bars that are associated with every

    such process. Ontological authority is granted to the machine irrespective of the errors in

    its decision-making process. At the same time the television viewer is all too often

    misled into thinking that they have epistemological privilege in view of the sharp

    reconstructed image with which they are presented. Many intelligent viewers believe

    they are seeing `what actually happened rather than a reconstruction based on an

    algebraic inference with inherent error.16 Actually, the visual reconstruction comes `after

    the fact of the decision based on the mathematical reconstruction; the decision is not

    based on anyone looking at the reconstruction. Viewers, or at least most of them, remain

    happy because they do not realise that the epistemological privilege they believe they

    have been granted is illusory. They all too easily believe they are seeing a replay of what

    really happened like a TV replay which puts them in a better position to make a

    16 Collins was astonished to discover that even senior physicists had not realised they

    were watching simulations based on inference when the saw the graphic displays produce

    by an RTD.

    15

  • 8/6/2019 Hawk Eye Public Understanding

    16/30

    judgement than the umpire can make in real time.17 This, as explained, is what we call

    false transparency.

    What viewers are actually seeing in the case of close calls, when the ball is truly within a

    millimetre or so of the line, is the outcome of an error-laden process with, at best, a very

    slight (but unknown) excess probability of the call being right rather than wrong. There

    is then, a lot going on which is not what is seen to be going on the visual reconstruction.

    Many viewers (and, apparently, some players) are being misled. However happy viewers

    are, it seems wrong that to mislead them.

    Still more worrisome is that professional tennis events seem no longer to show television

    slow-motion replays that can be compared with the RTDs reconstruction. It is no longer

    possible for viewers to compare the RTDs reconstruction with a television replay. It

    may be the case that a sensible comparison cannot be made in the case of genuinely close

    calls because of the limited frame rate of even high-speed cameras, but TV cameras are a

    good and cheap way of demonstrating that justice is being done where big mistakes are

    concerned. It is really strange that this ability to compare with TV replays, and provide a

    17 To look at the matter from the perspective of past mistakes in the progress of artificial

    intelligence, viewers, when they see an RTDs reconstruction, are watching, not the real

    world, but a `micro-world (Winograd and Flores, 1986). The well-defined ball, bouncepoint and clean sharp-edged lines shown by RTDs are constructed objects. A micro-

    world is a mathematically reconstructed world and, as such, can be as clear and sharp-

    edged as a set of ideal geometrical objects. But these objects are built upon statisticalinformation that by its very nature is inexact and represents the actual world in a

    misleading way. (For instance, on grass courts the line can become smeared and on clay

    courts it can become obscured and such changes are not likely to be represented in thereconstructed image of an RTD unless very special care is taken; under these

    circumstances, what is `in and what is `out becomes less and less like a decision that

    can be represented in a mathematically modelled world.)

    16

  • 8/6/2019 Hawk Eye Public Understanding

    17/30

    continued demonstration of the efficacy of the RTD has been abandoned.18 This is bad

    for the public because they cannot learn to compare one technology with another.

    The solution for tennis in the case of Hawk-Eye like devices

    Other makes of sports decision aids have told us that it is a mistake to promote millimeter

    accuracy as the goal because errors cannot be eliminated. The impetus for the

    introduction of RTDs, and earlier line-calling aids appears to have been, at least in part, a

    desire to avoid prolonged disputes between players and umpires. But if an RTD can

    merely show that the ball bounced very close to the line then it has shown that it is not

    appropriate to shout at the umpire because the umpires decision is reasonable. An

    umpires decision is only unreasonable if the mistake is obvious. In other words, in the

    absence of transparent injustice, presumptive justice is adequate and with close line calls

    there can be no transparent injustice. An RTD that automatically alerted the umpire only

    on occasions when a bad error was detected would be able to eliminate transparent

    injustice and would discourage players from appealing. As it is, on crucial points, they

    have a not unreasonable hope that the play of chance on a close call will benefit them

    irrespective of whether they really believe the ball was miscalled.

    The appropriate and simple solution for International Tennis Federation is to adopt the

    procedure of the International Cricket Council and introduce zones of uncertainty. Once

    more, if the trailing edge of the ball entered the zone of uncertainty the umpires original

    line call would be the default (see Figure 2). This would return ontological authority to

    18 There have been some large mistakes in cricket so we understand.

    17

  • 8/6/2019 Hawk Eye Public Understanding

    18/30

    the umpire except where a large mistake that is one that shows the trailing edge of the

    ball to be outside the zone of uncertainty was shown to have been made.19

    Figure 2: Zone of uncertainty on the crucial edge of a tennis line: If the trailing edge

    of the ball is shown to be in the zone the umpires initial decision stands (not

    to scale).

    How wide should the zone of uncertainty be? A reasonable minimum zone would the

    that used in the ITFs own test procedures: it should extend 5mm either side of the crucial

    edge of the line. To take account of less than optimum conditions, such as the darkness

    19 This solution is a modified version of that put forward in our 2008 paper. There we

    called for error bars (or error ovals) to be attached to the reconstructed images of the ball.

    We argued that these error marks should vary according to the speed and direction of theball, the frame speed of the cameras, the ambient conditions, and so forth. We no longer

    believe this is likely to happen. What we have suggested here, however, seems to be

    easy, quick and cheap to introduce and unobjectionable since it does not require thatanyone accept that it proves that RTDs are not accurate, they have only to accept that, as

    in the case of cricket, ontological authority should remain with the umpire on the case of

    close calls.

    18

  • 8/6/2019 Hawk Eye Public Understanding

    19/30

    that attended the 2008 Wimbledon Mens Final, and the `tails of the statistical

    distribution, it would, however, be wise to make the zone wider perhaps a centimeter

    either side. Or perhaps it should be somewhere in-between.20 The exact width of the

    zone ceases to be important once the obsession with accuracy is relaxed and ontological

    authority is given back to the umpire in the case of close calls. If the RTD shows a ball

    landing within, say, 7 or 8mm of the edge of a line, the initial umpires decision must

    have been reasonable we can settle for presumptive justice.21

    Television replays should also be brought back for calls outside the zone. Television

    replays would assure viewers that justice was being done in these cases. This would put

    an end to the embarrassment of having both players and umpire agree that the ball was

    miscalled by the RTD, as has happened, but still having to accept its decision.22 If the

    RTD had made a big mistake the TV replay would show it. If the RTD had made only a

    small mistake the initial call would stand anyway. The current problematic situation

    seems damaging to tennis, wrong in an absolute sense and easy to avoid. And most

    important of all, the introduction of a cricket-like zone of uncertainty would remove the

    20 There might be a case for making the zone asymmetrical to mimic the tendency of

    human umpires to see a ball as `out when it touches the line but skids onwards. As we

    argue in Collins and Evans (2007), technological aids should change the game as little aspossible.

    21 It is argued in Collins and Evans (2008) that the professional game should depart as

    little as possible from the amateur game. As it is, technology is set up to call ballsdifferently to the way they appear to the human eye because skidding can make a ball to

    bounce up from a different position to that in which it impacted. The correct solution is

    to go back to the human eye where skidding is significant and this means using the

    human eye for close calls.

    22 As reported on the Gulf News website of 3rd March 2007 in respect of a match in Dubai

    between Nadal and Youzhny. See:http://gulfnews.com/sport/tennis/hawk-eye-leaves-

    nadal-and-federer-at-wits-end-1.165119[accessed 9 February 2010]

    19

    http://gulfnews.com/sport/tennis/hawk-eye-leaves-nadal-and-federer-at-wits-end-1.165119http://gulfnews.com/sport/tennis/hawk-eye-leaves-nadal-and-federer-at-wits-end-1.165119http://gulfnews.com/sport/tennis/hawk-eye-leaves-nadal-and-federer-at-wits-end-1.165119http://gulfnews.com/sport/tennis/hawk-eye-leaves-nadal-and-federer-at-wits-end-1.165119http://gulfnews.com/sport/tennis/hawk-eye-leaves-nadal-and-federer-at-wits-end-1.165119http://gulfnews.com/sport/tennis/hawk-eye-leaves-nadal-and-federer-at-wits-end-1.165119
  • 8/6/2019 Hawk Eye Public Understanding

    20/30

    current situation of false transparency and give a proper idea to the public of how such

    technologies should be used.

    Are improvements in technology important?

    The essence of the argument of this paper (and see Collins, under submission), implies

    that accuracy is not important in RTDs so long as they are reliable. So long as they avoid

    large mistakes, small errors can be accounted for within the zone of uncertainty. It may

    be, however, that viewers and governing bodies would prefer to see the size of the zone

    of uncertainty reduced as the technology improves. It can never be eliminated because

    the very idea of millimeter accuracy in tennis makes no sense because the edges of lines

    and balls are not defined that accurately.23 If, however, it was thought appropriate to

    reduce the zone of uncertainty then it would make sense, and be of great benefit to public

    understanding, if an open competition were introduced between the available devices

    which might also take into account their advantages and disadvantages other than

    accuracy. Here we begin to indicate what kind of dimension such a competition might

    take into account basing it on such information as we have on a limited number of

    devices.

    TEL is the acronym for a device known as `Tennis Electronic Lines. It bases its

    estimate of ball bounce point on the interaction of a metalised ball with wires embedded

    in the courts surface. It is the only device in respect of which we have been shown an

    error-distribution (see footnote 4). The error distribution indicates that it has an average

    error of less than 2mm and, on the basis of the number of tests illustrated, no errors larger

    23 See Collins, under submission, for a longer explanation of this point.

    20

  • 8/6/2019 Hawk Eye Public Understanding

    21/30

    than 4.5mm. If this graph represents its performance in all conditions over long runs it

    would certainly perform reliably outside a narrow zone of uncertainty. It has the great

    advantage over all other systems that it can be used in automatic mode without camera

    operators or other attendants. It is, therefore, the only device that could be used to take

    the place of umpires in the lower levels of the game.

    `Hot Spot uses infra-red cameras to image the heat signature caused by the impact of the

    ball with the ground. It has not yet been used in tennis but is regularly used in cricket to

    indicate whether a ball has touched the bat or the glove of a batsman before being caught.

    Hot Spot should be the most accurate device for indicating the exact relationship of the

    impact footprint of the ball with the edge of the line so long as the ball is travelling fast

    enough to show a good image and conditions are such as to allow the heat image of the

    line to be markedly different to that of the ground.

    Auto-Ref is an RTD like Hawk-Eye but we have been told that it uses 12 high frame rate

    colour cameras (usually running at about 220 frames per second but capable of much

    higher speeds). We have been given to understand that this makes it more accurate than

    the rough estimates of the accuracy of RTDs presented in this paper and that it less likely

    to be confounded when the path of a ball is superimposed on a player or a line than

    systems (if these are still in use) that employ a smaller number of black-and-white

    cameras.

    Our own deductions from first principles tell us that, in general, devices such as Hot Spot

    and TEL, the data for which is generated by the interaction of ball and surface, are likely

    21

  • 8/6/2019 Hawk Eye Public Understanding

    22/30

    to be more accurate and less subject to `outlier errors than RTDs, which have to infer the

    bounce point of the ball from data taken remotely from the bounce point.

    We must make it clear that the evidence upon which the above indicators are based is thin

    and uneven. We have obtained considerable help from the makers of TEL and Auto-Ref

    (eg, see footnotes 2 and 4) very little from the makers of Hot Spot and virtually nothing

    from Hawk-Eye that was not already available in the public domain. Even from those

    from whom we have obtained most help, we can only report what they tell us. We have

    tried to add what we can deduce from first principles about the nature of the different

    technologies.

    To return to public understanding, we believe that nothing but good could come from a

    more open and public discussion of what these technologies can actually deliver. We

    believe that at least some makers of these devices would welcome such a discussion.

    Summary and conclusions

    Problems have arisen with the introduction of new technology which impacts on

    umpiring and refereeing. They come about because they distribute epistemological

    privilege leading to disharmony with ontological authority and to transparent injustice or,

    with some devices, false transparency. Many of these problems could be easily and

    cheaply resolved if the purpose of the technology was no longer seen as getting things

    exactly right but as a help to umpires and referees. In tennis in particular, the aim of the

    technological aids should be to ensure that large, visible, errors that lead to transparent

    injustice are eliminated while the normal traditions of judgement based on human

    perception and presumptive justice are maintained in the case of close line calls; this

    22

  • 8/6/2019 Hawk Eye Public Understanding

    23/30

    would also maximise the continuity with the traditions of the game and its practice

    outside of the international arena.24 Once these principles are accepted, only small and

    easy changes in procedure are needed. More detailed recommendations for changes in

    the employment of off-field umpires in cricket and the reintroduction of television

    replays in the case of tennis are to be found in the text.

    Public understanding would be enhanced, first, by the elimination of false transparency

    and second by a wider debate about the capabilities of the different available

    technologies. With the rapid merging of the appearance of synthetic `reality and the

    photographic image, the need to educate the public about the capabilities of these

    technologies is increasingly urgent.

    24 For a more detailed explanation see Collins, under submission.

    23

  • 8/6/2019 Hawk Eye Public Understanding

    24/30

    Appendix 1: Detailed description of the event of 15 and 16 th January 2010

    The 2009/2010 test series between South Africa and England was one of the first to

    introduce an `review procedure. In a test match each team of 11 men is allowed to bat

    twice each team takes turns to have a `first innings and a `second innings. In this

    series each team was allowed two failed `reviews of umpiring decisions during each

    innings. If they were batting they could ask for `out decisions to be reviewed while if

    they were bowling and fielding they could ask for `not out decisions to be reviewed. If a

    review was successful it did not affect the number of reviews remaining. In case of a

    demand for a review, the on-field umpire asked for advice from the `off-field, `third

    umpire who had access to television replays and, in this case, the output of a `stump

    microphone located in the wicket that the batman is defending which should pick up

    the sound of a ball brushing bat or glove, signifying a possible catch. The third umpire

    also had access to the output of `Hawk-Eye, an example of what we will call a

    `Reconstructed Track Device (RTD); Hawk-Eye will be discussed at much greater

    length below but does not figure in this section of the paper. Perhaps the most significant

    element of the review system, however, is the meta-rule that the on-field umpires

    decision is taken to be right unless the evidence showed that it was very badly wrong. If

    there was any doubt, the on-field umpires decision was to be upheld it was to be

    treated as the `default decision.

    England began by batting and the events in question concern Englands `first innings

    and South Africas succeeding first innings. The first problem arose when one of

    Englands batsmen, Alastair Cook, was adjudged `out in virtue of `leg before wicket

    (lbw). The main decision aid in cases of lbw is Hawk-Eye and in this case it showed that,

    24

  • 8/6/2019 Hawk Eye Public Understanding

    25/30

    according to the rules covering its use (of which more below), the umpires call had been

    correct. A prior decision has to be made, however, (not usually a subject of the demand

    for a review), and this concerns whether the ball that was bowled at the batsman was a

    legal `delivery. The television replay appeared to show that the bowlers heel was not

    behind the line that defined the furthest point that the front foot should reach in the course

    of the delivery. The heel was on the edge of the line but not behind and, if that was truly

    the case, the delivery should have been declared a `no-ball and the batsman given `not

    out. Whether it is reasonable to expect the on-field umpire to have spotted what would

    have been a very marginal no-ball is debatable but, as the replays were broadcast, every

    television viewer and the commentators could clearly see that it was at least a strong

    probability. In the end, the third umpire upheld the original decision and the batsman was

    given out. The best that can be said about the decision is that there was just enough

    possibility of doubt to make over-ruling the on-field umpire improper under the meta-

    rule. On the other hand, it has to be noted that many spectators and the majority of the

    commentators disagreed with this decision and felt that the no-ball could and should have

    been called. Here then, we have case of groups with equal epistemological privilege TV

    viewers, commentators and the third umpire disagreeing about the correct application

    of a ubiquitous skill, with the result that, in the view of many spectators, including

    unbiased spectators, the wrong decision was reached.

    The remaining events took place during South Africas first innings. Graham Smith, the

    South African captain, and a very good batsman who went on to score 105 runs, was the

    subject of a vociferous and unanimous appeal by bowler and close English fielders in

    respect of `caught behind (caught by the `wicket-keeper). Smith, who had scored only

    25

  • 8/6/2019 Hawk Eye Public Understanding

    26/30

    15 runs at the time, had swished at the ball and fielders close to the batsman, TV

    commentators and TV viewers had heard a sound, picked up by the stump microphone, as

    the ball appeared to brush the edge of the bat before being caught. Nevertheless, the on-

    field umpire gave `not out and the English team asked for a review. Once more the third

    umpire confirmed the on-field umpires decision and, once more, there were claims that a

    mistake had been made. The problem turned on the sound.

    Smith, interviewed on television later that day, agreed that there had been a sound but he

    thought it had been caused by something other than the ball touching the bat. These

    things can happen and the third umpires decision would have stood up to everyones

    scrutiny if the TV replay had shown clearly that the ball had not hit the bat, or if there

    was some other putative cause for the sound, but neither was the case. Still, the on-field

    umpires decision is the default so the third umpires decision could have been justified.

    The matter did not rest here, however, because it had been agreed beforehand by the

    authorities that sound would be taken to be an indicator of impact in the case of no

    obvious over-ruling feature.25 With no other cause of the sound identified, one would

    have expected the batsman to be given out by the third umpire. The really big problem,

    however, was the reason that gradually emerged to explain the third umpires decision to

    confirm the original not out decision; this was that he did not hear the sound! Various

    explanations for this have emerged but the one that seems to have stuck is a simple

    human error. The third umpire had the volume on the stump-microphone feed turned to

    `4 instead of `10. Once more an evident clash between parties of equal epistemological

    25 Interview on Radio 5 Live at lunchtime (10.30am UK time) on 16 Jan, with Giles

    Clarke, chairman of the English Cricket Board.

    26

  • 8/6/2019 Hawk Eye Public Understanding

    27/30

    status or in this case, superior status on the part of everyone with a television except the

    third umpire has led to the ontological authority of the umpire being questioned because

    of a transparent injustice.

    The third event occurred when the batsman, De Villiers, was given `out by the on-field

    umpire in virtue of the ball touching his glove before being caught by one of the fielders.

    De Villiers asked for a review and the third umpire advised `not out on the basis of TV

    replays (the putative contact was such that no clear sound was expected to be heard).

    Once more, commentators and viewers had a view that was equal to that of the third

    umpire and, unlike the no-ball issue discussed in the first example, there appeared little

    doubt that the evidence was ambiguous, suggesting that the ball could have touched both

    glove and bat and certainly not proving that it had missed both. Given the meta-rule that

    the on-field umpires decision is correct unless clearly shown to be wrong, many

    spectators and commentators believed that the correct decision was to uphold the original

    decision. As noted, this did not happen and the third umpire reversed the original

    decision, once again pitting the epistemological privilege of officials against the

    ubiquitous judgement of the spectators though in this case it was third umpires

    perception and knowledge of the rules that were challenged

    The fourth event occurred a little later and the same batsman De Villiers was

    concerned. In this case the appeal was again for `caught, the close fielders and bowler

    believing the ball had touched the inside edge of the bat before being caught by the

    wicket-keeper. The on-field umpire gave `not out. By this time England had used up

    their rights to review, one review having been wasted on a futile lbw appeal but one

    27

  • 8/6/2019 Hawk Eye Public Understanding

    28/30

    having been used up on the abortive case of the `Smith Sound. England could not,

    therefore, ask for a review. As it happens, television replays showed the ball had clearly

    hit the bat. Here the widely discussed problem was not the fact that it was the wrong

    decision TV viewers and players, as explained, have to some extent accommodated to

    umpires being wrong in difficult circumstances but the fact that the right to review had

    been used up by a badly processed decision on the part of the third umpire. Once more, it

    was the review process that was seen to be at fault.

    28

  • 8/6/2019 Hawk Eye Public Understanding

    29/30

    References

    Collins, H.M. (2010). Tacit and Explicit Knowledge, Chicago, IL: The University of

    Chicago Press)

    Collins, H.M. and Evans, R.J. (2007)Rethinking Expertise, Chicago, IL: The University

    of Chicago Press.

    Collins, H.M. and Evans, R.J. (2008) You Cannot be Serious! Public Understanding of

    Technology with Special Reference to Hawk-Eye,Public Understanding of

    Science, Vol. 17, No. 3, pp. 283308.

    Dreyfus, H.L. and Dreyfus, S.E. (1986) Mind Over Machine: The Power of Human

    Intuition and Expertise in the Era of the Computer, New York: Free Press

    Gower, David (2009) Third umpire strikes back as review system improves, The

    Sunday Times, 20 December 2009. Available online at:

    http://www.timesonline.co.uk/tol/sport/columnists/david_gower/article6962837.e

    ce [Accessed 7 April 2010]

    Jonkhoff, H.C. (2003) `Electronic line-calling using the TEL system in Stuart Miller (ed)

    Tennis, Science and Technology 2, International Tennis Federation, pp 377-383

    Miller, S. (2006) 'Evaluation Report, Auto-Ref 5,Appendix 1 - Accuracy Data, test #37 of

    120' (August 2006)

    Russell, J.S. (1997) The Concept of a Call in Baseball,Journal of the Philosophy of

    Sport, Vol. XXIV, pp. 21-37.

    29

    http://www.timesonline.co.uk/tol/sport/columnists/david_gower/article6962837.ecehttp://www.timesonline.co.uk/tol/sport/columnists/david_gower/article6962837.ecehttp://www.timesonline.co.uk/tol/sport/columnists/david_gower/article6962837.ecehttp://www.timesonline.co.uk/tol/sport/columnists/david_gower/article6962837.ece
  • 8/6/2019 Hawk Eye Public Understanding

    30/30

    Winograd, T. and Flores, F. (1986) Understanding Computers and Cognition: A New

    Foundation for Design, New Jersey: Ablex.

    30