View
216
Download
0
Category
Preview:
Citation preview
7/31/2019 Luke Bishop Research Project
1/40
Luke Bishop
Key Performance Indicators that discriminatewinning and losing in the knockout stages of
the 2011 Rugby World Cup!!E-mail: luke_michael_bishop@hotmail.com
7/31/2019 Luke Bishop Research Project
2/40
I have recently finished my BSc Sport and Exercise Science undergraduatedegree at Sheffield Hallam University. This is my final year dissertation,looking into the 2011 Rugby World Cup. I am looking to pursue a career inPerformance Analysis, and have been working as an analyst at Opta for the
past year.
If you choose to read my project, then thank you. Any comments or feedbackwould be much appreciated using the address provided on the first page.
Luke Bishop
!
7/31/2019 Luke Bishop Research Project
3/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page ii
Abstract
The objectives of the study were to investigate what performance indicators
discriminated winning and losing teams in the knockout stages of the 2011 Rugby
World Cup, looking to gain a better understanding of the influence of certain law
changes (Van Rooyen, Diedrick and Noakes 2010) on the tactical development of the
game. A review of preceding literature identified several performance indicators that
were found to be influential on match success in elite rugby; Lineout success,
turnovers conceded, frequency of kicks out of hand, tackle completion, line breaks,
penalties conceded and ruck frequency. An independent groups design was used,
comparing the means of winning and losing teams in the knockout matches (n=8). All
games were analysed by the primary investigator using a pre-set analysis template,
which allowed coding of the selected performance variables. Operational definitions
were assigned to each indicator prior to analysis. Intra- and inter-observer reliability
tests were completed, with correlation coefficients and percentage errors provided for
each variable. An effect size calculation was also applied to each variable. Following
a Mann Whitney U test, winners were found to concede a significantly higher
percentage of their penalties between halfway and the opposition 22m than losers (P
= 0.026) and losing teams were found to carry the ball significantly more than winning
teams (P= 0.035). Effect size calculations also identified meaningful, yet not
significant findings for several other variables. It was concluded that the ability to
concede penalties in more attacking positions and concede fewer turnovers, along
with a kicking, territory-based approach was more beneficial than a possession-
based one in the knockout stages of the 2011 Rugby World Cup.
7/31/2019 Luke Bishop Research Project
4/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 1
1.0 INTRODUCTION
Since its inception as a professional sport in the 1990s, Rugby Union has
advanced in terms of the scientific support that teams utilise. Nutritionists, strength
and conditioning coaches and psychologists have been introduced into modern day
rugby, all with the aim of enhancing the mental and physical performance of the
players and team. One of the most recent advances is the introduction of video
analysis support, aimed at identifying and improving match strategies by reviewing
video footage to gain a tactical advantage over the opposition.
Hughes (2004, p104) describes notational analysis as an objective way of
recording performance so that key elements of that performance can be quantified ina valid and consistent manner. The aim of notational analysis is to provide both
coaches and players with valuable information regarding sports performance that will
advance their decision-making (ODonoghue 2006) and understanding of tactical
issues in their sport, with an overall view to improving future performances (McGarry
2009). Prior to the introduction of performance analysis into sport, coaches would
have to make decisions based solely on their observation, which Franks and Miller
(1991) found to be poor. They reported that international level soccer coaches could
only recall 30% of what would be considered key events over the duration of a game.In addition to this, Bracewell (2002) suggests coaches struggle to remember rare
events and are unable to put events they do recall into context. It is suggested that
this is due to tension, emotion and individual bias a coach experiences during and
after a game. Jenkins, Morgan and ODonoghue (2007) successfully integrated
match analysis into the current coaching process of a netball team, highlighting that
using notational analysis as a way of overcoming the recall limitations of coaches, is
effective. Reilly and Gilbourne (2003) support these findings, suggesting that using
notational analysis as a coaching tool can provide detailed and accurate feedback forthe coach regarding positive and negative aspects of performance.
Lago-Ballesteros and Lago-Peas (2010) state that in order for identified
performance indictors to be useful, they have to be associated with success.
Therefore, studies have investigated how winning and losing teams differ with
regards to these performance indicators. There have been numerous studies across
several sports looking at the differences between winners and losers (Jones,
Mellalieu and James 2004, Csataljay et al. 2009, Lago-Peas, Lago-Ballesteros and
Rey 2011).
7/31/2019 Luke Bishop Research Project
5/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 2
To date limited research has been conducted into what performance indicators
differentiate winning and losing teams in Rugby Union. Indicators that distinguish
winning and losing found to be common across several studies are lineout success
(Ortega, Villarejo and Palao 2009, Vaz, Van Rooyen and Sampaio 2010, Jones,
Mellalieu and James 2004), turnovers conceded (Ortega, Villarejo and Palao 2009,
Vaz et al. 2011) and number of kicks out of hand (Stanhope and Hughes 1997,
Ortega, Villarejo and Palao 2009, Vaz et al. 2011). In addition, other secondary
indicators have been identified between winning and losing teams. These included
tackle completion percentage (Ortega, Villarejo and Palao 2009, Vaz et al. 2011), line
breaks (Ortega, Villarejo and Palao 2009), and penalties conceded (Jones, Mellalieu
and James 2004, Vaz et al. 2011). However, these studies have reported findings
from a mixture of international and domestic competitions, some of which are league
based and others are knockout based.
Van Rooyen, Diedrick and Noakes' (2010) study into ruck frequency is the only
article to look at knockout matches in the World Cup as a separate entity to the pool
stage matches. They found contrasting results for ruck frequency and match success
between the pool stages and knockout stages. This suggests that the format of the
competition may have an influence on a team's tactical approach and what
performance indicators are most important for success. Van Rooyen, Diedrick and
Noakes (2010) suggested that in the 2011 Rugby World Cup, the findings for ruck
frequency would be different to the 2007 tournament, due to the introduction of the
Experimental Law Variations (ELVs). Of the thirteen ELVs introduced in 2008, ten
were used at the 2011 World Cup.It is suggested that the ELVs may have an
influence on other performance indicators in modern rugby based on Van Rooyen,
Diedrick and Noakes (2010) findings.
This study sought to investigate the key performance indicators at the 2011
Rugby World Cup that distinguish winning and losing in the knockout stages. This
research will further our understanding of the tactical development of the modern
game, and how knockout rugby may or may not differ from other competition formats.
2.0 LITERATURE REVIEW
2.1. The use of Notational Analysis in Sport
2.1a. How Notational Analysis is used
Notational analysis aims to enhance performance by identifying the key
performance indicators (KPIs) in a particular sport. Performance Indicators are
7/31/2019 Luke Bishop Research Project
6/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 3
described as a combination of variables that identify a certain aspect of performance
and help to achieve success in a certain sport (Hughes and Bartlett 2002, Jones,
Mellalieu and James 2004). They are used to outline the differences between
winning and losing performances, thus allowing the coach to recognise which
elements are the most critical in determining the result (Csataljay et al. 2009).
Performance indicators for invasion games can be split into three groups: Match
Indicators (e.g. tries scored, number of lineouts), technical indicators (e.g. tackles
won/lost, number of completed passes) and tactical indicators (e.g. possession, pass
distribution) (Bartlett 2001, Hughes and Bartlett 2002). Match indicators help to
describe performance giving the analyst an idea of what the team or individual has
done successfully or unsuccessfully throughout a game. However, these indicators
can be misleading if presented on their own, offering no way of determining if
performance was good. They should be presented in comparison to previous
performances and/or the oppositions data, which puts the findings in context
(McGarry 2009). Technical indicators identify which particular skills a team or
individual are good or bad at. For example, if a rugby team consistently missed 50%
of their tackles in a game, it could be inferred that their tackling technique is poor,
meaning this could be addressed in future training sessions. Tactical indicators help
to identify how a team or individual has played the game, e.g. in soccer, a team may
focus their attack down the right wing, possibly to target the opposition left-back if
they have been identified as being poor. In rugby union, a team may kick the ball to
touch regularly if they feel that their opponents lineout is poor and can put the
opposition under pressure. Hughes and Bartlett (2002) note that all types of sports
(i.e. net and wall, striking and fielding) follow the same structure when identifying
performance indicators, although the indicators are adapted to the rules of each sport
(e.g. a match indicator in Tennis could be number of aces). However, the provision
and application of feedback has been suggested to be the most important factor in
improving performance (Hughes and Bartlett 2002, Liebermann et al. 2002), not
merely the identification of key performance indicators.
2.1b. Providing Feedback
Feedback is often given to the coach using video (ODonoghue 2006). Video
feedback improves clarity for coaches and players when trying to understand what is
being presented. This feedback is underpinned by statistics created from match
analysis. Statistics identify trends in performance indicators and give an unbiased
and objective record of the game, which can determine certain areas that require
7/31/2019 Luke Bishop Research Project
7/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 4
attention (ODonoghue 2007, Vaz et al. 2011). These trends govern what feedback is
given, providing an objective rationale for showing certain videos. It has been
suggested that any feedback given on the basis of statistical compilation must be put
into context (Bracewell 2002, McGarry 2009, Vaz et al. 2011) with regards to the
opposition, pitch position and previous performances. Vaz et al. (2011) and
Bracewell (2002) suggest using multivariate statistical techniques, by comparing
statistics with the opposition, in order to create more powerful results relative to that
particular match. This is supported by Tenga et al. (2009) who state that performance
analysis research must reflect the interaction between the two opponents, due to the
dynamic nature of invasion games, where the actions of each team or player is
influenced by that of their opposition (Grehaigne, Bouthier and Bernard 1997, Lames
and McGarry 2007). These statistics, if supported by video highlights, can provide an
important insight into relative match performance. If not put into context, statistics
produced are of little value, due to the lack of comparison with another amount. In
summary, the most effective form of feedback is using video, due to its facility to
improve clarity for both coaches and players. However, the underpinning of these
videos should be provided by trends identified in match statistics, which evidence
suggests should be reported in relation to the opposition and pitch position.
2.1c. Reliability and Validity of feedback
The most important methodological issue in performance analysis is ensuring
valid and reliable results. Valid and reliable results are imperative to making sure that
the aims and purposes of the study are effectively met (Tenga et al. 2009). Reliability
is described as the consistency of measurements made using an analysis system
(Wilson and Batterham 1999). In performance analysis, this would refer to the
consistency in which performance indicators are measured over time. James, Taylor
and Stanley (2007) report that reliability gives an indication of the validity of the
findings of a study. In performance analysis, validity is the extent to which the coded
events reflect what has happened during the analysed match.
James, Taylor and Stanley (2007) highlighted that there are three sources of error
in notational analysis; Operational Error, where the analyst presses the incorrect
button; Observational Error, where the analyst fails to code an event; Definitional
Error, where events are labelled incorrectly. It is suggested that to avoid definitional
errors in the analysis, that operational definitions are given to each variable. Hughes
(2004) states that these definitions need to be clear and precise, which will help to
enhance the reliability of the results. However, James, Taylor and Stanley (2007)
7/31/2019 Luke Bishop Research Project
8/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 5
suggest that poorly written definitions can lead to uncertainty for the analyst, meaning
events may be coded incorrectly; highlighting the influence they can have on the
findings of a study. An inter-rater reliability test can be used to address any errors
due to the misinterpretation of definitions. Atkinson and Nevill (1998) state that any
study that involves measurement has to have a measurement tool that is reliable. An
inter-rater reliability test will identify if the analyst is making regular mistakes.
However, if the second analyst is also coding these events incorrectly, it suggests
that there is possibly a problem with the operational definitions used (James, Taylor
and Stanley 2007). An intra-reliability test will not detect errors due to ambiguous
definitions, but will indicate how consistently the analyst codes events. Therefore, it is
suggested that if possible, both types of reliability tests should be completed to
provide a greater indication of the quality of the results presented.
Table 2.1. An analysis of different reliability tests carried out in recent performance
analysis research papers in Rugby Union.
(Eaves and Hughes 2003, Boddington and Lambert 2004, Jones, Mellalieu and James 2004,
Eaves, Hughes and Lamb 2005, James, Mellalieu and Jones 2005, Prim, Van Rooyen and
Lambert 2006, Van Rooyen, Lambert and Noakes 2006, Van Rooyen and Noakes 2006,
Sasaki et al. 2007, Williams et al. 2007, Ortega, Villarejo and Palao 2009, Van Rooyen,
Diedrick and Noakes 2010, Vaz, Van Rooyen and Sampaio 2010, Wheeler, Askew and
Sayers 2010, Williams, Hughes, ODonoghue 2010, Diedrick and Van Rooyen 2011, Vaz et
al. 2011).
The use of reliability procedures in previous performance analysis research has
been varied. Hughes, Cooper and Nevill (2002) considered the reliability of
procedures across 67 unspecified performance analysis studies. They found that
Reliability Test Reported Number %
Inter-analyst 3 17.5
Intra-analyst 7 41
Inter and Intra analyst 4 24
None 3 17.5
Total 17 100
7/31/2019 Luke Bishop Research Project
9/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 6
70% of studies did not mention any reliability procedures and 15% reported the use
of correlations. However, Bland and Altman (2010) suggest that using solely
correlations are often an inadequate method for confirming reliability. These studies
may have completed reliability tests, even though not reported. The minimal
discussion placed upon the reliability of their results suggests less importance was
placed upon this part of the research, with more discussion based on the overall
findings. It is suggested that more time should be spent discussing the reliability of
the study, as this helps to enhance the overall value of the presented findings
(Williams et al. 2007). A similar review (Table 2.1) is shown regarding the reported
reliability tests of recent Rugby Union literature.
2.2. Notational Analysis in Rugby Union
2.2a. Development of the Game at the World Cup
There has been a clear development in several areas of Rugby Union from
amateur rugby during the 1980's to the modern-day professional era (Eaves and
Hughes 2003, Eaves, Hughes and Lamb 2005, IRB 2003, 2005, 2007).
The first development has been the increase in total match time by approximately
five minutes from 1980's to 2000's. It has been suggested that this is due to an
advance in the laws in the modern game, with things such as blood replacements,yellow cards and the Television Match Official all adding on to the total match time
(IRB 2005). Half time is also now 3-4 times longer than during the 1980's (IRB 2005).
It is suggested that this is required in the modern game due to the increased ball-in-
play time (IRB 2005). Ball-in-play time has increased from averaging approximately
21-23 minutes in the 1980's (IRB 2005), compared to 35 minutes in the 2007 World
Cup (IRB 2007). An increased ball-in-play time leads to an increase in the number of
game actions, and therefore, the physical effort players exert is greater, meaning half
time is needed to be increased in order to allow players sufficient recovery time.In terms of game actions, there have been several changes identified over time.
Eaves and Hughes (2003) report the game has moved towards a quicker, more ruck
dominated game, with more phases of play in the professional era compared to when
the game was amateur. IRB (2003, 2007) support this, reporting that the average
number of rucks per game has increased from 69 per game (at the 1995 World Cup)
to 144 per game (at the 2007 World Cup). These findings coincide with how often
teams kick the ball, either in play or in to touch. Eaves, Hughes and Lamb (2005)
report a 22.5% decrease in the average number of kicks per game from the amateur
7/31/2019 Luke Bishop Research Project
10/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 7
to professional era. This is likely to be due to the introduction of laws that meant
teams gain no territory from kicking the ball directly in to touch from outside their 22m
area in the modern era. From an average of 80 passes per game in the 1980's (IRB
2005), numbers of more than 200 passes per game (IRB 2007) were reported at the
2007 World Cup. These findings relate back to the increased ball-in-play time in the
modern era, which result in increased game actions. Fewer handling errors per game
have been reported in the modern era (IRB 2005). It can be inferred that this relates
to an increase in skill level as the game has become professional. There has been a
decrease in the number of scrums over time, averaging 31 per game in the 1980's
compared to the 2000's where there is on average, 19 scrums per game (IRB 2005).
This could be linked to the reduction in handling errors, which are a common source
of scrums. In addition, a reduction in the number of lineouts is also apparent. Eaves,
Hughes and Lamb (2005) report a greater average frequency of lineouts during the
amateur era. An average of 52 per game was reported in the 1980's (IRB 2005)
compared to 31 per game at the 2007 World Cup (IRB 2007). This is possibly due to
the reduction of kicks in the modern era, which reduces the likelihood of a lineout
occurring. The most recent influence on the development of the game has been the
introduction of the ELVs in 2008. Of the thirteen laws introduced, three of them
(Table 2.2) were designed to influence how teams play, with their aim being to
reward attacking play.
In summary, it is suggested that due to law changes and an increase in
professionalism, teams have placed an emphasis on keeping possession of the ball,
through rucking and recycling the ball through several phase. There has become less
of a focus on kicking the ball either in play or in touch. These findings look at how the
game has changed in terms of the overall amount of game actions, but does not
identify what is the most preferential tactical approach to winning matches. The
constant development in teams tactical approaches due to increased fitness and skill
levels, and also due to changing laws, highlights the need for regular analysis into
what strategies and approaches are most suitable and relevant in elite Rugby Union.
7/31/2019 Luke Bishop Research Project
11/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 8
Table 2.2.Experimental Law Variations still in place during 2011 Rugby World Cup.
(IRB 2008)
2.3. Research in Rugby Union
Research into elite Rugby Union has varied. Some literature has looked at
patterns of play and its development over time (Eaves and Hughes 2003, Williams,
Hughes and ODonoghue 2005). Others have investigated the physiological
demands of sport, how this differs between positions and what the ideal physical
characteristics are at an elite level (Cunniffe et al. 2009, Austin, Gabbett and Jenkins
2011). More recently, the development and identification of performance indicators
has become more common (Hughes and Bartlett 2002, James, Mellalieu and Jones
2005). These indicators represent different aspects of the game, such as possession,
tackle completion and turnovers conceded. They are generally presented as either
times, frequency counts or as percentages. Developing from research into identifying
Law Effect on Game
If a team puts the ball back into its
own 22 and the ball is
subsequently kicked directly into
touch, there is no gain in ground.
This ensures that defending teams do not
have an unfair advantage over attacking
teams by encouraging tactical kicking and
counter-attacking skills.
A quick throw in may be thrown in
straight or towards the throwing
teams own goal line.
This increases the probability of a quick
throw-in, providing a positive opportunity for
the team taking the throw-in to run the ball
instead of choosing a lineout.
Introduction of an offside line 5
metres behind the hindmost feet
of the scrum.
This increases the space available to the
team who wins the ball at the scrum. By
having all the forwards committed at the
scrum itself and 10 metres between theback lines, more space is available to build
an attack in.
7/31/2019 Luke Bishop Research Project
12/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 9
general performance indicators, current research has begun to investigate what
performance indicators are related to success in elite Rugby Union (Ortega, Villarejo
and Palao 2009, Van Rooyen, Diedrick and Noakes 2010, Vaz, Van Rooyen and
Sampaio 2010). This is largely due to increased professionalism in the sport,
resulting in the need for improved scientific and analytic support aimed at improving
performance.
2.3a. Performance Indicators Associated With Success
Previous studies into which performance indicators determine winning and losing
in Rugby Union have found differing results, but some indicators have been found to
be common across several studies. Success at the lineout has been reported as
being a determining factor between winning and losing in Rugby Union. Ortega,
Villarejo and Palaos (2009) (p=
7/31/2019 Luke Bishop Research Project
13/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 10
lower percentage pass completion, which is likely to lead to situations where
turnovers will be conceded. However, both of these studies fail to suggest whether
pitch location of the turnovers have an influence on the match outcome, with neither
putting their findings into any context, other than comparing winners and losers. From
the limited evidence available, it appears that maintaining control of the ball when in
possession, and forcing errors by the opposition when in defence are related to
successful performance.
How teams use possession of the ball can help to identify their game tactics. A
team who produce a lot of passes and offloads may be typical of a team looking to
play fast, expansive rugby, whereas a team who kicks the ball a lot, but attempts
fewer passes, could be indicative of playing a more measured, territorial game.
Previous research (Ortega, Villarejo and Palao 2009, Vaz, Van Rooyen and Sampaio
2010 (p= 0.01)) has identified that winning teams produced more kicks in field and
kicks to touch than losing teams. Ortega, Villarejo and Palao (2009) also suggest
losing teams tend to pick and go from rucks, and pass the ball out wide more often
than winners. These findings suggest that kicking the ball as a form of attack appears
to be more beneficial than attacking with ball in hand. It could be suggested that if a
team kicks well, they will create a territorial advantage, thus increasing pressure on
the opposition. This will increase the likelihood of mistakes being made and point
scoring opportunities being presented to the team who has kicked the ball. Jones,
Mellalieu and James (2004) and Vaz et al. (2011) both report successful teams are
able to gain more penalties in attacking areas of the pitch. They suggest this is due to
a successful kicking and rucking game, which is used to build pressure. This
increases the likelihood of penalties being conceded by the defending team, again
suggesting a kicking oriented game is linked with match success. Van Rooyen and
Noakes (2006) in a comparison between South Africa, England, Australia and New
Zealand at the 2003 World Cup, reported that South Africa had more defensive kicks
than the other three teams, who had a higher number of kicks in attacking positions.
South Africa were the least successful out of these teams, so it could be inferred that
pitch position of kicks could also be an indicator of match success, not just the
frequency of kicks.
Diedrick and Van Rooyen (2011) suggest that try scoring is strongly linked with
overall success. This is supported by Jones, Mellalieu and James (2004) (p=
7/31/2019 Luke Bishop Research Project
14/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 11
Rooyen, Lambert and Noakes (2006) support the idea that game tactics alter during
knockout matches, as the most successful team at the 2003 World Cup, England
scored the most penalties and drop goals, with third placed New Zealand scoring the
most tries. However, Ortega, Villarejo and Palao (2009) (p= 0.01) report losing teams
to score more penalty goals than winning teams, perhaps devaluing their importance
for overall match success. This could possibly be due to the winning team having a
strong defence, meaning try scoring opportunities would be infrequent, leading to
goal kicks being their most favourable option to score points.
Ruck frequency was found to differ between winning and losing knockout matches
at the 2007 Rugby World Cup (Van Rooyen, Diedrick and Noakes 2010). They
reported that teams with a lower ruck frequency than their opponents won 100% of
the knockout matches. This finding was opposite to the pool stage matches, where
increased ruck frequency was associated with success. This suggests knockout
rugby requires a different tactical approach to league based matches. It appears that
the ability to take point scoring opportunities more often and the ability to defend well
as a team is more beneficial during knockout rugby, rather than having control of the
ball for longer than the opposition. Stanhope and Hughes (1997) found higher ruck
frequency led to more success throughout the 1991 World Cup. Eaves, Hughes and
Lamb (2005) suggest keeping possession and completing more rucks is more
important now than in previous years in an analysis of 5 Nations and 6 Nations
games from 1988 to 2002. They suggest a lineout almost guarantees possession to
the side throwing the ball in, meaning kicking to touch is effectively handing
possession to the opposition and is not an successful tactic. This suggests there has
possibly been a development in the importance of rucking and maintaining
possession of the ball, although there is not enough evidence to come to any definite
conclusions at present.
There have been other performance indicators reported to differ between winning
and losing teams, but have not been reported to be as influential as previously
mentioned indicators. Ortega, Villarejo and Palao (2009) report winning teams to
have a higher tackle completion percentage than losing teams, in addition to making
more tackles than their opponents during Six Nations matches. Vaz, Van Rooyen
and Sampaio (2010) support these findings, reporting successful teams to make
more tackles than the opposition during Super 12 fixtures. These findings suggest
winning teams spend long periods without possession, due to the greater number of
tackles they have to make. The fact winning teams are able to maintain a tackle
completion percentage greater than that of their opponents while making a greater
7/31/2019 Luke Bishop Research Project
15/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 12
total number of tackles, suggests the ability to defend is key to success in Rugby
Union. However, it could be that a combination of good attack from one team and
bad defence from the other is key to success in Rugby Union. Wheeler, Askew and
Sayers (2010) report top teams in the Super 12 competition are able to break tackles
with 19% of their carries, compared to 16% for the mid-table teams and 11% for the
bottom teams. These findings suggest top teams are more proficient in attack in
comparison to less successful sides. However, it remains unclear whether their
findings were as a result of good attacking by the successful sides or poor defending
from opponents.
Ortega, Villarejo and Palao (2009) report winning teams successfully break the
defensive line when attacking more often than losing teams do. This is supported by
Diedrick and Van Rooyen (2011) who reported that at the 2007 World Cup, an
increase in initial breaks was related to match success. However, Wheeler, Askew
and Sayers (2010) report that the number of line breaks achieved by a team is not
associated with success in Rugby Union, but the number of defenders beaten seems
to be a better predictor of success. However, these two studies looked at different
competitions (6 Nations and Super 12) where there are clear dissimilarities in how
teams play, offering an explanation as to why they reported contrasting results.
2.4. Aims and hypotheses
The purpose of this study was to examine which of the previously identified
performance indicators are significant in determining the result of Rugby World Cup
2011 knockout matches. These findings may provide a basis for possible training
interventions and suggestions in tactics for knockout rugby. A comparison of the
chosen performance indicators (Lineout success %, turnovers conceded, kicks out of
hand, total carries, total passes, ruck frequency, tackle completion %, line breaks and
penalties conceded) was made with findings from previous studies in order to allow
inferences to be made as made as to what the influence of the Experimental Law
Variations has been, and offer any suggestions as to how and why the game has or
has not changed tactically. Suggestions can then be made as to what was the most
suitable tactical approach during the 2011 Rugby World Cup knockout stages.
It is hypothesised that, based on findings and suggestions of previous research,
winning teams will have a more successful lineout, will concede fewer turnovers, kick
possession away more often and complete less attacking rucks than losing teams
during the knockout stages of the 2011 Rugby World Cup. In addition to these
primary performance indicators, secondary performance indicators will show that
7/31/2019 Luke Bishop Research Project
16/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 13
winning teams complete a higher percentage of their attempted tackles, break the
defensive line more often, and concede fewer penalties than losing teams.
3.0 METHOD
3.1. Design
The study was an independent groups design. Both winning and losing teams
from the knockout matches of the 2011 Rugby World Cup formed the basis for
comparison. The identified performance indicators for each group were compared in
an attempt to identify those factors that distinguish winning and losing teams.
3.2. Sample
Data were gathered from the knockout matches of the 2011 Rugby World Cup (n=
8), resulting in eight winning teams and eight losing teams. Only knockout matches
were chosen for analysis as teams at this stage are generally of a similar standard,
meaning tactical differences in a team's approach can be crucial in determining the
result. This differs from games in the pool stages, where a high proportion of games
are won due to clear differences in playing ability. Thus, meaning a teams tactics are
less important at this stage of the competition.
3.3. Procedure
The eight knockout matches were individually uploaded into the Rugby Union
DVD 11 (Opta Sports Data, Leeds, United Kingdom) analysis software. The primary
investigator undertook all of the match analysis. Each match was observed and
coded using a pre-set analysis template (Appendix 2), which allowed the coding of
the selected performance variables; Lineout success, Turnovers conceded, Kicks out
of hand, Passes, Carries (divided into types of carry; Pick and Go, One Out Drive,
Other Carry, Support Carry and Kick Return), Ruck frequency, Tackle completion,
Initial breaks, Penalties conceded. Each variable coded was recorded onto a
timeline; giving the match time it took place as well as the pitch co-ordinates, which
indicate where on the pitch the event took place.
After the games had been coded, the data recorded onto the timeline were
exported from the analysis software into Microsoft Excel, (Microsoft Corporation,
Washington, USA) where the final set of data for each match was displayed. Data
analysis was then completed.
7/31/2019 Luke Bishop Research Project
17/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 14
3.4. Identification of Performance Indicators
Performance Indicators were identified following a review of previous Rugby
Union literature in performance analysis. Primary indicators had been found to differ
between winning and losing teams common across several studies. Secondary
indicators were reported to discriminate between winners and losers in some studies,
but not to the extent of the primary indicators. The chosen indicators were adapted to
produce a list of the analysis variables for this study. Once these variables had been
identified, operational definitions (Table 3.1, 3.2) were given to each variable.
Hughes (2004) notes how it is important these definitions are clear and precise to
increase the reliability of the analysis.
3.5. Reliability Procedures
Both intra-observer (data compared between original analysis and re-analysis for
three matches) and inter-observer reliability (data compared with an analyst who is
equally familiar with the software for all matches) tests were undertaken. This was
done to ensure that the analyst reliably recorded each performance indicator, and
also to guarantee that the system used for the analysis had test-retest reliability
(James, Taylor and Stanley 2007). The inter-observer test was used to supplement
the information provided by the intra-observer test. Both reliability tests were
undertaken on each performance indicator individually, providing a percentage error
and correlation coefficient for each variable. Re-analyses for intra-observer reliability
were completed two weeks after the original analysis.
3.5a. Percentage Error
Each performance indicator was compared, and a percentage difference between
the two analyses calculated. This was measured using a simple test-retest
agreement method to find the total % error between the original test (A), and re-test
(B) scores, using the following equation (O'Donoghue 2010):
( A - B )
Total % Error = 100 x
( A + B ) / 2
7/31/2019 Luke Bishop Research Project
18/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 15
Using this method, the percentage error was calculated by expressing the
absolute error between two values as a percentage of the mean of the two values.
Cooper et al. (2007) suggest that there should be an agreement of!90% between
the compared analyses, i.e. "10% error.
3.5b. Correlation Coefficient
Pearson's coefficient of correlation was used for inter-observer reliability, whereas
an Intraclass Correlation was used for intra-observer reliability. These tests
determine the association between the sets of data. A correlation of 0.0 represents
no association between the values, whereas a correlation of 1.0 denotes perfect
agreement, and therefore good reliability. Values of#0.7 indicate a moderate
correlation, #0.8 is good and #0.9 denotes a very good association.
3.6. Data Analysis
Descriptive statistics (means, medians and standard deviations) were provided to
present the findings. The Shapiro-Wilk test showed that the majority of dependent
variables were normally distributed. However, several variables were not normally
distributed. This, accompanied by the small sample size, suggested a non-parametric
approach. A Mann Whitney U test was used to identify any statistical differences
between winning and losing teams at a statistical significance level of 95%.Due to the small sample size, effect sizes were reported for each variable, which
will help to identify trends in the data. From these trends, suggestions could be made
as to what variables may be found to be significant had the study been conducted
using more matches. Effect size (ES) was calculated using Cohens (1969) equation:
ES = (Mean1 Mean2) / Mean Standard Deviation
Cohen reports figures of 0.2 (small ES), 0.5 (medium ES) and!0.8 (large ES) as
guidelines for interpreting effect size.
All data analysis was completed using Excel 2011 (Microsoft Corporation,
Washington, USA) and SPSS 20.0.0 (IBM Corporation, New York, USA).
7/31/2019 Luke Bishop Research Project
19/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 16
Table 3.1. Operational definitions of Primary match indicators
Primary Match
Indicator
Definition
Lineout success (%) A successful lineout is determined when the attacking
team win the ball cleanly, win a free kick or penalty, or
win the ball in any other way. An unsuccessful lineout is
when the defending team wins the ball in any of the
methods identified above.
Turnovers conceded When the attacking team give possession to the
opposition through an error on their part (accidental
offside, bad pass, carried dead, forward pass, unforced
knock-on, carried in touch) or due to good work from
the defending side (jackal, forced in touch, forced
knock-on).
Kick in play When an attacker kicks the ball in open play, from
either inside or outside of their 22-metre area.
Passes When an attacker attempts a pass to a teammate. This
can be positive, i.e. complete, or negative (incomplete,
off target, forward, intercepted).
Carries When an attacker carries the ball towards the defensive
line. Types include; Pick and go, One out drive, Support
carry, Kick return and Other carry.
Pick and Go When an attacker carries the ball from the base of a
ruck or a maul.
One Out Drive A carry made following one (usually short) pass from a
ruck or maul, often to a forward, who takes the ball into
contact.
Support Carry A carry made following an offload from a team mate, or
after a receiving a pass from a team mate who has
broken the defensive line.
Kick Return A carry made in the first phase following the reception
of a kick by the opposition.
Other Carry Any carry made that does not fit into one of the other
four types of carry.
7/31/2019 Luke Bishop Research Project
20/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 17
Ruck Frequency The number of rucks the attacking team completes
successfully throughout the match. A ruck is where two
or more players from each team come together over the
ball (which is on the ground), often following a tackle.
Table 3.2 Operational definitions of Secondary match indicators
Secondary Match
Indicator
Definition
Tackle Completion (%). A completed tackle occurs when the ball carrier is held
by one or more defenders, causing the ball carrier to go
to ground, stay on their feet, or offload the ball from thetackle area. This will be recorded as a percentage of
the total number of attempted tackles.
Attempted Tackles Frequency of tackles attempted. This includes
successful tackles where the tackle is completed, and
unsuccessful ones, i.e. missed tackles.
Initial Breaks. When the ball carrier cleanly breaks the defensive line,
gaining a better territorial position for their team.
Penalties conceded This is when the referee awards a penalty against theteam or player who have broken the laws of the game.
4.0. RESULTS
4.1. Reliability
Reliability analyses undertaken on the data were completed in order to provide
information as to the value of the presented results. Pearsons Correlation Coefficient
(for inter-observer), Intraclass Correlation Coefficient (for intra-observer) and
percentage error were provided for the variables. Table 4.1 presents intra-observer
values, taken from the re-analysis of three matches. Table 4.2 shows inter-observer
values obtained from a comparison of all matches with another analysts data.
7/31/2019 Luke Bishop Research Project
21/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 18
Table 4.1. Correlation Coefficients and % Error for Intra-observer analysis.
aIntraclass Correlation Coefficient
4.1a. Intra-observer Reliability
Results from the Intra-observer analysis in Table 4.1 indicate that the majority of
variables for winners and losers showed at least a moderate to good relationship.
Four variables (Initial Breaks, Penalties Conceded, Turnovers Conceded and Lineout
Success %) showed a perfect correlation (1.000) between the two analyses for
winners and losers. The lone irregular correlation was found for tackle completion %
for losers (0.588), which, although is positively correlated, shows a less than
moderate relationship between the original analysis and re-analysis. Percentage
Error was found to be within the 10% limit suggested by Cooper et al. (2007) for
100% of the variables. The smallest percentage error was 0%, found for winners and
losers in initial breaks, penalties conceded, turnovers Conceded and lineout success
%. The largest error was reported for total passes (4.92%) for losing teams, which is
still well within the recommended 10% limit.
Winners Losers
Variable Correlation
Coefficient a% Error Correlation
Coefficient a%
Error
Tackle Completion (%) 0.758 2.56 0.588 2.98
Total Carries 0.990 2.91 0.879 3.72
Total Passes 0.998 0.42 0.993 4.92
Initial Breaks 1.000 0.00 1.000 0.00
Total Kicks 0.929 2.06 0.969 2.44
Penalties Conceded 1.000 0.00 1.000 0.00
Turnovers Conceded 1.000 0.00 1.000 0.00
Ruck Frequency 0.993 1.45 0.993 0.70
Lineout Success (%) 1.000 0.00 1.000 0.00
7/31/2019 Luke Bishop Research Project
22/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 19
Table 4.2. Correlation Coefficients and % Error for Inter observer analysis.
Winners Losers
Variable Correlation
Coefficient b% Error Correlation
Coefficient b%
Error
Tackle Completion (%) 0.905 1.09 0.789 2.36
Total Carries 0.994 2.98 0.982 2.00
Total Passes 1.000 0.00 1.000 0.00
Initial Breaks 1.000 0.00 1.000 0.00
Total Kicks 0.992 1.67 0.999 0.53
Penalties Conceded 1.000 0.00 1.000 0.00
Turnovers Conceded 1.000 0.00 1.000 0.00
Ruck Frequency 0.998 1.61 0.995 1.95
Lineout Success (%) 0.976 1.46 0.981 1.59
bPearsons Correlation Coefficient
4.1b. Inter-observer Reliability
Table 4.2 shows how good to perfect correlation coefficients were reported for all
variables for the inter-observer analysis. Perfect correlations (1.000) were reported
for four variables for winners and losers (total passes, initial breaks, penalties
conceded and turnovers conceded). Percentage error was found to be well within the
10% error limit, with the largest error being reported for Total Carries (2.98%).
Overall, there was very good agreement between the two analyses.
4.2. Key Performance Indicators
Table 4.3 presents descriptive statistics (Median, Mean and Standard Deviation
(SD)) for winners and losers. It also shows effect sizes from Cohens dtest and P
values from the Mann Whitney U test for each of the analysed performance
indicators. Variables are displayed as either frequency counts or percentages.
Overall, there were very few significant differences between winners and losers,
although there did appear to be some trends in the data. Two variables were found to
be significantly different between winners and losers. Winners conceded a
significantly higher percentage of their penalties between 50m and the opposition
22m than losers (P= 0.026) (Figure 4.2c) and losers carried the ball significantly
7/31/2019 Luke Bishop Research Project
23/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 20
more than winners (P= 0.035). While the total number of carries was significantly
higher for losing teams, there was no noticeable difference in the types of carries
between the groups (Figure 4.1).
Figure 4.1. Pie chart showing comparison between winners and losers in terms of
each type of carry as a percentage of total number of carries.
Figure 4.2. Bar chart showing comparison between winners and losers in distribution
of a) Turnovers conceded, b) Ruck frequency and c) Penalties conceded, as a
percentage of total number of each event. (* = Statistical significance, P= 0.026).
* *
7/31/2019 Luke Bishop Research Project
24/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 21
Effect sizes in Table 4.3 showed that some variables differed between winning
and losing teams, although not statistically significantly. Losers conceded more
turnovers than winners (ES = 0.89, P= 0.153); Winners conceded a higher
percentage of their turnovers between their own 22m and 50m than losers (ES =
0.87, P= 0.205) (Figure 4.2a); Winners kicked the ball out of hand more often than
losers (ES = 0.82, P= 0.223); Losers attempted more passes (ES = 0.91, P=0.078)
and had a higher pass completion (ES = 0.85, P= 0.157) than winners; Losers
completed more rucks than winners (ES = 0.81, P= 0.152), although no difference
was noted in terms of where on the pitch these rucks took place (Figure 4.2b); Losers
conceded a higher percentage of their penalties between their own 22m and 50m
than winners (ES = 0.92, P= 0.088) (Figure 4.2c)despite there being no difference inthe total number of penalties conceded.
.
7/31/2019 Luke Bishop Research Project
25/40
KeyP
erform
a
nceIn
dic
ators
thatdis
crimin
atewinning
an
dlo
sin
gin
thekn
ock
outstagesof
the2
011R
ugbyW
orld
Cup.
Luk
eBish
op
P
age22
Table4.3.D
ifferen
cesink
eyperform
an
cein
dicators
betweenwinn
ers
an
d
losers
durin
gthekn
ock
outstag
esof
the2
011R
ugbyW
orld
Cu
p.
Winners
Losers
Variable
Median
Mean
SD
Median
Mean
SD
EffectSize
P1Value
LineoutSucce
ss(%)
89.0
0
85.6
3
11.3
9
87.0
0
91.0
0
12.
80
0.11
0.6
6
4
TurnoversCon
ceded
13.5
0
13.7
5
3.11
16.5
0
17.1
3
4.4
5
0.89*
0.1
5
3
Own22m(%)
3.0
0
5.2
5
7.0
9
8.5
0
8.2
5
5.04
0.4
9
0.24
5
22m-50m
(%)
34.5
0
35.3
8
9.5
9
30.5
0
25.8
8
12.14
0.87*
0.2
0
5
50m-Opposition22m(%)
33.0
0
37.3
8
18.21
46.0
0
45.7
5
10.
59
0.5
8
0.27
7
Opposition
22m(%)
23.0
0
21.8
8
13.62
20.0
0
20.12
8.92
0.1
6
0.6
9
7
TotalKicksou
tofhand
28.5
0
29.6
3
5.21
26.0
0
24.4
0
7.3
3
0.82*
0.22
3
Kicksin22
m(%)
30.0
0
30.8
8
12.2
9
35.0
0
35.7
5
10.48
0.4
3
0.3
6
8
KicksinPlay(%)
70.0
0
69.1
3
12.2
9
65.0
0
64.2
5
10.48
0.44
0.3
6
8
Passes
97.0
0
100.3
8
39.81
154.5
0
139.0
0
45.
01
0.91*
0.07
8
PassComp
letion(%)
96.0
0
96.0
0
2.62
98.0
0
97.5
0
0.9
3
0.85*
0.1
5
7
Carries
92.0
0
91.0
0
26.27
114.0
0
11
6.8
8
32.
30
0.88*
0.035
**
PickandG
o(%)
16.0
0
15.6
3
7.87
12.5
0
14.1
3
6.62
0.21
0.64
4
OneOutDrive(%)
16.5
0
19.0
0
7.4
3
19.0
0
21.0
0
9.32
0.24
0.52
1
SupportCa
rry(%)
4.0
0
2.5
9
4.1
3
6.5
0
5.2
5
2.87
0.41
0.3
9
3
7/31/2019 Luke Bishop Research Project
26/40
KeyPerforma
nceIndicatorsthatdiscriminatewinningandlosingintheknockoutstagesofthe2011RugbyWorldCup.
LukeBishop
P
age23
KickReturn
(%)
11.50
13.00
6.97
9.00
10.00
4.14
0.54
0.51
4
OtherCarry
(%)
48.50
47.38
7.37
48.00
49.50
8.00
0.28
1.00
0
RuckFrequency
79.00
76.00
24.75
100.50
98.00
29.49
0.81*
0.15
2
Own22m(%
)
4.50
7.00
6.46
5.00
6.75
6.32
0.04
0.90
4
22m-50m(%)
22.50
27.38
13.76
22.00
25.13
6.24
0.23
0.89
8
50m-Oppo
sition22m(%)
47.00
43.63
13.55
45.00
47.00
14.59
0.24
0.62
7
Opposition
22m(%)
27.00
22.00
11.16
17.50
21.25
13.59
0.06
0.62
5
TackleComple
tion(%)
92.00
91.75
4.37
91.00
90.50
2.45
0.37
0.52
2
AttemptedTac
kles
124.5
122.75
32.87
103.00
98.13
39.75
0.68
0.16
9
InitialBreaks
2.00
2.50
1.77
2.50
2.63
1.06
0.09
0.62
1
PenaltiesConceded
7.00
7.75
2.19
8.50
8.38
2.67
0.26
0.55
5
Own22m(%
)
12.00
16.00
16.66
15.50
14.63
11.41
0.10
0.81
3
22m-50m(%)
27.00
29.38
20.87
43.50
45.44
13.99
0.92*
0.08
8
50m-Oppo
sition22m(%)
35.50
39.13
18.31
19.50
22.00
10.59
1.09*
0.026
**
Opposition
22m(%)
13.50
15.50
13.62
10.50
17.19
8.92
0.10
0.89
7
1MannWhitneyU.
*LargeEffectS
ize(>0.80).
**SignificantP
value(
7/31/2019 Luke Bishop Research Project
27/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 24
5.0 DISCUSSION
5.1. Summary
The purpose of the study was to identify performance indicators that discriminate
between winners and losers in the knockout matches at the 2011 Rugby World Cup.
This would then allow suggestions to be made as to the tactical development of the
game, and recommendations to be made about what was the most appropriate
tactical approach during the 2011 Rugby World Cup knockout stages.
The findings from the study indicate that there were very few clear differences
between winning and losing teams. The only significant differences were that winners
conceded a higher percentage of their penalties between the halfway line and the
opposition 22m than losers, and losing teams carried the ball more often than
winners. In addition to these significant differences, it was found that following an
effect size calculation, other variables may have been significantly different between
winners and losers had there been a larger sample size (winning teams kicked the
ball out of hand more often than losing teams; winners conceded a higher
percentage of their turnovers between their 22m and the halfway line than losing
teams; losing teams conceded more turnovers, attempted more passes, had a higher
pass completion, had a higher ruck frequency and conceded a higher percentage of
their penalties between their 22m and the halfway line than winning teams).
5.2. Differences between winners and losers
Winning teams conceded a significantly higher percentage of their penalties
between halfway and the opposition 22m (mean = 39.13%) than losing teams (mean
= 22%). When comparing this to the finding that losing teams conceded a higher
percentage of their penalties between their 22m and halfway (mean = 45.44%)
compared to winning teams (mean = 29.38%), it could be suggested that pitch
location of penalties conceded had an influence on the match outcome. Conceding
penalties in the defending half increases the likelihood of conceding points through
subsequent goal kicks, or from a possible territorial loss. This supports the view that
winning teams are able to gain more penalties in attacking positions than losing
teams (Jones, Mellalieu and James 2004, Vaz et al. 2011). Therefore, it is suggested
that a teams ability to concede penalties further up the field, outside the point scoring
range of the opposition, is indicative of success during the knockout stages of the
2011 Rugby World Cup. The pitch position seems to be of more importance than
7/31/2019 Luke Bishop Research Project
28/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 25
simply the frequency of penalties conceded, where there was a negligible difference
between winning and losing teams.
Losing teams (mean = 17.13) were found to concede more turnovers than winning
teams (mean = 13.75), which supports findings from previous research (Ortega,
Villarejo and Palao 2009, Jones, Mellalieu and James 2004). There was found to be
minimal difference in terms of pitch position of turnovers conceded between winners
and losers apart from between 22m and halfway in the defending half. Winning
teams (mean = 35.38) conceded more turnovers here than losing teams (mean =
25.88). This finding was not anticipated as tries most frequently originate from this
area of the pitch (Van Rooyen, Lambert and Noakes 2000, Boddington and Lambert
2004). This would suggest that a higher number of turnovers conceded here is likely
to be related to unsuccessful performance. However, it appeared not to be the case
in the 2011 Rugby World Cup.
Overall, findings from the current study suggest that the number of penalties
conceded did not discriminate between winning and losing teams at the 2011 Rugby
World Cup. However, pitch position of penalties conceded did appear to be influential
on the match outcome. The capability of teams to concede a high percentage of their
penalties in more attacking positions in comparison to their opponents was shown to
differentiate winning and losing teams. Contrary to this, the frequency of turnovers
conceded seems to help discriminate winners and losers more so than the pitch
position that these turnovers take place. A lower frequency of turnovers conceded
was apparent in winning teams.
5.2a. Style of Play
Losing teams (mean = 116.88) carried the ball significantly more than winning
teams (mean = 91). However, there was found to be no differences in the types of
carries by either group (Figure 4.2), which would give an indication of their tactical
approach, i.e. more Pick and Go carries would indicate a more narrow, less
adventurous approach, whereas more Other Carries would indicate a more
expansive approach. It is suggested that winning and losing teams adopted a similar
approach in terms of their distribution of carry types. These findings do not support
Ortega, Villarejo and Palao (2009), who suggested that losing teams pick and go
more often than winning teams. This difference is possibly due to the analysis being
completed on different competition formats (6 Nations and World Cup) where it has
appeared different tactical approaches are needed. Losing teams (mean = 139) were
also found to pass the ball more often than winning teams (mean = 100.38), which
7/31/2019 Luke Bishop Research Project
29/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 26
supports Ortega, Villarejo and Palao's (2009) findings that losing teams pass the ball
more frequently than winning teams. Losing teams percentage pass completion was
higher than winning teams, which contradicts the findings of Ortega, Villarejo and
Palao (2009), who reported a lower percentage pass completion for losing teams.
This was an unanticipated finding, as pass completion and turnovers conceded are
somewhat related (Ortega, Villarejo and Palao 2009), so it was expected that a
superior pass completion would result in an inferior number of turnovers conceded.
Winners (mean = 29.63) were found to kick the ball more often than losers (mean =
24.40), which supports the previous findings of Ortega, Villarejo and Palao (2009)
and Vaz, Van Rooyen and Sampaiao (2010) who all found winning teams to kick the
ball as a form of attack more frequently than losing teams. It does however disagree
with Eaves, Hughes and Lamb (2005) who suggested that keeping possession was
more related to match success than kicking the ball. Although it was found that
winning teams kicked more often in attacking positions than losing teams, this
difference was minor. While this agrees with Van Rooyen and Noakes' (2006)
findings that pitch position of kicks is linked to success, the findings from the current
study suggest that the frequency of kicks was more influential on match outcome
than the pitch position of these kicks. The findings suggest that kicking as a form of
attack is more associated with success than choosing to run with the ball. This
suggests that the impact that the Experimental Law Variations (Table 2.2) were
designed to have, has maybe not worked. The laws introduced were designed to
reward teams who look to counter-attack and run with the ball as opposed to kicking
it. Based on the results of this study, it is suggested that running and passing the ball
is not linked with success, signifying that the new laws have not favoured teams who
play more expansively. Contrary to this, perhaps winning teams have simply adapted
better to the law change with the use of better tactical kicking than losing teams and
as a result are being rewarded for this.
Losing teams (mean = 98) completed more rucks than winning teams (mean =
76). These findings are in agreement with what was found by Van Rooyen, Diedrick
and Noakes (2010) who reported that lower ruck frequency was linked to success in
the knockout stages of a World Cup. However, the findings in the current study do
not agree with the suggestion made by these authors that an increase in ruck
frequency will be related to success in the 2011 Rugby World Cup due to the
influence of the Experimental Law Variations. This again suggests that the
Experimental Law Variations have not had the influence the IRB had intended, with
more attacking play not appearing to relate to match success. It does seem that
7/31/2019 Luke Bishop Research Project
30/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 27
these laws are having an influence on the overall development of the game though.
There was an average of 174 rucks per game in the 2011 knockout stages compared
to 121 rucks per game in the 2007 knockout stages. Thus, suggesting that while the
laws have had an impact on the number of rucks completed during a game, perhaps
making the game a better spectacle for supporters, it appears that a rucking,
possession based approach is not linked with match success.
In terms of the style of play required for success in the knockout stages of the
2011 Rugby World Cup, it seems a game based around good tactical kicking and
territory was more effective than a possession dominated game plan, built around
rucking and carrying the ball.
5.3. Similarities between winners and losers
There were some variables that were found to be similar between winning and
losing teams, despite what previous research had suggested. These variables were;
Lineout success %, tackle completion % and initial breaks.
Firstly, a somewhat unexpected result was that the difference in lineout success %
between winners (mean = 85.63%) and losers (mean = 91%) was statistically
insignificant. In fact, the findings show that losing teams actually had a slightly better
average lineout success %. This suggests that success in the lineout was not
essential for success in the knockout stages of the 2011 Rugby World Cup. These
findings contrast to several studies, which all reported that success at the lineout was
indicative of match success (Jones, Mellalieu and James 2004, Ortega, Villaerjo and
Palao 2009, Vaz, Van Rooyen and Sampaio 2010, Vaz et al. 2011). A possible
reason for the findings of the current study opposing previous research is that the
previous studies looked at different competition formats, namely 6 Nations and Super
12, rather than specifically knockout matches. This further supports the idea that
different match indicators may be important in different competition formats.
There was also found to be no clear difference in tackle completion % between
winning (mean = 91.75%) and losing teams (mean = 90.50%), which differs from
what was reported by Ortega, Villarejo and Palao (2009). Despite, the tackle
completion % being similar, it was noted how winning teams (122.75) attempted a
greater average number of tackles per game than losing teams (98.13), although this
was not reported to be a significant difference. This agrees with Vaz, Van Rooyen
and Sampaio (2010) who report that winning teams make a greater number of
tackles than losing teams. From these results it could be suggested that during the
knockout stages of the 2011 Rugby World Cup, tackle completion % alone was not
7/31/2019 Luke Bishop Research Project
31/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 28
enough to give an indication of success. However, combining the percentage with the
number of attempted tackles gives a greater indication of a team's defensive
capabilities. This study suggests that maintaining a high tackle completion % whilst
attempting a greater number of tackles than the opposition is related to match
success. However, due to the insignificant difference between winners and losers
here, it is suggested that further research into this area will give a greater
understanding of the importance of this performance indicator.
There was also found to no clear difference in the number of initial breaks made
per game by winning (mean = 2.50) and losing (mean = 2.63) teams. This agrees
with Wheeler, Askew and Sayers (2010), who reported that the number of line breaks
by a team was not associated with success in Rugby Union. It does however,
disagree with Ortega, Villarejo and Palao (2009) and Diedrick and Van Rooyen
(2011) who both report winning teams to successfully break the defensive line more
frequently. It could be suggested that due to the mixed results found regarding this
performance indicator, that it perhaps is not vital to match success. It is suggested
that the frequency of initial breaks has no influence on the outcome of knockout
matches at the 2011 Rugby World Cup. Further research into this area is certainly
required to gain a clearer understanding as to the influence of initial breaks on match
outcome.
These findings have suggested that, despite previous research suggesting
otherwise, lineout success %, tackle completion % and initial breaks do not
discriminate between winning and losing teams at the 2011 Rugby World Cup
knockout stages. However, a high tackle completion % combined with a greater
number of attempted tackles than the opposition may contribute to discriminating
between winners and losers.
5.4. Limitations and Future Recommendations
5.4a. Sample Size
One particular limitation regarding the methodology of the study was the sample
size. As only the knockout matches were analysed, any firm conclusions can only be
made specifically to the 2011 Rugby World Cup. The issue with a small sample size
is that there is often not enough data to represent a population, meaning that results
that are presented as being significant are sometimes not the case. Effect size was
calculated in order to give an indication as to the meaningfulness of results, which
helps to overcome the issue of a small sample. Although this calculation suggested
7/31/2019 Luke Bishop Research Project
32/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 29
that some variables (turnovers conceded, % turnovers conceded between own 22m
and halfway, total kicks out of hand, total passes, pass completion, ruck frequency,
% penalties conceded between own 22m and halfway) would possibly be significant
had the sample size been larger, this calculation is not without its flaws. Cohen's d
calculation uses the means of the two groups to calculate effect size. This means
that a solitary unusual value may considerably affect the results (Thomas, Nelson
and Silverman 2011). This could give a false indication of the meaningfulness of the
results. Therefore, it is recommended that conclusions from this study should only be
applied to the 2011 World Cup and not to knockout rugby as a whole.
5.4b. Reliability
Overall, the reliability of the study was sound. However, what was unexpected was
that the inter-observer analysis showed more reliable results than the intra-observer
analysis (Table 4.1 and 4.2). A possible explanation for this was that the intra-
analysis was undertaken on only three matches in comparison to eight matches for
the inter-observer test, meaning that the slightest discrepancies were likely to be
magnified due to the very small sample size. It is recommended that reliability
analysis should be undertaken on the same size sample for both inter-observer and
intra-observer tests. Generally, the reliability of the study was strong, giving an
indication that the results presented possess good validity.
5.4c. Future Recommendations
As this was the first study to look at identifying what performance indicators
discriminate winners and losers in knockout rugby, specifically the 2011 World Cup, it
can act as a starting point for future research into this specific area. The review of
relevant literature, asserted by the findings from this study, have suggested that
performance indicators that relate to match success are varied across competition
formats. Therefore, future research should look to compare similar competition
formats to attempt to build profiles of what match indicators are key in each
respective format. It is suggested that to enhance the understanding of what
performance indicators are key in knockout rugby, follow-up studies should be
completed looking at recent knockout competitions, i.e. Heineken Cup, domestic
league play-off matches. This will then create a larger pool of information, meaning
better-founded conclusions can be made as to what is the best tactical approach in
knockout rugby. It is also recommended that a review of knockout rugby takes place
7/31/2019 Luke Bishop Research Project
33/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 30
annually in order to maintain awareness of the development of the game in this
format.
5.5. Conclusion
This study presents game statistics that help to identify what performance
indicators discriminate winning and losing teams during the knockout stages at the
2011 Rugby World Cup. These statistics can help to guide possible training
interventions for sides competing in future knockout tournaments. Key conclusions
are presented along with practical recommendations for each point.
Firstly, the results of the study showed that the Experimental Law Variations had
influenced the game in terms of the number of rucks that took place per game,
signalling that teams were more prepared to carry the ball at the 2011 World Cup in
comparison to previous years. However, despite this increase, it appeared that
carrying and rucking the ball was not the best approach to win matches. It was found
that kicking the ball, as a form of attack was more suitable, advocating a more
territory-based approach as opposed to a possession one. Therefore it is suggested
that kicking-based drills are included as an important part of training sessions prior to
knockout competitions.
In addition to this, a teams ability to concede fewer turnovers than the opposition
is related to match success. Therefore, offence sessions based around rucking and
ball retention in contact will help to reduce the incidence of conceding turnovers at
the breakdown. Defence sessions based around competing at rucks, slowing the ball
down and the jackal technique will also help to improve the chances of forcing
turnover ball from the opposition.
Conceding more penalties further away from the defensive try line also
discriminated between winning and losing teams at the 2011 Rugby World Cup. A
possible approach to help reduce the penalty frequency in the defensive half would
be to not commit many players to the breakdown when on defence. Fewer players
competing at the breakdown and more players set up in the defensive line will lead to
a reduction in the chance of conceding a penalty, whilst the defensive line will not be
outnumbered should the opposition move out wide in attack.
In summary, this study shows that there were few significant differences between
winning and losing teams at the 2011 Rugby World Cup. However, it was identified
that the most appropriate form of attack was kicking the ball more frequently,
whereas rucking, carrying and passing the ball more frequently was a less successful
approach. A good tactical kicking game, in combination with the ability to concede
7/31/2019 Luke Bishop Research Project
34/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 31
few turnovers and concede a higher proportion of penalties in the attacking half were
related to match success. A good defence was also found to have some importance,
with winning teams able to complete a similar percentage of tackles to losing teams,
whilst attempting a greater amount of tackles. It is important to note that the findings
of this study should be used carefully. They only represent the knockout stages at the
2011 World Cup, so to generalise them across different competition formats is ill-
advised. Further research should look at other knockout competitions in an attempt to
find the most effective tactical approach to knockout rugby.
7/31/2019 Luke Bishop Research Project
35/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 32
References
ATKINSON, G. and NEVILL, A. M. (1998). Statistical methods for assessing
measurement error (reliability) in variables relevant to sports medicine. Sports
Medicine (Auckland, N.Z.), 26 (4), 217-217.
AUSTIN, Damien, GABBETT, Tim and JENKINS, David (2011). The physical
demands of Super 14 Rugby Union. Journal of science and medicine in sport, 14 (3),
259-263.
BARTLETT, R. (2001). Performance analysis: Can bringing together biomechanics
and notational analysis benefit coaches? International Journal of Performance
Analysis in Sport, 1 (1), 122-122.
BLAND, J. Martin and ALTMAN, Douglas G. (2010). Statistical methods forassessing agreement between two methods of clinical measurement. International
Journal of Nursing Studies, 47 (8), 931-936.
BODDINGTON, Michele and LAMBERT, Mike (2004). Quantitative and qualitative
evaluation of scoring opportunities by South Africa in World Cup Rugby
2003. International Journal of Performance Analysis in Sport, 4 (2), 32-32.
BRACEWELL, P. (2002). Implementing statistics in a diagnostic coaching structure
for Rugby. Res. Lett. Inf. Math. Sci., 3 (1), 79-84.
COHEN, J. (1969). Statistical power analysis for the behavioral sciences. San Diego,
CA: Academic Press.
COOPER, Stephen-Mark et al. (2007). A simple statistical method for assessing the
reliability of data entered into sport performance analysis systems. International
Journal of Performance Analysis in Sport, 7(1), 87-87.
CSATALJAY, Gabor et al. (2009). Performance indicators that distinguish winning
and losing teams in basketball. International journal of performance analysis in
sport, 9 (1), 60-60.
CUNNIFFE, Brian et al. (2009). An evaluation of the physiological demands of elite
rugby union using global positioning system tracking software. Journal of Strength
and Conditioning Research / National Strength & Conditioning Association, 23 (4),
1195-1203.
DIEDRICK, Emile and VAN ROOYEN, Michele (2011). Line Break Situations in
International Rugby. International Journal of Performance Analysis in Sport, 11, (13)
522-534
7/31/2019 Luke Bishop Research Project
36/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 33
EAVES, Simon and HUGHES, Mike (2003). Patterns of play of international rugbyunion teams before and after the introduction of professional status. InternationalJournal of Performance Analysis in Sport, 3 (2), 103-103.
EAVES, Simon, HUGHES, Mike and LAMB, Kevin (2005). The consequences of theintroduction of professional playing status on game action variables in internationalnorthern hemisphere Rugby Union Football. International Journal of PerformanceAnalysis in Sport, 5 (2), 58-58.
FRANKS, I.M. and MILLER, G. (1991). Training coaches to observe andremember. Journal of Sports Sciences, 9 (3), 285-297.
GRHAIGNE, J. F., BOUTHIER, D. and DAVID, B. (1997). Dynamic-system analysisof opponent relationships in collective actions in soccer. Journal of Sports
Sciences, 15 (2), 137-149.
HUGHES, Mike (2004). Performance analysis - a 2004 perspective. InternationalJournal of Performance Analysis in Sport, 4 (1), 103-103.
HUGHES, Mike and BARTLETT, Roger (2002). The use of performance indicators inperformance analysis. Journal of Sports Sciences, 20 (10), 739-754.
HUGHES, M., COOPER, S-M. and NEVILL, A. (2002). Analysis procedures for non-parametric data from performance analysis. International Journal of PerformanceAnalysis in Sport, 2 (1), 6-6.
IRB (2003). RWC 2003: Statistical Review and Match Analysis. [online]. Lastaccessed 25th March 2012 at:http://www.irbplayerwelfare.com/pdfs/gameanalysis/RWC_Analysis_2003.pdf
IRB (2005). Changes in the Playing of International Rugby Over a 20 Year Period.[online]. Last accessed 25th March 2012 at:http://www.irb.com/mm/document/newsmedia/0/comparison1983to2003_541.pdf
IRB (2007). Statistical Review and Match Analysis: IRB game analysis. [online]. Lastaccessed 25th March 2012 at:http://www.irb.com/mm/document/home/0/071031ctrwc07statscomm_3811.pdf
IRB (2008). The IRB Guide to Experimental Law Variations. [online]. Last accessed19th March 2012 at:http://www.irb.com/mm/document/newsmedia/0/irbelvguideenfinal_5097.pdf
JAMES, Nic, MELLALIEU, Stephen D. and JONES, Nicholas M. P. (2005). Thedevelopment of position-specific performance indicators in professional rugbyunion. Journal of Sports Sciences, 23 (1), 63-72.
7/31/2019 Luke Bishop Research Project
37/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 34
JAMES, Nic, TAYLOR, Joe and STANLEY, Simon (2007). Reliability procedures for
categorical data in performance analysis. International Journal of Performance
Analysis in Sport, 7 (1), 1-1.
JENKINS, Rhian, MORGAN, Lynsey and O'DONOGHUE, Peter (2007). A case study
into the effectiveness of computerised match analysis and motivational videos within
the coaching of a league netball team. International Journal of Performance Analysis
in sport, 7 (2), 59-59.
JONES, Nicholas, MELLALIEU, Stephen and JAMES, Nic (2004). Team
performance indicators as a function of winning and losing in rugby
union. International Journal of Performance Analysis in Sport, 4 (1), 61-61.
LAGO-BALLESTEROS, J. and LAGO-PENAS, C. (2010). Performance in teamsports: Identifying the keys to success in soccer. Journal of Human Kinetics, 25 (3),
85-91
LAGO-PENAS, C., LAGO-BALLESTEROS, J. and REY, E. (2011). Differences in
performance indicators between winning and losing teams in the UEFA Champions
League. Journal of Human Kinetics, 27 (3), 137-148.
LAMES, Martin and MCGARRY, Tim (2007). On the search for reliable performance
indicators in game sports. International Journal of Performance Analysis in
Sport, 7 (1), 62-62.
LIEBERMANN, Dario et al. (2002). Advances in the application of information
technology to sport performance. Journal of Sports Sciences, 20 (10), 755-769.
MCGARRY, Tim (2009). Applied and theoretical perspectives of performance
analysis in sport: Scientific issues and challenges. International Journal of
Performance Analysis in Sport, 9 (1), 128-128.
O'DONOGHUE, Peter (2006). The use of feedback videos in sport. International
journal of Performance Analysis in Sport, 6 (2), 1-1.
O'DONOGHUE, Peter (2010). Research methods for sports performance
analysis. London, Routledge.
ORTEGA, E., VILLAREJO, D. and PALAO, J.M. (2009). Differences in game
statistics between winning and losing rugby teams in the Six Nations
tournament. Journal of Sports Science and Medicine, 8 (4), 523-527.
PRIM, Sebastian, VAN ROOYEN, Michele and LAMBERT, Michael (2006). A
comparison of performance indicators between the four South African teams and the
7/31/2019 Luke Bishop Research Project
38/40
Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011
Rugby World Cup. Luke Bishop
Page 35
winners of the 2005 super 12 rugby competition. What separates top from
bottom? International Journal of Performance Analysis in Sport, 6 (2), 126-126.
REILLY, Thomas and GILBOURNE, David (2003). Science and Football: A review of
applied research in the football codes. Journal of Sports Sciences, 21 (9), 693-705.
SASAKI, Koh et al. (2007). Scoring profiles and defence performance analysis in
Rugby Union. International Journal of Performance Analysis in Sport, 7 (3), 46-46.
STANHOPE, J. and HUGHES, M. (1997). An analysis of scoring in the 1991 rugby
union World Cup for men. In Hughes, M. (Ed.) (1997). Notational Analysis of Sport I
& II. University of Wales Institute Cardiff. Cardiff, Wales, pp. 167 176.
Recommended