Embed Size (px)
Citation preview
http://hfs.sagepub.com/Ergonomics Society
of the Human Factors and Human Factors: The Journal
http://hfs.sagepub.com/content/50/6/903The online version of this article can be found at:
DOI: 10.1518/001872008X375009
2008 50: 903Human Factors: The Journal of the Human Factors and Ergonomics SocietyGoodwin and Stanley M. Halpin
Eduardo Salas, Deborah DiazGranados, Cameron Klein, C. Shawn Burke, Kevin C. Stagl, Gerald F.Does Team Training Improve Team Performance? A Meta-Analysis
Published by:
http://www.sagepublications.com
On behalf of:
Human Factors and Ergonomics Society
can be found at:SocietyHuman Factors: The Journal of the Human Factors and ErgonomicsAdditional services and information for
http://hfs.sagepub.com/cgi/alertsEmail Alerts:
http://hfs.sagepub.com/subscriptionsSubscriptions:
http://www.sagepub.com/journalsReprints.navReprints:
http://www.sagepub.com/journalsPermissions.navPermissions:
http://hfs.sagepub.com/content/50/6/903.refs.htmlCitations:
What is This?
- Dec 1, 2008Version of Record >> at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
Attention to team training has grown exponentiallyduring the past 10–20 years. It is driven by at leastthe perception that the “high performance team”is a critical element in the design of organizationsthat will be effective in the global economy.
– Campbell & Kuncel (2001, p. 299)
INTRODUCTION
The nature of work has changed. Organizationsnow face increased competition and collaborationwithin and across organizational, geographic, andtemporal boundaries; a need to engage a demo-graphically heterogeneous workforce; a need todeal with advancements in information technol-ogy; a need to promote safety; and a need to fos-
ter enduring customer relations (Noe, 2002; Pfeffer& Sutton, 2000; Tannenbaum, 2002). One re-sponse to these changes has been the use of workteams as a preferred performance managementtechnique. “Teams are ubiquitous. Whether we aretalking about software development, Olympichockey, disease outbreak response, or urban war-fare, teams represent the critical unit that ‘getsthings done’ in today’s world” (Marks, 2006, p. i).
Both governmental agencies and private indus-try are increasingly relying upon work teams as apreferred performance arrangement to fulfill theirvisions, execute their complex missions, and ac-complish their goals (Salas, Stagl, & Burke, 2004).In fact, a recent random sample of U.S. organiza-tions indicated that nearly half (48%) used some
Does Team Training Improve Team Performance? A Meta-Analysis
Eduardo Salas, Deborah DiazGranados, Cameron Klein, C. Shawn Burke, and Kevin C.Stagl, University of Central Florida, Orlando, Florida, and Gerald F. Goodwin andStanley M. Halpin, Army Research Institute, Arlington, Virginia
Objective: This research effort leveraged the science of training to guide a taxonomicintegration and a series of meta-analyses to gauge the effectiveness and boundary con-ditions of team training interventions for enhancing team outcomes. Background:Disparate effect sizes across primary studies have made it difficult to determine the truestrength of the relationships between team training techniques and team outcomes.Method: Several meta-analytic integrations were conducted to examine the rela-tionships between team training interventions and team functioning. Specifically, weassessed the relative effectiveness of these interventions on team cognitive, affective,process, and performance outcomes. Training content, team membership stability, andteam size were investigated as potential moderators of the relationship between teamtraining and outcomes. In total, the database consisted of 93 effect sizes representing2,650 teams. Results: The results suggested that moderate, positive relationships existbetween team training interventions and each of the outcome types. The findings ofmoderator analyses indicated that training content, team membership stability, and teamsize moderate the effectiveness of these interventions. Conclusion: Our findings sug-gest that team training interventions are a viable approach organizations can take inorder to enhance team outcomes. They are useful for improving cognitive outcomes,affective outcomes, teamwork processes, and performance outcomes. Moreover, resultssuggest that training content, team membership stability, and team size moderate theeffectiveness of team training interventions. Application: Applications of the resultsfrom this research are numerous. Those who design and administer training can ben-efit from these findings in order to improve the effectiveness of their team traininginterventions.
Address correspondence to Eduardo Salas, Institute for Simulation & Training, University of Central Florida, 3100 TechnologyPkwy., Orlando, FL 32826; [email protected]. HUMAN FACTORS, Vol. 50, No. 6, December 2008, pp. 903–933. DOI 10.1518/001872008X375009. Copyright © 2008, Human Factors and Ergonomics Society.
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
904 December 2008 – Human Factors
type of team-based structure, with ongoing projectteams being the most frequently reported (Devine,Clayton, Philips, Dunford, & Melner, 1999).
The increasing frequency of team-based formsof organizing presents unique opportunities andcreates challenges. Opportunities via the use ofteams abound. For example, collectives can lever-age shared mental models, compensatory pro-cesses, and affective states such as cohesion to dealmore effectively with the complex, stressful, andsometimes chaotic contexts that typify modernoperations (Orasanu & Salas, 1993). The processgains yielded by teams can ultimately culminatein enhanced efficiencies, quality and safety im-provements, creativity, and/or adaptation (Banker,Field, Schroeder, & Sinha, 1996; Burke, Stagl,Salas, Pierce, & Kendall, 2006; Cohen & Led-ford, 1994; Foushee, 1984).
However, challenges inherent to the use ofteams exist. Most notably, how should organiza-tions compose, manage, and develop team mem-bers? More specifically, how does one design anddeliver team training? What kind of team train-ing should be implemented? And finally, does itwork? This research was motivated by the need toanswer these questions.
Team training targets latent teamwork knowl-edge, skills, and/or attitudinal competencies (KSAs)as well as manifest team processes and performancefor improvement (Salas & Cannon-Bowers, 1997,2000). Teams can be trained to make better deci-sions (Orasanu & Fischer,1997), to perform betterunder stress (Driskell, Salas, & Johnston, 1995),and to make fewer errors (Wiener, Kanki, & Helm-reich, 1993).
Enhancing teamwork through team training isa chief concern in many industries, including com-mercial and military aviation (Boehm-Davis, Holt,& Seamster, 2001; Helmreich & Foushee, 1993;Oser, Salas, Merket, & Bowers, 2001), health care(Baker, Salas, Barach, Battles, & King, 2007; Da-vies, 2001; Salas, DiazGranados, Weaver, & King,2008), and energy (Flin, 1995; Flin & O’Connor,2001), to name just a few. In these industries (andothers), a great deal of research has been generatedon team training. Unfortunately, no comprehensivesource exists in the team training literature thatdemonstrates whether or not team training evenmatters and, if it does, by how much. Further, nosource outlines the conditions under which teamtraining works. This research initiative seeks to fillthese voids.
The following sections of this paper are orga-nized as follows. We first describe the rationale forthe current meta-analytic initiative. Next, we ad-dress the conceptual foundation on which thisresearch is based. Included here is a short discus-sion centered upon the nature of teams and teamperformance, as well as a longer discourse on thecurrent state of team training efforts. Eight specificstudy hypotheses are then put forward, followedby a detailed accounting of the study methods. Anensuing description of meta-analytic results is sub-sequently presented, followed by implications forapplied work with teams – a discussion that fullyincorporates study limitations and directions forfuture research on team training.
WHY A TEAM TRAINING INTEGRATION?
This research effort includes a series of meta-analyses, each of which was conducted to gaugethe effectiveness of team training for enhancingteam outcomes. Specifically, we sought to (a) clar-ify and contribute to the science of training by pro-viding a clarification of the nature of team trainingvia a taxonomic integration of team training inter-ventions; (b) establish the boundary conditions ofteam training for enhancing team outcomes viamoderator analyses with additional variables ofinterest; and (c) impart valuable information fororganizational stakeholders charged with design-ing, delivering, and evaluating team training inter-ventions.
First, several limitations of the existing base ofresearch on team training and development haveserved to impede knowledge accumulation and in-tegration (Campbell & Kuncel, 2001). As Judge,Cable, Colbert, and Rynes (2007) recently noted,“Regardless of the quality of an idea, the abilityto draw inferences about a phenomenon is con-strained by the quality of the methods used togather data about it” (p. 493). In the team trainingarea, practical methodological limitations wit-nessed in primary studies that rely on relativelysmall samples of teams generally result in an unac-ceptably large sampling error – error that can ad-versely influence the consistency and quality ofindependent research studies’conclusions. At thesame time, disparate effect sizes across primarystudies have made it difficult to determine the truestrength of the relationships between team train-ing techniques and team outcomes.
The use of meta-analysis can help overcome at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 905
these issues. Specifically, meta-analytically inte-grating the results of independent studies allowsone to amalgamate, with increased precision andcertainty, the significance, strength, and predictablevariation of the effects of team training on teamfunctioning. Although recent attempts have beenmade to empirically (and meta-analytically) reviewthe influence of team development interventionson team functioning (e.g., Salas, Nichols, & Dris-kell, 2007; Salas, Rozell, Mullen, & Driskell,1999),this research advances (and expands upon) priorefforts by including more primary studies, employ-ing enhanced methodology, and examining keyvariables that will help clarify the existing knowl-edge base on team training.
We should note that most team training evalu-ations have been conducted in the military and inaviation. These sectors are the primary investorsof funding and effort to understand team perfor-mance and how to develop and deliver team train-ing. The current research includes team trainingevaluations from several sectors (i.e., military, avi-ation, industry, lab/classroom, and medical set-tings). As will be discussed at a later point, theprimary studies included in this meta-analysis areheavily represented by military research.
This meta-analysis also provides clarity andunderstanding concerning the amorphous empir-ical literature on team training. In practice, teamtraining appears in many shapes, sizes, and formsand is called by many different names and labels(e.g., assertiveness training, crew resource man-agement, cross-training, group process training,problem-solving training, task-focused simulationtraining). What has long been needed in this do-main is a taxonomic integration and empirical in-vestigation of the utility of these interventionsbased upon the content or focus of the interven-tions, rather than upon the myriad of names givento these strategies by independent researchers.
Fortunately, an inclusive taxonomy has alreadyemerged. Upon summarizing work conducted bySalas and colleagues (e.g., Cannon-Bowers, Tan-nenbaum, Salas, & Volpe,1995; McIntyre & Salas,1995; Salas, Dickinson, Converse, & Tannen-baum, 1992), Goldstein and Ford (2002) arguedthat [team] training methods are designed to en-hance taskwork, teamwork, and process improve-ment skills. Meta-analytic reviews have beenconducted of team interventions that focused onprocess improvement skills (i.e., team building;Klein et al., in press; Salas et al., 1999), so there is
now a need to empirically analyze the team train-ing literature along the teamwork-taskwork con-tinuum. The current research investigates theunique and relative value of these two distinctfocal points of team training.
A second impetus for this research is to inves-tigate moderators of team training effectiveness.These investigations will help provide answers toa number of important questions, including thefollowing:
• Does the effectiveness of team training vary basedupon the content of the training?
• Does team training work better with intact versus ad hoc teams?
• Do larger teams benefit more from team training thansmaller teams?
Answers to these questions will go a long way to-ward advancing the literature and providing deeperinsights into the effectiveness of team training.
Finally, although there is evidence to suggestthat training (and team training) has a significant,positive impact on key measures of business performance (e.g., George, Hannibal, & Hirsch,2004; Mathieu & Leonard, 1987) and safety(Salas, Burke, Bowers, & Wilson, 2001; Salas,Wilson, Burke, & Wightman, 2006), there is aneed to form a more accurate understanding of theimpact of team training, specifically. Accurate esti-mates of the efficacy of team training are impor-tant for both researchers and practitioners becausetens of millions of dollars are spent annually onteam training in military, aviation, health care, andother industry settings.
In short, the present research provides a sub-stantial contribution to the current understandingof the efficacy of team training. In comparisonwith other quantitative reviews (e.g., Salas, Nich-ols, et al., 2007) and qualitative reviews (e.g., Salaset al., 2001, 2008), the current effort uses more rig-orous methods to assess a greater number of pub-lished and unpublished primary studies; distillsknowledge from a wider range of interventions,outcomes, and moderator variables; and offersactionable guidance to organizational decisionmakers. Before describing in detail the researchquestions and hypotheses posed in the present ini-tiative, a brief discussion of teams, team perfor-mance, and team training is provided.
TEAMS, TEAM PERFORMANCE, ANDTEAM TRAINING
“Teams embedded in organizations exist to at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
906 December 2008 – Human Factors
perform tasks” (Ilgen, 1999, p. 131). These tasksare often complex, difficult, and dynamic. They re-quire expertise, experience, and the perspectivesof multiple individuals synchronizing their workto perform collectively in the pursuit of commongoals (Mohammed & Ringseis, 2001).
For the purposes of this meta-analysis, teamsare defined as a distinguishable set of two or moreindividuals who interact dynamically, adaptively,and interdependently; who share common goals orpurposes; and who have specific roles or functionsto perform (Salas et al., 1992). Teams can be dis-tinguished from work groups by their task, feed-back, and goal interdependencies (e.g., Saavedra,Earley, & Van Dyne, 1993) as well as by their formal structure and coordination demands (Tan-nenbaum, Beard, & Salas, 1992). That is, groupsare usually more loosely constituted. They are col-lectives that may be perceived as social entities,and they may even have common goals, but theyalso possess task connections that are less well de-fined (e.g., juries, committees, councils).
For the current research integration, team per-formance is defined as an emergent phenomenonresulting from the goal-directed process wherebymembers draw from their individual and sharedresources to display taskwork processes, team-work processes, and integrated team-level pro-cesses to generate products and provide services(Kozlowski & Klein, 2000; Salas, Stagl, Burke, &Goodwin, 2007).
Frameworks of team performance and effec-tiveness generally follow an input, throughput, andoutput format (e.g., Hackman,1983,1987; McGrath,1984; Nieva, Fleishman, & Rieck,1978; Salas, Stagl,et al., 2007; Steiner,1972; Tannenbaum et al., 1992)in which inputs might include individual and teamcharacteristics, capabilities, and states; through-puts include team communication, coordination,collaboration, and decision-making processes; andoutputs consist of the goods or services producedby a team. Stakeholders make judgments about theeffectiveness of the quality, quantity, and timelinessof the products or services produced, as well asabout the changes in the team and its members thatresult as performance unfolds (Hackman, 2002).
Training is a systematic, planned interventiondesigned to facilitate the acquisition of job-relatedKSAs (Goldstein & Ford, 2002). Concerning teamtraining, one widely cited definition positions it as“a set of tools and methods that, in combinationwith required [team-based] competencies and
training objectives, form an instructional strategy”(Salas & Cannon-Bowers, 1997, p. 254). Task-focused team training enables team members tobecome aware of, learn about, and practice requi-site team competencies (i.e., KSAs) and perfor-mance processes while receiving feedback on theirperformance. Moreover, similar to that of individ-ual training, the science of team training involvesidentifying the optimal combination of tools (e.g.,team task analysis), delivery methods (e.g., practicebased, information based, demonstration based),and content (e.g., knowledge, skills, attitudes; Salas& Cannon-Bowers, 1997).
Many narrative and empirical evaluations ofteam training have been conducted (e.g., Denson,1981; Dyer,1984; Marks, Zaccaro, & Mathieu, 2000;Mathieu, Heffner, Goodwin, Salas, & Cannon-Bowers, 2000; Salas et al., 2001, 2006). Althoughthe majority of these assessments have yieldedample support for team training, differences havebeen observed in the efficacy with which assortedteam training strategies improve team outcomes.In an effort to summarize the relative contributionsof various team training strategies, Salas, Nichols,et al. (2007) recently conducted a meta-analysison the subject. Specifically, they investigated therelative efficacy of three team training strategies:cross-training, coordination and adaptation train-ing, and team self-correction training.
Their database consisted of 28 effect sizes (fromseven studies) assessing the efficacy of team train-ing for team performance. For the overall test, therewas a low, significant tendency for team trainingto improve team performance (r = .29). However,when they examined the relative efficacy of dif-ferent team training strategies, cross-training didnot appear to be effective (r = –.09), whereas theefficacy of self-correction training (r = .45) andteam coordination and adaptation training (r = .61)was confirmed.
Although the findings from the Salas, Nichols,et al. (2007) team training meta-analysis suggestthat coordination and adaptation training is ef-fective for producing team performance im-provements, some caution should be exercised ininterpreting these results and directly comparingthem with those of other meta-analyses. The meta-analytic technique that Salas, Nichols, et al. (2007)utilized was based on procedures outlined byMullen (1989), Mullen and Rosenthal (1985), andRosenthal (1991). That approach to meta-analyticintegration is considerably different (e.g., use of
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 907
transformations, not calculating confidence inter-vals) from the procedure we utilized in the presentstudy, which followed the procedure provided inHunter and Schmidt (2004).
In sum, the findings from qualitative and lim-ited quantitative research on team training inter-ventions have largely supported the effectivenessof these techniques. However, none is as compre-hensive, robust, or systematic as this one. This is(to our knowledge) the first thorough research en-deavor undertaken to gauge the impact of teamtraining on team outcomes. Given the widespreaduse of team training in many important settings,it is also a timely and critical integration.
HYPOTHESES: THEIR RATIONALE ANDCONCEPTUAL FOUNDATION
As noted, several research questions concern-ing the importance of team training for enhancingtargeted team outcomes guided this effort. In aneffort to address these questions, we present test-able hypotheses and their theoretical rationale inthe following sections. A model that graphicallyillustrates these research questions is offered inFigure 1.
Does Team Training Work?
The first question posed in this meta-analysis iswhether team training works. This general ques-tion treats all forms of team training as equivalentand, likewise, treats all targeted outcomes of teamtraining interventions as interchangeable. Al-though simplistic from a scientific standpoint, thisoverarching issue is the single most importantquestion posed in the current study. It asks, in ageneral way, whether team training is useful as a lever to produce change. If the answer to thisquestion is no, then an enormous amount of sci-entific effort and limited organizational resourceshave surely been wasted. Moreover, such null re-sults would more broadly call into question thewidely accepted premise that planned interven-tions are useful in some measurable way.
Fortunately, research on instructional designtheory and, more recently cognitive, metacognitive,and macrocognitive theory, collectively suggeststhat team training is indeed useful for promptingmeaningful change and in helping teams achieveperformance levels deemed effective by organiza-tional stakeholders. Supporting this growing baseof theory are the mounting findings from primary
Figure 1. Model depicting study hypotheses (H1–H8).
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
908 December 2008 – Human Factors
team training studies (e.g., Ellis, Bell, Ployhart,Hollenbeck, & Ilgen, 2005; E. E. Entin, Serfaty, &Deckert, 1994; Klein, Stagl, Salas, Parker, & VanEynde, 2007). Given the abundance of theoreticalsupport and empirical evidence speaking directlyto the usefulness of team training, the followinghypothesis is advanced:
Hypothesis 1: Team training is positively re-lated to overall team outcomes.
Separating Outcomes
In any meta-analysis, a balance must be struckbetween casting too wide a net (i.e., includingwidely disparate independent and dependent vari-ables) and being too restrictive (i.e., reducing themeta-analytic database to a small number of stud-ies). Clearly, results likely to be obtained from theexamination of the first research question can becriticized for thoughtlessly combining “apples andoranges” and concluding that “fruit tastes good”(Kraiger, 1985). Thus, there was a need for morefinely tuned analyses that examined the relation-ships between more specific and restrictive defi-nitions of independent and dependent variables.This set of analyses is directed at the next questionposed in this study, namely, “Is team training, whenconceptualized as a single overarching interven-tion, an effective technique for producing plannedchanges in specific types of team-level outcomes?”
This issue is important, as team researchers haveoffered compelling propositions about (Marks,Mathieu, & Zaccaro, 2001), and meta-analytic evi-dence for (LePine, Piccolo, Jackson, Mathieu, &Saul, 2008), behavioral and cognitive processesconstituting teamwork. Team performance emergesas teams and their members draw upon individualand shared characteristics, capabilities, and cogni-tive and affective states to enact the core processesthat constitute teamwork (Burke, Stagl, Salas, et al.,2006). In turn, the performance outcomes result-ing from this unfolding, recursive process are eval-uated by organizational stakeholders in terms oftheir effectiveness (Hackman, 2002). If the utilityof team training is to be fully realized, it must be alever to affect all of the core components of team-work and produce performance outcomes that aredeemed effective in the workplace.
Gagné (1962) astutely argued that the most fun-damental design issue in training is the specifica-tion of what is to be learned. This is the specificationof training or instructional objectives. These objec-
tives specify the knowledge, skills, beliefs, atti-tudes, or choice behaviors that a trainee or traineesshould be capable of accomplishing upon success-ful completion of the training program (Campbell& Kuncel, 2001; Goldstein & Ford, 2002). Teamtraining is no different from individual trainingin this regard. Once these objectives are chosen,the training environment is tailored to achievethem in the most efficient way possible. Thus, in-structional objectives provide input for the designof training programs as well as the criteria usedto judge the programs’ adequacy (Goldstein &Ford, 2002).
Because the primary studies included in thepresent meta-analytic initiative addressed a widevariety of instructional objectives, a meaningfultaxonomy for organizing training outcomes wasneeded to guide this effort. In line with the teamtheory noted previously and with prior team train-ing meta-analyses (e.g., Salas, Nichols, et al.,2007) – and consistent with Kraiger, Ford, andSalas’s (1993) taxonomy – training outcomes canbe classified as cognitive, affective, and perfor-mance outcomes as well as behavioral processes.Each of the training outcomes investigated in theprimary studies included in the present meta-analysis were classified into one of these four taxons.
In the present study, team member cognitiveoutcomes predominantly consisted of declarativeknowledge gains from pre- to posttest measure-ment. Team member affective outcomes includedsocialization, trust, confidence in team members’ability, and attitudes concerning the perceived ef-fectiveness of team communication and coordina-tion processes. Team processes examined in thisreview included behavioral measures of commu-nication, coordination, strategy development, self-correction, assertiveness, decision making, andsituation assessment. Finally, the assessment ofteam performance integrated quantity, quality, ac-curacy, efficiency, and effectiveness outcomes. Tohelp answer the question posed at the beginningof this section, the following hypotheses are ad-vanced:
Hypotheses 2a to 2d: Team training is positivelyrelated to team-level (a) cognitive outcomes, (b)affective outcomes, (c) behavioral processes, and(d) performance outcomes.
Training Content
It is interesting to know whether team training at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 909
is useful for improving specific team outcomes,but it is perhaps more important to ask, “What typeof team training is most effective for improvingteam functioning?” To this end, the current re-search addresses the relative influence of specifictraining interventions on valued team outcomes.Although particular team training interventionsare often referred to by many different names, itcan be argued that every team training interven-tion targets some combination of taskwork orteamwork skills (e.g., Cannon-Bowers et al., 1995;Goldstein & Ford, 2002; McIntyre & Salas, 1995;Salas et al., 1992). Thus, three content categorieswere coded in the present study: taskwork, team-work, and interventions that utilized a mix oftaskwork and teamwork content. Illuminating therelationships between these three types of teamtraining and each of the four types of outcomesnoted previously is important to the objective ofthe present study.
Team training interventions targeting taskworkKSAs seek to develop the technical competen-cies of team members (Salas et al., 1992). Cross-training is a common example of a team trainingintervention that includes a heavy taskwork com-ponent. In cross-training, members learn aboutand sometimes practice the responsibilities of mul-tiple positions and/or roles, including those oftheir teammates. Having acquired interpositionalknowledge and honed teamwork KSAs, indi-viduals who have been cross-trained are betterprepared to recognize when their teammates areoverburdened and to step in and perform their co-workers’ duties when called upon.
In contrast, training interventions targetingteamwork KSAs are focused on improving howindividuals work together effectively as a team.For example, teamwork skills targeted by teamtraining may include mutual performance moni-toring, feedback, leadership, management, coor-dination, communication, and decision making(e.g., Cannon-Bowers et al., 1995; Stagl, Salas, &Fiore, 2007). Team coordination and adaptationtraining is one example of a team training inter-vention that is focused on providing teamworkskills content.
Hypotheses 3 to 5 expand upon the basic ques-tions addressed in the earlier hypotheses because,assessed in their entirety, they provide preliminaryestimates for determining which configurations ofteam training content are better than others. How-ever, given the absence of a priori expectations for
this set of hypotheses, the investigation into themoderating impact of training content is explora-tory. Nonetheless, the following hypotheses areadvanced to investigate these questions:
Hypotheses 3a to 3d: Taskwork training is pos-itively related to enhancements in team-level (a) cognitive outcomes, (b) affective outcomes, (c) behavioral processes, and (d) performanceoutcomes.
Hypotheses 4a to 4d: Teamwork training is pos-itively related to enhancements in team-level (a) cognitive outcomes, (b) affective outcomes,(c) behavioral processes, and (d) performanceoutcomes.
Hypotheses 5a to 5d: Team training that includesa combination of teamwork and taskwork con-tent is positively related to enhancements inteam-level (a) cognitive outcomes, (b) affectiveoutcomes, (c) behavioral processes, and (d) per-formance outcomes.
One truism of collectives is that teams of ex-perts often fail to evolve into expert teams (Salas,Cannon-Bowers, & Johnston, 1997). In otherwords, the possession of copious amounts of taskknowledge is insufficient to overcome the perfor-mance decrements that can result from the coor-dination demands teamwork imposes on teammembers.
In order to assist teams in fulfilling their poten-tial, it is often necessary to move beyond the pro-vision of mere task-specific content to featureteamwork and team interaction training as well(Hollenbeck, DeRue, & Guzzo, 2004). For exam-ple, a team of medical professionals that possessesa high degree of skill in mutual performance mon-itoring will not only have the task expertise to of-fer suggestions but will also know when to do soin a timely manner. Therefore, team training thattargets both task-specific content and genericteamwork competencies is expected to provide thegreatest benefit in terms of enhanced team out-comes. Given this line of thinking, the followinghypothesis is advanced:
Hypothesis 6: Team training that includes a com-bination of teamwork and taskwork content willbe more effective than interventions that targeteither phenomenon in isolation.
Team Membership Stability
The aforementioned research questions willhelp illuminate the importance of team training as
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
910 December 2008 – Human Factors
a lever for facilitating change in a variety of team-level phenomena. However, it is important to movebeyond these preliminary questions about themain effects of team training to begin to addressother pertinent questions, such as, “Under whatconditions is team training most effective as aplanned intervention?” Thus, in the remainder ofthis section, questions are posed and hypothesesare advanced to address additional variables thatmay moderate the effect of team training on team-level outcomes.
Of primary importance to the present meta-analysis are the potential moderating roles of teammembership stability and team size. Membershipstability addresses the length of time team mem-bers have worked together interdependently tosolve joint challenges, and team size indexes thenumber of team members assigned to a given col-lective.
In regard to membership stability, studies ofteam training interventions generally consist of twovarieties: (a) studies of training interventions con-ducted with intact teams, whose members have ashared history as a result of a commonly held as-signment to a given collective operating inside anorganization (e.g., an energy exploration team);and (b) studies of interventions delivered to a groupof ad hoc strangers purposively assembled to con-duct either basic or applied research in a contrivedsetting (e.g., a tank crew composed of randomlyselected and assigned students performing in auniversity lab). In comparison with their ad hoccounterparts, intact teams tend to have relativelystable membership and a shared history of work-ing together (Devine et al., 1999). The total sam-ple of primary studies included in the presentmeta-analysis is representative of both intact andad hoc teams.
In their recent meta-analysis on team training,Salas, Nichols, et al. (2007) made a consciousdecision to examine the effects only of interven-tions that were delivered to intact teams. How-ever, these researchers also suggested that futuremeta-analytic efforts should examine levels ofteam membership stability as a potential modera-tor variable. We accept that call and will investi-gate whether team training works better for intactor ad hoc teams.
Based upon prior research addressing the stagesof team development, it seems plausible that teamtraining will be more effective for intact teams.This is because intact collectives presumably have
already navigated at least some of the process chal-lenges that characterize the preliminary, and oftenrocky, stages of group development (e.g., storm-ing, norming [Tuckman, 1965]) that newly formedteams must overcome to develop cohesion. Thebaseline level of shared expectations, competen-cies, and emergent states developed from a sharedhistory will likely free up cognitive and interper-sonal resources, allowing intact teams to focusmore of their resources on skill acquisition duringteam training. Given these arguments, the follow-ing hypothesis is advanced:
Hypothesis 7: Team training will be more effectivefor intact teams than for ad hoc teams.
Team Size
This research also examines the relative impactof team training for teams of varying sizes. It hasbeen noted by several researchers that teams arehighly effective when they have a sufficient, butnot greater than sufficient, number of members toperform team tasks (Guzzo & Shea, 1992; Hack-man, 1990). The definition of sufficient, however,is somewhat unclear, in part because the researchfindings on the impact of team size on effective-ness have been mixed.
Some studies have found that larger teams aremore effective (Magjuka & Baldwin, 1991; Yetton& Bottger, 1982), whereas other studies have re-ported that larger teams suffer from coordinationand process losses (Gooding & Wagner, 1985;Mullen, Symons, Hu, & Salas, 1989). Kozlowskiand Bell (2003) summarized these mixed results,concluding that the benefits of larger teams aredependent on the nature of the task and the team’senvironment. For example, one study that evalu-ated team size and its impact on the performanceof tasks that required innovativeness found a negative correlation between team size and teamprocesses, r = –.18 to r = –.51 (Curral, Forrester,Dawson, & West, 2001).
Also bearing on this issue is the seminal workof Steiner (1972), who examined how increasingthe size of the team impacted team processes.Steiner’s (1972) findings suggested that an increasein team size resulted in increases in productivitybut also introduced inefficiencies (e.g., decreasedmotivation, poorer decision making, poorer coor-dination, and higher levels of conformity). Otherstudies have also found additional detrimentaleffects attributable to large-sized teams – for
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 911
example, an in-group bias effect (Mullen, Brown,& Smith, 1992), a participation leadership effect(Mullen, Salas, & Driskell, 1989), a decrease ingroup liking (Indik, 1965), and a decrease in per-formance (Mullen, 1987).
A common undesirable group phenomenon issocial loafing. It has been found that larger groupsare more prone to social loafing (Latané, 1981).Because larger groups lose the connection betweentheir inputs and the rewards that are received, in-dividuals in large teams become less motivated toperform (Kidwell & Bennett, 1993). Karau andWilliams (1993) discussed what they termed the“dilution” effect. This effect manifests itself in avariety of dysfunctional ways (e.g., free riding, get-ting lost in a crowd, shirking group work), all ofwhich are associated with an increase in team size.
Astudy by Hackman and Vidmar (1970) set outto find the perfect size of a team. Based on ques-tions asked of teams large and small, they foundthe optimal number of team members to be 4.6.Moreover, in a recent study by De Dreu (2007), aweak correlation was found between team size andthe measured outcomes of learning (r = .15) andteam effectiveness (r = .16). De Dreu also found anegative correlation between team size and infor-mation sharing (r = –.11). These results show thatthe key to enhancing team functioning does notnecessarily lie in increasing the size of the team.
The current research categorized teams intothree groups; small teams (n = 2), medium teams(2 < n < 5), and large teams (n ≥ 5). Arecent meta-analysis on team building interventions found thatlarge teams benefited the most from team buildinginterventions, resulting in a mean true score corre-lation of .66 (Klein et al., in press). Given this find-ing and past findings suggesting that medium-sizedteams will already be performing at a high level,we suggest that team training interventions willshow greater effects for dyads and large teams. Inother words, team training, whether it is geared to-ward teamwork or taskwork, will aid small andlarge teams more than medium-sized teams in im-proving their inefficiencies. Given this thinking,the following hypothesis is proposed:
Hypothesis 8: Team training will be more benefi-cial for small and large teams than for medium-sized teams.
METHOD
This study applied meta-analytical techniques
to previously conducted research in an effort toobtain a numerical estimate of the relationshipsbetween team training and team outcomes. Analy-ses were also conducted in order to determine theexistence of moderators. The following sectionsdescribe in detail the identification, selection, andcoding procedures for primary studies. In addition,a description of the method for calculating effectsizes is provided.
Literature Search
More than a half century ago, B. Glass (1955)observed that “no problem facing the individualscientist is more defeating than the effort to copewith the flood of published scientific research, evenwithin one’s own narrow specialty” (p. 583). Andeven within the relatively narrow specialty of teamtraining, there is indeed a rising tide of empiricalresearch. Surfing this tide required a calculatedline of attack. In total, four distinct and diverse ap-proaches were used to identify studies for poten-tial inclusion in the meta-analytic database.
First, comprehensive electronic searches ofcomputerized databases were conducted usingmultiple combinations of appropriate keywords(e.g., teams, cross-training, team performance;a full list of keywords used can be obtained fromthe authors). Specifically, the search engines,electronic databases, abstracting services, andproceedings of Google Scholar, ScienceDirect,EBSCOhost, Academic Search Premier, BusinessSource Premier, PsycINFO, PsycARTICLES, Mil-itary and Government Collection, SPORTDiscus,the Interservice/Industry Training, Simulation andEducation Conference, and the Defense TechnicalInformation Center were searched for articles pub-lished through August 2008.
Second, a targeted electronic search of the fol-lowing journals was conducted, using the key-words team training and group training, with norestriction on dates: Journal of Applied Psychol-ogy, Personnel Psychology, Military Psychology,Organization Development Journal, Academy ofManagement Journal, Small Group Research, andHuman Factors. Third, an ancestry approach wasemployed, whereby the bibliographies and refer-ence sections of relevant studies that had alreadybeen retrieved were searched to locate earlier rel-evant studies. Finally, an informal network of col-leagues was queried for potential unpublishedarticles, manuscripts, and conference papers. Thecomplete study identification and search process
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
912 December 2008 – Human Factors
resulted in more than 500 articles for possible in-clusion.
In the next phase of the literature review, twoauthors reviewed abstracts and online versions ofthe identified articles and came to a consensus con-cerning whether the full text document should beobtained. At this stage, articles containing the fol-lowing characteristics were excluded from furtheranalyses: (a) master’s theses; (b) use of clinicalpopulations; (c) use of children as participants;(d) collectives and groups characterized by a defi-cient degree of task interdependence; (e) studiesfailing to report a usable test statistic (e.g., F, t, χ2,z, d, r) or the raw data necessary to calculate thesestatistics (e.g., means, standard deviations, samplesize); and (f) studies failing to utilize an actualteam intervention.
As a result of this effort, 168 relevant publishedand unpublished papers were identified. Thesearticles, book chapters, dissertations, technicalreports, and conference proceedings were subse-quently obtained in full text and coded by studyauthors. A discussion of the specific coding strat-egy employed for these articles is presented next.
Coding Procedure
Various meta-analytic integrations often use a similar coding scheme to quantify study char-acteristics and results (Lipsey & Wilson, 2001;Stock, 1994). The coding strategy used in thepresent research included capturing 20 pieces ofinformation from each study: (a) publication type,(b) study setting, (c) study design type, (d) natureof the organization and participant sample, (e)team type, (f) team membership stability, (g) taskinterdependence, (h) number of teams, (i) averageteam size, (j) predictor level of analysis, (k) crite-rion level of analysis, (l) training content, (m) nameand description of training intervention, (n) methodof training intervention, (o) criterion report type,(p) criterion description, (q) reliability of predic-tor measures, (r) reliability of criterion measures,(s) effect size or sizes, and (t) recommendation forinclusion.
For the criterion level of analysis coding, theinitial database consisted of 19 effect sizes at theindividual level and 93 at the team level. Althoughit is desirable to include in the subgroup analysesthe largest possible number of effect sizes, re-searchers have strongly recommended againstmixing levels of analysis in research integrations(e.g., Gully, Devine, & Whitney, 1995; Hunter &
Schmidt, 1990). For the purposes of this research,only team-level outcomes were considered eligi-ble for analyses.
Rater Reliability
One or more study authors independentlycoded each of the 168 articles originally identifiedfor possible inclusion in the meta-analysis. Thecoders for this research had previously pilot testedthe coding scheme and possessed an adequatelevel of expertise in the substantive areas beingcoded. Moreover, before coding began, the au-thors met to discuss and develop detailed instruc-tions for coding the 20 categories of informationfrom primary studies. More specifically, a resourcemanual was developed in order to provide thecoders with standardized information to code theprimary studies. Coders met in order to discussthe coding manual and ensure a shared under-standing of the concepts being coded. Furthermore,the coders were trained by coding two articles andthen holding a meeting to discuss the informationextracted from the articles. Once a satisfactorylevel of understanding of the categories wasreached by all coders, the coding for the presentmeta-analysis began.
The coding was done in stages, with two checksperformed to assess interrater reliability. First, twoauthors and one of their colleagues (who was blindto the study hypotheses) each independently coded15 articles from the database. At this point, the in-terrater reliability for the three coders was assessedby calculating intraclass correlation coefficients(ICCs; Nunnally, 1978; Shrout & Fleiss, 1979).The ICC is a measure of reliability among two ormore raters. An ICC can be conceptualized as theratio of between-group variance to total variance.When an ICC approaches 1 it can be interpretedthat the variance between the raters is essentiallyzero; thus the raters give the same ratings. Further-more, the variation in the ratings or measurementsis attributable solely to the target being rated.
The reliability at this stage was deemed satis-factory (ICCs ranged from .75 to .85) and the fullcoding procedure continued. The second check oninterrater reliability was performed near the end ofthe coding process, when 10 more articles werechosen to be coded by each of the three coders.The reliability of the coding for these final 10 arti-cles was assessed at the end of the coding processand again was acceptable (ICCs ranged from .85to 1.0). Taken together, 25 of the 168 articles
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 913
were coded by two authors and a colleague. These25 articles represented approximately 15% of thetotal number of articles.
After the 25 reliability sample articles werecoded, the combined codings were assessed forrater reliability. We were particularly interested inthe coding for the data that would be contributingto the investigation of the hypotheses. We werealso keenly concerned with whether the coders re-liably coded the same effect sizes and whether theymade the same decisions in regard to including thearticle in the final database. Upon examination, thecoding of the data used for the hypotheses was sat-isfactory.
Specifically, for the criterion description cate-gory, the ICC (3, k) = .89. Similarly, for trainingcontent, team membership stability, and team size,the ICCs (3, k) were equal to .77, .94, and 1.00, re-spectively. Moreover, the coding for the effect sizeor sizes and inclusion/exclusion decisions werehighly reliable (ICCs = .99 and .89, respectively).These estimates of the reliability with which 25common articles were coded gave us confidencethat the coding for this research was performedreliably. The remaining 143 articles were codedindependently by one of the two authors who alsoserved as coders for the reliability sample. Anyinitial ambiguity or uncertainty about whether toinclude a particular study result or characteristicin the final meta-analytic database was resolvedthrough a consensus discussion by two or morestudy authors.
Meta-Analysis Procedure
The data analysis for this research was aided bythe Hunter-Schmidt Meta-Analysis Programs 1.1(Schmidt & Le, 2005). Effect sizes were first sortedwithin their associated subgroups (i.e., outcometype, training content, team membership stability,team size), and then a combined effect size esti-mate was generated for each subgroup and/or levelof each moderator (e.g., small, medium, and largeteams).
This software utilizes a random effects model,rather than a fixed effects model, to analyze thedata. This model allows the true effect sizes tovary, instead of assuming that the true effect sizeshave fixed values. Field (2001) noted that the ran-dom effects model is considered to be “more real-istic than the fixed-effect model on the majority ofoccasions” (p. 162). The data output for the pro-gram includes (but is not limited to) the mean true
score correlation, the standard deviation and vari-ance of true score correlations, credibility inter-vals, and the percentage of variance attributableto observed correlations after the removal of arti-facts. Confidence interval estimates were calcu-lated separately in Excel using formulas providedby Hunter and Schmidt (2004).
Effect Size Calculations
In order to aggregate findings across studies, alleffect size estimates culled from primary studieswere converted to a common metric, r (correla-tion). Formulas found in Hunter and Schmidt(2004) were used to transform primary study ef-fect sizes that had been reported as other statistics(e.g., t, F, d, χ2, or Z). Upon being placed on thiscommon metric of effect size (i.e., r), the primarystudy results can be pooled and evaluated for fitwith the predicted hypotheses in this research.
An important issue that presented itself was thatseveral studies contained more than one effect sizeestimate. This circumstance occurred for two rea-sons. First, several studies reported findings forseparate samples under investigation. In this in-stance, it is acceptable to include the findings asseparate effect sizes in the database. Alternatively,a large number of studies reported multiple effectsizes from single samples that were either (a) forthe same outcomes or (b) for different outcomes.For example, a study might have reported twoprocess outcomes and one performance outcome.
To be clear, however, stochastically dependenteffect sizes were not included in the current data-base (which, if included, would patently violate theassumption of independent effect sizes). In otherwords, findings generated on the same (or similar)outcomes from the same sample were averagedprior to entry in the database. In contrast, whenstudies reported findings from the same sample ondifferent outcomes, these findings were includedseparately in the overall database but were notincluded in any of the same subgroup analyses.Effect size estimates from primary studies thatwere combined prior to entry in the meta-analyticdatabase are denoted with an asterisk in Table 1.
Corrections for Unreliability and Range Restriction
Meta-analysis is especially useful for determin-ing whether conflicting results in the literature areattributable to artifactual or actual variation (G. V.Glass, 1976; G. V. Glass, McGaw, & Smith, 1981;
text continues on page 917 at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
914
TAB
LE 1
:Stu
die
s an
d E
ffec
t Si
zes
Incl
uded
in t
he M
eta-
Ana
lytic
Dat
abas
e
No
. of
Ave
rag
eC
rite
rio
nM
emb
ersh
ipTr
aini
ngA
utho
r(s)
Year
raTe
ams
Team
Siz
eD
escr
ipti
on
Stab
ility
Co
nten
t
Ad
elm
an, C
hris
tian
, Gua
ltie
ri, &
Bre
snic
k19
98.2
8*26
2.00
Pro
cess
Ad
ho
cM
ix19
98–.
17*
282.
00P
erfo
rman
ceA
d h
oc
Mix
Ale
xand
er, K
epne
r, &
Tre
go
e19
62.8
1*4
13.0
0P
erfo
rman
ceA
d h
oc
Mix
Ban
ks, H
ard
y, S
cott
, Kre
ss, &
Wo
rd19
77.6
4*16
9.00
Per
form
ance
Inta
ctTa
skw
ork
Blic
kens
der
fer,
Can
non-
Bo
wer
s, &
Sal
as19
97.4
5*40
3.00
Pro
cess
Ad
ho
cTe
amw
ork
1997
.03
393.
00P
erfo
rman
ceA
d h
oc
Team
wo
rkB
rann
ick,
Pri
nce,
& S
alas
2005
.59*
482.
00P
roce
ssA
d h
oc
Mix
2005
.12*
482.
00P
erfo
rman
ceA
d h
oc
Mix
Bro
wn
2003
.40
414.
50A
ffec
tive
Ad
ho
cTe
amw
ork
2003
.29
414.
50P
erfo
rman
ceA
d h
oc
Team
wo
rkB
uzag
lo &
Whe
elan
1999
.63
213
.00
Aff
ecti
veIn
tact
Team
wo
rk19
99.6
92
11.0
0P
roce
ssIn
tact
Team
wo
rk19
99.8
5*2
13.0
0P
roce
ssIn
tact
Team
wo
rk19
99.7
8*2
11.0
0P
erfo
rman
ceIn
tact
Team
wo
rk19
99.6
6*2
13.0
0P
erfo
rman
ceIn
tact
Team
wo
rkC
anno
n-B
ow
ers,
Sal
as, B
licke
nsd
erfe
r, &
Bo
wer
s19
98.7
636
3.00
Co
gni
tive
Ad
ho
cM
ix19
98.3
2*36
3.00
Pro
cess
Ad
ho
cM
ix19
98.3
9*36
3.00
Per
form
ance
Ad
ho
cM
ixC
ob
b20
00.2
5*36
4.00
Pro
cess
Ad
ho
cTe
amw
ork
2000
.28
364.
00P
erfo
rman
ceA
d h
oc
Team
wo
rkC
oo
ke, K
ieke
l, &
Hel
m20
01.8
2*10
3.00
Co
gni
tive
Ad
ho
cM
ix20
01.9
29
3.00
Per
form
ance
Ad
ho
cM
ixC
oo
ke e
t al
. (C
oo
ke e
t al
., 20
00)b
2003
.48*
333.
00C
og
niti
veA
d h
oc
Mix
Co
oke
et
al.
2003
.53
333.
00P
erfo
rman
ceA
d h
oc
Mix
2003
.43
93.
00P
erfo
rman
ceA
d h
oc
Mix
Dun
can
et a
l.19
96.7
68
5.00
Pro
cess
Inta
ctTe
amw
ork
1996
.88*
75.
00P
erfo
rman
ceIn
tact
Team
wo
rkE
llis,
Bel
l, P
loyh
art,
Ho
llenb
eck,
& Il
gen
2005
.56
654.
00C
og
niti
veA
d h
oc
Team
wo
rk20
05.3
1*65
4.00
Pro
cess
Ad
ho
cTe
amw
ork
E. B
. Ent
in, E
ntin
, Mac
Mill
an, &
Ser
faty
1993
.58
162.
00P
roce
ssIn
tact
Task
wo
rk
a Eff
ect
size
s (r
) den
oted
with
an
aste
risk
(*) r
epre
sent
an
aver
age
for
a si
ngle
sam
ple
and
a s
ing
le o
utco
me
and
hav
e b
een
com
bin
ed fo
r th
is m
eta-
anal
ysis
. bFo
ur e
ffec
ts s
izes
C
ontin
ued
on
next
pag
ein
dex
ing
tas
kwor
k kn
owle
dg
e fr
omth
e C
ooke
et
al. (
2000
) pro
ceed
ing
s p
aper
wer
e ad
ded
to
thre
e te
amw
ork
know
led
ge
effe
ct s
izes
from
the
Coo
ke e
t al
. (20
03) j
ourn
al a
rtic
le.
Thes
e co
gni
tive
effe
ct s
izes
wer
e av
erag
ed b
ecau
se t
hey
wer
e b
ased
upo
n th
e sa
me
sam
ple
of
par
tici
pan
ts.
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
915
TAB
LE 1
(co
ntin
ued
)
No
. of
Ave
rag
eC
rite
rio
nM
emb
ersh
ipTr
aini
ngA
utho
r(s)
Year
raTe
ams
Team
Siz
eD
escr
ipti
on
Stab
ility
Co
nten
t
E. E
. Ent
in &
Ser
faty
1999
.73*
65.
00P
roce
ssA
d h
oc
Mix
1999
.63
45.
00P
erfo
rman
ceA
d h
oc
Mix
1999
.57
45.
00P
erfo
rman
ceA
d h
oc
Mix
E. E
. Ent
in, S
erfa
ty, &
Dec
kert
1994
.53*
45.
00P
erfo
rman
ceA
d h
oc
Team
wo
rkFr
eem
an, C
ohe
n, &
Tho
mp
son
1998
.65*
142.
00P
roce
ssA
d h
oc
Mix
1998
.55
132.
00P
erfo
rman
ceA
d h
oc
Mix
1998
.75
132.
00P
erfo
rman
ceA
d h
oc
Mix
Gan
ster
, Will
iam
s, &
Po
pp
ler
1991
.05
434.
70P
erfo
rman
ceA
d h
oc
Team
wo
rkG
lickm
an e
t al
.19
87.4
113
5.33
Per
form
ance
Inta
ctTa
skw
ork
Gre
en19
94.8
114
5.00
Pro
cess
Ad
hoc
Team
wo
rk19
94.6
504
5.00
Per
form
ance
Ad
hoc
Team
wo
rkH
effn
er19
97.1
9*74
2.00
Per
form
ance
Ad
ho
cM
ixH
enry
19
97.2
810
4.70
Per
form
ance
Ad
ho
cTe
amw
ork
Iko
mi,
Bo
ehm
-Dav
is, H
olt
, & In
calc
ater
ra19
99.5
444
2.50
Per
form
ance
Inta
ctTe
amw
ork
Lam
pto
n et
al.
2001
.18
492.
00P
erfo
rman
ceA
d h
oc
Task
wo
rkLa
ssit
er, V
aug
hn, S
mal
tz, M
org
an, &
Sal
as
1990
.42*
452.
00P
erfo
rman
ceA
d h
oc
Team
wo
rkLe
Bla
nc, H
ox,
Sch
aufe
li, T
aris
, & P
eete
rs20
07–.
01*
2923
.00
Aff
ecti
veIn
tact
Team
wo
rk20
07.1
0*29
23.0
0P
roce
ssIn
tact
Team
wo
rkLe
edo
m &
Sim
on
1995
.53
162.
00A
ffec
tive
Ad
ho
cM
ix19
95.6
216
2.00
Pro
cess
Ad
ho
cM
ix19
95.6
1*16
2.00
Per
form
ance
Ad
ho
cM
ix19
95.1
015
2.00
Aff
ecti
veA
d h
oc
Mix
1995
.91
82.
00P
roce
ssA
d h
oc
Mix
1995
.60*
152.
00P
erfo
rman
ceA
d h
oc
Mix
Mac
Mill
an, E
ntin
, Ent
in, &
Ser
faty
1994
.57
82.
00A
ffec
tive
Inta
ctTe
amw
ork
1994
.48*
82.
00P
roce
ssIn
tact
Team
wo
rkM
arks
, Sab
ella
, Bur
ke, &
Zac
caro
2002
.43
453.
00C
og
niti
veA
d h
oc
Task
wo
rk20
02.3
649
3.00
Co
gni
tive
Ad
ho
cTa
skw
ork
Mar
ks, Z
acca
ro, &
Mat
hieu
2000
.44*
753.
00C
og
niti
veA
d h
oc
Mix
Mat
hieu
, Hef
fner
, Go
od
win
, Sal
as, &
20
00.0
7*56
2.00
Co
gni
tive
Ad
ho
cM
ixC
anno
n-B
ow
ers
2000
.35*
562.
00P
roce
ssA
d h
oc
Mix
2000
.19*
562.
00P
erfo
rman
ceA
d h
oc
Mix
a Eff
ect
size
s (r
) den
oted
with
an
aste
risk
(*) r
epre
sent
an
aver
age
for
a si
ngle
sam
ple
and
a s
ing
le o
utco
me
and
hav
e b
een
com
bin
ed fo
r th
is m
eta-
anal
ysis
. C
ontin
ued
on
next
pag
e
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
916
TAB
LE 1
(co
ntin
ued
)
No
. of
Ave
rag
eC
rite
rio
nM
emb
ersh
ipTr
aini
ngA
utho
r(s)
Year
raTe
ams
Team
Siz
eD
escr
ipti
on
Stab
ility
Co
nten
t
Min
ioni
s19
95.3
2*94
3.00
Co
gni
tive
Ad
ho
cTe
amw
ork
1995
.10*
903.
00A
ffec
tive
Ad
ho
cTe
amw
ork
1995
.17*
893.
00P
roce
ssA
d h
oc
Team
wo
rk19
95.3
2*90
3.00
Per
form
ance
Ad
ho
cTe
amw
ork
Mo
rey
et a
l.20
02.8
46
3.00
Aff
ecti
veIn
tact
Team
wo
rk20
02.8
56
3.00
Pro
cess
Inta
ctTe
amw
ork
2002
.43
63.
00P
erfo
rman
ceIn
tact
Team
wo
rk
Nay
lor
& B
rig
gs
1965
.46*
323.
00P
erfo
rman
ceA
d h
oc
Task
wo
rk
Par
sons
1981
.44
124.
00A
ffec
tive
Ad
ho
cTe
amw
ork
1981
.61*
124.
00P
erfo
rman
ceA
d h
oc
Team
wo
rk
Pea
rsal
l, E
llis,
& W
est
2004
.00*
544.
00C
og
niti
veA
d h
oc
Task
wo
rk
Pri
char
d &
Ash
leig
h20
07.8
116
3.00
Co
gni
tive
Ad
ho
cM
ix20
07.4
116
3.00
Aff
ecti
veA
d h
oc
Mix
2007
.54*
163.
00P
roce
ssA
d h
oc
Mix
2007
.26*
163.
00P
erfo
rman
ceA
d h
oc
Mix
Rap
p &
Mat
hieu
2007
.52
153.
38P
erfo
rman
ceA
d h
oc
Team
wo
rk
Ro
llins
& A
ngel
cyk
1995
.43
542.
50A
ffec
tive
Inta
ctTe
amw
ork
1995
.38
135
2.50
Aff
ecti
veIn
tact
Team
wo
rk19
95.2
613
2.50
Aff
ecti
veIn
tact
Team
wo
rk19
95.6
25
2.50
Aff
ecti
veIn
tact
Team
wo
rk
Schr
eib
er e
t al
.20
02.6
4*2
20.0
0A
ffec
tive
Inta
ctM
ix
Serf
aty,
Ent
in, &
Jo
hnst
on
1998
.63*
115.
00P
roce
ssA
d h
oc
Team
wo
rk19
98.6
1*11
5.00
Per
form
ance
Ad
ho
cTe
amw
ork
Shap
iro e
t al
.20
04.3
94
5.00
Pro
cess
Inta
ctTe
amw
ork
Sieg
el &
Fed
erm
an19
73.2
732
6.00
Per
form
ance
Inta
ctM
ix
Sto
ut, S
alas
, & F
ow
lkes
1997
.41
212.
00C
og
niti
veA
d h
oc
Team
wo
rk19
97.3
721
2.00
Aff
ecti
veA
d h
oc
Team
wo
rk19
97.6
2*21
2.00
Pro
cess
Ad
ho
cTe
amw
ork
1997
.68*
212.
00P
erfo
rman
ceA
d h
oc
Team
wo
rk
Volp
e, C
anno
n-B
ow
ers,
Sal
as, &
Sp
ecto
r19
96.3
6*40
2.00
Pro
cess
Ad
ho
cTa
skw
ork
1996
.39*
402.
00P
erfo
rman
ceA
d h
oc
Task
wo
rk
a Eff
ect
size
s (r
) den
oted
with
an
aste
risk
(*) r
epre
sent
an
aver
age
for
a si
ngle
sam
ple
and
a s
ing
le o
utco
me
and
hav
e b
een
com
bin
ed fo
r th
is m
eta-
anal
ysis
.
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 917
Hunter&Schmidt, 2004; Hunter, Schmidt,&Jack-son, 1982). Part of this determination stems froman assessment of the amount of unreliability andrange restriction that is present in the data. For ex-ample, it is standard practice for authors of meta-analyses to attempt to correct obtained reliabilitycoefficients for unreliability in measures of thepredictor and criterion (e.g., Hunter & Schmidt,2004). For the current research, it was an unfor-tunate reality that primary studies often failed toreport all of the auxiliary information necessary toperform corrections for study artifacts. Criterionreliability estimates were available for only 44%of the effect sizes in the database (41 of 93). Thus,in order to best deal with this issue, we utilizedartifact distribution meta-analysis, rather than per-forming corrections for unreliability on each indi-vidual effect size.
In addition to making corrections for the unre-liability of predictors and criteria in primary stud-ies, researchers using meta-analytic techniquesoften attempt to account for the effects of director indirect range restriction. In a recent article,Hunter, Schmidt, and Le (2006) argued that cor-recting for range restriction will generally resultin more accurate combined estimates of the rela-tionships between variables. However, the articlesincluded in the current database did not report thenecessary information on restricted and unre-stricted samples that would be needed to performcorrections for range restriction. As a result, wecould not make these corrections in the current re-search, and our effect size estimates may be con-servative. That is, they may underestimate the truenature of the relationships between team trainingand team functioning if the biasing effects ofrange restriction could be corrected.
Weighting
It is often recommended that studies with largersample sizes receive more weight in meta-analysesin order to increase the precision of the estimateof the average effect size across studies (Hunter &Schmidt, 2004; Shadish, Cook, & Campbell, 2002).Although estimates obtained from studies withlarger samples are not necessarily more valid, theyare presumed to be more stable (i.e., accurate). Atthe same time, recent simulation studies havefound that one can obtain a very accurate com-bined effect size through procedures employed toweight studies by sample size (e.g., Field, 2005).Taking these considerations into account, the pri-
mary study effect sizes that were utilized in thecurrent series of meta-analyses were weighted bytheir sample sizes.
RESULTS
Description of the Database
The final database for this research consisted ofa total of 93 correlations obtained from 45 primarystudies. These 93 effect sizes represented 2,650teams. Of these, 1,660 teams were teams from thelab or classroom setting, 762 teams were from the military sector, 138 teams were aviation teams,80 teams were medical teams, and 10 teams werefrom business organizations. It should be notedthese 93 effect sizes were not all from independentsamples (there were 52 independent samples).However, every subgroup analysis that was per-formed included only independent samples. Of the45 studies included in the meta-analytic database,31 were published (25 journal articles, 4 confer-ence proceedings, and 2 book chapters) and 14were unpublished (3 conference papers, 6 techni-cal reports, and 5 dissertations). Table 1 summa-rizes some of the key information that was recordedfrom each primary study.
Meta-Analytic Results
The meta-analytic results for the four primaryareas of investigation are reported in Tables 2through 5. In each of these tables, key pieces of in-formation from each analysis are displayed. Thisinformation includes the number of teams in eachanalysis (N), the number of independent effectsizes (correlations) in each analysis (k), the meanweighted observed correlation (r–), and the 80%confidence interval for that correlation. In addi-tion, the tables display the estimated true score cor-relation (ρ), the standard deviation of this truescore correlation (SDρ), the 80% credibility inter-val (10% CV and 90% CV), and the percentage ofvariance accounted for by statistical artifacts.
It is important to explain why both confidenceand credibility intervals are reported with this re-search. In general, they each provide informationthat contributes to the process of estimating thetrue nature of the relationships between two vari-ables (Whitener, 1990). However, confidence in-tervals are applied to observed scores, are centeredupon a single mean score, and take into accountthe effects of sampling error. Alternatively, cred-ibility intervals provide information concerning
text continues on page 922 at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TAB
LE 2
:Ana
lysi
s o
f th
e E
ffec
tiven
ess
of
Team
Tra
inin
g B
ased
Up
on
Out
com
e Ty
pe
% V
ar.
Acc
oun
ted
for
by
Out
com
e Ty
pe
Nk
r–C
I r 10
%C
I r90
%ρ
SDρ
10%
CV
90%
CV
Art
ifact
s
Co
gni
tive
554
12.3
8.3
0.4
6.4
2.1
9.1
8.6
734
.87
Aff
ecti
ve46
516
.32
.26
.37
.35
.00
.35
.35
100.
00b
Pro
cess
607
25.3
9.3
4.4
4.4
4.0
0.4
4.4
410
0.00
b
Per
form
ance
1,02
440
.33
.29
.37
.39
.09
.27
.50
86.6
3A
ll o
utco
mes
a1,
563
52.3
4.3
1.3
7.3
4.0
0.3
4.3
410
0.00
b
No
te.
N=
num
ber
of
team
s; k
= n
umb
er o
f co
rrel
atio
ns c
oef
ficie
nts
on
whi
ch e
ach
dis
trib
utio
n w
as b
ased
; r–
= m
ean
ob
serv
ed c
orr
elat
ion;
CI r
10%
= lo
wer
bo
und
of
the
confi
den
ce in
terv
al f
or
ob
serv
ed r
;C
I r90
% =
up
per
bou
nd o
f th
e co
nfid
ence
inte
rval
for
ob
serv
ed r
; ρ=
est
imat
ed t
rue
corr
elat
ion
bet
wee
n th
e p
red
icto
r co
nstr
uct
and
the
rel
evan
t cr
iterio
n (fu
lly c
orre
cted
for
mea
sure
men
t er
ror
in b
oth
the
pre
-d
icto
ran
d c
riter
ion)
; SD
ρ=
est
imat
ed s
tand
ard
dev
iatio
n o
f the
tru
e co
rrel
atio
n; 1
0% C
V =
low
er b
oun
d o
f the
cre
dib
ility
inte
rval
for
each
dis
trib
utio
n; 9
0% C
V =
up
per
bo
und
of t
he c
red
ibili
ty in
terv
al fo
r ea
chd
istr
ibut
ion;
% V
ar. A
cco
unte
d f
or
by
Art
ifact
s =
per
cent
age
of
ob
serv
ed v
aria
nce
acco
unte
d f
or
by
stat
isti
cal a
rtifa
cts.
a This
ent
ry r
epre
sent
s th
e en
tire
dat
abas
e o
f 52
ind
epen
den
t ef
fect
siz
es, w
hich
rep
rese
nted
93
anal
yses
fo
r d
isti
nct
out
com
es. H
ow
ever
, in
ord
er t
o e
stim
ate
an o
vera
ll ef
fect
of
team
tra
inin
g o
n te
am f
unc-
tioni
ng, w
e ne
eded
to
co
mb
ine
mul
tiple
eff
ect
size
est
imat
es t
hat
wer
e d
eriv
ed f
rom
sin
gle
sam
ple
s.bTh
ese
entr
ies
rep
rese
nt c
ases
in w
hich
the
per
cent
age
of
varia
nce
acco
unte
d f
or
by
artif
acts
res
ulte
d in
anu
mb
er g
reat
er t
han
100.
Thi
s ca
n b
e in
terp
rete
d s
uch
that
all
the
varia
nce
bet
wee
n ef
fect
siz
es f
rom
prim
ary
stud
ies
coul
d b
e ac
coun
ted
for
by
stud
y ar
tifac
t. W
e d
irect
the
rea
der
to
the
Met
a-A
naly
tic R
esul
tsse
ctio
nfo
r a
mo
re d
etai
led
exp
lana
tio
n.
918 at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TAB
LE 3
:Ana
lysi
s o
f th
e E
ffec
tiven
ess
of
Team
Tra
inin
g B
ased
Up
on
Trai
ning
Co
nten
t
% V
ar.
Acc
oun
ted
for
by
Trai
ning
Co
nten
tO
utco
me
Typ
eN
kr–
CI r
10%
CI r
90%
ρSD
ρ10
% C
V90
% C
VA
rtifa
cts
Task
wo
rk
Co
gni
tive
242
4.2
8.1
8.3
7.3
0.1
0.1
7.4
362
.13
Aff
ecti
ve90
1.1
0.1
0.1
0.1
1.0
0.1
1.1
1—
Pro
cess
145
3.2
7.1
6.3
7.2
8.0
2.2
6.3
198
.23
Per
form
ance
240
6.3
5.2
8.4
1.3
5.0
0.3
5.3
510
0.00
a
Team
wo
rk
Co
gni
tive
862
.52
.46
.58
.52
.00
.52
.52
100.
00a
Aff
ecti
ve32
611
.37
.32
.43
.41
.00
.41
.41
100.
00a
Pro
cess
236
13.4
0.3
3.4
7.4
4.0
0.4
4.4
410
0.00
a
Per
form
ance
374
17.3
5.2
8.4
2.3
8.0
8.2
9.4
889
.54
Mix
C
og
niti
ve22
66
.45
.31
.58
.51
.25
.20
.83
27.1
8A
ffec
tive
494
.36
.25
.48
.36
.00
.36
.36
100.
00a
Pro
cess
226
9.4
7.4
0.5
4.5
6.0
0.5
6.5
610
0.00
a
Per
form
ance
410
17.3
0.2
3.3
8.4
0.1
5.2
0.5
975
.17
No
te.
N=
num
ber
of
team
s; k
= n
umb
er o
f co
rrel
atio
ns c
oef
ficie
nts
on
whi
ch e
ach
dis
trib
utio
n w
as b
ased
; r–
= m
ean
ob
serv
ed c
orr
elat
ion;
CI r
10%
= lo
wer
bo
und
of
the
confi
den
ce in
terv
al f
or
ob
serv
ed r
;C
I r90
% =
up
per
bou
nd o
f th
e co
nfid
ence
inte
rval
for
ob
serv
ed r
; ρ=
est
imat
ed t
rue
corr
elat
ion
bet
wee
n th
e p
red
icto
r co
nstr
uct
and
the
rel
evan
t cr
iterio
n (fu
lly c
orre
cted
for
mea
sure
men
t er
ror
in b
oth
the
pre
-d
icto
ran
d c
riter
ion)
; SD
ρ=
est
imat
ed s
tand
ard
dev
iatio
n o
f the
tru
e co
rrel
atio
n; 1
0% C
V =
low
er b
oun
d o
f the
cre
dib
ility
inte
rval
for
each
dis
trib
utio
n; 9
0% C
V =
up
per
bo
und
of t
he c
red
ibili
ty in
terv
al fo
r ea
chd
istr
ibut
ion;
% V
ar. A
cco
unte
d f
or
by
Art
ifact
s =
per
cent
age
of
ob
serv
ed v
aria
nce
acco
unte
d f
or
by
stat
isti
cal a
rtifa
cts.
a Thes
e en
trie
s re
pre
sent
cas
es in
whi
ch t
he p
erce
ntag
e o
f va
rianc
e ac
coun
ted
fo
r b
y ar
tifac
ts r
esul
ted
in a
num
ber
gre
ater
tha
n 10
0. T
his
can
be
inte
rpre
ted
suc
h th
at a
ll th
e va
rianc
e b
etw
een
effe
ct s
izes
fro
mp
rim
ary
stud
ies
coul
d b
e ac
coun
ted
fo
r b
y st
udy
arti
fact
. We
dire
ct t
he r
ead
er t
o t
he M
eta-
Ana
lyti
c R
esul
ts s
ecti
on
for
a m
ore
det
aile
d e
xpla
nati
on.
919 at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TAB
LE 4
:Ana
lysi
s o
f th
e E
ffec
tiven
ess
of
Team
Tra
inin
g B
ased
Up
on
Team
Mem
ber
ship
Sta
bili
ty
% V
ar.
Acc
oun
ted
Mem
ber
ship
for
by
Stab
ility
Out
com
e Ty
pe
Nk
r–C
I r 10
%C
I r90
%ρ
SDρ
10%
CV
90%
CV
Art
ifact
s
Inta
ctC
og
niti
ve—
——
——
——
——
—A
ffec
tive
254
9.3
7.3
0.4
4.4
1.0
0.4
1.4
110
0.00
a
Pro
cess
758
.43
.30
.55
.48
.00
.48
.48
100.
00a
Per
form
ance
122
8.4
9.4
1.5
6.5
4.0
0.5
4.5
410
0.00
a
Ad
Ho
cC
og
niti
ve55
412
.38
.30
.46
.42
.19
.18
.67
34.8
7A
ffec
tive
211
7.2
6.1
8.3
4.2
8.0
0.2
8.2
810
0.00
a
Pro
cess
532
17.3
9.3
4.4
4.4
4.0
4.3
9.4
995
.05
Per
form
ance
902
32.3
1.2
6.3
6.3
8.1
0.2
5.5
083
.66
No
te.
N=
num
ber
of
team
s; k
= n
umb
er o
f co
rrel
atio
ns c
oef
ficie
nts
on
whi
ch e
ach
dis
trib
utio
n w
as b
ased
; r–
= m
ean
ob
serv
ed c
orr
elat
ion;
CI r
10%
= lo
wer
bo
und
of
the
confi
den
ce in
terv
al f
or
ob
serv
ed r
;C
I r90
% =
up
per
bou
nd o
f th
e co
nfid
ence
inte
rval
for
ob
serv
ed r
; ρ=
est
imat
ed t
rue
corr
elat
ion
bet
wee
n th
e p
red
icto
r co
nstr
uct
and
the
rel
evan
t cr
iterio
n (fu
lly c
orre
cted
for
mea
sure
men
t er
ror
in b
oth
the
pre
-d
icto
ran
d c
riter
ion)
; SD
ρ=
est
imat
ed s
tand
ard
dev
iatio
n o
f the
tru
e co
rrel
atio
n; 1
0% C
V =
low
er b
oun
d o
f the
cre
dib
ility
inte
rval
for
each
dis
trib
utio
n; 9
0% C
V =
up
per
bo
und
of t
he c
red
ibili
ty in
terv
al fo
r ea
chd
istr
ibut
ion;
% V
ar. A
cco
unte
d f
or
by
Art
ifact
s =
per
cent
age
of
ob
serv
ed v
aria
nce
acco
unte
d f
or
by
stat
isti
cal a
rtifa
cts.
a Thes
e en
trie
s re
pre
sent
cas
es in
whi
ch t
he p
erce
ntag
e o
f va
rianc
e ac
coun
ted
fo
r b
y ar
tifac
ts r
esul
ted
in a
num
ber
gre
ater
tha
n 10
0. T
his
can
be
inte
rpre
ted
suc
h th
at a
ll th
e va
rianc
e b
etw
een
effe
ct s
izes
fro
mp
rim
ary
stud
ies
coul
d b
e ac
coun
ted
fo
r b
y st
udy
arti
fact
. We
dire
ct t
he r
ead
er t
o t
he M
eta-
Ana
lyti
c R
esul
ts s
ecti
on
for
a m
ore
det
aile
d e
xpla
nati
on.
920 at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TAB
LE 5
:Ana
lysi
s o
f th
e E
ffec
tiven
ess
of
Team
Tra
inin
g B
ased
Up
on
Team
Siz
e
% V
ar.
Acc
oun
ted
for
by
Team
Siz
eO
utco
me
Typ
eN
kr–
CI r
10%
CI r
90%
ρSD
ρ10
% C
V90
% C
VA
rtifa
cts
Smal
l C
og
niti
ve77
2.1
6.0
3.3
0.1
6.0
0.1
6.1
610
0.00
a
Aff
ecti
ve24
96
.39
.34
.43
.39
.00
.39
.39
100.
00a
Pro
cess
253
10.4
8.4
2.5
4.5
9.0
0.5
9.5
910
0.00
a
Per
form
ance
418
12.2
8.2
0.3
6.3
9.1
8.1
5.6
261
.63
Med
ium
C
og
niti
ve47
710
.42
.34
.50
.46
.18
.23
.69
35.2
9A
ffec
tive
183
7.2
7.1
8.3
6.2
8.0
0.2
8.2
810
0.00
a
Pro
cess
288
7.3
1.2
4.3
7.3
3.0
0.3
3.3
310
0.00
a
Per
form
ance
461
15.3
4.2
8.4
0.3
4.0
9.2
3.4
677
.96
Larg
e C
og
niti
ve—
——
——
——
——
—A
ffec
tive
333
.07
–.09
.22
.08
.00
.08
.08
100.
00a
Pro
cess
668
.43
.29
.56
.49
.00
.49
.49
100.
00a
Per
form
ance
145
13.4
4.3
7.5
1.5
0.0
0.5
0.5
010
0.00
a
No
te.
N=
num
ber
of
team
s; k
= n
umb
er o
f co
rrel
atio
ns c
oef
ficie
nts
on
whi
ch e
ach
dis
trib
utio
n w
as b
ased
; r–
= m
ean
ob
serv
ed c
orr
elat
ion;
CI r
10%
= lo
wer
bo
und
of
the
confi
den
ce in
terv
al f
or
ob
serv
ed r
;C
I r90
% =
up
per
bou
nd o
f th
e co
nfid
ence
inte
rval
for
ob
serv
ed r
; ρ=
est
imat
ed t
rue
corr
elat
ion
bet
wee
n th
e p
red
icto
r co
nstr
uct
and
the
rel
evan
t cr
iterio
n (fu
lly c
orre
cted
for
mea
sure
men
t er
ror
in b
oth
the
pre
-d
icto
ran
d c
riter
ion)
; SD
ρ=
est
imat
ed s
tand
ard
dev
iatio
n o
f the
tru
e co
rrel
atio
n; 1
0% C
V =
low
er b
oun
d o
f the
cre
dib
ility
inte
rval
for
each
dis
trib
utio
n; 9
0% C
V =
up
per
bo
und
of t
he c
red
ibili
ty in
terv
al fo
r ea
chd
istr
ibut
ion;
% V
ar. A
cco
unte
d f
or
by
Art
ifact
s =
per
cent
age
of
ob
serv
ed v
aria
nce
acco
unte
d f
or
by
stat
isti
cal a
rtifa
cts.
a Thes
e en
trie
s re
pre
sent
cas
es in
whi
ch t
he p
erce
ntag
e o
f va
rianc
e ac
coun
ted
fo
r b
y ar
tifac
ts r
esul
ted
in a
num
ber
gre
ater
tha
n 10
0. T
his
can
be
inte
rpre
ted
suc
h th
at a
ll th
e va
rianc
e b
etw
een
effe
ct s
izes
fro
mp
rim
ary
stud
ies
coul
d b
e ac
coun
ted
fo
r b
y st
udy
arti
fact
. We
dire
ct t
he r
ead
er t
o t
he M
eta-
Ana
lyti
c R
esul
ts s
ecti
on
for
a m
ore
det
aile
d e
xpla
nati
on.
921 at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
922 December 2008 – Human Factors
the distribution of effect sizes after other researchartifacts have been removed. That is, credibility in-tervals use the estimated true correlation betweenconstructs as their basis. Moreover, credibility in-tervals, along with the estimate for the percentageof variance attributable to statistical artifacts, canprovide information on whether moderators maybe operating (Whitener, 1990). As the percentageof variance attributable to artifacts increases, themore certain it is that additional moderators arenot operating.
Abrief discussion explaining the percentage ofvariance accounted for by artifacts is appropriate.The meta-analytic procedure that we used is basedon the hypothesis that much of the variation instudy results is often based on statistical andmethodological artifacts, rather than true under-lying population relationships. These artifacts maydistort study findings in many ways. For exam-ple, sampling error distorts study findings ran-domly, whereas the effect of measurement erroris to systematically bias study results. Thus, thismeta-analytic procedure corrects for the artifactualdifferences in study results. The amount of vari-ance accounted for by study artifacts is based oncalculating the ratio of the variance attributable toartifacts to the total variance.
This index, as calculated by the Hunter-SchmidtMeta-Analysis Program1.1(Schmidt & Le, 2005),represents the amount of variance in the observedcorrelations (those gathered from the primarystudies) attributed to artifacts. In other words, thisnumber indicates the amount of variance that isaccounted for by sampling error and unreliability.This index allows one to judge whether substan-tial variation attributable to moderators exists. Anumber greater than 100% is not uncommon andis an indication that the variation in effect sizesfrom the primary studies could have been attribut-able to second-order sampling error and not exist-ing moderators (Hunter & Schmidt, 2004).
If the percentage of variance accounted for byartifacts is calculated to be 100 or greater, whetherthe number is 150 or 300, the correct conclusionwould be that all the total variance could be ac-counted for by artifacts (Hunter & Schmidt, 2004).The magnitude of the number does not reveal any-thing about the quality of the analysis. Rather, whatis meaningful is if the number is greater than 100%or less than 100%. Anumber under 100% indicatesthat some of the variance left to account for maybe attributable to unhypothesized moderators,
whereas a number over 100% indicates that thetotal variance could be accounted for by artifactssuch as sampling error or measurement error.
There is no relationship between this numberand the confidence in the estimated true correla-tion. Rather, this is another piece of informationthat is used in order to interpret the findings. More-over, when the percentage of variance accountedfor is greater than 100%, the credibility interval be-comes a point estimate. The credibility interval isyet another indicator of the existence of modera-tors. When a credibility interval is a point estimate,it is unlikely that moderators exist based on theanalysis.
In the following sections, the results derivedfrom our investigations into the four primary areasof interest in this research are presented. In meta-analysis there is no criterion that illustrates theminimum number of effect sizes to include in ananalysis. We do acknowledge and understand thatconducting an analysis with a small number ofeffect sizes does increase the possibility of second-order sampling error (Arthur, Bennett, & Huffcutt,2001; Hunter&Schmidt,2004). In order to be com-plete and to fully illustrate the database used in thismeta-analysis, we conducted all possible analyseswith one or more effect sizes. However, we do em-phasize, as did Arthur, Bennett, Edens, and Bell(2003, see p. 238), that any meta-analyzed effectsize based on fewer than five effect sizes should beinterpreted with caution.
Does team training work? The overall resultssupport our hypothesis that team training doeswork. For the combined (and averaged, when nec-essary) set of independent outcomes, team train-ing was shown to have a moderate, positive effecton team functioning (ρ = .34; 10% CV = .34; 90%CV = .34). This analysis was based on 52 effectsizes representing 1,563 teams and provided sup-port for Hypothesis 1.
Criterion description. The results of our analy-ses revealed team training to have a positive effecton each of the four outcomes under investigation(see Table 2). For the 12 available effect sizes forteam-level cognitive outcomes, the estimated truescore correlation (ρ) was .42. The 80% credibilityinterval (10% CV and 90% CV) ranged from .18to .67. For affective outcomes, the analysis of 16effect sizes (representing 465 teams) revealed anestimated true score correlation of .35. Next, teamtraining appeared to work best for process out-comes (ρ = .44; 10% CV = .44; 90% CV = .44);
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 923
this analysis was based on 25 independent effectsizes representing 607 teams. However, given theoverlapping confidence and credibility intervals,this assertion is merely suggestive and not defin-itive. Finally, team training was also shown to beeffective for performance outcomes (ρ = .39; 10%CV = .27; 90% CV = .50). Overall, these findingsprovide support for Hypotheses 2a through 2d.
Training content. The content of training in-terventions was also investigated as a potentialmoderator variable (see Table 3). For this analysis,the content of study interventions was coded asprimarily taskwork, teamwork, or both. Overall,the analyses provided support for Hypotheses 3through 5 but not necessarily Hypothesis 6. Thatis, although each type of intervention was shownto be useful, there is insufficient evidence to con-clude that mixed content interventions are supe-rior to taskwork or teamwork interventions alone.
Moreover, there was very little differenceamong taskwork (ρ = .35), teamwork (ρ = .38), ormixed content (ρ = .40) interventions for the im-provement of performance. However, for processoutcomes, taskwork training (ρ = .28) did not workas well as teamwork (ρ = .44) or mixed contenttraining (ρ = .56); caution should be exercised inthe interpretation of the taskwork effect size, as itis based on three effect sizes. Similarly, teamwork-focused training appeared to result in enhancedaffective outcomes in comparison with taskworktraining (ρ = .41 and .11, respectively). We againcaution the reader in the interpretation made onthe estimated correlation for taskwork content onaffective outcomes, as it was based on one pointestimate.
Team membership stability. The results for theanalyses on team membership stability providedpartial support for Hypothesis 7 (see Table 4). Spe-cifically, team training appeared to work just aswell (if not better) for intact teams. This was es-pecially the case for performance outcomes, forwhich ρ = .54 and .38 for intact and ad hoc teams,respectively. At the same time, the findings forprocess outcomes were highly similar across thetwo team membership configurations (ρ = .48 and.44 for intact and ad hoc teams, respectively).
Team size. For the analyses addressing the po-tential moderating effect of team size, we dividedstudies into three categories based on the averagesize of the teams under investigation: small, me-dium, and large. The most informative comparisonhere concerns performance outcomes. Our results
suggest that team size impacts the benefit fromteam training in terms of improved performance(ρ = .39, .34, and .50, respectively, for small, me-dium, and large teams; see Table 5). These resultsare based upon having 12, 15, and 13 effect sizesin each group, respectively. The findings from theother three outcomes with team size as a potentialmoderator are not as clear. For example, team train-ing appeared to influence cognitive outcomes toa larger extent in teams of medium size (ρ = .46).Conversely, small teams benefited the most interms of improvements in affective (ρ = .39) andprocess (ρ=.59) outcomes. This diverse set of find-ings does not provide any appreciable level of sup-port for Hypothesis 8.
Post hoc analyses. We conducted some post-hoc meta-analyses on effect sizes based on domainand strategy (see Tables 6 and 7). The five primarydomains that are represented in our sample arestudies conducted in the aviation, military, med-ical, and business sectors and those conducted inlaboratories with student samples. The trainingstrategies that were represented in our databaseconsisted of coordination training/crew resourcemanagement, cross-training, communication train-ing, critical thinking training, self-guided training,self-correction training, team adaptation and coor-dination training (TACT), stress training, tacticaltraining, and team knowledge training. These re-sults should be interpreted with caution but none-theless can be used to paint a picture of not onlyteam training in each of these domains but alsowhere the research and quantifiable data on train-ing effectiveness are lacking.
Most of the primary studies could be catego-rized into two domains: studies conducted in themilitary and those conducted in the lab with stu-dent samples. These analyses were based on 39and 37 effect sizes, representing 762 teams in themilitary domain and 1,660 student teams. Teamtraining appeared more effective in military set-tings (ρ = .59; 10% CV = .59; 90% CV = .59) thanin laboratory settings (ρ = .33; 10% CV = .22; 90%CV = .43).
The other three domains represented by the pri-mary studies in our analyses were from the med-ical, aviation, and business settings. These threedomains did not render a significant number of pri-mary studies that reported the necessary quantifi-able data to use in meta-analysis. Six independenteffect sizes were utilized to conduct the analysis formedical and aviation studies, and five independent
text continues on page 926 at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
924
TAB
LE 6
:Po
st H
oc
Ana
lysi
s o
f th
e E
ffec
tiven
ess
of
Team
Tra
inin
g B
ased
Up
on
Do
mai
n
% V
ar.
Acc
oun
ted
for
by
Do
mai
nN
kr–
CI r
10%
CI r
90%
ρSD
ρ10
% C
V90
% C
VA
rtifa
cts
Lab
/uni
vers
ity
1,66
037
.30
.27
.33
.33
.08
.22
.43
77.4
2M
ilita
ry76
239
.49
.45
.53
.59
.00
.59
.59
100.
00a
Bus
ines
s10
5.7
2.6
8.7
7.8
5.0
0.8
5.8
510
0.00
a
Avi
atio
n13
86
.29
.15
.43
.33
.20
.08
.58
56.8
3M
edic
al80
6.2
1.0
6.3
7.2
3.1
0.1
1.3
591
.17
No
te.
N=
num
ber
of
team
s; k
= n
umb
er o
f co
rrel
atio
ns c
oef
ficie
nts
on
whi
ch e
ach
dis
trib
utio
n w
as b
ased
; r–
= m
ean
ob
serv
ed c
orr
elat
ion;
CI r
10%
= lo
wer
bo
und
of
the
confi
den
ce in
terv
al f
or
ob
serv
ed r
;C
I r90
% =
up
per
bou
nd o
f th
e co
nfid
ence
inte
rval
for
ob
serv
ed r
; ρ=
est
imat
ed t
rue
corr
elat
ion
bet
wee
n th
e p
red
icto
r co
nstr
uct
and
the
rel
evan
t cr
iterio
n (fu
lly c
orre
cted
for
mea
sure
men
t er
ror
in b
oth
the
pre
-d
icto
ran
d c
riter
ion)
; SD
ρ=
est
imat
ed s
tand
ard
dev
iatio
n o
f the
tru
e co
rrel
atio
n; 1
0% C
V =
low
er b
oun
d o
f the
cre
dib
ility
inte
rval
for
each
dis
trib
utio
n; 9
0% C
V =
up
per
bo
und
of t
he c
red
ibili
ty in
terv
al fo
r ea
chd
istr
ibut
ion;
% V
ar. A
cco
unte
d f
or
by
Art
ifact
s =
per
cent
age
of
ob
serv
ed v
aria
nce
acco
unte
d f
or
by
stat
isti
cal a
rtifa
cts.
a Thes
e en
trie
s re
pre
sent
cas
es in
whi
ch t
he p
erce
ntag
e o
f va
rianc
e ac
coun
ted
fo
r b
y ar
tifac
ts r
esul
ted
in a
num
ber
gre
ater
tha
n 10
0. T
his
can
be
inte
rpre
ted
suc
h th
at a
ll th
e va
rianc
e b
etw
een
effe
ct s
izes
fro
mp
rim
ary
stud
ies
coul
d b
e ac
coun
ted
fo
r b
y st
udy
arti
fact
. We
dire
ct t
he r
ead
er t
o t
he M
eta-
Ana
lyti
c R
esul
ts s
ecti
on
for
a m
ore
det
aile
d e
xpla
nati
on.
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
925
TAB
LE 7
:Po
st H
oc
Ana
lysi
s o
f th
e E
ffec
tiven
ess
of
Team
Tra
inin
g B
ased
Up
on
Stra
teg
y
% V
ar.
Acc
oun
ted
for
by
Stra
teg
yN
kr–
CI r
10%
CI r
90%
ρSD
ρ10
% C
V90
% C
VA
rtifa
cts
Co
ord
inat
ion/
CR
M t
rain
ing
744
33.4
3.3
9.4
6.4
7.0
0.4
7.4
710
0.00
a
Cro
ss-t
rain
ing
432
14.4
0.3
3.4
7.4
4.1
4.2
6.6
259
.05
Co
mm
unic
atio
n tr
aini
ng86
3.1
3–.
02.2
8.1
3.0
8.0
2.2
383
.91
Cri
tica
l thi
nkin
g t
rain
ing
834
.34
.14
.53
.60
.40
.09
1.12
43.5
8Se
lf-g
uid
ed t
rain
ing
822
.35
.30
.39
.36
.00
.36
.36
100.
00a
Self-
corr
ecti
on
trai
ning
792
.24
.05
.43
.27
.16
.06
.48
51.4
6TA
CT
676
.50
.43
.57
.56
.00
.56
.56
100.
00a
Stre
ss t
rain
ing
582
.05
–.01
.10
.05
.00
.05
.05
100.
00a
Tact
ical
tra
inin
g20
2.6
7.6
1.7
3.6
7.0
0.6
7.6
710
0.00
a
Team
kno
wle
dg
e tr
aini
ng15
2.8
1.7
6.8
7.8
1.0
0.8
1.8
110
0.00
a
No
te.
N=
num
ber
of
team
s; k
= n
umb
er o
f co
rrel
atio
ns c
oef
ficie
nts
on
whi
ch e
ach
dis
trib
utio
n w
as b
ased
; r–
= m
ean
ob
serv
ed c
orr
elat
ion;
CI r
10%
= lo
wer
bo
und
of
the
confi
den
ce in
terv
al f
or
ob
serv
ed r
;C
I r90
% =
up
per
bou
nd o
f th
e co
nfid
ence
inte
rval
for
ob
serv
ed r
; ρ=
est
imat
ed t
rue
corr
elat
ion
bet
wee
n th
e p
red
icto
r co
nstr
uct
and
the
rel
evan
t cr
iterio
n (fu
lly c
orre
cted
for
mea
sure
men
t er
ror
in b
oth
the
pre
-d
icto
ran
d c
riter
ion)
; SD
ρ=
est
imat
ed s
tand
ard
dev
iatio
n o
f the
tru
e co
rrel
atio
n; 1
0% C
V =
low
er b
oun
d o
f the
cre
dib
ility
inte
rval
for
each
dis
trib
utio
n; 9
0% C
V =
up
per
bo
und
of t
he c
red
ibili
ty in
terv
al fo
r ea
chd
istr
ibut
ion;
% V
ar. A
cco
unte
d f
or
by
Art
ifact
s =
per
cent
age
of
ob
serv
ed v
aria
nce
acco
unte
d f
or
by
stat
isti
cal a
rtifa
cts.
a Thes
e en
trie
s re
pre
sent
cas
es in
whi
ch t
he p
erce
ntag
e o
f va
rianc
e ac
coun
ted
fo
r b
y ar
tifac
ts r
esul
ted
in a
num
ber
gre
ater
tha
n 10
0. T
his
can
be
inte
rpre
ted
suc
h th
at a
ll th
e va
rianc
e b
etw
een
effe
ct s
izes
fro
mp
rim
ary
stud
ies
coul
d b
e ac
coun
ted
fo
r b
y st
udy
arti
fact
. We
dire
ct t
he r
ead
er t
o t
he M
eta-
Ana
lyti
c R
esul
ts s
ecti
on
for
a m
ore
det
aile
d e
xpla
nati
on.
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
926 December 2008 – Human Factors
effect sizes were found from studies conducted inbusiness settings. The results indicate that teamtraining conducted with business groups (ρ = .85)resulted in greater effect sizes than did studiesconducted with medical teams (ρ=.23;10% CV=.11; 90% CV = .35) and aviation teams (ρ = .33;10% CV= .08; 90% CV= .58). Again, we cautionthe reader in interpreting these analyses.
Our last post hoc analysis was based on thetraining strategy used. The two most popular strat-egies represented in our database were coordina-tion training and cross-training. Thirty-three effectsizes were meta-analyzed in order to assess theimpact that coordination training had on team out-comes. The results indicated that this type of train-ing had a moderate positive effect on outcomes(ρ = .47; 10% CV = .47; 90% CV = .47). Cross-training was represented in our database by 14 ef-fect sizes and 432 teams. These results suggest thatcross-training does have a moderate positive effecton outcomes (ρ = .44; 10% CV = .26; 90% CV =.62). We direct the reader to Table 7, which in-cludes all the analyses based upon strategy. Andagain, we strongly caution the interpretation ofthese analyses and can state only that more re-search is needed based on strategy.
DISCUSSION
Meta-analysis is a valuable tool for navigatingthe research literature when conflicting findingsare the norm rather than the exception. Most areasof research fall prey to this issue, and the teamtraining literature is certainly not immune. Ameta-analysis of this area is beneficial so that the find-ings across studies can be integrated and revealsome simpler patterns of relationships. These pre-liminary patterns can then provide a basis for theory development. In addition to this benefit,meta-analysis can be particularly advantageous forresearch on teams because of the small-samplestudies that are typical of this area. The current in-vestigation therefore provides an overdue empir-ical accounting of the team training literature and,in turn, identifies important patterns of results.
What Works and Why?
The findings from the present study provide in-sight into the extent to which team training inter-ventions relate to team outcomes. That is, acrossa wide variety of settings, tasks, and team types,team training efforts were successful. Although
“a positive relationship between training and per-formance in team settings is and has been clear forsome time” (Hollenbeck et al., 2004, p. 360), thisresearch is, to the best of our knowledge, the firstto comprehensively and quantitatively evaluatethe relationships between these interventions andoutcomes.
These findings suggest that team training inter-ventions are a viable approach for organizations totake in order to enhance team outcomes. They areuseful for improving cognitive outcomes, affec-tive outcomes, teamwork processes, and perfor-mance outcomes. That is, team training accountedfor approximately 12% to 19% of the variance inthe examined outcomes. This percentage, thoughstatistically correct, is an underestimate of the prac-tical and theoretical significance of the relation-ships between the variables investigated in thepresent meta-analysis (e.g., Hunter & Schmidt,1990). Thus, the utility of team development tech-niques is likely understated by the present find-ings. Further, the large number of studies in thedatabase and the large overall observed effect sizemake these findings robust to disconfirmation, ex-cluding those analyses that included fewer thanfive effect sizes.
Across all outcomes, team training interven-tions were more effective for team processes thanfor the other outcome types. As a primary expla-nation, in dynamic input-throughput-output mod-els it is generally understood that team-memberand team-level processes such as communicationand coordination are determinants of team perfor-mance and effectiveness. Although the traininginterventions reviewed in the present study didappear to have a nearly equal impact on outcomesas processes, it stands to reason that improvingprocesses will also positively influence perfor-mance outcomes.
For example, training teams to communicate,coordinate, and make decisions should ultimatelyincrease the quantity of products produced. Finally,it is also recognized that team performance isshaped by additional environmental and/or orga-nizational characteristics and contingencies thatare out of the control of team members; team mem-bers actively execute team processes but are oftenat the mercy of a larger organizational system to produce more distal performance outputs. Ofcourse, team processes may also be influenced bythe larger organizational system, but probably notto the same degree as team performance outcomes.
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 927
In contrast to our hypothesis, team training in-terventions with a mixed training content (i.e.,teamwork and taskwork) were not superior to thosewith a taskwork or teamwork focus in isolation.Team training was deemed useful for improvingteam performance outcomes, regardless of the typeof training content. This finding could be inter-preted such that team training is helpful but thatcombining teamwork and taskwork content doesnot necessarily make the training incrementallymore beneficial. The problem could potentiallybe that too much information was included inthe training given to teams. As such, the teams ex-posed to mixed-content training may have beenoverwhelmed and could not successfully grasp theintended concepts and information that were pre-sented to them.
An additional explanation is available, but itmerits more research. Specifically, Salas, Burke,and Cannon-Bowers (2002) identified the impor-tance of both taskwork and teamwork training.They argued that it is best to develop a team’s task-work knowledge prior to the team’s acquisitionof teamwork skills. Alternatively, the results froma recent study by Ellis et al. (2005) suggested thatenhancing teamwork skills prior to taskworkknowledge leads to greater performance improve-ments. Although these explanations do not com-bine to provide a clear explanation for the lack ofsupport for Hypothesis 6, they do serve as a call toresearchers to further examine the impact of tem-poral factors when designing training that containsboth teamwork and taskwork components.
Our research also showed that membershipstability moderated the relationship between teamtraining and team outcomes, such that intact teamsthat underwent training improved the most onprocess and performance outcomes. Our hypoth-esis that intact teams would benefit most fromteam training interventions was only partially sup-ported, however. There was clear support, giventhe nonoverlapping confidence intervals, for ourassertion that intact teams would benefit the mostfrom training interventions when the focus was onperformance outcomes. This finding was consis-tent with our belief that those teams that had an op-portunity to work with one another would benefitthe most from team training.
Intact teams should have already overcomesome of the maturational challenges that newlyformed teams would not have had the opportunityto navigate. Ad hoc teams are likely forced to at-
tend to many more factors during their trainingsession than intact teams, and thus intact teamsshould benefit the most from team training. Anadditional explanation could be that intact teamsmay have seen the benefits of training more sothan ad hoc teams and, thus, were more likely toattend to the important aspects of training, whethertaskwork or teamwork focused in nature. In otherwords, intact teams may have perceived the train-ing as being more instrumental to their team’s per-formance improvement and, ultimately, to theattainment of valued outcomes in the workplace.
The moderator analysis on team size helped tofurther clarify when team training is most helpfulfor collectives. Team performance, as expected,improved the most for large teams. Moreover, teamprocesses improved the most for small teams.These findings partially supported our hypotheses.Intuitively, if the members of a small team arebehaving in a way that impedes their performance,once they are trained how to properly communi-cate and coordinate with one another, they mayhave an easier time maintaining those process en-hancements.
Asurprising finding was the extremely low ef-fect size (ρ = .08) for affective outcomes in largeteams. However, it is important to interpret thisfinding with caution because there were only threeeffect sizes to combine for this calculation. Addi-tionally, it speaks to the need for more research onthese phenomena in order to appropriately deter-mine the effect of team training on affective out-comes for large teams. Moreover, as there were noeffect sizes available to meta-analyze for cognitiveoutcomes for large teams, more research is neces-sary in this area as well.
Implications for Practitioners
We believe that much progress has been madewith respect to team training, both in research andin practice. The meta-analytic findings are encour-aging in that they show team training to be ef-fective. However, these findings highlight somepractical issues for organizational change agentsto consider when designing and implementingtraining. First, practitioners should relish the factthat team training is effective. The implications ofthis are critical. To know that team training can ex-plain 12% to 19% of the variance of a team’s per-formance (i.e., better-performing teams will exhibitmore effective team behaviors than their counter-parts) can mean reducing medical errors (in health
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
928 December 2008 – Human Factors
care), saving an aircraft (in aviation), increasingthe bottom line (in business), or saving lives (in the military).
Second, given the heightened interest in teamtraining in health care, change agents in health careinstitutions should utilize this information to bol-ster their argument for implementing such training.
Third, in deciding on the specific type and fo-cus of team training, practitioners need to be cog-nizant of the ultimate outcome they are interestedin, as our findings indicated differential effects.Sometimes these effects were rather small, but atother times they were rather large, depending onthe specific outcome targeted (i.e., cognition, be-havior/process, affect, performance).
In conclusion, the data we present, as we havestated several times, are encouraging. Of most im-portance, however, is what individuals do to im-prove the training and development initiatives thatthey manage. We hope that practitioners and re-searchers alike find useful and practical advice tofollow based on the findings presented.
Study Limitations
In general, a large number of methodologicaltools are available to researchers who attempt todiscover the meaning of organizational phenom-ena (McCall & Bobko, 1990); meta-analysis isjust one such tool. The validity of inferences con-cerning any relationship examined through meta-analysis is a function of several factors, including(a) the representativeness of the sample of primarystudies considered by the meta-analyst, (b) the de-gree of validity of each of the primary studies, and(c) the number of primary studies (Bobko & Stone-Romero, 1998).
Despite the strengths and contributions of thecurrent study, several potential limitations shouldbe noted. First, although extensive efforts weremade to be inclusive and a large number of pri-mary studies were included, definitive conclusionsregarding the validity of each included study can-not be made. To the extent that the primary stud-ies lacked proper control mechanisms, either viadesign features or statistically, it is not possibleto unequivocally conclude that team training in-terventions cause improvements in team func-tioning.
Second, as Rosenthal and DiMatteo (2001) sug-gested, every meta-analysis has some inherent biasattributable to the selection of inclusion/exclusion
criteria and the methods chosen to review the literature. To the extent that the studies in the data-base come from populations that are more homog-enous and nonrepresentative than is typical, themore likely it is that the external validity of thesefindings may be threatened. For example, the over-whelming majority of the studies included hereinwere conducted using college students acting asaction and performing teams in simulated en-vironments (e.g., aviation, military). Clearly, thisfocus has been influenced by years of generousfunding by the Department of Defense and var-ious military institutions (e.g., U.S. Army, U.S.Navy). Although these studies have obvious meritin advancing the science of this area, they often donot capture the unique dynamics and context inwhich other teams perform.
Finally, in our attempt to drill down and gaugerelationships at more finite levels, we unfortunatelyleft many of our subgroups lacking a robust num-ber of effect sizes to analyze. This was both a nec-essary evil and a cogent assessment of the currentstate of research on teams. It was necessary becausewe deemed it important to provide more prescrip-tive findings than simply saying, “team trainingworks to improve team outcomes.” The goal of ad-dressing when and where it works best is an im-portant undertaking. This research takes an initialstep in that direction, but it is worth repeating thatmore and better evaluations of the impact of teamtraining in organizations are needed.
In addition, we ask researchers and practition-ers, in any domain, to consider the data that theypresent when reporting their findings. Whereas p values and percentage improvement provide anindication of the impact of team training, theyleave the reader with no true measure of the mag-nitude with which team training improved desiredoutcomes. We call for researchers (and editors ofjournals) to report (and to demand) informativestatistics, such as the group means and standarddeviations, rather than solely p values or percent-ages. Researchers need to examine diverse sam-ples and settings and to elicit more published data,which would allow for additional meta-analysesin the future.
Recommendations for Future Research
The results of the present meta-analysis sug-gest several avenues for future research. First,most of the articles contained in the database
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 929
focused on teams working jointly on a task in thesame location. However, many organizations haveoperations in multiple countries. In multinationalorganizations, globally distributed teams are oftenresponsible for making and implementing impor-tant decisions. Research is needed that examineswhether team training improves team outcomesin distributed teams. Distributed teams have toresolve the same problems that other teams en-counter; however, these challenges are often morepronounced when team members are distributedacross temporal and geographic boundaries (Stagl,Salas, Rosen, et al., 2007). This type of researchwould provide useful information to managers andhuman resource professionals regarding the in-creasing use of globally distributed teams.
More than a decade ago, Stout, Salas, andFowlkes (1997) noted, “there are few systematicevaluations of the effectiveness of particular teamtraining efforts” (p. 169). Unfortunately, the largeamount of published literature that had to be ex-cluded from the meta-analytic database bears evi-dence that this critique still merits attention. Thatis, although the study of work teams has increasedin the past 2 decades (Cohen & Bailey,1997; Niel-sen, Sundstrom, & Halfhill, 2005; Sundstrom,McIntyre, Halfhill, & Richards, 2000), relativelyfew researchers properly report their findings. Inshort, there is a plethora of team training but littlesystematic evaluation and dissemination of results.Practitioners who facilitate team training inter-ventions and researchers who study teams andteam training should make increased efforts to col-laborate and empirically evaluate and publish theirfindings in peer-reviewed scientific outlets.
Another area meriting further attention involvesthe extent to which the team leader is incorporatedas a critical component of team training design.Research has demonstrated the beneficial influenceof team leader support, by way of leader briefings,for posttraining recall and the performance of train-ees (e.g., Mathieu et al., 2000; Smith-Jentsch, Salas,& Brannick, 2001). Following this further, a recentmeta-analysis found that person-focused and task-focused behaviors by the leader were related toteam effectiveness and team productivity (Burke,Stagl, Klein, et al., 2006). However, it was an un-fortunate fact that very few studies in the databaseincluded any discussion of the role of the leader inteam training. This is in contrast to the vast liter-ature on team leadership in general. Given theidentification of a sufficient number of empirical
articles on the topic, a moderator analysis inves-tigating the presence or absence of team leadersupport during training may be an informativeavenue of future research.
Also highlighted within the current findingswas the potentially important role of time in inter-vention delivery. This area has been recently high-lighted as one that needs more attention in theteam’s arena by several researchers. For example,with regard to developmental interventions, Hack-man and Wageman (2005) noted the importanceof timing. Specifically, they discussed how teamcoaching interventions should be made: “at timeswhen the team is ready for them and able to dealwith them” (p. 283). They suggested that coachingbe done at the beginning of the team’s task cyclefor effort-related interventions, near the midpointfor strategy-related interventions, and at the end ofa task cycle for interventions that address knowl-edge and skill (Hackman & Wageman, 2005). Wejoin with those authors and others in arguing forresearch that examines issues related to time anddevelopmental interventions – in our specific case,team training interventions.
Last, our meta-analysis did not consider theimpact of team composition on the effectivenessof team training. Most of the primary studies inthe database did not consider team composition.Thus, considering the dearth of reported data onteam composition variables in team training stud-ies, we make a call to researchers. Specifically, al-though much research has been done to evaluatethe moderating effects of individual characteristics(e.g., goal orientation, motivation to learn, consci-entiousness) on individual learning, how does ateam’s composition impact team learning?
Concluding Remarks
Team training works! The findings from this re-search are encouraging. In general, our results alsosuggest that training content, team membershipstability, and team size moderate the effectivenessof team training interventions. This study was thefirst to thoroughly and quantitatively summarizethe team training literature. We hope our findingsprovide valuable information to those stakehold-ers charged with making informed decisions re-garding the planning, design, development, anddelivery of team training interventions. We alsohope it inspires more and richer team training eval-uations from a variety of sectors.
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
930 December 2008 – Human Factors
ACKNOWLEDGMENTS
This work was supported by funding from theU.S. Army Research Institute for the Behavioraland Social Sciences (Contract W74V8H-04-C-0025). The views, opinions, and/or findings con-tained in this paper are those of the authors andshould not be construed as an official position, pol-icy, or decision of the Department of the Army orthe University of Central Florida.
We would like to thank Huy Le, John Mathieu,and Florian Jentsch for their insightful guidanceduring the course of this project. We would alsolike to acknowledge Shannon Scielzo and SamuelWooten for their assistance during the early cod-ing portions of the project.
REFERENCES
References marked with an asterisk indicate studiesincluded in the meta-analysis.
*Adelman, L., Christian, M., Gualtieri, J., & Bresnick, T. A. (1998). Ex-amining the effects of communication training and team compo-sition on the decision making of patriot air defense teams. IEEETransactions on Systems, Man, and Cybernetics – Part A: Systemsand Humans, 28, 729–741.
*Alexander, L. T., Kepner, C. H., & Tregoe, B. B. (1962). The effective-ness of knowledge of results in a military system-training program.Journal of Applied Psychology, 46, 202–211.
Arthur, W., Jr., Bennett, W., Jr., Edens, P. S., & Bell, S. T. (2003). Effec-tiveness of training in organizations: Ameta-analysis of design andevaluation features. Journal of Applied Psychology, 88, 234–245.
Arthur, W., Jr., Bennett, W., Jr., & Huffcutt, A. I. (2001). Conductingmeta-analysis using SAS. Mahwah, NJ: Erlbaum.
Baker, D. P., Salas, E., Barach, P., Battles, J., & King, H. (2007). The re-lation between teamwork and patient safety. In P. Carayon (Ed.),Handbook of human factors and ergonomics in health care andpatient safety (pp. 259–271). Mahwah, NJ: Erlbaum.
Banker, R. D., Field, J. M., Schroeder, R. G., & Sinha, K. K. (1996).Impact of work teams on manufacturing performance: A longitudi-nal field study. Academy of Management Journal, 39, 867–890.
*Banks, J. H., Jr., Hardy, G. D., Scott, T. D., Kress, G., & Word, L. E.(1977). Realtrain validation for rifle squads: Mission accomplish-ment (Research Rep. 1192, Army Project No. 2Q763743A773 and2Q763743A775, Tactical Team Performance). Alexandria, VA: U.S.Army Research Institute for the Behavioral and Social Sciences.
*Blickensderfer, E., Cannon-Bowers, J. A., & Salas, E. (1997, April).Training teams to self-correct: An empirical investigation. Paper pre-sented at the 12th Annual Meeting of the Society for Industrial andOrganizational Psychology, St. Louis, MO.
Bobko, P., & Stone-Romero, E. L. (1998). Meta-analysis may be anotheruseful tool, but it is not a panacea. Research in Personnel and HumanResources Management, 16, 359–397.
Boehm-Davis, D. A., Holt, R. W., & Seamster, T. L. (2001). Airline re-source management programs. In E. Salas, C. A. Bowers, & E. Edens(Eds.), Improving teamwork in organizations (pp.191–215). Mahwah,NJ: Erlbaum.
*Brannick, M. T., Prince, C., & Salas, E. (2005). Can PC-based systemsenhance teamwork in the cockpit? International Journal of AviationPsychology, 15, 173–187.
*Brown, T. C. (2003). The effect of verbal self-guidance training on col-lective efficacy and team performance. Personnel Psychology, 56,935–964.
Burke, C. S., Stagl, K. C., Klein, C., Goodwin, G. F., Salas, E., & Halpin,S. M. (2006). What type of leadership behaviors are functional inteams? A meta-analysis. Leadership Quarterly, 17, 288–307.
Burke, C. S., Stagl, K. C., Salas, E., Pierce, L., & Kendall, D. (2006).Understanding team adaptation: A conceptual analysis and model.Journal of Applied Psychology, 91, 1189–1207.
*Buzaglo, G., & Wheelan, S. A. (1999). Facilitating work team effec-tiveness: Case studies from Central America. Small Group Research,30, 108–129.
Campbell, J. P., & Kuncel, N. R. (2001). Individual and team training. InN. Anderson, D. S. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.),Handbook of work and organizational psychology (pp. 278–312).London: Blackwell.
*Cannon-Bowers, J. A., Salas, E., Blickensderfer, E., & Bowers, C. A.(1998). The impact of cross-training and workload on team func-tioning: Areplication and extension of initial findings. Human Fac-tors, 40, 92–101.
Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Volpe, C. E.(1995). Defining team competencies and establishing team train-ing requirements. In R. Guzzo, E. Salas, & associates (Eds.), Teameffectiveness and decision making in organizations (pp. 333–380).San Francisco: Jossey-Bass.
*Cobb, M. G. (2000). The impact of environmental complexity and teamtraining on team processes and performance in multiteam environ-ments. Dissertation Abstracts International, 60, 4281–4411.
Cohen, S. G., & Bailey, D. (1997). What makes teams work: Groupeffectiveness research from the shop floor to the executive suite?Journal of Management, 23, 239–290.
Cohen, S. G., & Ledford, G. E. (1994). The effectiveness of self-managing teams: Aquasi-experiment. Human Relations, 47, 13–43.
*Cooke, N. J., Cannon-Bowers, J. A., Kiekel, P. A., Rivera, K., Stout,R. J., & Salas, E. (2000). Improving teams’ interpersonal knowl-edge through cross-training. In Proceedings of the XIVth TriennialCongress of the International Ergonomics Association and 44thAnnual Meeting of the Human Factors and Ergonomics Society (pp.2.390–2.393). Santa Monica, CA: Human Factors and ErgonomicsSociety.
*Cooke, N. J., Kiekel, P. A., & Helm, E. E. (2001). Measuring teamknowledge during skill acquisition of a complex task. InternationalJournal of Cognitive Ergonomics, 5, 297–315.
*Cooke, N. J., Kiekel, P. A., Salas, E., Stout, R., Bowers, C., & Cannon-Bowers, J. (2003). Measuring team knowledge: A window to thecognitive underpinnings of team performance. Group Dynamics:Theory, Research, and Practice, 7, 179–199.
Curral, L. A., Forrester, R. H., Dawson, J. F., & West, M. A. (2001). It’swhat you do and the way that you do it: Team task, team size, andinnovation-related group processes. European Journal of Work andOrganizational Psychology, 10, 187–204.
Davies, J. M. (2001). Medical applications of crew resource manage-ment. In E. Salas, C. A. Bowers, & E. Edens (Eds.), Improving team-work in organizations (pp. 265–281). Mahwah, NJ: Erlbaum.
De Dreu, C. K. W. (2007). Cooperative outcome interdependence, taskreflexivity, and team effectiveness: A motivated information pro-cessing perspective. Journal of Applied Psychology, 92, 628–638.
Denson, R. W. (1981). Team training: Literature review and annotatedbibliography (Final Rep. AFHRL-TR-80-40). Brooks Air ForceBase, TX: HQ Air Force Human Resources Laboratory.
Devine, D. J., Clayton, L. D., Philips, J. L., Dunford, B. B., & Melner,S. B. (1999). Teams in organizations: Prevalence, characteristics, andeffectiveness. Small Group Research, 30, 678–711.
Driskell, J. E., Salas, E., & Johnston, J. (1995, April). Is stress traininggeneralizable to novel settings? Paper presented at the 10th AnnualConference of the Society for Industrial and Organizational Psychol-ogy, Orlando, FL.
*Duncan, P. C., Rouse, W. B., Johnston, J. H., Cannon-Bowers, J. A.,Salas, E., & Burns, J. J. (1996). Training teams working in complexsystems: A mental model–based approach. In W. B. Rouse (Ed.),Human/technology interaction in complex systems (Vol. 8, pp.173–231). Greenwich, CT: JAI Press.
Dyer, J. L. (1984). Team research and team training: A state-of-the-artreview. In F. A. Muckler (Ed.), Human factors review (pp. 285–323).Santa Monica, CA: Human Factors and Ergonomics Society.
*Ellis, A. P. J., Bell, B. S., Ployhart, R. E., Hollenbeck, J. R., & Ilgen,D. R. (2005). An evaluation of generic teamwork skills training withaction teams: Effects on cognitive and skill-based outcomes. Person-nel Psychology, 58, 641–672.
*Entin, E. B., Entin, E. E., MacMillan, J., & Serfaty, D. (1993). Struc-turing and training high-reliability teams (AD No. ADA302385).Burlington, MA: Alphatech.
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 931
*Entin, E. E., & Serfaty, D. (1999). Adaptive team coordination. HumanFactors, 41, 312–325.
*Entin, E. E., Serfaty, D., & Deckert, J. C. (1994). Team adaptation andcoordination training (TR-648-1). Burlington, MA: Alphatech.
Field, A. P. (2001). Meta-analysis of correlation coefficients: A MonteCarlo comparison of fixed- and random-effects methods. Psycho-logical Methods, 6, 161–180.
Field, A. P. (2005). Is the meta-analysis of correlation coefficients accu-rate when population correlations vary? Psychological Methods,10, 444–467.
Flin, R. H. (1995). Crew resource management for teams in the offshoreoil industry. Journal of European Industrial Training, 19, 23–27.
Flin, R. H., & O’Connor, P. (2001). Applying crew resource manage-ment on offshore oil platforms. In E. Salas, C. A. Bowers, & E. Edens(Eds.), Improving teamwork in organizations (pp. 217–233).Mahwah, NJ: Erlbaum.
Foushee, H. C. (1984). Dyads and triads at 35,000 feet: Factors affectinggroup process and aircrew performance. American Psychologist, 39,885–893.
*Freeman, J. T., Cohen, M. S., & Thompson, B. (1998). Effects of deci-sion support technology and training on tactical decision making.In Proceedings of the 1998 Command and Control Research &Technology Symposium [CD-ROM]. McLean, VA: Command andControl Research Program.
Gagné R. M. (1962). Military training and principles of learning.American Psychologist, 17, 83–91.
*Ganster, D. C., Williams, S., & Poppler, P. (1991). Does training in prob-lem solving improve the quality of group decisions? Journal ofApplied Psychology, 76, 479–483.
George, G. C., Hannibal, M. J., & Hirsch, W. L. (2004). Building thebrand through people. WorldatWork Journal, 13(1), 39–45.
Glass, B. (1955). News and notes. Science, 21, 583–596.Glass, G. V. (1976). Primary, secondary, and meta-analysis of research.
Educational Researcher, 5, 3–8.Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social
research. Beverly Hills, CA: Sage.*Glickman, A. S., Zimmer, S., Montero, R. C., Guerette, P. J., Campbell,
W. J., Morgan, B. B., et al. (1987). The evolution of teamwork skills:An empirical assessment with implications for training (Tech. Rep.Contract No. N00014-86-K-0732). Ft. Belvoir, VA: Defense Tech-nical Information Center.
Goldstein, I. L., & Ford, J. K. (2002). Training in organizations: Needsassessment, development, and evaluation (4th ed.). Belmont, CA:Wadsworth.
Gooding, R. Z., & Wagner, J. A., III. (1985). A meta-analytic review ofthe relationship between size and performance: The productivity andefficiency of organizations and their subunits. Administrative ScienceQuarterly, 30, 462–481.
*Green, L. R. (1994). The effectiveness of tactical adaptation and coor-dination training on team performance in tactical scenarios. Un-published master’s thesis, Naval Postgraduate School, Monterey,California.
Gully, S. M., Devine, D. J., & Whitney, D. J. (1995). A meta-analysisof cohesion and performance: Effects of levels of analysis and taskinterdependence. Small Group Research, 26, 497–520.
Guzzo, R. A., & Shea, G. P. (1992). Group performance and intergrouprelations in organizations. In M. D. Dunnette & L. M. Hough (Eds.),Handbook of industrial and organizational psychology (Vol. 3, pp.269–313). Palo Alto, CA: Consulting Psychologists Press.
Hackman, J. R. (1983). A normative model of work team effectiveness(Tech. Rep. No. 2). New Haven, CT: Yale University.
Hackman, J. R. (1987). The design of work teams. In J. Lorsch (Ed.),Handbook of organizational behavior (pp. 215–342). New York:Prentice Hall.
Hackman, J. R. (Ed.). (1990). Groups that work (and those that don’t):Creating conditions for effective teamwork. San Francisco: Jossey-Bass.
Hackman, J. R. (2002, August). New rules for team building. Optimize,pp. 50–62.
Hackman, J. R., & Vidmar, N. (1970). Effects of size and task type ongroup performance and member reactions. Sociometry, 33, 37–54.
Hackman, J. R., & Wageman, R. (2005). A theory of team coaching.Academy of Management Review, 30, 269–287.
*Heffner, T. S. (1997). The influence of team training methods on simu-lator performance. IEEE Transactions on Systems, Man, and Cyber-netics, 27, 4165–4170.
Helmreich, R. L., & Foushee, H. C. (1993). Why crew resource man-agement? Empirical and theoretical bases of human factors train-ing in aviation. In E. L. Weiner, B. G. Kanki, & R. L. Helmreich(Eds.), Cockpit resource management (pp. 3–45). San Diego, CA:Academic Press.
*Henry, L. J. (1997). The effect of group process training on team effec-tiveness. Unpublished doctoral dissertation, Texas Women’sUniversity, College of Health Sciences, Denton, TX.
Hollenbeck, J. R., DeRue, D. S., & Guzzo, R. (2004). Bridging the gapbetween I/O research and HR practice: Improving team composi-tion, team training, and team task design. Human Resource Manage-ment, 43, 353–366.
Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Cor-recting error and bias in research findings. Beverly Hills, CA: Sage.
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Cor-recting error and bias in research findings (2nd ed.). Newbury Park:Sage.
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis:Cumulating research findings across studies. Beverly Hills, CA:Sage.
Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct andindirect range restriction for meta-analysis method and findings.Journal of Applied Psychology, 91, 594–612.
*Ikomi, P. A., Boehm-Davis, D. A., Holt, R. W., & Incalcaterra, K. A.(1999). Jump seat observations of advance crew resource manage-ment (ACRM) effectiveness. In R. S. Jensen, B. Cox, J. D. Callister,& R. Lavis (Eds.), Proceedings of the 10th International Symposiumon Aviation Psychology (pp. 292–297). Columbus: Ohio StateUniversity.
Ilgen, D. R. (1999). Teams embedded in organizations: Some implica-tions. American Psychologist, 54, 129–139.
Indik, B. P. (1965). Operational size and member participation: Someempirical tests of alternative explanations. Human Relations, 18,339–350.
Judge, T. A., Cable, D. M., Colbert, A. E., & Rynes, S. L. (2007). Whatcauses a management article to be cited—Article, author, or journal?Academy of Management Journal, 50, 491–506.
Karau, S. J., & Williams, K. D. (1993). Social loafing: A meta-analyticreview and theoretical integration. Journal of Personality and SocialPsychology, 65, 681–706.
Kidwell, R. E., & Bennett, N. (1993). Employee propensity to withholdeffort: A conceptual model to intersect three avenues of research.Academy of Management Review, 18, 429–456.
Klein, C., DiazGranados, D., Salas, E., Le, H., Burke, C. S., & Lyons, R.(in press). The effect of team building interventions on team out-comes: An update and extension. Small Group Research.
Klein, C., Stagl, K. C., Salas, E., Parker, C., & Van Eynde, D. F. (2007).Returning to flight: Simulation-based training for the U.S. NationalAeronautics and Space Administration’s Space Shuttle MissionManagement Team. International Journal of Training and Develop-ment, 11, 132–138.
Kozlowski, S. W. J., & Bell, B. S. (2003). Work groups and teams in orga-nizations. In W. C. Borman, D. R. Ilgen, & R. J. Klimoski (Eds.),Comprehensive handbook of psychology: Industrial and organiza-tional psychology (Vol. 12, pp. 333–375). New York: Wiley.
Kozlowski, S. W. J., & Klein, J. K. (2000). A multi-level approach totheory and research in organizations: Contextual, temporal and emer-gent processes. In J. K. Klein & S. W. J. Kozlowski (Eds.), Multi-level theory, research, and methods in organizations: Foundations,extensions, and new directions (pp. 3–90). San Francisco: Jossey-Bass.
Kraiger, K. (1985). On learning from the past: A meta-analytic fable.Personnel Psychology, 38, 799–801.
Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive,skill-based, and affective theories of learning outcomes to newmethods of training evaluation. Journal of Applied Psychology, 78,311–328.
*Lampton, D. R., McDonald, D. P., Rodriguez, M. E., Cotton, J. E.,Morris, C. S., Parsons, J., et al. (2001). Instructional strategies fortraining teams in virtual environments (Tech. Rep. No. 1110).Alexandria, VA: U.S. Army Research Institute for the BehavioralSciences.
*Lassiter, D. L., Vaughn, J. S., Smaltz, V. E., Morgan, B. B., Jr., &Salas, E. (1990). A comparison of two types of training interven-tions on team communication performance. In Proceedings of theHuman Factors Society 34th Annual Meeting (pp. 1372–1376).Santa Monica, CA: Human Factors and Ergonomics Society.
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
932 December 2008 – Human Factors
Latané, B. (1981). The psychology of social impact. American Psychol-ogist, 36, 343–356.
*Le Blanc, P. M., Hox, J. J., Schaufeli, W. B., Taris, T. W., & Peeters,M. C. W. (2007). Take care! The evaluation of a team-based burnoutintervention program for oncology care providers. Journal of Ap-plied Psychology, 92, 213–227.
*Leedom, D. K., & Simon, R. (1995). Improving team coordination: Acase for behavior-based training. Military Psychology, 7, 109–122.
LePine, J. A., Piccolo, R. F., Jackson, C. L., Mathieu, J. E., & Saul, J. R.(2008). A meta-analysis of teamwork processes: Tests of a multi-dimensional model and relationships with team effectiveness criteria. Personnel Psychology, 61, 273–307.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thou-sand Oaks, CA: Sage.
*MacMillian, J., Entin, E. E., Entin, E. B., & Serfaty, D. (1994). Struc-turing and training high reliability teams (Year 2 Interim Tech. Rep.TR-632). Fort Rucker, AL: U.S. Army Research Institute/AviationResearch and Development Activity.
Magjuka, R. J., & Baldwin, T. T (1991). Team-based employee involve-ment programs: Effects of design and administration. Personnel Psy-chology, 44, 793–812.
Marks, M. A. (2006). The science of team effectiveness. PsychologicalScience in the Public Interest, 7(3), i.
Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). Atemporally basedframework and taxonomy of team processes. Academy of Manage-ment Review, 26, 356–376.
*Marks, M. A., Sabella, M. J., Burke, C. S., & Zaccaro, S. J. (2002). Theimpact of cross-training on team effectiveness. Journal of AppliedPsychology, 87, 3–13.
*Marks, M. A., Zaccaro, S. J., & Mathieu, J. E. (2000). Performance im-plications of leader briefings and team-interaction training for teamadaptation to novel environments. Journal of Applied Psychology,85, 971–986.
*Mathieu, J. E., Heffner, T. S., Goodwin, G. F., Salas, E., & Cannon-Bowers, J. A. (2000). The influence of shared mental models onteam process and performance. Journal of Applied Psychology, 85,273–283.
Mathieu, J. E., & Leonard, R. L. (1987). Applying utility concepts to atraining program in supervisory skills: A time-based approach.Academy of Management Journal, 30, 316–335.
McCall, M., & Bobko, P. (1990). Research methods and discovery inindustrial/organizational psychology. In M. Dunnette & L. Hough(Eds.), Handbook of industrial and organizational psychology (pp.381–418). Palo Alto, CA: Consulting Psychologists Press.
McGrath, J. E. (1984). Groups: Interaction and performance. Engle-wood Cliffs, NJ: Prentice Hall.
McIntyre, R. M., & Salas, E. (1995). Measuring and managing for teamperformance: Emerging principles from complex environments. InR. Guzzo & E. Salas (Eds.), Team effectiveness and decision mak-ing in organizations (pp. 149–203). San Francisco: Jossey-Bass.
*Minionis, D. P. (1995). Enhancing team performance in adverse con-ditions: The role of shared team mental models and team trainingon an interdependent task (Doctoral dissertation, George MasonUniversity, 1995). Dissertation Abstracts International, 56, 1139.
Mohammed, S., & Ringseis, E. (2001). Cognitive diversity and con-sensus in group decision making: The role of inputs, processes, andoutcomes. Organizational Behavior and Human Decision Processes,85, 310–335.
*Morey, J. C., Simon, R., Jay, G. D., Wears, R. L., Salisbury, M., Dukes,K. A., et al. (2002). Error reduction and performance improvementin the emergency department through formal teamwork training:Evaluation results of the MedTeams project (quality of care). HealthServices Research, 37, 1553–1582.
Mullen, B. (1987). Self-attention theory. In B. Mullen & G. R. Goethals(Eds.), Theories of group behavior (pp. 125–146). New York:Springer-Verlag.
Mullen, B. (1989). Advanced BASIC meta-analysis. Hillsdale, NJ:Erlbaum.
Mullen, B., Brown, R. J., & Smith, C. (1992). Ingroup bias as a functionof salience, relevance, and status: An integration. European Journalof Social Psychology, 22, 103–122.
Mullen, B., & Rosenthal, R. (1985). BASIC meta-analysis. Hillsdale,NJ: Erlbaum.
Mullen, B., Salas, E., & Driskell, J. E. (1989). Salience, motivation, andartifact as contributions to the relation between participation rateand leadership. Journal of Experimental Social Psychology, 25,545–559.
Mullen, B., Symons, C., Hu, L., & Salas, E. (1989). Group size, leader-ship behavior, and subordinate satisfaction. Journal of General Psy-chology, 116, 155–170.
*Naylor, J. C., & Briggs, G. E. (1965). Team-training effectiveness undervarious conditions. Journal of Applied Psychology, 49, 223–229.
Nielsen, J. M., Sundstrom, E. D., & Halfhill, T. R. (2005). Group dynam-ics and effectiveness: Five years of applied research. In S. A.Wheelan(Ed.), The handbook of group research and practice (pp. 285–311).Thousand Oaks, CA: Sage.
Nieva, V. F., Fleishman, E. A., & Rieck, A. (1978). Team dimensions:Their identity, their measurement and their relationships (ContractNo. DAHC 19-78-C-0001). Washington, DC: Advanced ResearchResources Organization.
Noe, R. A. (2002). Employee training and development. Boston:McGraw-Hill.
Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill.Orasanu, J., & Fischer, U. (1997). Finding decisions in natural environ-
ments: The view from the cockpit. In C. E. Zsambok & G. Klein(Eds.), Naturalistic decision making expertise: Research and appli-cations (pp. 343–357). Mahwah, NJ: Erlbaum.
Orasanu, J., & Salas, E. (1993). Team decision making in complex envi-ronment. In G. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok(Eds.), Decision making in action: Models and methods (pp.327–345). Norwood, NJ: Ablex.
Oser, R. L., Salas, E., Merket, D. C., & Bowers, C. A. (2001). Applyingresource management training in naval aviation: Amethodology andlessons learned. In E. Salas, C. A. Bowers, & E. Edens (Eds.), Im-proving teamwork in organizations (pp. 283–304). Mahwah, NJ:Erlbaum.
*Parsons, K. L. (1981). The effects of load sharing system training uponteam performance (Doctoral dissertation, Michigan State University,1980). Dissertation Abstracts International, 42(6-A), 2876–2877.
*Pearsall, M. J., Ellis, A. P. J., & West, B. J. (2004, August). Reducingthe negative effects of stress in team contexts: The impact of cross-training. Paper presented at the Annual Conference of the Academyof Management, New Orleans, LA.
Pfeffer, J., & Sutton, R. I. (2000). The knowing-doing gap: How smartcompanies turn knowledge into action. Boston: Harvard BusinessSchool Press.
*Prichard, J. S., & Ashleigh, M. J. (2007). The effects of team-skills train-ing on transactive memory and performance. Small Group Research,38, 696–726.
*Rapp, T. L., & Mathieu, J. E. (2007). Evaluating an individually self-administered generic teamwork skills training program across timeand levels. Small Group Research, 38, 532–555.
*Rollins, M. L., & Angelcyk, J. J. (1995). A descriptive study of crewresource management attitude change. In N. Johnston, R. Fuller, &N. McDonald (Eds.), Aviation psychology: Training and selection:Proceedings of the 21st Conference of the European Association forAviation Psychology (Vol. 2, pp. 45–50). Aldershot, UK: AveburyAviation.
Rosenthal, R. (1991). Meta-analytic procedures for social research.Newbury Park, CA: Sage.
Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: Recent devel-opments in quantitative methods for literature reviews. AnnualReview of Psychology, 52, 59–82.
Saavedra, R., Earley, P. C., & Van Dyne, L. (1993). Complex interde-pendence in task-performing groups. Journal of Applied Psychology,78, 61–72.
Salas, E., Burke, C. S., Bowers, C. A., & Wilson, K. A. (2001). Teamtraining in the skies: Does crew resource management (CRM) train-ing work? Human Factors, 43, 641–674.
Salas, E., Burke, C. S., & Cannon-Bowers, J. A. (2002). What we knowabout designing and delivering team training: Tips and guidelines.In K. Kraiger (Ed.), Creating, implementing, and managing effectivetraining and development: State-of-the-art lessons for practice (pp.234–259). San Francisco: Jossey-Bass.
Salas, E., & Cannon-Bowers, J. A. (1997). Methods, tools, and strategiesfor team training. In M. A. Quinones & A. Ehrenstein (Eds.), Train-ing for a rapidly changing workplace (pp. 249–279). Washington,DC: American Psychological Association.
Salas, E., & Cannon-Bowers, J. A. (2000). The anatomy of team train-ing. In S. Tobias & J. D. Fletcher (Eds.), Training and retraining: Ahandbook for business, industry, government, and the military (pp.312–335). New York: Macmillan Reference.
Salas, E., Cannon-Bowers, J. A., & Johnston, J. H. (1997). How can youturn a team of experts into an expert team? Emerging training
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
TEAM TRAINING EFFECTIVENESS 933
strategies. In C. E. Zsambok & G. Klein (Eds.), Naturalistic decisionmaking (pp. 359–370). Mahwah, NJ: Erlbaum.
Salas, E., DiazGranados, D., Weaver, S. J., & King, H. (2008). Does teamtraining work? Principles for health care. Academic EmergencyMedicine, 15, 1002–1009.
Salas, E., Dickinson, T. L., Converse, S. A., & Tannenbaum, S. I. (1992).Toward an understanding of team performance and training. In R. W.Swezey & E. Salas (Eds.), Teams: Their training and performance(pp. 3–29). Norwood, NJ: Ablex.
Salas, E., Nichols, D. R., & Driskell, J. E. (2007). Testing three teamtraining strategies in intact teams: A meta-analysis. Small GroupResearch, 38, 471–488.
Salas, E., Rozell, D., Mullen, B., & Driskell, J. E. (1999). The effect ofteam building on performance: An integration. Small Group Re-search, 30, 309–329.
Salas, E., Stagl, K. C., & Burke, C. S. (2004). 25 years of team effec-tiveness in organizations: Research themes and emerging needs. InC. L. Cooper & I. T. Robertson (Eds.), International review of indus-trial and organizational psychology (pp. 47–91). New York: Wiley.
Salas, E., Stagl, K. C., Burke, C. S., & Goodwin, G. F. (2007). Fosteringteam effectiveness in organizations: Toward an integrative theoret-ical framework of team performance. In R. A. Dienstbier (Series Ed.)& B. Shuart, W. Spaulding, & J. Poland (Vol. Eds.), Modeling com-plex systems. Vol. 52: Current theory and research in motivation(pp. 185–243). Lincoln: University of Nebraska Press.
Salas, E., Wilson, K. A., Burke, C. S., & Wightman, D. (2006). Does crewresource management training work? An update, an extension, andsome critical needs. Human Factors, 48, 392–412.
Schmidt, F. L., & Le, H. (2005). Hunter-Schmidt Meta-Analysis Pro-grams (Version 1.1) [Computer software]. Iowa City: University ofIowa, Department of Management & Organizations.
*Schreiber, M. A., Holcomb, J. B., Conaway, C. W., Campbell, K. D.,Wall, M., & Mattox, K. L. (2002). Military trauma training per-formed in a civilian trauma center. Journal of Surgical Research,104, 8–14.
*Serfaty, D., Entin, E. E., & Johnston, J. H. (1998). Team coordinationtraining. In J. A. Cannon-Bowers & E. Salas (Eds.), Making deci-sions under stress: Implications for individual and team training (pp.221–245). Washington, DC: American Psychological Association.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimentaland quasi-experimental designs for generalized causal inference.Boston: Houghton Mifflin.
*Shapiro, J. J., Morey, J. C., Small, S. D., Langford, V., Kaylor, C. J.,Jagminas, L., et al. (2004). Simulation based teamwork training foremergency department staff: Does it improve clinical team perfor-mance when added to an existing didactic teamwork curriculum?Quality and Safety in Health Care, 13, 417–421.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses inassessing rater reliability. Psychological Bulletin, 86, 420–428.
*Siegel, A. I., & Federman, P. J. (1973). Communications content train-ing as an ingredient in effective team performance. Ergonomics, 16,403–416.
Smith-Jentsch, K. A., Salas, E., & Brannick, M. T. (2001). To transfer ornot to transfer? Investigating the combined effects of trainee char-acteristics, team leader support, and team climate. Journal of AppliedPsychology, 86, 279–292.
Stagl, K. C., Salas, E., & Fiore, S. M. (2007). Best practices in cross train-ing teams. In D. A. Nembhard (Ed.), Workforce cross training hand-book (pp. 156–175). Boca Raton, FL: CRC Press.
Stagl, K. C., Salas, E., Rosen, M. A., Priest, H. A., Burke, C. S.,Goodwin, G. F., et al. (2007). Distributed team performance: Amulti-level review of distribution, demography, and decision making.Research in Multi-Level Issues, 6, 11–58.
Steiner, I. D. (1972). Group process and productivity. Orlando, FL:Academic Press.
Stock, W. A. (1994). Systematic coding for research synthesis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research syn-thesis (pp. 97–110). New York: Russell Sage Foundation.
*Stout, R. J., Salas, E., & Fowlkes, J. E. (1997). Enhancing teamworkin complex environments through team training. Group Dynamics:Theory, Research, and Practice, 1, 169–182.
Sundstrom, E., McIntyre, M., Halfhill, T., & Richards, H. (2000). Workgroups: From the Hawthorne studies to work teams of the 1990sand beyond. Group Dynamics, 4, 44–67.
Tannenbaum, S. (2002). A strategic view of organizational training andlearning. In K. Kraiger (Ed.), Creating, implementing, and manag-
ing effective training and development (pp. 10–52). San Francisco:Jossey-Bass.
Tannenbaum, S. I., Beard, R. L., & Salas, E. (1992). Team building andits influence on team effectiveness: An examination of conceptualand empirical developments. In K. Kelley (Ed.), Issues, theory, andresearch in industrial/organizational psychology (pp. 117–153).Amsterdam: Elsevier.
Tuckman, Bruce W. (1965). Developmental sequence in small groups.Psychological Bulletin, 63, 384–399.
*Volpe, C. E., Cannon-Bowers, J. A., Salas, E., & Spector, P. E. (1996).The impact of cross-training on team functioning: An empiricalinvestigation. Human Factors, 38, 87–100.
Whitener, E. M. (1990). Confusion of confidence intervals and credi-bility intervals in meta-analysis. Journal of Applied Psychology,75, 315–321.
Wiener, E. K., Kanki, B. G., & Helmreich, R. L. (Eds.). (1993). Cockpitresource management (pp. 479–501). San Diego, CA: AcademicPress.
Yetton, P. W., & Bottger, P. C. (1982). Individual versus group problemsolving: An empirical test of best-member strategy. OrganizationalBehavior and Human Performance, 29, 307–321.
Eduardo Salas is a Pegasus Professor and universitytrustee chair in the Department of Psychology and In-stitute for Simulation & Training at the University ofCentral Florida, Orlando. He received a Ph.D. in indus-trial/organizational psychology from Old DominionUniversity in 1984.
Deborah DiazGranados is a doctoral candidate andresearch associate in the Department of Psychology andInstitute for Simulation & Training at the University ofCentral Florida, Orlando, where she received her M.S.in industrial/organizational psychology in 2005.
Cameron Klein is a doctoral candidate in the Depart-ment of Psychology at the University of Central Florida,Orlando, and a consultant at Kenexa in Lincoln,Nebraska. He received his M.S. in industrial/organiza-tion psychology from the University of Central Floridain 2004.
C. Shawn Burke is a research scientist in the Institute forSimulation & Training, University of Central Florida,Orlando. She received her Ph.D. in industrial/organiza-tional psychology from George Mason University in2000.
Kevin C. Stagl is a senior talent advisor at Talent Thresh-old LLC, Orlando, and an adjunct professor at the Uni-versity of Central Florida, where he received his Ph.D.in industrial and organizational psychology in 2006.
Gerald F. Goodwin is a research psychologist in theBasic Research Unit at the Army Research Institute,Arlington, Virginia. He received his Ph.D. in industrial/organizational psychology from Pennsylvania StateUniversity in 1999.
Stanley M. Halpin is chief of the Fort LeavenworthResearch Unit, the leader development unit of the ArmyResearch Institute, Fort Leavenworth, Kansas. Hereceived his Ph.D. in social psychology from PurdueUniversity in 1970.
Date received: June 18, 2008Date accepted: November 25, 2008
at SUFFOLK UNIV on February 24, 2014hfs.sagepub.comDownloaded from
