AD-A263 407DTICELECTE

APR 27 1993Research Product 93-03 C

A Review and Annotated Bibliography

of Literature Relevant to ArmorSkill Retention

February 1993

Fort Knox Field Unit

Training Systems Research Division

U.S. Army Research Institute for the Behavioral and Social Sciences

Approved for public release; distribution is unlimited. 93-0788 1

93 4 14 138

U.S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency Under the Jurisdiction

of the Deputy Chief of Staff for Personnel

EDGAR M. JOHNSON

Acting Director

Technical review by

Gary S. ElliottRobert A. Wisher

Accesion For

NTIS CRA&I OTIC TABUadr'oO'~ced

By

uistribution I

Avjildibtlty Codes

IAvail ariGjorDIM specidl

NOTICES

FINAL DISPOSITION: This Research Product may be destroyed when it is no longer needed.Please do not return it to the U.S. Army Research Institute for the Behavioral and Social Sciences.

NOTE: This Research Product is not to be construed as an official Department of the Armyposition, unless so designated by other authorized documents.

iorm - ,", ;

REPORT DOCUMENTATION PAGEOunl.c nu~ c~C 'O r th;% rte acl etf , ''ac' 5 ST- at9 ec t,- 1 c--e t tre .'re ,V 'rs .- -amenrI'? anc 'T3,nc) Ire -c313 neaa. ndr e lo'e n rc 'e.. - e-irIr.C-'a,- Sena .m.,. I.o r.;t s.-c

ccetn '' I ' C .lls ~ ~ ,n ..'0I Cen _ . .,rf2Qn< Q;:., ".9 ,. , 3c~eC r.3 .'. ,,

5-- -. !'.a. C4. t'~ c- 20- i3 J*8 . . V e~;e ~ ~ e0,- e~: C ~ 1

1. AGENCY USE ONLY (Leave b/ank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED1993, February Final Jan 92 - Nov 92

4. TITLE AND SUBTITLE 5. FUNDING NUMBERSA Review and Annotated Bibliography of Literature 62785ARelevant to Armor Skill Retention 790

32046. AUTHOR(S) HO1Rowatt, Wade C. (University of Louisville); andShlechter, Theodore M. (ART)

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATIONU.S. Army Research Institute for the Behavioral and REPORT NUMBER

Social Sciences ART Research ProductATTN: PERI-IK 93-035001 Eisenhower AvenueAlexandria, VA 22333-56009. SPONSORING/ MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING i MONITORING

AGENCY REPORT NUMBER

11. SUPPLEMENTARY NOTESWade C. Rowatt is a graduate s:udent at the University of Louisville, working as aconsortium fellow at the U.S. Army Research Institute's Fort Knox Field Unit.

12a. DISTRIBUTION / AVAILABILITY STATEMENT 12b. DISTRIRUTION CODEApproved for public release;distribution is unlimited.

13. ABSTRACT (Maximum 200 words)This research product addresses the progress of research on armor skill

retention by reviewing 74 pertinent works. These works include book chapters,empirical studies, review and theoretical articles, and technical and researchreports. The main factors affecting military skill retention are categorized anddiscussed according to training, task, and individual difference factors. Thediscussion of factors indicates that predicting armor skill retention is difficultbecause (a) little is actually known about armor skill retention, and (b) inter-active effects were found for most of the previously identified elements of skillretention. Potential areas of research are discussed.

14. SUBJECT TERMS 15. NUMBER OF PAGESArmor retention 53Training and sirlilati- 16. PRICE CODE

Skill retention17. SECURITY CLASSIFICATION 18. SECURITY CLASSIFICATION 19. SECURITY CLASSIFICATION 20. LIMITATION OF ABSTRACT

OF REPORT OF THIS PAGE OF ABSTRACTUnclassified Unclassified Unclassified Unlimi ted

NSN 7540-01-280-5500 Standard Form 298 (iev 2-89)

"9 ANSI %to /13 IS

Research Product 93-03

A Review and Annotated Bibliography of LiteratureRelevant to Armor Skill Retention

Wade C. RowattUniversity of Louisville

Theodore M. ShlechterU.S. Army Research Institute

Field Unit at Fort Knox, KentuckyBarbara A. Black, Chief

Training Systems Research DivisionJack H. Hiller, Director

U.S. Army Research Institute for the Behavioral and Social Sciences5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600

Office, Deputy Chief of Staff for PersonnelDepartment of the Army

February 1993

Army Project Number Human Performance Effectiveness2Q162785A790 and Simulation

Approved for public release; distribution is unlimited.

III

FOREWORD

The U.S. Army Research Institute for the Behavioral andSocial Sciences (ARI) is dedicated to helping U.S. Armycommanders and trainers understand the most efficient proceduresfor maintaining a proficient force. This goal has becomeincreasingly important as training resources have become morelimited. Army commanders and trainers need to know the factorsthat affect soldiers' retention of job-related skills. Thisresearch product provides such information in relationship toarmor skills.

The work described in this research product is a part of theARI Fort Knox Field Unit's work program under the task entitled"Training Requirements for Combined Arms Simulators." This taskis supported by a memorandum of agreement entitled "The Effectsof Simulators and Other Research on Training Readiness," signed16 January 1989. Parties to this agreement are the U.S. ArmyTraining and Doctrine Command, the U.S. Army Armor Center at FortKnox, the U.S. Army Materiel Command; and ARI.

The information contained in this research product has beenmade available to commanders, trainers, and civilian trainingdevelopers at the U.S. Army Armor Center and School at Fort Knox.It has also been made available to armor unit trainers.

EDGAR M. JOHNSONActing Director

v

A REVIEW AND ANNOTATED BIBLIOGRAPHY OF LITERATURE RELEVANT TOARMOR SKILL RETENTION

CONTENTS

Page

INTRODUCTION ............. ....................... 1

METHOD OF THE LITERATURE SEARCH ........ .............. 1

SUMMARY OF FINDINGS ........... ................... 3

Training Factors ........... ................... 3Task Factors ............ .................... 9Individual Difference Factors ....... ............. 9

FINAL COMMENTS ......... ... ...................... 10

Implications for Armor Training ...... .. ......... 10Implications for Future Research ...... ........... ii

APPENDIX A. ANNOTATED BIBLIOGRAPHY: ARMOR SKILLRETENTION LITERATURE .... ............ A-i

B. ANNOTATED BIBLIOGRAPHY: REVIEWS OFTHE SKILL RETENTION LITERATURE ....... . B-i

C. THE PERTINENT MILITARY AND CIVILIAN

SKILL RETENTION LITERATURE .. ......... C-i

D. MISCELLANEOUS REFERENCES ... ......... . D-I

E. CORPORATE BIBLIOGRAPHIES .. .......... E-I

F. DOCUMENT REPRODUCTION SERVICES ....... . F-i

LIST OF FIGURES

Figure I. Major factors affecting skill retention . . . . 4

2. Summary of armor skill retention research . . . 5

vii

A Review and Annotated Bibliography of Literature Relevant to

Armor Skill Retention

Introduction

Military training has generated an enormous body ofliterature. A subset of this literature addresses issuesrelevant to skill retention, broadly defined as the maintenanceor sustainment of learned behaviors without practice (Schendel &Hagman, 1991). A search of computerized databases and existingbibliographies revealed hundreds of documents that have beenpublished on skill retention and the inverse, skill decay.Prophet (1976), for example, noted the existence of over 120articles on this topic. These documents provide a plethora ofinformation about skill retention, yet many questions and gaps inour knowledge remain.

One of the most common issues running throughout thisliterature involves predicting the rate of decline for specificindividual skills (see Rose, Radtke, Shettel, & Hagman 1985;Rigg, 1983; and, Wisher & Sabol, 1990). Rose et al., forexample, developed a method for predicting retention ofprocedural and perceptual-motor military skills over no-practiceintervals of up to 1 year. This prediction model was based onsuch factors as complexity of the task to be remembered, lengthof the retention interval, and presence of job aids.

Questions remain, however, regarding the validity of anymodel for predicting the rate of armor skill decay. In addition,a large portion of the literature focuses on individual skillretention rather than collective (team or crew) retention. Thisproduct attempts to address these issues by reviewing thepertinent material on armor skill retention. It identifies whatis not known about this topic and provides recommendations forareas of future research. It also includes an annotatedbibliography of the armor skill retention literature (AppendixA), the existing literature reviews of skill retention (AppendixB), and the pertinent military and civilian skill retentionliterature (Appendix C). This document should help armorcommanders, trainers, and researchers understand efficientprocedures for maintaining a proficient, trained force astraining resources become more limited and Army downsizingincreases the reliance on Reserve Components for futuremobilization.

Method of the Literature Search

To accomplish the goals set forth in the previous section, asystematic search of the armor skill retention literature wasconducted. This search entailed locating articles through theuse of (a) the Defense Technical Information Center (DTIC) database; (b) the Army Research Institute (ARI) corporate bibliographies

1

(1981, 1986, 1990, 1992); (c) the Human Resources ResearchOrganization's (HumRRO's) corporate bibliographies (1971, 1977);(d) reference sections of the existing literature, and (e)knowledgeable individuals in the armor training field.

Fourteen research efforts, two of which were literaturereviews, were found that directly addressed armor skillretention. Ten of these works were completed before 1984. Mostof the recent works, however, have limited value for armorcommanders and trainers. Lampton, Bliss, & Meert (1992), forinstance, used college students as their subject population.

It was decided to broaden the scope of this review toinclude pertinent skill retention research from other militaryand civilian sources. Computerized literature searches ofseveral data bases (DTIC, Educational Resources InformationCenter-ERIC; and PsychLIT) were conducted to locate additionalsources on skill retention. These articles were also found by(a) searching the corporate bibliographies of ARI, HumRRO, andthe Air Force Human Resources Laboratory (1971); (b) searchingthe on-line card catalogue system at the University ofLouisville; (c) going through proceedings of professionalconferences; (d) examining the reference sections of the existingliterature; and (d) contacting researchers from universities andmilitary laboratories.

Over 80 references were found to be potentially applicableto military skill retention. Seventy-four of these referenceswere included in the annotated bibliography. As stated, 15 ofthese articles directly addressed armor skill retention. Elevenof these articles were existing skill retention literaturereviews. The remaining 48 articles consisted of research effortsfrom the military and civilian research literature which wererelevant to armor skill retention. Of these 48 references, 11were not acquired by the present authors; however, existingsources provided sufficient information to provide an abstract ofthe material. For example, Naylor and Briggs' (1961) articlecould not be obtained from DTIC, but an existing in-housedescription was available. A list of potentially useful articleswhich either did not directly deal with skill retention or werenot procurable by the time of compilation is also attached.

Annotated bibliographies of the armor skill literature, theliterature reviews, and the non-armor skill retention literatureare found in Appendices A-C, respectively. A list of potentiallyuseful articles which either did not directly deal with skillretention or were not procurable by the time of compilation canbe found in Appendix D. Finally, Appendices E and F contain alisting of the corporate bibliographies and document reproductionservices which were used to conduct portions of this literaturesearch.

2

Summary of Findings

Analyses of the 11 different review articles by the presentauthors indicated the following salient variables in thisliterature: (a) original learning; (b) over tr .iing; (c) lengthof the retention interval; (d) refre-her training; (e) spacing ofpractice; (f) functional similarity; (g) knowledge of results(KR); (h) mnemonics; (i) continuous vs. procedural tasks; (j)task complexity; (k. task organization; (1) individual ability;and (m) prior experience. These variables were grouped into thefollowing factors of skill retention (see Figure 1):

1. training--factors associated with instructional

strategies.

2. task--inherent attributes of the retention task.

3. individual differences--factors associated withthe student.

This taxonomy, similar to one employed by Rose et al. (1981)and Schendel, Shields, & Katz (1978), is described in detail inthe next few pages. Results of articles which were not includedin previous reviews were also incorporated in this discussion.

The results of the armor skill retention research is furtherdelineated in Figure 2. Three main points regarding thisresearch literature can be extracted from this figure. One,nearly all of this research was done vis-a-vis the M60A1 tank.Two, all of these studies dealt with examining soldiers'retention of procedural skills with a majority of them dealingwith gunnery procedures. Three, the retention interval of thesestudies varied from a few weeks to a few months.

Training Factors

Level of original learning: As shown in Figures 1 and 2, apervasive finding in this literature is that the level oforiginal learning--level of proficiency prior to the retentioninterval--is an excellent predictor of retention performance. Infact, nearly all of these reviews have indicated that thisvariable is the single best determinant of skill retention withthe relationship between original learning level and retentionremaining highly positive and stable for an indefinite period oftime. Fleishman and Parker (1962), as cited in Schendel et al.,(1991), for example, found correlations ranging from .80 to .98between learners' initial proficiency on a three-dimensionalflight-control task and later retention, which ranged from 1month to 2 years.

3

a + +

IL -

C1

cncc

0

0

CL

C.0a

L-

0 a

.5

020

Z-0; + 0 . . 4

IF0CJ 6

.jM

-4 0)

a~CDSC

Source Tank Task(s) Retention Predictors of skill Predictors ofinterval retention skill decay

Army Trig. M60A1 MOS skills 2-25 wks Discrete tasks,Study (1978) Low ability,

Time

Golberg, M60A1 Gunnery 1-5 wks High mastery criterion Timeet al.(1981)

Gunmery,Knerr, et at. 1460A1 Comma, 4 wks High levels of mastery Discrete tasks(1984) Equipment

Lampton, M1AI Gunnery 10 mks Increased level ofet al(992) mastery criterion

Mach inegunMorrison 1460AI gunnery & Not Task structure Time(1982,1984) Comma stated

orrison & M6OAI Machinegun 6 wks Spacing practice and TimeBessemer gunnery video demonstrations(1981) did not aid retention

Osborn, M60A1 lank crew 16-32 wks Task relevancy toet al. (1979) position cremen

ski l ls

Rigg (1983) 160A1 NOS skills 19-22 wks High level of mastery Time

Rigg & Gray M60A1 Gunnery 1-5 wks Level of mastery Time(1983)

Shlechter "I Communication 2 wks Group learning(1988)

High SOT level, Gunnery skills,Wisher, "I NOS skills Varied soldiers self- Low SOT level,et at. (1991) assessment, refresher Time

training -.. ~...AcCesn For

All tasks were procedural. NTICUr;annOU ed U

Figure 2. Summary of armor skill retention research

Distribution5

Availability Codes

Ois Avail ardljorDs Spca,

However, the relationship between level of original learningand retention may be more complicated than previously thought.Schendel & Hagman (1991) have found that augmented feedback mayenhance acquisition but not retention. Holding & Collins (inpublication) have noted that intrinsic task cues have a positiveeffect on students' skill zetention (tested after a 15 minute no-practice period) but not upon their level of original learning.And, Farr (1987) has noted that learners with the same level oforiginal learning may differ markedly on their retentionperformance, depending on such factors as refresher training.

Over training: A related pervasive finding is that overtraining makes skill retention more cesistant to extinction.This term refers to providing students with additional practicetrials once they have reached the criteria signifying skillmaster,. Work by ARI (e.g., Schendel, Shields, & Katz, 1978;Goliber j, Drillings, & Dressel, 1981) has indicated severaladanta;es of overtraining. One, it is a most potentinstructional strategy for promoting retention of skills.Goldberg et al., 1981 have shown that the skill retention ofM60Al crewmen (measured after 1 or 5 weeks) is directly relatedto the amount of overtraining on different skills. Two, overtraining may be a cost-effective instructional strategy as itreduces the need for frequent refresher training. Three, thebenefits of overtraining may extend beyond skill retention byallowing the skilled performer to use his/her attentionalresources in the most judicious manner possible (Schendel, etal.).

Length of the retention interval. Another well documentedfinding is that skill decay is directly related to the length ofthe no-practice retention interval. Leonard, Wheaton, & Cohen(1976) found, for example, that soldiers' grenade-launchingskills were much poorer after a 17-week retention interval thanafter a 6-week retention interval. However, the actual rate ofskill decay varies from task to task. It may also be mitigatedby such factors as prior practice, type of task (discrete,procedural or continuous), and the individual's abilities.

Refresher training. Once optimal retention intervals forparticular skills are determined, more effective schedules ofrefresher training can be implemented. Refresher or sustainmenttraining has been found to promote long-term retention (nospecific length mentioned) by reinstating the student's level oftask proficiency to the levels obtained during the period oforiginal learning (Wells & Hagman, 1989). The amount of timerequired for refresher training is consistently less than 50% ofthat required for initial training (Mengelkoch, et al., 1971),but varies according to:

6

(a) The amount of time between initial and refresher

training;

(b) The frequency of prior refresher training sessions;

(c) Trainee ability level; and,

(d) The nature of the task to be retrained (proceduraltasks take longer to retrain than continuous tasks).

Spacing of Practice. Interest in the spacing of practice,which refers to the temporal distribution of training sessions,has been minimal for approximately 20 years (Adams, 1987). Pastresearch indicates that different repetition schedules ofpractice for motor tasks--massed versus spaced--produce about thesame positive effect on learning and retention (Holding, 1965, ascited in Schendel & Hagman, 1991). However, there are certaininstances when spacing training iterations (or distributingpractice), as opposed to massing practice, would be recommended.Wells and Hagman (1989) concluded that spaced training scheduleswould be recommended for:

(a) Dangerous tasks where fatigue could pose a safetyrisk;

(b) Poorly motivated trainees who are adverselyaffected by the rigorous nature of massedrepetitions; and,

(c) High-ability trainees, who tend to make moreresponses during massed scheduling, quickly becomefatigued, and accordingly respond at a lower levelof proficiency than trainees of lower ability (p.45).

Schendel, Shields, & Katz (1976) have also indicated that thenumber of practice trials, not their distribution, is a keyissue, especially when training time is limited. When time islimited, training should be massed to optimize original learning.

Functional Similarity: The functional similarity or fidelitybetween the training and transfer situations has long beenconsidered to be a critical variable for promoting successfultransfer/retention of the training materials. As noted byDruckman & Bjork (1991), transfer is positive when the trainingand transfer tasks contain many shared structural elements andfew distinctive components. Also, Hagman and Rose (1983) havesuggested that skill retention can be improved by tailoring thetraining situation to specific field environments. They noted,for example, that extension course training may be moreappropriate than conventional classroom training for promotingtask acquisition and retention for military field purposes.

7

The skill retention literature, however, has also indicatedproblems with drawing any definitive conclusions about theeffects of this variable on students' retention abilities(Druckman & Bjork, 1991; Farr, 1987). For one thing, the amountof transfer obtained between the training and retentionsituations is a function of the functional similarities perceivedby the students (Druckman & Bjork, 1991). Additionally, Farr(1987) has noted the lack of any empirical evidence regarding theeffects of fidelity on students' long-term skill retention.

Knowledge of results (KR): Mixed conclusions have beenfound in the skill retention literature regarding the effects ofKR upon students' skill retention. It was once widely believed,as indicated in the early reviews, that KR was a crucial variablein skill acquisition and retention (e.g., Annett, 1979; Hurlock &Montague, 1982). As stated by Hurlock and Montague:

The research literature is clear about one variable thatmust be presented during the learning process. This variableis called knowledge of results or "feedback" (p. 6).

The recent skill retention literature, however, suggests that theeffects of KR are not so clear (Druckman & Bjork, 1991; Schendel& Hagman, 1991). Schendel and Hagman have noted that providingstudents with too much augmented feedback (from an externalsource) may attenuate their retention performance. Also,Druckman and Bjork suggested that instructors should givefeedback either less frequently during training or eliminate itduring later trials; otherwise, performance becomes dependentupon the feedback. Hence, instructors should be cautious inproviding students with augmented feedback about theirperformance.

Also, recent works from the instructional psychologyliterature suggest that the effects of KR are not so clear.Kulhavy and his associates (e.g., Kulhavy & Stock, 1989) haveargued that the effectiveness of feedback is basically a functionof the students' perception of a need for such information. Thatis, students who are certain that their responses are correct donot pay attention to the feedback provided by the instructionalsystem.

Mnemonics: Mnemonics are strategies employed to imposemeaning and/or organization on complex materials or skills tofacilitate learning and retention (Wells & Hagman, 1991). Thereviews (Druckman & Bjork,1991; Wells & Hagman, 1991) suggestthat allowing learners to provide their own elaborations enhancestheir retention and transfer of procedural tasks. However,Bellezza (1981, as cited in Wells & Hagman, 1991) found thatmnemonics generated for tasks with less than 10 elements havelittle, if any, effect on students' skill retention.

8

Task Factors

Continuous vs. vrocedural tasks: Marked differences existin soldiers' abilities to retain continuous and procedural tasks.Continuous responses involve the repetition of a movement patternwith no clear beginning or end (e.g. steering, tracking, or bikeriding); procedural tasks are made up of a series of discreteresponses (e.g. flipping switches, pressing buttons).

Students have been found to have more problems retainingprocedural tasks than continuous tasks (Grimsley, 1969a, 1969b;McDonald, 1967; and, Vineberg, 1975). As a result, proceduraltasks must be the subject of regular refresher training (weeklyor monthly) to remain proficiently executable, whereas,continuous task movements are retained well over intervals ofmonths or years, and do not need to be the focus of concentratedrefresher training.

Task complexity: Researchers from ARI projects have arguedthat the number of steps required in a task is a salientpredictor of skill retention (Hagman & Rose, 1983; Shields,Goldberg, & Dressel, 1979). Shields et al., found that soldiers'rate of skill decay after 4-12 months for assembly/disassembly ofthe M16 rifle was significantly related to the number of steps inthe task, with the most rapid decay affecting tasks that were notcued by the equipment or previous steps. However, their findinghas not been replicated. Hagman & Rose (1983) noted that rapidskill decay occurred most for task steps which were not cued bythe equipment or the next step. Definitive conclusions about therelationship between retention and this predictor variable cannotbe made.

Task organization: Another task factor discussed in theliterature is the effects of the task's inherent organization onretention and transfer. Morrison (1983) found that armor crewmenorganize their memory for procedures according to the hierarchalgoal structures of the task. One would then expect that suchstructured tasks are acquired faster and retained longer thantasks with less structure. The effects of task organization uponretention, however, seem to be dependent upon the learningprocess. Schendel & Hagman (1991) have noted that taskorganization was only a salient variable for those responseswhich were not well learned during the period of originallearning.

Individual Difference Factors

Ability level. Level of ability in the military can beassessed using mental categories defined by the Armed ForcesQualification Tests (AFQT) and/or General Technical (GT) scores.With respect to military training, higher ability studentsusually learn faster and achieve higher levels of acquisition

9

than their lower ability counterparts. As a result, higherability soldiers typically have higher levels of skill retention.

However, the relationship between ability levels and skillretention may be mitigated by other training factors. Vineberg(1975) has found that retention differences do not exist betweenhigher and lower ability students when they are trained to thesame level of original learning. Shlechter (1988) found thathigher ability trainees are inclined to benefit more from self-paced training methods while lower ability students benefit morefrom small group instruction.

Experience. An individual's prior knowledge of andexperience with similar tasks have been demonstrated tofacilitate his/her acquisition and retention of new skills(Hurlock & Montague, 1982). However, dissimilar skills retainedby experienced soldiers can create interference or negativetransfer when they attempt to learn a new or related skill.

Final Comments

Implications for Armor Training

There are several reasons for armor commanders to becautious in making any assumptions about their soldiers' orunits' skill retention. First of all, the quantity of datadirectly relating to armor skill retention is limited and dated.As previously indicated, nearly all the available data on thistopic were collected during the era of the M60-series tanks.Second, nearly all of the research on skill retention focuses onindividual skills as opposed to collective/team skills.Furthermore, these data did not always provide clear answers forskill retention issues due to the usual constraints of conductingresearch in real-world environments and the contextual andinteractive nature of the factors associated with skillretention.

It is possible to make some general inferences about armorskill retention. Integrating these inferences with the judgmentsof experienced trainers can lead to important implications forarmor training. These inferences with their possibleimplications are summarized below:

1. Armor skills appear to decay rapidly relative to othermilitary skill areas. There are several reasons for thisinference regarding the rate of armor skill decay. One, Wisher,Sabol, Sukenik, & Kern (1991), reported that survey data fromreservists activated for the Persian Gulf War indicated that theArmor School had the highest proportion of reservists (22%) whoreported remembering only a few of their MOS skills and requirednearly complete retraining. These self-report data indicateproblems with their retention of basic armor skills. Two, armor

10

skills usually involve many discrete steps. Three, these skillsinvolve many higher-order cognitive components, which are themost prone to skill decay.

2. Armor commanders should be highly concerned about thesustainment of their crewmens' skill proficiency, and must takeadvantage of every opportunity to provide practice for theirsoldiers and for themselves. These practice or refreshertraining opportunities should include pretesting the soldiers'level of skill mastery. Such testing provides training in-and ofitself. It also enables commanders to identify those skillswhich need additional (re)training.

3. Armor skills will usually decay in direct proportion tothe rate of original learning. That is, high rates of originallearning lead to high rates of skill retention. Armor commandersshould thus provide every opportunity for their soldiers to reacha high level of original skill mastery. This goal can beachieved by providing mastery training (three consecutive correctperformances) instead of standard training (one correctperformance).

4. Armor skill retention is a function of severalinstructional and task factors. Armor commanders can maximizetheir soldiers' skill retention by following these trainingrules: (a) make the training requirements functionally similar tothe on-the-job performance requirements; (b) space the practiceopportunities to minimize fatigue; (c) provide performancefeedback without making the soldiers overly reliant on suchinformation; and (d) encourage the soldiers to develop anorganizational model of the task. Armor commanders shouldcorrespondingly realize that skill retention is usually best forcontinuous, well-structured tasks.

5. As previously stated, Armor skill retention is becomingan increasingly important issue because of the downsizing of theArmy and the additional reliance upon the Reserve Components.Few research efforts, however, have investigated the skillretention of Armor Reserve Component and National Guard units.

Implications for Future Research

Several important skill retention issues remain largelyunexplored. Future research should continue:

1. Examining the effectiveness of simulators and otherautomated training devices in augmenting skill retention;

2. Examining the retention of collective skills in armor andother military units;

11

3. Investigating the armor skill retention and decay ofmembers of Reserve Component and members of NationalGuard units.

4. Developing skill retention measurement systems (i.e.,self-assessment, expert-assessment, automatedassessment in devices and simulations);

5. Determining the retention intervals for specific armorskills;

6. Assessing how often to refresh training for specificskills;

7. Identifying important personality variables andcognitive styles;

8. Investigating retention performance for higher ordercognitive skills, such as planning, preparing, anddecisionmaking; and,

9. Examining ways to enhance skill retention in theReserve Components.

By examining these issues researchers can begin to close thegap between what we know and do not know about skill retention.Then more definitive conclusions can be drawn regarding the skillretention of armor personnel.

12

Appendix A

Annotated Bibliography: Armor Skill Retention Literature

1. Army Training Study: M60A1 Weapon System. (1978). Retentionof basic armor training skills within the unit: A studyreport. Fort Knox, KY: Author.

Two-hundred seventy crewmen were retested on 11 skills which theyhad recently mastered during their basic armor training (BAT).The retention interval between the BAT testing and the retentiontests varied from 2 to 25 weeks. The Go rates for the retentiontests ranged from nearly 50% of the soldiers passing thebreechblock test to 90% of them passing the advanced drivingtests with an average GO rate of nearly 80% across the differenttests. The lower GO rate on the breechblock tasks was attributedto the greater number of steps involved in completing this task.The results also indicated that the higher ability subjects'retention performance was better than their lower abilitycounterparts. Finally, the results did not indicate anysignificant relationship between retention performance and thesoldier's rank, age, marital status, education level, and careerintention.

2. Goldberg, S. L., Drillings, M., & Dressel, J. D. (1981).Mastery Traininc: Effect on skill retention (Tech. Rep.No. 513). Alexandria, VA: U.S. Army Research Institutefor the Behavioral and Social Sciences. (AD A120 762) -

This research sought to determine the effects of mastery trainingand length of retention interval on retention of a proceduralskill. Forty-two armor crewmen were individually trained toboresight and zero the main gun of the M60AI tank. Crewmen weretrained to either of two criteria: one correct performance(standard training) or three consecutive correct performances(mastery training). Retention was tested either 1 or 5 weeksafter training. The results indicated that: (a) there was asignificant effect of both amount of training and length ofretention interval on recall of the task, but no interactionbetween the variables; (b) c:ewmen performed better on theretention test after the shorter retention interval or after moreintensive training; and (c) differences in performance among thegroups were mostly caused by differences on the first retentiontrial and mental category. While mastery training aidedretention, the authors noted that only 15% of the mastery trainedcrewmen performed the task correctly on the first retentiontrial. They concluded that mastery training might be most usefulfor tasks with few steps and tasks that: (a) must be recalledfrom memory; (b) must be performed correctly on the first trial,and, (c) lack adequate refresher training resources. (Revisedfrom document abstract)

A-i

3. Knerr, M., Harris, J. H., O'Brien, B. K., Sticha, P. J., &Goldberg, S. L. (1984). Armor procedural skills:learning and retention (Tech. Rep. No. 621). Alexandria,VA: U.S. Army Research Institute for the Behavioral andSocial Sciences. (AD A153 227)

This research investigated learning and retention of 8 armortasks selected to represent tasks that varied in length,complexity, and extent of practice in operational units. Theresults supported previous findings that: (a) performance decaysduring the interval between completion of armor school and unitplacement; (b) tasks with more steps and complexity decay morerapidly; and, (c) frequent practice improves armor skillretention, provided that the tasks are performed the same way inthe unit as they were during initial training. If tasks areperformed differently in the unit from the way they wereoriginally trained, then performance will appear to decline evenwith high rates of practice. Appendixes contained tablespotentially useful in efforts to estimate the requirements forinitial and refresher training.

4. Lampton, D. R., Bliss, J. P., & Meert, M. (1992). Theeffects of differences between practice and test criteriaon transfer and retention of a simulated tank gunnerytask (Technical Rep. No. 949). Alexandria, VA: U.S.Army Research Institute for the Behavioral and SocialSciences.

This investigation examined the effects of different practice andtest criteria on the transfer and retention of a simulated tankgunnery task. Three groups of 15 male undergraduates practicedtrack-and-shoot tasks using the Videodisc Gunnery Simulator(VIGS). Practice difficulty was controlled by setting the killzone area (the percentage of target-silhouette area--50%. 100% or150%--that is scored as a kill when hit). After practice,participants were immediately tested with the kill zone set at100%, and then tested after a 10 week retention interval.Performance was measured in terms of target kills, aiming error,and time to fire. The results revealed that practice with a moredifficult criterion yielded larger kill percentages and slowerfiring times than did practice and testing with the samecriterion. This trend toward significance remained for transfer(2 = .061) and retention (p = .057).

5. Morrison, J. E. (1984). Methods for empirically derivingthe structure of memory for procedures. Journal ofADDlied Psychology, 690, 473-481.

To determine the structure of memory for procedures, 12 noviceand 12 experienced male soldiers verbally recalled and performedtwo procedural tasks. Proximity analyses revealed that soldiersorganized their memory for procedures according to a hierarchical

A-2

structure of task goals. Experienced personnel were lessaccurate than novice soldiers in their recall and performance oftasks; however, there were no systematic differences in derivedmemory structures between the groups. Morrison suggested thatthe poorer performance of experienced soldiers was due to thelack of recent practice, despite the experienced soldiers'advantage in overall frequency of performance. Structuresderived from verbal recall were highly indicative of taskperformance for the novice soldiers. Combined with a rationalanalysis of task goals and subgoals, proximity analysis canprovide a useful description of the memory requirements ofprocedural tasks. (Document abstract)

6. Morrison, J. E. (1982). Retention of armor procedures: A=tructural analysis (Tech. Rep. No. 591). Alexandria,VA: U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A140 003)

Proximity analyses were performed on verbal recall and hands-onperformance of selected procedures to determine the structure ofmemory for armor procedural tasks. These analyses confirmed thatarmor crewmen organized their memory for procedures according tothe hierarchical goal structures of the tasks. Comparisons ofentry-level and field unit armor personnel showed significantdecrements in skill performance over time; however, there were nosystematic differences in memory structure between the twogroups. Structures derived from verbal recall were highlyindicative of hands-on structures for crewmen still in training,but the relationships between verbal and hands-on structure wasnot as strong for armor crewmen in field units.

7. Morrison, J. E., & Bessemer, D. W. (1981). TraininQ andretention of armor machine gun tasks (Research Rep. No.1317). Alexandria, VA: U. S. Army Research Institutefor the Behavioral and Social Sciences. (AD A128 824)

This report examined the retention of armor machine gun tasks inplatoons within 3 Armor One Station Unit Training (OSUT)companies. Each platoon was assigned to one of three M85training schedules; a single four-hour block, two four-hourblocks received in one day, or two four-hour blocks separated byat least one week. One of the three companies was also shownvideotaped demonstrations of M85 tasks. GO/NO GO data on M85 andM240 tasks were gathered by evaluators from the Directorate ofPlans and Training (DPT) at Ft. Knox. M85 performance wasmeasured at the end of the OSUT cycle, whereas M240 scores weregathered at both mid and end-of-cycle tests. Execution times onM240 tasks were also collected. The findings indicated: (a) noeffect of training schedule and introduction of videotapeddemonstrations on M85 task petformance or M240 retention; (b)poor performance on M85 mechanical training tasks for the end-of-cycle test; (c) reliable degradation in M240 performance between

A-3

mid and end-of-cycle tests; and (d) task execution times revealedsubtle changes in performance, which were not indicated by theGO/NO GO scores. (Pevised from document abstract)

8. Morrison, J. E., Drucker, E. H., & Campshure, D. A. (1991).Devices and aids for training Ml tank crunnery in the ArmyNational Guard: A review of military documents and theresearch literature (Research Rep. No. 1586).Alexandria, VA: U. S. Army Research Institute for theBehavioral and Social Sciences. (AD A240 628)

This project analyzed six training devices and aids that arerelevant to gunnery training in an armory environment. Thedevices were M1 Top Gun; M1 Videodisc Interactive GunnerySimulator (VIGS); M1 Mobile Conduct-of-Fire Trainer (M-COFT);Guard Unit Armory Device Full-Crew Interactive SimulationTrainer, Armor (GUARD FIST I); Simulation Networking (SIMNET)battlefield simulation system, and Hand-Held Tutor (HHT). Theauthors discussed the training effectiveness of these devices andaids with respect to (a) skill acquisition; (b) skill retention;(c) prediction of performance; and (d) transfer of training.However, in their review of the literature, the authors notedthat there was "very little research that addressed the effectsof training devices and aids on the retention of gunneryknowledge or skills" (p. viii). The research that did addressskill retention was reported in the validation and verificationtests performed in 1984 with U-COFT by the General ElectricCompany and maintained that armor crews should be retrained using-U-COFT every 3-10 weeks.

9. Osborn, W. C., Campbell, C. H., & Harris, J. H. (1979).The retention of tank crewman skills (Research Rep. No.1234). Alexandria, VA: U. S. Army Research Institutefor the Behavioral and Social Sciences. (AD A086 399)

This study examined armor crewman skills between completion ofbasic armor training and the early months of field unitassignment. Performance tests were administered at the end offormal training and again after 4 to 8 months on the job. Forboth testing periods, overall proficiency remained unchanged fromthe school to field testing. When examined by task categoryproficiency declined for tasks common to all crewmen andincreased for tasks specific to a crew position. Specific testresults should aid in determining when to retrain specific armorskills. (Revised from document abstract)

A-4

10. Rigg, K. E. (1983). Optimization of skill retention in theU.S. Army through initial training analysis and design:Prototype skill sustainment guide (MGA-5181-PRO).Monterey, CA: McFann, Gray & Associates, Inc. (AD A132190)

This report summarized the results of a McFann, Gray & Associates(MGA) software test designed to estimate training and retentionoutcomes. Sixty-eight soldiers, who were randomly selected fromunits at Fort Campbell and Fort Lewis, participated in thisresearch effort. All soldiers were trained to criterion for40 selected armor tasks and then were retested prior to theirunit refresher training. The author reported that the MGAsoftware provided an accurate estimate of the soldier's trainingand retention performance. However, empirical data were notpresented in this document to substantiate this claim. (Revisedfrom the document's executive summary)

11. Rigg, K. E. & Gray, B. B. (1980). Estimating skill trainingand retention functions through instructional modelanalysis (MRO-FR-0378). Monterey, CA: McFann, Gray &Associates, Inc.

This document reports the findings of a study that applied amathematical model (the Markov chain model) to data collectedduring initial training in order to: (a) describe initialtraining; (b) predict training outcomes in terms of time, costs,efficiency, and end of course performance; and, (c) predict thelevel of field performance after a retention interval with nopractice. Forty-two armor crewmen were trained to boresight andzero the main gun of the M60AI tank. The study was designed withtwo levels of initial training: one correct performance or threeconsecutive correct performances; and with two retentionintervals: one or five weeks. Results obtained using the Markovmodel proved to be highly predictive of field performance basedon initial performance in training. The authors note that theMarkov model may not be applicable to skills that are learnedgradually or to skills that are repeated several times as sub-elements of another procedural skill (e.g. placing safety on FIREposition).

12. Shlechter, T. M. (1988). The effects of small group andindividual computer-based instruction on retention and ontraining lower ability soldiers (Research Rep. No. 1497).Alexandria, VA: U. S. Army Research Institute for theBehavioral and Social Sciences. (AD A203 793)

The effects of having various numbers of soldiers at a terminalfor computer-based instruction (CBI) in a procedural task wereexamined. Soldiers in each of two experiments were trained touse the Communications-Electronics Operating Instructions (CEOI)extracts. Twenty-four soldiers participated in Experiment 1.

A-5

Eight were non-systematically assigned to each of threeconditions: "GRP", or four at a terminal; "PR" or two at aterminal; and "IND", or one at a terminal. All soldiers took apretest, a posttest immediately after training, and a secondposttest 2 weeks later. Thirty-four soldiers who were insustainment training participated in Experiment 2. Twelve werein the GRP condition, 10 in PR, and 12 in IND. All werepretested and took three posttests: one immediately aftertraining, another 2 weeks later, and another 8 weeks aftertraining. Soldiers in the PR conditions retained moreinformation than those in the IND condition. Significantdifferences were not found for the soldiers' scores on theimmediate posttest.

13. Sticha, P. J., Edwards, T. D., & Patterson, J. F. (1984).An analytical model of learning and performance of armorprocedures (Research Note No. 84-24). Alexandria, VA:U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A138 000)

This report documented the development of a model to investigateissues regarding acquisition and retention of procedural armorskills. Learning and retention models were developed for eighttasks performed by the driver, gunner, and loaders of the M60A1tank. Sequencing control was modeled using the SAINT (SystemAnalysis of Integrated Networks of Tasks) simulation system.Models described acquisition, retention, retrieval, and choice oftask information and were validated by comparing theirpredictions to two samples of data, one composed of soldiers intraining, and one from soldiers in operational armor units.

14. Wisher, R. A., Sabol, M. A., Sukenik, H. K. & Kern, R. P.(1991). Individual ready reserve (IRR) call-up: Skilldecay (Research Report 1595). Alexandria, VA: U. S. ArmyResearch Institute for the Behavioral and SocialSciences. (AD A239 371)

Soldiers from the Individual Ready Reserve (IRR) who were calledup for the Persian Gulf War were tested to determine the extentof skill decay since release from active duty. Results fromtests (e.g. hands-on performance, written, weapon qualifications)were merged with data from personnel files and responses to a 31-item questionnaire on job experience, attitudes, and personalimpact of call-up. The major findings were that: (a) knowledgeabout Army jobs decayed mostly within 6 months; (b) weaponsqualification skills decayed mostly after 10 months; (c) previousskill qualification scores were the best predictor of skilldecay; (d) skill decay was higher in Armor and Combat Engineeringfields and lower in Infantry, Maintenance, and Supply fields; and(e) skill retention was higher for those who entered the IRRdirectly from active duty. (Revised from document abstract)

A-6

Appendix B

Annotated Bibliography: Reviews of the Skill RetentionLiterature

1. Adams, J. A. (1987). Historical review and appraisal ofresearch on the learning, retention, and transfer of humanmotor skills. Psychological Bulletin, 101, 41-74.

Adams has reviewed over 360 articles, which span the past 100years, on learning, retention, and transfer of human motorsskills. This review focused on: (a) knowledge of results; (b)distribution of practice; (c) transfer of training; (d)retention; and (e) individual differences in motor learning. Themain finding concerning retention was that performance on adiscrete procedural task, like throwing switches in sequence, wasforgotten more rapidly than performance in a continuous task liketracking. Forgetting of procedural responses was completed in ayear, although there can be savings, because relearning is rapid.

2. Annett, J. (1979). Memory for skill. In M. M. Gruneberg &P. E. Morris (Eds.), Applied problems in memory, (pp.215-247). New York: Academic Press.

This chapter reviewed skill acquisition and retention researchfrom 1885 (Ebbinghaus) to 1979. It also discussed severalresearch findings pertaining to memory for different kinds ofskill by distinguishing between natural vs. artificial tasks,motor vs. verbal tasks, continuous vs. discrete tasks, integratedvs. non-integrated tasks, type and amount of training, durationof the retention interval, interference and/or facilitation byactivities in the retention interval, conditions of recall, andindividual differences. The pertinent conclusions were thatrefresher training, level of skill at the end of originallearning, and duration of the retention interval are crucialfactors in skill retention.

3. Druckman, D. & Bjork, R. A. (1991). Optimizing long-term retention and transfer. In D. Druckman & R. A. Bjork(Eds.). In the mind's eye: Enhancing human performance(pp. 23-56). Washington, D.C.: National Academy Press.

This chapter reviewed major training variables that effect long-term retention and transfer of cognitive-motor procedural skills.The authors concluded that: (a) post-training performanceimproves as original learning increases; (b) automatic skillshave a higher likelihood of being retained without refreshertraining; and, (c) procedural task forgetting is affected by thetask's organization, number of steps, degree of cuedness, andamount of elaboration possible. They also concluded thattransfer is enhanced when: similarity between training and thetransfer task is high; and, sufficient feedback is provided.

B-1

4. Farr, M. J. (1987). The long-term retention of knowledgeand skills: A cognitive and instructional perspective.New York: Springer-Verlag.

Farr reviewed 133 articles that revealed several factors whichinfluence knowledge and skills over relatively long periods ofnonuse. He has identified major variables influencing long-termretention in order of importance as: (a) the degree of originallearning; (b) task characteristics (type and complexity oforganization); (c) length of the retention interval; (d)instructional strategies and conditions of learning; (e) methodsfor testing retention and conditions of retrieval; and, (f)individual differences. Farr proposed several ways to promotelong-term retention. First, one must consider the real-worldenvironment in which the learner must function. Second,selectively overtrain more difficult tasks. Third, present thematerial or skill to be learned in a way that elaborativeprocessing can create associative links. The appendices includedetailed analyses of 8 major literature reviews that dealt withlong-term retention.

5. Hagman, J. D., & Rose, A. M. (1983). Retention of militarytasks: A review. Human Factors, 25, 199-213.

This article reviewed thirteen experiments conducted or sponsoredby the Army Research Institute examining retention of militarytasks. Selected findings of this review were: (a) increasing theamount of task repetition enhanced retention; (b) testing duringtraining enhanced retention; (c) spacing, rather than massingrepetitions increases retention; (d) tailoring training forspecific training environments improved retention; and (f) usingmnemonic techniques during training did not necessarily promotebetter retention than rote memorization.

6. Hurlock, R. E., & Montague, W. E. (1982). Skill retentionand its implications for navy tasks: An analyticalreview (SR-82-21). San Diego, CA, Navy PersonnelResearch and Development Center. (AD A114 211)

This review identified variables contributing to skill loss inthe Navy. Findings were organized into five categories--personnel characteristics, task variables, training factors, jobconditions, and retraining factors. Significant retentionvariables included original learning, nature of the learningenvironment, overlearning, ability level, type of task, number ofsteps or subtasks of a job, quantity of practice (refreshertraining), quality of feedback, length of nonuse, and recallconditions. The authors concluded that the existing scientificliterature has had little application to naval skilldeterioration because the previous research conditions differedfrom those found in the complex naval working environment.

B-2

7. Prophet, W. W. (1976). Long-term retention of flyingskills: A review of the literature (Final Report 76-35).Alexandria, VA: Human Resources Research Organization.(AD A132 188)

This study reviewed the literature on long-term retention offlying skills. Findings were organized in three broadcategories: general retention factors; task or skill factors;and, retraining factors. General retention factors includedlevel of learning, length of retention interval, habitinterference and transfer, and rehearsal effects. Task/skillfactors included control and procedural tasks, instrument andcontact tasks, information processing tasks, and "other" taskfactors. Retraining factors included use of devices, nature oftraining, and individual factors in retraining. Resultssuggested that basic flight skills could be retained for extendedperiods of non-flying, but instrument and procedural skillsexhibited some decay.

8. Rose, A. M., McLaughlin, D. H., Felker, D. B., & Hagman, J.D. (1981). Retention of soldiering skills: A review ofrecent ARI research (Tech. Rep. No. 530). Alexandria,VA: U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A119 307)

This report reviewed 16 research studies sponsored or conductedby ARI on the acquisition and retention of Army tasks and skills.The findings suggest that-the most important training factors foracquisition and retention of such skills are the: (a) criterionfor proficiency or mastery upon completion of training, (b) useof practice and test trials during training, and (c) use ofstructured training materials. They also reported that soldierstended to have retention problems with difficult, uncued, andnon-critical tasks.

9. Schendel, J. D., Shields, J. L, & Katz, M. S. (1978).Retention of motor skills: Review (Tech. Paper No. 313).Alexandria, VA: U. S. Army Research Institute for theBehavioral and Social Sciences. (AD A061 338)

This review dealt with the variables known to affect theretention of learned motor behaviors over lengthy no-practiceintervals. These variables were dichotomized into task andprocedural variables. Task variables included: (a) duration ofthe no-practice period, or retention interval; (b) nature of theresponse required to accomplish a particular motor task; (c)degree to which the learner could organize or order the elementsthat defined the task; (d) structure of the training environment;and (e) initial or "natural" ability of the learner inperformance of a task without prior practice. Proceduralvariables included: (a) degree of proficiency attained duringinitial training; (b) amount and kind of refresher training; (c)

B-3

transfer of skills from previous tasks; (d) presence ofinterfering activities (e) distribution of practice duringretraining; (f) use of part-task versus whole-task trainingmethods; and (g) introduction of extra test trials prior to finaltesting.

10. Schendel, J. D., & Hagman, J. D. (1991). Long-termretention of motor skills. In J. E. Morrison, (Ed.),Training for Performance (pp. 53-92). New York: JohnWiley & Sons.

This chapter identified some of the key variables in the long-term retention of motor skills. For example, refresher trainingcan be used to offset the effects of forgetting. Also,retraining time has been shown to vary depending on suchvariables as length of retention interval, nature of the task,frequency of refresher training sessions, temporal spacing ofsuccessive sessions, and learners' initial ability levels. Anexample of a retention prediction matrix (Rose et al., 1985) for9 military tasks was presented.

11. Wells, R. & Hagman, J. D. (1989). Training procedures forenhancing reserve component learning, retention. andtransfer (Tech. Rep. No. 860). Alexandria, VA: U. S.Army Research Institute for the Behavioral and SocialSciences. (AD A217 450)

This report cites over 220 articles identifying specific trainingprocedures for enhancing learning, retention, and transfer ofmilitary skills within Reserve Components. The pertinentfindings are that: (a) repetition facilitates proficiency on allbut the simplest verbal and perceptual-motor tasks; (b) verbaltasks are retained better with distributed practice versus massedpractice; (c) perceptual-motor skills are retained at nearly thesame level regardless of the type of practice; (d) testing duringtraining promotes retention of verbal and perceptual-motorskills; and (e) elaboration on materials to be learned increaseslearning and retention.

B-4

Appendix C

The Pertinent Military and Civilian Skill Retention Literature

1. Begland, R. R. (1979). The effect of different guestiontypes on retention in audio-visual materials (NTIS Nr. AD-A075 652). Fort Monroe, VA: U.S. Army TrainingDevelopments Institute.

The effects of the insertion of different types of questions uponlearning and retention of an audiovisual lesson were examined. Atotal of 345 male soldiers attending Advanced Individual Training(AIT) viewed a Training Extension Course (TEC) lesson dealingwith the operation and nomenclature of a night vision device andthen completed a written post test. Four versions of the TEClesson were used: the standard lesson used by the Army, thelesson with all embedded questions and feedback deleted, thelesson with 21 comprehension questions inserted, and the lessonwith 21 simple verbatim questions inserted. The post-test wasgiven to 168 soldiers immediately after viewing the lesson and tothe other 177 soldiers after a delay of 48 to 126 hours. Themain result was that the immediate post-test soldiers scored anaverage of 25.3% of the possible points, while the delayed post-test group scored an average of 18.4%. This difference wassignificant, demonstrating the loss of knowledge gained from aTEC lesson within a few days. (Revised available in-houseabstract; hard copy not procured)

2. Berstein, B. R. & Gonzalez, B. K. (1971). Learningretention. and transfer in military training (NAVTRADEVCEN69-C-0253-1). Orlando, FL: Naval Training Device Center.(AD 733 964)

Five experiments explored the influence of imagery, fidelity ofsimulation, and retention interval on learning, retention, andtransfer. Experiments I and II piloted the methods andprocedures to be used for experiments III, IV, and V. Inexperiment III, assessing imagery, 10 undergraduates were trainedabout 1-1.5 hours to operate a synthetic communications system.Fourteen days after training, subjects returned for a transfersession. Experiment IV assessed the interaction between imageryand verbal skills. Thirty male introductory psychology studentstook a paper and pencil test. Experiment V investigated theeffect of simulation fidelity on transfer performance. Subjectswere trained using one of three levels of fidelity. The findingsprovided no evidence that imagery improves performance or thathigh physical fidelity enhances training effectiveness. Measuresof retention revealed considerable improvement (20-60 percent) inspeed and accuracy performance after 2 and 7 day retentionintervals. After 14 days, speed showed no difference andaccuracy decayed 30 percent from the first session level.

C-i

Students exhibited a 30 percent retention decay from their firstsession level. (Hard copy not procured)

3. Blaiwes, A. S., & Regan, J. J. (1970). An integratedapproach to the study of learninQ, retention. andtransfer: A key issue in training device researchdevelopment (NAVTRADEVCEN Tech. Rep. I-178). Orlando,FL: Naval Training Device Center. (AD A712 096)

This paper categorized 181 reports according to learning,retention, and transfer and briefly discussed the relationshipbetween these topics. Four general research issues werediscussed that characterized learning, retention, and transferresearch. These issues were identifying: (a) the subtasks thatshould be included in training simulation; (b) which variationsin stimulus and response characteristics of the training systemshould be incorporated; (c) which instructional devices,materials, and methods should be introduced; and (d) how muchgeneralization should be built into training devices. A summaryfocused primarily on the methods, materials, and devices thatenhance transfer.

4. Dressel, J. D., (1980). Mnemonically enhanced training(Working Paper). Alexandria, VA: U.S. Army ResearchInstitute for the Behavioral and Social Sciences.

This project compared the effectiveness of a rote-repetitionmnemonic training technique on the acquisition and retention ofan 18-step procedural task (installing an M14 anti-personnelmine). Fifty-one soldiers taking part in One Station UnitTraining (OSUT), learned the task with a mnemonic strategy (e.g.each task operation was represented by a letter and the sentencepreserved the correct order of task performance) or without theuse of the mnemonic strategy. Retention was tested one monthlater by a relearning procedure. Results revealed that retentiondid not vary as a function of the training method. Dresselconcluded that soldiers did not need to use the mnemonic method,because the task contained a manageable number of steps and couldthus be remembered from rote training. (Revised from abstract inHagman & Rose, 1983; hard copy not procured).

5. Gardlin, G. R. & Sitterley, T. E. (1972). Degradation oflearned skills: A review and annotated bibliograhyI(D180-15081-1, NASA-CR-128611). Seattle, WA: BoeingCompany.

This review investigated the decay of learned skills pertinent tospace flight. Detailed abstracts were included for 21 of the 116studies most relevant to NASA's interest. Short summaries wereincluded for 25 studies with less direct application. The majorfindings were that: (a) motor skill retention varied almostdirectly with the amount of original learning; (b) less trained

C-2

subjects exhibited improved retention for procedural tasks withhigher organization; (c) part-task training improved retentionwhen focused on temporal aspects of procedural tasks, whilewhole-task training proved superior for single tasks; and, (d)rapid relearning occurred for tracking tasks, even after 2 years.The authors concluded that the level of performance in the finaltraining period (initial mastery or original learning) was thebest predictor of motor skill retention regardless of theretention interval length. (Revised from existing in-houseabstract; hard copy not procured)

6. Goldberg, J. H., & O'Rourke, S. A. (1989). Prediction ofskill retention and retraining from initial training.Perceptual and Motor Skills, 6, 535-546.

This research examined a method for predicting retention andretraining performance of a manual skill after controlledtraining. Forty men were trained to criterion performance onlow- or high-workload tracking tasks, then retested and retrainedafter no-practice intervals of 1 to 6 weeks. Retention andretraining performance times were modeled as a function of(a) average tracking completion time during initial training and(b) a power-model point-estimate of training completion times.The power model predicted retention and retraining times slightlybetter than average times, but both models were significant (.64< R < .78). Performance times were not related to duration ofretention and were marginally related to training workload.Later performance was well predicted by earlier task performance,but retention was independent of the elapsed no-practice timeperiod. (Abstract)

7. Grimsley, D. L. (1969a). Acauisition, retention, andretraining: Effects of high and low fidelity in trainingdevices (Tech. Rep. No. 69-1). Alexandria, VA: HumanResources Research Organization. (AD 686 741)

This study extended the work of a previous study (RINGER) toinclude an examination of retention. The task to be learnedrequired performing 92 simple procedural tasks (operation ofswitches, plugging in headsets, making brief verbalannouncements, etc.) in proper sequence. Training was closelymonitored and assisted by instructors through unsystematicreinforcement and cuing. Training to error-free completion ofthe procedural task require a maximum of 3 hours per individual.Subject's were tested and scored by alternate instructors whomerely counted number of correct steps completed. Then after 4weeks, and again after an additional 2 weeks, subjects wereretested. Following the tests they were retrained to correcterrors and their retraining performance was recorded. There wereno significant differences between the various trainees'performances on any tests, nor were there significant differencesin their time and accuracy in relearning to correct tested

C-3

errors. All subjects retained material equally well for 6 weeksregardless of fidelity of device.

8. Grimsley, D. L. (1969b). Acquisition. retention andretraining: Group studies on usina low fidelity trainingdevices (Tech. Rep. No. 69-4). Alexandria, VA: HumanResources Research Organization. (AD 686 741)

This study investigated the effect of varying training devicefidelity on acquisition, retention, and retraining of aprocedural task. One hundred and twenty trainees in AdvancedIndividual Training were randomly divided into 10 groups.Subjects with Armed Forces Qualification Test scores below 30were not included. Subjects learned a 92-step procedural task ondevices of differing simulator fidelity (functioning duplicate,non-functioning duplicate, or artist's representation). Theresults showed no significant differences in training time,initial performance level, amount remembered after 4 and 6 weeks,or retraining time between groups trained on high versus lowfidelity devices. In addition, subjects who knew in advance thatthey would be retested exhibited no retest score advantage oversubjects without access to this information. Thus, Grimsley hasconcluded that the fidelity of training devices can be very lowwithout adverse effect on training time, level of proficiency,retention, or time to retrain. This conclusion remains truewhether the training time is administered individually or ingroups.

9. Grimsley, D. L. (1969c). Acquisition, retention andretraining: Training catecory IV personnel with lowtidelity devices (Tech. Rep. No. 69-12). Alexandria,VA: Human Resources Research Organization.

Seventy-two trainees enrolled in Advanced Individual Trainingwere individually trained to perform a 92-step procedural task onthe control panel of a Nike-Hercules guided missile system (thesame task as used in other studies by Grimsley). Resultsindicated that: (a) low aptitude personnel trained on lowfidelity training devices can achieve high performance levels oncomplicated procedural tasks; (b) category IV trainees neededmore time to master the task than higher ability men; (c) therewere no differences in training time, initial performance level,or amount retained after 4 and 6 weeks. He concluded that thetraining device fidelity can be low, with no adverse effect ontraining time, level of proficiency, amount remembered over time,or time to retrain.

C-4

10. Hagman, J. D. (1979). TvMewriting: Retention andrelearning (Research. Rep. 1211). Alexandria, VA:U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A072 369)

Retention and relearning of typewriting skills among 38administrative specialists (71L MOS) were examined after the no-practice retention interval between Advanced Individual Training(AIT) graduation and unit duty. Without practice, average typingspeed dropped 12% and errors increased 86% between AIT graduationand unit duty. As a result, net typing proficiency fell belowthe minimum AIT graduation standard. Twenty-five minutes ofadditional typing practice reinstated a significant amount (28%)of lost typing speed and 19% of lost net typing proficiency. Itwas concluded that brief refresher training (1.5 to 2.5 hrs) issufficient for Administrative Specialists to regain typewritingskill. (Revised from document abstract)

11. Hagman, J. D. (1980a). Effects of presentation and testtrial training on motor acquisition and retention (Tech.Rep. No. 431). Alexandria, VA: U. S. Army ResearchInstitute for the Behavioral and Social Sciences. (ADA086 406)

This experiment examined the relative effects of three differenttraining methods on the acquisition and retention of apositioning motor task. The first group of 15 subjects was giventhree alternating presentation and test trials. The second groupwas given five presentation trials followed by a test trial percycle. The last group had a single presentation trial followedby five test trials. Group acquisition performance was comparedat the last trial of each cycle while retention was compared 3minutes and 24 hours after acquisition. Absolute error scoresindicated that acquisition and short-term retention were bestwhen training emphasized presentation and test trial alternationand presentation repetition within cycles. Long-term retentionwas best when training stressed test trial repetition. Theseresults have suggested that testing is an effective way toenhance long-term retention of motor skills.

12. Hagman, J. D. (1980b). Effects of training task repetitionon retention and transfer of maintenance skill (ResearchRep No. 1271). Alexandria, VA: U.S. Army ResearchInstitute for the Behavioral and Social Sciences. (ADA101 859)

This experiment investigated the effects of task repetitions onthe performance of a simple task. Subjects were military fueland electrical repairmen. The task repetition levels varied from0 to 4 repetitions of the task. An apparent ceiling of-3repetitions of the tasks during training was found to enhanceboth acquisition of the skill and 2-weeks retention of the skill

C-5

in task performance. However, the task repetitions did notenhance transfer of the skill to similar equipment. Maintenancetask retention improved in terms of both speed and accuracy asthe number of task repetitions (up to 3) performed duringtraining increased.

13. Hagman, J. D. (1980c). Effects of training schedule andM uinment variety on retention and transfer ofmaintenance skill (Research Rep. No. 1309). Alexandria,VA: U. S. Army Research Institute for the Behavioral andSocial Science. (AD A120 167)

This experiment examined the effects of training schedule andequipment variety on maintenance task retention and transfer.During training, four groups of 15 student Fuel and ElectricalRepairers (63G MOS) performed the task of testing charging systemelectrical output. Each group performed three task repetitionsunder one of four training conditions formed by the factorialcombination of two training schedules (massed, spaced) and twolevels of equipment variety (present, absent). Massed schedulingallowed no rest pauses between successive task repetitions;spaced scheduling allowed one-day rest pauses betweenrepetitions. When equipment variety was present, studentsperformed one repetition on each of the three charging systems.When equipment variety was absent, all three repetitions wereperformed on the same charging system. The first retention testoccurred immediately after training and the second occurred anaverage of 14 days later. It was concluded that: (a) spacing oftask repetitions during training is an effective way to improveboth retention and transfer of maintenance skill; and that (b)added transfer improvements can be obtained by coupling spacedtask repetitions with increased equipment variety duringtraining.

14. Hagman, J. D. (1980d). Effects of presentation and testtrial training on acquisition and retention of movementend-location (Tech. Rep. No. 492). Alexandria, VA:U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A100 867)

This research on a linear positioning task examined the effectsof presentation and test trial training on acquisition andretention. The retention segment of the experiment consisted ofa single test trial performed at both 3 minutes and 24 hoursafter the 18th trial of acquisition. Long-term retention wasbetter when training emphasized test-trial repetition. Thus,testing proved to be an effective way to improve long-termretention of motor skill. The author concluded that trainingshould emphasize testing more than presentation.

C-6

15. Hagman, J. D. (1981). Testing during training: Why does itenhance lonQ-term motor task retention? (Tech. Rep. No.535). Alexandria, VA: U. S. Army Research Institute forthe Behavioral and Social Sciences. (AD A120 761)

This study examined the reasons why testing during training ismore effective than repeated presentation in promoting long-termmotor task retention. Sixty professional and clerical employeesof the Army Research Institute performed a linear positioningtask consisting of 18 training trials divided into three cyclesof six. Cycles included presentation-trials and test-trials.Presentation-trials were experimenter-defined movementsterminated by a mechanical stop. Test-trials were learner-defined recall movements unconstrained by the mechanical stop.Training methods varied in their emphasis on presentation or testtrials. Results indicated that groups which emphasized testingduring training showed no post-training recall error increases.Groups which emphasized presentation showed increases in recallerror over the post-training retention interval. Twenty fourhours after training the "test" groups exhibited better retentionthan the "presentation" groups. The author argued that theincreased retention due to testing can be attributed to learner-defined, as opposed to experimenter-defined movements and thattesting during training benefitted movement distance and end-point cue retention.

16. Hagman, J. D., Hayes, J. F., & Bierwirth, W., (1986). Amethod for estimating task retention: Research pays off.Army Trainer, 1, 54-55.

This article described and explained the "User's Manual forPredicting Military Task Retention" (Rose, Radtke, Shettle,Hagman, 1985) to the military trainer. It reviewed the threesteps (determining a task's retention rating, determining the no-practice interval, and predicting soldier retention performance)in the process of estimating task retention. This method forpredicting task retention can be used to assess how quicklyspecific tasks are forgotten, the percentage of soldiers who canperform a task correctly up to one year of no practice, and howoften sustainment training should be conducted.

17. Hall, E. R., Ford, L. H., Whitten, T. C., & Plyant, L. R.(1983). Knowledge retention among graduates of basicelectricity and electronic schools (Tech. Rep. No. 149).Orlando, FL: Training Analysis and Evaluation Group,Department of the Navy. (AD A131 855)

This study examined knowledge decay during the interval (about 28days) between graduation from Basic Electricity and Electronics(BE/E) School and entry into a Construction Electrician (CE) "A"School. The results showed that BE/E knowledge did decay andthat the amount of decay was related to student ability

C-7

characteristics. The average student lost about 24% of AC theoryknowledge and about 6% of DC theory knowledge. In addition,higher ability students exhibited less decay of BE/E informationthan lower ability students. Decay had no apparent affect onsubsequent "A" School achievement.

18. Healy, A. F.; Erickson, K. A.; Bourne, L. E., Jr. (1990).Ootimizina the long-term retention of skills: Structuraland analytic approaches to skill maintenance (ResearchNote 90-104). Alexandria, VA: U. S. Army ResearchInstitute for the Behavioral and Social Sciences.(AD A226 130)

This research identified the characteristics of knowledge andskill most resistant to decay and disuse. The research wasdivided into two parts. The first part was concerned withexperimental analysis of factors influencing and improvingretention of skill components. The second part was concernedwith analysis and assessment of the structure of acquired memoryand skills and how to monitor differential retention ofcomponents. For the analytic approach five methodologies weredeveloped and investigated. Four natural skills wereinvestigated. For the structural approach, an experimentalparadigm was designed to assess the detailed encoding of newknowledge at presentation and at delay using verbal reporttechniques and chronometric measurement of retrieval components.Several studies of retention of vocabulary items were completed.They found evidence for a surprising degree of long-term skillretention. A theoretical framework focusing on the importance ofprocedural reinstatement was formulated which enabledunderstanding of the memory performance. (Abstract from ARIbibliography; hard copy not procured)

19. Holmgren, J. E., Hilligoss, R. E., Swezey, R. W., & Eakins,R. C. (1979). Training effectiveness and retention oftraining extension course (TECg instruction in thecombat arms (Research Rep. No. 1208). Alexandria, VA:U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A069 942)

This project examined the training effectiveness and retention ofTEC instruction relative to conventional classroom instruction.TEC lessons in five subject areas were evaluated. The five tasksconsidered were: (a) field artillery--use of the gunner'squadrant; (b) air defense artillery--use of the Target Alert DataDisplay System (TADDS); (c) armor--M551 tank target engagementsimulation; (d) infantry--use of the squad radio; and (e) firingand zeroing the M60 machine gun. The retention intervals usedwere 8-9-weeks for Active Army soldiers and 7-12 weeks forNational Guard soldiers. Results indicated that TEC trainedsoldiers performed better than conventionally trained soldiers on

C-8

the initial and retention tests. Performance declined for bothgroups across all tasks.

20. Hughes, C. R., Butler, W. G. Sterling, B. S., & brgland, A.W. (1987). M1 Unit-Conduct of Fire Trainer (TRC-WSMR-TEA-16-87). White Sands Missile Range, NM: Departmentof the Army, U. S. Army TRADOC Analysis and Command.

The effect of the Unit-Conduct of Fire Trainer (U-COFT) onretention of live-fire gunnery skills was one of the issuesexamined in this Post Fielding Training Effectiveness Analysis(PFTEA). Six battalions participated in this experiment. Fivereceived Ml U-COFT nine months prior to the experiment, while thesixth received theirs after the experiment. One month after theresearch began, all six battalions were tested on a crewqualification exercise (Table VIII). Three months later, astratified sample of 15 crews from each battalion fired a specialTable VIII without firing the pre-qualification tables (VI andVII). The results revealed, for the non-U-COFT trained group, asharp division between those who were still in Reticle Aim Groups1-2 and those in Groups 3-6 (at retest). The majority of thosewho were still in the earlier matrix exercises showed a drop inTable VIII scores, whereas the majority of those who hadprogressed to Reticle Aim Group 3 and beyond showed gains inTable VIII scores. The authors conclude that exposure to U-COFTdoes not ensure skill retention and that the crews must makeprogress in the matrix. (Revised from Morrison et al., 1991;hard copy not procured)

21. Johnson, S. L. (1981). Effect of training device onretention and transfer of a procedural task. HumanFactors, 23, 257-272.

This experiment investigated the effectiveness of three differenttraining devices with respect to initial training, retention, andtransfer of learning. The training devices evaluated weredesigned to require varying degrees of visual imagery utilizationthrough reductions in the stimuli that provide visual cueing andfeedback. The procedural tasks were comparable to operating anindustrial plant's master control panel or an air vehicle. Theresults indicated that: (a) training devices do not need to havehigh fidelity in order to train procedural tasks, and (b)training strategies that require trainees to provide their owncueing and feedback from memory lead to increased retention (70days) of procedure-following skills. (Revised from documentabstract)

C-9

22. Jones, M. B. (1986). Nonimposed overpractice and skillretention (Research Note No. 86-55). Alexandria, VA:U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A168 572)

This project investigated the role of nonimposed overpractice inpredicting individual performance in skill retention. Noniuposedmeans that no external constraint or scoring convention preventsfurther improvement. Nonimposed overDractice refers to the shapeof individual performance curves late in acquisition. Flatterperformance curves late in acquisition depict the individual'slimits and outcomes of overpractice. Three experiments wereperformed. In Experiment I, 27 Navy enlisted volunteerspracticed each of six video-computer tasks for 15 daily sessions.Retention was measured after 4-6 months for two tasks, after 10-12 months for two tasks, and after 16-18 months for two tasks.It was found, in all three experiments, that nonimposedoverpractice improved retention. The more an individualpracticed, even though performance in acquisition no longerimproved, the better retention was likely to be. In addition,nonimposed overpractice even improved performance amongindividuals who performed well in acquisition, and would havebeen expected to perform better than others in retention.

23. Konoske, P. J., & Ellis, J. A. (1987). Cognitive factors inlearninQ and retention of procedural tasks (Tech. Rep.No. 87-14). San Diego, CA: Navy Personnel Research andDevelopment Center.

This project developed a taxonomy of qualitative explanations--linear, structural, and functional--for teaching procedural tasksand tested the effects of these explanations on learning, initialperformance, and retention of a procedural assembly task(constructing a model crane). Linear explanations providedstudent with information about what to do with the system or inthe situation. Structural explanations provided informationabout how and why the system/situation was constructed.Functional explanations provided information about how and whythe system/situation worked. Three experiments tested initiallearning and retention of the procedural assembly task. Theresults indicate that the advantages of qualitative explanationsdepend on the user's prior expertise and familiarity withassembling models.

24. Larson, J. A., Earl, W. K., & Henson, V. A. (1976).Assessment of U. S. tank crew traininq (TCATA TestReport FM 331). Fort Hood, TX, TRADOC Combined Arms TestActivity.

During a case study of a CONUS battalion, Table VIII gunneryresults were obtained at the completion of a formal gunnerytraining program, a tank degradation test, off-season firing, and

C-10

tank gunnery program. Even though many uncontrolled factors werepresent during this period, the results provided a generalassessment of crew gunnery performance at different times in thegunnery season. A significant degradation in off-season gunneryperformance was found. Mean first round firing times also showeda degradation during off-season firing, from 11.0 to 19.5 secondsfor day firing and from 12.6 to 20.4 seconds for night firing.The battalion experienced a good deal of turbulence during thestudy period, and it was not possible to determine the degree towhich degradation in performance was due to this factor as wellas to retention loss. The performance of 7 crews who were intactduring the one year time frame was approximately equal to thatfor crews experiencing turbulence. Thus, it appears that aformal gunnery program improves performance to the same generalextent for stable and turbulent crews. (Revised from existingin-house abstract; hard copy not procured)

25. Leonard, R. L., Jr., Wheaton, G. & Cohen, F. P. (1976).Transfer of training and skill retention (Tech. Rep. No.76-A3). Alexandria, VA: U. S. Army Research Institutefor the Behavioral and Social Sciences. (AD A036 059)

This experiment examined the retention of training coursematerial in a simulated operational situation and the effect ofrefresher training on transfer of hand grenade skills over a 17week period. Subjects were 150 enlisted Army personnel who weredivided into six groups. The intervals between training andtransfer (0, 6, or 17 weeks) and the intervals between refreshertraining (6 or 17 weeks) varied across groups. Training wasprovided by the Training Extension Course (TEC) materials onselection, maintenance, and the use of hand grenades. A "post-training" paper-and-pencil instrument consisting of 75 items wassynthesized from the testing materials provided with the TEClessons to assess retention. The post-test consisted of foursubtests: I--Selection of Grenades; II--Maintaining the HandGrenade; III--Arming the Hand Grenade; and IV--Throwing the HandGrenade. A fifth subtest involved Identifying Components of aHand Grenade. After a 6-week delay from initial training,subjects provided with refresher training outperformed thosehaving no refresher training on subtests I and II but not onsubtests III and IV. After 17 weeks subjects given refreshertraining outperformed unrefreshed subjects on subtests I, II, andIV. (Revised from document abstract)

26. MacPherson, D., Patterson, C., &, Mirabella, A. (1989).ApDlication of ARI skill retention model to wheel vehiclemaintenance tasks (Research Rep. No. 1538). Alexandria,VA: U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A219 684)

A Skill Retention Model has been developed which describes

forgetting behaviors for technical tasks in terms of the number

C-il

of steps in the task and the quality of the job aids. Sevensubject-matter experts (SMEs) regarding wheeled vehiclemechanical/instructors examined this model for nine wheel vehicletasks. The SMEs identified three out of nine tasks that wererapidly forgotten. The authors recommended ways to improvetraining and calculated the effects of provided training uponstudents' retention abilities. (Revised from ARI SR-li)

27. McDonald, R. D. (1967) Retention of military skills acquirodin basic combat training (Tech. Rep. No. 67-13).Alexandria, VA: Human Resources Research Organization.(AD 663 785) (ED 022 103)

Performance data were collected in the three general Basic CombatTraining (BCT) proficiency areas (rifle marksmanship, physicalcombat fitness, end-of-cycle tests) from independent groups ofsoldiers (60 per group) during BCT, Advanced Individual Training(AIT), and Combat Support Training (CST). The general militarysubjects measure was given as a paper-and-pencil test. All otherperformance measures were tested hands-on. Important findingswere that: (a) a significant performance decrement was found forthese skills across the year-long retention interval; (b) nosignificant decrement over the course of one year was evident forphysical combat proficiency; and (c) the mean percentage ofperformance measures passed fell from 86% to 69% over theretention interval.

28. Mengelkoch, R. F., Adams, J. A., & Gainer, C. A. (1971). Theforgetting of instrument flying skills. Human Factors,13, 397-405.

An aircraft simulator was used to assess the forgetting ofinstrument flying skills. There were 125 discrete proceduralitems per trial scored on a error/no-error basis. An error wasrecorded if the item was either omitted, performed incorrectly,or occurred out of place. Twenty-six ROTC male undergraduatestudents, naive to flying, participated in this study. After afour-hour academic training program and one familiarization trialin the simulator, Group 5 (N=13) received 5 trials duringoriginal training, resulting in intermediate proficiency. Group70 (N=13) received 10 trials during original training, resultingin high proficiency. The main findings were that discreteprocedural responses were more susceptible to forgetting thanflight control responses (maintaining altitude, bank, airspeed,level-off at altitude, roll-out on new heading). It was alsofound that the number of trials taken to relearn procedures wasless than the number during original training. This findingsuggests that flight training programs should focus on retrainingprocedures.

C-12

29. Mirabella, A. (1989). Contributions of the U. S. ArmyResearch Institute to the training technolo M fieldactivity-Aberdeen (Research Rep. No. 1536). Alexandria,VA: U. S. Army Research Institute for the Behavioraland Social Sciences. (AD A214 346)

This report summarized ARI's efforts to develop trainingguidelines for improved acquisition, retention, and transfer ofvehicle maintenance skills. The need for such guidelines cameafter previous studies showed very high failure rates for theperformance of expert and novice mechanics on tasks dealing withdiagnostics and removal/replacement of faulty parts. Forexample, the false removal rate (good parts mistakenly removed)was about 42% of all removals. The results of the TrainingTechnology Field Activity (TTFA) projects summarized includedvarions recommendations to improve the 63W10 course at theOrdinance School. One set of guidelines dealt with producingeffective review graphs for classroom instruction. Another setdealt with developing effective instructional principles.(Revised from ARI SR-i1)

30. Morgan, B. B., Jr., Lassiter, D. L., & Salas, E. (1989).Networked simulation application for team training andperformance research. Proceedinas of the Svmposium onInteractive Networked Simulation for Training (pp. 16-21). Orlando, FL.

This paper described a research program--its methodology andpotentidl applications--using interactive networked simulationprocedures. The authors noted that few studies have investigatedhow teams acquire, maintain, and lose teamwork skills. Inaddition, the authors discussed the development of a test-bed forthe study of team training and performance and the use ofSimulation Networking (SIMNET) as an existing simulation test-bedfor studies in this area.

31. Naylor, J. C., & Briggs, G. E. (1961). Long-term retentionof learned skills: A review of the literature (Tech.Rep. No. 61-390). Columbus, OH: Ohio State University,Laboratory of Aviation Psychology. (AD 276 043)

This review examined literature relating to flight skillretention. The authors noted that most of the experimentslocated dealt with verbal, rather than motor skill retention.The main results are that: (a) extra amounts of originallearning facilitate retention; (b) complex task performancedecays more than simple tasks over longer retention intervals;(c) continuous tasks are retained better than discrete orprocedural tasks; (d) task integration or organization may causethe superior retention associated with motor tasks; and (e)rehearsal facilitates skill retention. They concluded that many

C-13

of the studies reviewed lacked long enough retention intervals.(Revised from existing in-house abstract; hard copy not procured)

32. O'Hara, J. M. (1990). The retention of skills acquiredthrough simulator-based training. Ergonomics, 33(9),1143-1153.

This study examined merchant marine cadets' loss of watchstandingskills (standing on the bridge or deck to verify course steered,position, navigation, collision avoidance, equipment tests, andarrival preparation) across a 9-month retention period followinga simulator-based training program developed to enhance theseskills. The effects of retraining experience, as it influencedretention decay, were also evaluated. Two groups of merchantmarine cadets were tested immediately following the training andagain nine months later. One group was given a refresherexperience and tested 6 months into the retention interval. Theinstrument used to measure retention was composed of a series ofitems describing observable watchstanding behaviors. The resultsindicated that: (a) watchstanding skills improved followingtraining; (b) these skills declined over the nine month retentioninterval; and (c) retention decay was partially mitigated byrefresher training. (Revised from document abstract)

33. Palmer, R. L., & Buckalew, L. W. (1988). SINCGARS operatorperformance decay (Research Rep. No. 1501). Alexandria,VA: U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A210 716)

This study examined the decay of operational skills and knowledgein two groups of recently trained operators who went withoutexposure to Single Channel Ground and Airborne Radio (SINCGARS)for several weeks. Performance levels were measured with theSINCGARS Learning-Retention Test (SLRT), a simulated hands-onperformance test emphasizing skills and operational knowledgeretention. The results indicated that operators might lose about10 percent of their prior performance levels within the first fewweeks. However, this figure varied considerably, depending onthe type of soldier, the length of the non-exposure period, andother conditions. It was also found that performance level wascorrelated with soldiers' Armed Services Vocational AptitudeBattery (ASVAB) General Technical (GT) scores. Correlationsbetween GT and SLRT scores obtained at the different testingtimes were .43 and .50. A relationship, however, was not foundbetween performance decay and GT. (Revised from ARI SR-ll; hardcopy not procured)

C-14

34. Rose, A. M., Czarnolewski, M. Y., Gragg, F. E., Austin, S.H., Ford, P., Doyle, J., & Hagman, J. D. (1985).Acauisition and retention of soldering skills (Tech. Rep.No. 671). Alexandria, VA: U. S. Army Research Institutefor the Behavioral and Social Sciences. (AD A160 336)

This research investigated the acquisition and retention of fieldartillery tasks and the effects of overtraining, previoustesting, and soldiers' abilities on this task. It also developeda User's Decision Aid (UDA) based on ratings of taskcharacteristics that estimated retention functions for each task,and assessed the relationship between predicted and obtainedretention functions. One hundred and forty five Cannon Crewmen(NOS 13B10) were tested on 26 tasks. Each soldier was trained toproficiency on each task. After performing all tasks correctly,soldiers received mastery training (test-train-retest) for halfof the tasks. Of the five tasks, three--Boresight telescope,micrometer test, and sight target--were passed by less than 25%of the soldiers. Soldiers were most accurate on Crew tasks.Performance on Individual tasks varied. Retention tests wereadministered 2, 5, and 7 months after the acquisition phase.Almost all tasks showed forgetting after two months. After fivemonths all but one task showed considerable forgetting. Afterseven months soldiers performed better than after five months.The authors noted several confounding factors explaining thislast result. The UDA now exists in both paper and pencil and incomputer form and has been validated against existing retentiondata.

35. Rose, A. M., Radtke, P. H., Shettel, H. H., & Hagman, J. D.(1985). User's manual for predictinQ military task.retention (Research Product 85-26). Alexandria, VA:U. S. Army Research Institute for the Behavioral andSocial Sciences. (AD A163 710)

This manual represents an expanded version of a user's manual(ARI RP 85-13) for estimating proficiency on military tasks overperiods of no practice. The method requires that each task berated on how easy it is to remember. For practical purposes, themethod, developed for both paper-and-pencil and computer-basedformats, is geared toward helping trainers decide the: (a) tasksthat are most likely to be forgotten; (b) number of soldiers thatshould be able to perform a task correctly after given intervalsof no practice; and (c) frequency of sustainment training needed.

C-15

36. Schendel, J. D., & Hagman, J. D. (1980). On sustainingprocedural skills over prolonged retention intervals(Research Rep. No. 1298). Alexandria, VA: U. S. ArmyResearch Institute for the Behavioral and SocialSciences. (AD A120 758)

Task retention (M60 machine gun assembly/disassembly) wasexamined at two intervals (4 and 8 weeks), and between theoverlearning groups and the control (no overtraining) group. Theresults showed that soldiers forgot as much after four weeks asafter eight weeks. The data also indicated that the scores forthe two overtrained groups' scores were equivalent to each otherand substantially better than the control group's. Hence,retraining after four weeks was equivalent to providing initialovertraining.

37. Schendel, J. D. & Hagman, J. D. (1982). On sustainingprocedural skills over a prolonged retention interval.Journal of Applied Psychology, 67, 605-610.

This research assessed three alternative approaches forsustaining procedural skills over a prolonged retention interval.It also sought to determine if soldiers could estimate the amountof refresher training that they needed to regain proficiency.Soldiers (N=38) were assigned to groups according to pastexperience on the experimental task--disassembly/assembly of theM60 machine gun. The first group was trained to criterion andthen given 100% initial overtraining. This group's 8-weekretention and retraining performance was compared with a groupreceiving the same amount of additional training midway throughthe retention interval. The control group was trained tocriterion but received no additional training prior to theretention testing and retraining. Refresher training estimateswere collected immediately prior to the retention test. Cost andeffectiveness considerations heavily favored the overtraininggroup. In addition, most soldiers accurately assessed the amountof refresher training they required to regain proficiency.(Revised from document abstract)

38. Schendel, J. D., Morey, J. C., Granier, M. J., & Hall, S.(1983). Use of self-assessment in estimating levels ofskill retention (Research Rep. No. 1341). Alexandria,VA: U.S. Army Research Institute for the Behavioral andSocial Sciences. (AD A141 042)

This research examined self-assessment as one method forpredicting task retention and refresher training requirements.Before zeroing their M16Al's for annual rifle qualification, 147male and 6 female soldiers completed a questionnaire designed tocollect information about prior marksmanship experience andperceptions about how they were going to shoot at record fire.The main findings were that subjects' self-assessments were a

C-16

poor method for predicting task retention scores as they onlypredicted approximately 10% of the variance in the retentiondata. Also, most subjects tended to overestimate the success oftheir performance.

39. Schmidt, R. A., & Shapiro, D. C. (1988). Optimizingfeedback utilization in motor skills training (ResearchNote No. 88-05). Alexandria, VA: U. S. Army ResearchInstitute for the Behavioral and Social Sciences. (ADA191 559)

This project examined the use of feedback (i.e. knowledge ofresults (KR)] on goal achievement for the maximization of skilllearning and retention. Two variations of KR were studied: (a)relative frequency (the proportion of trials receiving KR) and(b) summary (the entire set of trials) KR. In each case,alterations in KR which degraded training performance actuallyproduced enhanced performance on a delayed retention test when KRwas not presented. These experiments suggest that enhanced KR inacquisition may generate over reliance by the learner on KR,which would prevent the learner from retaining important featuresof the task when feedback is removed or degraded (e.g.marksmanship). (Revised from ARI SR-il; hard copy not procured)

40. Schrenk, L. P., Daniels, R. W., & Alden, D. G. (1969).Study of long-term skill retention. NAVTRADEVCEN 18224,Contract N61330-1822, Honeywell, Inc. (AD A692 770)

This project sought to determine how individual and team skillsinvolved in anti-submarine warfare (ASW) degraded over time andwhich form of refresher training was most effective. Phase Icompared ASW skill retention between teams that had receivedshore based simulator training either 0, 4, 8, or 16 weeksearlier. Major performance measures were time and number ofshots required to hit a target. The main findings of Phase Iwere that: (a) crew performance markedly degraded over the 16week period; and (b) initial decrement slowed performance,resulting in subsequent losses in procedural skills. Phase IIexplored the effect of full-team refresher training. Some groupsreceived part-task refresher training and a control groupreceived no refresher training. All groups were retested afterinitial training and retested on a parallel form 16 weeks later.It must be noted that Phase I and II of this study were to beconducted on a "non-interfering" basis with standard navaltraining and operations. This mandate jeopardized the resultsbecause of a 35% team member turnover, instructor turnover, teamsnot receiving refresher training or retention testing at thescheduled times, and instructor/monitor overloads and errors inrecording team member responses. The authors concluded that partteam training was inferior to control group and full teamrefresher training, and procedural skills exhibited greatestdegradation.

C-17

41. Shields, J. L., Goldberg, S. L., & Dressel, J. D. (1979).Retention of basic soldierinQ skills (Research Rep. No.1225). Alexandria, VA: U. S. Army Research Institutefor the Behavioral and Social Sciences. (AD A075 412)

One hundred and eighty-two soldiers were tested on 20 commonsoldier procedural tasks that included: reporting enemyinformation, loading and firing grenade launcher, donning gasmask and learning cardiopulmonary resuscitation (CPR). Threehundred and forty-one soldiers were tested on the same tasksafter 5 to 12 months in a unit. Retention intervals ranged from4-12 months following initial basic training for 523 soldiers.The main finding was that soldiers forgot steps not cued by theequipment or by the previous steps, such as those involvingsafety.

42. Shields, J. L., Joyce, R. P. & Van Wert, J. R. (1979).Chaparral skill retention (Research Rep. No. 1205).Alexandria, VA: U. S. Army Research Institute for theBehavioral and Social Sciences. (AD A069 945)

This project evaluated retention of Chaparral skills to determinethe most effective schedule of refresher training. Soldiers weretested immediately after Advanced Individual Training (AIT) onseveral Chapparal tasks (pre-energizing, energizing, and de-energizing the launch station; before-operations performancemeasure checks on the M730 carrier; installing and operating afield telephone; and performing emplacement and operator checksand adjustments on the target alert data display set), andretested upon crrival in their battalions, and again 4 monthslater. Comparison groups were tested after one or two-monthretention intervals. Results showed that performance generallydid not decline. Authors attributed this finding to the use ofjob aids that helped the soldiers.

43. Singer, R. N., Gerson, R. F., & Ridsdale, S. (1979). Theeffects of various stratecies on the acquisition,retention, and transfer of a serial positioning task(Tech. Rep. No. 399). Alexandria, VA: U. S. ArmyResearch Institute for the Behavioral and SocialSciences. (AD A081 995)

This project analyzed the effectiveness of various learnerstrategies upon initial learning, retention, and transfer of amotor skill. Male and female undergraduates were randomlyassigned to one of five strategy conditions: imagery,kinesthetic, labeling, informed-choice, and control. The task,using a curvilinear repositioning apparatus, requiredparticipants to replicate six limb movements to predeterminedlocations. Following the learning trials, subjects were given aretention test and transfer task involving six new positions.Results indicated that "imagers" were more accurate in their

C-18

responses than the four other groups. Control subjects were moreaccurate and less variable than either the kinesthetic, labeling,or informed-choice groups.

44. Singer, R. N., Ridsdale, S., & Korienek, G. G. (1979). Theinfluence of learning strategies in the acquisition,retention. and transfer of a procedural task. (Tech. Rep.No. 408). Alexandria, VA: U. S. Army Research Institutefor the Behavioral and Social Sciences. (AD A086 405)

This project analyzed and compared the effectiveness of differentlearning strategies on a sequential procedural task. Subjectswere randomly assigned to one of five conditions: imagery,chunking, verbalization, informed-choice, and control. The taskrequired participants to manipulate a sequence of buttons andswitches on a computer-managed task apparatus. Results indicatedthat "imagers" performed better on acquisition and transfer thanthe chunking, verbalization, or informed-choice groups.

45. Thompson, T. J., Morey, J. C., Smith, S. & Osborne, A. D.(1981). Basic rifle marksmanship skill retention:Implications for retention research (Research Rep. No.1326). Alexandria, VA: U. S. Army Research Institutefor the Behavioral and Social Sciences. (AD A134 017)

Three experimental Basic Rifle Marksmanship (BRM) trainingprograms were developed and compared with the current program.All subjects (N=3.151) fired a Record Qualification-Course whichwas the standarL_ training performance exam. All availablesubjects (N=388) were retested approximately 6 weeks aftercompletion of the BRM. The record fire scores of experimentalprograms were significantly better than those of the standardprogram in the initial qualification training.

46. Vineberg, R. (1975)..A study of the retention of skills-andknowledge acauired in basic training (Tech. Rep. No. 75-10). Alexandria, VA: Human Resources ResearchOrganization.

This study assessed retention of basic combat skills. Thirteensubtests of the Comprehensive Performance Test (CPT) wereadministered to soldiers during the last week of BCT and againsix weeks later during AIT. Results indicated that theprobability of the average soldier passing a CPT subtest: (a) atthe end of BCT was .81; (b) during retention testing 6 weekslater was .63; and (c) at the end of basic training was .55.Proficiency decreased between 18% to 26%. These decrements inperformance varied across subtasks and mental categories.(Revised from in-house abstract; hard copy not procured)

C-19

47. Wisher, R. A., & Sabol, M. A. (1990). Predicting decay ofMobile Subscriber Equipment (MSE) operator skills.Proceedings of the 17th Army Science Conference, Durham,NC.

This project described the initial stages in developing a modelto predict the decay of complex skills [operation and maintenanceof the Mobile Subscriber Equipment (MSE) communication system].The authors hypothesized that conceptual understanding of asystem is the key to retention of skills over extended intervalsof non-use. Phase I involved application of current skillprediction techniques (Rose, et al., 1985) to 85 MSE skills. AnMSE Knowledge Test was constructed that assessed active andreserve soldier's conceptual understanding of the system. Themaximum score of active and reserve soldiers on the MSE test,taken 90 days after formal training, were 73% and 65%,respectively. Results indicated that the Switch Operators (31F)scored higher than the Transmission Operators (31D) on all of thesubtests.

Note: Phase one of this study is presented in:

Sabol, M., Chapell. L, & Meiers, C. (1990). Predicted decay ofMobile Subscriber Ecuipment (MSE) operator skills (ResearchProduct 90-11). Alexandria, VA: Army Research Institute forthe Behavioral and Social Sciences.

48. Wright, R. H. (1973). Retention of flying skills andrefresher training reguirements: Effects of nonflyingand proficiency flying (Tech. Rep. 73-32). Alexandria,VA: Human Resources Research Organization.

This project investigated the effects of periods of little or noflying on flying skills of Army Aviators. A survey questionnairewas submitted to Army Aviators who had experienced extendedperiods of flying excusal or proficiency flying status. Fifty-eight usable questionnaires were obtained from aviators withextended flying excusal, and 117 from aviators who had flown theminimum required to retain flying status. The responses providedjudgements on losses in visual flying rules (VFR) and instrumentflying rules (IFR) skills over varying periods of time and inrelation to whether or not proficiency flying was performed.Results indicated that flying minimums slightly reduced the rateand amount of reported skill loss with the effects morepronounced in the IFR than VFR. It was also found that thedegree of refresher training required to regain minimal orsufficient skills to be pilot-in-command was a negativelyaccelerated positive function of the nonflight interval until 16months where the curve flattened.

C-20

Appendix D

Miscellaneous References

The following documents that are not included in theannotated bibliography, but may be of some interest and use toresearchers and trainers.

Baker, R. A. & Winograd E. (1957). Retention of basic combatskills by active duty personnel. Alexandria, VA: HumanResources Research Organization.

Bellezza, F. S. (1981). Mnemonic devices: Classification,characteristics, and criteria. Review of EducationalResearch, A, 247-275.

Bodily, S., Fernandez, J., Kimbrough, J., & Purnell, S. (1986).Individual Ready Reserve (IRR) skill retention and refreshertraining options (Rand Note N-2535-RA). Santa Monica, CA:Rand Corporation.

Boldivici, J. A., Bessemer, D. W., & Haggard, D. F. (1985).Review of the Ml Unit-Conduct of Fire Trainer (U-COFT)validation and verification test report (Research Note No. 85-

56). Alexandria, VA: Army Research Institute for theBehavioral and Social Sciences. (AD A173 938)

Fleishman, E. A. & Parker, S. F. (1962). Factors in theretention and relearning of perceptual-motor skill. Journalof Experimental Psychology, E4, 215-276.

Gagne, R. M. (in press,1987). Research on learning and retainingskills (AFHRL Tech. Rep.). San Antonio, TX: Air Force HumanResources Laboratory.

Hagman, J. D. (1983). Presentation and test-trial effects onacquisition and retention of distance and location. Journalof Experimental Psychology: Learning. Memory. and Cognition,.9, 334-345.

Hagman, J. D., & Brosvic, G. M. (in press). Effects of repeatedtesting on acquisition and retention of motor skills (Tech.Rep.). Alexandria, VA: U. S. Army Research Institute for theBehavioral and Social Sciences.

Hagman, J. D., & Wells, R. (in press). Effects of refreshertraining session freguency and temporal separation onretention and relearning (Tech. Rep.). Alexandria, VA:U. S. Army Research Institute for the Behavioral and SocialSciences.

D-1

Hall, E. R., Willis, R. P., Thompson, J. R., & Schalow, S. R.(1986). Skill deterioration and retraining needs of Navy

individual ready reserve (Tech. Rep. No. 86-007). Orlando,FL: Naval Training Systems Center.

Holding, D. H. (1965). Principles of Training. New York:Pergamon Press.

Holding, D. H. & Collins, J. W. (In publication) Intrinsic cuesand knowledae of results. Manuscript submitted forpublication.

Johnson, S. L. (January 1978). Retention and transfer oftraining on a procedural task: Interaction of trainingstrateay and cognitive style (Calspan Rep. No. DJ-6032-M-1).Buffalo, NY: Calspan Corp.

Knerr, C. M., Berger, D. C., & Popelka, B. A. (1980). Sustainingteam performance: A systems model (Final Report).Springfield, VA: Litton Mellonics.

Kulhavy, R. W. & Stock, W. A. (1989). Feedback in writteninstruction: The place of response certitude. EducationalPsychology Review, 1, 279-308.

McCluskey, M. R. & Schmidt, S. A. (1978). Skill decay of sixteencommon tasks for MOS 11B and 11C (Final Report 78-13).Alexandria, VA: Human Resources Research Organization.

Roth, J. T. (1992). A taxonomy for predicting team and collectivetask performance change (Research Note No. 92-35).Alexandria, VA: Army Research Institute for the Behavioraland Social Sciences. (AD A251 568).

Sticha, P. J. & Knerr, C. M. (1983). Task-element and individualdifferences in Procedural learning and retention: a modelbased analysis (DDI/PR 83-1-334). Decisions and Designs,Inc., Final Report.

Sullivan, D. J., Casey, R. J., & Hebin, J. M. (October 1978).Acauisition and retention of cognitive versus perceptuallyoriented training materials (Tech. Rep. Draft). Alexandria,VA: U. S. Army Research Institute for the Behavioral andSocial Sciences.

Training effectiveness analysis (TEA). Volume I (Armor). FortBelvoir, VA. Armor Training Study, 1978.

Trainina effectiveness analysis (TEA). Volume III (REDEYE WeaponSystem. Fort Belvoir, VA. Armor Training Study, 1978.

D-2

t II I

Training effectiveness analysis (TEA). Volume IV (Ordinance).Fort Belvoir, VA. Armor Training Study, 1978.

Wetzel, S. K., Konoske, P. J., & Montague, W. E. (1983).Estimating skill degradation for aviation antisubmarinewarfare operators (AWs): Loss of skill and knowledge followingtraining (NPRDC SR-83-31). San Diego, CA: Navy PersonnelResearch & Development Center. (AD A129 407)

D-3

Appendix E

Corporate Bibliographies

Army Research Institute. (1980). List of research publications:1976-1980 (supplement). Alexandria, VA: Author.

Army Research Institute. (1981). List of research publications:1940-1980. Alexandria, VA: Author.

Army Research Institute. (1986). List of research publications:FY80-FY86. Alexandria, VA: Author.

Army Research Institute. (1990). List of U. S. Army ResearchInstitute Research and Technical Publications; January 1.1986. to September 30. 1989 (Special Report No. 11).Alexandria, VA: Author.

Army Research Institute. (1992). List of U. S. Army ResearchInstitute Research and Technical Publications: October 1.1990, to September 30, 1991. Alexandria, VA: Author.

Barlow, E. M. (Ed.). (1987). Annotated bibliography of the AirForce Human Resources Laboratory technical publications: 1981through 1985 (Tech. Rep. No. 86-73). Brooks Air Force Base,TX: Air Force Human Resources Laboratory, Technical ServicesDivision.

Human Resources Research Organization. (1971). Bibliography ofpublications as of 30 June 1971. Alexandria, VA: Author.

Human Resources Research Organization. (1977). Bibliography ofpublications and presentations during FY 1972-77. Alexandria,VA: Author.

Moses, A. M. (1981). Report of basic research publicationsproduced since June 1976 by the Army Research Institute.Alexandria, VA: U. S. Army Research Institute for theBehavioral and Social Sciences.

E-I

Appendix F

Document Reproduction Services

Several of the references cited can be obtained from thefollowing document services.

Defense Technical Information CenterCameron StationAlexandria, VA 22304-6145202-274-6434; Autovon 284-6434

National Technical Information Service5285 Port Royal RoadSpringfield, VA 22161703-487-4650

ERIC Document Reproduction Service7420 Fullerton, RoadSuite 110Springfield, VA 22153-28521-800-443-3742

F-1 930409