A practitioner's guide for designing performance support systems

The voice on the other end ofthe phone line says, “I believemy customer’s problem couldbenefit from a performance

support system. Unfortunately, I’m notsure where to begin. Can you help?”

As organizations seek to reduce the timeand cost associated with training, thistype of phone call has become increas-ingly common. Organizations are lookingmore toward noninstructional interven-tions, like electronic performance supportsystems (EPSS), to provide employeeswith the information they need and to do so in increasingly faster ways. Unfor-tunately, despite the potential for EPSSto mitigate performance problems, manypractitioners are not sure which systemswould be best suited to solving their current issue or sometimes even where to start.

The general human performance tech-nology (HPT) model available on the ISPIwebsite (http://www.ispi.org/services/whatshptmodel.pdf) provides practition-ers with a process to analyze performanceproblems and select the appropriate inter-vention. However, once an intervention isselected, the HPT model typically relieson the processes, technologies, and bestpractices commonly used in each particu-lar domain to guide the design anddevelopment of the selected intervention.For instance, when training is selected aspart of the intervention portfolio, a practi-tioner may rely on the ADDIE model oron other competing models from theinstructional design domain.

When it comes to electronic performancesupport, no clear and concise model toguide the practitioner currently exists.Models from other domains could cer-

tainly be applied to EPSS. For instance,Witt & Wager (1994) compared the EPSSdesign process to the general ADDIEmodel used in instructional design.Raybould (2000) introduced the Perfor-mance Support Mapping methodology.Although this approach is quite robustand heavily emphasizes analysis, map-ping, and carefully designed performancesupport interfaces, it is unfortunately notyet widely known or adopted by practi-tioners. As Fischer and Horn (1997) noted,“without tools that are primarily EPSStools and with no clear methodology forbuilding them . . . EPSS will be limited toan approach.”

This article addresses this perceived gap.Figure 1 illustrates a model that theauthors have developed, applied, andrefined over the course of working withvarious customers and performanceproblems at a Fortune 100 company. Thegoal of this model is not to be compre-hensive and all encompassing. Instead,the intent is to provide a model thathuman performance technologists canreadily apply in the design and develop-ment of performance support systems.Where possible, actual instruments andtemplates (identified in Figure 1 withshaded boxes) will be provided so thatpractitioners can use and adapt thesetools to their particular organization’sneeds and performance problems.

A Practitioner’s Guide for DesigningPerformance Support Systemsby Frank Nguyen and Craig A. Woll

37Performance Improvement, vol. 45, no. 9, October 2006

© 2006 International Society for Performance ImprovementPublished online in Wiley InterScience (www.interscience.wiley.com) • DOI:10.1002/pfi.019

Phase 1: Performance Analysis

Conduct HPT Analysis

In the first phase of solving any human performance prob-lem, a performance analysis should be completed. Somecommon tools for analyzing performance include organiza-tional, environmental, workflow, task, and gap analyses.The appropriate combination of these analyses will paint aclear picture of the performance problem to be addressed.Because this article is focused on performance support sys-tems, we will not attempt to address the general topic ofperformance analysis in detail. More information on thistopic can be obtained from the ISPI HPT Institute (seehttp://www.ispi.org/hpt_institute) or publications such asFirst Things Fast (Rossett, 1998).

Select EPSS Intervention

According to Gilbert’s (1978) Behavior Engineering Model(BEM) the root cause of a performance problem falls intoone or a combination of six categories: data, resources,incentives, knowledge, capacity, and motives. Chevalier(2003) updated the language of the Gilbert model to betterreflect the terminology used in today’s performance analy-sis, as is shown in Table 1. Placing the root cause(s) intothese categories simplifies the intervention selection pro-cess by clearly articulating the type of problem that has beenencountered. An EPSS intervention is most commonly asso-ciated with solving problems with a root in the informationcategory but can also be used to supplement interventionsin any of the other five categories.

Carr (1992) mentions five guidelines for the selection of anEPSS as an intervention: (1) “Skilled performers spend sig-nificant amounts of time helping and correcting unskilled

performers”; (2) “A lot ofdocumentation exists, sothat employees must doextensive searches in orderto find the right informa-tion”; (3)“Neophyte workersmust begin to perform effec-tively and training is eitherimpractical or unavailable”;(4) “Individual technicians,specialists, managers, orother performers need to beguided through complexprocesses”; (5) “Teams oftechnicians, specialists,managers, or other perform-ers need to be guidedthrough a complex process”(p. 37). Ladd (1993) devel-oped a checklist like the oneshown in Table 2 to assist in

determining if EPSS was right for the performance problemidentified in the analysis.

Check the items that apply to your organization. Your orga-nization is a candidate for the use of electronic performancesupport if you check three or more items in the first sectionor four or more items in the second section.

EPSS are best suited for tasks that are not done frequently.They are most effective when they are implemented in thecontext of the work itself. Research shows that adult learnersretain information better when it is related to their personalexperience and is applied to a context with which they arefamiliar (Knowles, 1984; Caffarella, 1994; MacKeracher,1996; Daley, 1998). EPSS are also a better fit with noncriticaltasks and objectives and can also be associated with tasks orprocesses that are not frequently performed. The selection ofan EPSS as an intervention to solve a problem or need iden-tified in a performance analysis is more likely to succeedwhen the human performance technologist adheres to thesimple principles previously stated.

Measure Performance Problem

The time and cost associated with some interventions, likeEPSS, often require strong management and organizationalsupport. One of the best ways to ensure that the project isproperly funded and supported is to identify key perfor-mance measures against which the success of theintervention can be measured. These measures can be financial(improved sales, reduced operating costs), organizationalperformance (employee retention or satisfaction), or indi-vidual performance (reduced errors, decrease in workflowcycle time). Ideally, the measures you choose should beones that are directly related to the performance problem

38 www.ispi.org • DOI:10.1002/pfi • OCTOBER 2006

Figure 1. A Practitioner’s Model for Designing EPSS.

you are trying to solve, that affect the organization in ameaningful way, and that are valued by the customer. Abaseline of these measures should be taken during Phase 1to allow you to calculate the impact that the EPSS interven-tion has made after implementation. Start by identifying thekey indicators and then measure them longitudinallythroughout the process, beginning prior to the implementa-tion of the intervention.

Webb (1999) identified four key factors in calculating theROI of a system: (1) isolating your data from other variables,(2) limiting the number of key metrics to avoid saturation,(3) converting the data collected into money, and (4) mea-suring the same indicators before and after the project. Themetrics selected can measure value in a number of ways, butthe most useful is generally a financial indicator shown indollars. Some examples of indicators are displayed in Table 3.

39Performance Improvement • Volume 45 • Number 9 • DOI:10.1002/pfi

Table 1. Updated Behavior Engineering Model. Source: Adapted from Gilbert, 1978, p. 88, by Chevalier, 2003, p. 9.

Table 2. Is Performance Support Right for Your Organization? Source: Adapted from Ladd, 1993, p. 24.


Table 3. Examples of EPSS Indicators. Source: Adapted from Hawkins, Gustafson, & Nielsen, 1998, pp. 19-20.

The final metrics selected should be understood and ratifiedby the key stakeholders of the project.

Phase 2: EPSS Analysis

Once EPSS has been selected as an intervention it is impor-tant to conduct a more specific needs assessment directlyrelated to the EPSS. This assessment will provide criticalinformation that is a key to the high-level design carried outin Phase 3. This assessment information is also important inpinpointing the exact need of the customers to guaranteethat the EPSS is properly outfitted to mitigate their perfor-mance problem.

The data for the EPSS analysis can be gathered through sur-veys, observations, collection of the company’s operationaldata, or any of a number of data gathering methods. Thisarticle will make one recommendation of how the analysiscan be administered. Customization of this method is rec-

ommended to match unique situations. Table 4 provides anexample of a simplified instrument for gathering the assess-ment data. A precursor survey may be administered prior tousing this instrument to understand the general audience.Once again, timeliness of the assessment, rollup, and analy-sis completion is important in maintaining a qualityrelationship with your customer.

Conduct Quantitative Assessment

The sample size for the assessment should be sufficient toreveal the need clearly but not so large that the data rollupbecomes overbearing. We recommend limiting the sample tono fewer than 10 and no more than 30 performers. Thismeans that for large populations the sample should be care-fully selected to be random yet representative of the targetaudience. The job roles of the individuals selected in thesample may include end users, managers, peer trainers, andothers, depending on the intended use of the system.

Conduct Qualitative Assessment

Data collected as part of the quantitative needs assessmentcan often reveal worker preferences, environmental condi-tions, and trends. However, data do not typically explainwhy these conditions may exist. To better understand theseresults, a performance technologist can use qualitativeresearch methods. This research can be easily conductedusing interviews, focus groups, job observations, or othermethods. This follow-up is a deterrent to costly defects thatcan occur in the development phase if the design does notmatch the true user need.

When the final assessment is complete and the data havebeen placed in a meaningful summary document, thatassessment should be validated by the key stakeholdersprior to moving into the succeeding phases of the project.The metrics collected during Phase 1 should also bereviewed as a reminder of what the project outcomes will bemeasured against.

Phase 3: EPSS Design

After collecting the requisite analysis information, the nextphase in the performance support design process is to usethese data to inform the design of the EPSS. As shown in

Figure 1, the design phase includes four steps: selecting thetype(s) of EPSS to deploy, designing a high-level architec-ture, developing a detailed design, and validating thesedesign products with the customer.

Select EPSS Type

Arguably the most important yet most often neglected stepin the EPSS design process is the careful and deliberateselection of the appropriate type of performance supportsystem. It is also important to note that in certain circum-stances, human performance technologists may choose toimplement more than one type of system. Although moreresearch needs to be conducted, some guidelines and empir-ical studies currently exist to guide practitioners in thisselection process.

Table 5 defines the three categories of performance supportsystems proposed by Gloria Gery (1995). These three typesof EPSS differ in the level of integration between the sup-port system and the users’ work interface. For instance,external systems have minimal integration and thereforerequire the learner to stop what he or she is doing, findinformation in the EPSS, learn it, and then return to the taskat hand. In contrast, intrinsic systems are so integrated intothe work interface itself that users do not have to interrupt


Table 4. Simplified EPSS Needs Assessment Template.

their workflow to learn. “They simply feel that they are justdoing the work” (Gery, 1995, p. 51).

Gery implicitly suggests that the more integrated a perfor-mance support system is to the work interface, the better it is.In fact, she provides practitioners with the goal of integrating“as much as 80% of the required performance support asintrinsic,” with the remaining percentage equally allocated

between extrinsic and external solutions (Gery,1995, p. 53). Raybould (2000) confirms thisnotion by arguing that “[a]s support moves fur-ther from the tool and requires more time offthe job it becomes less powerful” (p. 35). Theseassertions have been partially validated bymore recent empirical research by Nguyen,Klein, and Sullivan (2005). They found thatusers provided with intrinsic and extrinsic per-formance support performed significantlybetter on a software procedure than a controlgroup that was not provided with an EPSS. Inaddition, the intrinsic users accessed theirEPSS 1.5 times more than the control groupdid, whereas the extrinsic users accessed theirEPSS 3 times more often.

However, integrated performance support solutions likeintrinsic and extrinsic systems may not be possible in allcircumstances or may prove too costly to build. For exam-ple, the root cause of your customer’s performance problemmay be related to an off-the-shelf software application.Because the software was purchased from a vendor, theinterface may be compiled in such a way as to preventmodifications required by an integrated EPSS. In addition,

certain workers, such as thosein factories, warehouses, orrepair centers, perform tasksthat are not primarily computerbased. In fact, they may nothave immediate access to com-puters, making it inconvenientor impossible for them to useelectronic forms of perfor-mance support. As noted byRaybould (2000), “when a par-ticular [EPSS] proves infeasible,”practitioners may need to lookat less-embedded support sys-tems (p. 35).

Computer technologies and HPTinterventions have advancedtremendously since Gery firstintroduced her three EPSS cate-gories. Figure 2 provides anupdated taxonomy that illus-trates just some of the perfor-mance support systems thatpractitioners have available tothem today. Again, this taxon-omy is not meant to be anexhaustive list but a frameworkthat illustrates some technolo-gies that can be leveraged forperformance support purposes.


Table 5. Types of Electronic Performance Support Systems. Source: Adapted from Gery,1995, p. 51.

Figure 2. Taxonomy of Performance Support Systems.

As you can see, performance technologists now enjoy moreEPSS options for mitigating a performance problem thanthey did 15 years ago. To make sense of these choices, aneeds assessment was recently conducted in conjunctionwith ASTD Benchmarking Forum companies. This studyprovides insight into the particular types of EPSS that corpo-rate employees perceive as useful and potentially effective(Nguyen, 2005). As noted earlier, though, selecting the mosteffective EPSS for a given performance problem may be dif-ficult due to the current lack of more substantive research.

Develop High-Level Architecture

Once one or more types of performance support systemshave been selected, the next step in the process is to developa high-level architecture. Figure 3 shows the basic compo-nents that need to be considered when designing an EPSS:the performance problem, the end user, the type of elec-

tronic device (if any) theindividual can use to accessthe performance support sys-tem while on the job, theEPSS or work interface thathe or she will use to accessthe support content, and thedatabases or content reposi-tories he or she may haveaccess to from the EPSS.This architecture templatemay be downloaded, andthen adapted for your use,from http://www.frankn.net/epss/architecture.zip.

The information from theanalysis phase can be com-bined with the EPSS selectionto complete this architecturediagram. In particular thequantitative data should informthis step. You will notice thatthe items from the needsassessment instrument inTable 4 map directly to thecomponents of the archi-tecture diagram. Figure 4presents an example of anEPSS architecture. Thisdesign addresses an equip-ment repair problem wheretechnicians had accessonly to computer kiosks. Toimprove the technicians’ability to access contentwhile working on equip-ment, the architecture calls

for the adoption of wireless tablet computers. The needsassessment also revealed that the technicians wanted accessto manuals and training materials as well as the ability toshare their knowledge among peers. As a result the high-level design calls for an EPSS that can access a learningcontent management system (LCMS) and knowledge man-agement (KM) system.

Upon completing an initial version of the architecture, it isnot unusual to change the selected performance supportsystems. For example, the needs assessment data may sug-gest that users require access to more content repositoriesthan the work interface can realistically support. Toaccommodate this, a practitioner may choose to integratecertain content intrinsically or extrinsically in the workinterface. The remaining content may be accessed exter-nally through a list of frequently asked questions (FAQs)or a search engine.


Figure 3. Simplified EPSS Architecture Template.

Figure 4. EPSS Architecture Example.

Technicianencounters a repair error

code

Computerkiosk

ExternalEPSS:Search

engine forerror codesdisplayed byequipment

Wirelesstablet PC

VendorEquipmentManuals

TrainingLCMS

KnowledgeMgmt.

System

Develop Low-Level Design

The next step in the process is to define in more detail thedesign of the performance support system. Software devel-opers often refer to this step as low-level or detailed design.Because low-level design tends to be specific to a work inter-face and EPSS type, it is not possible in this article toprovide a generic framework or template for this activity.Nevertheless the goal of this activity remains the same in allsituations: develop a design of sufficient detail that a perfor-mance support development team can build or purchase asystem that will address the identified performance problem.

Validate with Customer

As with all the phases in the EPSS design model, the perfor-mance technologist should validate the EPSS selection andthe high-level and low-level design products with the cus-tomer before moving to the next phase.

Phase 4: EPSS Development

Once the design for the EPSS has been completed, the nextphase in the performance support design model is to use thehigh-level architecture and low-level design to build anEPSS system, purchase an off-the-shelf EPSS tool that meetsthe design requirements, or adapt an existing EPSS systemto the current performance problem. As shown in Figure 1,the development phase includes at least four steps: develop-ing or purchasing the EPSS, developing support content tobe accessed from the EPSS, integrating the EPSS into theusers’ work interface, and, of course, validating the devel-oped performance support system with the customer.

Develop or Purchase EPSS

Where possible, it may be more cost effective to purchase vendor-developed software packages to support your perfor-mance support system design. A relatively comprehensive listof EPSS tools is available from EPSScentral (at http://www.epsscentral.info/knowledgebase/desdev). However, it is quitecommon that performance support designs call for an EPSSthat does not currently exist or that cannot be met by off-the-shelf software. This circumstance is particularly likely forhighly integrated performance support, as found in intrinsicsystems. In these instances, organizations are forced to pursuea custom-developed performance support system.

Develop Content

Because extrinsic and external performance support sys-tems rely on content stored outside the work interface, itmay be necessary to develop new content to support userson the job, and store it in a database or repository. To opti-mize project timing and resources, this step could be donein parallel with the preceding EPSS development step.

In certain situations it may also be possible to leverage con-tent stored in other systems in the environment. Forinstance, if the performance problem being addressed iscaused by lack of knowledge or discipline in following busi-ness processes, one could look to workflow diagrams orother business process engineering documents. If the perfor-mance problem is related to software procedures, one couldlook to vendor-developed manuals, job aids, or simulations.Furthermore, a growing trend in the training world is theadoption of an LCMS. These systems allow training devel-opers to chunk e-learning and instructor-led content intoreusable learning objects (RLOs). Although the originalintent of these objects is to be combined and reused fortraining offerings, they can also be leveraged as content forperformance support systems. In this way performance sup-port content can be introduced and updated as trainingcontent is created, thereby reducing overall developmentand maintenance costs.

Integrate into Work Interface

In intrinsic and extrinsic EPSS designs, it is necessary tointegrate support content directly into the users’ work inter-face. It is sometimes helpful to consult with a usabilityexpert, human factors engineer, or industrial engineer. Bydoing so the human performance technologist can determinepotential problem areas, opportunities for interface redesign,and other strategic locations in the work interface to inte-grate performance support content. Even in externalperformance support designs, it may be helpful to provide alink in the users’ work interface to the EPSS. This is oftenseen in software applications that provide a Help button thatlaunches an external search engine, FAQ page, or help index.

Validate with Customer

As in the analysis and design phases the performance tech-nologist should validate the developed or purchasedperformance support system with the customer. This stepmay involve a number of tests common in the softwaredevelopment world, such as functional testing, performancetesting, and user acceptance testing. Some of these tests mayoccur during the EPSS development phase, and others mayoccur at the conclusion.

Phase 5: EPSS Implementation, Evaluation

The implementation and evaluation of the EPSS occurs fol-lowing the development. In reality, the foundation for bothimplementation and evaluation are laid long before the endof the project. The successful adoption and validation of theusefulness of the system is contingent on tying the previousphases together in this phase. In Phase 1 the metrics thatwill be referenced in the evaluation were created and vali-dated by the stakeholders. In Phase 2 the population thatwould use the EPSS as an intervention was identified and


an assessment of its needs was performed. A sample fromthat audience was carefully selected to represent the popu-lation. This same sample, or a small audience of choice, waslikely used in the functional testing of Phase 4. During theimplementation portion of Phase 5, emphasis must beplaced on communication, training, change management,support, and marketing of the system. Each componentplays an important role in the adoption of the performancesupport system by employees.

Although it is possible and potentially valuable to evaluatethe EPSS intervention using Kirkpatrick’s Level 1-4 frame-work, we believe it is particularly important to determinethe direct impact of the EPSS intervention on the perfor-mance problem. During Phase 1, baseline data about keyperformance metrics were gathered. During the evaluationportion of Phase 5, these baseline data should be comparedwith data collected during the implementation phase and atpredefined intervals for some time after implementation.The system’s success or failure in meeting the predeter-mined success criteria should be reported out to the keystakeholders in the form of a periodic report or a dashboard.This will not only be useful in performance managementand continuous improvement of the system but will also actas a data point for future performance analysis to determineif EPSS is the right intervention.

Conclusion

In the coming years new technologies will continue to pro-vide burgeoning opportunities to support the performanceof users in innovative ways. Our understanding of the fac-tors that influence the effectiveness of performance supportsystems will grow, and the field of EPSS will continue tomature. The design model introduced in this article pro-vides a simple framework that human performancetechnologists can readily apply, refine, and adapt as elec-tronic performance support systems continue to evolve.

References

Caffarella, R. S. (1994). Planning programs for adult learn-ers: A practical guide for educators, trainers, and staffdevelopers. San Francisco: Jossey-Bass.

Carr, C. (1992). PSS! Help when you need it. Training &Development, 46(6), 31–37.

Chevalier, R. (2003). Updating the behavior engineeringmodel. Performance Improvement, 42(5), 8–14.

Daley, B. J. (1998). Novice to expert: How do professionalslearn? In 39th Annual Adult Education ResearchConference Proceedings. San Antonio, TX: University ofIncarnate Word and Texas A&M University.

Fischer, O., & Horn, R. (1997). Electronic performance sup-port systems. Communications of the ACM, 40(7), 31–32.

Gery, G. (1995). Attributes and behaviors of performance-centered systems. Performance Improvement Quarterly,8(1), 47–93.

Gilbert, T. F. (1978). Human competence: Engineering wor-thy performance. New York: McGraw-Hill.

Hawkins, C. H., Jr., Gustafson, K. L., & Nielsen, T. (1998).Return on investment (ROI) for electronic performancesupport systems: A web-based system. EducationalTechnology, 38, 15–22.

Knowles, M. (1984). The adult learner: A neglected species(3rd ed.). Houston, TX: Gulf.

Ladd, C. (1993). Should performance support be in yourcomputer? Training & Development, 47(8), 22–26.

MacKeracher, D. (1996). Making sense of adult learning.Toronto: Culture Concepts.

Nguyen, F. (2005). Oops, I forgot how to do that: A needsassessment of electronic performance support systems.Performance Improvement, 44(9), 33–39.

Nguyen, F., Klein, J. D., & Sullivan, H. (2005). A compara-tive study of electronic performance support systems.Performance Improvement Quarterly, 18(4), 71–86.

Raybould, B. (2000). Building performance-centered web-based systems, information systems, and knowledgemanagement systems in the 21st century. PerformanceImprovement, 39(6), 32–39.

Rossett, A. (1998). First things fast: A handbook of perfor-mance analysis. San Francisco: Jossey-Bass/Pfeiffer.

Webb, W. (1999). Show me the return. Inside TechnologyTraining. Retrieved February 3, 2006, fromhttp://www.trainingsupersite.com.

Witt, C. L., & Wager, W. (1994). A comparison of instruc-tional systems design and electronic performance supportsystems design. Educational Technology, 34(6), 20–24.

Frank Nguyen is a doctoral candidate focusing on performance support sys-tems. He has managed the development and deployment of enterprisee-learning and performance support systems at Intel Corporation for the lastsix years. He is coauthor of Efficiency in Learning (with Ruth Colvin Clark and John Sweller, Jossey-Bass, 2006) and holds a master’s degree inEducational Technology from Arizona State University. He may be reached [email protected].

Craig A. Woll, PhD, is a senior learning technologist for Intel Corporation. Hereceived his PhD degree in Instructional Technology from Utah State University.He works with internal and external customers to develop strategic learningpackages to meet Intel’s strategic objectives. Craig may be reached [email protected].


Documents

A practitioner's guide for designing performance support systems