Download pdf - Work-Centered Support Systems: A Human-Centered Approach to Intelligent System Design

MARCH/APRIL 2005 1541-1672/05/$20.00 © 2005 IEEE 73Published by the IEEE Computer Society

H u m a n - C e n t e r e d C o m p u t i n g

Work-CenteredSupport Systems: A Human-CenteredApproach to IntelligentSystem DesignRonald Scott, BBN Technologies

Emilie M. Roth, Roth Cognitive Engineering

Stephen E. Deutsch, Erika Malchiodi, and Thomas E. Kazmierczak, BBN Technologies

Robert G. Eggleston and Samuel R. Kuper, Air Force Research Laboratory

Randall D. Whitaker, Northrop Grumman Information Technology

Ahallmark of human-centered computing (HCC) is its focus on domain practi-

tioners and their field of practice.1,2 Human-centered design depends on a deep

analysis of a field’s cognitive and collaborative demands and how people work individu-

ally, in groups, and in organizations to meet those demands.3,4 The objective is to leverage

what we know about human cognitive and collabo-rative processes to create systems that optimize theaffordances (direct perception of meanings) andeffectivities (knowledge-driven actions) for humans.5

We developed a software-agent-based system forweather forecasting and monitoring that exemplifiesthe HCC approach and demonstrates work-centeredsupport systems concepts.6–8 The WCSS paradigmoffers an approach for incorporating software agenttechnology in a manner that helps the user keep his orher “head in the work” and reduces the possibilitythat software agent states or actions will surprise theuser. To date, software-agent technology has focusedlargely on increased autonomy of individual softwareagents and increased coordination of activity amongmultiple software agents.9 As the technology matures,we need to focus on how to integrate agents intoteams that include both humans and agents and howbest to deploy agents to support human work.10,11

Work-centered support systemsA distinguishing characteristic of WCSSs is that

they focus on supporting all facets of work. Specif-ically, a WCSS includes

• decision support aiding problem solving and othercognitive work processes,

• product development support aiding deliverablework artifacts’ production,

• collaborative support aiding team and colleagueinteractions, and

• work management support aiding the metacogni-tive activities entailed in prioritizing and manag-ing multiple intertwined work activities (such asrequirements for attention shift and rememberingthe number and state of tasks in process).

A WCSS attempts to integrate support for each ofthese areas through the principle of joint aiding—thecoordinated use of direct and indirect aiding methodswithin a common, work-centered ontology.7 A coor-dinated set of software agents that interact with theuser and are clearly connected to or embedded inwork domain visualizations provide direct aiding.

The Work-Centered

Support System

approach to human-

centered computing

focuses on analyzing

and supporting the

multiple facets of work.

The WCSS for Global

Weather Management,

developed to support

weather forecasting

and monitoring in an

airlift service

organization,

exemplifies this

approach.

Indirect aiding is provided largely throughthe creation of work domain visualizationsthat reveal domain relationships, constraints,and affordances. By emphasizing the need tosupport all facets of work and to providedirect and indirect aiding, the WCSS para-digm falls squarely within the HCC tradition.

Designing a WCSSWe developed the Work-Centered Support

System for Global Weather Management(WCSS-GWM) to support weather forecast-ing and monitoring in a military airlift ser-vice organization. Building a WCSS requiresa development process that grounds systemrequirements in a deep understanding of thecognitive and collaborative demands of thework domain. We achieved this through closecollaboration among cognitive and softwareengineers and end-user domain practitioners,all of whom were involved from the project’sinception. The collective goal was to under-stand the workspace (work demands andpractices in the environment) and stretchwhat we already knew how to do to build anew work environment—a collaborativeeffort in envisioning, creating, and refininga new workspace.

Our project’s backgroundTraditionally, airlift pilots have been

responsible for their own flight planning,including obtaining preflight weather brief-ings. The organization we worked with initi-ated a new approach to reduce the amount oftime an aircrew must devote to these tasks.It created a flight manager (FM) positionwith the primary responsibility of planningand managing multiple flights, both preflightand en route. This includes obtaining aweather briefing and providing a completeflight plan to the pilot, including weatherforecast information. The FM is a virtualcrew member who supports the pilot.

Weather can significantly influence pre-flight and en route flight management deci-sions (for example, accelerating, delaying,or rerouting a flight because of unfavorableweather conditions). So, weather forecastersmust work closely with the FM to evaluateweather conditions at the departure andarrival airfields and along the planned route.We focused on developing an intelligent sys-tem to aid near-term weather forecasting tosupport planning and managing airlifts.

Cognitive analysisOur WCSS methodology emphasizes

acquiring an understanding of work as prac-ticed. Field observations and interviewsintended to help us understand the cognitiveand collaborative demands of the domainwere an integral part of our WCSS-GWMdevelopment process.12 We examined workpractices in terms of decision making, prod-uct development, collaboration, and workmanagement activities. These revealedaspects of cognitive and collaborative workthat could be more effectively supported,which then guided the design of the WCSS-GWM architecture and displays.

We performed three site visits, each twoto three days long and held approximatelytwo months apart. During these visits, weinterviewed forecasters, FMs, and senior per-sonnel to understand workflow and elicit

examples of the complications that can ariseand increase task demands. We also observedFMs and forecasters as they handled actualflights during planning and en route phasesto identify FM and forecaster activities andsources of complexity that the interviews didn’t reveal. We sampled as broad a rangeof domain practitioners and situations aspractical, including interviewing five or sixindividuals during each visit, interviewingindividuals with differing levels of experi-ence and expertise, and conducting observa-tions over several time periods to sample dif-ferent activity rhythms (morning and eveningshifts as well as shift turnovers). Observa-tions spanned routine and challenging cases.They revealed the intensive collaboration thatoccurs between FMs and forecasters whenweather conditions such as severe turbulenceor lack of anticipated tailwind require mod-ification to planned flight routes.

We presented storyboards (rapidly devel-oped prototypes) to FMs and forecasters fortheir comments, enabling them to actively par-ticipate in design. The storyboard prototypes

embodied candidate aiding concepts and dis-played increasing functionality and robustnesswith each site visit, reflecting what we’dlearned from the prior visit. They provided aconcrete vehicle to demonstrate our growingunderstanding and to obtain user feedback onthe evolving aiding concepts’ viability. Theyalso provided a stimulus for raising additionaldomain constraints and complications thatwe’d need to accommodate.

A multidisciplinary team—including cog-nitive engineers experienced in domainanalysis and user requirements specificationand software engineers who would designand implement the software—conducted thefield observations and interviews. The par-ticipation of both cognitive and softwareengineers facilitated dialogue among theteam members and enabled close collabora-tion in the specification and design of theWCSS-GWM system architecture and userinterface.

Workspace and work patternobservations

Typically, three trained forecasters are onduty each shift in the organization we stud-ied. Two sit in a weather-forecasting roomadjacent to and within sight of the FMs’area,and one sits in the flight management area toprovide collaborative support to the FMs. Theweather-forecasting room includes numerousworkstations, each with two or three CRTs,and several wall-mounted TV monitors andlarge-screen displays that can be slaved tosome of the workstation screens. As a matterof standard practice, the large-screen panelsdisplay loops of weather satellite imagery,focused on geographic regions of interest.They are also used to present weather brief-ings to FMs during shift changes, as well asto support forecaster collaboration.

The forecasters focus on near-term avia-tion weather forecasting—understanding andpredicting weather conditions that will affectflights in the air and those scheduled to takeoff in the next 12 hours. Near-term forecast-ing requires acquiring, interpreting, and inte-grating multiple weather data types, includ-ing satellite imagery, airfield and otherobservations (such as pilot reports of turbu-lence), upper-air forecasts, and computermodel projections from local and worldwidesources. These weather data types were typ-ically available to the forecasters from sepa-rate Web pages or map displays, requiringforecasters to mentally integrate informationfrom multiple disparate sources.


74 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

By emphasizing the need to

support all facets of work and to

provide direct and indirect aiding,

the WCSS paradigm falls squarely

within the HCC tradition.

In many cases, the available data areincomplete, ambiguous, stale, or conflicting.Weather-forecasting expertise includes theabilities to look for convergence among mul-tiple data types (to check that “things are lin-ing up”) and pursue additional sources of evi-dence to fill in missing data or resolveambiguous or conflicting data. For example,a forecaster can increase confidence in a fore-cast by checking with a location in a pre-dicted storm’s path to confirm that the stormhas materialized. Similarly, the forecaster canincrease confidence in a turbulence predic-tion by checking certain kinds of satelliteimagery and seeking pilot reports (PIREPs)confirming actual conditions.

We observed several cases where fore-casters had to reconcile conflicting forecasts,including cases where the forecaster neededto call the forecast’s source to understand thebasis for a prediction. In at least one case, theforecaster had to advocate an alternative fore-cast. Being able to resolve ambiguous or con-flicting weather data is critical because pre-dicting severe weather has risk/benefitconsequences. Being too conservative canmean serious delays or cancelled missions;being insufficiently conservative can result incostly mission diversions, plane damage, oreven risk to life. To minimize the impact onmission objectives, forecasters are sensitive toboth of these concerns and work intensely torefine their forecasts and communicate to FMsthe basis of and level of confidence in weatherpredictions. They must also identify ways towork around real or anticipated weather haz-ards—for example, by selecting alternate take-off or landing airfields or changing the takeofftime or flight route.

Our analysis revealed that FMs and fore-casters operate as a team to manage the air-lift missions in their work queue. The FM isthe primary player, constructing flight plans,preparing crew papers, and monitoring themany details involved in ensuring each mis-sion’s success. The forecasters engaged inmultiple activities:

• maintaining situational awareness ofweather conditions in multiple geographicregions worldwide,

• preparing general forecasts for theseregions and tailored forecasts for each mis-sion,

• responding to requests and providingtimely weather data to various parties inthe organization (including pilots who callfor weather updates),

• monitoring weather observations to assesstheir effects on current and upcoming missions,

• helping FMs develop options for workingaround hazardous weather to minimize itsimpact on mission goals, and

• negotiating with other weather-forecast-ing organizations regarding the appropri-ate interpretation of ambiguous weatherdata and advocating particular weatherinterpretations.

FMs and forecasters worked collabora-tively to understand the consequences ofunexpected situations (such as changing mis-sion requirements or weather conditions) anddetermine appropriate action (such as delay-

ing or rerouting a flight or changing the fuelload or cargo).

Identifying leverage points—opportunities to provide support

Our analysis identified a number of lever-age points or opportunities to more effec-tively support weather forecasting and mon-itoring. This provided the basis for definingwork-centered support requirements.

Decision support. Forecasters can use sup-port in collating and integrating informationfrom multiple, disparate aviation weathersources, including forecasts, satellite imagery,real-time weather updates, and particularlyflight plans for current and near-term flights.This can be accomplished by integrating themultiple weather sources on a single georef-erenced map.13 Providing an integrated visu-alization enables forecasters to more quicklyderive and update a situation model of sig-nificant weather factors in their geographicareas of responsibility. This supports the gen-

eral requirement for forecasters to achieveand maintain weather-related situationalawareness so that they can make sound deci-sions. An integrated visualization providesmore effective support for the cognitivelychallenging aspects of forecasting: identify-ing converging evidence in support of a fore-cast and identifying and resolving ambigu-ous or conflicting information.

Product development. The process by whichforecasters update and revise forecasts islabor intensive, so they don’t update them asoften as they’d like. Tools that enable morerapid recognition of changes in weather con-ditions and production of revised forecastswould enhance both forecasts’ accuracy andtimeliness.

Collaboration support. The forecaster-FMteam needs collaborative support in evaluat-ing weather’s impact on flight plans and mak-ing reroute decisions.Weather and flight infor-mation aren’t well integrated. The forecasterhas the best access to weather information,and the FM has the best access to plannedand en route flights’ status. Superimposingweather and flight information (for example,the flight route and organized tracks that con-stitute legal flight path options) on a singlegeoreferenced map that the FM and forecastercan view simultaneously would let them visu-alize the weather’s impact on a flight route andcollaboratively formulate reroute options.

Work management support. Forecastersmust monitor weather conditions in multiplegeographic regions of interest and supportplanning and monitoring of tens of flights.Notifying the forecaster and FM of signifi-cant changes in weather conditions and themissions that weather changes will affect canhelp direct their attention to potentially high-risk flights. Further, mechanisms that sup-port keeping track of missions to be moni-tored and easily shifting back and forthamong them are important to support themultiple interwoven activities that charac-terize forecasters’ work.

Using software agent technologyto create a WCSS

The WCSS-GWM focused on monitoringthe status of en route and near-takeoff mis-sions with respect to weather conditions andon achieving and maintaining weather-related situational awareness for geographicregions of interest.

MARCH/APRIL 2005 www.computer.org/intelligent 75

Tools that enable more rapid

recognition of changes in weather

conditions and production of

revised forecasts would enhance

both forecasts’ accuracy and

timeliness.

Our investigation into how weather per-sonnel performed these tasks suggestedwhich weather data types would be useful.The weather personnel already used some ofthese data types. In some cases, weather per-sonnel evaluated graphic images, and we’dhave to analyze the underlying numericaldata to achieve the same effect. To meet ourdesign objectives, the WCSS-GWM requiredthese functionalities:

• acquisition of real-time weather observa-tions, such as PIREPs and automated obser-vations of wind and turbulence informationsent through the Aircraft Communicationand Reporting System (ACARS).

• acquisition of worldwide airfield andupper-air forecasts produced locally andremotely. These include Significant Mete-orological Information bulletins (SIGMETs),which describe areas with weather that’spotentially hazardous to aviation; Meteo-rological Terminal Air Reports (METARs),which describe current surface weatherobservations at worldwide reporting sta-tions; and terminal area forecasts (TAFs),which are bulletins that give surface

weather forecasts for worldwide reportingstations.

• graphical integration of multiple datasources including maps, flight plans, fore-casts, point observations, and satelliteimagery. An important design requirementwas the ability to easily overlay any sub-set of data types on the same georefer-enced map for comparison purposes.

• automated comparison of real-timeweather observations with user-definedgeographical areas of interest (referredto as watch areas) and alerts to focusforecaster attention on operationally rel-evant changes in weather conditions.Achieving a general capability for anautomated alerting process would be dif-ficult. So, we limited alerting to specific,useful capabilities, such as generatingan alert for any report from PIREPs orACARS of turbulence or icing of at leasta defined severity level in the definedregion of interest (latitude, longitude,altitude, or time).

• automated and directed monitoring ofindividual missions and watch areas, aswell as generation of alerts to focus user

attention on changes in weather conditionsthat can affect flights.

The prototype we developed includes acentrally located, georeferenced-map-basedvisualization as a framework for providingdirect and indirect aiding. Users can selec-tively superimpose a variety of weather andflight plan information on the map. Weather-related alerts generated by software agentsare presented at their geolocations on themap. The display is built on top of OpenMap,an open-source, Java-based, geospatial dis-play toolkit (see http://openmap.bbn.com).Figure 1 illustrates the WCSS-GWM’s basicfeatures.

The WCSS-GWM’s “home view” is amap showing the geographical area of inter-est, surrounded by controls. The map con-trols let you pan, zoom, and change projec-tions, while layer controls let you viewmultiple layers of flight and weather infor-mation on the map. You can place flightplans, PIREPs, ACARS, observations, SIGMETs,and satellite images on the map and removethem. An altitude slider control lets you fil-ter observations by specifying the altitude.



Figure 1. The Work-Centered Support System for Global Weather Management’s map display and floating Sortie Palette.

You can access more details by hovering overan icon; for example, you can obtain a PIREP’stext by placing the cursor over the PIREP sym-bol on the map.

A floating Sortie Palette lists all missionsof interest and summarizes individual mis-sions’status. It provides the abilities to high-light and locate specific missions and to sortand organize them to suit the work context.It also lets users maintain awareness ofweather-related alerts, keeping track ofwhich ones they’ve already viewed andwhich ones they must still deal with. Fur-thermore, it’s integrated with the map dis-play such that you can highlight problemnotifications on the map with a simple buttonclick on the palette. Thus, the Sortie Paletteaids work management for work on a givenmission or a set of missions.

Software agents’ roleSoftware agents provide intelligent auto-

mation to directly support forecasters. TheWCSS-GWM includes agents that monitormissions and geographic regions of interestand notify the forecaster when operationallysignificant weather changes occur. A keyWCSS-GWM design element is that the fore-caster can create, monitor, and modify theseagents. For example, the forecaster can createan agent by drawing a polygon around awatch area on the map and specifying theagent behavior by setting parameters (desiredaltitude, start and stop time, hazard type, andseverity to watch for). Later, the forecastercan modify the polygon’s shape and positionand modify the agent behavior by changingthe parameters. Forecasters can also createand modify agents that monitor for weatherchanges around a flight path.

Figure 2 illustrates this ability to createand modify agents. The wide shading alonga flight path indicates the geographic area anagent is monitoring. Similarly, the transpar-ent geometric shapes off the eastern UScoastline indicate the watch areas agents aremonitoring. The Create Area and Edit AgentParameters pop-up windows illustrate theabilities to create new agents and to view andmodify the parameters that control agentbehavior.

User (forecaster) creation of agents is animportant aspect of a WCSS. Complex sys-tems with many automation features, such asauto-assistance in different operationalmodes, can lead to machine-induced usererrors10 in which the user thinks the systemis behaving in one way but the automation

condition causes it to behave differently. Sys-tem designers can mitigate this problem byproviding users with the capability to createand set the conditions for an agent. Becauseof this involvement in specifying and acti-vating the automation protocol, the usershould have increased awareness of what ser-vice the automation provides.

The watch area agent in Figure 2 isexpressed in work terms—a physical area tobe watched, indicated by the polygon. Usersdon’t have to translate from an agent icon tothe weather-based semantic intent of whatassistance the agent provides. They observethe agent directly in the work ontology, therebyreducing cognitive complexity and demand.

The WCSS-GWM contains three broadclasses of agents. Acquisition agents acquiredata from outside sources such as weatherbulletins, ACARS, SIGMETs, satellite imagery,mission details, and flight plans. Each acqui-sition agent is responsible for a particulardata type or source and periodically retrievesthe latest data from that source (anywherefrom once a minute to once every few hours,depending on how often new data are avail-able). Furthermore, each acquisition agentsignals other interested agents when it hasretrieved new data.

Analysis agents analyze data that acquisi-tion agents have retrieved to produce initialproblem indications (individual turbulenceor lightning strike reports, intersections of

flight plans with SIGMETs, and so on). Fore-casters trigger region analysis agents whenthey decide to monitor a geographic regionfor critical conditions (that is, to create awatch area) and then watch for observationsmatching given criteria. The presence of acurrent or upcoming flight mission automat-ically generates mission analysis agents,which watch for reports (such as PIREPs orACARS) close to the flight plan (in latitude,longitude, altitude, or time) that significantlyaffect the mission.

Presentation agents, based on the analysisagents’ results, decide what information topresent to the user. These agents work on ini-tial problem indications, clustering and prior-itizing them to create a high-level presenta-tion of problems. For example, the analysisagents might generate many related notifica-tions that the presentation agents must aggre-gate into a single notification message to avoidan “alarm avalanche” problem.14 Presentationagents also stage displays—that is, theyretrieve enough of the right kind of data so thatthe information the user needs can be quicklyrendered on the screen. We’ve implementedonly a limited presentation agent capability(as shown in Figure 2), but in a full-scaleWCSS-GWM implementation, these agentswould have two additional responsibilities:

• displaying data at different levels of aggre-gation, depending on the user’s role. For


Figure 2. A screen shot illustrating the ability to create and modify software agents.

example, a supervisor might get only atop-level summary view of areas of turbu-lence, while the user responsible for a par-ticular mission might see individualreports of turbulence close to that missionpath. The presentation agent would aggre-gate the same underlying data to differentlevels for different users.

• displaying different data depending on theuser’s role. In a global system, multipleFMs and forecasters would split the globeinto regions of responsibility. As analysisagents produced critical weather indica-tions, a presentation agent would presentthis information only to interested users.

A consideration in the agent architecturedesign was to create a structure that the fore-casters could understand, inspect, and mod-ify. While software agent research hastended to focus on the high-level tasks del-egated to agents and their level of autonomy,agent technology’s value from a softwaredevelopment perspective is that softwareagents are small, independent chunks ofsoftware. Each addresses a small, unified setof tasks and is separately controllable andmodifiable. In creating the agent architec-ture, we needed to structure the software sothat the agents’ capabilities would be mean-ingful to users in terms of their workdomain. We configured the agents to mirrorthe basic terms of reference the users employin addressing their work. This applies bothwith respect to the agents’ functions (suchas acquiring, analyzing, and presenting data)and to the domain objects the agents workon (such as missions, forecasts, and watchareas). Once the software is organized intodomain-meaningful chunks implemented asagents, users can more readily observe anddirect their operation.

Using D-OMAR to create agent-based systems

We built our agent system on top of theDistributed Object Model Architecture sys-tem, developed under the Air Force ResearchLaboratory’s sponsorship (see http://omar.bbn.com). D-OMAR was initially an event-based simulator that included representationlanguages designed to facilitate research inhuman performance modeling.15 It hasevolved into a full-featured distributed soft-ware infrastructure that provides a broadrange of services essential to agent-basedsystem development. Agents must be cre-ated, be able to launch procedures that imple-

ment their services, and be retired orremoved when no longer needed. A publish-subscribe protocol supports agent commu-nication. In addition to the basic function ofmoving data between agents, the publish-subscribe capability plays an essential rolein coordinating pairs or groups of agents’activities. An agent can also use the publish-subscribe protocol to coordinate the execu-tion of its multiple proactive and reactivebehaviors. We’ve found these language fea-tures essential to the rapid development ofsophisticated agent behaviors.

Two implementations of D-OMAR exist:the original Lisp implementation, now knownas OmarL, and the new Java implementation,OmarJ. We developed the WCSS-GWM

using OmarJ. Software agents developed ineither environment have similar capabilities;indeed, Java- and Lisp-based agents are inter-operable. A procedural language supportsagent behavior implementation. In OmarL,the Simulation Core language (SCORE) wasbuilt as an extension of the Lisp languageitself. ScoreJ, a set of Java class libraries, pro-vides a similar capability for Java imple-mentations. Important language featuressuch as race, join, and satisfy support the devel-opment of concurrent proactive and reactiveprocedures essential to individual agentbehaviors. Time management features suchas sleep support the scheduling of importantservices, and you can use them with concur-rency control to specify time-out conditions.These language features, used in combina-tion, can act as pattern matchers for eventsevolving in time. The languages also supportthe priority-based mediation of contentionbetween specified subsets of procedures.

The WCSS-GWM’s statusThe WCSS-GWM is in the late stage of

development. We’ve installed it on severalworkstations at the airlift service facility, andforecasters are using it. Users have reportedseveral cases where they successfullyrerouted flights on the basis of informationprovided by the WCSS-GWM.

Implications of using agenttechnology

Designing the WCSS-GWM required usto consider a number of questions related to integrating software agents into human-centered computing.

Observability and directabilityA growing body of cognitive-engineering

literature is showing that for automatedagents to be effective, they must act as teamplayers.10,11,16 For software agents to becometeam players, you need to design in two fun-damental characteristics from the begin-ning—observability and directability. Usersmust be able to see what the automatedagents are doing and understand what they’lldo next relative to the task’s state. They alsoneed to be able to control and redirect thesoftware agents as task requirements change.

We designed the WCSS-GWM agent-based architecture with these objectives inmind. The georeferenced map provides acommon-ground representation of the cur-rent situation that’s available to both humans(FMs and forecasters) and software agents.Furthermore, the user can see and controlagents’ activities—the display presents thegeographic area the software agents are mon-itoring, and the user can modify it. Similarly,the forecaster can inspect and modify theweather parameters that the agents are mon-itoring and the trigger points for alerts.

Software agents’ role in human-centered systems

Our discussion of agents in the WCSS-GWM points to their multiple roles inhuman-centered systems’ development. Atthe software development level, agent tech-nology is valuable because the agents aresmall, independent “chunks” of software,each of which addresses bounded tasks. Soft-ware agents can be controlled and modifiedseparately, which greatly facilitates softwaredevelopment, maintenance, and upgrades.

From a work-centered perspective, agentsare useful because they enable the develop-ment team to organize software componentsaround the elements of work. A detaileddomain analysis lets the team map out the work



A consideration in the agent

architecture design was to create

a structure that the forecasters

could understand, inspect, and

modify.

requirements and systematically allocate tasksto humans and software agents as appropriate.In the domain of weather forecasting to sup-port flight management, the kinds of work thathumans can allocate to software agents include“watch this region of the world and let meknow of any weather problems that arise”(region analysis agents), “watch this missionfor me and let me know of any problems thatarise” (mission analysis agents), or even“watch this weather system” or “interpret thisweather pattern” (weather-centered agents,which we have yet to implement), freeinghuman attention resources for other tasks.

A corollary of structuring agents in workdomain terms is that the agents give us a con-crete vocabulary of concepts and metaphors(agents as subordinates to whom you canassign specific tasks) that software and cog-nitive engineers and users can share. Thisfacilitates communication and enables closercollaboration during system design.

Software agents: Behind thescenes or up front?

Closely related to the question of softwareagents’role in human-centered systems is thequestion of their visibility in the interface.We continue to struggle with the extent towhich (and conditions under which) softwareagents should have an explicit screen pres-ence. Should we treat software agents as con-venient chunks of code that underlie systemfunctionality but have no explicit presencein the user interface, or should they beexplicit to let users interact with them?

Multiple stances are possible on this ques-tion. At one extreme, you could argue thatagents are a powerful tool for software imple-mentation but that users should never see orhear about them as such. Arguments for thisposition include concerns that users mightascribe a greater degree of expertise, compe-tence, and autonomy to the agents than is war-ranted, leading them to overly trust the agents.Conversely, ascribing more autonomy to theagents could also cause users to unjustifiablymistrust or even fear them. At the otherextreme, you could argue that if agents havea screen presence and even a personality(such as the animated Microsoft paperclip),users can more readily grasp their function-ality, seeing them as assistants or subordinatesto whom they can delegate tasks.

For the WCSS-GWM, we chose a middleground. Some agents are visible in the userinterface, while others operate behind thescenes. Agents with a visible presence are

organized around domain work: the forecast,region, and mission analysis agents. Users del-egate work to these agents that they would oth-erwise have to do themselves. The agents’presence in the user interface lets users mon-itor and control their performance. Our focuswas on letting users understand, observe, andcontrol the agents’ behavior. However, wemade no attempt to personify these agents.Nothing marks them as agents on the screen,and they have no personality or animated pres-ence. Rather, we adopted a work-centered per-spective where users interact with the inter-face and monitor and delegate tasks to agentsin the context of performing their work.

Other agents in the WCSS-GWM remainpredominantly behind the scenes, but users

can access them should the need arise. Forexample, although the data acquisition agentsoperate out of sight most of the time, fore-casters can bring up displays letting theminspect and modify the agents’ performanceas necessary. This becomes particularlyimportant for diagnosing agent behaviors (forexample, if a data source accessible over theWeb goes down or if data become corruptedor stale) or tailoring agent behaviors to newcircumstances (for example, to add or mod-ify a data source).

Finally, still other agents execute softwaretask elements, but users can’t see or controlthem. This category includes the presenta-tion agents, where a need for user interactionwith the agents hasn’t yet emerged.

Fostering collaboration in systemdesign

We found that an additional benefit of soft-ware agent technology is its potential to facil-itate collaboration among users and cogni-tive and software engineers. The adoption of

object-oriented design techniques over thelast 10 to 15 years has improved communi-cation among software and cognitive engi-neers and end users by providing easilyunderstandable descriptions of the domainobjects’software implementation and small-scale behavior. However, these descriptionshave, at times, fallen short in providing theuser with a clear understanding of large-scalesystem behavior—how objects hook togetherto produce the desired system results. As asystem increases in complexity, the user caneasily lose the big picture of how objectswork together, with the result that collabora-tion between the user and the software devel-opment team effectively stops.

The agent architecture provides the poten-tial for user-accessible descriptions not onlyof domain objects but also of workflow andlarge-scale interactions between domainobjects. All members of the team can under-stand and contribute to the software’s design.We expect that the dialogue among users andcognitive and software engineers couched interms of workflow for users and softwareagents will contribute to improved human-centered system design.

The WCSS-GWM exemplifies andexpands upon HCC concepts. It

embodies WCSS concepts that emphasizethe need to support the multiple facets ofindividual cognitive and collaborative workthrough the integration of software agentsthat support data acquisition, analysis, andvisualization. We successfully transitionedthe WCSS-GWM into the user organization,where it underwent a formal evaluation thatproduced highly favorable results.17 We’recontinuing our research program to developand exercise WCSS design concepts andprinciples through developing and evaluat-ing WCSS applications. Our new researcheffort targets the broader enterprise of flightmission planning and execution.

AcknowledgmentsThe Air Force Research Laboratory Human

Effectiveness Directorate supported the develop-ment of the Work-Centered Support System forGlobal Weather Management. We thank RobertHoffman for his thoughtful comments and sug-gestions on an earlier draft of this article.


We expect that the dialogue among

users and cognitive and software

engineers couched in terms of

workflow for users and software

agents will contribute to improved

human-centered system design.

References

1. Hoffman, P.J. Hayes, and K.M. Ford,“Human-Centered Computing: Thinking Inand Outside of the Box,” IEEE Intelligent Sys-tems, vol. 16, no. 5, 2001, pp. 76–78.

2. E.M. Roth, E.S. Patterson, and R.J. Mumaw,“Cognitive Engineering: Issues in User-Cen-tered System Design,” Encyclopedia of Soft-ware Engineering, 2nd ed., J.J. Marciniak, ed.,Wiley-Interscience, 2002, pp. 163–179.

3. K.J. Vicente, Cognitive Work Analysis:Toward Safe, Productive, and Healthy Com-puter-Based Work, Lawrence Erlbaum Asso-ciates, 1999.

4. R.R. Hoffman and D.D. Woods, “StudyingCognitive Systems in Context: Preface to theSpecial Section,” Human Factors, vol. 42, no.1, 2000, pp. 1–7.

5. R.R. Hoffman, J.W. Coffey, and K.M. Ford,The Handbook of Human-Centered Comput-

ing, tech. report, Inst. for Human and MachineCognition, Univ. of West Florida, 2001.

6. R.G. Eggleston and R.D. Whitaker, “Work-Centered Support Systems Design: UsingOrganizing Frames to Reduce Work Com-plexity,” Proc. Human Factors and Ergonom-ics Soc. 46th Ann. Meeting, Human Factorsand Ergonomics Soc., 2002, pp. 265–269.

7. R.G. Eggleston, “Work-Centered Design: ACognitive Engineering Approach to System



T h e A u t h o r sRonald Scott is a division scientist in BBN Tech-nologies’ Intelligent Distributed Computing department. His research interests include the appli-cation of distributed software technology and code-generation techniques to the design of human-centered applications. He received his PhD inmathematics from the University of Chicago. Con-tact him at BBN Technologies, 8778 Danton Way,Eden Prairie, MN 55347; [email protected].

Emilie M. Roth is a cognitive engineer and theowner of Roth Cognitive Engineering. Her researchinterests include the impact of support systems (suchas alarms, decision aids, and group view displays)on individual and team decision making in real-world environments (such as military command andcontrol, intelligence analysis, power plant operations,surgical teams, and railroad operations). She receivedher PhD in cognitive psychology from the Univer-

sity of Illinois at Champaign-Urbana. She’s on the editorial board of the jour-nals Human Factors and Le Travail Humain. Contact her at Roth CognitiveEng., 89 Rawson Rd., Brookline, MA 02445; [email protected].

Stephen E. Deutsch is a division scientist in theHuman Centered Systems Group at BBN Technolo-gies. His research interests include human perfor-mance modeling with related interests in agent-basedsystems and simulation. He’s the principal investiga-tor on a NASA Ames Research Center task in whichresearchers are refining models of human perfor-mance to help better understand the sources of air-crew error on commercial aircraft flight decks. He

also leads related research in modeling human decision making on the flightdeck. He received his BS in mathematics from the University of Wisconsin.Contact him at BBN Technologies, 10 Moulton St., Cambridge, MA 02138;[email protected].

Erika Malchiodi is a member of the technical staffat BBN Technologies. Her research interestsinclude the application of new ideas in distributedsystems, complex databases, and data integrationto military logistics. She received her MS in com-puter science from the State University of New Yorkat Stony Brook. Contact her at BBN Technologies,10 Moulton St., Mail Stop 6/4B, Cambridge, MA02138; [email protected].

Thomas E. Kazmierczak is a program managerat BBN Systems and Technologies and managesdevelopment for the Global Air Mobility AdvancedTechnologies research program, funded by the AirForce Research Laboratory. He received his BS ineducation from Cornell University. Contact him atBBN Technologies, 1337 Park Plaza Dr., Ste. 3,O’Fallon, IL 62269; [email protected].

Robert G. Eggleston is a senior scientist with theHuman Effectiveness Directorate of the Air ForceResearch Laboratory, where he’s been engaged inuser-centered research for advanced system con-cepts for over 25 years. He also holds adjunctappointments with the Air Force Institute of Tech-nology’s Electrical Engineering and ComputerEngineering Department and Wright State Uni-versity’s Biomedical and Industrial Engineering

Department. His research interests include cognitive systems engineering,design engineering, human and team performance, intelligent systems, oper-ator workload, decision making, user interface design, visual informationprocessing, and the development of the user interface as a fully integratedsupport system. He received his PhD in experimental psychology fromMiami University. Contact him at AFRL/HECS, 2255 H St., Wright-Patterson AFB, OH 54533-7022; [email protected].

Samuel R. Kuper is a senior industrial engineerand program manager for the development of user-centered interface displays as integrated work sup-port systems, including the Human Interaction withSoftware Agents, Work-Centered Support Systemfor Global Weather Management, and Agent Man-agement Tool technology demonstrations appliedto command and control environments. Hisresearch interests include cognitive systems engi-

neering, user interface design, and reusable interface design patterns. Hereceived his MBA in finance from Wright State University. Contact him atAFRL/HECS, 2698 G St., Wright-Patterson AFB, OH 45433; [email protected].

Randall D. Whitaker is a senior scientist withNorthrop Grumman Information Technology. Hisresearch interests include human-computer inter-action, computer-supported cooperative work, cog-nitive science, applied semiotics, information war-fare, and second-order cybernetics. He received hisPhD in informatics from Umeå Universitet. He’s amember of ACM SIGCHI and SIGGROUP and theVice President for Electronic Publications for the

American Society for Cybernetics. Contact him at Northrop Grumman Infor-mation Technology, 2555 University Blvd., Fairborn, OH 45324-6501; [email protected].

Design,” Proc. Human Factors and Ergonom-ics Soc. 47th Ann. Meeting, Human Factorsand Ergonomics Soc., 2003, pp. 263–267.

8. M.J. Young, R.G. Eggleston, and R.D.Whitaker, “Direct Manipulation InterfaceTechniques for Interaction with SoftwareAgents,” Proc. NATO/RTO Symp. Usabilityof Information in Battle Management Opera-tions, North Atlantic Treaty OrganizationResearch and Technology Organization,2000, pp. 19-1–19-10; ftp://ftp.rta.nato.int/PubFul l text /RTO/MP/RTO-MP-057/MP-057-19.PDF.

9. N. Jennings and M. Wooldridge, “SoftwareAgents,” IEE Review, vol. 42, no. 1, 1996, pp.17–20.

10. K. Christoffersen and D.D. Woods, “How toMake Automated Systems Team Players,”Advances in Human Performance and Cog-nitive Engineering Research, 2: Automation,E. Salas, ed., Elsevier, 2002, pp. 1–12.

11. J. Malin, “Preparing for the Unexpected:Making Remote Autonomous Agents Capa-ble of Interdependent Teamwork,” Proc.

XIVth Triennial Congress Int’l ErgonomicsAssoc. and 44th Ann. Meeting Human Fac-tors and Ergonomics Soc., Human Factorsand Ergonomics Soc., 2000, pp. 1-254–1-257.

12. E.M. Roth and E.S. Patterson, “Using Obser-vational Study as a Tool for Discovery:Uncovering Cognitive and CollaborativeDemands and Adaptive Strategies,” How DoProfessionals Make Decisions? H. Mont-gomery, R. Lipshitz, and B. Brehmer, eds.,Lawrence Erlbaum Associates, 2005, pp.379–393.

13. R.R. Hoffman, “Human Factors Psychologyin the Support of Forecasting: The Design ofAdvanced Meteorological Workstations,”Weather & Forecasting, vol. 6, no. 1, 1991,pp. 98–110.

14. D.D. Woods, “The Alarm Problem andDirected Attention in Dynamic Fault Man-agement,” Ergonomics, vol. 38, no. 11, 1995,pp. 2371–2393.

15. S.E. Deutsch, “Interdisciplinary Foundationsfor Multiple-Task Human Performance Mod-eling in D-OMAR,” Proc. 20th Ann. Conf.

Cognitive Science Soc., Lawrence ErlbaumAssociates, 1998, pp. 303–308.

16. E. Roth, J. Malin, and D. Schreckenghost,“Paradigms for Intelligent Interface Design,”The Handbook of Human-Computer Interac-tion, 2nd ed., M. Helander, T. Landauer, andP. Prabhu, eds., Elsevier, 1997, pp.1177–1201.

17. R.G. Eggleston, E.M. Roth, and R. Scott, “AFramework for Work-Centered Product Eval-uation,” Proc. Human Factors and Ergonom-ics Soc. 47th Ann. Meeting, Human Factorsand Ergonomics Soc., 2003, pp. 503–507.

For more information on this or any other com-puting topic, please visit our Digital Library atwww.computer.org/publications/dlib.


Advertising Sales OfficesMatthew BertholfPhone: +1 785 843 1234 ext. 267Fax: +1 785 843 [email protected]

Sandy Brown10662 Los Vaqueros Circle, Los Alamitos, CA90720-1314; phone +1 714 821 8380; fax +1 714 8214010; [email protected]

Advertiser/Product IndexMarch/April 2005

COMING July/August

Advanced Heuristics in Transportation and Logistics

Transportation and logistics organizations often face large-scale combinatorial problems on operational and strategiclevels, where all possible combinations of decisions andvariables must be examined to find a solution. A promisingapproach to these problems invloves metaheuristic methods,which imply a higher-level strategy controlling a lower-levelheuristic method. Metaheuristics exploit not only the problemcharacteristics but also ideas based on AI rationale, such asdifferent types of memory structures and learning mechanisms.This special issue will address all aspects of metaheuristicmethods as applied to transportation and logistics.