A case for the consideration of system related cognitive functions throughout design and development

A Case for theConsideration of SystemRelated CognitiveFunctions ThroughoutDesign and DevelopmentIain S. MacLeod

Aerosystems International, West Hendford, Yeovil, Somerset BA20 2AL, United Kingdom (email:

[email protected])

Received August 22, 1999; revised May 16, 2000; accepted June 21, 2000SYSTEM RELATED COGNITIVE FUNCTIONS IN DESIGN AND DEVELOPMENT

ABSTRACT

Modern avionics and digital computing are rapidly advancing engineering fields. However,

associated design and development methodologies lag behind these advances. As a result,

Man-Machine Systems (MMS) are increasingly entering service with performances less than,

or different from, that required by their specification. In United Kingdom military systems

procurement, this performance-associated problem is strongly related to a gap between the

contracted system requirements and the requirements related to the system�s �Fitness for

Purpose� as determined by the elected Ministry of Defence (MoD) system acceptance

authority. �Fitness for Purpose� requirements consider the safe usage of the system, its utility,

and its ease of use. Addressing this performance problem requires an understanding of what

is being missed in the subject gap and whether what is missing is related to engineering

methods or poor consideration of functionality. The problem is based partly on incomplete

system specification, specification that emphasizes the physical engineering of the system at

the expense of the cognitive functions needed for the skilled operation of the system.

Considering these differences, traditional Human Factors (HF) / Ergonomic based techniques,

and those of other disciplines related to system design, are becoming increasingly inappro-

priate for use in the design of new avionics systems. One avenue to promote HF value to

design would be to complement systems engineering by the early integration of cognitive

functions into the requirements capture processes, both user and system. These cognitive

functions should be considered as an integral part of an MMS�s functionality and, for conven-

Regular Paper

Systems Engineering, Vol. 3, No. 3, 2000

© 2000 John Wiley & Sons, Inc.

113

ience, will be termed System Cognitive Functions (SCFs). Advocated through the use of SCFs

is a human-orientated approach to assist the logical and physical SE processes, this to assist

in defining system capabilities and expected performance. Suggested is a new emphasis on

the early capture of the differences between the specified system performance requirements

and the requirements needed to satisfy the �Fitness for Purpose� criteria met at customer�s

formal acceptance of the system. These differences are seen as particularly related to MMS

management and control, and point toward a need to discover methods suitable for the

marriage, and associated trade-offs, between MMS human-associated functions and engi-

neered functions. It is argued that the capture and use of SCFs could assist in the knowledge-

able adoption of new technologies. © 2000 John Wiley & Sons, Inc. Syst Eng 3:113�127, 2000

1. INTRODUCTION

Modern electronics and digital computation techniques

continue to promote increased levels of engineered

system functionality, efficiency, and performance. In

parallel, traditional methods of engineering and pro-

gram management are arguably not really good enough

for application to Systems Engineering (SE) or to any

subsequently evolved design and development para-

digms. (For an early argument on this theme see Ackoff,

1973.)

For example, some form of allocation related to

engineered system functions will always be required as

part of the early iterative processes integral to all forms

of design. Thus, considering SE terms, an allocation is

made when logically assigning a performance require-

ment to a physical system function. However, the sup-

posed allocation of functions between man and

machine, as advocated by many Human Factors (HF)

standards and guidelines, is certainly not viable consid-

ering burgeoning system complexity and must be chal-

lenged (MacLeod and Scaife, 1997) as many system

functions are not addressed explicitly by design proc-

esses. These missing functions reside in a gap that exists

between the specified performance requirements for the

system and the requirements on which system accep-

tance is based, these latter predominately concerned

with system �Fitness for Purpose.�

The argument of this paper will be that many of these

poorly considered or missing functions reside as cogni-

tive-based activities. By cognitive activity we refer to

system-related activities that are currently predomi-

nately thinking based and not necessarily directly

linked to any discernible operator activity. Functions

associated with these cognitive activities include the

application to system operation of an operator�s knowl-

edge, mental strategies, and judgment. For conven-

ience, these functions will be termed System Cognitive

Functions (SCFs). The concept of SCFs will be consid-

ered by this paper under the contexts of Knowledge

Based Systems (KBS), Robotics, Electronic Crewman

(EC), and Automation.

It is given that with the introduction of new technolo-

gies to system design, the operator role in system work

performance is becoming less physical and involves

more cognitive activity than physical activity. The more

complex the system, the harder it is for a system opera-

tor to fulfil the cognitive needs of his or her role. Thus,

it is argued that any improved approach must include

the specification of system performance considering

cognitive-related system functions that, where appro-

priate and possible, may well be part allocated to engi-

neered components of the Man-Machine System

(MMS). It is important to note that cognitive functions

are usually in the form of heuristics, these considered

to be expertise-associated �rules of thumb� or schema

located in an operator�s mental processes, and are not

algorithmic in nature as might be found in conventional

software programs.

This paper focuses on a practical approach to the

consideration of the cognitive aspects of MMS design.

A primary tenet of the proposed approach is its close

integration into MMS design and development engi-

neering practices. It follows that a question pertinent to

HF as well as SE is whether the system functionality

and performance under consideration has been ade-

quately specified and considered. The initial function-

ality and performance specification should be based on

user requirements, should be implementation free, and

should logically approach all the user requirements for

the MMS. In traditional engineering terms, machines

are specified where the functionality and performance

is solely concerned with functions that can be physi-

cally engineered. In contrast, by SE terms a system

specification is considered to logically encompass the

totality of MMS functional requirements and perform-

ance. However, SE currently considers the human op-

erator to be mainly a constraint on system design at the

requirement-specification stage of the design life cycle,

a constraint usually having no associated performance

requirements. It is argued that, whilst it is necessary to

114 MACLEOD

consider system constraints where appropriate, this

constraint perspective on the human operator reflects a

limited understanding of the impact of operator-based

cognitive functions on system performance and system

effectiveness.

It is important that user requirements be carefully

specified at the onset, or as early as possible in the

design and development life cycle, and include cover-

age of both operator and system capabilities and per-

formance. In addition, system certification requires

additional evidence not only that requirements have

been met but that the system is safe to use and �Fit for

Purpose.� This evidence can be supplied through high-

quality system specification, periodic appreciation of

user requirements, maintenance of a requirements trace,

iterative processes in design, and appropriate HF evalu-

ation.

2. SYSTEM PERFORMANCEREQUIREMENTS AND CERTIFICATION

In UK military avionics systems procurement, any sys-

tem performance-related problem is strongly related to

the differences between the contracted system perform-

ance requirements and the requirements related to the

system�s �Fitness for Purpose,� these as determined by

the elected MoD system acceptance authority. �Fitness

for Purpose� requirements encompass the safe usage of

the system, its manifest utility, and its ease of use. All

these requirements have pertinence to the domain of

HF. These requirements must be satisfied if a full certi-

fication of the system is to be authorized.

Certification problems arise partly from incomplete

system specification, specification that emphasises the

physical engineering of the system at the expense of the

SCFs needed to support the skilled operation of the

system. Furthermore, these problems also arise from a

frequent unwillingness, inability, or contractual con-

straint on review and adjustment of requirements to

meet �Fitness for Purpose.� Partly, this is a skepti-

cism in certain quarters on the value of the �Soft

Sciences,� i.e. HF and Engineering Psychology, to

assist engineering processes. This unwillingness is

also associated with contracts being placed solely on

specified requirements and, especially under fixed

price contracts, failing to allow leeway for the con-

sideration of any additional functions or design con-

siderations related to system acceptance as �Fit for

Purpose.�

To illustrate the above, the burgeoning adoption of

Commercial Off The Shelf (COTS) equipments, as

components of complex systems, has added to many of

the problems inherent in the achievement of system

�Fitness for Purpose.� The use of COTS has many

advantages. However, if a system is composed of many

diverse subsystems, there is frequently a lack of consis-

tency within the designed Human Computer Interface

(HCI) and the protocols applicable to the system�s

engineered interfaces. As operation of systems increas-

ingly becomes more of a cognitive than physical nature,

a lack of design consistency will cause quandaries in

operator understanding of system operation and ad-

versely affect the operator�s ease of use of the system.

Inconsistency within a system will affect the ease of

operating the system, decrease its overall utility, and

thus adversely affect system acceptance and certifica-

tion. Another factor introduced by COTS is the potential

for the supply of over- and under-functionality with

respect to the initial user requirements of the system,

these unwanted levels of COTS-based functionality still

having to undergo system certification.

To minimize certification effort and cost required

downstream when moving from logical concept to

physical design and development, validation and certi-

fication issues must be considered throughout the de-

sign and development lifecycle. This necessitates for

the requirements to be secure, and it is argued that the

absence of SCFs does not support such security, and that

a trace on requirement consideration is maintained

throughout. To assist such a trace, existing lifecycle

models for conventional engineering practice must be

compared to models and practices supporting the intro-

duction of new technologies and their formal accep-

tance. The new technologies associated differences and

similarities to older technologies need to be equated to

allow a convergence or accommodation to meet air-

borne certification requirements. Further investigations

are, therefore, needed into the development and integra-

tion of models, tools, and methods. This area is dis-

cussed briefly at the end of this section.

2.1. Associated Frameworks

Encompassing the above, two strongly associated

frameworks are suggested as required for the design and

development of an avionics system. These are the sys-

tem contractor�s design and development frameworks,

using as a basis the contracted performance specifica-

tion of the system, and the certification framework. The

structure and performance associated with the design

and development framework supplies the evidence

needed to support the certification framework. As such

the structure and pertinent requirements placed on one

should support the other.

Under the certification framework, certification

comprises two distinct elements:

SYSTEM RELATED COGNITIVE FUNCTIONS IN DESIGN AND DEVELOPMENT 115

1. Ensuring that the system is fit for the purpose for

which it was designed, in that it has the utility

required to perform its task and that the system�s

ease of use is acceptable;

2. Ensuring that the system has been demonstrated

to be adequately safe to reside within its opera-

tional environment.

The certification framework must be associated with

both the contracted host system and any associated

subcontract development frameworks. The purpose of

the certification framework is to support the processes

required for the production of a certification certificate.

There are three closely associated products required for

the production of a certification certificate, namely:

a. A Compliance Matrix showing evidence of �Fit-

ness for Purpose� through compliance to agreed

MMS performance requirements;

b. The Safety Case (see UK Defence Standards

00-55 and 00-56) as a document referencing all

evidence required by the Certification Authority

to enable them to assess the safety of the system;

c. The achievement of satisfactory Trial Reports on

the performance of the avionics system.

It is argued that it is essential that user and system

requirements specification should additionally consider

important system functional and performance require-

ments associated with the granting of system certifica-

tion. The achievement of the three products outlined

above requires assessment and understanding of the

work environment, system control and direction, sys-

tem management, supervision of system task perform-

ance, and teamwork. Such achievements are reached

through the application of system-related cognitive-

type functions, the application of which has been tradi-

tionally relegated to the trained application of human

skills to the system, as mediated by human cognition,

in the support of required MMS performance. It is only

recently that some of these functions can be engineered,

or support to these functions engineered, within the

context of system operational performance. This paper

argues the need to logically specify these functions,

referred to in this paper as SCFs, within future user and

system requirement specifications, and their associated

specification of systems functionality and performance.

As one illustration of the nature of SCFs, the modern

technology of KBS is considered. Until recently KBS

build was only related to laboratory-based develop-

ments, but is now being seriously considered for inclu-

sion into future avionics systems. KBS is

unconventional with relation to conventional software

based systems in that the structured use of the knowl-

edge base of the KBS is largely on heuristic principles

rather than on the use of deterministic algorithms as

employed by most conventional computer-based sys-

tems. In reality, much of the knowledge base of a KBS

is composed of SCFs in that it encompasses facets of

human cognition, such as knowledge gleaned from

Subject Matter Experts (SMEs), and application rules

for that knowledge. Many methods have been devel-

oped to capture SME knowledge in a fashion suitable

for its coding into a database. Such methods are primar-

ily developments from traditional methods of knowl-

edge elicitation and are also applicable to the capture of

knowledge on any operator task-related cognitive func-

tion. Moreover, the knowledge elicited from the SMEs

is recorded for KBS this recording is in the form of an

engineered database and, therefore, is purely MMS-

performance related. Thus it is explicit for the database

to contains SCFs.

The certification process requires evidence of good

design and development practice, and trace of system

requirements and engineering decisions, as a baseline

for granting the certification certificate. With KBS this

may be presented by a design and development model

that not only considers the requirements for quality

build of the KBS, but also considers how the KBS build

processes fit into the model adopted for the host system

development. One such model is CommonKADS

(Schreiber et al., 1998). Further expansion on this

model is considered to be out of the scope of this paper.

3. PRACTICE AND THEORY

This paper is about the principles of practice, rather than

about the theory and details of design and development.

Theory is important for the furtherance of knowledge

but is not necessarily important to the furtherance of

expertise in system design, development, or operation.

However, the knowledge of theory is important to the

designer if for no other reason than to stop design

endeavors heading down nugatory avenues.

This paper is not about the study of hypothetical

constructs concerning topics such as decision making,

judgment, leadership, or even workload. What such

constructs mean is well understood by the majority, but

they all defy a common agreed definition. Moreover,

hypotheses and theories associated with such constructs

are invariably applied or considered retrospectively

with relation to their fit to real MMS design, develop-

ment, and work. The study of hypothetical constructs is

important to the furtherance of knowledge and has

contributed to the prediction of outcomes, causes, and

effects of work. However, such study has yet to show

any great practical contribution to new MMS design

116 MACLEOD

processes except for providing theory is support of HF

system evaluation methods.

It should be noted that this paper is about the overall

problems associated with the practice of HF within a

UK military avionics system design and development

environment. Moreover, it is not intended to solely

represent any particular HF practice such as Cognitive

Engineering, Engineering Psychology, or Usability En-

gineering.

Neither is the paper about possible specific improve-

ments to traditional HF methodologies. Traditional ap-

plication of HF fails to allow efficient HF program

integration during the system life-cycle with the work

programs of other disciplines. Further, the main empha-

sis of current HF practice is on physical system design

and not the logical specification of system performance.

Moreover, the burgeoning advances in technology re-

sult in a poor associated applicability of many of the

well-established HF methodologies. This failure, or

inability of HF to effectively contribute to design and

development, is becoming more obvious with relation

to the manifest poor usability of many modern complex

MMS. A better approach than the traditional is required

for consideration of the human operator work within

future MMS design processes. It is important that such

an approach must evolve and complement, and be as-

sociated with, the engineering trade-off processes used

to optimize design throughout the system life-cycle

including its design and development processes. The

recently introduced Integrated Product Team (IPT) ap-

proach by UK MoD is one of many approaches to

multi-disciplinary �concurrent� systems design that

may be pertinent here.

It is believed that the details and practice of the

proposed approach can only evolve and improve in time

with practice and refinement. As a lead-in to the argu-

ment, there follows a critique of the traditional HF

approaches to system design.

4. CRITIQUE OF TRADITIONAL HFAPPROACHES TO DESIGN

4.1. Critique of Standards

An example of a traditional HF approach to systems

design is contained in the UK Ministry of Defence DEF

STAN 00-25, Part 12. This part of the standard contains

a description of the UK MOD recommended process

for applying HF to systems. It describes a serial process,

including the traditional allocation of functions be-

tween man and machine, and refers the reader to the

work of Price (1985).

Thus, DEF STAN 00-25 describes a traditional, se-

quential approach to HF including an allocation of

functions. Its consideration of iteration practices in

design, design trade-off processes, and the traceability

of design decisions, is generally poor. This critique of

the standard leads to a deeper consideration on the

drawbacks of the traditional approach.

As long ago as 1965, Chapanis (1965) cites several

problems with the application of the Colonel Fitts initi-

ated �Man is Better At, Machine is Better At� approach.

Firstly, the approach does not attempt to account for

individual differences, either in terms of the machine or

the man. For example, the statement that humans are

superior to machines in the area of error correction is

not necessarily true for all situations, all machines, or

all human operators. Basic individual differences theo-

ries dictate that some humans will posses very highly

developed error correction abilities, whilst others will

be very poor.

Secondly, Chapanis questions the usefulness of a list

that attempts to define the relative capabilities of man

and machine. He suggests that the most salient ques-

tions are which component of the man-machine system

is good enough for the job in hand and is cost effective.

Lastly, Chapanis considers the factor of trade-offs.

A system is seldom designed solely on the basis of

which component is best for the job. Most often there

is a trade off against other factors such as weight, cost,

size, quality, or safety requirements.

In general the traditional application of HF fails to

consider the complementary nature of the contributions

of man and machine to system performance because the

HF approach is too parochial (for example, providing

task analyses that can be interpreted only by an HF

practitioner). As a second example, a more recent but

still traditional HF approach to the system design proc-

ess is given in the NATO STANAG 3994 (1999). This

STANAG, as is often the case with STANAGS, is still

under draft and revision. Nevertheless, in the UK it is

regularly cited in contracts as a baseline standard on

which industries must base their HF work. Indeed, its

citing is often in association with the DEF STAN 00-25

(1997) and MIL STD 1472D (1989). It should be noted

that these standards, originating as they do from diverse

sources, manifest many contradictions one to another.

STANAG 3994 provides a simplistic model of sys-

tem design process that contains little provision for

iteration processes throughout design and development.

System requirements are relegated to history, or possi-

bly considered to be the concern of some other disci-

pline, and possibilities of HF contributions to

multi-disciplinary design trade-offs are not considered.

The STANAG is representative of the parochial nature

of many standards that over-focus on their own specific

concerns and forget the broader context of their appli-

cation. The model�s stated consideration of early design


progresses in a sequential fashion through to a detailed

system design stage, as illustrated in Figure 1.

4.2. Critique of Traditional Practice

The emphasis in traditional HF practice is on the physi-

cal system and does not include logical consideration

of the system as considered during the initial implemen-

tation free capture of user requirements, system require-

ments, functions, and performance. Therefore, HF

input into the design process, and particularly an under-

standing of SCFs, occurs rather late in the design life-

cycle precluding any major HF influence in the

all-important early design processes.

Moreover, in the UK, the traditional and late use of

Ergonomists or HF practitioners results in some cases

of their derogatory and fruitless naming as �Flower

Arrangers.� This term is in popular use in the UK

aircraft industry where the Ergonomist is often forced

to pick the most appropriate design from a given physi-

cal set of alternatives, even though all the designs may

be fundamentally flawed. Traditional planning of the

HF approach to systems design and development is

through the formulation of a Human Engineering Pro-

gram Plan (HEPP). This form of plan details WHAT has

to be done, but seldom considers the nature or perti-

nence of gleaned evidence, or relevant details on the

HOW and WHY aspects of the work with relation to

system design and development schedules. The HEPP

is normally associated only with physical design proc-

esses. The author believes that in order to avoid such a

situation, system functionality currently defaulted to

human work should be considered from the onset of

system design. In addition, such consideration of HF

early in the system development life cycle would allow

a proper trace of HF-related functions throughout de-

sign and development. Furthermore, it would support

effective HF allocation of time and effort. Improved HF

consideration should also support improved iteration

and consideration of how HF-related system problems

should best be addressed and HF products included in

system design related trade-off processes.

Nevertheless, poor design is a result of more than ill

advised and late use of HF; it is a result of many factors

including poorly integrated team work, bad planning,

poorly conducted trade-offs in design, and inadequate

system requirements. More cross-education is needed

within the engineering disciplines. Historically, the fre-

quent result of military system design and development

was crisis as system acceptance approached (MacLeod

& Taylor, 1993). This crisis is still equated by many

Figure 1. STANAG 3994�General Model for a Human Engineering Program.

118 MACLEOD

forms of ill-advised compromises, forms too numerous

to detail here.

5. SOME REASONS FOR HF ADOPTIONOF THE SE MODEL

In contrast to the traditional HF approaches presented

earlier, the generic SE model described by Figure 2,

albeit at a high level, shows a controlled and iterative

approach to system design throughout a design life-cy-

cle. All aspects of design can be subject to a multi-dis-

ciplinary process. Such an approach also provides a

mechanism for maintaining traceability of system func-

tions back to the requirements specification.

However, HF is normally only seriously considered

beyond a very basic level past the processes of Func-

tional Analysis and has little traceability to earlier

stages outside those provided by HF-related Con-

straints. Note, however, that these HF Constraints will

always need to be considered by design, e.g., human

physical space requirements plus certain human per-

formance limits.

There are many other factors involved in the poor

integration of HF alongside engineering design proc-

esses. Some factors are dictated by traditional social and

professional practices. Nevertheless, part of the engi-

neering community�s failure to see the usefulness of HF

as a complementary process to other system design

specification approaches, outside the often-parochial

nature of HF practice, is confusion of terms promoting

poor communication between different design disci-

plines. As an example, there are different meanings for

�Function� and �Task� within and between different

disciplines including HF. For the sake of clarification,

the next section considers some of the meanings as-

cribed to the terms Function and Task.

6. DIFFERENCES IN THE MEANINGS OF�FUNCTION� AND �TASK�

The word �Function� has different meanings depending

on the context of use and the profession of the user. It

is worth emphasizing these differences because the

word is used too glibly in discussions related to any

defined system design life-cycle (MacLeod & Scaife,

1997). This paper will present its own definitions in

support of its arguments. However, it is not the purpose

of the paper to suggest an all-encompassing definition

of any term. It must be emphasized that with any one

approach to questions of design, design being an inter-

disciplinary activity, there should be a good apprecia-

tion of the other approaches and definitions that also

Figure 2. Example of a generic SE process model (after IEE P1220).


influence the design processes�no one definition be-

ing necessarily all encompassing.

Thus, functions may solely reside within the engi-

neered system or be unique to the work performed by

the system operator. The term Function has a common

underlying theme in purpose, but its interpretation is

influenced by diverse and undefined nuances.

6.1. Definitions of Function and Task

In support of the arguments in this paper a Function is

stated to be a system property that is latent until re-

quired, has an associated level of performance, and is

appropriately evoked by engineered automation or the

system operator through a requisite application of ef-

fort.

A loose definition of a Function is that it is anything

that a system might be required to do. For a coverage

of various forms of system functions see Jackson, 1997.

A Task can be considered as a system�s planned

application of its functionality toward the satisfaction

of explicit system goals. In contrast to a Function,

consideration or performance of a Task includes an

appreciation of the requisite time and effort needed to

support that task.

Furthermore, tasks may evoke one or more system

functions. Also the form of these functions may vary

between heuristic and algorithmic bases. Thus, the

eventual task-related performance of functions by man

and/or machine, partly set during the iterative and

physical processes of design, can be represented by

tasks undertaken by the engineered system, the system

operator, or both. For example, an engineered avionics

system can be designed to automatically adjust control

surfaces and engine power to safely maneuver an air-

craft, the associated system functions predominately

algorithmic in nature. However, the initiation of that

maneuver task is directed by the operator through some

form of control activity. This control activity can be as

simple as a single physical setting of a flight parameter

on a sub system or a system input by operator voice

command. Currently, the correct selection of a system

control activity is normally supervised and managed

through the operator�s use of their cognitive functions,

these system functions being predominately heuristic in

nature.

7. THE MEANING OF COGNITION

Cognition has commonly been associated solely with

aspects of human work. However, the term cognition (a

term covering the use of knowledge, operator�s under-

standing, judgment, anticipation, directing, application

of skills, and system control) can apply to functions and

tasks resident in either man or machine or both (Holl-

nagel & Woods, 1983).

Activities and actions are necessary to the actual

performance of tasks. Mediating between operator sys-

tem tasks and activities, to effect appropriate operation

of the system, are the operator�s system-related Cogni-

tive Functions or SCFs. The �Cognitive Function� in

the process refers to the specific SCFs associated with

operator cognition and related to the translation of the

�intended task� into the �activity.� Note that all opera-

tors will also have a repertoire of nonsystem-related

cognitive functions that may or may not affect their use

of SCFs, e.g., inter-crew communications. Thus:

Intended Task >> Cognitive Function >> Activity >>System Feedback

All tasks are governed by plans devised as guides

toward the achievement of work goals. In the case of

the human operator, the proactive heuristic-driven con-

sideration of these plans produces intended operator-

based tasks associated with a system. During system

operation, these intended tasks are reactively revisited

and revised considering the system�s operating environ-

ment and the intended quality of performance. This

revisit produces the dynamics of task-related activities

involving the redirection of system effort and functions

toward the fulfillment of system work goals. The per-

formance of these task-related activities is fundamental

to the assessment of system situation and the mainte-

nance of operator Situational Awareness (SA). Detailed

discussion on the properties of SA is beyond the scope

of this paper; the reader is referred to Taylor and Selcon,

1994; and Endsley, 1996. These activities are crucial in

support of the direction and control of the MMS to-

wards the satisfaction of the system work goals.

7.1. Situation Assessment and SystemCognitive Functions (SCFs)

System Cognitive Functions (SCFs) are cognitive func-

tions that are considered to be an integral part of an

MMS�s functionality. SCFs incorporate knowledge of

the architecture of the engineered system, that of the

expected system task performance, an ability to assess

that performance throughout task execution, and as-

sessment of the influences of the overall-operating en-

vironment. They direct and sustain MMS situation

assessments to assist in the determination and control

of the quality and quantity of performance required

from the whole system. Indeed, their essential role is in

ensuring the maintenance of overall stable and pertinent

MMS management and control. They represent a fo-

cused understanding of the state of system performance

with relation to MMS work goals and are associated

120 MACLEOD

with feedback to the operator, and/or engineered sys-

tem, on assessments of both the quality of overall

system performance and the quality of operator control

of that system.

7.2. Intrinsic System Differences BetweenAwareness and Assessment

Human resident SCFs reside under mental processes

and states of attention and are different from the SCFs

as might be related to engineered systems. The human

SCFs have unique properties of awareness and assess-

ment; the machine solely has properties of assessment.

Thus the SCFs related to engineered components of

systems do not imply any awareness (an engineered

system is not sentient), but SCFs are related to a de-

signed machine knowledge of a required performance

tag placed on many system functions. SCFs assist the

maintenance of overall SA by the system through assis-

tance to the system operator; however, the SA associ-

ated with a system does require consciousness. This can

be argued to exist in two forms. One form exists as

human consciousness that requires an awareness of the

system operating environment and system goals. The

other form is arguably a form of consciousness in that

it requires an �awareness,� within and between a sys-

tem�s engineering components, this of the engineered

system operating environment and some system goals

(Haig, 1998). This paper will consider only the first

form, as the second form is still hypothetical.

7.3. Consciousness, Intelligence,Cognition, and Expertise

�Consciousness� is the basic ingredient of awareness

and is the property that links the object that has con-

sciousness with its environment and community. The

quality of conscious behavior depends on intelligence.

�Intelligence� has many definitions but can be de-

scribed as an ability to adapt to different and changing

environments. �Cognition� encompasses the processes,

analysis, and knowledge/experience basis, currently as-

sumed to reside in the mind, that are sustained and

supported by intelligence and consciousness. Quality

manifestation of cognition, the product of many diverse

cognitive activities, is represented by high operator

expertise in work performance.

Currently, the fusion of the areas encompassed by

cognition requires the application of human expertise

in the workplace. The machine can accurately assess

situations and conditions within its immediate remit,

and can perform quality and timely work. Currently, the

machine on its own has no consciousness, knowledge

of the purpose of its inbuilt processes, or of the reasons

supporting its performance goals. The ultimate man-

agement, control, and direction of the machine reside

with trained human operators, supervisors, or manag-

ers.

8. AUTOMATION AND ROBOTICS

As the capabilities of technologies improve, and with it

an associated capability to use that technology, the share

of the technology in the control of the overall system

increases. It is a truism that new technology introduces

new and more numerous forms of system automation

which in turn change the nature of operator work within

an MMS. New technologies can also support Robotics.

8.1. Differences Between Robotics andAutomation

Robotics can be seen as different from automation in

that automation may be expressed as the designed ap-

plication of technology to engineered system work

tasks, whereas Robotics is purely concerned with the

application of technology to work tasks associated with

human machine systems. Further, Robotics is normally

seen as being concerned with the application of tech-

nology to work tasks previously performed by human

operators.

Robots require an integral understanding of human

work, or work related requirements, to perform effec-

tively. Most current Robots, e.g. as used in car assembly

lines, consist of algorithmic-based automation of activi-

ties previously undertaken by the use of human work

skills that showed a high physical component. How-

ever, increasingly fault detection systems are being

employed, replacing the human inspector, and such

systems need to be built with a knowledge of SCFs

through an understanding of pertinent operator heuris-

tics. Future intelligent systems, which may go beyond

the current definition of a Robot in that they perform

forms of work not previously performed by a human or

machine, will require a knowledge of SCFs as well as

an engineered emulation of other relevant properties

approaching consciousness, intelligence, and cogni-

tion. In the future, automation and robotics may become

synonymous.

8.2. Automation, Assessment, Awareness,and SCFs

Automation uses system situation assessment from the

standpoint of that system�s engineered design. How-

ever, the assessment of overall system performance

with relation to the achievement of goals still requires

system feedback to the operator to maintain the opera-

tor�s and, therefore, the MMS�s SA. Good SA is argued


to be an intrinsic part of complex human skills and is

necessary for the operator�s effective direction of the

system toward the satisfaction of work goals and the

maintenance of those skills (MacLeod, Taylor, and

Davies, 1995).

It is suggested that human functions cannot be de-

scribed or captured in the deterministic fashion cur-

rently required by mainstream engineering disciplines.

Human system-related functions must be described in

a fashion that is equitable to their nature and yet allows

their complementary specification within the engineer-

ing process, or to any equivalent specification and de-

sign process. The evocation of such functions to assist

human task and system task performance must rely on

human skill, expertise, and SA within the context of

both overall system design and its later operational

performance. Thus, it is not only the specific skills and

anthropometrics of the human operator that must be

specified during early requirements capture for a sys-

tem (i.e., as contained in a Target Audience Description

[TAD]), but also the necessary SCFs required by the

MMS.

The form that would be required of these SCFs

would logically depend on the associated suite of sys-

tem functions and their intended performance and ap-

plication. Additionally, an operator�s full suite of

cognitive functions will still be required to assist the

safe and effective direction and control of the system

and deal with the uncertainties of the environment. An

example of these latter functions would be those sup-

porting recovery from MMS-related operational errors.

Following from the above, the physical design of an

MMS would be performed by iteratively approaching a

refinement on the logically specified user, system, and

performance requirements, this as a means of optimiz-

ing the complementary contributions of man and ma-

chine to support an achievement of expected system

performance. Throughout the physical design phase,

the suite of system functions would be refined through

a series of controlled multi-disciplinary trade-offs. In

any sense of allocation of functionality to man and

machine during physical design, this would be mainly

an allocation of levels of control considering the de-

signed level of automation and the control/feedback

needs of the human in command of the system.

9. A MODEL OF MMS CONTROL

Figure 3 illustrates a simple model of an MMS system

in support of the previous discussion. From the model

it can be seen that regardless of the sophistication of an

MMS, SA and the catering for uncertainty are distinctly

placed in the area termed Cognitive Control. To what

eventual extent the complete system-related functions

Figure 3. A model of Control for an aircraft MMS (part acknowledgment to Neisser, 1976 and Hollnagel, 1995).

122 MACLEOD

in that area can be eventually imbued into the engi-

neered machine is problematical. However, many sim-

pler cognit ive functions can already be

engineered�for example, as shown in aircraft colli-

sion-warning systems, in Fault Detection, Inspection,

and Rectification (FDIR) systems, and in sophisticated

fighter fly-by-wire systems. The �Situation Assessment

Module� represents these simpler functions in the

model. Currently, a simple rule-based program, a more

sophisticated Knowledge Based System (KBS), a neu-

ral net, or/and some fuzzy logic routines might reside

in this module. For discussion on one use of a �Situation

Assessment Module� see Scaife and MacLeod, 1998.

The difficulty in communication between the Sensor

Control area and the Cognitive Control area relies on

the form, timeliness, and appropriateness of informa-

tion, advice, and feedback one to another. If properly

designed, the creation of SCFs should assist the design

of communication by bridging between, and alleviating

differences between, the viewpoints of the engineered

machine and the operator. An example is the advice

produced by a high quality KBS.

The greater the degree of system automation, the

greater will become the importance of system feedback

to the operator for the maintenance of correctly directed

system effort. With the burgeoning system complexity

associated with each new system, it is important that the

feedback to the operator be based on a system function-

ality that has embedded cognitive functions in its suite.

The engineered design must allow a degree of assess-

ment by the machine of the working environment, the

situation, and the quality of the performance of the

MMS components�including the system operator

where possible. Such SCFs are needed in order that a

system can include situation assessment as an integral

part of the engineered control of the system. Thus

system related SA could include an anticipation of

future events in the form of advice, and an associated

timely and effective feedback to the operator of the

implications and consequences of ongoing control

processes on system performance.

Certification standards for computer aircraft sys-

tems, such as ARP 4754 (1996), point out that require-

ments are allocated to people, as well as systems.

However, this allocation is the very area where engi-

neering design is flawed. As already argued, SE often

fails because in that it does not encompass system

pertinent human functionality within a system to allow

such a requirements allocation, although it effectively

supports the transition from system �logical� require-

ments to an allocation of �physical� engineering re-

quirements.

10. CONSIDERATIONS ON THESPECIFICATION OF MMS SCF

The level at which the SCFs must be considered by a

system specification, particularly at different stages of

the system design and development, will in itself be a

difficult task. However, there are many other issues

associated with the adoption of cognitive functions

within design processes. Some of these issues can be

amplified by considering future concepts such as the

EC.

The concept of the future Electronic Crewman (EC),

or Pilot Associate, is that the EC will perform many of

the tasks currently performed by the pilot. These tasks

will include aircraft and system control, under the tute-

lage of the pilot, but the EC would also aid the pilot with

the usage and interpretation of sensor data, the handling

of equipment faults, advice on the criticality of situ-

ations, and assistance to the operator when required in

the form of pertinent advice. All these tasks raise issues

related to the specification of SCFs. Taking the case of

the EC, how can it be made to appear aware of the pilot,

how can it present information or advice that comple-

ments the pilot�s own situation assessment, how is its

use validated, how is such a system debriefed after a

flight and then refined (trained), how can such a system

be certified for airborne use? (MacLeod, 1997).

There are many problem areas associated with the

concept of SCF specification of any advance avionics

system. However, the automation/robotics/EC created

by new technology cannot be sensibly introduced as

part of a quality product if its purpose in an MMS

cannot be adequately specified with relation to the

nature of the system functions of both the technology

and the human operator. As an associated example, a

current simple avenue for selecting system functional-

ity, appropriate for inclusion in Human Computer In-

terface (HCI) design, would be to ensure that in the

designed information forms are not confused and that

the operator is presented with information that is related

to the performance of current tasks. One information

classification scheme considers HCI functionality un-

der what the operator �Requires Constantly,� �Requires

as Appropriate,� �Requires by Request,� and �Never

Requires.�

10.1. Specification of SCFs

This short paper cannot fully delineate how SCFs can

be specified, although some suggested avenues have

been discussed. However, they must complement engi-

neered system functions. It has been argued that they

fill the gap, if any, between system-specified require-

ments and the requirements that have to be met to


support system certification. Examples of prospective

systems have been considered in the paper, e.g., KBS

and EC.

However, asking what contribution the human needs

to make to support MMS-specified performance re-

quirements must govern the approach to specifying

SCFs. Human support to a system is supplied by the

dynamic and goal-driven application of human cogni-

tive functions, behind which are the heuristic-based

processes of expertise not considered to be within the

context of this discussion. Generally this human sup-

port can be classified as residing in forms of analysis,

anticipation, direction, control, and management/su-

pervision of MMS tasks and their performance. All

these forms of human support involve anticipation or

prediction of the meaning of understood current system

performance to the future goal-associated operation and

performance of the system, often in the context of an

uncertain operating environment. Thus the contribu-

tions also involve an understanding of system function-

ality, at an appropriate level, and the level of system

effort required to convert that functionality into the

system activities necessary to achieve a desired per-

formance from the system.

It follows that particular levels of engineered, or

�hard,� system functions, when dynamically evoked as

activities, require specific forms of cognitive function-

ality in support of their associated performance. As a

simple example, considering a high level customer/user

requirement of: �The MMS shall be capable of (Pro-

vide) computation of aircraft position to aaa accuracy,

within lll limits, at rrr rate, within lll bands of Latitude�

gives no indication of the use and means of interpreting

the required computation. A high-level cognitive func-

tion might be used to fill this gap as: �The system shall

be capable of assisting and complementing (Provide)

aircrew interpretation, supervision, management, and

control of navigation systems.� From high-level func-

tion descriptions, the need for lower-level MMS-related

cognitive type functions can be made explicit when

considered alongside their associated �hard� MMS re-

quirement. Further illustration is given by some decom-

position examples shown in Table I.

11. FUTURE ASSOCIATED ISSUES

As previously argued, the more hidden and complex the

processes involved in a system�s software, and the

interfacing and interaction between systems, the more

difficult it is to present analogues of the processes to the

operator as adequate feedback in support of the desired

operator levels of control of the system. In current

aircraft systems, we are approaching the limits of the

aircrew ability to remain in control of the aircraft far

less for them to direct its employment (Burrett, 1996).

Technology is advancing at a fast pace, but the human�s

ability to properly utilize it follows far behind.

One possible avenue for giving technological assis-

tance and advice to the system operator, and to amelio-

rate operator problems in control, is for the technology

Table I. A Comparison of Complementary System Cognitive and System �Hard� Functions

124 MACLEOD

not only to be designed as a tool for the operator but

also to be engineered to assist the operator. KBS is one

of many currently mooted technologies intended to

provide assistance through advice to operators� skilled

use of their cognitive functions. A concept for such

assistance is encapsulated in the concept of an aircraft

EC in which the operator is assisted by technology that

emulates some of the teaming functions of an extra

crewman. However, within the context of system opera-

tion, advice has some important differences to the mere

presentation of information. It is argued that the devel-

opment of methods and tools to capture SCFs will

promote the appropriate specification/presentation of

advice.

11.1. Natures of Information and Advice

Firstly, to consider information. Currently engineering

design performs system tasks related to specified sys-

tem functions, and presents the results or status of these

tasks as a required feedback of data and information to

engineered subsystems and to the human operator of the

system. Engineered system data can be said to be a

collection of facts and figures that, by the application

of appropriate rules, is transformed into information.

Information in itself is primarily system function re-

lated and situation independent. Only in its timely HCI

presentation as task feedback has it any situation rele-

vance. However, system-generated advice is informa-

tion that is considered within a specified context and

translated into tips to the operator on the effective

present and future employment of the system, this from

the �point of view� of the engineered system. Moreover,

whereas an operator had to validate data and informa-

tion against certain rules and measures, advice will

additionally have to be critiqued by the operator with

relation to its operating context, its pertinence, and its

usefulness. Thus, advice must not only be related to the

designed intention of the advice, but also must consider

the recipients� range of expertise and the form of train-

ing required. Furthermore, advice must contain both

extensional and intensional meaning to make sense to

the operator (Hayakawa & Hayakawa, 1990). Table II

illustrates the basic differences between the intensional

and extensional worlds and the bridging nature of ad-

vice between these worlds. The intensional world con-

sists of knowledge and beliefs mainly received though

communication of the written or verbalized word. The

extensional world is the world as experienced physi-

cally by the system operator. Each world strongly ef-

fects changes on the other.

Thus, it is suggested that good advice can comple-

ment operator expertise to promote an overall system

SA. Moreover, it can be a means of reducing the bias of

operator beliefs and heuristics (Tversky & Kahneman,

1974) whilst supporting the work of differently skilled

operators than hitherto employed by the organization

utilizing the new system.

11.2. Comparisons Between Informationand Advice

Operator understanding and use of information and

advice is generated through cognitive activity and, with

relation to system operation, more specifically by SCFs.

Information and advice are both meant to provide inputs

to the operator assessment of the flight situation and

thus enhance SA, operator system-related work per-

formance, and the development of operator skills. How-

ever, advice is information that has been transformed to

present not only a timely situation assessment but also

an extrapolation from that assessment into a recommen-

dation to the operator on future action. Whilst informa-

tion can be said to sustain the operator�s mental model

of the current environment and be a basis for the opera-

tor�s anticipation of future activity requirements, sys-

tem advice additionally encompasses readiness and/or

prediction for future system activity. Such prediction

implies a good system-based knowledge of the current

Table II. Advice as the Mediator Between Intensional and Extensional World


situation and appropriate operational tactics (MacLeod,

1998).

Like information, advice should be presented

through a specified HCI dialogue and format. The qual-

ity of generated advice will not only depend on the

efficacy of the designed system, it will depend also on

the effectiveness of the HCI dialogue, format, and tim-

ing in the proffering of that advice. The HCI dialogue

will have to be devised to prevent ambiguity in its

interpretation by the operator�it should require mini-

mum translation by the operator and must not mislead.

It must be specific, pertinent, constructive, and not

�chatty� or cause annoyance. Following the require-

ments of all good advice or instruction, the advice must

be given in an active and positive form.

Advice can also be considered to reside in different

categories, each of which may deserve a separate con-

sideration on their quality. As an example, it can be

argued that generated advice could be assigned to the

following categories:

1. Reminders of important details related to antici-

pated future actions.

2. Timely predictions of required operator activi-

ties.

3. General work or tactical guidance.

4. Situation assessments.

5. Advice on recovery from consequences of imper-

fect operator or system actions.

6. Handling and feedback to the operator of routine

but onerous system management tasks.

The important differences between the nature of

advice and information imply that this topic area re-

quires investigation to determine its importance to fu-

ture aircraft system certification.

11.3. Advice-Associated CertificationQuestions

One issue related to system-generated advice that is

certain to be important in system certification is the

generation of hazards from advice. Could the system

provide advice that, if acted upon, would in some

conditions result in a hazard? Showing that this could

not occur may be essential to the production of the

System Safety Case in support of system certification.

In this case the �Fitness for Purpose� of the advice could

be adversely affected by the contents of the Safety Case.

Reduction of the probability of occurrence could be

tackled by training of the operators or by restrictions on

the use of the advice, but neither present an optimal

solution.

11.4. Section Summary

Future systems will need to be specified before they can

be accepted by the customer and certified as fit and safe

for use. However, the model for specification and the

design life-cycle is unlikely to stay with the traditional

waterfall model as manifest by most HF standards and

guidelines. More likely, the model used will be near the

general SE model modified to allow specification up-

date in the case of systems utilizing a large amount of

SCFs such as postulated for the EC

12. FINAL NOTES ON THE ARGUEDAPPROACH

The above arguments advocate a human-orientated

contribution that complements and arguably improves

the logical SE specification processes and subsequent

system design and development life-cycle processes.

MMS are increasingly entering service with perform-

ance capabilities less than those required. This problem

is partly the fault of current HF approaches to MMS

design, approaches that lack any sensible justification,

rigor, and purpose with relation to their application to

new technologies and systems. One possible avenue to

promote HF value to design is to complement systems

engineering by the early integration of SCFs into the

logical and implementation-free user and system re-

quirements capture processes. This would promote a

new emphasis on the capture of the heuristics of system

cognitive functions related to MMS direction and con-

trol, and highlight a need to discover effective methods

for the marriage of MMS human-associated functions

with engineered functions throughout a system�s design

and development life-cycle. It has been argued that the

mooted approach would assist in the knowledgeable

adoption of new technologies and allow a more user-

centered approach to system automation. Finally, if the

outlined approach were sensibly developed and ap-

plied, it is suggested that it would be less likely that the

operator would be left to somehow cater for necessary

system functions and tasks that the designer could not

see how to automate (Bainbridge, 1987).

REFERENCES

R. Ackoff, Science in the systems age: Beyond IE, OR, and

MS, Operations Research, vol. 21, 1973, pp. 661�671.

L. Bainbridge, �Ironies of automation,� in: New technology

and human error, J. Rasmussen, K. Duncan, and J. Leplat

(Editors), John Wiley & Sons, Chichester, 1987, pp. 276�

283.

126 MACLEOD

I. Burrett, A test pilot�s perspective on advanced avionics,

Contemporary Ergonomics 1996, Taylor & Francis, Lon-

don, 1996, pp. 20�24.

A. Chapanis, On the allocation of functions between man and

machines, Occupational Psychology, vol. 39 (1965), 1�

11.

M. R. Endsley, Theoretical underpinnings of situation aware-

ness: A critical review, in Experimental analysis and meas-

urement of situation awareness, D.J. Garland and M.R.

Endsley (Editors), Embry-Riddle University Press, Fla.,

1996, pp. 17�23.

N. Haig, The possible value of consciousness to guided

weapons and other unmanned platforms, Journal of De-

fence Science, DERA Technical Directorate, Farnbor-

ough, UK, 3 (1998), 451�454.

S.I. Hayakawa and A.R. Hayakawa, Language in thought and

action, 5th edition, Harcourt Brace Jovanovich College

Publishers, Florida, 1990.

E. Hollnagel and D.D. Woods, Cognitive systems engineer-

ing: New wine in new bottles, International Journal of

Man-Machine Studies, 18 (1983), 583�600.

E. Hollnagel, Lecture notes from Industrial Summer School

on Human-Centered Automation, August 21�25 1995,

Saint-Lary, Pyrenees, France.

IEE P1220, Standard for the Application and Management of

the SE Process, IEEE Standards Department, Piscataway,

NJ, 1994.

S. Jackson, Systems engineering for commercial aircraft,

Ashgate Publishing Ltd., England, 1997.

I.S. MacLeod and R. Scaife, What is functionality to be

allocated?, Proceedings of Allocation of Functions Con-

ference (ALLFN �97), Galway, Ireland, October 1�3,

1997, pp. 125�134.

I.S. MacLeod, Information and advice: Considerations on

their forms and differences in future aircraft systems, in

Proceedings of the Global Ergonomics Conference,

P.A.Scott, R.S.Bridger, and J. Charteris (Editors), Cape

Town, Elsevier Science, 1998, pp. 667�672.

I.S. MacLeod and R.M. Taylor, Does human cognition allow

human factors (HF) certification of advanced aircrew sys-

tems? in J.A. Wise and D.V. Hopkin (Editors), Human

factors in certification, Embry-Riddle University Press,

Fla., 1993, pp. 163�186.

I.S. MacLeod, Certification of the EC/aircrew team�A cog-

nitive control loop, Proceedings of 4th Human Electronic

Crew Conference, Kreuth, Germany, September 23�26,

1997 (in press).

I.S. MacLeod, R.M. Taylor, and C.L Davies, Perspectives on

the appreciation of team situational awareness, in D.J.

Garland and M.R. Endsley (Editors), Experimental analy-

sis and measurement of situational awareness, Embry

Riddle University Press, Fla. 1997, pp. 305�312.

MILitary STandardD (MIL-STD) 1472D, Human engineer-

ing design criteria for military systems, equipment and

facilities, US Government Printing Office, March 1989.

NATO STANdardisation AGreement (STANAG) 3994AI

First Draft, The application of human engineering to ad-

vanced aircrew systems, Edition 2, Notification Draft 1,

June 1999.

U. Neisser, Cognition and reality: Principles and implications

of cognitive psychology, W.H. Freeman and Company,

San Francisco, 1976, pp. 172.

H.E. Price, The allocation of functions in systems, Human

Factors, 27 (1985), 33�45.

R. Scaife and I.S. MacLeod, A KBS based safety engineering

and engineering psychology contribution to flight deck

safety, in Conference Volume of 1998 ERA Avionics

Conference, Heathrow, UK, November 18�19, 1998, pp.

3.2.1�3.2.11.

G. Schreiber, J. Akkermans, S. Anjewierden, R de Hoog, N.

Shadbolt, W van de Velde, and B. Wielinga, Engineering

and managing Knowledge: The CommonKADS Method-

ology version 1.0., Technical Report, Department of So-

cial Science Informatics, University of Amsterdam, 1998.

Society of Automotive Engineers (SAE) in cooperation with

the Federal Aviation Administration (FAA), Guidelines

for the certification of highly-integrated and complex

aircraft systems, ARP 4754, 1996.

R.M. Taylor, and S.J. Selcon, Situation in mind: Theory,

application and measurement of situational awareness, in

D. Gilson, D.J. Garland, and J.M. Koonce (Editors), Situ-

ational awareness in complex settings, Embry-Riddle Uni-

versity Press, Fla., 1994, pp. 67�98.

A. Tversky, and D. Kahneman, Judgement under uncertainty:

heuristics and biases, Science, 185 (1974), 1124�1131.

UK Defence Standard 00-25 (DEF STAN), Human factors for

designers of equipment, Part 12: Systems, issue 1, dated

15 July 1989.

UK Defence Standard 00-55/2, The requirements for safety

related software in defence equipment, part 1: Require-

ments and Part 2: Guidance, dated 1 August 1997.

UK Defence Standard 00-56/2, Safety management require-

ments for defence systems, part 1: Requirements and part

2: Guidance, dated 13 December 1996.

Iain S. McLeod was born in Scotland, has been a United Kingdom military aviator, a data processing

manager of a signal processing laboratory, and is now working as a Principal Consultant at Aerosystems

International Ltd, a UK information technology and systems house. He holds an MSc in Industrial

Psychology from Hull University. Mr. McLeod has worked in Aerospace for more than 35 years. He is a

UK Chartered Occupational Psychologist, a UK Chartered Engineer, and is also a Registered European

Ergonomist through the Council for Registration of European Ergonomists.


Documents

A case for the consideration of system related cognitive functions throughout design and development