Towards a Representation of AT Systems

John Gilligan Dublin Institute of Technology

Professor Peter Smith University of Sunderland

AT Systems• The classic definition of an AT system is that of

the HAAT model of Cook and Hussey.

Context

The HAAT model is an example of a PEO model

Representation Challenge

• The Challenge is to represent all elements of the model, namely PERSON, ENVIRONMENT and OCCUPATION.

• It is also imperative that the impact of ASSISTIVE TECHNOLOGY (AT) intervention be accommodated.

• Let this discussion begin with (AT)

Assistive Technology

• Assistive Technology is defined as: “Any product, instrument, equipment or

technical system used by a disabled person, especially produced or generally available, preventing, compensating, relieving or neutralizing the impairment, disability or handicap”. ISO 9999:1998

AT and State Change

• The central implication of this description of assistive technologies is that the technical aids bring about a state change. For example glasses bring about an improvement in what a user can see. Any description of the device must surely concern itself with this state change since this defines the enablement potential of the device.

• A representation scheme to model AT systems must embody State Change.

• The scheme chosen is Jensen’s Coloured Petri Nets (CPNS)

• CPNS Represent State Change -hence their suitability for this task

Example Petri Net

What CPNS Represent

• CPNS are directed graphs consisting of two kinds of nodes, places and transitions.

• Transitions represent events, processes, activities, actions.

• Places contain tokens which represent the states before and after the actions etc represented by the transitions.

How Petri Nets Work

• Tokens move across Transitions effecting State change

• This state change happens only if the guard is satisfied by the tokens at the input place

• The change is determined by the output expression

Tokens, Guards, Expressions and Outcomes

• Tokens represent components of the State. In our case, there will be tokens to represent Person, Environment and Intended Action and objects of the domain, at the very least.

• Transition Guards represent conditions which must be satisfied if the event is to take place.

• Outcome expressions describe the effects of the action

• The expression defines the subset of tokens to move across to the out put place.

P1 T1P2

G O

Places P1, P2

Transition T1

Expression E

Guard on T1 G

Outcome of T1 O

Elements of Petri Net

E

CPN Components

To Summarise

• CPNS are Graphs with Place and Transition nodes• Activity is represented by Transitions.• Prerequisites to Activity performance are described

in the Transition Guards.• The effects of the Activity are expressed in the

Outcome expressions• The domain state Before and After the Activity are

represente by tokens ate the Input and Output places.

P1 T1P2

Tokens Representing State Before ActionIncluding Person CapacitiesAnd Environment Factors

Tokens Representing State After Action

Transition Representing

Action

Outcome Expression Which defines the Change which will take place as a

result of Action

Guard Representing Barriers to Performance of Action

Input Expression Which defines

the Subset of the Input State

Currently in Focus

Petri Net to Represent Action

This is Summarised in the following Diagram

Example

In the following Diagram a CPN representing the activity of making tea is shown

P1 T1P2

Tokens Representing State Before ActionIncluding Person CapacitiesAnd Environment Factors such as location of milk sugar tea, kettle

Tokens Representing State After Action

Including Person has A Cup of Tea,-and they are happier.

ActivityMake Tea

Outcome Expression Which defines the Change which will take place as a

result of Action

Guard Person must be able to - Lift KettleWork buttonsOpen Milk carton etc

Input Expression Which defines

the Subset of the Input State

Currently in Focus

Activity Making A Cup Of Tea

Example of Activity Representation

Another Example

• The following CPN represents the activity of opening a drawer

P1 T1P2

Agent Token JohnAction Token Pull1Object TokensDrawer1 and Handle1Environment TokenE1

Agent Token JohnObject Tokens’’’

Drawer1 and Handle1Environment Token

E1

ActivityOpen Drawer

Outcome Expression Drawers position and status change

Guards Hand must fit handleAgent must be able to pull objects towards them etcRoom must have sufficient light

Activity Opening Drawer1

Tasks and Subtasks

• CPNs can represent the sub tasks of a task , the steps of an algorithm, the subroutines of a routine. For example consider.

Multi Step Petri Net

Note

• Different instances of the same subtask occur across the net

• For example We will lift the kettle and lift the sugar bowl at different times in the task

• This is important later when discussing a link to the ICF

In Summary

In order to model someone doing something in some environment, it is necessary to:

• Represent features of the person and the factors that impact on their capacity and performance of the activity.

• Characterise the activity in terms of its requirements on the person or barriers to its performance.

• Represent the outcomes or changes that take place as a result of the activity being performed.

Advantages of Scheme

• Intuitively seems to have all the elements required

• Barriers to Activity Performance can be articulated through Transition Guard

• Outcomes and Inputs are expressed as apart of same mechanism

• Rich scheme founded in strong formal computer science

AT and State Change Revisited

• State here refers to a snapshot of the world in which activity takes place.

• There are two important states , the state before the activity happens and the state afterwards.

State Before

• State Before reflects the status of the various components of the domain of activity prior to the execution of the action.

• These components include Person, Environment including objects of the domain and Action descriptors which describe the aspects of the intended activity.

• In CPNs State is represented by Tokens at the places of the net.

State Before Representation

• State Before is represented by the tokens at the Input place

• This is described in the following Diagram

Input Place

AgentTokens Representing

Person

State Before ActionEnvironmental

Tokens including Object Tokens

ActionTokens Other

Tokens

State Afterwards

• State Afterwards reflects the status of the various components of the domain of activity after the execution of the action.

• This is described by the tokens at the Output place

• This is shown in the following Diagram

Output Place

AgentTokens Representing

Person

State After Action

EnvironmentalTokens including Object

Tokens

ActionTokens

OtherTokens

Note some of these tokens are passed across the transition from input place to the output place after the action. Some are modified others not. The changes are described in the outcome expression of the transition

Moving from Input to Output Places

• This happens across a Transition• Transitions represent two aspects of Activity

through its guards and outcome expressions.• The structure of a Transition is shown in the

following

Transition

Guard which must

be satisfied

Expression of

Outcome

Guards

• In order for the activity to take place the guards on the transition must be satisfied.

• In other words the guard describes potential barriers to the execution of the action.

Outcome expressions

• The Outcome expression describes the effects of the action.

• In other words what changes happen as a result of the activity

Putting all this Together

By placing the input place to the left of the transition and the output place to the right we get a one transition CPN

Basic Structure of CPN unit

Input Place Output Place

TransitionGuard which must be satisfied

Outcome of Activity

To summarise this view of Action

• Action Has three components• Parameters of the Intended Action such as

Location, Duration etc• Guards of the Action• Outcomes of the Action• There is a Domain state before and after the

action• This is summarised in the following

Representation of Action

Person Capacities in The Context of that Action, Objects involved in the action and the environment in which the action is to take place before Action takes place

Capacity Demands of Action and Object

State of the world before Action –

State of the world After Action –

Person and environment Capacities After Action

Outcomes of Action

GuardOutcome expression

Intended Action

Modelling AT systems

• So far we have looked at using CPNs to model Activity. The next questions to answer is, how does AT impact on these models?

• So what does AT do? • In terms of Activity, what contribution can AT

make to the overall performance of the activity.

AT as Intervention

• The roles different technologies play reflect to some degree, different intervention strategies which someone like an OT can make to increase occupational performance.

1: Adapting the Task

• When the Task method is altered the same task objects are used in the same environment but the method of performing the task is altered to make the task feasible given the persons circumstances. Examples of this kind of intervention, include one handed techniques (ref) for tasks normally requiring two hands e.g. one handed dressing or typing.

• Mastering one handed methods require the capacity to learn and practice is a necessary component of this process.

Adapt the Task

CPN Structure for Task Adaptation

• In the previous example we see that task adaptation is represented in the CPN simply as an alternative path between two places.

• The idea is that if the guard isn’t satisfied on one path ,then it may be on this alternative path.

• In this way the activity may be realised in another way

2: Adapting the Environment

• This intervention emphasizes selecting and implementing an environment that enables the person to perform with current skills and abilities (Dunn et al)

• Examples of environmental adaptation include making changes to a home , to facilitate a wheelchair user perform every day activities. For example a bathroom could be adapted by the provision of a higher toilet to facilitate easier transfers, grab bars introduced for standing pivot transfers and recessed plumbing and drawers for easier access to the sink. (ref)

Environmental Change – Change Objects

CPN Structure for Environmental Adaptation

• In the previous example we see that Environmental adaptation is represented in the CPN simply as an alternative path between two places.

• The idea is that if the guard isn’t satisfied on one path ,then it may be on this alternative path.

• In this way the activity may be realised in another way by changing the object involved.

• There are also other structures which realise this

3 Prevent Barriers to Task performance

• Therapeutic interventions can prevent the occurrence or evolution of barriers to performance in context.

• Family members could be trained in a hierarchy of assists, for example verbal cues, that can help maintain the remaining skills that a person with a progressive dementing illness still has.(Rogers et al 2000)

• Technology can help prevent barriers resultant from discrepancies in task requirements and personal capacity. Consider a standard keyboard.

• If someone with poor targeting abilities is repeatedly hitting two keys at once , then a key guard can prevent this difficulty by placing raised borders around each key button, which make it impossible to hit more than one key at a time.

The CPN For Barrier Prevention

• This is complex which involves a discussion of how objects are represented and contextual capacities.

• This will be postponed until after this.

4 Restore or Establish Capacities

• Improve the capacities of the agent through some intervention

• This is really the classic exemplar of AT intervention.

Enhance Capacities

CPN Structure for capacity restoration

• A prerequisite to this task represented by the guard at the transition, is that the person be able to lift 5 kg which originally they cannot do.

• They then embark on an exercise program which builds up their basic strength.

• In our CPN we see that capacity is modified through another transition. The outcome of this transition is a change to the persons capacity. This in turn is tested against the original guard. Now the guard is satisfied and the activity can take place

• Instead of bypassing the guard we change capacity and re-test

4 Create an Environment to Enhance Performance.

• This therapeutic choice focuses on providing enriched contextual and task experiences that will enhance performance.

• Examples of these environments, include playgrounds which have soft surfaces and safe equipment which allow children to climb and swing and slide and push to the limits of physical activity without an enhanced risk of injuring themselves.

• Another example is a garden with raised flower beds (ref Holms et al)which allow ,say wheel chair users, tend to the plants and enjoy the experience and rewards of gardening.(Ref?)

CPN Structure

• In this case the enhanced environment should be reflected in the input tokens of the place.

• This should satisfy the guards for many activities

• There is no change to the basic structure of a standard CPN

Summary

• The informal CPNS presented here seem a good way to represent AT knowledge

• In surveys respondents were favourable to their use.

• Thought they were both intuitive and naturally expressive of AT knowledge

Possible Applications of This

• Model AT systems• Teach about AT systems• Investigate impact of AT on Capacity Demands

versus Capacities• Demonstrating how modelling AT systems is

rooted in Mainstream Computer Science• Can be used as the basis for Serious Games

and for Intelligent systems around AT

Issues

• Represent Agents, Objects, Actions, State• Issues representing Agents• Represent Static and Dynamic Attributes• Represent Capabilities• Use ICF, MPT and other instruments of Assessment

criteria• Some of this is done in representation of agents in

games • Borrow from these.