October, 2005 (Albacete) 1 Adaptive User Interfaces Based on Models and Software Agents Víctor M. López Jaquero Escuela Politécnica Superior de Albacete

October, 2005 (Albacete) 1

Adaptive User Interfaces Based on Adaptive User Interfaces Based on Models and Software AgentsModels and Software Agents

Víctor M. López JaqueroVíctor M. López JaqueroEscuela Politécnica Superior de Albacete Departamento de Sistemas Informáticos

Universidad de Castilla-La Mancha Campus Universitario, s/n.02071 – Albacete (SPAIN)

Email: [email protected]

Supervisors:Dr. Pascual González López

Dr. Antonio Fernández Caballero


1. Introduction

2. State of Art in Adaptive User Interfaces Design

3. AB-UIDE: A Method for Adaptive UIs Design

4. A MAS Architecture for UI Adaptation

5. Final Remarks

CONTENTS


1. Introduction1. Motivation

2. Objectives




5. Final Remarks

CONTENTS


MOTIVATION Interaction is changing, and it will keep on changing ...

Different Platforms

Wintel

Macintosh

Different Capabilities

Different Contexts

Different Users

PDA

PCAt Home

In the streets Expert users

Rookie users



There are different platforms

Different Platforms

Wintel

Macintosh


Different Contexts

Different Users

PDA

PCAt Home


Rookie users



They have different capabilities

Different Platforms

Wintel

Macintosh


Different Environments

Different Users

PDA

PCAt Home


Rookie users



They are used in different environments

Different Platforms

Wintel

Macintosh


Different Enviroments

Different Users

PDA

PCAt Home


Rookie users



They are used by different users

Different Platforms

Wintel

Macintosh



Different Users

PDA

PCAt Home


Rookie users



Different Platforms

Wintel

Macintosh



Different Users

PDA

PCAt Home


Rookie users

We need to face designing user interfaces able to work under these different situations (adaptive user interfaces)


Design for different situations– One interface per situation considered

• High monetary cost

• High maintenance cost

• Impossible to consider all the possible situations!

– A single user interface able to adapt to all (or at least many) situations

• Easy to keep consistency between versions

• Lower maintainace cost

• Lower monetary cost

MOTIVATION


Design for different situations– One interface per situation considered

• High monetary cost

• High maintenance cost

• Impossible to consider all the possible situations!

– A single user interface able to adapt to all (or at least many) situations

• Easy to keep consistency between versions

• Lower maintainace cost

• Lower monetary cost

MOTIVATION


Hardcoded adaptation vs. Engineered adaptation– Hardcoded adaptation rules

• Adaptation knowledge reusing is hard

• Difficult to modify adaptation rules

• Hard to apply methodological processes

• Multi-platform development is almost handmade

– Engineered adaptation• Adaptation knowledge can be reused

• A standard manner of editing adaptation rules

• Adaptation can be included within a methodological process

• Adaptation code can be automatically generated

MOTIVATION

Enter amount

<<action>>

Show balance

<<action>> start

InvalidAmount<<action>>

start

error

Bank Customer

Deposit

Withdraw

Transfer

Inquiry

Recharge cell phoneChange preferences

Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>> Customerid : Longname : StringlastName : Stringaddress : StringcontactPhone : StringcurrentLogin : StringcurrentPin : IntegercurrentDeposit : Double

checkLogin() : Booleanlogout()getCurrentAccount() : AccountgetCurrentAccountBalance() : DoublegetCurrentAccountStatement() : String []

transactionTypedepositwithdrawalpaymentpositiveTransfernegativeTransfer

<<enumeration>> Entryid : Longdate : Dateamount : Doublesource : Stringtarget : Stringtype : transactionType

addEntry(amount : Double, source : String, target : String, type : transactionType, date : Date)

uses

Accountid : Longbalance : DoublewithdrawLimit : Doublelogin : Stringpin : Integer

deposit(amount : Double) : Booleanwithdraw(amount : Double) : Booleantransfer(target : String, amount : Double) : Booleanstatement() : Entry[]rechargePhone(phoneNumber : String, amount : Double) : Boolean

modalityTypetextual : Stringgraphical : String

<<enumeration>>

fontTypeArial : StringTimes New Roman : StringCourier : StringVerdana : StringGeneva : String

<<enumeration>>

Preferencesfont : fontTypefontColor : StringmodalityPreference : modalityType

updatePreferences()

uses

uses


Hardcoded adaptation vs. Engineered adaptation– Hardcoded adaptation rules

• Adaptation knowledge reusing is hard

• Difficult to modify adaptation rules

• Hard to apply methodological processes

• Multi-platform development is almost handmade

– Engineered adaptation• Adaptation knowledge can be reused

• A standard manner of editing adaptation rules

• Adaptation can be included within a methodological process

• Adaptation code can be automatically generated

MOTIVATION

Enter amount

<<action>>

Show balance

<<action>> start


start

error

Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>> Customerid : Longname : StringlastName : Stringaddress : StringcontactPhone : StringcurrentLogin : StringcurrentPin : IntegercurrentDeposit : Double





uses




<<enumeration>>


<<enumeration>>


updatePreferences()

uses

uses


Design adaptive user interfaces able to adapt to:– User’s skills, preferences or characteristics– The platform where the application is running on– The physical environment where the interaction takes place– The adaptation process should preserve the usability

Include adaptation within a development process– Reuse adaptation knowledge– A standard manner of editing adaptation rules– Adaptation code can be automatically generated

An architecture for adaptive user interfaces execution– Execute designed adaptive user interfaces– At any time apply the best possible adaptation– Support multi-platform development

OBJECTIVES


1. Introduction

2. State of Art in Adaptive User Interfaces Design1. User Interfaces Design

2. Model-Based User Interfaces Design (MB-UID)

3. Adaptation in MB-UID

4. Adaptation process

5. Software Agents in User Interfaces Design



5. Final Remarks

CONTENTS


User interfaces design approaches– Language based approaches: the user interfaces is built by

programming it using a general purpose language (C/C++, Java, Pascal, etc).

– User interfaces integrated development environments: they allow the design of the user interface interactively by means of graphical tools (Borland Delphi, Borland JBuilder, Microsoft Visual Basic, ...).

– Model-based user interfaces development environments: they allow the specification of user interfaces out of a set of declarative models.

STATE OF ART

Hard and tedious programming task.Very low level of

abstraction.

Hard and tedious programming task.Very low level of

abstraction.



programming the user interface using a general purpose language (C/C++, Java, Pascal, etc).



STATE OF ART

Hard to apply a methodological approach. Low level of abstraction.

Hard to apply a methodological approach. Low level of abstraction.



programming the user interface using a general purpose language (C/C++, Java, Pascal, etc).



STATE OF ART

Higher level of abstraction.Easier to maintain. Automatic code generation.Methodological approach.

Higher level of abstraction.Easier to maintain. Automatic code generation.Methodological approach.


Model-based user interfaces design– Based on a set of declarative models

• Task model

• Domain model

• Context of use model– user, platform and environment

• Abstract user interface model

• Concrete user interface model

• Final user interface model

– The models are transformed into an executable/interpretable presentation automatically or semiautomatically.

STATE OF ART


Adaptation in model-based user interfaces design – Each approach allows the adaptation of

some specific features:• Context aware help systems• Look & Feel• Navigation• Many of them support no adaptation.

– Most of the adaptations are personalizations– No language to specify new adaptations– No intelligent adaptation process

STATE OF ART


Adaptation process

STATE OF ART

Detect User’s Goals

Detect User’s Needs

Proposal for Adaptation

Execute Adaptation

Detect Platform Changes

Detect EnvironmentChanges

Detect User Changes

Select Adaptation

Initiative stage

User initiated adaptation

Proposal stage

Decision stage

Execution stage




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage

Adaptation process can be fired by either the user (adaptability) or the system (adaptivity).



Adaptation process

STATE OF ART




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage




Adaptation process

STATE OF ART




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage

Propose feasible adaptations given the current situation and state of interaction.

Propose feasible adaptations given the current situation and state of interaction.


Adaptation process

STATE OF ART




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage

Select the best adaptations among the proposed adaptations.

Select the best adaptations among the proposed adaptations.


Adaptation process

STATE OF ART




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage

Execute the selected adaptations.

Execute the selected adaptations.


Adaptation process

STATE OF ART




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage




Execute Adaptation



Detect User Changes

Select Adaptation

Initiative stage


Proposal stage

Decision stage

Execution stage

The process requires reasoning about which adaptation to fire, choose the best adaptations, ....

The process requires reasoning about which adaptation to fire, choose the best adaptations, ....

Multi-agent systems


Software agents in user interfaces– Interface agents dwell in the user interface to improve

user’s interaction experience.

STATE OF ART

Application’s processingAgents’

processing

User input

User interf ace updating

mechanism

User interf ace and context of use

Agents’ output

Application output

Application’s processingAgents’

processing

User input

User interf ace updating

mechanism

User interf ace and context of use

Agents’ output

Application output

Our agents use BDI mental model. BDI model is a natural manner to deal with the required decision mechanism to execute adaptive UIs.

Our agents use BDI mental model. BDI model is a natural manner to deal with the required decision mechanism to execute adaptive UIs.


Software agents in user interfaces– The design of multi-agent systems require new

methodological approches:• Extensions of Object Oriented / Knowledge

Engineering methods and techniques– Tropos– Gaia– AUML– OASIS– Prometheus– Desire– MAS-CommonKADS– INGENIAS– ...

STATE OF ART

We used Prometheus because:• It supports the whole software life cycle.• Widely used.• It provides a visual design tool.• Code generation for JACK and JADE.

We used Prometheus because:• It supports the whole software life cycle.• Widely used.• It provides a visual design tool.• Code generation for JACK and JADE.


1. Introduction


3. AB-UIDE: A Method for Adaptive UIs Design1. A Study Case: ATM UI

2. Requirements Analysis stage

3. Analysis stage

4. Design stage

5. Implementation stage


5. Final Remarks

CONTENTS


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

AB-UIDE (Agent Based User Interface Development Environment) extends usual model-based user interface development methods to support the development of adaptive user interfaces in a seamless way.– User-centred approach

– Iterative

– Covers the whole development life cycle of the user interface

– The adaptive user interfaces designed are executed on an agent-based adaptation engine.

AB-UIDE: A Method for Adaptative UIs Design

Enter amount

<<action>>

Show balance

<<action>> start


start

error


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

AB-UIDE stages:


Enter amount

<<action>>

Show balance

<<action>> start


start

error

Use Case Model

User model

Platform Model

Domain model Roles Diagram

REQUIREMENTS

ANALYSIS

DESIGN

IMPLEMENTATION

Environment Model

Task model(task and actions

temporal relationships)

Interaction objects

AIO

AUIuses presented as

Adaptation usability trade-off

<window><button>....

<window>

XML UI specification

User interface

Agent programming language

Adaptation engine

Agent programming language

Adaptation engine

class 1

class 2 class 3

role 1

role 2 role 3

Caso 1

Actor Actor 2

Caso 2

class 1

class 2 class 3 AIO

AIO

CUI

CIOCIO

CIO

Adaptivity rules


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

A study case: ATM UI– ATM UI is a user interface for an automatic teller

machine, where the user can:• Withdraw money

• Make a deposit

• Transfer money

• Recharge a cell phone

• Get the account statement

• Change the preferences for the application

– The user interface should be able to run on different platforms (bank platform and mobile platform).


Enter amount

<<action>>

Show balance

<<action>> start


start

error


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Requirements analysis stage– Use case model

• Use case sequence diagram

– “Static” context of use model• User

• Platform

• Environment


Enter amount

<<action>>

Show balance

<<action>> start


start

error

Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

2: Logout OK

: Bank Customer

: System1: Logout

HighFrecuency

The user logs into the system providing a login and a password.

Description

AbstractType

FreeContainerPresentation

Participant.checkLogin()Postcondition

NULLPrecondition

LoginTask name

HighFrecuency

The user logs into the system providing a login and a password.

Description

AbstractType

FreeContainerPresentation

Participant.checkLogin()Postcondition

NULLPrecondition

LoginTask name

HighFrecuency

InputType

Participant.login!=“”Postcondition

NULLPrecondition

The user types in the login.Description

Input LoginAction name

HighFrecuency

InputType

Participant.login!=“”Postcondition

NULLPrecondition

The user types in the login.Description

Input LoginAction name

HighFrecuency

InputType

Participant.Password!=“”Postcondition

Participant.login!=“”Precondition

The user types in the password for the login.

Description

Input PasswordAction name

HighFrecuency

InputType

Participant.Password!=“”Postcondition

Participant.login!=“”Precondition

The user types in the password for the login.

Description

Input PasswordAction name


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>



Enter amount

<<action>>

Show balance

<<action>> start


start

error

Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>


• Use case sequence diagram


Enter amount

<<action>>

Show balance

<<action>> start


start

error

: Bank Customer

: System1: Login

2: Login OK

3: Perform operation

: Bank Customer

: System1: Deposit

2: Show balance

3: Withdraw

4: Show balance

5: Transfer

6: Show transfer info

7: Inquiry

8: Show info

9: Recharge cell phone

10: Show recharge info

11: Change preferences

12: Show preferences

13: Logout

14: Logout OK


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Requirements analysis stage– “Static” context of use model

• User

• Platform

• Environment


Enter amount

<<action>>

Show balance

<<action>> start


start

error

400x350screenSize

BankATMplatformName

0025PplatformId

trueisColorCapable

trueisImageCapable

Bank ATMPlatform

400x350screenSize

BankATMplatformName

0025PplatformId

trueisColorCapable

trueisImageCapable

Bank ATMPlatform

BankOfficeenvironmentName

0028EenvironmentId

lowlightingLevel

falseisNoisy

BankEnvironment

BankOfficeenvironmentName

0028EenvironmentId

lowlightingLevel

falseisNoisy

BankEnvironment

BankCustomeruserName

0025UuserId

GraphicalModality

ArialFont

BlackFontColor

CustomerUser

BankCustomeruserName

0025UuserId

GraphicalModality

ArialFont

BlackFontColor

CustomerUser

220x144screenSize

PDA01platformName

0026PplatformId

trueisColorCapable

trueisImageCapable

PDAPlatform

220x144screenSize

PDA01platformName

0026PplatformId

trueisColorCapable

trueisImageCapable

PDAPlatform

InTheStreetsenvironmentName

0029EenvironmentId

averagelightingLevel

trueisNoisy

StreetEnvironment

InTheStreetsenvironmentName

0029EenvironmentId

averagelightingLevel

trueisNoisy

StreetEnvironment


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Analysis stage– Domain model

– Roles model

– Usability trade-off


Enter amount

<<action>>

Show balance

<<action>> start


start

error

role 1

role 2 role 3

Customerid : Longname : StringlastName : Stringaddress : StringcontactPhone : StringcurrentLogin : StringcurrentPin : IntegercurrentDeposit : Double





uses




<<enumeration>>


<<enumeration>>


updatePreferences()

uses

uses


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Analysis stage– Domain model


Enter amount

<<action>>

Show balance

<<action>> start


start

error




uses




<<enumeration>>

Customerid : Longname : StringlastName : Stringaddress : StringcontactPhone : StringcurrentLogin : StringcurrentPin : IntegercurrentDeposit : Double



<<enumeration>>


updatePreferences()

uses

uses


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Analysis stage– Roles model


Enter amount

<<action>>

Show balance

<<action>> start


start

error

BankCustomer

MyBankCustomer


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Analysis stage– Usability trade-off


Enter amount

<<action>>

Show balance

<<action>> start


start

error BANK ATM PDA

Usability criteria to be preserved.

Usability criteria to be preserved.

Weight of the criterium in the usability trade-off.

Weight of the criterium in the usability trade-off.


Connector modelLightingIndicators

HeadLamps : BooleanFogLamps : BooleanLeftBlinker : BooleanRightBl inker : Boolean

DisplayerHeadlamps

DisplayerFoglamps

headlamps

foglamps

DisplayerLeftblinker

leftblinker

DisplayerRightblinker

rightblinker

textComponent“HeadlampsLabel”

imageComponent“HeadlampsImage”adjacency

textComponent“FoglampsLabel”

imageComponent“FoglampsImage”adjacency

textComponent“LeftblinkerLabel”

imageComponent“LeftblinkerImage”adjacency

textComponent“RightblinkerLabel”

imageComponent“RightblinkerImage”adjacency

Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Design stage– Adaptivity rules

– Task model

– Interaction objects specification

– Abstract User Interface (AUI)

– Concrete User Interface (CUI)


Enter amount

<<action>>

Show balance

<<action>> start


start

error

AIOAIO

AIO

CIOCIO

CIO

Watch speedometer

Watch RPM

Watch Oil Temperature

Watch lighting indicators

Watch system messages

Watch speedometer

Watch RPM

Watch Oil Temperature

Watch lighting indicators

Watch system messages

Show speedometer

<<action>>

carBrand : StringSpeed : intStatus : String

(1,1)Speed

Show speedometer

<<action>>

carBrand : StringSpeed : intStatus : String

(1,1)Speed


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Design stage– Task model


Enter amount

<<action>>

Show balance

<<action>> start


start

error

Login Perform operation

start start

( 1,* )end

Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

: Bank Customer

: System1: Login

2: Login OK

3: Perform operation

Transitions are labelled to specify the dialog.

Transitions are labelled to specify the dialog.


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Design stage– Task model


Enter amount

<<action>>

Show balance

<<action>> start


start

error

Login Perform operation

start start

( 1,* )end

Enter login/card

<<action>>

Enter pin<<action>>

Enter login/card

<<action>>

Enter pin<<action>>

Actions and tasks temporal relationships are described by using LOTOS operators (as defined in CTT).

Actions and tasks temporal relationships are described by using LOTOS operators (as defined in CTT).

Deposit

Withdraw

Transfer

Statement

Recharge cell phone

Change preferences

Logout

Deposit

Withdraw

Transfer

Statement

Recharge cell phone

Change preferences

Logout

Enter amount

<<action>>

Show balance

<<action>> start


start

error

Tasks and actions are described specifying a set of properties to help on UI generation.

Tasks and actions are described specifying a set of properties to help on UI generation.


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Design stage– Interaction objects specification


Enter amount

<<action>>

Show balance

<<action>> start


start

error

Customerid : Longname : StringlastName : Stringaddress : StringcontactPhone : StringcurrentLogin : StringcurrentPin : Integer


Enter login/card

<<action>>

Enter pin<<action>>

Name Login

Type String

Initial value

“”

(1,1)

Login

(1,1)

Pin

Name Pin

Type String

Initial value “”

Password true


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Design stage– Abstract User Interface (AUI)


Enter amount

<<action>>

Show balance

<<action>> start


start

error

Customerid : Longname : StringlastName : Stringaddress : StringcontactPhone : StringcurrentLogin : StringcurrentPin : Integer


Enter login/card

<<action>>

Enter pin<<action>>

(1,1)

Login

(1,1)

Pin

FreeContainer Login

Login

Login

Pin

OK

AUI consists of AIOs: Inputters, Displayers, Editors, ActionInvokers and Selectors.

AUI consists of AIOs: Inputters, Displayers, Editors, ActionInvokers and Selectors.

The AIOs are grouped in containers, that help on deciding a good final layout for the UI elements.

The AIOs are grouped in containers, that help on deciding a good final layout for the UI elements.


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Design stage– Concrete User Interface (CUI)


Enter amount

<<action>>

Show balance

<<action>> start


start

error

Window Login

FreeContainer Login

Login

Login

Pin

box Login

textComponentName=“loginLabel”

isEditable=falsedefaultContent=“Login”

textComponentName=“Login”isEditable=true

defaultContent=“”

textComponentName=“pinLabel”isEditable=false

defaultContent=“Pin”

textComponentName=“Pin”

isEditable=truedefaultContent=“”

OK

buttonName=“OK”

Event Name = “event010”device=“mouse”eventType=“OnClick”

postconditionName=“postcondition004”

expression=“Customer.checkLogin()”

CUI consists of CIOs. CIOs are derived from AIOs and Interaction Objects.

CUI consists of CIOs. CIOs are derived from AIOs and Interaction Objects.

Events represent the behaviour of the system.

Events represent the behaviour of the system.


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Design stage– Conectors model

• Automatically generated


Enter amount

<<action>>

Show balance

<<action>> start


start

error

InputterLogin

InputterPassword

Participant

Login : StringPassword : St ring

checkLogin() : BooleansetLogin(aLogin : String)setPassword(aPassword : String)

password

login

ActionInvokerOK

buttonName=“OK”

textComponent“Login”

textComponent“LoginLabel”

textComponent“LoginPassword”

textComponent“Password”

adjacency adjacency

Mapping between CUI and AUI.

Mapping between CUI and AUI.

Graphical relationship.

Graphical relationship.

Mapping between AUI and Domain model.

Mapping between AUI and Domain model.


Enter amount

<<action>>

Show balance

<<action>> start


start

error

environmentEvent userEvent

platformEvent

softwareEnvironmentSensor

softwarePlatformSensor

softwareUserSensor

sensorid : Integername : String

transformationRuleid : integername : String

contextEventid : Stringname : Stringdescription : StringdataType : Object

1..n1..n

1..n1..n

produces

transformationid : integername : String

1..n

1

1..n

1

data

contextPreconditionid : Integername : String

adaptivityRuleid : integername : Stringdescription : StringhandlingPriority : integer

1..n 1..n1..n 1..n

triggers

1

1..n

1

1..nproduces

0..n

0..n

0..ntriggers

0..n

0..n

0..n

0..n

0..n

reads / writes

0..n

0..n

0..n

0..n

produces

0..1

1

0..1

1

applicability context

softwareSensor

hardwareSensor


softwareEnvironmentSensor softwareUserSensor

Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Design stage– Adaptivity Rules

AB-UIDE: A Method for Adaptative UIs Design Sensors model the

information captured from the context of use.

Sensors model the information captured from the context of use.


Enter amount

<<action>>

Show balance

<<action>> start


start

error


platformEvent



softwareUserSensor




1..n1..n

1..n1..n

produces


1..n

1

1..n

1

data



1..n 1..n1..n 1..n

triggers

1

1..n

1

1..nproduces

0..n

0..n

0..ntriggers

0..n

0..n

0..n

0..n

0..n

reads / writes

0..n

0..n

0..n

0..n

produces

0..1

1

0..1

1


softwareSensor

hardwareSensor



Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>



Context events are produced by one or several sensors. They trigger adaptivity rules.

Context events are produced by one or several sensors. They trigger adaptivity rules.


Enter amount

<<action>>

Show balance

<<action>> start


start

error


platformEvent



softwareUserSensor




1..n1..n

1..n1..n

produces


1..n

1

1..n

1

data



1..n 1..n1..n 1..n

triggers

1

1..n

1

1..nproduces

0..n

0..n

0..ntriggers

0..n

0..n

0..n

0..n

0..n

reads / writes

0..n

0..n

0..n

0..n

produces

0..1

1

0..1

1


softwareSensor

hardwareSensor



Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>



Adaptivity rules will be available to be applied if the context precondition is met. The “real” adaptation is described by means of graph grammars transformations rules.

Adaptivity rules will be available to be applied if the context precondition is met. The “real” adaptation is described by means of graph grammars transformations rules.


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>



Enter amount

<<action>>

Show balance

<<action>> start


start

error

<softwarePlatformSensor id="SPS001" name="screenResolutionPlatformSensor" dataName="/contextInfo/platformInfo/hardwarePlatform/screenResolution" dataType="_2x2_" description="Senses whenever screen resolution changes."/>

<softwarePlatformSensor id="SPS001" name="screenResolutionPlatformSensor" dataName="/contextInfo/platformInfo/hardwarePlatform/screenResolution" dataType="_2x2_" description="Senses whenever screen resolution changes."/>

SENSOR CONTEXT EVENT

<adaptationRule id="AR002" name="radioGroupsOrientationChange" description="When screen resolution is reduced, tries to fit the presentation on screen by changing the orientation of the radio button to 'vertical'." > <contextEvents> <contextEvent id="CE001"/> </contextEvents> <contextPrecondition id="CP001" name="screenResolutionShrinked"> <logicalSentence id="LS001" name="OldGreaterThanNew" sentence="SPS001.oldValue > SPS001.newValue "/> </contextPrecondition> <transformation>

radioGroupsOrientationChange

Parte izquierda (LHS)

Parte derecha (RHS)

</transformation> </adaptationRule>

ADAPTIVITY RULE


Bank Customer

Deposit

Withdraw

Transfer

Inquiry


Login

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

<<extend>>

Logout

<<extend>>

Implementation stage– User interface specification

• User Interface eXtensible Mark-Up Language

• Stores the whole user interface specification

• The specification is rendered for the target platform

– Adaption engine• Multi-agent system based architecture

• Takes advantage of the user interface specification for the application of adaptations

• The architecture applies the adaptation facilities defined in the design process


Enter amount

<<action>>

Show balance

<<action>> start


start

error


1. Introduction



4. A MAS Architecture for UI Adaptation1. Initiative stage

2. Proposal stage

3. Decision stage

4. Execution stage

5. Implementing the MAS architecture

5. Final Remarks

CONTENTS


A MAS ARCHITECTURE FOR UI ADAPTATION

Platform

Environment

User

Task

Context::=

Adaptivity rules Usability trade-off

Multi-Agent System

Adapted UI

Current UI



Multi-agent system goals

Adaptation process stages have been refined to design the goals for the multi-agent system.

Adaptation process stages have been refined to design the goals for the multi-agent system.


DispatcherAgent and AgentContextOfUse sense the context of use by means of the designed sensors.

DispatcherAgent and AgentContextOfUse sense the context of use by means of the designed sensors.


Multi-agent system overview


AgentContextPlatform, AgentContexUser and AgentContextEnvironment process the incoming data and produce the context events.

AgentContextPlatform, AgentContexUser and AgentContextEnvironment process the incoming data and produce the context events.


Multi-agent system overview


AgentAdaptationProcess proposes the plausible adaptations, selects the best ones and executes them.

AgentAdaptationProcess proposes the plausible adaptations, selects the best ones and executes them.


Multi-agent system overviewThe MAS uses all the knowledge about the UI collected at design time.

The MAS uses all the knowledge about the UI collected at design time.



Initiative stage– Adaptation can be initiated by:

• The user (adaptability)

• The system (adaptivity)

– System initiated adaptation• Sensing the context of use

– Sensors detect the events produced in the context

» Software sensors

» Hardware sensors

• Detecting the user’s current goal– What is the task the user is carrying out at a moment?

– Recurrent task sequences

– Heuristics based on the interaction data collected



Proposal stage– A set of plausible adaptations for the current situation is

proposed.– The possible adaptations to be applied are those adaptivity

rules specified at design time.

– The adaptation applicable given a context of use change are those that:

• Are fired by the context events produced by the changes in the incoming sensors data.

• The context precondition is met.

Adaptivity rules



Decision stage– How to choose the best adaptation (plan) among the

proposed ones• Compute how good or bad an adaptation (plan) is for the

user:– Migration cost: represents the physical, cognitive and

conative effort the user needs to apply in order to migrate from one context to another.

– Adaptation benefit: represents how good an adaptation will be for the user in the new context.

• Choose the one that maximizes:

Adaptation benefit – Migration cost


Decision stage– Migration cost: represents the physical, cognitive

and conative effort the user needs to apply in order to migrate from one context to another.

Discontinuity

Cognitive load

Preferences

Migration cost

Conative effort

Cognitive effort

Physical effortDiscontinuity

Cognitive load

Preferences

Migration cost

Conative effort

Cognitive effort

Physical effort


User’s mental effort required to resume the task that was carrying out before adaptation took place.

User’s mental effort required to resume the task that was carrying out before adaptation took place.




Discontinuity

Cognitive load

Preferences

Migration cost

Conative effort

Cognitive effort


Cognitive load

Preferences

Migration cost

Conative effort

Cognitive effort

Physical effort


Amount of information the user needs to understand to perform the tasks using the adapted user interface.

Amount of information the user needs to understand to perform the tasks using the adapted user interface.




They are assessed by means of metrics based on the empirical results from GOMS-based (Goals, Operators, Methods, Selection rules) evaluations.

They are assessed by means of metrics based on the empirical results from GOMS-based (Goals, Operators, Methods, Selection rules) evaluations.

Discontinuity

Cognitive load

Preferences

Migration cost

Conative effort

Cognitive effort


Cognitive load

Preferences

Migration cost

Conative effort

Cognitive effort

Physical effort





Discontinuity

Cognitive load

Preferences

Migration cost

Conative effort

Cognitive effort


Cognitive load

Preferences

Migration cost

Conative effort

Cognitive effort

Physical effort


Preferences modify the other two parameters evaluation.

Preferences modify the other two parameters evaluation.


Decision stage– Adaptation benefit: represents how good an

adaptation will be for the user in the new context.


When a context of use situation is often found, the cost should be reduced since it will allow dealing with common situations.

When a context of use situation is often found, the cost should be reduced since it will allow dealing with common situations.

The adaptations can be rejected by the user. The more times the user rejects an adaptation the less likely that adaptation will be.

The adaptations can be rejected by the user. The more times the user rejects an adaptation the less likely that adaptation will be.

Context frequency

User Feedback

Adaptation benefit

Context frequency

User Feedback

Adaptation benefit


Proposal stage– Because of the limitations of the model the system needs to evolve at

run time to improve adaptation process. This evolution has been included as Bayesian learning (as in antispam filters, for instance).

– The formula below will be applied for each selectable adaptation (producing a ranking of rules).


)(

)(*)|()|(

RP

SPSRPRSP

P(S|R) represents the probability that when R is applicable, R is choosed.

P(R|S) quantifies the compatibility between the hypothesis (the adaptation selection) and the contents of the adaptation (the adaptation itself).

P(S) represents the a priori probability that R is selected to be applied.

P(R) represents the probability of R.



Execution stage– The system executes the first adaptation in the ranking

by means of the transformation engine.

– The system checks that the application of the adaptation doesn’t violate the usability trade-off for the current platform profile created at design-time.

– If the adaptation violates the usability trade-off• Undo last adaptation

• Repeat the execution and usability trade-off checking processes for the next adaptation in the ranking until:

– One adaptation meets usability trade-off

– Adaptation ranking list is empty (no adaptation could be applied)

– An adaptation is found where ranking value is too low



Execution stage<window>

<button>....

<window>Current UI specification

using USIXML::=

Adaptions rules using graph grammars transformations

::=

Adaptions rules using graph grammars transformations

<window><button>....

<window>Adapted UI specification

using USIXML

UI code in XUL or Java

Translate the XML specification into a graph representation.

Translate the XML specification into a graph representation.

Apply the graph grammar transformations on the UI graph representation.

Apply the graph grammar transformations on the UI graph representation.

Translate the UI graph representation into usiXML syntax.

Translate the UI graph representation into usiXML syntax.

Render the usiXML specification for the target platform.

Render the usiXML specification for the target platform.


1. Introduction




5. Final Remarks1. Conclusions

2. Contributions

3. Future work

CONTENTS


FINAL REMARKS

Conclusions & Outcomes– An adaptive UI design method (AB-UIDE)

– Adaptive user interfaces execution



• A specification to capture context data through sensors modelling.

• A metamodel for adaptivity rules to provide a common syntax for adaptations specification.

• A runtime quality model (usability trade-off) to preserve usability while adapting the user interface.

• A task model enriched with dialog specification.

• An abstract user interface (AUI) model and a set of heuristics to transform the domain and task/dialog model into the AUI and the CUI.

• A graphical syntax for model-to-model mapping that allows preserving traceability in the development process.

– Adaptive user interfaces execution

FINAL REMARKS



– Adaptive user interfaces execution• An architecture based on multi-agent system for adaptive user

interfaces execution.

• The integration of the adaptation facilities designed following AB-UIDE within the architecture in a seamless way.

• A model to assess how good or bad an adaptation is given a context of use state.

• The implementation of the MAS architecture proposed.

• The implementation of a tool for the transformation of user interfaces specifications by means of graph grammars transformations rules.

FINAL REMARKS


FINAL REMARKS

Acknowledgements– This work has been supported by

•The spanish grants:– CYCIT TIN2004-08000-C03-01 project

– JCCM PBC-03-003 project

•European networks– SIMILAR Network of Excellence

– Seven month stay at BCHI (Belgian laboratory of Computer Human Interaction).


FINAL REMARKS

Contributions– Topics

•Adaptive user interfaces development related papers

•Multi-agent systems related papers

•Study cases related papers


FINAL REMARKS

Contributions– Adaptive user interfaces development related papers (i)

• López-Jaquero, V., Montero, F., Molina, J.P., González, P., Fernández-Caballero, A. A Seamless Development Process of Adaptive User Interfaces Explicitly Based on Usability Properties. Proc. of 9th IFIP Working Conference on Engineering for Human-Computer Interaction jointly with 11th Int. Workshop on Design, Specification, and Verification of Interactive Systems EHCI-DSVIS’2004 (Hamburg, July 11-13, 2004). Lecture Notes in Computer Science, Vol. 3425, Springer-Verlag, Berlin, 2005.

• López Jaquero, V., Montero, F., Fernández Caballero, A., Lozano, M.D. Towards Adaptive User Interfaces Generation: One Step Closer to People. In Enterprise Information Systems V. Kluwert Academia Publishers, Dordrecht, Holanda, 2004. pp. 226-232. ISBN: 1-4020-1726-X.

• López Jaquero, V., Montero, F., Molina, J.P., Fernández-Caballero, A., González, P. Model-Based Design of Adaptive User Interfaces through Connectors. Design, Specification and Verification of Interactive Systems 2003, DSV-IS 2003. In DSV-IS 2003 : Issues in Designing New-generation Interactive Systems Proceedings of the Tenth Workshop on the Design, Specification and Verification of Interactive Systems. J.A. Jorge, N.J. Nunes, J. F. Cunha (Eds). Springer Verlag, LNCS 2844, 2003. Madeira, Portugal June 4-6, 2003.

• López Jaquero, V., Montero, F., Fernández, A., Lozano, M. Towards Adaptive User Interface Generation: One Step Closer To People. 5th International Conference on Enterprise Information Systems, ICEIS 2003. Proccedings of 5th International Conference on Enterprise Information Systems, ICEIS 2003, vol. 3, pp. 97-103. Angers, France, April 23-26, 2003.

• Montero, F., López Jaquero, V., Molina, J.P., González, P. An approach to develop User Interfaces with plasticity. Design, Specification and Verification of Interactive Systems 2003, DSV-IS 2003. In DSV-IS 2003 : Issues in Designing New-generation Interactive Systems Proceedings of the Tenth Workshop on the Design, Specification and Verification of Interactive Systems. J.A. Jorge, N.J. Nunes, J. F. Cunha (Eds). Springer Verlag, LNCS 2844, 2003. Madeira, Portugal June 4-6, 2003.


FINAL REMARKS Contributions

– Adaptive user interfaces development related papers (ii)• Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L., López-Jaquero, V., UsiXML: a Language

Supporting Multi-Path Development of User Interfaces, Proc. of 9th IFIP Working Conference on Engineering for Human-Computer Interaction jointly with 11th Int. Workshop on Design, Specification, and Verification of Interactive Systems EHCI-DSVIS’2004 (Hamburg, July 11-13, 2004). Lecture Notes in Computer Science, Vol. 3425, Springer-Verlag, Berlin, 2005, pp. 207-228.

• Montero, F., López-Jaquero, V., Vanderdonckt, J., González, P., Lozano, M.D., Solving the Mapping Problem in User Interface Design by Seamless Integration in IdealXML. 12th International Workshop on Design, Specification and Verification of Interactive Systems (DSV-IS’2005), Newcastle upon Tyne, England, July 13-15, 2005. Springer-Verlag, Berlin, 2005 (in print).

• Montero, F., López-Jaquero, V., Lozano, M., González, P. A User Interfaces Development and Abstraction Mechanism. Artículo seleccionado en el V Congreso Interacción Persona Ordenador para su publicación en Springer-Verlag, Berlin, 2005 (in print).



– Multi-agent systems related papers• López-Jaquero, V, Montero, F., Molina, J.P., González, P., Fernández-Caballero, A. A Multi-Agent System

Architecture for the Adaptation of User Interfaces. 4th International Central and Eastern European Conference on Multi-Agent Systems (CEEMAS 2005). 15-17 September 2005, Budapest, Hungary. In Multi-Agents Systems and Applications IV. M. Pechoucek, P. Petta, L. Zsolt Varga (Eds.) LNAI 3690, Springer-Verlag, Berlin.

• López-Jaquero, V., Fernández-Caballero, A. Métricas de Usabilidad y Sistemas Multiagente en Hipermedia Adaptativa. XIII Escuela de Verano de Informática. Tendencias Actuales en la Interacción Persona-Ordenador: Accesibilidad, Adaptabilidad y Nuevos Paradigmas. ISBN: 84-921873, pp. 21-34, Albacete, España, 2003.

• Fernández-Caballero, A., López Jaquero, V., Montero, F. , González, P. Adaptive Interaction Multi-agent Systems in E-learning/E-teaching on the Web. International Conference on Web Engineering, ICWE 2003. In Web Engineering: International Conference, ICWE 2003, Oviedo, Spain, July 14-18, 2003. Proceedings. J.M. Cueva Lovelle, B.M. González Rodríguez, L. Joyanes Aguilar, J.E. Labra Gayo, M. del Puerto Paule Ruiz (Eds.). Springer Verlag, LNCS 2722, pp. 144-154. ISSN:0302-9743. Oviedo, Spain, June, 2003.

• López Jaquero, V., Montero, F., Fernández, A., Lozano, M. Usability Metrics in Agent-Based Intelligent Tutoring Systems. Human-Computer Interaction: Theory and Practice (part 1). J. Jacko, C. Stephanidis (Eds.). Lawrence Erlbaum Associates. Londrés, Reino Unido, 2003. ISBN: 0-8058-4931-9. pp. 539-543.



– Study cases related papers• Robles, A., Molina, J. P., López-Jaquero, V., García, A. S. Even Better Than Reality: The Development of

a 3-D Online Store that Adapts to Every User and Every Platform. HCI International 2005, Las Vegas, Nevada, USA, July, 2005. Volume 7 - Universal Access in HCI: Exploring New Interaction Environments.

• López-Jaquero, V., Fernández-Caballero, A., Montero, F., Molina, J.P., González, P. Towards Adaptive E-learning / E-teaching on the Web. International Conference on Technology-Enhanced Learning (TEL 2003). Procedings of International Conference on Technology-Enhanced Learning (TEL 2003). Milán, Italia, noviembre, 2003.

• González, P., Montero, F., López Jaquero, V., Fernández, A., Montañés, J., Sánchez, T. A Virtual Learning Environment for Short Age Children. IEEE International Conference on “Advanced Learning Technologies”, ICALT 2001. Proccedings of the IEEE International Conference on Advanced Learning Technologies, ICALT 2001, Okamoto, T., Hartley, R., Kinshuk, Klus, J. (eds.). IEEE Computer Society, Los Alamitos, CA., Agosto 2001, pp. 283-285. ISBN:0-7695-1013-2. Madison, USA, August 6-8, 2001.


FINAL REMARKS Future work

– Collaborative adaptive user interfaces

– Adaptation in virtual environments

– Visual and intuitive graphical tools for adaptivity rules design

– Creating a corpus of adaptiviy rules big enough

– Porting the multi-agent system to a open source agent language like JADE.

– Adding user modelling techniques to make user model evolve automatically by inference.

– Make some more usability tests for adaptive user interfaces designed and executed by using our approach.


Adaptive User Interfaces Based on Adaptive User Interfaces Based on Models and Software AgentsModels and Software Agents

Thanks for your attention

QUESTIONS & ANSWERS

Víctor M. López JaqueroVíctor M. López JaqueroEscuela Politécnica Superior de Albacete Departamento de Sistemas Informáticos

Universidad de Castilla-La Mancha

Campus Universitario, s/n.

02071 – Albacete (SPAIN)

Email: [email protected]

Documents

October, 2005 (Albacete) 1 Adaptive User Interfaces Based on Models and Software Agents Víctor M. López Jaquero Escuela Politécnica Superior de Albacete