25 juin 2010 Interactive Genetic Algorithms for Creative Enhancement in UI design Dimitri Masson Alexandre Demeure Gaelle Calvary 1

25 juin 2010

Interactive Genetic Algorithmsfor Creative Enhancement in UI design

Dimitri MassonAlexandre DemeureGaelle Calvary

1

2

Problem statement

• User Interface (UI) design is a creative process

• General question: How to support designers’ exploration of the design space?

Instantmessenger

…

…Design spaceof possible UIs

My research proposal

• Presentation of generated design solutions for the application under study

• Motivation– Compliance with designers’ practice: samples browsing– Sources of inspiration

3

Outline

• State of the art

• Approach

• Prototype

• Early evaluation

• Conclusion and future work

4

State of the art

• Creativity– Creative person

– Creative product

– Creative process

• Creative evolutionary systems– Interactive Genetic Algorithms (IGA)

– IGA & HCI

• UI design process– Design process

– Tools

– Model Based UI

5

State of the art





– IGA & HCI


– Tools

– Model Based UI

6

State of the art





– IGA & HCI


– Tools

– Model Based UI

7

State of the art: Creative process

• A Darwinian process

8

Variation & recombination

Selective retention



9


Selective retention



10


Selective retention

State of the art





– IGA & HCI

• UI design process and places of creativity– Design process

– Tools

– Model Based UI

11

State of the art: IGA

Interactive Genetic Algorithms: Suitable for large problem space for which

– no analytical method can be applied– but candidate solutions can be evaluated

12


Selective retention

Performed by the system Performed by the designer

13

State of the art: IGA

Init a population of candidate solutions

Evaluate candidate solutions

Apply mutation and crossbreedings

Select promising candidate solutions

Design space

0.1

0.2

0.2

0.22

State of the art





– IGA & HCI


– Tools

– Model Based UI

14

• Three contributions: [Monmarché 1999 & Oliver 2002] [Quiroz 2007] [Plessi 2008]– Input: one UI sample– Dimensions of the design space: fonts, colors, layout– Output: definitive UIs for the end-user not for the

designer

15

State of the art: IGA & HCI


The exploration of the design space is limited

• Evolution is limited to predefined elements

• The UI remains basically the same

Absence of high level description of the UI.

16



As a result:


• The UI remains basically the same – No widget substitution

17

dd/mm/yy



As a result:


• The UI remains basically the same – No widget substitution– No UI enrichment

18

dd/mm/yy

dd/mm/yySelect a date:

State of the art: coverage





– IGA & HCI


– Tools

– Model Based UI

19

State of the art: Model based UI

• High level description of the UI: concept & task model

20

Transformation 1

Transformation 2

Transformation N

• Transformations

…

Outline


• Approach

• Prototype


• Conclusion and future works

21

My approach

• Three key points:#1 Combination of IGA and Model Based UI at C&T.

#2 Evolution of transformations from C&T UI

#3 Support from a knowledge database

22

Concept and task (C&T) Transformations(C&T UI)

IGA

Evolve Transformations(C&T UI)

uses

23

My approach: #2 transformation

• Transformation = one tree– Edges: Select C&T elements– Nodes: Translate C&T elements into UI elements

Root of transformation

Select choices

Translate into radiobuttons

Select abstract tasks

Translate into windows

Select descendants abstract tasks

Translate into frameswith a red background

Root of the task

24

My approach: #2 Genetic operations

• Mutation– Edge: changing selector– Node: Changing translation– Add/remove nodes

Select choices

Translate into radiobuttons

Select abstract tasks

Translate into windows

Select descendants abstract tasks

Translate into framesTranslate into comboboxes

Select children abstract tasks

24

25

My approach: #2 Genetic operations

• Crossbreeding

My approach: #3 Knowledge database

• Widget representations for each task

• Attributes for each widget and manipulation function

26

Choice of date

Entry textFormatBackground colorForeground color

CalendarWidthHeight

23/11/08

Outline


• Approach

• Prototype



27

• VIDEO

28

Prototype: Implementation

• TCL

• COMET: a widget toolkit [Demeure 2008]– Task level interactors supporting multiple presentations– Database storing design knowledge

29

Outline


• Approach

• Prototype



30

Early evaluation

• The purpose is to– assess the effectiveness of the approach by collecting as

many ideas and practices as possible– identify new areas for improvement

• The purpose is not to– assess the creativity improvement– evaluate the usability of the prototype

31

Early evaluation: Protocol

• Protocol defined with N. Mandran, sociologist

• Face-to-face semi-structured interviews– Questionnaire about design activity

• Personal activity, Design practices, tools, etc.

– Sketch design of an Instant Messenger• e.g.: Sketches done by E1, Appendix C

– Manipulation of Magellan• Thinking aloud technique

• 11 participants: Designers, HCI experts, Students

• 2h/interview + 4h/post-analysis (> 2 weeks)

32

Early evaluation: Results

I conducted an experiment to test 3 hypothesis

1. Designers find benefits from the IGA approach.– “Not perfect but interesting”.

– The process enable to reject bad UIs.

2. Designers find the evolution of widgets valuable.– Presentation of elements in the actual UI.

3. Designers find the UI enrichment valuable. – Few even notice the enrichment process.

As well as eliciting unforeseen needs about such approach.– Need for layout support.

– Need for User Interaction.

33

Outline


• Approach

• Prototype


• Conclusion and future work

34

Conclusion: My contributions

• Increment on previous approaches: IGA & MBUI– Transformation as individual representation– Widget substitution– UI enrichment

• Early inquiry of designers’ needs – Previous work focus more on the IGA than on the HCI

• Magellan

• [Masson 2010]: ACM EICS, Berlin, June 20-23

35

Conclusion: Difficulties

• Large scope of the state of the art

• Automatic generation of UI is complex

• Conducting experiment is complex– Very long to set up– Require experience

• IGA requires a good modeling of individuals and fine tuning of the process

36

Conclusion: Future work

• Direct edition of the samples presented – Integration of user sketches

• Support several level of fidelity

• Partial UI evaluation and evolution

• New evaluation

37

Questions

38

Research direction

• Presentation of generated design options.

39

• A faire nb de ligne de code.

40

Early evaluation: findings

• Purpose 1: Assess the effectiveness of the approach by collecting as many ideas and practices as possible

• H1: Designers find benefits from the IGA approach

– “Not perfect but interesting”.

– The process enable to reject bad UIs• H2

• H3

– identify new areas for improvement

41

Documents

25 juin 2010 Interactive Genetic Algorithms for Creative Enhancement in UI design Dimitri Masson Alexandre Demeure Gaelle Calvary 1