Prague, 11 Feb 2012, Slide 1
MaxSumo and MAxEvaTrain the trainer meeting
EPOMM-PLUS
Prague
13th February, 2012
Before
After
Prague, 11 Feb 2012, Slide 2
Program for today
14.00 Introduction Friso (done)
14.10 How to Use MaxSumo Friso (done)
14.40 What is MaxEva and how does it work Karl-Heinz
14.50 Entering projects into MaxEva Paul
15.10 = Short Break = Group formation
15.20 Workshop: entering projects into MaxEva
16.40 = Short Break
16.50 Sharing your experiences Friso
17.10 Benchmarking Karl-Heinz & Paul
17.25 Closure Friso
Prague, 11 Feb 2012, Slide 3
MaxSumo:
plan
monitor
evaluate
of MM projects
3
Prague, 11 Feb 2012, Slide 4
How to use MaxSumo
1. About MaxSumo
2. Why use MaxSumo?
3. Why monitoring & evaluation?
4. How to measure the effects of MM projects?
5. How to use MaxSumo?
6. Cases
Prague, 11 Feb 2012, Slide 5
About MaxSumo
5
Prague, 11 Feb 2012, Slide 6
Why use MaxSumo?
• get more results out of your MM projects• easier to adjust• demonstrate the effects• benchmarking between projects• it’s becoming the standardized way of
MM monitoring and evaluation
Prague, 11 Feb 2012, Slide 7
About MaxSumo
• “System för Utvärdering av MObilitetsproject”• From Sweden• Further developed in the Max project• On its way to become the tool for MM project design, monitoring & evaluation• Based an old, well known methods• Simple, logical & flexible• From project start to evaluation
Prague, 11 Feb 2012, Slide 8
Why monitoring & evaluation?
8
Prague, 11 Feb 2012, Slide 9
Why monitoring & evaluation?
• to monitor = to measure & to describe what is happening– collect, store and compile data
• to evaluate :– to determine the value in a systematic way– in order to answer questions:
• has the target been reached?• comparison with other projects• learn from results• explain causal relations
Prague, 11 Feb 2012, Slide 10
Why monitoring & evaluation?
• measure the progress of a project• make it possible to adjust the project• make clear what measures are (not) effective• satisfy project officers, politicians, … yourself• create support for further actions
MM effects are often invisible !!
• MM projects tend to be cost-effective
PROVE IT !!
Prague, 11 Feb 2012, Slide 11
How to measure the effects of
MM projects?
11
Prague, 11 Feb 2012, Slide 12
When is a MM project successful?12
profits
cost savingsemployee satisfactioncorporate responsibility
accessibilityclimate &environmenthealthregional economyparking problems
“joy”“benefits”“ease”
Prague, 11 Feb 2012, Slide 13
Core indicator for success
Permanent behaviour change
by the target group
reduction of car mileage
Effects
higher revenues
employeesatisfaction
congestionCO2 new clients
This is what MaxSumo measures!
Prague, 11 Feb 2012, Slide 14
How to use MaxSumo
14
Prague, 11 Feb 2012, Slide 15
How to use MaxSumo
selecting targets & measures
1 start
project design & measuering progress
2 design & monitor
show & explainthe results
3 evaluate
enter resultsin MaxEva
select cost-effective measures
Prague, 11 Feb 2012, Slide 16
MaxSumo: the step by step approach
Mobility Management
System impacts
Assessment
Levels
A Project activities and outputs
B Awareness of mobility services provided
C Usage of mobility services provided
Assessm
ent of services
provided
D Satisfaction with mobility services provided
E Acceptance of mobility option offered
F Take up of mobility option offered
Assessm
ent of m
obility
option
s offered
G Satisfaction with the mobility option offered
H Long-term attitudes and behaviour
Overall
effects I System impacts
your activities
the results!
what you want the target group to do
Prague, 11 Feb 2012, Slide 17
Defining projects with MaxSumo (template 1)17
what you offer the target group
how many people adapt their travel
behaviour?
what you want the target group to do
what you aim for
which group of travellersyou need
what actors you need
indirecttarget group
productpromotionpriceplace
Prague, 11 Feb 2012, Slide 18
Example- test traveller (template 1)18
one month free PT ticket + information
would we succeed?
change solo driving car trips to PT on commuting trips
CO2 reduction +less congestion
employess travelling by car >= 3x/week by car
company management
indirecttarget group
Prague, 11 Feb 2012, Slide 19
Test traveller: template 1
Projectplan
overall goals
CO2 reduction +less congestion
target groups indirect target groups
target groups
employees travelling by car>= 3 times /week by car
company management of4 employers
Mobility services provided
one month free PT ticket+ information
meeting with CEO’scommunication package
Mobility option(s) offered
Change solo driving car trips to public transport on commuting trips
desired behaviour
Prague, 11 Feb 2012, Slide 20
Example: Test Traveller / Job ticket projectFramework conditions
Person-rel. factors Travel behaviour before: 5 % PT, 95 % car
Assessm
ent o
f services p
rovid
ed
MM project details: 10 posters, Intra net info, 4 meetings, 10000 E
Awareness: 48 % (480 employees)
Usage: 28 % (280 employees) particip. in info-meet.
Satisfaction: 80 % satisfied with info and test concept
Mo
bility o
ptio
n o
ffered
Acceptance: 6 % (54 people) signed a contract
Take up: During test month: 85 % PT, 15 % car
Satisfaction: 90 % of TT satisfied with commuting by PT
Effects
Long-term att & beh: After 12 months: 40 % PT, 60 % car
System impacts: Car traffic: 110.000 km, CO2: 20 tonnes
20
Prague, 11 Feb 2012, Slide 21
Level Targets Indicators Methods When
Assessm. of
servicesprovided
A MM project details
B Awareness of mobility services provided
C Usage of mobility services provided
D Satisfaction with mobility services provided
Assessm.of
mobilityoption offered
E Acceptance of mobility options offered
F Take up of mobility option offered
G Satisfaction with the mobility option offered
Overall Effects
H Long-term attitudes and behaviour
I System impacts21
Monitoring and Evaluation Plan (template 2)
Prague, 11 Feb 2012, Slide 22
Level Targets Indicators Methods When
Assessm of
servicesprovided
A MM project details XXX XXX
B Awareness of mobility services provided
XXX XXX
C Usage of mobility services provided
XXX XXX
D Satisfaction with mobility services provided
XXX XXX
Assessm.of
mobilityoption offered
E Acceptance of mobility options offered
XXX XXX
F Take up of mobility option offered
XXX XXX
G Satisfaction with the mobility option offered
XXX XXX
Overall Effects
H Long-term attitudes and behaviour
XXX XXX
I System impacts XXX XXXFinal Conference WP B 22
Monitoring and Evaluation Plan (template 2)
Prague, 11 Feb 2012, Slide 23
Stage positioning (MaxSem)
verstokte automobilist net-niet reiziger
vaste gebruikerkeuzereiziger
pre-contemplation contemplation
maintenancepreparation/ action
Prague, 11 Feb 2012, Slide 24
Cases
24
Prague, 11 Feb 2012, Slide 25
Malmö: no ridiculous car trips
Prague, 11 Feb 2012, Slide 26
Malmö: no ridiculous car trips (Sweden)
levels result(s)
assessm
en
t of s
erv
ices
pro
vid
ed
A MM project details Write down your most ‘ridulous car trip ‘isinformation: website, billboard, campaign team
B awareness of mobility services provided
50% of inhabitants is aware of the campaign
C usage of mobility services provided
D satisfaction with mobility services provided
94% says it’s good that the city starts such a campaign
assessm
en
t of m
ob
ility o
ptio
n
off
ere
d
E acceptance of mobiltiy options offered
F take up of mobility options offered
16% has intended to drive less
G satisfaction with mobility options offered
overa
ll eff
ects
H long-term attitudes and behaviour
23% has changed attitudes9% drives less -= 14.000 persons
I system impacts
Prague, 11 Feb 2012, Slide 27
Carsharing promotion in Nijmegen East (NL)
Prague, 11 Feb 2012, Slide 28
Carsharing promotion in Nijmegen East (NL)
levels result(s)
assessm
en
t of s
erv
ices
pro
vid
ed
A MM project details 8 extra sharing cars in Nijmegen East3500 flyers; personal contacts with 5300 persons
B awareness of mobility services provided
50% is aware of the letter by the city26% is aware by message in the news paper9% knows the special event
C usage of mobility services provided
870 want to receive more information1522 consider carsharing
D satisfaction with mobility services provided
n/a
assessm
en
t of m
ob
ility o
ptio
n
off
ere
d
E acceptance of mobiltiy options offered
46 new subscribers (3rd april – 9th dec 2009)
F take up of mobility options offered
Figures available at Greenwheels
G satisfaction with mobility options offered
Figures available at Greenwheels
overa
ll eff
ects
H long-term attitudes and behaviour
46 new subscribers(growth of 27%) Rest of Nijmegen: 1% growth
I system impacts 4 of 8 sharing cars are put in place permanently16 less occupied parking places, 0,54 ton CO2 reduction, 46.000 less car kilometers, app. 5-20% increase of bike use/ PT use
Prague, 11 Feb 2012, Slide 29
E-bike promotion in the Netherlands
Prague, 11 Feb 2012, Slide 30
E-bike promotion in the Netherlands
levels result(s)
assessm
en
t of s
erv
ices
pro
vid
ed
A MM project details try an e-bike/ e-scooter for one monthcommunication via employers
B awareness of mobility services provided
C usage of mobility services provided
22 employers with 3500 employees participate
D satisfaction with mobility services provided
?
assessm
en
t of m
ob
ility o
ptio
n
off
ere
d
E acceptance of mobiltiy options offered
292 employees participate
F take up of mobility options offered
?
G satisfaction with mobility options offered
Employers and employees are enthusiastic
overa
ll eff
ects
H long-term attitudes and behaviour
?
I system impacts ?
Prague, 11 Feb 2012, Slide 31
MaxSumo gives you
- Help to manage and pilot
- Guidance on survey design:
- control group
- comparison group
- one group
- Practical help: Examples of questions
- Help throughout the whole process: stop and go
- Early results
- Comparable results
To be aware of…
- Clear objective
- Clear conceptions of services and options offered
- Best if used already in the planning stage
- You don´t have to use all levels
- Only provides a structure – still the same requirements on the competence of the evaluator
31
Prague, 11 Feb 2012, Slide 32
Program for today
14.00 Introduction Friso
14.10 How to Use MaxSumo Friso
14.40 What is MaxEva and how does it work Karl-Heinz
14.50 Entering projects into MaxEva Paul
15.10 = Group formation =
15.20 Workshop: entering projects into MaxEva
16.40 = Short Break
16.50 Sharing your experiences Friso
17.10 Benchmarking Karl-Heinz & Paul
17.25 Closure Friso
Prague, 11 Feb 2012, Slide 33
MaxEva – from MAX to today
• As project database modeled on „SARA“ in Sweden• Based on MaxEvas‘ data we would like to become able to predict outcomes of
MM• Product from Trivector (SE) for MAX end 2009 – no public testing• Very low usage of MaxEva – reason many bugs and bad usability• With support from Trafikverket and KpVV – transfer from Trivector to
AMOR/EPOMM and a LOT of additional programming in 2011• 2012 complete rewriting of the programme, finished last Friday
Prague, 11 Feb 2012, Slide 34
Prague, 11 Feb 2012, Slide 35
Prague, 11 Feb 2012, Slide 36
Features of MaxEva
• Project data are publicly accessible via maps and benchmark functions• A LogIn lets you view ALL projects, but only change your own projects• It is a general evaluation tool NOT limited to MaxSumo evaluation method• It supports MaxSumo „levels“• It supports control-group and comparison group evaluation method• It supports awareness „stages“• Automatic calculation of mileage effects and CO2 effects• In the background the fleet emission data from countries „work“• Useful additional information can easily be added:
– Location on a google map– Additional files– Links (for example to ELTIS case study)
• Powerful Benchmark function• Other language versions can easily be produced (within a day, after
translation has been done)
Prague, 11 Feb 2012, Slide 37
Prague, 11 Feb 2012, Slide 38
Prague, 11 Feb 2012, Slide 39
Prague, 11 Feb 2012, Slide 40
Prague, 11 Feb 2012, Slide 41
MaxSumo data into MaxEva?41
What you offer the target group? YES
How many people adapt their travel behaviour? YES
What you want the target group to do?
YES
What you aim for?YES
Which group of travellersyou need? YES
What actors you need? PARTLY
indirecttarget group
Product?Promotion?Price?Place ?PARTLY - MaxExplorer
Map, Benchmarking,
Effects, Publication
Prague, 11 Feb 2012, Slide 42
Program for today
14.00 Introduction Friso
14.10 How to Use MaxSumo Friso
14.40 What is MaxEva and how does it work Karl-Heinz
14.50 Entering projects into MaxEva Paul
15.10 = Group formation =
15.20 Workshop: entering projects into MaxEva
16.40 = Short Break
16.50 Sharing your experiences Friso
17.10 Benchmarking Karl-Heinz & Paul
17.25 Closure Friso