Embed Size (px)
Citation preview
How Busy is too Busy?: Validation of the Dutch Integrated Workload ScaleRobin Kramer BSc1, prof. dr. Addie Johnson2, Melcher Zeilstra MSc EurErg3
1University of Groningen, Faculty of Mathematics and Natural Sciences, 9747 AG, Groningen, The Netherlands, E-mail: [email protected]
2University of Groningen, Faculty of Behavioural and Social Sciences, 9712 TS, Groningen, The Netherlands, E-mail: [email protected]
3Intergo, Consultancy in Human Factors and Ergonomics, P.O. Box 19218,3501 DE, Utrecht, The Netherlands, E-mail: [email protected]
Optimal Workplace Workload
› Moderate level of workload ensures best performance[1]
• Workload too low affects level of situation awareness[2]
• Workload too high can negatively affect health[3]
› The goal is to moderate the level of workload• Identify the peaks and troughs during the task• A measure tailored to the task
[1] = Brookhuis and De Waard (2002)[2] = Endsley (2013)[3] = Spieker et al. (2002)
Subjective Assessment Scales
› NASA-Task Load Index[1]
• Multi-dimensional scale• Divert too much attention
[1] = Hart and Staveland (1988)
Subjective Assessment Scales
› NASA-Task Load Index[1]
• Multi-dimensional scale• Divert too much attention
› Rating Scale Mental Effort[2]
• One-dimensional scale• Anchor-points at uneven intervals
[1] = Hart and Staveland (1988) [2] = Zijlstra (1993)
Integrated Workload Scale (IWS)[1]
› Development• Developed for train dispatchers• Multi-dimensional descriptions of workload [2]
› Use• Periodical questioning• Concurrent monitoring of task
[1] = Pickup, Wilson, Norris, Mitchell, and Morrisroe (2005)[2] = Ames and George (1993)
The English IWS. Taken from Pickup et al. (2005).
The English IWS. Taken from Pickup et al. (2005). The original Dutch translation of the English IWS. Adaptation from Zeilstra (2007).
Original Item Dutch Translation Alternate Translation
Some spare time Enige tijd over Lichte inspanningModerate Effort Matige inspanning Redelijke inspanningVery busy Erg druk Behoorlijk druk
Behoorlijk inspannend
Enorm druk
Enorm inspannendStruggling to keep up
Moeite om het werk bij te houden
Extreme inspanning
Work too Demanding
Te belastend Overbelastend
Alternate Dutch translations
Method
› Verify that the Dutch and English scale are comparable• Magnitude Estimation[1]
[1] = Meek, Sennott-Miller, and Ferketich (1992)
Participants
› Students in an English-language program (n = 58 (30 female; age = 21.4 ± 2.2 years)
› Dutch students (n = 48 (28 female; age = 21.8 ± 1.9 years)› Train dispatchers (n = 19 (1 female; age = 43.2 ± 11.4 years)• University (26.3%), mid-vocational (42.1%), secondary school (31.6%)
› 15 participants excluded due to incompleteness, a further 12 participants due to noncompliance.
Results (1)
› 2 (Group) x 9 (Item) Repeated Measures ANOVA• Main effect on item (F(8, 672) = 517.35, p < 0.001)• No group effect, nor a Group x Item interaction.
Results (1)
› 2 (Group) x 9 (Item) Repeated Measures ANOVA• Main effect on item (F(8, 672) = 517.35, p < 0.001)• No group effect, nor a Group x Item interaction.
› Simple Linear Regressions• English language students:
R2 = 0.778, F(1,439) = 1536.45, p < 0.001• Dutch students
R2 = 0.804, F(1,331) = 1355.75, p < 0.001• Train Dispatchers
R2 = 0.823, F(1,106) = 491.70, p < 0.001
Results (2)
Alternate item Original item (item nr.) Dutch students Train dispatcherLichte inspanning Enige tijd over (3) 50.3 (20.1)* 53.8 (20.3)Redelijke inspanning Matige inspanning (4) 70.6 (21.4) 88.4 (19.1)Behoorlijk druk Erg druk (6) 92.4 (20.0) 96.2 (22.6)Enorm druk Erg druk (6) 108.7 (16.8) 112.6 (14.9)Behoorlijk inspannend Erg druk (6) 93.7 (22.8) 102.5 (21.2)Enorm inspannend Erg druk (6) 105.4 (21.1) 113.4 (19.9)Extreme inspanning Moeite om het werk bij te houden (8) 125.0 (18.4)* 127.3 (13.9)Overbelastend Te belastend (9) 141.9 (13.2) 144.8 (7.9)
Mean estimates (and SD) of the alternate Dutch translations. Asterix (*) indicates alternate items
that are significantly closer (p < 0.05) to the regression line compared to the original item.
› Simple Linear Regressions of the Alternate Dutch IWS• Dutch students
R2 = 0.831, F(1,331) = 1633.39, p < 0.001• Train Dispatchers
R2 = 0.843, F(1,106) = 568.74, p < 0.001
Results (3)
› Is the alternate Dutch IWS an improvement?• Compute the fit of the mean ratings of the two Dutch scales to the regression line of the
English IWS. Dutch Students Train Dispatchers
Original Dutch IWS
R2 = 0.813, F(1,331) = 1442.96, p < 0.001
R2 = 0.771, F(1,106) = 359.64, p < 0.001
Alternate Dutch IWS
R2 = 0.826, F(1,331) = 1579.89, p < 0.001
R2 = 0.800, F(1,106) = 427.67, p < 0.001
Results (3)
› Is the alternate Dutch IWS an improvement?• Compute the fit of the mean ratings of the two Dutch scales to the regression line of the
English IWS.
› Z-test for comparing two correlations• Not significant for students (roriginal = 0.902, ralternate = 0.909, p = 0.609),
nor for train dispatchers (roriginal = 0.878, ralternate = 0.894, p = 0.581)
Dutch Students Train Dispatchers
Original Dutch IWS
R2 = 0.813, F(1,331) = 1442.96, p < 0.001
R2 = 0.771, F(1,106) = 359.64, p < 0.001
Alternate Dutch IWS
R2 = 0.826, F(1,331) = 1579.89, p < 0.001
R2 = 0.800, F(1,106) = 427.67, p < 0.001
Discussion
› Considerable consistency of ratings on the original English and Dutch IWS• Evidence of roughly equal spacing
› Alternative items resulted in marginal improvements• Alternative items referred to “effort”, therefore removing multidimensional aspect of the
scale.
› Train dispatchers’ and Dutch students’ results were similar• Suggest broad field of application
