Embed Size (px)
Citation preview
ARTICLE IN PRESS
Applied Ergonomics 40 (2009) 230– 238
Contents lists available at ScienceDirect
Applied Ergonomics
0003-68
doi:10.1
� Corr
E-m
journal homepage: www.elsevier.com/locate/apergo
Attitudes and opinions of railway signallers and related staff, using the RailErgonomics Questionnaire (REQUEST)
Brendan Ryan a,�, John R. Wilson a,b, Sarah Sharples a, Theresa Clarke c
a Institute for Occupational Ergonomics, University of Nottingham, University Park, Nottingham NG7 2RD, UKb School of Risk and Safety Science, University of New South Wales, Australiac Ergonomics National Specialist Team, Network Rail, Melton Street, London, UK
a r t i c l e i n f o
Article history:
Received 9 July 2007
Accepted 3 April 2008
Keywords:
Rail human factors
Survey administration and analysis
Comparisons of roles
70/$ - see front matter & 2008 Elsevier Ltd. A
016/j.apergo.2008.04.010
esponding author. Tel.: +44115 9514040; fax
ail address: [email protected] (
a b s t r a c t
REQUEST, the Rail Ergonomics Questionnaire, has been designed to survey attitudes and opinions of
railway signallers and those in associated roles on a range of human factors such as job satisfaction, the
workplace, culture or stress. The development of the survey instrument has been described in Ryan
et al. [2008. Developing a rail ergonomics questionnaire (REQUEST). Appl. Ergon., doi:10.1016/
j.apergo.2008.04.006.]. The present paper presents an overview of findings from this national survey
which achieved a sample size of 3889 and a response rate of 83%. Findings are compared by different
roles (e.g. signaller, controller) on a range of main scales and responses to additional questions in the
survey. Ratings from the largest of the occupational groups, the signaller at 83.1% of the overall
population, have been compared according to different types of signalling system used, identifying
different characteristics in the use of lever, panel and VDU signalling systems. Comparisons of signallers’
ratings by geographical location have produced findings of interest at different layers of management of
the organisation, identifying locations with high or low ratings on a range of scales. The findings provide
valuable information for the client organisation, using direct input from frontline staff on a range of
human factors issues. This was a large survey, one of the largest ever civilian human factors surveys,
involving lengthy, often repetitive and frequently complex analyses. Lessons learned within the
administration of the survey, the analysis of the data and dissemination of the findings will be of
interest to researchers. An overview of the scope of analyses of the database is introduced, in addition to
proposals for further development of the survey instrument.
& 2008 Elsevier Ltd. All rights reserved.
1. Introduction
REQUEST (the Rail Ergonomics Questionnaire) has beendesigned to measure worker perceptions and opinions on a widerange of human factors scales and has been administered toworkers in a variety of roles in the railway industry in the UnitedKingdom. Earlier versions of the questionnaire have now beenused in four applications between 1998 and 2004, providingregular screening or audit data from frontline staff which can beused as an indicator of performance on a range of human factors.A shorter version of the questionnaire was developed for use inthe fifth and largest administration in a national rail survey,though this questionnaire retained wide coverage of the humanfactors scales that were within the earlier versions of the surveyinstrument. The development of REQUEST has been described inRyan et al. (2008).
ll rights reserved.
: +44115 846 6771.
B. Ryan).
This paper presents, firstly, an overview of findings from thequestionnaire. These are ‘‘top level’’ findings from the survey andgive examples of the range of analyses that are possible in relationto REQUEST. Commentary on the value of the findings to theorganisation will be of interest to researchers (e.g. ergonomists orpsychologists) or managers working within rail, and to otherswithin high-risk complex industries. This paper also discussesmajor lessons learned from administering the survey, highlightingthe practicalities in the analysis and dissemination of findingsfrom a very large and comprehensive survey of this kind.
2. Method of administration of the survey
2.1. Participants
The REQUEST questionnaire was given to all attendees at acomplete three-monthly cycle of the Network Rail company safetybriefings between September and November 2004, providing anational sample of 4686 participants. These company safety
ARTICLE IN PRESS
B. Ryan et al. / Applied Ergonomics 40 (2009) 230–238 231
briefings are carried out nationally and attended over the 3-month period by all employees in a number of roles(signaller, controller, mobile operations manager and crossingkeeper).
2.2. Procedure
The approach to administration of REQUEST was developed inan attempt to improve on response rates to the questionnaire inearlier surveys, taking advantage of the opportunity for comple-tion of the questionnaire during a 30 min time slot at the companysafety briefing programme. To maximise the benefits of supportfrom the client organisation, researchers liaised with a seniormanager with responsibility for the safety briefing programmeand developed support from the thirty or so company staff whowere to deliver the company safety briefing programme andencourage respondents to participate in the survey. Steps werealso taken to enhance the confidence of respondents in theanonymity and confidentiality procedures and the independenceof the researchers who would be carrying out the analysis, withclear statements included on the questionnaire of the process ofindependent analysis at the University. Arrangements wereagreed with the client for non-identified completion of ques-tionnaires and their return to the researchers using individuallysealed envelopes within bulk packages, coordinated by thoseresponsible for the company briefings. Practical arrangementswere made for the supply of necessary materials for the survey(questionnaires, reply envelopes, briefing notes on the survey)to all locations in the UK at which briefings would take place.The researcher attended two of the safety briefings to observethe administration of the survey and discussed the process ofadministering the survey, informally, with the staff that ledthe briefings. A brief feedback questionnaire on perceptionsof the administration of the survey was completed by 26company briefers who attended a subsequent company-trainingday.
2.3. Analysis
This was by far the largest REQUEST survey so far. Arrange-ments were made to ensure high levels of accuracy for data entry,including the design of a data entry form which was intuitive andeasy to complete, written instructions and training for data entrystaff, and rigorous checking and data back up procedures. Anindependent accuracy check on 2% of the electronic data foundvery few errors (minor errors in less than 1% of items checked),with no evidence of any systematic errors from any of the dataentry staff.
The primary method of analysis of the survey is on the basis ofa number of multiple item rating scales (job satisfaction, team-work, various job characteristics such as monitoring demandor control, workplace quality, work environment, communica-tions, usability of systems, workload, general and safety culture,working hours and fatigue, occupational stress). Mean ratingsfor scales and items in the questionnaire were calculated andused in a series of comparisons: by different roles within theindustry, different signalling types used and different geographi-cal locations. Comparisons are also made with previous REQUESTsurvey data to investigate trends in performance, and the ratingswere compared with normative data where these were available.Age-, experience- and grade-related analyses have also beendescribed. Other important information has been obtained fromthe analysis of free responses to specific questions in thequestionnaire.
3. Overview of findings from the survey
The survey achieved an overall response rate of 83% with 3889completed questionnaires and was a predominantly male sample(96% male).
3.1. Comparison by role
3.1.1. Roles compared in the survey
The roles compared within the survey were as follows:
�
Signaller (responsible for actual setting of routes and monitor-ing of individual train movements) (N ¼ 3230, 83.1% of thetotal responses). � Controller (responsible for larger geographical areas of routesand management of large-scale problems when they occur)(N ¼ 97, 2.5%).
� Mobile Operations Managers (MOMs) (a field-based role, alsoresponsible for attending incident sites) (N ¼ 264, 6.8%).
� Crossing keepers (responsible for operation of road/rail inter-face barriers) (N ¼ 148, 3.8%).
3.1.2. Findings on roles from the survey
Mean ratings and the standard error of the mean for the mainscales in the survey are shown for each of the roles in Fig. 1.Differences between mean ratings from respondents in thedifferent roles have been investigated by ANOVA and LSD testsand through inspection of differences in the figure.
Comparisons of responses from those in the different roles to aselection of additional questions in the survey are shown in Fig. 2.Overall differences in the distributions of responses between roleshave been identified by chi-square tests, with specific differencesin the responses determined through inspection of the figure.
3.1.3. Interpretation of role data
Focusing on the largest differences in ratings between theroles, crossing keepers reported ratings on a number of scales (e.g.job satisfaction, the work environment, culture, stress) andresponses to other questions (e.g. on working hours and fatigue),which are indicative of positive attitudes to the job and workconditions. The crossing keeper role is characterised by largelyindependent working, often in remote locations. Workers in thisrole are generally employed at low grades, for work which mightbe expected to be low in demands. Ratings on some of the jobcharacteristics are quite low (e.g. control, growth, task signifi-cance), but these need to be considered in conjunction with thelargely favourable ratings on the other scales. Any attempts tomake improvements on the modest ratings for these jobcharacteristics (e.g. building on opportunities for growth, improv-ing control in the role) should be made with caution, consideringthe aspirations of those in the role and the potential effects ofchanges on perceptions of other human factors which are coveredby the questionnaire.
Controllers’ responses (e.g. low opportunities for growth, highlevels of demand, inadequate breaks and problems with fatigue)highlighted concerns which need some consideration throughredesign of the job or work conditions. The role is workstationbased, involves responsibility for a large geographical area andrequires communication with those in other roles. Controllerslargely react to incidents or events, attempting to maintain andrestore normal and efficient services when incidents or problemsarise (see Farrington-Darby et al., 2006). The ratings suggest thatcontrollers perceive a greater degree of control than the signallers,
ARTICLE IN PRESS
0
1
2
3
4
5
6
7
job satisfaction(1-7)
control workplace workenvironment
(1-7)
communication workload -demand∗
culture
mean
signaller (S) controller (C) MOM (M) crossing keeper (CK)
CK>M†CK>C†††CK>S†††M>C††
C>CK†††C>M†††C>S†††S>M†††S>CK†††
M>C†††M>S†††
M>CK†††M>S††C>CK†S>CK†††
CK>M††CK>C†CK>S†††M>S†
CK>M†CK>C†††CK>S†††M>C††
CK>M†CK>C†††CK>S†††M>C††M>S†
C>CK†††C>S†††C>M†††M>CK†††S>CK†††
CK>M†††CK>C†††CK>S†††M>C††S>M††S>C†††
C>CK†††C>S†††S>CK†††M>S†††M>CK†††
† p<0.05†† p<0.01†††p<0.001
Low
High
teamwork growth tasksignificance
stress∗ (0-5)
Fig. 1. Comparisons of mean ratings from respondents in different roles on the main scales in the survey, also showing standard error of the mean and significant
differences (LSD post hoc tests following ANOVA) (1–5 point scale unless indicated). N.B. Findings are presented so that larger mean scores reflect greater/higher/ better
perceptions on the scales (e.g. better perceptions of job satisfaction, greater levels of control, higher task significance). Ratings on some of the scales were therefore reversed
during analyses in order to achieve this (see responses categories in the questionnaire). However, higher ratings on scales marked * (i.e. workload and stress) are indicative
of perceptions of problems (e.g. higher perceptions of mental workload, greater experience of stress).
0
10
20
30
40
50
60
70
unacceptableworkload
more than 48hours per week
frequently lessthan 12 hours
off
inadequatesleep
inadequatebreaks
moderate,quite a lot, agreat deal of
fatigue
physicalproblems
percentage
signaller controller MOM crossing keeper
Fig. 2. Comparisons of percentage of respondents in different roles providing responses to additional questions in the survey.
B. Ryan et al. / Applied Ergonomics 40 (2009) 230–238232
but this could be compromised by the additional demands of therole and more negative perceptions of the respondents on anumber of the main scales.
A high proportion of the MOMs reported working more than48 h each week, but very few from this group reportedunacceptable levels of workload or had problems taking adequatebreaks. MOMs operate in field-based roles, travelling to differentlocations to react to events, with the potential to work for longperiods when incidents arise. However, it would appear that theMOMs have some freedom in managing their time effectively.
Signallers provided many similar ratings to the controllers,though they seem to have fewer concerns with workload(demand), stress, breaks, fatigue and physical problems but moreproblems with control. Generally, the signallers are performingadequately on a number of the human factors scales and a steadyimprovement in ratings over the years of the surveys is
demonstrated below. A number of company initiatives may havecontributed to this, such as the introduction of the 13-week signallerbriefing programme, opportunities for coaching and mentoring ofstaff, involvement of staff in special projects and the introduction ofnew duties within the role. However, as with the controllers, thesignallers have poorer ratings than the crossing keepers and MOMson a number of the scales and there is still scope for furtherimprovement in relation to many of the issues (e.g. growth,workplace quality, communications, workload and breaks).
3.2. Comparison of signaller ratings by system type
3.2.1. Signalling system types compared
Signallers were by far the largest occupational group withinthis survey (83% of the survey population) and the comparisons
ARTICLE IN PRESS
B. Ryan et al. / Applied Ergonomics 40 (2009) 230–238 233
carried out between signalling system types or technologies inthis survey were as follows:
�
† p†† †††
Hi
Lo
Figsign
ind
Lever frame systems (N ¼ 1116 in the survey)—frequently inremote areas, with signallers working independently andhaving responsibility for a small geographical area of the railnetwork. Signallers often need high physical effort to pulllevers on the system which is the oldest type of signallingtechnology in the industry.
� Panel signalling systems (N ¼ 910)—also a quite dated tech-nology in the industry, controlling signals by pushing/pullingbuttons on a panel representation of the track layout. Work iscarried out in small teams in signal boxes of varying sizes withresponsibilities for variable densities of traffic.
� VDU signalling systems (N ¼ 252)—allow seated work withmore recent technology and can offer some assistance withautomated route setting. Usually involves responsibility for thecontrol of larger geographical areas.
0
1
2
3
4
5
6
7
job satisfaction(1-7)
teamwork tasksignificance
w
mean
lever (L) panel (P) VDU (V)
L>P†††L>V†††P>V†††
P>L†P>V††
L>V†††L>V†††
<0.05p<0.01p<0.001
gh
wcontrol growth
. 3. Comparisons of mean ratings from respondents operating different system type
ificant differences (LSD post hoc tests following ANOVA) (1–5 point scale unless
icative of perceptions of problems (e.g. higher perceptions of mental workload, grea
0
10
20
30
40
50
60
70
unacceptableworkload
more than 48hours per week
frequently lessthan 12 hours off
ina
percentage
lever panel VDU
χ2 = 40.4df = 2
p < 0.001
χ2 = 17.5df = 2
p < 0.001
χ2 = 8.7df = 2
p < 0.05
χ2
p <
Fig. 4. Comparisons of percentage of respondents operating different sys
Mean ratings and the standard error of the mean for the main
3.2.2. Findings on signalling system types from the surveyscales in the survey are shown for each of the signalling systemtypes in Fig. 3. Differences between mean ratings from respondentsoperating different system types have been investigated by ANOVAand LSD tests and through inspection of differences in the figure.
Comparisons of responses from operating the different systemtypes to a selection of additional questions in the survey areshown in Fig. 4. Overall differences in the distributions ofresponses between roles have been identified by chi-square tests,with specific differences in the responses determined throughinspection of the figure.
3.2.3. Interpretation of system type data
The signalling role consists primarily of route setting andmonitoring functions, but also responsibility for additionalfunctions, such as the coordination of access to the track formaintenance/engineering and reactive elements when incidentsoccur (Pickup, 2006). The role is likely to change when different
orkplace workenvironment
(1-7)
communication workload -demand∗
culture
L>P†††L>V†††P>V†††
L>P† L>P†††L>V††
P>L†††V>L†††V>L††
L>P†††L>V†††
P>L†††V>L†††
stress∗ (0-5)
s on the main scales in the survey, also showing standard error of the mean and
indicated). N.B. Higher ratings on scales marked * (i.e. workload and stress) are
ter experience of stress).
dequatesleep
inadequatebreaks
moderate, quite alot, a great deal
of fatigue
physicalproblems
= 33.1df = 2 0.001
χ2 = 21.7df = 2
p < 0.001
χ2 = 15.5df = 2
p < 0.001
χ2 = 1.1df = 2NS
tem types providing responses to additional questions in the survey.
ARTICLE IN PRESS
B. Ryan et al. / Applied Ergonomics 40 (2009) 230–238234
system types are used (e.g. lever, panel, VDU systems) due todifferences in the technology or differences in the geographicalarea of responsibility. In the current survey differences inrespondents’ ratings and perceptions by system type have beendemonstrated. For example, signallers operating VDU systemsreported lower ratings for growth than those involved in theoperation of panel and lever systems. Those operating VDU-basedsystems are often employed at high grades, working with complexand high traffic demands, and there may be perceptions of pooreropportunities for promotion or expansion in the role. Specificcharacteristics of the VDU system, such as the availability ofautomated route setting systems in some locations, may alter thesignalling role from one of regulating to one of monitoring traffic.
Signallers operating VDU systems also reported low ratings for thework environment (particularly for temperature and draughts), whichis interesting given that these work locations are likely to be the mostmodern signalling environments, where it might be expected thatenvironmental concerns would have been minimised during designand development. Many of the modern signalling locations have beenfound to suffer from environmental complaints within earlierREQUEST surveys, including problems with the effectiveness ofheating, ventilation and air conditioning systems and problems withlighting and glare in the use of VDU technology. However, there isevidence to show that dissatisfaction with ‘‘less tangible’’ aspects ofwork systems design (job characteristics, for instance) may findexpression in complaints about more tangible aspects in the physicalworkplace and work environment (Wilson, 1987; Wilson and Grey,1986).
Lever operators operating the oldest technology provided themost positive ratings on a number of scales (e.g. job satisfaction,culture, the work environment). This group of respondents oftenwork independently and in remote locations and usually takeresponsibility for signalling less frequent services over relativelysmall geographical areas. Consequently, they reported lowerlevels of workload (demand) and stress than those signallersoperating other systems. There were no differences in thenumbers of personal physical problems reported by leveroperators and those operating the alternative systems, in spiteof the heavy physical forces which are frequently required in theoperation of the levers (Muffett, 2007).
Working hours and fatigue were important areas of concern formany signallers responding to the survey and there were againsome important differences between the operators of differentsystem types. Quite high proportions of the panel operatorsreported that they could not get adequate sleep between shifts,were unable to take adequate breaks during their work, andreported problems with fatigue. VDU operators were the mostlikely group to report working 48 or more hours each week and,like panel operators, reported problems with fatigue. Paneloperators and VDU operators generally work in teams, often incentralised, metropolitan locations (which may require greatertravel time to work), and they control the movements of trainswhich can vary quite appreciably in terms of traffic frequency. Inresponse to open questions in the questionnaire, some respon-dents have raised concerns about availability of staff cover insome locations, increasing the need for extra shifts or additionalduties during nights and weekend work.
3.3. Comparison of signaller ratings by location
3.3.1. Locations compared in the survey
Ratings and opinions have been collected from signallers at 740different signalling work locations. In-depth analyses have beenconducted for this group, comparing ratings across different broadlocations of the organisation (subject to confidentiality issues at
locations where there are few operators); these are the eight mainroutes, 18 general manager areas and 159 signaller manager areas.
3.3.2. Findings on locations from the survey
An example of the type of comparison which has been carriedout by location grouping is given in Fig. 5. The figure shows theratings on four of the main scales for the 48 geographical areas(i.e. signaller manager areas) in one of the eight routes of theorganisation. Respondents in Area A reported high ratings for jobsatisfaction, teamwork and fatigue, and low ratings for stress.Respondents in Area B reported lower ratings for job satisfaction andstress, moderate ratings for teamwork, and high ratings for stress.The ratings from respondents can also be compared with percentilescores for the whole signalling population, as illustrated for one ofthe eight routes in Fig. 5, showing comparison with 25th (lower line)and 75th (upper line) percentile scores for the whole surveypopulation. Consequently, the highest and lowest performing areashave been identified across the range of main scales, providing auseful means of identifying priorities for more specific analyses.
3.3.3. Interpretation of findings on locations from the survey
Differences in responsibilities and demands in the signallingrole might be expected because of regional differences in a railnetwork which is distributed throughout the United Kingdom, withcontrasting management approaches, variable traffic densities oreven differences arising from staff shortages in some areas, whichcan potentially affect signallers’ hours of work. Signal box locationcan therefore influence a range of task, workplace or social factors.For example, track complexity and the timetable (including optionsfor diverting traffic during the control of incidents, the frequencyand predictability of train services, and the size of the geographicalarea of responsibility) might be expected to influence ratings ofsatisfaction, workload or stress. Similarly, the attitudes andapproaches of local management and the interactions amongteams could affect the respondents’ ratings on scales such asteamwork, culture and stress. Analyses of the respondents’ ratingson the main scales have provided descriptive statistics on a widerange of human factors, which will be useful to managers atdifferent parts and layers of the organisation. It is clear from theseanalyses that there are geographical areas (such as indicated inFig. 5) where respondents’ perceptions or ratings on a number ofthe main scales or question areas are consistently good or poorwhen compared with the responses from the national populationof signallers. It would be possible to focus the comparative analysesby location at a more precise level (i.e. at the level of the 740 signalbox locations), though this has not been carried out in this analysisin order to eliminate problems with anonymity of respondents insome of the smaller signal box locations (where only one or tworespondents may have participated in the survey).
3.4. Tests of association between personal factors (age, grade and
experience) and signallers’ ratings
3.4.1. Details of analyses for personal factors
Survey data from the largest group of respondents (signallers)have been examined, testing the association between thesepersonal factors and ratings on the main scales in the surveythrough calculation of Spearman rank correlation coefficients.
3.4.2. Findings on tests of association between personal factors and
signallers’ ratings from the survey
The Spearman rank correlation coefficients (for the associationbetween the personal variables (age, experience and grade) andthe ratings) in each of the main scales in the survey are given inTable 1.
ARTICLE IN PRESS
Table 1Relationships between personal characteristics and ratings on main scales within
the questionnaire for signallers
Correlation (Spearman’s rho) with personal variables
Scale Grade Age Experience
Job satisfaction �0.26��� �0.04� �0.27���
Teamwork �0.14��� �0.04� �0.17���
Control 0.06�� �0.06��� �0.05�
Workload-demand 0.50��� �0.01 0.19���
Growth �0.15��� �0.03 �0.14���
Task significance 0.07��� 0.04 �0.05�
Workplace quality �0.17��� �0.01 �0.13���
Work environment �0.17��� 0.02 �0.14���
Communication �0.18��� �0.01 �0.10���
Culture �0.36��� 0.02 �0.24���
Stress 0.30��� �0.06��� 0.16���
Grade �0.05�� 0.35���
Age �0.05�� 0.38���
Experience 0.35��� 0.38���
��� po0.001.�� po0.01.� po0.05.
Fig. 5. Ratings on four scales by signaller manager area in one of the eight main routes, also showing percentile scores for the whole survey population.
B. Ryan et al. / Applied Ergonomics 40 (2009) 230–238 235
There were many significant correlations between the personalvariables and the ratings in the main scales, in particular for thepersonal variables for grade and experience. The strongestrelationships between the variables are highlighted in the tableand these were found most commonly with the grade of therespondent (e.g. moderate to quite strong relationships betweenthe grade of respondent and both workload and stress; moderate,
negative relationships for ratings of job satisfaction and culturewith grade and experience of the respondents).
3.4.3. Interpretation of findings on tests of association between
personal factors and signallers’ ratings from the survey
Analysis of the effects of personal factors revealed morenegative ratings from those at higher grades on some of the keyissues in the questionnaire (e.g. lower ratings of general and safetyculture and job satisfaction, and higher ratings of stress and workdemands). These were findings which were not expected prior tothis survey. Carlopio and Gardner (1995) found respondent gradeto be positively associated with ratings of job satisfaction, andthey reported that this finding was consistent with a number ofstudies in the literature (citing Adams et al., 1977; Lindstrom,1991; Sawyer, 1988; Stellman et al., 1987). Carlopio and Gardneroffered a number of explanations for this type of positiveassociation between grade and ratings for job satisfaction,suggesting that those at higher grades, and particularly thoseoperating in a supervisory capacity, have the benefits of morefavourable workloads, more decision latitude in their roles, morevaried and interesting work, and more autonomy. However, in thecurrent survey, as indicated in Table 1, there was quite a strong,positive relationship between grade and workload, no relationshipbetween grade and ratings of control, and only a weak relation-ship between grade and ratings for variety in the role (not shownin Table 1), suggesting that those in the higher grade roles werenot experiencing the potentially more positive features of thesupervisory related roles. It is possible that the present survey hasnot collected responses from enough respondents at supervisory
ARTICLE IN PRESS
Table 2Comparison of ratings for job satisfaction from the current survey with ratings
from normative data from Mullarkey et al. (1999)
Type of rating Mean rating for
signallers (sd)
Mean rating for
controllers (sd)
Mean ratings in
normative data
(sd)
Job satisfaction 4.31 4.17 4.43–4.67
(0.97) (0.87) (0.8–0.85)a
4.33 (0.90)b
a Associate professional and technical—for male and female population, with
range for age groups from 16-29 to 50-65.b For male respondents from associate professional and technical.
B. Ryan et al. / Applied Ergonomics 40 (2009) 230–238236
or managerial grades, and there is some evidence from feedbackfrom those carrying out the company briefing programme thatsignaller managers did not attend safety briefings or were notgiven questionnaires to complete (even where they did attend thebriefings).
There were modest, negative relationships between thenumber of years of experience in the industry and ratings forjob satisfaction and ratings for culture. Grade and experience werecorrelated in the analysis (Table 1) and the more negativeperceptions of job satisfaction and culture at higher grades mightbe explained in part by higher expectations, perhaps coupled withincreased uncertainty in terms of growth in the role. This could beindicative of reaching a ceiling in career progression after anumber of years of experience in the industry.
Age was not strongly associated with any of the variables.Mathieu et al. (1993) previously found that there was norelationship between age and job satisfaction, but had originallyhypothesised that there would be a positive relationship (basedon Rambo (1982), James and Jones (1980) and Oldham andHackman (1981)).
3.5. Comparison of ratings with other data
3.5.1. Trends in ratings over years of the survey
Data from previous REQUEST surveys were used to identifytrends over the years of the surveys. Trends in ratings on four ofthe scales which have been used in each of the surveys since 1998,are shown in Fig. 6 (the scale for culture was introduced in 2001).The figure shows general improvements in ratings for jobsatisfaction, teamwork and culture. A trend of increased ratingsof stress between 1998 and 2004 has not been maintained in themost recent survey, which may be due to improvements in jobdesign or organisational culture, or may even be an artefact of thelarger survey size. For future administrations of REQUEST, we nowhave a comprehensive database which can be used in compar-isons and analysis of trends.
20052004200119991998year
5
4
3
2
1
Mean
stress∗cultureteamworkjob satisfaction
Fig. 6. Trends in ratings on four scales over years of the survey. N.B. Higher ratings
on the scale marked * (i.e. stress) are indicative of perceptions of problems (e.g.
greater experience of stress).
3.5.2. Comparisons with normative data
Normative data for different occupational groupings areavailable for one of the scales (i.e. job satisfaction ratings) inMullarkey et al. (1999) and these data were used to benchmarkratings for the survey population against other populations/industry groups. Comparisons of ratings from the signallers andthe controllers in the current survey with ratings for associateprofessional and technical staff in the normative data are given inTable 2.
The mean ratings from signallers, a predominantly male group,were fractionally below, and for controllers well below, the meanvalue in the normative data, suggesting that there is some scopefor improvements in ratings on this scale from parts of the surveypopulation. Whilst the normative data show a positive relation-ship between ratings of job satisfaction and age, there was noclear relationship between ratings of job satisfaction and age inthe current survey (Section 3.4.3).
4. General discussion
The REQUEST survey is a useful monitoring tool and the surveydata are a valuable resource for the organisation, using directinput from a highly representative sample of frontline staff thathas readily participated in the survey. A successful means ofadministering the survey nationally has been demonstrated.
4.1. Key findings from the survey
The survey has been effective in discriminating differences inratings on a number of important human factors from respon-dents in different roles, operating different signalling systems, indifferent locations and with different levels of seniority. Some ofthese findings have been somewhat counter-intuitive; for exam-ple, more positive perceptions have been obtained from those inthe least complex and lower grade roles or positions, and ‘‘newer’’is not always better in terms of workplace and environmentalconditions. More specifically:
�
By role, crossing keepers seemed to be happiest (reportinggreater satisfaction on a number of scales within the survey,with few respondents experiencing problems with fatigue orworking hours). � When comparing signalling systems, those operating leversystems (the oldest technology in the industry) were generallymore satisfied on many of the scales than their colleaguesoperating other system types.
� Locations have been identified where signallers have providedhigh and low ratings for a range of human factors.
� More negative ratings from those at higher grades on some ofthe key issues in the questionnaire (e.g. lower ratings of
ARTICLE IN PRESS
B. Ryan et al. / Applied Ergonomics 40 (2009) 230–238 237
general and safety culture and job satisfaction, and higherratings of stress and work demands) had not been expectedprior to the survey.
4.2. Key methodological findings
4.2.1. Effect of large sample size on analysis and dissemination of
findings
This was clearly a large survey. Considering its UK nationalcoverage (of the signallers and those in associated roles), theoverall numbers of participants and the range of human factorsconcepts and scales within the survey instrument, it is one of thelargest civilian human factors surveys known to the authors. Anumber of challenges have been overcome and lessons learned inthe administration of the survey, using the support and participa-tion of the client organisation. This support and participation hasbeen clear at three levels: senior management support has raisedthe profile of the survey and given unprecedented national accessto respondents during work time at the safety briefing process;day to day support has come from those delivering the briefings,handing out and collecting questionnaires, and encouragingparticipation; and the respondents themselves have empathisedwith the ethos and aims of the survey and demonstrated theirgeneral support through their participation. This support fromwithin the organisation, in conjunction with the improvements inthe design and layout of the questionnaire, will have contributedsignificantly to the high response rate.
The scale of the survey has also presented a number ofchallenges for the analysis of the responses. The findings from thesurvey have required somewhat lengthy and necessarily repetitiveanalyses, using outputs from multiple statistical procedures andinspection of numerous figures, culminating in a comprehensiverange of comparisons across multiple layers of the organisation.Furthermore, the analysis of qualitative responses (the needfor which had been restricted in the development of the survey,but which was still necessary for a number of questions withseveral thousand responses) was time consuming, but providedimportant information on a number of topics (such as factorsaffecting unacceptable workload, the main causes of fatigue andthe types of personal physical problems experienced by therespondents).
The timely analysis and reporting of findings was an importantconsideration during this survey, to inform management of resultsof analyses and to give feedback to the survey respondents,maintaining user confidence in the survey. Regular review meet-ings were held with key personnel at the client organisation todiscuss survey results and priorities for further analyses. Feedbackof preliminary findings was delivered to the survey respondents atthe company safety-briefing programme (i.e. using the samemechanism as was used for the survey). A further programme ofbriefings will be necessary to feedback the findings from morerecent analyses, and details of the organisational response to thefindings.
4.2.2. Effect of large sample size on statistical significance and
interpretation of findings
The large sample size obtained from this survey presented anumber of challenges when interpreting statistical results. As thedegrees of freedom were so high, very small differences orassociations were found to be statistically significant. Forexample, correlations of 0.08 were significant at the po0.01 level(in other words, would only have occurred by chance 1% of thetime). Whilst this indicates to some extent the power of the dataobtained, it is not useful when attempting to prioritise research
findings or identify particularly strong relationships betweenvariables. A correlation coefficient of 0.08 indicates that lessthan 1% of the variance in variable A can be accounted for byknowing the value of variable B. Therefore, for the clientorganisation, and for making general human factors conclusions,this is not a useful finding. Similarly, when interpreting thedifferences in ratings from those in different roles (Fig. 1), greatestattention was paid to those differences which were significant atthe level of po0.001. It was vital that a pragmatic and commonsense approach to the interpretation of the statistical data wastaken, alongside the use of statistics to identify data trends andstrong overall findings.
4.3. Further analysis and development of the survey instrument
4.3.1. Opportunities for further in-depth analyses
The comprehensive survey has provided opportunities toinvestigate a range of theoretically driven research questionsusing a variety of statistical procedures and tests. The analysesoutlined in this paper have indicated that respondents’ percep-tions on the various scales are likely to be influenced by a range ofphysical, psychosocial and personal factors.
Further analysis of the data is in progress, using multipleregression and structural equation modelling to investigatethe strengths of the relationships between the human factorsconstructs (such as job satisfaction and organisational culture),to help in understanding how the different constructs mayinfluence one another within the railway context. Relation-ships between ratings on the human factors scales and otherorganisational performance data and safety data are alsobeing investigated (c.f. Itoh et al. (2004) and Lee (1998)). Thiswill help to develop a better understanding of how thehuman factors constructs which are covered by scales withinthe survey might influence performance or safety within theorganisation.
4.3.2. Re-administration of the questionnaire and scope for use with
other occupational groups
Further consideration will also be given to the approach foradministration of the survey. The most recent survey has beenhighly successful in terms of the widescale participation ofstaff and the breadth of information collected using the surveyinstrument, though the time and resource implications foradministering and analysing such a survey are clear. It is likelythat the survey will be repeated at some time in the future(not at yearly intervals, but perhaps at 3-yearly intervals), butthere is also some scope for administering elements of the surveyat shorter intervals with specific groups or staff or in particularareas of the organisation. REQUEST scales or questions are nowavailable for a range of human factors concepts. Some ofthese (e.g. body part discomfort and longer question sets forcommunications and workload) were not included for practicalreasons within the version of the questionnaire which was usedmost recently, but these could be appropriate for use in othersituations.
REQUEST has been used to date with staff in railway signallingand associated roles. A version of questionnaire is currently beingdeveloped for use with railway engineering and maintenancestaff, covering a wide range of roles (such as those involved intrack labour, machine operation or roles associated with protect-ing workers from traffic). This version will include additionalquestions on issues such as safety, adequacy of equipment andtools, and vision and comfort in vehicles.
ARTICLE IN PRESS
B. Ryan et al. / Applied Ergonomics 40 (2009) 230–238238
5. Summary and conclusions
REQUEST has been administered successfully within theorganisation, collecting opinions and ratings from a large numberof frontline staff on a broad range of human factors. The findingscan complement those from other audit tools (e.g. those based onworkplace inspection) and assist with the targeting of resourcesor the effective planning of more in-depth field evaluations todevelop a better understanding of key areas of concern. Inter-pretations of the ratings on scales have been achieved primarilythrough a series of in-depth comparisons of the ratings (betweenroles, users of different system types, and different locations) andthrough comparisons with percentile scores for the surveypopulation. A generally positive trend in the ratings on scaleshas been observed over the years of the surveys.
There were many challenges in the administration of thisnational survey, and a pragmatic approach has been used in theanalysis and interpretation of the findings. Further analyses arebeing carried out to investigate the relationships between ratingson the different human factors constructs and relationships withother safety and performance data which are held by theorganisation. The survey is being developed for use with otheroccupational groups within the industry.
Acknowledgements
The authors would like to thank the many participants atNetwork Rail who readily participated in the survey, and alsothank Peter Nock and his colleagues, who were involved in thesmooth running of the administration of the questionnaire at thesafety briefing process. The research was funded by Network Rail.
References
Adams, E.F., Laker, D.R., Hulm, C.L., 1977. An investigation of the influence of joblevel and functional speciality on job attitudes and perceptions. J. Appl.Psychol. 62, 335–343.
Carlopio, J., Gardner, D., 1995. Perceptions of work and workplace: mediators of therelationship between job level and employee reactions. J. Occup. Organ.Psychol. 68, 321–326.
Farrington-Darby, T., Wilson, J.R., Norris, B.J., Clarke, T., 2006. A naturalistic study ofrailway controllers. Ergonomics 49, 1370–1394.
Itoh, K., Andersen, H.B., Seki, M., 2004. Track maintenance operators’ attitudes tojob, organisation and management, and their correlation with accident/incident rate. Cognit. Technol. Work 6, 63–78.
James, L.R., Jones, A.P., 1980. Perceived job characteristics and job satisfaction: anexamination of reciprocal causation. Personnel Psychol. 33, 97–135.
Lee, T., 1998. Assessment of safety culture at a nuclear reprocessing plant. WorkStress 12, 217–237.
Lindstrom, K., 1991. Well-being and computer mediated work of variousoccupational groups in banking and insurance. Int. J. Human–Comput. Interact.3, 339–361.
Mathieu, J.E., Hofmann, D.A., Farr, J.L., 1993. Job perception–job satisfactionrelations: an empirical comparison of three competing theories. Org. Behav.Hum. Decis. Processes 56, 370–387.
Muffett, R.J., 2007. Ergonomics assessment of lever operation in mechanicalsignaling. In: Wilson, J.R., Norris, B.J., Clarke, T., Mills, A. (Eds.), People and RailSystems: Human Factors at the Heart of the Railway. Ashgate Publishing,Abingdon, UK.
Mullarkey, S., Wall, T.D., Warr, P.B., Clegg, C.W., Stride, C.B. (Eds.), 1999. Measures ofJob Satisfaction, Mental Health and Job-Related Well-being. A Bench MarkingManual. Institute of Work Psychology, University of Sheffield, UK.
Oldham, G.R., Hackman, J.R., 1981. Relationships between organizational structureand employee reactions: comparing alternative frameworks. Admini. Sci. Q. 26,66–83.
Pickup, L., 2006. Understanding and assessment of mental workload in railwaysignalling. Ph.D. Thesis. University of Nottingham, Nottingham, UK.
Rambo, W.W., 1982. Work and Organisational Behaviour. Holt, Rinehart andWinston, New York.
Ryan, B., Wilson, J.R., Sharples, S., Morrisroe, G., Clarke, T., 2008. Developing a railergonomics questionnaire (REQUEST). Appl. Ergon., doi:10.1016/j.apergo.2008.04.006.
Sawyer, K.E., 1988. Measuring attitudes across job levels. When are scale scorestruly comparable? Org. Behav. Hum. Decis. Processes 42, 324–342.
Stellman, J.M., Klitzman, S., Gordon, G.C., Snow, B.R., 1987. Work environment andwell being of clerical and VDT workers. J. Occup. Behav. 8, 95–114.
Wilson, J.R., 1987. Whose attitudes to which aspects of work? Cool media for highparticipation. Work Stress 1, 385–395.
Wilson, J.R., Grey, S.M., 1986. Perceived characteristics of the work environment.In: Brown, Jr., O., Hendrick, H.W. (Eds.), Human Factors in OrganisationalDesign and Management. Amsterdam, North Holland.
