Australian Management Selection Practices: Closing the Gap between Research Findings and Practice

214

Australian management selection practices: Closing the gapbetween research findings and practice

Lee Di MiliaCentral Queensland University and CRC for Integrated Engineering AssetManagement, Australia

Three arguments were made for expecting changes in selection practice over thepast decade: 1) time necessary to transfer research findings into practice, 2)evidence that HRM has become more strategic, and 3) the increasedprofessionalism of the HRM community. In contrast to earlier findings, there wasstrong support for the use of structured interviewing. Interviews were mostcommonly undertaken by HR and line staff. Typically applicants underwentmore than one interview and at each, were interviewed by two to threeinterviewers. Cognitive tests were more frequently used than personalityassessments. There was some support for personality assessments based on the FiveFactor Model but there was similar support for measures without publishedpsychometric properties. In general, the government sector made greater use ofstructured interviewing, conducted a single interview with two to threeinterviewers present and made use of better quality assessment tools compared toother sectors. Recommendations are made for future studies.

Keywords: assessment, human resource management, interviewing, selection practice

Management selection practice has the potential to make a strategic contribu-tion to organisational performance. Studies of Australian managementselection practice using data collected in the 1980s (Patrickson and Haydon1988) and early 1990s (Di Milia and Smith 1997) concluded there was a relianceon the classic trio of interviewing, reference checking and application forms.With the exception of the standard application form, the other methods havegenerally been found to be associated with poor predictive ability depending

Correspondence to: Dr Lee Di Milia, School of Management, Central Queensland University,Rockhampton, Qld, Australia 4702; fax: +617 4930 9700; e-mail: [email protected]

Asia Pacific Journal of Human Resources. Published by Sage Publications (London, Thousand Oaks, CA andNew Delhi; www.sagepublications.com) on behalf of the Australian Human Resources Institute. Copyright © 2004Australian Human Resources Institute. Volume 42(2): 214–228. [1038-4111] DOI: 10.1177/1038411104045364.

Asia Pacific Journal of Human Resources 2004 42(2)

APJHR_42_2_DiMilia.qxd 7/06/2004 9:40 PM Page 214

www.sagepublications.com

on structure (Campion, Palmer and Campion 1997). Since the Australian datawere collected, a number of significant developments in selection practice haveoccurred and these changes take time to filter into practice. During the sameperiod the business environment has also undergone significant change. Largerorganisations are now operating globally and for some in a deregulated envir-onment. The ‘new’ business environment with an increased emphasis onpeople as a strategy for gaining competitive advantage has contributed to thetransformation of human resource management (HRM) (Boxall and Purcell2003) in recent years. In addition, there are a number of studies suggesting theincreased strategic orientation (Kramar 2003) and professionalism (Dowlingand Fisher 1997) of HRM. The wider recognition that people can contributeto organisational outcomes, combined with a more professional HRM practi-tioner may be reflected in the wider use of better quality selection tools. Theaim of this paper is to review contemporary approaches to managementselection in Australia and to test the hypothesis that selection practice haschanged.

Reviews of management selection practices have not been overlysupportive of good practice. A literature review covering the years 1954–78identified the interview to be the most dominant selection tool, followed by awritten application and reference checking (Patrickson and Haydon 1988). Asurvey of 123 South Australian companies during the 1980s concluded thatlittle had changed compared to the earlier findings. There was some supportfor psychological assessment and assessment centres (ACs), but these wererestricted to executive appointments in larger companies (Patrickson andHaydon 1988). Similar findings were obtained from a study of 39 Victoriancompanies but a key finding was the strong opposition to structured interviews(Vaughan and McLean 1989).

More recently, Di Milia and Smith (1997) surveyed 254 major Australianorganisations in 1992. This survey had three strengths: a) it was a nationalsurvey, b) responses were received from across the public and private sectors,and c) it had the highest response rate to date. Nevertheless, the authorsconcluded no major changes to practice compared to earlier reviews. Acrossall organisations, the following tools were ‘always’ used: interviews (91%),reference checking (77%) and application forms (60%). These tools were over-whelmingly preferred over personality assessments (12%), cognitive tests (11%)and ACs (2%).

The Australian studies, however, had some limitations. With the exceptionof Vaughan and McLean (1989), details concerning interview structure werenot obtained and therefore, it may have been misleading to have suggestedunstructured interviews were commonplace. Nevertheless, studies during the1980s and 1990s confirmed the dominant use of unstructured interviews inAustralia, New Zealand (Taylor, Mills and O’Driscoll 1993), and the UK(Bartram, Lindley, Marshall and Foster 1995). More recent studies have alsoconfirmed a preference for unstructured interviewing in both interviewers (van

Australian management selection practices 215


der Zee, Bakker and Bakker 2002) and interviewees (Conway and Penno 1999).A number of studies have shown that unstructured interviews are associatedwith less validity than structured interviews (Huffcutt and Arthur 1994; Taylorand Small 2002). A second limitation is that while each of the Australian studiesreported the use of various psychological assessment strategies, these werelimited to global assessments. There is much variability in the reliability andvalidity of various psychological assessment tools and therefore, it is importantto identify the actual instruments in use (Di Milia and Smith 1997),

Research findings from New Zealand during the 1990s showed a remark-able similarity to Australian findings. Harris, Toulson and Livingston’s (1996)survey of selection practice among personnel consultancies reported thatmethods associated with higher validity were infrequently used. Taylor, Millsand O’Driscoll (1993) found structured interviews, ACs, and cognitive testingwere not in general use. Recent findings from New Zealand, however, haveindicated some positive change in practice. Compared to the 1993 findingsTaylor, Keelty and McDonnell (2002) reported a substantial increase in the useof valid psychological tests. The Occupational Personality Questionnaire(OPQ) was most commonly used by organisations and recruitment firms. Lessused were the California Psychological Inventory, 16 Personality Factor (16PF)and the Myers Briggs Type Indicator (MBTI). ACs were used in approxi-mately 8 percent of their total sample. Behavioural based interviews were usedin 35 percent of organisations and 8 percent used situational interviewing.Although structured interviewing questions were used, the use of a rating scaleto assess the response was not used. Overall, this study suggested that contem-porary research findings are being transferred into practice.

Selection is a key job role for HR managers (Morehead, Steele, Alexander,Stephen and Duffin 1997). When 322 HR managers were asked to identify themost important policies and programs for the previous five years, selectionprocesses were ranked third highest (Fisher and Dowling 1999). In NewZealand organisations, the greatest change over the past seven years wasreported to be the standardisation of selection processes (Taylor, Keelty andMcDonnell 2002). Of importance is some correlational evidence linkingselection practice with business outcomes. Terpstra and Rozell (1993) reportedhigh correlations (0.70-0.80) between structured interviews, mental ability tests,biodata and higher annual profits, profit growth and sales growth. Huselid,Jackson and Schuler (1997) reported small (0.10–0.16) but significant correla-tions between global indicators of HR capability (including some specificselection items) and return on assets and profitability.

There are some grounds to expect changes to current selection practice inAustralia. The last major review was conducted in the early 1990s and it wasabout that time that significant research findings were being reported. Therewas mounting evidence suggesting the superiority of structured interviewingin terms of reliability (Conway, Jako and Goodman 1995) and validity(Campion, Campion and Hudson 1994; Wiesner and Cronshaw 1988). The

216 Asia Pacific Journal of Human Resources 2004 42(2)


poor regard for personality assessments was replaced with optimism suggestingthey are valid predictors of job performance (Barrick and Mount 1991) andespecially when the assessment was based on a job analysis (Tett, Jackson andRothstein, 1991). Findings such as these take time to be communicated to prac-titioners and subsequently to filter into practice.

A second reason for expecting change is that HRM in Australia has madesome progress towards being recognised for its strategic business contribution.An international comparison of HR managers in 22 countries suggested some49 percent of Australian HR managers were members of the board orexecutive team, 54 percent were involved in strategy development and 30percent were consulted in strategy development (Kramar 2003). The strategicmanagement of people linked to meeting business goals is reflected in resourcebased views of HRM (Boxall and Purcell 2003) and developments in humancapital theory (Lepak and Snell 1999).

The third reason for predicting change is the proposition that HRM prac-titioners are becoming more professional. Dowling and Fisher (1997) surveyed2795 members of the Australian Human Resources Institute (AHRI) in 1995and compared these data to a similar survey in 1985. The 1995 data indicatedan increased use of the term ‘HR’, a similar level of undergraduate qualifica-tions (34%) and a three-fold increase in graduate qualifications (23%).Graduate study may arguably be seen as an opportunity to develop higherorder cognitive skills. Members of AHRI are able to take advantage of regularprofessional development activities. These include conferences, workshops,study programs, a number of print publications and website content. Tertiarystudy and AHRI activities can both serve as vehicles for selection practicefindings to be transferred into practice.

In summary, existing Australian studies on management selectionpractices have not been overly supportive of good practice. However, thecurrent business environment is one in which organisations are seeking tomaximise the contribution of each employee. The business environmentcoupled with developments in the research literature and in the practice ofHRM suggests a shift in selection practice to methods with greater relevanceto job success. The aim of this paper is to conduct a broad assessment ofmanagement selection practice and compare the results to earlier findings.

The Business Review Weekly (BRW, November 2002) was used to provide alisting of Australia’s top 500 companies by revenue. Approximately two-thirds of these firms were contacted to identify the best person to receive thesurvey and the remainder were addressed to the human resource manager.Participation was voluntary and confidential.

Each firm received a package containing an introductory letter outlining


Participants and survey


the purpose of the study, an explanation of key terms, the survey and a replypaid envelope. Definitions were provided for the following terms to minimiseany misunderstanding between academics and practitioners; biodata, unstruc-tured, situational and behavioural interviewing. The survey was based on thatused by Di Milia and Smith (1997) and modified to collect specific data oninterview structure and personality instruments.

Chi-square was used to compare the present results to the findings reported byDi Milia and Smith (1997). Due to the number of comparisons alpha level wasset to 0.01 to minimise Type I error. Chi-square requires a minimum expectedfrequency and comparisons were not conducted if more than 20 percent ofcells had an expected frequency of less than five (Siegal and Castellan 1988).

Differences in selection practice between four organisational sectors wereexamined as percentages since minimum expected frequencies were notobtained for chi-square comparisons. The sectors were created by combiningsome ANZSIC divisions: manufacturing (mining and manufacturing); trade(wholesale and retail); business (finance, insurance, property and businessservices); and government (administration and defence).

Response rate and industry sector

A total of 218 surveys were received and twelve were returned to sender(response rate = 45%). Responses were received from the private (n = 150),public (n = 58) and the not-for-profit (n = 8) sectors. Overall, 162 responseswere from organisations employing more than 500 employees, 39 responsesfrom organisations employing 100 to 499 employees and 17 responses fromorganisations employing less than 100 employees. Table 1 lists the major sectorsthat responded to the survey according to the ANZSIC classification. Thethree main groups were manufacturing (21%), property and business services(18%) and mining (8%). These groups were consistent with those obtained byDi Milia and Smith (1997).

Interview structure and use for all organisations

Interviews were conducted in 93 percent of organisations. Behavioural inter-views were ‘always’ used in 47 percent of cases compared to unstructured(22%) and situational (22%) interviews. It was common for more than oneinterview to be conducted and the most frequent arrangement was for HR


Data analysis procedure

Results


and line staff (42%) to conduct the interviews. Although external consultantswere used, there was a significant reduction in their involvement (χ2 (4) =22.50, p = 0.001). Significant differences were also obtained for changes to theinterview format. The use of one-to-one interviews declined (χ2 (4) = 61.96, p= 0.001) and the use of two-to-three interviewers increased (χ2 (4) = 46.05, p =0.001). Complete details expressed as percentages can be found in table 2.

Use of non-interview selection methods for all organisations

The use of application forms, curriculum vitae and resumes was widespread.Less than 1 percent of respondents indicated they were ‘never’ used. A smallbut not significant increase in the use of references was observed.

The use of recognised personality assessments was not widespread. TheOPQ was ‘always’ used in less than 7 percent of organisations followed by‘other’ tests (6%). The main tests identified under ‘other’ included the ‘FifteenFactor Questionnaire’, ‘Occupational Personality Profile’, ‘16PF’ and the‘DISC’. These data can be found in table 3.


Table 1 Response rate by Australian and New Zealand Standard Industrial Classification (n = 218)

Industry classification Number of replies

Agriculture, forestry and fishing 5

Mining 18

Manufacturing 45

Electricity, gas and water supply 9

Construction 9

Wholesale trade 11

Retail trade 10

Accommodation, cafes and restaurants 2

Transport and storage 10

Communication services 14

Finance and insurance 39

Property and business services 11

Government administration and defence 4

Education 9

Health and community services 12

Cultural and recreational services 4

Personal and other services 3

Missing 3



Table 3 Use of non-interview based selection methods (%)

Selection tool Never < half About half > half Always

Application forms,CV, résumé 0.5 2.8 0.9 9.2 86.7

References 0.9 2.8 1.8 13.3 81.2

Personality tests*NEO 81.9 11.6 1.9 3.3 1.4PCI 71.8 15.7 5.1 4.6 2.8OPQ 64.4 17.6 6.5 4.6 6.9MBTI 71.4 16.1 2.3 6.9 3.2Other 80.9 7.0 1.9 4.2 6.0

Cognitive testsLogical reasoning 44.6 24.9 8.9 9.9 11.7Numerical reasoning 49.8 22.1 7.5 8.0 12.7Other 81.6 7.1 2.4 3.8 5.2

Biodata 93.4 4.2 0.5 0.5 1.4

Assessment centre 67.8 18.2 7.0 4.7 2.3

Graphology 94.0 0.9 2.3 1.4 1.4

Astrology 100.0 0.0 0.0 0.0 0.0

* NEO – NEO Personality Inventory; PCI – Personality Characteristics Inventory; OPQ – Occupational PersonalityQuestionnaire; MBTI – Myers-Briggs Type Indicator.

Table 2 Key characteristics of interviews in organisations (%)

Never < half About half > half Always

InterviewsUnstructured 43.1 17.3 8.4 9.4 21.8Behavioural 14.6 8.9 9.4 20.2 46.9Situational 34.7 17.8 9.9 16.3 21.3

No. of interviewsOne only 48.6 23.1 10.2 13.4 4.6More than one 4.6 12.0 10.6 24.5 48.1

ParticipantsHR 46.5 11.5 9.5 18.5 14.0Line managers 36.9 11.2 10.2 15.0 26.7HR & line 13.7 8.5 8.0 28.3 41.5Ext. consultants 33.3 25.2 15.2 19.5 6.7Outsourced 76.7 16.0 3.9 2.9 0.5

TypeOne to one 54.2 19.2 6.9 9.9 9.92–3 interviewers 6.0 5.1 11.1 35.5 42.4Panel 73.3 17.8 1.5 4.0 3.5


The use of biodata was rare and significantly lower (χ2 (4) = 21.29, p =0.001) than the previous findings. The reported use of ACs was higher than theprevious findings but was significant at the 0.03 level. There was a significantincrease in the use of graphology (χ2 (4) = 15.06, p = 0.001) and consistent withearlier results, astrology was not used.

Interview structure and use between organisational sectors

A comparison of organisational sectors by the ‘always’ category suggested tradeorganisations (37%) were more likely to use unstructured interviews and thegovernment sector least likely (8%). The use of behavioural interviews seemedsimilar across the sectors but highest in the business and government sector.These two sectors also recorded greater use of situational interviewing.Conducting more than one interview was common across all sectors but leastlikely in the government sector. The use of two-three interviewers at eachsession was most common in the government sector. For all sectors, the mostcommon composition was the involvement of HR and line staff in the inter-viewing process. These data can be found in table 4.

Use of non-interview selection methods between organisationalsectors

The use of application forms, curriculum vitae and resumes was common toall sectors and highest in the government sector. References were taken up inat least 80 percent of organisations in each sector.

The OPQ seemed to be the most commonly used personality instrumentacross all sectors and the MBTI was least used in the government sector.Cognitive testing was more frequently used than personality assessments in allsectors and ACs appeared to be more regularly used by the government sector.Although it was infrequently used, graphology was identified in the manu-facturing and business sectors, whereas astrology was not practised in anysector.

A key finding was that organisations reported making use of both behaviouraland situational interview formats. While it seems behavioural interviews arebetter predictors than situational interviews (0.51 vs 0.39; Campion, Campionand Hudson 1994), both are superior to an unstructured format. More recently,Taylor and Small (2002) reported mean validity estimates of 0.63 for behav-ioural and 0.47 for situational interviews when rating scales were used to assessan applicant’s response. A limitation of the present survey is that it did notgather data on the use of a rating scale. ‘Structure’ per se, however, does not


Discussion


22

2A

sia Pacific Journal of Hum

an Resources

2004 42(2)

Table 4 Selection methods used by industry sector (%)

Manufacturing (n = 63) Trade (n = 21) Business (n = 50) Government (n = 13)

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

InterviewsUnstructured 50.0 20.0 5.0 8.3 16.7 47.4 10.5 0.0 5.3 36.8 44.4 6.7 13.3 13.3 22.2 50.0 41.7 0.0 0.0 8.3Behavioural 9.7 11.3 6.5 27.4 45.2 15.0 15.0 5.0 15.0 50.0 10.0 8.0 8.0 18.0 56.0 8.3 8.3 0.0 25.0 58.3Situational 35.5 22.6 8.1 16.1 17.7 57.9 10.5 5.3 5.3 21.1 25.0 18.2 9.1 18.2 29.5 25.0 8.3 8.3 25.0 33.3

NumberOne 53.2 27.4 12.9 4.8 1.6 66.7 19.0 4.8 4.8 4.8 64.0 16.0 6.0 10.0 4.0 23.1 38.5 0.0 38.5 0.0> than one 0.0 3.2 11.3 30.6 54.8 4.8 0.0 4.8 28.6 61.9 2.0 8.0 10.0 16.0 64.0 7.7 30.8 0.0 38.5 23.1

ParticipantsHR 45.6 12.3 8.8 19.3 14.0 52.4 9.5 0.0 28.6 9.5 43.5 8.7 15.2 13.0 19.6 69.2 7.7 7.7 7.7 7.7Line 37.3 15.3 8.5 13.6 25.4 47.6 9.5 4.8 23.8 14.3 31.3 8.3 12.5 12.5 35.4 53.8 30.8 0.0 7.7 7.7HR & line 12.9 4.8 1.6 29.0 51.6 14.3 4.8 14.3 28.6 38.1 22.9 4.2 12.5 22.9 37.5 0.0 15.4 0.0 30.8 53.8External 31.7 11.1 15.9 27.0 14.3 28.6 33.3 19.0 19.0 0.0 34.8 19.6 17.4 23.9 4.3 30.8 30.8 15.4 15.4 7.7Outsourced 75.0 18.3 3.3 1.7 1.7 65.0 20.0 10.0 5.0 0.0 78.7 14.9 0.0 6.4 0.0 76.9 15.4 7.7 0.0 0.0

TypeOne 50.0 23.3 6.7 10.0 10.0 57.1 19.0 0.0 19.0 4.8 43.8 16.7 12.5 14.6 12.5 83.3 16.7 0.0 0.0 0.02–3 4.8 0.0 11.3 43.5 40.3 4.8 9.5 4.8 33.3 47.6 6.0 6.0 12.0 34.0 42.0 0.0 0.0 0.0 30.8 69.2Panel 80.0 15.0 0.0 3.3 1.7 71.4 23.8 0.0 4.8 0.0 72.3 21.3 0.0 4.3 2.1 91.7 8.3 0.0 0.0 0.0

Application form, CV, résumé 0.0 0.0 1.6 7.9 90.5 0.0 4.8 0.0 14.3 81.0 2.0 0.0 0.0 12.0 86.0 0.0 7.7 0.0 0.0 92.3

References 0.0 3.2 1.6 6.3 88.9 0.0 0.0 0.0 19.0 81.0 4.0 2.0 2.0 12.0 80.0 0.0 7.7 0.0 7.7 84.6Personality

NEO 88.9 6.3 1.6 2.6 1.6 76.2 14.3 4.8 0.0 4.8 77.1 14.6 2.1 4.2 2.1 75.0 16.7 0.0 8.3 0.0PCI 69.8 15.9 7.9 4.8 1.6 81.0 9.5 9.5 0.0 0.0 70.8 14.6 2.1 6.3 6.3 53.8 30.8 0.0 7.7 7.7OPQ 58.7 19.0 4.8 4.8 12.7 66.7 19.0 9.5 0.0 4.8 62.5 10.4 4.2 12.5 10.4 69.2 23.1 0.0 0.0 7.7MBTI 63.5 17.5 4.8 9.5 4.8 81.0 19.0 0.0 0.0 0.0 73.5 14.3 2.0 6.1 4.1 92.3 7.7 0.0 0.0 0.0

CognitiveLogical 35.5 22.6 16.1 11.3 14.5 35.0 35.0 15.0 10.0 5.0 38.8 22.4 6.1 12.2 20.4 41.7 16.7 8.3 16.7 16.7Numerical 34.4 24.6 11.5 9.8 19.7 45.0 30.0 15.0 5.0 5.0 51.0 18.4 6.1 6.1 18.4 41.7 16.7 8.3 16.7 16.7

Biodata 86.9 8.2 0.0 1.6 3.3 90.5 9.5 0.0 0.0 0.0 95.8 0.0 2.1 0.0 2.1 91.7 8.3 0.0 0.0 0.0Assessment centre 59.7 29.0 3.2 3.2 4.8 71.4 23.8 0.0 4.8 0.0 77.6 4.1 12.2 4.1 2.0 58.3 16.7 8.3 8.3 8.3Graphology 96.8 0.0 1.6 1.6 0.0 100.0 0.0 0.0 0.0 0.0 90.0 4.0 2.0 0.0 4.0 100.0 0.0 0.0 0.0 0.0Astrology 100.0 0.0 0.0 0.0 0.0 100.0 0.0 0.0 0.0 0.0 100.0 0.0 0.0 0.0 0.0 100.0 0.0 0.0 0.0 0.0

Key: 1 = Never; 2 = Less than half; 3 = About half; 4 = More than half; 5 = Always


necessarily result in a better outcome. Di Milia and Gorodecki (1997) reportedan inter-rater reliability of 0.55 from a trained workforce using a behaviouralbased format and an anchored rating scale. Equally, it is important to highlightthat there are a number of variables that impact on the general propositionthat structured interviewing is better (Posthuma, Morgeson and Campion2002).

The continued presence of unstructured interviews is not necessarilyproblematic. Whereas earlier research was largely dismissive of unstructuredinterviews, meta-analytic findings suggest validities of 0.38 and this placesunstructured interviews on par with validity data for biodata and ACs(Schmidt and Hunter 1998). In addition, preliminary evidence on the constructvalidity of interviewing seems to suggest that less structured interviews aremore highly correlated with mental ability and the Five Factor Model (FFM)of personality (Salgado and Moscoso 2002). Structured interviews in contrast,assessed job knowledge, situational judgement and social skills. Quite apartfrom any technical argument there are a number of reasons why unstructuredinterviews will remain in use, including: a) they provide the interviewer withgreater autonomy, b) they provide advantage for interviewers seeking power,c) they may allow a better ‘fit’ between the candidate and the organisation, d)they are a means for relaying organisational culture to the candidate, and e)their lower cost (Dipboye 1997).

Consistent with the earlier findings, this study confirmed the practice ofusing more than one interview and that interviews were typically conductedby HR and line staff. There are at least two explanations for this finding: a) assupport for Ulrich’s (1998) position that HR work is not the sole responsibilityof the HR department, and b) that line staff are necessary to provide technicalinput into the interview and ownership of the decision making process. Thesepossibilities are not mutually exclusive. The involvement of HR and line staffmay explain the increased use of two-three interviewers compared to a one-to-one format.

The use of personality measures in management selection was low. Thismay not be too surprising since a recent international comparison of eighteencountries ranked Australia 14th in use of personality measures (Ryan,McFarland, Baron and Page 1999). The OPQ was most commonly reportedfollowed by a number of instruments without known published psychometricproperties (Fifteen Factor Questionnaire, Occupational Personality Profile,DISC profiling). Also of concern was the use of the MBTI in this study and inNew Zealand (Taylor, Keelty and McDonnell 2002) since it is not a selectiontool (Briggs Myers, McCaulley, Quenk and Hammer 1998). The fact that someorganisations used personality instruments without empirical support is areminder of Smith and George’s (1992) lament that some practitioners arebeing ‘deluded by pseudo science’.

An alternative explanation for the infrequent use of personality instru-ments is that practitioners may not be aware of recent developments in this



area. Reviews published since the early 1990s have consistently reportedpersonality measures as valid predictors of job performance (Barrick andMount 1991; Robertson and Kinder 1993), and more so when the predictorsare chosen on the basis of job analysis (Barrick, Mount and Judge 2001; Tett,Jackson and Rothstein 1991) rather than global personality assessments. Thesestudies used the FFM of personality; the FFM is a taxonomy for organisingscales from various inventories. Meta-analytic studies using the FFM dimen-sions have suggested ‘conscientiousness’ to be strongly associated with workperformance followed by low ‘neuroticism’ (Barrick, Mount and Judge 2001).More recently, a meta-analysis concluded that FFM based inventories for‘conscientiousness’ and ‘neuroticism’ provided greater criterion validity thannon-FFM based inventories (Salgado 2003). The OPQ and NEO (Costa 1996)are both measures of the FFM and each received some support in this study.

The FFM is not without some criticism. Hough (1998) has suggested theconstructs are too broad and that additional differentiation of the factors allowsfor better prediction. Regardless of whether the FFM or other models arebetter, developments in personality research need to be better communicatedwith practitioners since they are able at the very least to offer incrementalvalidity and therefore, boost the overall efficacy of a selection process. Forexample, Salgado (1998) demonstrated that ‘conscientiousness’ and ‘neuroti-cism’ added incremental validity to mental ability measures by 12 percent and7 percent respectively.

The use of cognitive testing was similar to earlier results and morepopular than personality assessment. Although this study did not seek thespecific measures in place, the use of cognitive testing generally may be consid-ered a positive outcome. Schmidt and Hunter’s (1998) meta-analysis suggestedcognitive tests are among the best predictors of job performance (0.51)compared to ‘conscientiousness’ (0.31). Similar estimates for mental ability testswere recently provided in British and Spanish samples (Salgado and Anderson2001). Schmidt and Hunter suggested mental ability testing to be the ‘primarypersonnel measure for hiring decisions’ (p. 266) and other measures to be usedas supplements. ‘Conscientiousness’, work sample tests and structured inter-views were reported to provide incremental validity of between 0.12 and 0.14beyond cognitive testing.

The use of ACs showed a small increase compared to earlier reviews.However, the makeup of the ACs was not explored. Evidence from NewZealand suggested more than half of the exercises were developed in-house(Taylor, Keelty and McDonnell 2002). Well-constructed ACs are consideredto have good predictive ability (0.37, Schmidt and Hunter 1998), but less thanmental ability tests or structured interviews. Given the cost of conducting ACsand that less expensive technologies are producing better validity coefficients,it may not be too surprising that ACs are not widely used. While it was encour-aging to find that no organisation used astrology as a selection method, therewas a significant (albeit small) increase in the use of graphology. Although



graphology is widely used in France, there is little empirical support for its useas a selection tool (Schmidt and Hunter 1998).

Consistent with earlier findings selection practice in the governmentsector seemed to suggest a more rigorous structure (Di Milia and Smith 1997).It recorded the highest use of application forms and tended to rely on a singlestructured interview using multiple interviewers. While the use of a singleinterview was at odds with the other sectors, this may be due to the findingthat the government sector also recorded a greater use of psychological assess-ment data. These data may be used to supplement the interview data andtherefore, negate the need for additional interviewing that may introduce addi-tional bias. The government sector made use of better quality assessment toolsand minimised use of those not recognised as selection tools (e.g., MBTI) andignored instruments without published psychometric support. In general,practice in the government sector suggested a process that records the jobinterest and subsequently employs selection methods that maximise proceduralfairness.

Reviews using the methodology employed in this study provide a broad assess-ment of contemporary management selection practice. The use of structuredinterviews, FFM based personality measures and the absence, or infrequent useof poor predictive methods such as astrology and graphology are indicators ofacceptable selection practice. Future research can benefit by collecting detailedinformation on a range of selection tools so as to allow more confident state-ments to be made. For example, while this survey identified the dominance ofstructured interviewing, it did not collect data on the following dimensions:whether generic or tailored questions were used, the ‘freedom’ to ask followup questions or whether a rating scale was used to assess responses. Thesedimensions are part of a ‘structured’ process and potentially allow for a betteroutcome (Campion, Palmer and Campion 1997). Similarly, detailed data shouldbe collected to define the type of cognitive testing being undertaken, thecontents of the assessment centre and how reference checks are conducted.Additional research is required to examine how methods are actually used(Taylor, Keelty and McDonnell 2002) and how information from a number oftools is combined or weighted in decision-making.

It is also recommended that dialogue between academics, professionalbodies and practitioners be ongoing. While there were some positive findings,it is important to highlight that a number of methods with good predictivevalidities were not widely used and that methods with unknown psychometricproperties and poor predictive validity were in use. This recommendation,however, will not assist HR employees that are not members of a professionalbody. The challenge is how to communicate good practice across the widerHRM community.


Recommendations and conclusions


In contrast to earlier reviews of management selection practice, this studyhas presented some evidence that reliable and valid selection methods are inuse in Australia’s top organisations. These findings coincide with a period inwhich human resources are considered as a means for competitive advantage.This lends some support to the possibility that a strategic HRM perspectiveand a professional HRM practitioner may explain the changes in managementselection practice.

Lee Di Milia (PhD) is a Senior Lecturer in the School of Management. His research interests include

human resource management, shiftwork and driver fatigue. He has worked with a number of coal mines

in the Bowen Basin, Central Queensland Police and Queensland Transport. Prior to joining Central

Queensland University in late 2001, he was a work psychologist in the private sector.

References

Barrick, M.R., and M.K. Mount. 1991. The big five personality dimensions and jobperformance: A meta-analysis. Personnel Psychology 44: 1–26.

Barrick, M.R., M.K. Mount and T.A. Judge. 2001. Personality and performance at thebeginning of the new millennium: What do we know and where do we go next?International Journal of Selection and Assessment 9: 9–30.

Bartram, D., P.A. Lindley, L. Marshall and J. Foster. 1995. The recruitment and selection ofyoung people by small business. Journal of Occupational and Organizational Psychology68: 339–58.

Boxall, P., and J. Purcell. 2003. Strategy and human resource management. Basingstoke:Palgrave Macmillan.

Briggs Myers, I., M. McCaulley, N. Quenk and A. Hammer. 1998. Manual: A guide to thedevelopment and use of the Myers-Briggs Type Indicator. Palo Alto, CA: ConsultingPsychologists Press.

Campion, M.A., J.E. Campion and J.P. Hudson. 1994. Structured interviewing: A note onincremental validity and alternative question types. Journal of Applied Psychology 79:998–1002.

Campion, M.A., D.K. Palmer and J.E Campion. 1997. A review of structure in the selectioninterview. Personnel Psychology 50: 655–702.

Conway, J.M., R.A. Jako and D.F. Goodman. 1995. A meta-analysis of interrater and internalconsistency reliability of selection interviews. Journal of Applied Psychology 80: 565–79.

Conway, J.M., and G.M. Penno. 1999. Comparing structured interview question types:Construct validity and applicant reactions. Journal of Business and Psychology 13(4):485–506.

Costa, P. 1996. Work and personality: Use of the NEO-PI-R in industrial/organizationalpsychology. Applied Psychology: An International Review 45: 225–41.

Di Milia, L., and M. Gorodecki. 1997. Some factors explaining the reliability of a structuredinterview system at a work site. International Journal of Selection and Assessment 5(4):193–9.

Di Milia, L., and P. Smith. 1997. Australian management selection practices: Why does theinterview remain so popular? Asia Pacific Journal of Human Resources 35(3): 90–103.

Dipboye, R.L. 1997. Structured selection interviews: Why do they work? Why are they



underutilized? In International handbook of selection and assessment, ed. N. Anderson andP. Herriot, 455–73. New York: Wiley.

Dowling, P.J., and C. Fisher. 1997. The Australian HR professional: A 1995 profile. AsiaPacific Journal of Human Resources 35(1): 1–20.

Fisher, C., and P.J. Dowling. 1999. Support for an HR approach in Australia: The perspectiveof senior HR managers. Asia Pacific Journal of Human Resources 37(1): 1–19.

Harris, N.J., P.K. Toulson and E.M. Livingston. 1996. New Zealand personnel consultantsand the selection process. Asia Pacific Journal of Human Resources 34(2): 71–87.

Hough, L.M. 1998. Personality at work: issues and evidence. In Beyond multiple choice:Evaluating alternatives to traditional testing for selection, ed. M. Hakel, 52–65. Mahwah,NJ: Erlbaum.

Huffcutt, A.I., and W. Arthur, Jr. 1994. Hunter and Hunter (1984) revisited: Interviewvalidity for entry level jobs. Journal of Applied Psychology 79: 184–90.

Huselid, M.A., S.E. Jackson and R.S. Schuler. 1997. Technical and strategic human resourcemanagement effectiveness as determinants of firm performance. Academy ofManagement Journal 40: 171–88.

Kramar, R. 2003. Changes in human resource policies and practices: Management’s dreamcome true? In Human resource management: Challenges and future directions, eds R.Wiesner and B. Millett, 9–20. Milton, Qld: Wiley.

Lepak, D., and S. Snell. 1999. The HR architecture: Toward a theory of human capitalallocation and development. Academy of Management Review 24(1): 31–48.

Morehead, A., M. Steele, M. Alexander, K. Stephen and L. Duffin. 1997. Changes at work: The1995 Australian Workplace Industrial Relations Survey. Melbourne: Longman.

Patrickson, M., and D. Haydon. 1988. Management selection practices in South Australia.Human Resource Management in Australia 26: 96–104.

Posthuma, R.A., F.P. Morgeson and M.A. Campion. 2002. Beyond employment interview: Acomprehensive narrative review of recent research and trends over time. PersonnelPsychology 55: 1–81.

Robertson, I.T., and A. Kinder. 1993. Personality and job competencies. The criterion-relatedvalidity of some personality variables. Journal of Occupational and OrganizationalPsychology 66: 225–44.

Ryan, A., L. McFarland, H. Baron and R. Page. 1999. An international look at selectionpractices: Nation and culture as explanations for variability in practice. PersonnelPsychology 52: 359–91.

Salgado, J.F. 1998. Big Five personality dimensions and job performance in army and civiloccupations: A European perspective. Human Relations 11: 271–88.

Salgado, J.F. 2003. Predicting job performance using FFM and non-FFM personalitymeasures. Journal of Occupational and Organizational Psychology 76: 323–46.

Salgado, J.F., and N. Anderson. 2001. Cognitive and GMA testing in the EuropeanCommunity: Issues and evidence. Human Performance 15: 75–96.

Salgado, J.F., and S. Moscoso. 2002. Comprehensive meta-analysis of the construct validity onthe employment interview. European Journal of Work and Organizational Psychology 11:299–324.

Schmidt, F.L., and J.E. Hunter. 1998. The validity and utility of selection methods inpersonnel psychology: practical and theoretical implications of 85 years of researchfindings. Psychological Bulletin 124: 262–74.

Siegal, S, and N.J. Castellan Jr. 1988. Nonparametric statistics for the behavioural sciences. NewYork: McGraw-Hill.

Smith, M. and D. George. 1992. Selection methods. In International review of industrial andorganizational psychology, vol. 7, eds C.L. Cooper and I.T. Robertson, 55–97. Chichester,UK: Wiley.



Taylor, P., Y. Keelty and B. McDonnell. 2002. Evolving personnel selection practices in NewZealand organizations and recruitment firms. New Zealand Journal of Psychology 31:8–18.

Taylor, P., A. Mills and M. O’Driscoll. 1993. Personnel selection methods used by NewZealand organizations and personnel consulting firms. New Zealand Journal ofPsychology 22: 19–31.

Taylor, P.J., and B. Small. 2002. Asking applicants what they would do versus what they diddo: A meta-analytic comparison of situational and past behaviour employmentinterview questions. Journal of Occupational and Organizational Psychology 75: 277–94.

Terpstra, D.E., and E.J. Rozell. 1993. The relationship of staffing practices to organizationallevel measures of performance. Personnel Psychology 46: 27–48.

Tett, R.P., D.N. Jackson and M. Rothstein. 1991. Personality measures as predictors of jobperformance: A meta-analytic review. Personnel Psychology 44: 703–42.

Ulrich, D. 1998. A new mandate for human resources. Harvard Business Review Jan.–Feb.:124–34.

van der Zee, K.I., A.B. Bakker and P. Bakker. 2002. Why are structured interviews so rarelyused in personnel selection. Journal of Applied Psychology 87: 176–84.

Vaughan, E., and J. McLean. 1989. A survey and critique of management selection practicesin Australian business firms. Asia Pacific Journal of Human Resources 27(2): 20–33.

Wiesner, W.H., and S.F. Cronshaw. 1988. A meta-analytic investigation of the impact ofinterview format and the degree of structure on the validity of the employmentinterview. Journal of Occupational Psychology 61: 275–90.



Documents

Australian Management Selection Practices: Closing the Gap between Research Findings and Practice