PSYCHOLOGICAL ASSESSMENT SEMINAR - libvolume7.xyz · PSYCHOLOGICAL ASSESSMENT SEMINAR Rubik’s cubes of colour, shape and size: Ethical psychological testing in a multicultural work

PSYCHOLOGICAL ASSESSMENT SEMINAR

Rubik’s cubes of colour,

shape and size:

Ethical psychological testing

in a multicultural work

environment

Department of Industrial & Organisational

Psychology

18 August 2010

AGENDAMorning session: 08h30 - 13h00

• Welcome by Prof Dirk Geldenhuys

• Introduction: Welcome to the Rubik’s Cube

Adventure

• A short history of psychological assessment

• The areas of psychological assessment

• Psychological assessment myth busters

Tea break: 11h00

• Legislation and ethics

• Ethical freeze role play

Lunch: 13h00 - 13h45

• Current topics in psychological assessment

• Psychological assessment Crosswords

• The future of psychological assessment

Conclusion: 15h00

A short history of Psychological Assessment

� History influences current practice

� Relevance

� Evolved and progressed

� History is important

� Helps explain current practice

� Strengths and weaknesses

� Prevent repetitions of the ‘wrongs of the past’

Why look at the past?

History of testing: An International Perspective

� Over the years many authors, philosophers and scientist have explored

various avenues in an attempt to assess human attributes. To mention a

few;

� Astrology (positions of planets to describe the personality of individuals

and predict what may happen in their lives)

� Physiognomy (judging the character of a person from external features of

the body, especially the face.)

� Graphology (the systematic study of handwriting and the notion that it

can be used as an expression of personality characteristics)

� Humorology

� Phrenology

(Foxcroft, 2009)

Testing in South Africa: The Early Years

• Psychology emerged as an academic discipline in South Africa after World War 1.

• Many of the influential figures in the development of intellectual tests in South Africa where trained at American institutes.

• Therefore tests which where typically used in the early phases where also American such as; Goddards 1911 revision of the

Binet-Simons test, Termans 1916 Stanford revision, the Army Beta, the Porteus Maze Test etc.

• In the period between the two world wars, social and human sciences in South Africa became important contributors to debates on crucial social issues including ‘the Native question”

(Louw, 1997; Foxcroft, 2009; Classen, 1997)

Testing in South Africa: The Early Years

� Psychological knowledge was used as a tool legitimise a social order based on race as the perception was that psychological testing produced empirical data which supported certain explanations of this order.

� There was an increase in the use of science as a factor in regulating aspects of social economic life.

� Differential performance of black and white reinforced the idea of hierarchy of human societies and consequently differential treatment in terms of education, employment etc.

(Louw, 1997)

Key notes

Psychometric testing is applied worldwide and used in various

industries for recruitment, selection and counselling purposes

(Gregory, 2000).

For the purpose of this seminar four important areas for

assessment have been identified.

� Cognitive assessment

� Personality assessment

� Behavioral assessment

� Interest

Psychometric Testing

Range of tests and distributors available - Cognitive

MEASUREMENT

CONSTRUCT

CONCEPT TEST DISTRIBUTORS APPLICATION IN

INDUSTRIAL/ PSYCHOLOGICAL

FIELDS

Cognitive IQ,General

cognitive,Multiple aptitude,

Learning potential

WAIS, SSAIS, ASAT, RPM,

DAT Forms R, S, K & L

Mindmuzik Media Personnel screening and selection,

Career guidance and counseling,

Personnel developmentCPP, CTA Magellan Consulting

APIL-B, TRAM 1 & 2,

AproLabM&M Initiatives

LPCAT M&M Initiatives

TST Thomas International

COPAS Integrity International

Range of tests and distributors available -

Cognitive

Range of tests and distributors available - Cognitive

MEASUREMENT

CONSTRUCT



FIELDS

Cognitive IQ,General

cognitive,Multiple aptitude,

Learning potential

WAIS, SSAIS, ASAT, RPM,

DAT Forms R, S, K & L

Mindmuzik Media Personnel screening and selection,

Career guidance and counseling,

Personnel developmentCPP, CTA Magellan Consulting

APIL-B, TRAM 1 & 2,

AproLabM&M Initiatives

LPCAT M&M Initiatives

TST Thomas International

COPAS Integrity International

Range of tests and distributors available -

Cognitive

• Some consensus

– IOPs agree that cognitive ability test are valid and fair

– Cognitive ability tests provide good but inadequate

measures of intelligence

– Other attributes are necessary to account for multi

dimensional nature of performance (90% of respondents)

Cognitive Assessment

(Murphy, Cronin & Tam)


• Some controversy

- Societal concerns

- Cultural bias

- Education

- Socio-economic status

- Education levels of parents

- Effects of negative stereotypes

- The g-Ocentric vs MI

(McKay, Doverspike, Bowen-Hilton & McKay)

(Murphy, Cronin & Tam)


• Unifying theoretical approach

– 3 approaches

• Structural approach (factor analytic)

• Information-processing

• Dynamic (based on learning theory)

(Taylor)

“Learning as the critical factor underlying cognitive competenceand the mastery of problems and challenges”

Structural - Spearman (g – one general factor) and Thurstone 7&9

Primary mental abilities

•static

- fluid & crystallised (Cattell)

- mostly power tests

Info processing – e.g. Ravens progressive matrices

receiving, processing & retrieval speeds

computer admin essential

Learning / dynamic approach

• Vygotsky – Zone of proximal development (ZPD)

•less susceptible to cultural bias

•3 phases of learning – conceptual understanding, compilation of

execution procedures & automatization of processing.

•Test – teach – test or pre-test – mediation – post-test

Emphasis on potential and not achievement

MEASUREMENT

CONSTRUCT



FIELDS

Personality Personality preferences,

Management styles,

Communication styles,Interpersonal

styles,Coping styles

MMPI, TAT, SORT, VS , WLQ

Mindmuzik Media Personnel selection,

Career counselling,

Academic counselling,

Personal development,

Psychological growth,

Candidate - job match,

Management

development,

Stress management

OPQ 32 SHL

15 FQ, OPP, Jung Psytech

MBTI, Big Five, WPI, LSPMSP, CPI

Jopie van Rooyen

PAW Integrity International

Range of tests available: Personality

MEASUREMENT

CONSTRUCT



FIELDS

Personality Locus of control,

motivation and autonomy

LCIAMP

Jopie van Rooyen Personnel selection

Career counselling

Academic counselling,

Personal development

Psychological growth,

Candidate - job match,

Management

development,

Stress management

Emotional Intelligence

MP SHL, Magellan Consulting

EQI Jopie van Rooyen

15 FQ EI Psytech

TEI Thomas International

Integrity Giotto Mindmuzik Media

PSI Jopie van Rooyen

IP-200IMI

Integrity International

Range of tests available: Personality

MEASUREMENT

CONSTRUCT



FIELDS

Behavioural Underlying emotional

issues,Stress,

Anxiety,Depression,Conflict

BESS Jopie van Rooyen Screening and selectionTraining and development

PromotionManagement potential

Group-role analysisInterpersonal relationshipsCareer psychology

Personnel psychologyOrganizational psychology

BAIBDI-II

BDI Fast screen

Mindmuzik MediaJopie van Rooyen

SWSISRP

MBI

Jopie van Rooyen

SP Mindmuzik Media

TKI Jopie van Rooyen Conflict managementPersonal development

Range of tests available: Behavioral

MEASUREMENT

CONSTRUCT



FIELDS

Interest Career SDSCVS

CIP

Jopie van Rooyen Career guidance and counselling

Personal development and growth

19 FIICDQ

SAVII

Mindmuzik Media

IDEAS workbookCOP

??

Range of tests available: Interest

• Option 1: Purchasing a “report”

– Advised for non-accredited users and for individual testing on

once-off basis

– Some test can be expensive (up to R3,000 per report)

– Suppliers include SHL, Jopie van Rooyen, and Thomas

International

– Benefit: initial costs is low

Computer generated reports is time efficient

– Downfall: ongoing high costs

Potential misuse of reports due to lack of control

Cost Pricing Analysis

Cost Pricing Analysis

• Option 2: Purchasing Material: Software or Paper-and-Pencil version

– Advised for accredited users and group or large-scale testing on

ongoing basis

– Initial purchasing of material and software can be expensive

(need manuals, booklets, answer sheets, scoring stencils, etc –

between R2,000 – R25,000)

– Suppliers include Minimize, M&M Initiatives, PsyTech

– Benefit: ongoing assessments at minimal fee

– Downfall: initial start-up fees can be high

Software need ongoing purchasing of credits for reporting

Manual interpretation, scoring and reporting is time consuming

Key notes

Legal aspects of Psychological Assessment

The Constitution of the Republic of South

Africa:

– Chapter 2: Bill of Rights “is a cornerstone of democracy in South Africa - democratic values of human dignity, equality and freedom”.

– Section 9(3): “The state may not unfairly discriminate directly or indirectly against anyone on one or more grounds, including race gender, sex, and pregnancy, marital status, ethical or social origin, colour, sexual orientation, age, disability, religion, conscience, belief, culture, language and birth.

– Section 9 (4): “No person may unfairly discriminate directly or undirectly against anyone on one or more grounds in terms of subsection (3).

• Section 8: “Psychological testing and other

similar assessments of an employee are

prohibited unless the test or assessment being

used –

– Has been scientifically shown to be valid and reliable;

– Can be applied fairly to all employees;

– Is not biased against any employee or group.

Employment Equity Act, No.55

(1998)

• Section 11: Burden of proof.

• Section 15: Affirmative action.

• Subsection 20(3): “capacity to acquire, within a

reasonable time, the ability to do the job”

• Code of Good Practice on the Integration of

Employment Equity into Human Resources

Policies and Practices: Section 10.

EEA (continued)

• Chapter VIII: Unfair discrimination and unfair

labour practices – same grounds as the

Constitution.

• Chapter VIII: Unfair discrimination and unfair

labour practices – same grounds as the

Constitution.

The Labour Relations Act,

No. 66 (1995)

• Association of Test Publishers of South Africa Saville & Holdsworth (SA) and the Chairperson of the Professional Board for Psychology

• Dispute: ATP: Any individual (unregistered) should be able to conduct psychometric assessments? HPCSA: Only registered professionals !

• Ruling (19 February 2010): In favour of ATP SA. “The court order declares the notice published on the 10th November 2008, which stated that “it is not permissible to use unregistered persons to render psychological services including the administration of tests, instruments or techniques”, to be void and of no force and effect.

• Current status: This ruling will stand until further notice. WATCH THIS SPACE!!!!

Legal Cases

• Buthelezi v Amalgamated Beverage Industries[1999] JOL 5086 (LC)

• Hendricks v SA Airways[2002] JOL 10382 (LC)

• NUM obo Moeng / Douglas Colliery[2007] 7 BALR 647 (CCMA)

Legal Cases (continued)

• Fines: Employment Equity Act

• Costs: CCMA, Labour Court,

Constitutional Court

What will the long arm of the law do

to you?

• Assessments play an important role in the measuring of performance, selection, recruitment, etc. in the workplace, schools and various other institutions.

• Assessment Practitioners should today more than ever be aware of the responsibilities that the legal use of these powerful instruments require.

• Our history has not been without bumps, hurdles and mountains, but it is our responsibility to promote a fair and unbiased future and set an example for the rest of the world. The current limitations and challenges that we face can only be achieved through co-operation, adhering to legislation and high ethical standards.

Closing statement

We rest our case�

Regulation by

Legislation

Regulation by

Professional

Bodies

Self-

Regulation

The Regulation of Psychological Assessment

• LEGISLATION

� Employment Equity Act (1998) – different to all other countries, in that we are asked to proactively defend our position with regards to psychological assessment

• PROFESSIONAL BODIES

• (Responsibilities: Training, Accreditation, Test classification)

� HPCSA -- might not be in a position to efficiently regulate access to and use of tests

� Psychometric Committee of the board of Psychology – falls under HPCSA

� International Test Commission (2001)

� Society for Industrial and Organisational Psychology of South Africa

� People Assessment in Industry

� Human Sciences Research Council

• PROFESSIONAL BODIES (cont.)

� South African Board for People Practices

� Association of test publishers

� Test Developers/Publishers

� Any others? What about universities?

• SELF-REGULATION

� Strategy of self-regulation (Paterson & Uys, 2005) – practitioners empowered to make informed decisions through access to information and adequate training

� Training = better decision making with regards to selection of (registered) tests

� Continuing professional development

� Personal moral compass

• ADDITIONAL ISSUES TO CONSIDER

� Conflicting policies (e.g. Employment Equity Act (1998) –assessments must not discriminate against groups, but can be used to assist with addressing previously disadvantaged groups)

� Personal morals vs Ethical policies

� Canadian Psychological Association – ethical principle ranking to assist decision making

Competency Based Assessment -

• What is competency-based testing?

– Competencies

• Knowledge, Skills, Abilities

– Job Analysis

– Effective in Application

– Movement from intelligence

testing

Competency Based Assessment

Development

Defining Competency Based Assessment

• A way to monitor and assess the competences a person

has, no matter where or how these competences were

gained.

• Three major principles differentiating competence-based

assessment from other methods used to assess are:

– Assessment of evidence

– Current abilities

– Standards or competence

• An assessment of past behaviour can assist you in

predicting future behaviour

• Multiple assessment techniques are

used to verify the results obtained

• The competencies measured are related to successful

job execution

• Objectivity of evaluation is ensured through the utilisation

of trained assessors

Benefits of Competency Based Assessment

“Recognition of prior learning is a process whereby, through assessment, credit is given to learning which has already been

acquired in different ways”. – SAQA

• Discuss practical example of the RPL process

(real life case study)

Recognition of Prior Learning

RPL is seen to have the capacity:

• to contribute to redress equity by opening up more ways for

people to attain qualified status;

• enable more people to reach higher levels of qualification and

expertise by beginning with an acknowledgement of existing

skills and knowledge;

• contribute to enhancing international economic

competitiveness by building on often invisible and

unacknowledged workforce skills;

• and offer the first step in attaining the goal of developing a

multiskilled and flexible workforce by acting as an auditing tool

to quantify existing competence.

Benefits of RPL

Competency Based Assessment Series (CAS)

• Designed by JvR professionals and associates

• Purpose is to provide their clients with a range of uniquely South

African assessment centre exercises

• the exercises are designed to evaluate performance against certain

job relevant competencies and can contribute to an informed

selection decision.

• The exercises can be categorised into four groups, namely:

1. In-basket Exercises

2. Group Exercises

3. Strategy and/or Presentation Exercises

4. Role Play Exercises

• All the exercises are job relevant and assess current as well as

future potential.

Examples of Competency Based Assessment

COMPETENCY ASSESSMENT SERIES (CAS)

The aims of the CAS exercises include:

• To identify behavioural competencies needed for a specific job.

• To assist the assessor in gaining a comprehensive understanding of

the candidates’ behavioural competencies.

• Provide additional, job-relevant, “tangible” information to supplement

that gathered with traditional psychometric tools.

• To determine the participants’ strengths and development areas.

• To use the information in combination with collateral information for

selection and development purposes in the workplace.

Examples of Competency Based Assessment

• Case studies

• In-basket exercises

• Job sample tests/Skills Tests

• Direct Observation

• Oral/Written Tests

• Portfolios

• Simulation

Designing a Competency Based Battery

• Step 1: Planning

– Purpose

• Step 2: Establish Competencies

– Job Analysis

• Step 3: Develop assessment battery

– Structure

• Step 4: Implement and evaluate battery

• Within in the organisation

• In the South African context

– Advantages

– Disadvantages

• Fairness

– Unbiased, ethical

Uses of Competency Based Assessment

Key notes

Computerised Assessment

• CAPA

• CBA

• CBT

• CAT

• CBTI

• IDT

Before we start

Assessment

Generation

Assessment

Delivery

Assessment

Scoring and

Interpretation

Storage,

Retrieval and

Transmission

Computer-based Assessment

• International:

�1962

�1970’s

�1980’s

�1990’s

�2000’s

Taking a step back in History

• South-Africa

�1970

�Late 1970’s and early 1980’s

�1979

�1980’s

�1986

�1990

�1993-1994

Taking a step back in History

Advantages and Disadvantages

• Advantages:– More enjoyable

– Ultimate levels of standardization

– Biasing effect is eliminated

– Reduction of time

– More information about test-takers

– Spatial and perceptual measured to greater extend


• Advantages (continued)– Voice activated and touch screen responses are possible

– Assessments can be individually tailored

– Greater element of control

– Number of assessment practitioners and assistants

needed

– Scoring by a computer

– Increased test security

• Disadvantages

– Eish Eskom

– Hardware / software

compatibility & capability

– Screen size

– Machine availability

– Socio-cultural and linguistic

factors

– System failure



• Disadvantages (continued)– Copy right violation

– Lack of security

– Problems with confidentiality

– Skilled clinical judgement is overlooked

– Difficult to detect certain problems in software

– High costs in item development

– Costly

• Disadvantages (continued)

– Vital qualitative information

not assessed

– Human-computer interface

issues

– Lack of computer literacy


• Four approaches to CBTI are recognized:

– Scoring reports

– Descriptive reports

– Actuarial reports

– Computer-assisted clinical reports

Computer-based Interpretation

CAT

• What is it

• Purpose

• Characteristics

• Measurement Advantages

• What is it?

• Increased attention

• What does it involve?

• South Africa

Dynamic Assessment

• Effective management tool

• Time and cost effective

• Usage

Assessing Potential

• Why will we use it?

• Present and potential

• Opportunity to be measured fairly.

• Is suitable at all levels

• Scientifically proven valid and reliable

• Learning takes place during the assessment

• Is extremely time-saving

• Uses computerised adaptive testing

Assessing Potential

• Advantages:

– Fair

– Suitable for use at all levels of ability;

– Valid and reliable indication of potential to

cope with training;

– Time saving;

– Uses CAT which allows for all ability levels.

Assessing Potential

Item Response Theory

• Definition

• Higher an individual’s ability level, the

greater the chance of getting an item

correct.

• Each item is referenced

Item Response Theory

Good practice and Ethical considerations

• The International Test Commission (ITC)

– Give due regard to technological

– Attend to quality issues in CBT and internet

testing;

– Provide appropriate levels of control

– Make appropriate provision for security

• Minimum professional and ethical

standards:

– Competence

– Potential utility

– Choose a technically (psychometrically)

sound computer-based test

– Equivalence of paper and computer-based

version

– Consider the human factors and issues of

fairness



– Prepare test takers

– Verify the identity of test-takers

– Closely supervise the administration

– Unsupervised psychological test

– Contingency

– Securely stored.

– Computer scoring system

– Interpretation of results

– Debrief test-takers

The future – high definition video and

virtual reality

• Multimedia

• A five-dimension framework

– Innovations in item format

– Innovations in response

– Innovations in media inclusion

– Innovations in the extent of interactivity

– Innovations in scoring

– Authenticity

The Future of Psychological Assessments

Research building on existing knowledge

•Enhancing consumer psychology

•Reduced costs

•System generated reports

•Dynamic assessment of intelligence

•Construction of new adaptive tests

•Need for an all encompassing body to:

�Monitor test use

�Advise practitioners

�Research and review tests with information centrally

available

�To monitor and coordinate test development,

adaptation, and updating (Foxcroft, 2004)

• Control and regulation

•Increased availability of information via internet

The Future Role of the HPCSA

Artificial Intelligence

•Virtual Reality – a new tool for psychological assessment

(Fernandez-Ballesteros, 2009)

•Computer assisted assessment

•Ethical considerations

•Control and access to electronic assessments

•Closer examination of the tester-testee relationship

The Future of Assessments in South Africa

• Locally developed tests appropriate for all cultures within

SA

• Leadership tests appropriate for SA context

• Explore other intelligences – cosmic, spiritual, moral,

emotional, transcendental contributions

• Stimulating use of assessments by smaller organisations and NGOs

• Industrial psychologists making

significant contributions locally and

internationally


• Psychometric tests – most effective of predicting behaviour (Mittner, 1998)

• Tests 4 X more effective than screening interviews (Van der Walt, 1998)

• Employment Equity Act – definite move towards making assessment techniques more scientific (Eckstein, 1998)

• To be used as additional aid in decision making i.e. not in isolation

• Retain professional level for conducting psychometric assessment.


• (Bartram & Coyne, 1998) in a recent world-wide survey found lack of psychologists involvement in testing and use of tests

– Found 60% of test users were not specifically trained in use of psychological tests.

– Therefore recommend training and increasing public understanding.

– Gregoire, (1999), recommend academic education, continued education, and more publications on tests and assessment methods.


• Cook (1997), trend towards assessment centres and competency based assessment in changing socio-political and economic context. [some psychological factors that competence measures cannot provide, i.e. learning and potential]

• Consideration should be given to development of new tests with urgent special emphasis on culture-fairness.

• Validation of tests in different organisations that they are

used in. Cook (1997),

Conclusion: History

� In the past the same tests were

used for everyone.

� Psychologists used international

tests.

� Testing was for a long time viewed

as discriminatory and unjust.

� Practitioners are becoming aware

of the advantages of fair

assessment.

� The implementation of cross-culturally fair tests.

Conclusion: Assessment Areas

COGNITIVE PERSONALITY

BEHAVIOURAL INTEREST

Assessment

Areas

� Changes in legislation.

� The EEA has been established in

1998 to ensure more equitable

and fair Psychological testing.

� Tests need to be scientifically valid

and reliable;

� tests should be able to be applied

fairly to all employees; and

� tests should not be biased against

any employee or group.

Conclusion: Legislation

Conclusion: Ethics

� Psychological tests can act as

a disabling factor.

� Practitioners therefore have to

be more cautious and informed and should take responsibility

to ensure that a test can be

fairly applied.

� Practitioners should act in an

ethical way before, during and

after psychological tests.

Conclusion: Competency Based Assessment

� Focus on the essential

behaviours required to perform a

specific job.

� Focussing on individual

differences in terms of work-

related constructs relevant to job performance.

� This contributes to keep tests fair and relevant as required by the

law.

Conclusion: Computerised Assessment

� In our modern society there is new

technology on the market every

day and everything is being

computerised in one way or

another as well as psychological testing.

� There is a lot of advantages to

computerized testing we just need

to make sure that it remains unbiased, ethical, valid and

reliable for our rainbow nation.

Conclusion: Challenges

� According to research the most

frequently cited hindrance to the

administration of psychological

tests is language.

� When administering individual

intelligence tests, psychologists

often argue that it is justifiable to

administer the measure in

English, irrespective of whether English is the first or second

language of the test-taker.

Continue...


� It is important to determine

whether the performance on the

test reflects the testee’s actual

ability and not the testee’s

competence in the test language.

� Translation of a test is not a quick-

fix.

� Changing some wording in an

item/question can draw a question

mark over the construct, score and

predictive comparability.

Continue...


� Other factors that need to be taken

into account: cultural and

environmental factors, age and

gender.

� Test anxiety has an effect on test

resuls.

� Anxiety can hamper a person’s test performance.

� Employees may also start to view tests

as unfair and invalid.

� Explain everything in simple terms.

“If you talk to a man in a language he understands, that goes to his head. If you talk to him in his

language, that goes to his heart.” Nelson Mandela

“ The most important single ingredient in the

formula of success is knowing how to get along with people” Theodore Roosevelt.

Crossword Competition

Documents

PSYCHOLOGICAL ASSESSMENT SEMINAR - libvolume7.xyz · PSYCHOLOGICAL ASSESSMENT SEMINAR Rubik’s cubes of colour, shape and size: Ethical psychological testing in a multicultural work