Chapter 1

Psychological Testing and Assessment

Twelve Assumptions in Psychological Testing and Assessment

• Assumption 1: Psychological traits and states exist– Observed (manifest) vs. latent

– Can you study something invisible?

– Logical positivism, Behaviorism

• Assumption 2: Psychological traits and states can be quantified and measured– Can you really assign a numeric value to everything?

– Rasch: objective measurement

• Assumption 3: Various approaches to measuring aspects of the same thing can be useful (Methodology Triangulation)

Twelve Assumptions in Psychological Testing and Assessment

• Assumption 4: Assessment can provide answers to some of life’s most momentous questions– Tracking (in the UK system)– screening and placement

• Assumption 5: Assessment can pinpoint phenomena that require further attention or study– National (?) assessment: No Child Left

behind (The objective of assessment is…)– International assessment: PISA, TIMSS

What is PISA? Not these

• PISA: Program for International Student Assessment is a series of assessments on reading literacy, mathematics literacy, and science literacy, sponsored by Organization for Economic Cooperation and Development (OECD), and administered internationally to 15-year-olds

• TIMSS: The Trends in International Mathematics and Science Study (TIMSS) is administered every four years internationally to both fourth and eighth grade students.

Source: OECD/PISA and Wikipedia

Source: TIMSS and Wikipedia

Obama's "Sputnik Moment"

• In 1957 the USSR launched the first artificial Earth satellite, Sputnik. In order to fill the technological gap, the US government awarded financial support to promote math and science education.

• In response to the performance gap between the US students and Asian/European students, President Obama introduced the phrase "Sputnik moment" to argue for more investment in science, technology, engineering, and technology (STEM) education.

Twelve Assumptions in Psychological Testing and

Assessment (Cont’d)• Assumption 6: Various sources of data

enrich and are part of the assessment process (Data triangulation)

• Assumption 7: Various sources of error are part of the assessment process

Error is everywhere!

• Sources of error– Measurement error– Sampling error

• Types of error– Bias: Systematic e.g.

self-report height and weight

– Variance: Random

Twelve Assumptions in Psychological Testing and

Assessment (Cont’d)• Assumption 8: Tests and other measurement

techniques have strengths and weaknesses– Classical test theory– Item response theory– Rasch model

Twelve Assumptions• Assumption 9: Test-related behavior predicts non–test-related

behavior

• Assumption 10: Present-day behavior sampling predicts future behavior – Academically adrift: Limited learning on college campuses -- After 4

years, 36% college graduates showed no gains in higher order thinking skills. US corporations complain that college graduates don’t have the critical thinking skill required by the jobs.

– Corporations launch their own assessment programs e.g. MCSE, CCNA.

Examples of industry certification/assessment program

Twelve Assumptions in Psychological Testing and

Assessment (Cont’d)

• Assumption 11: Testing and assessment can be conducted in a fair and unbiased manner

• Assumption 12: Testing and assessment benefit society

Who, What and Why?

• Who Are the Parties?– The test developer– The test user– The test taker– Society at large

The test developer

• Subject matter expert/content expert: item authoring

• Psychometrician: item analysis, factor analysis

• Database programmer/IT personnel: item banking

• Program manager: put all together

Evaluating the Quality of Tests

• What is a “Good” Test?– Reliability– Validity– Other settings

• Reference Sources for Test Information– Test catalogues– Test manuals– Journal articles– Online and CD-Rom databases – Other sources

• http://www.creative-wisdom.com/pub/efficacy.pdf

Exercise (2 points)

• Form a group consisting of 3-4 people. One or two must have a Web-enabled laptop.

• Access the APU library• Look for Mental Measurements Yearbook (the

electronic version)• Search for instruments that measure psychological

constructs (e.g. anxiety, depression, happiness…etc)• Read the comments at the end• Discuss which one is the best.