2010 01 psyf 588 master syllabus

MASTER SYLLABUS PSYF-588

CARLOS ALBIZU UNIVERSITYSAN JUAN CAMPUS

MASTER SYLLABUSPSYF-588: THEORY OF TESTS AND TEST CONSTRUCTION

CREDITS: 3 CONTACT HOURS: 45

COURSE DESCRIPTION The goal of this course is to present the major principles of test construction in psychological measurement. Methods for determining validity and reliability will be examined by performing class exercises. The content also includes the study of scaling methods such as Guttman, Thurstone, and Likert scales. Moreover, students will apply knowledge from the course to construct their own assessment instruments. PRE-REQUISITES

PSYF-568 Applied Inferential Statistics

COURSE OBJECTIVES This course provides graduate students in psychology basic knowledge in measurement and test development. Students will be able to make responsible and professional decisions in selecting or developing instruments. REQUIRED TEXT BOOKS DeVellis, R.F. (2003). Scale development: Theory and application (2nd ed.).

London: Sage Publications. ISBN-10: 0761926054; ISBN-13: 978-0761926054

Kline, T. (2005). Psychological testing: A practical approach to design and evaluation. Thousand Oaks, CA: Sage Publications. ISBN-10: 1412905443; ISBN-13: 978-1412905442

Kline, P. (2000). Handbook of psychological testing (2nd ed.). New York: Routledge. ISBN-10: 0415211581; ISBN-13: 978-0415211581

Page 1 of 18


Tornimbeni, S., Pérez, E., Olaz, F., & Fernández, A. (2004). Introducción a los test psicológicos (3era ed. rev.). Argentina: Editorial Brujas. ISBN-10: 9871142242

ITINERARY OF CLASS UNITS

UNIT 1: Basic concepts, historical background, and measurement modelsUNIT 2: General steps of the test construction process UNIT 3: Item construction: Sensitivity to cultural and individual variablesUNIT 4: ValidityUNIT 5: Item analysisUNIT 6: ReliabilityUNIT 7: Development of the test manual and the test administration processUNIT 8: Review of statistical conceptsUNIT 9: Norms and standard scoresUNIT 10: Discriminatory power of the testUNIT 11: Ethical principles and their role in the test construction process

COURSE CONTACT HOURS

Professors who teach the course must divide the contact hours the following way:

1. Face-to-face time in the classroom must not be less than 40.0 hours (16 classes, 2.5 hours each class).

2. For the remaining hours (≥ 5.0 hours), students will conduct research projects or homework outside the classroom. These projects or homework will include, but are not limited to, literature review, field work (i.e. experts evaluation of item content validity, instrument administration), statistical analysis with SPSS, and writing the test manual.

METHODOLOGY The specific methodology will be selected by the professor who offers the course. These methodologies could include, but would not be limited to, conferences by the professor, group discussions of assigned readings, class research projects, student presentations, individual meetings with students and sub-groups in the classroom. EDUCATIONAL TECHNIQUES The specific educational techniques will be selected by the professor who offers the course. These techniques could include, but are not limited to, group or individual projects, debates, practical demonstrations, films/videos, simulations, slide shows and forums.

Page 2 of 18


EVALUATION The specific evaluation criteria will be selected by the professor who offers the course. These methodologies could include, but would not be limited to, term papers, projects, literature reviews, exams and class presentations. Three partial exams are recommended to examine the material discussed. The development of a scale or test in an area of interest for the student is highly recommended. The student should identify a psychological construct of his/her interest, develop items to measure it, administer these items to a sample, and analyze its psychometric properties (item discrimination index, validity, reliability, norms).

RESEARCH COMPETENCIES

1. Compare/contrast the principles of several theories pertaining to test use2. Evaluate and select research instruments that are appropriate for a particular

research project3. Design, develop, and validate research instruments4. Select statistical tests that are appropriate for data analysis5. Interpret the results of statistical data analysis, including descriptive and

inferential statistics6. Perform a literature research of the formulate research problem7. Evaluate and analyze critically quantitative research that is presented in the

literature

ATTENDANCE POLICY

Class attendance is mandatory for all students. After two unexcused absences, the student will be dropped from the class, unless the professor recommends otherwise. When a student misses a class, he/she is responsible for the material presented in class.

AMERICANS WITH DISABILITIES ACT (ADA)

Students that need special accommodations should request them directly to the professor during the first week of class.

Page 3 of 18


COURSE UNITS

UNIT 1: BASIC CONCEPTS, HISTORICAL BACKGROUND, AND MEASUREMENT MODELS

Upon successful completion of this unit, students will understand basic concepts commonly used in test theory, test historical background, and models of measurement. LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. Define the concept of test, measurement, and assessment2. Identify different types of tests3. Identify the purposes of tests4. Review the historic development of test theory and test development 5. Identify cultural sensitive problems most commonly encountered in test

development in Puerto Rico6. Identify the differences between classic test theory and modern test theory7. Discuss the concept of individual differences and its impact in assessment ASSIGNED READINGS: 1. Kline (2005)

Chapter 1 – The Assessment of Individuals: The Critical Role and Fundamentals of Measurement

Chapter 5 – Classic Test Theory: Assumptions, Equations, Limitations, and Item Analyses

Chapter 6 – Modern Test Theory: Assumptions, Equations, Limitations, and Item Analyses

2. DeVellis (2003) Chapter 7 – An Overview of Item Response Theory

3. Tornimbeni et al. (2004) Chapter 1 – Fundamentos de la Medición Psicológica Chapter 2 – Evolución Histórica de los Tests Chapter 3 – Paradigmas de la Psicometría

4. Cirino, G., Herrans, L.L. & Rodríguez, J.M. (1988). El futuro de la medición psicológica en Puerto Rico: Predicciones y recomendaciones. En Memorias Primer Simposio de Medición Psicológica en Puerto Rico. Asociación de Psicología de Puerto Rico.

Page 4 of 18


UNIT 2: GENERAL STEPS OF THE TEST CONSTRUCTION PROCESS Upon successful completion of this unit, students will learn how to construct a test. LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. List the steps in test construction2. Explain the procedures for preparing test specifications3. Discuss the importance of test specifications4. Examine the process followed in the preliminary item tryouts5. Discuss the importance of the test sensitivity review ASSIGNED READINGS: 1. DeVellis (2003)

Chapter 5 – Guidelines in Scale Development

2. Tornimbeni et al. (2004) Chapter 8 – Construcción de Pruebas Chapter 9 – Adaptación de Tests a Diversas Culturas

3. Cirino, G. (1992). Introducción al desarrollo de pruebas escritas. Río Piedras, PR: Editorial Bohío. Chapter 3 – Planificación de una Prueba Educativa Chapter 4 – Planificación de una Prueba de Selección de Personal

4. Murphy, K.R. & Davidshofer, C.O. (2001). Psychological testing: Principles and applications. New Jersey: Prentice Hall. Chapter 11 – The Process of Test Development

Page 5 of 18


UNIT 3: ITEM CONSTRUCTION: SENSITIVITY TO CULTURAL AND INDIVIDUAL VARIABLES

Upon successful completion of this unit, students will understand how to construct a list of items.

LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. Explain the procedures followed in the construction of test items:

a. essay test itemsb. two-option alternate-response test items (true or false)c. multiple choice test itemsd. matching test itemse. completion or fill-in itemsf. interest and personality inventories itemsg. attitude scales itemsh. projective techniques items

2. Discuss the important aspects of item reviews

ASSIGNED READINGS:

1. Kline (2005) Chapter 2 – Designing and Writing Items Chapter 3 – Designing and Scoring Responses

2. Kline (2000) Chapter 5 – Rasch Scaling and Other Scales Chapter 6 – Computerized and Tailored Testing Chapter 11 – Other Methods of Test Construction

3. DeVellis (2003) Chapter 5 – Guidelines in the Scale Development

4. Cirino, G. (1992). Introducción al desarrollo de pruebas escritas. Río Piedras, PR: Editorial Bohío. Chapter 5 – Formulación de Preguntas de Múltiples Alternativas

Page 6 of 18


UNIT 4: VALIDITY

Upon successful completion of this unit, students will understand the concept of validity.

LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. Define the concept of validity2. Define the three major approaches to test validation3. Explain the steps to be followed in content validation4. Explain the steps to be followed in criterion related validation5. Explain the construct validation process6. Apply the formulas for computing each type of validity7. Explain the results obtained from a validation process8. Describe and analyze the practical consideration in each type of validation

process9. Establish the relationship between validity and reliability

ASSIGNED READINGS:

1. Kline (2005) Chapter 9 – Assessing Validity Using Content and Criterion Methods Chapter 10 – Assessing Validity via Item Internal Structure

2. Kline (2000)

Chapter 2 – The Validity of Psychological Tests

3. DeVellis (2003) Chapter 4 – Validity

4. Tornimbeni et al. (2004) Chapter 6 – Validez

5. Lawshe, C.H. (1975). A quantitative approach to content validity. Personnel

Psychology, 28, 563-575.

6. Rungtunsanatham, M. (1998, July). Let’s not overlook content validity. Decision Line, 10-13.

Page 7 of 18


UNIT 5: ITEM ANALYSIS

Upon successful completion of this unit, students will understand the item analysis process. In addition, students will be able to understand the role of SPSS in the item analysis process. LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. Define item analysis2. Discuss the concepts of item difficulty (i.e. P and Delta) and item

discrimination (i.e. D and rbis)3. Describe the steps followed in item analysis4. Apply the formulae for computing item difficulty and discrimination5. Interpret the results obtained from an item analysis6. Use SPSS to perform item analysis

ASSIGNED READINGS: 1. Kline (2005)

Chapter 5 – Classic Test Theory: Assumptions, Equations, Limitations, and Item Analyses

2. Kline (2000) Chapter 10 – Test Construction – Factor Analytic and Item Analytic

Methods

3. DeVellis (2003) Chapter 7 – An Overview of Item Response Theory

4. Tornimbeni et al. (2004) Chapter 8 – Construcción de Pruebas

5. Cirino, G. (1992). Introducción al desarrollo de pruebas escritas. Río Piedras, PR: Editorial Bohío. Chapter 9 – Análisis de Ítems

6. Sayers, S., & Vélez, M. (2006, noviembre). Using SPSS for the final project of the PSYF-588 course. Unpublished manuscript, Carlos Albizu University, San Juan Campus, PR.

7. Field, A. (2005). Discovering statistics using SPSS for Windows (2nd ed.). London: SAGE Publications. Chapter 2 – The SPSS Environment Chapter 15.7 – Reliability Analysis

Page 8 of 18


UNIT 6: RELIABILITY

Upon successful completion of this unit, students will understand the concept of reliability and develop skills in the statistical procedures for its estimation. In addition, students will be able to understand the role of SPSS in the reliability analysis process LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. Define the concept of reliability2. Explain the procedures for estimating reliability: test-retest, equivalent forms,

internal consistency (i.e. split half, Cronbach's alpha), and scorer reliability3. Apply the formulas for computing the different types of reliability4. Explain the results of a reliability coefficient5. Identify the sources of unreliability6. Use SPSS to perform reliability analysis


Chapter 7 – Reliability of Test Scores and Test Items Chapter 8 – Reliability of Raters

2. Kline (2000) Chapter 1 – Reliability of Tests: Practical Tests

3. DeVellis (2003) Chapter 3 – Reliability

4. Tornimbeni et al. (2004) Chapter 5 – Confiabilidad

5. Field, A. (2005). Discovering statistics using SPSS for Windows (2nd ed.). London: SAGE Publications. Chapter 15.7 – Reliability Analysis

Page 9 of 18


UNIT 7: DEVELOPMENT OF THE TEST MANUAL AND THE TEST ADMINISTRATION PROCESS

Upon successful completion of this unit, students will understand how a test manual is prepared. Also, they will understand the process involved in developing the test administration process.

LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. Identify the information that should be part of a test manual2. Discuss the importance of standardization of procedures in test administration3. Define the concept of test anxiety4. Explain how motivation affects test performance5. Discuss the importance of preparation of the examiner and supervision in test

administration

ASSIGNED READINGS:

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Chapter 3 – Test Development and Revision Chapter 5 – Test Administration, Scoring, and Reporting Chapter 7 – Fairness in Testing and Test Use Chapter 8 – The Rights and Responsibilities of Test Takers Chapter 9 – Testing Individuals of Diverse Linguistic Backgrounds Chapter 10 – Testing Individuals with Disabilities

Page 10 of 18


UNIT 8: REVIEW OF STATISTICAL CONCEPTS

Upon successful completion of this unit, students will know the statistical concepts most commonly used in measurement. LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. Recapitulate previously learned statistical concepts: scales of measurement,

sampling, frequency distribution, measures of central tendency, variability, and correlation

2. Differentiate among methods of sampling3. Apply the formula for obtaining a sample from a known population and

interpret the results4. Apply the formula for stratified sampling and interpret the results5. Apply the guessing correction formula and interpret the results


Chapter 1 – The Assessment of Individuals: The Critical Role and Fundamentals of Measurement

Chapter 4 – Collecting Data: Sampling and Screening

2. Murphy, K.R. & Davidshofer, C.O. (2001). Psychological testing: Principles and applications. New Jersey: Prentice Hall. Chapter 4 – Basic Concepts in Measurement and Statistics

3. Daniel, W.W. (2006). Bioestadística: Base para el análisis de las ciencias de la salud (4ta ed.). México: Limusa Wiley. Chapter 1 – Introducción a la Bioestadística Chapter 2 – Estadística Descriptiva Chapter 9 – Regresión y Correlación Lineal Simple

Page 11 of 18


UNIT 9: DEVELOPMENT OF NORMS AND STANDARDIZED SCORES

Upon successful completion of this unit, students will understand how to develop test norms. In addition, students will learn the importance of using standardized scores. Finally, students will be able to explain how the standard error measurement is used to establish confidence levels for standard scores. LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. Discuss the basic steps in the development of test norms2. Assess the importance of describing the norms development process in the

test manual3. Describe common standard scores: z score, T score, stanines, percentiles,

and percentile rank4. Apply the formulae for the different standard scores and interpret the results5. Define the concept of standard error of measurement6. Apply the formula for computing the standard error of measurement7. Explain the results of the standard error of measurement8. Use SPSS to calculate standardized scores and the standard error of

measurement ASSIGNED READINGS: 1. Kline (2000)

Chapter 3 – The Classical Model of Test Error Chapter 4 – Standardizing the Test

2. Tornimbeni et al. (2004) Chapter 5 – Interpretación de las Puntuaciones: Tests Referidos a

Normas y Criterios

3. Murphy, K.R. & Davidshofer, C.O. (2001). Psychological testing: Principles and applications. New Jersey: Prentice Hall. Chapter 5 – Scales, Transformations, and Norms

4. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Chapter 4 – Scales, Norms, and Score Comparability

Page 12 of 18


UNIT 10: DISCRIMINATORY POWER OF THE TEST Upon successful completion of this unit, students will learn how to compute the discriminatory power of a test and interpret its results. LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to:

1. Apply Ferguson's Delta formula and interpret the results2. Understand the difference between the discriminatory power of a test and

the discrimination index of an item


Chapter 2 – The Validity of Psychological Tests

Page 13 of 18


UNIT 11: ETHICAL PRINCIPLES AND CONSIDERATIONS Upon successful completion of this unit, students will understand the ethical principles involved in test development. LEARNING OBJECTIVES: Upon successful completion of this unit, students will be able to: 1. Identify the ethical and professional principles involved in test development.2. Examine the impact of the violation of these principles.

ASSIGNED READINGS:

1. Kline (2005) Chapter 11 – Ethics and Professional Issues in Testing

2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Chapter 11 – The Responsibilities of the Test Users Chapter 12 – Psychological Testing and Assessment Chapter 13 – Educational Testing and Assessment Chapter 14 – Testing in Employment and Credentialing

Page 14 of 18


REFERENCES Álvaro Page, M. (1993). Elementos de psicometría. Madrid: EUDEMA Universidad.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

American Psychological Association. (1992). Ethical principles of psychologists and code of conduct. American Psychologist, 47, 1597-1611.

Anastasi, A. (1982). Psychological testing. New York: MacMillan.

Anstey, E. (1976). Los tests psicológicos. Madrid: Morova.

Burisch, M. (1984). Approaches to personality inventory construction. American Psychologist, 39, 214-227.

Camilli, G. y Shepard, L.S. (1994). Methods for identifying biased test items. California: SAGE Publications.

Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284-290.

Cirino, G. (1992). Introducción al desarrollo de pruebas escritas. Río Piedras, PR: Editorial Bohío.

Cirino, G., Herrans, L.L. & Rodríguez, J.M. (1988). El futuro de la medición psicológica en Puerto Rico: Predicciones y recomendaciones. En Memorias Primer Simposio de Medición Psicológica en Puerto Rico. Asociación de Psicólogos de Puerto Rico.

Clark, L.A. y Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319.

Cortada de Kohan, N. (1999). Teorías psicométricas y construcción de tests. Argentina: Lugar Editorial.

Cortada de Kohan, N. (2004). Teoría y métodos para la construcción de escalas de actitudes. Argentina: Lugar.

Cronbach, L.J. (1990). Essentials of psychological testing (5th ed.). New York: Harper and Row.

Daniel, W.W. (2006). Bioestadística: Base para el análisis de las ciencias de la salud (4ta ed.). México: Limusa Wiley.

Dahlstrom, W.G. (1993). Tests: Small samples, large consequences. American Psychologist, 48, 393-399.

Page 15 of 18


Denova, C.C. (1979). Test construction for training evaluation. New York: Van Nostrand Reinhold.

DeVellis, R.F. (2003). Scale development: Theory and application (2nd ed.). London: Sage Publications.

Endler, N.S. y Parker, J.D.A. (1994). Assessment of multidimensional coping: Task, emotion, and avoidance strategies. Psychological Assessment, 6, 50-60.

Fan, C.T. (1952). Item analysis table. New Jersey: Educational Testing Service.

Field, A. (2005). Discovering statistics using SPSS for Windows (2nd ed.). London: SAGE Publications.

Frederiksen, N. (1984). The real test bias. American Psychologist, 39, 193-202.

Geisinger, K.F. (1994). Cross-cultural normative assessment: Translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychological Assessment, 6, 304-312.

Haladyna, T. (1999). Developing and validating multiple-choice test items. New Jersey: Lawrence Erlbaum Associates.

Hambleton, R.K., Swaminathan, H. & Rogers, H.J. (1991). Fundamentals of item response theory. London: SAGE.

Haynes, S.N., Richard, D.C.S. y Kubany, E.S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7, 238-247.

Herrans, L.L. (2000). Psicología y medición. México: McGraw Hill.

Huebner, E.S. (1994). Preliminary development and validation of a multidimensional life satisfaction scale for children. Psychological Assessment, 6, 149-158.

Junta Examinadora de Psicólogos de Puerto Rico (1988). Código de ética. Revista Puertorriqueña de Psicología, 5, 71-82.

Kaplan, R.M. y Saccuzzo, D.P. (1993). Psychological testing. California: Brooks/Cole Publishing Company.

Kehoe, J.F. y Tenopyr, M.L. (1994). Adjustment in assessment scores and their usage: A taxonomy and evaluation of methods. Psychological Assessment, 6, 291-303.

Kline, P. (2000). The handbook of psychological testing (2nd ed.). New York: Routledge.

Kline, T. (2005). Psychological testing: A practical approach to design and evaluation. Thousand Oaks, CA: Sage Publications.

Page 16 of 18


Lara-Cantu, M.A., Verduzco, M.A., Acevedo, M.C. y Cortes, J. (1993). Validez y confiabilidad del Inventario de Autoestima de Coopersmith para adultos, en población mexicana. Revista Latinoamericana de Psicología, 25, 247-255.

Lawshe, C.H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563-575.

Likert, R. (1967). The method of constructing an attitude scale. In M. Fishbein (Ed.), Readings in attitude theory and measurement. New York: John Wiley & Sons, Inc.

López, N.J. y Domínguez, R. (1993). Medición de la autoestima en la mujer universitaria. Revista Latinoamericana de Psicología, 25, 257-273.

Matarazzo, J.D. (1992). Psychological testing and assessment in the 21st century. American Psychologist, 47, 1007-1018.

McIntire, A. y Miller, L. (2000). Foundations of psychological testing. New York: McGraw Hill.

Meliá, J.L., Oliver, A. y Tomás, J.M. (1993). El poder en las organizaciones y su medición. El cuestionario de Poder Formal e Informal. Revista Latinoamericana de Psicología, 24, 139-155.

Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' response and performance as scientific inquiry into score meaning. American Psychologist, 50, 741-749.

Murphy, K.R. & Davidshofer, C.O. (2001). Psychological testing: Principles and applications. New Jersey: Prentice Hall.

Nunnally, J.C. (1978). Psychometric theory. New York: McGraw Hill.

Ramos-Lira, L. y Andrade-Palos, P. (1991). La victimización: miedo, riesgo percibido y gravedad percibida. Construcción y validación de escalas de medición. Revista Latinoamericana de Psicología, 23, 229-246.

Rodríguez-Irlanda, D. (1998). Medición, “assessment” y evaluación. San Juan, PR: Publicaciones Puertorriqueñas.

Rungtunsanatham, M. (1998, July). Let’s not overlook content validity. Decision Line, 10-13

Sánchez-Viera, J.A. (2004). Fundamentos del razonamiento estadístico (3era ed. revisada). San Juan, PR: Universidad Carlos Albizu.

Sayers, S., & Vélez, M. (2006, noviembre). Utilizando SPSS para el trabajo final de PSYF-588. Manuscrito inédito, Universidad Carlos Albizu, Recinto de San Juan, PR.

Page 17 of 18


Silva, F. (1993). Psychometric foundations and behavioral assessment. California: SAGE Publications.

Thorndike, R.L. (1982). Applied psychometrics. Boston, MA: Houghton Mifflin Co.

Thurstone, L.L. (1967). Attitudes can be measured. En Fishbein, M. Readings in attitude theory and measurement. New York: John Wiley & Sons, Inc.

Tornimbeni, S., Pérez, E., Olaz, F., & Fernández, A. (2004). Introducción a los test psicológicos (3era ed. rev.). Argentina: Editorial Brujas.

Westgaard, O. (1999). Tests that work. San Francisco, CA: Jossey-Bass.

Zeidner, M. y Most, R. (1992). Psychological testing: An inside view. California: Consulting Psychologists Press, Inc.

Rev. /2004

Revised by: Sean K. Sayers Montalvo, Ph.D. (August, 2008; December 2009)

Page 18 of 18

Technology

2010 01 psyf 588 master syllabus