Copyright 2005 AMP

Update from AMP –New Broker Candidate

Testing Method

Presented to the GREC 2005 Annual School Meeting

December 2, 2005

Lawrence J. Fabrey, PhD

Vice President, Measurement Research

Applied Measurement Professionals, Inc.

Copyright 2005 AMP

Overview of Presentation

• Some (brief) principles of assessment

• Results of recent Job Analysis

• New Examination Specifications

• Purpose and structure of simulations

• Time frame for implementation

• Experience a sample simulation (or two or three)

Copyright 2005 AMP

Principles of MeasurementLinks in the Chain of Evidence Used to

Support the Validity of Examination Results: Job Analysis Test Specifications Item Writing Examination Development Standard Setting Test Administration Scoring

Copyright 2005 AMP

Job Analysis• Survey sent to a sample of 12,337 real

estate professionals– Received responses from 776 brokers and 427

salespersons

• 166 real estate job tasks and 16 real estate major issues rated for significance– Task ratings used to develop multiple choice

examination specifications– Issue ratings used to develop the simulation

examination specifications

Copyright 2005 AMP

New MC Exam Specs – overview

Copyright 2005 AMP

Detailed Outline –www.goAMP.com

Copyright 2005 AMP

Copyright 2005 AMP

Copyright 2005 AMP

Free Demo Test

Copyright 2005 AMP

• Free Sample CBT items:

www.lxr.com click on “take a demo web test now”

Copyright 2005 AMP

Copyright 2005 AMP

Simulations:Simulations:Purpose of Simulations

• To provide accurate assessment of higher order thinking related to a content area of interest

• That is, testing more than just recall• Addressing important issues facing

brokers (based on JA results)

Copyright 2005 AMP

Structure of Simulations• Opening Scenario

• Information Gathering (IG) Sections

• Decision Making (DM) Sections– Single or multiple DM

• All choices are weighted (+3 to –3)

• Passing scores relate to judgment of ‘minimal competence’

Copyright 2005 AMP

Simulation Specifications

Copyright 2005 AMP

Simulation Development

Copyright 2005 AMP

We’ll Have Time to Complete a Simulation or Two

Samples available now through the GA REP page on www.goAMP.com

First, a “simulated” login, and then, we’ll want your answers

Copyright 2005 AMP

Copyright 2005 AMP

Copyright 2005 AMP

Copyright 2005 AMP

In case we would need

help

Copyright 2005 AMP

Copyright 2005 AMP

Upon Completion of a Sim

Copyright 2005 AMP

Several score reports

Copyright 2005 AMP

Copyright 2005 AMP

Copyright 2005 AMP

Copyright 2005 AMP

Time line for Implementation

• Pilot testing has begun

– Please continue to encourage students to participate

• Live testing begins March 1, 2006

• Delayed score reporting likely

More information after the first of the year

Copyright 2005 AMP

Let’s take a simulation problem (or two or three)

Copyright 2005 AMP

Georgia Real Estate Examination Program

Contact InformationLawrence J. Fabrey, PhD, VP, Measurement Research

LFabrey@goAMP.com (913.495.4406)Applied Measurement Professionals, Inc.

8310 Nieman Road, Lenexa KS 66214