Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
More than Tests?: ExpandingMore than Tests?: Expandingdesign from tests to instructiondesign from tests to instruction
and learningand learning
Brian GongBrian GongCenter for AssessmentCenter for Assessment
Discussion of “Creating Tests that Measure WellDiscussion of “Creating Tests that Measure Well andand thatthatModel Good Instructional and Learning PracticeModel Good Instructional and Learning Practice” Session by” Session by
Randy Bennett and Edys QuellmalzRandy Bennett and Edys Quellmalz
CCSSO National Conference on Student AssessmentCCSSO National Conference on Student AssessmentMinneapolis, June 28, 2012Minneapolis, June 28, 2012
Overview of commentsOverview of comments
Two really strong projects of national standingTwo really strong projects of national standing
Comprehensive assessment systemsComprehensive assessment systems
Design lessonsDesign lessons Design lessonsDesign lessons
Process lessonsProcess lessons
2Gong – CCSSO - Tests that Measure & Model - 6/28/12
1. Contribution to Comprehensive1. Contribution to ComprehensiveAssessment SystemsAssessment Systems
One assessment taxonomy: formative, interim,One assessment taxonomy: formative, interim,summativesummative
Within interim, another taxonomy of purposes:Within interim, another taxonomy of purposes: Within interim, another taxonomy of purposes:Within interim, another taxonomy of purposes:instructional, evaluative, predictiveinstructional, evaluative, predictive (Perie, Marion,(Perie, Marion, &&
Gong, 2009,Gong, 2009, EMIPEMIP))
A different view: more contextualizedA different view: more contextualizedassessments; relation to curriculumassessments; relation to curriculum
3Gong – CCSSO - Tests that Measure & Model - 6/28/12
Assessment Relation toAssessment Relation toCurriculum/InstructionCurriculum/Instruction
More decontextualized More contextualized
Targeted to standards,generalized away from particularinstructional means to theoutcome (“ends”)
Targeted to standards withincontext of particular instruction(“means” = context)
outcome (“ends”)
Claim & interpretation typicallyfocused on “construct”
Claim & interpretation often interms of construct & content
Comparability & reliabilityvalued, leading to high degree ofstandardization
Some flexibility built in, alwaysin instruction, sometimes inelements of assessment (e.g.,administration)
4Gong – CCSSO - Tests that Measure & Model - 6/28/12
Assessment Relation toAssessment Relation toCurriculum/InstructionCurriculum/Instruction –– 22
More decontextualized More contextualized
Traditional psychometric quality,generalizability, credibility, andlow intrusion valued overtimeliness and detail
Balance shifted towardscontextual detail, timeliness, andlocalized control; try to maintaintraditional psychometric qualitytimeliness and detail traditional psychometric qualitybut not as generalized (?)
Main means of performanceinterpretation linked to constructand scale, and strongmeasurement models
Main means of performanceinterpretation in terms of (multi-dimensional) content/skills,learning theory, and curriculum
Assessment information use nottied to particular instruction
Assessment information used toinform particular instruction
5Gong – CCSSO - Tests that Measure & Model - 6/28/12
Degree of Contextualization,Degree of Contextualization,Amount of Curriculum SpanAmount of Curriculum Span
Curriculum Span - Large
-Sp
arse
Muc
h
Performance assessmentsoften are morecontextualized; assess
6Gong – CCSSO - Tests that Measure & Model - 6/28/12
Curriculum Span - Small
Co
nte
xtua
lizat
ion
-
Co
nte
xtua
lizat
ion
-
Typical large-scale
assessmentCollection of items
Individual item
contextualized; assessaspects of constructs notaddressed well in otherassessments; and oftenhave greater depth but lesscurricular breadth
2. Design Lessons2. Design Lessons
1. Good models of assessment1. Good models of assessment
2. Good models of learning2. Good models of learning
3. Good models of instruction3. Good models of instruction
4. Good models4. Good models of how these relateof how these relate 4. Good models4. Good models of how these relateof how these relate
More than ECDMore than ECD (evidence(evidence--centered design) whichcentered design) whichhas focused on assessment designhas focused on assessment design
7Gong – CCSSO - Tests that Measure & Model - 6/28/12
Design lessonsDesign lessons –– 22
Good modelsGood models of assessmentof assessment
MeaningfulMeaningful –– content, reportingcontent, reporting
UsefulUseful –– clear, actionable, timely, in theory of actionclear, actionable, timely, in theory of action
CredibleCredible CredibleCredible
BoundedBounded –– limitations acknowledged, e.g., areas still beinglimitations acknowledged, e.g., areas still beingworked onworked on
Spanning credibility for traditionalSpanning credibility for traditional psychometrics and thispsychometrics and thisinstructional/learning contextinstructional/learning context
8Gong – CCSSO - Tests that Measure & Model - 6/28/12
Design lessonsDesign lessons -- 33
Good models of learningGood models of learning
Learning progressions (stages of conceptualLearning progressions (stages of conceptualdevelopment, or development of conceptualdevelopment, or development of conceptualrepresentations)representations)representations)representations)
LearningLearning process (e.g., how conceptualprocess (e.g., how conceptualrepresentation andrepresentation and expertise develop)expertise develop)
Type ofType of scaffoldedscaffolded experience (Ramp in, consolidate out)experience (Ramp in, consolidate out)
Role ofRole of feedback, selffeedback, self--assessment, reflectionassessment, reflection
Role of practice, generalizationRole of practice, generalization
Deal with errors or less developed representationsDeal with errors or less developed representations
Etc.Etc.9Gong – CCSSO - Tests that Measure & Model - 6/28/12
Design lessonsDesign lessons –– 33 Good models of curriculum and instructionGood models of curriculum and instruction
Organized experiences to help learner developOrganized experiences to help learner developknowledge/skillknowledge/skill representations/expertise from whererepresentations/expertise from wherelearner is to goal statelearner is to goal state
E.g., Learn with feedback, perform with feedback, reflectE.g., Learn with feedback, perform with feedback, reflect E.g., Learn with feedback, perform with feedback, reflectE.g., Learn with feedback, perform with feedback, reflect
Role of learnerRole of learner (and rest of system, e.g., teach,(and rest of system, e.g., teach, help functions)help functions)
Decision, implementation of quality learning/teachingDecision, implementation of quality learning/teachingexperiencesexperiences (e.g., model of coaching, uses of technology)(e.g., model of coaching, uses of technology)
Good models of how assessment, learning,Good models of how assessment, learning,instructions relateinstructions relate
10Gong – CCSSO - Tests that Measure & Model - 6/28/12
3. Process lessons3. Process lessons
1. “Theory building”1. “Theory building”
Have a theory,Have a theory, model, framework, hypothesismodel, framework, hypothesis
Develop and refineDevelop and refine
Systematically and purposefully gather and use evidenceSystematically and purposefully gather and use evidence
Use the theory and experience (things) to improve eachUse the theory and experience (things) to improve eachotherother
2. Have a committed team2. Have a committed team
3. Use a theory of action about development and3. Use a theory of action about development anduseuse
Be part of the communityBe part of the community11Gong – CCSSO - Tests that Measure & Model - 6/28/12
RecapRecap
Two very good examplesTwo very good examples
Comprehensive assessment systemsComprehensive assessment systems
Design lessonsDesign lessons
Process lessonsProcess lessons Process lessonsProcess lessons
12Gong – CCSSO - Tests that Measure & Model - 6/28/12
For more information:For more information:
Center for AssessmentCenter for Assessment
www.nciea.orgwww.nciea.org
Brian GongBrian Gong
[email protected]@nciea.org
13Gong – CCSSO - Tests that Measure & Model - 6/28/12