Upload
emery-york
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
Robert DolanRobert DolanTracey HallTracey Hall
Developing Accessible Tests Developing Accessible Tests with Universal Design and with Universal Design and
Digital TechnologiesDigital Technologies
Restatement of the Problem:Restatement of the Problem:How to Standardize Assessment of How to Standardize Assessment of Nonstandard (i.e. Nonstandard (i.e. AllAll) Individuals?) Individuals?
• The challengeThe challengeEnsuring adequate individualization of test delivery Ensuring adequate individualization of test delivery without compromising comparabilitywithout compromising comparability
• Proposed solutions:Proposed solutions:– Application of universal design principlesApplication of universal design principles– Use of digital technologiesUse of digital technologies
Universal DesignUniversal Design
Originated in architecture as a means for Originated in architecture as a means for ensuring environments, and later products ensuring environments, and later products
and communications, be designed from and communications, be designed from the start to consider the needs of the the start to consider the needs of the
widest possible array of userswidest possible array of users(Ron Mace, 1996)(Ron Mace, 1996)
MatthewMatthew
Guggenheim MuseumGuggenheim Museum
Universal DesignUniversal Design
• NotNot one-size-fits-all, but rather one-size-fits-all, but rather provides users with provides users with alternativesalternatives..
• Designed in from the beginning, Designed in from the beginning, not added on later.not added on later.
• Increases access opportunities for Increases access opportunities for everyoneeveryone..
Universal Design for LearningUniversal Design for Learning
• Universal Design for Learning (UDL) Universal Design for Learning (UDL) extends universal design from a extends universal design from a physical space to a pedagogical space.physical space to a pedagogical space.
• New approach for teaching, learning, New approach for teaching, learning, and assessment that draws on new and assessment that draws on new brain research and new media brain research and new media technologies to respond to individual technologies to respond to individual learner differences.learner differences.
Universal Design for LearningUniversal Design for Learning
• UDL curricula, teaching practices, and UDL curricula, teaching practices, and policies are inherently flexible so as to policies are inherently flexible so as to support individual learning differences.support individual learning differences.
• Reduces the demand on educators to Reduces the demand on educators to develop and implement modifications develop and implement modifications and accommodations.and accommodations.
• Supports individual learner differences Supports individual learner differences through use of:through use of:– Multiple means of representation– Multiple means of expression– Multiple means of engagement
• Relies on scaffolds and accommodations Relies on scaffolds and accommodations to support students’ challenges and thus to support students’ challenges and thus improve access to learning.improve access to learning.
Universal Design for LearningUniversal Design for Learning
Multiple Means of RepresentationMultiple Means of RepresentationTo give learners various ways ofTo give learners various ways of
acquiring information and knowledgeacquiring information and knowledge
• Provide multiple examplesProvide multiple examples• Highlight critical features Highlight critical features • Provide multiple media and formats Provide multiple media and formats • Support background contextSupport background context
Multiple Means of ExpressionMultiple Means of ExpressionTo provide learners alternatives for To provide learners alternatives for
demonstrating what they knowdemonstrating what they know
• Provide flexible models of skilled performance Provide flexible models of skilled performance • Provide opportunities to practice with Provide opportunities to practice with
supports supports • Provide ongoing, relevant feedback Provide ongoing, relevant feedback • Offer flexible opportunities for demonstrating Offer flexible opportunities for demonstrating
skillskill
Multiple Means of EngagementMultiple Means of EngagementTo tap into learners' interests, challenge them To tap into learners' interests, challenge them
appropriately, and motivate them to learnappropriately, and motivate them to learn
• Offer choices of content and tools Offer choices of content and tools • Offer adjustable levels of challenge Offer adjustable levels of challenge • Offer choices of rewards Offer choices of rewards • Offer choices of learning contextOffer choices of learning context
Role of Digital TechnologyRole of Digital Technology
New media, such as digital text, New media, such as digital text, sound, images, and the Web,sound, images, and the Web,can be adjusted for different can be adjusted for different individuals and can reduce individuals and can reduce
construct-irrelevant barriers,construct-irrelevant barriers,thus opening doors to learning. thus opening doors to learning.
http://www.cast.org/teachingeverystudent/mlk
- Folktales Example -- Folktales Example -
Universal Design for Learning Universal Design for Learning and Assessmentand Assessment
Universal Design for Learning Universal Design for Learning principles apply globally across principles apply globally across
standards, instruction, materials, standards, instruction, materials, and assessments.and assessments.
FormativeEvaluation• Analyze Errors• Monitor Progress• Modify Instruction
Instructional Delivery• Access• Participation and Engagement • Monitor performance and pace • Immediate Feedback
Instructional Design• Determine Content• Instructional Strategies• Materials Selection and Sequence• Select examples• Provide for cumulative review
Initial Assessment• Archival Assessment• Formal Standardized Measures• Diagnostic Assessments
Multiple means of: • Representation• Expression• Engagement
UDL
UDL & Instructional / Assessment CycleUDL & Instructional / Assessment Cycle
Universal Design & AccommodationsUniversal Design & Accommodations
• Accommodations critical to reduce construct-Accommodations critical to reduce construct-irrelevant variance.irrelevant variance.
• Universal design requires that accommodations Universal design requires that accommodations be be considered from the startconsidered from the start to ensure to ensure effectiveness.effectiveness.
• Ideally accommodations should be seamlessly Ideally accommodations should be seamlessly integrated across integrated across instruction and assessmentinstruction and assessment..
The goal is to reduce accommodations The goal is to reduce accommodations asas retrofitsretrofits..
Universal Design and Universal Design and AssessmentAssessment
Current EffortsCurrent EffortsTwo complementary efforts:Two complementary efforts:
– Improving current state of testing through Improving current state of testing through application of original Mace principles.application of original Mace principles.
– Improving future of testing by increasing Improving future of testing by increasing flexibility and individualization usingflexibility and individualization usingdigital technologies.digital technologies.
UDA Directed ResearchUDA Directed Research
Compare and contrast two states, Compare and contrast two states, Kentucky and Maine,Kentucky and Maine,
with respect to universal designwith respect to universal designand technology-based testing.and technology-based testing.
Kentucky Online SystemKentucky Online System
Technology Skills ChecklistTechnology Skills Checklist
• Basic computer skillsBasic computer skills• Keyboarding skillsKeyboarding skills• Word processing skillsWord processing skills• Text-reader / screen-reader skillsText-reader / screen-reader skills• Online assessment skillsOnline assessment skills
Fleming, Kearns, Dethloff, Lewis & Dolan JSET 2006Fleming, Kearns, Dethloff, Lewis & Dolan JSET 2006
Research on Universal Design Research on Universal Design and Assessmentand Assessment
• Limited research to date.Limited research to date.• Accommodations research in Accommodations research in
general shows mixed results.general shows mixed results.• Technology-based testing Technology-based testing
research in general shows mixed research in general shows mixed results.results.
Keyboarding vs. HandwritingKeyboarding vs. Handwriting
• Study by Boston College Lynch School Study by Boston College Lynch School of Education (Michael Russell & Walt of Education (Michael Russell & Walt Haney) and Wellesley, MA, Public Haney) and Wellesley, MA, Public Schools (Tom Plati).Schools (Tom Plati).
• Significant improvement in MCAS Significant improvement in MCAS scores for typically-achieving 4th, 8th, scores for typically-achieving 4th, 8th, and 10th graders’ when using and 10th graders’ when using keyboarding for open-response keyboarding for open-response questions.questions.
Keyboarding vs. HandwritingKeyboarding vs. Handwriting
Study of Universally Designed Study of Universally Designed Technology-Based TestingTechnology-Based Testing
• Pilot study with high-school Pilot study with high-school students with learning disabilities.students with learning disabilities.
• Test of civics and U.S. history.Test of civics and U.S. history.• Paper-and-pencil vs. computer-Paper-and-pencil vs. computer-
based testing with optionalbased testing with optionaltext-to-speech read-aloudtext-to-speech read-aloud(CBT-TTS).(CBT-TTS).
Prototype CBT SystemPrototype CBT System
Effect of CBT-TTS on Test ScoresEffect of CBT-TTS on Test Scores
Dolan, Hall, et Dolan, Hall, et al. JTLA 2005al. JTLA 2005
Research Take Home PointsResearch Take Home Points
• Both technology and universal Both technology and universal design promising for reducing design promising for reducing construct-irrelevant variance.construct-irrelevant variance.
• Importance of matching Importance of matching instruction and assessment instruction and assessment techniques and technologies.techniques and technologies.
• Additional research needed!Additional research needed!
Future of Universal Design, Future of Universal Design, Technology, and AssessmentTechnology, and Assessment
• Providing students with choice Providing students with choice during testing.during testing.
• Developing richer, more authentic Developing richer, more authentic assessment situations.assessment situations.
• Progress monitoring.Progress monitoring.
Providing Students with Providing Students with Choice During TestingChoice During Testing
What is the effect of providing students What is the effect of providing students choice of equivalent questions /stimuli?choice of equivalent questions /stimuli?(i.e. (i.e. multiple means of representationmultiple means of representation))
Mei Ling gave the following description of a three-dimensional figure.
• The solid has 6 faces• The solid has 8 vertices.• The solid has 12 edges.
8. Which of the following figures matches Mei Ling’s description? Click on any figure to see it spin in 3D.
Developing Richer, MoreDeveloping Richer, MoreAuthentic AssessmentsAuthentic Assessments
Can universal design and Can universal design and technology be used to develop technology be used to develop
assessment items that reduce not assessment items that reduce not only physical and sensory barriers only physical and sensory barriers
but cognitive barriers as well?but cognitive barriers as well?
Progress MonitoringProgress Monitoring
Can technology improveCan technology improveformative assessments throughformative assessments through
real-time feedback and individualization real-time feedback and individualization of instruction?of instruction?
Commercial Tools available Commercial Tools available for Progress Monitoringfor Progress Monitoring
• AIMSWEBAIMSWEB• DIBELSDIBELS
Progress Monitoring Built in a Supported Writing Environment
Technology Limitations & ConcernsTechnology Limitations & Concerns
• Availability of technology and the Availability of technology and the infrastructure to support it.infrastructure to support it.
• Need to retain traditional administration Need to retain traditional administration modes for use under certain situations.modes for use under certain situations.
• Potential for inequities between students.Potential for inequities between students.• Comparability issues.Comparability issues.• Students’ ability to exercise choice wisely.Students’ ability to exercise choice wisely.
ConclusionConclusion
• Technology can help realize the Technology can help realize the flexibility necessary to accurately flexibility necessary to accurately and fairly assess diverse students.and fairly assess diverse students.
• The principles of Universal Design The principles of Universal Design for Learning can inform the for Learning can inform the appropriate use of technology in appropriate use of technology in assessment.assessment.