Aligning Instruction to Standards and Assessments for English Language

Learners Meeting # 3 Aug 26-27 2008

U.S. Department of Education, Title I, Enhanced Assessment Grant

Iowa SEC-ELL State Consortium

SEC-ELL Consortium Iowa, Florida, Idaho, Maine, Minnesota, Ohio,

Utah, Virginia, Wisconsin

CCSSO WCER

WestEd

PD consultantsExpert Advisers

Study Questions1. To what extent do English language learners have

opportunity to learn the subject content specified in state academic standards and assessed on state assessments?

2. What is the relationship of academic English language development standards and assessments to state academic standards?

3. What instructional practices and strategies are used to teach academic English language skills across different subject areas or courses?

4. What is the relationship between the extent of alignment of instruction to standards and student achievement?

Study Goals/Methods

Overall Purpose: Improve academic achievement of English language learner students

Use Data in Ed improvement: Use SEC data tool/methods—augmented with linguistic analysis--to measure and report the degree of alignment between instruction, state standards, and student assessments for English language learners in a sample of classrooms and schools in the Consortium states.

R&D ELL Assessment/standards: Advance and improve the SEC alignment procedures and tool to increase capacity for applying the method to English language development practices and materials.

Technical assistance/Training (PD)

State level: Provide expert technical assistance for Consortium States to apply alignment analysis results to improve the quality and validity of assessment instruments for ELL students.

Technical assistance/Training (PD)

Local level: Demonstrate/lead PD and TA with study districts and school leaders toward using data for improving practices for ELLs and other students.

Evaluate: Work with states to analyze growth in student achievement in relation to the degree of alignment of classroom instruction to standards (i.e., differences in “opportunity to learn”).

Or, Analyze change in practices /alignment

Address State Questions and Needs –improving ELL education

e.g. Differences in practices, Alignment of content/language, Variation by ELL programs and by district/school, Teacher prep/conditions

Project Outcomes: Disseminate products and findings to education leaders across states.

Implement tools/processes with other states

SEC ELL ProjectMeeting #3 Objectives

Initial Data Analysis / Leader Training

Describe

Interpret

Apply

State project development Workshops, training, schedule

Identify findings, issues, questions What have we learned

Plan next steps, calendar

Project TimelineProject Planning / State questions Dec-Jan 08

Survey / Method development Feb – Mar

Alignment coding/ Data collection Apr – June

Initial analysis – year 1 Jul – Aug

*********

State & within state workshops/plans Sept – Nov

Technical assistance Jan – Feb 09

Evaluate progress / results March 09

[Extend project ]

Project Timeline– 2 Year 2 Data collection Apr – May 09

Analysis of Follow-up or New data June – July

Report study findings and recommendations Aug – Sept

State workshops– Disseminate, Implement Fall 09

Provide training

Evaluate results

SEC-ELL Study Design: 2008-10

(2) How to Measure Language Demand

ELD Standards,Assessments,

Classroom Instr.Var. ELL practices

5) What are the R & D results? Formative, Implementation

Validate surveys, codingChange in Practices (yr 2)

Expand use

(1) DevelopSEC–ELL Tool

SEC model:Practices + ContentOnline data system State ELL needsStudy Questions

(3) Alignment analysis/coding by Grade level ELD + Subject Stands, Assess.

Training 4 person teams

Are the tools and model working?

(4) Collect/ Analyze Databy State-- districts, schools

Facilitator trainingWeb-based tool

ELD teacherContent teacher w/ELL

How to report and use data? PD model?

How to measure alignment? S x A x I

SEC Rationale

AssessmentStandards

Curriculum

Key Question -- SEC ToolsResearch into Practice

• How can Educators obtain reliable, valid data to determine Alignment of instruction with required standards and assessments?

Questions Addressed by SEC Data

• How can in-depth data on content of instruction be collected and reported in a reliable, comparable survey tool to analyze teaching content in relation to standards, assessments, achievement?

• How can methods of teaching practices be compared across classrooms, schools, districts, and states?

• How can enacted curriculum data be reported in a manner to encourage use by teachers to improve instruction?

Applications

• Alignment analysis --instruction, standards, assessments

• Instructional improvement in schools• Needs assessment/ Evaluation• Indicators – monitoring change over time

• School & Class Description• Instructional Activities

• General• Problem Solving Activities

• Pairs & Small Group Work• Use of Hands-on Materials• Use of Calculators/Computers & other Ed. Tech.

• Assessment Use• Instructional Influences

• Instructional Readiness• Teacher Opinions• Professional Development

• Types, Frequency• Content , Active,• Collegial, Coherence

• Instructional Content•Topic x Cog. Demand

Survey Sections

English LanguageArts & Reading

The SEC Data-set

SUMMARY MEASURES

Content Practice Climate Prof. Dev./ / /

Science

InstructionalContent

Instr.Activities

Tchr.Char./Opinions/Beliefs

Professional DevelopmentMathematics

Social Studies

Uses a multi-dimensional language fordescribing instructional content

SEC Instructional Content

Topics

byCognitive Demand

(Expectations for Student Learning)

MemorizePerform

CommunicateConjecture

Connect0

0.020.040.060.08

0.1

Operations

Number Sense

Measurement

Geometric Concepts

Algebraic Concepts

Data Analysis

Instructional Technology

MemorizePerform

CommunicateConjecture

Connect

State J Grade 8Mathematics Instruction

Content Maps

Steps in SEC Development

• Models: TIMSS, NAEP, Analyze assessment• OTL interest of States, Porter/ Smithson

research, Studies of curriculum-- 1990s• CCSSO Science Assessment project• Develop Math/Science surveys-’98-’01 (NSF)• Field study, Reports format, Alignment method • SEC Collaborative – 12 to 18 state members• MSP-NSF Longitudinal Eval. of Teacher PD• Data PD study – 5 urban districts, 40 mid schs ’02-’06

Cont. -- Steps in SEC • 2003-04 Online data collection and reporting

www.SEConline.org

10,000 teacher surveys / year

100 + content alignment documents / year• ELAR survey/framework ’02-’04• Social studies survey/framework • 3rd Annual summer conf “Using Data” • Iowa/9 state SEC ELL project 07-09

SEC Collaborating Organizations

•Council of Chief State School Officers www.SECsurvey.org

•Wisconsin Center for Education Research www.SEConline.org

•Learning Point Associates/NCREL www.SECsupport.org

•TERC Regional Alliance DEC Project www.ra.terc.edu/DEC

•Measured Progress (assessment development and PD)

SEC Collaborative offers

• Core Components of SEC tools/services1) Surveys with teachers – report instruction

2) Alignment analysis – content code standards and assessments

3) How to use Data – knowledge, skills

4) Leader development – 2-3 meetings/yr

4) In-state workshops: a) orientation

b) use of data

SEC Surveys and Alignment:2006-07

20+ States 10,393 SEC Surveys 06-07

4674 Math

1993 Science

3646 ELAR

80 Soc Stud

131 Alignment content analysis Standards or Assessments

1000~ Schools Data use

The intended curriculum: State content standards—What students should learn

The enacted curriculum: What teachers teach

The learned curriculum: Student outcomes based on school learning

The assessed curriculum: State (and other) assessments—tested learning

Analyzing Alignment

Linquistic Complexity by Language Demands

A neutral linguistic taxonomy

A neutral content language

Topics by Cognitive Demand

K-12 Mathematics Content

K-12 Mathematics ContentBasic Algebra

The Goal

To render quantitative descriptions of instruction, standards, and assessments using a common language in order to facilitate comparisons and analyses of the three domains of a standards-based approach to education reform and their relationship to one another.

To Facilitate Comparisons

Academic Content

ELL Lang. Acquisition

Academic Content

ELL Lang. Acquisition

Academic Content

ELL Lang. Acquisition

The Two Dimensions Of Content

What students should know[Topics]

And…

Be Able to Do [Expectations for student performance]

Describing Instructional Content

SEC utilizes a two-dimensional taxonomy based on:Topic

by Cognitive Demand

1

2

3

4

5

B C D E F

Coarse Categories of Cognitive Demand

Content Areas

Memorize Perform Procedures

Demonstrate Understanding

Conjecture, Hypothesize

Non-routine problems

Number Sense

Operations

Measure-ment

Basic Algebra

Advanced Algebra

Geometric Concepts

The Content Matrix

Fine Categories of Cognitive Demand

Topics Memorize Perform

Procedures Demonstrate

Understanding Conjecture, Hypothesize

Non-routine problems

Place Value

Whole Nbrs.

Fractions

Decimals

Percents

Ratio & Proportion

The Content Matrix

… adding levels of relative emphasis yields a 3-D construct

Coarse Categories of Cognitive Demand

Content Areas

Memorize Perform Procedures

Demonstrate Understanding

Conjecture, Hypothesize

Non-routine problems

Number Sense

Operations

Measure-ment

Basic Algebra

Advanced Algebra

Geometric Concepts