Facilitated by WISExplore Team Members:

The Data Disconnect Between Classroom Tasks & Standards

Leading for Learning Summit Breakout SessionJune 28, 2019

Jim Lee, CESA 12Lisa Arneson, CESA 3

1

Session Outcomes

● Understand how scores from misaligned tests can lead educators to make faulty inferences and predictions.

● Utilize new research that suggests educators should be concerned with alignment of instruction and assessment practices.

● Explore various data practices and tools that inform whether expectations, instruction, assignments, and learning align with grade-level standards.

● Plan an alignment investigation in their school.

2

Validity

3

Importance of Validity

Validity: The extent to which an assessment accurately measures what it is intended to measure.Content Validity: The tested content is aligned with what was taught. Content validity is increased when assessments require students to make use of as much of their classroom learning as possible.Predictive Validity: The extent to which a test score predicts performance on some criterion measure.

What are the consequences if test results are NOT VALID based on what we’ve taught?

4

Making Inferences

Valid measures give us the ability to make inferences from the data about what the student knows, can do or understands.“Based on these test scores, we can infer that the student

has mastered this content.”

“Validity is the degree to which all the accumulated evidence supports the intended interpretation of test scores for the proposed purpose” (AERA, 2008, p.11).

What happens when our inferences from test scores are incorrect?

5

Ensuring Validity

Test Content & Rigor

Adopted Curriculum &

Materials Content & Rigor

Standards Content & Rigor

Taught Curriculum

Content & Rigor

6

Example: Grade Level Misalignment

● All special ed students in 5th grade earned “Below Basic” on the Forward Exam

● We infer that they haven’t mastered tested content because the test is too hard for them. These students continue to be placed in remedial courses throughout their school career

● In reality, we find that students are pulled out of core classes and they are taught from reading and math materials that are at the 3rd grade level

● In this instance, the students are not exposed to grade-level instruction that is tested on the state test

Don’t go down a rabbit hole!

What can be done in this situation for

better alignment and student learning at

expected levels?7

See the Difference in ELA

CCSS.ELA-LITERACY.RI.3.8Describe the logical connection between particular sentences and paragraphs in a text (e.g., comparison, cause/effect, first/second/third in a sequence).

CCSS.ELA-LITERACY.RI.5.8Explain how an author uses reasons and evidence to support particular points in a text, identifying which reasons and evidence support which point(s).

What is a 5th grader taught that a third grader isn’t?Do these skills rely on a student’s ability to

read at the 5th grade level?

8

Faulty Inference from aTest ScoreTest Score:Dibels ORF (Oral Reading Fluency) number of correct words per minute: 16

https://dpi.wi.gov/sites/default/files/imce/standards/pdf/ela-stds-app-a-revision.pdf

Faulty Inference: Infer that the student has not mastered reading comprehension (informational text).

9

Grade 2 ELA Standards RI

Do No Harm

When your students sit down to take an assessment, will the task types be familiar

enough to boost their confidence?

If the test content and rigor are unfamiliar to the student -- what harm might it cause?

10

Research

11

Research:Students Not Taught at Grade-level

“Low-income students, English learners, students of color and those with disabilities are far less likely to be given the chance to do grade-level work.” (Educationdive.com article)

12

Opportunity Myth

In the nearly 1,000 lessons observed:

● Students were working on activities related to class 88 percent of the time.

● They met the demands of their assignments 71 percent of the time, and ● More than half brought home As and Bs. ● Yet students only demonstrated mastery of grade-level standards 17

percent of the time. ● That gap exists because so few of their assignments actually gave them

the chance to demonstrate grade-level mastery.

Also, regarding teacher expectations:

● 82% of teachers support their state's standards, only 44% say they expect that their students can reach those standards.

(TNTP: Opportunity Myth) 13

“Myth” Found Four Things Missing in Schools

● Grade-appropriate assignments● Strong instruction● Deep engagement, and● Teachers who hold high expectations.

14

Fordham Institute

While many students get good grades in the subject, far fewer

earn top grades on end-of-course algebra tests, and grade

inflation has grown worse over time at schools serving more

affluent students.

(Fordham Institute: Grade Inflation in High Schools: 2005-2016)

15

Starting with WISEdash

16

Forward Tested Topics Results by Grade Level

17

Tools and Practices

18

Process for Alignment ValidationWhat information do you have?● Grade level standards● Content Curriculum● State Assessment (content) blueprint & technical guides● Classroom assessments, item specifications● Materials and/or lesson plans with activitiesGuiding Questions:1. Does the curriculum include the same content as what is tested in

the assessment?2. Is there a difference between the adopted curriculum and what is

taught in the classroom?3. Does the curriculum align with the standards? How do you know?4. Are the learning expectations for students at the same level of rigor

as the standards? 19

Tools to Examine Test Specifications

Provides:● Test blueprint● Test Maps● Item types and descriptions● Number of items/points● Standards alignment● Depth of Knowledge (DOK)

20

● Forward Exam Tech Specs● Aspire Summative Technical Manual● ACT Alignment Report● ACT Technical Manual Supplement● ACCESS for ELLs Interpretive Guide

Use technical manuals for:

● Deep content and DOK alignment

Allow ample time for thorough study.

Test Blueprints

● English Language Arts● Mathematics● Science● Social Studies

21

Use blueprints for:

● Distribution of content

● DOK levels

● Number of points

Allow ample time for study.

Data Inquiry Using Test Specifications

Assessment Technical Guides can help you answer:● How can we understand how the assessments were

constructed?● What types of tasks are students asked to complete or

answer?● What depth of thinking is required in the assessment?● How well does our curriculum align with the assessment

items?● Which standards are assessed more frequently? Where are

these taught and assessed in our curriculum? Are these identified as “power standards?”

22

Tools to Examine Instructional Materials

● EdReports● Achieve the Core’s

○ Materials Alignment Quick Check (ELA)

○ EQuIP Suite of Tools

○ Instructional Materials Evaluation Tool (IMET)

● CCSSO, Achieve, and Student Achievement Partners have jointly developed a Toolkit for Evaluating the Alignment of Instructional and Assessment Materials to the Common Core State Standards

● DPI IMPL resources

23

Data Inquiry Using Materials Alignment Tools

Instructional materials tools can help you answer:● To what degree is our materials aligned to grade-level

standards?● To what degree is our materials aligned to the district

adopted curriculum?● Are the materials designed to meet the diverse needs of all

students? (i.e. SwD, English Learners, Gifted, etc.)

24

Tools to Examine Curriculum Alignment● Wisconsin State Standards● Learning Progressions

○ Mathematics■ Arizona: Progressions Documents for the Common Core Math Standards ■ NC State: Learning Trajectories for the K-8 Common Core Math

Standards○ ELA

■ Delaware: ELA Learning Progressions ○ Science

■ Sense Publishers: Learning Progressions in Science○ Social Studies

■ Lewis Central College: K-12 Learning Progressions in Social Studies● Performance Level Descriptors

○ ACT Performance Level Descriptors (ELA, Math, Science)

○ Forward Performance Level Descriptors (ELA, Math) Science, Social Studies

○ DLM Performance Level Descriptions

25

Data Inquiry Using Curriculum Alignment Tools

Curriculum alignment tools can help you answer:● How well does the adopted curriculum align with the district

adopted standards?● To what degree is our curriculum aligned with the content

learning progressions?● What evidence is there that we teaching and assessing all

students with grade-level standards? ● Are our interventions and programs based on appropriate

standards learning progressions?● What is the rigor of what is taught and assessed?● Are there gaps in instruction and assessment?

26

Depth of Knowledge

From ELA 8th grade Item Sampler, P. 3

27

Sample Activity: Assessment Blueprint

28Item Number

Response Type

Assessment blueprints are a valuable alignment tool. They can be used to plan an assessment or to adjust an assessment.

Discussion: Determining Proficiency

29

Layers of Alignment

30

Layer 1 - Simple Alignment

Layer 2 - Alignment with Standards Rigor

Layer 3 - Alignment with Local Curricula and Resources

Layer 4 - Alignment with Classroom Instruction and Assessment

Activity

31

Sample Activity

Compare Forward Test Specs to Local Curriculum, Instruction, and Assessment at one grade level1. Open Forward ELA Test Specs and choose a grade to study2. Examine the spreadsheet. What do you notice?3. Which standards are tested multiple times on the assessment?4. Are these same standards assessed with the same frequency in the

grades above and below?5. Where are those standards taught and assessed in your curriculum? What

is your evidence of student mastery?6. Examine the Depth of Knowledge column. 7. How many items are assessed at a level 3 DOK?8. Which standards are assessed at a higher DOK level? Which ones aren’t?9. Where do local assessments include items at the same level of rigor?

32

33

Next Steps

Contact your local CESA for support with these activities.

Each has a data specialist to help you!

34

There are many ways to continue your learning after the Summit, that also

connect to this session!

35

Mark your calendars!

Registration done through your local CESA

36

37

● Multi-day Academies● Reimbursement for school

and district leaders through Title II

● Individual and team-based opportunities

● Direct connection to 3 tenets

Wrap Up & Feedback Form

38

Please take a few minutes to complete this feedback form to help us get better.