Massachusetts Department of Elementary & Secondary Education Waiting Room Today’s webinar will begin shortly. Massachusetts Department of Elementary &

Massachusetts Department of Elementary & Secondary Education

Waiting Room

Today’s webinar will begin shortly.


REMINDERS: Dial 800-503-2899 and enter the passcode

6496612# to hear the audio portion of the presentation

Download today’s materials from the sign-in page: Webinar Series Part 7 PowerPoint slides Using Current Assessments in DDMs

Ramping Up for Next Year:Strategies for Using Current Assessments in DDMs

Webinar Series Part 7

DDM and Assessment Literacy Webinar Series# Title Date Length Time

1Introduction: District-Determined Measures

and Assessment Literacy3/14 60 minutes 4-5pm

2 Basics of Assessment 4/4 90 minutes4-

5:30pm 3 Assessment Options 4/25 60 minutes 4-5pm

TA and Networking Session I 7/11 3 hours9am-12pm

4Determining the Best Approach to District-

Determined Measures7/18 60 minutes 4-5pm

5Measuring Student Growth and Piloting

District-Determined Measures8/15 60 minutes 4-5pm

TA and Networking Session II 9/19 3 hours2:30pm-5:30pm

6Integrating Assessments into Educator

Evaluation: Developing Business Rules and Engaging Staff

10/24 60 minutes 4-5pm

7Ramping up for Next Year: Strategies

for Using Current Assessments in DDMs.

12/5 60 minutes 4-5pm

TA and Networking Session III 12/12 3 hours2:30pm-5:30pm

8 Communicating Results 1/23 60 minutes 4-5pm 9 Sustainability 2/27 60 minutes 4-5pm


4

Audience & Purpose Target audience

District teams that will be engaged in the work of identifying, selecting, and piloting District-Determined Measures.

After today participants will understand: Strategies for using current assessments as a

starting place for District Determined Measures

Educator Evaluation Data SY 2012-2013


6

Data Release – 11/21/13 2012-13 educator evaluation

performance rating data released for 213 RTTT districts 37,940 educators

Majority of Educators Received Solid Ratings 85.2% Proficient

Early evidence of differentiation is encouraging 7.4% Exemplary; 6.8% Needs Improvement;

and 0.7% Unsatisfactory

Ramping Up for DDM ImplementationIdentifying DDMs beyond the minimum pilot areas


8

Key Messages for Stakeholders


9

Ramping Up Goal by June 2014: To identify at least

two measures of student growth to match with each educator for use in the 2014-15 school year.


10

Ramping Up Strategies:

Collaborate with other districts (e.g. network at the upcoming TA sessions, work with area collaboratives)

Engage educators in the process of identifying DDMs

Use assessments currently in use in the district as a starting point for DDMs


11

Why use Current Assessments?

FeasibilityPotentially cost effectiveFamiliarity and experience with the assessments

They are Aligned to ContentAssess what is most important for students to learnAssess what the educators intend to teach

They are InformativeTeachers already use the assessments to measure

student learning Final Step

Provide valuable information to schools and districts about an educator’s impact


12

Considerations in DDMs Performance vs. Growth

Most current assessments are measures of performance and not growth

Growth takes into account the different levels of student achievement

Measures of growth should provide all students an equal opportunity to demonstrate growth

Unit vs. Year Most current assessments are designed to assess

short units DDMs that represent a full year provide more accurate

representations of student growth and of an educator’s impact


13

Potential Solutions Bring together multiple assessments of

similar content to measure performance at different points in time

Administer one assessment (or similar versions) at multiple points during the year

Combine multiple assessments of growth into a single DDM (e.g. pre-post tests of multiple units)


14

The larger DDM contextPreparing

Step Key Questions

1. Create a Team Who will be on the team? What are the key responsibilities?

2. Determine Content to Be Measured

What standards and objectives are the most important to measure?

3. Identify Measure What measures are already in use in the district? Do components need to be developed or modified before piloting?

4. Prepare to Implement Measure

What is the timeline? How will measure be scored? How will the data be collected and stored?

ImplementingStep Key Actions

5. Test Administer the measure Score the results Collect the data

6. Analyze Analyze student results Analyze the administration and scoring processes

7. Adjust Modify components as needed

8. Repeat Test, analyze, and adjust


15

New Guidance from ESE Using Current Assessments in District-

Determined Measures: Leveraging the Curriculum-Embedded Performance Assessments From the Model Curriculum Units

Available at http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf

15

http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf


16

Using Current Assessments: Three Steps

1. Identify key content.

2. Ensure that change in performance represents student growth.

3. Select an approach for measuring growth.


17






18

Step 1: Identify Key Content Key Content may include content

that is: A meaningful sample of the

learning Challenging to teach and learn. Under represented in assessment District priorities.


19

Example: 3rd Grade ELA Looking across the curriculum map to

identify a meaningful sample of content


20

Example: 3rd Grade ELAModel Curriculum Unit Name

Writing Standards Covered in Each MCU

1 2 3 4 5 6 7 8 9 10

Independent Readers X

Author Study X X X X

Biography X X X X

Extreme Weather X X

Newspaper X X X

Poetry X

Reading Inquiry X X

Stories Matter

Whose Story X X


21

Example: 3rd Grade ELA Standard 3 reads (W3.3) Write narratives

to develop real or imagined experiences or events using effective technique, descriptive details, and clear event sequences.


22

Example: 9th Grade US History

Are there commonalities in content covered across multiple units aligned to the MA Frameworks.


23

Example: 9th Grade US History

Model Curriculum Unit Name Standards

Constitutional Rights ELA9-10 W.1: Write arguments to support claims in an analysis of substantive topics or texts, using valid reasoning and relevant and sufficient evidence.

Economies of North & South RH 10.1: Cite specific textual evidence to support analysis of primary and secondary sources, attending to such features as date and origin of the information. WHST 10.9: Draw evidence from informational texts to support analysis, reflection and research

Industrial Revolutions W.H. 6-8.1 b: Support claims with logical reasoning and relevant, accurate data and evidence that demonstrate an understanding of the topic or text, using credible sources.

Westward Expansion HSS Writing.9-10-1: Introduce precise claim(s), distinguish the claim(s) from alternate or opposing claims, and create an organization that establishes clear relationships among the claim(s), counterclaims, reasons, and evidence.


24

Step 1: Identify Key Content Key content may be taught repeatedly

across the year

Key content may be taught once during the year


Step 1: Identify Key ContentQuestion for Engaging Educators

What can a student do after they take your class that they couldn’t do before?

25


26






Step 2: Does change represent growth?

What is Growth?

Educators understand growth. Growth is about improvement and

learning. There is a difference between growth

and how we measure growth. We can measure growth in both simple

and sophisticated ways. 27


28

Step 2: Does change represent growth? Are the assessments similar enough to

support meaningful inferences about student growth during the year? Do early assessments provide meaningful

information about what students do not understand?

Do future assessments provide meaningful information about what students have learned?

Do students have the opportunity to demonstrate different levels of growth?


29

Example: 3rd Grade ELA

In the third grade example Each unit has a CEPA covering the same

content Rubrics are different, hard to compare

Solution Identify similar evidence Modify rubrics by adding consistently worded

rubric items that assess the same content


30

Looking Across Assessments

Unit Name Rubric Items

1 2 3 4 5 6

Author Study Topic developme

nt and clarity

Evidence and

Content Accuracy

Organization

Standard English

Conventions

Variety of Sentences

Precise Use of

Vocabulary

BiographyOrganization: relevant,

well organized

and detailed

Opinion: Includes

persuasive details that support the

writer’s opinion

Illustrations: Includes engaging

illustrations on each page

Text Features: Includes

text features

Voice: Turns the events into a story

NewspaperThe Writing

Process Use of

Technology

Language and

Conventions

Interviewing and

Quotes

Descriptive Details

Organization

PoetryIncludes

description of topic of

poem

Includes senses or emotions evoked by

poem

Specific words or

phrases to support

response

Personal reflection

Whose StoryTopic

development

Evidence and

Content Accuracy

Illustrations Standard

English Conventions


31

Example: 3rd Grade ELA Conclusion:

The rubrics are similar enough to conclude that change in performance represents growth.

Advantages: Do not need to change the tasks students

will complete Rubrics are on a consistent scale Rubric do not need to be identical

Still include important rubric items that are present in only one CEPA (for example, use of technology in the Newspaper Unit)


32

Example: 9th Grade US History Constitutional Rights: Presentation where

students take on different roles Economies of North & South: Write a

journal, letter, … etc. Industrial Revolutions: Persuasive Writing Westward Expansion: Work in small

groups making exhibit


33

Example: 9th Grade US History Conclusion:

The performance tasks are too dissimilar to conclude that change in performance represents growth.

Next Steps: Identify a way to assess the key content

consistently. For example, administer a single assessment

of the key content at multiple times during the year.


34






35

Step 3: Select an approach Think about the scoring approach that

will best capture student learning given the type of measure you have chosen.

For example: Pre-Test/Post Test Repeated Measures Holistic Evaluation Post-Test Only

Learn more Webinar 5 http://www.doe.mass.edu/edeval/ddm/webinar.html Technical Guide B http://www.doe.mass.edu/edeval/ddm/TechnicalGuide-AppxB.pdf

http://www.doe.mass.edu/edeval/ddm/webinar.html

http://www.doe.mass.edu/edeval/ddm/TechnicalGuide-AppxB.pdf


36

Example: 3rd Grade ELA Each student has five time points. Classroom teacher rates all five time

points, which provides information to parents and teachers about growth across the year.

First and last assignments are rated by two additional teachers. The difference is used to create a growth score. (The sum of the four rubric items. They decide to set standards after the pilot year.)


37

Example: 9th Grade US History Each student completes four writing

assignments. The two US History teachers create a

holistic rubric that looks at four aspects of the writing.

The two US History teachers flag disagreements between ratings and meet to resolve these differences.


38

More Information Using Current Assessments in District

Determined Measures: Leveraging the Curriculum-Embedded Performance Assessments From the Model Curriculum Units

Available at http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf.



Closing


40

Attend Next Week’s TA Session Date: December 12, 2013 Time: 2:30pm – 5:30pm Locations: Southeastern MA: Norwell High

School, Little Theatre, 18 South Street, Norwell, MA 02061

Northeastern MA: Groton-Dunstable Regional High School, 703 Chicopee Row, Groton, MA 01450

Western MA: Frontier Regional School, Media Center, 113 North Main Street, South Deerfield, MA 01373

Register: http://www.surveygizmo.com/s3/1462056/Registration-DDM-Technical-Assistance-Networking-Session-3-December-12th-2013

http://www.surveygizmo.com/s3/1462056/Registration-DDM-Technical-Assistance-Networking-Session-3-December-12th-2013




41

Sharing Locally-Developed DDMs

We need your help! Please submit: Assessment directions, materials, items and

prompts Scoring resources (rubrics, answer keys) Other information (steps for administration,

lesson plans, examples of how results are used)

To submit your examples, complete the survey and upload your materials at https://app.smartsheet.com/b/form?EQBCT=52e3762ac7954f93aa40cc879bc38855.

https://app.smartsheet.com/b/form?EQBCT=52e3762ac7954f93aa40cc879bc38855




42

Register for Webinar Series Part 8

Part 8: Communicating Results Date: January 23, 2014

Time: 4-5pm EST (60 minutes)Register: https://air-event500.webex.com/air-event500/onstage/g.php?d=596783122&t=a

https://air-event500.webex.com/air-event500/onstage/g.php?d=596783122&t=a

https://air-event500.webex.com/air-event500/onstage/g.php?d=596783122&t=a


43

Questions Contact

Craig Waterman at [email protected]

Ron Noble at [email protected]

Tell us how we did: http://www.surveygizmo.com/s3/1469270/District-Determined-Measures-Assessment-Literacy-Webinar-7-Feedback

mailto:[email protected]

mailto:[email protected]

http://www.surveygizmo.com/s3/1469270/District-Determined-Measures-Assessment-Literacy-Webinar-7-Feedback



Documents

Massachusetts Department of Elementary & Secondary Education Waiting Room Today’s webinar will begin shortly. Massachusetts Department of Elementary &