Introduction to Assessment of Student Learning … 1 Alternate Route to Certification Introduction to Assessment of Student Learning • Target Objective: Locate points on a number

11/23/2015

1

Alternate Route to Certification

Introduction to Assessmentof Student Learning

• Target Objective: Locate points on a number line through understanding of their value

• Task: Locate on the number line provided:

0 1

Opener

• Target Objective: Locate points on a number line through understanding of their value

• Task: Locate 1 on the number line provided:

0

Opener

11/23/2015

2

• Identify expectations for assessment of student learning as defined in the Common Core of Teaching (CCT)

• Recognize the difference between standardized tests and classroom assessments

• Recognize advantages and disadvantages of the major categories of assessments

• Recognize the characteristics of an effective test or quiz question

• Identify the characteristics of effective rubrics to evaluate performance tasks

• Recognize the format and content of state assessments (CMT & CAPT, SBAC)

Objectives

• Overview of Assessment• Types of Assessment• Break• Creating Effective Assessment Questions• Lunch• Discussion about Rubrics• State Testing• Break• Teacher Accountability• Framework for Effective Assessment Planning and

Design

Agenda

“What students learn depends as much on your tests as your teaching.”

WJ McKeachie

University of Michigan

11/23/2015

3

Pop Quiz

2. Explain how the results of the race would change if the race used a clock that rounded to the nearest tenth

1. For each equation, choose true or false.

Imagine you are a student on the first day of class and your teacher is addressing classroom norms and shares the following:

Although we all know you should use the bathroom during your break to maximize learning time, you will each receive five tokens that you can use throughout the semester whenever “nature calls” during class time. Once you’ve exhausted your five tokens, one point will be deducted from your participation grade for each bathroom visit. Because I believe in fairness, the converse will be true as well; for every token you have left at the end of the marking period, one point will be added to your participation average.

Discuss with a classmate your stance on the above policy.

Philosophy of Assessment

Adopted from M Dueck, Grading Smarter not Harder (2014).

Based on your experience as a student, what would you say is the purpose of assessment?

CSDE: Assessment for Learning

• Teachers use multiple measures to analyze student performance and to inform planning and instruction.

How does your experience compare to the purpose implied within the CCT expectations for assessment?

Philosophy of Assessment

11/23/2015

4

Assessment Grading

• What makes for “good” assessment practice?

Brainstorm Activity

• Phase I – Generation

– Focus on quantity, not quality

– Time based

– Limited or no discussion

– Piggybacking / opposing ideas are okay

• Phase II – Evaluation

– Group process

– Criterion based

Rules for Brainstorming

11/23/2015

5

12 Components of Good Classroom Assessment

• Determining the Assessment Task

– Engagement

– Alignment

– Relevance

• Communicating the Standards of Performance

– Transparent and Precise

• Assessing Prior Knowledge before Instruction

– To inform instructional practice

The Skillful Teacher (Saphier, et al, 2008)


• Frequent Data Collection

• Frequent High Quality Feedback

– Direct and Specific

– Used by students to improve work

• Student Self-Assessment

• Student Progress Monitoring



• Frequent Error Analysis by Teacher

– Identify patterns of error / misconceptions

– To inform instructional practice

• Error Analysis by Students

• Planning and Reteaching

• Student Goal Setting

• Reporting Systems


11/23/2015

6

• Reference pg 2 of the Assessing Student Learning Handout

• What to assess?

• How much to assess at one time?

• How often to assess?

Key Questions About Assessment

• Formative Assessment

• Interim Assessment

• Summative Assessment

Types of Assessment

• Assessment used to evaluate students’ knowledge and understanding of particular content; the results are used by the teacher to adjust and plan instruction to improve achievement in that particular area– Anything that informs instruction– Provides diagnostic information– Occurs prior to or during instruction– Typically short; quick feedback turnaround; not used

for grading purposes– Integral to teaching-learning process

Formative Assessment

11/23/2015

7

• Find 3 ways to represent the number 53 to demonstrate your understanding of place value

• Identify one property for bases and one for acids

• Looking at a set of letters on the board, recite them in random order

Formative Assessment

• Assessment used to evaluate students’ knowledge and understanding of particular content; the results are used by the teacher to adjust and plan instruction to improve achievement in that particular area

– More formal than formative assessments

– Often used for progress monitoring / predictive

– Can be analyzed for programmatic purposes

– May or may not be used for grading

– Positioned widely enough to alter instruction and produce measureable progress before next assessment

Interim Assessment

• Assessment used to document student achievement at the end of a unit or course, or to evaluate the end product of a learning activity or unit of study

– Occurs after material is taught

– Often used for grading purposes

– Can take many forms (paper and pencil test, paper, performance, etc.)

Summative Assessment

11/23/2015

8

• What are some different ways to assess?

• How do you assess using performances / projects?

• How do you reduce subjectivity in assessment?

Key Questions about Assessment

• Thinking critically and making judgments

(developing arguments, reflecting, evaluating, assessing, judging)

• Essay

• Report

• Journal

• Letter of Advice to .... (about policy, public health matters .....)

• Present a case for an interest group

• Prepare a committee briefing paper for a specific meeting

• Book review (or article) for a particular journal

• Write a newspaper article for a foreign newspaper

• Comment on an article's theoretical perspective

Variety within Assessment

• Solving problems and developing plans

(identifying problems, posing problems, analyzing data, designing experiments, applying information)

• Problem scenario• Group Work• Work-based problem• Prepare a committee of

inquiry report• Draft a research bid to a

realistic brief• Analyze a case• Conference paper (or

notes for a conference paper plus annotated bibliography)


11/23/2015

9

• Performing procedures and demonstrating techniques

(Computation, taking readings, using equipment, following laboratory procedures, following protocols, carrying out instructions)

• Demonstration• Role Play• Make a video (write script

and produce/make a video)• Produce a poster• Lab report• Prepare an illustrated

manual on using the equipment, for a particular audience

• Observation of real or simulated professional practice


• Managing and developing oneself

(Working co-operatively, working independently, learning independently, being self-directed, managing time, managing tasks, organizing)

• Journal

• Portfolio

• Learning Contract

• Group work


• Accessing and managing information

(Researching, investigating, interpreting, organizing information, reviewing and paraphrasing information, collecting data, searching and managing information sources, observing and interpreting)

• Annotated bibliography

• Project

• Dissertation

• Applied task

• Applied problem


11/23/2015

10

• A test that is administered, scored, and interpreted in a manner predetermined by the test producer

• Connecticut Mastery Test (CMT)

• Connecticut Academic Performance Test (CAPT)

• Smarter Balanced

• SAT, GRE, LSAT

• AP

• TOEFL

• Stanford-Binet

Standardized Assessment

• Aptitude

– Measure of cognitive ability

– Stanford Binet

– SAT I

– GRE

• Achievement

– Measure of knowledge or skills

– SAT II

– AP exams

– CMT & CAPT

– SBAC

– Classroom assessments

Types of Standardized Tests

• Pre-Assessment– Typically formative– Pre / Post mirrored or pre-requisite knowledge

• Formative Assessment– Process vs static

• Summative– Varying formats based on purpose

Classroom Assessments

11/23/2015

11

Characteristics of High Quality Assessments

Reliable

• CONSISTENT RESULTS ON MULTIPLE ADMINISTRATIONS OR WITH OTHER EVALUATORS

Valid• CLEARLY FOCUSED ON WHAT IT IS SUPPOSED TO MEASURE

Fair

• THE NATURE OF THE ASSESSMENT ITSELF DOES NOT PREVENT STUDENTS FROM ACHIEVING EXCELLENCE

Unbiased

• THE ASSESSMENT DOES NOT OFFEND OR PENALIZE ANY STUDENTS BECAUSE OF GENDER, ETHNICITY, RACE, SOCIOECONOMIC STATUS, OR RELIGION

Characteristics of High Quality Assessments

• The type and design of the assessment should be determined by the level of thinking you wish your student to demonstrate.

11/23/2015

12

Bloom’s Taxonomy

Bloom’s Taxonomy [1956 ] & Bloom’s Cognitive Process Dimensions [2001]

Knowledge -- Define, duplicate, label, list, name, order, recognize, relate, recall

Remember Retrieve knowledge from long-term memory, recognize, recall, locate, identify

Comprehension -- Classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, review, select, translate

Understand -- Construct meaning, clarify, paraphrase, represent, translate, illustrate, give examples, classify, categorize, summarize, generalize, predict…

Application -- Apply, choose, demonstrate, dramatize, employ, illustrate, interpret, practice, write

Apply -- Carry out or use a procedure in a given situation; carry out or use /apply to

an unfamiliar task

Analysis -- Analyze, appraise, explaincalculate, categorize, compare, criticize, discriminate, examine

Analyze -- Break into constituent parts,

determine how parts relate

Synthesis -- Rearrange, assemble, collect, compose, create, design, develop, formulate, manage, write

Evaluate -- Make judgments based on criteria, check, detect inconsistencies/fallacies, critique

Evaluation -- Appraise, argue, assess, choose, compare, defend, estimate, explain, judge, predict, rate, core, select, support, value

Create -- Put elements together to form a coherent whole, reorganize elements into new patterns/ structures

• DOK-1 – Recall & Reproduction - Recall of a fact, term, principle, concept, or perform a routine procedure

• DOK-2 - Basic Application of Skills/Concepts - Use of information, conceptual knowledge, select appropriate procedures for a task, two or more steps with decision points along the way, routine problems, organize/display data, interpret/use simple graphs

• DOK-3 - Strategic Thinking - Requires reasoning, developing a plan or sequence of steps to approach problem; requires some decision making and justification; abstract, complex, or non-routine; often more than one possible answer

• DOK-4 - Extended Thinking - An investigation or application to real world; requires time to research, problem solve, and process multiple conditions of the problem or task; non-routine manipulations, across disciplines/content areas/multiple sources

Webb’s Depth of Knowledge

11/23/2015

13

• While verbs may appear to point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level.

– Describe the process of measuring to the nearest unit

– Describe how two characters are alike and different; describe an observation you made about these materials

– Describe the evidence that supports your solution or conclusions, using words, data, diagrams, etc.

– Describe the evidence you found in 2 or more texts that shows different perspectives on this topic; describe the most significant effect of WWII on the nations of Europe

DoK is about complexity, not difficulty.

• Depth of Knowlege – Hess Matrix

Hess Cognitive Rigor Matrix

Depth + Thinking

Level 1Recall & Reproduction

Level 2Skills & Concepts

Level 3Strategic Thinking/ Reasoning

Level 4Extended Thinking

Remember - Recall, locate basic facts, details, events

Understand - Select appropriate words to use when intended meaning is clearly evident

- Specify, explain relationships- summarize– identify main ideas

- Explain, generalize, or connect ideas using supporting evidence (quote, example…)

- Explain how concepts or ideas specifically relate to other content domains or concepts

Apply - Use language structure (pre/suffix) or word relationships (synonym/antonym) to determine meaning

– Use context to identify meaning of word- Obtain and interpret information using text features

- Use concepts to solve non-routine problems

- Devise an approach among many alternatives to research a novel problem

Analyze - Identify whether information is contained in a graph, table, text feature, etc.

– Compare literary elements, terms, facts, events– analyze format, organization, & text structures

- Analyze or interpret author’s craft (literary devices, viewpoint, or potential bias) to critique a text

– Analyze multiple sources- Analyze complex/abstract themes

Evaluate – Cite evidence and develop a logical argument for conjectures

- Evaluate relevancy, accuracy, & completeness of information

Create - Brainstorm ideas about a topic

- Generate conjectures based on observations or prior knowledge

- Synthesize information within one source or text

- Synthesize information across multiple sources or texts

11/23/2015

14

• Consider the fairy tale, Little Red Riding Hood.

– Write a basic comprehension question to assess students’ understanding of the story.

– Consider where you would place your question on the Hess CRM.

– Write a more rigorous question (higher cognitive demand).

– Be prepared to share your examples.

Practice w/ Hess CRM

Depth + Thinking

Level 1Recall & Reproduction

Level 2Skills & Concepts

Level 3Strategic Thinking/ Reasoning

Level 4Extended Thinking

RememberWhat color was Red’s cape?Who is this storyabout?

UnderstandWho are the characters?What was the story’s setting?

Retell or summarize the story in your own words.

Apply

Identify words/phrases that helped you to know the sequence of events in the story.

AnalyzeIs this a realistic or fantasy story?

Compare the wolf character to the character of Red. How are they alike-different?

Is this a realistic or fantasy story? Justify your interpretation using text evidence.

Are all wolves (in literature) like the wolf in this story? Support your response using evidence from this and other texts.

EvaluateWhat is your opinion about the intelligence of the wolf? Justify your opinion using textevidence.

CreateWrite a telephone conversation between Red and her mother toexplain the wolf incident.

Take a 15 minute break!

11/23/2015

15

• Selected Response – Select correct answer from response provided

• Multiple Choice• True / False• Matching

• Constructed Response– Produce an answer in response to a question

• Fill in the blank• Short answer• Essay

Type of Assessment Prompts

• Performance Task

– Demonstrate new learning by completing a task• Performing or presenting

• Experiment or Demonstration

• Creating

• Portfolio Assessment

– A collection of samples of student work to demonstrate learning

Type of Assessment Prompts

• Refer to pg 6 of your handout

• Separate into groups of 4

• Within groups of 4, pair off and jigsaw the problems with one pair working evens and the other working odds

• You will have 15 minutes to work in pairs and 10 minutes to report out within your group of 4.

Activity – Creating Effective Assessment Questions

11/23/2015

16

Tips

Refer to pg 10 of your handout

Tips


11/23/2015

17

Tips


Grade 11 Key Details

Read this sentence from the passage.

• “Besides being beautiful to contemplate, space diamonds teach us important lessons about natural processes going on in the universe, and suggest new ways that diamonds can be created here on Earth.”

• Explain how information learned from space diamonds can help scientists make diamonds on Earth. Use evidence from the passage to support your answer.

ELA sample

Read this sentence from the opening of our presentation.

• What students learn depends as much on your tests as your teaching.

• Explain how information learned from this presentation exemplifies the sentiment of this statement. Provide examples from the presentation to support your thinking.

Closure

11/23/2015

18

• End Session I

– Break for lunch

• Preview of Slides for Session II

– Subject to change pending review and student feedback on Session I


It is impossible to determine teaching effectiveness without determining learning results.

No Child Left Behind

11/23/2015

19

A TOOL FOR ASSESSING THE QUALITY OF A STUDENT GENERATED PERFORMANCE OR PRODUCT CREATED IN RESPONSE TO AN ASSIGNED TASK

Rubrics fit within a conversation regarding assessment of content vs process

Rubrics

Rubric

ABOVE

EXPECTED LEVEL

AT

EXPECTED LEVEL

BELOW

EXPECTED LEVEL

EYE

CONTACT

Made contact with most of the audience

Made contact with some of the audience

Rarely or never looked up from notes

VOLUME

Could be heard by everyone and used volume for emphasis

Could be heard by everyone

Hard to hear

• Analytical

– Identify and assess components of a finished product

• Holistic

– Assess student work as a whole

• Scoring or Grading

– Used to delineate consistent criteria for grading

3 Types of Rubrics

11/23/2015

20

Sample Holistic Rubric

Activity

Specific Skill Highly Effective Adequate Inadequate

Language &Mechanics Precise, standard

words & sentences; no errors

Well organized with standard words &

sentences; only one or two non-distracting

errors

Hard to follow; awkward; non-standard English; many misspellings; too

many errors distract

ContentStrong Adequate Weak

Brevity One page that includes all the necessary

details and only the necessary details

One page with no more than two unneeded

details

More than two unneeded details and/or

runs longer than one page

ClarityEasy to follow Pretty easy to follow Confusing

FormatStandard memo format Standard memo format

Nonstandard memo format

NeatnessNeat and word-processed

Neat but handwritten Messy

Class Requirements

On time with proper heading

Late and/or improper heading

ClassParticipation

Actively participated in the class discussion on writing effective memos

Moderately involved in the class discussion on writing effective memos

Not involved in the class discussion on writing effective memos

Identify two strengths and two weaknesses of the sample analytical rubric

• Clear, precise and easy to understand

• Observable and measurable criteria

• Focused on key learning

• Reasonable expectations

• Singularity of criterion

• Relevant criteria

Effective Rubrics

11/23/2015

21

• http://rubistar.4teachers.org/

Rubric samples

CAPT Science Holistic Rubric

Analytic Rubric

11/23/2015

22

Music Sample

History Teaching Portfolio Rubric

Integrating Points into Analytic Rubric

Criteria Below Level 1 Level 1 Level 2 Level 3 Level 4 Total

Know l edge

Using historicallyaccurateinformation

Use of historical information not passable

Limited use of accurate historical information

Accurate for the most part; some important information is not accurate

All information used is historically accurate; most is relevant to topic

All informa-tion used is historically accurate and relevant

0 2 4 5 5.5 6 6.5 7 7.5 8 9 10

Observe how point values are assigned across performance levels.

11/23/2015

23

Grading Rubric Sample

• Rubrics design offers significant flexibility.

• Teachers should be prepared to revise their rubrics multiple times to insure the tool measures the characteristics they seek to assess.

• Rubrics are most effective when shared with students in advance and offered with annotated student work samples.

• Rubrics can be used in a formative sense to provide feedback and guidance for revision.

Final Word


11/23/2015

24

• Overview of the CMT and CAPT

• Achievement Gap and NCLB

• Smarter Balanced Assessment

• CSDE Performance Index

State Assessment

• First introduced in 1985

• Phase Out Math, Reading, and Writing by 2014

• Science added as a test in grades 5 and 8 in 2008; will be retained despite advent of SBAC

• CMT originally developed as a minimum proficiency examination

• CAPT originally developed as a catalyst to shape instructional change

Overview of CMT and CAPT

2013 Math Science Reading Writing

CMT gr5 Goal 69.4% 62.5% 66.9% 65.6%

SWD 31.1% 22.4% 29.2% 21.4%

Non-SWD 73.0% 68.1% 70.0% 71.7%

CAPT gr 10 Prof

78.6% 81.7% 81.0% 88.9%

BLA or AA 51.3% 57.7% 57.9% 77.3%

HIS 56.1% 60.3% 61.9% 77.1%

WHI 89.4% 92.1% 90.1% 94.2%

2013 State Assessment Data

11/23/2015

25

ESEA Waiver

• NCLB Targets

• Waiver provided

– Release from NCLB targets

– Release from NCLB sanctions

• Waiver required establishment of

– Teacher Development and Evaluation System tied to Student Outcomes

– Student Accountability System ground in Mastery

Smarter Balanced

September 28, 2012

Summative assessment in grades 3-8 and 11During last 12 weeks of school year

Computer adaptive testingSelected response, short constructed response, extended constructed response, technology enhanced, and performance tasks

Performance tasksReadingWritingMath

Nationally established standard setting

SBAC highlights

11/23/2015

26

Computer Adaptive Assessment and Performance Tasks

INTERIM ASSESSMENT

SBAC Assessment System

Optional Interim assessment system —no stakes

Summative assessment for accountability

Last 12 weeks of year*

DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model curriculum units; educator training; professional development tools and resources; an interactive reporting system; scorer training modules; and teacher collaboration tools.

Scope, sequence, number, and timing of interim assessments locally determined

PERFORMANCETASKS

• Reading• Writing• Math

COMPUTERADAPTIVE

ASSESSMENT

Re‐take option

* Time windows may be adjusted based on results from the research agenda and final implementation decisions.

English Language Arts and Mathematics, Grades 3 – 8 and High School

Computer Adaptive Assessment and Performance Tasks

INTERIM ASSESSMENT

Claims Assessment Targets

Task Models Evidence

Evidenced-Centered Assessment Design

SBAC Accommodations Concept

11/23/2015

27

Navigation Toolbar

Technology Enhanced Items

• Presentation

– Animations

– Simulations

– On-line access to information

– Video or Audio stimulus

– Moveable models

Innovative Item Types


• Student Response

– Selecting one or more points on a graphic

– Dragging or dropping a graphic from one location to another

– Manipulating a graph


11/23/2015

28


• Cognitive Rigor

– Used to assess content, cognitive complexity and depth of knowledge not accessible otherwise


• SBAC has provided a training test that gives students and teachers an opportunity to become familiar with the format and software that will be used in the new assessment platform.

Smarter Balanced Training Test

Classroom Activity

ELA Gr 6

11/23/2015

29

Sample ELA – selected response

• Read this sentence from the passage. Why does the author use this sentence to describe what Coyote feels?– To explain how…– To describe how…– To compare the effect of …– To contrast the effect of…

• Based on Coyote’s actions to get the honey, which sentence best matches his personality?– …happy but silly– …lonely but angry– …hungry but slow– …clever but foolish

Sample ELA – constructed response

• Coyote’s plan to get the honey fails. Support this statement with two details from the passage. (2 point response)

Sample ELA – performance task

• Now that you have read the sources, you will take a position and present your findings at your school's yearly mock Congressional Session.

– For your presentation, analyze the arguments and make a claim as to whether or not the penny should be preserved.

– Make sure you address potential counterarguments in your essay and support your claim with information from the sources you have examined.

11/23/2015

30

ELA Rubrics

Sample Math – selected response

• Which of the following expressions have a sum equal to ?

a.

b.

c.

d.

e.

Sample Math – selected response

11/23/2015

31

Sample Math – constructed response

• Drag one number into each box to complete the subtraction problem shown. (1 point response)


• Kayla asked 10 students in her class whether they owned a dog, a cat, or both.

– 40% own a dog

– 30% own a cat

– 10% own both


• Jack, Max, and Annie are hiking on the same trial. – Jack is ½ finished – Max is 1/3 finished – Annie is 5/8

finished

• Drag each person to his or her location on the trial.

11/23/2015

32

• Javier says that all odd numbers greater than 2 and less than 20 are prime.

• Find an odd number greater than 2 and less than 20 that is not prime. Explain why the number is not prime. (1 point response)


Sample Math – technology enhanced item

• The school is 100 meters from Jason’s house. – He walked 50 m

toward school in 2 minutes.

– He turned around and walked home at the same speed.

– He spent a minute at home looking for a forgotten book.

– He walked all the way to school at twice his original speed.

• Use the Line tool to finish a graph that accurately displays Jason’s trip.

Sample Math – performance task

• Anna and her family go to the zoo. The zoo ticket prices, snack shop menu, and gift store prices are in the tables.

11/23/2015

33

English Language Arts

• Reading– Literary Text– Informational Text

• Writing– Organization– Providing Evidence– Conventions

• Oral Language– Listening – Speaking (not necessarily each

year)

• Inquiry / Research

• Language Usage

Mathematics

• Concepts and Procedures

• Problem Solving, Modeling, & Data Analysis

• Communicating Reasoning

Proposed Score Reporting Categories

Considerations

• Technology Infrastructure

• Policy

• Curriculum and Technology Integration

• Technology Literacy

• Professional Development

Technology Literacy Skills

• Point, navigate and click (rightand left click)

• Screen Navigation: scroll bars, arrows

• Keyboard commands

• Radio buttons for answering

• Position arrows: mathematics questions

• Highlighting tools

• Text Expansion

• Click, drag and drop

• Numerical entry – keyboard on-screen

• Typing / Keyboarding

Implications for Instruction

State Assessment and Teacher Evaluation

11/23/2015

34

• Accountability expectations

• SRBI

• Universal Design Principles

– Representation

– Action & Expression

– Engagement

Challenges

• Design an assessment for your content discipline

– Work within a small group

– Apply what you have learned

• Alternative: visit NAEP at http://nces.ed.gov/nationsreportcard/about/naeptools.asp

Activity

Questions

11/23/2015

35

• CT State Dept of Ed Student Assessmenthttp://www.sde.ct.gov/sde/cwp/view.asp?a=2748&Q=334726

• Smarter Balanced Assessment Consortiumhttp://www.smarterbalanced.org/

• Assessment for Learning – Rick Stigginshttp://www.murrieta.k12.ca.us/cms/lib5/CA01000508/Centricity/Domain/2241/Assessment%20for%20the%20learning%20gap%20in%20effective%20schools.pdf

Resources

• National Assessment of Educational Progresshttp://nces.ed.gov/nationsreportcard/

• AP Central – The College Boardhttp://apcentral.collegeboard.com/apc/Controller.jpf

• Program for International Student Assessmenthttp://nces.ed.gov/surveys/pisa/

• Foreign Language Assessment Directoryhttp://www.cal.org/calwebdb/flad/

Resources

Documents

Introduction to Assessment of Student Learning … 1 Alternate Route to Certification Introduction to Assessment of Student Learning • Target Objective: Locate points on a number