Published on
13-Feb-2017
View
214
Download
2
Embed Size (px)
Transcript
January 7, 2015
Smarter Balanced Assessment Consortium:
Achievement Level Setting Final Report
ALS Final Report
1
Contents Executive Summary ................................................................................................................................ 5
Overview of Tasks and Processes ........................................................................................................... 5 Online Panel Results ................................................................................................................................ 5 In-Person Workshop Results ................................................................................................................... 7 Results of Cross-Grade Review ............................................................................................................ 10 Impact by Subgroup .............................................................................................................................. 12 Action by Chiefs ..................................................................................................................................... 16
Section 1. Introduction ........................................................................................................................ 18 Smarter Balanced Background ............................................................................................................ 18 Purpose and Organization of this Report ............................................................................................ 19
Section 2. Communications Plan ........................................................................................................ 20 Purpose and goals ................................................................................................................................ 20 Targets ................................................................................................................................................... 20 Research and Planning ......................................................................................................................... 21
Environmental scan. ......................................................................................................................... 22 Focus group study. ............................................................................................................................ 23
Recruitment and Outreach ................................................................................................................... 23 Social media campaign/paid promotion. ........................................................................................ 24
Media Strategy/Communications Planning ........................................................................................ 24 Communications planning for key activities and state approval. .................................................. 24
Contingency planning ........................................................................................................................... 25 Intake interviews: Development of contingency plan, messaging and protocols. ........................ 25 Ebola contingency response. ........................................................................................................... 26
Digital and Creative Support ................................................................................................................ 26 Expansion of video production scope. ............................................................................................. 27
Achievement Level Naming Research ................................................................................................. 27 Interviews with assessment experts. ............................................................................................... 28 Focus groups among teachers. ........................................................................................................ 29 Online surveys among parents and students. ................................................................................. 29
Section 3. Logistics ............................................................................................................................. 31 Schedule ................................................................................................................................................ 31 Facilities ................................................................................................................................................. 32 Hardware and Software Support.......................................................................................................... 33 Meetings ................................................................................................................................................ 33
ALS Final Report
2
In-person workshop........................................................................................................................... 33 Cross-grade review. ........................................................................................................................... 36
Technical Advisory Committee Meetings ............................................................................................. 36 Collaboration Conferences ................................................................................................................... 36 Advisory Panel Meetings ....................................................................................................................... 37 National Conference on Student Assessment .................................................................................... 37 Other Meetings ...................................................................................................................................... 37 Security .................................................................................................................................................. 38
Section 4. Audit ................................................................................................................................... 40 Purpose of the Audits ........................................................................................................................... 40 Selection of the Auditors ...................................................................................................................... 40 Major Audit Functions ........................................................................................................................... 41
Review of tools and materials. ......................................................................................................... 41 On-site review of procedures. ........................................................................................................... 41 Review of documentation. ................................................................................................................ 42
Legal review ........................................................................................................................................... 42 Section 5. Panelist Recruitment .......................................................................................................... 43
Online Panel for Achievement Level Setting ....................................................................................... 43 Implementation. ................................................................................................................................ 43
Findings and Refinements ................................................................................................................... 46 In-Person Panel for Achievement Level Setting .................................................................................. 46
Implementation. ................................................................................................................................ 47 Selection Process .................................................................................................................................. 48 Findings and Refinements ................................................................................................................... 50 Cross-Grade Review Panel ................................................................................................................... 54
Implementation. ................................................................................................................................ 54 Findings and Refinements ................................................................................................................... 55
Section 6. Standard Setting Design and Implementation ................................................................... 57 The Bookmark Procedure ..................................................................................................................... 57
Creating ordered item booklets ....................................................................................................... 57 Item mapping. ................................................................................................................................... 59 Typical application of the bookmark procedure. ............................................................................. 59
Design and Implementation of the Online Panel ................................................................................ 60 Software development. ...................................
