Smarter Balanced Member Portal - Smarter Balanced Assessment

Embed Size (px)

Text of Smarter Balanced Member Portal - Smarter Balanced Assessment

  • January 7, 2015

    Smarter Balanced Assessment Consortium:

    Achievement Level Setting Final Report

  • ALS Final Report

    1

    Contents Executive Summary ................................................................................................................................ 5

    Overview of Tasks and Processes ........................................................................................................... 5 Online Panel Results ................................................................................................................................ 5 In-Person Workshop Results ................................................................................................................... 7 Results of Cross-Grade Review ............................................................................................................ 10 Impact by Subgroup .............................................................................................................................. 12 Action by Chiefs ..................................................................................................................................... 16

    Section 1. Introduction ........................................................................................................................ 18 Smarter Balanced Background ............................................................................................................ 18 Purpose and Organization of this Report ............................................................................................ 19

    Section 2. Communications Plan ........................................................................................................ 20 Purpose and goals ................................................................................................................................ 20 Targets ................................................................................................................................................... 20 Research and Planning ......................................................................................................................... 21

    Environmental scan. ......................................................................................................................... 22 Focus group study. ............................................................................................................................ 23

    Recruitment and Outreach ................................................................................................................... 23 Social media campaign/paid promotion. ........................................................................................ 24

    Media Strategy/Communications Planning ........................................................................................ 24 Communications planning for key activities and state approval. .................................................. 24

    Contingency planning ........................................................................................................................... 25 Intake interviews: Development of contingency plan, messaging and protocols. ........................ 25 Ebola contingency response. ........................................................................................................... 26

    Digital and Creative Support ................................................................................................................ 26 Expansion of video production scope. ............................................................................................. 27

    Achievement Level Naming Research ................................................................................................. 27 Interviews with assessment experts. ............................................................................................... 28 Focus groups among teachers. ........................................................................................................ 29 Online surveys among parents and students. ................................................................................. 29

    Section 3. Logistics ............................................................................................................................. 31 Schedule ................................................................................................................................................ 31 Facilities ................................................................................................................................................. 32 Hardware and Software Support.......................................................................................................... 33 Meetings ................................................................................................................................................ 33

  • ALS Final Report

    2

    In-person workshop........................................................................................................................... 33 Cross-grade review. ........................................................................................................................... 36

    Technical Advisory Committee Meetings ............................................................................................. 36 Collaboration Conferences ................................................................................................................... 36 Advisory Panel Meetings ....................................................................................................................... 37 National Conference on Student Assessment .................................................................................... 37 Other Meetings ...................................................................................................................................... 37 Security .................................................................................................................................................. 38

    Section 4. Audit ................................................................................................................................... 40 Purpose of the Audits ........................................................................................................................... 40 Selection of the Auditors ...................................................................................................................... 40 Major Audit Functions ........................................................................................................................... 41

    Review of tools and materials. ......................................................................................................... 41 On-site review of procedures. ........................................................................................................... 41 Review of documentation. ................................................................................................................ 42

    Legal review ........................................................................................................................................... 42 Section 5. Panelist Recruitment .......................................................................................................... 43

    Online Panel for Achievement Level Setting ....................................................................................... 43 Implementation. ................................................................................................................................ 43

    Findings and Refinements ................................................................................................................... 46 In-Person Panel for Achievement Level Setting .................................................................................. 46

    Implementation. ................................................................................................................................ 47 Selection Process .................................................................................................................................. 48 Findings and Refinements ................................................................................................................... 50 Cross-Grade Review Panel ................................................................................................................... 54

    Implementation. ................................................................................................................................ 54 Findings and Refinements ................................................................................................................... 55

    Section 6. Standard Setting Design and Implementation ................................................................... 57 The Bookmark Procedure ..................................................................................................................... 57

    Creating ordered item booklets ....................................................................................................... 57 Item mapping. ................................................................................................................................... 59 Typical application of the bookmark procedure. ............................................................................. 59

    Design and Implementation of the Online Panel ................................................................................ 60 Software development. ...................................