Embed Size (px)
Citation preview
North CarolinaOnline Assessments
Northwest Regional MeetingNorth Wilkesboro, NC
October 12, 2007Presenter: Scott Ragsdale
NC State University, CUACS/TOPS
Purpose
This presentation will cover information on online assessments currently implemented in North Carolina. Topics will include:
North Carolina Online Test of Computer Skills
North Carolina EOC Online Assessments
Why Online?
Lifting the Veil of Mystery Assessing students online is no longer shrouded in
mystery or a perilous journey across a barren desert with no water
Successful implementation of large-scale online assessment programs is not uncommon…in fact, it is the norm
Fundamental advantages over traditional assessment formats
The student viewpoint versus the adult viewpoint…who is really afraid of the Big Bad Wolf?
The Future is Today…Not Tomorrow We live, work, and LEARN in a 21st Century world…why
should we not assess what we learn using 21st Century technology?
Innovative assessments delivered in an innovative format for innovative students in a global commonwealth that demands innovation
Infinite advantages for infinite possibilities
21st Century SkillsPartnership for 21st Century Skills http://www.21stcenturyskills.org/ Vision for 21st Century education Business-Education-Policymakers Information and Communication Technology [ICT]
Literacy Creation of assessments that reveal students applied
cognitive skills in conjunction with using technology Use technology to deliver higher-order assessments
Why NOT Online?
A Thought…
“Once a new technology rolls over you, if you’re not part of the steamroller, you’re part of the road.”
- Steward Brand
North CarolinaOnline Test of Computer Skills
Notable Data from Fall 2006 205 students tested in one day at one school [2005-06: 139] 52 schools tested 100 or more students in one day [2005-06: 7] 5,068 maximum testing sessions used in a single day [2005-06:
3,063] 23 days of testing in which volume of used sessions was 3,000
or more ~133,980 total students tested ~106,035 8th grade students tested ~2,228 different test administrators with 52 logging in 300 or
more students ~3,446 students had the Computer Reads Aloud option
Online Progress IndicatorsMaximum Number of Students Tested in a Day
0
1000
2000
3000
4000
5000
6000
2005-06 Fall 2006
Number of Schools Testing 100 or More Students in a Day
0
10
20
30
40
50
60
2005-06 Fall 2006
Maximum Number of Students Tested in a Day at a School
0
50
100
150
200
250
2005-06 Fall 2006
Average Minutes on Test for Students with One Login
0
15
30
45
60
75
90
105
120
2005-06 Fall 2006
Platform Shifting
Fall 2006
Mac OS X (10.3.3-9)1.544%
Windows Me (4.90)0.002%
Windows 2000 (5.0)9.178%
Mac OS X (10.4.1-8)5.734%
Windows 98 (4.10)5.495%
Windows XP (5.1)78.046%
Fall 2005
Windows XP (5.1) 65.22%
Windows Me (4.90) 0.01%
Windows 98 (4.10) 16.05%
Windows 2000 (5.0) 10.04%
Mac OS X (10.4.x) 1.88%
Mac OS X (10.3.x) 6.73%
Mac OS X (10.2.x) 0.07%
2007-08 School Year Schedule
Testing Calendar First Semester [Fall 2007]
September 17, 2007 – January 11, 2008 Second Semester [Spring 2008]
March 10, 2008 – June 20, 2008 Summer 2008
TBD
NCDeskStable environment from 2006-07 school year, however…
New version is available for 2007-08 school year Windows computers – updated August 24, 2007 Macintosh OS X computers – updated August 29, 2007
NCDeskNew and Noteworthy for 2007-08
Local dictionary Two separate spell check dictionaries
Original dictionary functional during test mode New additional local dictionary improves spell check
ability of the NCDesk editor when being used in non-test mode
http://ncdesk.ncsu.edu/ncdesk/technote/dictionaries.asp
NCDeskNew and Noteworthy for 2007-08
Test Simulation New Items Number of items in each section increased from 5 to 10
3 sections, 10 items each section, 30 items total Item/task expectations will be released in the near future
NCDeskNew and Noteworthy for 2007-08
Supports Windows Vista Two versions of Java Runtime Environment [JRE]
Java 5.0_08 – required for Windows 98 [may be problems with Windows 98 running on Java 6.0_02]
Java 6.0_02 – required for Windows Vista [Vista will not run on Java 5.0_08]
Full client details can be found at: http://cskills.ncsu.edu/ncdesk/technote/client.asp
NCDeskNew and Noteworthy for 2007-08
Automatic font verification upon launch Identifies if client computer is missing any of the four
required fonts for testing Times New Roman, Courier New, Arial, Verdana NCDesk will prompt user with a warning message if one
or more of the required fonts are missing Application can still be used for non-testing purposes
without required font(s)
NCDeskNew and Noteworthy for 2007-08
Audio verification within NCDesk Verifies that audio can be played on client computer, but… Does not verify bandwidth capacity of a network or capability
to deliver audio files across a network Note: Audio files have been reduced in size from 96-bit
compression [compression size delivered in the 2006-07 school year] to 32-bit compression to reduce, NOT ELIMINATE, the demand on bandwidth when administering a test using the Computer Reads Test Aloud accommodation
NCDeskNew and Noteworthy for 2007-08
Auto-update message
NCDeskNew and Noteworthy for 2007-08 Confirmation check box on test start page
2007-08 School YearReporting Reporting schedule mirrors 2006-07 school year CSV files with student Pass/Fail status and scale score results
continue to be produced and delivered from NCRegistration Individual Student Reports (ISRs) and labels will no longer be
available via NCRegistration Starting with the 2007-08 school year, results for the North
Carolina Online Test of Computer Skills will be delivered in a WINSCAN format
ISR design may incorporate student-level feedback by strand [or equivalent] during the 2007-08 school year
LEAs will be able to generate ISRs using the INDIVRPT32 Individual Student Report software
Just A Reminder2006-07 Implementations continue… SIQs must be filled out at least one day prior to testing
If SIQs are not complete, the students will not appear in the testing application
Once a student begins testing, SIQ can no longer be edited
Tests must be started and completed on the same day unless student is registered as multiple test sessions Forced completion of tests on a nightly basis
Test Information
StrandPercent of Test
Items
Societal/Ethical Issues 12-14%
Database 22-25%
Spreadsheet 22-25%
Keyboard Utilization/WP/DTP 18-20%
Multimedia/Presentation 10-12%
Telecommunications/Internet 10-12%
Resources http://cskills.ncsu.edu/nccs
Link to home page of the North Carolina Online Test of Computer Skills
http://ncdesk.ncsu.edu/ncdeskDirect link to the home page for the NCDesk application suite
http://cskills.ncsu.edu/ncdesk/technote/Direct link to the home page for the NCDesk technical notes
http://www.ncpublicschools.org/curriculum/computerskillsLink to Computer/Technology Skills Standard Course of Study on the North Carolina Department of Public Instruction website
Resources http://community.learnnc.org/dpi/tech
Link to Computer/Technology Skills page for Curriculum and School Reform on the North Carolina Department of Public Instruction website
http://www.ncpublicschools.org/accountability/testing/computerskillsLink to computer skills testing information on the North Carolina Department of Public Instruction website
http://tps.dpi.state.nc.us/Link to Technology Implementation & Planning Services page on the North Carolina Department of Public Instruction website
http://www.ncpublicschools.org/techservicesLink to Technology Services page on the North Carolina Department of Public Instruction website
North CarolinaEOC Online Assessments
NCTestNCTest is a web browser-based testing system that relies on Adobe/Macromedia Flash to display test content. Except for the North Carolina Online Test of Compuer Skills, the NCTest web application will be used to deliver online assessments for the North Carolina Testing Program for the 2007-08 school year
http://cuacs8.mck.ncsu.edu/nctest/
NCTest Tests that can be accessed with NCTest:
NCEXTEND2 English I Civics & Economics U.S. History Algebra I Algebra II Geometry Biology Physical Science Chemistry Physics
2006-07 School Year Participation Physics [Field Test]: 8597
Fall: 3733 Spring: 4864
Algebra I: 3919 Algebra II: 17 Biology: 1513 Civics & Economics: 1457 English I: 2775 Geometry: 57 US History: 1524
General EOC Online Information All EOC tests are available in an online format, but are
NOT required to be administered online, with the exception of…
Physics – Physics is ONLY available in an online format Why? Physics includes innovative/interactive,
performance-based items [virtual simulations] within the assessment
Sufficient traditional materials printed for all students statewide
General EOC Online Information Scrap paper is required A Flash-based scientific calculator will be provided
within the test interface for tests where calculators are appropriate. Physical calculators are also required and procedures for using located in the appropriate Test Administrator's Manual must be followed
Students MUST NOT open another web browser, local application, or other Internet-based service application (Instant Messager, etc.) while taking an online EOC assessment
General EOC Online Information For Physics ONLY: If necessary, test takers can receive
assistance with getting the simulations to work, but not working the simulations
For Physics ONLY: Flash-based virtual Reference Tables for Physics are provided within the test interface for physics. Physical copies of the reference tables are also required
EOC Online Technology Browser-based online assessment - should work with
recent versions of Internet Explorer and Firefox (these browsers have been used during development work)
Adobe/Macromedia Flash 9 or better required FlashPaper for standard item presentation Flash content for interactive items
JavaScript must be allowed in browsers used to access the test(s)
Session state cookies must be allowed in browser used to access the test(s)
EOC Online Technology LEAs/schools must allow all web traffic from
data.ncsu.edu (standard and secure) Pop-up windows have to be allowed from data.ncsu.edu
in order to force the browser into full-screen mode Pop-up windows must be allowed to open in a new
window. Pop-up windows must NOT open in a tabbed window
Test takers WILL NOT have to move to windows outside of the test administration window
EOC Online Technology Bandwidth Footprint (at this time):
At this time there are no detailed metrics established for footprint analysis. We will be reporting these when available.
However, at this time, the downstream estimate for simple questions is 40-50 Kb per question. Questions with larger graphics or text passages and questions (e.g., English), can be 100 Kb or higher. Interactive items downstream estimate is between 20-70Kb. Upstream footprint for most test questions is very small, but interactive items could equal the amount of the download or greater.
EOC Online Technology Soft section breaks
This is unlike the North Carolina Online Test of Computer Skills which has “hard” section breaks
Test takers can navigate throughout the entire test until completed
Innovative/Interactive performance-based items [virtual simulations] are used in Physics, but are not used in any other EOC online assessment at this time
Likelihood of background color changed to yellow for all EOC online assessments [like NCEXTEND2 feasibility]
Registration and Reporting Eligible participants are students currently enrolled in the
course either at the school or through a virtual online course for which the test is required
Registration and SIQ entry of participants in NCRegistration Scheduling of testing sessions in NCRegistration Participants must be registered, SIQs completed, and sessions
scheduled in NCRegistration at least one day prior to the administration
Standard administrations must be completed on the same day. Reporting output timeline and format is TBD
WINSCAN
2007-08 School Year Schedule
TBD
NCEXTEND2A Brief Update Feasibility study currently being conducted Feasibility study closes October 19, 2007 One EC Teacher and one general education teacher
choose one student with disabilities to complete the feasibility study
More information on the feasibility study will be provided once analysis of results is conducted
Online AssessmentsChallenges Technology Resources Logistics Communication Time Complacency Fear
Online AssessmentsKeys to Success Support
Strong foundation must be built and maintained Learning
Familiarity = Knowledge = Understanding = Confidence Plan
Don’t drop down the rabbit-hole without at least considering how to get out
COMMUNICATION and COOPERATION Share ideas, opinions, and concerns Ask questions Seek resources and act as a resource
CommunicationTESTING TECHNOLOGY
STATE LEA
LEA SCHOOL
SCHOOL SCHOOL
Contact InformationScott Ragsdale
Project Manager, North Carolina Computer Skills AssessmentsNC State University, CUACS/[email protected]
Randy CravenDirector, Technology ImplementationNC State University, CUACS/[email protected]
http://cskills.ncsu.edu/helpdesk/
Final Thought
“Let us not look back in anger or forward in fear, but around in awareness.”
James Thurber
Questions and Comments
