Upload
christopher-terry
View
214
Download
0
Embed Size (px)
Citation preview
Measuring the Effectiveness of Virtual School Programs
Research and BeyondJuly 10 2007
2
Introductions and Program Overviews
bull VHS Global Consortium ndash Liz Pape
bull IDLA -Idaho Digital Learning Academy ndash Donna Vakili
bull eHigh School Cobb County GA ndash Becky Nunnally
3
What does an effective virtual school program look like
What are the indicators for measurement and how are they defined
How are virtual school programs held accountable
How do we communicate if we all define ourselves by different standards and definitions
4
EVALUATION is integral to a continuous quality improvement process
What do we evaluate
Online Course (Becky)Online Instruction (Donna)Student Learning (Liz)Online Program ( All)
5
ldquoFor many students online access has changed the way they see the world and the way they work and play Consideration of these student issues is now critical in designing and delivering quality online instructionrdquoStandards for Quality Online Courses SREB November 2006
Why do we evaluate an online course
To improve the course
To improve student learning
6
How do we evaluate an online course
Clearly defined objectives aligned to standards
Linking learning objectives to specific learning activities course
materials assessment and measurement
Opportunities for communication and interaction including student to
student student to instructor and student to content
Highly engaging activities that meet the needs of all learning styles
and engage the student
Developed in teams and meets standards and quality measures by
peers
7
Ex eHigh School Course Evaluation Process
Technical design
Curriculum alignment
Rigor depth and breadth
Student performance
Student participation and interaction
Ongoing
8
Why do we evaluate online course instruction
To improve course designTo improve delivery of course contentTo improve course facilitation and managementTo meet the needs of our customers
9
How do we evaluate online course instruction
Focuses on whether the objectives of the course were met amp informs practice
Pre-Mid Post-course surveys to all stakeholdersLearner reflectionsPeer and Administrator reviewInstructor self-evaluation
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
2
Introductions and Program Overviews
bull VHS Global Consortium ndash Liz Pape
bull IDLA -Idaho Digital Learning Academy ndash Donna Vakili
bull eHigh School Cobb County GA ndash Becky Nunnally
3
What does an effective virtual school program look like
What are the indicators for measurement and how are they defined
How are virtual school programs held accountable
How do we communicate if we all define ourselves by different standards and definitions
4
EVALUATION is integral to a continuous quality improvement process
What do we evaluate
Online Course (Becky)Online Instruction (Donna)Student Learning (Liz)Online Program ( All)
5
ldquoFor many students online access has changed the way they see the world and the way they work and play Consideration of these student issues is now critical in designing and delivering quality online instructionrdquoStandards for Quality Online Courses SREB November 2006
Why do we evaluate an online course
To improve the course
To improve student learning
6
How do we evaluate an online course
Clearly defined objectives aligned to standards
Linking learning objectives to specific learning activities course
materials assessment and measurement
Opportunities for communication and interaction including student to
student student to instructor and student to content
Highly engaging activities that meet the needs of all learning styles
and engage the student
Developed in teams and meets standards and quality measures by
peers
7
Ex eHigh School Course Evaluation Process
Technical design
Curriculum alignment
Rigor depth and breadth
Student performance
Student participation and interaction
Ongoing
8
Why do we evaluate online course instruction
To improve course designTo improve delivery of course contentTo improve course facilitation and managementTo meet the needs of our customers
9
How do we evaluate online course instruction
Focuses on whether the objectives of the course were met amp informs practice
Pre-Mid Post-course surveys to all stakeholdersLearner reflectionsPeer and Administrator reviewInstructor self-evaluation
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
3
What does an effective virtual school program look like
What are the indicators for measurement and how are they defined
How are virtual school programs held accountable
How do we communicate if we all define ourselves by different standards and definitions
4
EVALUATION is integral to a continuous quality improvement process
What do we evaluate
Online Course (Becky)Online Instruction (Donna)Student Learning (Liz)Online Program ( All)
5
ldquoFor many students online access has changed the way they see the world and the way they work and play Consideration of these student issues is now critical in designing and delivering quality online instructionrdquoStandards for Quality Online Courses SREB November 2006
Why do we evaluate an online course
To improve the course
To improve student learning
6
How do we evaluate an online course
Clearly defined objectives aligned to standards
Linking learning objectives to specific learning activities course
materials assessment and measurement
Opportunities for communication and interaction including student to
student student to instructor and student to content
Highly engaging activities that meet the needs of all learning styles
and engage the student
Developed in teams and meets standards and quality measures by
peers
7
Ex eHigh School Course Evaluation Process
Technical design
Curriculum alignment
Rigor depth and breadth
Student performance
Student participation and interaction
Ongoing
8
Why do we evaluate online course instruction
To improve course designTo improve delivery of course contentTo improve course facilitation and managementTo meet the needs of our customers
9
How do we evaluate online course instruction
Focuses on whether the objectives of the course were met amp informs practice
Pre-Mid Post-course surveys to all stakeholdersLearner reflectionsPeer and Administrator reviewInstructor self-evaluation
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
4
EVALUATION is integral to a continuous quality improvement process
What do we evaluate
Online Course (Becky)Online Instruction (Donna)Student Learning (Liz)Online Program ( All)
5
ldquoFor many students online access has changed the way they see the world and the way they work and play Consideration of these student issues is now critical in designing and delivering quality online instructionrdquoStandards for Quality Online Courses SREB November 2006
Why do we evaluate an online course
To improve the course
To improve student learning
6
How do we evaluate an online course
Clearly defined objectives aligned to standards
Linking learning objectives to specific learning activities course
materials assessment and measurement
Opportunities for communication and interaction including student to
student student to instructor and student to content
Highly engaging activities that meet the needs of all learning styles
and engage the student
Developed in teams and meets standards and quality measures by
peers
7
Ex eHigh School Course Evaluation Process
Technical design
Curriculum alignment
Rigor depth and breadth
Student performance
Student participation and interaction
Ongoing
8
Why do we evaluate online course instruction
To improve course designTo improve delivery of course contentTo improve course facilitation and managementTo meet the needs of our customers
9
How do we evaluate online course instruction
Focuses on whether the objectives of the course were met amp informs practice
Pre-Mid Post-course surveys to all stakeholdersLearner reflectionsPeer and Administrator reviewInstructor self-evaluation
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
5
ldquoFor many students online access has changed the way they see the world and the way they work and play Consideration of these student issues is now critical in designing and delivering quality online instructionrdquoStandards for Quality Online Courses SREB November 2006
Why do we evaluate an online course
To improve the course
To improve student learning
6
How do we evaluate an online course
Clearly defined objectives aligned to standards
Linking learning objectives to specific learning activities course
materials assessment and measurement
Opportunities for communication and interaction including student to
student student to instructor and student to content
Highly engaging activities that meet the needs of all learning styles
and engage the student
Developed in teams and meets standards and quality measures by
peers
7
Ex eHigh School Course Evaluation Process
Technical design
Curriculum alignment
Rigor depth and breadth
Student performance
Student participation and interaction
Ongoing
8
Why do we evaluate online course instruction
To improve course designTo improve delivery of course contentTo improve course facilitation and managementTo meet the needs of our customers
9
How do we evaluate online course instruction
Focuses on whether the objectives of the course were met amp informs practice
Pre-Mid Post-course surveys to all stakeholdersLearner reflectionsPeer and Administrator reviewInstructor self-evaluation
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
6
How do we evaluate an online course
Clearly defined objectives aligned to standards
Linking learning objectives to specific learning activities course
materials assessment and measurement
Opportunities for communication and interaction including student to
student student to instructor and student to content
Highly engaging activities that meet the needs of all learning styles
and engage the student
Developed in teams and meets standards and quality measures by
peers
7
Ex eHigh School Course Evaluation Process
Technical design
Curriculum alignment
Rigor depth and breadth
Student performance
Student participation and interaction
Ongoing
8
Why do we evaluate online course instruction
To improve course designTo improve delivery of course contentTo improve course facilitation and managementTo meet the needs of our customers
9
How do we evaluate online course instruction
Focuses on whether the objectives of the course were met amp informs practice
Pre-Mid Post-course surveys to all stakeholdersLearner reflectionsPeer and Administrator reviewInstructor self-evaluation
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
7
Ex eHigh School Course Evaluation Process
Technical design
Curriculum alignment
Rigor depth and breadth
Student performance
Student participation and interaction
Ongoing
8
Why do we evaluate online course instruction
To improve course designTo improve delivery of course contentTo improve course facilitation and managementTo meet the needs of our customers
9
How do we evaluate online course instruction
Focuses on whether the objectives of the course were met amp informs practice
Pre-Mid Post-course surveys to all stakeholdersLearner reflectionsPeer and Administrator reviewInstructor self-evaluation
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
8
Why do we evaluate online course instruction
To improve course designTo improve delivery of course contentTo improve course facilitation and managementTo meet the needs of our customers
9
How do we evaluate online course instruction
Focuses on whether the objectives of the course were met amp informs practice
Pre-Mid Post-course surveys to all stakeholdersLearner reflectionsPeer and Administrator reviewInstructor self-evaluation
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
9
How do we evaluate online course instruction
Focuses on whether the objectives of the course were met amp informs practice
Pre-Mid Post-course surveys to all stakeholdersLearner reflectionsPeer and Administrator reviewInstructor self-evaluation
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
10
Ex IDLA Teacher Evaluation External (Fall lsquo04-Fall lsquo05)
bull Student Surveys (N=715)ndash 87 of students had positive and rewarding interactions with their
online teacherndash 88 said that the online teacher was very supportive and helped
them learnndash 89 said that the teacher was good at moderating course
discussionsndash 86 felt that the teacher met their needsndash 85 felt that the teacher was prompt and helpful with feedbackndash 84 felt encouraged to express themselves throughout the coursendash 84 felt satisfied with the extra online help received from the
teacherndash Areas of Improvement Declining trend in experience with teachers
(up 11 for Summer rsquo06) Increase prof dev of teachers in moderating online discussions more teacher interaction
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
11
Ex IDLA Teacher Evaluation External (Spring 2006)
bull Administrators (N=152)ndash Overall 86 of administrators rate IDLA as good or
excellentndash 85 of administrators rate responsiveness of IDLA teachers
to needs of their school and students as good or excellentndash 64 rate the quality of the IDLA teachers as average to
above avg (30 Donrsquot Know)ndash Areas of improvement Improve student progress reporting
clear lines of communication and schools know their responsibilities
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
12
Ex IDLA Teacher Evaluation Internal Methods
bull Independent Contractorsndash Non-Renewablendash Teaching Expectations
bull Bonus Teaching Rubricndash Administrative Requirements (25)ndash Professional Development (20)ndash Communication and Intervention (45)ndash Student Evaluations (10)
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
13
Ex Evaluating Online Course Instruction Informs Practice at IDLA
bull Mandated Online Site Coordinator Coursebull Instituted Teacher Rubricbull Systemized and clarified role of online principal and
evaluationbull Established systems and consistency for reporting such
as phone logs and progress reportsbull Guided Online and Summer Conference Professional
Development for Facultybull Focused Emphasis on Quality amp Customer Servicebull Increased Value for External Evaluations
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
14
Why do we evaluate student learning
Mastery of course content
Application of course content to the real world
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
15
How do we evaluate student learning
Pre-MidPost-course surveysLearner reflectionsPeer reviewInstructor self-evaluationEnd-of-course assessments
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
16
Ex VHS Student Learning Measures
bull AP pass rates 70
bull AP exam take rates79
bull Course completion rates 79
bull Credit recovery rate (summer)88
bull TeacherSCPrincipalSuperStudent Survey input
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
17
Ex VHS AP exam take rate with of Students
AP Exam Take Rates
7989
60
49
80 302
124
68
1843
0102030405060708090
100
0001 0102 0203 0304 0405
0
50
100
150
200
250
300
350
AP ExamParticipationRate AP Students
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
18
Ex VHS Course Completion Rate
Course Completion Rate
818383
8179807979
0102030405060708090
100
9798 9899 9900 0001 0102 0203 0304 0405
CourseCompletionRate
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
19
Why do we evaluate an online program
AccountabilityStewardship of fundsImprove student achievementMeet the needs of the learning communityPlan for the future
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
20
How do we evaluate an online program
Program UtilizationEnrollment (seats)Number of students servedNumber of students served with no other opportunityCourseTeacher Evaluations
By studentsBy parents
Counselor and Administrator Surveys and Interviews
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
21
Ex VHS Growth GoalNumber of Students
VHS students (Projected 0707)
710 943
25163675 3736 4063
50696114
7604
9500
0
2000
4000
6000
8000
10000
9798 9899 9900 0001 0102 0203 0304 0405 0506 0607
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
22
Ex VHS Professional Development Measures ndashHard Data Points Research and Survey Data
bull Professional development grad rate 89bull 1st-semester teacher success rate 94bull of teachers remaining with FA 4bull of external evaluationsbull Satisfied with VHS professional development
ndash 99 principals
bull Applying VHS pedagogy to f2f teachingndash 79 teachers
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
23
Ex VHS Performance Metrics-Program and Services Quality
bull Membership retention rates 87bull Seat utilization 83bull of external evaluationsbull Satisfaction with electronic communications email
ndash Teachers 99 SCs 96
bull Websitendash Teachers 97 SCs 98
bull NewsletterAnnual Reportndash Superintendents 8668ndash Principals 7879ndash Teachers 7954ndash SCs 8454
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
24
Ex VHS Quality Benchmark Indicators
Indicators of Success
0
20
40
60
80
100
AP Pass Rate CourseCompletions
Satisfactionw ith VHS
High CourseQuality
01-05(cum)all APstudents
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
25
eHigh School Program Measures of Effectiveness
Student success - completions - passing - exceedsmeets EOCT Meets local school needs - scheduling conflicts - graduation requirements Meets student needs - flexibility - acceleration - credit recovery - hospitalhomebound - family obligations
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
26
Tools of Tools of EngagementEngagementWhole Course Whole Course
DeliveryDeliveryBlended Blended LearningLearningBlackboard
Live Classroom
BlogsWikis
Podcasts
Polar Watches
Turnitincom
AP Courses
Work Study Programs
Career Education
High School
Middle School
Faculty Collaboration
Mathematics
Language Arts
Science
Social Studies
Health amp PE
Business
Meeting our 21Meeting our 21stst Century Students Century Students OnlineOnline
Cobb County School Districtwwwcobbk12ehighschoolorg Marietta Georgia
ehighschoolcobbk12org
ISTE NETSS Standards 1 Creativity amp Innovation 2 Communications amp Collaboration
3 Research and Information Fluency 4 Critical Thinking Problem-Solving amp Decision-Making5 Digital Citizenship 6 Technology Operations amp Concepts
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
27
Ex Virtual Charter Schools QualityEvaluation
bull Similar to Supplementalndash Inputs Sourcing Structure Alignmentndash User Input Surveys Feedback ldquoPopularityrdquondash Results Successful Completion
bull Different from Supplementalsndash Total Student Progress Monitoring (and Intervention
with Multiple Tools) ndash NCLB Accountability ndash Test Resultsndash Systemic Improvement ndash Courses Curriculum
Platform Instruction
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
28
Measuring the Effectiveness of Virtual School Programs Research amp Beyond
ISSUESIndicators for measurementDefinition of indicatorsTimeframe for evaluationExample Completion Rates
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
29
Measure of Effectiveness
bull Policy Changesndash Establishment of State Virtual Schoolsndash Michigan Requirement for High School
Graduation
bull Fundingndash Changes to State Law and funding formulasndash Examples IDLA Georgia and Florida Virtual
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
30
And still more questions
bull Are we measuring what we should be measuring
bull What else should we measurebull Who should be measuringbull How do we publish results of measuresbull Where is the Consumer Reports for virtual
schools and online courses
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
31
Will the growing national investment in Kndash12 online learning particularly implementing and expanding the
use of online high school courses provide a basically sound high-quality educational
experience for online students Or will this new investment in Kndash12 online
learning be yet another waste of scarce national educational resources
on todayrsquos newest educational technologies that is destined to be abandoned
in some virtual tomorrowrdquo
(A Synthesis of New Research on Kndash12 Online Learning)
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg
32
References
bull eSchool wwwcobbk12ehighschoolorg bull Idaho Digital Learning Academy
wwwidahodigitallearningorg bull Keeping Pace with K12 Online Learning A Review of
State Level Policy and Practice bull North American Council for Online Learning (NACOL)
wwwnacolorg bull A Synthesis of New Research on Kndash12 Online Learning
httpwwwncrelorgtechsynthesissynthesispdfbull Virtual High School (VHS) wwwgoVHSorg