Technology to TEACH

Kathy MaksimovCurriculum Specialist

Modern Computing Technology

Manufacturers needed consistency

Scientists needed perfect recall and delivery

Businesses needed the ability to scale

Turned to Technology

Consistent, replicable

Perfect recall and delivery

The ability to scale

As educators, we need consistency, perfect recall, the ability to scale, and

…

… the time to focus on just one child.

Our StudentsPre-Literacy Training by First Grade

Marilyn Jager Adams, Beginning to Read, 1990

Middle Class

Low Income

3000 Hours

200 Hours

Our StudentsEarly Predictor

Our Students

- Hart and Risley, Meaningful Differences (1995)

Welfare Parents

13 million words

2:1 negative to positive

Working Class Parents

26 million words

2:1 positive to negative

Professional Parents

45 million words

6:1 encouraging

Vocabulary at age 4

A normal classroo

m

Average Students

Exceptional Students

Troubled Students

The Average School Day

- Eaton H. Conant (1973)

Time at school

Actual Instructio

n

Individualized

Instruction

7 Hours

2 Hours

1 Minute!

“The Work Problem”

w = p*ew is the work produced

by a system

p is the potential of the system to create work

e is efficiency of the system in creating work

Efficiency (e)

• How well workers work.• Maximum = 100%• Examples:

– Lesson manuals– Professional development– Mastery learning– Managed schools, charter schools, etc.– Accountability– Grouping students

Potential (p)

• Workers and tools• Limited only by the

worker or tool• Examples:

– Teacher– Paraprofessionals– Manipulatives– Chalkboards or digital whiteboards– Books– Software …

w = p*eExample: Digging a Foundation

The Work Problem

What if your students need this much work?

An Ideal Solution?• Do I scale?• Can I

individualize?• Am I interactive?

• Perhaps you just need more of me …

• Costs too much …• Can’t find enough experts

…• Too hard to consistently

train existing resources …• Not enough time …

Why can’t we solve the work problem?

The Ideal Solution Described

• Scalable• Affordable • Perfect recall• Consistently replicable• Never tired, impatient, or frustrated• Always the very best performance• Constantly improving

What if your students need this much work?

The Work Problem …

The Work Problem … Solved!

Technology fundamentally changes potential …

Communication example:1. Pony Express2. Telegraph (45.4 million

times faster than a horse)

3. Telephone4. Radio5. Television6. Optical fiber / Internet

Musical Performance

A pioneer father goes to see a concert …

Teaching is a Performance

What if you could capture and always deliver the best teaching?

The Formation of Waterford - Dusty’s Epiphany

Moore’s Law

And so on…

Doubling Checkmate

$184,000,000,000,000,000.00

In the beginning, most people only saw

a penny.

Decision Science

Learner Profiles

• Can we apply decision science in education?

Approximation to Precision

Linear vs. Exponential Growth

Source: Kurzweil 2005, The Singularity

is Near

Leveraging Technology Requires …

•Commitment

•Willingness to change …–How we view the classroom

–How we view the role of the teacher

–What we teach and when

–How we use student data

Leveraging Technology Delivers …

the very best education

individualized

for each student on the curve

“It’s independently run. I turn it on in the morning and it pretty much through the rest of the day, gives them their time on it and evaluates where they need to be the next day.”

- Shannon Skipper, Pre-K Teacher, Gadsen, Alabama

A Teacher’s Perspective …

Teacher Experience

> 450 Hours of Instruction

Ways Programs Deliver Instruction

Menu Linear / Predetermined

Adaptive (Mastery-based)

1

2

3

4

From Chutes and Ladders by Milton Bradley

Waterford Delivers Instruction

Automatically individualized for

each student

From Chutes and Ladders by Milton Bradley

Waterford’s Sequencing

Lesson

Pre-assessmentSongBook

InstructionPractice

Extended PracticeAssessment

Did the student master the learning objective?

Sequencing within a Lesson

“Successful” SaraContinue to the next lesson

Sequencing within a Lesson

“Needs Help” Sam

Mark this lesson to automatically try again later

Sequencing between Lessons

1 2 3 1 4

1 2 3 2 1

Automatic Review

Automatic Review

Try Again

“Successful” Sara

“Needs Help” Sam

Teacher Reports

• Averages• Student

progress, usage, and skill performance

• Highlighted areas of concern

Meet “Miss Waterford”

• One-on-one instruction tailored for each student …

… Proven methods, endlessly patient, FUN, responsive, private, equitable

The Data

1. Understand types of efficacy studies

2. Review examples with Waterford

Understanding Terms

• Random Assignment: – a technique for assigning subjects to

different treatments (or no treatment).

• Control Group– the group that does not receive the new

treatment being studied.

Study Designs

Quasi Experiments

Pretest Posttest Nonequivalent Group. 

– Control and treatment– Group assignment by convenience – Pretest and posttest

Commons Lane Elementary

• Pre and Posttest (Terra Nova)• Participants (K and 1st):

– Commons Lane (treatment) – 20 students per class; approx. 80

– Halls Ferry (control) – 13 students per class; approx. 80

• Non-equivalents– Class size (favors the control)– Pretest scores (Commons Lane kindergarteners had

lower pretest scores)

Commons Lane – Kindergarten Results

Commons Lane =

3 times the gains!

Commons Lane – 1st Grade Results

0

5

10

15

20

25

30

35

Gains

ControlExperiment

Commons Lane = 2 times the gains!

Hecht and Close (Florida)

• Pre and Posttest• Participants: inner city & rural public

schools with low SES (“at risk” students)– 42 Kindergarteners (treatment)– 34 Kindergarteners (control)

• Treatment:– Six months on Reading Level One

Hecht and Close - Definitions

• Effect sizes (ES): tell how different two groups are.– ES = 0.2: small difference – ES = 0.5: medium difference– ES = 0.8: large difference

• Finding: Best Predictors of Future Reading Ability– Segmenting and blending phonemes

Hecht and Close - Results

Segmenting ES = 1.14 Blending ES = 1.13 Word Reading ES = 1.11 Invented Spelling ES = 1.19

Print Concepts Letter Name Letter Sound Letter Writing

“… Computer assisted instruction provides a cost effective way to teach at-risk children.”

Hecht and Close - Results

• Confident in Results:– Exact amount of time

each student used Waterford

– Computer delivers identical experience

– Individualized– Reports show exactly

how students performedStudying Computer-Based

InstructionStudying Classroom Instruction

Hecht and Close

Sited in:Developing Early Literacy: Report of the National Early Literacy Panel

“Found that … Amount of exposure children had to [Waterford] contributed to individual differences in phonemic awareness and spelling.”

Quasi Experiments

• Time Series Designs.  – One group of subjects– Pretested and posttested at different intervals.  – The purpose might be to determine long-term effect of

treatment and therefore the number of pre- and posttests can vary from one each to many. 

Hillcrest Elementary

• Demographic:– Title 1 School (low SES)

• Before Waterford:– Below district average reading scores– 75% of students were in two lowest

reading categories: below basic (more then 50%) and basic

Hillcrest Elementary - Results

• Two years after Waterford: trend reversed.– 75% students in top two categories: Proficient

and Advanced

• Three years after Waterford:– the first class to use all three levels of

Waterford from kindergarten to second-grade reached the third-grade and had the highest reading scores of all 36 schools in the district!

0

5

10

15

20

25

30

35

40

1995 1997 1998

BelowBasicProficientAdvanced

Hillcrest Elementary - ResultsKindergarten

Student Rankings on the Utah State Core Assessment Test

Nu

mb

er

of

Stu

den

ts

1996*

State of Idaho

• Participants: 8 Idaho School Districts– 3,394 students

(treatment)– 2,413 students

(historical control)

• Test scores (IRI) over 4 years

State of Idaho – ResultsMore Use = Higher Gains

3637

3839

44

49

34

36

38

40

42

44

46

48

50

Control 0–1000 1001–1500

1501–2000

2001–2500

2500+

Usage (Minutes)

Av

era

ge

Ga

in

Waterford recommends 15 min per day = 2250 min

State of Idaho - Results

• Lowest third (at-risk) experienced the most gains (>1.0 effect size).

• Finishing the level had a larger effect size than SES, motivation, and tutoring.

A Midwest School (Indiana)

• Pretest and Posttest• Participants:

– 46 first grade students (year 2001) - Treatment

– 47 first grade students (year 2000) - Control

• Historical control is nice because it reduces the variance from teachers.

A Midwest School - Results

• Evaluated students by how they performed on the pretest:– High scores– Moderate scores– Low scores

• All treatment students outperformed their control counterparts … but the low treatment outperformed the moderate control on the posttest!

A Midwest School - Results

660

640

620

600

580

560

540

520

500Grade 1 Grade 2

Control – High

Control – Moderate

Control – Low Exp – High

Exp – Moderate

Exp – Low

True Experiments

• “The Gold Standard”• Random treatment and

control• Testing to measure change

in both groups• Only research method that

can adequately measure the cause and effect relationship

True Experiments

• Post Equivalent Groups.– Treatment and control– Randomized assignment to groups– Posttest administered to measure

differenceR = Randomized participants

N = Not-randomized participants

O = Test

X = Treatment

True Experiments

• Pretest Posttest Equivalent Groups– Treatment and control– Randomized assignment to group– Pretest to measure difference before the

study takes place– Posttest to measure effect of treatment

R = Randomized participants

N = Not-randomized participants

O = Test

X = Treatment

True Experimentsin Education

One review showed that not even 1 percent of dissertations in education or of the studies archived in ERIC Abstracts involved randomized experiments. http://www.hoover.org/publications/ednext/3384446.html

True Experimentsin Education

Challenges for True Experiments in Education

• Random sample– Parents– School staff / Well-

meaning teachers

• Fidelity of implementation– Teacher abilities– Classroom set up – Scheduling

Tucson – Math and Science

• Participants:– 5 Title 1 schools in Tucson Unified School

District• Free and reduced lunch rate 88.5%-97.5%• 22 classrooms• 338 students total

– Treatment and Control– Random assignment of classrooms– Pretest and Posttest

• SAT10 Math and the environment (science) tests

Tucson Results by classroom

School Gain Diff Free & Reduced Lunch

ELL

A WEMS Control

8.84 .99

7.85 88.5% 16.0%

B WEMS Control

13.43 2.99

10.44 90.6% 39.2%

C WEMS Control

14.23 2.86

11.37 98.3% 22.5%

D WEMS Control

13.02 6.92

6.10 97.5% 49.0%

E WEMS Control

6.21 1.26

4.95 92.6% 29.1%

Tucson Results define terms

• NCE (Normal Curve Equivalent)– Where a student falls on a normal curve

• Indicates a student’s rank compared to other students on the same test

– Range from 1-99 with mean of 50– In a normally distributed population, if all

students make exactly one year of progress, NCE gain would be zero even though raw score increased.

Tucson Resultsby subject

Math

Science

Math - Tucson Results

35

37

39

4143

45

47

49

51

Pretest Posttest

Me

an

NC

E

Waterford

Control

Science - Tucson Results

35

37

39

41

43

45

47

49

Pretest Posttest

Me

an

NC

E

Waterford

Control

Tucson Resultsby genderTucson Results - by Gender

11.71

10.00

5.84

0.45

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

Boys Girls

Mea

n N

CE

Gai

nWaterford

Control

Tucson Resultsby ELL status

Tucson Results - ELL Status

0

2

4

6

8

10

12

14

16

18

20

Waterford ELL Waterford Non ELL Control ELL Control Non-ELL

Mea

n N

CE

Gai

n

Tucson Resultsby ELL status

Tucson Results - ELL status

30

35

40

45

50

55

Pretest Posttest

Mea

n N

CE

Waterford ELL

Waterford Non ELL

Control ELL

Control Non-ELL

Waterford ELL students had the lowest pretest scores and the highest posttest scores!

Qualitative Research• Uses “naturalistic” methods

– interviewing– observation– focus groups

• No statistical or quantitative procedures

• Goals – behavior in natural setting– perspective of the research

participant– meanings people give to their

experience

Madisonville Consolidated Independent School District

• Teachers report higher interest in reading– They report that children now argue over who is allowed to go

to the reading centers, when previously there was little interest shown in reading activities.

• Teachers report improved home/school connection; parents support program 100% (survey)

• Increased student academic self-esteem• Waterford supports and supplements existing

curriculum• Waterford is user friendly• Anecdotes of improved phonemic awareness and

reading readiness skills

Questions?