11

Adult ESL Literacy Adult ESL Literacy Impact Study:Impact Study:Description and Description and

ImplicationsImplicationsLarry CondelliLarry Condelli

Stephanie CronenStephanie CronenAmerican Institutes for American Institutes for

Research, USAResearch, USA

LESLLA LESLLA Fifth Annual SymposiumFifth Annual Symposium

Banff, AB, CanadaBanff, AB, CanadaSeptember 29, 2009September 29, 2009

22

Overview of PresentationOverview of Presentation

Summary of Purpose, Research Summary of Purpose, Research Questions and Design Questions and Design

Present descriptive data on students Present descriptive data on students and teachersand teachers

Discussion and implications of Discussion and implications of potential findingspotential findings

33

Evaluation of an ESL Literacy Evaluation of an ESL Literacy Intervention: Intervention: Sam and PatSam and Pat

Structured language approach Structured language approach Adapted from Wilson Reading Adapted from Wilson Reading

SystemSystem Never before evaluated for ESL Never before evaluated for ESL Literacy activities organized Literacy activities organized

around basal reader/workbookaround basal reader/workbook Heavily phonics-basedHeavily phonics-based

44

Sam and Pat: Instructional Sam and Pat: Instructional ApproachApproach

Direct instruction, transparentDirect instruction, transparent Rules explained, modeled, practicedRules explained, modeled, practiced

Controlled text, vocabulary and Controlled text, vocabulary and grammargrammar Words match phonics already learnedWords match phonics already learned

Sequential Sequential Easy to hard in defined stepsEasy to hard in defined steps

Multi-sensory, kinestheticMulti-sensory, kinesthetic ESL instruction to support literacyESL instruction to support literacy

55

Sam and Pat: Literacy and Sam and Pat: Literacy and Language Skills CoveredLanguage Skills Covered

Phonics for reading and writingPhonics for reading and writing Sight wordsSight words Oral reading for Oral reading for

accuracy and fluencyaccuracy and fluency Reading comprehensionReading comprehension VocabularyVocabulary Speaking and listeningSpeaking and listening GrammarGrammar

66

Structured Two Part LessonStructured Two Part Lesson

Part 1:Part 1: Rereading familiar Rereading familiar

texttext Pre-reading (picture Pre-reading (picture

story)story) Letters and phonicsLetters and phonics Vocabulary and Vocabulary and

grammargrammar Sight wordsSight words ConversationConversation

Part 2:Part 2: Review of Part 1 Review of Part 1

contentcontent Read storyRead story Written exercisesWritten exercises

77

Research QuestionsResearch Questions How effective is instruction using the How effective is instruction using the

intervention in improving the English intervention in improving the English reading, speaking and listening skills of reading, speaking and listening skills of low-literate adult ESL learners?low-literate adult ESL learners?

Is the intervention more effective for Is the intervention more effective for certain groups of students (e.g., certain groups of students (e.g., language, literacy level)?language, literacy level)?

Do differences in level of implementation Do differences in level of implementation of of Sam and PatSam and Pat and other instruction and other instruction relate to variation in impacts?relate to variation in impacts?

88

Study DesignStudy Design 10 adult ESOL centers across USA10 adult ESOL centers across USA Paired intervention and “normal” ESL Paired intervention and “normal” ESL

literacy classes (34 total)literacy classes (34 total) Random assignment of students and Random assignment of students and

teachersteachers Minimum 5 hours/week 10-12 weeks Minimum 5 hours/week 10-12 weeks

instruction with approachinstruction with approach Other instruction also Other instruction also provided (5-10 hours/week)provided (5-10 hours/week) Each class conducted Each class conducted twice over a year twice over a year

99

Student Flow in the StudyStudent Flow in the Study

1.1. Student applies to centerStudent applies to center2.2. Assessed for NLLAssessed for NLL3.3. Recruited into studyRecruited into study4.4. Gives informed consent Gives informed consent 5.5. Random assignment to class Random assignment to class 6.6. Pretests administeredPretests administered7.7. InstructionInstruction8.8. Posttests administeredPosttests administered

IntakeNL Literacy

Post-test

Random Assignment

Recruited into StudyInformed consent

Pre-test

Instruction

1010

Students AssessmentsStudents Assessments

Phonics and decoding Phonics and decoding Word attackWord attack Letter/Word IDLetter/Word ID

Reading comprehensionReading comprehension Vocabulary (ROWPVT)Vocabulary (ROWPVT) Listening, oral expressionListening, oral expression

1111

Sam And Pat Sam And Pat Teachers in the Teachers in the StudyStudy

All teachers randomly assignedAll teachers randomly assigned 3-day teacher training on 3-day teacher training on

curriculumcurriculum Follow-up visits by trainersFollow-up visits by trainers Classroom observationsClassroom observations

to monitor fidelityto monitor fidelity Refresher webinar at Refresher webinar at

start of second term start of second term

1212

Classroom ObservationsClassroom Observations

Literacy Literacy development development instruction:instruction: Pre-literacyPre-literacy PhonicsPhonics FluencyFluency Reading strategies Reading strategies

& comprehension& comprehension WritingWriting

ESL Instruction:ESL Instruction: Oral language Oral language

developmentdevelopment Grammar, etc.Grammar, etc. VocabularyVocabulary Socio-cultural Socio-cultural

knowledgeknowledge Functional literacyFunctional literacy

1313

Students in the Study Students in the Study “True” Literacy (LESLLA) “True” Literacy (LESLLA)

Little or no literacy in native languageLittle or no literacy in native language Limited oral EnglishLimited oral English Education: 0-6 yearsEducation: 0-6 years Languages:Languages:

Haitian-CreoleHaitian-Creole SpanishSpanish Burmese Burmese Others Others

1414

Students in the StudyStudents in the Study Non-Roman Alphabet Literate Non-Roman Alphabet Literate

Some Literacy in native language with Some Literacy in native language with non-Roman scriptnon-Roman script

Mean education: 6 and more yearsMean education: 6 and more years Limited oral English Limited oral English Languages:Languages:

ArmenianArmenian ArabicArabic FarsiFarsi Chinese Chinese Others Others

Description of SitesDescription of Sites Ten sites participated:Ten sites participated:

1 site in San Francisco CA area1 site in San Francisco CA area 1 site in Los Angeles CA area1 site in Los Angeles CA area 3 sites in Miami FL area3 sites in Miami FL area 2 sites in Chicago IL area2 sites in Chicago IL area 3 sites in Houston TX area3 sites in Houston TX area

Sites included adult ed centers, Sites included adult ed centers, high schools, and community high schools, and community centerscenters

Type of programs included…??Type of programs included…??1515

Description of StudentsDescription of Students 1,344 students participated for one 1,344 students participated for one

termterm They attended a range of low-level They attended a range of low-level

classes in the study, with ESOL classes in the study, with ESOL Literacy being the most common:Literacy being the most common: ESOL LiteracyESOL Literacy Low Beginning ESOLLow Beginning ESOL Beginning ESOL (combined w/Literacy)Beginning ESOL (combined w/Literacy)

Students in the sample had been in Students in the sample had been in the U.S. for an average of 4.7 yearsthe U.S. for an average of 4.7 years

1616

Description of StudentsDescription of Students Spanish was the most common Spanish was the most common

language, followed by Armenianlanguage, followed by Armenian

1717

Description of Students, Description of Students, Cont.Cont.

Average years of educationAverage years of education 6.7 years of education outside the U.S.6.7 years of education outside the U.S. 7.7 years of education total7.7 years of education total

Why so high?Why so high? As a group, only Haitian students were As a group, only Haitian students were

“true literacy”“true literacy” Spanish-speakers ranged from true Spanish-speakers ranged from true

literacy to beginning ESL levelliteracy to beginning ESL level Non-Roman alphabet students ranged Non-Roman alphabet students ranged

from beginning ESL level to much from beginning ESL level to much higherhigher

1818

Description of Students, Description of Students, Cont.Cont.

Over half (58%) of Ss were Over half (58%) of Ss were femalefemale

Student ages ranged from 18 to Student ages ranged from 18 to 8484

1919

Teachers in the StudyTeachers in the Study 33 teachers participated in the study in 33 teachers participated in the study in

Fall, and 31 participated in Winter/Spring Fall, and 31 participated in Winter/Spring 1 class pair/2 teachers were dropped before 1 class pair/2 teachers were dropped before

the second term due to insufficient numbers the second term due to insufficient numbers of new students enrolling in their classesof new students enrolling in their classes

1 teacher taught 2 treatment classes both 1 teacher taught 2 treatment classes both termsterms

Teachers had 7.5 years of experience Teachers had 7.5 years of experience teaching adult ESOL, on averageteaching adult ESOL, on average 5.3 years of experience teaching literacy-5.3 years of experience teaching literacy-

levellevel

2020

Teachers in the StudyTeachers in the Study The majority of teachers (85%) had The majority of teachers (85%) had

some form of certificationsome form of certification

2121

Teachers in the StudyTeachers in the Study Most teachers’ highest degree Most teachers’ highest degree

received was either a Bachelor’s received was either a Bachelor’s (42%) or a Master’s (48%)(42%) or a Master’s (48%)

61% of teachers were male61% of teachers were male Race/ethnicity variedRace/ethnicity varied

40% white, nonHispanic40% white, nonHispanic 30% Black or African American30% Black or African American 24% Hispanic or Latino24% Hispanic or Latino 6% Asian or American Indian6% Asian or American Indian

2222

Teachers in the StudyTeachers in the Study Most teachers spoke a language Most teachers spoke a language

other than Englishother than English

2323

Findings and Impacts….Findings and Impacts….

2424

SORRY!!!SORRY!!!

We’re not sure yet (and We’re not sure yet (and couldn’t tell you now couldn’t tell you now

anyway….)anyway….)

2525

2626

Observations and Observations and DiscussionDiscussion

Implications of Possible Implications of Possible Findings for Practice and Findings for Practice and

Research for LESLLA Research for LESLLA StudentsStudents

2727

Implications for LESLLA Implications for LESLLA Practice Practice

Let’s talk about the 3 most likely Let’s talk about the 3 most likely results scenarios:results scenarios: Positive impact on decoding; no other Positive impact on decoding; no other

impactsimpacts No impacts on any outcomesNo impacts on any outcomes Negative impacts on some ESL Negative impacts on some ESL

outcomesoutcomes

2828

Implications for LESLLA Implications for LESLLA Practice Practice

Positive impact on decoding; no other Positive impact on decoding; no other impacts. Given Sam & Pat strong phonics impacts. Given Sam & Pat strong phonics focus this is the expected and most likely focus this is the expected and most likely scenario. scenario.

But is improving only decoding after 3 months of But is improving only decoding after 3 months of instruction enough to warrant using it? instruction enough to warrant using it?

Would the impact differ by type of student?Would the impact differ by type of student? What would the implications of these results be for What would the implications of these results be for

teachers considering Sam & Pat or a similar teachers considering Sam & Pat or a similar approach?approach?

2929

Implications for LESLLA Implications for LESLLA Practice Practice

Gains for all but no differences Gains for all but no differences between groupsbetween groups Sam & Pat is no more effective than Sam & Pat is no more effective than

other types of ESL literacy instructionother types of ESL literacy instruction Under what conditions, then, would Sam Under what conditions, then, would Sam

& Pat be a good choice for instruction? & Pat be a good choice for instruction? What are some potential explanations What are some potential explanations

for these results from your perspective? for these results from your perspective?

3030

Implications for LESLLA Implications for LESLLA Practice Practice

Negative impacts on some ESL Negative impacts on some ESL outcomesoutcomes Some teachers expressed concern about Some teachers expressed concern about

taking 5 hours of instruction away from taking 5 hours of instruction away from ESL activities, so we could see a negative ESL activities, so we could see a negative impact on vocab or listening testsimpact on vocab or listening tests

How likely is this scenario?How likely is this scenario? What other explanations could we What other explanations could we

investigate in our analyses?investigate in our analyses?

3131

Context of LESLLA ResearchContext of LESLLA Research

The study took place in a challenging The study took place in a challenging environment that makes having/finding an environment that makes having/finding an impact difficult, but that represents realityimpact difficult, but that represents reality Low exposure and instructional time in adult Low exposure and instructional time in adult

ESL classESL class Short class durationShort class duration Limited instructional time and irregular attendanceLimited instructional time and irregular attendance

Training teachersTraining teachers Lack of specialized training in literacyLack of specialized training in literacy Short training time availableShort training time available

3232

Thank you!Thank you!

ContactsContacts lcondelli@air.orglcondelli@air.org scronen@air.orgscronen@air.org

Enjoy Banff and LESLLAEnjoy Banff and LESLLA