40
Understanding our First Years Three studies and a comparison

Understanding our First Years

  • Upload
    ossie

  • View
    28

  • Download
    0

Embed Size (px)

DESCRIPTION

Understanding our First Years. Three studies and a comparison. CARL Conference, Friday April 4, 2008, Irvine CA. Understanding our First Years. Karen Brodsky, Instruction and Outreach Coordinator, Sonoma State University - PowerPoint PPT Presentation

Citation preview

Page 1: Understanding our First Years

Understanding our First YearsThree studies and a comparison

Page 2: Understanding our First Years

Understanding our First Years

Karen Brodsky, Instruction and Outreach Coordinator, Sonoma State University

Linda J. Goff, Head of Instructional Services,California State University, Sacramento

Gabriela Sonntag, Coordinator, Information Literacy Program, California State University, San Marcos

Sharon Hamill, Professor of Psychology, California State University, San Marcos

CARL Conference, Friday April 4, 2008, Irvine CA

Page 3: Understanding our First Years

Agenda – Part One

• Overview -handouts

• Definition of iSkills

• Demographics

• Similarities and Differences in campus projects

Page 4: Understanding our First Years

Our Three Projects - Similarities

• Focused on Entering Freshman

• Each a 2 year project

• Common Goal – establish baseline under-standing of first-year students’ knowledge of information competence/literacy

• Used iSkills as pre and post test

Page 5: Understanding our First Years

Sacramento - Differences• 132 F’06 & 107in F’07 (239)

did both pre & post test • Faculty groups worked on

InfoLit lesson plans• Intervention lesson was one

class session• Student incentive of $50 gift

certificateTesting at Sacramento

Page 6: Understanding our First Years

San Marcos - Differences• Freshman Seminars and

Oral Communication• 3 weeks IL vs. 1 hour IL• Institutional data

gathered included analysis of retention factors

Page 7: Understanding our First Years

Sonoma - Differences

• Year-long class over 2 pilot years (A2, A3)

• ~ 8 sessions per class over the year

• Librarians integral in curriculum development

• Faculty Development Instruction at Sonoma

Page 8: Understanding our First Years

Agenda – Part Two

• Data review

• Test scores

• ETS reports

Page 9: Understanding our First Years

Sacramento Pre/Post-test ScoresFall2006 Fall2007

Page 10: Understanding our First Years

Sacramento Scores• 2006 Aggregate ETS reports indicate

strongest need for instruction in skill areas: Evaluate, Create and Communicate

• 2007 scores included Honors classes

• 2007 Aggregate ETS reports therefore slightly higher

Page 11: Understanding our First Years

Sac - Comparison of Average ScoresEOP – HonorsHonors- GE classes

0

50

100

150

200

250

300

2006

Pre

2006

Post

2007

Pre

2007

Post

EOP (1+1)

Honors(0+2)

GE (5+2)

Page 12: Understanding our First Years

San Marcos Pre/Post-test ScoresGEL(2006) GEO (2006)

Page 13: Understanding our First Years

San Marcos Scores• Lesson plans were not matched to test

content• All students show considerable improvement.• GEO students outperformed GEL• Students not needing remediation take GEO

in Fall

Page 14: Understanding our First Years

0

10

20

30

40

50

60

70

80

90

Pe

rce

nt

Co

rre

ct

Pre Test

Post Test

GEL pre and post test scores

Access

0

20

40

60

80

100

120

Chosecorrect

database onfirst search

Selected allof the

appropriateitems for the

customer

Used searchterms precise& useful in

Websearches

Rankedpotential

alternativesearches

accurately

Skills

% co

rrec

t Pre-Test

Post-Test

Evaluate

0

10

20

30

40

50

60

70

80

Chose bestWeb pages

as mostuseful forresearchproject

Judgedrelevance

Judgedauthority

Judgedpoint of

view

Judgedcurrency

Selectedbest sites

for researchtopic

Judged authority

Judged bias

Skill

% C

orre

ct

GEO pre and post test scores

0102030405060708090

Skill

perc

ent c

orre

ct

pre-test

post-test

Page 15: Understanding our First Years

Comparing GEL and non-GEL

                     

  Fall 2000 Entrants Fall 2001 Entrants Fall 2002 Entrants Fall 2003 Entrants Fall 2004 Entrants

Percent Number Percent Number Percent Number Percent Number Percent Number

                     

Cumulative GPA          

All GEL 101 Students 2.73 (305) 2.70 (307) 2.59 (487) 2.66 (486) 2.73 (344)

Non-GEL 101 2.75 (204) 3.14 (176) 2.73 (258) 2.73 (331) 2.73 (317)

One Year Continuation Rates          

All GEL 101 Students 72.0 (321) 69.7 (323) 76.7 (520) 76.2 (509) 71.9 (358)

All Freshmen* 60.0 (575) 62.0 (545) 70.7 (837) 71.0 (890) 69.5 (722)

Two Year Continuation Rates

All GEL 101 Students 65.1 (321) 59.8 (323) 63.5 (520) 66.2 (509) n/a n/a

All Freshmen* 51.3 (575) 52.7 (545) 58.4 (837) 60.6 (890) n/a n/a

Page 16: Understanding our First Years

Pre - Fall 2006• 103 participants (68%)• Scheduled outside class• Required for “participation

points”• Slightly above average –

except for “Define”

Post - Spring 2007• 44 students participated

– 33 students repeat• Results not statistically

relevant• Scheduled out class• Required for “participation

points”

Sonoma Year Fall 2006 pre/post test

Page 17: Understanding our First Years

Sonoma – Year 2 pre/post testFall 2007• 144 participated (84%)• Scheduled during Class• Slightly above average

- better on “Define”

Spring 2008

• No post-test

Page 18: Understanding our First Years

Sonoma – pre-test comparison

Fall 2006– Slightly lower in “Define”– Slightly higher in “Access,” and “Create”

Fall 2007– Slightly higher in “Define” and “Integrate”

Both groups equivalent for “Evaluate”

Page 19: Understanding our First Years

Agenda – Part Three• Have you assessed first

years? • Have you used the

iSkills test?• What other instruments

were used? • What were your

experiences?

Page 20: Understanding our First Years

Sacramento conclusions

Page 21: Understanding our First Years

iSkills is not based on ACRL-IL

• Lesson plan objectives based on ACRL Standards: (5.2.f- plagiarism) and (3.2.a evaluation of bias)

• Mismatch with iSkills test content – lessons and Standards

Page 22: Understanding our First Years

Problem with Data-driven Decisions

• Aggregate data available from ETS only for 100+ users

• Disconnect – class-level data not available for faculty

• Statistical significance vs. practical significance?• One lesson intervention isn’t enough

Page 23: Understanding our First Years

Faculty Feedback

“ETS needs to give us a results section that includes students weaknesses and strengths so that we can apply it to the development of lesson plans that will adequately address areas that need remediation.” RG, Sacramento

Page 24: Understanding our First Years

Outcomes (Sacramento)• Freshman Programs Director and FS

faculty have developed stronger working rapport with librarians and positive influence on information literacy program.

• Data useful for Freshman Program assessment and re-design

Page 25: Understanding our First Years

San Marcos Conclusions

Page 26: Understanding our First Years

One Year Analysis• Sample as a whole scored higher on post-test.• GEO students outperformed GEL students but

most variance accounted for by the pre-test• Only prior difference between groups was HS

GPA (not EPT, ELM, or SAT)• No difference in college GPA, units completed,

units enrolled in or likelihood of enrolling

Page 27: Understanding our First Years

Further Analysis• GEL students more likely to

– Go to the career center and math lab– Go see a professor during office hours and outside of

class• GEO students more likely to

– Have a job• No difference in going to the writing center,

visiting with an academic advisor or going to the academic advising web page

Page 28: Understanding our First Years

Sonoma Conclusions

Page 29: Understanding our First Years

iSkills - What we learned• SSU students scored near average or above in

all categories • Can’t assume entering classes will score the

same• Pre-test used to scaffold assignments, faculty

training, etc.• “Evaluation” was focus of several sessions• Other assessment confirmed some of iSkills

results – “Define”

Page 30: Understanding our First Years

iSkills - What else we learned• iSkills could be one more indicator – must

have multiple assessment tools

• Question of correlation of categories

• Pre/post test didn’t work for us

• Could be more useful tied to entire freshmen program -- not this particular class

Page 31: Understanding our First Years

Sonoma -- Outcomes• Commitment to working with first year

students.

Fall planning . . .

• What do we really want our students to learn?

• What do we really want our faculty to learn?

Page 32: Understanding our First Years

Our Combined Conclusions

Page 33: Understanding our First Years

Overall Outcomes

• We have established a baseline measure of freshman ICT/iSkill, but not of InfoLit

• All 3 Libraries have developed stronger working relationships with Freshman Programs and grants have had positive influence on information literacy programs

• Data useful for Freshman Program assessment and re-design

Page 34: Understanding our First Years

Overall Conclusions

• Programmatic data not individual or by class

• Best used as a diagnostic

• Advanced iSkills test could be given to Major in upper division

• Faculty and students have greater awareness of Information Literacy

• Timing of testing critical

• Student motivation for assessment is lacking

About iSkills About First Years

Page 35: Understanding our First Years

Overall Conclusions

• More appropriate to test through Assessment Office or Testing Center

• Multiple measures are needed• Recommitted to working with

freshmen programs

Page 36: Understanding our First Years

Agenda – Part Four

• Next steps

• Your questions

Page 37: Understanding our First Years

Sacramento -- Next Steps• Will add new First Year/Outreach

Librarian position• Will look at what worked at San Marcos

and Sonoma IL programs• Will invite other CSU’s to attend our Fall

2008 Freshman Seminar convocation, with sessions on FS/IL

Page 38: Understanding our First Years

San Marcos -- Next Steps• Reviewing our homegrown Computer

Competence Requirement exam• Continue to analyze and review multiple

sources of data• Continue to revisit our curriculum for FY

students throughout GE

Page 39: Understanding our First Years

Sonoma -- Next Steps• FYE class approved by Academic Senate

• Developing online modules for basics

• iSkills won’t be administered through the library

• Continue integrating IL with freshmen curriculum across campus

Page 40: Understanding our First Years

Understanding our First Years

Karen Brodsky, Instruction and Outreach Coordinator, Sonoma State University

Linda J. Goff, Head of Instructional Services,California State University, Sacramento

Gabriela Sonntag, Coordinator, Information Literacy Program, California State University, San Marcos

Sharon Hamill, Professor of Psychology, California State University, San Marcos

CARL Conference, Friday April 4, 2008, Irvine CA