View
219
Download
0
Category
Tags:
Preview:
Citation preview
Assessing Integrated Learning
Maureen Pettitt, Ph.D.Skagit Valley College
Beth Hartsoch, M.A.Western Washington University
PNAIRP – Oct ‘07
Session Overview Learning Communities at SVC Established Approaches
Student Writing, Course Satisfaction Survey & CCSSE
SVC/WWU Project WELS Survey & Focus Groups
Themes from the Data New Studies in Progress
Interdisciplinary Learning at SVC
Interdisciplinary courses are a means of delivering instruction and fostering student learning
A response to curricular issues: Faculty felt that students did not see connections between and among
disciplines, and needed to engage subjects more fully, to see
education as a dynamic and interconnected process of exploration and discovery
CCSSE
Student Writing
Student Satisfaction Survey
Past Approaches:Multiple Assessment Methods
1. Student Writing Study
Faculty/IR team developed two-year research project to assess student attainment of overarching General Education learning outcomes
Student Writing Study: GoalsAssess students' ability to: Apply a variety of concepts/texts/contexts
and perspectives to solving problems, thinking about issues
Connect one’s own life experience, ideas and abilities with those that others bring
Understand and value the learning process for oneself and for others
Connect to external, lifelong social issues
Student Writing Study: Method
Identified courses being taught in Learning Communities, stand-alone, and DE that could be “matched” over a two-year period
Faculty agreed to participate and give students course credit
Developed a set of questions based on Gen Ed goals and faculty input
Students surveyed beginning-, mid-, and end-course
End of Course Questions
1. Have your learning expectations been met?2. Have you learned things that you hadn't
anticipated? If so, please describe.3. What do you think are the most important
aspects of your experiences in this course that account for your learning?
4. What have you learned in this course that will matter to you five years from now?
Student Writing Study: Findings
Students’ responses regarding valuing learning, connecting classroom learning with the world, and connecting with other were fairly similar in both stand-alone and LC courses.
Students in LC courses were far more likely to cite applying or appreciating a variety of perspectives to problem-solving (55% versus 15%)
Student Writing Study: Student Comments
“After taking this course I feel that I can make connections to various things, such as history, influences, people, and culture. This course taught me the value of making connections and things from my own perspective.”
Student Writing Study: Student Comments
“By combining course topics you get the ‘bigger picture’ and are able to sort of apply what we are learning better. By applying a subject or topic to another subject or topic you have to comprehend what you are learning and apply it to other things.”
Student Writing Study: Comparisons
Comparing student responses in the stand-alone courses with students in collaborative courses—same courses with the same instructor(s): Students in stand-alone courses were
less likely to write about these connections or about learning, and
tended to focus on personal growth, liking faculty, etc.
2. Quarterly Surveys Faculty-developed pen-and-paper survey Scale from “strongly disagree” to
“strongly agree” Faculty are provided their course results
and comments, plus the cumulative for the quarter for all courses
Several years ago, we created a version that could be scanned
Learning CommunitySurvey Questions
1. "This learning community has been a valuable educational experience."
2. "I believe it was probably more valuable to have taken these classes together than it would have been to take them separately."
Mean Responses to Value Questions (N = 5156)
4 .1
3 .93 .8
3 .6
3 .33 .4
3 .5
3 .6
3 .7
3 .8
3 .9
4
4 .1
V a lu a b le M o re V a lu e T o g e t h e r
L C C o m p L C
Linked & Federated Composition LCs
(N = 2780)
4 .0
3 .63 .7
3 .4
3 .0
3 .2
3 .4
3 .6
3 .8
4 .0
V a lu a b le M o re V a lu e T o g e t h e r
L in k e d F e d e ra t e d
Student Comments LC: “This class has been fun and not
sucky at all. I think I have learned a lot.” Link: “I did not learn too much other
than how to explore more in depth, how to write more.”
LC: “If they had been separate, I would have known the what and where, but not the why, and the why is always the most important question.”
3. Community College Survey of Student Engagement
Administered the CCSSE in 2003, 2005, and 2007
Went to this survey because it most closely matched the institution’s focus on student learning and engagement
The LC question was a bonus extra!
CCSSE LC Question Stem: “Which of the following have you
done, are you doing, or do you plan to do while attending this college?”
Category: “Organized learning communities (linked courses/study groups led by faculty or counselors)”
Response Categories: I have done I plan to do I have not done nor plan to do
Method
Used T-test for independent samples to examine differences in effort and engagement between students who had taken Learning Communities and those who had not done nor planned to do.
Results are only for the latest survey administration (Spring 2007)
Learning Community
Sig.Variable HaveTaken
(n = 173 )
Not Taken
(n = 232)
Asked questions in classor contributed to classdiscussions
3.00 3.00 1.000
Made a classpresentation
2.43 2.27 .069
Prompt: “In your experiences at this college during the current school year, about how often have you done each of the following?”Responses range from 1 (“Never”) to 4 (“Very Often”)Responses range from 1 (“Never”) to 4 (“Very Often”)
Findings
Learning Community
Sig.Variable Have Taken
(n = 173 )
Not Taken
(n = 232)
Prepared two or more drafts of a paper or assignment before turning it in
2.96 2.45 .000
Worked on a paper or project that required integrating ideas or information from various sources
3.14 2.75.000
Worked with other students on projects during class
2.88 2.69 .021
Worked with classmates outside of class to prepare class assignments
2.49 2.10 .000
Discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.)
2.87 2.69 .038
Learning Community
Sig.Variable Have Taken
(n = 173 )
Not Taken
(n = 232)
Worked harder than you thought you could to meet an instructor’s standards or expectations
2.78 2.52 .003
Used email to communicate with an instructor
3.10 2.67 .000
Discussed grades or assignments with an instructor
2.76 2.58 .039
Discussed ideas from your readings or classes with instructors outside of class
2.11 1.88 .012
Worked with instructors on activities other than coursework
1.74 1.45 .000
The advantage of using multiple methods….
……you you get to get to do a do a “Reality “Reality Check” Check” on a on a periodic periodic basisbasis
What we didn’t have….
Perceptions of students after they transferred from Skagit
Collaboration with Western Washington University’s Office of Survey Research WELS (Western Educational
Longitudinal Study) Data Focus Groups
WELS Transfer Study Process began in 2001; baseline data Fall
2003 Survey items related to students’
expectations, pre-WWU experiences, college preparedness, etc.
Three separate surveys of incoming transfer students administered on-line, with phone follow-up Prior to starting classes After the first quarter After finishing three quarters
Results for SVC/Non-SVC Transfers
Percent responding “5 or
more times”
Not SVC (n = 507 )
SVC(n = 45)
Talked to a professor outside of class 54.6% 66.7%
Personally contributed to a classroom discussion
76.3% 80.0%
Volunteered on or off campus outside of coursework
20.1% 33.3%
Participated in a club or activity 16.8% 24.4%
Wrote a substantial scientific or other research paper of 5 pages or more
17.9% 17.8%
Wrote any other type of substantial paper of 5 pages or more
25.6% 4.4%
Focus Groups WWU generated list; SVC did
transcript analysis to identify student course taking, esp. Learning Communities and English Links
Identified two groups for interviews:1) a mix of none, one, or two or more LCs taken at SVC2) two or more LCs taken at SVC
Questions for all SVC transfers
What aspects of your learning experiences at Skagit Valley College have helped you in your courses at Western Washington?
If you could change anything about your learning experiences at Skagit Valley College so that you would be better prepared to succeed at Western, what would that be?
Can you talk a little bit about how Learning Community courses have helped or hindered your academic progress at Western?
If you took an English course that was linked with another course, can you describe how that learning experience has been useful or not in your studies at Western?
Additional Group Questions For the first group:
Of you who have taken more than one Learning Community or English Link, how did taking a second or third learning experience of this kind impact the development of your skills or knowledge?
For the second group: How did taking the second or third learning
experience of this kind impact the development of your skills or knowledge?
Helpful in Transition to WWU
Students say the following aspects have helped: interdisciplinary learning, analytical/critical thinking, research and writing, group work, one on one interaction with instructors, and course content in specific courses
Additional helpful aspects noted: completion of GURs, knowing people who have attended Western, and the financial benefit of attending SVC
Interdisciplinary Learning Students say that the interdisciplinary
aspect of learning communities helps to make classes more interesting and fun at SVC.
Exposure to interdisciplinary learning at SVC helps students to feel more confident in their classes at Western. Students say they have learned to make connections and to apply what they’ve learned to another topic or to the real world.
English Links Most students say English links were
useful in their studies at SVC and in transferring to Western—because English links challenged students to think and write analytically, outside of the box.
Students say English was more interesting because it was linked to a content area which provided something to write about. They felt better prepared for researching and writing analytical papers at Western.
English Links A few students say their English
link was not useful in their studies at SVC. They say the focus was more on grammar and punctuation, as it was in high school.
Research and Writing While some students say that Skagit
prepared them to think critically and write analytical papers (discussed earlier), other students say they were not prepared for writing analytical papers at Western.
Research and writing experiences in English links and learning communities at SVC helped to prepare students for research and writing assignments at Western.
Other Student Outcomes Analytical/critical thinking
The emphasis on critical thinking at SVC helps students in their classes at Western because they learned how to analyze information, make connections, and synthesize major ideas.
Group work Group work in links and learning
communities helped to prepare students for group work assignments at Western.
Challenge & Workload An interesting side note is students’
perceptions about the challenge and workload involved in learning communities and links: Some students say that learning communities
and links are more challenging because of the level of analyzing involved. For some students, there is also a greater workload than expected.
Other students say there is not a greater workload.
Faculty Coordination When instructors do not coordinate well
together, the classes seem separate—not linked. Students find that the lack of coordination between some instructors means lack of integration between courses that are supposed to be linked.
Learning communities work best for students when the instructors coordinate well together, and particularly when instructors are in the classroom teaching together.
Changes Students Would Make
Students say the biggest issue for them is academic advising—because advising directly impacts students’ learning experiences at SVC as well as their preparation for Western.
Students also say they would finish a DTA to avoid difficulties in transferring, make sure they had learning experiences in a large lecture class, and do something about disruptive students.
Future WELS Analyses Comparisons
SVC degree vs. no degree (or ranked by number of LCs taken)
SVC vs. other CCs LC comparison between CCs SVC vs. other CC vs. non-transfer
WWU third-year students
Future WELS Analyses Surveys
Baseline – administered prior to start of Fall qtr Focus = experiences at previous college &
expectations at WWU Transitions – administered at the end of Fall qtr
Focus = first quarter WWU experiences Spring Follow-Up – administered at the end of
third quarter Focus = first year experiences at WWU
Analyses Descriptives and regression
GPA Analysis – LCs Taken
0
0 .5
1
1 .5
2
2 .5
3
3 .5
E n t e r in g G P A Q P A a f t e r 2 Q t r s
N o L CO n e L CT w o + L C s
There were no significant differences between groups on either variable.
GPA Analysis - Degree
0
0 .5
1
1 .5
2
2 .5
3
3 .5
E n t e r in g G P A G P A a f t e r 2 Q t r s
N o S V C D e g re eS V C D e g re e
There were no significant difference between groups on entering GPA but there was for GPA after two quarters (p=.000)
N = 60 Entering GPA
GPA After 2
Qtrs
Sig.
Degree Completed
No 3.15 2.54 .000
Yes 3.19 3.14
LCs Completed
None 3.32 2.43
One 2.87 2.77 .362
Two or more 3.24 2.98
Convergence of the Data
Findings from surveys, focus groups, and student writing strongly suggest that taking Learning Communities results in higher levels of effort and engagement collaborate with peers interaction with faculty
Convergence of the Data
The WWU data about transfer students also suggest that students who complete Learning Communities at Skagit or the degree perform better than those who do not
However, we may need to look at the kinds and level of writing students are required to do in their courses
It’s all about perceptions…
Latest SVC Efforts…. Learning Communities Assessment
Project (Washington Center for the Improvement of Undergraduate Education) Protocol developed at Harvard (Project Zero) Rubric being tested
Developmental Learning Communities Project SVC project to research student success
based on developmental course delivery methods, including counseling-enhanced
Thanks for coming!
Recommended