WELCOME!!!

WELCOME!!!WELCOME!!!Disclosures:Disclosures:Suzanne Cashman – no financial disclosuresSuzanne Cashman – no financial disclosuresBarbara Gottlieb – no financial disclosuresBarbara Gottlieb – no financial disclosures

1

Service-LearningService-LearningCCPH Annual CCPH Annual ConferenceConference

Houston, TexasHouston, TexasApril 18, 2012April 18, 2012

Suzanne CashmanSuzanne CashmanUniversity of Massachusetts Medical SchoolUniversity of Massachusetts Medical School

Barbara GottliebBarbara GottliebBrookside Community Health CenterBrookside Community Health Center

Harvard Medical School and School of Public HealthHarvard Medical School and School of Public Health2

Today’s AgendaToday’s Agenda1:00-1:301:00-1:30 Introductions and plan for the dayIntroductions and plan for the day

Definitions – Service, Service-LearningDefinitions – Service, Service-Learning

1:30-2:101:30-2:10 PartnershipsPartnerships

2:10-2:502:10-2:50 ReflectionReflection

BREAKBREAK 5 minutes5 minutes

2:55-3:352:55-3:35 EvaluationEvaluation

3:35-4:003:35-4:00 Summary, Service-learning in Summary, Service-learning in perspective, wrap-up, feedback perspective, wrap-up, feedback

3

Learning objectivesLearning objectives►Describe the theoretical basis and key Describe the theoretical basis and key

components of service-learningcomponents of service-learning►Articulate the varied ways in which Articulate the varied ways in which

service can be viewedservice can be viewed►Apply the principles of partnership to Apply the principles of partnership to

service-learningservice-learning►Explain the key role of reflection in Explain the key role of reflection in

service-learningservice-learning

4

Learning objectivesLearning objectives►Demonstrate a variety of reflection Demonstrate a variety of reflection

modalitiesmodalities►Understand how to evaluate service-Understand how to evaluate service-

learning activities and programslearning activities and programs►Articulate the place and role of Articulate the place and role of

service-learning in the context of service-learning in the context of community-engaged scholarly community-engaged scholarly activitiesactivities

5

IntroductionsIntroductions

6

I slept and dreamt that life was I slept and dreamt that life was joy. I awoke and saw that life was joy. I awoke and saw that life was

serviceservice. I acted and behold, . I acted and behold, serviceservice was joy. Rabindranath was joy. Rabindranath

TagoreTagore

7

What is service?What is service?

What are some examples of What are some examples of service?service?

8

A type of A type of economic activityeconomic activity that is that is intangible, is not stored and does not result intangible, is not stored and does not result in ownership. A service is consumed at the in ownership. A service is consumed at the point of sale. Services are one of the two point of sale. Services are one of the two key components of economics, the other key components of economics, the other

being goods. Examples of services include being goods. Examples of services include the transfer of goods, such as the postal the transfer of goods, such as the postal service delivering mail, and the use of service delivering mail, and the use of

expertise or experience, such as a person expertise or experience, such as a person visiting a doctor.visiting a doctor.

Basically, when you engage in an unpaid Basically, when you engage in an unpaid activity intended to benefit others, that’s activity intended to benefit others, that’s

serviceservice..9

What makes service What makes service service?service?►Brainstorming exerciseBrainstorming exercise

10

What is service-learning?What is service-learning?

11

What is Service-Learning? What is Service-Learning? ► Service-learning is a structured learning Service-learning is a structured learning

experience that combines community experience that combines community service with preparation and reflection. service with preparation and reflection. Students engaged in service-learning Students engaged in service-learning provide community service in response to provide community service in response to community-identified concerns and learn community-identified concerns and learn about the context in which service is about the context in which service is provided, the connection between their provided, the connection between their service and their academic coursework, and service and their academic coursework, and their roles as citizens. (CCPH Website)their roles as citizens. (CCPH Website)

12

Service-Learning Service-Learning ►Strives to balance service and learning Strives to balance service and learning

objectivesobjectives►Addresses community concerns and Addresses community concerns and

broad determinants of health broad determinants of health ► Integrates community partners Integrates community partners ►Emphasizes reciprocal learning - Emphasizes reciprocal learning -

traditional definitions of "faculty," traditional definitions of "faculty," "teacher" and "learner" are intentionally "teacher" and "learner" are intentionally blurredblurred

13

Service-Learning Service-Learning (cont’d)(cont’d)►Emphasizes reflective practice - Emphasizes reflective practice -

fosters critical thinking and self-fosters critical thinking and self-awarenessawareness Integrates reflection at all stages of Integrates reflection at all stages of

learninglearning►Develops citizenship skills and Develops citizenship skills and

focuses on achieving social changefocuses on achieving social change Provision of health services is not often the Provision of health services is not often the

most important factor; students place their most important factor; students place their roles as health professionals and citizens in roles as health professionals and citizens in a larger societal contexta larger societal context 14

Service-Learning Service-Learning (cont’d)(cont’d)

►Focus is on partnershipsFocus is on partnerships►Quality of service-learning Quality of service-learning

opportunities develop and evolve opportunities develop and evolve over timeover time

15

Service-Learning

Field ExperienceCommunity Service

Volunteerism Practicum, Internship, Clerkship

Furco, Service-Learning. 1996

Recipient ProviderBeneficiary

What Distinguished Service-Learning from Other Forms of Experiential

Education?

Service Focus Learner

16

Distinguishing service-learning with other types of experiential learning

(CCPH, 1999)

Community Community ServiceService

AcademicAcademicLearningLearning

Purposeful Purposeful Civic learningCivic learning

Clinical Clinical TrainingTraining

MayMay YesYes Generally NotGenerally Not

Volunteer Volunteer WorkWork

YesYes NoNo NoNo

InternshipInternship MayMay YesYes NoNo

Service- Service- learninglearning

YesYes YesYes YesYes

17

Points of Departure: Points of Departure: S-L and Traditional EducationS-L and Traditional Education

► Balance between service and learningBalance between service and learning► Integral involvement of community Integral involvement of community

partners partners ► Emphasizes :Emphasizes :

Importance of addressing community-identified Importance of addressing community-identified concerns and broad determinants of healthconcerns and broad determinants of health

Strength/asset-based approachStrength/asset-based approach Reciprocal learningReciprocal learning Reflective practiceReflective practice Developing citizenship skills and achieving Developing citizenship skills and achieving

social changesocial change 18

The Yoder WheelThe Yoder Wheel

19

Evidence: HPSISNEvidence: HPSISN ► Health Professions Schools in Service to Health Professions Schools in Service to

the Nation (HPSISN) Program, 1994-98the Nation (HPSISN) Program, 1994-98 Three-year grants to 17 health professions Three-year grants to 17 health professions

schoolsschools Program goals:Program goals:

► Strengthen partnerships between health professions Strengthen partnerships between health professions schools and communitiesschools and communities

► Address unmet community needsAddress unmet community needs► Instill ethic of community service and social Instill ethic of community service and social

responsibility in health professions schools, students responsibility in health professions schools, students and facultyand faculty

► Equip next generation of health professionals with Equip next generation of health professionals with community- oriented competencies necessary to community- oriented competencies necessary to practice in a changing health care environment. practice in a changing health care environment.

20

HPSISN Partnership FocusHPSISN Partnership Focus► School-based health School-based health

educationeducation► Health promotion Health promotion

and disease and disease preventionprevention Teenage Teenage

pregnancypregnancy Domestic violenceDomestic violence Oral healthOral health

► Worksite-based Worksite-based health educationhealth education

► CompanionshipCompanionship► Case managementCase management► Mentoring and Mentoring and

tutoringtutoring► Rural access to careRural access to care

21

Findings: StudentsFindings: StudentsTransformational learning experiencesTransformational learning experiences

Clarification of values, sense of selfClarification of values, sense of selfTaken more seriously when it’s Taken more seriously when it’s

requiredrequiredGreater gains when non-clinical:Greater gains when non-clinical:

Awareness of determinants of healthAwareness of determinants of healthSensitivity to diversitySensitivity to diversityKnowledge of health policy issuesKnowledge of health policy issuesLeadership developmentLeadership development

22

Findings: Community Findings: Community PartnersPartners

► Service, economic and social benefits Service, economic and social benefits ► awareness of institutional awareness of institutional

assets/limitationsassets/limitations► High value placed on relationship with High value placed on relationship with

facultyfaculty► Eager to be seen as teachers and expertsEager to be seen as teachers and experts

Campus involvement limitedCampus involvement limited► Benefits of SL outweighed the burdensBenefits of SL outweighed the burdens► Concerns re: communication, logistics, Concerns re: communication, logistics,

needs-based and expert approachesneeds-based and expert approaches23

Findings: FacultyFindings: Faculty►Stronger relationships associated with:Stronger relationships associated with:

Joint planningJoint planning Partners being offered specific and active Partners being offered specific and active

rolesroles Genuine sense of reciprocityGenuine sense of reciprocity Student preparation and orientationStudent preparation and orientation Single point of contactSingle point of contact Consistent, accessible communicationConsistent, accessible communication

24

Findings: Institutional Findings: Institutional CapacityCapacity

Link to mission and strategic goalsLink to mission and strategic goals Clear definition of service-learningClear definition of service-learning Supportive leadership at all levelsSupportive leadership at all levels Effective institutional structures and policy Effective institutional structures and policy Investment in faculty developmentInvestment in faculty development Integration of SL into existing coursesIntegration of SL into existing courses Long-term community relationshipsLong-term community relationships Ongoing assessment and improvementOngoing assessment and improvement

25

Lessons LearnedLessons Learned► Service learning is powerful pedagogyService learning is powerful pedagogy► SL can contribute to competencies needed SL can contribute to competencies needed

for health professions practicefor health professions practice► SL can benefit students, faculty, the SL can benefit students, faculty, the

community and community-university community and community-university relationshipsrelationships

► Community can be effective educatorsCommunity can be effective educators► Community assets are often overlookedCommunity assets are often overlooked► SL requires schools to give up “control”SL requires schools to give up “control”

26

Why Service-Learning?Why Service-Learning?

27

2121stst Century Challenges for Century Challenges for EducatorsEducators

► Knowledge explosionKnowledge explosion► Cost, resource constraintsCost, resource constraints► Multiple stakeholders in educationMultiple stakeholders in education► Demand for new paradigmsDemand for new paradigms► Recognition of multiple learning styles and Recognition of multiple learning styles and

“intelligences”“intelligences”► Complex problems, but historical constraints on Complex problems, but historical constraints on

innovationinnovation► Educated consumers/publicEducated consumers/public► Disparities and inequitiesDisparities and inequities► Demand for equity and social justiceDemand for equity and social justice

28

Are we meeting these challengesAre we meeting these challenges??► Do our teaching methods …Do our teaching methods …

promote and develop the thinking skills that our promote and develop the thinking skills that our students will need to work compassionately, safely students will need to work compassionately, safely and effectively? and effectively?

speak to multiple learning styles and intelligences?speak to multiple learning styles and intelligences? provide the skills and encourage a commitment to provide the skills and encourage a commitment to

life-long learning?life-long learning? promote the exploration of values that are promote the exploration of values that are

consistent with social justice and equity?consistent with social justice and equity? promote altruism and help students learn that they promote altruism and help students learn that they

can do well by doing good?can do well by doing good? promote teamwork, collaboration, communication promote teamwork, collaboration, communication

skills?skills?29

Service-learning Service-learning vsvs traditional traditional learninglearning

TraditionalTraditionalFact-orientedFact-orientedMy-side biasMy-side biasAlgorithmicAlgorithmicGroup thinkGroup thinkStay in comfort zoneStay in comfort zoneAbility to focus may Ability to focus may

obscure the big pictureobscure the big pictureExisting knowledge may Existing knowledge may

compete with compete with acquisition of newacquisition of new

knowledgeknowledge(Ritchhart, Perkins)(Ritchhart, Perkins)

Service-learningService-learningProcess-orientedProcess-orientedDiverse perspectivesDiverse perspectivesDiverse strands of Diverse strands of

informationinformationTake risksTake risksHeightened awareness Heightened awareness

of experienceof experienceBig-picture orientationBig-picture orientationOpenness, self-Openness, self-

awareness, critical awareness, critical thinkingthinking

30

PartnershipsPartnerships

31

ActivityActivity►Using the word magnets we’ve Using the word magnets we’ve

distributed, compose the following:distributed, compose the following:• A sentence or phrase that describes your A sentence or phrase that describes your

experiences with community-campus experiences with community-campus partnershipspartnerships

• A sentence that describes your dream A sentence that describes your dream partnershippartnership

32

Why partnerships?Why partnerships? Community-campus partnerships – a Community-campus partnerships – a

strategy for social changestrategy for social change Establish missing but critical connectionsEstablish missing but critical connections Identify new/better way to solve problemsIdentify new/better way to solve problems Link complementary skills and resources of Link complementary skills and resources of

diverse people and organizationsdiverse people and organizations Plan and carry out comprehensive actions Plan and carry out comprehensive actions

that coordinate reinforcing strategies and that coordinate reinforcing strategies and systemssystems

33

Partners: Faculty/CampusPartners: Faculty/CampusFaculty & campuses involved in partnerships:Faculty & campuses involved in partnerships: Primary motivators: personal values, belief in Primary motivators: personal values, belief in

improved learning, new knowledgeimproved learning, new knowledge Enhanced relationships - students, communityEnhanced relationships - students, community Linkage of personal/professional livesLinkage of personal/professional lives understanding of community issuesunderstanding of community issues New career and scholarship directionsNew career and scholarship directions Concerns re: time, controlConcerns re: time, control

34

Partners: CommunitiesPartners: Communities

Communities involved in partnerships:Communities involved in partnerships: Service, economic and social benefits Service, economic and social benefits awareness of institutional assets/limitationsawareness of institutional assets/limitations High value placed on relationship with facultyHigh value placed on relationship with faculty Eager to be seen as teachers and expertsEager to be seen as teachers and experts Campus involvement limitedCampus involvement limited Benefits outweigh the burdensBenefits outweigh the burdens Concerns re: communication, logistics, needs-based and Concerns re: communication, logistics, needs-based and

expert approachesexpert approaches35

The Power of PartnershipsThe Power of Partnerships► Initial principles adopted 1998Initial principles adopted 1998►Revised 2005-2006 (October 2006)Revised 2005-2006 (October 2006)

Based on practice and feedbackBased on practice and feedback CCPH board discussionCCPH board discussion Community Partner Summit Community Partner Summit

(Wingspread Conference Center, (Wingspread Conference Center, Racine, Wisconsin)Racine, Wisconsin)

36

Principles of Principles of GoodGood Community- Community-Campus PartnershipsCampus Partnerships

► Partnerships form to Partnerships form to serve a specific purpose serve a specific purpose and and may take on new goals over time. may take on new goals over time.

► Partners have agreed upon Partners have agreed upon mission, values, goalsmission, values, goals, , measurable measurable outcomes outcomes and and accountabilityaccountability for the for the partnership. partnership.

► The relationship between partners is characterized by The relationship between partners is characterized by mutual mutual trust, respect, genuineness, and trust, respect, genuineness, and commitmentcommitment. .

► The partnership builds upon identified strengths and The partnership builds upon identified strengths and assets, but also works to address needs and assets, but also works to address needs and increase increase capacity of all partners. capacity of all partners.

► The partnership The partnership balances power balances power among partners and among partners and enables resources among partners to be shared. enables resources among partners to be shared.

37

Principles (cont’d)Principles (cont’d)► Partners make clear and open Partners make clear and open communicationcommunication an an

ongoing priority by striving to understand each other's ongoing priority by striving to understand each other's needs and self-interests, and developing a needs and self-interests, and developing a common common language. language.

► Principles and processes for the partnership are Principles and processes for the partnership are established with the input and agreement of all established with the input and agreement of all partners, especially for partners, especially for decision-making and conflict decision-making and conflict resolutionresolution. .

► There is There is feedback feedback among all stakeholders in the among all stakeholders in the partnership, with the goal of continuously improving the partnership, with the goal of continuously improving the partnership and its outcomes. partnership and its outcomes.

► Partners Partners share the benefits share the benefits of the partnership's of the partnership's accomplishments. accomplishments.

► Partnerships can dissolve and need to plan a process for Partnerships can dissolve and need to plan a process for closureclosure

38

Partnerships - exercisePartnerships - exercise►Working in pairs – choose one of the Working in pairs – choose one of the

principles of partnership and discussprinciples of partnership and discuss how you have seen this principle in actionhow you have seen this principle in action

►What has gone well?What has gone well?►Challenges?Challenges?

(if relevant) – what this principle might look (if relevant) – what this principle might look like early in a partnership, and how it might like early in a partnership, and how it might evolve in a more developed partnershipevolve in a more developed partnership

does this seem like a “core” principle?does this seem like a “core” principle?

39

ReflectionReflection

40

Kolb’s learning cycleKolb’s learning cycle► Inventory of Inventory of

dominant learning dominant learning stylesstyles

► Cycle of learning Cycle of learning (for all learners)(for all learners)

► Cycle of teaching Cycle of teaching (for educators)(for educators)

41

Challenges in higher Challenges in higher educationeducation

► Do our teaching methods …Do our teaching methods … promote and develop the thinking skills that our promote and develop the thinking skills that our

students will need to work compassionately, safely students will need to work compassionately, safely and effectively? and effectively?

provide the skills and encourage a commitment to provide the skills and encourage a commitment to life-long learning?life-long learning?

promote the exploration of values that are promote the exploration of values that are consistent with social justice and equity?consistent with social justice and equity?

promote altruism and help students learn that promote altruism and help students learn that they can do well by doing good?they can do well by doing good?

promote teamwork, collaboration, communication promote teamwork, collaboration, communication skills?skills? 42

How do learners learn best?How do learners learn best?► ExperientialExperiential► Deliberate ecology of learnersDeliberate ecology of learners

MentoringMentoring Reciprocal learningReciprocal learning

► Embedded learning Embedded learning GroundedGrounded Values-basedValues-based

► Developmentally appropriateDevelopmentally appropriate► Guided reflection supports all of these areasGuided reflection supports all of these areas

43

DefinitionsDefinitions

44

ReflectionReflection

Slow consideration, serious thoughtSlow consideration, serious thought

45

It is one thing to absorb a fact, to situate it alongside other facts in a It is one thing to absorb a fact, to situate it alongside other facts in a configuration, and quite another to contemplate that fact at leisure, configuration, and quite another to contemplate that fact at leisure, allowing it to declare its connectivity with other facts, its thematic allowing it to declare its connectivity with other facts, its thematic destiny, its resonance…..destiny, its resonance…..

Sven Birkerts, The Guttenberg ElegiesSven Birkerts, The Guttenberg Elegies 46

ReflectionReflection► Dewey, 1933: ‘‘active, persistent and careful Dewey, 1933: ‘‘active, persistent and careful

consideration of any belief or supposed form of consideration of any belief or supposed form of knowledge in the light of the grounds that support knowledge in the light of the grounds that support it and the further conclusion to which it tends” it and the further conclusion to which it tends”

► Bond, 1985: ‘‘a generic term for those intellectual Bond, 1985: ‘‘a generic term for those intellectual and affective activities in which individuals engage and affective activities in which individuals engage to explore their experiences in order to lead to a to explore their experiences in order to lead to a new understanding and appreciation”new understanding and appreciation”

► Moon, 1999: ‘‘a form of mental processing with a Moon, 1999: ‘‘a form of mental processing with a purpose and/or anticipated outcome that is applied purpose and/or anticipated outcome that is applied to relatively complex or unstructured ideas for to relatively complex or unstructured ideas for which there is not an obvious solutionwhich there is not an obvious solution

47

Reflection definitions - Reflection definitions - themesthemes

► Critical analysis of experienceCritical analysis of experience► Search for deeper meaningSearch for deeper meaning► Active construction of lessons and Active construction of lessons and

meaningsmeanings► Guide for future actionGuide for future action► Affective processingAffective processing► 2 dimensions (Mann, 2 dimensions (Mann, et alet al))

Iterative – new understanding leads to new Iterative – new understanding leads to new action or changed response in future practiceaction or changed response in future practice

Vertical – deepening of understanding and Vertical – deepening of understanding and analysisanalysis

48

Evidence base for Evidence base for reflectionreflection

►Commentary/essaysCommentary/essays►TestimoniesTestimonies►Descriptive studiesDescriptive studies►Studies with measured outcomesStudies with measured outcomes►Comparisons of 2 or more approaches Comparisons of 2 or more approaches

to reflectionto reflection►Comparisons of reflection vs no Comparisons of reflection vs no

reflectionreflection49

Reflection activitiesReflection activities

50

ReflectionReflection►How do you like to reflect?How do you like to reflect?►What modalities have you used in your What modalities have you used in your

lives?lives?►What modalities have you used in your What modalities have you used in your

teaching?teaching?►Do you prefer some over others? Do you prefer some over others?

Why?Why?

51

Mapping S-L ReflectionMapping S-L ReflectionBeforeBefore During During AfterAfter

AloneAlone

With With ClassmateClassmatessWith With CommunitCommunity Partnersy PartnersWith With academic academic communitcommunityy

(From: Eyler, Creating your Reflection Map)

52


AloneAlone What are my What are my concerns? What concerns? What challenges do I challenges do I anticipate? Why do I anticipate? Why do I want this want this experience?experience?

Double-entry journal Double-entry journal (personal thoughts (personal thoughts re experience/re experience/class discussion class discussion responses)responses)

Advocacy letter, Advocacy letter, letter to editor, letter to editor, personal reflections personal reflections on how I’ve changed on how I’ve changed and grownand grown

With With ClassmateClassmatessWith With CommunitCommunity Partnersy PartnersWith With academic academic communitcommunityy (From: Eyler, Creating your Reflection Map)

53


AloneAlone

With With ClassmateClassmatessWith With CommunitCommunity Partnersy PartnersWith With academic academic communitcommunityy


54

Reflection exerciseReflection exerciseScenarioScenario: Your students are going to : Your students are going to participate in a semester-long project in a participate in a semester-long project in a nursing home. They will take turns doing a nursing home. They will take turns doing a variety of activities, including leading an variety of activities, including leading an exercise class, doing arts and crafts exercise class, doing arts and crafts activities, shadowing and assisting various activities, shadowing and assisting various staff members, and simply sitting and staff members, and simply sitting and visiting with the residents in an unstructured visiting with the residents in an unstructured manner. manner.

55

Reflection exercise: using the grid, design reflection Reflection exercise: using the grid, design reflection activities for each of the boxes that would enhance activities for each of the boxes that would enhance students’ learning from their experiences in the students’ learning from their experiences in the nursing home. Try in particular to think of activities nursing home. Try in particular to think of activities that might benefit the community partners, and that might benefit the community partners, and develop the students’ understanding of their roles develop the students’ understanding of their roles as citizens.as citizens.

BeforeBefore During During AfterAfter

AloneAlone

With With ClassmatesClassmates

With With Community Community PartnersPartners

With With academic academic communitycommunity


56


AloneAlone Letter to selfLetter to self Structured journalsStructured journals Summary reflective Summary reflective essayessayAdvocacy letterAdvocacy letter

With With ClassmateClassmatess

Hopes and fears Hopes and fears discussiondiscussion

TheatreTheatreRole playRole playMixed team Mixed team discussiondiscussion

Team presentationTeam presentationCollage, mural, Collage, mural, video, photo essayvideo, photo essay

With With CommunitCommunity Partnersy Partners

Asset mappingAsset mappingPlanning w/ Planning w/ communitycommunity

Lessons learned, Lessons learned, debriefingdebriefing

Presentation to Presentation to community group, to community group, to school school administrators/administrators/leadersleaders

With With academic academic communitcommunityy

Review of theory Review of theory and anticipated and anticipated practicepractice

Ways in which Ways in which actual practice actual practice does/does not does/does not reflect theoryreflect theory

Where theory might Where theory might be amendedbe amendedRecommendationsRecommendations

e


57

Engaging with the Engaging with the reflectorreflector

58

Reflection - How to get Reflection - How to get startedstarted

►Create a culture for reflectionCreate a culture for reflection►Establish clear expectationsEstablish clear expectations►Reflection activitiesReflection activities►System for respondingSystem for responding►System for sharing with other learners System for sharing with other learners ►System for sharing with community System for sharing with community

59

Establishing a culture for Establishing a culture for reflectionreflection

► Explain learning objectivesExplain learning objectives► InspirationInspiration““I celebrate such writing and the impulse to write, the impulse I celebrate such writing and the impulse to write, the impulse

to share some transformative incident that I am privileged to to share some transformative incident that I am privileged to have witnessed. In my own writing, I often feel that I write in have witnessed. In my own writing, I often feel that I write in order to understand what I am thinking. Mysteriously, insight order to understand what I am thinking. Mysteriously, insight comes (when it does come) in the very comes (when it does come) in the very actact of writing, as if of writing, as if only by sitting with pen and pad can we snatch it out of the only by sitting with pen and pad can we snatch it out of the ether.” ether.” Abraham Verghese, forward in Abraham Verghese, forward in Narrative MattersNarrative Matters (2006) (2006)

► ModelingModeling► Feedback/gratification/rewardFeedback/gratification/reward

60

Evaluation of reflection - Evaluation of reflection - issuesissues

►StandardizationStandardization►SafetySafety►Who will see (and respond?) to written Who will see (and respond?) to written

reflections?reflections? Instructors?Instructors? Peers?Peers? Community preceptors?Community preceptors? Others? Who?Others? Who?

61

Principles of assessmentPrinciples of assessment►Linked to goals and objectivesLinked to goals and objectives

Project/activity Project/activity Learning objectives of the courseLearning objectives of the course Individual learning objectives of the studentIndividual learning objectives of the student

►Clear and transparentClear and transparent PurposePurpose

► FeedbackFeedback► Track progress toward goalsTrack progress toward goals

►Learner-centerLearner-center Appropriately timedAppropriately timed Provides basis for change/growthProvides basis for change/growth

62

Domains of assessmentDomains of assessment►ContentContent

Breadth and depthBreadth and depth►Diversity and creativityDiversity and creativity

Depends on structure/non-structure of Depends on structure/non-structure of reflection activitiesreflection activities

►Domains of learningDomains of learning DescriptiveDescriptive AnalyticAnalytic AffectiveAffective Self-exploration, developmentalSelf-exploration, developmental

63

Assessment planningAssessment planning► Establish clear goalsEstablish clear goals► Establish clear learning objectives for the courseEstablish clear learning objectives for the course► Allow students to set individual learning objectivesAllow students to set individual learning objectives► Ask – how can reflection enhance the goals and Ask – how can reflection enhance the goals and

objectives above?objectives above?► Design reflection activitiesDesign reflection activities

When? (before, during, after)When? (before, during, after) What?What?

► Schedule for reflection activities and Schedule for reflection activities and feedback/responsefeedback/response

► Communicate to studentsCommunicate to students► Communicate to community partnersCommunicate to community partners 64

Getting realGetting real

65

Making time for reflectionMaking time for reflection►ChallengesChallenges►StrategiesStrategies

66

EvaluationEvaluation

67

68

““Good evaluation” is nothing Good evaluation” is nothing more than “good thinking”more than “good thinking”

It is the systematic collection of information It is the systematic collection of information about activities, characteristics and about activities, characteristics and outcomes of programs, personnel, and outcomes of programs, personnel, and products to use to reduce uncertainties, products to use to reduce uncertainties, improve effectiveness and make decisions. improve effectiveness and make decisions.

Patton, 1997Patton, 1997

69

Why Are You Engaged in Why Are You Engaged in Evaluation?Evaluation?

►What have been some of your…What have been some of your… positive experiences?positive experiences? negative experiences?negative experiences?

70

Why EvaluateWhy Evaluate?? Reduce uncertaintiesReduce uncertainties Measure program and/or student Measure program and/or student

achievementachievement Improve effectivenessImprove effectiveness Demonstrate accountabilityDemonstrate accountability Make programmatic and curriculum Make programmatic and curriculum

decisionsdecisions Build constituencyBuild constituency

71

Why EvaluateWhy Evaluate? (cont’d)? (cont’d)

Hold people accountableHold people accountable Identify areas needing improvementIdentify areas needing improvement Use scarce resources wiselyUse scarce resources wisely Build capacityBuild capacity Influence policyInfluence policy It can be fun…and interestingIt can be fun…and interesting

72

Continuous ImprovementContinuous Improvement►EvaluationEvaluation

Provides opportunity for continuous Provides opportunity for continuous improvementimprovement

Identifies where in the course Identifies where in the course improvemenet is neededimprovemenet is needed

73

Comparison of Academic Research and Practical Evaluation

Academic Research

Practical Evaluation

Purpose Test hypotheses Improve program/ practice

Method Controlled environment

Context sensitive

Statistics Sophisticated Simpler

74

Evaluation should:Evaluation should:►Strengthen projectsStrengthen projects►Use multiple approachesUse multiple approaches►Address real issuesAddress real issues►Create a participatory processCreate a participatory process►Allow for flexibilityAllow for flexibility►Build capacityBuild capacity

WKKellogg Foundation, 1998WKKellogg Foundation, 1998

75

Guiding Principles

Ongoing processMeans, not an endCollaboration based on trust and respectSensitive to cultural, socioeconomic, ethnic,

lifestyle, life-span pluralisms Allow for combination of methodologiesRemain flexibleBuild capacity

76

Planning An EvaluationPlanning An EvaluationDevelop a logic modelDevelop a logic modelEnumerate stakeholdersEnumerate stakeholdersCreate an evaluation planCreate an evaluation plan

Decide on methods, strategiesDecide on methods, strategies Specify data collection approaches Specify data collection approaches

and indicatorsand indicatorsDevelop dissemination planDevelop dissemination plan

77

Logic ModelsLogic Models►WHAT?WHAT?

Systematic and visual way to present and Systematic and visual way to present and share your understanding of the share your understanding of the relationship among resources, activities, relationship among resources, activities, changes/resultschanges/results

►WHY?WHY? Provide stakeholders with road map Provide stakeholders with road map

describing sequence of related events describing sequence of related events connecting need with resultsconnecting need with results

Students

CommunityPartners

Faculty

ServiceRecipients

PLANNING COMPONENT• Funding

• Risk management• Service-learning objectives

•Protocols• Plans for research

and reporting•Principles of Partnership

• Reflection

TEACHING COMPONENT FOR

STUDENTS• Geographic community

•Agency mission / function• Service recipients

• Clinical and interpersonal skills• Role and responsibility

• Risk management• Reflection

ORIENTATIONCOMPONENTFOR AGENCY

• Student role & responsibilities•Agency role & responsibilities

• Student clinical skills &time commitment

• Risk management•Reflection

Inputs

Service-LearningProgram

OutcomesOutputs

Service and Learning

Service report anddissemination

Assessment and evaluationof student learning

Impact of reflectionon program revisions

Publication inprofessional journal

CQI implementationand work toward

sustainability

Evaluation -- Continuous Quality Improvement (CQI) -- Reflection

Logic model for service-learning curriculum (K.Yoder, Ind University)

Health policy activism

79

Logic Model

INPUTS/RESOURCESThe resources required for this program to operate.

ACTIVITIESStrategies you will use or services you will provide to try to achieve your goal.

OUTPUTSBasic data on program, product, or service

OUTCOMESDesired changes in the population as a result of the program activities.

IMPACTChanges that can be expected in organizations, communities, or systems

80

Logic Model

INPUTS/RESOURCESThe resources required for this program to operate.

ACTIVITIESStrategies you will use or services you will provide to try to achieve your goal.

OUTPUTSBasic data on program, product, or service

OUTCOMESDesired changes in the population as a result of the program activities.

IMPACTChanges that can be expected in organizations, communities, or systems

StaffTimeMoneyMaterialsEquipmentTechnologyPartners

Meet with youth group directorsMeet with youthObtain relevant cultural materialsDevelop and implement culturally relevant dating violence prevention program

Collect baseline dataUse data to inform culturally relevant violence prevention programEvaluate programShare results

Youth show increased recognition of dating violenceYouth increase skills to help others in abusive relationshipsYouth report attitudes unaccepting of dating violence

Decreased gender stereotypingReduced rates of dating violence

81

StakeholdersStakeholders► People who have a “stake” in what will be People who have a “stake” in what will be

learned from an evaluation and what will be learned from an evaluation and what will be done with the knowledgedone with the knowledge

► They include:They include: People who manage or work in the People who manage or work in the

program/organizationprogram/organization People who are served or affected by the program, People who are served or affected by the program,

or who work in partnership with the programor who work in partnership with the program People who are in a position to do or to decide People who are in a position to do or to decide

something about the programsomething about the program Who else…?Who else…?

82

StakeholdersStakeholders►Stakeholders’ information needs and Stakeholders’ information needs and

intended uses focus the evaluationintended uses focus the evaluation►Variety of stakeholders may mean:Variety of stakeholders may mean:

more than one focus (policy implications more than one focus (policy implications vs documentation of local activities)vs documentation of local activities)

varied levels of involvementvaried levels of involvement

83

StakeholdersStakeholders►Who are your stakeholders?Who are your stakeholders?►What do they want and need to know?What do they want and need to know?►How will you make sure that they have How will you make sure that they have

data they want and information they data they want and information they need to know?need to know?

84

Methods, Strategies: Thinking Methods, Strategies: Thinking about dataabout data

►Match the data to the questions – what Match the data to the questions – what kinds of information would be worthwhile?kinds of information would be worthwhile?

►As much as possible, use data that are As much as possible, use data that are being created as a regular part of the being created as a regular part of the programprogram

►Collect and analyze data from multiple Collect and analyze data from multiple perspectivesperspectives

►Keep available resources in mindKeep available resources in mind

85

DataData

►Where might we find them?Where might we find them?►How might we obtain them?How might we obtain them?►What types should we consider?What types should we consider?

Quantitative and qualitativeQuantitative and qualitative►What do we do now that we have What do we do now that we have

them?them?

86

Who can help us collect and Who can help us collect and make sense of data?make sense of data?

Community partnersCommunity partners Student participantsStudent participants College administrative officesCollege administrative offices Faculty colleagues (and their students)Faculty colleagues (and their students) Students who participated in previous Students who participated in previous

programsprograms Campus service-learning centersCampus service-learning centers

87

Outcome IndicatorsOutcome Indicators►The specific, measurable The specific, measurable

characteristics or changes that characteristics or changes that represent achievement of an outcomerepresent achievement of an outcome

► Indicators are measurable and Indicators are measurable and observable and answer the question: observable and answer the question: “How will we know?”“How will we know?”

88

Program Outcome IndicatorsProgram Outcome Indicators► Short term OutcomesShort term Outcomes

Youth show increased Youth show increased recognition of dating recognition of dating violence and healthy violence and healthy relationshipsrelationships

Youth increase their Youth increase their anger management anger management skillsskills

Youth increase their Youth increase their communication skillscommunication skills

► IndicatorsIndicators Pre/post attitude and Pre/post attitude and

knowledge surveyknowledge survey

Police and school Police and school reports reports

Teacher reportsTeacher reports

Learner Outcome IndicatorsLearner Outcome Indicators► Short-term OutcomesShort-term Outcomes

Learners report/Learners report/ demonstrate greater demonstrate greater

cultural humilitycultural humility--Learners demonstrate--Learners demonstrate increased advocacy increased advocacy

skillsskills--Learners link theory to --Learners link theory to

actionaction

► IndicatorsIndicators Pre/post survey/Pre/post survey/ community faculty community faculty

observationsobservations--Letter to the editor --Letter to the editor

pre and postpre and post

--Reflective journals--Reflective journals

90

Creating an Evaluation PlanWhat do we want to know?

What will we measure?

How will we measure?

Who will be responsible? When?

How will we use results?

91

Creating an Evaluation PlanWhat do we want to know?

What will we measure?

How will we measure?

Who will be responsible? When?

How will we use results?

Impact on clients/service users

Client/Service user attitudes

Pre and post survey

EvaluatorBeginning of course and at the end

Demonstrate power and influence of student involvement in reaching program goals

92

Closing the LoopClosing the LoopCollaborate with stakeholders for Collaborate with stakeholders for

meaningfulmeaningful::Communication of results (process and Communication of results (process and

outcome)outcome)Decisions based on resultsDecisions based on resultsNew assessment plans emerging from New assessment plans emerging from

resultsresultsReflection on the assessment processReflection on the assessment process

Summary and wrap upSummary and wrap up►Where are you now? Where do you Where are you now? Where do you

want to go?want to go?►Where does service-learning fit into Where does service-learning fit into

the full spectrum of community-the full spectrum of community-engaged scholarship?engaged scholarship?

►Final reflectionsFinal reflections►FeedbackFeedback

93

Using with the Yoder Wheel…Using with the Yoder Wheel…

94

Categories of the Service-Categories of the Service-Learning JourneyLearning Journey

PartnershipsPartnerships► SustainedSustained► S-L objectivesS-L objectives► Broad preparationBroad preparation► Apply principlesApply principles► EvaluationEvaluationResourcesResources► FundingFunding► SpaceSpace► StaffStaffCurriculumCurriculum► Service objectivesService objectives► Learning objectivesLearning objectives► S-L objectivesS-L objectives► Guided reflectionGuided reflection► EvaluationEvaluation

95

Categories of the Service-Categories of the Service-Learning JourneyLearning Journey (continued) (continued)

ScholarshipScholarship► Community engagedCommunity engaged► Parity Parity Faculty and staff recognitionFaculty and staff recognition► Protected timeProtected time► Promotion/tenurePromotion/tenureCommunity benefitCommunity benefit► CapacityCapacity► ResourcesResources► Improved health outcomesImproved health outcomes

96

Stages in the journeyStages in the journey1 Precontemplative1 Precontemplative

2 Contemplative2 Contemplative

3 Planning3 Planning

4 Early stage implementation4 Early stage implementation

5 Mid-stage implementation5 Mid-stage implementation

6 Well-established6 Well-established

7 Fully institutionalized (locally)7 Fully institutionalized (locally)

8 National/international recognition8 National/international recognition97

Where am I now? Where do I want to Where am I now? Where do I want to be?be?

►Working in small groups….Working in small groups…. Briefly catalogue strengths, resources, Briefly catalogue strengths, resources,

challengeschallenges Look at the service-learning journey and Look at the service-learning journey and

see where you are, where your institution see where you are, where your institution is, for several of the spokes of the Yoder is, for several of the spokes of the Yoder wheelwheel

Think of 2 action steps you can take when Think of 2 action steps you can take when you returnyou return

98

Community-engaged Community-engaged scholarshipscholarship

Where does service-learning fit in?Where does service-learning fit in?

CBPR Community based

research

Volunteerprojects

Community Service

Service-Learning

99

CBPR Volunteer projects

Community service

Community based

research

Final reflectionsFinal reflections

100

FeedbackFeedback

101

Selected referencesSelected referencesFurco A. Service-Learning: A Balanced Approach to Furco A. Service-Learning: A Balanced Approach to

Experiential Education Experiential Education Expanding Boundaries Expanding Boundaries 1996.1996.Eyler J, Giles D, Stenson C, Gray C. At a Glance: What Eyler J, Giles D, Stenson C, Gray C. At a Glance: What

We Know about the Effects of Service-Learning We Know about the Effects of Service-Learning 1993-2000. Third Edition. Corporation for National 1993-2000. Third Edition. Corporation for National Service.Service.

Seifer S. Service-learning: Community-Campus Seifer S. Service-learning: Community-Campus Partnerships for Health Professions Education Partnerships for Health Professions Education Acad Acad MedMed 1998;73(3):273-7. 1998;73(3):273-7.

Yoder K. A Framework for Service-Learning in Dental Yoder K. A Framework for Service-Learning in Dental Education. Education. Journal of Dental EducationJournal of Dental Education 2006;70(2):115-122. 2006;70(2):115-122.

http://www.ccph.info/http://www.ccph.info/http://depts.washington.edu/ccph/http://depts.washington.edu/ccph/

servicelearningres.htmlservicelearningres.html102

103

references …references …► Centers for Disease Control and Prevention. Centers for Disease Control and Prevention.

Practical Evaluation of Public Health ProgramsPractical Evaluation of Public Health Programs. . 1998.1998.

► Patton M. Patton M. Utilization-Focused EvaluationUtilization-Focused Evaluation. Sage . Sage Publications. 1997.Publications. 1997.

► WKKellogg Foundation. WKKellogg Foundation. Evaluation HandbookEvaluation Handbook. 1998.. 1998.► WKKellogg Foundation. Logic Model Development WKKellogg Foundation. Logic Model Development

Guide. 2001.Guide. 2001.► http://depts.washington.edu/ccph/http://depts.washington.edu/ccph/►American Evaluation Association American Evaluation Association

www.eval.orgwww.eval.org

references …references …► Smith CS, Irby DM. The roles of experience and Smith CS, Irby DM. The roles of experience and

reflection in ambulatory care education. Acad reflection in ambulatory care education. Acad Med (1997); 72: 32-35.Med (1997); 72: 32-35.

► Mann K, Gordon J, MacLeod A. Reflection and Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions reflective practice in health professions education: a systematic review. Adv in Health education: a systematic review. Adv in Health Sci Educ (2009) 14:595–621.Sci Educ (2009) 14:595–621.

► Boenink, A, Oderwald A, de Jonge P, van Tilburg Boenink, A, Oderwald A, de Jonge P, van Tilburg W, Smal J. (2004). Assessing student reflection W, Smal J. (2004). Assessing student reflection inmedical practice. The development of an inmedical practice. The development of an observer-rated instrument: Reliability, validity observer-rated instrument: Reliability, validity and initial experiences. Medical Education and initial experiences. Medical Education (2004); 38:368-377.(2004); 38:368-377.

referencesreferences► Glaze J. Reflection as a transforming process: Glaze J. Reflection as a transforming process:

students advanced nurse practitioners’ students advanced nurse practitioners’ experiences of developing reflective skills as experiences of developing reflective skills as part of an MSc programme. J of Adv Nursing part of an MSc programme. J of Adv Nursing (2001); 34(5): 639-647.(2001); 34(5): 639-647.

► Leung D, Kember D. The relationship between Leung D, Kember D. The relationship between approaches to learning and reflection upon approaches to learning and reflection upon practice. Educational Psychology (2003); 23: practice. Educational Psychology (2003); 23: 61–71.61–71.

► Williams R, Sundelin O, Foster-Seargeant E, Williams R, Sundelin O, Foster-Seargeant E, Norman G. (2000). Assessing the reliability of Norman G. (2000). Assessing the reliability of grading reflective journal writing. J of Physical grading reflective journal writing. J of Physical Therapy Education (2000); 14: 23–26.Therapy Education (2000); 14: 23–26.

Documents

WELCOME!!!