145
Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations

Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

Evaluating the Wesley Mission In charge of my money

financial literacy program for vulnerable populations

Page 2: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

This publication was made possible by a grant from Financial Literacy Australia

How to cite this publication:

Zanoni, L., Warburton, W., Russell, R., Warburton, M., & Flynn, L. (2016). Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

Team involved in this project:

Dr Wayne Warburton, Senior Lecturer, Department of Psychology, Macquarie University

Ms Maisie Warburton, Program Manager Financial Literacy Education, Wesley Training

Ms Lynne Flynn, Financial Literacy Educator and Financial Counsellor, Wesley Financial Literacy Program, Wesley Training

Professor Roslyn Russell, School of Economics, Finance and Marketing, College of Business RMIT University

Dr Lee Zanoni, Researcher, Department of Psychology, Macquarie University

Published in September 2016 by Wesley Mission, Sydney, Australia

© Copyright Wesley Mission, Macquarie University, RMIT University 2016 www.wesleymission.org.au 1800 676 039

Page 3: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

3

Table of Contents

ExecutiveSummary........................................................................................................................................8

KeyFindings...............................................................................................................................................9

Istheprogramcontentrelevantandevidence-based?.........................................................................9

Doestheprogramfacilitatesustainedimprovementinfinancialbehaviours?...................................10

Howmuchknowledgeandunderstandingdidparticipantsretainseveralmonthsaftertheprogram?.............................................................................................................................................................11

Didtextmessageremindersincreasethechanceofsustainedchangeinfinancialbehaviours?.......11

Conclusion............................................................................................................................................12

Introduction.................................................................................................................................................13

ProgramEvaluationBackground..............................................................................................................13

ProgramEvaluationDescription&Aims..................................................................................................13

LiteratureReview.........................................................................................................................................16

Bestpracticeforfinancialliteracyprogramstargetedatvulnerablepopulations...................................16

Istheprogramcontentrelevantandevidence-based?...........................................................................22

Deliveredbytherightpeople..............................................................................................................24

Deliveredattherighttime...................................................................................................................25

Adheretoprinciplesofadultlearning.................................................................................................25

Empowerment&self-efficacy..............................................................................................................27

Conclusion............................................................................................................................................28

UsingTextMessagestoSupportImprovementsinFinancialLiteracy.....................................................28

Conclusion............................................................................................................................................38

TheInChargeofMyMoneyProgram..........................................................................................................39

ProgramBackground................................................................................................................................39

ProgramDescription................................................................................................................................42

InChargeofMyMoneyProgramEvaluation...............................................................................................46

GoalsoftheStudy,KeyFactors&OutcomeMeasures...........................................................................46

ParticipantDemographics........................................................................................................................48

ProgramEvaluationResearchMethodology................................................................................................53

Procedure.................................................................................................................................................53

Attheendoftheprogram(Time1).....................................................................................................53

Several(5-7)monthsaftertheprogram(Time2)................................................................................54

Questionnaires.........................................................................................................................................57

Page 4: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

4

DataAnalysis............................................................................................................................................59

ProgramEvaluationResults.........................................................................................................................60

ParticipantEngagementwiththeProgram..............................................................................................60

Reasonforattendingtheprogram......................................................................................................60

Mainmoneyissue................................................................................................................................61

Levelofmotivation..............................................................................................................................62

Programattendance............................................................................................................................62

ParticipantFeedbackontheProgram......................................................................................................63

Programratingattheendoftheprogram..........................................................................................63

Recommendingtheprogramtofriends...............................................................................................65

Suggestedareasforimprovement.......................................................................................................68

WhatParticipantsRemembered5-7MonthsAftertheProgram............................................................70

ImprovementinFinancialBehaviours......................................................................................................76

Wasthereevidenceofimprovedfinancialbehavioursforprogramparticipantsimmediatelyafterand5monthsaftertheprogram?........................................................................................................78

Whatdothestatisticsshow?...................................................................................................................79

IdentifyingandAchievingFinancialGoals................................................................................................80

Settinggoals.........................................................................................................................................80

Whatwereparticipant’sgoals?...........................................................................................................81

Didtheprogramhelpparticipantsachievetheirfinancialgoals?........................................................82

Whathelpedparticipantsachievetheirgoalsthemost?....................................................................82

Towhatextentdidparticipantsachievethespecificgoaltheyidentifiedduringtheprogram?........86

ImprovementinFinancialKnowledge,Attitudes,Self-EfficacyandEmotionalSpending........................88

Wasthereevidenceofimprovedgeneralandpersonalfinancialknowledgeforprogramparticipantsimmediatelyafterandfive-sevenmonthsaftertheprogram?..........................................................89

Wasthereevidenceofimprovedfinancialattitudesandfinancialself-efficacy,andlessemotionalspending,forprogramparticipantsimmediatelyafterandfivemonthsaftertheprogram?.............90

Participants’FinancialStrain....................................................................................................................94

Wasthereevidenceoffewerproblemsduetoashortageofmoneyfrombeforetheprogramto5-7monthslater?.......................................................................................................................................94

Wasthereevidenceofreduceddebt5-7monthsaftertheprogram?................................................96

Didparticipantsfeellessstressedabouttheirfuturesincedoingthisprogram?...............................96

Wastheregreaterfinancialinclusionforparticipantsseveralmonthsaftertheprogram?................97

Didmoreparticipantshavemoneysetasideforemergencies?..........................................................99

Wasthereachangeinparticipants’financialresilience?....................................................................99

Page 5: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

5

Wasthereimprovementinotherindicatorsoffinancialstrain?.......................................................101

ParticipantFactorsMostRelatedtoImprovementsinFinancialBehavioursandStrain.......................104

Improvementinattitude...................................................................................................................105

Improvementinfinancialself-efficacy...............................................................................................105

Programengagement........................................................................................................................106

General&personalfinancialknowledge...........................................................................................106

Emotionalspending...........................................................................................................................107

Participantfactorsnotassociatedwithimprovementinfinancialbehavioursorfinancialstrain.....107

Themostpredictiveparticipantfactors.............................................................................................108

TheEffectivenessofTextMessageNudges...............................................................................................110

Methodology..........................................................................................................................................110

Participants........................................................................................................................................110

Procedure...........................................................................................................................................111

Textmessagedelivery........................................................................................................................111

Textmessagewording.......................................................................................................................111

Measures...........................................................................................................................................113

Examiningtheeffectsoftextmessagesonchangeinbehaviours.....................................................114

TextMessagingResults..........................................................................................................................115

Whatthestatisticsshow........................................................................................................................117

Pearson’scorrelations.......................................................................................................................120

SummaryofResultsandImplications........................................................................................................121

Istheprogramcontentrelevantandevidence-based?.........................................................................121

Isthereevidencethattheprogramfacilitatessustainedimprovementsinfinancialbehaviours?.......126

Isthereevidenceofknowledgeandunderstandingretentionafterseveralmonths?..........................126

Didtextmessagenudgesincreasethechanceofsustainedchangeinfinancialbehaviours?...............127

AdditionalFindings:ImprovementsinAttitudes,Self-Efficacy,EmotionalSpending,andfinancialStrain...............................................................................................................................................................128

AdditionalFindings:ParticipantFactorsThatWereNotAssociatedwithOutcomes............................130

Discussion...................................................................................................................................................132

LimitationsoftheStudy.........................................................................................................................132

Futureresearch......................................................................................................................................133

Conclusion..............................................................................................................................................134

References..................................................................................................................................................135

Appendices.................................................................................................................................................139

Appendix1.............................................................................................................................................139

Page 6: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

6

Appendix2.............................................................................................................................................140

Appendix3.............................................................................................................................................141

Appendix4.............................................................................................................................................142

Appendix5.............................................................................................................................................143

Appendix6.............................................................................................................................................144

ListofTables

Table1:Participantdemographics...............................................................................................................49Table2:Reasonswhyparticipantsattendedtheprogram..........................................................................61Table3:Numberofworkshopsattendedbyparticipants(dosage).............................................................63Table4:Whichpartsoftheprogramneedtobeimproved?.......................................................................68Table5:Evidenceofimprovedfinancialbehaviours5monthspostprogram.............................................78Table6:Wastheresignificantimprovementovertime?.............................................................................89Table7:Wastheresignificantimprovementinparticipants'financialattitudesovertime?......................91Table8:Wastheresignificantimprovementinparticipants'financialself-efficacyovertime?..................92Table9:Wastheresignificantimprovementinparticipants'emotionalspendingovertime?...................93Table10:Wasthereasignificantreductioninthenumberofissuesduetoshortageofmoneyovertime?......................................................................................................................................................................95Table11:Wastheresignificantimprovementinfinancialinclusionovertime?.........................................98Table12:Wastheresignificantimprovementsseeminindicatorsoffinancialstrain?.............................103Table13:Theeffectsoftextmessagesonchangeinbehaviour................................................................115

ListofFigures

Figure1:Knowit,understandit,doit,reviewitmodeloflearning.............................................................44Figure2:Ageofparticipants........................................................................................................................50Figure3:Participants'highestlevelofeducation........................................................................................51Figure4:Annualincomeofparticipants......................................................................................................51Figure5:Issuesaddressedbycentrewhereprogramwasdelivered..........................................................52Figure6:Whatwasthemainmoneyissueyouhopedthisprogramwouldhelp?......................................61Figure7:Participants'levelofmotivationduringtheprogram...................................................................62Figure8:Outof10,howwouldyouratethisprogram?..............................................................................63Figure9:Whydidyougivetheprogramthatrating?..................................................................................65Figure10:It'sbeen5monthssinceyoudidtheInChargeofMyMoneyprogram.Whatdoyourememberfromtheprogram?.......................................................................................................................................71Figure11:Changestomoneybehavioursacrossthethreemeasuredtimepoints.....................................79Figure12:Participantgoalsettingbeforeandaftertheprogram...............................................................80Figure13:Howmuchdidthisprogramhelpyouachieveyourfinancialgoals?..........................................82Figure14:Whatpartoftheprogramhelpedyoutoachieveyourgoalthemost?......................................83Figure15:Onascalefrom1-10,towhatextenthaveyoumadeprogresstowardsthefinancialgoalthatyouhadattheendoftheprogram?............................................................................................................87Figure16:Levelsoftotal(combinedgeneralandpersonal)financialknowledgeacrossthethreemeasuredtimepoints..................................................................................................................................90

Page 7: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

7

Figure17:Inthepast5months,didanyofthefollowinghappentoyoubecauseofashortageofmoney?......................................................................................................................................................................95Figure18:Overthepastfewmonths,hasyourlevelofdebtincreased,decreasedorstayedthesame?..96Figure19:Doyoufeellessstressedaboutthefuturesincedoingthisprogram?.......................................97Figure20:Doyouhaveanyofthefollowingtypesoffinanceproducts?....................................................98Figure21:Ifyouhad1weektoraise$1000foranunexpectedexpense,howwouldyouraisethemoney?....................................................................................................................................................................101Figure22:Ifyouwerereceivingregularremindertextmessages,howusefuldidyoufindthetextmessage?....................................................................................................................................................114Figure23:Changestomoneybehavioursovertimebytextmessage'nudge'group................................116Figure24:Changestoemotionalspendingovertimebytextmessage'nudge'group..............................116

Page 8: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

8

Executive Summary

InresponsetoWesleyMission’s2010thirdfinancialstressreport(WesleyMission,2010),

WesleyMissiondevelopedandlauncheditsfinancialliteracyProgramInChargeofMy

Money,targetedatvulnerablepopulations,in2011.Thisreportdetailsthefirstevaluation

oftheInChargeofMyMoneyprogram,whichwasconductedbyMacquarieUniversityand

fundedbyFinancialLiteracyAustralia.Inadditiontoexaminingtheeffectivenessofthe

program,thisresearchalsoexploredtheusefulnessoftextmessagereminders(‘nudges’)

aftertheprogram.

Thefourmaingoalsoftheprojectwereto:

- Ensuretheprogramcontentisrelevantandevidence-based;

- Identifywhethertheprogramfacilitatessustainedimprovementinfinancial

behaviours;

- Estimatechangestoparticipantfinancialknowledgeandunderstandingattheendof

theprogram,andestimatetheirretentionafterseveralmonths;

- Ascertainhowincorporatingapracticalstrategy,suchastextmessagereminders,

increasesthechanceofsustainedchange.

Thisprogramevaluationemployedamixedmethodapproach,collectingbothquantitative

andqualitativedata.Participantsweresurveyedtwice-Time1wasimmediatelyafterthe

programandTime2wasapproximatelyfivetosevenmonthslater.TheTime1sample

included113participantsandtheTime2sampleincluded54participants.Studyparticipants

weredrawnfrom11differentnot-for-profitservicesinSydneyandsurroundingregions

addressingdrug,alcoholandgamblingaddictions,homelessness,orfamilydysfunction.

Theseparticipants,onaverage,hadverylowincomes,withgovernmentsupportbeingthe

mainsourceofincomefor85%.Textmessagesaffirmingpositivemoneybehaviourswere

sentevery3-4weekstohalftheparticipantstotestifreminders(a‘nudge’intervention)

hadanimpactonthefinancialbehavioursofparticipants.

Page 9: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

9

Key Findings

Is the program content relevant and evidence-based?

ThisevaluationfoundbothquantitativeandqualitativeevidencethattheInChargeofMy

Moneyprogramcontentanddeliveryisrelevantforsomeofthemostsociallyand

economicallydisadvantagedandvulnerablepeoplewithintheAustralianpopulation.More

specifically:

• Immediatelyaftertheprogram,participantsratedtheInChargeofMyMoney

programveryhighlyandhadfewcomplaints.

• Whenaskedseveralmonthslateriftheywouldrecommendtheprogramtotheir

friends,themajority(85%)saidtheywouldrecommenditbecauseitwaseasyto

understand,relevant,andhelpful.Thissuggeststhattheparticipants’highopinionof

theusefulnessandrelevanceoftheprogramimmediatelyaftertheprogramdidnot

declineovertime.

Theevaluationalsofoundthattheprogramisevidence-based,conformingtowhathasbeen

identifiedasbestpracticeintheliteraturetodate.Tentativebestpracticeforfinancial

literacyprogramstargetedatvulnerablepopulationsinvolvestailoringthecontentand

deliverystyletotheparticularaudience,havingtheprogramdeliveredbytrainerswhoare

knowledgeableandnon-judgemental,providingtheprogramatanopportunetime,

adheringtotheprinciplesofadultlearning,andempoweringprogramattendees.TheIn

ChargeofMyMoneyprogramisconsistentwithmostoftheseguidelinesforbestpractice:

• ThecontentanddeliverystyleoftheprogramistailoredtotheneedsofWesley

Missionclients.

• Appropriatepracticalexamplesareusedduringthepresentation,andinformationis

oftenpresentedaseasytoremember“rulesofthumb”.

• Thereisanemphasisonteachinggoodmoneymanagementskillsandhabits,rather

thanjustimpartingknowledge.

Page 10: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

10

• ThelanguageoftheprogrammaterialsissuitableforYear10levelofliteracy,to

ensureitisunderstandablebythosewithrelativelylowlevelsofliteracy.

• InthisstudytheInChargeofMyMoneyprogramwasdeliveredbywell-trained,very

financiallyliterate,non-judgementalstaff.Feedbackfromstudyparticipantsshows

thattheyfoundthepresenterstobeknowledgeable,abletoanswertheirquestions,

helpfulandempathic.

• Theprogramisdeliveredwhilstparticipantsareengagedinaservice(e.g.seeking

employmentorhousingorotherservices),thusmakingthemostofwhatcouldbe

considered‘teachablemoments’intheirlife.

• TheInChargeofMyMoneyprogramincorporatedtheprinciplesofadulteducation

intoitsdesign,givingprogramattendeesguidanceratherthaninstruction,

encouragingquestionsanddiscussion,andincludingworksheetstoactivelyengage

attendees.

Does the program facilitate sustained improvement in financial

behaviours?

TherewasevidencethattheInChargeofMyMoneyprogramfacilitatedsustained

improvementsinfinancialbehaviours.Statisticalanalysesrevealedasubstantial

improvementinfinancialbehavioursfrombeforetheprogramtoseveralmonthsafterthe

program.Therewassustainedimprovementoverseveralmonthsin:

• savingbehaviour;

• planningbehaviour;

• monitoringofexpenses;and,

• othermoneybehaviours(e.g.comparingpricestogetthebestdeal,notbuying

thingsifunabletoaffordthem).

Togiveoneexample,overhalfofstudyparticipantssaidthattheprogramhelpedthem

achievetheirfinancialgoals‘alot’or‘verymuch’,andonly11%saiditdidnothelpatall,

indicatingthatmostparticipantsbelievedthattheprogramassistedtheminpracticalways

severalmonthsaftertheprogram.

Page 11: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

11

How much knowledge and understanding did participants retain

several months after the program?

Therewasevidenceofretentionofknowledgeandunderstandingseveralmonthsafterthe

program.Infact,averylarge,statisticallysignificantimprovementwasfoundinboth

generalfinancialknowledgeandpersonalfinancialknowledge(knowledgeaboutone’sown

financialcircumstances)frombeforetheprogramtoimmediatelyaftertheprogram.These

gainswerestillapparent5-7monthsaftertheprogram,demonstratingthatstudy

participantsretainedmostoftheknowledgetheyhadlearntintheprogramoverseveral

months.

Whenaskedtodescribewhattheyrememberedabouttheprogram5-7monthslater:

• asubstantialproportionofparticipantsrememberedspecificcatchphrases,suchas

“payyourselffirst”;

• justoveraquarterofparticipantsrecalledinformationaboutthedifferenttypesof

loansandhowtomanagedebt;

• approximatelyonequarterrememberedspecificadviceaboutbeingmindfulof

spendinghabits;and,

• onlysixparticipantsreportedrememberinglittleornothingfromtheprogram.

Consideringthatover50%ofparticipantswerelivinginresidentialrehabilitationcentresat

thetimeofcompletingtheprogram,thishighlevelofknowledgeretentionfromthe

programisquiteimpressive.

Did text message reminders increase the chance of sustained change

in financial behaviours?

Sendingeffectivetextmessagenudgesinthispopulationwaschallengingbecausesome

participantsdidnothavereliablemobilephoneaccess.Duringthetrialonly65%oftext

Page 12: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

12

messageswereactuallyreceivedbyparticipantsinthe‘nudge’group.Threeparticipants

optedout.Thisreducedthe‘dosage’oftheinterventionforsomeparticipantsandresulted

inaslightlysmallerinterventiongroup.

Despitetheseissues,whichwouldhavetendedtoreducetheimpactofthetextmessage

intervention,textmessagenudgesstillhadasignificantandpositiveimpactonparticipants’

financialbehaviours(i.e.,onmoneybehaviour,saving,monitoringofexpensesand

emotionalspending).However,thesebenefitsdidnotextendtoalloutcomes.Textmessage

nudgesdidnothaveanyeffectonparticipants’financialstrain,orhelpparticipantsachieve

theirgoalsorreducetheirdebt.Giventhedifficultiesencounteredusingthenudge

interventioninthispopulation,furtherstudiesshouldattempttoreplicatetheseresultsina

larger,similarpopulation.

Conclusion

ThisprogramevaluationfoundevidencethattheInChargeofMyMoneyprogramis

relevantandevidence-based,doesfacilitatesustainedimprovementinfinancialbehaviours,

anddoesproducestrongretentionoffinancialknowledgeandunderstandingforpeople

facingvulnerabilitiesincludingdrug,alcoholandgamblingaddictions,homelessness,and

familystrain.Thisstudyalsofoundtextmessagenudgesmayincreasethechanceof

sustainedchangeinfinancialbehavioursaftertheprogram,butthisfindingneedstobe

replicatedwithalargersample.

Page 13: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

13

Introduction

Program Evaluation Background

WesleyMissionhascaredforthemarginalisedandvulnerablefor150years,andforthe

past30yearshasdeliveredfinancialcounsellingservices.Forexample,in2014-2015,

WesleyCounsellingServicesprovidedfinancialcounsellingsupportfor1,339people(Wesley

Mission,2015).In2011,WesleyMissiondevelopedafinancialliteracyprogramto

complementitsfinancialcounsellingprogram.Financialliteracyisdefinedhereasthe

developmentofknowledge,understandingandskillsthatassistindividualstomake

informedjudgementsandtotakeeffectivedecisionsregardingtheuseandmanagementof

money(WesleyMission,2010).WesleyMission’sfinancialliteracyprogram,called“In

ChargeofMyMoney”,wonaMoneySmartawardin2013andhasbeendeliveredtoover

4,000peoplesinceitsinceptioninAugust2011(WesleyMission,2016b).

AlthoughprogramparticipantshavereportedpositiveoutcomesinWesleyMission’s

evaluationquestionnairesprovidedattheendofeachsessionoftheprogram,the

program’sefficacyhasnotbeentestedempirically.Theprincipalaimofthisprojectwasto

evaluatetheeffectivenessoftheInChangeofMyMoneyprograminvulnerable

populations,intermsofsustainedpositivechangesinfinancialbehaviourandknowledge

retentioninparticipants.

ThisprogramevaluationwasfundedbyFinancialLiteracyAustralia.

Program Evaluation Description & Aims

AnexternalevaluatorfromMacquarieUniversity,Sydney,wasengagedtoindependently

evaluatetheeffectivenessoftheInChargeofMyMoneyprogramasafinancialliteracy

educationalresource.MacquarieUniversityresearcherscollaboratedwithWesleyMission

financialliteracyprogramadvisorsandanacademicexpertinthefieldfromRMITUniversity

Page 14: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

14

inMelbourne,todeveloptwoquestionnaires.Thefirstquestionnairewaspersonally

administeredbyoneoftheMacquarieUniversityresearchersto128programattendeesat

theendofthelastsessionoftheprogramat18programdeliverycentres–NGOsfrom

MetropolitanSydneyorregionalcentreswithina300kmradiusoftheSydneyCBDthat

assistpeoplewithdrug,alcoholorgamblingaddictions,orprovidesupportforthehomeless

orforat-riskfamilies.Betweenfivetosevenmonthsaftertheprogram,thesameresearcher

attemptedtocontactallinitialstudyparticipants,bytelephoneoremail,askingthemto

completeasecondquestionnaireverysimilartothefirst.Sixty-threeparticipantscompleted

thesecondquestionnaire.

InadditiontotheprimarygoalofevaluatingtheeffectivenessoftheInChargeofMyMoney

programinvulnerablepopulations,thisprojectexploredthelinksbetweenvarious

participantfactorsandchangesinfinancialbehaviours,knowledgeandattidudes,inan

attempttoidentifythereasonswhytheprogrammighthaveanimpact.Furthermore,

participantswereaskedhowtheythoughttheprogramcouldbeimproved,sothatWesley

Missioncanrefinetheprogramtobettermeettheneedsoftheirtargetaudience.

Participants’feedbackonwhattheythoughtwasmosthelpfulabouttheprogramandwhat

theyrememberedfromitseveralmonthslateralsoprovidesusefulinformationregarding

theaspectsoftheprogramthatweremostrelevanttothesevulnerablepopulations.Finally,

theprojectinvestigatedwhetherornotregulartextmessagereminders,alsoreferredtoas

“nudges”,wouldincreasethechanceoflong-termimprovementsinfinancialbehaviours.

Theoutcomesofthisprojectareintendedtoassistthelong-termsustainabilityoffinancial

literacyeducationforvulnerablepopulations.Itishopedthatthelearningswillalsobe

relevanttootherfinancialliteracyprogramstargetingawideraudience.Thisprojectmay

alsoassistthefinancialliteracysectorthroughincreasedaccesstoevidence-basedfinancial

literacyeducationalmaterials.

Thisreportwillcoverthefollowing:

• Aliteraturereviewofbestpracticeforfinancialliteracyprogramstargetedat

vulnerablepopulations.

Page 15: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

15

• Areviewofhowtextmessagenudgesmaybeabletosupportimprovementsin

financialbehaviour.

• ThebackgroundanddescriptionoftheInChargeofMyMoneyprogram.

• TheaimsoftheInChargeofMyMoneyprogramevaluation,keyfactorsand

outcomes,andtheparticipants’demographics.

• Theprogramevaluationresearchmethodology.

• Theresultsfromtheprogramevaluation,includingparticipants’engagement

with,feedbackon,andknowledgeretentionfrom,theprogram.

• Changeinfinancialbehaviours,knowledge,attitudes,self-efficacyandemotional

spending,severalmonthsaftertheprogram.

• Theeffectivenessoftextmessagenudges.

• Asummaryofthekeyfindingsfromtheprojectandtheirimplications.

• Thelimitationsofthestudy.

• Suggestionsforfutureresearch.

• Conclusion.

Page 16: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

16

Literature Review

Best practice for financial literacy programs targeted at

vulnerable populations

Onlyadecadeagotherewaslimitedinterestinthetopicoffinancialliteracy(Miller,

Reichelstein,Salas,&Zia,2014).Now,however,thereareinitiativestoincreasefinancial

literacylevelsworld-wide(Blue&Brimble,2014).Financialliteracycanbedefinedasa

combinationofawareness,knowledge,skills,attitudes,andbehavioursnecessaryto

effectivelymanageone’smoneyandmakesoundfinancialdecisions(Atkinson&Messy,

2011;Collins&Holden,2014).Financialliteracy,therefore,isnotsimplyaboutknowledge

andunderstanding,butimpliestheabilitytotranslatefinancialknowledgeintoeffective

financialdecisionsandbehaviours(Collins&Holden,2014;ConsumerFinancialProtection

Bureau,2015).Theneedforindividualstohavegreaterfinancialliteracyhasgrownfora

numberofreasons.First,therehasbeenashiftinmanydevelopedcountriesfromstateto

individualresponsibilitytoprovideadequateresourcesforoldageandtoprovidefinancial

securityduringsickness,disabilityorunemployment(Atkinson,McKay,Collard,&Kempson,

2007).Furthermore,financialproductsandmarketshavebecomeincreasinglycomplex,and

therehasbeenwidespreadgrowthofalternativeandpredatoryfinancialservices(Blue&

Brimble,2014).Inaddition,thereisevidenceoflowlevelsoffinancialliteracyglobally

(Brimble&Blue,2013).

Althoughfinancialliteracyfocusesonindividualresponsibilitytouseresourceseffectively,it

isimportanttorememberthataperson’sfinancialwell-beingisnotsolelydeterminedby

theirabilitytomanagetheirmoneyandmakesoundfinancialdecisions.Financialwell-being

canbedefinedashavingcontroloverday-to-dayfinances,havingthecapacitytoabsorba

financialshock,trackingtomeetone’sfinancialgoals,andhavingthefinancialfreedomto

makechoicesthatallowapersontoenjoylife(ConsumerFinancialProtectionBureau,

2015).Therearemanyfactorsbeyondanindividual’scontrolinachievingfinancialwell-

being,suchastheglobalandnationaleconomicenvironment,familywealthand

Page 17: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

17

connections,accesstoeducation,income,andgeographiclocation.Thesefactorscreatea

contextthateitherprovidesorlimitsanindividual’sopportunitiestoputtheirfinancial

literacyintopractice(ConsumerFinancialProtectionBureau,2015).Therefore,thegoalof

financialliteracyeducationistohelppeoplemakethebestoftheirsituationgiventheir

currentfinancialcircumstances(ConsumerFinancialProtectionBureau,2015).Promoting

financialliteracyisnotintendedtoimplythatindividualsarecompletelyresponsiblefor

theirownfinancialsituations,orthatfinancialliteracycaneliminatepoverty,sincesomuch

isoutsidethecontrolofindividuals.Inthismonetisedsociety,financialliteracyeducation

shouldbeseenasabasicright,liketherighttobasicliteracyandnumeracyeducation,so

thateveryonehastheknowledgeandskillstoactintheirbestfinancialinterests,giventheir

backgroundandcontext(Johnson&Sherraden,2007).

Australiareflectstheworld-widephenomenonoftherecognitionoflowlevelsoffinancial

literacyinthegeneralpopulationandtheneedtoimprovefinancialliteracylevels.The

AustralianandNewZealandBank(ANZ)surveysfrom2003and2010showedthatthereisa

substantialdeficiencyintheleveloffinancialliteracyamongstmanyAustralians(Taylor&

Wagland,2013).The“FinancialResilienceinAustralia2015”report,basedona

representativesampleofalmost1,500Australians,foundthatalmosthalfofthepopulation

(48%)reportedthattheyonlyhavea‘basicunderstanding’offinancialproductsandservices

andmorethan9%reportedthattheyhad‘nounderstanding’(Muiretal.,2016).Recent

Australiansocialpolicyhasprioritisedfinancialliteracy,withafocusonmoneymanagement

education,asawaytoaddressfinancialstressandover-indebtedness(Brackertz,2014).The

FinancialLiteracyTaskforceandtheFinancialLiteracyFoundationwereestablishedin2004,

andin2011theNationalFinancialLiteracyStrategywasreleasedbythegovernment

regulator,AustralianSecuritiesandInvestmentsCommission(ASIC)(Brackertz,2014).

AkeystrategytoimprovelowlevelsoffinancialliteracyinAustraliaandworld-wideis

throughadultfinancialliteracyprograms.Theseprogramsusuallyaddresspersonalfinance

topicssuchasbudgeting,saving,andcreditmanagement(Collins&O'Rourke,2012;Fox&

Bartholomae,2008).Financialeducationforadultsisusuallydeliveredthroughcommunity

Page 18: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

18

oremployerbasedprogramsthatprovideattendeeswithinformation,ontheassumption

thatthisknowledgewillresultinimprovementsinfinancialbehaviours(Collins&Holden,

2014).Althoughtherehasbeenaproliferationoffinancialliteracyprograms,workshops,

andseminars,littleresearchhasmeasuredtheimpactandusefulnessoftheseprograms

(Milleretal.,2014;Vyvyan,Blue,&Brimble,2014).Therearefewfinancialliteracy

programsthathavebeenevaluatedwithrobustresearchdesignsthatdemonstrateifand

howimprovedfinancialknowledgeinfluencesfinancialbehaviourslaterinlife,witha

paucityofstudiesmeasuringbehaviouralchangesoveraperiodoftime(Collins&Holden,

2014).Positivecorrelationshavebeenfoundbetweenfinancialknowledgeandfinancial

practices,butverylittleresearchhasbeenconductedusingexperimentaldesignsthatprove

thatthealteredbehaviourshavebeencausedbytheprogramandnootherfactors(Collins

&Holden,2014;Schuchardtetal.,2009).Inaddition,theliteraturesuggeststhatcombining

financialeducationwithaccesstoappropriatefinancialproductsandservicesisthemost

effectivewaytohelppeopleachievefinancialwell-being(Birkenmaier&Maynard,2016).

Withoutaccesstofinancialproducts,andwithouttheopportunitytosaveduetolowor

unpredictabledisposableincome,theabilityoffinancialliteracyprogramstoproduce

greaterfinancialsecurityislimited(Birkenmaier&Maynard,2016;Kempson&Finney,

2009).

Ofthefinancialliteracyprogramevaluationsthathavebeenconducted,notallhave

demonstratedprogrameffectiveness.Evenprogramsthatarepopularandappeartobe

helpfulmaybefoundtohavenoeffect,orinsufficienteffect,tojustifytheexpenditureof

resources,whentested(Reich&Berman,2015).Ameta-analysisoftheliteraturefoundthat

morethan140ofthe188studiesindicatedthatfinancialeducationcanbehelpfulin

improvingfinancialoutcomes.However,approximately40ofthestudiescitedeitherno

impactoronlyamodestimpactfromtheinterventionthatmaynotbejustifiedrelativeto

thecostoftheprogram(Milleretal.,2014).Thismeta-analysisconcludedthatfinancial

educationcanconsistentlyimprovesomeoutcomes,suchassavingsandrecordkeeping

behaviour,butdoeslesswellinpreventingoutcomessuchasloandefault(Milleretal.,

2014).Anothermeta-analysisshowedthatstudiestendtooverstatetheeffectsoffinancial

Page 19: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

19

educationonsubsequentbehaviour,withonly0.1%ofthevarianceinfinancialbehaviour

beingattributabletofinancialeducationinterventions(Fernandes,Lynch,&Netemeyer,

2014).Therefore,therearemajorchallengestodrawingconclusionsabouttheeffectiveness

offinancialliteracyeducationanditremainsunclearifandhowincreasedfinancial

knowledgetranslatesintomorepositivefinancialbehaviours(Collins&Holden,2014;Fox,

Bartholomae,&Trombitas,2012;Schuchardtetal.,2009).TheOECDconcludedin2013that

thereisapossiblecausallinkbetweenfinancialeducationandoutcomes,andthatimproved

levelsoffinancialliteracycanleadtopositivebehaviourchange[italicsadded](Brimble&

Blue,2013).

Financialliteracyeducationisbelievedtobeparticularlyimportantforlow-income

populations.Thisisnottosay,however,thathouseholdslivingonlow-incomesare

responsibleforthefinancialhardshiptheyexperience.Indeed,ithasbeensuggestedthat

thefocusofpoliciesshouldbetodirectlyaddresssocietalinequalityratherthanfocusingon

thefinancialincompetenceofthepoor(Johnson&Sherraden,2007).

Asdiscussedpreviously,manyfactorscontributingtofinancialwell-beingareoutsidean

individual’scontrol,limitingtheiropportunitiestobecomefinanciallysecure,despitetheir

leveloffinancialliteracy.Higherincomeandeducationarelikelytoprotectindividuals

againstseverefinancialproblemssincetheyprovideabufferandgreateroptionsintimesof

economicdownturnorpersonaldifficulties(vonStumm,FentonO’Creevy,&Furnham,

2013).

Nevertheless,theresearchdoesshowanassociationbetweenlow-incomesandlow

financialliteracy.Forexample,generalpopulationsurveysintheU.S.havefoundthat

peoplewithlowincomesarelessfinanciallyknowledgeablethanthosewithhigherincomes

(Anderson,Zhan,&Scott,2005).Thosewithlowincomesarealsomoresusceptibleto

paydayloans,pawning,andaccessingother‘predatory’lendersthatchargemuchhigher

Page 20: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

20

interestratesthanmajorlenderssuchasbanks(Andersonetal.,2005).Studieshavealso

shownthatthoseonlow-incomesarelesslikelytosavemoney(Andersonetal.,2005).

InAustralia,the2003ANZfinancialliteracystudyfoundthatthelowestlevelsoffinancial

literacywereassociatedwiththosehavingloweducationqualification(Year10orless),

thosecategorisedas‘unskilledworkers’,andthosewithlessthan$20,000inhousehold

income(Taylor&Wagland,2013).Thisfindingwasrepeatedinthe2005and2008ANZ

surveys(Taylor&Wagland,2013).Similarly,theFinancialResilienceinAustralia2015report

(Muiretal.,2016)foundthatmostpeoplewithveryloworloweconomicresourcesalso

haveloworverylowlevelsoffinancialknowledgeandbehaviour.GiventhatAustralia’s

NewstartAllowanceisthelowestunemploymentpaymentintheOECD,atjust40%ofalow

full-timewageaftertax,beingabletomanageaverysmallincomewelliscrucialforsurvival

(Brackertz,2014).Infact,someresearchhasshownthatfinancialliteracyprogramsare

particularlyeffectiveforlow-incomeandverydisadvantagedparticipants,sinceevensmall

changesinfinancialbehaviourscanhavelargeimpactsontheirlives(Fry,Mihajilo,Russell,

&Brooks,2008;Grinstein-Weiss,Guo,Reinertson,&Russell,2015).

Althoughanumberoffinancialeducationprogramshavebeendevelopedgloballyto

addressthefinancialliteracyneedsoflow-incomepopulations,thereislittleresearch

measuringtheeffectivenessoftheseprograms(Lyons,Chang,&Scherpf,2006).Oneofthe

mainreasonsforthesparseamountofresearchisthatvulnerablepopulationgroupsmay

bemoredifficulttotrack,theytypicallyhavehighprogramdropoutratesandlowsurvey

responserates(Collins,2010;Lyonsetal.,2006).Forexample,anationalstudyofthe

effectivenessoffinancialcounsellinginAustraliaonlyhada5.3%responseratetothe1,200

questionnairessentoutto48sitesineachstateandterritory(Brackertz,2014).Theauthor

consideredthisasuccessduetothedifficultiesinherentinengagingthemost

disadvantagedmembersofsociety(Brackertz,2014).

Page 21: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

21

Additionally,mostprogramstargetinglow-incomepopulationsareoperatedbynon-profit

organisationswithlimitedresourcesandresearchcapability(Lyonsetal.,2006).

Furthermore,ithasbeenarguedthatmeasuresof‘success’offinancialliteracyprograms

targetedatlow-incomeclienteleneedtoberealisticandrecognisethatsomefinancial

behaviourscannotchangebecausetheirextenuatingfinancialcircumstanceshavenot

changed(Lyonsetal.,2006).Forexample,someprogramattendeesareunabletosaveas

theirincomeisinsufficienttomeeteventheirbasicexpenses(Brackertz,2014;Loibl,Jones,

Haisley,&Loewenstein,2016).

Ofthemorerecentresearchevaluatingtheeffectivenessoffinancialliteracyprograms

targetedatlow-income,disadvantagedpopulations,twoofthesestudieswereexperimental

indesignandsowereabletoshowcausality.Asmall(N=33)experimentalstudyofa

financialliteracyprogramdeliveredthrougharesidentialserviceforhomelessfamilies

foundthattheprogramincreasedbothfinancialknowledgeandpositivefinancial

behavioursattheendofthefourweeks,comparedtothoseinthecontrolgroupof

residentswhodidnotattendtheprogram(Reich&Berman,2015).However,therewasa

highlevelofattritionfromthestudy,with46%ofthe61initiallyrecruitednotcompleting

thestudyforavarietyofreasons(Reich&Berman,2015).

Anotherexperimentalstudyexaminedtheeffectivenessofafinancialliteracyprogram

designedforanddeliveredto300survivorsofintimatepartnerviolence(Hetling,Postmus,

&Kaltz,2015).Thisstudyfoundastrongprogrameffectonself-reportedfinancial

knowledgeandbehavioursonemonthaftercompletionoftheprogram.Thisstudyalsohad

quiteahighattritionrate,with34%notcontinuinginthestudyatTime2,onlyonemonth

aftertheendoftheprogram.Themainreasonfortheattritionwastheinabilitytocontact

participantsforthesecondinterview(Hetlingetal.,2015).Althoughnotanexperimental

study,Grinstein-Weissetal.(2015)examinedthelongitudinaldataofapproximately2,000

low-incomeparticipantsofamatchedsavingsprogramwithafinancialeducation

componentintheU.S.Theyfoundthatfinancialeducationwasastrongpredictorofsavings

outcomesforlow-incomeprogramparticipants.Theauthorsconcludedthatevenshort

Page 22: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

22

financialeducationcoursescanprovidelow-incomehouseholdswiththeknowledgeand

motivationtoaccumulatesavings.

Duetothelimitedamountofresearchonfinancialliteracyprogramsdesignedforand

deliveredtolow-incomepopulations,theliteraturehasnotyetconfirmeddefinitebest

practicesinfinancialeducationforthispopulation(ConsumerFinancialProtectionBureau,

2015;Grinstein-Weissetal.,2015;Tuominen&Thompson,2015).Nevertheless,some

tentativebestpracticeguidelinesfordeliveringfinancialliteracyprogramstolow-income

andvulnerableclientshaveemergedfromtheliterature(Robson,2012).Theseguidelines

recommendthatfinancialliteracyprogramsforlow-incomeanddisadvantagedpopulations

aremosteffectivewhenthey:

• aretargeted&relevanttotheiraudience;

• aredeliveredbytherightpeople;

• aredeliveredattherighttime;

• areconsistentwithprinciplesofadultlearning;and,

• empowerattendees.

Is the program content relevant and evidence-based?

Themostimportantbestpracticetobehighlightedintheliteratureistheneedforfinancial

programstobewelltargetedtotheiraudience(Blue&Brimble,2014;Hathaway&

Khatiwada,2008).Intheirreviewoftheliterature,HathawayandKhatiwada(2008)

concludethattheoverallevidencefortheeffectivenessoffinancialeducationremains

unclear;however,inboththeshortandlongterm,highlytargetedprograms,ratherthan

generalprograms,tendtobemoreeffectiveinchangingpeople’sfinancialbehaviours.

Thereisaparticularneedtotailorfinancialliteracyprogramswhentheaudiencearethose

whoaremarginalisedinsocietyanddonothaveaccessto,oropportunitywith,mainstream

financialservices.Sometimesmainstreaminformationandadvicefromfinancialservices

Page 23: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

23

canbeirrelevantorunhelpfulforvulnerableclients(Robson,2012).Therefore,theprogram

contentneedstobetailoredtoclients’livesandcircumstances,addressingtheirmainneeds

andconcerns(Friedman,2005;Robson,2012).Theinformationprovidedneedstobe

accurateandappropriateforthetargetaudience(Robson,2012).Themorerelevantthe

material,themorelikelyitisthatprogramparticipantswillremainengagedinthetraining

(Friedman,2005).Forexample,someresearchhasfoundthatthosewhoaredependenton

governmentpaymentsarenotawareofthefullservicesandbenefitsavailabletothemand

thereforethisisatopicrelevanttothemostdisadvantagedinsociety(Andersonetal.,

2005).

Inadditiontothecontentoftheprogramneedingtoberelevant,theteachingstyleshould

alsobetargetedtotheaudience.Forexample,usingpracticalexamplestoillustratethe

informationisrecommended,asthisenablesparticipantstopersonalisetheconceptsbeing

taughtandapplythemtotheirownsituations(Friedman,2005).Theuseofrelevant,

practicalexamplesmayalsohelppeoplebetterunderstandfinancialconceptsandretain

whattheyhavelearnt(Friedman,2005).However,anyillustrationsorexercisesneedto

reflectthelimitedeconomicresourcesof,andconstraintsplacedon,peoplelivingonlow-

incomes(Andersonetal.,2005).Aswellasappropriateexamples,teachingsimple“rulesof

thumb”,hasbeenshowntobeveryeffectiveininfluencingfinancialbehaviours(Consumer

FinancialProtectionBureau,2015;Datta&Mullainathan,2014).Thiscanbeaparticular

issueforpeopleinsubstanceabuserehabilitationprograms.Rulesofthumb,suchas“pay

yourselffirst”,areeasytorememberandsoaremorelikelytoberecalledwhenmaking

decisions(ConsumerFinancialProtectionBureau,2015).Furthermore,thelanguageusedto

presentthetrainingmaterialshouldbeconsistentwithparticipants’educationaland

culturalcontexts(Andersonetal.,2005).

Tailoringcontentanddeliverytothetargetaudienceshouldalsoincludeculturaladaptation

(Brimble&Blue,2013).Forexample,afinancialliteracyprogramwasadaptedfordeliveryat

sixindigenoussitesinQueensland(Brimble&Blue,2013).Thisprogramrecognisedthe

culturalandtraditionalvaluesystemsofindigenouspeoples,wheresharingresourceswith

Page 24: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

24

extendedfamily,kinshipandreciprocityarecentraltoAboriginalidentity.Furthermore,the

programwasadaptedtothelearningstyleofprogramparticipants,withmoretimegivento

groupdiscussionsandstorytelling.Suchculturaladaptationsarevitaltoensuringthatthe

contentanddeliverystyleoftheprogramarerelevantandappropriateforculturallydiverse

groupsinsociety.

Delivered by the right people

Notonlydoesthecontentoftheinformationneedtoberelevantanddeliveredina

culturallyappropriatemanner,butwhodeliversthematerialisalsoimportant.The

trustworthinessoftheprovideroftheprogram,particularlyintheirmotivesandthe

capacitytoprovideaccurateandappropriateinformation,issignificant(Robson,2012;

Russelletal.2011).Financialliteracyprogramsdeliveredbycommunity-basedorganisations

aretrustedthemostbyvulnerableindividualssincetheytendnottoarousesuspicionsthat

therearehidden,self-servingmotivesbehindtheinformationbeingpresented(Friedman,

2005;Robson,2012).Furthermore,targetingthemostvulnerablepopulationsmaybebest

achievedthroughthecommunitysector,sincethissectorunderstandstheneedsand

circumstancesoflow-incomeindividualsbetterthanothers(Russelletal.,2011).

Inadditiontotheprogramprovider,theindividualswhodelivertheprogramandtheir

communicationstylealsomatterstoprogramefficacy.Theliteraturesuggeststhatbest

practiceincludeshavingwell-prepared,well-trainedinstructorswhoarefirstfinancially

literatethemselves(Collins&Holden,2014;Robson,2012).

Additionally,itiskeythatfinancialliteracystaffhaveagoodunderstandingoftheneedsof

vulnerablegroupsandarenon-judgemental(Robson,2012).Therearerealriskstoteaching

financialliteracybadlybyprovidinginaccuratecontent,ordeliveringinsensitiveor

inappropriateguidancetoclients(Robson,2012).Theserisksincludeencouragingchanges

thatareinfactdetrimentaltoparticipants’financialwell-being,discouragingfurther

learningaboutpersonalfinance,causingclientstoviewlearningaboutfinancialknowledge

Page 25: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

25

asirrelevant,orprovidingapainfulreminderofparticipants’owndisadvantageand

marginalisation(Robson,2012).Therefore,itisimportanttochooseandtrainfinancial

literacyprogramstaffcarefully.

Delivered at the right time

Inadditiontorelevantcontentandappropriatedeliverybytherightpeople,thetimingof

financialliteracyeducationisalsoimportant(Kiviat&Morduch,2012).Peoplearemost

likelytobeattentiveandengagedwithaprogramwhenthecontentisrelevanttotheir

greatestneedsatthetime(Frentzeletal.,2010;Friedman,2005).Conversely,unwanted

informationmaybeneithervaluednoractedupon(Russelletal.,2011).Ithasbeen

suggestedthatfinancialliteracyprogramsaremosteffectivewhentheyinterveneat

“teachablemoments”inparticipantslives(Robson,2012).Theseteachablemomentsare

whenfinancialchoiceisimminentandwhenthecontentoftheprogramhasimmediate

practicalapplications(Robson,2012).

Ithasalsobeenproposedthatembeddingfinancialliteracyprogramsintootherexisting

programsandservicesmaybeworthwhile,asclientsarepresumablyatavulnerableperiod

intheirliveswhentheyseektheseotherservices,andthereforemaybemotivatedto

improvetheirmoneymanagementskills(Robson,2012).Furthermore,includingfinancial

literacyeducationwithinotherprogramsthatsimultaneouslyaddressmultipleneeds

enablesorganisationstosupportclientsinaholisticmanner(Robson,2012).

Adhere to principles of adult learning

Adultslearndifferentlytochildrenandyouth,sofinancialliteracyprogramsbeingdelivered

toadultsneedtoadheretotheprinciplesofadultlearning(Robson,2012).Thebasic

principlesarethatadults:

• respondtoguidancenotinstruction;

• selectivelyattendtoinformationtheyfeelisrelevanttothem;

• bringindependentideas,previousknowledge,andlifeexperiencetotheirlearning;

Page 26: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

26

• learnwhentheyaremotivatedtodoso;and,

• learnbestbydoing(Robson,2012)

Themostimportantprinciplefromadultlearningthatappliestofinancialliteracyprograms

isthatadultsdonotwanttobeinstructedorlecturedaboutwhatthey‘should’do.Instead,

adultlearnerswishtoreceiveguidancethatisrelevanttotheircircumstances(Robson,

2012).Adultshaveexistingknowledgeandlifeexperiences,soadaptingprogramcontentto

buildonparticipants’existingknowledgeandinsightsiskeytosuccessfuladultlearning

(Robson,2012).Therefore,amoreparticipatoryandgroupdiscussionstyleofteachingthat

elicitsparticipants’inputandfeedbackislikelytobemoreeffectivethanalecturestyle

method(Robson,2012).Thisalsoallowsthepresentertomoreaccuratelyadaptthe

programmaterialtothecurrentneedsandinterestsofthegroup,therebyincreasing

programrelevance(Andersonetal.,2005).Ithasbeensuggestedthatthestyleoflearning

infinancialliteracyprogramswiththegreatestimpactfordisadvantagedgroupsisamutual

exchangebetweenparticipantsandprogramproviders(Robson,2012).Anotheradvantage

ofgroupdiscussions,andelicitingparticipationandfeedbackfromprogramattendees,is

thatitdemonstratesaninterestintheopinionsandlivesofattendees,andsobuildstheir

confidenceandsupportstheirempowerment(Andersonetal.,2005).

Adultlearnersdobestwhentheyaremotivatedandvolunteertobepartofafinancial

educationprogram(Robson,2012).Mandatoryfinancialliteracyprogramsmaycreate

barrierstolearningandfacilitatorsmayhavetofirstwinoverreluctantparticipants

(Robson,2012),althoughthishasnotbeendemonstratedbyresearchasyet.However,

peoplewithverylowfinancialliteracyhavebeenfoundtolackmotivationtoseekfinancial

counselling,sovoluntaryparticipationamongthispopulationgroupmightresultin

programsnotbeingdeliveredtothosewhoneeditthemost(Frentzeletal.,2010).

Regardingtheprincipleofadultslearningbestbydoing,someresearchhasfoundthat

‘learningbydoing’isthesinglemostimportantfactorinacquiringfinancialcapabilityandis

consideredbysometobeavitalpartoffinancialeducation(Russelletal.,2011).Hence,itis

importantthatfinancialliteracyprogramsprovideactivitiesforparticipantstoengagein

Page 27: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

27

duringtheprogram,andthatattendeesareencouragedtoactontheirnewknowledge

immediately.Foradults,knowinghowtodothingsisoftenmoreimportantthanthe

knowledgeofparticularfacts(ConsumerFinancialProtectionBureau,2015).Therefore,

teachingfinancialliteracyskills,suchashowtodoabudgetandpracticalwaystoreduce

spending,ismorelikelytoproducechangesinpeople’sbehaviourthanfactualknowledge

(ConsumerFinancialProtectionBureau,2015).

Empowerment & self-efficacy

Finally,theliteraturesuggeststhatitisimportanttoempowerprogramparticipants

(Andersonetal.,2005).Indeed,empoweringindividualshasbeendescribedasbedrockto

theworkoffinancialeducators(Kiviat&Morduch,2012).Byprovidingtoolsandteaching

skills,thepurposeofempoweringparticipantsistobuildtheirabilityandconfidencetobe

abletobettercontrolandmanagetheirfinances(Andersonetal.,2005).Incorporating

empowermentlanguageintoprogramsmaybeespeciallysalientforpeoplewithlow-

incomeswhofeelmarginalisedfromtherestofsociety(Andersonetal.,2005).Forthose

whomaybedependentongovernmentincomesupportand/orcharitiestobeableto

survive,financialliteracyeducationmaypromotesomesenseofself-sufficiencyandcontrol

intheirlives(Tuominen&Thompson,2015).

Relatedtoempowermentistheconceptofself-efficacy(Bandura,1982).Self-efficacyisthe

beliefinone’sabilitytosucceedatperformingaspecifictask(Frentzeletal.,2010).Those

withhigherlevelsofself-efficacyaremorelikelytobeproactiveandtakeinitiativebecause

theybelievetheiractionswillbeeffective(Frentzeletal.,2010).Healthmanagement

studieshavefoundassociationsbetweenhighself-efficacyandincreasedlikelihoodsof

followingadietandexerciseprogram,higherqualityoflifeforasthmaticsandhigher

medicationadherence(Frentzeletal.,2010).Therefore,increasingpeople’sfinancialself-

efficacycouldplayanimportantroleintheeffectivenessoffinancialliteracyeducationin

changingbehaviour(Frentzeletal.,2010).Regardlessofwhethertheterms‘empowerment’

or‘self-efficacy’areused,theimportantpointisthatfinancialprogramsaremorelikelyto

Page 28: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

28

producechangesinbehaviourifprogramparticipantsbelievetheyhavetheabilityto

changetheircircumstances.

Conclusion

Insummary,thereisevidencethatfinancialliteracyprogramsareeffectiveatcontributing

toimprovedfinancialbehavioursforlow-incomeandmarginalisedpopulations.Recent

experimentalstudieshavedemonstratedcausationbyfindingthatthosewhoparticipated

infinancialliteracyprogramshavebetterfinancialknowledgeandbehavioursafterthe

programscomparedtothosewhodidnotreceivetheeducation.Althoughthereis

insufficientresearchasyettoconfirmasetofbestpracticesforfinancialliteracyprograms

targetingseverelyfinanciallyvulnerableclients,ithasbeenrecommendedthatfinancial

literacyprogramsaremosteffectivewhentheyoffertailored,contentisrelevant,anditis

deliveredinanappropriatemannerbytrustedtrainerswhoareknowledgeableandnon-

judgemental(Robson,2012).

Using Text Messages to Support Improvements in Financial

Literacy

Humansoftenbehaveinwaysthataredetrimentaltotheirownwell-being.Forexample,

despiteincreasededucationandadvertisingefforts,obesity,smoking,alcohol,andlackof

physicalactivity,continuetohaveadeleteriousimpactonpublichealthworldwide

(Marteau,Ogilvie,Roland,Suhrcke,&Kelly,2011).So,howcanpolicymakerschangehuman

behaviourtoimprovewell-beingacrossawholerangeofdomains?Oneapproachis

throughtheuseof“nudges”.

BehaviouraleconomistsRichardThalerandCassSunsteinpopularisedtheterm“nudge”,

whichtheydefinedasanymechanismthataltersorprejudicespeople’sdecisionsand

behaviourinapredictablewaywithoutprohibitinganyoptions(Marteauetal.,2011;

Sunstein,2014;Thaler&Sunstein,2008).Theoriginaldefinitionofnudgingexcluded

Page 29: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

29

legislation,regulationandinterventionsthataltereconomicincentives(Marteauetal.,

2011).However,thereisnoprecise,operationaldefinitionofnudginganditisbestthought

ofasatypeofbehaviouralchangeintervention(Marteauetal.,2011).Thekeycharacteristic

ofnudgesisthattheysteerpeopleinaparticulardirection,butstillallowfreechoice

(Sunstein,2014).Nudgespromptandactivelyguidechoices,ratherthandictate,byaltering

socialorphysicalenvironmentstomakecertainbehavioursmorelikely(Marteauetal.,

2011).

Theconceptofnudgesdrawsonbehaviouraleconomicsandsocialpsychologythathelp

explainwhypeopleoftendonotbehaverationally(Marteauetal.,2011;Pathak,Holmes,&

Zimmerman,2011).Althoughbasedinbehaviouraleconomicsandsocialpsychology,

Marteauetal.(2011)proposethatnudgingbridgesthegapbetweentwohumansystems,

thegoal-orientedsystemandtheautomatic,affectivesystem.Thegoal-orientedsystemis

drivenbyourcognitivecapacity,ourthoughts,valuesandintentions.Theautomaticsystem,

incontrast,requireslittleornocognitiveengagementandisdrivenbyimmediatefeelings

andistriggeredbyourenvironment.AccordingtoMarteauetal.,nudgingmainlyoperates

throughthisautomatic,affectivesystemandassumesthatalteringenvironmentalcues

shouldalterpeople’schoicesandbehaviours.

Nudgingasawayofchanginghumanbehaviourhasgeneratedgreatinterestamongstpolicy

makersworldwideinrecentyears,withboththeU.K.andU.S.governmentsembracingthis

tool(Marteauetal.,2011).TheU.K.nowhasaBehaviouralInsightsteam(sometimes

referredtoasthe“NudgeUnit”)andtheU.S.hasaWhiteHouseSocialandBehavioural

Sciencesteam(Sunstein,2014).Thegrowinginterestinnudgescanbeattributedtothelow

costoftheinterventions,thepotentiallyfastdeliveryandresults,thepersonalfreedom

theymaintain,andtheireffectiveness(Sunstein,2014).Thereareawiderangeoftypesof

nudges,withSunstein(2014)listingthetenmostimportantnudgesas:

• defaultrules(e.g.automaticenrolmentsinprograms);

• simplification(e.g.thesimplificationofformstoenterprograms);

Page 30: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

30

• useofsocialnorms(e.g.comparingyourelectricitybilltothatofothersofthe

samehouseholdsize);

• increasesineaseandconvenience(e.g.puttingfruitatthecheckoutcounter

insteadoflollies);

• disclosure(e.g.remindingcustomersoftheinterestrateontheircreditcard);

• warnings(e.g.graphicphotosoncigarettepackets);

• pre-commitmentstrategies(e.g.committingtostopsmoking);

• reminders(e.g.textmessageappointmentreminders);

• elicitingimplementationintentions(e.g.asking“doyouplantoimmuniseyour

child?”);and,

• informingpeopleofthenatureandconsequencesoftheirownpastchoices(e.g.

theirelectricitybillforthesameperiodtheyearbefore).

Remindernudgesareofparticularrelevancetothepresentstudy.Remindersare

hypothesizedtobeeffectivenudgesbecauseoftheireffectonattention(Datta&

Mullainathan,2014;Pathaketal.,2011).Sometimespeoplehavegoodintentionstochange

theirbehaviour,butfollowingthroughonthoseintentionsrequiresrememberingtomake

thenecessarychangesorperhapsprioritisingtheaction(Datta&Mullainathan,2014).A

timelyremindercanmitigatethismemorylapse(Datta&Mullainathan,2014).Reminders

alsoaddressthe‘availabilitybias’,whichisthetendencyforpeopletomakedecisionsbased

onthefrequencyofinformationortheexamplesthemindimaginesorretrieves(Pathaket

al.,2011).Nudgescanbringgoalsandintentionstoone’simmediateattention,makingit

morelikelythattheywillactonthoseintentions(Pathaketal.,2011).Therefore,reminders

canbeusedinsituationswherepeopleforfeitsubstantialbenefitswhentheydonotdo

something,attherighttime(Datta&Mullainathan,2014).

Oneofthemosteconomical,easy,rapidandeffectivewaystosendreminderstoatargeted

groupisthroughtextmessaging(Milward,Day,Wadsworth,Strang,&Lynskey,2015).Easy

touseandinexpensivesoftwareisavailabletofacilitateautomatedtextmessage

communication(Thirumurthy&Lester,2012).Mobilephonesarepervasiveinthe

Page 31: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

31

developedworld,with91%ofadultsintheU.S.and94%intheU.K.owningamobilephone

(Milwardetal.,2015).StatisticsbytheSwissFederalStatisticsOffice(2014)indicatedthatin

2012,thereweremoremobilephonesubscriptionsthaninhabitantsinmanyEuropean

countries(Billieux,Maurage,Lopez-Fernandez,Kuss,&Griffiths,2015).InAsia,theOfficeof

theCommunicationAuthorityofHongKong(2013)statedthatin2012,theaveragenumber

ofcellularphonesownedwas2.21perperson(Billieuxetal.,2015).Mobilephoneshave

alsohadaconsiderableimpactindevelopingcountries,sincecommunicationbymobile

phoneislessexpensivethanlandlinetelephonesorstandardinternet(Cole-Lewis&

Kershaw,2010b).Approximately75%ofadultsworldwidehaveaccesstoamobilephone,

withthree-quartersoftheworld’ssevenbillionmobilephonesubscriberslivinginlow-and

middle-incomecountries(Kamisetal.,2015).Therefore,textmessagingholdspromiseasa

usefultoolforbehaviourchangeinterventionsworldwide(Cole-Lewis&Kershaw,2010b).

Despitealltheinterestandpotentialadvantagesofusingnudgesinsteadofregulationand

monetaryincentivestoalterbehaviour,theevidencesupportingtheeffectivenessofnudges

isnotstraightforward.Textmessagingisarelativelynewmethodofdeliveryforbehaviour

changeinterventions,andthereforethereisapaucityofresearchontextmessagenudges

(Cole-Lewis&Kershaw,2010b).Todate,textnudgeshavebeenmostlyusedfordisease

preventionandmanagement.Inameta-analysisof12randomisedcontrolledtrialsusing

textmessagingforthemanagementofweightloss,smokingcessationanddiabetes,eightof

thestudiesfoundevidenceofshort-termeffects(Cole-Lewis&Kershaw,2010b).These

effectswerefoundforadolescentsandadults,minorityandnon-minoritygroups,and

acrossnationalities.However,ofthe12studiesinthemeta-analysis,threereportednot

beingstatisticallypowerfulenoughtodetectdifferencesinoutcomesandtherefore

producedinconclusiveresults.

Sincethismeta-analysisbyCole-LewisandKershaw(2010),otherstudiesusingtextmessage

nudgesinthehealthdomainhaveadvancedtheliteratureinthisarea.Forexample,inChina

almost2,000randomlyselectedcommunityresidentswereassignedtoreceivethreeweeks

oftextmessagesabouteitherinfluenzapreventionorstoppingsmoking(Chaietal.,2013).

Page 32: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

32

Participantswhoreceivedtheinfluenzapreventionmessageshadahigherself-reported

uptakeofinfluenzavaccination,reportedreducedinfluenza-likeillness,andshowed

improvedknowledgeandattitudesaboutinfluenzapreventionandcontrolattheendofthe

studycomparedtothosewhohadreceivedthesmokingmessages.Further,morethan95%

ofparticipantsfoundthetextmessageprogramusefulandtrustworthy.Similarly,two

randomisedcontrolledtrialsinKenyashowedthatregulartextmessagescanimprove

adherencetoantiretroviraltherapyforpeoplewithhumanimmunodeficiencyvirus(HIV)

(Thirumurthy&Lester,2012).Inthefirststudy,textmessagesweresentonceaweek

enquiringaboutpatients’healthandsayingtheycouldreplyandseekadvicefromhealth-

careproviders.Patientsreceivingthisnudgeadheredtotheirtreatmentbetterthanthosein

thecontrolgroup.Inthesecondstudy,aone-waytextmessagewassentremindingpatients

totaketheirmedication.Halftheparticipantsreceivedthismessageweeklyandtheother

halfreceiveditdaily.Surprisingly,thosewhoreceivedthetextsweeklyadheredtothe

treatmentbetterthanthosewhoreceivedthemessagesdaily,suggestingthefrequencyof

thenudgesisimportant.

Ameta-analysisofninetext-messaginginterventionsdesignedtopromoteadherenceto

antiretroviraltherapydemonstratedthattextmessaginginterventionssignificantly

improvedHIVtreatmentadherencecomparedtocontrolconditions(Finitsis,Pellowski,&

Johnson,2014).Themeta-analysisalsofoundthatmessagingparticipantsonceormore

timesadayresultedinlessadherencecomparedtosendingmessagesafewtimesaweekor

weekly.Largeradherenceeffectswerealsofoundwhenthetextmessagessupported

bidirectionalcommunication,includedpersonalisedmessages,andwerematchedto

participants’dosingschedule.Implicationsfromthismeta-analysisarethattextmessages

shouldbetailoredtotheindividual,encouragetherecipienttoreplytothetext,andbesent

lessfrequentlythandaily.

Incontrasttothesepositivefindings,otherstudiesoftextmessageinterventionshave

demonstratedmixedresults.Forexample,astudyontextmessageremindersafterrapid

diagnostictesting(RDT)foralmost700peopleseekingtreatmentformalariaatdrugstores

Page 33: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

33

inNigeriafoundthattheeffectsofthetextmessagesdifferedforchildrencomparedto

adults(Liu&Modrek,2016).Inthisstudy,participantswererandomlyassignedtothree

groups;thosewhoreceivedabasictextreminderrepeatingtheRDTresultandappropriate

treatmentaction,thosewhoreceivedanexpandedtextmessageindicatingthatthe

governmentmightallowchemiststodoRDTs,oracontrolgroupthatdidnotreceiveany

textmessages.Thestudyfoundthatthetextremindersincreasedtreatmentadherencefor

RDT-positiveadultsanddecreasedtheuseofunnecessaryanti-malarialsforRDT-negative

adults.However,thebasictextreminderinterventiondidnothaveanimpactontreatment

adherenceortheuseofunnecessaryanti-malarialsforcaregiversofsickchildren.Infact,

theexpandedtextmessageactuallyledtopoorertreatmentadherenceandincreaseduse

ofunnecessaryanti-malarialsforsickchildren.

Mixedresultswerealsofoundinarandomisedcontrolledtrialstudywhereapproximately

800collegefreshmanweresent12personalisedtextmessagereminderseverycoupleof

weekspromptingthemtore-applyfortheFreeApplicationforFederalStudentAid

(FAFSA)(Castleman&Page,2015).Thestudyfoundlargeandpositiveeffectsamong

freshmanattendingcommunitycolleges,witha14%increaseinthoseremainingenrolledin

college.Theimpactoftheinterventionwasparticularlystrongforcommunitycollege

freshmanwithlowergradesinhighschool.Incontrast,thetextmessagesdidnotimprove

enrolmentcontinuanceamongfreshmanatfour-yearinstitutions.Inthecontrolgroup,

freshmanfromcommunitycollegeswerenearlythreetimesmorelikelytofailtoreapplyfor

theFAFSAcomparedtofreshmanoffour-yearcollegeswheretherateofpersistencewas

alreadyhigh(87%).Therewasalsoevidencethatthetextmessageshaddifferenteffectsfor

maleandfemalefreshmanatfour-yearinstitutions.Comparedtothecontrolgroup,thetext

messagesincreasedtheprobabilitythatmalefreshmanwouldremaininfour-year

institutions,butactuallydecreasedtheprobabilitythatfemalefreshmanwouldremainin

four-yearinstitutions.Therefore,thisstudy,aswiththestudyinNigeria,suggeststext

messagesmayhavedifferentialeffectsondifferentsub-groupswithinapopulation.

Otherstudieshavefoundthattextmessageremindershadnostatisticallysignificanteffects.

Forexample,anexperimentalstudyof63universitystudentsinEuropefoundthatdailytext

Page 34: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

34

messageremindersencouragingparticipantstobephysicallyactivedidnotincrease

students’levelofphysicalactivity(Schwerdtfeger,Schmitz,&Warken,2012).However,this

wasonlyasmallandbriefpilotstudy,sothesefindingsneedtobereplicated.Alargescale

randomisedevaluationofarecyclingprograminPerualsofoundnoeffectfortextmessages

onbehaviour(Chong,Karlan,Shapiro,&Zinman,2015).Inthisstudythetreatmentgroup

wasdividedintosubgroupsthatreceivedarangeofdifferenttextmessagewording,but

noneofthemessageswereeffectiveinchangingrecyclingbehaviour.Incontrast,thestudy

foundthatprovidinghouseholdswitharecyclingbindidimpactonrecyclingbehaviour.The

authorscommentthatmanyofthetextmessagesweresimilartomessagesfoundtobe

effectiveinothercountries,andsotextmessaginginterventionsmaynotgeneraliseacross

contextsandcountries(Chongetal.,2015).Inaddition,textmessageremindersmaynotbe

thebestinterventiontoalterbehaviourinsomecontexts,withotherinterventions,suchas

providingarecyclingbin,beingmoreeffective.

Asimilarlydisappointingresultwasfoundinarecentexperimentalstudytestingthe

effectivenessofnudgeremindersonincreasingthechildsupportpaymentsfrom

noncustodialparents(Baird,Cullinan,Landers,&Reardon,2016).Noncustodialparentsnot

currentlypayingchildsupportthroughincomewithholdingwererandomlysplitintothree

groups:acontrolgroupthatwasnotsentanyreminders,atextgroupthatwassenttwo

textmessagereminderseachmonth,andamailinggroupthatwaspostedamonthly

redesignedremindernotice.Althoughboththemailingandtextmessagereminders

increasedthepercentageofparentswhomadeapaymentbyastatisticallysignificant

amount,thisincreasewassmall(anincreaseof3.2%forthemailinggroup,and2.5%forthe

textinggroup).Furthermore,thetotalamountscollectedperparentdidnotincreasefor

eitherremindergroups.Therewasnostatisticallysignificantdifferenceineffectiveness

betweenthemonthlymailednoticesandthetwiceamonthtextmessages.Hence,it

appearsthattextmessagenudgesarenotalwaysaneffectivetoolforbehaviourchange.It

isimportanttoconsiderthattextmessagesaremeantonlyasreminderstobringadesired

behaviourintofocusandshouldnotbeexpectedtochangealltypesofbehaviour.Withall

interventions,itisimportanttoknowwhatworksforwhomandunderwhatcircumstances.

Page 35: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

35

Usingtextmessagingshouldinthefirstinstancebeappropriatetothebehaviouralcontext.

Therefore,testingtheuseoftextmessagingasanudgeindifferentcontextswillhelp

produceanunderstandingofwhentextmessagingcanbemosteffective.

Inthedomainoffinancialliteracy,theevidencefortheeffectivenessoftextmessage

nudgesinchangingpeople’sfinancialbehavioursisalsomixed.Inafieldexperimentwith

almost900microentrepreneurswhohadopenedbankaccountsinChile,participantswere

randomlyassignedtoreceiveaweeklytextmessagegivingpersonalisedtransaction

feedbackontheiraccount,ortoreceivenoreminders.Thestudyfoundsignificanteffects

forthosereceivingweeklytextmessages,withalmosttriplethenumberofweeklydeposits

comparedtothecontrolgroup.However,oncethetextmessagesstopped,therewasno

significantdifferenceinsavingsbehaviourbetweenthetreatmentandcontrolgroups,

demonstratingthattheeffectsdidnotpersistaftertreatment(Karlan,McConnell,

Mullainathan,&Zinman,2010).

Similarly,fieldexperimentswithdifferentbanksinthePhilippines,BoliviaandPeru

investigatedtheeffectofremindersonsavings(Karlanetal.,2010).Ineachexperiment,

afterindividualsopenedsavingsaccounts,thebankrandomlyassignedsomeindividualsto

receivemonthlyremindersviatextmessageorletter,ortoreceivenoreminders.Overall,

reminderssignificantlyincreasedthelikelihoodofreachingasavingsgoalby3%,and

marginallysignificantlyincreasedthetotalamountsavedinthebankby6%(Karlanetal.,

2010).InPeru,whereclientsstatedaspecificsavingsgoal,remindersthatmentionedthe

specificgoalincreasedsavingsbyanestimated16%comparedtothecontrolgroup,while

generalremindersthatdidnotmentionthespecificgoalhadnoeffectonsavingsamount.

Inaddition,neitherspecificnorgeneralremindershadaneffectongoalattainment.

Inanotherfieldexperiment,thistimeinUganda,arandomisedcontroltrialcomparedthe

impactofthreedifferentincentivesforloanrepayment(Cadena&Schoar,2011).Borrowers

wererandomlyassignedtoeitherreceivealumpsumcashrewarduponthecompletionof

Page 36: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

36

theloan,a25%reductionintheinterestrateforthenextloantheborrowertakesfromthe

bank,oramonthlytextmessagereminderthreedaysbeforeeachloanpaymentduedate.

Thestudyfoundthattheaveragesizeofthetreatmenteffectwassimilaracrossall

treatmentconditions,suggestingthattextmessagereminderscanbeaseffectiveas

monetaryincentivestohelpborrowersbettermanagetheirloanrepayments.Moreover,

textmessageremindersshowedsignificantlystrongereffectsforborrowersbelowtheage

of30yearsoldcomparedtoolderborrowers,suggestingthattheeffectivenessoftext

nudgesmightberelatedtoage(Cadena&Schoar,2011).

Otherstudieshavefoundmixedsupportfortheeffectivenessoftextmessagenudgeson

financialbehaviours.Inanotherfieldexperimentexaminingtheeffectivenessofreminders

forloanrepaymentsinthePhilippines,textmessagereminderswereonlyfoundtobe

effectiveunderonecondition(Karlan,Morten,&Zinman,2012).TwolargePhilippinemicro

lenderssentrandomlyassignedclientsweeklytextmessageremindersabouttheirweekly

repaymentobligation.Thecontentandtimingofthemessageswerevaried.Themessages

weresenteitherontheduedateoracoupleofdayspriortotheduedate,andthecontent

ofthemessagewaseitherframedinlossorgainterms,andeitherincludedtheloan

officer’snameornot.Therewasnooveralltreatmenteffect,withnoevidencethatthe

averagemessageimprovedrepaymentperformancerelativetothecontrolgroup.Theonly

treatmentconditionthatshowedanyeffectwasmentioningtheloanofficer’sname,which

significantlyimprovedrepaymentperformance.Eventhiseffectwasonlysignificantfor

borrowerswhohadbeenservicedbythementionedloanofficer.Theauthorssuggestthat

thisdifferentialeffectcanbeexplainedbythemessagestriggeringpersonalrelationship

schemas(Karlanetal.,2012).Itispossiblethatthiseffectisatleastpartlyculturalandmay

notoccurinothercountries.

AU.S.fieldexperimentexaminedtheeffectoftextmessageremindersoncreditscoresof

low-to-moderate-incomeindividuals(Bracha&Meier,2014).Approximately250individuals

wererandomlyassignedtoreceiveeithermonthlytextmessagereminderscontaining

individualizedfinancialinformationornomessages.Thetextmessagesremindedpeopleto

Page 37: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

37

paybillsontime(andatleasttheminimumamount),remindedthemoftheirtop-priority

financialgoalfortheyear,andgavetheircreditscorerange.Thisstudyfoundnooverall

effectoftextmessageremindersonchangeincreditscores.However,whenthe

participantsweredividedintosubgroupsaccordingtotheirinitialcreditratingscores,itwas

foundthattextmessageshadapositiveandsignificanteffectonindividualswhoinitially

hadlowcreditscores.Incontrast,thetextmessageshadamarginallysignificantnegative

effectonindividualswhoinitiallyhadhighcreditscores.Theauthorsofthestudysuggest

thattheremindersmayhavecausedhigh-credit-scoreindividualstolosesomefinancialself-

discipline,leadingtoincreasedcollectionaccounts.Therefore,thesefindingssuggestthat

textremindersmayhelpthosewhoneeditthemost,thatis,individualswithlowcredit

scores,buttheymayhaveanegativeeffectonpeoplewhodonotneedreminders.The

authorscautionthatnudgeremindersarenotamagicpillbutratheratoolthatcanhave

bothpositiveandnegativeeffects(Bracha&Meier,2014).

RodriguezandSaavedra(2015)conductedafieldexperimentwithlow-incomeyouthsin

Colombia.Approximately10,000youthswithbankaccountswererandomlyallocatedinto

oneoffourexperimentalconditions:

• Group1received12,monthlytextmessagescontainingfinancialeducationasthe

content.Eachmonththemessagevaried,andincludedthemessuchasthe

differencebetweenwantsandneeds,reducingunnecessaryexpenses,developinga

savingshabit,andsavinginsecureplaces.

• Group2received12,monthlysavingsreminders,allwiththesamewording,

“RemembertosaveyourTuticuenta!Thiswayyouwillbeonestepclosertoyour

goalsandmakeyourdreamscometrue.BancoCajaSocial”.

• Group3received24savingsreminderstwiceamonth,usingthesamemessageas

forGroup2.

• Group4wasthecontrolgroupthatreceivednomessages.

After12months,thestudymeasuredtheimpactoftextmessagesonaccountclosings,

accountdormancy,depositsandwithdrawals,andnetaccountbalance.Noneofthetext

messagingconditionseffectedaccountclosingsoraccountdormancycomparedtothe

Page 38: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

38

controlgroup.However,therewerelargedifferencesovertimeinaveragenetaccount

balancesbetweenyouthsassignedtoeitherofthetwosavingsremindergroups,compared

tothecontrolgrouporthefinancialeducationgroup.Therewasnostatisticallysignificant

differenceinthenetaccountbalancesbetweenmonthlyandtwice-monthlyreminder

treatmentgroups.Interestingly,however,theincreasesinnetaccountbalancesinboth

remindertreatmentgroupswasmainlyduetoasignificantdecreaseintheamountof

moneywithdrawnfromaccountscomparedtothecontrolgroup.Noneofthetextmessage

treatmentshadasignificanteffectonthenumberofwithdrawalsoronthenumberor

amountofdeposits.Furthermore,theeffectsoftheremindersonbehaviourdidnotlast

oncetheremindersceasedbeingsent,withparticipantsreturningtosimilaramountsof

withdrawalsoncethestudyended(Rodriguez&Saavedra,2015).

Conclusion

Insummary,theresearchevidencefortheeffectivenessoftextmessagenudgesacrossthe

domainsoffinancialbehaviours,educationandpublichealth,isquitevariable.Indeed,

insteadofaskingwhetherornotnudges‘work’,amoreappropriatesetofquestionsare

whatworks,forwhom,inwhatcircumstances,andforhowlong(Marteauetal.,2011)?The

timing,content,andfrequencyoftextmessagescanimpactontheeffectivenessofnudges.

Theage,gender,culturalbackgroundandsocioeconomicstatusoftherecipientsofthe

nudgescanalsodeterminetheirusefulness.Importantly,theactualbehaviourthatthe

nudgeisaimedatchangingisperhapsthemostcriticalelementtoconsider.Forexample,

somebehaviourssuchasnotpayingchildsupportisreflectiveofarangeofotherissuesthat

cannotbechangedbytextmessages.Finally,itisimportanttobecognizantofthefactthat

textmessagenudgescanhaveunintentional,adverseeffectsonsomesubgroupswithina

population.Overall,then,textmessagenudgescanbeausefultoolforshortterm

behaviouralchangeinsomesituations,forsomepeople,someofthetime,andfurther

researchisrequiredtoidentifyasetofguidelinesfortheiruse.

Page 39: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

39

The In Charge of My Money Program

Program Background

TheWesleyFinancialLiteracyProgramInChargeofMyMoneywasdevelopedandlaunched

in2011.ThecatalystfordevelopingtheprogramwasWesleyMission’s2010ThirdFinancial

StressReport(WesleyMission,2010).Theaimofthisreportwastounderstandhow

financialstresshadchangedsincethepreviousreportsin2006and2008,andtoidentify

whowasmostatrisk,andwhy.Dataforthereportwasgatheredfromanumberofsources,

includingfocusgroupswithWesleyMissionfinancialcounsellorstodiscussexperienceswith

theirclients,aquantitative,onlineandtelephonesurveyof621adultsinNSW,andcase

studyinterviewswithpeoplewhosoughtfinancialcounsellingfromWesleyMission(Wesley

Mission,2010).Theonlineandtelephonesurveydatawasweightedbyhouseholdincome

andlocationtomoreaccuratelyreflecttheNSWpopulation.

Keyfindingsfromthe2010reportwerethatmorethanathirdofrespondentsreported

beingfinanciallystressed,morethanhalfreportedfeelinganxiousabouttheirabilityto

meettheirfuturefinancialobligations,andcreditcarddebtwastwotothreetimeshigher

amonghouseholdsunderfinancialstress.Theseresultsrepresentedanincreaseinfinancial

problemsinNSWcomparedtoearlieryears.Thereportfoundthatmoneymanagement

skillswerenotwelldeveloped,with40%ofhouseholdsnothavingabudget,andathirdof

thosewithbudgetsnotadheringtothemandoverspending.Attitudestomoneywerefound

tobeunhelpfulinmanycases,with80%ofpeoplenothavingasavingsmindset,andmany

strugglingwiththeconceptofreducingexpensestodealwithfinancialhardship.

Inaddition,the2010surveyfoundthatfinancialstressisnotjustaboutmoney.Almosthalf

ofthoseunderfinancialstressinthesurveyhadsufferedfromillhealth,bothphysicaland

mental,andathirdexperiencingrelationshipissues,suchasdivorceanddomesticviolence.

Aquarterofrespondentsturnedtoalcoholanddrugstocopewiththefinancialstrain,and

onein10exacerbatedtheirfinancialproblemsbygambling.Themajorityofthosesurveyed

Page 40: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

40

didnotseekassistancewhenexperiencingfinancialstress,ordelayedobtaininghelpuntilit

wastoolate(WesleyMission,2010).

Thekeyrecommendationfromthe2010“MakingEndsMeet”reportwastheneedto

increasepeople’sabilitytoself-managetheirfinancesbyenhancingfinancialliteracyin

NSW.Thereportrecommendedaneedfortargetedstrategiestoassistthosestrugglingwith

financialproblems.WesleyMissionrecognisedtheneedformorefocusedfinancialliteracy

trainingandresourcestobedisseminatedthroughcommunityservicesthatcareforthose

experiencingthegreatestfinancialstress.SinceWesleyMissionwasnotabletofindan

existingfinancialliteracyprogramthatspecificallyaddressedtheneedsoftheirmarginalised

clients,theydecidedtocreateaconsistentcommunityeducationprogramforWesley

Missionfinancialcounsellorsandfinancialliteracyeducatorstodeliver.

In2011,WesleyMission,togetherwithStGeorgeBank,begantheprocessofcreatinga

financialliteracyprogramtargetedatvulnerablecommunitieswheretherewerecomplex

problemsincludingaddictions,mentalandphysicalhealthissues,homelessness,

unemployment,anddomesticviolence.ThegoalofthisWesleyMissionandStGeorgeBank

financialliteracyeducationprojectwasto:

- provideeducation,materialsandresourcesthatwouldbuildbasicknowledge,

understandingandfinancialmanagementskills;

- developsociallyresponsiblebehavioursinpersonaldecisionmakinginmanaging

theirmoney;and,

- assistpeopletomakeinformedjudgementsandeffectivefinancialdecisions,

leadingtoimprovedself-managementoffinancesovertime.

ThefinancialliteracyeducationprojectwaspartoftheStGeorgesustainabilityagendaand

wasanextensionofthepartnershipwhichalreadyexistedbetweenStGeorgeBankand

WesleyMissionthroughdonationstofinancialeducation.TheInChargeofMyMoney

programwasbasedonarangeofexistingresourcesandgovernmentproducedmaterials

Page 41: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

41

whichweremodifiedandextendedtoaddressthemoreacuteneedsofWesleyMission’s

clientele.

WesleyMission’sfinancialcounsellorsworkingacrosstheSydneymetropolitanand

Newcastleareaswereconsultedtoobtainanaccuratepictureoftheneedsoftheirclients.

Thefinancialcounsellorsidentifiedthemajorcontributortofinancialstressasbeingpoor

moneymanagementthroughtherelianceoncreditcards.Furthermore,aninternetsurvey

wasdevelopedtoexplorewhatsupportfinancialcounsellorsneededtodeliverafinancial

literacyeducationprogramandalsotoidentifywhatareaofunderstandingwasapriority

forthedevelopmentofprogrammaterials.Thesurveyhighlightedthatpersonalmoney

managementandmanagingdebtneededtobeapriorityforinitialprogramdevelopment.

TheprogramwasdevelopedusingtheadultlearningprinciplesofKnowles(Knowles,

Holton,&Swanson,1998)andfacilitatorsweretrainedtoincorporatethismodelintheir

deliveries.Theprogramwaspilotedfora12monthperiodinliaisonwithfinancial

counsellorsandcommunitygroups.WesleyMission’sfinancialcounsellorsweretrainedto

familiarisethemwiththenewmaterialsandresourcesforeffectivedeliverytomeetthe

needsofWesleyMission’sclients(WesleyMission,2011).

Page 42: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

42

Program Description

InChargeofMyMoneywasdesignedbyMaisieWarburtonandlaunchedin2011.Theprogram

providesinformationabouthowtobettermanagemoney.Itisdesignedtobepractical,easyto

understand,andensureparticipantsareequippedwiththerelevantskillstomanagetheir

money.Thematerialsforpresentersincludefacilitatorguides,PowerPointslides,andaDVD.

Programparticipantsaregivenworksheets,acalculator,apen,amoneybox,andasmall

notebook.ThelanguageusedinthematerialsisataYear10literacylevel.In2013this

programreceivedtheHighlyCommendedAwardfortheannualASICMoneySmartAwards

(WesleyMission,2016b).

ThecontentofWesley’sfinancialliteracyprogramisarrangedintothreemodulesandeach

modulehascorrespondingworksheetstoengageparticipants.Thematerialsincorporate

informationfromtheMoneySmartwebsitewithpermissionfromAustralianSecuritiesand

InvestmentCommission(ASIC)(WesleyMission,2016a).

Page 43: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

43

Thethreeindependentmodulesoftheprogramare:

Module1:Knowit—Yourincomeandexpenses

Thissectionexaminessourcesofincomeandexpenses,whatpeoplespendtheirmoneyon

andhowtheypayforthings.

Part1:Knowyourincomeandexpenses

Part2:Knowwhereyourmoneygoes

Part3:Knowhowyoupayforthings

Module2:Doit—Managingyourmoney

Thissectionexamineshowtomanagemoneybyidentifyingpersonalgoals,saving,planning

andpreparingabudget.

Part1:Knowwhatyouwant—goalsetting

Part2:Payyourselffirst—savings

Part3:Planningandbudgeting—manageyourmoney

Module3:Reviewit—Managingyourdebt

Thissectionexploreshowpeoplemanagetheirdebtthroughexaminingtheirloansandthe

costofborrowing.Itprovidesanopportunitytoreviewexpensesandmanagedebtthrough

budgeting.

Part1:Gettingintoandoutofdebt

Part2:Knowtherealcostofborrowing

Part3:Monitorandmanageyourdebt

Part4:Wheretogoforhelp

Thekeymessagetoprogramattendeesistoapplywhattheylearntoeverydaylife.The

underlyingrefrainthatappearsconsistentlythroughoutallworksheetsis:Knowit,

Understandit,Doit,Reviewit.Thispracticalandsimplestructureisintendedtoreinforce

thelearningduringtheprogram(WesleyMission,2016a).

Page 44: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

44

Figure1:Knowit,understandit,doit,reviewitmodeloflearning

Theprogramalsoprovidesinformationon“wheretogoforhelp”andhighlightsthe

availabilityoffreefinancialcounsellingservices.Thereisanaddedbenefitofexposing

vulnerablepopulationstofinancialcounsellorsthroughthefinancialliteracyprogramas

manydonotknowthisfreeserviceexists.Thefacetofacecontactandexposureofthe

programencouragesprogramattendeestoaccessfinancialcounsellingandmakesitless

threateningtoseekthesupporttheyneed.

Programdelivery

TheInChargeofMyMoneyprogramisdeliveredbyqualifiedtrainingprofessionals,usually

financialcounsellors,andallmaterial,toolsandresourcesareprovidedfortheattendee.During

theevaluationstudy,allprogrampresenterswereeithercurrentWesleyMissionfinancial

counsellorsortheprogramdeveloperwhoisalsoafinancialliteracyeducator.Thepresentersare

trainedtodelivertheprograminanon-judgementalandnon-patronisingmanner,offering

Knowit

DoitUnderstand

it Reviewit

“TheprogramgotmethinkingaboutmoneyandhowIusedit.BeforeIjustuseditwithoutthinking,butnowIthinkaboutwhatIneedthemoneyfor.Inevertookthatintoaccountbefore.NowI'mholdingontomymoneymore.Itusedtoallbegonein2-3days,butnowmyunemploymentbenefitlastsaweekanda

half…”

Page 45: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

45

informationandsuggestionswithouttellingattendeeswhatthey‘should’do.Thepresentersare

carefultorespecttheagency,opinions,andlifesituationsofprogramattendees.Flexibilityof

deliveryisalsocrucialtotakeintoconsiderationtheattentionspanofclientswithmentalhealth,

disability,addictionandgamblingissues.Presentersencourageparticipantstoaskquestionsand

discusstheirlifesituationsduringthesessions.Theprogramcanbedeliveredinoneday,butis

mostcommonlydeliveredweeklyinthreesessionsofthreehourseach.Eachsessioncoversone

module.Duringthisstudy,however,theprogramwasdeliveredinfoursessionsratherthan

threetoallowtimeattheendofthelastsessionforthestudytobedescribedandfor

participantstocompletethequestionnaires.

Informalprogramevaluationshavesuggestedthattheprogramiswell-receivedbyattendees.

However,thisprogramhasneverbeenempiricallyevaluatedforitsefficacy.Thepurposeofthis

studywastoascertainwhetherthisprogramiseffectiveandwhereitsstrengthsandweaknesses

lie.

“TheprogramshowedmewhereIwasgoingwrongandmademethinkaboutit,beresponsibleforwhatIwasdoingand

thinkingabouthowtousemymoney,beconsciousofthebillsIhave,puttingawayextra,learningtomanagemymoney.Idon'ttakemoneywithmewhenIwalkpasttheshopsnow.Iusedtobuy6-7coffeesaweek,nowIonlyhave1or2aweek,andnowI

knowIcanaffordit.”

Page 46: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

46

In Charge of My Money Program Evaluation

Goals of the Study, Key Factors & Outcome Measures

TheprimarygoalofthisstudywastoassesstheoutcomesofWesleyMission’sfinancial

literacyprogramintermsofpositivechangestofinancialbehaviourinalow-income,

vulnerablepopulation.

Morespecifically,thisstudyaimedto:

- Ensuretheprogramcontentisrelevanttothecohortandevidence-based.

- Identifywhethertheprogramfacilitatessustainedimprovementinfinancial

behaviours.

- Estimatewhetherparticipantknowledgeandunderstandingareretainedafter

severalmonths.

- Ascertainhowincorporatingapracticalstrategy,suchastextmessagereminders,

increasesthechanceofsustainedchange.

Inordertoachievethesegoals,avarietyofparticipantfactorswereinvestigated,suchas:

• Demographics(gender,age,countryofbirth,highestlevelofeducation,income

bracket,mainsourceofincomebeinggovernmentbenefits).

• Programengagement(i.e.howmanyworkshopsattended,levelofmotivationduring

theprogram,whetherattendingtheprogramwascompulsory,andparticipants’

programrating).

• Changeinfinancialknowledge,attitudes,self-efficacy,andemotionalspending.

Themainquantitativeoutcomesthatweremeasured5-7monthsaftertheprogramwere:

• Changesinfinancialbehaviours:

- Changeinmoneymanagementbehaviour

- Changeinlevelofsavingbehaviour

- Changeindegreeofplanningbehaviour

Page 47: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

47

- Changeinlevelofmonitoringexpenses

• Changeinreachingfinancialgoals:

- Howmuchtheprogramassistedinachievinggoals

- Theextentofprogresstowardsthespecificgoalidentifiedduringthe

program

• Changeinfinancialstrain:

- Thenumberofproblemsduetolackofmoney

- Thedegreeofdifficultyinpayingbills

- Howmuchmoneyisleftoverbythenextpayday

- Frequencyofaskingfamilyformoney

- Numberoffinancialproducts(i.e.financialinclusion).

• Changeinlevelofdebt

Thestudyinvestigatedalltheseparticipantandoutcomefactors,andtherelationships

betweenthem.

Inadditiontothesequantitativeoutcomes,participantsansweredseveralquestionsintheir

ownwords,therebycontributingrichqualitativedataasasupplementtothequantitative

data.Theoutcomesthatwereabletobededucedfromparticipants’qualitativeresponses

were:

- Theprogramelementsthatweremostrelevantto,andusefulfor,program

attendees.

- Programcomponentsthatwerebestremembered.

- Howtheprogramcanbeimproved.

Obtainingtheopinionsofprogramattendeeswascrucialinthisevaluationstudybecause

theprogramistargetedataspecificaudience(i.e.themostvulnerableinsociety).Aswillbe

demonstratedinthefollowingsection,studyparticipantsweredrawnfromvery

disadvantagedandmarginalisedgroupswithinthegeneralpopulation.

Page 48: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

48

Participant Demographics

Ofthe128participantswhowererecruitedintothestudyatTime1,63completed

questionnaires5-7monthslater.Datafromninestaffparticipantswereexcludedfromthe

analysesbecausethosestaffweresosocio-demographicallydistinctfromtheother

participants.Hence,thefinalsamplesizeforthefullevaluationwas54.Therewereslightly

moremalesthanfemales,duetosomeprogramsbeingrunthroughservicestohomeless

men.TheparticipantsweremostlyborninAustraliaorborninEnglish-speakingcountries

(78%and92%respectively).ThenumberofparticipantsidentifyingasAboriginal(11%)was

overrepresentedcomparedtotheNSWstateaverageof3%(AustralianBureauofStatistics,

2013).NoparticipantsidentifiedasbeingTorresStraitIslander.Participantshadlowerlevels

ofeducationcomparedtotheaverageAustralian.In2015,26%oftheAustralianadult

population’shighesteducationalattainmentwasYear11orless,comparedtoatleast41%

ofparticipantsinthisevaluation(AustralianBureauofStatistics,2015a).Fiftyninepercent

oftheAustralianadultpopulationcompletedanon-schoolqualification,comparedto45%

ofstudyparticipants(AustralianBureauofStatistics,2015a).

ParticipantsreceivedfarlowerincomesthantheaverageAustralian.Mostparticipants

(77%)receivedanannualincomeoflessthan$32,000in2016.Thisissubstantiallylower

thantheaveragegrosshouseholdincomeforNSWin2013-2014,whichwas$111,280,and

theaverageequivalisedannualdisposablehouseholdincomeforAustraliansin2013-2014,

whichwas$51,896(AustralianBureauofStatistics,2015b).Governmentbenefitswerethe

mainsourceofincomefor85%ofparticipants,comparedto25%ofhouseholdsinAustralia

in2013-14(AustralianBureauofStatistics,2015b).Whenaskedaboutdebt,70%of

participantssaidtheywereindebtwhentheycommencedtheprogram.Mostdidnothave

childrenlivingwiththem.Participantsweredrawnfromprogramsdeliveredincentres

addressingissuesofdrugandalcoholabuse,gamblingaddiction,homelessnessandfamily

dysfunction.Someofthesecentreswereresidentialrehabilitationcentres,asshownin

Table1.Morethanhalfthestudyparticipants(54%)weredealingwithsubstanceabuse

Page 49: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

49

and/orgamblingaddictions.Thirtyfivepercentofparticipantswerehomeless,and11%

werereceivingassistanceintheirabilitytocarefortheirchildren.

Table1:Participantdemographics

Participantcharacteristic Numberofparticipants

Percentageofparticipants

Gender

Male 32 59%

Female 22 41%

BorninAustralia 39 78%

BorninAustralia,NewZealand,UK,Ireland,USorCanada

46 92%

Aboriginalorigin 6 11%

Agebracket

21-25 7 13%

26-30 10 19%

31-35 6 11%

36-40 13 24%

41-45 6 11%

46-50 3 6%

51-55 4 7%

56-60 3 6%

61-65 2 4%

Highestlevelofeducation

Year7-10 22 41%

Year11-Year12 7 13%

TAFEortechnicalcollege 16 30%

Universityundergraduate 7 13%

Universitypostgraduate 1 2%

Incomebracket(annual)

Noincome 1 2%

$1–10,399 2 4%

$10,400–15,599 12 26%

$15,600–20,799 15 32%

$20,800–31,199 6 13%

$31,200–41,599 3 6%

Page 50: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

50

Participantcharacteristic Numberofparticipants

Percentageofparticipants

$41,600–51,999 2 4%

$65,000–77,999 1 2%

$78,000–103,999 2 4%

$104,000+ 3 6%

Mainsourceofincomeisthegovernment 45 85%

Typeofcentre

Addictionrehabilitation 29 54%

Homelessness 19 35%

Familysupport 6 11%

Numberofchildrenunder18yearsoldlivingwithyou

0 36 69%

1-2 11 21%

3-4 4 8%

5-6 1 2%

Figure2:Ageofparticipants

0% 5% 10% 15% 20% 25% 30%

21-25

26-30

31-35

36-40

41-45

46-50

51-55

56-60

61-65

Percentageofparvcipants

Page 51: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

51

Figure3:Participants'highestlevelofeducation

Figure4:Annualincomeofparticipants

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Year7-10

Year11-Year12

TAFEortechnicalcollege

Universityundergraduate

Universitypostgraduate

Percentageofparvcipants

0% 5% 10% 15% 20% 25% 30%

Noincome

$1–10,399

$10,400–15,599

$15,600–20,799

$20,800–31,199

$31,200–41,599

$41,600–51,999

$65,000–77,999

$78,000–103,999

$104,000+

Percentageofparvcipants

Insum,theparticipantsofthisevaluationwereamongstthe

mostsociallyandeconomicallydisadvantagedandvulnerablewithintheAustralianpopulation.

Page 52: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

52

Figure5:Issuesaddressedbycentrewhereprogramwasdelivered

0% 5% 10% 15% 20% 25% 30% 35% 40%

Homelessness

Substanceabuse-residenval

Substanceabuse-non-residenval

Substanceabuse&gambling-residenval

Familysupport

Percentageofparvcipants

Page 53: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

53

Program Evaluation Research Methodology

Procedure

Studyparticipantsweresurveyedattwoseparatetimes:

• Attheendofthelastsessionoftheprogram(Time1)

• 5-7monthsaftercompletingtheprogram(Time2).

At the end of the program (Time 1)

BetweenJulyandDecember2015aresearcherfromMacquarieUniversityattended18

programdeliveriesinmetropolitanareasofSydneyandalsoinCessnockandWollongong.

TheseprogramsweredeliveredbyWesleyMissionfinancialliteracyeducatorsandfinancial

counsellorsatcollaboratingresidentialanddrop-inrehabilitationcentresforaddiction,

homelesscentres,andfamilysupportcentres.Theprogramswerealsodeliveredtosome

MissionAustraliaandWesleyMissionstaff,sothesewereincludedintheTime1data

collection.

Attheendofthelastsessionoftheprogram,aftertheprogramfacilitatorhadfinished,the

facilitatorinvitedtheresearchertojointhegroup.Theresearcherthenexplainedthestudy,

answeredanyquestions,andhandedoutconsentformsandpaperstudyquestionnaires.

Onceparticipantshadcompletedtheconsentformsandquestionnairesbyhand,theygave

thembacktotheresearcher.Theresearcherhandedeachparticipanta$20groceryonlygift

voucherinappreciationoftheirtime.Thequestionnaireswerelaterenteredintoa

computerforanalysisbytheresearcherwhoattendedtheprogramsessions.

Page 54: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

54

Time1Participants

Atotalof128participantscompletedquestionnairesatTime1,including15staff(12%).It

wasexplainedthatparticipationinthestudywasvoluntaryandthreepeoplechosenotto

participate,givinga97.71%studyparticipationrate.

Several (5-7) months after the program (Time 2)

BetweenFebruaryandMay2016thesameresearcherattemptedtocontactallTime1

participantsbyemailand/ortelephone,atleastfivemonthsfromthetimeofthefirst

survey.Ifanemailaddresshadbeenprovidedthenparticipantswereemailed,requesting

thattheyparticipateinthesecondpartofthestudy.Intheemailparticipantswereasked

whenwouldbethebesttimefortheresearchertocall,andaskingthemtoconfirmtheir

telephonenumber.Alinktotheonlinesurveywasalsoprovidedintheemailsotheycould

completeitonlineiftheypreferredthattocompletingitbytelephone.Eachemailwas

individualisedasparticipantshadtobegiventheiruniqueidentificationnumberforthe

survey.Toensureconfidentiality,identificationnumberswereusedtomatchparticipants’

responsesfromTime1toTime2ratherthannames.Ifamobilephonenumberwasgivenat

Time1,thentheresearchersentatextmessageremindingtheparticipantoftheirinclusion

inthestudyandaskingwhenwouldbethebesttimetocalltodothesurveyoverthe

phone.Afewdayslater,theresearchertriedcallingtheparticipant,regardlessofwhether

ornottheyrespondedtothetext.

Forthetelephonesurvey,theresearcheraskedparticipantsthequestionsandrecorded

theiranswersdirectlyintoacomputer.Thistookapproximately15minutes.Participants

alsohadtheoptionofdoingthesurveyonlinebythemselves,butnotmanyexercisedthis

option.Eachparticipantwassenta$30groceryonlygiftvouchertothankthemfortheir

time,whethertheycompletedthesurveybytelephoneoronline.

Page 55: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

55

DifficultiescontactingparticipantsatTime2

Itprovedverychallengingtocontactstudyparticipantsafewmonthsaftertheyhad

completedthefinancialliteracyprogram.Manyoftheemailaddresses‘bounced’andmany

ofthephonenumbersweredisconnectedornotreceivingincomingmessages.Oftheemail

addressesthatdidnotbounce,fewpeopleresponded,andofthephonenumberswhereit

waspossibletoleaveamessage,fewreturnedthecall.Theresearcherresentemailsatleast

twiceandtriedcallingeachparticipantatleastthreetimesatdifferenttimesofthedayand

overaperiodofafewweeks.Sinceparticipantsinrehabilitationresidentialprogramswere

onlypermittedtousetheirphonesonweekends,theresearchermademanyofthephone

callsafter5pmonFridays,inthehopeofincreasingthechancesofmakingcontact.

Althoughthisrepeatedemailingandcallingseemedtoirritatesomeoftheparticipantswho

werecharitystaffmembers,itdidnotappeartobothertheotherparticipants,mostlikely

becausetheydidnotreceivetheearlieremails,textsorvoicemessages.Acoupleof

participantsmentionedthattheyhadbeeninrehabilitationformonthsandhadabacklogof

hundredsofemails.Oneparticipantcompletedthesurveyonlinetwomonthsafterthelast

attempttocontacthim.Someparticipantswerehappytocompletethesurveywhenthe

researcherfinallymadecontactbymobilephone,eventhoughshehadhadnoresponseto

textmessagesandvoicemessages.Otherparticipantsaskedtheresearchertocallbackata

certaintime,butthendidnotanswertheirphoneatthattimeorinthedaysafterwards.

Twoparticipantswereinhospitalwhentheresearchercalled,oneofthembeingunableto

dothesurveyduetopain.Anotherparticipantwasreportedbyhismothertobenon-

contactablebecausehewasinprison.Therewasconfusionabouttheidentityofone

participantbecauseanotherstudyparticipanthadgivenhimhismobilephoneafterthe

program,andtheyhappenedtohavethesamefirstname.

Time2participants

Ofthe128Time1participants,only63completedtheTime2telephone/onlinesurvey.Four

peopleansweredtheirtelephonebutdeclinedtoparticipateintheTime2survey.The

Page 56: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

56

responserateforTime2is49%ofthosewhocompletedtheTime1questionnaire.

However,itisunknownhowmanypeopledidnotparticipateatTime2duetochoiceand

howmanysimplydidnotreceiveanymessage,despiterepeatedandvariedattemptsto

contactthem.

Itwasapparentfromcommentsmadebyparticipantsthattheonlyreasonasmanyas49%

agreedtocompletethesurveyatTime2wasbecausetheywantedthe$30giftvoucher.

Withoutthatincentive,theresponseratewouldhaveundoubtedlybeenmuchlower.

Itwasdecidedthatthestaffparticipantsweretoodemographicallydissimilartotheother

participantsinthestudytobeabletocombinetheirresults,sothedatafromtheninestaff

whocompletedtheTime2surveywasexcludedfromfurtheranalyses.TheTime2sample

sizewasthereforereducedto54.

TheTime2participantswerenotreliablydifferentfromthosestudyparticipantswhoonly

completedtheTime1questionnaireonage,income,ormotivationlevel.However,those

participantswhocompletedTime2questionnairesweremarginallymorehighlyeducated

thanthosewhoonlycompletedTime1questionnaires(t(124)=-1.86,p=.065).

Consent,confidentiality,andvouchers

ThequestionnairesandstudyprocedurewerereviewedbytheMacquarieUniversityHuman

EthicsCommitteeandtheprojectwasgrantedethicsapproval(ethicsapprovalnumber

5201500453).AtTime1theMacquarieUniversityresearcherexplainedthestudyandits

purposetothoseattendingtheInChargeofMyMoneyprogram,andstatedthat

participationinthestudywasvoluntaryandthatparticipantscouldwithdrawatanytime.

Programattendeeswerethenaskedtosignandreturnaconsentformiftheywishedtobe

involvedinthestudy.

Names,contactnumbersandemailaddresseswerecollectedatTime1sothatthe

researchercouldcontacttheparticipantsinseveralmonths’time.Toprotectparticipants’

Page 57: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

57

confidentiality,thesecontactdetailswerewrittenonaseparatesheetofpaperandentered

intothecomputerindependentlyofthesurveyresponses.Eachparticipantwasassigneda

uniqueIDnumbersothattheirresponsesatTime1couldbecomparedwiththoseatTime

2,withoutusingnames.PostaladdresseswereonlycollectedatTime2sothatthe$30gift

vouchercouldbeposted.

Vouchersweresentviaregisteredmail,toensuretheirdelivery.Despitethis,fivevouchers

werereturnedtosenderandparticipantsneededtobecontactedagainandthevouchers

resent.Oneparticipantaskedforhisvouchertobesenttothehospitalwherehewas

currentlyapatient.TwoparticipantsaskedtocollectthevouchersfromWesleyMission,

onebecauseheneedediturgentlyandtheotherbecausehewashomelessandhadno

address.

Questionnaires

ThequestionnairesatTime1andTime2werealmostidentical.Themaindifference

betweenthemwasthattheTime1demographicquestionswerenotrepeatedintheTime2

questionnaire.Thecontentofthequestionnairesdrewuponvalidatedinstrumentsusedin

long-runningfinancialliteracyprogramssuchasSaverPlus(Russell,Stewart,Cull,2015)and

theUKfinancialcapabilitysurvey(Atkinson,2011).Relevantadaptationsweremadetosuit

thecohortandcontextoftheInChargeofmyMoneyprogram.

Theareascoveredbythequestionnaireswere:

• Demographics(gender,age,placeofbirth,highestlevelofeducation,numberof

childrenlivingwithyou,mainsourceofincome,incomebracket)

• Reasonsforattendingtheprogramandmotivationlevel

• Programratingandfeedback

• Financialself-efficacy(i.e.self-confidenceinthefinancialdomain)

• Moneybehaviour

• Generalfinancialknowledge

• Personalfinancialknowledge

Page 58: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

58

• Degreeofemotionalspending

• Financialstrain

• Howoftenparticipantsaskfamilyorfriendsformoney

• Financialinclusion(i.e.thenumberoffinancialproducts)

• Financialresilience(i.e.couldparticipantsraisemoneyforanunexpectedexpense)

• Howdifficultithasbeentopaybillsandcoverexpenses

• Howtheparticipanthasbeenmonitoringtheirspending

• Howtheparticipanthasbeensavingtheirmoney

• Howfarinadvancetheparticipantplanstheirspending

• Howoftentheyhavemoneyleftoverbythenextpayday

• Questionsaboutfinancialgoals

• Whetherornottheyhavemoneyputasidetocoverexpensesinanemergency

• Changeindebtfivemonthsaftertheprogram.

Thequestionnairescollectedinformationthatwasbothquantitative(i.e.abletobe

measuredinnumbers)andqualitative(i.e.quotesfromparticipants).Bygathering

quantitativedata,improvementsinfinancialbehaviours,knowledge,attitudes,self-efficacy

andemotionalspendingwereabletobemeasuredandstatisticallyanalysedtoascertainif

thechangeswerestatisticallysignificantandthuslikelytobegenuinechanges.

Thequalitativedataprovidedrichinformationregardinghowtheprogramactuallyimpacted

participants’livesandabouttheaspectsoftheprogramthatweremostrelevantand

helpful.AtTime1participantswrotetheiranswerstothequalitativequestionsonthe

questionnaire,andatTime2thosewhocompletedthesurveyonlinetypedintheiranswers.

ForthosewhodidtheTime2surveyviathetelephone,theMacquarieUniversityresearcher

typedparticipants’wordsintothecomputerastheywerespeaking.

ThesewerethequalitativequestionsparticipantswereaskedatTime1:

• Whydidyougivetheprogramthisscore(rating)?

• Howcouldtheprogrambeimproved?

Page 59: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

59

• Whatisyourfinancialgoal?

AtTime2participantswereasked:

• Whatdoyourememberfromtheprogram?

• Whywouldorwouldn’tyourecommendthisprogramtoyourfriends?

• Whatpartoftheprogramhelpedyouachieveyourgoalsthemost?

Data Analysis

ThequantitativeanalyseswereconductedusingtheSPSS21statisticalprogram.Descriptive

statisticswereusedtofindthepercentagesofparticipantsthathadparticular

characteristicsandtheaveragescoresforthemeasures.RepeatedMeasuresANOVAswere

usedtodetermineifchangesinthefactorsmeasuredovertime(e.g.,tofinancial

behaviours)werestatisticallysignificant(i.e.thattheyhada95%probabilityofnot

occurringbychance).Pearson’scorrelationswereconductedtoexploreassociations

betweenparticipantfactorsandimprovementsinfinancialliteracy.Multipleregressionwas

employedtoascertainwhichparticipantfactorsweremoststronglyrelatedtothefinancial

literacyoutcomes,afteraccountingforotherfactors.Thequalitativedatawereanalysedby

themesandbycountingthenumberoftimeseachtopicwasmentionedbyparticipants.

Quotesfromstudyparticipantsareprovidedasexamplesoftheirfeedback.Aplainlanguage

explanationofthemeaningofeachstatisticaltestwillbeprovidedwiththestatistical

analyses.

Inshort,thereareanumberofdifficultiesandcomplicationsconductingresearch

withdisadvantagedandvulnerablecohorts

Page 60: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

60

Program Evaluation Results

TheprimaryaimofthisstudywastoevaluatetheeffectivenessoftheInChargeofMy

Moneyprogramforvulnerablepopulations.However,otherareasofinterestwerealso

explored.

Theresultswillbepresentedinfivemainsections:

• Participantprogramengagement,feedbackandknowledgeretention

• Changesinfinancialbehavioursandreachinggoals

• Changesinfinancialknowledge,attitudes,self-efficacyandemotionalspending

• Changesinfinancialstrain

• Theparticipantfactorsmostrelatedtochangesinfinancialbehavioursandstrain

• Theeffectivenessoftextmessagenudgesinmaintainingbehaviourchangeover

time

Participant Engagement with the Program

ThereasonswhyparticipantsattendedtheInChargeofMyMoneyprogramwerecollected

toinvestigatewhetherornottherewereoutcomedifferencesbetweenthosewhoattended

theprogramvoluntarilyandthosewhoseattendancewascompulsory.

Reason for attending the program

AsseeninTable2,theprimaryreasonwhyparticipantsattendedtheInChargeofMy

Moneyprogram(65%)wasbecauseitwasacompulsorypartofanotherprogramtheywere

enrolledin.Someparticipants(18%)attendedbecauseitwassuggestedbytheircase

worker.

Page 61: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

61

Table2:Reasonswhyparticipantsattendedtheprogram

Reasonforattendingtheprogram Numberofparticipants

Percentageofparticipants

ItwasacompulsorypartofanotherprogramIwasattending

35 65%

Itwassuggestedbymycaseworker 10 18%

Itwassuggestedbymyemployer 1 2%

Iheardaboutitandwantedtoattend 8 15%

Main money issue

Attheendoftheprogram,participantswereaskedwhatthemainmoneyissuewasthat

theyhadhopedtheprogramwouldassistthemwith.AsseeninTable3below,themain

moneyissuesrelevanttotheparticipantswererelatedtobudgetingandmoney

management.

Figure6:Whatwasthemainmoneyissueyouhopedthisprogramwouldhelp?

0% 5% 10% 15% 20% 25% 30% 35%

Helpmetomakeendsmeet

Helpmewithbudgevng

Helpmetosave

Helpmetosetmoneygoals

Helpmetoreducedebt

Helpmetomanagemoneybexer

Helpmetotrackexpenses

Other

PercentageofParvcipants

Page 62: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

62

Level of motivation

Approximatelyhalftheparticipantsweremoderatelymotivated,withnearlyonequarter

onlybeingalittlemotivated,andonefifthbeingextremelymotivated(seeTable4).

Therefore,themotivationlevelofparticipantswasgenerallyquitegood.Itwaspredicted

thatparticipantswithhigherlevelsofmotivationwouldhavebetteroutcomesthanthose

withlowerlevelsofmotivation.

Figure7:Participants'levelofmotivationduringtheprogram

Program attendance

Thenumberofprogramworkshopsattendedwasofinterestbecauseitwassurmisedthat

themoreworkshopsaparticipantattended(i.e.thegreatertheprogramdosage),thebetter

theoutcomeswouldbe,astheprogramwouldhaveagreaterimpactonparticipants.As

showninTable5,themajorityofparticipants(67%)hadattendedallfourworkshops,and

onlyfiveparticipantshadonlyattendedoneworkshop.Therefore,programattendance

ratesweregenerallyquitehighandtherewasnotmuchvariationinprogramdosage.

0% 10% 20% 30% 40% 50% 60%

Notatallmovvated

Alixlemovvated

Moderatelymovvated

Extremelymovvated

Percentageofparvcipants

Page 63: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

63

Table3:Numberofworkshopsattendedbyparticipants(dosage)

Numberofworkshopsattended

Numberofparticipants

Percentageofparticipants

1 5 9%

2 7 13%

3 6 11%

4 36 67%

Participant Feedback on the Program

Program rating at the end of the program

Attheendofthelastworkshopoftheprogram,participantswereaskedtoratetheprogram.Participantsratedtheprogramveryhighly,withanaveragescoreof8outof10(seeTable6).

Figure8:Outof10,howwouldyouratethisprogram?

Whenaskedwhytheyhadgiventheprogramthatscore,29participants(54%)commented

thattheinformationwashelpfulorinterestingorrelevant.Forexample:

“Allusefulinformation”

“Bearinginmindthetargetaudienceitiswellsuited”

“BecauseitwasinformativeandmademethinkbeforeIspent”

0

2

4

6

8

10

12

14

16

18

1 2 3 4 5 6 7 8 9 10

Num

bero

fparvcipants

Score

Page 64: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

64

“Itwasclear,conciseandbestofall,relevant”

“Ifoundittobeinformative,andinsomecases,confronting”

“Ifounditveryinformativeanduseful”

“Iknewalotoftheinformationalready,butrevisitingitwashelpful”

“Itgavemeideasaboutgettingmyfinancesbackinorder”

“Ithasalotofusefulinformationtohelpthosewhowanttohelpthemselves”

“Wasveryinformativeandcoveredabroadrangeofareas”

“Veryinformativeaboutdebt,interestratesanddifferentformsofdebt/credit”

Eightparticipantscommentedthattheprogramhelpedthemlearnspecificmoneymanagementskills:

“ItwasveryinterestingandIlearnedalotofinformationabouthowtoreallymanagemymoneybetterthanIhavebeen”

“Ithelpedmetolearnaboutmanagingmymoney”

“BecauseIlearntalotofskillssurroundingmydebts”

“Gavemeincentivetosave,thinkwhatIamspendingon(Iamanimpulsiveandcompulsivespender)andbudgetinginformation,whichIwilluse”

“Becausetheprogramhelpedmelearnhowtosavemoney”

“HelpmerealizehowIcanmanagemymoney”

“Becausetheygavemetheinformation.Learnaboutbudgetingandmanagingmoney”

Severalparticipantsmadepositivecommentsregardingthepresentersorpresentation:

“[Facilitator’sname]’sknowledge”

“Facilitatorwasveryinformativeandusedeasilyunderstandableterms”

“Greatpresenter,presentation”

“Itwasveryeducationaland[facilitator’sname]presenteditreallywell.Answeredallmyquestions”

“[Facilitator’sname]wasgreat”

“Veryinformative,wellpresentedandspokenby[facilitator’sname]:)”

“Itwasverywellconstructedandinformative.Verywellpresented”

“Visualtools,lecturer,andinformation”

Page 65: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

65

Sevenparticipantsgavelesspositivereasonsfortheirscore:

“BecauseIwasn'tthatinterested”

“Wasn'trelevantatallforme”

“Ibasicallyknowalltheinfo”

“Becauseitneedssomegames,moreinteractive”

“Itmaybeexplainedinalittlemoredetail”

“Itonlyhelpedmeabitsofar”

“Speakercouldbetterexplainthings.Savingmentionedbutnotreallyhowtoorhowmuchyoushouldsave”

Figure9:Whydidyougivetheprogramthatrating?

Recommending the program to friends

Severalmonthsaftercompletingtheprogram,participantswereaskediftheywould

recommendtheInChargeofMyMoneyprogramtotheirfriends.Themajority(85%)said

thatyes,theywouldrecommendtheprogramtotheirfriends.Onlyfourparticipants(7.5%)

saidtheywouldnotrecommendittotheirfriends.Anotherfourparticipants(7.5%)saidit

woulddependonwhetherornottheirfriendsneededtheprogramorwereinterestedinit.

Thefactthat85%ofparticipantswouldrecommendtheprogramtotheirfriendsis

consistentwiththehighprogramratingnotedearlier.

0% 10% 20% 30% 40% 50% 60%

Informavonrelevant/interesvng/helpful

Learntmoneymanagementskills

Goodpresentavon/presenters

Notinterested/helpful

Percentageofparvcipants

Page 66: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

66

Whenaskedwhytheywouldorwouldnotrecommendtheprogram,participantsgavethe

followingresponses,consistentwiththecommentsontherelevanceandusefulnessthe

programmadebyparticipantsseveralmonthsearlier:

Yes,Iwouldrecommendtheprogramtomyfriends

“Definitely,itjustsortofhitshome,makesyouthinkaboutthingsmore.Iusedtoliveonedayatatime,butnowI’mthinkinginthenext20years”

“Forsure,becauseitislesssortoftextbook,it’smorerelevantthansomeoftheothercoursesI’vedone(I'vedonebusinesscoursesbefore)butthiswasmorerelevanttomySES

level”

“Yesasitwasveryinformative”

“Yesbecauseitmakesfinancialplanningeasierandlessstressful,andalsoinformsyouofhiddeninterestinplaces”

“Yesbecauseitmayhelpthem”

“YesIsupposeIwould.Iworkedinthebankfor10yearsbutIstilllearntstuff”

“YesIwould,becauseit'spracticallikealotofpeopledon'tthinkaccordingtowhatwe'velearnt,notmanythinklikethat,theyjustspendwhattheygetwithoutthinkingaboutit.This

programteachesyoutothinkabouthowyouspendyourmoney”

“Yes,becauseIthinkI'mmoneywisebuttherearealotofpeoplethataren't.It'sgoodbasicinformationthateveryoneshouldknow”

“Yes,becauseitdoesgiveyouaninsightintowhatyouweredoingwrong,spendingmoremoneythanyou'vegot”

“Yes,becauseitwasbeneficialtomesoitmightbebeneficialtothem”

“Yes,becauseitwaseasytounderstand,helpful,simpleandclear,nottoocomplicatedatall”

“Yes,I’vetoldmysonanddaughteraboutit.Becauseit’suseful,theneedtobudgetandsticktoit”

“Yes,Ithoughtitwaswonderful,quickandsimplebutlearntindepthhowtomanageyourmoney,andtolearnaboutthebudgetwebsite”

“Yes,I’vetoldafewpeopleaboutit.Iunderstoodit.It’seasiertounderstandthanotherprograms”

“Yes,wastalkingtomynieceaboutittheotherday.Atthetimeitwasveryhelpfulanditdidmakemethinkaboutthings”

Page 67: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

67

“Yes!Asitreallymakesyouthinkaboutthemoneyyou’respendingandhownottogetintomoredebt”

“Yes.ItwasprettybasicbutIneededtohearit”

“Yes.Itwasveryinformative,withuseful&practicaltipsonmoneymanagement”

No,Iwouldnotrecommendthisprogramtomyfriends

“Ihavealotoffriendswhoarebusinesspeoplesotheydon'tneedit”

“No,ithasn'tgotenoughstrategies”

“Nobecausemostofmyfriendsworkandhavetheirownfinancialplanners–theydon'tneedit”

“Nottomyfriends,butbeneficialforthoseongovernmentbenefitsorthosewhoconsistentlystrugglefinancially”

“Mostdefinitely.Ifoundwithmedoingthecourseit'staughtmetofocusonwheremymoneyisgoingandwhereI'mspending.Idon'tjustthrowcrapinthetrolley,likechocolatesandlollies,anymore.NowIaskmyself,‘doIneedthat?’andIthink,‘noIdon’t’,soIputitback.It'salsotaughtmetoplanmymeals.”

Page 68: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

68

Itdepends

“IftheyneededhelpIwould,becauseitgivesadviceaboutmoney”

“IftheywereinterestedIwould”

“Itdependsonthem-ontheircircumstances,iftheyneedit”

“Possibly,ifavailable,theyshoulddoit,mostarealreadyawareoftheirspending,butlittle

thingshelpevenpeoplewhoareOKwithmoneye.g.budgeting,beingmoreconsciousof

whatyouarespending,thenitbecomesahabittothinkbeforespendyourmoney”

Suggested areas for improvement

Whenasked“WhichpartsoftheInChargeofYourMoneyProgramneedtobeimproved”,

halfansweredthatnoimprovementisneeded,theylikeditjustthewayitis(seeTable4).

Table4:Whichpartsoftheprogramneedtobeimproved?

Areasforimprovement Numberofparticipants

Percentageofparticipants

Noimprovementneeded,Ilikeitjustthewayitis

28 52%

ThePowerPointslides 8 15%

Theinformationcouldbemorerelevant 6 11%

Other 5 9%

Thepresenters 4 7%

Theworksheets 3 6%

Theknowit,understandit,doit,reviewitstructure

0 -

Insum,most(85%)ofparticipantssaidtheywouldrecommendtheprogramto

theirfriendsbecauseitiseasytounderstand,relevant,andhelpful.

Page 69: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

69

TheeightpeoplewhothoughtthePowerPointslidesneededtobechangedsaidthefollowing:

- “Addpictures/videosforstimulation”- “Couldbeeasiertoreadandunderstand”- “Icouldn'tseethem-toosmall”- “Lesscreditcardinfonoteligible”- “Morepictures”- “MostIcouldn'tread-toosmall”- “Theyarequiterepetitive”

Thoseparticipantswhoticked“other”asneedingimprovement,said:

- “Session3wasinformationoverload”- “Games,prizes”- “Moregroupparticipation”- “Theinformationcouldbemoreindepth”- “Toomuchrepetition”

Commentsfromthefourwhosaidthepresenterneededimprovement,were:

- “Boringandnotwellsuitedforpublicspeaking”- “explainedmore”- “Repeatsthemselvestoomuch”

Twoofthethreepeoplewhosuggestedtheworksheetsneededtobeimprovedsaid:

- “Iwasn'tsurewhattowrite”- “Therearen'tenoughnotesintheworksheets”

Nocommentsweremaderegardinghowtoimprovetherelevanceoftheinformation,

exceptthatcreditcardinformationwasnotrelevanttomostparticipantsastheywerenot

eligibleforacreditcard.

Insum,participantswereverysatisfiedwiththecontent,relevanceand

presentationoftheInChargeofMyMoneyprogram,ratingitveryhighly,onaverage,

andhavingfewcomplaints.

Page 70: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

70

What Participants Remembered 5-7 Months After the

Program

Knowledgeandunderstandingretentionfromtheprogramseveralmonthslaterwasakey

areaofinterest.Aneffectivefinancialliteracyprogramisonewherethebenefitsofnew

knowledgeandunderstandingendureovertimeratherthanbeingquicklyforgotten.In

ordertoexaminethisretention,participantswereasked5-7monthsaftercompletionofthe

InChargeofMyMoneyprogram,“Whatdoyourememberabouttheprogram?”

Almosttwothirdsofparticipants(65%)rememberedthattheprogramwasaboutbudgeting

andmoneymanagementgenerally,withnospecifics.Justoveraquarterofparticipants

recalledmoredetailedinformationaboutthedifferenttypesofloans,howtomanagedebt,

andthedangerofpaydayloans.Approximatelyonequarteralsorememberedspecific

adviceaboutbeingmindfulofspendinghabits,includingimpulsebuying,spending“leaks”,

askingoneself“doIreallyneedthis?”,onlytakingthemoneythatyouneedtotheshops,

andplanningwhatyouwillspendbeforegoingtotheshops.Almostaquarterrecalledtips

abouthowtosave,suchas“payyourselffirst”,putchangeinthemoneybox,saveevery

dollar,andeverylittlebitcounts.Fourparticipants(7%)mentionedresourcesandnow

knowingwheretogetfinancialhelp,suchastheMoneySmartwebsiteandawarenessof

financialcounsellors.Onlysixparticipants(11%)rememberedlittleornothingfromthe

program.

Page 71: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

71

Figure10:It'sbeen5monthssinceyoudidtheInChargeofMyMoneyprogram.Whatdoyourememberfromtheprogram?

N.B.Thesepercentagesdonotaddupto100%becausesomeparticipantsrememberedmultipleaspectsoftheprogram.

BudgetingandMoneyManagementGenerally

Sixty-fivepercentofstudyparticipantsrememberedthattheprogramwasaboutbudgetingandmoneymanagementingeneral,asshownintheseexamples:

“Budgetingandbeingmoreawareofwhat'sgoingoutversuswhat’scomingin”

“Budgetingreally,andtryingtostartsaving”

“Irememberittellingmehowtohandlemyfinances,thingstolookoutfor,likefees,andhowtocreateyourownbudget”

“Howtobudget”

“Reinforcingthefactthatmanagingmoneyneedsagoodbudget-differencebetweenincomeandexpenses.I’llnevergetapaydayloanafterthecourse”

“Howtoplanabudgetandkeepitsimple”

“Ittaughtmehowtofocusonwhatwasimportantandnot,andhowtobudgetbetterthanIwas”

“Itwaslikeabasiconhowtomanagemoney”

“Learningaboutdifferentwaystomanageyourmoney”

“Moneymanagement,saving,budget,saveadollareveryday”

0% 10% 20% 30% 40% 50% 60% 70%

Budgevng&moneymanagement

Spendinghabits

Loans&managingdebt

Howtosave

Resources

Notmuch

Percentageofparvcipants

Page 72: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

72

SpendingHabits

Twenty-sixpercentofparticipantsrecalledlearningaboutspendinghabitsorbecomingmoreawareofhowtheyspenttheirmoney:

“Basicallypointstothinkaboutwhenbuying,likeimpulsebuyingwasthebigthingItookawayfromthat”

“IrememberthatIwasshockedtoseejusthowmuchmoneyIwastedonthingsIdidn'treallyneed”

“BasicallysortofIsupposebemorecarefulwhenIspendmymoney.Insteadofbuying2ofsomethingonlybuyone.SonowIonlybuy1packofsmokesinsteadof2,soI’msmoking

lesstoo”

“Discoveringwheremymoneyleaks”

“Learningtobemoremindfulofspendinghabits”

“ThesedaysIhavemoneyleftover.IttaughtmehowtobudgetandhowtobemorecarefulinwhatIspendmymoneyon”

“Yougottoplanwhatyou'regoingtospendbeforeyougototheshops.NowIavoidgoingtoshopsthatsellavarietyofthings.Askyourself,like,doIreallyneedthis,question

yourselffirst”

Alot’shappenedsincethen(theprogram)asIwasinrehab.It’shardtobudgetbecauseI’mstillonlimitedmoney.Carproblems

havebeenexpensive.Irememberthespreadsheetandthewebsitetodoyourbudgetandenterexpenses.Iremember

[facilitator’sname],she’saverygoodpresenter.AlotwasstuffIalreadyknew,butit’shardtogettothepointofbeingabletohaveabudget.Ihadsomecreditcards,whichIamnowpayingoff.I’vereducedmydebtbyhalfbecauseIpaiditoffinonego.I

saved$450dollarsbydoingthat(insteadofpaying$10perfortnight).”

Page 73: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

73

Loans&ManagingDebt

For28%ofparticipants,informationabouthowtoreducedebt,thedifferencebetweenthetypesofloansandbeingmoreawareofinterestrates,werethetopicstheyremembered:

“AdsaboutbaddebtonTV,steerclearofthem,paydayloans,steerclearofthem,howimportantitistodoabudget”

“Gettingtoknowwhatourneedsare,andthetypesofloans,goodandbadloans”

“Learningaboutdifferenttypesofloansandwhatnottotouch”

“Someaccountsthatyoucanopenup,smallloansprogram(NILS)-IendedupgettingaNILSloan”

“Irememberthebudgetingaspectofit,abitaboutthehelpfinancialcounsellorscangiveyou,differentaccountsandwhattheirbenefitsare,andalsolookingatwhereyourmoneyisgoingandhowmuchisspentonthingsyoudon'tneedorotherpeople”

“Irememberitbeingmorehelpfulthananyotherprogramandhelpedmegetfamiliarwithinterestandcreditcards.Themostmemorablethingwasaboutdebt,it’snevertoolatetosortout

debt”

Page 74: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

74

HowtoSave

Tipsonhowtosavemoneywererememberedby24%ofstudyparticipants:

“Alotofdetailaboutsavings,alotaboutpayingyourbills,likeputtingmoneyasideand

workingoutabudget,avoidingloans(especiallythoseonTV),budgetingmainly.Iwantedto

learnhowtosavebecauseI’veneverbeengoodatdoingthat.”

“Howtosavemoney,howtobudget”

“Justaboutpayingyourselffirst,makingsureyouhavemoneyforyourbills.BecauseI’mon

thepension,Ihavebillstakenoutfirst.WhenIgoshopping,IonlytakewhatIneed(cash)

andusemycardaslittleaspossible.It’sgoingwell,I’mevensavingabit”

“Justthatyou'reincontrolofyourownmoney.Payyourselffirst”

“Irememberittaughtushowtolookatourfinancesandhowtore-evaluateitandhowto

setgoals”

“Justabouthowyousavemoneyandwhatnotandmakeaplan,stufflikethat”

“rememberbudgetingandpayingmyselffirstandstoppingimpulsebuying,puttingloosechangeinmymoneybox.EverydayIputaside$3atleast,soIcanpaythebills.It’sworkingformereallywell,especiallybecauseI'monalimitedbudget.I’m

notgoingintodebt,I’mnotborrowinganymore.””

Page 75: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

75

ResourcesAvailable

Sevenpercentofstudyparticipantsrecalledinformationaboutwhereandhowtofindfinancialhelpandresources:

“Writingdownyourincomeandexpensesandseeingwhetheryouspendmorethanyou

earn.Irememberthewebsiteswhereyoucangotoseeifyouborrowmoneyhowlongthe

loanwilllastandhowmuchyou’llhavetopay”

“Avenuesforfinancialhelpandsupportcreditratings-howtheywork,howtoviewand

updateloans-shorttermloanpitfallsetc.”

RememberedVeryLittlefromtheProgram

Only11%ofparticipantsreportedrememberingnothingorverylittlefromtheprogram:

“Notmuchatall”

“Notmuchreally,justthateverylittlebitcounts”

“Ididitinrehab,I’mnotthereanymore.Ican'trememberalot”

“Notmuch”

“Notmuchtobehonest,somethingaboutbudgeting”

“Ican'trememberwhatitwasabout”

Inshort,mostparticipantsrememberedthattheInChargeofMyMoneyprogramwasaboutbudgetingandmoneymanagement

generally,andmanyparticipantsalsorememberedspecificknowledgeandspendingorsavingtips.Onlysixparticipantssaidtheyremembered

littleornothingfromtheprogram.

Consideringthatmostofthesestudyparticipantswereinresidentialdrug,alcoholorgamblingaddictionrehabilitationcentresorhomelessatthetimeofdoingtheprogram,thislevelofknowledgeretentionis

impressive.

Page 76: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

76

Improvement in Financial Behaviours

Themainaimofthisevaluationprojectwastoidentifytheoutcomesoftheprogramon

participants’financialknowledge,attitudesandbehaviours,andwhethertherewas

sustainedchangeovertime.Consistentwiththeconceptualmodelforfinancialcapability

(Kempson,Collard,&Moore,2006),changeinactualmoney-relatedbehaviourwas

consideredthemostimportantindicatorofprogrameffectivenessandwastheprimary

focusofthisevaluation.Toassessimprovementinfinancialbehaviours,thefollowing

specificfinancialbehavioursweremeasured:

MoneyBehaviour

Participantswereaskediftheystronglydisagreed,disagreed,neitheragreednordisagreed,

agreedorstronglyagreedwithfourstatementsrelatingtomoneybehaviours.Theywere

askedthesequestionsinrelationtothreepointsintimes;beforetheydidtheprogram,

immediatelyaftertheprogram,and5-7monthsaftercompletingtheprogram.Asking

participantsattheendoftheprogramtoanswersomequestionsastheybelievetheywould

havebeforetheydidtheprogram,isreferredtoas‘retrospective’questioningandis

consideredsuitableforlow-incomepopulationswhereresearchparticipationisdifficultto

obtain(Lyonsetal.,2006).Thestatementsmeasuringmoneybehaviourwere:

• BeforeIbuysomethingIcarefullythinkaboutwhetherIcanaffordit

• WhenIshopIspendtimecomparingpricestogetthebestdeal

• IbuythingsevenwhenIcan’treallyaffordthem

• Ibuythingsoncreditratherthanwaitingandsavingup

MonitoringExpenses

Tomeasurethedegreetowhichparticipantsmonitoredtheirexpenses,bothbeforethe

programandthen5-7monthsaftertheprogram,theywereaskedthisquestion:

• Whichoneofthefollowingbestdescribeshowyoumonitoryourexpenses?

o Idon’tkeepaneyeonexpensesatall

Page 77: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

77

o Ikeepmyeyeonexpensesabit

o WithoutkeepingwrittenrecordsIkeepafairlycloseeyeonexpenses

o Iusewrittenrecordstokeepaneyeonexpenses

SavingBehaviour

Tomeasureparticipants’moneysavingbehaviour,bothbeforetheprogramandthen5-7

monthsaftertheprogram,theywereaskedthisquestion:

• Whichofthefollowingbestdescribesyoursavingbehaviour?

o Isaveasetamountonaregularbasis

o Isavewhatisleftoveronaregularbasis

o IsaveoddamountswhenIcan

o Iamunabletosave

PlanningBehaviour

Tomeasurethewayparticipants’plantheirspending,bothbeforetheprogramandthen5-

7monthsaftertheprogram,theywereaskedthisquestion:

• Howdoyouplanyourspending?

o Overthenextweekorfortnight

o Overthenextfewmonthsto1year

o Overthenext2-4years

o Overthenext5yearsormore

o Idon’tplan

Page 78: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

78

Was there evidence of improved financial behaviours for program

participants immediately after and 5 months after the program?

Asthetablebelowshows,therewassignificant,positivebehaviourchange5-7monthsaftertheprogramacrossallfourbehaviouraldomains.Theeffectsizesofthesechangesweresubstantialandrobust.

Table5:Evidenceofimprovedfinancialbehaviours5monthspostprogram

Measure Wastheresignificantimprovementovertime? Fstatistic EffectSize

(partialeta2)

Moneybehaviour

Yes,therewasasignificantimprovementbetweenbehaviourbeforetheprogramandattheendoftheprogram,andbeforetheprogramandafterseveralmonths.Therewasnosignificant

differencebetweenimmediatelyaftertheprogramandseveralmonthslater,indicatingthattheimprovementsweresustainedovertime(see

Figure11

F(2,94)=15.71

p<.0005

n=48

.250

(verylarge)

Monitoringexpenses

Yes,therewasasignificantimprovementbetweenmonitoringexpensesbehaviourbefore

theprogramandafterseveralmonths.

F(1,53)=24.35

p<.0005

n=54

.315

(verylarge)

Savingbehaviour

Yes,therewasasignificantimprovementbetweensavingbehaviourbeforetheprogramand

afterseveralmonths.

F(1,53)=6.09

p=.017

n=54

.103

(medium)

Planningbehaviour

Yes,therewasasignificantimprovementbetweenplanningbehaviourbeforetheprogram

andafterseveralmonths.

F(1,53)=6.74

p=.012

n=54

.113

(medium)

Page 79: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

79

Figure11:Changestomoneybehavioursacrossthethreemeasuredtimepoints

What do the statistics show?

TheF-Statisticisameasureofhowdifferenttheaveragescoreforameasuredfactor(for

example,financialbehaviour)isatthedifferenttimesitwasmeasured.AlargerF-statistic

numbermeansthedifferenceisbigger.Thep-valueindicateshowlikelyitisthatthis

differenceisduetochance,withalowerp-valueindicatingalowerlikelihood.Theeffect

sizenumberreflectshow‘strong’androbusttheeffectis(e.g.,thedifferenceinaverage

scoresatthe3times).Aneffectsizebetween.01and.05isconsideredasomewhatweak

effect;between.06and.13amedium(moderatelystrong)effectandover.14astrongand

robusteffect.Thestrongeraneffectisthemorelikelyitwouldbefoundagainifthesame

studywererunagain(forexample,withadifferentgroupofprogramparticipants).

Thus,thefindingsforfinancialbehaviours(seeTable5andFigure11)showthat:

• Therewasaclearimprovementinfinancialbehavioursfrombeforetheprogramto

immediatelyafterit;

• Fivetosevenmonthslater,thosegainsinpositivebehaviourshadnotdeclined;

• Thelikelihoodthesegainswereduetochanceisverylow(lessthan1in1000);

Page 80: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

80

• Thesizeofthiseffectwasverylargeandrobust;

• Togetherthesestatisticssuggestthattherewasasignificantimprovementin

financialbehavioursduringtheprogram,thatthisimprovementwasretainedover

time,thattheimprovementwassubstantialandrobust,andthattheresultscanbe

trusted;

• MonitoringexpensesandMoneybehaviourswerethemostimprovedbehaviours.

Identifying and Achieving Financial Goals

Beingabletoidentifyafinancialgoalandsuccessfullyworktowardsitisconsideredakey

indicatoroffinancialcapability.Therefore,whetherornotparticipantshadidentifiedagoal,

howmuchtheInChargeofMyMoneyprogramhelpedthemachievetheirgoals,andthe

extentofactualprogresstowardsthespecificgoaltheyhadchosenduringtheprogram,

wereinvestigated.

Setting goals

Ofthe38participants(70%)whodidnothaveafinancialgoalbeforedoingtheprogram,30

ofthese(79%)hadagoalbytheendoftheprogram.AsshowninTable10,onlyeightstudy

participants(15%)didnothaveafinancialgoalbytheendoftheprogram.

Figure12:Participantgoalsettingbeforeandaftertheprogram

0% 10% 20% 30% 40% 50% 60%

Nogoalbeforeprogram,nogoalazerprogram

Nogoalbeforeprogram,goalazerprogram

Hadgoalbeforeprogram,hasgoalazerprogram

Percentageofparvcipants

Page 81: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

81

What were participant’s goals?

DuringtheInChargeofmyMoneyprogram,participantsareaskedtoidentifyafinancial

goaltoworktowards.Thegoalstendedtofallintothreecategories:

• Tosavemoneyortosaveforsomethinginparticular,forexample:

o “tosave”

o “TosaveandhaveaccesstomyownmoneyifandwhenIneedit”.

o “trytosavealittle”

o “Savecloseto30%ofmoneyIearn.15%emergency15%luxuries”

o “Saveforahouse”

o “saveforanoverseasholidayin2016”

o “Savemoneyformynewhome,likefurniture,bondmoney,etc.”

o “Savemoneytogoawayforaweekend”

o “Buynewelectricshaver”

o “GobacktoThailand”

o “Newcar”

50%ofparticipantsidentifiedsavingmoneyorsavingforsomethingspecific,astheir

goal.

• Toreducedebt,forexample:

o “Addressdebts”

o “Bedebtfree”

o “Contactfinancialcounsellortorepayandmanagedebts”

o “Payloans”

o “payoffmortgage”

o “Reduceallmydebt”

Tenparticipants(19%)saidthatpayingofftheirdebtswastheirgoal.

• Tobeabletobudgetandgenerallymanagetheirmoneybetter,forexample:

o “Budgetandsave”

o “Budgetbetter”

o “Getmorefinanciallystable”

Page 82: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

82

o “Keepabettereyeonthings”

o “Tomanagemymoneybetterandstoplending”

Sevenparticipants(13%)reportedthatbudgetingandimprovedmoney

managementwastheirgoal.

Did the program help participants achieve their financial goals?

Toanswerthatquestion,participantswereaskedseveralmonthsaftertheprogram:

• Howmuchdidthisprogramhelpyouachieveyourfinancialgoals?

Only6participants(11%)saidtheprogramdidnothelpthemachievetheirfinancialgoalsat

all,andmorethanhalf(56%)saidtheprogramhelpedthem‘alot’or‘verymuch’.

Figure13:Howmuchdidthisprogramhelpyouachieveyourfinancialgoals?

What helped participants achieve their goals the most?

Whenaskedwhichpartoftheprogramhelpedthemtoachievetheirfinancialgoalsthe

most,participants’responsesfellintofivemaincategories:

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Notatall

Alixle

Alot

Verymuch

Percentageofparvcipants

Page 83: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

83

• Becomingawareoftheirspendinghabitsandtheneedtoplanandprioritizetheir

spending

• Budgetingingeneral

• Informationaboutloans,debtandcredit

• Goalsettingandhowtosave

• Theprogramdidnothelpverymuchortheycouldnotremember.

Figure14:Whatpartoftheprogramhelpedyoutoachieveyourgoalthemost?

N.B.Thepercentagesdonotaddupto100%becausesomeparticipantsrespondedwithmorethanone

partoftheprogram.

Spendingbehaviour

Thirty-fivepercentofparticipantsreportedthatlearningpositivespendinghabitsandbecomingmoreawareoftheirownspendinghabits,wastheaspectoftheprogramthattheyfoundthemosthelpful.Forexample:

“Thinkbeforeyouspend–IonlytakethemoneyIneedtotheshopsnow”

“Understandingmyspendingpatterns”

“NotspendingonthingsIdon'tneed”

“Notusingyourmoneywastefully,havingpriorities”

“Prioritizingandthinkingahead(e.g.prepayingbillsinsteadofspendingthemoney)”

0% 5% 10% 15% 20% 25% 30% 35% 40%

Spendingbehaviour

Budgevng

Loan&debtinformavon

Goals&howtosave

Nothingmuch

Percentageofparvcipants

Partso

fthe

program

Page 84: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

84

“ThehandoutsreflectingbackandhowIcanchangemyhabits”

“Idon'tspendasmuchoncoffeeandcigarettesnow”

“IthinkjustwritingeverythingdownthatIspend,keepingreceipts-Ispendalotofmoneyonfood”

Budgeting

Twenty-eightpercentofstudyparticipantssaidtheinformationonbudgetinghelpedthemachievetheirgoalthemost:

“Definitelythebudgetingone,nowIwriteashoppinglistandmakesureIworkitallout”

“Creatingandworkingtoabudget”

“Budgetingbasically”

“Justbudgeting,weeklyandfortnightly”

“Thebudgetingpartandhowtosavemoney”

Loananddebtinformation

Thesamenumberofparticipants(i.e.28%)reportedthatitwastheinformationabouthowtoreducedebtandunderstandinghowdifferentloansworkedthatmosthelpedthemachievetheirfinancialgoal:

“Learningaboutloansandhowbadtheyare”

“Wheretogetinterestfreeloans(Ihadtoborrow$500)-Ididn'tknowaboutthatbefore”

“Gettingtoknowthetypesofloanswehave,thegoodandbadloans”

“Notgettingintotroubleloaningmoneywithhighinterest”

“LookingatwhatIspendmymoneyone.g.takeawaycoffee,

watchingwhatIspendmymoneyone.g.doIneedthismagazine?”

Page 85: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

85

“Debt,budgeting,paydayloaninformationinparticular”

“Toknowthatyoucanringandaskforextensionsandaskforapayplan”

“Waystomanagemydebt”

Howtosave

Someparticipants(17%)recountedthatitwassettingsomegoalsandlearningabouthowtosavethatmostassistedtheminachievingtheirgoal.Forexample:

“Savingacertainamountofmoneyeachweek”

“Savings”

“KnowingthatIcanhaveasavingsaccountthatIcan'ttouch”

“I’vestartedputtingmoneyintoasavingsaccount.BeforeIwithdrewthemoneybeforeI

sawit.Ididn'twanttodothecoursebecauseIthoughtIknewitall.ButIstartedputting

moneyasidethatI’veneededandI’vealreadyusedit(foranunexpectedexpenses)”

“Ithelpedalotbecauseitputsyouinabettermindset,notwastingyourmoney,heading

towardsyourgoalsbetter.Idon'twanttogointodebt,soI’lljustwaitandsaveandpayfor

thingsoutrighte.g.carandholiday”

“Putitinperspective,payingoffasmallamountisbetterthanbecomingoverwhelmedandnotdoinganything”

Page 86: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

86

Nopartoftheprogramhelpedmuch

Only13%ofstudyparticipantsstatedthattheprogramdidnothelpthemachievetheirgoal:

“Nothing”

“Notatall”

“Can'trecallthatmuchaboutitspecifically”

“Idon'tremember”

To what extent did participants achieve the specific goal they

identified during the program?

Severalmonthsaftertheprogram,participantswerealsoaskedtorateonascalefrom1-10

howmuchprogresstheyhadmadetowardsthespecificfinancialgoalthattheyhad

identifiedduringtheprogram.

ItcanbeseeninFigure15thatactualprogresstowardsaspecificgoalwasmorechallenging

forparticipants,withanaveragescoreof5.7outof10.Someparticipantswhenaskedthis

questionexplainedthatprogresshadbeenhamperedbyunexpectedexpenses(e.g.car

repairs)orpersistentunemployment,orotherissuesoutsidetheircontrol.

“I’msavingeveryfortnightnow,andhavesaved$2000.I’mveryhappywiththat.AndIstillhavemybillspaid,I’mnotshort,asIhavedirectdebitcomingoutofthebank.Idon'twastemoneyonthingsIdon'tneedanymore.I’mactuallybecomingabitofa

miser”

Page 87: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

87

Insummary,theInChargeofMyMoneyprogramhelpedthemajorityofthoseparticipantswithoutfinancialgoalstoidentifyone,andoverhalfsaidtheprogramhelpedthemachievetheirfinancialgoals‘alot’or‘verymuch’.Adviceonhowtochangespendinghabitswasreportedtobethepartoftheprogramthatmostassistedparticipantsachievetheirgoals.However,theextentofactualprogresstowardstheirspecificgoalwasoftenhamperedbyunexpected

and/oruncontrollablecircumstances.

Figure15:Onascalefrom1-10,towhatextenthaveyoumadeprogresstowardsthefinancialgoalthatyouhadattheendoftheprogram?

0

1

2

3

4

5

6

7

8

9

10

0 1 2 3 4 5 6 7 8 9 10

Num

bero

fparvciaptns

Score

Page 88: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

88

Improvement in Financial Knowledge, Attitudes, Self-Efficacy

and Emotional Spending

Theneedforincreasedfinancialknowledgeandimprovedattitudestowardspersonal

financesinordertodeveloppositivefinancialbehaviourshasbeenrecognised(Robson,

2012).Factualknowledgebyitselfisnotenoughtoproducechangesinbehaviour.Thelink

betweenknowledgeandbehaviourismediatedbypersonalcharacteristicssuchasattitudes

andself-efficacy(ConsumerFinancialProtectionBureau,2015).

Inthisevaluationstudy,financialknowledgewasdividedintotwotypesofknowledge;

generalfinancialknowledgeandpersonalfinancialknowledge.

Bothtypesofknowledgeweremeasuredbyaskingparticipantstorespondtocertain

statementsona5-pointscale,where1=stronglydisagreeand5=stronglyagree.They

wereaskedthesequestionsinrelationtothreepointsintimes;beforetheydidthe

program,immediatelyaftertheprogram,and5-7monthsaftertheprogram:

Thegeneralfinancialknowledgestatementswere:

• IknowwheretogethelpifIhavefinancialproblems

• Ihaveagoodunderstandingofthedifferenttypesoffinancialproducts(suchas

loans,creditcards,paydayloansorsavingsproducts)

• WhenIwanttoborrowmoneyIknowthemostimportantquestionstoask(suchas

findoutthecostofborrowing)

Thepersonalfinancialknowledgestatementswere:

• IknowhowmuchIneedformydailylivingexpenses

• IknowhowmuchmoneyIowe

• Iknowwhenmybillsaredue

• Iknowhowtobudget

Page 89: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

89

Was there evidence of improved general and personal financial

knowledge for program participants immediately after and five -

seven months after the program?

Ascanbeseeninthetablebelow,therewasstatisticallysignificantimprovementinbothgeneralandpersonalfinancialknowledgefrombeforetheprogramtoimmediatelyaftertheprogramandtoseveralmonthsaftertheprogram.Theeffectsizesoftheseimprovementswereverylargetoextremelylarge.

Table6:Wastheresignificantimprovementovertime?

Measure

Wastheresignificantimprovementovertime?

Fstatistic

EffectSize

(partialeta2)

Generalfinancialknowledge

Yes,therewasasignificantimprovementbetweengeneralfinancialknowledgebeforetheprogramandattheendoftheprogram,andbeforetheprogramandafterseveralmonths.Therewasnosignificantdifferencebetweenimmediatelyaftertheprogramandseveralmonthslater,indicatingthattheimprovementsweresustainedovertime.Theeffectsizeofthisdifferencewasextremelylarge.

F(2,98)=104.27

p<.0005

n=50

.680

(extremelylarge)

Personalfinancialknowledge

Yes,therewasasignificantimprovementbetweenpersonalfinancialknowledgebeforetheprogramandattheendoftheprogram,andbeforetheprogramandafterseveralmonths.Therewasnosignificantdifferencebetweenimmediatelyaftertheprogramandseveralmonthslater,indicatingthatthechangesweresustained.Theeffectsizeofthisdifferencewasverylarge.

F(2,96)=24.09

p<.0005

n=49

.334

(verylarge)

AsTable6aboveandFigure16belowshow,participants,onaverage,hadaconsiderableincreaseinfinancialknowledge,bothgeneralfinancialknowledgeandknowledgeabouttheirownfinancialsituation,thatwasretained5-7monthsaftertheprogram.Theincreaseingeneralfinancialknowledgewasbothsubstantialandveryrobust,andmusthavebeenexperiencedbymostoftheparticipantstoproducesuchstrongstatisticalresults.Again,the

Page 90: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

90

tinyp-valuesshowthatthereisanextremelylowlikelihoodthatthesefindingswereduetochance.

Figure16:Levelsoftotal(combinedgeneralandpersonal)financialknowledgeacrossthethreemeasuredtimepoints

Was there evidence of improved financial attitudes and financial self-

efficacy, and less emotional spending, for program participants

immediately after and five months after the program?

AttitudestoMoney

Attitudestomoneyweremeasuredbyaskingparticipantsiftheystronglydisagreed,

disagreed,neitheragreednordisagreed,agreedorstronglyagreedwiththefollowingthree

statements.Theywereaskedthesequestionsinrelationtothreepointsintimes;before

theydidtheprogram,immediatelyaftertheprogram,and5-7monthsaftertheprogram:

• Ibelievemoneyistheretobespent

• IbelievethewayImanagemymoneytodayaffectsmyfuture

• Iammoreofasaverthanaspender

Page 91: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

91

Thetablebelowshowsthatnosignificantimprovementwasfoundinthefinancialattitudes

ofparticipants,onaverage,frombeforetheprogramtoseveralmonthsaftertheprogram.

Table7:Wastheresignificantimprovementinparticipants'financialattitudesovertime?

Measure

Wastheresignificantimprovementovertime?

Fstatistic

EffectSize

(partialeta2)

Financialattitudes

Althoughtherewasasignificantimprovementinattitudesbeforetheprogramcomparedtoattheendoftheprogram(seestatistics),therewasnosignificantimprovementfrombeforetheprogramtoseveralmonthslater.Furthermore,therewasasignificantdeclineinattitudebetweenimmediatelyaftertheprogramandseveralmonthslater,indicatingthattheimprovedattitudesfromtheendoftheprogramwerenotsustainedovertime.

F(2,98)=18.26

p<.0005

n=50

.271

(verylarge)

Althoughnosignificantimprovementwasfoundinfinancialattitudesofparticipantson

average,datapresentedintheforthcomingsection(“Participantfactorsmostrelatedto

improvementsinfinancialbehavioursandstrain”)showsthatchangeinattitudeswas

significantlycorrelatedwithchangeinfinancialbehaviours,suggestingthatwhenprogram

participantsdidhaveimprovementsinfinancialattitudestheyalsohadmorepositive

financialbehaviours.

FinancialSelf-Efficacy

Higherlevelsoffinancialself-efficacy,definedashavingconfidenceinone’sabilitytodeal

withvarioussituationsinthefinancialdomain,havebeenshowntobeassociatedwith

betterfinancialoutcomesinpastresearch(Lown,2011).TheFinancialSelf-Efficacy(FSE)

scalewasusedtomeasurethisfactorinthepresentstudy,andhasbeenfoundtohave

adequatereliabilityandvalidity(Lown,2011).Itiscomprisedofsixstatementsand

participantswereaskediftheystronglydisagree,disagree,neitheragreenordisagree,agree

orstronglyagree,withthefollowingstatements:

Page 92: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

92

• Itishardtosticktomybudgetwhenunexpectedexpensescomeup

• Itishardtomakeprogresstowardsmyfinancialgoals

• WhenIhaveunexpectedexpenses,Iusuallyhavetousecredit

• WhenIhavefinancialproblems,itishardtoworkoutasolution

• Ilackconfidenceinmyabilitytomanagemyfinances

• Iworryaboutrunningoutofmoneyinretirement

ItcanbeseeninTable8belowthatnosignificantimprovementinfinancialself-efficacy,on

average,wasfound.However,similartofinancialattitudes,whilenosignificant

improvementwasfoundforparticipantsonaverage,thisfactorwassignificantlycorrelated

withimprovedfinancialbehaviours(seesectiontitled“Participantfactorsmostrelatedto

improvementsinfinancialbehavioursandstrain”),suggestingthatwhenprogram

participantsdidhaveimprovementsinfinancialself-efficacytheyalsohadmorepositive

financialbehaviours.

Table8:Wastheresignificantimprovementinparticipants'financialself-efficacyovertime?

Measure

Wastheresignificantimprovementovertime?

Fstatistic

EffectSize

(partialeta2)

Financialself-efficacy

Onaverage,therewasnosignificantimprovementinfinancialself-efficacyfrombeforetheprogramtotheendoftheprogram,norbeforetheprogramtoseveralmonthslater,norfromtheendoftheprogramtoseveralmonthslater.However,theimprovementinfinancialself-efficacyfrombeforetheprogramtoseveralmonthsaftertheprogramdidapproachsignificance,withp=.066.Itispossiblethatwithalargersamplesizeasignificantimprovementacrosstimemayhavebeenfound.

F(2,86)=2.107

p=.128

n=44

.047

Emotionalspending

Somepeoplespendmoneyforpsychologicalreasons,suchastorelievestressor

unhappiness(e.g.,Desarbo&Edwards,1996).Therefore,wemeasuredtheextentof

Page 93: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

93

emotionalspendingbyaskingparticipantsiftheystronglydisagreed,disagreed,neither

agreednordisagreed,agreedorstronglyagreedwiththefollowingtwostatements:

• WhenI’mfeelingdownIspendmoneytofeelbetter

• WhenI’mfeelingstressedIspendmoneytofeelbetter

Thefollowingtableshowsthat,onaverage,therewasnostatisticallysignificant

improvementinemotionalspending.

Table9:Wastheresignificantimprovementinparticipants'emotionalspendingovertime?

Measure

Wastheresignificantimprovementovertime?

Fstatistic

EffectSize

(partialeta2)

Emotionalspending

Therewasnosignificantimprovementinemotionalspendingfrombeforetheprogramtotheendoftheprogram,norbeforetheprogramtoseveralmonthslater,norfromtheendoftheprogramtoseveralmonthslater.

F(2,96)=1.48

p=.233

n=49

.030

Althoughthestatisticsshowthat,onaverage,therewaslittledeclineinemotionalspending,

itisimportanttonotethatthisfindingislikelyanartefactofthedifferingresultsofthe

groupswhodidanddidnotreceiveatextmessagenudge.Participantswhoreceivedthe

textmessagenudgehadadecreaseinemotionalspendingduringthecourseoftheprogram

andthenafurtherdecrease5-7monthslater(seeFigure24,insection“Theeffectivenessof

textmessagenudges”).Incontrast,thosewhodidnotreceivethenudgehadadecreasein

emotionalspendingacrossthecourseoftheprogrambut5-7monthslaterhadreturnedto

emotionalspendinglevelssimilartothoseatthestartoftheprogram.Thusitseemsthat

thedifferingresultsforthesetwogroupsmayhave‘cancelledeachotherout’whenthe

datawerecombined.Nevertheless,itseemstheprogramdidnotfacilitatereductionsin

emotionalspendingwithoutbeingcomplementedwithafurtherintervention.

Page 94: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

94

Insummary,therewaslittlestatisticalevidenceofsustainedchangeinfinancialattitudes,financialself-efficacy,and

emotionalspending,aftertheprogram.

Participants’ Financial Strain

Aswellasexaminingparticipantschangeinfinancialbehaviours,knowledge,attitudes,self-

efficacyandemotionalspendingseveralmonthsaftertheprogram,thisstudyinvestigatedif

participantsexperiencedchangestolevelsoffinancialstrainbyaskingparticipantsseveral

questionsregardingtheircurrentfinancialcircumstances.

Thefirstquestionaskedparticipantsiftheyhadexperienced,overthepastfewmonths,a

numberofindicatorsoffinancialstrain.

Was there evidence of fewer problems due to a shortage of money

from before the program to 5-7 months later?

ItcanbeseenfromTable10thattherewasasubstantialdecreaseinthenumberof

problemsduetoashortageofmoneyfrombeforetheprogramtoseveralmonthsafterthe

program,with26%ofparticipantsnowreportingthatnoneofthesethingshappenedto

themduetoashortageofmoney,comparedto11%inthefewmonthsbeforethey

commencedtheprogram.Thereweredecreasesacrossallindicatorsoffinancialstrainfrom

beforetheprogramtofivetosevenmonthsaftertheparticipantscompletedtheprogram.

Forthiscohort,thisisanimportantoutcome.Althoughthereareanumberoffactorsthat

mayhavecontributedtothisimprovementinfinancialcircumstances(e.g.beingfreeof

addiction,nowhavingajob),itdoessuggestthattheprogrammayhavehadapositive

impactonparticipants’financialcircumstances.

Page 95: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

95

Figure17:Inthepast5months,didanyofthefollowinghappentoyoubecauseofashortageofmoney?

Table10:Wasthereasignificantreductioninthenumberofissuesduetoshortageofmoneyovertime?

Measure

Wastheresignificantimprovementovertime?

Fstatistic

EffectSize

(partialeta2)

Numberofissuesduetoshortageofmoney

Yes,participantsreportedsignificantlyfewerissuesduetoashortageofmoneyfrombeforetheprogramcomparedtoseveralmonthsaftertheprogram.Theeffectsizeofthisimprovementwasmoderate.

F(1,53)=4.55

p=.038

n=54

.079

(medium)

0% 10% 20% 30% 40% 50% 60%

Couldnotpayelectricity,gasortelephonebills

Couldnotpaytherentormortgageonvme

Pawnedorsoldsomething

Wentwithoutmeals

Askedforfinancialhelpfromfamilyorfriends

Askedforhelpfromwelfareorcharives

Wenttoashopfrontpaydaylender

Usedanonlinepaydaylender

No,noneofthesethingshappenedtome

Percentageofparvcipants

Beforeprogram 5-7monthsazerprogram

Page 96: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

96

Was there evidence of reduced debt 5-7 months after the program?

ReducingdebtisaprimaryfocusoftheInChargeofMyMoneyprogramandimportantin

reducingfinancialstress.

After5-7months,participantswereasked:

• Overthepastfewmonths,hasyourlevelofdebt:

o Increased

o Stayedthesame

o Decreased?

AscanbeseeninFigure18,ofthe70%ofparticipantswhocommencedtheprogramwith

debt,almosthalf(47%)hadreducedtheirdebtseveralmonthsaftertheprogram.

Figure18:Overthepastfewmonths,hasyourlevelofdebtincreased,decreasedorstayedthesame?

Did participants feel less stressed about their future since doing this

program?

AsreportedinWesleyMission’s2010thirdfinancialstressreport(WesleyMission,2010),

financialstressimpactsonpsychologicalstressaboutthefuture.Therefore,participants

wereaskediftheyfeltlessstressedaboutthefutureafterdoingtheInChargeofMyMoney

program.ItcanbeseenfromFigure19belowthatalmosttwothirds(63%)ofparticipants

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Increased

Stayedthesame

Decreased

Percentageofparvcipantswhocommencedprogramwithdebt

Page 97: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

97

saidtheyfeltlessstressedaboutthefuturesincedoingtheprogram,suggestingthatthe

programmayhavehadsomeeffectongeneralstressaboutthefuture.

Figure19:Doyoufeellessstressedaboutthefuturesincedoingthisprogram?

Was there greater financial inclusion for participants several months

after the program?

Financialinclusionreferstohavingaccesstoarangeoffinancialproductsandservices,and

isanindicatorofavailablefinancialresources(Muiretal.,2016).Inthisstudy,financial

inclusionwasmeasuredbyaskingparticipantsiftheyhadanyofalistoffinancialproducts,

thensummingthenumberofproductstocreateafinancialinclusionscore.

0%

10%

20%

30%

40%

50%

60%

70%

No Yes

Percen

tageofp

arvcipants

Answer

Page 98: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

98

Figure20:Doyouhaveanyofthefollowingtypesoffinanceproducts?

Table11:Wastheresignificantimprovementinfinancialinclusionovertime?

Measure

Wastheresignificantimprovementovertime?

Fstatistic

EffectSize

(partialeta2)

Financialinclusion

Yes,participantsreportedsignificantlymorefinancialproductsseveralmonthsaftertheprogramcomparedtoattheendoftheprogram.Theeffectsizeofthisincreasehoweverwassmall.

F(1,53)=12.23

p=.001

n=54

.019

(small)

AscanbeseeninFigure20,oneparticipant(2%)didnotknowwhatfinancialproductsthey

had.Only8participants(15%)hadacreditcard,andthisdidnotchangeovertime.Five

people(9%)hadamortgage,andthisdidnotchangeoverseveralmonths.Onlythree

participants(6%)hadapersonalorcarloan5-7monthsaftertheprogram,comparedtosix

people(11%)beforetheprogram.Forallotherfinancialproductstherewassignificantly

greaterinclusionseveralmonthsaftertheprogram.Asshownintheabovetable,therewas

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Transacvonaccount(e.g.everydayaccount)

Savingsaccount(notusedforeverydaycosts)

Mortgage

Creditcard

Superannuavon

Personalloanorcarloan

Insurance(e.g.car,home)

Visa/Mastercarddebitcard

Idon'tknow

Percentageofparvcipants

Financialprodu

ct

Beforetheprogram 5-7monthsazertheprogram

Page 99: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

99

significantimprovementinfinancialinclusionforstudyparticipants5-7monthsafterthe

program.

Did more participants have money set aside for emergencies?

Havingmoneyavailableincaseofemergenciesisanindicatorofparticipants’financial

resilience.Participantswereasked,“Doyouhavemoneyputasideforemergenciesand/or

savingstocoveryourexpensesifyougotsickorlostyourjob?”

Attheendoftheprogram,35%ofparticipantssaidtheyhadmoneysetasidefor

emergencies,but5-7monthsaftertheprogram,52%ofparticipantssaidtheyhadmoney

reservedforemergencies,showingsubstantialimprovement.Havingafinancialbufferisa

keycomponentoffinancialwell-beingandservestoreducestress.Individualsliving

financiallyprecariouslivesareatmostriskofpoorhealthandlong-termfinancialhardship.

Was there a change in participants’ financial resilience?

Participantswereaskedthefollowingquestioninordertogaugeanunderstandingoftheir

financialresilience(abilitytocopewithfinancialshock):“Ifyouhad1weektoraise$1,000

foranunexpectedexpense,howwouldyouraisethemoney?”

AsseeninFigure21,noparticipantssaidtheywoulduseacreditcard,ateithertimepoint.

Noonesaidtheywoulduseanonlinepaydaylenderseveralmonthsaftertheprogram,

whichwasaslightdecreasefromtwowhosaidtheywouldusethismethodattheendof

theprogram.Thenumberwhosaidtheywouldborrowfromashopfrontpaydaylender

decreasedmarginallyfromthreetotwoparticipants.Therewasanincreasein‘usemy

savings’and‘borrowmoneyelsewhere’fromtheendoftheprogramtoseveralmonths

aftertheprogram.Ofthefive(9%)whosaidtheywouldborrowmoneyelsewhereseveral

monthsaftertheprogram,threeparticipantssaidtheywoulddoovertimeatworkorobtain

themoneyfromworkinotherways,suggestingsomeparticipantshadfoundemployment

betweentheendoftheInChargeofMyMoneyprogramandthesurvey5-7monthslater.

Page 100: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

100

The“FinancialResilienceinAustralia2015”report(Muiretal.,2016),basedona

representativesampleof1,496Australians,foundthat17%ofAustralianswouldnotbeable

toraise$2,000withinaweekincaseofanemergency,comparedto37%ofparticipantsin

thepresentstudyatTime1.However,atTime2,only20%ofstudyparticipantswouldnot

beabletopayanunexpectedexpense,whichissimilartotheAustralianaverage,although

theamountinthisstudywasonly$1,000not$2,0001.Inthepresentstudy50-54%of

participantswouldusetheirsavingstopayforanunexpectedexpense,comparedto83%of

Australians(Muiretal.,2016).Furthermore,31%ofAustralianswoulduseabankloanor

creditcardtopayforanunexpectedexpense,comparedtonostudyparticipantswhowould

useacreditcard.Thesefindingsreflecttheverylimitedresourcesofsavingsandcredit

cardsavailabletothesestudyparticipants,comparedtotheaverageAustralian.

1Theloweramountof$1,000waschosenforthisstudygiventhedifficultfinancialcircumstancesofthecohort.If$2,000wasusedasintheABSmeasure,therewouldbelittletonochangeinparticipants’abilitiestoraisetheamount.

Page 101: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

101

Figure21:Ifyouhad1weektoraise$1000foranunexpectedexpense,howwouldyouraisethemoney?

Was there improvement in other indicators of financial strain?

Otherindicatorsoffinancialstrainandhardshipwerethefrequencythatparticipantsasked

familymembersformoney,thelevelofdifficultlyinpayingbillsandcoveringexpenses,and

howoftentheyhadmoneyleftoverbythenextpayday.

Askingfamilymembersformoney

Participantswereasked:

• Howoftendoyouaskotherfamilymembersformoney(notincludingyourpartnerorchildren)?

0% 5% 10% 15% 20% 25% 30% 35% 40%

AdvanceonCentrelinkbenefit

Usemysavings

Usemycreditcard

Borrowmoneyfromfamilyorfriends

Nointerestloanscheme

Shopfrontpaydayloan

Onlinepaydayloan

Iwouldn'tbeabletopayit

Borrowmoneyelsewhere

Percentageofparvcipants

Attheendoftheprogram 5-7monthsazertheprogram

Page 102: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

102

Participantscouldanswerwithoneofthefollowing:

• Never• Occasionally(aboutoncepermonth)• Regularly(aboutevery2weeks)• Frequently(everyweek)

Difficultypayingbills

Participantswereasked:

• Inatypicalmonth,howdifficultisitforyoutocoveryourexpensesandpayallyourbills?

Participantscouldanswerwithoneofthefollowing:

• Notatalldifficult• Alittledifficult• Moderatelydifficult• Verydifficult• Extremelydifficult

Moneyleftover

Participantswereasked:

• Inatypicalmonth,howdifficultisitforyoutocoveryourexpensesandpayallyourbills?

Participantscouldanswerwithoneofthefollowing:

• Always• Mostofthetime• Sometimes• Rarely• Never

Asseeninthetablebelow,althoughparticipants,onaverage,askedfamilyformoneyless

frequently,founditlessdifficulttopayalltheirbills,andhadmoremoneyleftover5-7

monthsaftertheprogramthanbeforetheprogram,theseimprovementsdidnotreach

statisticalsignificance.Theseresultsshowthatimprovingfinancialliteracycannotbea

stand-alonestrategyforimprovingthefinancialwell-beingofindividuals,especiallythose

whoareexperiencingpoverty.Sufficientincomeiscriticaltobeabletoputintopracticethe

behavioursnecessaryforfinancialwell-being.

Page 103: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

103

Table12:Wastheresignificantimprovementsseeminindicatorsoffinancialstrain?

Measure

Wastheresignificantimprovementovertime?

Fstatistic

EffectSize

(partialeta2)

Askingfamilyformoney

Participantsaskedfamilymembersformoneylessfrequentlyseveralmonthsaftertheprogramcomparedtoattheendoftheprogram,butthisdifferencewasnotstatisticallysignificant.Thisimprovementapproachedsignificanceandmayhavebeensignificantifthesamplesizewaslarger.

F(1,52)=3.07

p=.086

n=53

.056

Difficultypayingbills

Participantsfounditlessdifficulttopaytheirbillsandmeettheirexpensesseveralmonthsaftertheprogramcomparedtoattheendoftheprogram,butthisdifferencewasnotstatisticallysignificant.

F(1,53)=2.52

p=.118

n=54

.045

Moneyleftover

Participantshadmoremoneyleftoverbythenextpaydayseveralmonthsaftertheprogramcomparedtoattheendoftheprogram,butthisdifferencewasnotstatisticallysignificant.

F(1,53)=1.86

p=.179

n=54

.034

Insummary,theresultsforimprovementsinfinancialstrainandhardshipseveralmonthsaftertheprogramweremixed.Therewasevidencethatparticipantsexperiencedfewerproblemsduetoa

shortageofmoney,hadreduceddebt,feltlessstressedaboutthefuture,hadmorefinancialproducts,andweremorelikelytohavemoneysetasideforemergencies,5-7monthsaftertheprogramcomparedtobeforetheprogram.However,therewasnostatisticallysignificantimprovementinaskingfamilyformoney,difficultypayingbills,or

havingmoremoneyleftover.

Page 104: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

104

Participant Factors Most Related to Improvements in Financial

Behaviours and Strain

Inadditiontoexaminingwhetherornotthereweresignificantandsustainedchangesin

participants’financialbehavioursandfinancialstrainseveralmonthsaftertheInChargeof

MyMoneyprogram,wewereinterestedinwhichparticipantfactorsweremostclosely

associatedwiththesechanges.Thisisdonebymeasuringthecorrelationbetweentwo

measuredfactors–thedegreetowhichonefactorchangessystematicallywithchangesin

another2.

Identifyingsignificantassociationsbetweenparticipantfactorsandoutcomessuggests

whichfactorsweremostinfluentialonparticipants’subsequentchangesinbehaviour.Using

correlationalanalyses,thefactorsmostcloselyassociatedwiththefollowingoutcomesas

measuredatTime2(5-7monthsaftertheprogram)wereinvestigated:

• Improvedmoneybehaviours

(i.e.thinkingcarefullyaboutwhetherIcanaffordsomethingbeforebuyingit,

spendingtimecomparingpricestogetthebestdeal,notbuyingthingsifIcan’t

reallyaffordthem,andwaitingandsavingupratherthanbuyingthingson

credit)

• Improvedlevelofsavingbehaviour

• Improvedlevelofplanningbehaviour

• Improvedlevelofmonitoringofexpenses

• Howmuchtheprogramhelpedachievefinancialgoals

2Correlationscanrangefrom-1.0to+1.0,withnumbersclosertozeroshowingalowcorrelationandnumberscloserto+/-1ahigher

correlation.Forexample,thecorrelationofheightwithagemightbeveryhighinchildhood(say.70orgreater)becauseaschildrengrow

theygenerallyalsogettaller.Oncepeopleareadultshowever,heightwouldchangeverylittlewithage,andsothecorrelationwouldbe

verysmall(maybe<.03)andreflectonlytinydecreasesinheightwitholdage.Correlationsarethoughtofasweakiftheyarelessthan.30,

moderatebetween.30and.60,andstrongabove.60.Itisimportanttobeawarethatcorrelationalanalysesdonotshowcausation(i.e.if

twofactorsarecorrelated,itdoesnotmeanthatonecausestheother,asfactorAcouldcausefactorB,factorBcouldcausefactorA,or

therecouldbeanotherfactorthatcausestheassociationbetweenfactorsAandB).

Page 105: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

105

• Theextentofprogressmadetowardsthespecificfinancialgoalchosenduringthe

program

• Fewerproblemsduetoashortageofmoney

• Lessdifficultypayingbills

• Moremoneyleftoverbythenextpayday

• Lessfrequentlyaskingfamilyformoney

• Reduceddebt,forthosewhohaddebtatthecommencementoftheprogram.

Improvement in attitude

AsshowninAppendix1,ofalltheparticipantfactorsinvestigatedinthisstudy,

improvementinattitudestospending(i.e.degreeofagreementtothestatements,‘Ibelieve

moneyistheretobespent’,‘IbelievethewayImanagemymoneytodayaffectsmyfuture’,

and‘Iammoreofasaverthanaspender’)wasthemostreliablystatisticallysignificantly

factorrelatedtoimprovementsinfinancialbehaviours.

Specifically,improvedattitudesweresignificantlyandmoderatelyrelatedtoimprovements

inmoneybehaviour,savingbehaviour,monitoringexpenses,achievinggoals,andreducing

debt.Attitudestospendingwerealsomarginallyassociatedwithimprovedplanning

behaviour.Itisinterestingthatimprovedattitudeswereconsistentlyassociatedwitha

rangeofimprovedfinancialbehavioursconsideringthat,onaverage,theimprovementin

attitudesattheendoftheprogramwasnotsustainedovertime(seep.91).These

correlationssuggestthateventhoughtherewasnosustainedimprovementinattitudes

overseveralmonthsforstudyparticipantsonaverage,therewasarelationshipbetween

improvedattitudestospendingandimprovedfinancialbehavioursovertimeforthose

participantswhodidexperiencechange.

Improvement in financial self-efficacy

Improvementinfinancialself-efficacy,whichisconfidenceintheabilitytomanagefinances,

wassignificantlyandmoderatelyrelatedtoimprovementsinmoneybehaviourand

monitoringexpenses,howmuchtheprogramhelpedthemachievetheirgoals,progress

Page 106: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

106

towardsaspecificfinancialgoal,andfewerproblemsduetoashortageofmoney.Similarto

improvementinattitudes,findingthatimprovedfinancialself-efficacywasassociatedwitha

numberofimprovedfinancialbehaviourswassomewhatsurprisinggiventhat,onaverage,

nostatisticallysignificantimprovementinfinancialself-efficacywasfoundaftertheprogram

(seep.92).Thesecorrelationssuggestthateventhoughtherewasnosustained

improvementinfinancialself-efficacyoverseveralmonthsforstudyparticipantson

average,therewasastatisticallysignificantrelationshipbetweenimprovedfinancialself-

efficacyandimprovedfinancialbehavioursovertimeforthoseparticipantswhodid

experiencechange.

Program engagement

Howhighlyaparticipantratedtheprogramattheendofthelastsessionoftheprogram

wasconsistentlyandmoderatelyassociatedwithimprovedfinancialbehaviours,butitwas

notrelatedtoreducedfinancialstrain.Specifically,higherratingsoftheprogramwere

linkedtoimprovedmoneybehaviours,improvedplanningandmonitoringbehaviours,and

theperceptionthattheprogramhadhelpedthemachievetheirfinancialgoals,several

monthslater.Thelevelofmotivationduringtheprogramwassignificantlyandmoderately

relatedtoonlyoneoutcome;improvedmoneybehaviours.Thesefindingsdemonstratethat

theextentthatprogramattendeesbelievedtheprogramwasbeneficialattheendofthe

programwasassociatedwithhowmuchtheirfinancialbehavioursimprovedmonthslater,

andthatmotivationduringtheprogramappearedtohavelittleinfluenceonmost

subsequentbehaviours.

General & personal financial knowledge

Verylargestatisticallysignificantandsustainedimprovementsinbothgeneralandpersonal

financialknowledgefrombeforetheprogramtoseveralmonthslaterwerefound.The

subsequentquestion,however,waswhetherornottheserobustimprovementsin

knowledgewereassociatedwithimprovementsinfinancialbehaviours,sinceincreased

knowledgedoesnotreducefinancialstrainexceptthroughchangedbehaviours.Asshown

inAppendix1,improvedfinancialknowledge,combininggeneralandpersonalfinancial

Page 107: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

107

knowledge,weremoderatelysignificantlycorrelatedwithimprovedmonitoringofexpenses

andhowmuchtheprogramhelpedparticipantsachievetheirfinancialgoals.Financial

knowledgewasalsomarginallyrelatedtoimprovedmoneybehaviourandhavingmore

moneyleftoverbythenextpayday.Thesesignificantassociationsbetweenknowledgeand

financialoutcomessuggest(butdonotprove)thatincreasedfinancialknowledgemayplay

anecessaryroleinchangingfinancialbehaviours.

Emotional spending

Somewhatsurprisingly,giventhattherewasnodecreaseinemotionalspendingfiveto

sevenmonthsaftertheprogram,reducedemotionalspending(i.e.lessspendingmoneyto

feelbetterwhenfeelingdownorstressed)wasmoderatelysignificantlyassociatedwith

improvedmoneybehaviour,howmuchtheprogramhelpedtheparticipantachievetheir

financialgoals,andfewerproblemsduetoashortageofmoney.Thesefindingssuggestthat

addressingemotionalspendingmaybeimportantinfinancialliteracyprograms,butthatthe

benefitsmaybemostsignificantforthoseforwhomemotionalspendingisanissue.

Participant factors not associated with improvement in financial

behaviours or financial strain

Itisimportanttonotethoseparticipantfactorsthatwerenotsignificantlyassociatedwith

improvementsinfinancialbehavioursaftertheprogramasthesemayalsoprovideuseful

informationforfinancialliteracyprograms.Nodemographicfactors,includinggender,

countryofbirth,age,highestlevelofeducation,andincome,werestatisticallysignificantly

associatedwithanyimprovementsinfinancialbehavioursorreducedfinancialstrain.These

findingssuggeststhattheinfluenceoftheprogramonimprovedfinancialbehaviourswas

notrelatedtogender,age,nationality,educationlevelorincome,andthereforetheIn

ChargeofMyMoneyprogrammayberelevanttoabroadrangeofpeoplewithinvulnerable

populations.However,itshouldberememberedthattherewaslittlevariationincultural

andlinguisticbackgroundinthiscohort,withmostbeingborninAustraliaorotherEnglish

speakingcountries.Thestudyparticipantswerealsoquitehomogenousonindicatorsof

Page 108: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

108

socioeconomicstatus,withgovernmentbenefitsbeingthemainsourceofincomefor85%

ofparticipants.Therefore,thisevaluationcannotcommentontheeffectivenessofthe

programforculturallyandlinguisticallydiversepopulations,orforthosefromhigher

socioeconomicpopulations.

Inadditiontodemographicfactors,twoprogramengagementfactorswerenotstatistically

significantlyassociatedwithanyimprovementsinfinancialbehavioursorreducedfinancial

strain.Thenumberofworkshopsattended,alsoreferredtoastheprogram‘dosage’,was

notrelatedtoprogramoutcomes.However,themajorityofparticipants(78%)attendedall

orthreeofthefourworkshops,sotherewaslittlevariationinthisfactor.

Theotherprogramengagementfactorthatwasnotrelatedtoimprovementsinfinancial

knowledgeorbehaviours,ortoreducedfinancialstrain,waswhetherornotparticipants

attendedtheprogrambecauseitwasacompulsorypartofanotherprogramtheywere

involvedin,suchasadrugandalcoholrehabilitationprogram.Thisknowledgeisrelevant

becauseitsuggeststhatthosepeoplewhodonotvoluntarilychoosetodotheInChargeof

MyMoneyprogram,maybenefitfromittoasimilarextentasthosewhochoosetoattend

voluntarily.Thisisconsistentwiththefindingthatmotivationlevelduringtheprogramwas

onlyrelatedtooneoutcomevariable,asnotedpreviously.Takentogether,thesefindings

suggestthatincludingtheprogramasacomponentofotherprogramsandservicesmaybe

aneffectivewaytoimprovethefinancialliteracyofalargenumberofmarginalisedand

vulnerablepeople.

The most predictive participant factors

Inadditiontodiscussingthesignificantassociationsbetweenparticipantfactorsandarange

ofprogramoutcomes,itisalsousefultoidentifywhichparticipantfactorsremain

significantlyassociatedwiththeoutcomesaftertakingtheotherfactorsintoaccount.

Appendix2showsthatsomeparticipantfactorsaresignificantlyandmoderatelycorrelated.

Thetwoprogramfactors,motivationlevelandprogramrating,werestronglycorrelated(r=

.66,p<.0005),aswerethetwoknowledgefactors,generalandpersonalknowledge(r=.61,

p<.0005).Multipleregressionswereconductedasawayofworkingoutwhichfactorsmost

Page 109: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

109

stronglypredictedbetteroutcomeswhenallthefactorswereconsideredtogether.

Improvedgeneralfinancialknowledge,improvedpersonalfinancialknowledge,improved

attitudestospendingmoney,improvedfinancialself-efficacy,lessemotionalspending,

programratingatcompletion,andlevelofmotivationatthebeginningoftheprogram,

wereallinitiallyenteredintoastepwise,backwardmultipleregressionforeachofthekey

outcomemeasures.TheresultsofthesemultipleregressionsaregiveninAppendix1.

Consistentwiththecorrelationalresults,financialself-efficacyandattitudestowards

spendingwerethemostfrequentlyindependentlypredictiveparticipantfactorsinthefinal

models.Theotherfactorsthatwereimportantpredictorsofgoodoutcomesinsome

analyseswereprogramrating,personalfinancialknowledge,andemotionalspending.

Page 110: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

110

The Effectiveness of Text Message Nudges

Withtheubiquitoususeofmobilephonesandtextmessaginghascomeanincreased

interestinhowtextmessagescanbeusedtoprompt,or‘nudge’,peopletowardspositive

behaviours.Forexample,textmessaginghasbeenusedtoincreasetheuptakeofinfluenza

vaccinations(Chaietal.,2013),helppeoplestopsmoking,encouragemorephysicalactivity,

reminddiabetespatientstotaketheirmedications,assistinweightloss(Cole-Lewis&

Kershaw,2010a;Fjeldsoe,Marshall,&Miller,2009),encourageadherencetoadaily

medicationregimenforthoselivingwithHIV(Finitsisetal.,2014),remindschoolleaversof

therequiredtasksneededtoentercollege(Castleman&Page,2015),promotesexual

healthinyoungpeople(Gold,Lim,Hellardetal.,2010),improvecreditscoreratings(Bracha

&Meier,2014),andimproveloanrepayment(Karlanetal.,2012).

Sincetherehasbeenevidenceofsuccessinusingtextmessagenudgestoencourageawide

rangeofdesirablebehaviours,itwasdecidedtotrialtheuseoftextmessagereminders

aftertheInChargeofMyMoneyprograminthisstudytoinvestigateifthosewhoreceived

textnudgeshadbetterfinancialoutcomesthanthosewhodidnot,severalmonthsafterthe

program.

Methodology

Participants

The128initialparticipantsintheprojectwererandomlyassignedtoeitherreceivetext

messageremindersinthefollowingfivemonthsortoreceivenotexts.Ofthe54

participantswhocompletedthesecondquestionnaireatTime2andwereincludedinthe

dataanalyses(i.e.excludingstaffdata),24participants(44%)receivedatleastonetext

message.Threeofthe54participants(6%)optedout(i.e.askednottoreceiveanymore

textmessages)duringthestudy,includingonewhooptedoutbeforethefirsttextmessage

wassent.

Page 111: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

111

Procedure

Participantsweresentsevendifferenttextmessagesinthefivemonthsaftercompletionof

theInChargeofMyMoneyprogram.Onetextmessagewasaccidentallysenttwicetoone

groupofparticipants,hencethenumberoftextmessagesreceivedbyparticipantsranged

from0to8.Participantsweregiventheoptionofoptingoutatanytime.

AnonlineSMSsystemwaspurchasedforthestudysothatthetextmessagescouldbesent

outautomatically.Participants’namesandphonenumbers,andtextmessagewording,

wereenteredintotheSMSsystemandthetextmessageswerescheduledtobesentout

every3-4weeks.Theliteraturerecommendedsendingtextmessages3-4weeksafteran

intervention(Schwerdtfegeretal.,2012),anditwasthoughtthatsendingtextsevery3-4

weeksforfivemonthswouldbesufficientandnotconsideredintrusiveorirritating.

Text message delivery

Only65%ofthetextmessagesthatweresentoutwereactuallyreceivedbyparticipants.

Twentypercentofthemessages‘bounced’andanother15%were‘pending’andnever

delivered.Studyparticipantsreceived5.5textmessagesonaverage,and68%received6or

moretextmessagesoverthedurationofthestudy.Thelowmessagedeliveryratewas

partlyduetothefactthatapproximately50%ofparticipantswerelivinginresidential

addictionrehabilitationcentreswheretheuseofmobilephonesandtheinternetwas

strictlylimited.Furthermore,inmarginalisedpopulationssuchastheparticipantsinthis

study,mobilephonenumberstendtobeunreliable.

Text message wording

Muchdeliberationwasgiventothewordingofthetextmessagenudges.Accordingtothe

literature,effectivetextmessagesneedtobe:

• personalised,

• briefandfocused,

• easytounderstand,avoidingslangandabbreviations,

• conversationalintone,

Page 112: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

112

• positivenotnegative,

• haveanintrinsiclocusofcontrol(e.g."Youareresponsibleforachievingyour

goals"),

• directive(e.g."Planaheadsoyoucanreachyourgoals"),

• statementsnotquestions(e.g."Committingtoyourgoalstodaywillhelpyouinthe

long-run",ratherthan,"Howwillcommittingtoyourgoalstodayhelpyouinthelong

run?"),

• polite,

• nonaggressive,

• supportingself-efficacy,and

• grammaticallycorrect(Gold,Lim,Hellardetal.2010;Latimer-Cheungetal.,2013;

Marketo,2015;Muench,vanStolk-Cooke,Morgenstern,Kuerbis,&Markle,2014;

Ranneyetal.,2014;Scharl,Dickinger,&Murphy,2005;Schwerdtfegeretal.,2012).

Additionally,thesenderofthemessagesshouldidentifythemselvesasitgivescredibilityto

themessagebecauseitisfromatrustworthysource(JudyGoldetal.,2010;Marketo,2015).

Belowisthewordingoftheseventextmessagesthatweresentout:

1. Hi(FirstName)rememberyouareinchargeofyourmoneyandreachingyourgoals.

Allthebest,WesleyMission(Opt-out-info).

2. Hi(FirstName)whenyoukeeptrackofwhereyourmoneyisgoing,youarein

control.Allthebest,WesleyMission(Opt-out-info).

3. Hi(FirstName)identifyyourgoalandputitonthefridgetoremindyou.Allthebest,

WesleyMission(Opt-out-info)

4. Hi(FirstName)trackyourspendingleakageandbeinchargeofyourmoneytoday!

Allthebest,WesleyMission(Opt-out-info)

5. Hi(FirstName)reduceyourdebtandstressbypayingbackalittlemoreeachmonth.

Itwillmakeadifference.Allthebest,WesleyMission(Opt-out-info)

6. Hi(FirstName)payyourselffirstandsave10%ofyourincome.Doittodayand

watchyourmoneygrow!Allthebest,WesleyMission(Opt-out-info)

Page 113: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

113

7. Hi(FirstName)youaremorelikelytoachieveyourgoalifyoutellafriend-sotella

friendtoday!Allthebest,WesleyMission(Opt-out-info)

Messagenumbers1,2and4remindedparticipantsthattheywereincontroloftheirmoney

andencouragedthemtomonitortheirexpenses(“..keeptrackofwhereyourmoneyis

going..”,“..trackyourspendingleakage..”).Messages1,3and7remindedparticipantsof

theirgoalsandprovidedtipstohelpachievethosegoals(“…putitonthefridgetoremind

you…”,“…tellafriendtoday!..”).Message5wasaboutmanagingdebt,andmessage6was

aremindertosave.Therefore,themaintopicsoftheprogramwerecoveredbythetexts,

withanemphasisonbeingincontrolofyourmoneyandachievinggoals.

Messagenumber4wasnotreceivedbysixparticipantsbuttwoparticipantsreceivedit

twice.

Measures

Theimpactofthetextmessagenudgeswasmeasuredinthreeways:

• Thenumberoftextmessageseachparticipantreceived(asanumericvariable

rangingfrom0to8)

• Adichotomousvariableofwhetherornoteachparticipantreceivedfourormore

textmessages

• Participantratingofhowusefulthetextmessageswere,forthosewhoreceivedtext

messages.

ItcanbeseeninFigure22thatalmosthalfthe25participants(12)whoreceivedtext

messagesreportedthatthemessageswereonlyalittlebituseful.Onlyfiveparticipants

(20%)saidthatthetextswereveryorextremelyuseful.

Page 114: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

114

Figure22:Ifyouwerereceivingregularremindertextmessages,howusefuldidyoufindthetextmessage?

Examining the effects of text messages on change in behaviours

Althoughtheparticipantratingoftheusefulnessofthetextmessageswasnotvery

promising,itwaspossiblethattherewereimplicit,unconsciouseffectsofthenudgeson

participants’behaviour.Toinvestigatethispossibility,two-waymixedANOVAswere

conducted,withtimeasonerepeatedmeasuresfactorandreceivingatleast4text

messagesversusreceivingnomessages(controlgroup)asabetween-groupsmeasure.

UsingANOVAsallowedforanexaminationofinteractioneffectsbetweentimeandtext

nudges.Thatis,wetestedwhetherthepatternofchangesfrombeforetheprogramtoafter

theprogramandthento5-7monthsaftertheprogram,wasdifferentforthosewho

receivedthetextmessagenudgescomparedtothosewhodidn’t.

AfurthermannerofinvestigatingtheinfluenceoftextnudgeswastoperformPearson’s

correlationstoseeifthenumberoftextmessagesreceived,orparticipants’ratingofthe

usefulnessofthetestmessages,weresignificantlyassociatedwithanyimprovementin

financialbehavioursoranyreductioninfinancialstrain.

0% 10% 20% 30% 40% 50% 60%

Optedout

Notatalluseful

Alixleuseful

Veryuseful

Extremelyuseful

Percentageofparvcipants

Page 115: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

115

Text Messaging Results Table13:Theeffectsoftextmessagesonchangeinbehaviour

Measure n Result

Moneybehaviour

48Therewasasignificantcurvilinear(quadraticcurve)interaction,F(1,46)=

4.80,p=.034,wherebythosewithanudgeshowedanimprovementinmoneybehaviourimmediatelyaftertheprogramandafurtherincrease5monthslater,whereasthosewithoutthenudgepeakedaftertheprogramandhadadeclineingoodmoneybehaviours5monthslater(seeFigure23).Theeffectsizeofthisinteractionwasmoderate(Partialetasquared=.094).

Savingbehaviour

54Therewasanalmostsignificantinteractionbetweentimeandnudge,F(1,52)

=3.28,p=.076,wherebythosewithanudgehadagreaterincreaseinsavingbehaviour5monthsaftertheprogramcomparedtothosewhodidnotreceivethenudge.Theeffectsizeofthisinteractionwasmoderate(Partialetasquared=.059)

Planningbehaviour

54Therewasnosignificantinteractionbetweentimeandnudge,F(1,52)=

.005,p=.942,meaningthatthetextmessagenudgehadnoimpactonplanningbehaviour.

Monitoringexpenses

54Therewasasignificantinteractionbetweentimeandnudge,F(1,52)=4.33,

p=.042,wherebythosewithanudgehadamoremarkedincreaseinmonitoringbehaviour5monthslater.Theeffectsizeofthisinteractionwasmoderate(Partialetasquared=.077).

Emotionalspending

49Therewasasignificantinteractionbetweentimeandnudge,F(2,94)=

3.962,p=.022,wherebythosewithanudgehadadecreaseinemotionalspendingbehaviour5monthsaftertheprogrambutthosewithoutanudgehadanincrease(seeFigure24).Theeffectsizeofthisinteractionwasmoderate(Partialetasquared=.078).

Numberofproblemsduetoshortageofmoney

54Therewasnosignificantinteractionbetweentimeandnudge,F(1,52)=

.09,p=.768,showingthatthetextmessagenudgehadnoimpactonreducingthenumberofproblemsduetoashortageofmoney.

Difficultypayingbills

54Therewasnosignificantinteractionbetweentimeandnudge,F(1,52)=

.58,p=.449,indicatingthatthetextmessagenudgehadnoimpactonreducingthedifficultyinpayingbills.

Howoftendoyouhavemoneyleftover

54Therewasnosignificantinteractionbetweentimeandnudge,F(1,52)=

.05,p=.830,meaningthatthetextmessagenudgehadnoimpactonhavingmoneyleftoverbynextpayday.

Frequencyofaskingfamilyformoney

53Therewasnosignificantinteractionbetweentimeandnudge,F(1,51)=

.14,p=.709,indicatingthatthetextmessagenudgehadnoimpactonreducingthefrequencyinaskingfamilyformoney.

Page 116: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

116

Figure23:Changestomoneybehavioursovertimebytextmessage'nudge'group

Figure24:Changestoemotionalspendingovertimebytextmessage'nudge'group

Page 117: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

117

What the statistics show

ThefoursignificantinteractionsshowninTable13aboveindicatethatthepatternofmoney

behavioursandattitudesacrosstimewasdifferentforthetwogroups–thosewhoreceived

atextmessage‘nudge’andthosewhodidn’t.Thosewhoreceivedthetextmessageswent

ontofurtherimprovementsinfinancialbehavioursat5-7months,whereasthosewhodid

notreceivethetextmessagestendedtohaveadecreaseindesiredfinancialbehaviours

acrossthesameperiod.Forexample,ascanbeseeninFigure23,thosewhoreceivedatext

messagenudgehadanimprovementinmoneybehavioursimmediatelyaftertheprogram,

andafurtherimprovementafter5-7months.ThiscanbeseenasalinearincreaseinFigure

23.Forthosewhodidn’treceivethetextmessagenudge,moneybehavioursimproved

immediatelyaftertheprogram,butafterthis‘peak’declinedagainafter5-7months.This

canbeseenasamorecurved,ᴧ-shaped(curvilinear)patterninFigure23.

Thesamepatterncanbeseeninreverseforemotionalspending.Thosewhoreceivedatext

messagenudgehadareductioninemotionalspendingimmediatelyaftertheprogram,and

afurtherreductionafter5-7months(seeFigure24).Forthosewhodidn’treceivethetext

messagenudge,emotionalspendingreducedimmediatelyaftertheprogram,butthen

increasedagainafter5-7months,returningtothesamelevelsasbeforetheprogram.

ItcanbeseenfromTable13thattextmessagenudgesalsohadasignificantandpositive

impactonsavingbehaviourandthemonitoringofexpenses.Thesizeoftheseeffects,

wherebythegroupswereshowntohavedifferentoutcomesovertimedependingon

whetherornottheyreceivedatextmessagenudge,was‘moderate’.Thissuggeststhatthe

impactoftextmessagesontheseoutcomeswasfairlyrobustandthatthe‘nudge’

interventionhasareasonablelikelihoodofhavingthesamepositiveimpactonthesame

behavioursifitweredoneagaininanewgroupofprogramparticipants.

Page 118: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

118

Planningbehaviour,thenumberofproblemsduetoashortageofmoney,thedegreeof

difficultypayingbills,howoftenparticipantshadmoneyleftover,andhowoftentheyasked

familyformoney,werenotimpactedbyreceivingtextmessagenudges.

Itwasnotpossibletotesttheinteractioneffectofnudgesonhowmuchtheprogram

helpedparticipantsachievetheirfinancialgoals,theextentofprogresstowardsfinancial

goals,orchangeindebt,becausetheseoutcomeswereonlymeasuredatonepointintime

andatleasttwotimepointsarerequiredforANOVAstatisticalanalyses.Instead,

independentsamplest-testswereusedtocomparetheaveragescoresofthosewhohad

receivedatleast3textmessagesweredifferenttothosewhohadnot(thatis,toseeif

thosewhoreceivedthenudgeshadmadebetterprogressafter5-7months).Theresults

indicatedthattherewasnostatisticallysignificantdifferencebetweenthetwogroups(and

thusnocomparativeimprovementforthenudgegroup)intermsofchangeindebt(t(36)=

1.15,p=.256),extentofprogresstowardsaspecificfinancialgoal(t(52)=-1.57,p=.122),

orhowmuchtheprogramhelpedparticipantsachievetheirfinancialgoals(t(52)=.89,p=

.380).

Thefactthattherewasasignificantimpactofnudgesafterseveralmonthsonparticipants’

money,saving,andmonitoringbehaviours,andontheiremotionalspending,couldpossibly

beexplainedbythewordingofthetextmessages.Threeofthemessagesreminded

participantsthattheywereincontroloftheirmoney,withtwooftheseremindingthemto

monitortheirspending.Thiscouldexplainwhytherewasanimpactoftextnudgesonboth

monitoringbehaviourandmoneybehaviour,since‘moneybehaviour’wasmeasuredbythe

extentofagreementregardingthinkingcarefullybeforebuyingsomething,comparingprices

togetthebestdeal,notbuyingthingsifonecannotaffordthem,andwaitingtobuy

somethingratherthanusingcredit.Threeofthetextmessagesremindedparticipantsof

theirgoals,withanadditionalmessageremindingparticipantstosave10%oftheirincome.

Sincethegoalsof50%ofstudyparticipantsweretosavemoneyorsaveforsomethingin

particular,itisnotsurprisingthatanalmostsignificanteffectoftextmessagesonsaving

behaviourwasfound.Inconsistentwiththisexplanation,however,wasfindingnosignificant

Page 119: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

119

differencebetweennudgerecipientsandnon-recipientsinhowmuchtheprogramhelped

participantsachievetheirgoalsandtheextentoftheirprogresstowardstheirspecificgoal,

consideringthreemessageswerereminderstoworktowardstheirgoals.

Therewasasignificantinteractionbetweenreceivingtextmessagenudgesandareduction

inemotionalspendingovertime.Emotionalspendingwasmeasuredbyparticipants’levelof

agreementwithwhetherornottheyspendmoneytofeelbetterwhentheyarefeeling

downorstressed,thereforeitisconceivablethatbeingremindedthattheyareincontrolof

theirmoney,andthattheyneedtomonitortheirspendingandsavefortheirgoals,helped

reduceimpulsiveemotionalspending.However,furtherresearchisneededtotestthese

tentativeexplanations.

Ifthereisinsufficientincomethennoamountoftextingornudgescanhelpreducefinancial

strainorreducedebt.Again,thepointtoberememberedisthatfinancialeducationneeds

tobeaccompaniedbyanadequateincomeforfinancialwell-beingtobeachieved.

Insummary,textmessagenudgesappearedtohaveapositiveimpactonparticipants’money,saving,andmonitoringbehaviours,andonemotionalspending,buttheydidnothaveaneffectonparticipants’

financialstrain,orhelpthemachievetheirgoalsorreducetheirdebt.Atentativeexplanationforthe

findingsisthatthewordingofthemessagesremindedparticipantstocontrolandmonitortheirspending,

andremindedthemoftheirgoaltosave.

Page 120: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

120

Pearson’s correlations

Pearson’scorrelationswereconductedtoinvestigateassociationsbetweenthenumberof

textmessagesreceived,participants’ratingsofhowusefulthetextmessageswere,anda

rangeofoutcomes:

• Improvedmoneybehaviour

• Improvedsavingbehaviour

• Improvedplanningbehaviour

• Improvedmonitoringbehaviour

• Lessemotionalspending

• Howmuchtheprogramhelpedachievegoals

• Extentofprogresstowardsaspecificgoal

• Fewerproblemsduetoashortageofmoney

• Lessdifficultypayingbills

• Moremoneyleftover

• Lessoftenaskingfamilyformoney

• Reduceddebt

Theonlysignificantrelationshipfoundwasbetweenthenumberoftextsreceivedand

emotionalspending,wherebyreceivinganincreasednumberoftextmessageswas

moderatelycorrelatedwithreducedemotionalspending(r=.45,p=.027).Asmentioned

above,itishypothesisedthatbeingremindedthattheywereincontroloftheirmoney,

needtomonitortheirspendingandaimfortheirgoals,amelioratesspendingforemotional

reasonstosomeextent.Thefactthatnootherfinancialbehaviours,orprogresstowards

financialgoals,orreductionindebt,wereassociatedwiththenumberoftextmessages

receivedsuggeststhattherewaslittledifferenceineffectivenessbetweenreceiving4texts

andreceiving8textsover5months.Findingnoassociationsbetweenparticipants’ratingof

theusefulnessofthetextmessagesandanyoutcomesafter5-7monthssuggeststhatthe

effectofthenudgesonbehaviourwasimplicitandoutsideparticipants’awareness.

Page 121: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

121

Summary of Results and Implications

TheprincipalaimofthisprojectwastoevaluatetheeffectivenessoftheInChangeofMy

Moneyprograminvulnerablepopulations.Theevaluationaimedtoidentifyparticipant

outcomesrelatedtofinancialbehaviourandknowledgeretentioninparticipants.

Thefindingsofthisstudywillbesummarisedaccordingtothefourkeygoalsoftheproject,whichwereto:

- ensuretheprogramcontentisrelevantandevidence-based;

- identifywhethertheprogramfacilitatessustainedimprovementinfinancial

behaviours;

- estimatechangestoparticipantfinancialknowledgeandunderstandingattheendof

theprogram,andestimateretentionafterseveralmonths;and,

- ascertainhowincorporatingapracticalstrategy,suchastextmessagereminders,

increasesthechanceofsustainedchange.

Is the program content relevant and evidence-based?

Accordingtofeedbackfromstudyparticipantsbothimmediatelyaftertheycompletedthe

programandseveralmonthslater,theInChargeofMyMoneyprogramdidprovide

informationthatwasrelevantanduseful.Thecontentappearedtobeparticularlyrelevant

forsomeofthemostsociallyandeconomicallydisadvantagedandvulnerablepeoplewithin

theAustralianpopulation.Immediatelyaftertheprogram,participantsreportedbeingvery

satisfiedwiththecontent,relevanceandpresentationoftheInChargeofMyMoney

program,ratingitveryhighly,onaverage,andhavingfewcomplaints.Whenaskedseveral

monthslateriftheywouldrecommendtheprogramtotheirfriends,themajority(85%)said

theywouldrecommenditbecauseitwaseasytounderstand,relevant,andhelpful.Only

fourpeoplesaidtheywouldnotrecommendittotheirfriends,mainlybecausetheythought

theirfriendsdidnotneedit.Findingthatmostparticipantswouldrecommendtheprogram

totheirfriendsseveralmonthsaftertheyhadcompletedtheprogramsuggeststhat

participants’highopinionoftheusefulnessandrelevanceoftheprogramimmediatelyafter

theprogramdidnotdeclineovertime.

Page 122: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

122

Inadditiontooverallrelevanceandusefulness,thisstudyinvestigatedwhichaspectsofthe

programweremostrelevantandbeneficialtoparticipants.Whenaskedwhatpartofthe

programwasmosthelpfulinachievingtheirfinancialgoalsseveralmonthsaftercompleting

theprogram,participantsmostfrequentlyreportedthatadviceonhowtochangespending

habitswasmosthelpful.Learninghowtobudgetwasalsoabeneficialcomponentofthe

program,aswasunderstandingthedifferenttypesofloans,beingawareofinterestrates,

andstrategiesforreducingdebt.Addressingfinancialgoalsandhowtosavemoneywas

reportedtobethemosthelpfulpartoftheprogrambyonlyafewparticipants.

Furthermore,whenaskedatthebeginningoftheprogramwhattheymosthopedthe

programwouldassistthemwith,themostfrequentanswerswerehelpinmanagingmoney

andbudgeting.Helpwithsavingmoneyandsettingmoneygoalsweremuchlesscommon

responses.Apossibleexplanationforthesefindingsisthatparticipantsinthisstudyfoundit

difficulttoactuallypaytheirdailybills,soworkingtowardsagoalandsavingmoneywasa

lowerprioritythanlearningwaystopayfortheirdailylivingexpensesandreducetheirdebt,

ashasbeenfoundinotherstudies(Brackertz,2014;Loibletal.,2016).Apossible

implicationofthesefindingsisthattheInChargeofMyMoneyprogram,andotherfinancial

literacyprogramstargetingtheverymarginalisedinsociety,mayneedtofocuson

informationregardingbudgeting,moneymanagement,spendinghabitsandreducingdebt,

morethanongoalsettingandhowtosavemoney.However,forsome,thehabitofsaving

evenafewdollarswasanachievement.

Whenaskedhowtheprogramcouldbeimproved,onlysixparticipants(11%)saidthatit

couldbemorerelevant.Theonecommentgivenrelatingtorelevancewasthatmost

programattendeesdonotqualifyforacreditcard,thereforelesstimecouldbespent

discussingcreditcardusage.Thefinancialinclusionmeasureverifiedthatonlyeight

participants(15%)hadacreditcard.However,WesleyMission’s2010ThirdFinancialStress

Report(WesleyMission,2010)highlightedthatcreditcarddebtwastwotothreetimes

higheramonghouseholdsunderfinancialstresscomparedtothosenotunderfinancial

stress,suggestingthatthecreditcardmaterialmayonlybeoflimitedrelevancewhenthe

programisdeliveredtothemostextremelydisadvantagedinsociety,suchasthe

Page 123: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

123

participantsofthisstudy.Animplicationofthisfindingisthatthepresentationofthe

materialoncreditcardscouldbedependentonhowmanyparticipantsineachgroup

actuallyhaveacreditcard.Thispracticeisnowbeingimplemented.

Fromareviewoftheliterature,tentativebestpracticeforfinancialliteracyprograms

targetedatvulnerablepopulationsinvolvestailoringthecontentanddeliverystyletothe

particularaudience,havingtheprogramdeliveredbytrainerswhoareknowledgeableand

non-judgemental,providingtheprogramatanopportunetime,adheringtotheprinciplesof

adultlearning,andempoweringprogramattendees.TheInChargeofMyMoneyprogramis

consistentwithmostoftheseguidelinesforbestpractice.Thecontentoftheprogramwas

tailoredtotheneedsofWesleyMissionclients,whoareamongstthemostmarginalisedand

vulnerableinsociety.Addressingthetopicsofpaydayloans,reducingdebt,howtosavejust

adollarortwoaday,areexamplesofcontentthatisrelevanttothosewhosemainsource

ofincomeisgovernmentbenefits.Payingoffamortgage,increasingsuperannuation,and

investinginthestockmarketarenotincludedintheprogramsincemostattendeesdonot

owntheirownhome,arenotworking,anddonothavelargeamountsofmoneytoinvest.

Appropriatepracticalexamplesareusedduringthepresentation.Thereisanemphasison

teachinggoodmoneymanagementskillsandhabits,suchashowtocreateandmaintaina

budget,usingASIC’sMoneySmartbudgetingtool,howtouseamoneyboxtosavecoins,

andhowonlytakingtheamountofmoneyrequiredtobuywhatyouneedpreventsimpulse

spending.ThelanguageoftheprogrammaterialsissuitableforYear10levelofliteracy,not

higher,toensureitisunderstandablebythosewithrelativelylowlevelsofliteracy.

Furthermore,theprogramisdesignedtobedeliveredtopeoplewithreducedcognitive

capacity,suchasthosefromdrugandalcoholrehabilitationcentres.Informationisoften

presentedas“rulesofthumb”,suchas“payyourselffirst”,“everylittlebitcounts”,“ask

myself,doIreallyneedthis?”Thesesimpleandpracticalmessagesareeasyforprogram

attendeestoremember,andindeedtherewasevidencefromthequalitativedatathat

manyparticipantsdidrememberthem.

Page 124: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

124

InthisstudytheInChargeofMyMoneyprogramwasdeliveredbywell-trained,very

financiallyliterate,non-judgementalandtrustedpersonnel,withaprofoundunderstanding

oftheneedsoftheiraudience,namely,WesleyMissionfinancialliteracyeducatorsand

somefinancialcounsellors.Feedbackfromstudyparticipantsreinforcethatparticipants

foundthepresenterstobeknowledgeable,abletoanswertheirquestions,helpfuland

empathic.Itisnotknowniftheprogramwouldbeanywherenearaseffectiveifitwas

deliveredbypersonnelwithlessfinancialknowledge,lessunderstandingoftheaudience,

andlessexperienceindealingwiththefinancialissuesofthemostmarginalisedinsociety.

Inthisstudytheprogramwasdeliveredthroughexistingservicestovulnerablepopulations.

Theliteraturehasrecommendedembeddingfinancialliteracyprogramswithinother

programs,providedthestaffwhodeliverthefinancialliteracyprogramhavetheknowledge

andunderstandingtobeabletodoso.Theseservicesweremostlyaddictionrehabilitation

orhomelessnessassistanceservices.Theliteraturehighlightstheimportanceofdelivering

theprogramat‘teachablemoments’inaperson’slife,anditcouldbearguedthattryingto

overcomeaddiction,movingonandseekingemploymentandhousing,aresuchmoments.

Furthermore,providingfinancialliteracyeducationaspartofotherserviceshasthebenefit

ofassistingclientsmoreholistically,ashasbeenrecommendedintheliterature.

TheInChargeofMyMoneyprogramwasdeliveredbyfinancialliteracyeducatorsandsome

financialcounsellorswhohavebeentrainedtoguideclientsandgivethemchoiceintheir

decisions,ratherthantellingclientswhatthey“should”do.Thisapproachisconsistentwith

theprinciplesofadultlearning,whichpositsthatadultswantguidancenotinstruction.In

addition,thecontentoftheprogramisrelevanttoattendees’lives,andthedeliverystyleis

participatory,withattendeesbeingencouragedtoaskquestionsandtalkabouttheir

situations.Completingtheprogramworksheetspromotesactive,learningbydoing,asis

recommendedbytheprinciplesofadultlearning.

Theoneprincipleofadultlearningthatthisprogramhasnotfollowedisthatofvoluntary

participation.Fortherehabilitationresidentialparticipantsofthisstudy,attendingthe

programwascompulsory.However,vulnerablepopulationshavebeenshowninthe

Page 125: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

125

literaturetobelesslikelytoseekoutandattendfinancialliteracyprograms,andhave

higherdropoutrates,comparedtolessvulnerablepopulations.Therefore,makingthe

programacompulsorypartofanotherprogramislikelytoincreasethenumberofpeople

onlow-incomeswhoreceiveappropriatefinancialeducation.However,importantly,this

studyfoundthatparticipantswhoattendedtheInChargeofMyMoneyprogrambecauseit

wascompulsorydidnotbenefitfromtheprogramlessthanotherparticipants.Thissuggests

thattheprogram,and/orthepresenters,wereabletoeffectivelyengagetheattentionof

participantsdespitetheirreasonforattending.Additionally,becauseapproximatelyhalfof

thestudyparticipantsdidnotvoluntarilyattendtheprogram,itreducedtheprobabilityof

selectionbias.Selectionbiascanoccurwhenonlythosealreadymotivatedandpredisposed

toimprovetheirfinancialliteracyenrolinaprogram.Selectionbiascaninflatethe

effectivenessofaprogrambecauseonlythemostmotivatedattend(Collins&Holden,

2014).

Finally,itisnotestablishedwhethertheInChargeofMyMoneyprogramengendersasense

ofempowermentandfinancialself-efficacyinattendees.Thisstudyfoundthat,when

lookingataveragesallparticipants,therewasnostatisticallysignificantincreaseinfinancial

self-efficacy,althoughincreasedfinancialself-efficacywasassociatedwithimproved

financialbehaviours.Thissuggeststhatsomeparticipantsexperiencedanincreasein

financialself-efficacy,andthattheseparticipantsbenefitedfromthisincreaseintermsof

positivefinancialbehaviours.Furtherresearchisneededtounderstandwhysomeprogram

attendeesdonotexperienceincreasedfinancialself-efficacyandhowtoimprovefinancial

self-efficacyforthisgroup.

Insummary,thisevaluationhasfoundbothquantitativeandqualitativeevidencethattheIn

ChargeofMyMoneyprogramcontentanddeliveryisrelevantandevidence-based,

conformingwithwhathasbeenidentifiedasbestpracticetodate.

Page 126: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

126

Is there evidence that the program facilitates sustained

improvements in financial behaviours?

TherewasevidencethattheInChargeofMyMoneyprogramfacilitatedsustained

improvementsinfinancialbehaviours.Statisticallysignificantimprovementwasfoundin

financialbehavioursfrombeforetheprogramtoseveralmonthsaftertheprogram,

measuringfinancialbehavioursinfourdifferentways.Therewassustainedimprovementin

moneybehaviours,savingbehaviour,planningbehaviourandmonitoringofexpensesover

severalmonths.Theeffectsizesoftheseimprovementswerelarge.Furthermore,47%of

participantswhohaddebtwhentheycommencedtheprogramreportedareductionindebt

5-7monthslater,suggestingthattheprogrammayhaveassistedparticipantstobetter

managetheirdebt.Additionally,overhalf(56%)ofstudyparticipantssaidthattheprogram

helpedthemachievetheirfinancialgoals‘alot’or‘verymuch’,andonly11%saiditdidnot

helpatall,indicatingthatmostparticipantsbelievedthattheprogramassistedthemin

practicalwaysseveralmonthsaftertheprogram.Finally,theseimprovementsinfinancial

behaviourswerenotrelatedtogender,age,nationality,educationlevelorincome,implying

thattheInChargeofMyMoneyprogrammayberelevanttoabroadrangeofpeoplewithin

vulnerablepopulations.

Is there evidence of knowledge and understanding retention

after several months?

Therewasstrongevidenceofknowledgeandunderstandingretentionseveralmonthsafter

theprogram.Averylargeandhighlysignificantimprovementwasfoundinbothgeneraland

personalfinancialknowledgefrombeforetheprogramtoimmediatelyaftertheprogram

andtoseveralmonthsaftertheprogram,demonstratingthatstudyparticipantsretained

mostoftheknowledgetheyhadlearntintheInChargeofMyMoneyprogramoverseveral

months.

Page 127: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

127

Whenaskedtodescribewhattheyrememberedabouttheprogram5-7monthslater,

almosttwothirdsrememberedthattheprogramwasaboutbudgetingandmoney

managementgenerally.Alargenumberofparticipantsrememberedspecificcatchphrases,

suchas“payyourselffirst”andspecificsavingtips,suchasputtingadollarortwoina

moneyboxeachday.Justoveraquarterofparticipantsrecalledinformationaboutthe

differenttypesofloansandhowtomanagedebt,andapproximatelyonequarter

rememberedspecificadviceaboutbeingmindfulofspendinghabits,includingimpulse

buying,spending“leaks”,askingoneself“doIreallyneedthis?”,onlytakingthemoneythat

youneedtotheshops,andplanningwhatyouwillspendbeforegoingtotheshops.Onlysix

participantsrememberedlittleornothingfromtheprogram.Consideringthatover50%of

participantswerelivinginresidentialrehabilitationcentresatthetimeofcompletingthe

program,thishighlevelofknowledgeretentionfromtheprogramisquiteimpressive.

Indeed,theInChargeofMyMoneyprogramwasdesignedtobedeliveredwithflexibility,in

ordertotakeintoconsiderationthereducedattentionspanofclientswithmentalhealth,

disabilityandaddictionissues.

Thestudyalsoexploredtheconnectionbetweenimprovedknowledgeandchangein

financialbehavioursovertime.Itwasfoundthatgeneralandpersonalfinancialknowledge

togetherweremoderatelysignificantlycorrelatedwithimprovedmonitoringofexpenses

andhowmuchtheprogramhelpedparticipantsachievetheirfinancialgoals,andmarginally

relatedtoimprovedmoneybehaviourandhavingmoremoneyleftoverbythenextpayday.

Thesesignificantassociationsbetweenknowledgeandfinancialoutcomessuggest(butdo

notprove)thatincreasedknowledgemaytranslateintoimprovedfinancialbehavioursover

time,supportingthevalueoffinancialliteracyprograms.

Did text message nudges increase the chance of sustained

change in financial behaviours?

Textmessagenudgesdidappeartoincreasethechanceofsustainedchangeinfinancial

behavioursovertime.Inthisstudythetextmessagenudgeshadasignificantimpacton

Page 128: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

128

participants’money,saving,andmonitoringbehaviours,andonemotionalspending.These

effectsappearedtobeunconscious,sincethemajorityofthosereceivingtextmessages

reportedthatthetextmessageswereonly‘alittlebit’helpful,supportingthehypothesis

thatnudgesoperatebytriggeringpeople’sautomatic,ratherthancognitive,system

(Marteauetal.,2011).Thenudges,however,didnothaveaneffectonparticipants’

progresstowardstheirgoals,orlevelofdebt,orreductioninfinancialstrain.Atentative

explanationforthemixedfindingsisthatthewordingofthemessagesreminded

participantstocontrolandmonitortheirspending,andremindedthemoftheirgoaltosave.

Furthermore,thefactthatnootherfinancialbehaviours,orprogresstowardsfinancial

goals,orreductionindebt,wereassociatedwiththenumberoftextmessagesreceived

suggeststhattherewaslittledifferenceineffectivenessbetweenreceiving4textsand

receiving8textsover5months.

Althoughtheseresultsarepromising,itisnotpossibletodrawstrongconclusionsand

implicationsfromthesefindings.Only24participantsreceivedatleastonetextmessage

andonly65%ofthetextmessagessentoutwereactuallyreceivedbystudyparticipants,

reflectingthepracticaldifficultiesexperiencedincommunicatingwiththisvulnerable

populationovertime.Thesefindingsneedtobereplicatedwithalargersampleinorderto

confirmtheeffectivenessoftextmessagenudges.Adetailedcostbenefitanalysisisalso

recommended,toascertainwhetherthedegreeofpositiveinfluenceofthetextmessage

nudgesisjustifiedbythecostoftheresourcesneededtocreateanddeliverthem.

Additional Findings: Improvements in Attitudes, Self-Efficacy,

Emotional Spending, and financial Strain

Althoughtherewaslittlestatisticalevidenceofsustainedchangeinfinancialattitudes,

financialself-efficacy,andemotionalspending,forparticipantsonaverageafterthe

program,thereweresignificantrelationshipsbetweenimprovementsintheseparticipant

factorsandthekeyoutcomes5-7monthslater.Inparticular,improvementinfinancialself-

efficacyandimprovedattitudestospendingwerethetwoparticipantfactorsmost

Page 129: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

129

frequentlysignificantlyassociatedwithprogramoutcomes,evenaftercontrollingforother

closelyrelatedfactors.Thesefindingssuggestthatencouragingprogramattendeestoadopt

adifferentattitudetospendingmoney,andbuildingattendees’financialself-efficacyby

helpingthemfeelmoreconfidentintheirabilitytomanagetheirmoney,maybothbe

crucialcomponentsinthesuccessofthisandotherfinancialliteracyprograms.These

findingsareconsistentwiththestagesofchangeinfinancialliteracymodel(Robson,2012).

Inthismodel,someattitudinalchangeisrequiredinordertolearnnewknowledgeand

behaviour,andthenincreasedcompetenceandconfidencesupportsfurtherbehavioural

change(Robson,2012).

However,itisimportanttorememberthatevenifattitudesandfinancialself-efficacyare

significantlyassociatedwithimprovementsinfinancialbehaviours,itdoesnotshowthat

theycausedtheseimprovements.Itisequallyvalidtosaythatimprovedfinancial

behavioursmayhavecausedparticipantstofeelmoreself-efficaciousandtohavebetter

attitudestomoney.Thedirectionoftherelationshipbetweentheparticipantfactorsand

outcomesisunknown.Itisalsopossiblethatanotherunknownfactorcausedboth

participantfactorsandoutcomestoimprove.Hencecorrelationalresultssuggestpossible

causalassociationsbutcannotprovethem.Inpracticalterms,however,itmaymakelittle

differencewhichwaycausalitylies,anditseemslikelythatitisbidirectional.Forexample,

betterfinancialbehaviourscouldincreaseself-efficacy,andbetterfinancialself-efficacymay

supportimprovementsinfinancialbehaviours,creatinganupwardcycleofpositivefinancial

behaviours.

Inadditiontoimprovedfinancialattitudesandfinancialself-efficacy,higherratingsofthe

programwerealsolinkedtoimprovedfinancialbehavioursandtheperceptionthatthe

programhadhelpedthemachievetheirfinancialgoals,severalmonthslater.However,the

levelofmotivationduringtheprogramwassignificantlyandmoderatelyrelatedtoonlyone

outcome.Thesefindingssuggeststhathowinteresting,relevantandhelpfulprogram

attendeesfindtheprogramisfarmoreimportantthantheirmotivationlevelduringthe

program,intermsofsubsequentimprovementsinfinancialbehaviours.

Page 130: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

130

Reducedemotionalspendingwasfoundtobemoderatelysignificantlyassociatedwith

improvedmoneybehaviour,howmuchtheprogramhelpedtheparticipantachievetheir

financialgoals,andfewerproblemsduetoashortageofmoney.Thesefindingssuggestthat

addressingemotionalspendingmaybeimportantinfinancialliteracyprograms.

Theresultsforimprovementsinparticipants’financialstrainweremixed.Therewas

evidencethatparticipants,onaverage,experiencedfewerproblemsduetoashortageof

money,hadreduceddebt,feltlessstressedaboutthefuture,weremorefinanciallyincluded

andhadhigherlevelsoffinancialresilience5-7monthsaftertheprogramcomparedto

beforetheprogram.However,althoughparticipants,onaverage,askedfamilyformoney

lessfrequently,founditlessdifficulttopayalltheirbills,andhadmoremoneyleftover

beforethefollowingpayday5-7monthsaftertheprogramthanbeforetheprogram,these

improvementsdidnotreachstatisticalsignificance.Therewasalsoanecdotalevidencethat

someparticipantswhohadbeenunemployedduringtheprogramwereworkingseveral

monthslater,andsomeparticipantswhohadbeenaddictedtosubstancesorgamblinghad

completedrehabilitationprogramsandwerecapableofbettermoneymanagement.

Therefore,thereweretoomanyadditionalinfluencesoutsidetherealmoftheprogramand

externaltoparticipants’ownfinancialbehavioursthatimpactedonparticipants’financial

circumstancestobeabletodrawanyconclusionsabouttheabilityoftheInChargeofMy

Moneyprogramtoreducefinancialstrain.

Additional Findings: Participant Factors That Were Not

Associated with Outcomes

Itisworthnotingthoseparticipantfactorsthatwerenotsignificantlyassociatedwith

improvementsinfinancialbehaviours.Nodemographicfactorswerestatistically

significantlyassociatedwithanyimprovementsinfinancialbehavioursorreductionin

financialstrain.Thisfindingsuggeststhattheimpactoftheprogramonimprovedfinancial

behaviourswasnotrelatedtogender,age,nationality,educationlevelorincome,and

thereforetheInChargeofMyMoneyprogrammayberelevanttoabroadrangeofpeople

Page 131: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

131

withinvulnerablepopulations.However,therewaslittlevariationincountryofbirthor

socioeconomicstatusinthiscohort,whichcouldexplainthelackofassociations.Inaddition

todemographicfactors,thenumberofworkshopsattended(i.e.dosage)wasnotrelatedto

financialbehavioursorfinancialstrain,althoughthiscouldbeduetothelimitedvariationin

dosagesincethemajorityofparticipantsattendedallorthreeofthefourworkshops.

Surprisingly,whetherornotparticipantsattendedtheInChargeofMyMoneyprogram

becauseitwasacompulsorypartofanotherprogramwasnotassociatedwith

improvementsinfinancialbehaviours,improvementsinknowledgeorreducedfinancial

strain.ThisimpliesthatthosepeoplewhodonotvoluntarilychoosetodotheInChargeof

MyMoneyprogram,maybenefitfromittoasimilarextentasthosewhochoosetoattend

theprogramvoluntarily.Thisisconsistentwiththefindingthatmotivationlevelduringthe

programwasonlyrelatedtooneoutcomevariable.Takentogetherthesefindingssuggest

thatincludingtheprogramasacomponentofotherprogramsandservicesmaybean

effectivewaytoimprovethefinancialliteracyandoutcomesofalargenumberof

marginalisedandvulnerablepeople.

Page 132: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

132

Discussion

ThisprogramevaluationfoundevidencethattheInChargeofMyMoneyprogramis

relevantandevidence-based,doesfacilitatesustainedimprovementinfinancialbehaviours,

anddoesproducehighlevelsofknowledgeandunderstandingretentionafterseveral

months,forpeoplefacingthevulnerabilitiesofdrug,alcoholandgamblingaddiction,

homelessness,andfamilystrain.Thisstudyalsofoundthatincorporatingapractical

strategy,suchastextmessagenudges,mayincreasethechanceofsustainedchangein

financialbehavioursaftertheprogram.However,therearesomelimitationstothisstudy.

Limitations of the Study

Themainlimitationsofthestudyrelatetoitsnon-experimentaldesign,itsgeneralisability,

andtheinformationcollectedatTime2.Firstly,thisstudywasnotanexperimentasthere

wasnomatchedcontrolgroupforcomparison.Therefore,weareunabletoshowthatthe

programcausesimprovementsinfinancialbehavioursandreducedfinancialstrainbecause

itispossiblethatparticipants’financialbehavioursandfinancialcircumstanceswouldhave

improvedregardlessofwhetherornottheyattendedtheInChargeofMyMoneyprogram.

Onlystudieswithanexperimentaldesigncanclaimtodemonstratecausation.

Furthermore,thisprogramevaluationcannotbegeneralisedtoapplytothosefrom

culturallyandlinguisticallydiverse,orhighersocioeconomicpopulations,sincethe

participantsinthisstudywerequitehomogenousindemographicfactors.Therefore,this

evaluationcannotcommentontheeffectivenessoftheprogramforculturallyand

linguisticallydiversepopulations,orforthosefromhighersocioeconomicpopulations.An

additionallimitationofthisstudyisthatwedidnotcollectinformationaboutchangesin

participants’non-financialcircumstancesseveralmonthsaftertheycompletedtheprogram.

Thiswasalimitationbecausewewerenotabletoisolatetheeffectsoftheprogramfrom

theeffectsofchangedlifecircumstances,suchashavingrecoveredfromaddiction,now

havingstablehousing,ornowbeingemployed.Thehighrateofattritioninsamplesizefrom

Time1toTime2wasalsoalimitation,asitreducedthestatisticalpowerofthestudyto

detectsignificanteffects,particularlyfortheexperimentwiththetextmessagenudges.

Page 133: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

133

Highratesofstudyparticipationattritionarecommonwhenconductingresearchwithlow-

incomepopulations,however(Collins,2010;Lyonsetal.,2006).Itishopedthatfuture

researchwilladdressthelimitationsinthisprogramevaluation.

Future research

Wewouldliketomakesuggestionsforfutureresearchforfinancialliteracyprogram

evaluationstargetingvulnerablepopulations.Firstly,futureresearchshouldincludea

matchedcontrolgroupwhoaresimilarineverywaytothestudyparticipantsexceptthat

theydonotattendthefinancialliteracyprogram.Theyshouldbegiventhesame

questionnairesatthesametimesasthosedoingtheprogramsothatchangesinfinancial

knowledge,behaviours,attitudes,andstraincanbecomparedbetweentheexperimental

andcontrolgroups.Thisistheonlywaytodemonstratethattheprogramcausesthe

changes.Secondly,thisprogramevaluationneedstobereplicatedwithparticipantswhoare

moredemographicallydiversetobeabletoshowthattheInChargeofMyMoneyprogram

iseffectiveforculturallyandlinguisticallydiverse,andlesssocioeconomically

disadvantaged,Australians.Thirdly,futurefinancialliteracyprogramevaluationsshould

collectinformationaboutparticipantschangedlifecircumstanceswhentheyaresurveyed

monthsaftertheprogram,sothatthesechangescanbetakenintoaccountintheanalyses.

Itispossiblethatrecoveryfromaddictionorfindingstablehousingarethemainreasonsfor

theimprovementsinfinancialbehaviours,ratherthantheprogram.Inaddition,some

participantsbenefittedfromtheprogrammorethanothers,sofutureresearchshouldtryto

identifythosewhobenefittheleastfromtheprogram,whythisisthecase,andhowthe

programcouldbetterengagethoseparticipants.Finally,thoseconductingresearchwiththe

marginalisedofsocietyshouldbeawareofthedifficultiesincontactingparticipantsfor

longitudinalstudiesandtheneedforadditionalresources.

Page 134: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

134

Conclusion

ThisprogramevaluationfoundevidencethatWesleyMission’sfinancialliteracyprogram,In

ChargeofMyMoney,isrelevantandbeneficialtoitstargetaudienceofthemostvulnerable

insociety.Forthosewithverylimitedincomesandresources,thesmalltipsonhowto

changespendinghabitsandsaveevenjustadollaraday,canmakeasignificantdifference

inpeople’slives.Forexample,oneparticipantsaidonthephonethathehadbeenplacing

hiscoinsinthemoneyboxhewasgivenattheInChargeofMyMoneyprogram.After

severalmonthsheopenedthemoneyboxandfoundthathehadover$100,andthatwas

enoughforhimtobeabletovisithisdaughter.Suchasmallchangeinfinancialbehaviour

hadmadeasignificantdifferenceinhislife.WesleyMissionandthecreatorsandpresenters

oftheprogramshouldbecommendedfortheirwork.

Page 135: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

135

References

Anderson,S.G.,Zhan,M.,&Scott,J.(2005).Developingfinancialmanagementtraininginlow-incomecommunities:Assessingneedsandcommunitypracticeimplications.JournalofCommunityPractice,13(4),31-49.

Atkinson,A.(2011).Measuringfinancialcapabilityusingashortsurveyinstrument:InstructionManual.PersonalFinanceResearchCentre,UniversityofBristol.

Atkinson,A.,McKay,S.,Collard,S.,&Kempson,E.(2007).LevelsofFinancialCapabilityintheUK.PublicMoney&Management,27(1),29-36.doi:10.1111/j.1467-9302.2007.00552.x

Atkinson,A.,&Messy,F.-A.(2011).Assessingfinancialliteracyin12countries:anOECD/INFEinternationalpilotexercise*.JournalofPensionEconomicsandFinance,10(4),657-665.doi:10.1017/S1474747211000539

AustralianBureauofStatistics.(2013).3238.0.55.001-EstimatesofAboriginalandTorresStraitIslanderAustralians,June2011fromhttp://www.abs.gov.au/ausstats/[email protected]/mf/3238.0.55.001

AustralianBureauofStatistics.(2015a).6227.0-EducationandWork,Australia,May2015fromhttp://www.abs.gov.au/AUSSTATS/[email protected]/mf/6227.0/

AustralianBureauofStatistics.(2015b).6523.0-HouseholdIncomeandWealth,Australia,2013-14Baird,P.,Cullinan,D.,Landers,P.,&Reardon,L.(2016).NudgesforChildSupport:Applying

BehavioralInsightstoIncreaseCollections.OPREReport,1.Bandura,A.(1982).Self-efficacymechanisminhumanagency.AmericanPsychologist,37(2),122-

147.doi:10.1037/0003-066X.37.2.122Billieux,J.,Maurage,P.,Lopez-Fernandez,O.,Kuss,D.J.,&Griffiths,M.D.(2015).CanDisordered

MobilePhoneUseBeConsideredaBehavioralAddiction?AnUpdateonCurrentEvidenceandaComprehensiveModelforFutureResearch.CurrentAddictionReports,2(2),156-162.doi:10.1007/s40429-015-0054-y

Birkenmaier,J.,&Maynard,B.(2016).InterventionsDesignedtoImproveFinancialCapabilitybyImprovingFinancialBehaviorandFinancialAccess:ASystematicReview.

Blue,L.,&Brimble,M.(2014).Reframingtheexpectationsoffinancialliteracyeducation:bringingbackthereality.JASSA(1),37.

Bracha,A.,&Meier,S.(2014).Nudgingcreditscoresinthefield:theeffectoftextremindersoncreditworthinessintheUnitedStates.

Brackertz,N.(2014).TheImpactofFinancialCounsellingonAlleviatingFinancialStressinLowIncomeHouseholds:ANationalAustralianEmpiricalStudy.SocialPolicyandSociety,13(03),389-407.doi:doi:10.1017/S1474746413000511

Brimble,M.,&Blue,L.(2013).Tailoredfinancialliteracyeducation:Anindigenousperspective.JournalofFinancialServicesMarketing,18(3),207-219.doi:10.1057/fsm.2013.16

Cadena,X.,&Schoar,A.(2011).Rememberingtopay?Remindersvs.financialincentivesforloanpayments:NBERWorkingPaperNo.17020.

Castleman,B.L.,&Page,L.C.(2015).Summernudging:Canpersonalizedtextmessagesandpeermentoroutreachincreasecollegegoingamonglow-incomehighschoolgraduates?JournalofEconomicBehavior&Organization,115,144-160.doi:http://dx.doi.org/10.1016/j.jebo.2014.12.008

Chai,S.J.,Tan,F.,Ji,Y.,Wei,X.,Li,R.,&Frost,M.(2013).Community-LevelTextMessagingfor2009H1N1PreventioninChina.AmericanJournalofPreventiveMedicine,45(2),190-196.doi:http://dx.doi.org/10.1016/j.amepre.2013.03.014

Page 136: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

136

Chong,A.,Karlan,D.,Shapiro,J.,&Zinman,J.(2015).(Ineffective)MessagestoEncourageRecycling:EvidencefromaRandomizedEvaluationinPeru.TheWorldBankEconomicReview,29(1),180-206.doi:10.1093/wber/lht022

Cole-Lewis,H.,&Kershaw,T.(2010a).TextMessagingasaToolforBehaviorChangeinDiseasePreventionandManagement.EpidemiologicReviews,32(1),56-69.doi:10.1093/epirev/mxq004

Cole-Lewis,H.,&Kershaw,T.(2010b).TextMessagingasaToolforBehaviorChangeinDiseasePreventionandManagement.EpidemiologicReviews.

Collins,J.M.(2010).ImprovingFinancialLiteracy:Theroleofnonprofitproviders.PensionResearchCouncilWP,39.

Collins,J.M.,&Holden,K.C.(2014).MeasuringtheImpactsofFinancialLiteracy:ChallengesforCommunity-BasedFinancialEducation.NewDirectionsforAdultandContinuingEducation,2014(141),79-88.doi:10.1002/ace.20087

Collins,J.M.,&O'Rourke,C.(2012).Stillholdingoutpromise:Areviewoffinancialliteracyeducationandfinancialcounselingstudies.NetworkFinancialInstitute:WorkingPaperSeries.

ConsumerFinancialProtectionBureau.(2015).Financialwell-being:ThegoaloffinancialeducationReport,IowaCity,IA:ConsumerFinancialProtectionBureau.

Datta,S.,&Mullainathan,S.(2014).BehavioralDesign:ANewApproachtoDevelopmentPolicy.ReviewofIncomeandWealth,60(1),7-35.doi:10.1111/roiw.12093

Desarbo,W.S.,&Edwards,E.(1996).Typologiesofcompulsivebuyingbehavior:Aconstrainedclusterwiseregressionapproach.JournalofConsumerPsychology,5,231-262.

Fernandes,D.,Lynch,J.G.,&Netemeyer,R.G.(2014).Financialliteracy,financialeducation,anddownstreamfinancialbehaviors.ManagementScience,60(8),1861-1883.

Finitsis,D.J.,Pellowski,J.A.,&Johnson,B.T.(2014).Textmessageinterventiondesignstopromoteadherencetoantiretroviraltherapy(ART):ameta-analysisofrandomizedcontrolledtrials.PloSone,9(2),e88166.

Fjeldsoe,B.S.,Marshall,A.L.,&Miller,Y.D.(2009).BehaviorChangeInterventionsDeliveredbyMobileTelephoneShort-MessageService.AmericanJournalofPreventiveMedicine,36(2),165-173.doi:http://dx.doi.org/10.1016/j.amepre.2008.09.040

Fox,J.J.,&Bartholomae,S.(2008).FinancialeducationandprogramevaluationHandbookofconsumerfinanceresearch(pp.47-68):Springer.

Fox,J.J.,Bartholomae,S.,&Trombitas,K.S.(2012).EvaluatingfinancialeducationprogramsStudentFinancialLiteracy(pp.141-166):Springer.

Frentzel,E.,Ganachari,D.,Bookhout,M.,Moon,M.,Galdo,J.,&Robinson,S.(2010).ConsumerEducationInitiativesinFinancialandHealthLiteracy.

Friedman,P.(2005).Providingandfundingfinancialliteracyprogramsforlow-incomeadultsandyouth.TheFinanceProject,EconomicSuccessforFamilies&Communities.

Fry,T.R.L.,Mihajilo,S.,Russell,R.,&Brooks,R.(2008).TheFactorsInfluencingSavinginaMatchedSavingsProgram:Goals,KnowledgeofPaymentInstruments,andOtherBehavior.JournalofFamilyandEconomicIssues,29(2),234-250.doi:10.1007/s10834-008-9106-y

Gold,J.,Lim,M.S.,Hellard,M.E.,Hocking,J.S.,&Keogh,L.(2010).What'sinamessage?DeliveringsexualhealthpromotiontoyoungpeopleinAustraliaviatextmessaging.BMCPublicHealth,10(1),1-11.doi:10.1186/1471-2458-10-792

Gold,J.,Lim,M.S.,Hocking,J.S.,Keogh,L.A.,Spelman,T.,&Hellard,M.E.(2010).DeterminingtheImpactofTextMessagingforSexualHealthPromotiontoYoungPeopleSexTransmDis.

Grinstein-Weiss,M.,Guo,S.,Reinertson,V.,&Russell,B.(2015).FinancialEducationandSavingsOutcomesforLow-IncomeIDAParticipants:DoesAgeMakeaDifference?JournalofConsumerAffairs,49(1),156-185.doi:10.1111/joca.12061

Hathaway,I.,&Khatiwada,S.(2008).Dofinancialeducationprogramswork?

Page 137: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

137

Hetling,A.,Postmus,J.L.,&Kaltz,C.(2015).ARandomizedControlledTrialofaFinancialLiteracyCurriculumforSurvivorsofIntimatePartnerViolence.JournalofFamilyandEconomicIssues,1-14.doi:10.1007/s10834-015-9479-7

Johnson,E.,&Sherraden,M.S.(2007).FromFinancialLiteracytoFinancialCapabilitysmongYouth.J.Soc.&Soc.Welfare,34,119.

Kamis,K.,Janevic,M.R.,Marinec,N.,Jantz,R.,Valverde,H.,&Piette,J.D.(2015).AstudyofmobilephoneuseamongpatientswithnoncommunicablediseasesinLaPaz,Bolivia:implicationsformHealthresearchanddevelopment.GlobalizationandHealth,11(1),1-10.doi:10.1186/s12992-015-0115-y

Karlan,D.,McConnell,M.,Mullainathan,S.,&Zinman,J.(2010).Gettingonthetopofmind:Howremindersincreasesavings.NBERWorkingpaperNo.16205.

Karlan,D.,Morten,M.,&Zinman,J.(2012).Apersonaltouch:Textmessagingforloanrepayment:NationalBureauofEconomicResearch.

Kempson,E.,Collard,S.,&Moore,N.(2006).Measuringfinancialcapability:AnexploratorystudyfortheFinancialServicesAuthority.ConsumerFinancialCapability:EmpoweringEuropeanConsumers,39.

Kempson,E.,&Finney,A.(2009).Savinginlower-incomehouseholds.AReview.Kiviat,B.,&Morduch,J.(2012).Fromfinancialliteracytofinancialaction.FinancialAccessInitiative,

RobertF.WagnerGraduateSchoolofPublicService.(NewYorkUniversity).Knowles,M.S.,Holton,E.F.,&Swanson,R.A.(1998).TheAdultLearner.TheDefinitiveClassicin

AdultEducationandHumanResourceDevelopment.Latimer-Cheung,A.E.,Rhodes,R.E.,Kho,M.E.,Tomasone,J.R.,Gainforth,H.L.,Kowalski,K.,...

Duggan,M.(2013).Evidence-informedrecommendationsforconstructinganddisseminatingmessagessupplementingthenewCanadianPhysicalActivityGuidelines.BMCPublicHealth,13(1),1-13.doi:10.1186/1471-2458-13-419

Liu,J.X.,&Modrek,S.(2016).EvaluationofSMSremindermessagesforalteringtreatmentadherenceandhealthseekingperceptionsamongmalariacare-seekersinNigeria.HealthPolicyandPlanning.doi:10.1093/heapol/czw076

Loibl,C.,Jones,L.E.,Haisley,E.,&Loewenstein,G.(2016).TestingStrategiestoIncreaseSavingandRetentioninIndividualDevelopmentAccountPrograms.AvailableatSSRN2735625.

Lyons,A.C.,Chang,Y.,&Scherpf,E.(2006).Translatingfinancialeducationintobehaviorchangeforlow-incomepopulations.JournalofFinancialCounselingandPlanning,17(2).

Marketo.(2015).StrategicmobilemarketingwithSMS&MMS:MarketoInc.Marteau,T.,Ogilvie,D.,Roland,M.,Suhrcke,M.,&Kelly,M.(2011).Judgingnudging:cannudging

improvepopulationhealth?BMJ,342,d228.doi:http://dx.doi.org/10.1136/bmj.d228Miller,M.,Reichelstein,J.,Salas,C.,&Zia,B.(2014).CanYouHelpSomeoneBecomeFinancially

Capable?AMeta-AnalysisoftheLiterature.WorldBankPolicyResearchWorkingPaperNo.6745Retrievedfromhttp://ssrn.com/abstract=2380391.

Milward,J.,Day,E.,Wadsworth,E.,Strang,J.,&Lynskey,M.(2015).Mobilephoneownership,usageandreadinesstousebypatientsindrugtreatment.DrugandAlcoholDependence,146,111-115.doi:http://dx.doi.org/10.1016/j.drugalcdep.2014.11.001

Muench,F.,vanStolk-Cooke,K.,Morgenstern,J.,Kuerbis,A.N.,&Markle,K.(2014).UnderstandingMessagingPreferencestoInformDevelopmentofMobileGoal-DirectedBehavioralInterventions.JMedInternetRes,16(2),e14.doi:10.2196/jmir.2945

Muir,K.,Reeve,R.,Connolly,C.,Marjolin,A.,Salignac,F.,&Ho,K.(2016).FinancialResilienceinAustralia2015.Sydney:CentreforSocialImpact(CSI),UniversityofNewSouthWales,forNationalAustraliaBank

Pathak,P.,Holmes,J.,&Zimmerman,J.(2011).Acceleratingfinancialcapabilityamongyouth:Nudgingnewthinking.NewAmericaFoundation.

Page 138: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

138

Ranney,M.L.,Choo,E.K.,Cunningham,R.M.,Spirito,A.,Thorsen,M.,Mello,M.J.,&Morrow,K.(2014).Acceptability,Language,andStructureofTextMessage-BasedBehavioralInterventionsforHigh-RiskAdolescentFemales:AQualitativeStudy.JournalofAdolescentHealth,55(1),33-40.doi:http://dx.doi.org/10.1016/j.jadohealth.2013.12.017

Reich,C.M.,&Berman,J.S.(2015).DoFinancialLiteracyClassesHelp?AnExperimentalAssessmentinaLow-IncomePopulation.JournalofSocialServiceResearch,41(2),193-203.doi:10.1080/01488376.2014.977986

Robson,J.(2012).Thecaseforfinancialliteracy:AssessingtheeffectsoffinancialliteracyinterventionsforlowincomeandvulnerablegroupsinCanada.CarletonUniversity,Canada:CarletonUniversity

Russell,R.,Bailey,D.,Wall,L.,&Brooks,R.(2011).MoneyMindedsummaryreport:FeaturingacasestudyofMissionAustralia’sfinancialliteracyprogramforyouthat-risk.Melbourne:Retrievedfromhttps://www.anz.com/resources/c/b/cb9ea0804692672e97cd9fff59dc78ef/Money+Minded+Summary+Report+2010-FINAL.pdf?MOD=AJPERES.

Russell,R.,Stewart,M.,&Cull,F.(2015).SaverPlus:Adecadeofimpact.ReportproducedfortheANZBank,Australia.

Scharl,A.,Dickinger,A.,&Murphy,J.(2005).Diffusionandsuccessfactorsofmobilemarketing.ElectronicCommerceResearchandApplications,4(2),159-173.doi:http://dx.doi.org/10.1016/j.elerap.2004.10.006

Schuchardt,J.,Hanna,S.D.,Hira,T.K.,Lyons,A.C.,Palmer,L.,&Xiao,J.J.(2009).Financialliteracyandeducationresearchpriorities.JournalofFinancialCounselingandPlanning,20(1).

Schwerdtfeger,A.,Schmitz,C.,&Warken,M.(2012).UsingTextMessagestoBridgetheIntention-BehaviorGap?APilotStudyontheUseofTextMessageReminderstoIncreaseObjectivelyAssessedPhysicalActivityinDailyLife.FrontiersinPsychology,3(270).doi:10.3389/fpsyg.2012.00270

Sunstein,C.R.(2014).Nudging:averyshortguide.JournalofConsumerPolicy,37(4),583-588.Taylor,S.,&Wagland,S.(2013).TheSolutiontotheFinancialLiteracyProblem:WhatistheAnswer?

AustralasianAccountingBusiness&FinanceJournal,7(3),69.Thaler,R.H.,&Sunstein,C.R.(2008).Nudge:Improvingdecisionsabouthealth,wealth,and

happiness:YaleUniversityPress.Thirumurthy,H.,&Lester,R.T.(2012).M-healthforhealthbehaviourchangeinresource-limited

settings:applicationstoHIVcareandbeyond.BulletinoftheWorldHealthOrganization,90(5),390-392.

Tuominen,M.C.,&Thompson,E.L.(2015).“ThereWasNoMoneyLefttoSave”:FinancialLiteracyandtheLivesofLow-IncomePeople.JournalofProgressiveHumanServices,26(2),148-165.doi:10.1080/10428232.2015.1018104

vonStumm,S.,FentonO’Creevy,M.,&Furnham,A.(2013).Financialcapability,moneyattitudesandsocioeconomicstatus:Risksforexperiencingadversefinancialevents.PersonalityandIndividualDifferences,54(3),344-349.doi:http://dx.doi.org/10.1016/j.paid.2012.09.019

Vyvyan,V.,Blue,L.,&Brimble,M.(2014).Factorsthatinfluencefinancialcapabilityandeffectiveness:Exploringfinancialcounsellors’perspectives.AustralasianAccounting,BusinessandFinanceJournal,8(4),3-22.doi:http://dx.doi.org/10.14453/aabfj.v8i4.2

WesleyMission.(2010).Makingendsmeet:Financialstressisnotjustaboutmoney.Sydney:WesleyMission.

WesleyMission.(2011).WesleyMissionandStGeorgeBankFinancialLiteracyEducationProject:Strategypaper.Sydney:WesleyMission.

WesleyMission.(2015).Annualreport2015;Strength,spirit,hope.Sydney:WesleyMission.WesleyMission.(2016a).Financialliteracytraining.Sydney:WesleyMission.WesleyMission.(2016b).Wesleyfinancialliteracyprogramflyer.Sydney:WesleyMission.

Page 139: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

139

Appendices

Appendix 1 TableA1:Pearson’scorrelationsbetweenparticipantfactorsandoutcomesatTime2

ParticipantFactorsImprovementsfrombeforeprogramto5-7monthsafter

programProgramengagementatTime1

OutcomesatTime2

GeneralFinancialknowl-edge

Personalfinancialknowl-edge

Attitudestospending

FinancialSelfEfficacy

Emotio-nalspending

Motivati-onduringprogram

Programrating

Improvedmoneybehaviour

r=.27p=.064n=48

ns

r=.42p=.003n=48

r=.30p=.046n=46

r=.28p=.048n=49

r=.43p=.002n=49

r=.36p=.010n=49

Improvedsavingbehaviour

ns

ns

r=.30p=.030n=51

ns

ns

ns

ns

Improvedplanningbehaviour

ns

ns

r=.26p=.060n=51

ns

ns

ns

r=.27p=.043n=54

Improvedmonitoringbehaviour

ns

r=.35p=.013n=51

r=.42p=.002n=51

r=.33p=.020n=48

ns

ns

r=.27p=.050n=54

Howmuchdidtheprogramhelpachievegoals

r=.33p=.019n=51

ns

r=.36p=.009n=51

r=.41p=.003n=48

r=.30p=.031n=51

ns

r=.400p=.003n=54

Extentofprogresstowardsspecificgoal

ns

ns

r=.31p=.029n=51

r=.45p=.001n=48

ns

ns

ns

Fewerproblemsdueshortageofmoney

ns

ns

ns

r=.36p=.012n=48

r=.29p=.038n=51

ns

ns

Lessdifficultypayingbills

ns

ns

ns

ns

ns

ns

ns

Moremoneyleftover

ns

r=.27p=.059n=51

ns

ns

ns

ns

ns

Lessoftenaskfamilyformoney

ns

ns

ns

ns

ns

ns

ns

Changeindebt ns

ns

r=-.32p=.051n=37

ns

ns

ns

ns

Page 140: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

140

Appendix 2

TableA2:Pearson’scorrelationsbetweenparticipantfactors(ns=non-significant

ParticipantFactorsImprovementsfrombeforeprogramto5-7monthsafter

programProgramengagementatTime1

Generalfinancialknowledge

Personalfinancialknowledge

Attitudestospending

Financialself-efficacy

Emotionalspending

Motivationduringprogram

Programrating

Generalfinancialknowledge

-

Personalfinancialknowledge

r=.61p<.005n=50

-

Attitudestospending

r=.43p=.002n=50

r=.40p=.004n=50

-

Financialself-efficacy

ns

ns

r=.34p=.022n=46

-

Emotionalspending

ns

ns

ns

r=.47p=.001n=46

-

Motivationduringprogram

ns

ns

r=.28p=.044n=51

ns

ns

-

Programrating

r=.44p=.001n=51

ns

r=.29p=.042n=51

ns

ns

r=.66p<.005n=54

-

Page 141: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

141

Appendix 3

TableA2:Backwardsstepwisemultipleregressionfinalmodels

Outcomemeasure F p R2 Variablesremaininginthefinalmodel

Improvedmoneybehaviour

(2,41)=12.33 <.005 .38 LessemotionalspendingProgramrating

Improved

savingbehaviour

(1,42)=3.57 .066

(onlymarginallysignificant)

.08 Improvedattitudetospending

Improvedplanningbehaviour

(1,42)=3.72 .061

(onlymarginallysignificant)

.08 Improvedpersonalfinancialknowledge

Improvedmonitoringbehaviour

(2,41)=6.24 .004 .23 Improvedattitudetospending

Improvedfinancialself-efficacy

Howmuchdidtheprogramhelpachievegoals

(2,41)=11.86 <.005 .37 Improvedfinancialself-efficacy

Programrating

Extentofprogresstowardsspecificgoal

(1,42)=9.70 .003 .19 Improvedfinancialself-efficacy

Fewerproblemsduetoshortageofmoney

(1,42)=8.45 .006 .17 Improvedfinancialself-efficacy

Lessdifficultypayingbills Nomodelwassignificant

-

-

-

Moremoneyleftover Nomodelwassignificant

-

-

-

Lessoftenaskfamilyformoney

(3,39)=2.95 .044 .19 Improvedpersonalfinancialknowledge

Improvedattitudetospending

Improvedfinancialself-efficacy

Changeindebt F(1,30)=3.63 .067

(onlymarginallysignificant)

.11 Improvedattitudetospending

Page 142: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

142

Appendix 4

FigureF1:TheinteractionbetweentimeandreceivingthreeormoretextmessagesonMoneyBehaviour

Page 143: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

143

Appendix 5

FigureF2:TheinteractionbetweentimeandreceivingthreeormoretextmessagesonSavingBehaviour

Page 144: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

144

Appendix 6

FigureF3:TheinteractionbetweentimeandreceivingthreeormoretextmessagesonMonitoringExpenses

Page 145: Evaluating the Wesley Mission - financial capability · Evaluating the Wesley Mission In charge of my money financial literacy program for vulnerable populations. Sydney: Wesley Mission

EM

T452

9_16

08