What does management focused results-based management? · oriented culture • RBM for learning is...

(UNJIUReportDecember2017AnnexII)-AdaptedfromJohnMayne,“Resultsmanagement:canresultsevidencegainafootholdinthepublicsector?”inTheEvidenceBook:Concepts,GenerationandUseofEvidence,OlafRieper,FransLeeuwandTomLing,eds.(NewBrunswick,TransactionPublishers,2010))Planning(results-based)

•  Planning(results-based)•  Delivery•  Monitoring,analysis+evaluation•  Learning+action•  Improvingdesign•  Improvingmeasurementstrategy

Whatdoesmanagementconsiderinoutcome-focusedresults-based

management?

OngoingIssueswithRBM:

(Remember,wehavebeenatthisfor30+years!)JohnMayne

John.mayne@rogers.com

KeyProblemswithRBM

• RBMmadetoappearstraightforward/simple• Useforaccountabilitydrownsoutlearning• Notunderstoodwithinorganizations• NoincentivestouseRBM• Basedonindicators•  ITsystemscan’thandleresultsinformation• BeliefthatRBMdoesn’trequireadditionalresources

BUT

• RBMisnotstraightforward,notsimple• RBMrequiresthinking,understanding,experienceandprofessionalexpertise

• RBMrequiresongoingsupport,incentives,resources:aresult-orientedculture

• RBMforlearningisasignificantchallenge•  IndicatorsareNOTenough:

Youcannotlearnandmanagewithindicators

Disappointingly

Wearen’tlearningwellaboutwhatisneededtogetRBMto

workeffectively

Butmaybethereishope

PrinciplesforRBMImportant•  ThinkingaboutanddoingRBMrequiresPrinciplesforguidance• NeedtounderstandwhatRBMisallabout,whatispossibleandhowresultsinformationcanbeused

• UNandOECDhavebeendevelopingRBMprinciplesformanyyears.•  JIU2017reportsthemostrecentversionandthemostinsightful

KeyNewFeaturesofJIUPrinciplesRBMPrinciplesrecognizetheneedto:•  Focusonoutcomes• Appreciatethecomplexityofinterventions• DevelopanduseToCs• Understandhowinterventionsareinfluencingresults,howtheymakeacontribution

•  Linkmonitoringandevaluation•  Thinkbeforesettingoutmeasurementapproaches

NickKowbel,currentevaluatorintheUnitedNationsSecretariat,andformerevaluatorintheGovernmentofCanada“viewspresentedaremyownanddonotnecessarilyreflecttheofficialviews

oftheOfficeofInternalOversightServices”

Who conducts Evaluation in the UN?

Source:OIOS-IEDManual2014edition

Donors

OIOS evaluations find challenges with fully implementing RBM

Implications for JIU

principles of RBM

Source: OIOS-IED internal assessment of 2018-2019 reports

•  Need to focus more

on theory of change

(validity of results chain);

•  use of strategies

that acknowledge

niche roles; •  resolve challenges

with M&E and use of

evidence in decisions

ManagingForResults:UNIDO’sIntegratedResultsand

PerformanceFrameworkPPXSymposiumOttawa,Canada

15May2019

GiuseppeDeSimoneOfficeofStrategicPlanning,Coordination

andQualityMonitoring

11

Content

• Theimperativeofmanagingforresults• TheIRPFandtheUNIDOstrategicplanningframework• OurrenewedtakeonRBM:anactor-based,behavioralchangemodel

• TellingtheUNIDOperformancestory• Benefitsandchallengesgoingintoimplementation

12

Theimperativeofmanagingforresults

• Inthe2030Agendaera,OrganizationsfordevelopmentmustbeabletoshowtheircontributiontoSDGs

• SDGs:country-ownedtransformational,systemicchange!• TheUNDSreformgoesinthesamedirection:Organizationstodemonstratecontributionàbutattributionisachallenge!

• Aresultsframeworkismuchmorethanareportingtool:itmeasuresprogresstowardsmanagementforchange,atscale

• UNIDO’sintegratedresultsandperformanceframework(IRPF):directemanationofUNIDO’sstrategicplan

13

Thestrategicplanningframework:integrationandscaleup•  Betterintegrationacrosscorefunctionstoenablescalingupofresults

•  Strengtheningknowledgeandinstitutionsaskeyenabler

•  Strengthenresultsorientation•  BettershowcontributiontotransformationalchangeandSDGs

•  BetterequipUNIDOforUNDSreformandSDGs

14

MakingRBMworkformorethanindicatorsWefollowthesequenceofRBMprinciples(JIU2017):1.   Visionandclarityofdesiredendproductorimpact2.   Causallinkageinahierarchyofresults(inputs,activities,outputs,

outcomes,impact)basedontheoryofhowchangehappens3.   System-operationthatgoesbeyondlinearlogicandconsiderscontext,

espousesequifinality,andaddressesrisksandconditionsforsuccessforachievinghigher-levelresults

4.   Performancemeasurementfortransparency,consensusbuilding,andhavingacommonperspectiveonresults,aswellasforaccountability

5.   Performancemonitoringforsinglelooplearning,andevaluationfordouble-looplearning

15

OurrenewedtakeonRBM

• Anactor-based,behavioralchangemodeltocrediblycapturecomplexity

•  Importanceofkeyactorsinthesystematthemicro,mesoandmacrolevel

•  Focusonchangeofbehaviors,whichtriggerhigher-levelresults• Amixofindicatorsandnarrativestotelltheperformancestory• EmbedsaTheoryofChangeapproachwithrisksandexternalfactors

16

Telling the UNIDO performance story

UNIDOisthekeyOrganizationthatprovidesanintegratedapproachtosupportinclusiveandsustainableindustrialization,toaddressthecomplexityoftheentailedsystemictransformation

UNIDOaddressesthecomplexityofISIDbycloselyworkingwithallrelevantactors:individuals&firms;intermediaryinstitutions;policymakersandglobalplayers

UNIDOsupportstheseactorsinstrengtheningtheirtechnical,institutionalandpolicy knowledge&understandingofISIDissues,andassociatedskills&capacities

Withavailableresourcesandhighlyspecializedstaff,UNIDOsupportsby:TC;Policyanalysis&adv.;Norms&standards;Convening&partnerships

UNIDOthereforecancrediblyclaimitscontributiontoISIDandtheSDGsbysupportingcountriesin:

UNIDOsupportstheseactorsinchangingkeybehaviors:

Knowledge&Capacities

Reach&Reactions

Sex,ageandcountrydisaggregateddatawhereverpossible

17

Level3:Programmaticoffer&programmemanagement

effectiveness

Level4:Organizationalresourcesandefficiency

Level2:CountryandglobalresultwithUNIDOsupport

Level1:Globalindustrialdevelopmentcontext

UNIDO-relatedSDGindicators(UNIDOascustodianagency)

18

Howdoesitwork?Clearinghouse

anddataanalysis

SPQand

IRPFTaskForce

Data*sourceandqualityassurance

Researchandstatisticsdepartment

TCDirectorate(projectmanagers,alldepartments,fieldoffices)Policy,ResearchandStatisticsOfficeofindependentevaluation

TCDirectorate(projectmanagers,alldepartments,fieldoffices)CorporateservicesOfficeofDirectorGeneral(strategy,quality)

Corporateservices(procurement,finance,HR,etc.Externalrelations/CommunicationInternalOversight/evaluation

Level1

Level2

Level3

Level4

*Note:KPIdatafromnewPPMorin-housesurveyswhenRBbased

19

Benefits1.  BettertellUNIDO’sperformancestory,whereittrulyis

•  Ownournarrative,changetheconversation(from$$$toimpact)•  SpelloutUNIDO’scontributiontotransformationalchange,ISID,theSDGs•  Provideconvincingevidencefornonproject-based,lessearmarkedfunding

•  MeetthechallengeofUNDSreform(crediblyshowcontributiontoSDGs)

2.   Harmonizescorporateresultsindicatorswithprojectresultstoavoidduplicationandfacilitatealignment Noadditionalburden!

4.   Aligntostrategy:•  Bemorethanthesumoffragmentedprojects•  LinkprogrammaticKPIstoorganizationalresults

6.   Theprocessitself:•  developedandagreedwiththewholehouse(top-downmeetsbottom-up):groundedonacomprehensiveprogrammemapping

20

Amanagementtool,withalreadyvisibleresults

•  IRPFispartofthebiggerpolicyarchitecture• AtooltocomplementandreinforceothereffortstoreformUNIDO:

•  QualityAssuranceFramework•  Adaptivemanagement•  Standardization•  Reengineeredprojectmanagement•  Alignmenttostrategicdocuments•  Integrationandscaleup(MTPF)

Changeisalreadyhere:newprogrammesbeingdevelopedarealreadyusingtheIRPFframework:sustainablecities,migration,youth,MICs,GEEWstrategy,AGRservicemodules,PCPEthiopia…

21

Showingourcontributionstory

• WearedevelopingavisualforUNIDO’sISIDandSDGcontribution

•  Scalableforaproject,aprogramme,acountryprogramme/PCP,UNIDOasawhole.

The width of the arrows corresponds to the data in the IRPF framework

ConsideraTC-heavyUNIDO• WhatifUNIDOgoesallupstream?UNIDO’sapproach:Integrationandscaleup

The impact contribution arrows will depend on theoretical linkages and empirical evidence from UNIDO interventions

22

Goingintoimplementation•  Changes are being introduced in a gradual manner, after testing and

validation. •  Consistent signals from policies, indicators and narratives to

facilitate the required shift in mindsets and processes •  Challenges:

•  Change management – (not again!!!) •  Capacity – training needs •  Financial investments – but not massive •  Timing – UNDS reform, donor assessments etc. •  Data availability: time lag in achieving impact

•  The consistent leadership of senior and middle management will continue to be crucial during implementation

•  Constant engagement with Member States is also be necessary

23

Conclusions1.  Acomprehensivesystemtodrivechange,basedonanactor-

basedbehavioralchangemodel,embeddingaToCapproach–scalablefromprojectstocorporate

2.  StrengthenstheabilityofUNIDOtorealizeandshowitsuniquecontributiontoISIDandSDGsanddrivefurtherintegrationandscaleupofresults

3.  TheIRPF(indicatorsandnarratives)providesmeasuresofprogress–amanagementtoolforUNIDOandareportingtoolforexternalparties

24

Thankyou!

g.desimone@unido.org

25

AnnexesTesting

withprojects,programmesandcountryreports

26

IRPFtesting–itworks!•  AtestingexercisewasconductedwithPTCdepartmentstesttherelevance,robustnessandusefulness.Examples:

•  TII–SMARTFishproject

•  ENV–Eco-industrialparkprogramme•  ENE–EnMSprogramme,SustainableCities•  AGR–Servicemodules

•  RequestfromPCP–EthiopiatosupportPCPresultsframeworkalignedtoIRPF

•  Conclusions:•  GreatmatchofIRPFindicators–relevance,robustnessandusefulnessconfirmed•  RequestsformoregranularitytocapturedetailsandmorevalueofUNIDOwork•  Eagernesstomoveawayfromvolumesapproachandfindwaystobettershowresults•  WealthofknowledgefromUNIDOstaff(incl.Field!)–sofarpartlyuntapped

27

ExampleoftheIRPFtestingatprojectlevel

28

Indicators tell the UNIDO performance story at PROJECT level (e.g. Smart Fish project)

UNIDOisthekeyOrganizationthatprovidesanintegratedapproachtoISIDservices,toaddressthecomplexityofthesystemictransformationitentails

•  Farmersandfirmsalongthreevaluechains•  Privatesectorrepresentatives•  National,provincialanddistrictlevelgovernments•  Multilateralandbilateralagencies

•  5401participantsin127capacitybuildingrelatedactivities

•  3valuechainroundtabledialogues•  EstablishmentofTropicalSeaweedInnovationNetwork•  Improvedmanagementpracticesalongthevaluechains(ISPIREDtool,standardoperating

procedures,branding);e.g.590farmersadoptedtheSOP•  4Women-ledhomebasedunitsestablishedregisteredtoproducesea-weedbasedfoods•  Morethan3000farmsand13processingcompaniesjoinedthetraceabilityplatform•  iPride4Fishcentreestablishedasasemi-autonomouscentrewithintheuniversityoffisheries•  18seaweedbasedfood/drinkproductscommerciallyproducedvaluedatUSD350,000peryear•  USD6,426,000newinvestmentraised

•  20,460USDpermonthestimatedvalueofproductionfornewmicroenterprises(undercomponent2:seaweedqualityandproductivitywhichincludedMSMEdevelopment)

•  1.2millionUSDestimatedannualsalesofexistingSMEs•  Productionofqualityfishincreasedfrom30to69-80%inTulungagun•  Participatingcompanieshaveover80%ofmarketshare

•  127capacitybuildingactivities(trainings,workshops,seminars,roundtablesorganized,etc.)•  TrainingonINSPIREDtoolandonstandardoperatingprocedures•  Policyrecommendationsandpolicydialogues

3.1 Nr. of trainings/workshops provided

IRPFindicators

2.1 Nr. of firms reporting economic gains 2.5 Nr. of MSMEs reporting increased market inclusion

2.14 Nr of firms introducing improved management practices (incl. business models/business plans, products) 2.15 Nr. of firms reporting adoption of new technologies 2.20 Value ($) of new investments leveraged 2.21 Nr. of firms participating in enhanced collaboration settings

2.25 Nr. of actors gaining awareness/ knowledge/skills on entrepreneurial/industrial skills, management practices, standards,etc.

2.26 (a) Nr. of firms/entrepreneurs/individuals reached 2.26 (b) Nr. of intermediary institutions reached 2.26 (c ) Nr. of governments/policymakers reached 2.26 (d) Nr. of international actors reached

29

Indicators tell the UNIDO performance story at PROGRAM level (e.g. Eco-Industrial Parks)

UNIDOisthekeyOrganizationthatprovidesanintegratedapproachtoISIDservices,toaddressthecomplexityofthesystemictransformationitentails

•  Industrialparksmanagementstaff•  Relevantnationalinstitutionsandgovernmentalagencies•  Residentfirms•  SMEstafftrained

•  onEIPpolicyandmanagement

•  ImprovementintermsofEIPinternationalframework•  Policiesreviewedandamended•  Synergyopportunitiesimplemented•  InvestmentsonRECP/EIP

•  Target:Atleast25%oftotalfirmworkersinindustrialparkareemployedthroughdirectemploymentandpermanentcontracts.

•  Target:Atleast25%ofresidentfirmsuselocalsuppliersorserviceprovidersforatleast80percentoftheirtotalprocurementvalue.

•  Target:Atleast90%oftotalprocurementvalueofparkmanagemententityaresuppliedbylocalfirmsorserviceproviders

•  Ratioofrenewableenergyused•  Ratioofwaterreused/recycled•  GHGemissionsandreductiontCO2eq./year

•  Capacitybuildingworkshopsorganized(EIPpolicy,industrialsynergy)•  Studytoursorganized•  Policyreview•  RECPassessment•  Conductindustrialsynergyfeasibilitystudy•  Outreachandawarenessraisingcampaigns

3.1 Nr. of trainings/workshops provided 3.7 Nr. of analytical publications released

2.26 (a) Nr. of firms/entrepreneurs/individuals reached 2.26 (b) Nr. of intermediary institutions reached

2.1 Nr. of firms reporting economic gains 2.5 Nr. of MSMEs reporting increased market inclusion 2.6 Number of people reporting improved access to manufactured goods 2.8 Indicator on reduced emissions 2.14 Nr of firms introducing improved management practices (incl. business models/business plans, products)

IRPFindicators

2.25 Nr. of actors gaining awareness/ knowledge/skills on entrepreneurial/ industrial skills, management practices, standards, innovation, gender, etc.

30

Indicators tell the UNIDO performance story at COUNTRY level (e.g. Egypt)

UNIDOisthekeyOrganizationthatprovidesanintegratedapproachtoISIDservices,toaddressthecomplexityofthesystemictransformationitentails

•  Ministries,policymakersandnationalinstitutions•  Youthandwomen•  MSMEsandproductiveclusters•  Publicandprivatebusinessdevelopmentserviceproviders•  Training/Educationalinstitutions•  Farmers•  Europeanmarketsforexport•  CivilsocietyandNGOs•  DevelopmentpartnersandIOs

•  3,500youthwithskillstorealizeopportunitiesfordecentemploymentandentrepreneurship•  7ministries,80policymakers,and39nationalinstitutionswithstrengthenedcapacitiesonISID

•  Over700women-ledstart-upsestablished•  AdoptionofISO50001standardonenergymanagement•  20,000farmersimprovedproducequality,foodsafetypracticesandaddedvaluetocrops•  EGP67millionmobilizedforanindustrialzoneinLuxor•  23linesofnewleatherandhabitatproductsdevelopedandexported

•  $852,000additionalincomeforruralfarmers•  Accesstocleanwaterfor150,000peopleinUpperEgypt•  10,000womencanaccessEGP2millionincommunitysavingsandloans•  1560OPDtonsreducedsince1993•  3.58MtCoeqlesscarbonemissionsemittedbyindustry

•  Trainingofyouthandwomenentrepreneurs•  Trainingoftrainersatthemesolevel•  Publicationsproducedwithlessonslearned(MENApublication)•  Holdeventsatnationalandregionalleveltoinformpolicydialogue

3.1 Nr. of trainings/workshops provided 3.7 Nr. of analytical publications released 3.12 Nr. of events held

2.26 (a) Nr. of firms/entrepreneurs/individuals reached 2.26 (b) Nr. of intermediary institutions reached 2.26 (c ) Nr. of governments/policymakers reached 2.26 (d) Nr. of international actors reached

2.25 Nr. of actors gaining awareness/knowledge/skills on entrepreneurial/industrial skills, management practices, standards, innovation, gender, etc.

2.14 Nr of firms introducing improved management practices (incl. business models/business plans, products) 2.15 Nr. of firms reporting adoption of new technologies 2.17 Nr. of new standards adopted 2.20 Value ($) of new investments leveraged 2.21 Nr. of firms participating in enhanced collaboration settings

2.1 Nr/Rate of firms reporting economic gains 2.3 Nr/Rate of firms reporting an increase in exports 2.5 Nr/Rate of MSMEs reporting increased market inclusion 2.8 Indicator on reduced emissions

IRPFindicators