Measuring what matters: Approaches to modernizing grant performance measurement that promote innovation, transformation and deliver impact Alex Sirota,

Measuring what matters:Approaches to modernizing

grant performance measurement that promote innovation,

transformation and deliver impact

Alex Sirota, Team Lead, eService DeliveryMinistry of Economic Development and Trade

@Results Based Management For Government Conference

Ottawa, Ontario, Canada December 2009

2http://www.ontario.ca/economyDecember 2009

4 objectives + 1 case study

Is your program a candidate for modernization?

How people, process and technology work to transform

How to achieve credible program outcomes

How to measure the results of your program

A real-world example


A candidate for modernization?

Are there specific pains to address?

Is there support from senior management for a change?

Are there technologies that can be readily reused or applied?

Does your organization need a repeatable, agile and measurable process to run a grant program and measure its performance?


People, Process & Technology

People– Understand what your staff can and cannot do– Understand what management is willing to risk, spend &

achieve

Process– What are the really painful parts of the process?– Continuous improvement

Technology– Technology is readily available as are skill sets – Choose tried and true solutions– Choose what fits your organization and iterate


Credible Program Outcomes

Can you communicate objectives and results?– Press-release worthy?– Celebrate wins– Survey perception from internal and external

stakeholders

New challenges or opportunities as a result of transformation?– Grant programs can be perceived as sustainable and

scalable– Project team engages to achieve a shared goal– Raises expectations for program (scrutiny, volume,

marketing)


Measure – Iterate - Innovate

Identify what you want to improve, and then do it– Step by step and allow for team to integrate into workflow

Things to consider – Process

– Review & grant approval processing time– # of grants processed pear year (if budget allows)– Privacy concerns, Accountability & transparency– Ability to measure detailed information pre and post approval– Determining when a proponent is “successful” or “not successful”

– People– Satisfaction of program officers, satisfaction of grant recipients– Impact of improvements to daily job

– Technology– Flexible and easily adapted application process– Collecting too much, not enough information?


There is no such thing as “best practice.”

Best-practice is yesterday’s news and contextual.

Emerging practices are tomorrow’s news and you can claim as your own.

Every year more grant programs are modernizing as technology matures.


What does transformation look like?


What is Summer Company?

An entrepreneurship experience for 15-29 year old students in Ontario - www.ontario.ca/summercompany

$3000 award for the duration of the summer to start and operate a business

Delivered by a network of program providers across Ontario– Alternate service delivery– Small Business Enterprise Centres + Non profit

organizations


A good candidate for modernization

In 2001, paper-based application and approval process– 190 approved awards

Inconsistent application and delivery process across Ontario

Time & resource intensive pre- and post-approval process

Difficult to determine “successful completion” of awards

Many young entrepreneurs wait until the last minute to apply, leading to a “python” effect of recommended applications

Modernization started 2003


Before modernization


Summer Company, today

5 years to fully implement process and system, by choice

Version 9 is in operation, delivered by 60+ delivery partners

Processed over 15,000 application inquiries and over 2,000 approved grants as of 2009

Major issues or opportunities– Avoid resistance to change by involving end users– Privacy Impact Assessment completed, recommendations underway– Improve reporting and business intelligence functionality

Next Steps– Business process and technology can be made generic for similar grant

programs– Collecting broader public sector interest to adopt business process

and/or technology


Results

2009 had over 600 approved awards, 200% increase over 2001

Manage more awards with same or less amount of staff

Significant workflow improvement

Transparency & accountability (Cashflow, Journals, Checklists)

Improved standard of service delivery, internally and throughout Ontario, reduced paper burden


Our “best-practices”

No grand vision for what the program would be in 5 years– build in quality through listening and iteration

Get feedback from end users, constantly and implement priority changes

Checks and Balances done throughout process leads to improved application rate and reduced default rate on awards


After modernization

Happy, productive program teams.


Thank you for your time!

For more information, contact– Alex Sirota, Team Lead, eService– MEDT – Entrepreneurship Branch– 416 212 0549 – [email protected]

Offer– Do you want to see the system in operation to learn

more?– Self running “Flash” demo– UAT system is always available via web conference-call


DEMO SCREENS






Documents

Measuring what matters: Approaches to modernizing grant performance measurement that promote innovation, transformation and deliver impact Alex Sirota,