View
216
Download
0
Category
Preview:
DESCRIPTION
Summary Diagnostic Conducting policy research with federal departments and agencies on measuring and reporting on the impacts of federal S&T is essential –a key commitment under the federal S&T Strategy announced in May 2007 –federally-performed S&T is considerable (about $5 billion annually) –a significant recent policy literature to draw from 3
Citation preview
1
Improving Measurement and Reporting on the Impacts of Federal Science and Technology
Presentation to the FPTT 2008 National MeetingJune 3, 2008
André DownsDirector General
About the Policy Research Initiative• Horizontal policy research that supports strategic policy
development and medium-term planning
• Overseen by a Deputy Ministers Steering Committee
• Support CCDM policy committees and PCO planning
• Research portfolio: economic policy social policy climate change environmental scanning (e.g., Canada@150)
2
Summary DiagnosticConducting policy research with federal departments and agencies on measuring and reporting on the impacts of federal S&T is essential
– a key commitment under the federal S&T Strategy announced in May 2007
– federally-performed S&T is considerable (about $5 billion annually)
– a significant recent policy literature to draw from
3
The Project in 1 Slide• Draw implications from recent policy research, the
experience of practitioners, and from original analysis, to suggest how the impact of the S&T performed by the federal government can be better measured and reported, and thereby potentially enhanced over time.
• A forum or “neutral-space” for federal departments and agencies to identify and explore: concepts that can be applied to draw a more complete
picture of the diverse impacts of S&T activities performed by the federal government;
options for improving reporting on those activities; and approaches for how to maximize the benefits to
Canadians of those activities in relation to costs.
4
Governance• Oversight by an inter-departmental
steering committee
• Consensus-driven
• Preliminary findings tested inside and outside the federal government
• Public report by February 2009
5
Project Deliverables1. Research report on how publicly-funded research organizations
transmit technology and knowledge to innovators. (StatCan lead)2. Identify and assess approaches for additional reporting. (PRI lead)3. A “think piece” on identifying, describing, and measuring the
impacts of federally-performed S&T. (IC lead)4. Synthesis reports on the market and non-market impacts on
Canadians of federal S&T investments. (PRI lead)5. Four events to present and discuss results.6. Lessons learned and best practices for assessing and reporting
impacts. (NRC lead)7. Additional work on non-market impacts. (EC lead)
6
1. Public labs and innovators• Research report by Frances Anderson from
Statistics Canada
• Applies international definitions and data from the 2005 Canadian Survey of Innovation in Manufacturing to examine how public labs (universities, federal labs, and provincial labs) transmit knowledge and technology to innovative firms
7
1. Public labs and innovators
Source Some importance
High importance
Clients or customers 91.2%* 51.9%*
Suppliers 89.7%* 33.4%*
Conferences, trade fairs and exhibitions 81.7%* 14.3%*
Universities 45.3% 3.8%
Federal research labs 35.2% 1.3%
Provincial research labs 34.0% 1.0%
Percent of Manufacturing Innovators Indicatingthat the Source was Important for Innovation
Source: Statistics Canada, Survey of Innovation 20058
1. Public labs and innovators• Sectors more likely to identify public labs as important:
natural resource processing machinery and equipment aerospace
• The innovators most strongly linked to public labs tended to:
be larger; have employees devoted to R&D; be engaged in more innovation activities; collaborate more with public and private organizations;
and receive funding from more public and non-conventional
private sources.
9
1. Public labs and innovators
Percent of Manufacturing Innovators Indicating thattheir Innovations had an Important Impact, by Impact Area
Impact Strongly linked
Not strongly linked
Improving job quality 43.1%* 33.2%Reducing environmental impacts 29.4%* 18.2%Improving health and safety 42.1%* 31.5%Meeting regulatory requirements 40.7%* 26.6%Reducing materials or energy per unit output
42.2%* 26.7%
Source: Statistics Canada, Survey of Innovation 2005
10
2. ReportingDefinitions
Resources are allocated to …
… Activities that lead directly to …
… Outputs that are intended to achieve …
… Outcomes (or Results) that benefit the country.
Impacts encompass Outputs and Outcomes
11
2. ReportingProportion of Federal Technology Commercialization Programs
by Type of Indicator in Estimates and Related Documents
All programs Larger programs
None 51% 35%
Activity* 19% 27%
Output 17% 19%
Outcome 13% 19 %
Total 100% 100%
* Funded activity described, but no indication provided of its impact
Source: PRI research, based on a database prepared by Industry Canada reported in The Impact Group (July 2005), Commercialization Activities of the Federal Government: Program Synopsis.
12
2. ReportingThe Relationship among Planning,
Reporting, and Impact Type
Kinds of Impact
Outcome Activity Output
PlanningPlanned outcomes
≈ Objectives ≈ Strategic outcomes
Planned activities
Planned outputs
Reporting
Results of evaluations, audits and
reviews
Actual activities
Actualoutputs
13
2. ReportingPossible Approaches
1. Explicitly incorporate S&T in the Program Activity Architectures (PAAs) of departments and agencies by:A. Treating S&T as an internal service;B. Portraying S&T activities as a single program within each
department and agency;C. Describing S&T in terms of activities that in some cases fall
under broader programs; orD. Describing S&T in terms of projects (where possible).
2. Expand the annual Statistics Canada survey to collect data in terms of the higher-order PAA categories of departments and agencies.
3. Prepare one government-wide S&T report.
4. Prepare annual S&T reports for each department and agency.
14
3. Think piece on identifying, describing and measuring the impacts of federally-performed S&T
• Federal S&T Strategy committed to increasing accountability by improving ability to measure impacts of federal S&T investments
• Considers lessons learned based on past efforts by federal S&T performers to measure impacts
• Paper designed to help better capture full range of impacts for federally-performed S&T
– Analytical framework linking impacts to Strategy’s Entrepreneurial, Knowledge and People Advantages
– Currently being finalized
15
4. Synthesis reportsThe Market Impacts of Federal S&T Investments
• No recently prepared synthesis has fully reflected the sizeable empirical literature
• Report will identify: Policy implications based on the synthesis; The methodologies that have been most
successful in deriving results; Outstanding policy-relevant research questions;
and Suggestions for further research and data
development.
16
4. Synthesis reportsThe Non-Market Impacts of Federal S&T Investments
• Report will provide: An overview of federal S&T; Conceptual discussion of the non-market impacts of
S&T investments; Literature review with a focus on health, safety and
security, and the quality of the environment; Empirical evidence on the linkages between market
and non-market impacts; Outstanding research issues and knowledge gaps; and Policy implications.
17
5. Project events• Preliminary findings in July 2008• Public Science and the Marketplace: Connecting,
Measuring and Reporting in September 2008 How do governments support researchers and
innovators to develop new ideas and bring them to the marketplace, and how can they measure and report on these efforts?
Presentations by Howard Alper, Chair of the Science, Technology and Innovation Council, Ken Carlaw, Richard Lipsey's co-author and collaborator, and Paul Johnston, President and CEO of Precarn, among others
• Synthesis reports in October 2008• Draft project report in early 2009
18
Next steps
• Further work on deliverables
• Events from July 2008 to early 2009
• Final report by February 2009
19
Recommended