PSI OnlineThe Socioeconomic Effects of Public Sector Information on Digital Networks: Toward a Better Understanding of Different Access and Reuse Policies

5 February 2008, OECD Headquarters Conference Centre, ParisChairs: A) Eivind Lorentzen, B) Jean-Jacques SahelRapporteurs: A) Juan Carlos de Martin, B) Tilman Merz

Summary of Session Three: Measuring the Economic and Social Costs and Benefits of PSI: Evaluation of the Existing Approaches and Suggestions for Future Work

OverviewI. Questions for

DiscussionII. Break Out Session A–

Government and PSI Supply-Side Experts

III. Break Out Session B – Industry Representatives and Users

IV. Summary

I. Questions for Discussion1. What are the commonalities and differences among the various

analytical methods identified and presented in Session two and in this Session?

2. What are the most effective metrics/indicators that can assess particular kinds of information/policies? What approaches and metrics/indicators effectively measure the network effects of the use of PSI online?

3. What are the main strengths and weaknesses of these approaches, e.g., their accuracy, comprehensiveness, relevance, validity and reliability?

4. What still needs to be known about the application of these methods to the evaluation of public information policies in the online environment?

5. What are theoretical frameworks, models and best practices used in assessing other information products or services that can be applied to the assessment of different policies of access to and reuse of digital PSI?

6. What are some future directions for the better study and measurement of access to and reuse of PSI online?

7. What other questions or issues should be raised in this context?

II. Break Out Session A – Government Representatives and PSI Supply-Side Experts

Session ASession A

Session A – Overview

Revised Questions for Discussion

1. What are the main strengths and weaknesses of these approaches, e.g., their accuracy, comprehensiveness, relevance, validity and reliability?

2. What are theoretical frameworks, models and best practices used in assessing other information products or services that can be applied to the assessment of different policies of access to and reuse of digital PSI?

3. What are possible future directions for the better study and measurement of access to and reuse of PSI online?Session ASession A

Question 1 – Existing Approaches

1. What are the main strengths and weaknesses of these approaches, e.g., their accuracy, comprehensiveness, relevance, validity and reliability?

• At this stage, no methodology seems clearly superior or general enough to be singled out.

• Need to proceed with more research using multiple methodologies.

Session ASession A

2. What are theoretical frameworks, models and best practices used in assessing other information products or services that can be applied to the assessment of different policies of access to and reuse of digital PSI?

• Existing frameworks could facilitate/cross-fertilise PSI analysis:

o Open Access / Scientific Publishingo Open Source / Free Softwareo Assessing experience in specific sectors

(meteorological services, geo-information, etc.)

• Existing research on the role of the public sector

Question 2 – Relevant Frameworks

Session ASession A

3. What are possible future directions for the better study and measurement of access to and reuse of PSI online?

a)OECD PSI Manual

b)PSI “Repository”

c)Academic Research

Question 3 – Future Directions

Session ASession A

a) OECD PSI Manual

• Progress in measurement of PSI policies needed

• Project would ideally involve statisticians, national accountants as well as PSI experts

• A similar model was successfully used by OECD DSTI in co-operation with Eurostat in 1999 to produce a manual on data collection and analysis in the environmental goods and services industry

Working towards a manual for PSI data collection and analysis of PSI policies

Session ASession A

a) OECD PSI Manual

An OECD PSI Manual would…

• take into account the OECD PSI Policy Principles;• address different audiences, such as

• Practitioners• Re-users• PSI Holders• National accountants;

• clarify the extent and kinds of PSI;• make measurement practices more comparable;• promote involvement of additional stakeholders

(regional, global, statistical entities, etc.).

Session ASession A

a) OECD PSI Manual

Stakeholders from the following groups need to be identified and involved

• PSI Holders• PSI Users and Re-users• NGOs• Libraries• Publishers• Sectoral bodies/communities (meteorological,

geo-information, etc.)

Session ASession A

b) PSI Repository

Requires Clarification of Several Dimensions• Content

• Format (portal, database, forum, etc.)

• Ideally not a passive repository but a platform for proactive knowledge creation (e.g. Wiki, forum, etc.)

• Link to existing EPSIplus Project (EU-specific, future funding issues)

• Language

• How can a general PSI portal serve specific communities?

Session ASession A

c) Future Academic Research

Considerations

1. Specific topics (see following slide)

2. Research funders: EC, foundations, etc.

3. Workshop/conferences

4. Online journal

5. Awards for theses and dissertations

6. The PSI manual

Session ASession A

c) Future Academic Research

Specific Topics

• Special attention should be paid to research on individual users and on social effects of PSI use/re-use

• Network effects and network externalities (both positive and negative) need new assessment methods

• Role of automated knowledge extraction and re-use

• Pilot projects to test different approaches

• Promoting involvement of young researchersSession ASession A

III. Break Out Session B – Industry Representatives and Users

Session BSession B

Session B – Overview

1. General Points on Future Directions and Measurement of PSI Policy Impacts

2. OECD PSI Principles3. PSI Definitions4. Repository for Mutual Support in

Measuring PSI5. Data Collection6. Mapping the Value Chain7. Potential further Studies

Session BSession B

1. General Points on Future Directions and Measurement of PSI Policy Impacts

• Session aims to generate ideas on next steps for research, measurement and analysis; also for follow-up to the OECD policy principles

• Need to produce facts and analysis for policy makers, businesses and users

• Need policy evaluation over time or across countries, using the same methodologies

• Possibly focus on top-5 PSI sectors across a number of countries – although these may differ by country

• Impact of PSI policy can depend on different political and institutional contexts – including at the sub-national level

• Broad support for looking at comparative experiences, e.g. telecoms liberalisation, environmental goods

Session BSession B

2. OECD PSI Principles

• Broad interest in OECD PSI principles• Theme is recommendation to “disseminate

widely at lowest cost possible”• Importance of preparing framework and

gathering information for any future review (once approved)

• Pay attention to the immediate obstacles which the application of the principles might pose for public finances

Session BSession B

3. PSI Definitions

• Importance of understandable and comparable definitions, e.g. product definitions, to go forward with the manual (on data collection and analysis)

• Important to make research more easily comparable and more focussed

• Potentially an approach that classifies types of PSI by function (end-use) is useful, rather than or in addition to, a product or activity approach

Session BSession B

4. RepositoryElements could include• Coordination of the different levels of surveys• Exchange of approaches to, and tools for, data collection

(questionnaires; model questionnaires; how to contact users)

• Exchange of survey results• Database of best practices for PSI policy (e.g. on low-cost

diffusion of PSI)• Efforts towards a classification of types of PSI (e.g. by

product or by function)• Function as reference for survey design, classifications,

and other activities

Needs structuring and long-term management; not discussed how it would be made operational

Session BSession B

5. Data Collection (1/2)• Accounting practices, different product/activity/functions

classifications make analysis of PSI market development difficult

• Some bodies can more easily collect data (e.g. national competition bodies compared with academics)

• Care must be taken in designing and conducting surveys

Surveys• OECD successfully develops model questionnaires (e.g. use of

ICT in businesses, use of ICT in households)• Ensure consistency and adherence to best practices in surveys• Survey design depends on understanding how people use PSI• What information to gather? E.g. obstacles to re-use and use• Grouping survey respondents: E.g. in analysing survey data,

OFT’s contractor grouped PSIHs by pricing regime

Session BSession B

Sampling methods – Survey points of contact with users and re-users

• Must take into account potential (currently unsatisfied) demand for PSI use and re-use

• Potentially make public announcement about PSI research in fora used by current and potential data requesters

• Publicly funded libraries serve as distributors of much government information in North America and Scandinavia – good opportunity to learn about users, particularly citizen users

• Data from applications based on Freedom of Information legislation may provide information on PSI use and (potentially unsatisfied) demand

5. Data Collection (2/2)

Session BSession B

6. Mapping the Value Chain• Value created via re-use depends very much on the product• Map PSI supply and demand, building on previous work• Analyse value chains for re-use: Where are costs incurred,

where is most value created• Model value chains in terms of activities/business processes• Attach costs to value chain components• Can reveal at which step of the value chain obstacles to

value creation lie

• How to value social and economic impacts: Obstacles to re-use and use can be ranked by the number of people affected or by the magnitude of the economic cost

Session BSession B

7. Further Studies

a) Transition costs of changing policy regimes (e.g. from/to UK trading fund model)

b) Substitution of PSI by non-government information: Pay attention to where and why this is happening and quality of created data

c) Examine PSI licensing costs across sectors/access regimes/countries

d) Network externalities of PSI-based user-created contente) Draw parallels to other areas and comparative

situations: e.g. impacts of telecommunications liberalisation

f) Include PSI policy in measuring government performance

Session BSession B

IV. Summary– Sessions A & B

Session BSession BSession ASession A

IV. Summary – Sessions A & B

• Importance of OECD PSI Principles and follow-up

• Manual to improve measurement: strong case• Repository: strong case but some issues to be

clarified• Academic research: strong case, new topics

and approaches proposed: Clarify PSI definitions, further data collection, mapping the value chain

Session BSession BSession ASession A