Embed Size (px)
Citation preview
www.uis.unesco.org
Innovation data collection:Advice from the Oslo Manual
South East Asian Regional Workshop on Science, Technology and Innovation StatisticsHanoi, Viet Nam
5-8 December 2011Luciana Marins
www.uis.unesco.org
Ch 8 OM - Innovation Survey Procedures
Guidelines - collection and analysis of innovation data;
Comparable results over time and across countries;
Particular circumstances may require other methodology comparability.
www.uis.unesco.org
The survey approach
The “subject” approach:
• Innovative behaviour and activities of the firm as a whole;
The “object” approach:
• Specific innovations (“significant innovation” of some kind, firm’s main innovation).
www.uis.unesco.org
Populations (1)
The target population:The target population:
• Innovation activities in the business enterprise sector (goods-producing and services industries);
• Minimum: all statistical units with at least ten employees:
• Small: 10-49;
• Medium: 50-249;
• Large: 250 and above.
www.uis.unesco.org
Populations (2)
The frame population:The frame population:
• Units from which a survey sample or census is drawn;
• Based on the last year of the observation period for surveys;
• Ideal frame = up-to-date official business register NSOs;
• If the register forms the basis for several surveys (innovation, R&D, general business), the information can be restricted to innovation.
www.uis.unesco.org
Survey methods (1)
Mandatory Mandatory surveys increase response rates;
Census or sample surveys?Census or sample surveys?
• Sample surveys - representative of target population (industry, size, region) stratified sample;
• Census - costly but unavoidable in some cases.
www.uis.unesco.org
Survey methods (2)
Domains (sub-populations):Domains (sub-populations):
• Subsets of the sampling strata;
• Potential sub-populations: industry groupings, size classes, regions, units that engage in R&D and innovation-active;
• Guidelines:» Same statistical units and classifications;
» Consistence of the methods for results calculation;
» Documentation of deviations in data treatment or differences in the quality of the results from the domains.
www.uis.unesco.org
Survey methods (3)
Sampling techniques:Sampling techniques:
• Stratified sample surveys: size and principal activity;
• Sampling fractions should not be the same for all strata;
Cross-sections: Cross-sections: standard approach - new random sample for each innovation survey;
Panel data:Panel data: alternative/supplementary approach.
www.uis.unesco.org
Survey methods (4)
Suitable respondents:Suitable respondents:
• Methods - ex: postal surveys, personal interviews;
• Unit’s most suitable respondent - very specialised questions that can be answered by only a few people;
• Try to identify respondents by name before data Try to identify respondents by name before data collection starts.collection starts.
www.uis.unesco.org
Survey methods (5)
The questionnaire:The questionnaire:• Pre-test;
• Simple and short;
• Order of the questions;
• Questions on qualitative indicators - binary or ordinal scale;
• International innovation surveys - attention to translation and design;
• Short-form questionnaires - units with little/no innovation activity previously reported.
www.uis.unesco.org
Survey methods (6)
Combination of Innovation and R&D surveys:Combination of Innovation and R&D surveys: Reduction in the overall response burden;
Scope for analysing the relations between R&D and innovation activities;
Increase in the frequency of innovation surveys;
Country experiences - it is possible to obtain reliable results for R&D expenditures;
Longer questionnaire;
Units not familiar with the concepts of R&D and innovation may confuse them;
Different frames for the two surveys.
www.uis.unesco.org
Survey methods (7)
Guidelines for conducting combined surveys:Guidelines for conducting combined surveys:
• Questionnaire - two distinct sections;
• Smaller individual sections;
• Comparison of results from combined and stand-alone surveys should be done with care - surveying methods should be reported;
• Samples extraction from a common business register.
www.uis.unesco.org
Estimation of results (1)
Weighting methods:Weighting methods:
• Weighting by the inverse of the sampling fractions of the sampling units, corrected by the unit non-response;
• If a stratified sampling technique with different sampling fractions is used, weights should be calculated individually for each;
• Based on the number of enterprises in a stratum;
• International and other comparisons: same weighting method.
www.uis.unesco.org
Estimation of results (2)
Non-response:Non-response:
• Unit non-response: reporting unit does not reply at all;
• Item non-response: response rate to a specific question - percentage of blank or missing answers;
» Disregarding missing values + applying simple weighting procedures based on the responses received assumes that respondents and non-respondents are distributed in the same way biased resultsbiased results;
» Possibility: imputation methodsimputation methods.
www.uis.unesco.org
Presentation of results
Variance for the results: (average) values for innovation indicators and their coefficients of variation and/or confidence intervals;
Results presentation: metadata (including data collection procedure), sampling methods, procedures for dealing with non-response and quality indicators.
www.uis.unesco.org
Frequency of data collection
Every two years;
If not economically feasible frequency of three or four years;
Specify an observation period;
• The length of the observation period for innovation surveys should not exceed three years nor be less than one year.
www.uis.unesco.org
Annex A - 5. Methodological issues for developing country contexts (1)
Information system specificities:
• Relative weakness of statistical systems:
» Absence of linkages between surveys and data sets;
» Lack of official business registers;
• Involvement of NSOs;
• When lacking, basic variables about firms’ performance can be included in the innovation survey.
www.uis.unesco.org
Annex A - 5. Methodological issues for developing country contexts (2)
General methodological considerations:
• Survey application:» In-person;
» Trained personnel;
• Questionnaire design: » Sections can be separated to allow different persons in the
firm to reply them;
» Guidance/definitions;
» Language and translation of technical terms.
www.uis.unesco.org
Annex A - 5. Methodological issues for developing country contexts (3)
General methodological considerations:
• Frequency:» Every three to four years (e.g., timed to CIS rounds);
» Update a minimum set of variables every year;
• Purpose of surveys;
• Clear questions;
• Adequate legislative base;
• The results should be published and distributed The results should be published and distributed widely.widely.
www.uis.unesco.org
Example - product innovation/degree of novelty
(Product innovations) Yes No
New or significantly improved goods? ( ) ( )
New or significantly improved services? ( ) ( )
(Degree of novelty) Yes No
New to your market? ( ) ( )
Only new to your firm? ( ) ( )
During the three years 200X to 200Y, did yourenterprise introduce:
Were any of your product innovations during the threeyears 200X to 200Y:
www.uis.unesco.org
Example - innovation activities and expenditures for product and process innovations
(Innovation activities and expenditures forprocess and product innovations)
Yes NoAmount of
expenditureIn-house R&D ( ) ( ) $
External R&D ( ) ( ) $
Acquisition of machinery, equipment and software ( ) ( ) $
Acquisition of external knowledge ( ) ( ) $
Training for innovative activities ( ) ( ) $
Market introduction of innovations ( ) ( ) $
Other (preparations) ( ) ( ) $
During the three years 200X to 200Y, did your enterprise engage in thefollowing innovation activities:
www.uis.unesco.org
Example - organisational innovation
(Organisational innovations) Yes NoNew business practices for organisingprocedures
( ) ( )
New methods of organising workresponsibilities and decision making
( ) ( )
New methods of organising external relationswith other firms or public institutions
( ) ( )
During the three years 200X to 200Y, did your enterpriseintroduce:
www.uis.unesco.org
Example - co-operation
Within country
Abroad
A. Other enterprises within your enterprise group ( ) ( )
B. Suppliers of equipment, materials, components, or software ( ) ( )
C. Clients or customers ( ) ( )
D. Competitors or other enterprises in your sector ( ) ( )
E. Consultants, commercial labs, or private R&D institutes ( ) ( )
F. Universities or other higher education institutions ( ) ( )
G. Government or public research institutes ( ) ( )
LocationType of innovation co-operation partner
During the three years 200X to 200Y, did your enterprise co-operate on anyof your innovation activities with other enterprises or institutions?( ) Yes ( ) No
www.uis.unesco.org
Example - hampering factors
High Medium LowNot
relevantExcessive perceived economic risks ( ) ( ) ( ) ( )
Innovation costs too high ( ) ( ) ( ) ( )
Lack of appropriate sources of finance ( ) ( ) ( ) ( )
Organisational rigidities within the enterprise ( ) ( ) ( ) ( )
Lack of qualified personnel ( ) ( ) ( ) ( )
Lack of information on technology ( ) ( ) ( ) ( )
Lack of information on markets ( ) ( ) ( ) ( )
Insufficient flexibility of regulations or standards ( ) ( ) ( ) ( )
Lack of customer responsiveness to new goods or services ( ) ( ) ( ) ( )
Economic factors
Internal factors
Other factors
Degree of importanceHampering Factors
If your enterprise experienced any hampering factors during the period 200X-200Y, please gradethe importance of the relevant factors.
