Surveying eValues: Experiences and Challenges Measuring Information Society in the Community Survey on ICT Usage and e-Commerce in Enterprises Fernando

Surveying eValues: Surveying eValues:

Experiences and Experiences and

ChallengesChallenges

Measuring Information Society in the Measuring Information Society in the

Community Survey on ICT Usage and Community Survey on ICT Usage and

e-Commerce in Enterprisese-Commerce in Enterprises

Fernando Reis

EUROPEAN COMMISSIONEUROSTATDirectorate F: Social Statistics and Information SocietyUnit F-6: Information Society and Tourism Statistics

19th Bled

eConference

"eValues"

June 5 - 7, 2006

Bled, Slovenia

www.europa.eu.int/comm/eurostat/

EUROPEAN COMMISSIONEUROSTAT

Directorate F: Social Statistics and Information SocietyUnit F-6: Information Society and Tourism Statistics

Information Society Statistics in the Internet:EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics

Information Society statistics at EurostatInformation Society statistics at Eurostat

The role of EurostatProvide the European Commission with statistics

The role of official statisticsLarge scale representative and harmonised statistics

Origins of IS statistics in various ICT surveys in the Member States

Big boost: Lisbon process and eEurope Action Plans, which included a benchmarking exercisefor coherent and comparable statistics at European level

Two surveys on ICT usage:- Enterprise Survey (since 2001)- Household / Individuals Survey (since 2002)

Legal basis:Framework Regulation April 2004, implementing regulation (for the 2006 survey) published on 13/07/2005




The Community Survey on ICT Usage The Community Survey on ICT Usage and e-Commerce in Enterprisesand e-Commerce in Enterprises

Network of national official surveys on ICT uptake 30 countries (EU MS + IS + NO + RO + BG + TR) Net sample of around 150 000 enterprises (5 000 per

country) 66 variables collected:

– Basic ICT uptake (e.g. computers, LAN)– Internet use– e-Commerce– e-Business (e.g. integration of business processes)

2 background variables used to breakdown results:– Size (3 categories)– Economic activity (14 categories)




Contents of the surveyContents of the survey

Type of questions:– Simple format: most questions/items only require a

“Yes” or “No” answer– e-Commerce questions asked in terms of percentages– But sometimes difficult topics … (e.g. advanced uses of

e-business solutions sometimes difficult to understand by some respondents)

Scope / Target population– Sections D to K of NACE + Divisions 92 and 93– Financial sector covered by a specific model

questionnaire– Enterprises with 10 or more persons employed

(optionally enterprises with less than 10)




Results of collections from 2003 to 2005Results of collections from 2003 to 2005

Computer systems and their integration - (E.U.)

96%95%94%

37%35%43%

28% 31% 34%

11% 13% 15%

0%

20%

40%

60%

80%

100%

2003 2004 2005

Computer Usage

Computer systems tomanage orders

Internal integration

External Integration




Coordination of ICT statistics in the EUCoordination of ICT statistics in the EU

Every year: model questionnaire developed by Eurostat in coordination with the national statistical institutes (via Task Forces and Working Groups) and other services of the European Commission (DG INFSO and DG ENTR)

Data collection by NSI’s and transmitted to Eurostat

Publications by the NSI’s as well as by Eurostat




Survey and reference periodSurvey and reference period

Reference period– previous calendar year for the monetary variables (turnover

and purchases) and e-government;– January of survey year for other variables.

Survey period– recommendation is first quarter of the survey year to keep it

as close as possible to the reference period (reduce recall bias) and to allow the release of the results as soon as possible

– But for practical reasons sometimes different: Internal planning of the NSI Data collection linked to other surveys




Survey period Survey period (2005 survey)(2005 survey)

Czech Republic

Belgium

Denmark

Germany

Estonia

Spain

Italy

Cyprus

Greece

Ireland

Latvia

Lithuania

Luxembourg

Hungary

Malta

Portugal

Slovenia

Netherlands

Austria

Slovak Republic

Finland

Poland

Sweden

Norway

J anuary February March April May J une J uly

J anuary February March April May J une J uly




Sampling: frameSampling: frame = the frame from which the sample will be drawn

Should be as close as possible to the target population as both overcoverage and undercoverage can induce bias and affect the reliability and comparability of the survey results. – E.g. overcoverage due to inclusion of enterprises with less than

10 persons employed in the sampling frame: as ICT usage in these enterprises is lower than the average, the population total will show a downwards bias.

Due to: Low quality of the register; Instability of the population (high percentage of enterprises

changing size class and/or economic activity). Correction:

1. Collection of the background variables in the survey;2. Reclassification of the respondents in the new/correct categories

before grossing-up. Recommendation:

Use the same frame as the structural business statistics.




Sampling: designSampling: design Highly harmonised between countries:

Stratified random sampling in most countries (smaller countries – e.g. LU, CY, MT - may use census)

Stratification based on the background variables (persons employed and economic activity) and for the majority of the countries also the region.

Sometimes the stratification is more detailed

But different approaches are allowed … assuming that different ways to draw a representative

sample should nevertheless lead to comparable and harmonised data

… NSI’s are best placed to decide on sampling design (competence, ‘local’ knowledge, …)




Sampling: sample sizeSampling: sample size In practice:

Output (indicators): proportions or percentages Requires smaller sample sizes then for estimating totals;

Very dependent on population size: small countries with small populations require higher sampling rates;

EU25 countries: average net sample size around 5000 enterprises;

Gives very good results at overall level;

Sometimes problematic for more detailed information; E.g.: the distribution of the enterprises by class of the percentage

of turnover from e-commerce, cannot be accurately estimated for several countries because not enough respondents;

Decision also depends on national data needs (e.g. specific need for data at regional level)

E.g.: DE, ES, IT which produce regional breakdown have average net sample size of 14000 enterprises.




Survey type (I)Survey type (I) Data collection method:

The vast majority of EU25 Member States use postal surveys (self-administered)

Many countries when launching the survey in the firsts years used face-to-face interviews

Other collection methods are combined to increase response rate (contact by telephone, electronic questionnaire, etc.)

Face-to-face or telephone interviews are usually preferred to postal surveys (interaction between the respondent and the interviewer can be important for the understanding of the questions and the answers)

Postal surveys: lower survey costs (and can give the respondent the time to reflect on his/her responses) and e-commerce information require enterprises to consult their accounts or other registers

But … in general budgetary constraints will be the determining factor …




Survey type (II)Survey type (II) Survey vehicle:

Independent, stand-alone survey Embedded survey

Embedded survey was sometimes used in the first years (e.g. structural business statistics survey - SBS);

Currently, all EU25 Member-States use stand-alone surveys to collect ICT usage and e-commerce variables

+ SBS surveys are usually slow to release results and timeliness is very important for ICT statistics

- requires the coordination of the economic variables (employment, turnover, purchases) with the SBS with which ICT statistics need to be coherent

- Crossing of ICT use data with economic performance data (from SBS) is more difficult because samples don’t overlap




Response and non-responseResponse and non-response

Countries’ experience show that the final response rate is on average 74%

When aiming at a net sample (= number of elements in the final database) of e.g. 3000 individuals, it may be necessary to select 4000 individuals.

Risk of biased results: non-users of ICT may be more inclined not to return the questionnaire filled in

Importance of imputation (e.g. post-stratification methods to improve the representativeness of the estimates)

Item non-response is low (excepting for a few complex questions)

Some methods: see the Methodological Manual




Questionnaire designQuestionnaire design

How to measure such a complex subject as e-business– Needs to be general – applicable to all

economic activities and all enterprise sizes;– Needs to be specific – e.g. references to

examples;– Needs to be explicit – no references to

abstract concepts, but to events or situations easily identifiable by the respondents.




Three main problems to be tackled in the Three main problems to be tackled in the next five yearsnext five years

1. High sensitivity of the results to questionnaire wording

2. Measurement of the impact of ICT in enterprises

3. Development of a conceptual framework for statistical measurement of ICT usage and the Information Society




Contact and Community Survey in the Contact and Community Survey in the InternetInternet

[email protected] - epp.eurostat.cec.eu.int




ICT usage data for the EUICT usage data for the EU

Statistics are available free of charge on Eurostat’s website:

http://europa.eu.int/comm/eurostat

Click « Data »

Click « Science and technology »

Via the NewCronos database, tables are defined by the user on extraction

Metadata including a first release of the Methodological Manual:

http://europa.eu.int/estatref/info/sdds/en/infosoc/infosoc_base.htm

Documents

Surveying eValues: Experiences and Challenges Measuring Information Society in the Community Survey on ICT Usage and e-Commerce in Enterprises Fernando