IC Literacy Test - Information and Communication Literacy measurement instrument for a general

secondary school graduates

IC Literacy Test - Information and communication literacy measuring tool

IC Literacy Test – is a tool which has been specially developed by the National Training Foundation (NTF) with support from the Ministry of Finances and in collaboration with the World Bank for the READ project. The test provides an opportunity to assess the level of proficiency in information and communication technologies, which today are widely used in order to access new knowledge, perform research activity and to communicate in the digital world.

What is IC Literacy?

IC Literacy – is an ability to use digital technologies to access information, then to define, manage, integrate, evaluate and communicate said information in a creative manner, while following ethical aspects and legal norms in order to live, work and be successful in today’s modern, knowledge-based Information Age.

Components of the IC Literacy

Components of the IC Literacy

Define (information): the ability to discover a specific topic based on a broad range of information, and then develop a concise research statement in order to facilitate an electronic search

Access (information): the ability to search for and find information from various sources

Manage (information): the ability to purposefully organize information

Integrate (information): the ability to identify main ideas and combine them with information from other sources

Evaluate (information): he ability to assess quality, relevancy and usefulness of information and its sources

Create (information): he ability to create information or to manipulate existing information for a specific purpose

Communicate (information): he ability to adapt information for a specific audience

Goals of the assessment

The goal of the ICL Test is to ensure a realistic and complex assessment of IC competence by using practical, scenario-based tasks.

Main tasks: Assess the level of the IC Literacy of the secondary school graduates Identify the factors, which affect IC Literacy Provide recommendations based on the test results Check whether the recommendations increase the level of the IC Literacy

Target audience: Students transitioning from secondary school to high school (nine-grade school students).

Features of the IC Literacy Test

Interactivity, automated process of the testing and evaluation of its results

Focus on a test-taker’s ability to successfully work with information and solve practical tasks using information and communication technologies, without assessing technical skills

The ICL Test has been developed in the Evidence Centered Design framework

Automatic processing of test results, based on the use of Baeysian

network

Compliance with legal and ethical norms while using digital technologies and communication tools is a necessary part of the ICL Test

Test Structure

Test includes 16 tasks based on the real life situations in both academic and personal contexts

While doing the tasks, the test taker needs to:• perform a web-search• perform different actions with the data and communicate• select and analyze information• create and select materials to present for different audiences • make a determination on how legal and ethical it is to use the information in a certain way.

Test Structure

Task complexity Number of tasks Task duration

simple(measures level of 1

proficiency)13 3 - 4

moderate (measures levels of 2

proficiencies)2 10 -15

complex (measures levels of 4

proficiencies)1 20 - 30

16 tasks of different complexity and time duration16 tasks of different complexity and time duration

Total time to complete the test – 2 hoursTotal time to complete the test – 2 hours

Wide range of digital technologies: range of digital technologies, including e-mail, text editors, presentation tools, electronic tables, graphs, databases, multimedia services, social networks and other Internet services.

Wide range of digital technologies: range of digital technologies, including e-mail, text editors, presentation tools, electronic tables, graphs, databases, multimedia services, social networks and other Internet services.

ICL Test – is a test which is based on the Evidence Centered Design model

The core of framework of the ICL Test is a method of collecting evidences called Evidence Centered Design

for each of 7 IC proficiencies a certain number of evidences and indicators are used to measure its level

data received allows to measure what can’t be measured directly - levels of each of 7 IC proficiencies separately and the total level of the IC literacy

Task Example «Discovery of America»

Task ModelTargeted proficiency: CreateDuration: 4 minutesContext: academicContent: Social StudiesTask: to choose the most suitable title for the article from three suggested titles, select a relevant illustration, identify geographical names that require hyperlinks to the relevant encyclopedia articles, then create the links.

Task Example «Email sorting»

Task ModelTargeted proficiency: ManageDuration: 4 minutesContext: personalContent: practical affairesTask: save attached files in a way that it won’t get lost and will be easily accessible later.

13

14

15

Standard setting for the IC iteracy

Test results

5 levels of IC Literacy •advanced•above basic•basic•below basic•developing

5 levels of IC Literacy •advanced•above basic•basic•below basic•developing

Individual results include: General assessment of IC Competence Individual recommendations on skills that require further development

Individual results include: General assessment of IC Competence Individual recommendations on skills that require further development

Opportunity for comparative study(example school)

Data from the research analyses of the IC Literacy of the 9 th grade students in Moscow, 2013

Opportunity for comparative study (Regions of Russian Federation)

Opportunity to use sociological survey

1. Basic characteristics of students and their families: gender, language, parents education, socio-economical status of a family, educational resources at home, parents usage of ICT tools.

2. Extra curricular computer accessibility and its practical usage: availability of computers at home, purpose of computer usage, attitude to a computer, friends involvement in ICT tools usage, preferred leisure time.

3. School’s influence on the IC Literacy: availability of computers at school, computer usage for different classes, attitude to computer usage for different classes and its efficiency evaluated by students, frequency of work in groups.

Longitudinal Study

Data from the research analyses of the IC Literacy of the 9 th grade students in Republic of Tatarstan, 2010-2013.

IC Literacy measurement of the secondary school graduates the Republic of Armenia Operational testing

The two-step stratified cluster sample of the test-takers

1. Stratum – town/villageselection of urban schools separately and separately for rural, in proportion to the total size of the sample and their representation in the general population;number of ninth-graders in the school is a weighting factor and it increases the likelihood of getting large schools;in case of unavailability / failure of some schools, school selection is performed to replace (replace up to 2);2. Cluster – the group of students (class)using a random number generator a class within the school is selected in which all students are tested;replacement of the class within the school is not available.

The sample size depends on the capabilities of administrators and the desired accuracy of the results: ~ 2000 students. Random selection is carried out using the SAS.

IC Literacy measurement of the secondary school graduates the Republic of Armenia Results of the Operational testing

1834 students from the 94 schools of 11 regions of the Republic of Armenia participated in the testing

Согласно оценкам учащихся:

IC Literacy measurement of the secondary school graduates of the Republic of Armenia

According to the survey 96% of students who participated in the testing enjoyed taking the test. 90% mentioned, that the test score reflected correctly their computer and Internet usage skills level, а 81% said that the interface of the test is user-friendly

Psychometric studies Test validity and reliability of the results

Validity study (convergent, discriminant, concurrent)

• correlation analysis • factor analysis• analysis of the internal consistency

Task model analysis DIF analysis (different functionality of the tasks) Bayesian network analysis (model criticism)

Test reliability was measured with Chronbach’s Alpha and corrected split-half reliability coefficients, which showed results ranging 0.77–0.83

The ICL Test Advantages

1. Assessment of cognitive rather than technological skills

2. Analysis complexity

3. Real-life connection

4. User-friendliness affordability

5. Automatic data processing

6. High validity and reliability of assessment

7. Practical character of the test results and possibility for their

wide use

8. Successful approbation

9. International experts approval

10. Responsible approach to the task development

www.ictlit.com