1

Deep learning in

practice: Use of

evidence to track

student progression,

enhance learning and

create professional

development

The overview and the

activities makes me feel

professional and the

students love it.- Scandinavian teacher

2

• Founded in 2001 and 1996

• 100 employees

• Software

• Content

• School development

• We work in:

– Norway (national systems and 90% of the schools)

– Denmark (started 2014)

– Sweden (started 2015)

– Singapore (started 2015)

• Starting in 2016:

– 2 Asian countries

– 2 European countries / North American states

About Conexus and

LearnLab

3

Deep learning society

• 2 years

• Deep learning data analytics

• Co-learning between teachers,

school leaders, municipalities,

politicians, researchers and

ed-tech companies

• Over 400 Schools in

Scandinavia

• Both open and paid content

and technology

4

Academic feedback from

• Professor Peter Mortimore

• Professor Knut Roald

• Professor Kjell B. Hjertø

• Professor Louise Stoll

• Professor Andy Hargreaves

• Professor Viviane Robinson

• Professor Michael Fullan

• Professor Lorna Earl

• Professor Jan Merok PaulsenYngve Lindvig, Louise Stoll, Michael Fullan, Andy Hargreaves, Jarl Inge Wærness,

Lorna Earl and Anders Fosnæs. Toronto, 2015.

5

Atlantic Rim Collaboratory

THOUGHT LEADERS

Vicky Colbert Andy Hargreaves Pak Tee Ng Jeannie Qakes Sir Ken Robinson Pasi Sahlberg

6

7

8

Using evidence and customized educational technology

systematically for releasing the potential in every learner:

Findings presented at ICSEI Jan 2016

1. Assemble and present data from multiple sources

leads to better professional learning?

2. In-depth processes with a broad evidence perspective

leads to better professional learning?

3. Use of digital reflection tools

leads to better professional learning?

9

Empowerment

• Scandinavia

• Canada

• New Zeeland

• Finland

• Singapore

• Scotland

We built on evidence form:

Relational Knowledge map

SelfEmotionsEmpathy

Critical thinking

Learn to learn

Hand-craft

Digital skills

Mathematics

System thinking

Education have to focus on all 3 levels…….

….to prepare for all 3 real life dimensions Society

Work

Family

ME

Deep Learning

Skills and competences

11

What is the

future in

learning

analytics?

Learning

analytics two

dimensions,

tree levels

Individual

analysing

Aggregated

analysing

• All relevant information in one system

• API’s

• Data delay

• The user is the analytical engine

• All relevant information in one system

• API’s

• Data delay

• The user is the analytical engine

• Information seen in relation

• Interpretation and process support

systems

• Live data

• Dynamic reporting

• Individual dashboards with learning

recourses

• Information seen in relation

• Interpretation and process support

systems

• Live data

• Dynamic reporting

• School development templates

• Individual profiling measuring depth

and progression

• User generated assessment

components

• Adaptive path to third party generated

content

• Aggregated profiling measuring depth

and progression

• User generated process components

• Adaptive path to school development

templets

• Standardised data exchange

• Predictive analysis of longterm student

outcome

1

2

3

12

School adm

inis

tration

syste

m

Learn

ing m

anagem

ent

syste

m

User-

genera

ted

conte

nt

Conte

nt

from

publis

hers

Self-a

ssessm

ent

Learn

ing s

tandard

s (

curr

iculu

m g

oals

)

Individual

Aggregated

Learning Analytics –

the ecosystem

13

14

15

16

17

50% - 10 years

18

19

”

so engage is about individual

data and learning content

20

Empowerment

Reinforcing and

modelling

what matters

21

22

23

24

25

26

27

Learning Progress

28

Empowerment

Reinforcing and

modelling

what matters

29

”

For the first time the management was

inquiring into our professional learning

activities, instead of just evaluating our

results.

- European group of teachers

30

PURPOSE

based on a sense of the issues

SELECTING DATA

Broad range seeing the problem from

many perspectives

INTER-ROGATINGDATA FOR

DEEP LEARNING

How data is presented, good

questions for insights and to develop

hypotheses

DESIGNINGPROCESSES AND TAKING

ACTION

Testing theories of action, further

professional learning

CHECKING

Knowing there’s enough of a

difference to students’ learning and

improvements in practice

Stoll 2016

Data-enriched

collaborative enquiry and

knowledge-building

cycle

31

PROFESSIONAL QUALITY

Student-Teacher Relations

Formative assessment

Student Involvement

Teacher Feedback

Order in Class

Teacher Encouragement of Creativity

Career counselling

Scientifically

tested

surveys

INPUT

Parental Support

Parental Expectations

Parental socioeconomic background

Input grades

OUTPUT

Intrinsic Motivation

Self-efficacy

Satisfaction

Tests and/or grades

School Attainment

DEEP LEARNING

Self-Assessment

Digital skills

Decision-making skills

Critical Thinking and Problem Solving

Creative Thinking

Communication

Collaboration

Learning To Learn

Sustainable Development

Democracy and Citizenship

Life Skills and Health

STUDENT BEHAVIOUR

Mastery-approach goal items -MAP

Mastery-avoidance goal items – MAV

Performance-approach goal items - PAP

Performance-avoidance goal items - PAV

Collectivism

Individualism

Win-lose

Commitment

LEARNING ENVIRONMENT

Social well-being

Bullying

Trust

Safety

CONFLICTS

Conflicts – (Emotional Relationship Conflict)

Conflicts – (Emotional Task Conflict)

Conflicts – CT (Cognitive Task Conflict)

Conflicts - (Cognitive Relationship Conflict)

EMOTIONAL INTELLIGENCE

EQ – SEA (Self Emotion Appraisal)

EQ - OEA (Other Emotion Appraisal)

EQ – UOE (Use of Emotion)

EQ – ROE (Regulation of Emotion)

32

Research

data

Register

data

Survey

data

Process

data

Students • Student learning

• Motivation

• Self-efficacy

• Test

• Grades

• Attendance

• Discipline records

• Learning environment

• Teaching appraisal

• Self-assessment

• Online learning

resources

• Adaptive learning

tools

• Student work

Professionals • Professional learning/

capital/ communities

• Educational

leadership

• Social demographic

• Human resource

• School facts

• Learning/ work

environment

• Leadership

evaluations

• Self-assessment

Descriptions of goals,

processes, quality and

actions

Parents • Home-school

collaboration

• Learning environment

• Highest level of

completed education

• Social demographics

• Home-school

collaboration

• Learning environment

Summary of

descriptions of meetings

between school and

parents

We would argue that to stimulate deep learning conversations it is insufficient just to collect

test and attendance data. For deeper learning, we need different types of data.

The model above provides an overview.

33

34

Step-by-step: color analysis

1. Testing of scales

2. Factor-analysis

3. Correlation between data sets

4. Testing of significant

differences

5. Path-analysis

6. Color-settingFinal estimation of fixed effectsFixed Effect  Coefficient t -ratio d.f.  p -value

For INTRCPT1, β 0

    INTRCPT2, γ 00 3,78 169,44 115 <0.001

    TRINN_ME, γ 01 0,03 0,65 115 0,52

     K_MILJ4, γ 02 0,13 2,53 115 0,01

    LAERER17, γ 03 0,33 4,21 115 <0.001

For KJONN slope, β 1

    INTRCPT2, γ 10 0,09 2,58 1757 0,01

For KARAKT6 slope, β 2

    INTRCPT2, γ 20 0,08 4,33 1757 <0.001

For VEIL3_11 slope, β 3

    INTRCPT2, γ 30 0,19 11,97 1757 <0.001

For HJEM3_09 slope, β 4

    INTRCPT2, γ 40 0,14 8,59 1757 <0.001

35

We have conducted a multilevel analysis (HLM) of 25 variables, investigating 5 dependent variables

- Intrinsic motivation (4 items),

- Grades (one item),

- Satisfaction (four items),

- Self-efficacy (four items),

- Well-being (two items),

All dependent and independent variables, including the

two-item measures, had alphas between above .70 to

above .90. The factor structure is excellent.

36

37

38

39

40

41

42

43

0

44

45

46

47

48

49

50

51

Assessment for learning

Level: Criteria to achieve level:

Level 4

All students know what they have achieved (achieved

competence) and what they should strive for (next skill

level). Students employ high-quality self-assessment

frequently and use feedback from teachers and peers in

ways that promote their learning.

Level 3

Teachers prepare students for what they will learn and

teachers give students feedback on what they have

learned. Students are more engaged in self-assessment.

Level 2

The school has a clear focus on assessment for learning.

The teacher feedback to students is often too general and

the quality of self-assessment among students is highly

variable.

Level 1Teachers' practice of assessment for learning is

unsystematic and they lack a clear collective

understanding of the concept.

52

Assessment for learningLevel: Criteria to achieve level:

Level 4

“WE THRIVE AT ASSESSMENT FOR LEARNING". All students know what they have achieved

(achieved competence) and what they should strive for (next skill level). Students employ high-quality

self-assessment frequently and use feedback from teachers and peers in ways that promote their

learning. Teachers make use of assessment information about their students to adapt the teaching to

meet student needs. A high level of coherence in the assessment work from grades 1-10 is now

established. Teachers take collectively responsibility for the principles and goals that they have

developed. Parents receives regular information about their child's learning progression. Parents

provide supportive feedback to their own children. We continually inquire into our practices and

student learning results in light of evidence-based knowledge of excellent assessment practices.

Level 3

We have established a professional language and began building a culture for assessment for learning.

Students are familiar with learning goals and criteria, and get feedback on what they have learned and

how they can improve. Students are engaged in assessing their learning. Teachers work systematically

and share good practice. The good half-year assessment / final assessment / student interview are

defined and implemented. Most parents understand the purpose of assessment for learning. We use

evidence-based knowledge of assessment practices.

Level 2

The school has defined the good lesson. The plan for the project is known throughout the staff.

Sufficient time is set aside to work with assessment for learning. A challenge is that teacher feedback to

students is often too general and the quality of self-assessment among students is highly variable.

Some teachers experiment with new principles and methods. Parents are aware of the schools work

with improving assessment practices. We collect evidence-based knowledge of assessment

practices.

Level 1Teachers' practice of assessment for learning is unsystematic and teachers need collective

understandings of concepts and practices.

53

Interpretation is paramount

Data provide tools for thinking

Interpretation is thinking, formulating possibilities, developing convincing arguments, locating logical flaws, establishing a feasible and defensible notion of what data mean

Interpretation requires time, thoughtfulness, reservation of judgements, open challenge and support for ideas

Earl and Katz (2006)

Data to information, to knowledge to wisdom

54

Process Design

An approach where educators could themselves select and combine

different types of data.

This would enable educators to select relevant evidence and connect this

to problems of practice and questions for deep reflection.

When participants first began using the process designer, they focused

specifically on designing data-informed meeting requests. We quickly

realized that designing these requests turned out to be difficult, and saw

the need to create prototypes of these requests. Together with

researchers we devised prototypes for common problems of practice.

55

Process Design

57

58

59

60

Drammen 2,00

Bergen 1,43

Kristiansand 1,40

Fredrikstad 1,11

Trømsø 1,09

Bærum 1,00

Sandnes 0,84

Oslo 0,80

Stavanger 0,50

Trondhjem 0,47

Results?

61

Using evidence and customized educational technology

systematically for releasing the potential in every learner:

Findings presented at ICSEI Jan 2016

1. Assemble and present data from multiple sources

leads to better professional learning?

2. In-depth processes with a broad evidence perspective

leads to better professional learning?

3. Use of digital reflection tools

leads to better professional learning?

62

Contact info:

Yngve Lindvig

yngve@learnlab.net

yngve@conexus.net

+47 90785069

63

Leading digital infrastructure

for education• Integration (APIs)

• Learning content made relevant

• Surveys tools

• Advanced models for learning

analytics

• Tools for professional development

• Prototypes for processes

“ This is probably… No, this is for sure

the most sophisticated and elegant

educational solution I’ve ever seen.

And the thinking behind it is very

much in line with ours.”

North-American

Deputy Education Minister

Conexus products are

an excellent way to

broaden the focus in

education to include

well-being, 21 century

skills and holistic

learning.

- Asian group of

Educational Directors