Teacher attitudes and practices in international studies and their relationship to PISA performance: Nordic countries in an

international context

Ragnar F. Ólafsson and Júlíus K. Björnsson

Educational Testing Institute, Iceland.

Northern Lights, Oslo, May 19th and 20th

• „teaching is cultural; most teachers within a culture use similar methods“ (Stigler and Hiebert, 2004, p. 16)

• „teachers are proxies for an educational system‘s values (...) mathematics teachers in one country behave in ways that identify them more closely with teachers in their own country than teachers elsewhere (Andrews, 2010, p. 21)

Aims:

• Explore cultural differences in teacher practices and attitudes across Europe

• Examine whether such differences are associated with progress in PISA

Three datasets: Teacher questionnaires

TALIS 2008: Questions/items related to Collective Teacher Efficacy in mostly European 23 countries, co-authors Auður Pálsdóttir and Allyson Macdonald, University of Iceland.

TIMSS and PIRLS 2011: Questions/items about the background, teaching practices of teachers in reading, mathematics and science in 24 countries. In Northerns Lights on TIMSS and PIRLS, co-author Júlíus K. Björnsson.

PIRLS 2006: Questions on teaching practices and teacher background of reading teachers in 27 countries, mostly European.

Analysis

• Multidimensional Scaling

• Country means for each question/item were computed with IDB Analyzer.

• Country means were rescaled 0-1: Country mean divided by top score, multiplied by 100

Analysis

•MDS and HCA: Dimensions and country groups (clusters) identified and interpreted.

•Dimension scores correlated with country progress in PISA reading literacy, mathematics and science.

TALIS 2008

TALIS 2008

Country clusters

• Eastern European

• Nordic

• Mediterranean

• Mid-European

• and outliers

Dimension 1

Culture of observation, feedback and improvement This dimension has the highest correlations with items overall. It correlates most strongly with items assessing whether the principal or other management team members observe teaching in classes and give suggestions on how to improve teaching. It also correlates with items assessing the teachers’ confidence in the principal’s methods to determine whether the teacher is doing well or badly… (cont.)

Dimension 1 (cont.)

Culture of observation, feedback and improvement

…It also correlates with measures assessing frequency and effect of feedback received, e.g. on teachers’ understanding of their main subject field, instructional practices, teaching in multicultural setting, and whether feedback led to the development of a training plan to improve teaching, and whether innovative teaching is rewarded.

• Eastern: Receive more professional development, yet express more need for it. They believe that reading professional journals has greater effect on their teaching.

• Nordics and Mid-Europe reject idea that “teaching facts is necessary” or that “instruction should be about problems with clear, correct answers that students can grasp quickly”

• Teachers in the West believe more often that professional development plan is put in place for bureaucratic purposes only while teachers in the East reject more often the notion that the feedback has little effect on their teaching practices.

TALIS 2008

• Culture of observation, feedback and improvement (COFI)

• Correlates with progress in PISA reading literacy; rho=-0.62

• Interestingly, COFI also correlates with country participation rates in TALIS (r=-0.67)

• TIMSS and PIRLS 2011: Teacher questionnaire regarding background, teaching practices and attitudes of reading, mathematics and science teachers of representative samples of students in participating countries.

• Aims:

• Are country groups from TALIS 2008 replicated in TIMSS and PIRLS 2011?

• Are similar dimensions differentiating between countries?

• Do the dimensions predict progress in PISA?

Country clusters

• Eastern European

• Mediterranean

• Anglo

• Nordic

• Germanic

• West: Better access to computers• West: Less communication with other teachers

about teaching practices, own teaching experience, less cooperation between teachers on new teaching projects, they more seldom observe others teaching

• West: Teachers less often summarise at the end of lessons what has been covered in the lesson, and more seldom encouraged pupils to improve their own performance.

• West: Meet parents less often, send them progress reports less often.

• East: Use textbooks more, children‘s books, magazines for children.

• East: Pupils read aloud more, are systematically taught new vocabulary, learn scanning and skimming text.

• West: Pupils read books of „own choice“.

• East: Pupils asked to link reading material to own experience, link it with other material read earlier, explain and argue their interpretation of the text, identify main ideas, draw conclusions from text, describe author‘s style, author‘s viewpoint, author‘s intentions etc.

• East: More assigment of home reading and follow up whether homework is done.

• East: If pupils gets behind in reading, they get more time with teacher. In West, they are referred more often to specialists.

• East: Overall, more examinations and assessments.

• East: Mathematics teachers ask pupils more often to provide justifications for their answers, to memorize rules, work together (the whole class) more often with direct guidance from teacher.

• East: More time spent on professional development.

Correlation between PISA progress and dimensions 1 and 2

Dimension 1 Dimension 2

Progress_Reading 0.606*p=0.013n=16

-0.075p=0.782n=16

Progress_Mathematics 0.573*p=0.026n=15

-0.407p=0.132n=15

Progress_Science 0.410p=0.073n=20

-0.530*p=0.016n=20

• PIRLS 2006: Teacher questionnaire pertaining to teacher practices, attitudes and their background.

• Aims: Replicate country groups and dimensions – and link with PISA progress

PIRLS 2006: Teacher practices and attitudes in European countries

Country clusters

• Eastern European

• Nordic/Anglo

• Nordic

• Mid-European, incl. BeNeLux

• West: More books available in classroom reading corner and more time spent in reading corner.

• East: Assign homework in reading more often and expect students to spend more time on reading homework.

• East: Students with reading difficulties are more often assigned extra homework to improve reading, and sometimes other students called in to assist them.

• West: Greater access to specialised staff to attend to students to with reading difficulties. In the East, the teachers themselves say they respond by devoting more time to the students having difficulties.

• West: Teachers report more often having the student with difficulties work in the regular classroom with a teacher aid or reading specialist or in a remedial reading classroom, compared with the East.

• East: More assessments and testing, classroom tests, teacher-made or textbook tests, and greater use of diagnostic reading tests.

• East: Grades given more often, and use various other material to assess progress such as portfolios

• East: Also, greater variety of methods used for reading assessment, e.g. multiple-choice questions, short open questions, asking students to write short paragraph about what they just read.

• East: Teacher listened more often to pupils reading aloud, asked them verbally about what they had read, listened to the pupils give overview of what they read, met them to discuss what they had read.

• East: Meet the parents more often to discuss progress, send more often progress reports to parents

• East: Teachers report having studied a number of areas more often than their coutnerparts in the West, e.g. the language of the test, literature, the pedagogy/teaching reading, psychology, remedial reading and reading theory

• East: More professional development, e.g. more in-service professional development workshops or seminars that dealt directly with reading (reading theory, instuctional methods) compared with teachers in the west

• East: More reading of books, professional journals related to teaching

• West: Teachers prepare more in school while teacher in East prepare more at home

• East: Teachers in the admit to have had more enthusiasm for teaching when they began, compared with their enthusiasm now

Link with progress

• The Pearson correlation between Progress in PISA literacy (from 2000 to 2012) and dimension 1 was r=-0.62; rho= -0.76.

Caveat

•Social desirability and extreme response generally higher in poorer countries.

•Countries high on the Human Development Index (e.g. Nordic countries and Canada) generally display less social desirability or extreme responses.

•On the other hand: Eastern countries value humilty, avoid being boastful.

• Experience of foreign students that come to Iceland indicates that there are more examinations and homework in Eastern European countries than in Iceland.

• Confirmation from TALIS NPMs from Eastern Europe countries

• Correlation between dimensions and progress does not say anything about individual countries or questions/items.

• Correlation at country level does not imply correlation at the level of the individual.

Summary• Country groupings and dimensions broadly

replicated across data sets and questionnaires

• Eastern vs. Western Europe dimension

• Nordic, Mediterranean, Germanic, Anglo, Eastern European country groups

Summary

• Culture of observation, feedback and improvement - TALIS

• Engagement – PIRLS and TIMSS

• Link with progress in PISA

Summary• More homework, greater emphasis on

examinations. Teachers relate material more to daily life. More professional development. Teachers meet the parents more often, get more feedback from principals, other management staff and colleagues. Give more feedback to students, Employ more textbooks, less computers, less choice in Eastern Europe.

• Correlation, not necessarily causal relationship

• But the relationship between certain teaching practices and progress are not surprising…

• … see National Reading Panel

• Teaching vocabulary and a number of techniques to engage with the text… increases reading comprehension

• Teacher co-operation – greater student progress (Rutter et al. 1979).

• Better teacher education, more student progress (Greenwald et al. 1996)

Future directions

• Correlate with progress on other measures of achievement

• Examine in more depth and breadth the nature of the cultural differences between the country clusters (e.g. Nordic, Anglo, Germanic, East European, Mediterranean)

• How are the teaching practices related to other fundamental practices in these societies

Future directions

• Examine cultural differences within countries. – Can we replicate the local geographies of these

countries with MDS of teaching practices at school or community level?

– Does the same main dimension emerge if we work with school or district means? – and does it correlate with progress. The terms used to compare communities or countries may not be the same that we use to compare individuals within those communities

• One can change teaching practices

• “by using methods known to change culture. Primary among these methods is the analysis of practice, which brings cultural routines to awareness so that teachers can consciously evaluate them and improve them.” (Stigler og Hiebert, 2004).

• Thank you!

Nordic countries: Reading literacy