Studies in Educational Evaluation 34 (2008) 73–81

Examining the evidence from TIMSS: Gender differences in Year 8 scienceachievement in Australia

Sue Thomson

Australian Council for Educational Research, Melbourne, Australia

Contents lists available at ScienceDirect

Studies in Educational Evaluation

journal homepage: www.elsevier.com/stueduc

A R T I C L E I N F O

Keywords:

TIMSS

Gender differences

Australia

Multilevel modelling

A B S T R A C T

Australia’s continuing participation in international science studies such as TIMSS provides a useful lens

through which to monitor achievement in science over time. Gender differences in science were not

evident in the early years of TIMSS but appear to be growing. This article examines gender differences in

science achievement in early secondary schooling in Australia using data from the TIMSS 2003 study. The

aim is to explore the cultural, social, structural and institutional factors that may differentially influence

science achievement, with the aim of identifying which are more important for males and for females.

� 2008 Elsevier Ltd. All rights reserved.

Introduction

The study of science has been a focal area of concern foreducators at most levels in Australia and internationally for thepast few decades. Over this time, science has played anincreasingly important role in terms of economic developmentfor countries in an information technology age. For nationaldevelopment to progress quickly, science could no longer stay theprovince of an elite group – all sections of society needed to beencouraged to study science – particularly girls. In order to obtainsome international benchmark of national achievement in science,Australia participated in the First International Science Study (FISS)in the early 1970s. The results for the oldest group of students whoparticipated in the FISS drew attention to gender differences infavour of boys in participation and achievement in physicsespecially, in a number of countries including Australia (Comber& Keeves, 1973). To redress these inequities as well as theperceived shortage of scientists, government policy in Australiaduring the 1970s and 1980s focussed on encouraging morestudents, particularly girls, into non-traditional areas of study,including mathematics and science (e.g. McKinnon, 1975; Miland,1984). The Second International Science Study (SISS) in the early1980s found that while similar patterns were still observable (boysoutperforming girls), the magnitude of the differences haddecreased in many countries, including Australia, where programsto reduce the gender gap had been implemented.

Australia next participated in an international science studywith the combined Third International Mathematics and Science

E-mail address: thomsons@acer.edu.au.

0191-491X/$ – see front matter � 2008 Elsevier Ltd. All rights reserved.

doi:10.1016/j.stueduc.2008.04.003

Study (TIMSS) in 1994–1995. A feature of the Australian results,found in only a handful of countries, was that no gender differenceswere found either for the younger (Year 4) or mid-secondary (Year8) students (Beaton et al., 1996).

However despite government efforts and increasing retentionrates to Year 12, the number of students studying science at boththe secondary and tertiary level in Australia has declined (Dekkers& De Laeter, 1997; DeLaeter, Malone, & Dekkers, 1989; Dobson &Calderon, 1999; FASTS, 2002; Fullarton, Walker, Ainley, & Hillman,2003; Rennie, Fraser, & Treagust, 1999). And despite more girlsstaying on to complete Year 12, it is not enough for governments tomandate gender equity of outcomes. Girls’ level of participation insecondary and tertiary science courses and the continued malepredominance in subjects such as physics and chemistry inparticular remains an issue (Dawson, 2000; DeLaeter et al., 1989;Fensham, Corrigan, & Malcolm, 1989). Australian findings areconsistent with those found in, for example, the United Kingdom(Gallagher, McEwen, & Knipe, 1997; Kelly, 1981; Stables & Stables,1995), the United States (Sellinger, 2002), and The Netherlands(Bosker & Dekkers, 1994). The OECD Global Science Forum hastargeted the declining interest in science studies among youngpeople as a priority issue, and their interim reports show that theratio of females to males in many areas of science and technology,in particular computer science and engineering, is dramaticallylow (Duby, 2005).

Australia’s continuing participation in international sciencestudies such as TIMSS provides a useful lens through which tomonitor achievement in science over time. Australia has partici-pated in all three combined mathematics and science studies—theThird International Mathematics and Science Study (TIMSS, 1995),the partial (Grade 8 and 9) repeat in 1999, (TIMSS-R) and the

Table 1Descriptive statistics for science, content areas, self-confidence and valuing science,

TIMSS 2003—Australia

Females Males Effect size

Science 518 (4.5) 538 (4.5) 0.27

Life science 527 (4.6) 538 (4.5) 0.15

Chemistry 498 (5.1) 515 (4.9) 0.25

Physics 510 (4.5) 532 (4.5) 0.31

Earth science 516 (4.8) 547 (4.9) 0.43

Environmental science 528 (4.4) 543 (4.0) 0.22

Self-confidence 1.26 (0.03) 1.37 (0.03) 0.14

Valuing science 1.03 (0.03) 1.17 (0.03) 0.18

Notes: Cohen’s effect sizes are classified as follows: negligible effect (��0.15 and

<0.15), small effect (�0.15 and <0.40), medium effect (�0.40 and <0.75).

Coefficients shown in bold are significantly different at p < 0.05.

S. Thomson / Studies in Educational Evaluation 34 (2008) 73–8174

Trends in International Mathematics and Science Study (TIMSS,2003). These data provide researchers with the facility to examinetrends in science achievement and using common achievementitems allows the calibration of different tests onto the same scale toexamine differences in achievement over time.

During the 1970s and 1980s in Australia there were strongcampaigns to encourage girls’ participation in mathematics andthe sciences, and the success of these programs was reflected in thelack of significant gender differences in either science ormathematics in Australia in either the TIMSS 1995 or TIMSS1999 assessments. However, despite the major gains made by girlsduring the eighties and nineties, the main trends noted in theAustralian National Report on TIMSS Science, Examining the

evidence: science achievement in Australian schools in TIMSS 20021

(Thomson & Fleming, 2004), were that Australia’s average scienceachievement had increased significantly since TIMSS 1995, andthat this was largely due to a significant increase in scores formales, and that at Year 8 level boys significantly outperformedgirls.

However, as time goes by educational focus changes, and inlatter years the educational focus has been on other issues. Thisarticle examines gender differences in science achievement inearly secondary schooling in Australia using data from the TIMSS2003 study. The aim is to explore the cultural, social, structural andinstitutional factors that may differentially influence scienceachievement, with the aim of identifying which are moreimportant for males and for females. In this way we may beginto isolate reasons why neither male nor female students appear tobe rushing to enrol in the sciences, and the reasons for thedeclining levels of girls’ achievement.

Gender differences in science achievement in TIMSS 2003

In TIMSS 2003 in Australia, there was no significant genderdifference in science achievement at primary school level, nor werethere gender differences in attitudes to science. By early in theirsecondary schooling, however, girls were outscored by boys in allof the science content areas, significantly in all but life science.2

They also exhibited lower levels of self-confidence in science thanboys, and did not value science to the same extent as boys. Agender gap appears to develop both in science achievement and inattitudes towards science during the latter years of primaryschooling, a gender gap that had appeared to have been eradicated.In contrast, no such gender differences were evident for mathe-matics achievement in TIMSS 2003.

For Australia, the means, standard errors and effect sizes for thedifferences in science overall, the science content areas andattitudes to science can be seen in Table 1. The table shows that thegender differences are most pronounced in physics and earthscience, and least pronounced in life science and environmentalscience. Most of the effect sizes are small (only for Earth Science isthe effect size moderate); however they are consistently in thesame direction. If there were large effect sizes it would indicatelarge gaps between males and females in achievement andattitudes. In this case, the effect sizes reflect gaps that should beable to be bridged.

1 For comparability across countries and assessments, testing for TIMSS is carried

out at the end of the school year. For most of the countries involved, this occurred in

May to June 2003, and thus the international study retains the name ‘TIMSS 2003’.

For most southern hemisphere countries, including Australia, the end of the school

year coincides with the end of the calendar year, so testing occurred at the end of

2002. Local reports all used the year of testing in Australian schools in the title.2 Internationally, males outscored females significantly in physics, earth science

and environmental science, females significantly outscored males in life science,

and scores were not significantly different for chemistry.

Implementation of the TIMSS 2003 study

Data

Data for the current TIMSS study were collected for Australiain late 2002, at both Year 4 and Year 8. The sample design forTIMSS is a two-stage stratified cluster sample design. The firststage consists of a random sample of schools, which in Australiais stratified by state and by sector (government, Catholic andnon-Catholic-independent schools); the second stage consists ofa random sample of one mathematics class from the targetyear in each sampled school. To ensure unbiased data, theInternational Study Centre set minimum participationrates of 85% of sampled schools and 85% of sampled students.Australia achieved these targets at Year 8 level. Sampleweights were calculated by Statistics Canada to ensure thatthe population was appropriately represented by thestudents participating in TIMSS. Using this methodology, 5355students in 210 schools participated in the TIMSS testing, and536 science teachers and 192 principals completed schoolquestionnaires.

The science assessment for TIMSS was framed by twoorganising dimensions; a content dimension and a cognitivedimension (Mullis et al., 2003). There were five content domains:life science, chemistry, physics, earth science and environmentalscience, and three cognitive domains: factual knowledge, con-ceptual understanding and reasoning and analysis.

It is important for a study such as TIMSS that studentachievement is not considered in isolation from the economic,social, cultural and educational contexts in which it occurs. Toensure that these data were available, TIMSS included threemain background questionnaires. The school questionnairesought information about school characteristics such as locationand size as well as resources, time for curriculum planning andcurriculum offerings. The teacher questionnaires asked aboutqualifications and preparation, pedagogical practices andviews on current issues in mathematics and science. As theTIMSS sample is based on the mathematics class, these surveyswere completed by all of the science teachers of the students inthe class, as long as they taught more than five students in theclass. The student questionnaire collected demographic infor-mation, data on how students spent their time in and out ofschool, and their attitudes and beliefs about mathematics andscience.

Variables used in this paper

This section briefly describes the variables that were used inthis paper. A full description of all of the variables used and theircoding is provided in Appendix A in Table A1.

S. Thomson / Studies in Educational Evaluation 34 (2008) 73–81 75

Achievement variables

The TIMSS tests are intended to generate achievement data thatare reliable and valid for their intended purpose. TIMSS used itemresponse theory (IRT) methods to summarise the achievementresults with a mean of 500 and a standard deviation of 100.

Student background variables

Previous IEA studies of science achievement have shown thatstudent achievement is related to home background factors, andto students’ activities and attitudes (Beaton et al., 1996; Martinet al., 2000). These studies (as well as many others) have foundthat students in homes with extensive educational resourceshave higher achievement levels in science than those from lessadvantaged backgrounds. Educational resources measured inTIMSS include number of books in the home (BOOKS), presence ofparticular study aids such as a computer and a desk for thestudent’s use (HER), and probably one of the most importanteducational resources in the home: parents’ level of education(PARED). Because having a computer in the home does notnecessarily mean that they are able to use it, students were askedwhere they actually used a computer. An index (COMPUSE) wasderived to examine the extent to which students used acomputer. Those who used computers at school only weredeemed to use it less than those who used them at home andthose who used them at home and school scored highest on theindex. In Australia, indigenous status (whether a studentidentifies as Aboriginal or Torres Strait Islander (INDIG) or not)is a strong negative correlate of achievement, and to examine theeffects of the combination of students’ country of birth andlanguage spoken at home, a variable (COBLAN) was derivedwhich uses data from each of those items.

Student attitudes and beliefs

Internationally, there was a strong relationship betweenstudents’ aspirations to higher education and their parents’ levelof education. A composite variable was derived (STUDASP)describing four combinations of student’s aspirations and parents’educational level, ranging from a student having no aspiration tofinish school, and parents not having completed secondary school,through to a student wanting to finish university and at least oneparent having done so.

Student self-confidence in science (CONFIDENCE), the extent towhich students valued and enjoyed science (VALUE), and students’perceptions of their own safety in the school (SAFETY) were alsoused as controlling factors. If students are confident in theirabilities, place a value on what they are learning and enjoy whatthey are learning, then learning should be easier and soachievement higher.

Teacher, class and school effects

Teachers’ perceptions of school safety (STPSS) and schoolclimate (STPSC), and the extent to which they believed they wereable to teach without being hindered by factors such as disruptivestudents or lack of resources (SCWOLF), were all measured throughthe teachers’ or schools’ questionnaires. These factors wereincluded because they showed positive or negative relationshipswith student achievement, and because they are perhaps amen-able to change.

From a school’s perspective, there were several factors, whichhave been shown to be related to achievement. An indexrepresenting the overall percentage of students from economicallydisadvantaged backgrounds (IPED) was developed, in which twolevels were defined: less than and more than 25% of students fromeconomically disadvantaged homes. An index representing theprincipals’ perception of the school climate (PPSC) was also used.

Data analysis

In Australia there were significant relationships found betweenscience achievement and each of the variables described in theprevious section (for further details see Thomson & Fleming, 2004).In order to investigate the factors associated with variance inscience achievement scores between males and females, it wasnecessary to use multilevel modelling techniques (Goldstein,2003; Snijders & Bosker, 1999). Multilevel modelling is adevelopment of the common statistical technique of regression,which finds a straight-line relationship, which allows us to predictthe values of a particular outcome measure given the values of oneor more related values. Multilevel modelling acknowledges thatdata are often grouped into clusters at several levels. For example,with the TIMSS data, students are grouped within classes, withinschools. Multilevel modelling allows us to take this hierarchicalstructure into account and produce more accurate predictions andestimates.

Results of this study

Table 1 shows that there were significant gender differences inmany areas of science achievement and beliefs. But is this blanketinformation helpful to principals or teachers, or really useful atanything other than the national level? Perhaps it would be moreuseful to schools if we were able to dig deeper into the data using avariety of techniques. The beginning of this exploration is shown inTable 2.

Gender differences

The results presented in Table 2 are a clear indication that thereis a variety of gender effects amongst these variables. One of themost striking effects is that of gender and indigenous status onscience achievement. There are significant gender differenceswithin both the indigenous and non-indigenous groups, howeverwhile the effect size for the non-indigenous population is small,there is a medium effect for the indigenous population. Indigenousfemales’ average achievement was almost half a standarddeviation lower than the average achievement level for indigenousmales.

For students in homes with few books, the gender difference islarger than for students in homes with many books. This meansthat girls achieve outcomes more similar to those for boys in moreaffluent homes, while in homes with fewer resources boys’ scoresare much larger than those for girls. Similarly level of education ofparents has an effect on student achievement in science, albeit onlya small one. There is a significant gender difference in homes withparents with low levels of education that is not apparent forstudents in homes with well-educated parents. Similar results canbe seen for students with low and medium levels of homeresources. In poor home situations it appears that it is girls whosuffer more in terms of science education.

Medium effect sizes for gender differences are also apparentfor those students who aspire to complete a university degreealthough neither of their parents did so. Clearly, having a parentwho attended university is greatly beneficial to girls, with anaverage score increase between this group and the group whoplanned to finish university when neither parent had of 26 scalepoints, and in homes where parents have a high level ofeducation, there are no significant differences in scienceachievement.

Self-confidence in science and valuing of science has a ‘booster’effect on achievement, particularly for males. The average increasein achievement scores for students on the lowest level of self-

Table 2Percentage of students and average science achievement for variables with a significant influence

Percentage students Average science achievement Effect size

Females Males Females Males

Indigenous (INDIG)

Yes 3 3 447 (13.1) 493 (9.3) 0.60

No 97 97 521 (4.4) 540 (4.5) 0.26

Books in the home (BOOKS)

Low 16 16 472 (6.9) 499 (6.7) 0.37

Medium 31 29 505 (5.8) 530 (5.3) 0.35

High 53 55 540 (4.1) 555 (4.7) 0.22

Parents’ educational level (PARED)

Low 38 40 497 (5.2) 521(4.4) 0.33

Medium 42 36 520 (4.5) 539 (5.7) 0.28

High 20 24 556 (5.3) 572 (6.8) 0.24

Home education resources (HER)

Low 3 2 431 (13.5) 456 (15.0) 0.35

Medium 83 83 512 (4.5) 533 (4.5) 0.29

High 14 15 565 (5.2) 579 (7.0) 0.22

Country of birth/language background (COBLAN)

Born NESC, English not spoken at home 3 6 517 (12.9) 547 (17.4) 0.35

Born ESC, English not spoken at home 4 3 490 (14.6) 544 (21.6) 0.63

Born NESC, English spoken at home 4 4 515 (10.1) 539 (14.7) 0.30

Born ESC, English spoken at home 89 87 521 (4.4) 541 (4.1) 0.28

Student aspirations relative to parental education (STUDASP)

Do not know, regardless of parents’ education 15 17 502 (9.3) 529 (6.5) 0.35

Not finish university, regardless of parents’ education 45 45 500 (4.2) 519 (4.4) 0.28

Finish university, neither parent attended university 25 20 537 (5.6) 568 (4.8) 0.48

Finish university, at least one parent completed uni 15 17 563 (5.5) 584 (6.6) 0.33

Computer usage (COMPUSE)

Do not use computer at all 1 1 458 (19.6) 465 (18.8) 0.10

Use computer only at places other than home 1 1 511 (21.0) 485 (24.3) 0.39

Use computer at school but not at home 6 4 467 (10.4) 483 (10.9) 0.20

Use computer at home but not at school 8 12 509 (10.1) 535 (7.7) 0.36

Use computer both at home and at school 85 82 523 (4.5) 544 (4.6) 0.29

Students’ perception of school safety (SAFETY)

Low 12 24 514 (6.2) 529 (5.4) 0.20

Medium 39 41 513 (4.9) 545 (4.9) 0.44

High 49 36 524 (5.4) 541 (5.4) 0.23

Self-confidence in science (CONFIDENCE)

Low 19 15 498 (6.1) 500 (6.6) 0.03

Medium 35 34 507 (4.7) 521 (3.9) 0.20

High 46 52 536 (4.9) 563 (5.1) 0.37

Valuing of science (VALUE)

Low 30 23 499 (5.7) 515 (5.5) 0.23

Medium 37 37 516 (5.3) 529 (5.3) 0.18

High 33 40 538 (4.4) 562 (5.1) 0.33

Science teachers perception of school climate (STPSC)

Low 31 28 489 (7.3) 511 (7.8) 0.31

Medium 54 60 522 (6.2) 545 (6.4) 0.32

High 15 13 542 (8.2) 571 (10.3) 0.44

Science teachers perception of school safety (STPSS)

Low 3 4 492 (18.8) 522 (21.0) 0.37

Medium 26 25 514 (11.0) 545 (9.1) 0.41

High 71 71 516 (5.2) 538 (5.2) 0.31

Science teachers’ report of science classes without limiting factors (SCWOLF)

Low 15 16 497 (8.0) 510 (8.7) 0.19

Medium 34 36 512 (8.7) 533 (5.7) 0.29

High 51 48 529 (5.1) 552 (7.3) 0.33

Index of percentage of students from economically disadvantaged classes (IPED)

Fewer than 25% of students 66 69 532 (5.6) 551 (5.8) 0.26

More than 25% of students 34 31 497 (7.2) 515 (8.3) 0.27

Principals’ perception of school climate (PPSC)

Low 7 8 480 (18.2) 473 (15.7) 0.09

Medium 64 59 516 (6.6) 545 (4.5) 0.41

High 30 33 535 (6.4) 546 (10.4) 0.15

Notes: Cohen’s effect sizes are classified as follows: negligible effect (��0.15 and<0.15), small effect (�0.15 and<0.40), medium effect (�0.40 and<0.75). Coefficients shown

in bold are significantly different at p < 0.05.

S. Thomson / Studies in Educational Evaluation 34 (2008) 73–8176

S. Thomson / Studies in Educational Evaluation 34 (2008) 73–81 77

confidence in science to the highest level of self-confidence was 38score points for females and 63 score points for males. An increasein the level of valuing science was similarly reflected in an increasein achievement. From the low level of valuing science to the highlevel resulted in an average score increase of 39 score points forfemales and 47 score points for males.

In all other ways, the effects of the variables were as would beexpected. Achievement was higher for students from backgroundswith high levels of resources, both physical and in terms ofparental education. Achievement was also higher for students whowere confident in science, who valued science, and who had highaspirations. It may well be, however, that some of these factors aremore important for males than females, or vice versa, holding allother variables constant.

Results of multilevel analysis

A two-step multilevel analysis was conducted to investigateseparately but simultaneously for males and females whichfactors were significant influences on science achievement atYear 8. The initial (null) model was used to estimate the amountof between-class and within-class variance. This found that,for both males and females, about 35% of the variance wasattributable to differences between schools/classrooms(between-class) and 65% of the variance in science achievementwas attributable to differences between individuals (within-class). In the subsequent models the school/class and studentvariables were included and the models were tested iteratively.At each iteration, variables that were not statistically significantin either model were removed until the models only containedvariables that were statistically significant in one or both of themodels.

The normalised coefficients for the multilevel models con-ducted for females and males are presented graphically in Fig. 1,and in Table A2 of Appendix A. To assist interpretation, all variableswere recomputed as normal scores, namely as ‘normal equivalentdeviates’ (NEDs) under the normal distribution, to ensure that suchvariables are measured on a common metric. This allows the

Fig. 1. Normalised coefficients from multilevel model for Australian females and male

Table A1.

effects of the different variables to be compared in terms of theirapparent influence on the outcome, when all the other variablesare simultaneously taken into account.

For each of the variables, the estimated normalised coefficient isplotted as a square, with a vertical line indicating the 95%confidence interval for the estimate. Any variable whose lineintersects the horizontal zero axis can be considered to be notstatistically significant (at the 5% level). Positive values imply apositive relationship with science achievement; negative valuesimply that science achievement scores tend to decrease withhigher levels of the variable. The first bar in a pair represents thecoefficient for females, the second for males.

Also shown in Table A2 are pseudo effect sizes. While thenormalised coefficient is a representation of the overall impact ofeach factor across the whole population, the pseudo effect sizeprovides us with a measure of the impact of each factor for thosewhom it affects. For binary variables, the pseudo effect size showsthe impact of going from one state to the other on the binaryvariable, as a percentage of the standard deviation in the outcomevariable (100 in this case). For non-binary variables, it representsthe ‘average switch’ between one state and another (Schagen,1994).

There are several strong relationships with achievement, whichcan be seen from Fig. 1. There are also school and class factors,student attitudinal factors and student background factors. Themain school factor that influences achievement for both femalesand males, all other things equal, is the proportion of students fromeconomically disadvantaged homes in the school (IPED). Thisprovides (as Table 2 shows) an advantage of about 20 scale scorepoints for those students attending a school with a low proportionof economically disadvantaged students. This is an importantfinding which needs to be dealt with separately. This result is aclear indication that the goal of egalitarian education in Australiahas not been reached.

The other strong influence on achievement, particularly formales, is that of indigenous status (INDIG). The negative influenceof indigenous status on achievement for males, all other thingsequal, has an effect size of 39, indicating that the difference in

s fitted to overall science score. Coding and details of all variables is described in

S. Thomson / Studies in Educational Evaluation 34 (2008) 73–8178

achievement between indigenous males and non-indigenousmales is about 39 scale points. For females, while there is adifference, this difference could have come about by chance, as theconfidence interval crosses zero on the horizontal axis. For thoseindigenous students with high levels of home education resources,however, the story is a little different. Although still not achievingas well as males, the gap between females and males was muchlower (differences were non-significant, but to be cautious this isprobably an effect of low numbers and thus high standard errors).While there were only 2% of female and 2% of male non-indigenousstudents with a low level of home educational resources, 15% offemale and 11% of male indigenous students reported this low levelof resources. The implication of these data is that the lowerachievement of indigenous students may be more about povertythan it is about ethnicity.

The other student background factors associated withachievement for females were parental education and books inthe home. Males and females from families with high levels ofparental education had about a 20 score point advantage overthose from families with low levels of parental education, andthose from family backgrounds with high numbers of booksachieved scores of about 46 points for females and 42 points formales higher than their counterparts with low numbers of booksin the home. Books in the home has long been used as a proxymeasure for home educational background, and the effects ofthese two variables together indicate that families with well-educated parents perhaps actively encourage their daughters toparticipate in science as they are more aware of the narrowing ofopportunities for girls who do not undertake any studies inscience.

The effect size for self-confidence in science is almost fourtimes greater than the effect size for valuing science for females,and almost three times greater for males. While the differencebetween females with a low level and a high level of valuingscience is around 10 scale score points, the difference betweenfemales with a low level and those with a high level of self-confidence is around 38 score points. For males the difference inself-confidence is reflected in a rise of around 48 scale scorepoints, almost half a standard deviation. This suggests that weshould focus on boosting self-confidence across the board, not justwith female students.

Conclusions and discussion

One of the benefits to a country participating in internationalstudies such as TIMSS that are carried out with representativesamples of the population over a period of a number of years is thatit is possible to monitor national trends. For Australia in the early1970s, the FISS showed that boys outperformed girls in science. Inthe 1970s and 1980s, governments pursued policies thatencouraged girls into the sciences, with some effect—the SecondInternational Science Study showed that the gender gap had closeda little. The Third International Science Study, conducted in theearly 1990s found that the gender gap had closed even more, andthat the difference in achievement between girls and boys wasnon-significant. The partial repeat of this study in the late 1990sconfirmed this finding.

What has the evidence from the most recent TIMSS study toldus about gender differences in Australian students’ scienceachievement in the early part of the twenty-first century? InTIMSS 2003 gender differences in both achievement and attitudesto science are not evident at junior primary school level inAustralia but are significant in favour of males at early secondaryschool level. In contrast, gender differences in mathematics haveremained non-significant since TIMSS 1995.

For around half of the students who participated in TIMSS 2003in Australia, the testing was carried out at the end of their first yearof secondary schooling, for the remainder, at the end of the secondyear of secondary schooling. This suggests that the differences thatare observed are likely to be a result of practices in the latter half ofprimary schooling. This needs further investigation. There is asmall window of opportunity here for intervention. If the windowis missed, self-concept and confidence of female students begin togo into decline.

Many of the factors that were hypothesised to affectachievement were not significant. Teacher factors did notappear to play a part in achievement, nor did many schoolfactors such as class size, urban or rural locality, or availability ofresources for science instruction. Further analyses could beconducted examining the effect of teaching methods onachievement.

A number of student background factors had strong relation-ships with achievement. Indigenous status was the most striking ofthese, and it was of interest that this large effect was for males butnot for females, all other things equal. While it is not measuredexplicitly, the lack of educational resources provides someindication of a level of poverty in students’ backgrounds, eitherpoverty financially or at the very least an impoverished educa-tional background. It is likely that this poverty in the indigenouscommunity is a confounding factor, particularly for girls. This isanother issue that raises questions for further research. Is theresomething about being indigenous that has a strong effect onmales but not such a strong effect on females? Is it tied in withroles that children are expected to play when families are inpoverty?

Two of the most important background factors for females werefound to be parental education and number of books in the home.There were no gender differences in achievement for thosestudents with a high number of books in the home and a highlevel of parental education. This is another finding worthy offurther investigation. Does the higher educational level of parentsencourage more of a feeling of equity in their daughters, so thatthey do not see science as a gendered subject? Another explanationis that the higher level of education of parents is more conducive toa home background of enquiry that encourages females to exploreand find answers for themselves and studying science at school is anatural extension of this.

The findings regarding the strong positive influence of self-confidence on science achievement, for both male and femalestudents, warrants our attention. Self-confidence is an attributethat is amenable to influence by teachers, parents and othersignificant people in student’s lives. It is one area in which wemight be able to bridge the gap. Furthermore, attitudes andself-concept become particularly important when studyingscience is no longer compulsory in secondary school, whenstudents must make an active choice to study science, and againin the pursuit of science-related courses at tertiary level(Thomson, 2005).

Students’ attitudes towards science and its study and thesubsequent influence on science achievement also warrant furtherinvestigation and intervention. If female students’ attitudestowards science were taken as a predictor of participation thenthe results from TIMSS 2003 would indicate that we are unlikely tosee any amelioration in girls’ attitudes, participation or achieve-ment in science over the next few years.

Appendix A

See Tables A1 and A2.

Table A1Coding of all variables used

Indigenous (INDIG) 0 = not indigenous

1 = indigenous

Books in the home (BOOKS) 1 = low (0–25 books)

2 = medium (26–200 books)

3 = high (more than 200 books)

Parents’ educational level (PARED) 1 = low (did not finish secondary school)

2 = medium (finished upper secondary school or post-secondary education)

3 = high (finished university or higher degree)

Home education resources (HER) 1 = low (25 or fewer books, fewer than three of the study aids, both parents

secondary educated at most)

3 = high (more than 100 books, all three study aids – computer, desk for own use,

dictionary – either parent finished university)

2 = medium (all other combinations)

Country of birth/language background (COBLAN) 0 = born non-English speaking country, English not spoken at home

1 = born English-speaking country, English not spoken at home

2 = born non-English speaking country, English not spoken at home

3 = born English-speaking country, English spoken at home

Student aspirations relative to parental education (STUDASP) 0 = do not know, regardless of parents’ education

1 = not finish university, regardless of parents’ education

2 = finish university, but neither parent attended university

3 = finish university and either parent went to university or equivalent

Computer usage (COMPUSE) 0 = do not use computers at all

1 = use computer only at places other than home or school

2 = use computer at school but not home

3 = use computer at home but not school

4 = use computer at home and at school

Students’ perception of school safety (SAFETY) Index based on responses to five statements about incidents at school during the last month

(Yes/No): something of mine was stolen, I was hit or hurt by other students, I was made to do things

I did not want to do by other students, I was made fun of or called names, I was left out of

activities by other students

0 = low (student answered Yes to three or more)

2 = high (student answered No to all five)

1 = Medium (all other combinations)

Self-confidence in science (CONFIDENCE) Index based on responses to four statements: I usually do well in science, science is more difficult

for me than for my classmates (reversed), science is not one of my strengths (reversed), I learn

things quickly in science. Average is computed across the items based on a four point scale: (1)

agree a lot; (2) agree a little; (3) disagree a little; (4) disagree a lot

0 = low (average agree a little or lot to all items)

2 = high (average disagree a little or lot to all items)

1 = medium (Other combinations)

Valuing of science (VALUE) Index based on responses to seven statements about science: I would like to take more science at

school, I enjoy learning science, I think learning science will help me in my daily life, I need

science to learn other school subjects, I need to do well in school to get into the university of my

choice, I would like a job that involved using science, I need to do well in science to get the job I

want. Average is computed across the items based on a four point scale: (1) agree a lot; (2) agree

a little; (3) disagree a little; (4) disagree a lot

0 = low (average agree a little or lot to all items)

2 = high (average disagree a little or lot to all items)

1 = medium (other combinations)

Science teachers perception of school climate (STPSC) Index based on teachers’ responses to eight questions about their school: teachers’ job satisfaction;

teachers’ understanding of the schools’ curricular goals; teachers’ degree of success in implementing

the school’s curriculum; teachers’ expectation for student achievement; parental support for

student achievement; parental involvement in school activities; students’ regard for school

property; students’ desire to do well in school. Average is computed across the items based

on a five point scale: (1) very high; (2) high; (3) medium; (4) low; (5) very low

0 = low (average greater than 3)

1 = medium (average more than 2 and less than or equal to 3)

2 = high (average less than or equal to 2)

Science teachers perception of school safety (STPSS) Index based on teachers’ responses to three statements about their schools: this school is located

in a safe neighbourhood; this school’s security policies and practices are sufficient; I feel safe

at this school.

0 = low (teacher disagrees or disagrees a lot to all three statements)

2 = high (teacher agrees or agrees a lot to all three statements)

1 = medium (all other combinations)

Science teachers’ report of science classes without

limiting factors (SCWOLF)

Index based on teachers’ responses to six statements about student factors limiting science

instruction: students who come from a wide range of backgrounds; students with different

academic abilities; students with special needs; uninterested students; low morale amongst

students; disruptive students

S. Thomson / Studies in Educational Evaluation 34 (2008) 73–81 79

Table A2Normalised coefficients and pseudo effect sizes for significant background variables related to science achievement in TIMSS 2003 (Australia)

Range S.D. Coefficient S.E. Pseudo effect size

Indigenous 0,1 Females 0.1 �10 7.6 �13

Males 0.2 �30 7.7 �39

Books in the home 0,2 Females 0.7 16 2.0 23

Males 0.7 16 2.2 21

Parental educational level 0,2 Females 0.7 8 2.4 11

Males 0.8 3 2.6 4

Home educational resources 0,2 Females 0.4 �1 4.8 �1

Males 0.4 5 5.2 3

Country of birth/language 0,3 Females 0.7 5 2.4 6

Males 0.8 4 2.4 6

Student aspirations 0,4 Females 0.9 9 1.7 15

Males 1.0 10 1.9 18

Computer use 0,4 Females 0.6 6 2.2 7

Males 0.6 10 2.4 11

Student perception of safety 0,2 Females 0.7 4 1.9 5

Males 0.7 2 1.9 2

Self-confidence in science 0,2 Females 0.8 13 1.9 19

Males 0.7 18 2.3 24

Valuing science 0,2 Females 0.8 3 1.9 5

Males 0.8 6 2.2 9

Science teachers’ perception of school climate 0,2 Females 0.6 10 3.9 12

Males 0.6 10 4.4 11

Science teachers’ perception of school safety 0,2 Females 0.5 �4 4.2 �4

Males 0.5 �10 4.6 �10

Science classes without limiting factors 0,2 Females 0.7 3 3.1 4

Males 0.7 10 3.3 13

Per cent students from economically disadvantaged homes 0,1 Females 0.5 �15 6.1 �21

Males 0.5 �16 6.6 �20

Principals’ perception of school climate 0,2 Females 0.1 11 5.3 1

Males 0.6 7 5.6 8

Table A1 (Continued )

Average is computed across the items based on a four point scale: (1) not at all; (2) a little; (3) some;

(4) a lot

0 = low (average greater or equal to 3)

1 = medium (average more than 2 and less than 3)

2 = high (average less than or equal to 2)

Index of percentage of students from economically

disadvantaged homes (IPED)

Principals’ report of the percentage of students coming from economically disadvantaged backgrounds

0 = less than 25% of students

1 = more than 25% of students

Principals’ perception of school climate (PPSC) Index based on teachers’ responses to eight questions about their school: teachers’ job satisfaction;

teachers’ understanding of the schools’ curricular goals; teachers’ degree of success in implementing

the school’s curriculum; teachers’ expectation for student achievement; parental support for student

achievement; parental involvement in school activities; students’ regard for school property; students’

desire to do well in school. Average is computed across the items based on a five point scale: (1)

very high; (2) high; (3) medium; (4) low; (5) very low

0 = low (average greater than 3)

1 = medium (average more than 2 and less than or equal to 3)

2 = high (average less than or equal to 2)

S. Thomson / Studies in Educational Evaluation 34 (2008) 73–8180

References

Beaton, A. E., Martin, M. O., Mullis, I. V. S., Gonzalez, E. J., Smith, T. A., & Kelly, D. L.(1996). Science achievement in the middle school years: IEA’s Third InternationalMathematics and Science Study. Chestnut Hill, MA: Boston College.

Bosker, R. J., & Dekkers, H. P. J. M. (1994). School differences in producing gender-related subject choices. School Effectiveness and School Improvement, 5(2),178–195.

Comber, L. C., & Keeves, J. P. (1973). Science education in nineteen countries: An empiricalstudy. Stockholm: Amquist and Wiksell.

Dekkers, J., & De Laeter, J. R. (1997). The changing nature of upper secondary schoolscience subject enrolments. Australian Science Teachers’ Journal, 43(4), 35–41.

DeLaeter, J. R., Malone, J., & Dekkers, J. (1989). Female science enrolment trends inAustralian senior secondary schools. The Australian Science Teachers Journal, 35(3),23–33.

Dawson, C. (2000). Upper primary boys’ and girls’ interests in science: Have theychanged since 1980? International Journal of Science Education, 22(6), 557–570.

Dobson, I. R., & Calderon, A. J. (1999). Trends in science education: Learning, teaching andoutcomes 1989–1997. Canberra: Australian Council of Deans of Science. (availablewww.acd.edu.au).

S. Thomson / Studies in Educational Evaluation 34 (2008) 73–81 81

Duby, J.-J. (2005). Study on declining interest in science studies: Preliminaryreport on the quantitative analysis. Presentation at the 10th OECD/Japanseminar.

Federation of Australian Scientific and Technological Societies (FASTS). (2002). Aus-tralian science: Investing in the future. Canberra: FASTS.

Fensham, P. J., Corrigan, D. J., & Malcolm, C. (1989). Science for everybody? A summary ofresearch findings for curriculum development Canberra: Curriculum DevelopmentCentre.

Fullarton, S., Walker, M., Ainley, J., & Hillman, K. (2003). Patterns of participation in Year12. LSAY Research Report No. 15. Melbourne: ACER.

Gallagher, T., McEwen, A., & Knipe, D. (1997). Science education policy: A survey ofthe participation of sixth-form pupils in science and other subjects over a 10-year period, 1985–1995. Research Papers in Education: Policy and Practice, 12(2),121–141.

Goldstein, H. (2003). Multilevel statistical models. London: Arnold.Kelly, A. (1981). Girls and science education—Is there a problem? In A. Kelly (Ed.), The

missing half: Girls and science education. Manchester: Manchester UniversityPress.

McKinnon, K. R. (1975). Girls, School and Society. Report by a Study Group to the SchoolsCommission, Canberra.

Martin, M. O., Mullis, I. V. S., Gonzalez, E. J., Gregory, K. D., Smith, T. A., Chrostowski, S. J.,et al. (2000). TIMSS 1999 international science report: Findings from IEA’s repeat of theThird International Mathematics and Science Study at the eighth grade. Chestnut Hill,MA: Boston College.

Miland, M. (1984). Girls and tomorrow: The challenge for schools. The Report of theCommonwealth Schools Commission’s Working Party on the Education of Girls,Canberra.

Mullis, I. V. S., Martin, M. O., Smith, T. A., Garden, R. A., Gregory, K. D., Gonzalez, E. J.,et al. (2003). TIMSS assessment frameworks and specifications 2003. Chestnut Hill,MA: Boston College.

Rennie, L. J., Fraser, B. J., & Treagust, D. F. (1999). Research into science education. In J. P.Keeves & K. Marjoribanks (Eds.), Australian education: Review of research 1965–1998(pp. 171–203). Melbourne: ACER.

Sellinger, P. G. (2002). Why more women aren’t becoming engineers. Education Week,21(29), 42.

Schagen, I. (1994). Presenting the results of complex models—Normalised coefficients,star wars plots and other ideas. In I. Schagen & K. Elliot (Eds.), But what does itmean? The use of effect size in educational research (pp. 25–41). London: NFER andUniversity of London, Institute of Education.

Snijders, A. B., & Bosker, R. J. (1999). Multilevel analysis. An introduction to basic andadvanced multilevel modeling. London: Sage.

Stables, A., & Stables, S. (1995). Gender differences in students’ approaches to A-levelsubject choices and perceptions of A-level subjects: A study of first year A-levelstudents in a tertiary college. Educational Research, 37(1), 39–51.

Thomson, S. (2005). Pathways from school to further education or work: Examining theconsequences of Year 12 course choices. Melbourne: ACER.

Thomson, S., & Fleming, N. (2004). Examining the evidence: Science achievement inAustralian schools in TIMSS 2002. Melbourne: ACER.