TIMSSTIMSS International ResultsTIMSS Released ItemsTIMSS Technical Reporthttp://timssandpirls.bc.edu/timss2011/international-results-mathematics.htmlPersonal Communication with IEA

S.Kanageswari Suppiah Shanmugam, PhD.TIMSSTrends in International Mathematics and Science StudyInternational assessment on Maths/Science for grade 4/Grade 8Conducted by International Association for Evaluation of Educational Achievement (IEA)Every four years since 1995 (TIMSS 2011- 5th cycle)Development and Administration Maths/Sc Test Booklets Student/Teacher/School/Curriculum Questionnaires Maths QuestionnairesStudent Questionnaire(S)Home ExperienceSchool ExperienceTeacher Questionnaire (T) EducationProfessional DevelopmentTeaching ExperienceSchool Questionnaire(P)Availability of ResourcesTypes of ProgrammesLearning EnvironmentCurriculum Questionnaire(C)Organisation of Maths/Sc Curriculum Content of Maths/Sc Curriculum

Student Testing Time Student Achievement Booklet Part 1 45 minutesBreakStudent Achievement Booklet Part 2 45 minutesBreakStudent Questionnaire 30 minutes

Student Achievement Booklet28 item blocks 14 Maths14 ScAssigned to 14 BookletsEach booklet has 4 blocks2 maths2 ScEach block has 12- 18 items and in two booklets (Linking)MCQStructured Response QuestionsProvide explanationSupport an answer with reasonsNumerical evidenceDraw diagramDisplay dataOne block- score points of 18 (on average)Maths/Sc blocks alternately begin (balance position effect)

TIMSS Mathematics Assessment Framework TIMSS 2011 Mathematics Frameworkis organized around two dimensions a content dimension specifying the domains or subject matter to be assessed within mathematics (number, algebra, geometry, and data and chance) Has several topic areasEach topic area is presented as a list of objectives a cognitive dimension specifying the domains or thinking processes to be assessed (knowing, applying, and reasoning)The cognitive domains describe the sets of behaviors expected of students as they engage with the mathematics content.TIMSS Maths Content DomainNumber (30%)Algebra(30%)Whole NumFractions/DecimalsIntegersRatio/Proportion/ PercentPatternsAlgebraic ExpressionsEquation/Formula and FunctionGeometry(20%)Data & Chance(20%)G ShapesG MeasurementLocation & MovementData organisation &representationData InterpretationChance Using mathematicsDepends on mathematical knowledgeFamiliarity with mathematics conceptFacts - factual knowledge that provides the basic language of mathematics, and the essential mathematical facts and properties that form the foundation for mathematical thought.Procedures- entails recall of sets of actions and how to carry them out & computational procedures and toolsKnowledge of concepts - make connections between elements of knowledge, judge the validity of mathematicalstatements and methods, and create mathematical representations

Cognitive DomainKnowing(35%)application of mathematical tools in a range of contexts. The facts, concepts, and procedures often are very familiar to the student, with the problems being routine ones.apply mathematical knowledge of facts, skills, and procedures or understanding of mathematical concepts to create representationsProblem solving is central but the problem settings are more routine in the implemented curriculum.have been standard in classroom exercisestextbook problemsApplying(40%)Cognitive Domainset in real-life situationspurely mathematical questionsemphasis more familiar and routine tasksfor logical, systematic thinking. includes intuitive and inductive reasoningbased on patterns and regularities that can be used to arrive at solutions to non-routine problems.Non-routine problems - problems that are very likely to be unfamiliar to students. They make cognitive demands over and above those needed for solution of routine problems, even when the knowledge and skills required for their solution have been learned.Cognitive DomainReasoning(25%)novelty of the context the complexity of the situation, any solution to the problem must involve several steps, drawing on knowledge and understanding from different areas of mathematicsinvolve transfer of knowledge and skills to new situationsInteractions among reasoning skillsCognitive DomainKnowing(35%)Applying(40%)Reasoning(25%) TOPIC AREA FRACTIONS AND DECIMALSCONTENT DOMAIN NUMBER COGNITIVE DOMAIN KNOWING TOPIC AREA FRACTIONS AND DECIMALSCONTENT DOMAIN NUMBER COGNITIVE DOMAIN KNOWING TOPIC AREA FRACTIONS AND DECIMALS CONTENT DOMAIN NUMBERCOGNITIVE DOMAIN APPLYING TOPIC AREA FRACTIONS AND DECIMALS CONTENT DOMAIN NUMBERCOGNITIVE DOMAIN APPLYINGTOPIC AREA RATIO, PROPORTION AND PERCENT CONTENT DOMAIN NUMBERCOGNITIVE DOMAIN APPLYING TOPIC AREA GEOMETRIC SHAPES CONTENT DOMAIN GEOMETRYCOGNITIVE DOMAIN REASONING TOPIC AREA INTEGERCONTENT DOMAIN NUMBERCOGNITIVE DOMAIN REASONING TOPIC AREA GEOMETRIC SHAPES CONTENT DOMAIN GEOMETRYCOGNITIVE DOMAIN REASONINGTOPIC AREA DATA CONTENT DOMAIN DATA AND CHANCE COGNITIVE DOMAIN REASONING INTERNATIONAL BENCHMARK ADVANCED (625)-reason with information, draw conclusions, make generalizations, and solve linear equations. -solve a variety of fraction, proportion, and percent problems and justify conclusions. -express generalizations algebraically and model situations. -solve a variety of problems involving equations, formulas, and functions. -reason with geometric figures to solve problems and with data from several sources or unfamiliar representations to solve multi-step problemsContent Domain: GeometryCognitive Domain: Reasoning Description: Solves a word problem involving filling a three-dimensional shape with rectangular solids

INTERNATIONAL BENCHMARK HIGH (550)-apply understanding and knowledge in complex situation-use information from several sources to solve problems involving different types of numbers and operations. -relate fractions, decimals, and percents to each other. -show basic procedural knowledge related to algebraic expressions. -use properties of lines, angles, triangles, rectangles, and rectangular prisms to solve problems. -analyse data in a variety of graphs.

Peter, James, and Andrew each had 20 tries at throwing balls into a basket. Complete the missing boxes below.

Name Number of Percentage of Successful Shots Successful Shots

Peter 10 out of 20 50 %

James 15 out of 20

Andrew out of 20 80% Content Domain: Number Cognitive Domain: Knowing Description: Given the part and the whole, can express the part as a percentage, and given the whole and the %, can find the part

-apply basic mathematical knowledge in a variety of situations. -solve problems involving decimals, fractions, prop,% -understand simple algebraic relationships. -relate a two-dimensional drawing to a three-dimensional object. -read, interpret, and construct graphs and tables. -recognise basic notions of likelihood. International Benchmark intermediate (475) What does xy + 1 mean? a Add 1 to y, then multiply by x. b Multiply x and y by 1. c Add x to y, then add 1. d Multiply x by y, then add 1. Content Domain: Algebra Cognitive Domain: Knowing Description: Knows the meaning of a simple algebraic expression involving multiplication and addition

Content Domain: Number Cognitive Domain: Knowing Description: Adds a two-place and a three-place decimal Have some knowledge of whole numbers and decimals, operations, and basic graphsInternational Benchmark low (440)42.65 + 5.748 =

Answer: ________ Administration in Malaysia45 countriesFour countries in SEA (Indonesia, Malaysia, Singapore, Thailand)180 Schools5733 Form Two studentsTest Booklets Bahasa Malaysia and EnglishStudent QuestionnaireBahasa MalaysiaMaths Teacher QuestionnaireEnglishSchool QuestionnaireEnglish

Malaysia Performance TrendOnly in Grade 8Participated since 19991999- 16th (39 countries) - BM2003- 10th (48 countries) - BM2007- 20th (46 countries) - BM/ENG2011- 26th (45 countries) - BM/ENGRecorded as one of the countries that declined greatly (40 points or more)

Performance Trend- Content DomainNum Geometry Data & Chance / AlgebraNum OverallGeometry / Data & Chance / Algebra Overall Performance Trend - Cognitive DomainLow performance in ReasoningNot much difference between Knowing/Applying Performance Trend -Content Domain by genderGirls Overall Boys

Performance Trend- Cognitive Domain by genderGirls Overall Boys

Achievement for malaysia at International BenchmarkAt every benchmark, there is a decline TIMSS Mathematics Items Question Types and Scoring Procedures2 question formats- MCQ & constructed-responseEach MCQ is worth 1 score point. Constructed-response questions worth 1 or 2 score points, depending on the nature of the task and the skills required to complete it. The choice of item format depends on the mathematics or being assessed, and the format that best enables students to demonstrate their proficiency.Multiple-Choice Questions (MCQ)MCQ provide students with 4 response options. Select one correct responsecan be used to assess any of the behaviors in the cognitive domains. allow valid, reliable, and economical measurement of a wide range of content in a relatively short testing time. do not allow for students explanations or supporting statementsless suitable for assessing students ability to make more complex interpretations or evaluations. linguistic features need to be appropriate.questions are written clearly and concisely. response options also are written to minimise the reading load of the question. The options that are incorrect are written to be plausible, but not deceptive.Constructed-Response Questions (CRQ)students are required to construct/give a written response, rather than select a response from a set of optionsallow students to provide explanations, support an answer with reasons or numerical evidence, draw diagrams, or display data, well-suited for assessing aspects of knowledge and skills that require students to explain phenomena or interpret data based on their background knowledge and experiencereal-world setting - the setting is familiar to students.need to prepare a Scoring RubricScoring Rubricdescribes the essential features of appropriate and complete responses.focus on evidence of the type of behavior the question assesses. describe evidence of completely correct, partially correct and completely incorrect responses. student responses at each level of understanding provide important guidance to those who will be rating the students responses. In scoring, the focus is solely on students achievement with respect to the topic being assessed, not on their ability to write well. students need to communicate clearlyTypes of Items- Grade 8Types of Items- Grade 8Scoring RubricTypes of Items- Grade 8Types of Items- Grade 8Types of Items- Grade 8TIMSS Item Writing Process Item Writing Process and Guidelinesconsider the timing, grade appropriateness, difficulty level, potential sources of bias (cultural, gender, or geographical, ). Make sure that item validity is not affected by factors that unnecessarily increase the difficulty of the item, such as unfamiliar or difficult vocabulary, grammar, directions, contexts, or stimulus materialsbe sensitive to the possibility of unintentionally placing particular groups of students at an unfair disadvantagediagrams and graphs are drawn accurately (to scale unless otherwise noted), and are correctly and fully labeled.What should the student know? What should the student be able to do?MCQStem is the initial part of the item in which the task is defined.Options refer to the entire set of labeled response choices presented under the stem.Key is the correct response option.Distracters are the incorrect response options.ask a direct question with only one correct answer, and provide plausible distractersthe question must be able to stand alone, and be answerable without the response optionsdo not include extraneous information that can confuse studentsavoid questions for which a wrong method yields the correct answer (e.g., a question about a circle with a radius of 2, since computing either area or circumference get 4)avoid writing items where students can work backwards from the response options to find the correct answer (e.g., solving for x in an equation).(CRQ is more appropriate)

CRQWrite a full credit answer in terms of the language, knowledge, and skills that student could be expected to possess & determine whether to allocate 1 or 2 score points.Develop a specific scoring guide

Take Home informationEach of the items needs to contribute to the overall mathematics testSome relatively easy items and some challenging itemsAvoid items that almost all students or almost no students are able to answer correctlyChecklist- MCQChecklist-CRQEach item1The Content Domain, topic area, and objective the item measures2. The Cognitive Domain 3. The item number (1, 2, 3, etc.)4. The key (multiple-choice items only) or5. The scoring guide12