Appendix 1 On line training resource€¦ · QCA-RSS Centre Project December 2006 20 Appendix 1 On line training resource 1 2 3 4 5 6

QCA-RSS Centre Project December 2006 20

Appendix 1 On line training resource

1 2

3 4

5 6


7 8

9 10

11 12


13 14

15 16

17 18


19 20

21 22

23 24


25 26

27 28

29 30


31 32

33 34

35 36


37 38

39 40

41 42


43 44

45 46

47


Appendix 2

Online feedback form for teachers trialling resources

1 2

3 4

5 6


7 8

9 10

11 12


13 14

15 16

17 18


19 20

21 22

23 24


25 26

27 28

29 30


31 32


Appendix 3

James Nicholson’s comments on initial set of resources Scene of Crime Evidence: I think there is a structural weakness in this which is referred to in the notes for slide 4, that is that the relationship between foot length is weak and probably worse for age; it might be useful for gender if the observation was at an extreme, but very little use when it is a pretty average length. The footprint itself would be useful as and when someone was suspected of the crime, because it is a multidimensional piece of information, and could be used for comparison with a footprint obtained from a suspect – therefore the ‘expert witness for the defence’ is not going to be able to argue just on the basis of not being able to tell age from the length. I think the context of forensic evidence is potentially a very rich one in which to explore the handling data cycle, but I am not convinced that this activity does it justice. The activities currently in the task are listed in order they occur in the presentation notes, and I have then made comments as to how I think these activities could or should be assessed, using these numbers as reference points, along with some suggestions for other activities which could be incorporated for assessment purposes. NB: In slide 12 notes, the end of the first paragraph has date instead of data. In slide 18 notes it talks about possible extensions related to left / right dominance, but I am at a loss to see the connection. Where’s worst? The activities currently in the task are listed in the order they occur in the presentation notes, and I have then made comments as to how I think these activities could or should be assessed, using these numbers as reference points, along with some suggestions for other activities which could be incorporated for assessment purposes. NB: On slide 8, I had no idea of the functionality you have built in to the sheet, but I was going to suggest having a place users enter the criteria and it would be copied automatically into all the locations: when I started it to illustrate what I meant I found that you had this there, but without any indication of it ….. (or at least that I have been able to find). On slide 14 the notes say: Stem and leaf diagrams can be quickly drawn in Autograph by entering the data as grouped data and then choosing stem and leaf on the object menu. This is incorrect – you have to enter the raw data and then group the data set to be able to produce a stem and leaf diagram. On slide 20, ‘Which are of school …’ should be area or areas. How safe is your area? The activities currently in the task are listed in order they occur in the presentation notes, and I have then made comments as to how I think these activities could or should be assessed, using these numbers as reference points, along with some suggestions for other activities which could be incorporated for assessment purposes.


NB: Downloading data has some funny features …. If you ask for a csv file, it does not matter what period you put in, you get all 6 years. If you ask for a pdf file you can’t get all six years. In neither case can you save the file directly – you have to download it and then use ‘save as’: this seems very odd and undesirable. A bizarre feature was that downloading the csv files produced a text list, but closing the window and clicking on the download again produced the file in Excel …. I don’t know the whys of this, but many teachers will not know what to do with the text list just appearing having not been able to save it directly [where they would then open it in Excel]. It is conceivable it is something in my computer, but it isn’t something I have had a problem with before. Other documents: Graphs showing comparison of actual crime stats across 20 counties between 2000 and 2006 [data analysis] and tables from survey in NE in Word doc [survey data analysis] along with Excel file for the NE Survey, with pivot tables [HSIYA north East Survey data] Virtual reality gloves. This is a nice enough context, but it is a very ‘simple problem’ as stated at present not requiring much, if any, insight. It would be good if it could be a little more open – or have a related second stage which is rather more open. The activities currently in the task are listed in order they occur in the presentation notes, and I have then made comments as to how I think these activities could or should be assessed, using these numbers as reference points, along with some suggestions for other activities which could be incorporated for assessment purposes. James Nicholson 2nd October 2006.


Appendix 4 Dotplots of the teachers’ feedback from the second trial

Q1. The guidance materials (online guidance and

teachers’ overview sheet) were clear. Q2. The guidance materials were easy to use.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100

Q3. The guidance materials were written with an appropriate level of detail.

Q4. The online guidance helped me understand the background to the resources.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100

Q5. I knew who I could ask for assistance in using the resources.

Q6. The online guidance helped me understand the importance of the problem solving approach

in teaching statistics and handling data.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100

Q7. The teaching materials (power point presentation, teaching notes, overhead slides, pupil

worksheets and supporting data) were easy to access from the website.

Q8. The teaching notes were written with an appropriate level of detail.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100

Q9. The presentation of the teaching materials was clear.

Q10. The structure of the teaching materials was easy to use.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100

Q11. The delivery options (overhead transparency and PowerPoint) were suitable for use in my

classroom.

Q12. I agree with the levels allocated to the activities given on the teachers overview sheet.

disagree agree1009080706050403020100

disagree agree1009080706050403020100

Q13. The activity took the suggested amount of lesson time (compared with the teachers overview

sheet).

Q14. The activity context was relevant to my group.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100

Q15. The activity context was engaging for my Q16. The pupils enjoyed the work.


group.

disagree agree1009080706050403020100

disagree agree1009080706050403020100

Q17. The pupils were able to understand the tasks quickly.

Q18. The pupil worksheet(s) was written with an appropriate level of detail for my group.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100

Q19. The structure of the pupil worksheet(s) was

clear. Q20. The structure of the pupil worksheet(s)

was easy to use.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100

Q21. I believe the resources had a positive effect on the learning in my classroom.

Q22. I will use the same materials again.

disagree agree1009080706050403020100

disagree agree1009080706050403020100

Q23. If I taught the same materials again I would use them as they are.

Q24. I would use these resources with older students.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100

Q25. I would find resources of this style useful for

older students. Q26. I would appreciate more resources of this

style for use in my classroom.

disagree agree1009080706050403020100

disagree agree

1009080706050403020100


Appendix 5

Referees summarised comments on the four new resources. This document provides a summary of the observations made by the referees after considering the four new resources developed for this project. Some of the changes have been made and others will be made after the submission of this report. Verbatim comments from the referees are available on the CR-Rom accompanying the project report. JN = James Nicholson DC = Doreen Connor PH = Peter Holmes JW = Jo Walker

Online Guidance module The Online Guidance module has not been altered according to the referees’ comments. The module will need to be remade to accommodate these factors. The comments will be taken into account if/when it is remade. Note that the slide references contained in these comments relate to the position of the comment in the slide dump of the module in Appendix 1 for reference purposes. The Comments Generally the guidance has a lot to commend it – the list of details below seems long and I would not want the overview that it is generally helpful and reasonable to use to get obscured JN

Str

uct

ura

l o

bse

rvati

on

s

Comment about the mix of information about functionality and background as well as difficulties encountered in navigating the module JN Speed of transitions too fast JN/DC

Suggestion that could have an automatic version and a manual one as well as having a sectioned and an un-sectioned version. DC

The top or bottom of some of the lines of wording is missing so giving an amateurish feel which is a shame given the high quality of all the rest of the materials DC Lots of factual information. Suggestion of having a downloadable document to support the module containing the same, relevant information JN

The online guidance still appears to relate only to the 4 original problems? DC

Note that the speed of the transition through this module can be, to some degree, controlled by the teacher using the pause and play buttons. This may need to be emphasised/explained in greater detail.


Sp

eci

fic

Pro

ble

ms

Should be “teachers’ notes” in the documentation. JN Typo “relevant” (slide 15) JN

‘freely use’ might be better as ‘use freely’ ? (location unsure) DC Slide 25 is about content, not style. Suggestion that the top and bottom are reversed and the 2 statements appear in a box so that it looks like a screen dump. Arrow unnecessary? JN

Slide 27 moves on so quickly that you miss what was on the screen. JN In Additional resources it refers to “additional formatting” which may be required for data downloaded in csv/xls, but gives no advice about what may be involved. JN

In the additional resources the ‘s’ needs removing from spreadsheets DC

Slide 42 Additional resources the explanation of the drop-down list is different to the actual options. JN

Ob

serv

ati

on

s

‘End of Section’ slide awkward. Menu button unresponsive. Would prefer a return to the menu screen. JN

I disliked the vaguely flashing menu button at the transitions between the sections. This could be improved by a ‘Click here for Next Section’ button and a ‘Return to Main Menu’ button DC

Teaching materials section – first slide is overwritten very quickly. Disconcerting (because they won’t yet have seen the materials) JN Watching the mouse move and reading the text is hard to follow despite having used the website previously. JN

Moving arrow showing the phases of the cycle is unclear –clarification comes late(slide 23) JN

General Comments about the resources: Teachers’ overview should be more specific in exactly what the downloadable material is and how many pages etc it contains DC

The teachers’ overview should also say what certain symbols on the slides such as the speech caption mean i.e. – discussion/questioning point. DC Include a symbol in the PowerPoint where it is appropriate for students to complete their sheet? DC Although I assume you wanted to keep the teachers’ overview one side long I think it should say how & when the pupil sheets would be best used or at least make some suggestions about this. DC

I feel that the Learning Objectives need to be on a slide somewhere to enable the teacher to make these very explicit when delivering the lesson and share them with the learners. DC Could a general question: What have you learnt about the Handling Data Process? be included at the end of the lesson or on the pupil’s worksheet? This would help with the AFL strategies for teachers using the lesson and also focus back onto the learning objectives at the end of the lesson. DC


Generally the new resources gave very positive vibes and I would love to have some classes to try them out on!!!!! DC

You, rightly, have a screen which encourages the students to consider what variables they will be using and what data they will be collecting. But similarly, if we are trying to get the students to appreciate the whole process, there ought to be a screen where they consider what processes they are going to use in the analysis This step is always done for them. PH In all the modules the process stage seems to imply that they already know about the technique/concept being used and are just being asked to use it. I did not see anywhere where the technique/concepts were being specifically taught. One of the aims of teaching through the four stage cycle is that the techniques are themselves taught in the context, not just used. PH All the modules leave a lot of development for the teachers to do – HFHFHH in particular. PH Response We will try to incorporate some of these changes into the materials when

further development occurs.

World Population

Comment that it is unclear how the teachers are to use the PP presentation – in one sitting or more. PH

Not emerged through trials. No immediate action.

Activities Much simpler than the main body of work – confusion about target group? PH Lack a clear reference in the general supporting materials – where they could be best used. DC

Querying where the teachers’ notes are for these. DC

Make references more obvious.

Activity 1 Good task to get them thinking and realising that different sources give different answers. JW Magnitude and accuracy of figures not population growth PH Do they do this in geography? JW

Activity 2 Much simpler than modelling in presentation. Would be helpful to consider different values of p. PH Like this activity. It could be used as a stand alone task to introduce modelling. JW

Consider extending activity.

Slide 4: How would we plot a graph to show this data? JW Why does time go on the x-axis? JW

Make references to worksheet clearer

Co

mm

en

ts

Slide 16/17: Comment that guidance needs to be clearer on converting csv file to excel. PH

Investigate ICT possibilities.

Activity 1 q2 Use of pyramid misleading in terms of size. Could include “NOT TO SCALE” and some calculations showing China is more than 20x the size of the UK PH

Teachers’ notes need a comment that web access would be useful. JW

Activity altered.

Sp

eci

fic

Pro

ble

ms

Slide 4: simulation – either explain what is going to happen i.e. each 20 minutes the number of bacteria will be counted, or it should be advanced to the next line of the table on a click. DC

Altered


Slide 12: This button to the primary/secondary data did not work for me. PH Version of pp?1

Slide 13: was one where the text overlaid and was slightly misaligned. PH Version of pp?

Activity 2 In the third paragraph of section 5 the word loosing should be losing. PH

The first bullet point should include the phrase ‘on average’ PH

Altered

The x scale on the graph which appears on the pupil sheet should go up in 20’s and the graph on the PowerPoint as well. DC

Altered

Slide 24: faulty buttons. PH

Website reference difficult to use PH Consider supporting web use.

Slide 26: A key is required to interpret this graph. JW Key added

Activity 2: Suggestions regarding how to combine a birth and death model as well as varying p provided. PH

Consider including

Refers to fish in a pond. This could be people or bacteria so as not to introduce another factor. PH

Altered

The last 2 bullet points. In 2 years it grows by a factor of 2.25 – not just double. You could ask (after bullet point 2) how much does the population grow on average in one year; then bullet point 3 is how much on average does it grow after 2 years. PH

Altered

Slide 2: Could the population clock and questions be simultaneously displayed on slide DC

Investigate

Slide 3: questions could appear one at a time? DC Altered

Slide 4: notes - Plotting line graph I prefer asking to telling. JW Altered

Slide 7: Students may consider epidemic and war which will effect the curve. When modelling should these be included or ignored? Eg 1918 – End of WW1 – how many died in this war? – Spanish Flu – Killed approximately 50 million worldwide. JW

Consider incorporating

Suggestion for further activity sheets on: • Time series on population growth • The effect of household size • Interpreting population pyramids PH

Consider including

Slide 6: The hint should only appear on a click not with the question DC Altered

Slide 16/17: There needs to be an activity sheet for this page – or some hint at graphing the data using the excel wizard. They could also return to this after the work on slides 19 & 20. PH

Consider including

Slide 24: Suggestion that more focus is needed in moving to household size (off the main thrust of the investigation) PH

More information is needed on the choice of further activities DC

Develop this further

Slide 26: A key is required to interpret this graph. JW Added

Su

gg

est

ion

s

Suggestion that a link could be included to the WHS website where there are detailed instruction sheets for using Excel and Autograph JW

Consider including

1 PH uses a Macintosh. There are sometimes conflicts with Microsoft software. This has not emerged as a problem during trialling.


What’s in your bowl?

Generally a very polished resource. DC The materials are certainly accessible to the students of all abilities and I would expect to see differentiation by outcome in that there are open-ended possibilities in the context and better students will be able to develop aspects much further than the average student. JN The context is relevant – thanks to Jamie Oliver. The activities are likely to appeal to teachers and to students, and the data is easily understood conceptually, but there is scope for analysis on different levels e.g. looking at how much variation there is between brands of similar cereals compared with the difference between types of cereal, and one could get into investigations regarding costs as well. JN Like the way this is related to a current issue. DC Emphasis on figures in the right range for ‘healthy’ and you have to compare relative healthiness. Have omitted to consider the quantity consumed as a relevant factor. Need to think about the manner used for sampling (the statistics are strange if a random sample not used) and how much people eat particular cereals. The health of the nation’s breakfast is better measured by things like the mean sugar content actually eaten at breakfasts. PH Fairly uncritical attitude to statistics provided by external suppliers. Breakfastcereal.org They have an agenda. How were the data collected, what do they mean? Similar but different concern with eatwell.gov Lots of information here, NO statistics. http://www.eatwell.gov.uk/healthissues/foodintolerance/foodintolerancetypes/ to see a list of possible allergies. It is absolutely frightening until you remember that the proportion of people having some of these allergies is miniscule. This is another important statistical point in making decisions. PH The structure is reasonably simple, and generally the accompanying notes give a helpful commentary on what is going on and why – and what the students are expected to be able to do. JN

Co

mm

en

ts

There is plenty of scope for collecting primary data here as well as looking at secondary data – perhaps the context makes it harder to construct reasonable and interesting hypotheses that students will be able to test e.g. is there a difference between certain groups? Does price affect the uptake? …. but I think there are some valuable activities in here. JN

There are some helpful points made

here. Investigate further and

consider including.

Clearer structure needed for pupil worksheet – which graphs are appropriate to draw? JN

Is there any data on total consumption of different cereals available? I am not sure where the ‘popularity data’ comes from and therefore how reliable it is an indicator of what people will think it represents. JN Slide 3: Query objective in teachers’ notes. Stated that breakfast is important rather than given reasons. Suggestion for including ref to objective information about breakfast as well as some figures on diabetes. PH

Investigate

Su

gg

est

ion

s

It is worthwhile reading through the PowerPoint notes (and print them off) before going through the PowerPoint presentation. PH

Ensure this is highlighted


Pupil sheet: needs to be printed in colour to be effective. When printed in black and white the red and green look exactly the same intensity. This could be a major problem when it comes to schools using this sheet. DC

Look into this

on page 3 ‘your graphs’ JN Altered

Not sure this one would take 3 hours? I’d say 2 hours for this one. DC Altered

Why have the coloured bars in the traffic light system got arrows on different ends? DC Slide 4: Can students be expected to answer this without checking at home? JN Include as

h/w?

In last bullet it should be proportion. JN Altered

Nice to have national vs. class data comparison. Could you provide an excel sheet to help? JN

Create this

Slide 5: Misquote. Should read 98% (not more than 95%) buy breakfast cereal. Questioning data. How did they get these data? PH

Altered

Slide 9: Salt overwritten. Missed teaching point with the fibre – more is better. The scale is not ‘reversed’ as it says in the notes. JN

Altered Emphasise later

Trouble with transitions. PH PP version?

Suggestion about rephrasing the guidance on sodium/salt content. Altered

Slide 12: links go to the blank answer table to be filled in (slide 15). Similarly slide 24 links directly to the PSA cycle on slide 27. PH

PP version?

Suggestion that the problem of how much is eaten could be included here. PH Consider including

Slide 13: How do the brands differences compare with the type differences? JN Consider including

Slide 15: The notes talk about efficiency and sampling. See above for comments on the strangeness of sampling in this context. PH

Reference removed

Slide 20: At the end of the first line of notes under Activities the word is is missing. PH

Altered

Slide 21: Since even the mean and median are not particularly informative, the mode is certainly not useful. I would not even refer to it. PH

Altered

Sp

eci

fic

Pro

ble

ms

Slide 22: Query about the use of the pie chart and the inappropriate manner these data are selected for the slide. JN

Altered


How old is your height?

Lovely historical photos and the historical context is presented extremely well. Interesting data used. DC The notes on the PowerPoint need to refer to the pupil sheets as it is unclear when they are meant to be filling parts in. DC I am not sure that these lessons would take three hours. JW I would use this activity with my groups. JW No substantial mention is made of the shortcomings of the whole process and method. Specifically the two types of error that the inspectors could make (small children old enough not being allowed to work; tall underage children being allowed to work) PH The fact that the data are being read backwards is problematic – ie they were collected to give heights for given ages and being interpreted as ages for given heights. PH The word data is consistently used as a singular noun. PH There is too little mention in the main part of the project on variability and the disadvantages of using the mean. Suggestion for including the US growth charts. PH The aim of this unit appears to be the planning and discussion elements since apart from copying and writing what they notice they do not do any graph work or averages. JW Slide 8: Comment on the problem of using teeth to estimate age. PH Slide 14: Why is the mean a good average to use here? Was the data in each group symmetrically distributed? Is the idea to recognise that heights generally form a symmetrical distribution? JW The results for the years are skewed so many pupils will not pick them up. JW Pupils might guess the mean because of the detail of the answers but at NC level 5 to 7 I feel that it is unlikely that pupils would know that heights are symmetrically distributed. JW Slide 21. The last question mentions able to work. Query whether 8/9 boundary for being able to work at all or to be able to work up to 12 hours a day, ie the 13 year-olds? PH It is not the line that determines whether they are allowed to work but the mean heights at the 8/9 and 12/13 boundaries. PH Slide 24: This first cycle is far too simple for many students in its treatment of variability and the problems of reverse reading of the data. PH

Com

men

ts

Slides 28/9: I could not get directly to these slides from the presentation. The work seems to imply prior knowledge of medians/quartiles/box plots since they are not specifically taught here. Box plots are not really useful here since the sort of questions we want to answer are (a) what proportion of 8 year-olds are over 3’11” so could be working illegally and (b) what proportion of 9 year-olds are under 3’11” and so would be stopped working legally PH



consider including.


Slide 2: Helpful to provide urls for more information on the Industrial PH Added

“25 thousand in 1772 to 455 thousand in 1851” might increase impact DC Altered

Slide 3: The Activities note that says Discussion points from the slide could be more helpful. PH

Altered

Using an audio file to accompany the Newspaper cutting would be a nice touch and provide something for the auditory learners! DC

Consider later

Slide 4: Fuller references would be helpful. PH Added

Slide 14: In the discussion bring out that it is important to have large numbers in the sample at the 8/9 and 12/13 boundaries since these are used for the legal decisions about working. PH

Added

Could add a class activity to stand in height order and look at the shape created. JW

To add later

Slide 16: Why not ask them what single number could be used to represent the group and explain why they chose it? JW

Altered

Slide 18: For the less able it may be useful to brainstorm all the graphs they know and then ask which will allow them to compare two different variables. JW

To alter later

If you are going to give them the graph to save time then ask them how they would draw it. JW

To alter later

Slide 19: The points on the initial graph (the pink ones) would be better a little larger DC

To alter later

Slide 20: Expand on the nature of the discussion of the likely age of the girl. Again there is a need to emphasise variability and not just point estimates PH

To alter later

Some pupils may suggest error bands as in baby growth charts or looking at growth charts may enrich the discussion at this point. The best ones on the web are on the US site www.cdc.gov/growthcharts DC

To alter later

Slide 21: Could have a blank graph and ask for some heights from the class to plot. JW

To alter later

Su

gg

est

ion

s

Could do with a follow up activity. I have included a suggestion. It could be adapted for this activity by giving them a situation to investigate. JW

Consider adapting later

Slide 2: Under Activities the second sentence does not make sense. PH Altered

The notes say ‘Activities be soft discussion’ – I’m not sure what this means? DC Altered

Slide 3: needs a reference about the illustration. PH Added

Slide did not come up when I was going through the ppt as a presentation PH PP version?

Slide 4: The last 2 questions and the illustration in the background came up all at once and were not easy to read. PH

Altered

Slide 10 Buttons not working PH PP version?

Slide 18 I had problems with overwriting on the slide. PH PP version?

Slide 21. Unable should be able at the end of the first paragraph. PH Altered

Slide 25 There needs to be some explanation of the dotted line. Is it the heights of today’s boys/girls or both? PH

Added

Slide 26 The button that says Extra Processing took me to a slide that said End of Presentation. PH

PP version?

Sp

eci

fic

Pro

ble

ms

Pupil sheet: table might benefit from a column for the mid points included. DC Altered


How Far? How Fast? How High?

The structure is reasonably simple, and generally the accompanying notes give a helpful commentary on what is going on and why – and what the students are expected to be able to do. JN The context is hugely relevant - with the 2012 Olympics coming to London this is likely to appeal to teachers and to students, and the data is easily understood conceptually, but there is scope for analysis on different levels e.g. comparing % improvements over time. JN The materials are certainly accessible to the students of all abilities and I would expect to see differentiation by outcome in that there are open-ended possibilities in the context and better students will be able to develop aspects much further than the average student. JN Given a broad context, and access to a plentiful source of detailed data, there is considerable scope for making hypotheses and testing them. There is a temptation however to use the data to generate the hypothesis and then to use the same data to test the hypothesis. I think this is a problem in some of this work - which seems to push students towards it at times JN There is not huge scope in the way this is presented to undertake the full cycle - to go back and start a second cycle, which I think is important [partly important per se and partly because this is the bit which is so often missing in other materials] and I think there are some rich opportunities here. JN Slide 7: Highlights the danger in suggesting a hypothesis needs to have a ‘because’ JN Slide 9: what is the same rate of improvement? is it proportionally or absolutely? This hypothesis does not have any 'becauses' with it. JN Slide 10: (and elsewhere) under objectives. You say that children should learn where they can get data from to investigate their hypotheses. In fact they are given one internet source to use – this does not teach them where in general they might get appropriate data nor how to get them. PH The website opened behind PowerPoint for me - if this is deliberate, or a requirement of the programme structure then it needs to be mentioned in the notes. JN Slide 11: - nice open-ended task; can do few / many of them over varying periods of time. Can also consider different types of event, and have some differentiation by outcome. JN Slide 13: this is where I think the hypothesis comes from the data - I would expect students to know men's records are better than women's but I would be amazed if they thought it was 1 second for the 100 metres. JN Slide 25: not all the WRs are actually being used. JN Slide 32: I am not sure of the comparison of gradients offered. JN

Com

men

ts

The two routes seem to relate to different content is this made explicit anywhere or possible content coverage listed?

o Hypothesis route; using what a hypothesis is, averages…… o Questions route; scatter graphs, lines of best fit … DC



consider including.


The records are very tied into the current 2006 position so will this mean the PowerPoint will need constant updating as records are broken or it might be better to say ‘In 2006 the world record was’ on the slides and mention in the Teachers overview that the teacher could update the PowerPoint to the current position if they wished to? DC

Alter later

The bulk of the module is on various aspects of fast rather than long or high. This module is more open-ended than the other ones. I think you may need to recommend that teachers work through the hypothesis route in detail first; then the students can move to the other options with some structure under their belt. PH

Alter later

One of the things about this investigation is that it only looks at 'exceptional performances' - resulting in a new world record. There would be some very interesting development work available to look at other performances - how many people do better than the precious record marks? JN

Consider later

Slide 4: Would it be possible to ask this type of question before starting to tell the students what the records are e.g. do you thing anyone in the world could jump over the bar of a full-size soccer goal without touching it? any woman? JN

Alter later

Slide 6: the notes should give some indication of who should follow the Hypothesis and Question routes. Is one for more able or one for a certain age group or pupils with specific learning needs? They do seem to have different content? DC

Alter later

Slide 15: takes 12 clicks to work through - why not have first two on clicks and then 'run' the rest - you can stop the simulation if you want. I think it would be good to show where the mean comes from. JN

Temporarily altered, look again

Slide 17: Activities. Not only might you consider expanding from continents to countries but you also need to expand over time. PH

Altered

Slide 21: in the activity section - 'who and why might be interested in them?' does not read well - can you break it into 2 sentences for the who and the why? JN

Altered

Pupil sheet: ‘What do you think is happening to the world records for your chosen event?’ - they may not have decided to investigate this and done one of the men can run faster than women type questions so you may want to say this is an example and for them to write their known hypothesis/question down and explain it. DC

Alter later

Su

gg

est

ion

s

Slide 32: I don’t think it is reasonable to use only the period up to the last WR was created in doing the calculation – why is it not up to the current date? JN

Consider later

Slide 5: The sentence below the pictures is tautology. Interesting questions arise from why the women's record has lasted so long - was the 'record' a drug fuelled / enhanced legacy of the Eastern European regimes? The men's record was broken in 1994, 1996 and 2005 - not 'a few times during the last 10 years' according to this slide but I don’t think these data are correct.. JN

Altered

Slide 6: Lots of data is plural so it is Lots of data have been collected…. PH Altered

link to 'questions' then does not use 'go back' to do that ..... JN Consider later

Slide 8: Activities. It should be practise (verb) not practice (noun) PH Altered

Slide 9: ‘gotten smaller’ might be better as ‘reduced’ DC Altered

Slide 19: in the slide notes, activities 'right' should be 'write'. in points to note, hypothesis should be hypotheses. JN

Altered

Slide 22: the web address given is not hyperlinked - but not showing all WRs in bold! JN

Altered

Sp

eci

fic

Pro

ble

ms

Slide 28: this graph needs to show a broken axis at least on the y-xis. JN Alter later


Appendix 6

Table 1: Revised Taxonomy and the Problem Solving Approach

Stage of problem solving approach

Description Level in taxonomy

1. Specify the problem and plan:

formulate questions in terms of the data needed, and consider what inferences can be drawn from the data; decide what data to collect (including sample size and data format) and what statistical analysis is needed

Remember: recognising; recalling Understand: interpreting; exemplifying; explaining Apply: executing Analyse: differentiating; organising Create: planning; producing; generating

2. Collect data

collect data from a variety of suitable sources, including experiments and surveys, and primary and secondary sources

Remember: recognising; recalling Understand: classifying; comparing Apply: executing; implementing Analyse: organising Create: planning

3. Process and represent the data:

turn the raw data into usable information that gives insight into the problem

Remember: recognising; recalling Understand: interpreting; exemplifying; classifying; summarising Apply: executing; implementing Analyse: differentiating; organising;

4. Interpret and discuss the data:

answer the initial question by drawing conclusions from the data

Remember: recalling Understand: interpreting; exemplifying; summarising; inferring; comparing; explaining Analyse: differentiating; organising; attributing Evaluate: checking; critiquing Create: generating; producing


Appendix 6

Table 2: Mapping learning objectives into two way table Cognitive Process Dimension

Remember Understand Apply Analyze Evaluate Create

Fact

ual kn

ow

ledge

To be able to list the four stages of the problem solving approach To be able to distinguish between discrete and continuous variables To know the implications of variable type for the data collection method (collection sheets)

To be able to identify and list what constitutes each stage of the problem solving approach

To be able to calculate summary statistics

To be able to identify the four stages and explain how they fit together as the problem solving approach

To be able to compare numerical data values

To be able to evaluate the relevance of conclusions to (wider) population

Kn

ow

led

ge D

imen

sio

n

Conce

ptu

al kn

ow

ledge

To be able to list the four stages of the problem solving approach

To be able to give an overview of the problem solving approach and its purpose. To be able to summarise the problem To be able to summarise requirements for data from the plan To be able to explain what can be inferred from each numerical summary/table and graph/plot To be able to articulate conclusions as a result of discussion

To be able to distinguish between the need for primary and secondary data To be able to use summarised data (including plots) in support of the conclusion

To be able to organise data into appropriate tables/summaries

To be able to explain what can be inferred from each numerical summary/table and graph/plot

To be able to identify variables that may be relevant to the questions/problem To be able to articulate conclusions as a result of discussion


Pro

cedura

l kn

ow

ledge

To be able to identify and list what constitutes each stage of the problem solving approach To be able to recognise availability of primary and/or secondary data To know the implications of variable type for the data collection method (collection sheets) To be able to organise data into appropriate tables/summaries To be able to construct summary tables appropriate to the data and the questions from the plan To be able to calculate summary statistics appropriate to the data and the questions from the plan To be able to relate the plan and questions raised to desirable graphical presentations To be able to construct appropriate graphical representations

To be able to organise data into appropriate tables/summaries To be able to construct summary tables appropriate to the data and the questions from the plan To be able to calculate summary statistics appropriate to the data and the questions from the plan To be able to relate the plan and questions raised to desirable graphical presentations To be able to construct appropriate graphical representations

To be able to undertake data collection appropriate to the above To be able to construct summary tables appropriate to the data and the questions from the plan To be able to calculate summary statistics appropriate to the data and the questions from the plan To be able to relate the plan and questions raised to desirable graphical presentations To be able to construct appropriate graphical representations

To prepare a plan for organising and recording the data To determine an appropriate source(s) for the data To be able to design data collection sheets appropriate to the variable type To be able to organise data into appropriate tables/summary

To be able to construct summary tables appropriate to the data and the questions from the plan To be able to calculate summary statistics appropriate to the data and the questions from the plan To be able to relate the plan and questions raised to desirable graphical presentations To be able to construct appropriate graphical representations

To be able to construct/develop a list of data that needs to be collected To be able to design data collection sheets appropriate to the problem questions


Met

acognitiv

e k

now

ledge

To be able to identify the four stages and explain how they fit together as the problem solving approach

To be able to give an overview of the problem solving approach and its purpose. To be able to relate inferences from tables and plots to questions raised and original problem decisions To be able to make decisions/come to conclusions about the original questions

To be able to construct summary tables appropriate to the data and the questions from the plan To be able to calculate summary statistics appropriate to the data and the questions from the plan To be able to relate the plan and questions raised to desirable graphical presentations To be able to construct appropriate graphical representations

To be able to evaluate the relevance of conclusions to (wider) population

To be able to decide if data was appropriate for the questions To be able to relate inferences from tables and plots to questions raised and original problem decisions To be able to make decisions/come to conclusions about the original questions To be able to evaluate the relevance of conclusions to (wider) population

To be able to articulate and construct questions that the problem raises To be able to decide/conceive if additional problems/questions are raised


Appendix 7

Comments on assessment from consultants (a) James Nicholson comments on assessment of materials Scene of Crime Evidence: The activities pupils undertake at present are: 1. Discuss why a coin was placed next to the footprint before the photo was taken. 2. Discuss what the footprint might say about the thief – what relationships might there be, and what would you expect to see, including how strong are these going to be? 3. Choose which relationship they want to investigate [height, gender, age], and whether to look at correlations or averages for only those with that foot length. 4. Describe what they expect the relationships to be, along with reasoning why this might be the case. 5. Decide what data to collect and how to do it – random samples? Sample size? 6. Discuss the use of secondary data. 7. Collect data, either as primary data or extract it from secondary source. 8. Represent data graphically – scatter diagrams or stem and leaf diagrams for distributions using only 25 cm footlength. 9. Discuss what it tells you about your hypothesis. 10. Discuss what has been done, limitations of evidence in this case. 11. Discuss possible extensions / improvements. 1 – 6 are really integral to getting the task understood sufficiently that the teacher is confident the pupils know what the context is, and what they are expected to do, so no formal assessment. 7. Can be assessed by traditional methods, but it may be that the effort in determining if data has been extracted correctly / validating the data collection is not reasonable for the teachers, in which case the assessment would be concentrated on what they do with the data: this would have the rider that if this part of the process is discovered to be flawed then the credit derived from it is at risk [and should be judged on the merits of the individual situation]. 8. Credit for appropriateness of representation and accuracy of graphs – including labelling etc. 9 – 11. At the moment there is no ‘report’ as there was in WW and HSIYA, which means at present there is little to base any assessment on: it would be possible to extend the task to requiring a brief written summary to be submitted for the police files, in which they have to summarise their findings – in this case, suitable language relating to the degree of uncertainty they consider to be in their conclusions would be an important factor in the rubric. Where’s worst? The activities pupils undertake at present are: 1. Discuss which areas of the school need improving. 2. Work out maximum and minimum scores possible on a simple EQI. 3. Consider what areas of the school to look at, what criteria to judge environment by, the range of scores to use and how many people to survey. 4. Data collection – either from photos, by going round school or from memory. 5. Calculating total scores from survey results, creating stem and leaf diagram to represent results for each area. 6. Describe stem and leaf diagrams and explain the distribution shape contextually. 7. Write a mini-report, including exploring aspects of continuing the cycle by looking at plan / collect in the light of the work undertaken. 1: This is the introduction to the problem, with class discussion, so no formal assessment.


2: Low-level activities designed to ensure pupils understand what the process means at a very basic level, so no formal assessment. 3: At the moment, a considerable number of possible criteria are available for inspection, but a number of these might be considered to have very substantial overlap e.g. boring / interesting and monotonous / varied or dirty / clean and polluted / unpolluted, and it is hard to see what others could be added which are distinctively different from those listed – graffiti, smelly, dangerous have elements which are not quite covered. This is a possible assessment activity, but I think it would need to be structured carefully – a smaller list of suggestions has already been given in the example: I would perhaps replace untidy / tidy by dirty / clean in this set of four, and then discuss whether polluted / unpolluted would be a suitable addition to this set [in a school context I don’t think there is much difference, and that including similar criteria will place disproportionate weight on one aspect of the environment]. Pupils could then be asked to nominate 2 or 3 more to add to the list, and ones which are really just variations on an existing criterion would not gain as much credit as ones which are reasonable measures of environmental quality – and suggestions which are not reasonable measures of EQ would not gain credit. [I am not convinced that this is really appropriate for assessment purposes – I think there may be merit in these changes for the activity’s sake, but it is difficult to see how it would be managed as a ‘formal assessment’]. 4: As the activity is structured currently, I don’t think it can be assessed – there is no guidance as to how to rate any environment on any of the scales. It could have some slides available, with assigned scores according to various criteria so pupils have some point of reference against which to make judgements. If these were provided it would then be possible to provide other slides which pupils have to rate by the same criteria – but the marking would have to be relatively crude – some leeway would be required but a ‘competence’ standard would be possible. 5: This could be assessed by traditional methods – can pupils correctly process the raw data to get EQI scores and construct stem and leaf diagram, either manually or using ICT. I would prefer the activity to include a broader discussion of suitable ways to process the available data, and for the pupils to then make a choice which they justify, which would provide a much richer assessment tool. 6: The activity in worksheet 2 stands alone as an assessment activity – however interpreting the stem and leaf plots requires that the pupils know what criteria were being applied, and I don’t think this is clear. I would have guessed that it used the full list of criteria on slide 9, with a scale of 0 to 4 or of 1 to 5 – but with no idea which [I think it is unhelpful that the example in slides 7 and 13 uses 1 to 5 and elsewhere, including the Excel sheet provided for data collection it is 0 to 4]. However, that gives ranges of 0 to 52 or of 13 to 65, neither of which can be the context of the data since there is a 7 and a 55. At present there are 43, 33, 38 and 42 data values in the 4 stem and leaf plots. I think this makes it very, very hard for pupils to interpret. It would actually be much easier if there was a much bigger difference in the sample sizes because it would be obvious that they were different; obviously having the same sample size for each would remove this problem, as well as implying that the same people had judged all 4 environments [which is after all the context of the problem they are looking at] avoiding the can of worms which exists if different people have judged the 4 environments. I have tried to think what it is that you want to assess in this activity, and I think there is more than one thing, which might be better addressed separately – I think there would be an argument for showing a number of pictures of the same place [or similar places] in different states so the comparisons are like for like, and then showing a number of distributions for the one picture, but for different groups of people who would be expected to have different views on it e.g. teenagers, mother with young children, senior citizens etc. Both of these could be assessed on a partial credit scale based on reasoning provided , but many pupils find this very hard to know what is expected of them, and I would suggest one example of each using the ‘match these cards ….’ style currently in the materials, which would assess the basic level of competence, followed by an open-ended one to assess higher levels. 7: Worksheet 1 is a sort of a mini-report, but the space available in some sections is not appropriate. If all the class are involved in collecting the data, there is only going to be one answer on everyone’s report as to the areas to be looked at, the criteria, the scoring scale and the sample size, so I think this part needs to be in a separate document to the ‘report’ document.


In its present structure I think the opportunity for differentiation is limited, as all students are directed towards compiling stem and leaf diagrams of qualitative data, and any fine-grading of the outcomes from that process will be extremely problematic I think. A more open discussion of possible analyses, and of possible quantitative measures, would provide better assessment opportunities in the report stage. For it to be manageable for teachers, I think a pro-forma structure to the report is advisable with a number of marks to be awarded in each section of the report – on a partial credit scale. It would be good if this could be structured to include some discussion of the measures to be taken, and even some informal references to comparison of the costs of different measures. How safe is your area? The activities pupils undertake at present are: 1. Discuss what crimes make headlines and why. 2. Discuss what the crime categories mean in terms of offences such as manslaughter and shoplifting. 3. Discuss why the 6 categories listed are the government as ‘key crimes’. 4. Discuss what information is needed to answer what crimes people are worried about in your area, and whether people should be worried about them. 5. Think about personal response to survey on how worried you are about various crimes. 6. Discuss which key crimes to be asked about, and why other information about age, gender might be collected also. 7. Collect data – conduct survey; extract relevant statistics from the database provided. 8. Analyse data 9. Comparison of perception and actual crime figures. Use of rates per 1000. 10. Summary report. 11. Think about reviewing the cycle – could the process have been improved?, could other things be investigated [age or gender effects, are rates increasing or decreasing, how does area compare with national picture? 1 – 6 are really integral to getting the task understood sufficiently that the teacher is confident the pupils know what the context is, and what they are expected to do, so no formal assessment. 7 and 8. These could be assessed partly by traditional methods. I have some reservations about data collection in this context, for the reasons identified in the notes for slide 6, and practical issues about who is going to take part in the survey – is it just the students in the class doing the task?, will these be independent of one another? [an anonymous written response would help with this] and will the responses from a fairly homogeneous group such as a class provide much variation in the responses to these questions? The survey data by region from CaS is very useful here as a tool, but see notes about downloading below and in sheet showing tabulations for North East. I would be inclined to have the data available in Excel as well as in a pdf, so students can analyse it more easily [even without using pivot tables, they can sort by gender and then score on any crime and count numbers quickly, or use other Excel functions to count …] You want them to extract the information in usable form, but without too much tedium. Likewise with the actual crime statistics – I think a number of different levels of aggregation would be good, available in different pages of a workbook, so teachers / pupils can choose what level of detail they want to consider – but it should be possible for a user to quickly identify the crime data for a number of years for a region which matches the survey data – so for the North East I would like a table which gives the crime stats per thousand: the pdf for crime stats gives totals in 04-05 or 05-06 by county, with the population, so I could calculate 2 years stats for the NE region to compare with the survey data, but not without quite a lot of work, and a fairly sophisticated process. However, it is this question [perception / reality] which seems to be at the heart of the task, and it would be good if the data were more accessible. If you adapted the data to provide more detail readily accessible, then you would open up a richer assessment opportunity – you could ask some quite directed questions to evaluate basic procedural


competencies, followed by some more open questions for which a partial credit rubric rewarding different skills could be devised. 9. This can be done on a number of different levels so a partial credit rubric needs to be developed to identify different standards of performance. 10 & 11. A pro-forma structure to the report is advisable with a number of marks to be awarded in each section of the report – on a partial credit scale e.g. discussion of the relative importance of crimes in terms of the impact on your life. This pro-forma should include the review / extend aspects of the cycle. Virtual reality gloves. The activities pupils undertake at present are: 1. Discuss what virtual reality is. 2. Discuss the notion of calibration of interfaces. 3. Discuss possible combinations of writing hand / control thumb, and how likely they are to occur. 4. Decide what data to collect, and how big a sample to collect. 5. Collect data. 6. Analyse data 7. Summary report to manufacturer. 8. Think about reviewing the cycle – could the process have been improved? Could it be extended? 1 – 4 are really integral to getting the task understood sufficiently that the teacher is confident the pupils know what the context is, and what they are expected to do, so no formal assessment. 5. Pupil completed data sheets could be included with summary report, but I would grade it pretty crudely – if the blank data sheets have been provided it is then a matter of whether they have worked legibly and thoroughly. 6. In the analysis, I ran into a problem using the tool provided to draw a pie chart: I could not copy and paste it into Word, and if I printed the page out directly the pie chart was well over twice as high as it was across. Of the two representations shown as illustrations, the comparative bar chart shows the nature of the 2 way table – I was very surprised to see the two way table illustrated as a pie chart, and I am still not convinced that it is appropriate: the form that it is in on slide 14 where colour is used to identify one categorisation and texture is used to identify the second I could live with, but these 4 colours provide a graph which I think you have to spend a long time looking at to get the information. 7. This is pretty well a right / wrong question of proportional reasoning unless pupils get into some discussion of uncertainty. However, a big question which is not addressed is whether data from the general population is appropriate as a basis for characteristics of a very specific and specialised profession. One curiosity is that the report pro-forma does not have space provided to answer the question posed in the problem, namely recommending how many of each type the manufacturer should make in a batch of 100 sets of VR gloves. Thought could be given as to whether any sort of prompt might be given so that pupils can be expected to touch on issues of uncertainty in the prediction, otherwise the assessment is going to be very routine. 8. One problem with this particular context at this stage [which is one of its strengths elsewhere!] is that the problem is very narrowly defined, so there is not a whole lot more which can be done, unless there was a slightly wider remit here that the company were going to expand into another area – see if they have learnt something from this process that they could apply in planning a similar investigation. James Nicholson 2nd October 2006.


(b) Peter Holmes comments on assessment of the problem solving approach Assessment of the Data Handling Cycle There are many aspects to this. Various forms of formative assessment can be fairly easily devised and used by teachers as part of the teaching and learning process. Difficulties only really arise when we consider the need for summative assessment that can be objectively applied to a large number of students with a wide range of ability in many different schools and colleges. As well as being valid and reliable it is a basic principle of such assessment that it should encourage good teaching and not distort the process. Any solution we come up with has to be practicable and fit for purpose. At present the sole means of assessing a student’s ability to use the data-handling cycle in practice is via the required statistical coursework project. This is part of their final assessment. The two major objectives to this are (i) it has caused a lot of resentment and resistance among mathematics teachers and (ii) it has turned out in practice not to be fit for purpose. The first of these objections is often linked with a statement that the course work takes up too much time. Since it carries 10% of the marks awarded for GCSE mathematics, it would appear reasonable that it might tame of the order of 10% of the mathematics time over the whole course. Anecdotal evidence is that it does not take up as much time as this and the real resistance is rooted elsewhere. The second objection is more substantial. If the teaching were done largely through the data-handling cycle, as envisaged in the programmes of study, then students would amass experience throughout the course of carrying out a statistical investigation. Any one of these, or a different topic of their own choosing, could then be developed into the final assessed piece of coursework. Instead, most schools find their students doing one of a small number of topics for their assessed coursework and there are real problems of so much cooperation that it is difficult to distinguish the individual’s ability in the project report. Examination boards have been reported as giving rules such as if you want to get a particular mark then you must include a specific technique. What do we want from students? The current method of assessment implies that we want all our students to be able to carry out the whole statistical process on a specific topic and to communicate their work and conclusions in a report. We are also presumably hoping that the student could transfer these abilities to other topics. Altogether this is a set of high level skills. Even if we expect this from our best students, is it truly what we want from all our students? It may be possible to separate out the communication aspect from the statistical process. Apart from that what do we want? Os it enough that the student (i) has experienced the statistical process in a few guided examples; (ii) knows the process and can suggest how it might be applied in some different contexts; (iii) can constructively criticise reports of statistical investigations done by others? For some students can we expect more than a basic understanding of the process or even only an appreciation of the process? Maybe we would like all of the above, but at the moment our assessment focuses on the highest level of integration and communication and may not be appropriate for assessing the lower level of skills implied by the is it enough questions above. What can we do? If we can persuade teachers to teach through answering statistical problems holistically, as envisaged in the Programmes of Study, then some of the levels of experience described in the previous section could be answered by the teacher using tick boxes on a questionnaire. It would be possible for an


individual student to build up a portfolio of work showing experience and competence with all aspects of the cycle. These could be assessed internally and moderated externally. It is possible to devise questions to assess understanding of various aspects of the data-handling cycle (See Holmes, 2002). These are not the type of question currently included in GCSE papers in mathematics, and they may have to be part of a different structure. AQA Chemistry has an interesting paper with questions flowing from a major context. This idea could be adapted for data-handling and statistics. What structures? Currently the structure for the objective summative assessment of data-handling in mathematics is in two parts. There are largely technique oriented questions on the examination papers and there is the individual project which carries half of the data-handling marks (10% of the total mathematics marks). One possibility is to replace the project by a portfolio, tick boxes for having had various experiences and a short paper (maybe online) like AQA chemistry which could include questions requiring extended answers, and questions requiring criticism of other statistical reports. If we did this we might have something less open to abuse, having less resistance from mathematics teachers and better able to distinguish different levels of understanding the statistical process. Reference Holmes, P (2002) Assessment: New Ways of Pupil Evaluation Using Real Data Teaching Statistics 24.3 pp 87 – 89. Peter Holmes 23/12/05


Appendix 8

The online assessment

1 2

3 4

5 6


7 8

9 10

11 12


13 14

15 16

17 18


19 20

21 22

23 24


25 26

27 28

29 30


31 32

33 34

35 36


37 38

39 40

41 42


43 44

45 46


Appendix 9

Teachers’ comments on online assessment

(a) Yvonne Sutton: Head of Maths and AST at Hillcrest School, Birmingham Outcomes of discussion about assessment:

1. How will teachers get feedback? Yvonne commented that it is important teachers get meaningful feedback from the assessment. The possibilities we discussed were:

a. Providing a spreadsheet of the answers that students completed, this will allow the teachers to compare individual responses and see the range in a group. This could also be built with a mail merge option which allows teachers (where appropriate) to merge the responses with a written version of the test. This could be either emailed to individual students, printed out so they can use it in class or used on screen.

b. It would be an interesting (and useful) exercise to try to separate the questions which contribute to MA1 (using and applying maths) and those which apply to MA4 (handling data).

c. In addition, we discussed the possibility of providing feedback through the medium of a checklist for all of the marking criteria (similar to the four headings and sub-headings that are currently used) and checking those for which there is some evidence for.

2. General structure: this was felt to be good and in general, language was considered to be at an appropriate level for the targeted age group. The sections were considered good, as well as appropriate. It was suggested that there might be more flexibility built into the assessment, with the option of teachers guiding students to complete one or more of the sections. They are in order of increasing difficulty and cover a range of levels of thinking. By adding in the option of choosing one or more sections to complete, this would allow some flexibility in terms of completing the assessment in one or more sittings. This would have a knock-on effect on concentration issues for the length of time the test takes to complete.

3. Accessibility was considered important as, although mostly by necessity, there is a lot of language involved. It was suggested that a reading option on each page could be available. I think this would be fairly straightforward to include. Yvonne commented that MP3 players and Ipods are becoming so common in schools that they could be given the option of using their own headphones which should take the pressure off (lack of) school resources.

4. Question numbering – it was noted that q1 isn’t actually q1. Numbering would be made more cohesive if numbered within sections – A1, A2, A3…; B1, B2, B3,…etc

5. Instructions at the start will be skipped by many pupils. Was suggested that these are read corporately – much like you would do at the start of any exam – going through the initial instruction pages.

6. Cosmetics a. Box around the diagram of the cycle b. Andy’s plan – make it look like an image – definitely distinct

Overall impression that Yvonne gave was that the assessment tests the skills that are being

learned and does so within an accessible, non-gender-specific context. She was generally very positive about it and felt it would be a useful tool within the classroom. She is willing to test it out on some of her students. When some alterations have been made (hopefully this week) that will be easier for her to do.

(b) Judith Engelhardt: Director of Studies, Claremont Fan Court School, Esher

Issues emerging from the year 9 students completing the assessment: □ Divide button on calculator – “/” – bit of confusion –should be ‘÷’ □ Lots of students asked about creating the frequency chart – in particular, what frequency means.

The students suggested a glossary on the screen (however, we are testing whether they know what


frequency is, so this wouldn’t be appropriate). They also suggested having an example/some of the table completed already – this would be a better option.

□ A few students asked what the mode is. This is subject content. □ One students asked about rounding when she calculated the mean. This should be addressed. □ The general opinion was that the last question is really quite difficult. □ The students felt there were some complicated ideas in the assessment. For example, are the deals

the same or different? □ A number of students had problems with the screens jumping – and missing out questions. I

suspect this might be a problem with the sensitivity of the laptops, but would need to be considered for the assessment to progress further.

□ The students ‘in general’ found the questions that referred to a previous slide tricky – could we overcome this by repeating things (e.g. the statements) – it may look as if the questions change rather than the actual slide which could work well

□ A couple of grammatical errors were spotted – ‘given’ and ‘different to’ □ The question “What else do you think she could do with the data?” was highlighted as a difficult

question – didn’t understand what it was asking. Could clarify this question further. □ Too much English/a lot of writing □ Size of things on the screen (the laptops had very small screens – difficult, in particular to complete

the frequency chart) It was also suggested that when completing the frequency chart, you could insert the first number (correctly) and then the rest might appear

(c) Debbie Turner: Mathematics strand PGCE tutor, NTU (Former Head of Mathematics) □ Suggested highlighting key information in a different colour (1 hour) □ Add that they need to complete it ‘on their own’ □ We don’t say who pays for Ayesha’s phone at the moment □ How old is Andy? □ Put instructions for the calculator in a different colour or in a box – highlight in some way □ C13 – on the frequency chart below to see the data ‘below to see’ □ Last question will be the most variable – ways of making this easier to assess could

include phrasing the question as: “write a ten point plan” – or include these words/phrases/points.

□ Ensure teachers prepare them before hand so that they read the questions □ Debbie was positive about the look and feel of the assessment she commented that: □ Although there’s a lot of words, the language isn’t too complex □ The use of ICT, imagery and a relevant and easily accessible context should result in the

children enjoying it □ It’s linked to something they are involved in – good. □ It will appeal to Visual, Auditory and Kinesthetic learners. □ She also raised the issue of what level it is appropriate for and discussed possibilities for

differentiation – making a version where you can access limited sections or having a link to a dictionary

□ Should highlight/emphasise the fact that you won’t be able to return to sections once you leave them.


Appendix 10

Response to issues arising in teachers’ comments on the assessment

Assessment Development

In light of the comments made by the teachers listed below and their classes, the following solutions and amendments have either been incorporated into the current assessment process or are being considered for future develolpment. DT = Debbie Turner JE = Judith Engelhardt YS = Yvonne Sutton

Navigation & Layout

Problems

Highlighting key information in a different colour DT

Should highlight/emphasise the fact that you won’t be able to return to sections once you leave them. DT

Solution The text file containing the questions is read in and interpreted as html by the assessment flash file. Additional tags (<b>bold</b>) could be used for coloured text (<red>important info</red>).

Problem Put instructions for the calculator in a different colour or in a box – highlight in some way DT

Solution

As with the example below, the instructions can be reversed in white on black or this could use the <red>code</red> as mentioned above.

Problem The students found the questions that referred to a previous slide tricky. JE


Solution

Currently pupils are allowed to move back and forth between slides that introduce information and then present information. With the example below the information needed for both screens only appears on the first one (LHS).

The slides could be re-designed to contain no more than two questions per page. This would allow the appropriate information to appear as a common feature on each related page as shown below:

+ +

Problem

Lots of students asked about creating the frequency chart – in particular, what frequency means. The students suggested a glossary on the screen JE

What level it is appropriate for and discussed possibilities for differentiation – making a version where you can access limited sections or having a link to a dictionary DT

Solution

A pop up glossary could be made available through a button on the screen – provided the access to this information does not detract from the assessment needs.

In the same way that Next and Back buttons can be switched on and off to control navigation, the glossary button could be made active at only certain times. It would also be possible to monitor how often the pupil uses the glossary.

According to the log on code that the teacher gives to their pupils – for example MCR-011-L and MCR-011-H – the final letter or number could be used to inform the flash file to read in different text files. The alternative text files would contain the same basic questions although the language used or hints given could be differentiated.

Problem Diagram of the PSA cycle requires box to help highlight its importance YS


Solution

Corrected as shown below:

Problem Andy’s plan – make it look like an image – definitely distinct YS

Solution

Corrected as shown below:


Problem Accessibility was considered important as, although mostly by necessity, there is a lot of language involved. It was suggested that a reading option on each page could be available. With MP3 players and I-Pods becoming so common in schools that they could be given the option of using their own headphones. YS

Solution Addition of interactive spoken text – when the mouse is held down whilst over the text it is read out to the pupil.

Problem Question numbering – it was noted that q1 isn’t actually q1. Numbering would be made more cohesive if numbered within sections – A1, A2, A3; B1, B2, B3, etc YS

Solution All question numbers can be generated from the external text file rather than being pre-set within the flash programme. This allows changes to be made easily without having to re-code the flash file.

Problem Many pupils skip the instructions at the start. YS

Solution These could be read corporately in the same way that instructions are read aloud at the start of any exam. The next buttons could be made to appear only after a time delay that depends on amount of information presented.

Content & Language Problem Last question will be the most variable – ways of making this easier to assess could

include phrasing the question as: “write a ten point plan” – or include these words/phrases/points. DT

Solution

Problem The question “What else do you think she could do with the data?” was highlighted as a difficult question – didn’t understand what it was asking. Could clarify this question further. JE


Solution

Problem It was suggested that there might be more flexibility built into the assessment, with

the option of teachers guiding students to complete one or more of the sections. YS

They are in order of increasing difficulty and cover a range of levels of thinking. By adding in the option of choosing one or more sections to complete, this would allow some flexibility in terms of completing the assessment in one or more sittings. This would have a knock-on effect on concentration issues for the length of time the test takes to complete.YS

Solution Initial buttons could be placed on the log on screen to allow pupils to jump to the later sections.

ICT issues & Interactivity

Problem A number of students had problems with the screens jumping – and missing out

questions. I suspect this might be a problem with the sensitivity of the laptops, but would need to be considered for the assessment to progress further. JE

Solution Every time a new page is loaded the next button can be greyed out for a number of seconds. This would stop students from accidentally clicking on Next before the page is completed

Problem Size of things on the screen (the laptops had very small screens – difficult, in

particular to complete the frequency chart) JE


Solution Currently the Flash Content Menu (right click on the mouse) is disabled. This prevents the user from moving on to the next page if certain activities are required. It is possible to allow a number of items in the Flash Content Menu to be active and others not.

This allows pupils to zoom in (anywhere from 100% to 800%) and move the page around appropriately.

Problem It was also suggested that when completing the frequency chart, you could insert the

first number (correctly) and then the rest might appear. They also suggested having an example/some of the table completed already. JE

Solution The final suggestion is perhaps the most suitable and could be built into the glossary of terms so that “frequency” is defined and an example table is shown.


Additional Comments Problem Divide button on calculator – “/” – bit of confusion –should be ‘÷’ JE

Solution Corrected as shown below:

Solution: to use pictures on top of each button instead of text since some characters like � do not always display correctly.

Problem How will teachers get feedback?

Solution This requires careful choice of the logon code that they give to pupils. For example if an individual uses MCR6-011-L and another uses MCR6-012-H, the first four letters in this code identifies the teacher/school, the middle numbers for particular individuals and the final letter could be a level of difficulty. The teacher needs to pre-register with the website to generate a suitable unique teacher/school code. This is important for confidentiality and for being able to return the spreadsheet to only the registered user. A separate log on page can then be made available to teachers – only available to teachers through a URL we email them (in the same way that the assessment page is not linked to any other page). They would enter the first four letters (eg MCR6) into an online form and password. This would generate a spreadsheet (CSV file) of the answers that students from just their class completed. The return of data could be by email or in an equivalent way to the live sampling of data from the CensusAtSchool site. Since a password and logon code are required this makes access to the data reasonably secure. The most important issue here is that of confidentiality and storage of information on pupils. Only the teacher should be able to take the full code used and identify individual pupils. We only hold the logon code and the responses.


Appendix 11

Assessment sheet for Getting the Best Deal

Domain Mark Question Description Comment Mark Holistic view

1 1 1

A D&D A1 B D&D

Places descriptions in correct locations. Clear statement of what cycle means and why it is important. Places statements into correct stages of the cycle.

Ayesha Plan 1

1 1

B1 & B2 B3 B4

Gives a clear justification for choice of response to B1 Correctly identifies the statement that is the problem to be solved. Gives a clear description of an alternative method that could be used to choose a day at random.

Collect 1 1 1

B5 B6 B7

Correctly calculates the mean Clearly comments on the choice made for the shop. Presents clear discussion of whether collecting more information would help.

Process 1 1 1

B8 B9 B10

Clearly states why the number 30 was used Gives a clear justification for choice of response to whether phone use could be expected to be the same in other months Clear statement of other uses that should be made of the data

Discuss 1 1 1

B11 B12 B13

Gives a clear justification for choice of advice on whether deal should be accepted Gives a clear statement of possible reasons for data to be an outlier Makes a clear statement of how data should be checked

Andy Plan 1

1 1 1 1 1 1

C1 C2 C3 C4 C5 C6 C7

Selects the correct answer Clearly states an example of how one of the deals could be better Clearly states which variables should be measured Gives clear explanation of the choice of time period over which data should be collected Clearly describes how the data could be collected Gives a clear explanation of why a bill is chosen at random Gives a clear explanation of why phone bills are a good data source

Collect 1 C8 Makes a clear suggestion of how data tabulation can be made less messy


Domain Mark Question Description Comment Mark Process 1

1 1 1 1 1 1 1 1 1

C9 C10 C11 C12 C13 C14i C14ii C14iii C14iv C15

Correctly calculates call durations Correctly identifies mistake in data Gives clear explanation of why mistake is there Gives clear statement of whether this poses a problem Correctly completes the frequency table Makes a sensible response to question about time series plot Makes a sensible response to question about bar chart Makes a sensible response to question about pie chart Makes a sensible response to question about scatter plot Correctly calculates summary statistics

Discuss 1 1 1 1

C16 C17 C18 C19

Gives clear interpretation of the mode Gives correct interpretation of results from frequency table Clearly states preferred choice of package and clearly explains reasons Gives a clear interpretation of the new graphical information

Plan 1 C20 Constructs a clear plan for a further investigation

Fractional marks may be awarded.

Documents

Appendix 1 On line training resource€¦ · QCA-RSS Centre Project December 2006 20 Appendix 1 On line training resource 1 2 3 4 5 6