Upload
abi-james
View
481
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Presented by Paul Nisbet and Abi James at the BDAN International Conference, March 2014. Since 2012 the JCQ Access Arrangements have acknowledged that candidates using a Computer Reader or text-to-speech technology are reading independently making such provision available to candidates in exams that test reading skills for the first time. While use of digital exams with text-to-speech has been widely supported in Scotland through the work of SQA and CALL Scotland for a number of years, the rest of the UK has not had equivalent access. From 2013/14 exam boards in England, Wales and Northern Ireland propose to provide digital versions of exam papers to schools for text-to-speech users. This paper will draw on experiences in Scotland and the work of the BDA New Technology Committee to identify processes and best practices within schools for using these digital papers and to identify the best text-to-speech technology to maximise the benefits for students.
Citation preview
Paul Nisbet, CALL ScotlandUniversity of Edinburgh
Abi James, BDA
USING TEXT-TO-SPEECH IN EXAMS - PITFALLS AND
PRACTICAL SOLUTIONS
Reading difficulties in secondary
• “20% of 11 year olds have poor reading comprehension” (data from SAT results)
• “by 14 years of age 33% of pupils have unsatisfactory reading comprehension”
Chris Singleton, Special Children 182 April/May 2008
“While four-fifths of pupils at Key Stage 2 reached national expectations over the last three years, one in five primary pupils did not achieve the expected standard in English.”
Moving English forward: action to raise standards in English, Ofsted 2012
Reading difficulties in secondary
SQA Assessment Arrangements
“Assessment arrangements allow candidates who are disabled and/or who have been identified as having additional support needs appropriate arrangements to access the assessment without compromising its integrity.”
Introduction to Assessment Arrangements for Schools and Colleges
SQA January 2010, revised June 2013
Assessment Arrangements 2013
• Requests for Assessment Arrangements made for –Scotland: 17,263 candidates
(11.3% of candidates)–Rest of UK: 145,430 candidates (6.9%)
• Requests made for 61,680 entries (exams)(8.3% of all entries in Scotland)
Assessment Arrangements 2013
Scotland Rest of UK
Requests for Access
Arrangements
17,263 candidates
(11.3%)
145,430 candidates(6.9% - many arrangements
delegated to schools)
Extra Time 76% 59%
Reader 30% 27%
Scribe 24% 15%
Use of ICT including digital
papers
19% n/a - use of word processing delegated to
schools
Computer readers
2% 0.4%
Coloured paper (excludes coloured
overlays)
3% 2%
Computer Reader (TTS) vs. Human Reader
• TTS offers an independent means to decoding text.• TTS provides greater consistency.• Human readers can be more flexible (can read
anything).• Some learners prefer to use TTS.• Studies have shown candidates are more likely to
check questions with TTS than using a human reader, resulting in higher scores (Dolan et al, 2005)
• Computer Readers are allowed in ALL exams, including all GCSE, A-level & Functional Skills exams including those testing reading skills “since it allows the candidate to independently meet the requirements of the reading standards”
Does text-to-speech help reading?
A 6th year pupil at Denny High in Falkirk was assessed using the Neale Analysis, reading with and without text-to-speech. Reading herself, her
comprehension age was 6 years 9 months. With text-to-speech, it was over 13 years.
2002 need for Digital Question Papers with TTS identified
2003-04 research into specification; development of papers
2004-05 evaluation in 6 schools2005-06 pilot trials #1 - 31 students used
digital papers in 105 examinations 2006-07 pilot trials #2 - 80 candidates used
490 digital papers in 200 examinations
2014 Rest of UK – PDF papers will be provided by exam boards (not interactive); UKAAF guidance
The Scottish Experience:Research, Development and Trial
Type your answers
Click to tick
Use on-screen drawing tools
Read questions with text-to-speech
SQA Digital Question Papers
Software and AppsWindows• Acrobat Reader (free)• Text-to-speech:
– ClaroRead, Co:Writer, Ivona MiniReader , Penfriend, Read and Write Gold, Adobe Read Out Loud, etc
iPad • ClaroPDF (£0.69 + £1.49 for extra voices)• PDF Expert (£6.99)• Adobe Reader (free)
www.bdatech.org/what-technology/text-to-speech/exams/computer-readers
Digital Paper Requests 2008-13
2008 2009 2010 2011 2012 2013
Number of requests 514 1,167 2,000 2,832 3,694 4,291
Number of centres 46 73 101 149 173 188
Number of candidates 204 422 675 1,069 1,327 1,677
Mean number of requests per centre 11.17 15.99 19.80 19.01 21.35 22.82
Mean number of candidates per centre
4.43 5.78 6.68 7.17 7.67 8.92
Mean number of requests per candidate
2.52 2.77 2.96 2.65 2.78 2.56
Digital Question Papers 2008-2013
2008 2009 2010 2011 2012 20130
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Number of requestsNumber of can-didates
Scottish assessment:Costs of readers/scribes
• 19,058 reader requests; 14,905 scribe requests
• ~ 36,000 individual exams• ~ 54,000 hours• Say average £20/hour for
reader/scribe?= £1,080,000
• Say £10/hour for invigilator?= £540,000
• TOTAL = £1.62m in Scotland last year
1. Become confident with the technology and text-to-speech tool they will be using in the exam.
- Use PDFs of past papers to gain experience on how to navigate the papers
- Use TTS in class
2. Centre applies for digital paper1. In Scotland via the online AAR system.2. In the rest of the UK through the Modified Paper Route
3. Digital exam paper arrives.1. In Scotland on CD with papers2. In the rest of the UK exam boards differ but most allow
for download 1 hour before exam from secure website
How do schools and candidates “use” a computer reader?
Resolving issues with use of digital papers & text-to-speech
1. Voice quality
Problem Solution
“many of the students commented that the synthetic voice was of poor quality and was difficult to understand”
Nisbet, P.D., Aitken, S., Shearer, N. (2004) Trial of External Papers in Accessible PDF for Candidates with Additional Support Needs. http://www.adapteddigitalexams.org.uk/Downloads/Reports/
1.Scottish ‘Heather’ voice licenced for use by Scottish Schools (2007).
2.Scottish ‘Stuart’ voice developed and licenced for Scottish schools (2011).
2. Pronunciation
Problem Solution
“Text to speech doesn’t read every word accurately, 1928 would be read as one thousand nine hundred and twenty-eight. Words are mispronounced, in particular names and places, which can affect the smooth reading of longer passages. Words containing an apostrophe are not read correctly.”
Nisbet, P.D. (2010) SQA Digital Papers 2010 Report.
1.2011 and 2012 papers (154 papers; 2,044 pages; 235,205 words!) analysed for mispronunciations
2.308 terms identified.3.CereProc updated the
voices to fix the pronunciation.
4.Or use a TTS tool with a pronunciation editor.
3. Unreadable image textProblem SolutionSome text is an image and cannot be read by the computer:
1.SQA desktop publishers now replace image text with text boxes with selectable, readable text.
2.By 2012, analysis showed that almost all image text elements were readable.
3.2014 – UKAAF guidance on accessible PDF exam papers
Impact on practice
Reliant on Readers?
Stuck with Scribes?
or
Independent with ICT?