7/27/2019 speaking asseemengt ppt.pptx
1/58
Assessing Speaking
7/27/2019 speaking asseemengt ppt.pptx
2/58
7/27/2019 speaking asseemengt ppt.pptx
3/58
It is simply the ability to parrot
back a word or phrase or a sentence.
7/27/2019 speaking asseemengt ppt.pptx
4/58
It is the production of short stretches of oral language.
Examples include directed response tasks, readingaloud, sentence and dialogue completion, limited
picture-cued tasks.
7/27/2019 speaking asseemengt ppt.pptx
5/58
The tasks include interaction and test comprehension
but at the limited level of short conversations, standardgreetings, small talk, requests, and comments.
7/27/2019 speaking asseemengt ppt.pptx
6/58
The length and complexity of the interaction are more
in interactive tasks than in responsive ones. The tasksometimes includes multiple exchanges and/or multiple
participants.
7/27/2019 speaking asseemengt ppt.pptx
7/58
The tasks include speeches, oral presentations, and
story-telling. Oral interaction from listeners is eitherhighly limited or ruled out altogether.
7/27/2019 speaking asseemengt ppt.pptx
8/58
What tasks to assess imitative speaking?
Typical listen and repeat.
What kinds of prompts would you use?
Examples: Test-takers hear: beat/bit bat/vat
I bought a boat yesterday.
The glow of the candle is growing.
Test-takers repeat the stimulus.
What tasks to assess imitative speaking?
Typical listen and repeat.
What kinds of prompts would you use?
Examples: Test-takers hear: beat/bit bat/vat
I bought a boat yesterday.
The glow of the candle is growing.
Test-takers repeat the stimulus.
7/27/2019 speaking asseemengt ppt.pptx
9/58
How would you score these tasks?
Example scoring scale:
2 acceptable pronunciation.
1 comprehensible, partially correct. 0 silence, seriously incorrect.
How would you score these tasks?
Example scoring scale:
2 acceptable pronunciation.
1 comprehensible, partially correct. 0 silence, seriously incorrect.
7/27/2019 speaking asseemengt ppt.pptx
10/58
Could you design a test that is done over the
phone?
Where the test-taker talks to a computer?
What tasks could you include to test all thefive basic types of speaking?
Lets look and listen at a test called
PHONEPASS (Versant English Test now
offered by Pearson Education, Inc.)http://www.ordinate.com/samples/english.jsp
http://www.ordinate.com/samples/english.jsphttp://www.ordinate.com/samples/english.jsp7/27/2019 speaking asseemengt ppt.pptx
11/58
What do you think about this test? Use the
five basic criteria.
7/27/2019 speaking asseemengt ppt.pptx
12/58
What speaking skills can be assessed/inferred
from the PHONEPASS test?
Examples?
7/27/2019 speaking asseemengt ppt.pptx
13/58
What range of tasks can be used to assess
intensive speaking?
List some with your partner
7/27/2019 speaking asseemengt ppt.pptx
14/58
Typical: Directed Response Tasks
What kinds of prompts would you use?
Examples: Test-takers are directed to
respond: Tell me he went home.
Tell me that you like rock music.
Tell me that you arent interested in tennis.
Tell him to come to my office at noon. Remind him what time it is.
7/27/2019 speaking asseemengt ppt.pptx
15/58
Typical: Reading aloud
What kinds of prompts did PHONEPASS use?
What other tasks could you design? List two
other tasks with your partner
7/27/2019 speaking asseemengt ppt.pptx
16/58
Reading a scripted dialogue.
Reading sentences containing minimal pairs.
Examples:
The man beat his dog. The man bit his dog.
Reading information from a table or chart.
Reading a scripted dialogue.
Reading sentences containing minimal pairs.
Examples:
The man beat his dog. The man bit his dog.
Reading information from a table or chart.
7/27/2019 speaking asseemengt ppt.pptx
17/58
What elements of speech would you assess?
List two of the major ones.
Typical:
Pronunciation Fluency
7/27/2019 speaking asseemengt ppt.pptx
18/58
Pronunciation: what scale would you design
for scoring?
Example:
0.00.4 frequent errors and unintelligible. 0.51.4 occasionally unintelligible.
1.52.4 some errors but intelligible.
2.53.0 occasional errors but always intelligible.
Pronunciation: what scale would you design
for scoring?
Example:
0.00.4 frequent errors and unintelligible. 0.51.4 occasionally unintelligible.
1.52.4 some errors but intelligible.
2.53.0 occasional errors but always intelligible.
7/27/2019 speaking asseemengt ppt.pptx
19/58
Fluency: what scale would you design for
scoring?
Example:
0.0 0.4 slow, hesitant, and unintelligible.0.5 1.4 non-native pauses and flow that
interferes with intelligibility.
1.5- 2.4 non-native pauses but the flow is
intelligible.2.5- 3.0 smooth and effortless.
Fluency: what scale would you design for
scoring?
Example:
0.0 0.4 slow, hesitant, and unintelligible.0.5 1.4 non-native pauses and flow that
interferes with intelligibility.
1.5- 2.4 non-native pauses but the flow is
intelligible.2.5- 3.0 smooth and effortless.
7/27/2019 speaking asseemengt ppt.pptx
20/58
Look at the PHONEPASS sample score report
and answer these questions.
1. What is the range of possible scores?
2. What are the different bands?3. What did the sample candidate score?
4. What does the score tell the candidate?
5. How does PHONEPASS measure up against
other speaking tests?
7/27/2019 speaking asseemengt ppt.pptx
21/58
What are the advantages?
Comparisons between students are quite
simple.
Tests are easy to prepare and to administer. Predictable output, practicality, and
reliability in scoring.
What are the advantages?
Comparisons between students are quite
simple.
Tests are easy to prepare and to administer. Predictable output, practicality, and
reliability in scoring.
7/27/2019 speaking asseemengt ppt.pptx
22/58
What are the disadvantages?
It is not really authentic is it?
Exceptions
A parent reading to a child, sharing a story (news) with someone
giving a scripted oral presentation.
It is not communicative in real contexts.
7/27/2019 speaking asseemengt ppt.pptx
23/58
Lets take a look at some speaking tasksamples from the TOEFL iBT TEST.
As you listen to the sample, make a note ofthe similarities and differences to the PHONEPASS
test
what tasks are used to assess the different types
of speaking
See research paper for background of test design.
You can download the full sample iBT test.
http://www.youtube.com/watch?v=Ux2Q0upiD_0http://www.youtube.com/watch?v=Ux2Q0upiD_0http://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=4121af5e44df4010VgnVCM10000022f95190RCRD&vgnextchannel=d35ed898c84f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=4121af5e44df4010VgnVCM10000022f95190RCRD&vgnextchannel=d35ed898c84f4010VgnVCM10000022f95190RCRDhttp://www.youtube.com/watch?v=Ux2Q0upiD_0http://www.youtube.com/watch?v=Ux2Q0upiD_07/27/2019 speaking asseemengt ppt.pptx
24/58
First, test-takers are given time to readthrough the dialogue to get its gist, then thetape/teacher produces one part orally andthe test-taker responds.
Advantages? More time to anticipate an answer
No potential ambiguity created by auralmisunderstanding (oral interview).
7/27/2019 speaking asseemengt ppt.pptx
25/58
TOEFL iBT identifies three broad areas to
assess in their scoring standards. What are
they?
D E L I V E R Y
L A N G U A G E U S E
T O P I C D E V E L O P M E N T
TOEFL iBT identifies three broad areas to
assess in their scoring standards. What are
they?
D E L I V E R Y
L A N G U A G E U S E
T O P I C D E V E L O P M E N T
7/27/2019 speaking asseemengt ppt.pptx
26/58
Oral Interview:
a test administrator and a test-taker
direct face-to-face exchange proceed through a protocol of questions and
directives.
It can vary in length from 5 to 45 minutes,
depending on purpose and context.
Placement interviews may need only 5 minutes
Proficiency Interviews may require an hour.
Oral Interview:
?
a test administrator and a test-taker
direct face-to-face exchange proceed through a protocol of questions and
directives.
It can vary in length from 5 to 45 minutes,
depending on purpose and context.
Placement interviews may need only 5 minutes
Proficiency Interviews may require an hour.
7/27/2019 speaking asseemengt ppt.pptx
27/58
Lets take a look at the IELTS TEST, part 1
and IELTS TEST, part2.
As you listen to the sample, make a note of
the the differences between IELTS and PHONEPASS
the role of the examiner
what different tasks are used, and why
http://www.youtube.com/watch?v=uHyHdRVe5NUhttp://www.youtube.com/watch?v=4Lj-ks9SxcAhttp://www.youtube.com/watch?v=4Lj-ks9SxcAhttp://www.youtube.com/watch?v=uHyHdRVe5NU7/27/2019 speaking asseemengt ppt.pptx
28/58
IELTS has five broad areas in scoring:
C O M P R E H E N S I O N
G R A M M A R
V O C A B U L A R Y
P R O N U N C I A T I O N C O M M U N I C A T I V E C O M P E T E N C E
IELTS has five broad areas in scoring:
C O M P R E H E N S I O N
G R A M M A R
V O C A B U L A R Y
P R O N U N C I A T I O N C O M M U N I C A T I V E C O M P E T E N C E
7/27/2019 speaking asseemengt ppt.pptx
29/58
Lets take a look at the IELTS TEST, part 2.
As you listen to the sample, try to assess the
speaker using the scoring grid provided.
Note your score for each aspect Compare with your partner at the end
7/27/2019 speaking asseemengt ppt.pptx
30/58
How do these relate to the 5 criteria?
Clear administrative PRACTICAL
procedures
Focusing the questions and VALIDprobes on the purpose of
the assessment
Biased for best AUTHENTIC & WASHBACK
performance
Creating a consistent, RELIABLE
workable scoring system
How do these relate to the 5 criteria?
Clear administrative PRACTICAL
procedures
Focusing the questions and VALIDprobes on the purpose of
the assessment
Biased for best AUTHENTIC & WASHBACK
performance
Creating a consistent, RELIABLE
workable scoring system
7/27/2019 speaking asseemengt ppt.pptx
31/58
Lets take a look at another totally different
approach to assessing speaking.
Linguistic profiling suggests that language
development (first or second) follows a standard
schedule.
Speech samples collected from learners
Analysed to locate the patterns
Patterns matched from the sample to the regularities
of the standard development schedule.
See RAPID PROFILE as an example:
http://groups.uni-paderborn.de/rapidprofile/
http://groups.uni-paderborn.de/rapidprofile/http://groups.uni-paderborn.de/rapidprofile/http://groups.uni-paderborn.de/rapidprofile/http://groups.uni-paderborn.de/rapidprofile/7/27/2019 speaking asseemengt ppt.pptx
32/58
7/27/2019 speaking asseemengt ppt.pptx
33/58
7/27/2019 speaking asseemengt ppt.pptx
34/58
7/27/2019 speaking asseemengt ppt.pptx
35/58
7/27/2019 speaking asseemengt ppt.pptx
36/58
For your reference, what follows is a
summary of various tasks that can be used to
assess different aspects of speaking.
These have been summarized from Brown(2003) Language Assessment Principles.
7/27/2019 speaking asseemengt ppt.pptx
37/58
Question and Answer
Examples:
1. What is this called in English? ( to elicit apredetermined correct response)
2. What are the steps governments shouldtake, if any, to stem the rate of de-forestation in tropical countries? ( givenmore opportunity to produce meaningful
language in response)
7/27/2019 speaking asseemengt ppt.pptx
38/58
1. What do you think about the weather today?
2. Why did you choose your academic major?
3. Personal questions:
a. Have you ever been to the U. S. before?b. What other countries have you visited?
c. Why did you go there? What did you like best
about it?
7/27/2019 speaking asseemengt ppt.pptx
39/58
Examples:
how to operate an appliance
how to put a bookshelf together, or
how to create a dish.Scoring: based on
(1) Comprehensibility
(2) Specified grammatical/discourse categories.
7/27/2019 speaking asseemengt ppt.pptx
40/58
Test-takers hear:
1. Describe how to make a typical dish
2. Whats a good recipe for making _____?
3.
How do you access email on a PC computer?4. How do I get from ___ to ____ in your city?
Test-takers respond.
7/27/2019 speaking asseemengt ppt.pptx
41/58
The task should require the test-taker to
produce at least 5 or 6 sentences.
Use familiar topics and test linguistic
competence. Paraphrasing, e.g. paraphrasing a story or a
phone message
7/27/2019 speaking asseemengt ppt.pptx
42/58
1. elicit short stretches of output
2. the criterion being assessed:
a. Is it a listening task more than production?
b. Does it test short-term memory rather thanlinguistic ability?
c. How does the teacher determine scoring of
responses?
7/27/2019 speaking asseemengt ppt.pptx
43/58
Four stages: Warm-up, Level check, Probe,
and Wind-down.
1. Warm-up
the interviewer directs mutualintroductions, helps the test-taker become
comfortable with the situation, apprises
the format, and reduces anxieties.
2. Level check Through preplanned Qs, the test-takers
respond using expected forms and functions.
Linguistic target criteria are scored.
7/27/2019 speaking asseemengt ppt.pptx
44/58
3. Probe: In this phase, test-takers go to the heights of
their ability and extend beyond the limits ofthe interviewers expectation.
Through probe questions, the interviewerdiscovers the test-takers proficiency. At thelower levels of proficiency, probe items maydemand a higher range of vocabulary andgrammar than predicted. At the higher levels,
probe items will ask the t-t to give an opinion,to recount a narrative or to respond toquestions.
7/27/2019 speaking asseemengt ppt.pptx
45/58
4. Wind-down
the interviewer encourages the test-taker to
relax with some easy questions, sets the t-ts
mind at ease, and provides information about
when and where to obtain the results of the
interview. This part is not scored.
7/27/2019 speaking asseemengt ppt.pptx
46/58
1. Warm-up:How are you?/Whats your name?/What countryare you from?/Let me tell your about thisinterview.
2. Level check:How long have you been in this city?/tell me aboutyour family./What is your major?/How long haveyou been working at your degree?/What are yourhobbies or interests?/Why do you like your hobby?
What is your favorite food?/Tell me about your
exciting experience youve had.
7/27/2019 speaking asseemengt ppt.pptx
47/58
3. Probe:What are your goals for learning English in thisprogram?/Describe your academic field to me.What do you like or dislike about it?/Describesomeone you greatly respect, and tell me why you
respect that person./If you were [president, primeminister] of your country, what would you like tochange about your country?
7/27/2019 speaking asseemengt ppt.pptx
48/58
4. Wind-down:Did you feel okay about this interview?/Youll getyour results from this interview next week./Doyou have any question to ask?/It was interesting totalk with you. Best wishes.
7/27/2019 speaking asseemengt ppt.pptx
49/58
A picture-cued stimulus requires a
description from the test-taker. It may elicit
a word, a phrase, a story, or incident.
Scoring scale for intensive tasks:2 comprehensible; acceptable target form
1 comprehensible; partially correct
0 silence; or seriously incorrect
7/27/2019 speaking asseemengt ppt.pptx
50/58
Translation is a communicative device incontexts where English is not a native lang.
English can be called on to be interpreted asa second language.
Conditions may vary from an instanttranslation of a native word, phrase, orsentence to a translation of longer texts.
Advantages: the control of the output &
easily specified scoring.
7/27/2019 speaking asseemengt ppt.pptx
51/58
It is a popular pedagogical activity in
communicative language-teaching classes.
The test administrator must determine
the assessment objectives of the role play,then devise a scoring technique that
pinpoints those objectives.
Examples: Pretend that youre a tourist
asking me for directions, You are buyinga necklace from me in a flea market, and
want a lower price.
7/27/2019 speaking asseemengt ppt.pptx
52/58
As informal techniques to assess learners,D & C offer a level of authenticity andspontaneity that other assessmenttechniques may not provide: clarifying, questioning, paraphrasing,
intonation patterns, body language, eyecontact, and other sociolinguistic factors
7/27/2019 speaking asseemengt ppt.pptx
53/58
Extensive speaking tasks are frequently
variations on monologues, usually with
minimal verbal interaction.
Oral Presentations: Examples: presenting a report, a paper, a
marketing plan, a sales idea, a design of a new
product, or a method.
7/27/2019 speaking asseemengt ppt.pptx
54/58
Rules for effective assessment: (a) specify thecriterion, (b) set appropriate tasks, Elicitoptimal output, and (d) establish practical,reliable scoring procedures.Oral presentation checklist
3 excellent 2 good 1 fair 0 poorContent: The purpose or objective of the presentation
was accomplished. The introduction was lively and got my attention.
The main idea or point was clearly stated towardthe beginning. The supporting points were clearlyexpressed and supported well by facts andargument.
The conclusion restated the main idea or purpose.
7/27/2019 speaking asseemengt ppt.pptx
55/58
Delivery The speaker used gestures and body language
well. The speaker maintained eye contact with the
audience. The speakers language was natural and fluent. The volume of speech was appropriate. The rate of speech was appropriate. The pronunciation was clear and comprehensible. The grammar was correct and didnt prevent
understanding. Used visual aids, handouts, etc., effectively. Showed enthusiasm and interest. Responded to audience questions well.
7/27/2019 speaking asseemengt ppt.pptx
56/58
At this level, a picture/a series ofpictures is used as a stimulus for a longerstory or description.
The objective of eliciting narrative
discourse needs to be clear. (p. 181) (Tell& use the P. tense)For example, are you testing for oral
vocabulary, (girl, telephone, wet) fortime relatives (before, after, when), forsentence connectors (then, so), for pasttense of irregular verbs (woke, drank,rang), or for fluency in general?
Criteria for scoring need to be clear.
7/27/2019 speaking asseemengt ppt.pptx
57/58
Test-takers hear /read a story or news event
that they are asked to retell.
It differs from the paraphrasing task
discussed above in that it is a longer stretch
of discourse and a different genre.
7/27/2019 speaking asseemengt ppt.pptx
58/58
Longer texts are presented for the test-taker to read in the native language andthen translate into English.
Texts vary in forms: dialogue, directions,play, movie, etc.
Advantages: the control of the content,vocabulary, the grammatical anddiscourse features.
Disadvantages: a highly specialized skill isneeded.
Recommended