speaking asseemengt ppt.pptx

7/27/2019 speaking asseemengt ppt.pptx

1/58

Assessing Speaking


2/58


3/58

It is simply the ability to parrot

back a word or phrase or a sentence.


4/58

It is the production of short stretches of oral language.

Examples include directed response tasks, readingaloud, sentence and dialogue completion, limited

picture-cued tasks.


5/58

The tasks include interaction and test comprehension

but at the limited level of short conversations, standardgreetings, small talk, requests, and comments.


6/58

The length and complexity of the interaction are more

in interactive tasks than in responsive ones. The tasksometimes includes multiple exchanges and/or multiple

participants.


7/58

The tasks include speeches, oral presentations, and

story-telling. Oral interaction from listeners is eitherhighly limited or ruled out altogether.


8/58

What tasks to assess imitative speaking?

Typical listen and repeat.

What kinds of prompts would you use?

Examples: Test-takers hear: beat/bit bat/vat

I bought a boat yesterday.

The glow of the candle is growing.

Test-takers repeat the stimulus.

What tasks to assess imitative speaking?

Typical listen and repeat.


Examples: Test-takers hear: beat/bit bat/vat

I bought a boat yesterday.

The glow of the candle is growing.

Test-takers repeat the stimulus.


9/58

How would you score these tasks?

Example scoring scale:

2 acceptable pronunciation.

1 comprehensible, partially correct. 0 silence, seriously incorrect.

How would you score these tasks?

Example scoring scale:

2 acceptable pronunciation.

1 comprehensible, partially correct. 0 silence, seriously incorrect.


10/58

Could you design a test that is done over the

phone?

Where the test-taker talks to a computer?

What tasks could you include to test all thefive basic types of speaking?

Lets look and listen at a test called

PHONEPASS (Versant English Test now

offered by Pearson Education, Inc.)http://www.ordinate.com/samples/english.jsp
http://www.ordinate.com/samples/english.jsphttp://www.ordinate.com/samples/english.jsp


11/58

What do you think about this test? Use the

five basic criteria.


12/58

What speaking skills can be assessed/inferred

from the PHONEPASS test?

Examples?


13/58

What range of tasks can be used to assess

intensive speaking?

List some with your partner


14/58

Typical: Directed Response Tasks


Examples: Test-takers are directed to

respond: Tell me he went home.

Tell me that you like rock music.

Tell me that you arent interested in tennis.

Tell him to come to my office at noon. Remind him what time it is.


15/58

Typical: Reading aloud

What kinds of prompts did PHONEPASS use?

What other tasks could you design? List two

other tasks with your partner


16/58

Reading a scripted dialogue.

Reading sentences containing minimal pairs.

Examples:

The man beat his dog. The man bit his dog.

Reading information from a table or chart.

Reading a scripted dialogue.

Reading sentences containing minimal pairs.

Examples:

The man beat his dog. The man bit his dog.

Reading information from a table or chart.


17/58

What elements of speech would you assess?

List two of the major ones.

Typical:

Pronunciation Fluency


18/58

Pronunciation: what scale would you design

for scoring?

Example:

0.00.4 frequent errors and unintelligible. 0.51.4 occasionally unintelligible.

1.52.4 some errors but intelligible.

2.53.0 occasional errors but always intelligible.

Pronunciation: what scale would you design

for scoring?

Example:

0.00.4 frequent errors and unintelligible. 0.51.4 occasionally unintelligible.

1.52.4 some errors but intelligible.

2.53.0 occasional errors but always intelligible.


19/58

Fluency: what scale would you design for

scoring?

Example:

0.0 0.4 slow, hesitant, and unintelligible.0.5 1.4 non-native pauses and flow that

interferes with intelligibility.

1.5- 2.4 non-native pauses but the flow is

intelligible.2.5- 3.0 smooth and effortless.

Fluency: what scale would you design for

scoring?

Example:

0.0 0.4 slow, hesitant, and unintelligible.0.5 1.4 non-native pauses and flow that

interferes with intelligibility.

1.5- 2.4 non-native pauses but the flow is

intelligible.2.5- 3.0 smooth and effortless.


20/58

Look at the PHONEPASS sample score report

and answer these questions.

1. What is the range of possible scores?

2. What are the different bands?3. What did the sample candidate score?

4. What does the score tell the candidate?

5. How does PHONEPASS measure up against

other speaking tests?


21/58

What are the advantages?

Comparisons between students are quite

simple.

Tests are easy to prepare and to administer. Predictable output, practicality, and

reliability in scoring.

What are the advantages?

Comparisons between students are quite

simple.

Tests are easy to prepare and to administer. Predictable output, practicality, and

reliability in scoring.


22/58

What are the disadvantages?

It is not really authentic is it?

Exceptions

A parent reading to a child, sharing a story (news) with someone

giving a scripted oral presentation.

It is not communicative in real contexts.


23/58

Lets take a look at some speaking tasksamples from the TOEFL iBT TEST.

As you listen to the sample, make a note ofthe similarities and differences to the PHONEPASS

test

what tasks are used to assess the different types

of speaking

See research paper for background of test design.

You can download the full sample iBT test.
http://www.youtube.com/watch?v=Ux2Q0upiD_0http://www.youtube.com/watch?v=Ux2Q0upiD_0http://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=4121af5e44df4010VgnVCM10000022f95190RCRD&vgnextchannel=d35ed898c84f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=00c534d0459c9110VgnVCM10000022f95190RCRD&vgnextchannel=ab16197a484f4010VgnVCM10000022f95190RCRDhttp://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=4121af5e44df4010VgnVCM10000022f95190RCRD&vgnextchannel=d35ed898c84f4010VgnVCM10000022f95190RCRDhttp://www.youtube.com/watch?v=Ux2Q0upiD_0http://www.youtube.com/watch?v=Ux2Q0upiD_0


24/58

First, test-takers are given time to readthrough the dialogue to get its gist, then thetape/teacher produces one part orally andthe test-taker responds.

Advantages? More time to anticipate an answer

No potential ambiguity created by auralmisunderstanding (oral interview).


25/58

TOEFL iBT identifies three broad areas to

assess in their scoring standards. What are

they?

D E L I V E R Y

L A N G U A G E U S E

T O P I C D E V E L O P M E N T

TOEFL iBT identifies three broad areas to

assess in their scoring standards. What are

they?

D E L I V E R Y

L A N G U A G E U S E

T O P I C D E V E L O P M E N T


26/58

Oral Interview:

a test administrator and a test-taker

direct face-to-face exchange proceed through a protocol of questions and

directives.

It can vary in length from 5 to 45 minutes,

depending on purpose and context.

Placement interviews may need only 5 minutes

Proficiency Interviews may require an hour.

Oral Interview:

?

a test administrator and a test-taker

direct face-to-face exchange proceed through a protocol of questions and

directives.

It can vary in length from 5 to 45 minutes,

depending on purpose and context.

Placement interviews may need only 5 minutes

Proficiency Interviews may require an hour.


27/58

Lets take a look at the IELTS TEST, part 1

and IELTS TEST, part2.

As you listen to the sample, make a note of

the the differences between IELTS and PHONEPASS

the role of the examiner

what different tasks are used, and why
http://www.youtube.com/watch?v=uHyHdRVe5NUhttp://www.youtube.com/watch?v=4Lj-ks9SxcAhttp://www.youtube.com/watch?v=4Lj-ks9SxcAhttp://www.youtube.com/watch?v=uHyHdRVe5NU


28/58

IELTS has five broad areas in scoring:

C O M P R E H E N S I O N

G R A M M A R

V O C A B U L A R Y

P R O N U N C I A T I O N C O M M U N I C A T I V E C O M P E T E N C E

IELTS has five broad areas in scoring:

C O M P R E H E N S I O N

G R A M M A R

V O C A B U L A R Y

P R O N U N C I A T I O N C O M M U N I C A T I V E C O M P E T E N C E


29/58

Lets take a look at the IELTS TEST, part 2.

As you listen to the sample, try to assess the

speaker using the scoring grid provided.

Note your score for each aspect Compare with your partner at the end


30/58

How do these relate to the 5 criteria?

Clear administrative PRACTICAL

procedures

Focusing the questions and VALIDprobes on the purpose of

the assessment

Biased for best AUTHENTIC & WASHBACK

performance

Creating a consistent, RELIABLE

workable scoring system

How do these relate to the 5 criteria?

Clear administrative PRACTICAL

procedures

Focusing the questions and VALIDprobes on the purpose of

the assessment

Biased for best AUTHENTIC & WASHBACK

performance

Creating a consistent, RELIABLE

workable scoring system


31/58

Lets take a look at another totally different

approach to assessing speaking.

Linguistic profiling suggests that language

development (first or second) follows a standard

schedule.

Speech samples collected from learners

Analysed to locate the patterns

Patterns matched from the sample to the regularities

of the standard development schedule.

See RAPID PROFILE as an example:

http://groups.uni-paderborn.de/rapidprofile/
http://groups.uni-paderborn.de/rapidprofile/http://groups.uni-paderborn.de/rapidprofile/http://groups.uni-paderborn.de/rapidprofile/http://groups.uni-paderborn.de/rapidprofile/


32/58


33/58


34/58


35/58


36/58

For your reference, what follows is a

summary of various tasks that can be used to

assess different aspects of speaking.

These have been summarized from Brown(2003) Language Assessment Principles.


37/58

Question and Answer

Examples:

1. What is this called in English? ( to elicit apredetermined correct response)

2. What are the steps governments shouldtake, if any, to stem the rate of de-forestation in tropical countries? ( givenmore opportunity to produce meaningful

language in response)


38/58

1. What do you think about the weather today?

2. Why did you choose your academic major?

3. Personal questions:

a. Have you ever been to the U. S. before?b. What other countries have you visited?

c. Why did you go there? What did you like best

about it?


39/58

Examples:

how to operate an appliance

how to put a bookshelf together, or

how to create a dish.Scoring: based on

(1) Comprehensibility

(2) Specified grammatical/discourse categories.


40/58

Test-takers hear:

1. Describe how to make a typical dish

2. Whats a good recipe for making _____?

3.

How do you access email on a PC computer?4. How do I get from ___ to ____ in your city?

Test-takers respond.


41/58

The task should require the test-taker to

produce at least 5 or 6 sentences.

Use familiar topics and test linguistic

competence. Paraphrasing, e.g. paraphrasing a story or a

phone message


42/58

1. elicit short stretches of output

2. the criterion being assessed:

a. Is it a listening task more than production?

b. Does it test short-term memory rather thanlinguistic ability?

c. How does the teacher determine scoring of

responses?


43/58

Four stages: Warm-up, Level check, Probe,

and Wind-down.

1. Warm-up

the interviewer directs mutualintroductions, helps the test-taker become

comfortable with the situation, apprises

the format, and reduces anxieties.

2. Level check Through preplanned Qs, the test-takers

respond using expected forms and functions.

Linguistic target criteria are scored.


44/58

3. Probe: In this phase, test-takers go to the heights of

their ability and extend beyond the limits ofthe interviewers expectation.

Through probe questions, the interviewerdiscovers the test-takers proficiency. At thelower levels of proficiency, probe items maydemand a higher range of vocabulary andgrammar than predicted. At the higher levels,

probe items will ask the t-t to give an opinion,to recount a narrative or to respond toquestions.


45/58

4. Wind-down

the interviewer encourages the test-taker to

relax with some easy questions, sets the t-ts

mind at ease, and provides information about

when and where to obtain the results of the

interview. This part is not scored.


46/58

1. Warm-up:How are you?/Whats your name?/What countryare you from?/Let me tell your about thisinterview.

2. Level check:How long have you been in this city?/tell me aboutyour family./What is your major?/How long haveyou been working at your degree?/What are yourhobbies or interests?/Why do you like your hobby?

What is your favorite food?/Tell me about your

exciting experience youve had.


47/58

3. Probe:What are your goals for learning English in thisprogram?/Describe your academic field to me.What do you like or dislike about it?/Describesomeone you greatly respect, and tell me why you

respect that person./If you were [president, primeminister] of your country, what would you like tochange about your country?


48/58

4. Wind-down:Did you feel okay about this interview?/Youll getyour results from this interview next week./Doyou have any question to ask?/It was interesting totalk with you. Best wishes.


49/58

A picture-cued stimulus requires a

description from the test-taker. It may elicit

a word, a phrase, a story, or incident.

Scoring scale for intensive tasks:2 comprehensible; acceptable target form

1 comprehensible; partially correct

0 silence; or seriously incorrect


50/58

Translation is a communicative device incontexts where English is not a native lang.

English can be called on to be interpreted asa second language.

Conditions may vary from an instanttranslation of a native word, phrase, orsentence to a translation of longer texts.

Advantages: the control of the output &

easily specified scoring.


51/58

It is a popular pedagogical activity in

communicative language-teaching classes.

The test administrator must determine

the assessment objectives of the role play,then devise a scoring technique that

pinpoints those objectives.

Examples: Pretend that youre a tourist

asking me for directions, You are buyinga necklace from me in a flea market, and

want a lower price.


52/58

As informal techniques to assess learners,D & C offer a level of authenticity andspontaneity that other assessmenttechniques may not provide: clarifying, questioning, paraphrasing,

intonation patterns, body language, eyecontact, and other sociolinguistic factors


53/58

Extensive speaking tasks are frequently

variations on monologues, usually with

minimal verbal interaction.

Oral Presentations: Examples: presenting a report, a paper, a

marketing plan, a sales idea, a design of a new

product, or a method.


54/58

Rules for effective assessment: (a) specify thecriterion, (b) set appropriate tasks, Elicitoptimal output, and (d) establish practical,reliable scoring procedures.Oral presentation checklist

3 excellent 2 good 1 fair 0 poorContent: The purpose or objective of the presentation

was accomplished. The introduction was lively and got my attention.

The main idea or point was clearly stated towardthe beginning. The supporting points were clearlyexpressed and supported well by facts andargument.

The conclusion restated the main idea or purpose.


55/58

Delivery The speaker used gestures and body language

well. The speaker maintained eye contact with the

audience. The speakers language was natural and fluent. The volume of speech was appropriate. The rate of speech was appropriate. The pronunciation was clear and comprehensible. The grammar was correct and didnt prevent

understanding. Used visual aids, handouts, etc., effectively. Showed enthusiasm and interest. Responded to audience questions well.


56/58

At this level, a picture/a series ofpictures is used as a stimulus for a longerstory or description.

The objective of eliciting narrative

discourse needs to be clear. (p. 181) (Tell& use the P. tense)For example, are you testing for oral

vocabulary, (girl, telephone, wet) fortime relatives (before, after, when), forsentence connectors (then, so), for pasttense of irregular verbs (woke, drank,rang), or for fluency in general?

Criteria for scoring need to be clear.


57/58

Test-takers hear /read a story or news event

that they are asked to retell.

It differs from the paraphrasing task

discussed above in that it is a longer stretch

of discourse and a different genre.


58/58

Longer texts are presented for the test-taker to read in the native language andthen translate into English.

Texts vary in forms: dialogue, directions,play, movie, etc.

Advantages: the control of the content,vocabulary, the grammatical anddiscourse features.

Disadvantages: a highly specialized skill isneeded.

Documents

speaking asseemengt ppt.pptx