View
8
Download
0
Category
Preview:
Citation preview
11/15/2016
1
WASH ı E LT SS T E P BACK
M E H TA P I N C E , I S TA N B U L Ş E H I R U N ı V E R S ı T YT U R K E Y
PAıR DıSCUSSıON
• Does your institution give an in-house or an internationally recognized test to assess students’ proficiency? Why? What about your preference?
• What do you know about the content and specifications of IELTS?
11/15/2016
2
READINGCOMPONENT
3 texts of different genres and lengths, from journals, books, magazines and newspapers
Total: 40 questions, 60 minutes
Question Types: MC,T/F/NG,Yes/No/NG, Matching, Sentence/ Summary/Table/Diagram Label Completion, SA
LISTENINGCOMPONENT
While Listening: 4 sections, each with 10 questions, audio heard onceThe first two are related social contexts; a conversation btw 2 & a monologueThe last two are related to academic contexts; a conversation btw 4 & a monologue
Total: 40 questions , 45 minutes (10 minutes for transferring answers)
Question Types: MC,T/F/NG,Yes/No/NG, Matching, Sentence/ Summary/Table/Diagram Label Completion, SA
WRITINGCOMPONENT
Task 1: 150 words, 20 minutes, description of a process or a diagramTask 2: 300 words, 40 minutes, presenting a solution to a problem/ justifying an opinion/ evaluating ideas and evidence
SPEAKINGC0MPONENT
Part 1: Interview with the examiner, personal questions, about 4 minutes
Part 2: A task card, 1 min. for preparation, monologue, about 1 min, followed by more questions on the same topic
Part 3: The examiner and examinee discuss more abstract topics linked with the Part 2 questions, about 4 minutes
Total: about 11-14 minutes
ıELTS
11/15/2016
3
READINGCOMPONENT
SEHIR UNIVERSITY TEST OF ENGLISH PROFICIENCY (STEP)
READING TEXT 1: 15%, about 1100 words, 10 MC questions
READING TEXT 2: 15%, about 1500 words, 10 MC questions
Total: 60 minutes
LISTENINGCOMPONENT
Listening 1 (While Listening): 10%, 10 MC questions, 10 minutes, audio heard once
Listening 2 (Note-taking): 15%, 10 MC questions, 25 minutes, audio heard once
Total: 35 minutes
WRITINGCOMPONENT
Argumentative Essay Writing: 25%, 60 minutes, 300+ words
SPEAKINGCOMPONENT
Part 1: Interview with the examiner, personal questions, about 2 minutes
Part 2: Monologue, answering an academic question for about 1 minute, about 4 minutes
Part 3: Discussion with the other examinee, a discussion prompt, about 8 minutes
Total: about 15 minutes
11/15/2016
4
11/15/2016
5
T H ı N KD ı S C U S SR E F L E C T
S H A R E
CHOOSE A HAT FOR YOUR HEAD
11/15/2016
6
DıSCUSSıON QUESTıONSSTUDENTS TEACHERS MID-ADMIN
1. How do you think students have felt about this shift from IELTS to an in-house test?
1. How do you think teachers have felt about this shift from IELTS to an in-house test?
1. How do you think levelcoordinators and testing membershave felt about this shift from IELTS to an in-house test?
2. Do you think there have been any changes in their attitudes, motivation, or learning/studying styles and habits?
2. Do you think there have been any changes in their lesson planning, teaching materials, and teaching methods? If yes, what changes have happened?
2. Do you think they have needed to make any changes in the curriculum and test specifications?
3. Which one do you think students would prefer, STEP or IELTS? Why?
3. Which one do you think teachers would prefer, STEP or IELTS? Why?
3. Which one do you think mid-admin group would prefer, STEP or IELTS? Why?
PURPOSE OF THE STUDY
• to observe different washback patterns among teachers, students, and mid-admin group; level coordinators and test writers
• to identify how the new in-house test is perceived
• to support innovations in curriculum and assessment within the program, through looking at the issue with the eyes of all the parties involved as much as possible
11/15/2016
7
PARTıCıPANTS AND PROCEDURE
Interview
7 experienced teachers in the exit level
Questionnaire
Teachers who have taught the exit level, at least once before
and once after
Students who have taken both IELTS and the new in-house exit
test
Mid-admin group: level coordinators, level test writers and STEP test
writers
Informal Interview
The exit level (PFAC) coordinator and an experienced instructor in the exit level
2 experienced test writers of the in-house exit test (STEP)
11/15/2016
8
•
11/15/2016
9
• Students are happy as«... guided well and they think STEP is more achievable due to MC questions,....it’s their perception.»« they can object and get feedback.»«their success rate is higher and they are less afraid.....its content is more relevant.»«they are not looking for outside exam oriented courses and they’re aware they need to improve their English to pass»• Students are not happy as «they are now panicked. They used to feel more comfortable with IELTS.»
• Admin people are happy because«students used to get stuck with IELTS more.»«they now have their own high stakes exam. It is prestigious.»
• Instructors are happy because «they now have more self-dependent learners. Students try to meet program expectations.»«they can plan their lessons more competently as STEP is more predictable and familiar to them...»«IELTS used to confuse me. I teach Academic English but IELTS assessed what I did teach. I used to feel I had to teach to the test, which made my lessons more exam oriented. I feel more flexible.«.... Readily accessible materials lack. They don’t complain much, though.»
11/15/2016
10
• «Since IELTS is gone, students' chances are more limited now. So, they tend to have much more exam anxiety.
• «Instructors also have more duties.»• «The only drawback for our level is preparing extra materials.»• «..faculties all demand... It's unfair.»
*«shared responsibility lacks in teachers; see grading and invigilation as extra work»*«ss’ pressure on admin and us»*«security of tests and personal computers» *«ss’ objections and asking for feedback»*«worries about whether the level is adjusted to the standards»*«fear of covering some items/objectives in class while teaching»
• «The program is not bound to the IELTS calendar any more....more flexibility....»• «All respectable schools have their own proficiency.. this shows the degree of quality...»• «In terms of practicality, it's better(cutting down on expenses, flexibility with timing and objectives).»• «...students' reactions.They used to demand IELTS practice to do in class. However, they now
demand academic reading texts, note-taking exercises and academic writing feedback.»• «better suited exam considering the needs of our program and departments»
*«It is highly motivating to engage in a new and challenging process.»*«better analysis of students’ profile and needs, test specs based on principles of already existing curriculum»*«active involvement of testers in curriculum innovations»*«less room for surprises»*«high face validity»*«testers are not outsiders»
11/15/2016
11
REFLECTıONS & ıMPLıCATıONS• The new in-house exit test has encouraged more in-depth analysis of students’ strenghts and
weaknesses. Reading booklets and reading quizzes have been integrated.
• Upper levels have started emphasizing academic tone of writing more and added sections to their booklets.
• Note-taking listening skills, interaction skills and critical thinking skills assessed in speaking, academic tone of voice in writing, more emphasis on academic skills in both writing and speaking criteria
• Sense of ownership among admin, mid-admin people, and instuctors
• Inevitable journey from ‘validity over reliability’ to ‘practicality over validity’.
• «.. to avoid negative washback from exam preparation, teachers should be aware of the exam requirements. They need to have a sound knowledge of the aims, specifications and administrative procedures of the exam they are preparing for.» Tsagari
GOıNG FURTHER – FOLLOW-UP
Replicating the study a year later, integrating student
interviews and class observations
Disseminating the results within and outside the institution
Comparing the percentage of students completing the program but failing the in-house exit test
and IELTS
Searching ways to enhance teachers’ and students’ literacy skills (self and
peer-feedback) because this may help positive washback, more than a reliable
and a valid test can
Discussing the results with both admin and mid-admin people to raise their awareness of how the exit test could
encourage innovations in the curriculum without dominating it
11/15/2016
12
YOUR CONTRıBUTıONS
REFERENCESAlderson, J. C. & Hamp-Lyons, l. (1996). TOEFL preparation courses: a study of washback. Language Testing. 13, p. 280-297. Retrieved from: https://www.researchgate.net/publication/240738514
Cheng, L. (2014). Consequences, Impact, and Washback. Retrieved from: https://www.researchgate.net/publication/270703143_Consequences_Impact_and_Washback
Cheng, L. , Watanabe, J. & Curtis, A. (Eds.) (2004). Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum.
Green, A. B. (2007). IELTS washback in context: Preparation for academic writing in higher education.
Feast, V. (2002). The Impact of IELTS Scores on Performance at University. International Education Journal, 3(4). Retrieved from: http://ehlt.flinders.edu.au/education/iej/articles/v3n4/feast/paper.pdf
Pan, Y. (2008). A critical review of five language washback studies from 1995-2007: Methodological considerations. SHIKEN: JALT Testing & Evaluation SIG Newsletter. 12 (2). p. 2-16. Retrieved from: http://jalt.org/test/PDF/Pan1.pdf
Tsagari, D. (2011) Washback of a high-stakes English exam on teachers’ perceptions and practices. Retrieved from: http://www.enl.auth.gr/symposium19/19thpapers/042_Tsagari.pdf
Recommended