A MESSAGE FROM THE FOAI ORGANIZING COMMITTEE · 2017. 3. 10. · exams in this case, have been proposed, such as increased consistency of scoring, the possibility to facilitate valid

NOVEMBER 2017

FOAI

1

FORUM ON ASSESSMENT ISSUES

FOAI-9

DESIGNING SPEAKING RUBRICS

NOVEMBER 3rd – 4th, 2017

2

A MESSAGE FROM THE FOAI ORGANIZING COMMITTEE

FOAI Organizing Committee: Mehtap İnce, Esin Çağlayan, Berna Akpınar Arslan

FOAI Webmaster: Esin Çağlayan

Dear Participants,

We would like to welcome you to the 9th FOAI event at Social Sciences University of Ankara. The theme of this event has

been chosen considering the responses of participants to the survey conducted at the previous FOAI event. We hope

that this event will serve as a platform through which the needs of the participants are met by getting to the depth of

the principles of speaking assessment. For FOAI events, it always seems to be that the more participants put into it, the

more they get out of it. To this end, we would like you to do pre-event tasks by the due date and reflect on your own

practices related to the theme of the event. We hope that the pre-tasks have already helped to focus you on the theme

and that the sharing of experiences and ideas in the focus groups will help you further your knowledge. So we wish you

inspirational one and a half days and hope that you all go back to your institutions with many new ideas to experiment

with for the improvement of your assessment practices.

If you have not visited the website yet, please do so at https://foaionline.wordpress.com/. There you can find all the

documents relating to all previous events. Please also feel free to post comments, questions and suggestions about FOAI

or any assessment issues on the website or email us at [email protected].

We hope to continue to welcome you or other representatives from your institutions in the future. However, we still get

many people who tell us that they have not heard of FOAI; hopefully, you will all help us spread the word about these

events to your contacts in Testing Teams in other universities.

We would like to thank Social Sciences University of Ankara, School of Foreign Languages, for hosting us and we are

grateful to the host team for their smooth and efficient organization.

PREVIOUS FOAI EVENTS

FOAI DATE INSTITUTION THEME

FOAI-1 February 2013 Sabancı University, İstanbul

Common Practices and Issues in Testing and

Assessment of English and Possible Responses to

them

FOAI-2 October 2013 İstanbul Şehir University,

İstanbul

Common Practices and Issues in Assessing

Productive Skills

FOAI-3 March 2014 Özyeğin University, İstanbul Standardization Practices and Marker Training

FOAI-4 November 2014 Bilkent University, Ankara Reading Assessment – Issues Related to Setting

the Level of Reading Texts and Tasks

FOAI-5 April 2015 İzmir University of

Economics, İzmir

Listening Assessment – Issues Related to Setting

the Level of Listening Texts and Tasks

FOAI-6 April 2016 Bahçeşehir University,

İstanbul Placement And Proficiency Exams

FOAI-7 November 2016 Pamukkale University, Denizli Validity and Reliability in Assessment and Testing

FOAI-8 May 2017 Karabük University, Karabük Speaking Assessment

FOAI-9 November 2017 Social Sciences University

of Ankara Designing Speaking Rubrics

FOAI-10 Spring 2018 If you wish to see your institution’s name here, please let us know and ask your

administrators to confirm their interest in an email to [email protected]

https://foaionline.wordpress.com/

3


RATIONALE/BACKGROUND

The Assessment Team of Sabancı University School of Languages set up the first FOAI for representatives from a range

of universities in Turkey and North Cyprus to come together to discuss assessment issues in university foundation

EAP/preparatory programmes. The forum aims to take a hands-on approach to the practical issues related to assessment

work, with a view to complement the formal presentation of papers or workshops to be found at other professional

events such as conferences, seminars, and workshops. Although the forum was initiated by Sabancı University School of

Languages, it has become a regular event, hosted by different institutions.

AIMS OF THE FORUM

The forum was set up specifically for representatives from the assessment teams of a range of foundation/preparatory

programmes in English medium universities in Turkey. The forum aims to provide an opportunity for different state,

foundation and private universities within the Turkish context to come together in order to:

discuss issues concerning the design, development, and administration of low and high stakes exams;

share practices, experiences, ideas, and challenges faced relevant to testing, evaluation, and assessment related issues

from diverse contexts;

increase awareness of the activities and approaches to assessment and evaluation of learning in other programmes;

build relationships and links between the assessment teams in different institutions;

exchange ideas on how members can cooperate in order to improve testing and assessment practices in their

institutions.

FOR MORE INFORMATION: Please visit our website https://foaionline.wordpress.com/

https://foaionline.wordpress.com/

4

A MESSAGE FROM THE HOST TEAM

Dear FOAI-9 Participants,

It is a great pleasure to host the FOAI-9 event at Social Sciences University of Ankara, School of Foreign

Languages. We hope it will be a fruitful platform by coming together, sharing our experiences, and

learning from each other to gain deeper insight into issues regarding speaking rubrics.

Like the previous eight events, we hope that this meeting will also enhance links between participants as well

as creating the opportunity to exchange ideas in order to improve assessment practices in our institutions.

Please find below the PARTICIPANTS of FOAI 9 – NOVEMBER 3rd and 4th, 2017.

Ayşe YÖNKUL Neslihan KOCAMAN ORTAÇ

5

PARTICIPANTS (in alphabetical order) University participant participant’s email

1 Abant İzzet Baysal University FATİH YALÇIN [email protected]

2 Altınbaş University MERVE SELÇUK [email protected]

3 Anadolu University SERCAN SAĞLAM [email protected]

4 Atılım University EYLEM ÖZTEKİN İBLİKCİ [email protected]

5 Bahçeşehir University MÜRÜVVET ÇOLAKOĞLU [email protected]

6 Başkent University RUHAN GÜLİZ ULUDAĞ [email protected]

7 Bülent Ecevit University HATİCE TOPTAN [email protected]

8 Düzce University ELİF TAŞKIN [email protected]

9 Eastern Mediterranean University MÜNEVVER N. KAHVECISOY [email protected]

10 Ege University FATMA B. KERİMOĞLU [email protected]

11 Eskişehir Osmangazi University DENİZ HANDE ÇAKMAK [email protected]

12 Gendarmerie and Coast Guard Academy YUNUS İNAN [email protected]

13 Ibn Haldun University MELTEM ÖZ [email protected]

14 Istanbul Bilgi University IŞIL ALATAN [email protected]

15 Istanbul Medeniyet University BURCU UZUNÖNER [email protected]

16 Istanbul Medipol University İSMAİL ARICI [email protected]

17 Istanbul Sehir University MEHMET AKINCI [email protected]

18 Istanbul Technical University KATHERINE PANTON [email protected]

19 Izmir University of Economics BEATRICE B. YAVAŞER [email protected]

20 Karabük University HATİCE GÜR [email protected]

21 Koç University SARAH KILINÇ [email protected]

22 Manisa Celal Bayar University BURAK BALABAN [email protected]

23 MEF University GREGORY O. CAMPBELL [email protected]

24 Ozyegin University NURGÜL KESKİN [email protected]

25 Sabancı University PINAR GÜNDÜZ [email protected]

26 TED University ESMA CEREN OCAK [email protected]

27 TOBB University of Economics and Technology HANDE AKALAN [email protected]

28 Ufuk University FATMA ZEHRA ÖZKAHYA [email protected]

29 University of Turkish Aeronautical Association ÇİSEM ALTUĞ [email protected]

30 Yaşar University GÜLCE GÖREN [email protected]

HOST TEAM

31 Social Sciences University of Ankara Ayşe Yönkul [email protected]

32 Social Sciences University of Ankara Neslihan K. Ortaç [email protected]

FOAI ORGANIZING COMMITTEE

33 Izmir University of Economics Esin Çağlayan [email protected]

34 Koç University Mehtap İnce [email protected]

35 Sabancı University Berna Akpınar Arslan [email protected]

6

PROGRAMME for FOAI-9 @ SOCIAL SCIENCES UNIVERSITY OF ANKARA

FRIDAY, 3rd November 2017

TIME ACTIVITY LOCATION

09.30 - 10.00 ARRIVAL & WELCOME

(refreshments provided) Prep Building, Floor -1, foyer area

10.00 – 10.15 OPENING REMARKS & ANNOUNCEMENTS Prep Building, Floor -1, big

lecture hall

10.15 – 11.00

WHOLE GROUP

Plenary: Dr. Keith Hoodless / Director of Operations and

Qualification Development at Learning Resources Network /

Britain

Many advantages, and disadvantages of using scoring rubrics

in performance assessments, and specifically ESOL Speaking

exams in this case, have been proposed, such as increased

consistency of scoring, the possibility to facilitate valid

judgment of complex competencies, and promotion of

learning.

This paper discusses the difficulties in assessing speaking, and

why we use rubrics for this purpose leading to an outline to

form conclusions that suggest:

(1) the reliable scoring of performance assessments can be

enhanced by the use of rubrics;

(2) an investigation of own designed as compared to readily

available rubric models

(3) rubrics seem to have the potential of promoting learning

and/or improve instruction, this potential lying in the fact that

rubrics make expectations and criteria explicit, which also

facilitate feedback and self-assessment.

Prep Building, Floor -1, big

lecture hall

11.00 – 11.15 TEA/COFFEE BREAK Prep Building, Floor -1, foyer area

11.15 – 12.15

WHOLE GROUP

Wrokshop: Dr. Reza Vahdani / Head of Professional

Development Unit at Social Sciences University of Ankara

The workshop aims to (1) generate a meaningful and

sustainable dialogue among the participants and presenters

concerning the methods and logic for developing an in-house

speaking rubric, (2) provide a platform for teacher trainers to

consider how they can increase their contributions to the

assessment and testing unit of their setting.

Prep Building, Floor -1, big

lecture hall

12.15 – 13.30 LUNCH Diner’s room of the university

13.30 – 13.45 GROUP PHOTO

If the weather is fine, in the grand

square.

If not, Prep Building, Floor -1, big

lecture hall

13.45 – 15.00 FOCUS GROUP DISCUSSIONS Prep Building -1

Rooms

15.00 – 15.15 TEA/COFFEE BREAK Prep Building, Floor -1, foyer area

15.15 – 16.45 FOCUS GROUP DISCUSSIONS (continued) Prep Building -1

Rooms

18.00 FOAI DINNER Ziverbey Konağı, Muhsin Yazıcıoğlu

Caddesi No:17/1 Çukurambar-Ankara

(sponsored by Plus Edu, Fatih Ceylan)

7

SATURDAY, 4th November 2017

TIME ACTIVITY LOCATION

09.00 ARRIVAL

(refreshments provided)

Prep Building,

Floor -1, big

lecture hall

09.00 – 10.00 ANNOUNCEMENTS & FINALIZING GROUP WORK

10.00 – 11.15

FOCUS GROUPS

FOCUS GROUPS presentations

(15 minutes each + 5 minutes Discussion and Q&A)

Group A: 10.00 – 10.20

Group B: 10.25 – 10.45

Group C: 10.50 – 11.10

11.15 – 11.30 TEA / COFFEE BREAK Prep Building,

Floor -1, foyer area

11.30 – 12.30 REFLECTION & ROUND-UP SESSION

ANNOUNCEMENT OF THE THEME FOR FOAI-10

Prep Building,

Floor -1, big

lecture hall

8

FOCUS GROUPS

AIMS

To discuss in detail findings and areas of common concern

To exchange ideas to find out if any institution has addressed these concerns

To share successes and failures in the related issues

To brainstorm different suggestions on how to better respond to these concerns

To summarize the highlights of the work of the focus group

To present these highlights to the whole group in the final session

STRUCTURE

Minute-taker: The host university will assign a minute-taker from their own institutions to each group. The

minute-taker will take notes of important ideas highlighted in the discussion and s/he will help the presenter of

the group prepare for the whole group presentation. At the end of the event, they will share their minutes with

the host univeristy’s organizing team.

Facilitator: FOAI Organizing Team will assign a facilitator from FOAI participants to each group. The

facilitator will guide group members to the achievement of the shared goals stated for each group.

Facilitators’ main duty is to ease the process. They are expected to keep the discussion on and lively with

interventions when needed. They need to make sure all participants contribute and take shared

responsibility for the outcome.

Presenter: Each focus group chooses one person to present their group work to the whole group.

FRIDAY

MORNING: The sessions in the morning will help participants to hear from some experts in the field, invited by the host

institution, and learn about the relevant practices applied in some institutions.

AFTERNOON: The sessions in the afternoon are an opportunity for participants to discuss the theme and share ideas

and experiences in smaller groups. It is hoped that the pre-task and the presentation sessions may provide some food-

for-thought in this process. Contemplating the responses that participants gave in the survey, we have decided to give

each group some focus areas to encourage participants to come up with some concrete suggestions for addressing

these areas and some practical solutions which we may be able to put into practice in our own institutions.

SATURDAY

MORNING: FOAI Organizing Team introduces the schedule of the day, and then the focus groups finalize their

presentations. The powerpoint must be e-mailed by the facilitators to [email protected] by 09.40 at the

latest. After all the focus groups are ready, the presenter of each group will have 15-20 minutes to present and there

will then be 5-10 minutes for questions and discussions about issues raised by each group. The presentations and follow-

up discussions will help all participants reflect on their own practices and the suggestions offered. And eventually, they

will be more likely to feel more competent and confident to stimulate innovations in their institutions.

9

FOCUS GROUP PARTICIPANTS

FOCUS GROUP A Facilitator: Berna Arslan, SABANCI

UNIVERSITY

Minute-taker: Özlem Akçay, SOCIAL SCIENCES

UNIVERSITY of ANKARA

1 RUHAN GÜLİZ ULUDAĞ BAŞKENT UNIVERSITY

2 IŞIL ALATAN ISTANBUL BILGI UNIVERSITY

3 BURCU UZUNÖNER ISTANBUL MEDENIYET UNIVERSITY

4 HATİCE GÜR KARABÜK UNIVERSITY

5 BEATRICE B. YAVAŞER IZMIR UNIVERSITY of ECONOMICS

6 İSMAİL ARICI ISTANBUL MEDIPOL UNIVERSITY

7 YUNUS İNAN GENDARMERIE and COAST GUARD ACADEMY

8 SARAH KILINÇ KOÇ UNIVERSITY

9 ELİF TAŞKIN DÜZCE UNIVERSITY

10 ESMA CEREN OCAK TED UNIVERSITY

FOCUS GROUP B

Facilitator: Esin Çağlayan, IZMIR

UNIVERSITY of ECONOMICS

Minute-taker: Özge Deretarlası, SOCIAL SCIENCES


1 MÜRÜVVET ÇOLAKOĞLU BAHÇEŞEHIR UNIVERSITY

2 MELTEM ÖZ IBN HALDUN UNIVERSITY

3 BURAK BALABAN CELAL BAYAR UNIVERSITY

4 GREGORY O. CAMPBELL MEF UNIVERSITY

5 NURGÜL KESKİN ÖZYEĞİN UNIVERSITY

6 ÇİSEM ALTUĞ UNIVERSITY of TURKISH AERONAUTICAL ASSOCIATION

7 HANDE AKALAN TOBB ETU

8 MERVE SELÇUK ALTINBAŞ UNIVERSITY

9 FATMA ZEHRA ÖZKAHYA UFUK UNIVERSITY

10 DENİZ HANDE ÇAKMAK ESKISEHIR OSMANGAZI UNIVERSITY

FOCUS GROUP C

Facilitator: Ayşe Yönkul, SOCIAL SCIENCES


Minute-taker: Burcu Torun, SOCIAL SCIENCES


1 EYLEM ÖZTEKİN İBLİKCİ ATILIM UNIVERSITY

2 FATMA B. KERİMOĞLU EGE UNIVERSITY

3 MEHMET AKINCI ISTANBUL ŞEHIR UNIVERSITY

4 KATHERINE PANTON ISTANBUL TECHNICAL UNIVERSITY

5 PINAR GÜNDÜZ SABANCI UNIVERSITY

6 GÜLCE GÖREN YAŞAR UNIVERSITY

7 MÜNEVVER N. KAHVECİSOY EASTERN MEDITERRANEAN UNIVERSITY

8 HATİCE TOPTAN BÜLENT ECEVİT UNIVERSITY

9 SERCAN SAĞLAM ANADOLU UNIVERSITY

10 FATİH YALÇIN ABANT İZZET BAYSAL UNIVERSITY

10

FOCUS GROUP TASKS

GROUP A: Design of Rubrics

GROUP B: Levels and Assessment

Types

GROUP C: Implementation of

Rubrics

Does CEFR play an important role in

your speaking rubric design? If yes,

how?

Should rubrics for external exams be

used as a model?

Who should be involved in rubric

design?

Which type of rubric outweighs the

other in terms of advantages?

Holistic or Analytic?

There is a myth that analytic rubrics

are much stricter than holistic ones.

Do you think it is correct? Or is it

vice versa?

What discrete elements/bands

should be included in speaking

rubrics?

How detailed should the descriptors

be?

What are the advantages of having

different rubrics for each level?

What are the advantages of having

one type of rubric catered for all

levels?

Should the rubrics used for

achievement (level exams) and

proficiency exams be different?

Should the rubrics be based on the

task types? E.g. pair task/individual

task, presentation, debate, etc.

How do task types affect the content

of rubrics? What descriptors would

be required for specific task types?

Should rubrics be independent of

task types? If yes, what should our

rubrics be based on?

What are some ways to monitor the

effectiveness of speaking rubrics

and ensure that speaking objectives

are assessed in a reliable way?

How can we ensure the success of

norming sessions done for

instructors’ training?

Should students be familiarized with

the rubrics used by assessors before

the assessment? If yes, how?

How should scoring be done to

ensure reliability and validity of

assessment?

Should each band have the same

weighting? Should points awarded

for each band be equal?

11

SUGGESTED AGENDA FOR FOCUS GROUPS

Feedback from previous events suggested that a proposed agenda for the focus groups might help in ensuring that the

work of the groups is efficient and effective. We have suggested such an agenda below, and we look forward to seeing

how this works.

1. INTRODUCTIONS

Participants introduce themselves and give some information about the assessment of speaking skills at their

institution, focusing on the principles behind the speaking rubric design. Here, participants had better refer to

the speaking rubrics section at the back of this booklet.

2. AGREE ON A PRESENTER

It is good to agree early on who will be presenting, if not at this stage then certainly before the group begins

planning the presentation.

Neither facilitators nor FOAI Organizing Team members should be considered as presenters.

3. INDIVIDUAL BRAINSTORMING

Some quiet time for each participant to go through the bullet points and make notes of ideas and issues.

4. GROUP BRAINSTORMING

Open discussion of the task in detail, maybe prioritising certain bullet points which seem to be more

controversial or fruitful. Other issues might also be raised.

All participants should note ideas and issues they feel are most important (having just one note-taker means

only one person’s interpretation of the discussion is recorded).

5. ORGANISING IDEAS

The group lists, categorises and organises the main ideas agreed upon and identifies areas which were more

controversial or could not be agreed upon. Mind-maps, diagrams etc might be useful.

6. PLANNING PRESENTATION

The group decides how best to present the ideas from the discussion and prepares powerpoint slides.

The presentation should be sent to [email protected] by 09.40 on Saturday.

Go over the presentation and if possible do a practice. A copy of the slides should be printed for the presenter

to go over and add notes.

While the presenter is rehearsing, team members of each focus group prepare questions for the other focus

groups to answer.

ROUND UP SESSION

AIMS:

To reflect on the event

To suggest topics for future events

To announce the theme and the host university for the next FOAI event

FORMAT:

Each participant will decide on a few “key items” that they wish to take away from today’s event. This could be

an idea they would like to take back to their own institution; or an area they would like to explore in further detail

in their own context.

They will, then, be asked to put their ideas onto an online platform and through this platform, all participants

will have the chance to see what others want to explore more. If time allows, the group will discuss some ideas

emphasized and some practical suggestions shared on this platform.

We will finally ask the participants to get together in groups of 3 or 4 to brainstorm some suggestions for themes

which they would like to see in future FOAI events. Again, the ideas will be collated on this platform to be referred

to while designing the next FOAI event.

12

FOAI-9 @ SOCIAL SCIENCES UNIVERSITY OF ANKARA

LOCATION OF THE EVENT

Address: ANKARA SOSYAL BİLİMLER ÜNİVERSİTESİ, ORAN KAMPÜSÜ, HAZIRLIK OKULU BİNASI, YUKARI DIKMEN

MAHALLESI, 648. CADDE, 649. SK. NO: 8, 06450 TURAN GÜNEŞ BULVARI, ORAN/ÇANKAYA/ANKARA

LOCATION: Ankara Social Sciences University Prep School GOOGLE MAPS COORDINATES: 39.856988, 32.845790

TRANSPORTATION:

From Esenboğa Airport: If you plan to travel by plane, you can take Havaş buses or taxi from the airport to the

campus after your arrival in the airport.

From AŞTİ (Ankara Şehirlerarası Otobüs Terminali): If you plan to travel by bus, you can take subway, taxi or bus to

the campus.

From the railway station: If you plan to travel by train, you can take taxi or bus to the campus.

ACCOMMODATION

At the campus, there are not any accommodation options available.

1. For the guesthouses, please click on the link below.

Ankara misafirhaneleri için tıklayınız.

2. Hotels nearby are; (ordered by considering the location of the school)

Atlı Hotel http://hotelatli.com/tr/ (single room 200 TL / double room 250 TL, special price for this event. Please

make sure you remind the receptionist that you’re coming for Ankara Sosyal Bilimler Üniversitesi)

Swiss Hotel Ankara http://www.swissotel.com.tr/hotels/ankara/

Büyükhanlı Park Hotel & Residence http://www.buyukhanliparkhotel.com/index/index.php?lang=tr

https://www.google.com.tr/maps/place/Ankara+Sosyal+Bilimler+%C3%9Cniversitesi+Haz%C4%B1rl%C4%B1k+Binas%C4%B1/@39.8945423,32.7639293,12z/data=!4m15!1m9!4m8!1m0!1m6!1m2!1s0x14d345aef86e2a35:0x54447f6c00f9c1e6!2sYukar%C4%B1+Dikmen+Mahallesi,+Ankara+S

https://www.google.com.tr/search?rlz=1C1GTPM_trTR604TR604&rlla=0&tbm=lcl&hotel_occupancy=2&q=ankara+misafirhaneleri&oq=ankara+M%C4%B0SA&gs_l=psy-ab.3.0.0l4.15223.18102.0.19765.12.12.0.0.0.0.133.1250.5j7.12.0....0...1.1.64.psy-ab..0.12.1245...0i67k

http://hotelatli.com/tr/

http://www.swissotel.com.tr/hotels/ankara/

http://www.buyukhanliparkhotel.com/index/index.php?lang=tr

https://www.google.com.tr/maps/place/Ankara+Sosyal+Bilimler+%C3%9Cniversitesi+Haz%C4%B1rl%C4%B1k+Binas%C4%B1/@39.8686867,32.8384427,12.33z/data=!4m15!1m9!4m8!1m0!1m6!1m2!1s0x14d345aef86e2a35:0x54447f6c00f9c1e6!2sYukar%C4%B1+Dikmen+Mahallesi,+Ankar

13


(FOAI-9)

3rd & 4th November, 2017

PREPARATORY PROGRAMMES

SPEAKING RUBRICS

14

ABANT İZZET BAYSAL UNIVERSITY

SYSTEM Semester-based or Modular

Please briefly describe how speaking assessment is integrated into your program. (i.e. the focus given to teaching and assessing speaking in the overall program)

During an academic year, there are four mid-term exams in which speaking skills are tested, and each of them has 20% of weight. Furthermore, students are required to deliver one poster, two powerpoint presentations covering academic topics. Each of these presentations is equivalent to one quiz. Speaking skill is assessed in 3 out of 11 quizzes in one academic year.

Please briefly describe the types of speaking assessment in your program, their weighting, the types of rubrics used (i.e. holistic/analytical, etc), the subskills/areas tested, bands/ scales, etc.

PLACEMENT TEST Proficiency test at the beginning of the year is used as a placement test.

PROFICIENCY TEST There is no speaking part in Proficiency Test.

PROGRESS ASSESSMENT

Analytic rubrics are used. The parts in the rubrics are use of language, vocabulary, discourse management, pronunciation and interactive communication. For mid-terms, the tasks include general daily topics, picture description and a talking about a question related to the picture and giving their opinion about a topic. In presentations, two topics are academic, one is general.

OTHER (if any)

Please briefly describe the principles behind the speaking rubrics used at your institution. Please also include information about the following:

who are involved in speaking rubric design

how the design process is carried out

whether rubrics are transparent to test takers

how the evaluation and re-design process is managed (if applicable)

The rubric is designed by the instructors who are in charge of exam preparation. The design is based on the feedback from all instructors. Instructors share their ideas and experiences in every end-of-term meeting. The rubrics are modified accordingly if needed. The rubrics are transparent to students.

15

ALTINBAS UNIVERSITY



20 % of the program and assessment


PLACEMENT TEST No speaking subtest in the test battery due to practicality issue

PROFICIENCY TEST

- 20 % of the battery

- Analytical rubric

- Constructs tested fluency organisation content linguistic knowledge *lexical richness, and grammatical accuracy*

- Multi level tasks

PROGRESS ASSESSMENT

- The same of the above plus at certain levels we include psycholinguistic component

(Elicited immitation task) to our speaking test (Van Moere 2012) (also to maintain positive washback) as memory plays a large role in language processing and automatization is necessary for proficiency in L2

- Construct of speaking depends on purpose we use different rubrics for our EDP EMP or ENG coded courses

- Level specific tasks

OTHER (if any)






how the design process is carried out how the evaluation and re-design process is managed We try to follow test Development Cycle of Downing & Haladyna (2006). There is a strict close link between our rubrics & tasks and how we define speaking ability/ constructive alignment’ is essential for us in all tests. We defined specific purposes, task types, discourse types (based on CEFR) and needs analysis. To identify our test takers future actions/functions we conducted needs analysis with all faculty members and 10 per cent of faculty students / we also conducted focus group interviews with all faculty members to design our curriculum and test framework and specifications. We cover construct and learning outcomes in test specifications. We have a strict framework for the interlocutors to follow during the test/ The interlocutor follows a script so that the speaking test structure is similar for each candidate. Our speaking task typology (depends on the level and the task characteristics)

Task Orientation – Open (outcomes dependent upon speakers) – Guided (by instructions) – Closed (outcomes dictated by input or instructions) • Interactional Relationship – Non-interactional (monologue) – Two-way (interlocutor – student; student – student) – Multi-way (interlocutor – student – student) • Interlocutor – face-to-face – no interlocutor whether rubrics are transparent to test takers? Yes, to familiarize learners with the rubric we have in class writing and per assessment tasks/ Students are graded on THE QUALITY of the FEEDBACK THEY GAVE TO THEIR PARTNER.

16

ANADOLU UNIVERSITY



In one semester, students take one midterm and final speaking exam. The exam is based on the GSE outcomes covered in the program. There are two parts in the exam. In the first part, the students talk on their own on a given topic. There are two questions. In the second part, there is a negotiation task. Students discuss a topic and make to agreement. Apart from the formal assessment of speaking, there are presentation and recording tasks that students prepare out of class.


PLACEMENT TEST No speaking in the placement

PROFICIENCY TEST

Speaking exam makes up the 20% of the overall grade. The exam covers the GSE outcomes of range 53-61. The rubric used is analytical and students’ spoken performance is assessed on five components, namely content, fluency, grammatical competence, lexical competence and interaction.

PROGRESS ASSESSMENT

The students take one midterm and one final speaking exam. The score they get on these tests make up the 20% of their overall grade on midterm and final exam. Students’ performance is assessed using an analytical rubric.

OTHER (if any) There are presentation and recording tasks that students prepare out of class. Tasks make 40% of the midterm II grade.





how the evaluation and re-design process is managed (if applicable) The testing unit prepared one version of the rubric in 2013. The design process was simple. First, the testing unit members identified papers representing students’ performance. All testing unit members assessed the performance and wrote rationale for their assessment. After this, all unit members came together and thought about the descriptors for each component of the criteria. They agreed on the descriptors first, then, they evaluated other students’ performances with the criteria. Before making the rubric available for public use, the team shared their experience about the easiness and fairness of assessment with the new criteria and made necessary changes. The rubric was implemented for two years. Based on oral feedback from the teacher, it was revised in 2015. The revised version addressed issues brought up by the teachers. The revised rubric is in use for two years. This year, the testing unit is carrying out standardization workshops to train the teachers about the rubric.

17

ATILIM UNIVERSITY



Students are expected to produce the language in the given tasks. Almost every day there is a productive task, either written or spoken. Throughout the module students are assessed during these tasks and they get 20% of their grade on the first achievement test from speaking. At the end of the module there is a face-to-face speaking test. This also makes the 20% of the second achievement test.


PLACEMENT TEST

PROFICIENCY TEST

PROGRESS ASSESSMENT

Formative Assesment during the module – analytical rubric is used – 20% Summative Assessment at the end of the module – holistic rubric is used – 20%

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) First, we talked to the instructors one by one and asked their opinions about what makes a good speaker, what must be assessed, what should be the weighting, what type of questions can be asked and what should be the ideal duration. After analysing the data, we prepared two different rubrics, one for in-class assessment and one for the end of the module. After that, we recorded some videos of students and carried out a standardisation process and checked if the rubrics worked or not. A group of teachers from each level group – we have five (A1, A2, B1, B1+ and Proficiency) prepared some tasks using the topics the students covered in the lessons. Then we trained our teachers on the rubric and carried out another standardisation with the same videos. When the term began we also trained the students on the rubrics by explaining each item and showing a sample video so that they can use the rubric and try to evaluate the performance in the video. After each exam at the end of each module we revised the tasks by taking the feedback coming from the test givers into consideration.

18

BAHÇEŞEHİR UNIVERSITY



2 hours of weekly speaking input is dedicated to each level covering the in-house produced speaking booklets, as well as the supplementary materials provided by the level coordinator. To maximize learners’ opportunities to develop this skill and to ensure that adequate amount of time is allocated for it, speaking is focused on exclusively. Speaking task results contribute 10% towards the total achievement score.


PLACEMENT TEST NO SPEAKING

PROFICIENCY TEST

Speaking component of the exam assesses the candidates’ ability to express their opinions about familiar topics, giving reasons, details and examples to support them. The weighted value of the speaking component in the Proficiency exam is 10 points. An analytical rubric is used to assess performance in the exam. The following are the areas tested in the exam: TASK, LANGUAGE, VOCABULARY, FLUENCY, DELIVERY For each area tested, the examiner is provided with a rubric describing each band score ranging from 0 to 4.

PROGRESS ASSESSMENT

For each level, different speaking tasks are prepared along with a holistic or analytical rubric.

OTHER (if any)






who is involved in speaking rubric design o Testing unit members, level coordinators, teachers

how the design process is carried out o We work hard to ensure that the speaking assessment reflects the objectives and content

covered in the classroom. Students are not assessed on one type of speaking; we try to include different types of speaking depending on the level of students being assessed.

whether rubrics are transparent to test takers o Students have the opportunity to over the rubric in class with the teacher prior to the

speaking task/ exam.

19

BAŞKENT UNIVERSITY



All prep students take one speaking exam once in each semester as a part of one of the progress tests. Those who pass all progress tests take another speaking exam as an integral part of End of Module Test (At the end of A level) and Exit Test (at the end of B level) / double marking (a scorer and an interlocutor) through an analytical rubric Teaching of the skill is supported by classroom activities and materials provided by Curriculum Development Unit.


PLACEMENT TEST - Made up of direct test items; may be described as integrative testing; double marking (a scorer and an interlocutor) through an analytical rubric -Tests communicative functions, fluency, lexical and grammatical accuracy & range, repair, discourse markers, coherence - 20% - Applied to 100% English-medium programs

PROFICIENCY TEST

PROGRESS ASSESSMENT

- Made up of direct test items; may be described as integrative testing; double marking (a scorer and an interlocutor) through an analytical rubric -Tests communicative functions, fluency, lexical and grammatical accuracy & range, repair, discourse markers, coherence - 10% - Applied to all PREP students once each semester

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) While designing the rubric, points of consideration were: - harmony between the rating scale and learning objectives - its ability to serve the desired learning outcomes - its ability to overcome personal bias, halo effect or logical errors. - its clarity - its reliability - its appropriateness The rubric was originally designed by the Testing Unit then went through a trial and re-design process where 10 different student mock-exam sample videos were used and 40 ELT instructors (from different age-experience-speciality field-gender-courses taught groups) involved. The rubric is transparent to the students. At the end of each academic year, an evaluation and re-design process is managed through student and instructor

feedbacks and through an analysis of the overall speaking test results.

20

BİLGİ UNIVERSITY (to be added soon)




PLACEMENT TEST

PROFICIENCY TEST

PROGRESS ASSESSMENT

OTHER (if any)






21

BÜLENT ECEVİT UNIVERSITY



All skills are equally integrated in the overall program.


PLACEMENT TEST No speaking assessment

PROFICIENCY TEST No speaking assessment

PROGRESS ASSESSMENT

Student Portfolio / %1(20 points) / holistic rubric / Content and Overall Oral Presentation tested. Content (40pts= Writing+Speaking+Reading+Vocabulary+Listening). Content is assessed considering the delivery of the items on time and the number of the items put into the portfolio. Overall Oral Presentation (10pts) is assessed based on the student’s performance in a 5 minute - oral presentation.

OTHER (if any)

There are four Midterms and one Final Exam in the overall program. In each exam, there is a different type of speaking assessment in the program. Their weighting is 20 points (%5). Analytic rubrics are used. The subskills – Fluency /Pronunciation, Vocabulary Range, Grammatical Range/Accuracy, Spoken Production, Spoken Interaction are tested. The scale is 4 to 1. 4(Very Good to Good), 3(Good to Fair), 2(Fair to Poor), 1(Poor).





how the evaluation and re-design process is managed (if applicable) The coordinators of the speaking office are involved in speaking rubric design. The rubrics are proofread by the members of the office. The rubrics are designed based on CEFR descriptors determined for each level of learners. Rubrics are transparent to test takers. They are assigned in classes one week in advance. The evaluation of the exams is carried out by all the instructors at the institution. The instructors are assigned as interlocutors, assessors or ushers to carry out the speaking exams. The re-design process is managed by the coordinators of the speaking office and the assessment office.

22

DUZCE UNIVERSITY



During a year, there are four midterms and one final exam. Students speaking skill is assessed in these exams. In the first semester, students study course books, and they are asked to answer questions in speaking exams. In the second semester, there are skill courses, and students are asked to talk about a topic on the exams. As a part of lessons, students are also required to prepare presentations, and their speaking performance also assessed on their presentations.


PLACEMENT TEST There is no speaking assessment on placement test.

PROFICIENCY TEST

Before speaking exam, there are two written exams which the students have to get a specific grade to be able to get the speaking exam. The rubric which is used in speaking assessment is holistic. For speaking exam, students are required to choose two questions in the first semester; two topics in the second semester. The students’ performance is assessed based on their speech about topic and the questions asked by the teachers in the exam.

PROGRESS ASSESSMENT

To be able assess students’ speaking performance, there are four midterms and scales in rubrics show some differences based on the proficiency of the students.

OTHER (if any) On final exam, holistic rubric is used and students’ performance is assessed based on their answers.





how the evaluation and re-design process is managed (if applicable) The speaking rubric was adapted from Edinburgh University based on the students’ needs. The rubric was adapted by the instructors. There are two instructors in the room in speaking exams and they can both have conversation with the students on topic. Rubrics are transparent to test takers. Students can see the rubrics on the door of the speaking room. After exams, instructors come together and discuss on the efficiency of the rubric. If there is need for change,

they decide to adapt the rubric based on their discussions.

23

EASTERN MEDITERRANEAN UNIVERSITY

SYSTEM Semester-based Modular

(Foreign Languages) (English Preparatory School)


EMU-FLEPS (Foreign Languages and English Preparatory School) has a productive-based syllabus so teaching and assessing speaking is very important. Students studying at Foreign Languages (FL) division have one speaking test during the academic year while students studying at English Preparatory School (EPS) have two speaking tests in each module. Thus, teachers plan their classroom instructions accordingly. They utilize speaking tasks that aim to foster the speaking ability of the students. Communicative tasks are integrated into the syllabus. Moreover, students have communicative tasks that they need to complete outside the class such as recording their voice and/or video recording themselves while interviewing with a partner and sending it to the teachers. They provide the students with feedback so that students have a chance to see their progress.


PLACEMENT TEST There is no speaking test in the placement test of EPS.

There is no placement test at FL division.

PROFICIENCY TEST

Types of Speaking Assessment: Students who have successfully completed B1 level have a chance to take the Proficiency Test. There are 3 tasks in the Proficiency Test and students are interviewed in pairs. In the first task, students are asked 2 general questions about different topics. In the second task, each student is given a slip of paper with an argument and expected to give their opinions about it and give their reasons to support their opinions. In the last task, the interlocutor introduces a problem based on an imaginary situation and students are expected to find possible solutions for the problem. The students are asked to discuss solutions and consequences of the solutions. Weighting: 25 % The types of rubrics: Analytic rubric is used. It resembles a grid with the criteria that provides

specific feedbackalongseveraldimensions. The subskills such as turn taking, maintaining

the interaction are also tested.

PROGRESS ASSESSMENT

EPS: Only for A2, B1 and UB1 students have a sit speaking exam. Other levels have a ‘Conversation Skills’ section in their written exams. Weighting: 5% FL: As mentioned above, there is only one speaking exam in one academic year for the TMP courses. It takes place before the final exam.

OTHER (if any)

FINAL EXAM EPS: A2, B1 and UB1 students have a speaking test in their Final Exam. Except for the Proficiency Exam, the same criteria are used for all speaking tests. Weighting: 10 % FL: The speaking test takes place before the final exam. Furthermore, there is a ‘Conversation Skills ‘ section in Progress and Achievement Tests Weighting: 10 % Tasks: There are 3 tasks in the test and students are interviewed individually. In the first task, students are asked 3 general questions about their hobbies, friends, family, weekends, etc. The second task is a role-play task. Test takers and the interlocutor have dialogues. In the final task, test takers are given a topic to talk about for a minute.

24





how the evaluation and re-design process is managed (if applicable) Speaking rubric design was carried out by the testing teams. The team members came together and worked on the tasks. Having analysed the tasks and their objectives thoroughly, the rubrics were designed. During the design process, Grant Kempton (the writer of Language Leader PEARSON) was leading the team. After the rubrics were written and approved by the administrator, they were shared with the teachers. Teachers at EPS have been using the same criteria for three years. Having used the criterion for several tests, it was realized that it was not applicable for the FL division. For the speaking tests in FL, the assessor and the interlocutor are the same person and the criterion was not user friendly. Thus, it was simplified to maintain consistency. For having consistency between graders, mock speaking tests are carried out and recorded before the real exam. All teachers come together, watch the videos and grade the students by using the rubrics. The standardisation meetings take place before the real exams. Rubrics are shared with the test takers at EPS. They are attached to students’ portfolios. However, for the FL division it is teachers’ choice whether to share them with the test takers or not.

25

EGE UNIVERSITY



In our program, as far as the speaking course is concerned, class hours in a week and exam points for all levels per semester are as follows: For Alpha & Beta Groups:

Course Class Hours Exam Points Total

Note-taking 2 10 + 10 + 10 (Exams) + 10 (Task) 40

Speaking - 30 (Presentation) + 30 (Exam) + 15 (Task) 75

Total 28 355 355

For Gamma Groups:


Note-taking 4 10 + 10 + 10 (Exams) + 10 (Task) 40

Speaking - 30 (Presentation) + 30 (Exam) + 15 (Task) 75

Total 28 375 375

For Delta Groups:


Listening 8 15 + 15 + 15 (Exams) 45

Speaking 15 + 15 (Exams) + 15 (Task) 45

Total 28 340 340

There are four different kinds of speaking exams implemented to all levels in an academic year. Students take all these exams individually except for one which is carried out in pairs. In addition, students are supposed to complete one task each semester. The speaking exams and tasks are prepared in parallel to the materials and tasks the students get familiar with throughout the academic year. The overview of all speaking exams/tasks each term are stated below: FALL TERM Speaking Exam I - Picture Description (15 pts / ~ 4 minutes) This is to be an individual speaking exam. Part 1: Greeting and introducing yourself (about 30 sec.) Part 2: Describing the picture (about 1 min. + 30 sec.) Part 3: Answering follow-up questions about the theme of the picture (about 1 min. + 30 sec.) Speaking Task I – Video Recording (15 pts / ~ 4 minutes) Speaking Exam II - Situational Conversation (15 pts / ~ 5 minutes) This is to be a speaking exam carried out in pairs. Part 1: Greeting, introducing yourself and answering instructor’s questions (about 1 min. + 30 sec.) Part 2: Discussing the situation and making a decision (about 4 min.) SPRING TERM Speaking Exam III - Presentation (15 pts / ~ 5 minutes) This is to be an individual speaking exam that requires each student to make an oral presentation. In our program, as far as the speaking course is concerned, class hours in a week and exam points for all levels per semester are as follows: Speaking Exam IV - Cue Cards (15 pts / ~ 4 minutes) This is to be an individual speaking exam. Part 1: Greeting, telling your name and talking about the cue card topic (about 3 min.) Part 2: Answering follow-up questions about the topic (about 1 min.) Speaking Task II – Poster Presentation (15 pts / ~ 4 minutes)


PLACEMENT TEST Proficiency test is used in order to determine the level of students.

PROFICIENCY TEST

In the proficiency exam, speaking session is given only to the group consisting of the language students whose departments include 100% English courses. A 100-point speaking exam is given to these students and its weighting is 30%. Then it is added to the total (Use of Eng. + Reading + Writing + Listening = 70%).

26

PROGRESS ASSESSMENT

In our institution, the speaking rubrics are designed according to the analytical approach. There are three types of rubrics designed for the specific skills to be tested through the different speaking exams. General speaking assessment criteria are used for the types of exams including picture description, situational conversation and cue cards (15 points in total, 3 points max. for each criterion). Oral and poster presentation are assessed through presentation assessment criteria (30 points in total, 3 points max. for each criterion). Lastly, video recording assessment criteria, which are almost the same with the presentation assessment criteria, are used for the video recording task (30 points in total, 3 points max. for each criterion). GENERAL SPEAKING ASSESSMENT CRITERIA (15 points)

Very good (3pts) Satisfactory (2pts) Needs improvement (1pt)

Interaction

Spoken Grammar

Vocabulary

Fluency& Coherence

Pronunciation

TOTAL 15 pts 10 pts 5 pts

PRESENTATION ASSESSMENT CRITERIA (30 points)

Very good (3 pts)

Satisfactory (2 pts)

Needs improvement (1 pt)

Not observed (0 pt)

Content

Organization

Vocabulary skills

Use of English

Speech

Visual aids

Body language

Responsiveness

Time management

Overall impression

TOTAL 30 pts 20 pts 10 pts 0 pts

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) who is involved in speaking rubric design

o Speaking rubrics used in our institution are designed by the Curriculum Development Unit members educated in the field.

how the design process is carried out o The design process of exam types and rubrics to assess speaking skills was carried out simultaneously. By doing a thorough

research on speaking exam types in parallel to the course material and examining various rubrics, the most suitable criteria were determined with respect to the selected exam types.

whether rubrics are transparent to test takers o Test takers get familiar with the rubrics used for each exam type through the mock exams given before each speaking

exam. The rubrics are also available to test takers on cloud account and in our website.

how the evaluation and re-design process is managed (if applicable) o In the light of the oral and written feedback from both the instructors and students, the rubrics are evaluated and re-

designed with the collaborative work of the Head of English Unit, Curriculum Development Unit and Testing Unit.

27

ESKISEHIR OSMANGAZI UNIVERSITY



We have adopted a program in which all the skills are integrated. That is why, after each of the target point of the language, we reinforce it with practicing all of the skills in the classroom environment; and we assess speaking with quizzes and a speaking part of each mid-term. The tasks of the speaking are parallel to the points that are covered in the classroom.


PLACEMENT TEST We do not apply any speaking part for this exam.

PROFICIENCY TEST

We use an analytical rubric in which task management, use of language/ vocabulary/ discourse markers are taken into consideration. The bands start from 10 and they decrease according to the performance of the student. The tasks generally include talking about some general daily topics, picture description and a talking about a question related to that picture and giving their opinion about a topic presented.

PROGRESS ASSESSMENT

We use an analytical rubric in which task management, use of language/ vocabulary/ discourse markers are taken into consideration. The bands start from 10 and they decrease according to the performance of the student. The tasks generally include talking about some general daily topics, picture description and a talking about a question related to that picture and giving their opinion about a topic presented.

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) Testing Unit members are the ones who are involved in the rubric design process. Firstly, the speaking rubrics of the international exams such as TOEFL and the ones that are applied by other universities are examined; and the rubric which is the most appropriate for our aims is determined and shaped according to our needs. Lastly, the rubric formed by the Testing Unit is presented to the other instructors in our institution and it is shaped according to the feedback from them.

28

GENDARMERIE and COAST GUARD ACADEMY



We have four terms starting from A1 to B1+ and each term has two speaking assessment. The first one is formative assessment in which we allocate task in classes and ask students to practice. Students are marked three times for this assessment. The second one is summative assessment. When the term is over, we plan an individual or pair speaking tasks and assess student’s performance with two assessors. The second one lasts for 7-10 minutes for each student/pair.


PLACEMENT TEST Analytical rubric

PROFICIENCY TEST Analytical Rubric

PROGRESS ASSESSMENT

Analytical Rubric

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) We aim to give proper feedback by using analytical rubric since students need to be aware of their weaknesses and strengths. The content of the rubric is first developed by test office staff and then revisited and contributed to by all teachers. For each term and level, we hold regular meetings to develop, adapt and modify the content of the rubrics since the expectations changes. When all teachers have the opportunity/responsibility to contribute to it, there arises no transparency issue for them. This building-modifying and rebuilding process is managed by level-heads and test office members. Teachers share their ideas to change/modify any parts of the content with their reasons and a negotiation process starts and a final decision is made by all teachers.

29

IBN HALDUN UNIVERSITY



Conversation clubs (extracurricular) Contemporary topics (in-class) Mid-term exam: No speaking assessment Final exam: Speaking assessed through a set of different question types (slight differences from one level to another) Placement & Proficiency tests: Pearson machine scoring (answering questions, reading tables/charts, retelling lectures, etc.)


PLACEMENT TEST Versant Placement Test (Computer-based, speaking assessed)

PROFICIENCY TEST PTE Academic (Computer-based, speaking assessed)

PROGRESS ASSESSMENT

Book project presentations (10%), Oral presentations (15%), Group discussions (10%), Lecture projects (15%): analytical rubrics Final test-speaking exam (20%): analytical rubric (fluency, accuracy, vocabulary, communication)

OTHER (if any) --






Testing and Assessment Unit, Administration

Rubrics are given to the students beforehand.

Feedback from the instructors

Testing office meetings for evaluation and re-design processes (if necessary)

30

ISTANBUL MEDENIYET UNIVERSITY



We do not have a separate speaking lesson. We cover the speaking parts in the main course books we use. That’s why we try to assess the students considering the tasks they do in the class.


PLACEMENT TEST -

PROFICIENCY TEST Speaking part is 20 points out of 100. We use an analytical rubric. We assess grammar, vocabulary, pronunciation & fluency, discourse management & interaction

PROGRESS ASSESSMENT

Midterms – They are the same as proficiency tests.

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) Previous testing coordinator and the members designed the speaking rubric. We revise the rubric when needed. Rubrics haven’t been shared with the test takers since it has not been asked for so far. However, there is no such a rule like not sharing it. It can be considered in such situation. There are two assessors in speaking exam; one interlocutor and one assessor. It can be either one student or a pair taking the exam depending on their level. A group takes the exam one by one. There is a question-answer session including 3 parts. Questions vary according to the topics covered in their main course and reading & writing books. B groups are accepted into class in pairs. Tasks may change. Here is an example: first, there is a question and answer session with the interlocutor and the students will answer some questions individually. Then, there is an information gap session. The pairs talks to each other. Both of the students are given a cue card. Student A has the information about a place or sth. Student B has the question prompts and ask questions according to the given clues. Neither of them is able to see each other's cards. After first information gap session finishes, the pairs switch their roles with a new set of cards. Both interlocutor and assessor score the students. If there is a gap more than 3 points, they listen to the recording –after the exam finishes- and score the students again.

31

ISTANBUL MEDIPOL UNIVERSITY



Since our program is based on a “skill based” design, speaking is assessed as part of Listening and Speaking classes, overall participation in other classes and a part of Proficiency and Preparatory Language Achievement Tests. To elaborate, once student intake is over, proficiency exam is offered. This exam is held in two phases. The first phase tests students on reading, grammar and vocabulary. Students who score 70 or above take the second phase via which they are tested on reading, grammar, vocabulary, writing and Speaking. Speaking is also a part of an exam that is conducted at the end of the year (Preparatory Language Achievement Test). In addition to exams, during the year, students are asked to prepare presentations as part of Project classes. Moreover, they are constantly assessed during the classes and their overall participation forms their Teacher Assessment.


PLACEMENT TEST N/A

PROFICIENCY TEST

As mentioned above, speaking is a part of the exam and has 20% of weight. In scoring students, the rubric attached is employed.

PROGRESS ASSESSMENT

Through Project Classes and Teacher Assessment

OTHER (if any)

As mentioned above, speaking is a part of the exam named Preparatory Language Achievement Test which is applied at the end of the year. The exam has 20% of weight.





how the evaluation and re-design process is managed (if applicable) Speaking rubrics are designed or re-designed by instructors who are assigned to do so. After the instructors are assigned, they work on their tasks by taking our syllabi and objectives into consideration. Once the preparation is over, the final work is proofread and approved by Supervisors. The speaking rubrics employed are shared with students by making use of exam contents. Students are clearly informed about on which they are going to be scored in terms of their speaking skills. Finally, once a speaking exam is complete, instructors share their opinions as well as suggestions regarding the rubric, and necessary changes are made before the next time that rubric is employed.

32

ISTANBUL SEHIR UNIVERSITY



We measure our students’ speaking ability through a variety of speaking measurement tools and the weighting and nature of the exams varies from level to level. The tools include video project, oral presentation, group discussion, issue of interest, and final speaking exams. Starting from A2 level, the speaking exam is integrated into our final exams and our proficiency exam also includes a speaking part.


PLACEMENT TEST No direct speaking assessment

PROFICIENCY TEST

The weighting is 20%. Students sit the exam in pairs and the exam consists of three sections; interview, individual-long run, and discussion. An analytic rubric is employed and it has four different bands: language use (grammar & vocabulary), delivery (pronunciation & fluency), content (expression of ideas), and communication (functions & strategies).

PROGRESS ASSESSMENT

In-module evaluation, speaking ability is measured through video project, oral presentation, group discussion, or issue of interest. In module-final exams, speaking is measured as well. The weighting varies from 10% to 20%. The nature of the exam is similar to the one employed in proficiency exam. The same rubric is used for our final speaking exams.

OTHER (if any) No direct speaking assessment


who are involved in speaking rubric design how the design process is carried out whether rubrics are transparent to test takers how the evaluation and re-design process is managed (if applicable)

Four test item writers, level coordinators, and upper administration were involved in the process. Many speaking exams were analysed and a literature review was done. Accordingly, the most effective and efficient one was developed, and a rubric was designed. The test writers came together and did the literature review and analysed the existing standardized or nonstandardized speaking exams and rubrics. They also interviewed the instructors. After developing the exams, they conducted pilot studies to test if the rubric and the exam works as expected. Piloting study made it possible to determine the timing and wording of both the script and the rubric of the exam. Students have access to the rubric.

33

ISTANBUL TECHNICAL UNIVERSITY



Teaching: Spread throughout all the lessons, without a lesson that makes it a main focus. **Until 2016-17, students had 4 hours of "speaking-listening" classes per week, in one block per day. Testing: 1 exam per semester, based on the Cambridge Mainsuite exams and the IELTS. Assessment: Some video training, on-going development.


PLACEMENT TEST None

PROFICIENCY TEST None

PROGRESS ASSESSMENT

Varies year to year and is included in the participation grade.

OTHER (if any) A holistic scale with descriptors similar to the IELTS public ones is used.


who are involved in speaking rubric design how the design process is carried out whether rubrics are transparent to test takers how the evaluation and re-design process is managed (if applicable)

ITU has to deal with huge classes and understaffing, so it adopts the theory of Cambridge/IELTS to facilitate training, procedure, video examples for students, levels, and assessment. This has been successful and students and teachers have both responded well.

34

IZMIR UNIVERSITY OF ECONOMICS



o Our institution has adopted the integrated skills approach through communicative teaching. o As a productive skill, speaking is as equally important as the receptive skills. o There are five in-modular assessments and speaking is one of them. The module exit exam, “Gateway Exam”

also has a speaking component. In addition to these, the Proficiency in English exam has a speaking component as well.

o Throughout the program, students have the opportunity to practice speaking in the classroom and outside the classroom by attending extracurricular activities.


PLACEMENT TEST N/A

PROFICIENCY TEST

The speaking exam is composed of the following parts: PART 1: Question and answer PART 2: Role-play PART 3: Discussion PART 4: Extended Individual long turn 20 points – analytical rubric – between an interlocutor and a candidate The speaking rubric includes the following: Content, Discourse Management, Pronunciation, Grammar & Lexis.

PROGRESS ASSESSMENT

In-modular assessment: 20 points – analytical rubric Tasks vary according to the level; picture description, information gap, discussion, personal questions, to name a few. The speaking rubric includes the following: Content, Lexis, Fluency & Pronunciation, Grammar

OTHER (if any)

Gateway / Module Exit Exam: 20 points – analytical rubric Tasks vary according to the level; prompted talk, discussion, personal questions, to name a few. The speaking rubric includes the following: Content, Interaction, Pronunciation & Discourse Management, Lexis, Grammatical Structures





how the evaluation and re-design process is managed (if applicable) o The members of the testing unit and an external assessment body was involved in designing both the

tasks and rubrics for each assessment tool.

o Extensive research was done during the design process.

o The rubrics are transparent to both test takers and examiners as the rubrics and sample assessments are available for all parties involved to refer to.

o If the rubric/tasks need to be revised, this will be done based on feedback from test takers and examiners.

35

KARABUK UNIVERSITY



Modules: Module Average %100 M1(A1/A2) Speaking Portfolio %10 M2 (A2/B1) Speaking Portfolio % 10 M3 (B1) Speaking Portfolio % 10 + End of Module Test (%45) speaking asses. (%20 of EMT) *M4(B1+/B2) Speaking Portfolio % 10 + End of Module Test (%45) speaking asses. (%20 of EMT) *English Language and Literature exit level In M1, M2 and M3, core lesson is integrated skills (IS) lesson where students engage with four skills activities as well as sub-skills like vocabulary. Students have 20 hours of IS in a week in each module. Every day they have 4 hours of IS lesson, which means they have the opportunity to utilise activities to promote their speaking skills. In M4, students engage with speaking activities mainly in listening & speaking skills lesson, which is 6 hours a week, but speaking skill is also supported in reading & writing skills lesson, which is 10 hours a week. During these lessons, not only question-answer based activities but also pair and group work tasks are often included in lesson plans especially as part of production activities. Besides, in order to be able to apply such activities in class more effectively, the class size is limited to 18- max. 24 students. As both in curriculum and in every lesson, speaking skill is given an important place, as much as the other skills, it is also included in the assessment process. The details about assessment have been described below.


PLACEMENT TEST Not applicable

PROFICIENCY TEST Not applicable (organized but did not manage to apply due to the number of students)

PROGRESS ASSESSMENT

Speaking Portfolio comprises %10 of Module Average. In every module, speaking portfolio comprises of three tasks. The tasks are designed according to level objectives. Some of the tasks are done in class (such as short presentation / role-play) and some of them are video recorded outside the class. During integrated skills lesson, students engage with various speaking skills activities and they are given a task in which they can perform the speaking skills they have gained; such as, opinion giving discussions, restaurant conversations role-play, mini presentation of hometown description and etc. Students are informed about the objectives and expectations of the task when assigning.

OTHER (if any)

As Module 3 and Module 4 exit levels, %20 of End of Module Test (EMT) is speaking skills assessment. (EMT is %45 of Module Average and speaking exam is % 20 of EMT). The exam includes 2 parts: 1) interview with interlocutor: question and answer, 2) expressing opinion on a given topic and commenting on second examinee’s opinion. Students are informed about all the steps of EMT Speaking exam with sample videos, exam specification and rubrics before the examination.

36





how the evaluation and re-design process is managed (if applicable) Each speaking portfolio task is designed by level coordinators according to level objectives. Each task has its own rubric, though might include common items, as each task requires different level of language expectations. The rubrics are designed by standardization unit and checked with level coordinators. During the academic year, depending on the effectiveness of the task &rubric and on feedback from both teachers and students, rubrics might be changed by standardization unit. Rubrics for speaking portfolio tasks are published in Portfolio Tasks Handbook which includes information and guidance on how to prepare a task, how to keep portfolio, task objectives with language expectations, deadlines and rubrics. The handbooks are delivered to every classroom. Instructors give students advice and guide them about every detail of the task using the handbook. At the end of every module, level coordinators organize a meeting with both instructors and students (class representatives) on the effectiveness of the tasks. Depending on the feedback and grade statistics of students, the tasks are completely changed or re-designed for the next term. Speaking Exam of EMT Students are informed about the speaking exam via website announcements and in-class presentations. The parts, the weight, length, objectives of the exam are described in exam specification. With a sample video, it is presented in class by the instructors. After the exam, depending on the interlocutors and assessors feedback, parts of the exam can be re-designed. As an instance, a third part (role-play) of the speaking exam was eliminated after instructors’ feedback.

37

KOÇ UNIVERSITY



At Koc, we have speaking exams (midterms/finals) as achievement assessments as well as cumulative speaking production tasks with individualized grading rubrics. As a testing committee member, I do not get involved in the creation/adaptation of these rubrics directly, though feedback is offered at the beginning of each semester to the Course Leaders, who are responsible for producing common grading rubrics for all instructors of the course. As a testing committee member, I coordinate the development and modification of the proficiency exam speaking grading rubric, tasks, and general specs of this exam (called KUEPE – Koc University English Proficiency Exam). The proficiency exams are offered 4 times each year, after each semester (fall, spring, summer) and once again before classes begin in September (only for potentially returning students).


PLACEMENT TEST No speaking

PROFICIENCY TEST

Individual speaking exam with 2 examiners and 1 proctor similar to the IELTS format: holistic rubric with subskills (task achievement, grammar range & accuracy, lexical range & accuracy, delivery & intelligibility) and 7 bands to score from 0-20.

PROGRESS ASSESSMENT

Many Course Leaders have designed the speaking production assessment rubrics to look similar to the KUEPE rubrics but they have the autonomy to alter them and adjust them to the level being taught (Foundation/Pre-Int/Int/UpperInt). Individual speaking tasks are also custom designed depending on the task such as academic discussion, presentation, debate, etc. The Koc University prep program allows its teachers a great deal of autonomy in this sense to reinforce a sense of team and horizontal organization rather than supporting a vertical hierarchy of decision-making about classroom management. As student GPAs do not play a direct role in their proficiency performance, we feel that this system works well at our institution.

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) The rubrics we currently use for our proficiency exam were developed several years ago (I was not directly involved at that time) but our current Assessment Committee Head was a part of that process, along with 3 other testing team members. It was piloted and tested, and we continue to encourage feedback from instructors each time it is used. That feedback and the testing committee’s own revision of the rubrics (both writing and speaking) takes place at the beginning of each academic year. While feedback continues to be elicited, changes are not made until the end of that academic year.

We share the KUEPE rubrics on our course webpages, and use them during office hours and practice sessions that are held before each proficiency exam, so they are publically accessible to all students and instructors.

Speaking rubrics for achievement assessment are designed by individual Course Leaders. They may request input by other instructors teaching the course, but they often develop the rubric individually. Feedback sessions by instructors who taught the course are held at the end of each semester, and speaking rubrics are adjusted and managed at that point.

38

MANISA CELAL BAYAR UNIVERSITY



Speaking exams are essential part of Midterm and Final Exams in MCBUSFL. They are executed on the following day of written exam. On the speaking exam day, students attend as pairs according to the announced schedule; and one assessor and one interlocutor are assigned for each session.


PLACEMENT TEST Not Available

PROFICIENCY TEST Not Available

PROGRESS ASSESSMENT

We have a speaking part in our Midterm Exams (16%) and Final Exam (16%) in which analytic rubric is used; and content & discourse, pronunciation & intonation, and control of grammatical forms and range of general and topic vocabulary are tested.

OTHER (if any)

Teachers may assign Ss to prepare videos/interviews as extra-curricular activity, however these homework and/or projects remain ungraded.





how the evaluation and re-design process is managed (if applicable) Curriculum and Assessment Units worked together in designing the rubric, and continuously collaborate on updates taking the feedback from the students and teachers into consideration. Rubrics and Pair Lists are announced on the notice board beforehand so the Ss can see the criteria and speaking exam dates.

39

MEF UNIVERSITY



Speaking is one of the primary focuses in our classroom. We even receive training on how to make our lessons more communicative. We have included a level of English and amount of English criteria to our classroom participation rubric. However, we only assess speaking twice a module – the beginning and end of the module. We have shied away from including it in our End of Module exams because of logistical problems and the highly subjective nature of speaking exam.


PLACEMENT TEST N/A

PROFICIENCY TEST

Our Rector has elected to use the Versant English Placement Test as our proficiency. This exam includes a speaking component.

PROGRESS ASSESSMENT

2 assessments – 1 in the middle of the module and one at the end. Both are worth 2.5% totalling 5% of the final grade.

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) The speaking rubric was designed by a few teachers. I am not sure how they carried out the process. When I became an assessment coordinator, I was only allowed to make minor changes. We use an analytic rubric with 4 criteria: Grammar range and accuracy, Vocabulary range and accuracy, Fluency, Coherence and Pronunciation and Interactive Communication-Task Completion. There is different a rubric for level 1, Level 2 and a separate rubric for levels 3 and 4. We have a session in the beginning of the school and we go over the speaking rubric with all the students. At the moment, there is no procedure to change the rubric, but I am interested in changing the rubrics to match our student profiles and to find out what the minimally competent student is for our institution.

40

ÖZYEĞİN UNIVERSITY



Speaking assessment is integrated into the lessons. It is a part of continuous assessment.

In A2 level, we attach importance to having speaking practice in the lessons so that students can give simple and short descriptions, can provide information about familiar and everyday topics and can make simple claims supported by basic reasons in predictable situations. They can use simple phrases to ask and answer questions and exchange ideas.

In B1 level, speaking is assessed through two different tasks that are a presentation and a discussion. By the end of this level, students can give coherent descriptions and provide factual information, and can make claims supported by reasons and explanations about general topics such as health, education, or work. They can communicate with some confidence by initiating, maintaining, and closing interactions on general topics including less routine and unprepared situations.

In B2 level, speaking is assessed through two different tasks. One of them is a task related to chart interpretation. By the end of this level, students can verbalize data in a chart/table to analyze and interpret data and trends, can give detailed, fluent, and well-structured talks on a wide range of topics to fulfill a variety of purposes. They can interact spontaneously in tasks and situations that are less predictable, and can extend their ideas and sustain the communication. The other one is a speaking test which is given at the end of the semester.


PLACEMENT TEST None

PROFICIENCY TEST None

PROGRESS ASSESSMENT

A2 - 5% of the total grade (An analytical rubric is used to evaluate oral presentations. – Organization, content, delivery, and language are tested. – The bands are marked according to these criteria: Good - Average - Needs improvement - Completed/done - Not completed/Not done.) (A holistic rubric is used to evaluate conversations. – Interactive communication, grammar, vocabulary, and pronunciation are tested. – The bands are marked with ratings from 0 to 5.) B1 - 5% of the total grade (An analytical rubric is used to evaluate oral presentations. – Organization, content, delivery, and language are tested. – The bands are marked according to these criteria: Good - Average - Needs improvement - Not completed.) (A holistic rubric is used to evaluate discussions. – Interactive communication, fluency and coherence, grammar, vocabulary, and pronunciation are tested. – The bands are marked with ratings from 0 to 5.) B2 - 5% of the total grade (An analytical rubric is used to evaluate chart interpretation. – Task achievement (includes numerical info description and reasoning), grammar, vocabulary, fluency and coherence, and pronunciation are tested. – The bands are marked according to these criteria: None - Poor - Average - Satisfactory - Good.)

OTHER (if any)

Speaking Test - only in B2 level - 15% of the total grade (A holistic rubric is used for the test. – Chart interpretation (includes numerical info description and reasoning), interactive communication, grammar, vocabulary, fluency and coherence, and pronunciation are tested. – The bands are marked with ratings from 0 to 15.)






Curriculum and assessment team designs the rubrics and feedback from the instructors is incorporated.

The first step is to match the objectives with the tasks in related levels. Then comes deciding all the general headings according to the intended outcome. Under the headings expectations are written clearly using appropriate language taking the scales into consideration. Later, after collecting sample student outcomes, the existing rubrics are adjusted.

Rubrics are transparent to the test takers from the very beginning of the course. Each band is explained to them and then they have a chance to use the rubrics to assess their own or classmates’ performances.

After using the rubrics for the tasks, according to the instructors’ feedback and classroom experiences, we make revisions on the rubrics. Depending on the feedback if there is no need to make changes, rubrics are used as they are for the upcoming tests.

41

SABANCI UNIVERSITY



Each level (Route) has one or two speaking exams, which are based on the themes covered in the coursebooks. Speaking is an important aspect of language curricula but in our program, we do not give too much emphasis on teaching or assessing speaking (i.e. teaching micro skills of speaking and testing them). We aim at helping students develop the skill in a natural way through interactive activities and communicative components of the program.



PROFICIENCY TEST Not applicable

PROGRESS ASSESSMENT

Learning Portfolio (LP)- Speaking Tasks:

1-2 tasks in the LP but it is only graded as part of the task completion section of the LP and does not have a separate weighting. (LP is 5%)

Task completion criteria derived from the OA rubric depending on the nature of the task

Oral Assessment (OA):

Routes 1, 2, 3: 10% and Route 4: 5% of the overall course grade

Analytical rubric

Bands: Task fulfilment/ idea development, Fluency & Coherence, Interaction, Use of English (each band out of 4, total is converted to 100)

OTHER (if any)

Exit level /Route 4: CTSS- Critical Thinking module has a presentation component. Proj 001- Project based learning has a presentation component.





how the evaluation and re-design process is managed (if applicable) The descriptors are based on the CEFR level descriptors, and reflect the subskills that are taught in the course. For instance, stress and intonation is not mentioned in the criteria since we’re not really teaching them apart from awareness raising. Rubrics were designed by a task group, and the assessment team members revise them if there are any issues through collecting feedback from teachers As with all of our exam rubrics, speaking criteria/rubrics are shared with test takers who are also trained through in class tasks and LP tasks. We do not have a separate “student version”.

42

SOCIAL SCIENCES UNIVERSITY OF ANKARA



We focus on speaking in classroom teaching as much as we focus on other main skills of the language. In line with this principle, it is assessed from both formative and summative aspects.


PLACEMENT TEST There is not any weighting of speaking in the placement test.

PROFICIENCY TEST

The weighting of speaking is 25 % and both holistic and analytic rubrics are used. The sub-skills tested consist of sharing ideas with reasons and examples as well as participating in an acamic discussion. The bands are content of message, pronunciation, quality of interaction, fluency, and volcabulary & language control.

PROGRESS ASSESSMENT

In addition to the speaking portfolio, the speaking performance of the students are assessed via the speaking quiz and the End of Course Test (ECT) in each level. The weighting of speaking quiz and the speaking part of the ECT are 5 % in each. The subskills change depending on the level expectations and objectives of the weekly program covered before the test. The bands for elementary and pre-intermediate groups are content of message, comprehensibility of message and vocabulary & language control. For intermediate and higher levels, the bands are content of message, pronunciation, quality of interaction, fluency, and volcabulary & language control.

OTHER (if any) The weighting of speaking portfolio is 12 % in every level. The subskills change depending on the focus on the curriculum content of the related level.





how the evaluation and re-design process is managed (if applicable) In the design of speaking rubric, both classroom instructors and Testing Office members are involved. Class instructors share their ideas with the test developers via a questionnaire and interviews. Based on their ideas, the content of the rubric is defined by the test developers. The PDU office organizes sessions to familiarize the class teachers with the new rubric. The rubric is shared with the test takers. At the end of each academic year, the feedback from the class instructors are collected and the rubric is revised or redesigned if necessary.

43

TED UNIVERSITY



Speaking assessment is integrated into our program by weekly speaking tasks which are collected in students’ learning portfolio and three speaking exams throughout the semester. We have three different levels. Although the focus is not mostly on speaking in elementary level, speaking is equally distributed in terms of teaching hours in intermediate and upper levels. For both of these groups, we allocate 10 hours to teaching listening/speaking. At the end of each unit, the students (including elementary students) are asked to perform a speaking task. They collect these tasks in their portfolios which are graded at the end of the semester.


PLACEMENT TEST No speaking assessment

PROFICIENCY TEST

25% of the proficiency exam is spared for assessing speaking. We are using an analytical rubric which has three bands: content, delivery, lexical, pronunciation and grammar and each of these bands weighs equally (5 pts).

PROGRESS ASSESSMENT

Our elementary, intermediate and upper students receive 15% of their cumulative grades from three speaking exams which are held in every 4-5 weeks. Each of these exams is out of 15 points and we calculate their average at the end of the term. They also have analytical rubrics which differ from level to level. In elementary level, we have three bands: accuracy in lexis, accuracy in grammar and delivery. In intermediate and upper levels we have: content, accuracy in lexis and grammar, delivery

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) First of all, the objectives of the books that we study are examined thoroughly. After we integrate them into our curriculum, we decide on our speaking tasks and how we are to assess them. We first prepare our speaking tasks criteria and try to align our speaking exam rubric accordingly to be standard both in our in-class practices and in assessment. For the design period, the Core Team (who is responsible for testing and curriculum) starts working on the rubric after the books and the plans are finalized and before the semester begins. Although there are some people who are working on one specific level, these people cross check their rubrics to make sure that the rubrics are somehow parallel across levels. We try to share the rubrics with the students usually 1-2 weeks before their first speaking exam to make sure that they are familiar with how they are going to be tested and graded.

44

TOBB UNIVERSITY of ECONOMICS and TECHNOLOGY



In the speaking assessment, students are expected to interact at conversational English in a range of contexts and topics to be evaluated by two instructors, which counts as one of the four total quizzes in one term. It contains three parts; personal interview, picture description and discussion on a randomly chosen subject respectively. Students take the Speaking Test individually and they are provided with blank note papers and pens so that they can take notes during the exam. At the end of this process, two instructors evaluate their speaking skills by using analytical rubrics and take the average of the score that they give. If the difference between the scores is above 15, the student is reevaluated.


PLACEMENT TEST There is no speaking part in the placement test.

PROFICIENCY TEST There is no speaking part in the proficiency test.

PROGRESS ASSESSMENT

Speaking assessment is both intensive, responsive, interactive and extensive varying in relation to the part of the evaluation. The first part of the exam in which students are interviewed is both responsive and intensive. The second part where students describe a picture is intensive and sometimes responsive. The third part which is impromptu speech is extensive. Speaking assessment is carried out using an analytical rubric comprising five sub-skills that are grammatical accuracy and range, vocabulary, content, fluency, pronunciation and an overall part. Each of these parts is evaluated on a 5 point scale and then the total score over 30 points is calculated using a conversion table over a hundred points. Speaking quiz’s weighting is 7% in the overall progress. Stimulus in the quiz is oral, visual and written.

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) Speaking rubric is developed by Measurement and Evaluation Unit and then approved by Administrative Unit. The rubric was first created by gathering data from a variety of sources including other universities’ speaking rubrics. Taking the needs and profile of the students of our institution into consideration, the rubric was created. After implementing the rubric, feedback was taken from instructors and final version of the rubric was shaped. Speaking rubrics are shared with instructors at the beginning of each term so that they can share it with students. In case of any feedback from practitioners rubrics are revised by Measurement and Evaluation Unit.

45

UFUK UNIVERSITY



The program is based on course materials (only coursebooks and worksheets). Hence, the speaking parts in the coursebooks are the only time that the program mentions speaking skill. Therefore, the teaching and the assessment of the speaking skill is only based on the coursebook and it has no place in proficiency, placement, midterms, quizzes or any other assessment types.


PLACEMENT TEST There is no part for speaking in the placement test.

PROFICIENCY TEST There is no part for speaking in the proficiency test.

PROGRESS ASSESSMENT

There is no part for speaking in the progress assessment.

OTHER (if any) ---






-----

46

UNIVERSITY OF TURKISH AERONAUTICAL ASSOCIATION


Please briefly describe how speaking assessment is integrated into your program. (i.e. the focus given to teaching and assessing speaking in the overall program) Speaking assessment is handled in 3 different ways. First of all, there are mid-course and end-of-course speaking tests which are parallel to the course books (New Language Leader series). Secondly, there are performance assignments in 2 parts. One of them is different speaking tasks which are assigned to students in each unit in the process. These tasks are parallel to the themes of the book units and also a part of a portfolio process. The other is that in each period students have one project to present at the end of that period. These are PPT Presentation, My Series, Project Based Learning (PBL) Presentation, and Class Magazine Presentation. Last but not least, as formal tests there are end-of-term speaking tests, and at the end of the program a speaking assessment takes place as a part of the proficiency test.



PROFICIENCY TEST

Speaking assessment has 3 parts which are practiced in the end-of-term speaking tests at the end of each period as mentioned in detail in “Other” section below. It covers 25% of the total test grade. The rubric is similar to the one used in the end-of-term tests. It has grammar-vocabulary, pronunciation, and discourse management parts and each part is grades out of 5 bands.

PROGRESS ASSESSMENT

Throughout the academic year, students have speaking tasks parallel to the units of the course book which is New Language Leader series. These speaking tasks are a part of NLL Portfolio. They cover 30% of the total portfolio grade, and in each period a separate NLL portfolio takes place. Related to the themes in each unit, a speaking task is assigned to the students. Based on the task requirements, they speak 2 to 5 minutes, audio/video record it and send to the instructor (individual, dialogue, discussion etc.). They are holistically evaluated based on their length and quality. Instructors also give written feedback for each task. The second component in this part is performance assignments. In each period, students are given a different project and asked to present it in the class. As mentioned above, these are PPT presentation, My Series, Class Magazine presentation, and PBL presentation. These presentations cover 10% of the total grade in that period. Holistic rubrics are used and they include parts such as content-organization, language use, visuals, fluency, and overall.

OTHER (if any)

At the end of each period, an end-of-term speaking test takes place. It is basically the same as the one for proficiency test. There are different parts in each test chosen from the following: warm-up and introduction, picture description, production, and role play. Production and role play parts are used interchangeably. The tests cover 5% of the total grade in each period. A holistic rubric is used. The rubric also changes as the level of students increases. Mainly it has grammar-vocabulary, pronunciation, discourse management, and interaction parts and each part is graded out of 5 bands.





how the evaluation and re-design process is managed (if applicable) In the process of rubric design, the members of Curriculum and Testing Unit come together and design rubrics separately for each speaking task-test. Task requirements, skills needed and student level are taken into consideration to be able to decide on parts and bands to evaluate student performance. Before putting them into application, some/all instructors give feedback to those rubrics, and based on the feedback the rubrics are revised if necessary. All rubrics are shared with students before the tasks/tests. After application, instructors who are involved in assessing speaking performance process give feedback on how much rubrics are useful and how well they worked in assessing performance. Then again if necessary, the rubrics are revised.

47

YASAR UNIVERSITY



--


PLACEMENT TEST NA

PROFICIENCY TEST We have just revised the speaking part of our proficiency test.

PROGRESS ASSESSMENT

We do not assess speaking in the Final or Midterm exams. Each module, one speaking quiz is conducted in Week 9.

OTHER (if any)





how the evaluation and re-design process is managed (if applicable) We use holistic rubrics for the speaking quiz and proficiency exam. Both of our rubrics are transparent to test takers. Instructors can explain speaking quiz rubric in their L/S lessons. Flat Speaking Rubric is available online. Speaking tests are conducted by EFL instructors. Testing office members conduct standardization meetings each year for both speaking quiz and proficiency exam. Speaking Quiz: Examiners use assessment criteria to award a band score for each of the four criteria: Range, Accuracy, Fluency, Task Assessment The criteria are weighted equally and the Speaking Band score is the sum grade given on each. Throughout the test, students are assessed on their individual performance. They are awarded marks by one examiner. Speaking Rubrics were designed by curriculum office members. During the accreditation process, the syllabi were designed initially. Finding relevant scales and descriptors in the CEFR, curriculum office members then stated the language proficiency level at which students are expected to be able to achieve the objectives. Then, the purpose of the speaking test was identified and the test was planned. Then, a thorough research was conducted. CAE Speaking Assessment Grid was adapted to fit our context. Proficiency Exam: Examiners use assessment criteria to award a band score. (Beginner, Developing, Satisfactory, Very Good, Excellent) Each candidate takes the test individually. They are awarded marks by two examiners. Speaking rubric was designed when our institution was accredited in 2011. Speaking Rubric is more compact version of the Speaking Quiz Rubric.

48


Documents

A MESSAGE FROM THE FOAI ORGANIZING COMMITTEE · 2017. 3. 10. · exams in this case, have been proposed, such as increased consistency of scoring, the possibility to facilitate valid