Upload
weston
View
36
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Online CEF-Based Assessment of Oral Proficiency for Intercultural Professional Communication Lutgart Dusar Prof.dr Lut Baten dr. Jan Van Maele . 1. CEFcult, the sequel. With CEFcult we want to contribute to - PowerPoint PPT Presentation
Citation preview
Online CEF-Based Assessment of Oral Proficiency for Intercultural Professional Communication
Lutgart Dusar <[email protected]> Prof.dr Lut Baten <[email protected]>dr. Jan Van Maele <[email protected]>
2-12-2010 www.cefcult.eu
1. CEFcult, the sequel
• With CEFcult we want to contribute to• ongoing development & validation of CEFR (scaling
language proficiency) and INCA (intercultural competence assessment) frameworks
• a new e-learning culture bringing together self-learning, the worlds of work and education
• mainstreaming e-Portfolio-based assessment
2-12-2010 Www.Cefcult.Eu
2. From CEFR (‘93-96) to WebCEF (‘06-09)
• WebCEF established a social learning community for self, peer and expert assessment in which both productive and interactive performance in a foreign language is video recorded and online displayed to be assessed by CEFR-scales.
• http://www.webcef.eu/
2-12-2010 www.cefcult.eu
… to CEFcult (2009 – 2011)
+ Intercultural Communicative Competence Assessment: Allowing a set of communication tasks (scenarios)
through which foreign language learners can assess their speaking skills for intercultural professional purposes
Map the road to a tested and viable model for online collaborative assessment among HEi, enterprises and various partnership brokers.
2-12-2010 www.cefcult.eu
e-Portfolio (Europass: Language, Mobility)
Video sample
ScenarioScenario Database
Social media
(LinkedIn)
Export to:
Selfassess
Peerassess
Expertassess
Select scenario
Add profile
Create sample
Followscenario
Feed-back
Invitation to:
Add scenario
2-12-2010 www.cefcult.eu
CEFcultUser
Groups(Alumni,..)
3. Assessing ‘Intercultural Competence’
• The model: INCA (EU 2001-04) criterion based assessment model, linking Byram’s framework to learning needs of (engineering) industry
• Scaling: to what extent INCA criteria including value systems, are developed and put into action: Full – Intermediate – Basic– Respect for otherness– Ambiguity tolerance– Knowledge discovery– Empathy– Behavioral flexibility– Communicative Awareness
• Warning “INCA tasks and grid are tentative “
2-12-2010 www.cefcult.eu
- INCA based online scenarios
2-12-2010 www.cefcult.eu
• Simulations designed in a way to guarantee fairness, reliability and accuracy
• Work-sample approach: a selected real-world task transformed into assessment task
• Challenge: lack of definition in INCA descriptors
- Different values, different ratings
‘I am honest’:– Considered a quality in low context cultures: ‘we say what we mean,
we mean what we say’– Considered ‘blunt’ in high context cultures where a meaning is stated in an indirect way
What matters: cultural (self) awareness:
• O bserve• D escribe• I nterpret• S uspend your judgment, look for and verify other perspectives before E valuating
(S.Ting Toomey)
2-12-2010 www.cefcult.eu
- Pilot Leuven ‘Screening interview’1st experiences(October, November, December 2010)
Positives • Descriptors are written generically,
can be used with many tasks• Corresponding rating criteria
determined through collaboration of language assessment specialists and (lay) stakeholders from the workplace
• Expert raters from the world of work tend to apply scales consistently, resulting in more reliable measurement.
Critical• Summative work-sample testing
mainly indicates a degree of overall communication competency, much less in detail
• Performances may meet INCA criteria in two or more categories, making it difficult to select the one best description
• Peer reviews (student community) do not provide specific feedback to test takers about the strengths and weaknesses of their performance.
2-12-2010 www.cefcult.eu
- Task specific descriptor writing & review cycle
Peer assessment
Task descriptor acceptable, understood?
Flag(What’s the problem?)
Commentexplanation
recommendation
Task specific
Yes
NoInternal review
4. A Tested Model
Specification •Experiment with INCA scales for the most meaningful feedback•Outcomes: development of graphic profile, bank of scales
Standard Setting •Benchmarking
•Test validity: getting the CEF&INCA rating levels right
Validation •Writing a living ‘CEFcult Technical manual’ containing recommended usage and flexible implementation guidelines
2-12-2010 www.cefcult.eu
5. A Viable ModelOutcome: e-Portfolio based assessment
1) The emergence of new global actors will make our world more colorful and more polyvalent. There is a need for foreign languages learning, inter-personal & inter-cultural monitoring and development tools in a safe, respectful environment.
2) Existing ICC-assessment tools mostly address criterions and references for individual competence development; not enough the organizational needs and seldom those of development partnerships (N-S).
3) E-Portfolio tools must go mainstream, embedded in the internal learning and monitoring processes of companies, organizations and higher education institutions.
2-12-2010 www.cefcult.eu
References• John De Jong & Glyn Jones , Getting the levels right: deriving item writer
guidelines from CEFR descriptors (Pearson, Item Writer Guidance-extract ppt)
• Peter Lenz and Raphael Berthele, Assessment in Plurilingual and Intercultural Education. Paper Policy Forum ‘The right of learners to quality and equity in education – The role of linguistic and intercultural competences’, Geneva, Switzerland, 2-4 November 2010
• Byram• …
2-12-2010 www.cefcult.eu
www.mymahara.com
References• Peter Lenz and Raphael Berthele, Assessment in Plurilingual and
Intercultural Education. Paper Policy Forum ‘The right of learners to quality and equity in education – The role of linguistic and intercultural competences’, Geneva, Switzerland, 2-4 November 2010
• Elisabeth Prechtl and Anne Davidson Lund, Assessing and developing intercultural competence. Perspectives from the INCA Project in Handbook of Intercultural Communication, Section 5, ch.22, p.467-490(2007) eBook ISBN: 978-3-11-019858-4
• John De Jong & Glyn Jones , Getting the levels right: deriving item writer guidelines from CEFR descriptors (Pearson, Item Writer Guidance-extract )
• www.cefcult.eu - www.incaproject.org - http://www.webcef.eu/
2-12-2010 www.cefcult.eu