Upload
seful-bahri
View
51
Download
4
Embed Size (px)
Citation preview
Joan Jamieson’s Putting Principles
into Practice an article summarized by:
SEFUL BAHRI16705261003
Yogyakata State University student
This article is written by Joan Jamieson, Carol A Chappele, and Sherry Preiss
Purpose of the studyTo illustrate how a set of principles informed a series of processes in the evaluation of Longman English Online, a commercial CALL product for teaching intermediate English as a second or foreign language (ESL/EFL).
CALL Product to Evaluate: Longman English Online 3 (LEO 3). it’s the
intermediate level of the video-based multimedia, integrated language skills program.
The units not only contain listening comprehension practice, but also include practice with grammar, vocabulary, pronunciation, speaking, reading, research on the web, and writing.
Quizz in every unit, longer tests at the end of every four units (“module test”), and one at the end of all twelve units (“the end –of-level test).
Nature of the evaluation : The evaluation was focused on the materials
rather than on the tasks that the leaners actually carried out as they were using the materials
It was judgmental evaluation (: based on the logical analysis of a CALL activity), NOT empirical evaluation (: based on the quantitative or qualitative analysis of a CALL activity through observed data which are summarized by the evaluator)
CALL Evaluation Principles and their associated criteria & vriables
Principles Criteria Variables Examples
1. Language Learning potential should be the central concern when developing & evaluating
* to indicate CALL quality. (Chapele. 2001)
Enhanced input
Input saliece, modification, elaboration
Text highlighted on the screen
interaction Between people, between a person & computer, within a person
Learner-learner communication tasks, grammar help available
production Planning, correcting production, using help
Taking notes suggested, model for constructed responses
Principle Criteria Variables Examples of Operationalized variables
2. CALL should be evaluated in terms of the quality assessment in addition to instruction
*to indicate the usefulness of CALL test. (Bachmer & Palmer)
Authenticity Domain Sampling
Correspondence between tasks on the test and in courseware
Construct Validity
Score interpretation
Meaningfulness of scores
Principle Criteria Variables Examples of Operationalized variables
3. The criteria to conduct the analysis should be drawn from theory and practice in SLA & Lang. Assessment4. Judgemental evaluation through defined variables is valable for examining materials in a way that can improve future design
All of the previously mentioned criteria were examined, to find whether or not they had been operationalized in LEO 3
These criteria were investigated as they have a relatively strong theoretical, pedagogical, and empirical base:
Theory and research on SLA converge to suggest that the likelihood of learners acquiring linguistic input increases if their attention is drawn to linguistic features, using techniques such as marking salience, modification, or elaboration. (Schmidt, 1990; Sharwood Smith, 1993; Robinson, 1995; Skehan, 1998).
Benefits and types of interaction for langugage learning (Ellis, 1999; Kol & Schcolnik, 2000; Lantolf & Appel, 1994, Long, 1996, Pica, 1994)
Swain (1985): From the cognitive perspective, the benefits of producing language may be enhanced when learners have the opportunity to plan before speaking or writing and to correct linguistic output, which can be prompted by feedback from others or from self-evaluation.
From sociocognitive perspective, Swain (1998) stated that the help from the interlocutor allows for production beyond what the learner can accomplish alone.
Authenticity is explained as the degree of correspondence between the characteristics of a test-task and the features of a task in “target language use” (Bachman & Palmer, 1996).
Achievement test is considered authentic in terms of the degree to which its tasks & directions mirror the content & skills in an insructional unit (:domain sampling)
Construct Validity dals with the appropriateness and meaningfulness of the interpretations which are made on the basis of a test score.
Summary of Judgemental Evaluation of LEO 3
Area Criteria Variables Operationaizations
Evidence Judgement
Language Larning Potential
Enanced input
salience Font size/type/color, highlighting, repetition
Font size/highlighting graphics, animation, audio repetition
Good
modification Simplification, repetition, clarification, L1 transl., L2 definitions, image/video
Images/video, transcripts, L2 definitions
Good
elaboration Adding grammatical phrases or clauses to texts
Adding grammatical phrases
Fair
Language Learning Potential
Interaction
between people
Amount/types of co-constructed and attention to form
Meaningful tasks
Good
People and computer
Potential of providing enhanced input, obtaining help using the language
Program correction of learner’s explanations, culture notes
Good
an inner voice
choices
Language Learning Potential
Production
planning Instructions to plan, directed activities, knowledge of expectations
Directions to plan, pre-writing exercises
Good
correcting Amount of self-correction, amount/type of signals from computer or another person
Self-correction of speaking/ pronunciation teacher
Fair
help Accessing dictionary, grammar, models
Glossaries, transcripts, grammar help, models of speaking and writing
Good
CALL assessmentAthenticityConstruct validity
domain sampling
Correspondence between tasks on tet & in courseware
Table of specifications, directions, texts, item types
Fair
score interpretation
Appropriate, meaningful
Reporting scheme
Good
CONCLUSIONbased on judgments on the criteria of
language learning potential as well as authenticity and construct validity of assessment, LEO 3 received a good evaluation.
This evaluation also points to areas in which LEO 3 might be improved in the future and it suggests that LEO 3 might be worth pursuing further through the addition of an empirical evaluation.
=== end of the slides ===