L1-L2 sentence translation in classroom grammar tests

L1–L2 sentence translation inclassroom grammar tests

Ilana Salem

L1–L2 translation of separate sentences is one kind of task format used bymainstream EFL teachers to assess their learners’ grammatical accuracy. Aimedat improving teacher-written translation items, this study analyses linguisticfeatures potentially causing such decontextualized cues (and their targetresponses) to sound odd or untypical of naturally used language. The findings showthat some items elicit structures that are marked in their contexts, others aretranslationally overchallenging, and yet others have textual flaws. The followingresearch method was employed: English-Hebrew bilinguals, mostly EFL teachers,translated seven cues from Hebrew to English. This resulted in numeroustranslation options for each cue. The analysis of these multiple translations andtheir comparison with corpus frequency data yielded most of the above-mentionedfindings. In addition, another group of Hebrew-English bilinguals translated theprevious group’s most frequent translations back into Hebrew. This addedinformation about the textual quality of the Hebrew cue.

Introduction Writing classroom tests is a routine task for many EFL teachers inmainstream schools. Classroom grammar tests and quizzes (henceforth‘tests’) help consolidate recently presented structures, keep studentsattentive and aware of the language points covered, and provide evaluativeinformation regarding learners’ progress in forming and usinggrammatical structures. This information is obtained by elicitation ofparticular language forms, often by presentation of individualdecontextualized sentences generating the target form by triggeringdevices.

One of the popular elicitation techniques is based on trigger-structureassociation. A‘trigger’ could be a word or an expression typically collocatingwith the target form. For example the trigger ‘yet’ in ‘She ______ (not come)yet’ signals the target response: ‘She hasn’t come yet’. Logical relationshipsbetween clauses may also serve as triggering devices: ‘The children playedquietly as ______ (well, long, nice, far) as mother was in the room’. Testtakers complete such items by recognizing the trigger and forming orchoosing an appropriate structure. Practising this trigger-form associationtechnique in the classroom and in classroom tests enhances learners’ test-taking skills, which are instrumental in completing the grammar part ofsome Israeli school-leaving examinations.

Eliciting language by means of triggers runs counter to the languageproduction process in communicative situations. Language is not

ELT Journal; doi:10.1093/elt/ccr044 1 of 9ªª The Author 2011. Published by Oxford University Press; all rights reserved.

ELT Journal Advance Access published June 23, 2011 at Indiana U

niversity Libraries T

echnical Services/Serials Acquisitions on N

ovember 24, 2013

http://eltj.oxfordjournals.org/D

ownloaded from

http://eltj.oxfordjournals.org/


normally elicited by triggers; rather, it is produced to convey meaning. Ina way, inclusion of L1–L2 translation items in classroom tests opens upan opportunity for test takers to express meaning in English. However, asis well known to translators, the meaning of a source text can usually beformulated in numerous ways in the translated text (House 2009: 29–30). This makes elicitation of particular forms in grammar testsproblematic. The present study investigates how elicitation of particularforms affects the linguistic authenticity of translation test items and theirtarget responses.

The notion of ‘linguistic authenticity’ refers to the level of naturalness of anutterance (Salem 2009). An ‘authentic’ utterance is a potentially usefulcommunicative token, whose possible situational or textual context isreadily recreatable. Its word strings and collocations are relatively frequentin naturally used language. Their reoccurrence potential can be verified bycorpus frequency data, as was done in Salem (ibid.), where I employeda corpus-based technique to evaluate the linguistic authenticity of grammartest items. In the present study, the linguistic authenticity of translation testitems is judged by analysing teachers’ translations of these items and bycomparison of teachers’ translation choices with corpus data.

This paper aims at assisting teachers in devising linguistically authentic,and at the same time effective, translation grammar test items. Thefollowing question is addressed: Which features affect the linguisticauthenticity and the elicitation potential of translation grammar testitems?

The data and howthey were obtained

This study is based on analysis of seven L1–L2 translation test items. Theywere selected out of 160 translation grammar test cues (a ‘cue’ is a Hebrewsentence test takers are expected to translate) comprising part of my larger,1,000-item collection of various formats of test items. The cues hadappeared in classroom grammar quizzes or grammar parts of larger testsdevised for elementary and high-school Hebrew-speaking students by theirHebrew-English-speaking teachers. They were collected randomly fromstudents, parents, and teachers. The particular grade or proficiency level ofthe target test-taker population was unknown to me, but I could infer thisinformation through my familiarity with the EFL curriculum in schoolswhere the tests were administered.

To acquaint myself with the linguistic features of these 160 translation testitems, I first translated all of them into English. While translating, I noticedthat many of them lent themselves to multiple translation options. Out ofthe multiple-translation-option items, I chose (rather arbitrarily, relying onmy own linguistic intuitions and test-writing experience) representatives ofvarious language structures taught or reviewed at high-school level; theseare Cues 3–7. For comparison, I added Cues 1 and 2, which I consideredtranslationally unproblematic. Cues 1–7 are presented below. They do notconstitute a statistically representative sample; they merely serve todemonstrate some linguistic features of translation grammar test items.The first line presents transliteration of the Hebrew cue. The second line ismy own translation of the cue and is one of numerous translationpossibilities.

2 of 9 Ilana Salem

at Indiana University L

ibraries Technical Services/Serials A

cquisitions on Novem

ber 24, 2013http://eltj.oxfordjournals.org/

Dow

nloaded from



1 ech tagiyu lachatunaHow will you get to the wedding?

2 ein lanu shi’urei bait hayomWe have no homework today.

3 hu kana dira acharei shehu machar et dirato hayeshanaHe bought an apartment after selling his old one.

4 hasavta sheli ka’et hitkashra le’achel li nesiya tova ki yesh li ti’ul macharMy grandmother (has) just called to wish me a safe trip because we havean outing tomorrow.

5 liknot besupermarket bimkom liknot bamakolet ozer lemishpachot lachsochkesefShopping in the supermarket instead of the grocery store helps familiessave money.

6 ha’ima hizhira et ruti lo lehafriya bashi’urMother warned Ruthi not to disrupt the class.

7 bizman sheshatafti et haritspa dibarti batelofon im chaverti hatovaWhile (I was) washing the floor, I was talking to my good friend on thephone.

The seven cues comprised Questionnaire One. It was completed by ninenative and two near-native speakers of English, ten of whom were EFL

teachers and one a translator. This convenience sample of English-Hebrewbilinguals will be referred to as ‘the teachers’. (Teachers who devised Cues 1–7 did not participate in the present research.) I was interested to see whichtranslation choices these participants, as a group, opted for. Would theychoose the target form? Would they experience any difficulties in theirtranslating? The participants were asked to translate the seven cues intoEnglish, without being given any more specific instructions. They wereinvited to comment and allowed to offer more than one option. As a result,over 11 translations were received for each sentence. I compared teachers’translations of each cue. (Numerals in brackets will signify the number oftimes each form occurred. WFW stands for word-for-word translation.) Forexample, Cue 1 ech tagiyu lachatuna—WFW ‘how you-will-get to-the-wedding’—was translated as ‘how will you get to the wedding’ (7), ‘how areyou going to get to the wedding’ (2), and ‘how are you getting to the wedding’(2). Then I created Questionnaire Two consisting of the most frequentEnglish translations of the seven cues, to be translated back into Hebrew.Translating back into source language, so-called ‘back translation’, is oftenused for quality check of translated documents (Grunwald and Goldfarb2006); I applied this method in order to see to what extent the back-translated sentences matched the original Hebrew cues.

The English–Hebrew back translation was carried out by 11 NSs of Hebrew1

with high proficiency in English: six EFL teachers, four Hebrew experts(writers or tertiary Hebrew teachers), and one translator. Again, this wasa convenience sample of Hebrew-English bilinguals. The back translationprovided some insights into textual quality of the cues, as will be shown in

L1–L2 sentence translation 3 of 9





Dow

nloaded from



the section ‘Textual shortcomings’, but it was the Hebrew–English ‘forward’translation that proved more informative.

Each cue’s multiple translation options were analysed and compared.Teachers’ linguistic choices were contrasted with frequency data of TheCorpus of Contemporary American English (Davies 2008), henceforth ‘thecorpus’. The corpus, containing more than 410-million words of text equallydivided among spoken, fiction, popular magazines, newspapers, andacademic texts, represents for my purposes naturally used Englishlanguage. I consider a structure recurring frequently (‘frequently’ and‘rarely’ are relative terms, as the following section shows) in the corpus asa useful language token. Structures rarely appearing or non-existent in thecorpus are of low, or no, usability potential, and they should, as far aspossible, be avoided in pedagogic materials.

Results Some test items allow for free variation of translations, such as theabove mentioned Cue 1, and also Cue 2, where ein lanu (WFW ‘[there]is-not to-us’) was translated as ‘we have no’, ‘we don’t have any’, ‘we don’thave’, ‘we haven’t got any’, ‘there is no’, and ‘no’. Although suchtranslation options may be situation and register bound, they wouldprobably pass as good choices in the EFL context under discussion, withoutpresenting any pedagogical difficulty. Translations of Cues 3–7(seeprevioussectionfor reference),on theotherhand, raiseseveraldoubtsaboutthese cues’ efficiency and linguistic authenticity, as discussed below.

Eliciting a markedtense form

Tenses (past, present, and future) and aspects (continuous and perfect) arenormally generated by trigger words and/or typical logical relationshipsbetween two actions or states. For instance, in Cue 3, test takers are expectedto produce past perfect ‘had sold’, triggered by the connector acharei she‘after’ and by the fact that the selling took place before the buying. Teachers’translations revealed, however, that ‘had sold’ is not necessarily the onlyoption. While seven translations did use past perfect, four used past simple,and one the gerund ‘selling’. In other words, in teachers’ translations, thetarget past perfect is the most frequent form, but past simple and gerund arealso possible. Thus, the item’s elicitation potential is weakened by allowingalternative structures in place of the target form.

Furthermore, corpus (Davies op.cit.) frequency search reveals that pastperfect is actually not the preferred form. Entering ‘had sold’/‘sold’/‘selling’ + up to six words + past simple into the search box enablescomparison of frequencies of the three forms. In doing so, we discoverthat past perfect appears only in 10 per cent of cases, followed by pastsimple (40 per cent), while the most frequently occurring form in thisposition is ‘after selling’ at 50 per cent. In this way, the corpus frequencydata confirm that in fact Cue 3 elicits a marked (not the most typical)structure and questions the utility of the pre-taught trigger-tensecorrespondence.

This finding holds also for the present perfect and past continuous in Cues 4and 7, respectively. I will refer to perfect and continuous forms as ‘aspectualforms’. Three ‘other forms’ (neither past simple, nor aspectual forms)produced by teachers, namely: ‘after selling’ in Cue 3, and ‘while I was on the

4 of 9 Ilana Salem





Dow

nloaded from



phone’/‘while I was busy talking’ in Cue 7, will be disregarded here for thesake of comparison between past simple and aspectual forms.2

I employed the following method to verify the level of typicality of pastsimple and aspectual forms in their structural environment: I calculated theproportional percentage of past simple versus aspectual forms appearing inCues 3, 4, and 7, first in teachers’ translations and then in the corpus data.After averaging the percentages for past simple versus aspectual forms, thefollowing picture emerged. In 39 per cent of translations teachers opted forpast simple, although the aspectual form was the target structure that theseitems were meant to elicit from test takers. In the corpus, the past simple iseven more popular: it appears in 74 per cent of cases (in structures similar tothe cues), as opposed to 26 per cent of aspectual forms. This shows again, ona more general basis, that the aspectual form is not the only candidate forCues 3, 4, and 7 and that the past simple is actually preferable in most cases.It seems, therefore, that aspectual forms tend to be over-tested (and over-taught), and the trigger-form association overused.

Translationdifficulties due tointerlanguagedifferences

The construction of translation test items faces additional, translation-related challenges due to interlanguage differences. Teachers’ translationsof lehafriya bashi’ur—WFW ‘disturb in-the-lesson’—in Cue 6 demonstratea translation difficulty stemming from lexico-grammatical incompatibilitybetween Hebrew and English. The Hebrew lehafriya has several Englishcontext-dependent equivalents: disturb, interrupt, interfere, bother, hinder,hamper, to name but a few. While some teachers’ translations opted for‘disrupt’, ‘bother’, and ‘misbehave’ (1 each), ‘disturb’ (11) was by far the mostpopular translation of lehafriya. Syntactically, the English ‘disturb’ is directlyfollowed by a noun phrase, except for some special uses such as the ‘Do notdisturb’ sign (Hornby2005). In Hebrew, on the other hand, the verb lehafriyamay be used intransitively, i.e. without any noun immediately following, asin lehafriya bashi’ur. In six teachers’ translations, ‘disturb’ followed theHebrew-like intransitive pattern ‘disturb in/during the lesson’, non-existentin English.

The Hebrew-like pattern in teachers’ translational output suggests thattheir NS linguistic intuitions are affected by their everyday use of Hebrew.One participant expressed her reluctance regarding the correct wording as:‘The correct answer in my opinion is without the ‘‘in’’ but I might findmyself saying ‘‘in the lesson’’ because I’ve gotten used to the Israelitranslation’. This language-aware teacher, a NS of English living in Israel,realized the effect of language interference on her own English. Herstatement and other teachers’ use of ‘disturb in/during the lesson’ reinforcefindings of previous error analysis research (Salem 2004), in which NS EFL

EFL teachers accepted intransitive use of purely transitive verbs as correct.

Moreover, none of the other teachers’ translation options—‘disturb herteacher/the class’, ‘disrupt the lesson’, ‘misbehave in class/in school’, ‘botherduring the lesson’—appear in the corpus. The closest equivalent I found inthe corpus for lehafriya bashi’ur was ‘disrupt the class’ (7), yet no teacheroffered this option. So it seems that due to lexico-grammatical differencesbetween English and Hebrew, Cue 6 is inherently difficult to translate.Professional translators might solve such lack of equivalence by sentence






Dow

nloaded from



rephrasing (Gerding-Salas 2000), but our target test takers are not normallytrained to face translation challenges of this kind. It could be assumed thatunless specifically pre-taught, translation of lehafriya bashi’ur exceeds thelevel of our learners’ translation capability and that Cue 6 would therefore bedeficient in assessing the target test takers’ grammatical accuracy.

Textualshortcomings:discourse featuresand lexicalredundancy of theHebrew cue

Other cues are efficient in eliciting the target form, but present (andconsequently elicit) textually odd language. In real-life communication onthe one hand, each utterance has a communicative purpose (Lambrecht1994: 14–15), which mostly becomes evident from the textual or situationalcontext (Hewings and Hewings 2005: 18). The function of test cues, on theother hand, is primarily to elicit a particular form rather than serve ascommunicative tokens. As such, they lack a genuine communicativepurpose and are devoid of context clues that would help the translatorformulate the message in the target language.

Yet, it is possible to devise discrete test items whose context andcommunicative purpose can be recreated with ease. In case of Cue 1 echtagiyu lachatuna (‘How will you get to the wedding?’), for example, we caninfer that A and B are talking about an approaching wedding. A knows thatB is planning to go and wants to know how B is getting there: this is thecommunicative purpose of A’s utterance.3 Not only is the contextrecreatable, and the communicative purpose readily derivable, but thereader can also distinguish known information shared by the parties fromwhat is unknown to A (Lambrecht op.cit.: 51). Cue 5, on the other hand, doesnot exhibit the above discourse features (Table 1).

Transliterated

Hebrew cue:

liknot besupermarket bimkom liknot bamakolet ozer lemishpachot lachsoch kesef

WFW: To-buy in-

supermarket

instead to-

buy

in-the-

grocery

helps to-families to-save money

table 1Cue 5

It is hard to imagine any co-text this complex sentence may belong to. Couldit appear in a text dealing with advantages of supermarkets over groceries,or one providing practical advice to families, or one suggesting ways of savingmoney? And what new information is conveyed? Some teachers tried torectify its lack of focus by complete rewrites such as: ‘In order to savemoney, families do the (weekly) shopping at the supermarket instead of thegrocery’, or ‘Many families find shopping in the supermarket much moreeconomical than the local grocery’. The diffuse information structurerenders Cue 5 textually odd. (Similarly, Cues 4 and 7 are also informationallydense.)

Although professional translators would need contextual information inorder to express the meaning of such sentences adequately in the targetlanguage, experience shows that learners can arrive at the target form byapplying pre-taught grammar rules. (‘Elicitator’ will refer to the [part of ]source text whose translation constitutes the target form.) Learnersrecognize the word liknot ‘to buy’, appearing twice in Cue 5, as an elicitator of

6 of 9 Ilana Salem





Dow

nloaded from



a gerund form, following the textbook rule: ‘We can use gerunds as a subjectof the sentence, or after a preposition . . . ’ (Zelenko 1994: 228).4 The doubleoccurrence of liknot in this sentence, however, amplifies its oddity; inteachers’ back translations, the sentence-initial infinitive liknot becamea verbal noun such as kniya, arichat kniyot in all but one cases,a phenomenon implying that the infinitive form liknot sounds peculiar in itssentence-initial position. The second liknot is meant to elicit a gerund aftera preposition. This elicitator was omitted in 9 out of 14 teachers’translations, and its absence in back translations points to the possibilitythat it was redundant in the original Hebrew cue. Thus, on the one hand,Cue 5 successfully elicits the target form; on the other hand, it is of deficienttextual quality due to lexical redundancy, morphosyntactic peculiarities, anddiffuse information structure.

Summary of results The present study reveals several sources of imbalance between theelicitation potential of translation test cues and their linguistic authenticity.

1 Triggering devices tend to be efficient in eliciting the target forms, yetmany structures elicited by means of triggering devices turn out to behighly marked in their contexts.

2 Some word combinations are poor elicitators because they aretranslationally over-challenging.

3 Other cues are poorly worded in Hebrew, supposedly due to teachers’efforts to elicit more than one target form in a single sentence.

Additional insights gained by this research are not reported here due tospace limitations.

Pedagogicimplications

First of all, let us remind ourselves of the obvious: L1–L2 sentencetranslation is practicable in monolingual classes where EFL teachers sharetheir students’ native language. The inclusion of translation tasks in testsshould be preceded by classroom practice of L1–L2 translation. In my 12thgrade grammar-review classes, brief translation tasks serve as an awareness-raising eye opener to L1–L2 differences and are generally appreciated bylearners. Beginners or weak learners respond well to translation test itemsrestricted to specifically pre-taught forms. Translation practice goes hand inhand with enhancement of dictionary skills; students are trained tointerpret the syntactic, lexico-grammatical, and semantic informationbilingual and monolingual dictionaries provide.

Teachers themselves would benefit from practising translation; thiswould familiarize them with the complexity of the translation process(Gerding-Salas op.cit.). Many or most EFL mainstream education teachershave been living in non-English-speaking countries for extended periods oftime and their NS intuitions are affected by the local language (Salem2004). They should, therefore, become ‘corpus-literate’ in order to updatetheir NS intuitions and verify their own judgment of linguistic authenticityand acceptability.

The present findings concur with other corpus researchers such as Mindt(1996), proposing that EFL curriculum devisers and textbook writersreorder their priorities in presentation of linguistic structures in accordancewith their relative frequency in relevant corpora. Furthermore, external






Dow

nloaded from



examination boards might wish to consider restricting the inclusion oftrigger-form association items, which often produce linguisticallyunauthentic tokens and which encourage practice of communicativelyfutile elicitation techniques in the classroom. My own test-writingexperience has shown, however, that presenting authentic language in teststends to result in items allowing for more than one acceptable answer. Thisis impractical in grammar tests because by accepting more than one option,the item loses its evaluative power. So it is the balance between linguisticauthenticity and high elicitation potential of test items that is called for.

Conclusion This study encourages teachers to pay attention to the linguistic quality ofthe language presented to and elicited from EFL learners. The findingsinvite further research addressing questions such as: How does the‘linguistic authenticity’ discussed above relate to other researchers’ view oftextual and corpus data authenticity (Mishan 2004)? To what extent doescontrasting L1 with L2 and learning grammar rules based on the trigger-form association assist second language acquisition (Scheffler and Cincia1a2011)? How could the ‘elicitation potential’ of classroom grammar test itemsbe defined in terms of validity and reliability (Hughes 2002)? It is hoped thatlearners will benefit from this enquiry by increased exposure tolinguistically authentic and usable language in teacher-produced materials.

Final revised version received April 2011

Notes1 It is often claimed (for example Translation

Advisor 2010) that translation should beperformed by NSs of the target language.Therefore, in this study, Hebrew cues weretranslated mostly by NSs of English, and Englishsentences by NSs of Hebrew.

2 In fact the ‘other forms’ might be even moreunmarked than past simple and aspectual forms.

3 Wider context could provide more specificinformation about the communicative function ofCue 1, depending on A’s motive for asking and thelevel of formality of the interlocutors’ relationship.This would help in deciding which of the threeforms suggested by the teachers is most relevant.

4 In Zelenko’s book, grammar rules are simplifiedfor the sake of weaker learners.

ReferencesDavies, M. 2008. The Corpus of ContemporaryAmerican English (COCA). Available at http://www.americancorpus.org (last accessed on 26 May2011).Gerding-Salas, C. 2000. ‘Teachingtranslation—problems and solutions’. TranslationJournal 4/3: Available at http://www.accurapid.com/journal/13educ.htm (last accessed on 26 May 2011).

Grunwald, D. and N. M. Goldfarb. 2006. ‘Backtranslation for quality control of informed consentforms’. Journal of Clinical Research Best Practices 2/2:1–6.Hewings, A. and M. Hewings. 2005. Grammar andContext. London: Routledge.Hornby, A. S. 2005. Oxford Advanced Learner’sDictionary (Seventh edition). Oxford: OxfordUniversity Press.House, J. 2009. Translation. Oxford: OxfordUniversity Press.Hughes, A. 2002. Testing for Language Teachers(Second edition). Cambridge: CambridgeUniversity Press.Lambrecht, K. 1994. Information Structure andSentence Form. Cambridge: CambridgeUniversity Press.Mindt, D. 1996. ‘English corpus linguistics and theforeign-language teaching syllabus’ in G. Sampsonand D. McCarthy (eds.). Corpus Linguistics: Readingsin a Widening Discipline. London: Continuum.Mishan, F. 2004. ‘Authenticating corpora forlanguage learning: a problem and its resolution’.ELT Journal 58/3: 219–27.Salem, I. 2004. ‘Teacher differences in perception ofstudent error’. English Language Teacher Educationand Development 8: 48–65.

8 of 9 Ilana Salem





Dow

nloaded from

http://www.americancorpus.org

http://www.americancorpus.org

http://www.accurapid.com/journal/13educ.htm

http://www.accurapid.com/journal/13educ.htm



Salem, I. 2009. ‘Corpus-based evaluation of testitems’ linguistic authenticity’. Humanising LanguageTeaching 11/6: Available at http://www.hltmag.co.uk/dec09/idea.htm (last accessed on 26 May 2011).Scheffler, P. and M. Cincia1a. 2011. ‘Explicit grammarrules and L2 acquisition’. ELT Journal 65/1: 13–23.Translation Advisor. 2010. ‘How to choose a freelanceEnglish-French translator or French-Englishtranslator’. Available athttp://www.translationadvisor.com/english-french-translator.html (last accessed on 26 May 2011).Zelenko, E. 1994. Develop Your Grammar. Ra’anana:Eric Cohen books.

The authorIlana Salem, a high-school EFL teacher in Israel,is involved also in teacher education andtranslation. She holds an MA in AppliedLinguistics and TESOL from the University ofLeicester. Her research interests include quality ofEFL tests and materials, bilinguals’ linguisticchoices, and using corpora for both evaluation ofteachers’ linguistic intuitions and also forassessment of language markedness orappropriateness.Email: [email protected]






Dow

nloaded from

http://www.hltmag.co.uk/dec09/idea.htm

http://www.hltmag.co.uk/dec09/idea.htm

http://www.translationadvisor.com/english-french-translator.html

http://www.translationadvisor.com/english-french-translator.html



Documents

L1-L2 sentence translation in classroom grammar tests