View
217
Download
0
Category
Tags:
Preview:
Citation preview
Holistic vs Analytical Assessment in Legal Translation
Carmen Valero-Garcés & Francisco Vigier – University of AlcaláMary Phelan – Dublin City University
• Assessment in Translation Studies and Professional Practice
• Introduction to HA
– Research Study on HA in Legal Translation
• Introduction to AA
– Research Study on AA in Legal Translation
• Conclusions2
Assessment in Translation Studies and Professional Practice
• Underresearched area
• Common problems in TQA (Williams 2009)
– The evaluator
– Level of target language rigour
– Seriousness of errors
– Sampling vs full-text assessment
– Quantification of quality
– TQA purpose 3
4
What is Holistic Assessment?
• The evaluator gives a TT a rating (0-10) or evaluative letter (e.g. A = excellent, B = very good) based on an overall impression
• Frequently used in both academia and industry
• Advantages less time-consuming and assessment of translations at the discourse/text level not at the sentence/word level (Garant 2009)
• Some attempts of systematization (Waddington, 2001)
• Disadvantages subjective, hence arbitrary, intuitive, unscientific, unsystematic and unreliable; does not provide a clear justification of the result (Waddington 2001)
5
Research Study on HA in Legal Translation
• Analyse strengths and weaknesses of holistic methods for the assessment of legal translation ( interrater reliability)
• One of the WS1 essential documents translated into SP by a student on MA in Translation
• That translation assessed numerically (0-10) by ten evaluators
• Evaluators surveyed on their assessment method
8
Results (2)• Survey
– Most evaluators ranked pragmatic errors as those with highest relevance and linguistic errors as those with lowest relevance
– Very different opinions expressed by respondents as to the translation’s strengths and weaknesses (i.e. “The message is appropriately conveyed. It fulfills its communicative function” vs. “Errors regarding sense, coherence, punctuation... A poor quality translation” assessment is based on personal criteria, thus subjective and variable
11
ATA Grid
ATA CERTIFICATION PROGRAM Exam Number:
FRAMEWORK FOR STANDARDIZED ERROR MARKING Exam Passage:
Version 2009 Check here if for Review
1 2 4 8 16 Code Reason
Errors that concern the form of the exam
Treat missing material within the passage as an omission UNF Unfinished (if a passage is substantially unfinished, do not grade the
exam)
ILL Illegibility
IND Indecision, gave more than one option
Translation/strategic/transfer errors: Negative impact on understanding/use of target text
MT Mistranslation (use a subcategory if possible)
MU - Misunderstanding of source text (if identifiable)
A - Addition
O - Omission
T - Terminology, word choice
R - Register
F - Faithfulness
L - Literalness
FA - Faux ami (false friend)
COH - Cohesion
AMB - Ambiguity
ST - Style (inappropriate for specified type of text)
OTH - Other (describe)
Mechanical errors: Negative impact on overall quality of target text. Points may vary by language. Maximum 4 points
G Grammar
SYN - Syntax (phrase/clause/sentence structure)
P Punctuation
SP/CH Spelling/Character (usually 1 point, maximum 2, if more than 2 points,
another category must apply)
D Diacritical marks/ Accents
C Capitalization
WF/PS Word form/ Part of speech
U Usage
OTH Other (describe)
0 0 x 2
=0
0 x 4 = 0 0 x 8 = 0 0 Column totals
A grader may stop marking errors when the
score reaches 46 error points
A grader may award a quality point for each of
up to three specific instances of exceptional
translation
Quality points are subtracted from the error point total to yield a final
score. A passage with a score of 18 or more points receives a grade of
Fail.
Total error points (add column totals): 0 Quality points (maximum 3) 0 Final passage score (subtract quality points from error points) 0
13
• UAH text – holistic - 532 words in ST• DCU text – analytical – 256 words in ST
• 5 assessors – three in Europe plus two ATA assessors
14
Evaluators’ VerdictsAssessor Score Verdict
Spanish evaluator 2 9 Pass
Spanish evaluator 3 16 Pass
Spanish evaluator 1 23 Would accept it with reservations
ATA evaluator 1 45+ Fail
ATA evaluator 2 43 Fail
15
Conclusions
• HA: subjective method with a low degree of inter-rater reliability
• Cost and time efficiency HA as supplementary method for LT assessment?
• AA: even though the system appears self-explanatory, there is a lot of variation in the overall result.
• AA: The ATA evaluators have years of experience of using this method.
16
References
Garant, M. (2009). A case for holistic assessment. AFinLA-e Soveltavan kielitieteen tutkimuksia 2009, 1, 5-17.
Waddington, C. (2001b). Should translations be assessed holistically or through error analysis? Hermes, Journal of Linguistics, 26, 15-38.
Williams, M. (2009). Translation Quality Assurance. Mutatis Mutandis, Vol 2, No 1., 3-23
Recommended