Upload
jo-sharon-udoh
View
216
Download
0
Embed Size (px)
Citation preview
8/12/2019 Quality Me as 2002
1/31
The Measurement of Quality
in TranslationUsing Statistical Methods
The Measurement of Qualityhe Measurement of Quality
in Translationn TranslationUsing Statistical Methodssing Statistical Methods
43 rd ATA Conference Atlanta 200243 rd ATA Conference At lanta 2002
Riccardo Schiaffino President Aliquantum
Franco Zearo Worldwide Director, Language Services Lionbridge Technologies
2002 by J.D. Edwards and Lionbridge Technologies, Inc.
8/12/2019 Quality Me as 2002
2/31
Overviewverviewverview
Measuring QualityTranslation Quality AssessmentQuality Assurance Forms
Error CategoriesSampling
Translation Quality IndexQuestions and Answers
8/12/2019 Quality Me as 2002
3/31
You cannot measure qualityYou cannot measure qualityou cannot measure quality
This is not true:There are certain components of translationquality that will always remain subjective .
However ,There are other elements that can be objectivelymeasured.
By concentrating of these, we believe we canachieve a satisfactory measurement of translationquality.
8/12/2019 Quality Me as 2002
4/31
Why is Quality Measurement Important?hy is Quality Measurement Important?hy is Quality Measurement Important?
It is difficult to improve something if youcannot measure it.
Such measurement should berepeatable and objective .
Different evaluators should arrive atsimilar assessment for the same piece oftranslation.
8/12/2019 Quality Me as 2002
5/31
Who Benefits from Reliable TranslationQuality Measurement?
Who Benefits from Reliable Translationho Benefits from Reliable TranslationQuality Measurement?uality Measurement?
Professional Translators
Translation Companies and
In-House Translation Departments
Translation Customers and Users
8/12/2019 Quality Me as 2002
6/31
Our Definition of Qualityur Definition of Qualityur Definition of Quality
Functional approach to quality
Quality is defined as
consistently meetingthe needs and expectations
of the customer or user
8/12/2019 Quality Me as 2002
7/31
Product Process Assessmentroduct Process Assessmentroduct Process Assessment
Translation quality assessment must apply to
both :The translated text(the product)
The translation process(the process)
Aim:
Inputs Outcomes
Boundary Boundary
Feedback
Conversion Process
Process Process Process Process Process
8/12/2019 Quality Me as 2002
8/31
QC vs QACC vss QAA
Quality Control (QC)Quality verification over the whole text.Example: Editing.
Quality Assurance (QA)Sampling techniques, control of quality over a
(statistically significant) sample of the wholetext. Appropriate use: Quality measurement.
8/12/2019 Quality Me as 2002
9/31
Ideas from other disciplinesdeas from other disciplinesdeas from other disciplines
Software project management techniquesW. Edwards Deming and other qualityassurance experts
8/12/2019 Quality Me as 2002
10/31
Real Life Exampleseal Life Exampleseal Life Examples
Development of translation qualitymeasurement at J.D. EdwardsUse of sampling techniques for quality
assurance at Lionbridge
8/12/2019 Quality Me as 2002
11/31
Development of Translation QualityMeasurement at J D Edwards
Development of Translation Qualityevelopment of Translation QualityMeasurement at J D Edwardseasurement at J D Edwards
From the concept of checklists to aspreadsheet of measurementsChecklists are appropriate to control whether acertain action has been performed or not (e.g.,
spell check done or not as opposed to ameasurement of how many spelling mistakeswere found)
Based on LISA model (www.lisa.org)Flexibility (different settings for differentlanguages)
8/12/2019 Quality Me as 2002
12/31
Use of Quality Assurance Formsse of Quality Assurance Formsse of Quality Assurance Forms
The LISA Quality Assurance Form
Language: Reviewer: Date: Result: Pass Comments:
Client NameProject Name
Project Number Project Manager
Critical max. error points + 1Number of words 0 Major 5 pointsMax error points allowed 0 Minor 1 point
Error Category Minor Major Critical total max. allowed0 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0
Total 0 0More elaborate descriptions of the error criteria can be found in the LISA QA model version 1.0 Reference Manual.
Quality Assurance Form
Consistency
Mistranslation AccuracyTerminologyLanguageStyleCountry
8/12/2019 Quality Me as 2002
13/31
The J.D. Edwards QA FormLanguage Customization
The J.D. Edwardshe J.D. Edwards QA FormA FormLanguage Customizationanguage Customization
Weighting the major categories
Accuracy 50%Style 15%Grammar 30%Formatting 5%
100%
1 - Give appropriate weight to the four following categories (total must add up to 10
Language Setup
Total
Categories
8/12/2019 Quality Me as 2002
14/31
The J.D. Edwards QA FormLanguage Customization
The J.D. Edwardshe J.D. Edwards QA FormA FormLanguage Customizationanguage Customization
Weighting the items within the major
categoriesIncorrect meaning 40%Non-standard terminolgy 20%Inconsistent terminolgy 20%Untranslated SL 20%
100%
Wrong register 40%Inappropriate anglicisms 30%Inappropriate use of passive/active voice 30%
100%
Spelling errors 20%Typos 15%Grammar errors 35%Syntax errors 25%Punctuation errors 5%
100%
Layout errors 50%Font errors 40%Double spaces 10%
100%
2 - Within the Accu ra cy category, give appropriate weight to the four following items (total must add up to 100%)
3 - Within the Style category, give appropriate weight to the three following items (total must add up to 100%)
4 - Within the Grammar category, give appropriate weight to the five following items (total must add up to 100%)
Total
Total
Accuracy
Style
Total
Formatting
Total
5 - Within the Formatting category, give appropriate weight to the five following items (total must add up to 100%)
Grammar
8/12/2019 Quality Me as 2002
15/31
The J.D. Edwards QA FormLanguage Customization
The J.D. Edwardshe J.D. Edwards QA FormA FormLanguage Customizationanguage Customization
Weighting the items within the major
categories (detail)
Incorrect meaning 40%
Non-standard terminolgy 20%Inconsistent terminolgy 20%Untranslated SL 20%
100%Total
Accuracy
8/12/2019 Quality Me as 2002
16/31
How We Worked to Develop OurSpreadsheet
How We Worked to Develop Ourow We Worked to Develop OurSpreadsheetpreadsheet
Determine type of errors, issues or problems
Determine relative importance of issues (may bedifferent for different languages; e.g., spellingerrors in English, French or Italian)
Determine which are the responsibility oftranslation
Determine tolerance limits for various levels ofquality
8/12/2019 Quality Me as 2002
17/31
Inspection Pointsnspection Pointsnspection Points
Key Principle: Reject defective material at its lowestKey Principle: Reject defective material at its lowestvaluevalue
QCQC
Proof Proof
Translation/EditTranslation/Edit
SourceSourceContentContentDevelopmentDevelopment(GIGO)(GIGO)
$ V a
l u e o
f S e r v
i c e
$ V a
l u e o
f S e r v
i c e
Stages of ProductionStages of Production
8/12/2019 Quality Me as 2002
18/31
Purposes of sampling according to LISAurposes of sampling according to LISAurposes of sampling according to LISA
To determine whether something has been
done or not.To accept / reject the batch of product at
hand.To determine if the process that producedthe product at hand was within acceptablelimits.
8/12/2019 Quality Me as 2002
19/31
Guidelines for Samplinguidelines for Samplinguidelines for Sampling
Select a sampleSelection criteria (e.g. random, systematic)Size considerationsCost considerations
Evaluate the sampleRepeatable, reproducible, objective
Investigate the outcome / causesCorrect / Improve
8/12/2019 Quality Me as 2002
20/31
Summary: Error Categorizationummary: Error Categorizationummary: Error Categorization
Select a (small) set of categories
CTQ: Critical-To-Quality categoriesProvide clear definitions
Set tolerance limitsMin / Max # of errors per X words
Assign a weight
Critical, Major, Minor
8/12/2019 Quality Me as 2002
21/31
Whats left to do ?hathat s left to do left to do ?
h f
8/12/2019 Quality Me as 2002
22/31
The concept of aTranslation Quality Index
The concept of ahe concept of aTranslation Quality Indexranslation Quality Index
Translation Quality Index (TQI)
A numberobtained by the rigorous
application of a QA formthat isindicative of the quality of a given
translation
8/12/2019 Quality Me as 2002
23/31
Index / Indicesndex / Indicesndex / Indices
Depending on ones purpose, there may
be more than a single TQI.E.g., a TQI may be developed for
external purposes (to standardize thework obtained from outsourcing).
Another TQI may be primarily for internalpurposes (to measure the quality of agiven special process).
l fA E l fE l f
8/12/2019 Quality Me as 2002
24/31
An Example of aTranslation Quality Index (1)
An Example of an Example of aTranslation Quality Indexranslation Quality Index (1)1)
LISA QA Model ver. 1.0 (1995)
3,000 words (12 pages @ 250 words)30 error points
30 error pts / 3,000 words = 1.0%
10,000 error pts out of 1 million wordsDPMO = 99.0% = TQI
A E l fA E l fE l f
8/12/2019 Quality Me as 2002
25/31
An Example of aTranslation Quality Index (2)
An Example of an Example of aTranslation Quality Indexranslation Quality Index (2)2)
Microsoft Quality Standards for Print
ver. 1.0 (1998)10,000 words (40 pages @ 250 words)
0 major errors15 minor errors
15 errors / 10,000 words = 0.15%1,500 errors out of 1 million wordsDPMO = 99.85% = TQI
A E l fA E l fE l f
8/12/2019 Quality Me as 2002
26/31
An Example of aTranslation Quality Index (3)
An Example of an Example of aTranslation Quality Indexranslation Quality Index (3)3)
2,000 words (8 pages @ 250 words)1 critical error 2 major errors3 minor errors
6 errors / 2,000 words = 0.3%
3,000 errors out of 1 million wordsDPMO = 99.7% = TQI
8/12/2019 Quality Me as 2002
27/31
Control Chartsontrol Chartsontrol Charts
Concept of statistical control
98.50
98.75
99.00
99.25
TQI Control Chart
C2
C 3
C4
C 5
C 6
C7
C 8
C 9
C1 0
C1 1
C1 2
C1 3
UCL LCL Mean Data Point
8/12/2019 Quality Me as 2002
28/31
Process Flow Diagramrocess Flow Diagramrocess Flow Diagram
Aim:
Inputs Outcomes
Boundary Boundary
Feedback
Conversion Process
Process Process Process Process Process
How to Apply Statistical Methods for QualityHow to Apply Statistical Methods for Qualityow to Apply Statistical Methods for Quality
8/12/2019 Quality Me as 2002
29/31
How to Apply Statistical Methods for QualityImprovement
How to Apply Statistical Methods for Qualityow to Apply Statistical Methods for QualityImprovementmprovement
1. Define error categories and tolerances
2. Create a QA form
3. Obtain a TQI index4. Use the TQI index to improve the
translation process
8/12/2019 Quality Me as 2002
30/31
ContactsContacts
[email protected] Aliquantum, Inc.
[email protected] Lionbridge Technologies, Inc.
For more information about translation issues, visit
TranslationQuality.com
Biographical Notes on the Authorsiographical Notes on the Authorsiographical Notes on the Authors
8/12/2019 Quality Me as 2002
31/31
Biographical Notes on the Authorsiographical Notes on the Authorsiographical Notes on the Authors
Riccardo SchiaffinoRiccardo Schiaffino worked as translator, translation manager and special software translationproject lead for a major software company, and now leads a small company he established with a
few experienced colleagues. As a translation manager, Riccardo worked on the improvement oftranslation quality and on translation quality metrics and tools. He holds an MA degree inTranslation, and has been working in translations for over 20 years, first in Italy and then in theU.S. Riccardo is ATA accredited.
Franco Pietro ZearoFranco Pietro Zearo is a project manager with Lionbridge Technologies in Boulder, Colorado. Heholds a degree in translation from the Advanced School of Modern Languages for Translators andInterpreters at the University of Trieste, Italy, and earned an MBA from the University of Phoenix.Before joining Lionbridge in 1996, he worked as a freelance technical translator in Italian, English,and Russian. At Lionbridge, he has held positions in translation, localization analysis, presales,and cultural and globalization consulting. He has been responsible for translation quality onnumerous projects for many Fortune 500 clients. In his previous role as senior technical translator,he helped define best practices for the translation department.