8/8/2019 HaPham_MESThesisSummary
2/2
PHAM, Thi Thu Ha,
A2 OR B1? COMPARING DIFFERENT METHODS OF STANDARD SETTING BASED
ON
ITEM CLASSIFICATION FOR THE COMMON EUROPEAN FRAMEWORK OF
REFERENCE
FOR LANGUAGES.
Masters thesis presented to obtain the degree of Master of
Educational Studies, September 2009
Promoter: Prof. Dr. Rianne Janssen
The Common European Framework of Reference for languages (CEFR)
has been initiated and
popularized by the Council of Europe as a reference scheme and
guideline for the language teaching,
learning and assessment. Within this framework, language
learners are categorized into six different
levels of language competence, namely A1, A2, B1, B2, C1 and C2.
The question on how we can classify
students into different language proficiency levels has long
been posed and discussed in language testing
research. In this present master thesis, four standard-setting
methods that are based on item classification
are compared for a test on reading comprehension in French.
In the first chapter, a brief summary of recent developments in
language testing research and of the use of
the CEFR in assessing language proficiency is presented. The
next chapter introduces different
conceptions of language mastery, different measurement tools for
language mastery assessment, and
different methods to set standards for different mastery levels.
In Chapter 3, an empirical analysis usingdata of the Flemish
national assessment on students reading comprehension in French is
carried out to
investigate the feasibility of four different standard-setting
methods to allocate students in accordance
with the three different language levels of the CEFR, namely the
Item-Descriptor Matching method, the
Basket method, Schulz method and the hierarchical Item Response
Theory (IRT) method. For the present
test, the conditions for deriving a cutoff score according to
the Item-Descriptor Matching method were
not fulfilled. The Basket method led to really high cutoff
scores, consequently no student reached level
B1. The two IRT-based methods led to comparable results but the
number of students reaching a
particular CEFR level was always lower for the Schultz methods.
Hence, the present investigationsuggests that the IRT-based methods
for standard setting are more advisable.
