Evaluation of Iranian TEFL M.A. Program in Terms of ...eltvoices.in/Volume6/Issue1/EVI_61_4.pdf · Program Evaluation and the Approaches ... Management- Oriented Approach (CIPP):

ELT Voices- International Journal for Teachers of English Volume (6), Issue (1), 31-51 (2016)

ISSN Number: 2230-9136

(http://www.eltvoices.in)

Evaluation of Iranian TEFL M.A. Program in Terms of Compatibility

with Common Standards: Teachers' Perspectives

1Neda Taheri,

2Gholamreza Abbasian

1 Department of English Language, Shahr-e-Qods Branch, Islamic Azad University, Tehran, Iran.

2Imam Ali University & South Tehran Branch, Islamic Azad University, Tehran, Iran

Article reference:

Taheri, N., & Abbasian, G. (2016). Evaluation of Iranian TEFL M.A. program in terms of compatibility with common

standards: Teachers’ perspective. ELT Voices, 6 (1), 31-51.

Abstract: The main purpose of this study was to evaluate the alignment of Iranian TEFL Master of Arts Program with

standards of a conventional educational program from the teachers' perspectives, and the extent to which their expectations

are met. To this purpose, two separate and validated questionnaires one addressing the Program compatibility with common

standards along with a number of open-ended questions and the other measuring the Program and its Course features were

used. The data were collected from four universities in Tehran, in which the program is still implemented. Quantitative data

analyses primarily based on frequency and chi-square analyses revealed that the educational Program is mainly aligned

with the determined standards and it meets the teachers' needs in a sense that the participants positively evaluated both

macro dimensions. However, based on the qualitative data analyses, a number of insightful points were explored sug-

gesting 1) inclusion of Teaching English for Children and Adults Courses, 2) revising Advanced Writing Course, 3) remov-

al of Contrastive Linguistics/ Error Analysis and Sociolinguistics courses, 4) inclusion of internet technology, 5)

modifying the manner courses are practically rendered, 6) evaluating the learners' through the teachers' comments, 7) hiring

good quality teachers, and 8) focussing on research-based education.

Index Terms: program evaluation, TEFL program, program standards

INTRODUCTION

Throughout the time English has been established as the international language of research, and it has also been im-

plemented as the language of the instruction in a majority of universities and institutes all over the world. However, simi-

lar to any educational program, ESL/EFL programs developed and offered at various academic levels follow or at least are

supposed to follow certain well-established standards of curriculum development and syllabus design. According to Rich-

ards (2001, p. 286), "the overall and interlinked system of elements (i.e. needs, goals, teachers, syllabuses, materials, and

teaching) is known as language curriculum". Similarly Kucuk (2010, as cited in Abbasian & Afsharimani, 2013, p.41)

states that "a language teaching program is a series of foreign language courses teaching the language through some kind of

methodology so as to fulfill aim/ aims such as communication or passing a proficiency exam."

Any educational systems in general and language educational programs in particular need to be evaluated to see how

well its objectives are achieved. In this regard, for example, "Council of Chief State School Officers’ (CCSSO) Interstate

32 Taheri & Abbasian (2016)

Teacher Assessment and Support Consortium (2010) articulates teaching standards under four core categories:

The Learner and Learning: learner development, learning differences, learning environments,

Content: content knowledge, innovative applications of content,

Instructional Practice: assessment, planning for instruction, instructional strategies, and

Professional Responsibility: reflection and continuous growth, collaboration."

Simultaneously, Richards (2001) states that there are four factors which play an important role in the success of lan-

guage teaching programs: instructional factors, teacher factors, teaching factors, and learning factors.

Program Evaluation and the Approaches

Defined as analyzing different aspects of a program or a course of study and making decisions on the basis of evidence

of program functioning and outcomes, evaluation, according to International Center for Educational Policies (2014), fulfills

a number of functions:

1. Measure the program's outcomes and impact,

2. Inform future program planning and design,

3. Provide important internal lessons for those conducting programs,

4. Ensure transparency and accountability, and

5. Provide broader lessons about good practice.

Educational evaluation is characterized by a rich theoretical literature such that different approaches have been intro-

duced which guide the researchers’ deeds, but the choice of the most appropriate approach/s is controlled by several factors

including the availability of resources, the researcher's tendency, and purpose of study. A brief presentation of evaluation

approaches emerged in 21 century is put forwarded as follows:

Objectives- Oriented Approach: focuses on evaluating the goals and objectives of a program and determines

whether the objectives have been achieved or not (Hogan, 2007).

Management- Oriented Approach (CIPP): tries to evaluate the three most important factors of a program including

Content, Input, Process, and Product (CIPP).

Consumer- Oriented Approach: according to Scriven (1967, as cited in Stake, 1973), tries to give information and

input to consumers (purchasing agents, taxpayers, and parents) on how good an educational or any other human

service products are.

Expertise- Oriented Approach: in this approach, according to Abbasian and Afsharimani (2013) the program is

evaluated by expert evaluators. They judged the program based on their perceptions and give feedback.

Adversary- Oriented Approach: tries to appreciate the perceptions of the critics as "Stake 1973 believes that …

truth emerges when opposing forces submit their evidence to cross- examination directly before the eyes of judges

and juries." (p.8)

Participant- Oriented Approach: welcomes to involve the participants of a program in its evaluation. In this respect

Luluquised and Zukoski (2002, as cited in Abbasian & Afsharimani, 2013) held that the participants who mostly

involved in evaluation in this approach are; stakeholders, policy makers, and curriculum developers, who play ac-

tive roles in all phases of evaluation and the implementation.

CIRO Evaluation Approach: it evaluates four distinct aspects of a program including context, input, reaction, and

output which all are abbreviated as CIRO (Hogan, 2007).

Kirkpatrick's Evaluation Approach: refers to a four-level program evaluation including Reaction which deals with

learners' expectations; Learning which considers the design of the program; Transfer which concentrates on trans-

ferring the acquired knowledge to the job positions; and Results which focuses on the evaluation of learners and

program (Hogan, 2007).

Phillips' Evaluation Approach: considered as the advanced version of Kirkpatrick's evaluation approach, it ac-

commodates a fifth level dealing with Finance of a program (Hogan, 2007).

ELT Voices-Volume (6), Issue (1), (2016) 33

Language Program Evaluation

Balint (2009, as cited in Abbasian & Afsharimani, 2013) asserts that language program evaluation has recently fo-

cused towards the formative evaluations with emphasis on process-oriented methods, featured by:

1. A shift from an exclusive focus on measurement of outcomes,

2. Increased attention to classroom processes,

3. Evaluation as the domain of professional practice,

4. Development of teachers' skills, and

5. Inclusion of baseline and formative evaluations.

Richards (2001) specifies two major purposes for language program evaluation including program accountability, and

program development. Weir and Roberts (1994, as cited in Richards, 2001) distinguish between these two purposes: Ac-

countability-oriented approach focuses on the evaluation of those who are involved in the program and Develop-

ment-oriented approach which concentrates on the program and takes it as a whole.

Global TEFL MA Program

TEFL master program offers a unique opportunity to study for a qualification which links teaching English to wider issues

of education. It is aimed at individuals with knowledge and understanding of current debates relevant to the teaching and

learning of English as a Foreign Language (University of Birmingham, 2014). In many universities various graduate profiles,

depending on socio-political and educational policy requirements, have been established and are put into force with regard to

their TEFL/TESL programs. For example, "the University of Jambi 2014 has set up graduate profiles for TEFL MA program."

The following is the summary of these profiles:

1. Speak in English fluently and accurately,

2. Write qualified scientific writings,

3. Calculate statistical data (Linguistics data) at least on the descriptive level,

4. Master the concepts of acquiring English language based on each learner's features,

5. Expertly teach English in terms of skills, components, and linguistics,

6. Apply ICT for English teaching,

7. Professionally teach English for young learners,

8. Dominate sociolinguistic fields such as the relationship between a language and its society,

9. Master the theory and practice for teaching English in a multicultural classroom, and

10. Overcome problems concern language functioned (pragmatics) with a specific context.

Iranian TEFL MA Program

Iranian TEFL MA Program is a two- year postgraduate course implemented in four semesters. The program requires

thirty two credits of which eighteen credits are required courses; eight credits are optional clinical experience; and six cred-

its are met in the light of master’s thesis (Approved TEFL MA Program Syllabus in Iran, 2007). Nevertheless, the success

of this educational program in achieving the predetermined objectives as well as its compatibility with academic standards

of successful educational programs has raised controversies among educators. Following the tenets of Zukoski and Lulu-

quised's (2004, as cited in Abbasian & Afsharimani, 2013) participant-oriented approach in order to investigate teach-

ers-learners' perceptions as two important target/participants in any course of study can result in big changes and improve-

ments. Although many research studies have been done on the subject of curriculum evaluation, it seems that Iranian

TEFL MA Program hasn't been evaluated empirically. Such a gap based the rationale behind this study to approach the MA

TEFL program evaluation from the perspectives of at least one group of consumers; those of the teachers to see the extent

of its compatibility with standards of a conventional educational program.

Problem and Purpose

Given the introduction to the issue of MA program evaluation, this study was more solidly designed to address the gap

in the Iranian in order to investigate the sources of controversies as to the nature of the program and to see the extent to


which the common standards have been included in the program the manner it is implemented. To address the problem

quantitatively and qualitatively, two macro research questions each accommodating six and fourteen minor questions,

respectively, were posed and investigated in the form of respective research hypotheses.

Research Questions

Major Research Question One: Addressing the Common Standards

Is Iranian TEFL MA Program aligned with the standards of an effective educational program?

Minor Research Question 1: Are the teachers consistent in their description of TEFL MA Program?

Minor Research Question 2: Do TEFL MA Program Admission Requirements meet the standards of an effective

educational program?

Minor Research Question 3: Does TEFL MA Program Content meet the standards of an effective educational pro-

gram?

Minor Research Question 4: Do TEFL MA Program Resources meet the standards of an effective educational pro-

gram?

Minor Research question 5: Do TEFL MA Program Implementation and Evaluation Methods require some revi-

sions to meet the standards of an effective educational program?

Minor Research Question 6: Do TEFL MA Program Graduation Requirements meet the standards of an effective

educational program?

Major Research Question Two: Addressing the Program Features and Courses

Does Iranian TEFL MA Program meet the teachers' expectations?

Minor Research Question 1: Does TEFL MA Program schedule meet the teachers' expectations?

Minor Research Question 2: Do TEFL MA Program objectives meet the teachers' expectations?

Minor Research Question 3: Is TEFL MA Program compatible with Learner-Oriented Education?

Minor Research Question 4: Are TEFL MA Program Evaluation Methods in concordance with Program Objec-

tives and Content?

Minor Research Question 5: Do TEFL MA Program Learners and Admission Requirements meet the teachers'

expectations?

Minor Research Question 6: Does Issues in Linguistics Course meet the teachers' expectations?

Minor Research Question 7: Does Phonology of TESOL Course meet the teachers' expectations?

Minor Research Question 8: Does Methods of Teaching a Foreign Language Course meet the teachers' expecta-

tions?

Minor Research Question 9: Does Methods of Research Course meet the teachers' expectations?

Minor Research Question 10: Does Testing a Foreign Language Course meet the teachers' expectations?

Minor Research Question 11: Does Practice Teaching Course meet the teachers' expectations?

Minor Research Question 12: Does Teaching Language Skills Course meet the teachers' expectations?

Minor Research Question 13: Does Materials Preparation Course meet the teachers' expectations?

Minor Research Question 14: Does Seminar Course meet the teachers' expectations?

METHOD

This research study was conducted during May 2014 to August 2014 with a randomly selected participation of 30 ex-

perts holding an MA in TEFL from four different public and private Iranian universities in Tehran. To collect the required

data (Quantitative and Qualitative) two distinct questionnaires named Curriculum Evaluation Questionnaire (Questionnaire

No.1) and Approved Syllabus Evaluation Questionnaire (Questionnaire No.2) were distributed. Questionnaire No.1 with six

items was a modified version of a curriculum evaluation questionnaire for the Private Training Institution which had al-

ready been validated and used by Abbasian and Afsharimani (2013). In order to ease the data analysis procedure, open-


ended questions were used as supplementary mechanism along with a Likert-Scale (Strongly Agree, Agree, Neutral, Disa-

gree, and Strongly Disagree). Questionnaire No. 2 had two main sections and each section had five and nine subsections

respectively. First section was a modified version of Approved Syllabus Evaluation Questionnaire for Defensive Works

Program at Imam Ali University (2014). Second section was developed on the basis of Overall Characteristics and Ap-

proved Syllabus for Iranian TEFL MA Program in 2007. The reliability index was estimated through alpha Cronbach

showing high enough index (i.e., .93). As to the Validity analysis, the first section of this questionnaire was previously

validated and used by Abbasian and Rostamzadeh Torghabeh (2014). Regarding the instrument Construct Validity, Varimax

rotation method via Factor analysis was run for the questionnaire subsections separately. Sampling Sufficiency Hypothe-

ses on the basis of KMO index and Bartlet (KMO=.90>.60) (x2 (36) =954.52, P<.05) approved the high unity index among

the variables=KMO. Meanwhile, there was one factor with 55.73 of the Variance.

RESULT

Major Research Question One: Addressing the Common Standards

Since evaluation of Iranian TEFL MA Program required the assessment of six educational dimensions, first main re-

search question was accompanied with six minor research questions, so in this section all the questions are presented with

the teachers' responses and their statistical analysis respectively.

Is Iranian TEFL MA Program aligned with the standards of an effective educational program?

Among the 30 teachers who participated in the research, 49.8% expressed their agreement, 22.2% stated disagreement

and 28.0% had no idea. Table 1 shows the answers given by the teachers to this question.

Table 1

Frequencies and Percentages TEFL MA Program

The chi-square observed value of 170.11 (P= .000< .05) (Table 2) indicates that the differences observed in Table 1 are

meaningful, i.e. the differences are not obtained by chance. Based on these results, the first null-hypothesis was rejected.

Table 2

Analysis of Chi-Square Test

CHOICS

Chi-Square 170.117a

Df 4

Asymp. Sig. .000

a. 0 cells (.0%) have expected frequencies less than

5. The minimum expected cell frequency is 120.0.

Minor Question 1: Teachers' consistency in program description

The findings revealed that 51.6% of the teachers indicated their satisfaction, 18.3% reported their dissatisfaction and

30.0% held neutral position (Table 3).

Frequency Percent

Strongly Agree

Agree

Neutral

Disagree

Strongly Disagree

Total

95 15.8

204 34.0

168 28.0

115 19.2

18 3.0

600 100.0


Table 3

Frequencies and Percentages TEFL MA Program Description

Frequency Percent

Strongly Agree 11 18.3

Agree 20 33.3

Neutral 18 30.0

Disagree 11 18.3

Total 60 100.0

The results of the analysis of chi-square; 4.40 (P= .221> .05) (Table 4) indicate that the frequencies and percentages do

not show significant differences between the respondents' views, so the first minor null-hypothesis was retained.

Table 4

Analysis of Chi-Square Description of TEFL MA Program

CHOICES

Chi-Square 4.400a

Df 3

Asymp. Sig. .221

a. 0 cells (.0%) have expected frequencies less than 5.

The minimum expected cell frequency is 15.0.

Minor Question 2: Program Admission Requirements

According to Table 5, 61.1% of the teachers stated their agreement, 12.2% held opposite view and 26.7% held neutral

position.

Table 5

Frequencies and Percentages TEFL MA Program Admission Requirements

Frequency Percent


Agree 29 32.2

Neutral 24 26.7

Disagree 10 11.1

Strongly Disagree 1 1.1

Total 90 100.0

The chi-square observed value of 31.88 (P= .000< .05) (Table 6) indicates that the frequencies and percentages ob-

served show significant differences between the respondents' views. Thus, the second minor null-hypothesis was rejected.

Table 6

Analysis of Chi-Square Admission Requirements of TEFL MA Program

CHOICES

Chi-Square 31.889a

Df 4

Asymp. Sig. .000




Minor Question 3: Program Content

For the TEFL MA Program Content, 48.9% of the teachers expressed agreement, 30.0% of the participants selected

disagree and 21.1% chose neutral scale (Table 7).

Table 7

Frequencies and Percentages TEFL MA Program Content

Frequency Percent


Agree 60 33.3

Neutral 38 21.1

Disagree 48 26.7


Total 180 100.0

The chi-square observed value of 46.88 (P = .000 < .05) (Table 8) indicates that the frequencies and percentages

observed show significant differences between the respondents' views. As a result, it can be concluded that the third mi-

nor null-hypothesis was rejected.

Table 8

Analysis of Chi-Square Content of TEFL MA Program

CHOICES

Chi-Square 46.889a

Df 4

Asymp. Sig. .000



Minor Question 4: Program Resources

Based on the information displayed in Table 9, 44.6% of the teachers indicated their agreement, 22.7% held opposite

view and 32.7% held neutral position.

Table 9

Frequencies and Percentages TEFL MA Program Resources

Frequency Percent


Agree 56 37.3

Neutral 49 32.7

Disagree 28 18.7


Total 150 100.0

The chi-square observed value of 65.93 (P = .000 < .05) (Table 10) indicates that the frequencies and percentages ob-

served show significant differences between the respondents' views, so the fourth minor null-hypothesis was rejected.


Table 10

Analysis of Chi-Square Resources of TEFL MA Program

CHOICES

Chi-Square 65.933a

Df 4

Asymp. Sig. .000



Minor question 5: Program Implementation and Evaluation

Concerning the Program implementation and evaluation, 54.4% of the teachers rated this item agree, 13.3% of the par-

ticipants selected disagree and 32.2% chose neutral scale (Table 11).

Table 11

Frequencies and Percentages TEFL MA Program Implementation and Evaluation Methods

Frequency Percent


Agree 31 34.4

Neutral 29 32.2

Disagree 9 10.0


Total 90 100.0


served show significant differences between the respondents' views. Thus, it can be concluded that the fifth minor

null-hypothesis was rejected.

Table 12

Analysis of Chi-Square of TEFL MA Program Implementation and Evaluation Methods

CHOICES

Chi-Square 33.111a

Df 4

Asymp. Sig. .000



Minor Question 6: Program Graduation Requirements

Based on the information displayed in Table 13, 36.7% showed their disagreement, 30.0% held opposite view, while

33.3% held neutral position.


Table 13

Frequencies and Percentages TEFL MA Program Graduation Requirements

Frequency Percent


Agree 8 26.7

Neutral 10 33.3

Disagree 9 30.0


Total 30 100.0

The chi-square observed value of 11.66 (P= .020< .05) (Table 14) indicates that the frequencies and percentages

observed show significant differences between the respondents' views, so the sixth minor null-hypothesis was rejected.

Table 14

Analysis of Chi-Square Graduation Requirements of TEFL MA Program

CHOICES

Chi-Square 11.667a

Df 4

Asymp. Sig. .020



Qualitative Data: Teachers' Comments

Responding the Questionnaire No.1, teachers suggested number recommendations for TEFL MA program improve-

ments that the most frequent ones are represented in Table 15 as below:

Table 15

Teachers' Qualitative Comments

Item Teachers' Comments

Program Length 3 yrs (71.4%)

4 yrs (14.2%)

Academic Qualifications TOEFL Score (31.8%)

IELTS Score (27.2%)

Pre-Requisite Courses Teaching English for Children (29.4%)

Teaching English for Adults (23.5%)

Insufficient Number of Credits Methods of Teaching a Foreign Language (25%)

Teaching Language Skills (15%)

Revised/Removed/Added

Courses

Advanced Writing (Revised) (12.9%)

Contrastive Linguistics & Error Analysis (Removed)

(16.1%)

Basic Statistics (Added) (12.9%)

Program Resources Usage of authentic materials (25%)

High speed internet access (18.7%)

Instructional Methods Giving lecture (19.2%)


Table 15 continued….

Cooperative activities (15.3%)

Evaluation Methods Teachers' comments (27.5%)

Learner-made books (20.6%)

Graduation Requirements Cut-off scores for modules (26.6%)

Grades for developing materials (20%)

Occupational Positions English Language Teacher (24%)

Translator (8%)

Major Research Question Two: Addressing the Program Features and Courses

As we mentioned before, Questionnaire No.2 had fourteen subsections. Thus, in order to investigate the second main

research question, fourteen sub-questions were designed to fulfill the whole dimensions of Iranian TEFL MA Program Ap-

proved Syllabus.

Does Iranian TEFL MA Program meet the teachers' expectations?


and 21.3% were neutral. Table 16 shows the answers given by the teachers to this question.

Table 16

Frequencies and Percentages TEFL MA Program

Frequency Percent

Strongly Agree

Agree

Neutral

Disagree

Strongly Disagree

Total

166 7.8

864 40.6

454 21.3

522 24.5

124 5.8

2130 100.0

The chi-square observed value of 846.59 (P= .000< .05) (Table 17) indicates that the differences observed are mean-

ingful, i.e. the differences are not obtained by chance. Based on these results, the second null-hypothesis was rejected.

Table 17

Analysis of Chi-Square

CHOICES

Chi-Square 846.592a

Df 4

Asymp. Sig. .000



Minor Question 1: Program schedule

The findings revealed that 45.8% of the teachers indicated their dissatisfaction, 30.8% reported their satisfaction and



Table 18

Frequencies and Percentages TEFL MA Program Schedule

Frequency Percent


Agree 34 28.3

Neutral 28 23.3

Disagree 48 40.0


Total 120 100.0

The results of the analysis of chi-square; 59.25 (P= .000< .05) (Table 19) indicate that the differences observed are

meaningful, i.e. the differences are not obtained by chance, so the first minor null-hypothesis was rejected.

Table 19

Analysis of Chi-Square Schedule of TEFL MA Program

CHOICES

Chi-Square 59.250a

Df 4

Asymp. Sig. .000



Minor Question 2: Program objectives

According to Table 20, 49.3% of the teachers stated their agreement, 31.3% held opposite view and 19.3 percent held

neutral position.

Table 20

Frequencies and Percentages TEFL MA Program Objectives

Frequency Percent


Agree 141 47.0

Neutral 58 19.3

Disagree 78 26.0


Total 300 100.0

The chi-square observed value of 193.90 (P= .000< .05) (Table 21) indicates that the frequencies and percentages

observed show significant differences between the respondents' views. Thus, the second minor null-hypothesis was re-

jected.


Table 21

Analysis of Chi-Square Objectives of TEFL MA Program

Chi-Square 193.900a

Df 4

Asymp. Sig. .000

a. 0 cells (.0%) have expected frequencies less than 5. The

minimum expected cell frequency is 60.0.

Minor Question 3: Program Learner-Orientedness

Based on the information displayed in Table 22, 46.0% of the teachers indicated their agreement, 28.5% held opposite

view and 25.6% held neutral position.

Table 22

Frequencies and Percentages TEFL MA Program Learner-Qrientedness

Frequency Percent


Agree 116 43.0

Neutral 69 25.6

Disagree 66 24.4


Total 270 100.0


served show significant differences between the respondents' views, so the fourth minor null-hypothesis was rejected.

Table 23

Analysis of Chi-Square Education of TEFL MA Program

Chi-Square 151.444a

Df 4

Asymp. Sig. .000



Minor Question 4: Program Evaluation Validity

Concerning the TEFL MA Program Evaluation Validity, 47.4% of the teachers rated this item agree, 28.1% of the par-

ticipants selected disagree and 24.4% chose neutral scale (Table 24).

Table 24

Frequencies and Percentages TEFL MA Program Evaluation Validity

Frequency Percent


Agree 103 38.1

Neutral 66 24.4

Disagree 67 24.8


Total 270 100.0



served show significant differences between the respondents' views. Thus, it can be concluded that the fifth minor


Table 25

Analysis of Chi-Square of TEFL MA Program Evaluation Validity

Chi-Square 103.333a

Df 4

Asymp. Sig. .000

a. 0 cells (.0%) have expected

frequencies less than 5. The

minimum expected cell frequency is 54.0.

Minor Question 5: Program Admission Requirements

Based on the information displayed in Table 26, 50.5% showed their disagreement, 20.6% held opposite view, while


Table 26

Frequencies and Percentages TEFL MA Program Admission Requirements

Frequency Percent

Strongly Agree 1 .6

Agree 36 20.0

Neutral 52 28.9

Disagree 60 33.3


Total 180 100.0


served in Table 40 show significant differences between the respondents' views, so the sixth minor null-hypothesis was

rejected.

Table 27

Analysis of Chi-Square Admission Requirements of TEFL MA Program

CHOICES

Chi-Square 57.833a

Df 4

Asymp. Sig. .000



Minor Question 6: Issues in Linguistics Course


and 19.2% had no idea. Table 28 displays the answers given by the teachers to this question.


Table 28

Frequencies and Percentages Issues in Linguistics Course

Frequency Percent


Agree 48 40.0

Neutral 23 19.2

Disagree 25 20.8

Strongly Disagree 1 .8

Total 120 100.0

The chi-square observed value of 46.16 (P= .000< .05) (Table 29) indicates that the differences observed are meaning-

ful, i.e. the differences are not obtained by chance. Based on these results, the first null-hypothesis was rejected.

Table 29

Analysis of Chi-Square Issues in Linguistics Course

CHOICES

Chi-Square 46.167a

Df 4

Asymp. Sig. .000



Minor Question 7: Phonology Course

The findings revealed that 56.6% of the teachers indicated their satisfaction, 18.3% reported their dissatisfaction and


Table 30

Frequencies and Percentages Phonology Course

Frequency Percent


Agree 55 45.8

Neutral 30 25.0

Disagree 19 15.8


Total 120 100.0


meaningful, i.e. the differences are not obtained by chance, so the eighth minor null-hypothesis was rejected.


Table 31

Analysis of Chi-Square Phonology Course

CHOICES

Chi-Square 66.000a

Df 4

Asymp. Sig. .000



Minor Question 8: TESOL Methodology Course

According to Table 32, 85.0% of the teachers stated their agreement, 3.3% held opposite view and 11.7 percent held

neutral position.

Table 32

Frequencies and Percentages TESOL Methodology Course

Frequency Percent


Agree 38 63.3

Neutral 7 11.7

Disagree 2 3.3

Total 60 100.0


served show significant differences between the respondents' views. Thus, the ninth minor null-hypothesis was rejected.

Table 33

Analysis of Chi-Square TESOL Methodology Course

Chi-Square 51.067a

df 3

Asymp. Sig. .000



Minor Question 9: Methods of Research Course

For the Methods of Research Course, 76.7% of the teachers rated this item agree, 5.0% of the participants selected

disagree and 18.3% chose neutral scale (Table 34).

Table 34

Frequencies and Percentages Research Methods Course

Frequency Percent


Agree 36 60.0

Neutral 11 18.3

Disagree 2 3.3


Total 60 100.0



served show significant differences between the respondents' views. As a result, it can be concluded that the tenth minor


Table 35

Analysis of Chi-Square Research Methods Course

CHOICES

Chi-Square 66.833a

Df 4

Asymp. Sig. .000



Minor Question 10: Foreign Language Testing Course

Based on the information displayed in Table 36, 81.7% of the teachers indicated their agreement and 18.3% held neutral

position.

Table 36

Frequencies and Percentages Foreign Language Testing Course

Frequency Percent


Agree 40 66.7

Neutral 11 18.3

Total 60 100.0


served show significant differences between the respondents' views, so the eleventh minor null-hypothesis was rejected.

Table 37

Analysis of Chi-Square Foreign Language Testing Course

CHOICES

Chi-Square 30.100a

Df 2

Asymp. Sig. .000



Minor Question 11: Practice Teaching Course

Concerning the Practice Teaching Course, 65.5% of the teachers rated this item agree, 24.5% of the participants select-

ed disagree and 10.0% chose neutral scale (Table 38).


Table 38

Frequencies and Percentages Practice Teaching Course

Frequency Percent


Agree 48 53.3

Neutral 9 10.0

Disagree 15 16.7


Total 90 100.0


served show significant differences between the respondents' views. Thus, it can be concluded that the twelfth minor


Table 39

Analysis of Chi-Square Practice Teaching Course

CHOICES

Chi-Square 64.444a

Df 4

Asymp. Sig. .000



Minor Question 12: Teaching Language Skills Course

Based on the information displayed in Table 40, 75.0% showed their agreement, 10.0% held opposite view, while


Table 40

Frequencies and Percentages teaching Language Skills Course

Frequency Percent


Agree 35 58.3

Neutral 9 15.0

Disagree 4 6.7


Total 60 100.0


served show significant differences between the respondents' views, so the sixth minor null-hypothesis was rejected.


Table 41

Analysis of Chi-Square Teaching Language Skills Course

CHOICES

Chi-Square 58.833a

Df 4

Asymp. Sig. .000



Minor Research Question 13: Materials Preparation Course


and 26.7% had no idea. Table 42 displays the answers given by the teachers to this question.

Table 42

Frequencies and Percentages Materials Preparation Course

Frequency Percent


Agree 26 28.9

Neutral 24 26.7

Disagree 26 28.9


Total 90 100.0

The chi-square observed value of 23.55 (P= .000< .05) (Table 43) indicates that the differences observed are mean-

ingful, i.e. the differences are not obtained by chance. Based on these results, the fourteenth null-hypothesis was rejected.

Table 43

Analysis of Chi-Square Materials Preparation Course

Chi-Square 23.556a

Df 4

Asymp. Sig. .000



Minor Question 14: Seminar Course

The findings revealed that 52.2% of the teachers indicated their dissatisfaction, 28.9% reported their satisfaction and


Table 44

Frequencies and Percentages Seminar Course

Frequency Percent

Agree 26 28.9

Neutral 17 18.9

Disagree 35 38.9


Total 90 100.0



meaningful, i.e. the differences are not obtained by chance, so the fifteenth minor null-hypothesis was not rejected.

Table 45

Analysis of Chi-Square Seminar Course

CHOICES

Chi-Square 13.733a

Df 3

Asymp. Sig. .003



Discussion and Conclusion

This study addressed two dimensions as to the Iranian TEFL MA Program evaluation; compatibility with the commonly

known standards and features of specific courses. These two areas were investigated posing two macro research questions

each accommodating six and fourteen minor research questions, respectively. Each macro research question along with its

respective minor questions was followed up using two separate questionnaires along with a number of open-ended ques-

tions.

As far as the first macro question was concerned the findings statistically indicated that from the participants’ perspec-

tives the Program mainly meets the common standards including program description, admission requirements, content,

resources, implementation and evaluation requirements. Then, all of the respective hypotheses were statistically rejected.

Concerning the second macro question followed through fourteen sub-questions including program schedule, objec-

tives, compatibility learner-oriented education, evaluation methods, admission requirements and specific courses like Issues

in Linguistics, Phonology, TESOL Methodology, Research Methods, Foreign Language Testing, Practice Teaching,

Teaching Language Skills, Materials Preparation, and Seminar the teachers expressed their satisfaction except as to the

Program schedule, admission requirements, and Seminar Course, meaning that all the respective hypotheses were statisti-

cally rejected. However, run based on Mixed-methods research whereupon the questions were triangulated, the qualita-

tive side of the coin revealed interesting findings. Teachers commented on nine main areas such as the Program length,

academic qualifications, and necessity of inclusion of pre-requisite courses, revision/removal and addition of some course,

resource quality, program implementation procedures, evaluation system, and graduation requirements. Majority of the

teachers believed that class time allocated for teaching required courses such as Methods of Teaching a Foreign Language

and Teaching Language Skills is not enough. They also indicated that TOEFL or IELTS scores in addition to the entrance

exams provide a better view towards the learners' selection. From their viewpoints, learners should be placed on the basis

of their language competency. Thus, it can be concluded that entrance exams just test learners' theoretical knowledge and

reveal nothing regarding their competencies and performance.

On the Program content and sources, they supported the addition of some other courses like Teaching English for Chil-

dren and Adults justified based on the fact that most of the TEFL MA graduates are employed in private institutes, so they

should be trained to cope with such situations. As a result, it can be concluded that the existing program courses provide

little to average knowledge and information about what if learners actually encountered in the real situation, so that curric-

ulum developers should take the teachers' and learners' views into account and remove the irrelevant modules and replace

them with other courses which can be more practical and useful. Regarding the program implementation procedures and

evaluation methods, the teachers mostly expressed their agreement toward research-based and assessment-based education.

Most of the teachers criticized the Seminar module headlines and teachers' inconsistency in this course teaching style.

They also suggested exercise of some modifications in order to enhance the Program for example lengthening it at least to

four years, removal of some modules from the syllabus, inclusion of internet technology, and modifying the manner classes


can be held. They also believed that teachers and learners are two important target groups in any educational course, so

they should be involved more in all steps decision-making and implementation of the Program.

In short, the following suggestions and modifications as the implications of the study are made to overcome the above

mentioned problems and improve the TEFL MA Program:

1. Pre-requisite courses such as Teaching English for Children and Adults, courses for learners with limited English

proficiency, and Basic courses for computer should be added.

2. The number of credits for some courses such as Methods of Teaching a Foreign Language, Teaching Language

Skills, Advanced Writing, Practice Teaching, and Methods of Research be increased.

3. Classes be held in seminars with more emphasis on giving lectures, project, and research-based education.

4. Institutionally-prepared materials, making lesson plan, writing conference, writing portfolio, doing team-building

teaching, and incorporation of teachers' comments during the course be taken into account.

5. Universities should have access to the high speed internet and other modern technologies.

6. Other requirements such as TOEFL and IELTS scores be considered in learners' admission.

Finally, as far as the findings of this study and those of the related literature are concerned, we can sum up the following

areas of match and mismatch:

1. Concerning the related literature on the evaluation programs and standards, another study was conducted by Ab-

basian and Afsharimani (2013) on the Modular English Language Educational Program. The findings of this study also

revealed the alignment of this educational program with the conventional standards.

2. As the results show, although TEFL MA Program content or Approved Syllabus is in line with the conventional

standards, it needs some modifications such as removal or revising of some of the courses to make this program delivery

stronger. In this regard, Rostamzadeh Torghabeh (2005) did another study to evaluate the status of translation education

at BA level in Azad University. The findings revealed that BA English Translation Program Approved Syllabus was not

successful in training the learners and increasing their abilities in translation.

3. Regarding the educational program resources, the findings of this study revealed that TEFL MA Program resources

are satisfactory and would strengthen the program delivery. As to the review of the related literature, Shafiee Nahrkhalaji

(2012) conducted a study which evaluated the EFL textbooks such as Interchange, Headway, and Top Notch. The results

of this study revealed that all skills are perfectly covered in Top Notch book.

4. On the evaluation requirements, although most of the teachers and learners stated their agreement towards TEFL

MA Program evaluation methods, some alternative evaluation methods such as project-based assessment were recom-

mended by teachers and learners. This finding is in line with Jabbarifar (2009) study which investigated the importance of

classroom assessment. The findings of this study indicated some positive aspects of the class evaluation such as increasing

the learners' motivation and making better educational decisions.

5. The results of this study revealed that TEFL MA Program education is in line with learner-oriented education.

Regarding the review of the related literature, a further evaluation study was carried out by Brady, Dickinson, Hirschler,

Cross, and Green (1998) on the child-centered and collaborative education. The findings of this study revealed that usage of

cooperative and step by step education and also democratic practices is growing up rapidly.

REFERENCES

Abbasian, G. R., & Afsharimani, S. (2013). Modular educational program in teaching English as a foreign language:

Modular education and EFL. Saarbrucken, Germany: LAP Lambert Academic Publishing.

Abbasian, G. R., & Rostamzadeh Torghabeh, A. (2014). Evaluation of Imam Ali Officer University approved syllabus with

emphasis on the secondary components. Periodical Military Management, 54 (14), 71-100.

Brady, J., Dickinson, D., Hirschler, J., Cross, T., & Green, L. (1998). Evaluation of the step by step program: Executive


summary. Arlington, Virginia: Education Development Center.

Council of Chief State School Officers. (2010). Interstate teacher assessment and support consortium model core teaching

standards: A resource for state dialogue. Washington, DC: Author.

Curriculum Evaluation Questionnaire PTI 11. (2010). Pursuant to the private training institutions act and regulations divi

sion of institutional and industrial education. Retrieved from

http://www.aes.gov.nl.ca/forms/privatetraining/PTI11.pdf

Hogan, R. L. (2007). The historical development of program evaluation: Exploring the post and present. Journal of Work

force Education & Development, 11 (4), 1-14.

International Center for Educational Policies. (2014). What is evaluation? Retrieved from http://www.icap.org.

Jabbarifar, T. (2009). The importance of classroom assessment and evaluation in educational system. Proceedings of the 2nd

International Conference of Teaching and Learning, 1- 9. Malaysia: INTI University College.

Rostamzadeh Torghabeh, A. (2005). Evaluation of undergraduate translation program in Azad University. The Journal of

Quality in Education, 2, 1-19.

Shafiee Nahrkhalaji, S. (2012). An evaluation of a global ELT textbook in Iran: A two- phase approach. International

Journal of Humanities and National Science, 2 (3), 184-191.

Richards, J. C. (2001). Curriculum development in language teaching. Cambridge: Cambridge University Press.

Stake, R. E. (1973). Program evaluation particularly responsive evaluation. Retrieved from

www.ed.uiuc.edu/circe/Publications/Responsive_eval.pdf.

Further Reading

Approved Defensive Works Program Syllabus and Overall Characteristics (2014). Imam Ali University. Retrieved from

http://www.magiran.com/view.asp?Type=pdf&ID=1317821&l=fa

Approved TEFL MA Program Syllabus and Overall Characteristics in Iran (2007). Iranian Ministry of Science, Research,

and Technology. Retrieved from http://www.msrt.ir/fa/prog/ApprovedCourses/Forms/AllItems.aspx

MA Teaching English as a Foreign Language (2014). University of Birmingham. Retrieved from

www.birmingham.ac.uk/tefl.

Masters of Education in Teaching English as a Foreign Language (2014). University of Jambi. Retrieved from

http://unja.ac.id/pps

http://www.aes.gov.nl.ca/forms/privatetraining/PTI11.pdf

http://www.icap.org/

http://www.ed.uiuc.edu/circe/Publications/Responsive_eval.pdf

http://www.msrt.ir/fa/prog/ApprovedCourses/Forms/AllItems.aspx

http://www.birmingham.ac.uk/tefl

http://unja.ac.id/pps

Documents

Evaluation of Iranian TEFL M.A. Program in Terms of ...eltvoices.in/Volume6/Issue1/EVI_61_4.pdf · Program Evaluation and the Approaches ... Management- Oriented Approach (CIPP):