A Catalyst for Teaching Critical Thinking in a Large University Class in Taiwan Discussions Online With Teaching Assistant

7/31/2019 A Catalyst for Teaching Critical Thinking in a Large University Class in Taiwan Discussions Online With Teaching Ass

1/25

R E S E A R C H A R T I C L E

A catalyst for teaching critical thinking in a large

university class in Taiwan: asynchronous onlinediscussions with the facilitation of teaching assistants

Ya-Ting C. Yang

Published online: 2 August 2007 Association for Educational Communications and Technology 2007

Abstract This study was designed to investigate the effects of teaching critical thinking

skills (CTS) in a large class through asynchronous discussion forums (ADFs) with the

facilitation of teaching assistants. A pretest and posttest quasi-experimental design with a

comparison group was employed to validate the effectiveness of the proposed approach.

The independent variable was the structured ADFs with two levelswithout Socratic

dialogues, and with Socratic dialogues, modeled and facilitated by the teaching assistants

via structured ADFs, while the dependent variable was the students levels of CTS asmeasured by two different evaluations: (a) the California Critical Thinking Skills Test, to

holistically examine students gains in their CTS, and (b) the Coding Scheme for Evalu-

ating Critical Thinking in Computer Conferencing, to investigate students interaction

patterns and the depth of their critical thinking (CT) demonstrated via the ADF. The

evaluation data were collected from 278 college students in Taiwan. The qualitative

analysis provided a detailed description of how students discussions moved from the

lower to the higher phases of CT. Results indicated that an inspired instructor and

some energetic teaching assistants who use Socratic dialogues during small-group online

discussions can successfully develop students CTS in a large university class.

Keywords Critical thinking Computer-mediated communication General education Large class size Graduate teaching assistants

One of the greatest experiences for students in higher education is to have the opportunity

to think freely and challenge other students ideas with their own. This ability to think

critically and argue logically is a precious, invaluable asset that benefits students as they

journey through life. Thus, the hallmark of higher education is to teach and develop

students critical thinking skills (CTS) (Barnett 2004; Facione 2007; Garrison et al. 2001;

Education Tech Research Dev (2008) 56:241264

DOI 10.1007/s11423-007-9054-5
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


2/25

Yang et al. 2005; Yeh 2006). Ennis (1985) offered a widely accepted definition of critical

thinking (CT) as reasonable, reflective thinking that is focused on deciding what to

believe or do (p. 46). Several teaching strategies, such as cooperative learning, reciprocal

teaching (Brown and Palincsar 1989), communities of practice (Wenger 1998), case study

pedagogy, and the integration or the direct teaching of CTS, have been proposed to helppromote CTS (Daud and Husin 2004).

Meanwhile, King (1990) and Taba (1966) have suggested that the level of thinking that

occurs within a given situation is influenced by the level of questions asked. That is,

thoughtful questions play an important role in inducing students higher-level cognitive

processes, which include self-reflection, revision, social negotiation (Brown and Palincsar

1989; Wenger 1998), and conceptual change of student misconceptions, all of which are

integral to CT. However, when students are asked to generate questions on their own,

factual rather than thought-provoking questions are generally posed (Dillon 1988; Flammer

1981; King 1990). Socratic questioning is one of the most popular and powerful teaching

approaches to use in guiding students to generate thoughtful questions that will foster their

CTS (Maiorana 19901991; Paul 1995). Instead of providing direct answers, the Socratic

questioning approach stimulates students minds by continually probing the subject with

thought-stimulating questions (Paul). Through interactive discussions that include rational

dialogue and questioning between the instructor and students and among students, Socratic

questioning can facilitate students CTS by the exchange of ideas and viewpoints, giving

new meaning to content, exploring applications to problems, and providing implications

for real-life situations (Yang et al. 2005).

However, in a traditional classroom, discussions are often hindered due to limited class

time and unequal access of interaction (e.g., a small number of students dominate thediscussion). If collaborative inquiry tasks are assigned outside of class time, it is difficult if

not impossible for the instructor to monitor collaborative discussions and guide students

through the thinking process. However, the advancement in information technology has

changed the way teaching and learning are traditionally conducted. For instance, emerging

technologies such as asynchronous discussion forums (ADFs) (i.e., text-based computer-

mediated communication tools) have become promising tools that extend opportunities for

interactive discussions outside the classroom, allowing students to compose their thoughts

in writing, and at the same time, providing opportunities for the instructor to moderate such

discussions (Duffy et al. 1998). It has been argued that appropriately designed online

discussions can facilitate interaction between the instructor and learners and among thelearners themselves. However, while student participation is necessary for successful

online discussions, it is hardly sufficient. In an unstructured online discussion where no

facilitators organize or guide students discussion, students may talk for hours or post many

messages without learning anything of substance. Therefore, the use of well-organized and

well-facilitated discussions within structured online forums is just the component needed in

higher education to achieve the larger instructional goal of developing students CTS

(Duffy et al. 1998; Harasim et al. 1995). At the same time, further research is required in

order to address questions about the components of truly successful online discussions

using structured ADFs, and to determine whether structured ADFs can promote thedevelopment of students CTS.

Angeli et al. (2003) investigated the quality of ADFs. The results of their study showed

242 Y.-T. C. Yang
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


3/25

Khine 2006) or significantly higher performance for students just making passing grades

(Davies and Graff 2005). It was suggested that the lack of an assessment strategy or well-

trained mentors to guide, model, and evaluate students online contributions might have

influenced the quality of messages posted (Gilbert and Dabbagh 2005).

Related to this research, Gunawardena et al. (1997) studied the quality of onlineinteractions in a list-serve debate format, from both the interactive and cognitive dimen-

sions. Based on constructivist learning theories, they developed an assessment tool, the

Interaction Analysis Model (IAM), to assess the exchanges made among class members

and the way these exchanges moved from the lower to the higher phases of CT. In a related

study, Newman et al. (1995) presented a content analysis method to measure the depth of

CT in face-to-face and computer-supported group learning. Both Gunawardena et al.s and

Newman et al.s studies were able to provide some evidence that CT can occur in computer

conferencing and that such processes occur as a direct result of exchanges among par-

ticipants. Using case study and qualitative research methods, Bullen (1998) and Salmon

(2002) likewise concluded that reflection-on-action and CT development can be demon-

strated through computer conferencing. Other related research includes that of Jeong

(2003) and Walker (2004), which showed that structured ADFs can support the develop-

ment of CTS. These studies suggest that with appropriate assessment strategies and

well-trained mentors for guiding student learning, students CTS can be fostered and

demonstrated through structured online discussions. Using a quasi-experimental design,

Yang et al. (2005) made a first step toward ascertaining the effectiveness of structured ADF

discussions in developing students CTS. Positive gains in students CTS and attitudes

provide empirical evidence that instructional designs incorporating asynchronous online

discussion and interaction can be effective and conducive to the development of CTS.Most of the above research was conducted in small regular classroom contexts or in

distance learning environments. Although ADF discussions tend to become the norm in

these learning environments, the difficulty of increasing students CTS in large classes is

the challenge most instructors face when using this strategy, such as in many general

education courses (i.e., those with more than 100 students). This is because an open

discussion may be impossible to manage or may greatly limit students participation. As a

result, it becomes easy for most students to assume a passive role, merely recording the

facts that the instructor conveys in the lecture, thus leading the students to a shallow

understanding of the course materials. The typical solution to this problem is to have

students pair up to discuss a question or problem for 5 min and then come back for a full-group discussion. Unfortunately, the questions asked in this short period of time are usually

very brief and shallow. With the advent of networked computers and Internet technology,

new teaching methodologies are, therefore, needed so that students do not fall prey to the

inadequacies of traditional studentteacher-based models of interaction (Preston and

Shackelford 1998). ADF tools seem to extend opportunities for interaction, provide all

students with a chance to contribute, and motivate students to compose their thoughts in

writing.

To extend prior research and explore applications of long-used teaching and learning

methods in online instruction, the current study, based on Yang et al. (2005) study, thustakes a further step by applying both quantitative and qualitative analyses to assess the

impact of improving students CTS via ADFs in a large class. The major goal was to

Teaching critical thinking in a large university class 243
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


4/25

1. In a large university class, will students levels of CTS improve after they participate

in Socratic dialogues, as modeled and facilitated by the facilitator via ADFs?

2. In a large university class, will students who participate in Socratic dialogues, as

modeled and facilitated by the facilitator in ADFs, have higher levels of CTS than

those who participate in ADFs, where Socratic dialogues are not fostered?

Method

Design and participants

A pretest and posttest quasi-experimental design with a comparison group was employed to

answer the above two research questions. Students enrolled in Electricity and Life, an

undergraduate-level general education course offered by the Electrical Engineering

Department at a large university in Taiwan, participated in the study. The purpose of this

course was to foster understanding by non-engineering students of the fundamental con-

cepts and terminology for electricity that are used frequently in daily life. The contents

included eight units: (1) nature and origin of electricity, (2) electricity and magnetism, (3)

classification of batteries, (4) direct current (DC) generators and DC motors, (5) alternating

current (AC) generators and AC motors, (6) transformers, (7) concepts of circuit, and (8)

characteristics and applications of electrical power. The course typically draws a diverse

group of students from different fields of study, such as Statistics, Economics, Business,

History, Chinese Literature, Medicine, and Biochemistry. There were two sections for thiscourse: Section I (experimental group) and Section II (comparison group). These two

sections were randomly assigned to experiment/comparison groups and had the same

instructor, course content, schedule, examinations, and online discussions to complete.

Two credit hours were awarded for successful completion of this course.

Initially 150 and 142 students enrolled in this course in Sections I and II, respectively.

Excluding students non-responses and unintentional skips on some items of the test/

survey, 278 complete sets of data were collected: 145 in Section I (49% males and 51%

females) and 133 in Section II (51% males and 49% females). The students ages varied

from 19 to 22, with an average age of 19.55 years old in Section I and 20.16 years old in

Section II. All were full-time students, had previous computer experience, and had accessto the Internet at home or at the university.

Independent and dependent variables

The independent variable was the structured ADFs, which refer to a variety of organized

dialogues in which a facilitator leads the students and facilitates exercises that can help

bring about more productive conversations to achieve the desired instructional goal via

online discussions. The structured ADFs have two levels: without Socratic dialogues, and

with Socratic dialogues, modeled and facilitated by the facilitator via structured ADFs. The

dependent variable was the students levels of CTS as measured by the Chinese version of

244 Y.-T. C. Yang


5/25

domain of the five cognitive skillsanalysis, evaluation, inference, deductive reasoning,

and inductive reasoning. One point is given for each correct answer, and the maximum

possible score is 34. In addition, the test has two parallel conceptually and statistically

equivalent forms that are suitable for a pretestposttest research design. The internal

consistency reliability (KuderRichardson 20) of the published version of Form A is KR-20 = .70, and of Form B is KR-20 = .71. As to the construct and content validity, the

CCTST is based on the conceptualization of CT articulated in the Expert Consensus

Statement on College Level Critical Thinking (Facione 1990b), known as the Delphi

report. This concept was supported by an independent replication research study of policy-

makers, employers, and academics, which was conducted at Penn State University and

sponsored by the US Department of Education. A sample question from CCTST, Form A

(Facione 1990a, 1992) is as follows:

Passage: The micro-organisms in this pond are of the kind which generally reproduce

only in water with a temperature above freezing point. Now its winter time and this pond

is solid ice. So if there are any micro-organisms of the kind we are researching in the pond,

they arent reproducing right now. Assuming all the supporting statements are true, the

conclusion of this passage

A = could not be false.

B = is probably true, but may be false.

C = is probably false, but may be true.

D = could not be true.

The content of the class discussion posted on the ADFs was analyzed qualitatively by using

the Coding Scheme for Evaluating Critical Thinking in Computer Conferencing, which isdiscussed in detail in the Data Analysis section.

Procedure

Training of instructor and teaching assistants (TAs)

There were one researcher, one instructor, and five-experienced TAs in this study. All of

the TAs were graduate students in electrical engineering. The researcher conducted the

training for the instructor and TAs to ensure that they were capable of teaching andmodeling Socratic dialogues during the online discussions. The training had four 2-h

sessions during which the researcher introduced the concept of CT, explained the rationale

for using it, and discussed and modeled the use of the Six Categories of Socratic Ques-

tioning Prompts (see Fig. 1), which are different but correlated categories of questioning

prompts to guide students in exploring ideas or statements in depth and breadth. The

categories included questions that (a) are for clarification, (b) probe assumptions, (c) probe

reasons and evidence, (d) are about viewpoints or perspectives, (e) probe implications and

consequences, and (f) are about the question.

The instructor and TAs practiced Socratic questioning prompts to guide the students in

exploring ideas or statements in depth and breadth during the pilot study (one semester long),

which was the same course as the main study, but one semester earlier than the main study.

http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


6/25

empirical evidence for a theory and the ability to integrate personal values with evidence).

The instructor closely monitored the interaction and offered pedagogical support for the TAs.

The preliminary results of the pilot study indicated that the instructor and TAs were

successfully trained to teach and model Socratic dialogues during the online discussions,

resulting in higher CT levels of students at the end of the pilot study.

Small-group structure

s

b. Questions That Probe

Assumptions

What are you assuming?

All of your reasoning depends

on the idea that ____. Why have

you based your reasoning on

____ rather than ____?

You seem to be assuming ____.

How would you justify taking

this for granted?

Is it always the case? Why do

you think the assumption holds

here?

c. Questions That Probe

Reasons and Evidence How did you come to believe

that?

Why did you say that?

How do you know?

What led you to that belief?

Why do you think that it is true?

Do you have any evidence for

that?

But is that good evidence to

believe that?

What are your reasons for

saying that?

Are these reasons adequate?

Is there reason to doubt that

evidence?

What other information do we

need?

What would change your mind?

Can someone else give evidence

to support that response?

By what reasoning did you come

to that conclusion?

How could we find out whether

that is true?

d. Questions about Viewpoints

or Perspectives

You seem to be approaching this

issue from ____ perspective.

Why have you chosen this ratherthan that perspective?

How could you answer the

objection that ____ would make?

Can/did anyone see this another

way?

What would someone who

disagrees say?

What is an alternative?

How are Kens and Marys ideas

alike? Different?

f. Questions about the

Question

Why is this question

important?

How can we find out?

What does this question

assume?

Is this the same issue as ____?

Is the question clear? Do we

understand it?

Do we all agree that this is the

question?

To answer this question, whatquestions would we have to

e. Questions That Probe

Implications and

Consequences

What are you implying by

that?

When you say ____, are you

implying ____?

But if that happened, what else

would happen as a result?

Why? What effect would that have?

If this and this are the case,

then what else must also be

true?

Categories

of SocraticQuestions

a. Questions of Clarification

What do you mean by ___?

What is your main point?

Could you give me an

example?

Could you explain thatfurther?

Why do you say that?

How does this relate to our

discussion (problem, issues)?

Fig. 1 Six categories of Socratic questioning prompts (summarized from Paul 1995, pp. 341344)

246 Y.-T. C. Yang
http://-/?-http://-/?-


7/25

research group) and five TAs in this study, with each TA facilitating two-group discussions

in the comparison group and another two in the experimental group.

Online discussions

At the beginning and end of the semester, the students were required to take the 45-min

CCTST in class (see Fig. 2). Two structured ADFs were separately set up for the comparison

and for the experimental groups. The instructor posted four specific discussion topics on both

ADFs for students to complete during the semester. Discussions 1 and 2 (O2 and O3) took

place during the first half of the semester, while Discussions 3 and 4 (O4 and O5) took place

during the second half of the semester. Of these four discussions, two were case studies

(Discussions 1 and 3), whose topics were home electrical appliances with anion function,

and electrical ground: why three prongs? respectively. The other two were debates (Dis-cussions 2 and 4), whose topics were Are electromagnetic waves/radiation that come from

the electrical appliances (e.g., cell phones, computers, microwave ovens, and hair dryers)

harmful to human health? and pros and cons of constructing a nuclear power plant,

respectively. Each discussion lasted 2 weeks. During the discussions, the TAs asked the

students to identify and post at least one argument or strong example to support their com-

ments about the discussion issue. Each student was also required to respond to at least one

other students posting by exploring the issue at hand and widening the discussion. Finally, at

the end of the discussion, the students were asked either to summarize the points that were

made during the discussion or to write a short reflection on the discussion. Student partici-

pation in the discussion counted for 20% of the total course grade in both research groups.

In the comparison group, the students could interact with each other and ask questions

of their peers, instructor, or TA on the structured ADF, but the TAs did not attempt to

facilitate the discussion.

In the experimental group, at the beginning of the semester, the instructor taught and

provided the students the Six Categories of Socratic Questioning Prompts (see Fig. 1). Both

the TAs and students adapted the prompts to generate thought-provoking questions by

filling in the blanks with specific content linked to the topic covered. Following are some

examples (with the course content underlined) of the Socratic questions that the TAs

modeled and asked so as to help students examine their thinking from a case study ofelectrical ground: why three prongs? and a debate of the pros and cons of constructing

a nuclear power plant

Comparison Group O1 O2 O3 O4 O5 O6

Experimental Group O1 X1 O2 X1 O3 X1 O4 X1O5 O6

Note:X1 = independent variable (the teaching and modeling of Socratic questioning in ADFs);O1 = scores on CCTST;O2, O3, O4, and O5 = CTS demonstrated in online discussions 1, 2, 3 and 4, respectively;O6 = scores on CCTST at the end of the semester.

First half of the semester Second half of the semester

Fig. 2 Pretest and posttest quasi-experimental design

http://-/?-http://-/?-http://-/?-http://-/?-


8/25

TA: Could you give us some examples of the disadvantages of nuclear power

generation?

(b. Questions that probe assumptions)

Student A: Because computers are more sophisticated than hair dryers, we usually see

grounds with computers more than with hair dryers.TA: You seem to be assuming that if the appliance is more sophisticated (e.g., computer

vs. hair dryer), the ground is more needed for that appliance . How would you justify

taking this for granted?

(c. Questions that probe reasons and evidence)

Student A: Recent surveys have indicated that there is currently more support for

construction of nuclear power plants than for halting construction.

TA: What is the source of your information? What other information do we need?

(d. Questions about viewpoints or perspectives)

Student A: We need to have grounds for computers or washing machines more than

for hair dryers because the former are more sophisticated than the latter.

TA: You seem to be approaching this issue from the sophistication of the appliance

perspective. Why have you chosen this rather than the prevention of electric shock

perspective?

(e. Questions that probe implications and consequences)

Student A: I would support a nuclear power plant. As demand for electricity soars, the

pollution produced from fossil fuel-burning plants is heading towards dangerous levels.

Unlike nuclear power plants, coal, gas, and oil burning power plants are already

responsible for half of the worlds air pollution. Burning coal produces carbon dioxide,

which depletes the protection of the ozone.TA: When you say Unlike nuclear power plants, air pollution., are you implying

that nuclear power plants have less or no pollution? Or that they cause a different kind

of pollution rather than air pollution? If so, what effect would that have?

(f. Questions about questions)

Student A: Im on the con side of the debate. Can you imagine a nuclear power plant

turning into a mushroom cloud? Images of the exploded reactor represent an almost

primal fear of technology out of control.

TA: Do we all agree that this is the main question (imagining the explosion of a nuclear

power plant or having a nuclear fear) for deciding whether we should construct a

nuclear plant or not? Why?

Throughout the online discussions, the TAs attempted to explicitly model Socratic

questioning during the ADF. Then the students practiced these questioning techniques by

composing comments, examining their own thinking, and challenging others in the

ADFs.

Data analysis

The quantitative data, the scores on the CCTST, were analyzed using two-way mixed

design ANOVA to identify whether there was a difference between the comparison and

248 Y.-T. C. Yang


9/25

Table 1 Coding scheme for evaluating critical thinking in computer conferencing

Part A: Interaction analysis model (IAM)

Phase I: Sharing/comparing of information

A. A statement of observation or opinion [IA]

B. A statement of agreement from one or more other participants [IB]

C. Corroborating examples provided by one or more participants [IC]

D. Asking and answering questions to clarify details of statements [ID]

E. Definition, description, or identification of a problem [IE]

Phase II: Discovery and exploration of dissonance or inconsistency among the ideas, concepts, or statements

advanced by different participants

A. Identifying and stating areas of disagreement [IIA]

B. Asking and answering questions to clarify the source and extent of disagreement [IIB]

C. Restating the participants position, and possibly advancing arguments or consideration in

its support by references to the participants experience, literature, formal data collected,or proposal of relevant metaphor or analogy to illustrate point of view

[IIC]

Phase III: Negotiation of meaning and/or co-construction of knowledge

A. Negotiation or clarification of the meaning of terms [IIIA]

B. Negotiation of the relative weight to be assigned to types of argument [IIIB]

C. Identification of areas of agreement or overlap among conflicting concepts [IIIC]

D. Proposal and negotiation of new statements embodying compromise and co-construction [IIID]

E. Proposal of integrating or accommodating metaphors or analogies [IIIE]

Phase IV: Testing and modification of proposed synthesis or co-construction

A. Testing the proposed synthesis against received fact as shared by the participants and/or

their culture

[IVA]

B. Testing against existing cognitive schema [IVB]

C. Testing against personal experience [IVC]

D. Testing against formal data collected [IVD]

E. Testing against contradictory testimony in the literature [IVE]

Phase V: Agreement, statement(s), and applications of the newly constructed meaning

A. Summarization of agreements [VA]

B. Applications of new knowledge [VB]

C. Metacognitive statements by the participants illustrating their understanding that their

knowledge or ways of thinking (cognitive schema) have changed as a result of theconference interaction

[VC]

Part B: Analysis model for analyzing depth of critical thinking

R+ Relevance

R+ Relevant statements to the issue discussed

R Totally irrelevant statements to the issue discussed

I+ Importance/significance

I+ Important/significant points/issues

I Totally unimportant, trivial points/issue

N+ Novelty

N+ Provide new information, ideas or solutions that have never been mentioned (even if they

are not important or useful)



10/25

analyzed using a chi-square test to investigate whether the students demonstrated CTS

from the online class discussions.

Coding scheme for evaluating critical thinking in computer conferencing

Constructivist learning theories are becoming widely accepted in all fields of education,

including the application of technology to teaching and learning. This interest is related to

the capacity of the computer to provide an interactive environment that creates an

effective means for implementing constructivist strategies that would be difficult to

accomplish in other media (Driscoll 1994, p. 376). Two widely accepted constructivist-

learning theories are critical constructivism and social constructivism (Kanuka and

Anderson 1998; Young 1997). While critical constructivism assumes that knowledge is

constructed as an integration of internal contradictions resulting from environmental

interactions, the assumption of social constructivism is that knowledge is generatedthrough social intercourse, and through this interaction, we gradually accumulate advances

in our levels of knowing. In this view, meanings emerge from the patterns of our social

experiences that occur over time in a contextual, situated, and continually changing

synthesis. Based on a constructivist paradigm, Gunawardena et al. (1997) developed the

Interaction Analysis Model (IAM) for the evaluation of the process of knowledge

construction that occurs through social negotiation in computer-mediated communication

(p. 400). This model is designed to detect evidence of knowledge construction and is

focused on assessing the quality of interactions in a CMC environment from both the

interactive and cognitive dimensions. As shown in Table 1 Part A, their model includesfive phases of interactions and 21 subcategories for use in the study of critical discussions

in case study and debate forums Gunawardena et al theorized that the active construction

Table 1 continued

A+ The references/literature used or information/data collected to support the participants

position are accurate and true

A The references/literature used or information/data collected to support the participants

position are clearly falseJ+ Justification

J+L+ Provide a logical statement of opinion, agreement or disagreement with supporting reasons/

examples/justifications/proof

J+L Provide an illogical statement of opinion, agreement or disagreement with supporting reasons/examples/justifications/proof

J Statement with simple agreement, disagreement or alternative opinions without elaboration

C+ Critical assessment

C+L+ Critical assessment/evaluation of ones own previous statements/reflection or others

contributions toward the issue discussed with logical thinking process

C+L Critical assessment/evaluation of ones own previous statements/reflection or otherscontributions toward the issue discussed with illogical thinking process

C Uncritical or unreasoned acceptance/reject

250 Y.-T. C. Yang
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


11/25

knowledge creation. Based on these phases, a model was developed that could be used to

analyze the construction of knowledge (constructivism) in computer conferencing tran-

scripts. These five phases are summarized as follows (Kanuka and Anderson 1998, p. 65):

1. Phase I. Sharing/comparing of information. In everyday transactions, this might take

the form of ordinary observations, statements of problems, or questions. This phase

may include an observation, opinion, agreement, corroborating example, clarification,

and/or identification of a problem.

2. Phase II. Discovery and exploration of dissonance or inconsistency among the ideas,

concepts, or statements advanced by different participants. This phase concerns

inconsistency between a new observation and the learners existing framework of

knowledge and thinking skills. Operations which may occur in this phase might

include identification of differences in understanding of terms, concepts, schemas, and/

or questions to clarify the extent of the disagreement.

3. Phase III. Negotiation of meaning and/or co-construction of knowledge. This phaseincludes negotiation or clarification of the meaning of terms, identification of areas of

agreement, and proposal of a compromise or co-construction.

4. Phase IV. Testing and modification of proposed synthesis or co-construction. Events

that occur in this phase include testing against an existing cognitive schema, personal

experience, formal data experimentation, or contradictory information from the

literature.

5. Phase V. Phrasing of agreement, statement(s), and applications of the newly

constructed meaning. This phase encompasses summarizing agreement(s) and

metacognitive statements that illustrate new knowledge construction and application.

The focus of IAM (1997) was to understand and describe the processes of negotiating the

meaning, knowledge co-construction, and conceptual change of student misconceptions in

a collaborative online discussion environment. All of such processes are important to

promote CT in online discussions (Marra et al. 2004; Yang et al. 2005). According to

Gunawardena et al. (1997), movement from one phase to the next shows that knowledge is

constructed by the process of social negotiation. The transcript analysis procedure involves

reading and coding the messages into one or more of the five phases.

Newman et al. (1995) provided a content analysis method to measure CT in computer-

mediated group learning, as shown in Table 1 Part B. To measure the frequencies of

specific critical (+) and non-critical () skills demonstrated in the discussions, theydeveloped a set of paired CT indicators (x+), such as R+ and N+, where x representsCT indicators such as relevance (R) and novelty (N). The symbol x+ represents the count

of positive statements of a CT indicator, while x is the count of negative statements in atranscript. Statements from the discussion transcripts were analyzed and scored across the

list of indicators. Once the scripts were marked, the total for each + or indicator wascounted, and a depth of CT ratio (x ratio) was calculated for each of the CT indica-

tors:x ratio x x = x x ; converting the counts to a 1 (all uncritical) to +1 (allcritical) scale. For example, 90 positive statements of relevance (R+) and 10 negative

statements of relevance (R) were found in one transcript. The CT ratio of relevance (xratio) is 0.80 as computed by: (90 10)/(90 + 10). This measure was designed to beindependent of the quantity of participation reflecting only the quality of the messages

http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


12/25

conferencing to promote deep learning, they did not attempt to measure the interactive

dimension taking place in the discussion.

Gunawardena et al. (1997) and Newman et al. (1995) both focused on the problem of

assessing the quality of online discussions. Although the latter provided focused and

segmented coding on certain potential indicators of CT, the former provided a moreholistic view of discussion flow and knowledge construction (Marra et al. 2004). Thus, the

tag codes with critical (+) and uncritical () valences used in Newman et al.s codingscheme (see Table 1, Part B) were incorporated into Gunawardena et al.s interaction

model (see Table 1, Part A) to form a new coding system called The Coding Scheme for

Evaluating Critical Thinking in Computer Conferencing for the current study (see

Table 1) (Yang et al. 2005). The combined coding system evaluates whether the students

discussion moves from the lower to the higher phases of CT, and whether the students

demonstrate more in-depth CTS within their online discussions.

Training session for establishing inter-rater reliability

Two ratersthe researcher and the instructor, followed the coding manual, detailing the

criteria and procedure for assigning a code for the different units. The raters reviewed and

negotiated the definitions of 21 CT categories and 14 different tags in the coding theme one

by one. The raters separately evaluated and assigned ratings to the first online discussion in

the pilot study. Using Miles and Hubermans (1994) inter-rater reliability formula (number

of agreements/(total number of agreements + disagreements)), the initial inter-rater reli-

ability coding had an 83.66% agreement. When differences occurred, two raters discussedthe discrepancies in the coding results until a consensus was reached. An independent

coding of the second discussion achieved a 91.35% inter-rater reliability. Based on this

rating, both the accuracy and reliability of using this coding instrument met the general

check-coding standard, which requires a 90% range (Miles and Huberman).

Coding procedure

One of the first tasks in the coding procedure was to parse the discussion transcripts into a

unit of analysis, that is, the portion of communication that would be used as the smallestunit to analyze. A unit of analysis was determined to be a phrase, sentence, paragraph, or

message, which illustrates any one of the indicators (see Table 1). The researcher and

instructor discussed, negotiated, and then parsed the discussion transcripts into units of

analysis together. Then they independently rated each unit across the Gunawardena et al .

(1997) category of interactions and all six of Newman et al.s (1995) criteria, if applicable.

The following examples are used to explain how each unit of the online postings was

coded. For example, if Student A started a new (N+) discussion (IA), which was relevant

(R+) and important (I+) to the discussion issue but the statement was inaccurate (A) andwithout supporting arguments (J

), the coding was [IA/R+/I+/N+/A

/J

]. Please note

that the indicator, critical assessment (C), was not suitable in this case because Student

A was the first person to post a message in the discussion forum; hence, no other students

252 Y.-T. C. Yang
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


13/25

with logical supporting arguments (J+L+). Therefore, it was coded [IIA/R+/I+/N+/A+/

J+L+/C+J+]. After the coding, we counted the total number of units of analysis that

occurred in each of the phases and subcategories according to the Coding Scheme for

Evaluating Critical Thinking in Computer Conferencing (see Table 1, Part A). In addi-

tion, similar to what Newman et al. (1995) had done, the total for each + or indicatorwas counted, and a depth of CT ratio (x ratio) was calculated for each criterion (see

Table 1, Part B). Both the researcher and the instructor coded all of the four online

discussions in the comparison and experimental groups and calculated the inter-rater

reliabilities for each discussion separately. Using Miles and Hubermans (1994) formula,

the inter-rater reliabilities for the eight coded online discussions ranged from 91.25% to

94.38%.

Results

California critical thinking skills test (CCTST)

A two-way mixed design ANOVA was performed to identify whether there was a dif-

ference between the comparison and experimental groups on the CCTST, and whether

there was a difference between the pretest and posttest of CCTST in each group. The mean

scores of CCTST (see Fig. 3) on the pretest were 16.79 (SD = 3.58) and 17.21 (SD = 3.36)

in the comparison and experimental groups, respectively. On the other hand, the mean

scores on the posttest of CCTST were 16.92 (SD = 3.84) and 18.23 (SD = 3.72) in the

comparison and experimental groups, respectively.The CCTST scores were then analyzed using a 2 2 mixed design ANOVA in

which the instructional group was a between-subjects factor, and occasion was a

within-subjects factor. The instructional group was divided into two levels (comparison

group and experimental group), and occasion was also divided into two levels (pretest

and posttest). The results indicate that both the main effects of instructional group,

F (1, 276) = 4.87, p = .03, g2 = .02, and occasion, F (1, 276) = 8.96, p < .001,

g2 = .03, and the interaction between them, F (1, 276) = 5.26, p = .02, g2 = .02, were

all significant.

Because the interaction effect was significant, follow-up comparisons of means wereconducted. Results revealed that the posttest mean (M = 18.23) in the experimental group

was significantly higher than the pretest mean (M = 17.21) in the experimental group

(F (1, 276) = 16.27, p < .001, g2 = .06), but there were no differences in the comparison

group (F (1, 276) = .21, p = .65, g2 =


14/25

(F(1, 552) = 1.01, p = .32, g2 =


15/25

Table 2 Results of asynchronous online discussions by interaction category for comparison group and

experimental group

Interaction

category

First half of the semester

(Discussions 1 and 2)

Second half of the semester


Total %

# of unit of analysis % # of unit of analysis %

Comparison group

Phase I IA 2,013 1,843

IB 124 130

IC 96 90

ID 198 206

IE 87 89

Sum 2,518 77.62 2,358 72.00 4,876 74.80

Phase II IIA 178 255IIB 77 93

IIC 5 7

Sum 260 8.01 355 10.84 615 9.43

Phase III IIIA 24 36

IIIB 2 7

IIIC 32 55

IIID 13 15

IIIE 2 3

Sum 73 2.25 116 3.54 189 2.90

Phase IV IVA 5 7

IVB 7 8

IVC 12 18

IVD 10 10

IVE 6 8

Sum 40 1.23 51 1.56 91 1.40

Phase V VA 25 27

VB 112 128

VC 216 240

Sum 353 10.88 395 12.06 748 11.47

Total 3,244 100.00 3,275 100.00 6,519 100.00

Experimental group

Phase I IA 2,894 2,203

IB 205 179

IC 86 90

ID 335 341

IE 86 88

Sum 3,606 62.91 2,901 48.63 6,507 55.63

Phase II IIA 384 465

IIB 340 597



16/25

Table 2 continued

Interaction

category

First half of the semester


Second half of the semester


Total %

# of unit of analysis % # of unit of analysis %

Phase III IIIA 116 145

IIIB 14 31

IIIC 156 296

IIID 74 115

IIIE 5 13

Sum 365 6.37 600 10.06 965 8.25

Phase IV IVA 23 42

IVB 21 26

IVC 56 84

IVD 35 42

IVE 15 21

Sum 150 2.62 215 3.60 365 3.12

Phase V VA 95 108

VB 41 72

VC 680 904

Sum 816 14.24 1,084 18.17 1,900 16.24

Total 5,732 100.00 5,965 100.00 11,697 100.00

77.62%

8.01%

2.25%1.23%

10.88%

72.00%

3.54% 1.56%

12.06%10.84%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Phase I Phase II Phase III Phase IV Phase V

comparison group (1st half)

comparison group (2nd half)

Fig. 4 Online discussion results by interaction categories for the comparison group

256 Y.-T. C. Yang


17/25

Table 3 Depth of critical thinking by indicator for comparison group and experimental group

Tag Code First half of the semester


Second half of the

semester (Discussions 3

and 4)

Total

Number

of codes

Depth of CT

ratio (%)

Number

of codes

Depth of CT

ratio (%)

Number

of codes

Depth of CT

ratio (%)

Comparison group

R (Relevance) R+ 1,932 85.77 2,043 89.87 3,975 87.85

R 148 109 257

I (importance) I+ 1,910 84.90 2,036 88.96 3,946 86.97

I 156 119 275

N (novelty) N+ 1,341 60.98 1,360 66.06 2,701 63.50

N 325 278 603

A (accuracy) A+ 1,245 62.75 1,250 66.89 2,495 64.80

A 285 248 533

J (justification) J+L+ 1,088 44.68 1,202 50.44 2,290 47.65

J+L 147 153 300

J 269 243 512

C (critical

assessment)

C+L+ 161 45.70 168 48.02 329 46.88

C+L 15 17 32

C 45 42 87

Total 9,067 9,268 18,335

Experimental group

R (Relevance) R+ 4,431 95.67 4,881 97.61 9,312 96.68

R 98 59 157

I (importance) I+ 3,612 94.98 3,631 97.66 7,243 96.31

I 93 43 136

N (novelty) N+ 2,343 90.02 2,256 96.09 4,599 92.95

N 123 45 168

A (accuracy) A+ 3,521 89.05 3,706 93.58 7,227 91.34

A 204 123 327

J (justification) J+L+ 3,229 85.90 3,354 93.76 6,583 89.82

J+L 27 23 50

J 218 85 303

C (critical

assessment)

C+L+ 724 82.83 748 90.09 1,472 86.45

C+L 20 18 38

C 48 21 69

Total 18,691 18,993 37,684

Note. Depth of CT ratio = (positive indicator negative indicator)/(positive indicator + negative indicator)



18/25

Experimental group

In the experimental group, 63, 14, 6, 3, and 14% of the analysis units were coded in Phases

IV, respectively, in the first half of the ADF discussion (see Table 2). In the second half,

51, 20, 10, 4, and 18% of the units were coded in Phases IV, respectively. Figure 6 shows

that the students discussion progressively and significantly moved from the lowermental functions (Phase I) to the higher mental functions (Phases IIV), v2(4,

n = 11,697) = 248.29, p < .001, / = .15.

Moreover, the depth of CT ratios (see Table 3 and Fig. 7) for each indicator in the

second half of the ADF was significantly higher (v2(1, n = 37,684) = 149.00, p < .001,

%19.26

%78.31

%73.6

%42.41

%35.91

%36.84

%60.01

%71.81

%00.01

%00.02

%00.03

%00.04

%00.05

%00.06

%00.07

%00.08

%00.09

%00.001

)flahts1(puorglatnemirepxe

)flahdn2(puorglatnemirepxe

85.77% 84.90%

60.98%62.75%

44.68%

89.87%

66.06%66.89%

50.44%

88.96%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Relevance Imoprtance Novelty Accuracy Justification

comparison group (1st half)

comparison group (2nd half)

Fig. 5 Online discussion results by CT indicators for the comparison group

258 Y.-T. C. Yang
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


19/25

/ = .06) than in the first half of the ADF: R (relevance) (96% vs. 98%), I (importance)

(95% vs. 98%), N (novelty) (90% vs. 96%), A (accuracy) (89% vs. 94%), J (justification)

(86% vs. 94%), and C (critical assessment) (93% vs. 90%). These results show that during

the second half of the ADF, the positive impact of the teaching and modeling of Socratic

questioning via the ADF increased over time in the experimental group.

Comparison group versus experimental group

When comparing the quality of interaction between the comparison and experimental

groups (see Fig. 8), a statistically significant difference was found, v2(4, n = 18,216) =

709.30, p < .001, / = .20. The results show that the instructional treatment was significantly

associated with the quality of interactions: students who participated in Socratic dialogues

had a higher quality of interaction than students who participated in the comparison group.

95.67% 94.98%

90.02% 89.05%

85.90%

96.31%96.68%

92.95%91.34%

89.82%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Relevance Imoprtance Novelty Accuracy Justification

experimental group (1st half)

experimental group (2nd half)

Fig. 7 Online discussion results by CT indicators for the experimental group

74.80%

9 43% 11.47%

55.63%

8 25%

16.24%16.76%20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%comparison group (total)

experimental group (total)

http://-/?-http://-/?-


20/25

In addition, as shown in Fig. 9, the analysis of the depth of CT indicated that both the

comparison and experimental groups had high R (relevance) (88% and 97%) and I(importance) (87% and 96%) ratios. The participants, in general, brought in relevant

materials and seemed to have adopted a serious style when taking part in the discussions.

There were notable differences for N (novelty) (64% vs. 93%), A (accuracy) (65% vs. 91%),

J (justification) (48% vs. 90%), and C (critical assessment) (47% vs. 86%). The chi-square

test of the relationship between the instructional treatment and the depth of CT demonstrated

that the structured ADF was significant, v2(1, n = 56,019) = 2,275.93, p < .001, / = .20.

Depth of overall CT in the experimental group was higher than that in the comparison group.

Discussion

In this study, we investigated the effects of teaching CT in a large class through ADF with

the facilitation of teaching assistants. The results of the data analyses are discussed in detail

in the following sections.

California critical thinking skills test (CCTST)

Results indicated that the comparison group did not significantly enhance their CTS levels

after actively participating in the online discussions. However, the students in the exper-

imental group significantly improved their levels of CTS and also outperformed the

comparison students. This outcome probably occurred because teaching and modeling of

%58.78 %79.68

%05.36%08.46

%56.74

%86.69

%43.19%28.98

%54.68

%59.29

%00.04

%00.05

%00.06

%00.07

%00.08

%00.09

%00.001

noitacifitsuJycaruccAytlevoNecnatrpomIecnaveleR

)latot(puorgnosirapmoc

)latot(puorglatnemirepxe

Fig. 9 Online discussion results by CT indicators for the two research groups

260 Y.-T. C. Yang
http://-/?-http://-/?-


21/25

is accomplished as students interact with their peers or instructor and benefit from

combining their levels of expertise, offering support, distributing the thinking load, and

confronting alternative points of view. The results indicate that learning is also accom-

plished as students elaborate, clarify, and justify their personal responses/solutions.

Online discussions

In the structured ADF experienced by both the comparison and experimental groups, the

students demonstrated very high levels of interaction, a social interaction that reduces

students reliance on the passive viewing mode of learning. Interpretations of the pat-

terns for each group follow separately.

Comparison group

Results show that the comparison students improved the quality of their online discussion

in the second half relative to the first half. Thus, even without the instructional intervention

(Socratic dialogues), the students also improved the quality of their online discussion in the

second half. These results disconfirm the findings of some previous studies (Davies and

Graff 2005; Sing and Khine 2006) in which greater online interaction did not lead to more

in-depth interaction or better learning performance. The students in the comparison group

seemed to ask and answer questions more critically to clarify ideas and negotiate meaning

as well as to identify areas of agreement or overlap among conflicting concepts. These

behaviors might have been fostered by students learning in the structured ADF how tonegotiate the meaning and co-construction of knowledge through the social learning

experience. Their classmates feedback provided beneficial information and questions,

which made them re-analyze some of their answers, eventually increasing the quality of the

discussion and their thoughts.

Although there were positive results in the online discussions when the students worked

as a team in the comparison group, the individual CTS of students did not improve as

reported in the CCTST section. This outcome implies that although the students improved

their CTS when the TAs and the students probed the meaning, justification, or logical

strength of other students statements in the group discussion, they did not improve their

CTS when working independently on their CCTST. Thus, without the external prompts,stimulus, or guidance of the instructor and TAs in teaching and modeling CTS, an inter-

active dialogue alone cannot effectively increase students CTS over a short period of time.

More time and practice might be needed for students to learn and practice such skills from

their interaction with classmates before these skills become self-monitoring and

independent.

Experimental group

When students participated in Socratic dialogues, as modeled and facilitated by the facil-

itator via ADFs during the first half of the semester, they learned how to use Socratic

questioning prompts to articulate and argue their positions in detail. Thus, students provided



22/25

experience of thinking critically, students seemed to think more logically and thoroughly

before posting their messages in the second half of ADF, resulting in a better quality of

critical discussion. This result suggests that the positive impact of the teaching and mod-

eling of Socratic questioning via the ADF increased over time in the experimental group.

Comparison group versus experimental group

When comparing the quality of students online discourse to reveal the process of CT

between two research groups, we found that the experimental group demonstrated sig-

nificantly higher phases of interaction and a higher depth of CT ratios than did the com-

parison group. In the experimental group, Socratic questioning was taught and modeled to

stimulate students minds by continually probing the subject with thought-provoking

questions. The students therefore seemed to think longer before posting their messages. As

a result, the number of negative statements for CT indicators (such as novelty, accuracy,

justification, and critical assessment) was reduced, which greatly increased the depth of CTratios for these indicators. These results suggest that instructional intervention (teaching

and modeling of Socratic questioning) should be employed during structured online

discussions to encourage learners to critically engage in learning issues.

Conclusions

Contributions of the study

Faced with todays ever-changing information technology and knowledge economy, testingthe validity of claims made by journalists, politicians, researchers, and other citizens is a

constant challenge. When citizens work together to solve public problems, a lot of effort

goes into understanding the problem and deciding what to do or believe about it. This type

of interaction requires excellent CTS. Thus, the importance of CT and the teaching of such

skills have been widely emphasized (Daud and Husin 2004; Ennis 1985; Yeh 2006).

However, relatively few researchers in the past decades have tried to teach CT in a large

university class. Because of the number of students in a large class, it is difficult to foster

CT using the traditional didactic approach to teaching. With the rapid development of

computer and network technologies, much research has revealed that structured ADFs have

tremendous potential to foster communities of inquiry and trigger a critical spirit

(Angeli et al. 2003; Duphorne and Gunawardena 2005; Sharma and Hannafin 2004).

Therefore, this study was designed to investigate an instructional strategy (i.e., teaching

CTS with the facilitation of TAs via ADFs) that can be used to overcome obstacles in the

development of CTS in a large class setting. The empirical results indicate that with an

instructor who teaches CTS at the beginning of the class and TAs who adopt Socratic

questioning prompts to model CTS during small online group discussions, students CTS

can be successfully developed in a large class environment. In addition, unless the

instructor and the TAs play a pedagogical role in teaching, modeling, and prompting

Socratic dialogues, an interactive dialogue alone cannot effectively help students becomeindependent critical thinkers over a short period of time. Finally, based on the results of

this study we strongly recommended that intellectual online discussions be designed

262 Y.-T. C. Yang
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


23/25

Limitations and suggestions for future research

There are several limitations that might have affected the outcome of this study. The first

limitation of this study is that the students levels of CTS were measured by the results

from the CCTST and their contributions to the online discussions. Future studies shouldinclude interviews to probe more in-depth opinions, comments, or attitudes on students

reactions to the teaching and modeling of Socratic Questioning via structured WBBs.

A second limitation is that the sample used in this study was a convenience rather than a

random sample. Still, the findings of this study can be transferable to similar contextsa

large undergraduate level general education course at a large university.

Third, CT is a mental process that seeks to clarify as well as evaluate the action and

activity that one encounters in life. In general, good critical thinkers should possess both

CTS and CT dispositions (CTD), which refer to the willingness, desire, and disposition to

base ones actions and beliefs on reasons (Siegel 1988). Facione (2007) mentioned that

CTS and CTD might be two separate things in people: Being skilled does not assure one is

disposed to use CT, and being disposed toward CT does not assure that one is skilled. This

study only investigated the mental processes but not the thinking dispositions (i.e., will-

ingness/inclinations). Additional research is needed to focus on how both to develop

students CTS and to nurture students consistent internal motivation to use these skills.

This effort will, thus, be a complete endeavor in cultivating students CT.

Acknowledgments The funding for this research was provided by the National Science Council of

Taiwan, ROC under Grant NSC92-2520-S006-001. We extend our special thanks to National Cheng Kung

University (NCKU) for supporting us through a free and reliable e-learning system to make this study a

success. The Academic Affairs Division of NCKU has selected this course as having the best design andimplementation of an e-learning course in 2004.

References

Angeli, C., Valanides, N., & Bonk, C. J. (2003). Communication in a web-based conferencing system: The

quality of computer-mediated interactions. British Journal of Educational Technology, 34(1), 3143.

Barnett, R. (2004). Learning for an unknown future. Higher Education Research and Development, 23(3),

247260.

Brown, A. L., & Palincsar, A. S. (1989). Guided, cooperative learning and individual knowledge acquisition.

In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 393451). Hillsdale, NJ: Erlbaum.

Bullen, M. (1998). Participation and critical thinking in online university distance education. Journal of

Distance Education, 13(2), 132.

Daud, N. M., & Husin, Z. (2004). Developing critical thinking skills in computer-aided extended reading

classes. British Journal of Educational Technology, 35(4), 477487.

Davies, J., & Graff, M. (2005). Performance in e-learning: Online participation and student grades. British

Journal of Educational Technology, 36(4), 657663.

Dillon, J. T. (1988). Questioning and teaching: A manual of practice. New York: Teachers College Press.

Driscoll, M. P. (1994). Psychology of learning for instruction. Toronto, ON: Allyn and Bacon.

Duffy, T. M., Dueber, B., & Hawley, C. (1998). Critical thinking in a distributed environment: A peda-

gogical base for the design of conferencing systems. In C. J. Bonk & K. S. King (Eds.), Electronic

collaborators: Learner-centered technologies for literacy, apprenticeship, and discourse (pp. 5178).Mahwah, NJ: Lawrence Erlbaum Associates.

Duphorne, P. L., & Gunawardena, C. N. (2005). The effect of three computer conferencing designs on



24/25

Facione, P. A. (1990a, 1992). The California critical thinking skills test (CCTST): Form A (1990) and form

B (1992). Millbrae, CA: California Academic Press.

Facione, P. A. (1990b). Critical thinking: A statement of expert consensus for purposes of educational

assessment and instruction. Research findings and recommendations. ERIC Document Reproduction

Service No. ED315423.

Facione, P. A. (2007). Critical thinking: What it is and why it counts. Millbrae, CA: California Academic Press.Flammer, A. (1981). Towards a theory of question asking. Psychological Research, 43, 407420.

Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer

conferencing in distance education. American Journal of Distance Education, 15(1), 723.

Gilbert, P. K., & Dabbagh, N. (2005). How to structure online discussions for meaningful discourse: A case

study. British Journal of Educational Technology, 36(1), 518.

Gunawardena, C., Lowe, C., & Anderson, T. (1997). Analysis of global online debate and the development

of an interaction analysis model for examining social construction of knowledge in computer con-

ferencing. Journal of Educational Computing Research, 17(4), 397431.

Harasim, L., Hiltz, S., Teles, L., & Turoff, M. (1995). Learning networks: A field guide to teaching and

learning online. Cambridge, MA: MIT Press.

Jeong, A. (2003). Sequential analysis of group interaction and critical thinking in online threaded discus-

sions. The American Journal of Distance Education, 17(1), 2543.King, A. (1990). Enhancing peer interaction and learning in classroom through reciprocal questioning.

American Educational Research Journal, 27(4), 664687.

Kanuka, H., & Anderson, T. (1998). Online social interchange, discord, and knowledge construction.

Journal of Distance Education, 13(1), 5774.

Maiorana, V. P. (19901991). The road from rote to critical thinking. Community Review, 11(12), 5363.

Marra, R., Moore, J., & Klimczak, A. (2004). Content analysis of online discussion forums: A comparative

analysis of protocols. Educational Technology Research and Development, 52(2), 2340.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand

Oaks: Sage.

Newman, D., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in

face-to-face and computer supported group learning. Interpersonal Computing and Technology: An

Electronic Journal for the 21st Century, 3(2), 5677.Paul, R. W. (1995). Critical thinking: How to prepare students for a rapidly changing world. Santa Rosa,

CA: Foundation for Critical Thinking.

Preston, J., & Shackelford, R. (1998). A system for improving distance and large-scale classes. ACM

SIGCSE Bulletin Inroads, 30(4), 193198.

Salmon, G. (2002). Mirror, mirror, on my screenExploring online reflections. British Journal of Edu-

cational Technology, 33(4), 379391.

Sharma, P., & Hannafin, M. (2004). Scaffolding critical thinking in an online course: An exploratory study.

Journal of Educational Computing Research, 31(2), 181208.

Siegel, H. (1988). Educating reason: Rationality, critical thinking, and education. New York: Routledge

Chapman & Hall.

Sing, C. C., & Khine, M. S. (2006). An analysis of interaction and participation patterns in online com-

munity. Educational Technology & Society, 9(1), 250261.Taba, H. (1966). Teaching strategies and cognitive functioning in elementary school children (Cooperative

Research Project No. 2404). San Francisco: San Francisco State College.

Walker, S. A. (2004). Socratic strategies and devils advocacy in synchronous CMC debate. Journal of

Computer Assisted Learning, 20, 172182.

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge

University Press.

Yang, Y.-T. C., Newby, T. J., & Bill, R. L. (2005). Using Socratic questioning to promote critical thinking

skills through asynchronous discussion forums in distance learning environments. The American

Journal of Distance Education, 19(3), 163181.

Yeh, Y. (2006). The interactive effects of personal traits and guided practices on preservice teachers

changes in personal teaching efficacy. British Journal of Educational Technology, 37(4), 513526.

Young, G. (1997).Adult development, therapy, and culture: A postmodern synthesis. New York: Plenum Press.

264 Y.-T. C. Yang


25/25

Documents

A Catalyst for Teaching Critical Thinking in a Large University Class in Taiwan Discussions Online With Teaching Assistant