Kim, C., Park, S. W., Cozart, J., & Lee, H. (2015). From Motivation to Engagement: The Role of Effort Regulation of Virtual

High School Students in Mathematics Courses. Educational Technology & Society, 18 (4), 261–272.

261 ISSN 1436-4522 (online) and 1176-3647 (print). This article of the Journal of Educational Technology & Society is available under Creative Commons CC-BY-ND-NC

3.0 license (https://creativecommons.org/licenses/by-nc-nd/3.0/). For further queries, please contact Journal Editors at ets-editors@ifets.info.

From Motivation to Engagement: The Role of Effort Regulation of Virtual High School Students in Mathematics Courses

ChanMin Kim1*, Seung Won Park2, Joe Cozart3 and Hyewon Lee1 1Learning, Design, and Technology, University of Georgia, Athens, Georgia, USA // 2Department of Medical

Education, Sungkyunkwan University, South Korea // 3Georgia Virtual Learning, Georgia Department of Education,

Atlanta, Georgia, USA // chanmin@uga.edu // parkswon@skku.edu // joe.cozart@gavirtualschool.org //

ehyewon@uga.edu *Corresponding author

(Submitted June 5, 2014; Revised December 10, 2014; Accepted May 3, 2015)

ABSTRACT Engagement and motivation are not one and the same, but motivation can be transformed into engagement with

proper design of support. In this study, we examined the differences between high performers and low

performers with regard to changes in their motivation, regulation, and engagement throughout the semester.

Participants were 100 students enrolled in online self-paced asynchronous mathematics courses offered at a

virtual high school in the United States. A survey was administered to participants at three times throughout the

semester. Data were analyzed using repeated measures MANOVAs. Overall, high performers and low

performers differed with regard to their changes in motivation and regulation throughout the course, specifically,

in self-efficacy and effort regulation. The study findings offer implications for teaching and research on creating

potentially effective support for virtual learning.

Keywords Virtual high school, Motivation, Regulation, Engagement, Mathematics education

Introduction

In recent years, online education has drastically increased, including at the K-12 level (Watson, Murin, Vashaw,

Gemin, & Rapp, 2011). The enrollment of K-12 school students in online courses continues to grow along with the

popularity of virtual schooling (Tucker, 2007). Every state in the United States and the District of Columbia has a K-

12 virtual school (Kennedy & Archambault, 2012). The rapid growth of virtual schooling has been attributed to

numerous factors, especially its perceived benefits such as provision of individualized instruction and broadening

educational access (Barbour & Reeves, 2009). The effectiveness of online and face-to-face education is now largely

considered equal, which may have added momentum to the growth of virtual schooling (Hughes, McLeod, Brown,

Maeda, & Choi, 2007). However, as with face-to-face schooling, high enrollment does not necessarily imply a high

success rate. Challenges in virtual schooling include low performance and high course dropout rates (Barbour &

Reeves, 2009).

Motivation is critical in learning. This is no less true in online learning (Carpenter & Cavanaugh, 2012). However,

motivated students do not always engage in learning (Keller, 2008). Motivation to learn is only a desire to be

involved in activities for learning (Kim & Bennekin, 2013). What makes students actually learn is their mindful

engagement in those learning activities because “engagement leads to outcomes such as achievement” and

“motivation underpins engagement” (Martin, 2012, p. 305).

There has been much research on motivation and engagement in a variety of face-to-face learning contexts (e.g.,

Junco, Elavsky, & Heiberger, 2013). However, what has been learned from such research may not apply to virtual

schooling because of the unique characteristics of online learning environments (Cho, Demei, & Laffey, 2010) such

as the lack of social presence, defined as “the degree of salience of the other person in the interaction and the

consequent salience of the interpersonal relationship” (Short, Williams, & Christie, 1976, p. 65). Social presence and

its lack have been researched in many studies to understand learning processes in online courses (e.g., Shea &

Bidjerano, 2010). Student motivation can be different depending on the quantity and quality of social presence

(Borup, Graham, & Davies, 2012; Shea & Bidjerano, 2010). This may apply even more to adolescents who tend to

heavily weigh the importance of peers (Berten, 2008). In fact, the K-12 online education literature highlights the role

of students’ interactions with their instructor and classmates (e.g., DiPietro, Ferdig, Black, & Preston, 2008). In

262

addition, mathematics educators and researchers have underscored social aspects of mathematics learning (Davydov

& Kerr, 1995).

In sum, there is a need to understand how students’ motivation and engagement influence their achievement in virtual

high school mathematics courses so that support can be planned and implemented accordingly. The aims of the

present study were to (a) explore and document how students’ motivation and engagement were related to their

mathematics achievement at a virtual high school and (b) determine what support is needed in order to improve their

motivation, engagement and achievement. This research can potentially provide a new lens through which to view

how motivation and engagement interrelate with student achievement in virtual schooling. In the following sections,

we discuss the definitions of engagement with an emphasis on its difference from motivation. We then discuss what

is needed to transform motivation into engagement. Our research question is then posed.

What is engagement?

There is no straightforward way of defining the construct of engagement. Rather, it may be reasonable to define

engagement as a multi-component construct comprised of subsets with associated indices. The engagement definition

of Fredricks, Blumenfeld, and Paris (2004) encompasses three kinds of engagement: behavioral, cognitive, and

emotional engagement. Behavioral engagement refers to involvement in learning tasks and environments such as

time-on-task and attendance; cognitive engagement refers to psychological investment in the process of learning

such as the use of learning strategies; and emotional engagement refers to affective reactions to learning tasks and

environments such as emotions (Fredricks et al., 2004). The multi-component approach to considering engagement

as a meta-construct can be conceptually and practically useful in research on and development of interventions to

improve student engagement (Fredricks et al., 2004). Such an approach can broaden understanding of engagement

(Finn & Zimmer, 2012; Fredricks et al., 2004; Lawson & Lawson, 2013). For example, if students’ emotional

experience is examined along with their off-task behaviors such as disrupting a peer (Skinner, Kindermann, & Furrer,

2008), one could better understand how to improve their engagement by providing relevant support for negative

emotions such as boredom.

In the present study, we define engagement as cognitive and affective participation in learning activities. We included

only cognitive engagement (i.e., using shallow and deep cognitive strategies; Pintrich, Smith, Garcia, & McKeachie,

1993) and emotional engagement (i.e., experiencing boredom, anxiety, enjoyment, anger, shame, pride, and

hopelessness; Pekrun, Goetz, & Frenzel, 2005) in our definition. We recognize that behavioral engagement is critical.

However, in asynchronous online education, there are no face-to-face or synchronous virtual classes to attend and

thus, the notion of behavioral engagement is not conceptually clear. For example, students’ login time does not

necessarily mean how many hours they studied. They may log in just to download course materials. In addition,

although Fredricks and her colleagues’ (2004) view of engagement as a meta-construct was applied in the present

study, we excluded motivation from cognitive engagement unlike their definition of cognitive engagement.

Engagement does not occur without desire to engage (Martin, 2012) but engagement and motivation are not one and

the same.

How can motivation be transformed into engagement?

Motivation and engagement do not always coexist. In other words, there could be motivation but without

engagement (e.g., only wanting something but not actually doing it). What transforms motivation to engagement is

the effort and metacognitive regulation that students put into the process of their learning (Pintrich et al., 1993).

Effort regulation is to control “one’s effort expenditure” (Halisch & Heckenhausen, 1977, p. 724). Metacognitive

regulation is to control “one’s own cognition” (Pintrich et al., 1993, p. 803). Effort regulation is part of resource

management (Pintrich et al., 1993). To display the role of the effort and metacognitive regulation in transforming

motivation to engagement, here is an example. Reviewing class notes over and over (i.e., rehearsal, one of the

cognitive strategies) is one way to engage in learning activities (Fredricks et al., 2004). This action of rehearsal (i.e.,

engagement) would not happen without the desire to learn (i.e., motivation); at the same time, the desire alone does

not guarantee engagement and the student should also make an effort to rehearse (i.e., effort regulation) and monitor

263

when to rehearse (i.e., metacognitive regulation). Managing both cognition (i.e., metacognitive regulation) and effort

(i.e., effort regulation) is important in learning (Pintrich et al., 1993) because it transforms motivation to engagement.

Such regulation happens more easily when students engage in the learning tasks that are (a) perceived easy to

execute and (b) interesting and enjoyable. Self-efficacy is defined as one’s perceived ability to successfully complete

a task (Bandura, 1977). Intrinsic task value is defined as the value one perceives in a task that is inherently

interesting and enjoyable (Schunk, Pintrich, & Meece, 2008). In many different learning environments, self-efficacy

has been steadily found to be a strong predictor for motivation and performance (e.g., Multon, Brown, & Lent,

1991). Self-efficacious students also tend to control their learning process (Bandura, 1977). Thus, when a task is

perceived to be easy to perform, students are likely to perceive high self-efficacy and to self-regulate. Self-efficacy

influences motivation directly and engagement indirectly (Schunk & Mullen, 2012). Students engage in tasks also for

their own interests (Ainley, 2012) and enjoyment (Csikszentmihalyi, 1988) when the intrinsic value of the tasks is

high (e.g., Deci & Ryan, 2008).

Not every student enjoys mathematics. Still, students can engage in learning tasks for which they do not perceive

high intrinsic value when there is no obstacle that they believe they cannot overcome. In other words, when students

have high expectancy of success (Wigfield & Eccles, 2000), motivation can be transformed into engagement.

However, not every task is easy. Especially in online mathematics courses, not only do many students not enjoy

math, but also they are not self-efficacious due to previous failure of math courses. Thus, such students often

experience negative emotions like anger in math classes (Kim, Park, & Cozart, 2014).

Research question

This study investigated how differently virtual high school students engage and achieve in mathematics courses and

what quality of theirs makes such differences. We addressed the following research question: How do high

performers and low performers differ with regard to their changes in motivation, regulation, and engagement

throughout the course? We compared such changes from the beginning of the semester to the middle of and the end

of the semester. In this study, motivation variables included self-efficacy and intrinsic value, regulation variables

included metacognitive regulation and effort regulation, and engagement variables included cognitive engagement

(i.e., using deep cognitive strategy use and shallow cognitive strategy use) and emotional engagement (i.e.,

experiencing boredom, anxiety, enjoyment, anger, shame, pride, and hopelessness). Table 1 summarizes each

construct and variable.

Table 1. Variable description (Operationalization of the constructs in this study)

Construct Construct definition Variable Variable description

Motivation

Desire to be involved with learning

activities/tasks

Self-efficacy Beliefs about own abilities to

complete learning tasks in a

certain circumstance

Intrinsic value Perception of the value of

learning tasks in relation to his or

her interest

Regulation Management of cognition and other

resources such as effort, emotions,

and environments

Metacognitive

regulation

Management of cognition in

learning activities

Effort regulation Management of effort in learning

activities in the face of difficulties

Engagement Cognitive and affective

participation in learning activities

Cognitive

engagement

Involvement with learning

activities using shallow and deep

cognitive strategies

Emotional

engagement

Emotional reactions, such as

boredom, anxiety, enjoyment,

anger, shame, pride, and

hopelessness, to learning

activities

264

Methods

Participants and setting

Participants were students enrolled in online self-paced asynchronous mathematics courses offered at a virtual high

school in the southeastern United States. The virtual high school is run by the State Department of Education.

Students who are enrolled in the virtual high school courses either take courses for an entire curriculum or

supplement courses that they take at their local school. A survey was administered to participants at three times

throughout the semester. One hundred participants who completed the survey all three times were included in the

study. The participants (n = 100) were from Math 1 (n = 13), Math 2 (n = 4), Math 3 (n = 9), Algebra (n = 31),

Geometry (n = 7), Pre-Calculus (n = 5), Calculus (n = 14), Statistics (n = 16), and Applied Math (i.e., Problem

Solving and Money Management) (n = 1) courses. The average age was 15.9. Sixty-eight out of 100 were female.

71% of the participants were Caucasian, 11% were Black/African American, 8% were Asian American, 3% were

Hispanic/Latino, and 7% were multiracial. Those who had no prior experience with online math courses (n = 82)

outnumbered those with experience.

Data collection

The Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich, & DeGroot, 1990) was used to measure

motivation, regulation, and cognitive engagement. Participants responded to each of the 40 items using a 7-point

Likert scale ranging from (1) “Not at all true of me” to (7) “Very true of me.” The reliability of scores on these sub-

scales of the MSLQ ranged from .52 to .93 (Pintrich, Smith, Garcia, & McKeachie, 1991) and validity of the items

were tested in a variety of school settings (e.g., Wolters & Pintrich, 1998). Some items were reworded to reflect the

online context of this study (e.g., the item “When reading I try to connect the things I am reading about with what I

already know” was revised to “when reviewing online course materials I try to connect the things I am reviewing

with what I already know”). Scale reliability coefficients with the reworded items ranged from .59 to .90 in a

previous study (Kim et al., 2014) and from .50 to .88 in the current study (see Table 2).

Table 2. Sample items and scale reliability

Measure Scale Sample item Scale reliability

(Cronbach’s α)

Motivation

Self-efficacy I am sure I can do an excellent job on the problems

and tasks assigned for this class. .88

Intrinsic value Even when I do poorly on a test I try to learn from

my mistakes. .88

Regulation

Metacognitive

regulation

Before I begin studying I think about the things I

will need to do to learn. .50

Effort regulation I work hard to get a good grade even when I don’t

like a class. .66

Engagement

Deep cognitive

strategy use

When reviewing online course materials I try to

connect the things I am reviewing with what I

already know.

.71

Shallow cognitive

strategy use

When I study for a test I try to remember as many

facts as I can. .73

Boredom Just thinking of my math homework assignments

makes me feel bored. .88

Anxiety I’m so scared of my math assignments that I would

rather not start them. .93

Enjoyment The material we deal with in mathematics is so

exciting that I really enjoy my class. .75

Anger I am so angry that I would like to throw my

homework into the trash. .86

Shame I feel ashamed when I realize that I lack ability. .83

265

Pride After having done my math homework, I am proud

of myself. .76

Hopelessness I would prefer to give up. .91

The Achievement Emotion Questionnaire in Mathematics (AEQ-M) (Pekrun, Goetz, & Frenzel, 2005) was used to

measure emotional engagement. Nineteen items were excluded from the current study because they were pertinent to

attending a physical classroom (e.g., “When I say something in my math class, I can tell that my face gets red.”).

Only the items asking about emotional experiences before, during, and after studying (18 items) and taking an exam

(23 items) were included. Participants responded to each of 41 items using a 5-point Likert scale ranging from (1)

“Strongly disagree” to (5) “Strongly agree.” Some items were reworded to reflect the online context of this study.

Internal consistency coefficients of scores on the various sub-scales of the AEQ-M ranged from .84 to .92 in a

previous study (Frenzel, Pekrun, & Goetz, 2007) and validity of the scores was tested in a variety of applications

(e.g., Frenzel, Thrash, Pekrun, & Goetz, 2007). Scale reliability coefficients with the reworded items ranged from .67

to .93 in a previous study (Kim et al., 2014) and from .75 to .93 in the current study (see Table 2).

Achievement was measured using students’ final grades. The possible range of the final grades was 0 – 100. Final

grades were determined using scores from asynchronous discussions, homework assignments, quizzes, tests, and the

final exam. There was no grade directly tied to attendance. Each course used a standard weighting system to

distribute grades across discussions, assignments, quizzes, tests, and exams.

Procedure

We recruited participants in the first and second weeks of the Fall 2011 semester. In the course website, we posted a

URL of a webpage containing an online survey that includes (a) the study description, (b) consent forms, (c)

demographic questions, and (d) 1st survey questions on motivation, regulation, and engagement. Students who

submitted signed parental consent and student assent forms proceeded to respond to demographic and 1st survey

questions. The same survey on motivation, regulation, and engagement was administered two more times throughout

the semester: one was in the middle of the semester and the other was toward the end of the semester. We collected

the final grade scores of the participants when the semester ended. Figure 1 illustrates the two groups, three

measurement points, and six variables of the study.

Motivation:

Self-efficacy

Intrinsic value

Regulation:

Metacognitive regulation

Effort regulation

Engagement:

Cognitive engagement

Emotional engagement

Low Performers’

High Performers’

Measured at Time 1

(beginning of the semester)

Measured at Time 2

(middle of the semester)

Measured at Time 3

(end of the Semester)

Figure 1. A summary of data collection

Data analyses

Four separate 3 (time) × 2 (group) MANOVAs were conducted with time (Measurement Points 1, 2, and 3) as a

repeated measure to investigate differences in changes in motivation, regulation, cognitive engagement, and

266

emotional engagement between the high-performer and low-performer groups. The participants were categorized

into high, middle, and low performer groups based on their final grade scores (M = 79.11, SD = 19.43). Because the

variability of the final scores was relatively high, we were concerned that grouping the participants based upon the

mean ± one standard deviation may not include students who scored high enough to be considered as high

performers. Thus, we categorized participants using the conventional letter grade assignment: participants with final

grade scores higher than 90 (equivalent to a letter grade A) were included in the high-performer group (M = 94.13, n

= 38) while participants with final grade scores lower than 80 (or, those who received a letter grade C or below) were

included in the low-performer group (M = 61.68, n = 40). The rest were regarded as the middle performers. For the

purpose of examining differences between high performers and low performers, the middle performers were

excluded in these analyses. Partial eta- squared (ηp2) was used to calculate effect size: Small: .01 ≤ηp2 < .06;

Medium: .06 ≤ηp2 < .14; Large: ηp2 ≥ .14).

Results

The results of repeated measures MANOVAs indicated that high performers and low performers differed with regard

to their changes in motivation, regulation, and engagement throughout the course, specifically, in self-efficacy (part

of motivation) and effort regulation (part of regulation). The descriptive statistics of all dependent variables

examined are presented in Table 3. The analysis results of the repeated measures are summarized in Table 4.

Table 3. Descriptive statistics

Low performer (n = 40) High performer (n = 38)

Measurement

time point 1 2 3 1 2 3

Self-efficacy a 41.40

(8.09)

37.70

(10.57)

34.20

(9.93)

46.97

(8.84)

46.36

(8.80)

46.18

(7.81)

Intrinsic value b 46.90

(8.14)

43.12

(9.65)

40.87

(9.91)

48.57

(7.80)

45.73

(10.19)

44.42

(8.94)

Effort regulation c 19.00

(3.63)

18.62

(4.18)

17.65

(3.98)

21.60

(3.71)

21.07

(3.58)

20.73

(3.87)

Meta. regulation d 22.97

(3.87)

22.30

(4.24)

21.72

(4.29)

23.28

(4.07)

22.60

(4.66)

22.21

(5.05)

Deep strategy e 40.57

(5.57)

40.90

(6.61)

38.45

(7.60)

42.36

(5.73)

40.89

(7.17)

40.28

(7.99)

Shallow strategy f 25.12

(5.07)

25.10

(6.09)

23.37

(6.61)

26.39

(4.87)

24.92

(5.55)

24.21

(6.24)

Boredom g 8.72

(3.94)

9.27

(4.00)

9.27

(3.72)

7.86

(4.04)

8.15

(3.83)

8.52

(4.26)

Anxiety h 35.80

(11.44)

34.40

(8.41)

37.12

(12.43)

31.26

(11.49)

32.71

(9.00)

29.63

(14.50)

Enjoyment i 17.05

(4.66)

17.20

(3.22)

15.67

(4.83)

18.26

(4.49)

17.00

(3.75)

17.63

(3.92)

Anger j 12.92

(5.06)

13.45

(4.60)

14.05

(5.84)

12.02

(6.27)

13.00

(5.27)

12.36

(6.28)

Shame k 14.00

(5.30)

12.82

(4.14)

14.65

(5.22)

10.36

(5.16)

12.10

(4.45)

11.26

(5.26)

Pride l 13.42

(3.72)

13.65

(2.96)

12.35

(3.51)

15.28

(3.57)

13.71

(2.70)

14.15

(3.72)

Hopelessness m 18.45

(7.05)

17.12

(6.96)

20.27

(7.04)

13.47

(7.07)

19.02

(7.54)

14.60

(8.02)

Notes. aPossible range of Self-Efficacy score: 9-63; bPossible range of Intrinsic Value score: 9-63; cPossible range of

Effort regulation: 4-28; dPossible range of Metcognitive Regulation: 5-35; ePossible range of Deep Cognitive

267

Strategy: 8-56; fPossible range of Shallow Cognitive Strategy: 5-35; gPossible range of Boredom score: 3-15; hPossible range of Anxiety score: 11-55; iPossible range of Enjoyment score: 6-30; jPossible range of Anger score: 5-

25; kPossible range of Shame score: 5-25; lPossible range of Pride score: 4-20; mPossible range of Hopelessness: 6-

30.

Table 4. Summary of univariate analyses of repeated measures

Group effect Time effect Time x Group effect

F P F P F P

Self-efficacy 22.74 .000 10.07 .000 6.32 .003

Intrinsic value 1.89 .173 26.57 .000 .843 .435

Effort regulation 12.52 .001 4.91 .010 N/A N/A

Meta. regulation .192 .662 3.45 .037 N/A N/A

Deep strategy N/A N/A 5.11 .008 N/A N/A

Shallow strategy N/A N/A 6.71 .002 N/A N/A

Boredom N/A N/A 1.610 .207 N/A N/A

Anxiety N/A N/A .011 .989 N/A N/A

Enjoyment N/A N/A 2.66 .077 N/A N/A

Anger N/A N/A 1.00 .371 N/A N/A

Shame N/A N/A 1.34 .268 N/A N/A

Pride N/A N/A 5.27 .007 N/A N/A

Hopelessness N/A N/A 3.03 .054 N/A N/A

Note. Significant effects are in bold.

The first 3 (time) × 2 (group) repeated measures MANOVA was conducted with two motivation variables: self-

efficacy and intrinsic value. One important assumption of a repeated measures MANOVA is the equality of

covariance. Results of Box’s Test of Equality Covariance Matrices yielded X2(21) = 36.76, p = .01, providing

evidence of a violation of the equal covariance assumption. Nevertheless, because the natural logs of covariance

matrices were found to be similar, we proceed with the usual MANOVA tests following Huberty and Olejnik’s

(2006) suggestion. Preliminary analyses were conducted to examine if there were any differences between two

groups at the beginning of the semester (e.g., Time 1), and we found a significant difference in self-efficacy (p < .01)

between two groups: high performers demonstrated higher self-efficacy than low performers at Time 1.

On the main analysis of the repeated measures MANOVA with two motivation variables, there were a significant

main effect of time, Wilks’ Lambda = .579, F(4, 73) = 13.23, p < .001, a main effect of group, Wilks’ Lambda = .759,

F(2, 75) = 11.86, p < .001, and a significant time x group interaction, Wilks’ Lambda = .847, F(4, 73) = 3.29, p < .05.

To further inspect the significant effects on the multivariate analysis, follow-up univariate analyses of repeated

measures were conducted for each motivation variable. Follow-up univariate analyses for self-efficacy yielded a

significant main effect of time, F(2, 75) = 10.07, p < .001, ηp2 = .21, a main effect of group, F(1, 76) = 22.74, p

< .001, ηp2 = .23, and a significant time x group interaction, F(2, 75) = 6.32, p < .01, ηp2 = .14. Further analyses

indicated that self-efficacy among the low-performer group gradually diminished from Time 1 to Time 3 (p < .001);

the self-efficacy of the high-performer group did not change over time. Last, follow-up univariate analyses for

intrinsic value yielded a significant main effect of time, F(2, 75) = 26.57, p < .001, ηp2 = .41, indicating that both

high- and low-performer groups reported a gradual decrease in intrinsic value over three measurement times.

The second 3 (time) × 2 (group) repeated measures MANOVA was conducted with two regulation variables:

metacognitive regulation and effort regulation. The equality of covariance matrices was upheld as indicated by X2(21)

= 30.65, p = .07. Preliminary analyses indicated a significant difference in effort regulation between two groups (p

< .05) at Time 1: high performers showed significantly higher effort regulation at Time 1 than low performers.

Results of the repeated measures MANOVA revealed a significant main effect of time, Wilks’ Lambda = .851, F(4,

73) = 3.17, p < .05, and a main effect of group, Wilks’ Lambda = .757, F(2, 75) = 12.02, p < .001. Follow-up

univariate analyses for effort regulation yielded a significant main effect of time, F(2, 75) = 4.91, p < .05, ηp2 = .11,

and a main effect of group, F(1, 76) = 12.52, p < .01. While high performers maintained superior effort regulation to

low performers throughout the semester, both groups demonstrated diminished effort regulation from Time 1 to Time

3 (p < .01, ηp2 = .11). Similarly, univariate analyses for metacognitive regulation indicated that both high and low

performers gradually reported lesser metacognitive regulation from Time 1 to Time 3 (p < .05, ηp2 = .08).

268

The third 3 (time) × 2 (group) repeated measures MANOVA was conducted with two cognitive engagement

variables: deep strategy use and shallow strategy use. Preliminary analyses indicated that high and low performers

demonstrated the similar level of both deep and shallow strategy use at Time 1. Given the equal covariance indicated

by X2(21) = 26.44, p = .18, a significant main effect of time was found from the repeated measures MANOVA, Wilks’

Lambda = .829, F(4, 73) = 3.75, p < .01. Follow-up univariate analyses for deep and shallow strategies also yielded a

significant time effect (p < .01, ηp2 = .12; p < .01, ηp2 = .15; respectively) indicating that both high and low

performers decreased their use of deep and shallow strategies over time.

The last 3 (time) × 2 (group) repeated measures MANOVA analysis was conducted with seven emotional

engagement variables: boredom, anxiety, enjoyment, anger, shame, pride, and hopelessness. Results of Box’s Test of

Equality Covariance Matrices provided the evidence of covariance equality. Preliminary analyses indicated that the

high and low performers differed in the level of shame (p < .01), pride (p < .05), and hopelessness (p < .01) in the

beginning of the semester. Two groups were not different in the levels of any other emotion variables at Time 1.

Results of the repeated measures MANOVA with emotional engagement variables indicated a significant time effect,

Wilks’ Lambda = .669, F(14, 63) = 2.23, p < .05. Conducting follow-up univariate analyses, pride was the only

variable that yielded the main effect of time, F(2, 75) = 5.27, p < .01, ηp2 = .12. Both high and low performers

diminished pride over time.

Discussion

Findings and interpretations

First, we found that high performers and low performers differed throughout the course: high performers started the

semester with the higher level of effort regulation than low performers and they maintained their superior level of

effort regulation to low performers’ throughout the semester. The higher the level of effort regulation that students

had, the higher their achievement was. This finding is aligned with that of Puzziferro’s study (2008) with community

college students enrolled in liberal arts online courses. Even when students’ perception of intrinsic task value was

low in the current study, those who reported greater effort regulation tended to perform better than those who

reported the lower level of effort regulation.

Given these findings, supporting students’ effort regulation may be one way to help them do better in an online

learning environment. Designing support for effort regulation could involve online instructors’ scaffolding for

student effort regulation that includes monitoring and guiding student efforts (Cho & Shen, 2013). Volition theories

and models can be helpful in creating such support as well since they explain how efforts can be better regulated

(e.g., implementation intentions in Gollwitzer & Sheeran, 2006; action control in Kuhl, 1985). Volition refers to “a

dynamic system of psychological control processes that protect concentration and directed effort in the face of

personal and/or environmental distractions” (Corno, 1993, p.14).

Second, we found that the metacognitive regulation of both high performers and low performers decreased

throughout the semester. This contradicts previous findings on the role of metacognitive regulation in online learning

(Artino, 2007; Cho & Shen, 2013). However, this finding along with discussions on effort regulation above suggests

that students’ effort regulation may have compensated for the impact of decreased metacognitive regulation on

achievement. This supports the notion that achievement depends not only on “cognitive control and regulation,

especially the different cognitive, metacognitive, and learning strategies that students may use to control their own

cognition and learning” but also on “how students control their own motivation, emotions, behavior (including

choice, effort, and persistence), and their environment” (Pintrich, 1999, p. 336). Although in the current study we did

not examine students’ regulation of other aspects such as emotions and environment, the inclusion of effort

regulation is an attempt to understand the path from student motivation to achievement. This attempt may be critical

especially in online learning environments where more qualities are expected than just knowing how to study (e.g.,

cognitive strategy use) (Kim & Bennekin, 2013).

Third, high performers started the semester with higher self-efficacy than low performers. Low performers’ self-

efficacy gradually diminished over time while there was no change in self-efficacy among high performers. The

indirect effect of self-efficacy on achievement has been well documented (e.g., Multon, Brown, & Lent, 1991) also

in the literature involving online learning contexts (e.g., Cho & Shen, 2013). It is conceivable that effort regulation

269

may have influenced self-efficacy (Komarraju & Nadler, 2013). The role of effort regulation as a mediator is pointed

out in some studies (Artino, 2007; Cho & Shen, 2013; Shea & Bidjerano, 2010). Thus, combined with the finding on

effort regulation, it seems that there could be other ways to promote self-efficacy than structuring learning

environments to provide vicarious experiences, autonomy, clear expectations, goal specificity, and balanced task

difficulty (Bandura, 1997; Jang, Reeve, & Deci, 2010; Locke & Latham, 2002). The effect of self-efficacy can be

improved through effort regulation.

Fourth, there was no difference between high performers and low performers in intrinsic value. This finding is

counter-intuitive and we can only speculate what has happened based on relevant literature. The motivation literature

describes that people tend to be persistent when they perceive intrinsic value in a certain task that satisfies their

interest (Ainley, 2012) and which they enjoy (Csikszentmihalyi, 1988). Such perceived intrinsic value enhances the

quality of motivation (Deci & Ryan, 2008) and provides momentum for participating in the task. With enjoyment,

full engagement can occur without even a conscious effort (e.g., flow experience; Csikszentmihalyi, 1988).

Nonetheless, without enjoyment and interest in a given task, people can be still engaged in a task and come to a

successful completion depending on regulatory styles (Ryan & Deci, 2000). Along this line of literature, our finding

on intrinsic value may have arisen as such: (a) the learning environment may have allowed students’ perceived

intrinsic value to fade away considering that both high and low performer groups showed gradual decreases in

intrinsic value throughout the semester and (b) even without enjoyment and genuine interest, students with effort

regulation could succeed considering high performers had superior effort regulation to low performers. Not every

student has the capability to “reshape tasks and to make them more palatable” in suboptimal learning contexts

(Corno & Kanfer, 1993, p. 302) and those without such a capability such as low performers in this study can be

educated about how to optimize contexts for themselves (e.g., exercising effort regulation) (Byman & Kansanen,

2008).

Last, both high and low performers’ pride and uses of deep and shallow strategies significantly diminished

throughout the semester. The use of shallow cognitive strategies should be better than nonuse but when shallow

cognitive strategies are used without deep cognitive strategies, learning tends to stay at a shallow level. Waning pride

may have been due to the decreased use of cognitive strategies and/or the lack of intrinsic value. However, resiliency

occurs when negative emotions serve as a warning for students with clear goals (Turner, & Schallert, 2001). The

steadily superior level of effort regulation that the high performers had may have allowed them to be resilient from

decreased pride and still be successful in the course.

Implications for research and practice

The findings offer implications for research on and teaching at virtual schools. Understanding how students’

motivation and engagement as well as regulation contribute to their learning provides information of how support

can be planned accordingly in virtual high school math courses. That is, comparing high and low performers’

changes in their motivation, regulation, and engagement provides direction for creating potentially effective support,

especially for student effort regulation, for online education in K-12 virtual schools. For example, support for

students’ effort regulation may help not only with a lack of motivation from not viewing the intrinsic value of

learning tasks but also with disengagement such as nonuse of cognitive strategies, which would in turn improve

achievement. This is a unique way of improving motivation, engagement, and achievement especially when every

learning environment cannot be optimal for every student (Kinshuk, Liu, & Graf, 2009). Improving learning through

effort regulation can also contribute to greater capacity for lifelong learning. Along this line, other qualities of

students could be studied also. For example, students’ beliefs about intelligence “valuing effort and hard work” could

be used to improve effort regulation (Komarraju & Nadler, 2013, p. 70). Also since self-efficacious students tend to

believe that their performance can be improved by exerting effort (Komarraju & Nadler, 2013), improving self-

efficacy can lead to improved effort regulation. Even when tasks are difficult, self-efficacious students tend to be

persistent (Komarraju & Nadler, 2013).

Limitations and suggestions for future research

There are several limitations in this study. First, mainly self-reported data were used. The social desirability issue

(Crowne & Marlowe, 1960) remains. Future research should consider individual or focus group interviews as well as

270

online behavioral observations using learning analytics and asynchronous communications such as emails. Second,

differences among courses in which participants were enrolled were not investigated due to the small sample size per

course. The study findings should be interpreted with caution especially due to these limitations that also make it

hard to generalize the study findings to other US virtual school contexts. A study with a larger sample size would

increase statistical power. Alternative sampling methods should be considered in future studies. Third, individual

differences among participants may have contributed to the difference in performance such as prior knowledge,

parental support, tutoring help, gender and socioeconomic status. Fourth, regulation of other resources such as

motivation, emotions, and environment (Pintrich, 1999) was not investigated in the current study. Last, social

presence was not empirically examined in this study to see if social presence actually lacks in the virtual learning

environment of this study.

References Ainley, M. (2012). Students’ interest and engagement in classroom activities. In S. L. Christenson, A. L. Reschly, & C. Wylie

(Eds.), Handbook of Research on Student Engagement (pp. 283–302). New York, NY: Springer US.

Artino, A. R. (2007). Self-regulated learning in online education: A Review of the empirical literature. International Journal of

Instructional Technology & Distance Learning, 4(6), 3-18.

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191–215.

Bandura, A. (1997). Self-efficacy: The Exercise of control. New York: Freeman.

Barbour, M. K., & Reeves, T. C. (2009). The Reality of virtual schools: A Review of the literature. Computers & Education,

52(2), 402-416.

Berten, H. (2008, August). Peer influences on risk behavior: A Network study of social influence among adolescents in Flemish

secondary schools. Paper presented at the Annual Meeting of the American Sociological Association Annual Meeting, Boston,

MA.

Borup, J., Graham, C. R., & Davies, R. S. (2012). The Nature of adolescent learner interaction in a virtual high school

setting. Journal of Computer Assisted Learning, 29(2), 153-167. doi: 10.1111/j.1365-2729.2012.00479.x

Byman, R., & Kansanen, P. (2008). Pedagogical thinking in a student’s mind: A Conceptual clarification on the basis of self-

determination and volition theories. Scandinavian Journal of Educational Research, 52(6), 603–621.

Carpenter, J. K., & Cavanaugh, C. (2012, April). An Exploratory study of the role of teaching experience in motivation and

academic achievement in a virtual ninth-grade English I course. Paper presented at the American Educational Research

Association (AERA) Annual Meeting, Vancouver, Canada.

Cho, M. H., Demei, S., & Laffey, J. (2010). Relationships between self-regulation and social experiences in asynchronous online

learning environments. Journal of Interactive Learning Research, 21(3), 297-316.

Cho, M. H., & Shen, D. (2013). Self-regulation in online learning. Distance Education, 34(3), 209-301.

Corno, L. (1993). The best-laid plans: Modern conceptions of volition and educational research. Educational Researcher, 22(2),

14–22. doi:10.3102/0013189X022002014

Corno, L., & Kanfer, R. (1993). The Role of volition in learning and performance. Review of Research in Education, 19, 301-341.

Crowne, D. P., & Marlowe, D. (1960). A New scale of social desirability independent of psychopathology. Journal of Consulting

Psychology, 24, 349–354.

Csikszentmihalyi, M. (1988). The Flow experience and its significance for human psychology. In M. Csikszentmihalyi & I. S.

Csikszentmihalyi (Eds.), Optimal experience (pp. 15–35). New York, NY: Cambridge University Press.

Davydov, V. V., & Kerr, S. T. (1995). The Influence of L. S. Vygotsky on education theory, research, and practice. Educational

Researcher, 24(3), 12–21.

Deci, E. L., & Ryan, R. M. (2008). Facilitating optimal motivation and psychological well-being across life’s domains. Canadian

Psychology, 49, 14-23.

DiPietro, M., Ferdig, R. E., Black, E. W., & Preston, M. (2008). Best practices in teaching K-12 online: Lessons learned from

Michigan Virtual School teachers. Journal of Interactive Online Learning, 7(1), 10-35. Retrieved June 8, 2012 from

http://www.ncolr.org/jiol/issues/PDF/7.1.2.pdf

Finn, J. D., & Zimmer, K. S. (2012). Student engagement: What is it? why does it matter?. In S. L. Christenson, A. L. Reschly, &

C. Wylie (Eds.), Handbook of research on student engagement (pp. 97–131). New York, NY: Springer US.

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of evidence. Review

of Educational Research, 74, 59-109.

271

Frenzel, A. C., Thrash, T. M., Pekrun, R., & Goetz, T. (2007). Achievement emotions in Germany and China: A Cross-cultural

validation of the Academic Emotions Questionnaire–Mathematics. Journal of Cross-Cultural Psychology, 38(3), 302-309.

Frenzel, A. C., Pekrun, R., & Goetz, T. (2007). Perceived learning environment and students’ emotional experiences: A Multilevel

analysis of mathematics classrooms. Learning and Instruction, 17(5), 478-493. doi:10.1016/j.learninstruc.2007.09.001

Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A Meta-analysis of effects and

processes. Advances in Experimental Social Psychology, 38, 69-119.

Halisch, F., & Heckhausen, H. (1977). Search for feedback information and effort regulation during task performance. Journal of

Personality and Social Psychology, 35(10), 724-733.

Huberty, C. J., & Olejnik, S. (2006). Applied MANOVA and discriminant analysis. Hoboken, NJ: John Wiley & Sons, Inc.

Hughes, J. E., McLeod, S., Brown, R., Maeda, Y., & Choi, J. (2007). Academic achievement and perceptions of the learning

environment in virtual and traditional secondary mathematics classrooms. American Journal of Distance Education, 21(4), 199-

214.

Jang, H., Reeve, J., & Deci, E. L. (2010). Engaging students in learning activities: It is not autonomy support or structure, but

autonomy support and structure. Journal of Educational Psychology, 102, 588-600.

Junco, R., Elavsky, C. M., & Heiberger, G. (2013). Putting twitter to the test: Assessing outcomes for student collaboration,

engagement and success. British Journal of Educational Technology, 44(2), 273–287. doi:10.1111/j.1467-8535.2012.01284.x

Keller, J. M. (2008). An Integrative theory of motivation, volition, and performance. Technology, Instruction, Cognition, and

Learning, 6, 79-104.

Kennedy, K., & Archambault, L. (2012). Offering preservice teachers field experiences in K-12 online learning: A national survey

of teacher education programs. Journal of Teacher Education, 63(3), 185–200.

Kim, C., & Bennekin, K. N. (2013). Design and implementation of volitional control support in mathematics courses. Educational

Technology Research & Development, 61(5), 793–817. doi:10.1007/s11423-013-9309-2

Kim, C., Park, S. W., & Cozart, J. (2014). Affective and motivational factors of learning in online mathematics courses. British

Journal of Educational Technology, 45(1), 171–185. doi:10.1111/j.1467-8535.2012.01382.x

Kinshuk, Liu, T. C., & Graf, S. (2009). Coping with mismatched courses: Students’ behaviour and performance in courses

mismatched to their learning styles. Educational Technology Research & Development, 57(6), 739–752.

Komarraju, M., & Nadler, D. (2013). Self-efficacy and academic achievement: Why do implicit beliefs, goals, and effort

regulation matter?. Learning and Individual Differences, 25, 67–72. doi:10.1016/j.lindif.2013.01.005

Kuhl, J. (1985). Volitional mediators of cognition-behavior consistency: Self-regulatory processes and action versus state

orientation. In J. Kuhl & J. Beckmann (Eds.), Action control: From cognition to behavior (pp. 101-128). doi:10.1007/978-3-642-

69746-3_6

Lawson, M. A., & Lawson, H. A. (2013). New conceptual frameworks for student engagement research, policy, and practice.

Review of Educational Research, 83(3), 432-479. doi:10.3102/0034654313480891

Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation. American

Psychologist, 57(9), 705–717. doi:10.1037/0003-066X.57.9.705

Martin, A. J. (2012). Part II Commentary: Motivation and engagement: Conceptual, operational, and empirical clarity. In S. L.

Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 303–311). New York, NY:

Springer US.

Multon, K. D., Brown, S. D., & Lent, R. W. (1991). Relation of self-efficacy beliefs to academic outcomes: A Meta-analytic

investigation. Journal of Counseling Psychology, 38(1), 30-38.

Pekrun, R., Goetz, T., & Frenzel, A. C. (2005). Achievement emotions questionnaire—mathematics (AEQ-M): User’s manual.

Unpublished manuscript, Department of Psychology, University of Munich, Munich, Germany.

Pintrich, P. R. (1999). Taking control of research on volitional control: Challenges for future theory and research. Learning and

Individual Differences, 11(3), 335-354.

Pintrich, R. R., & DeGroot, E. V. (1990). Motivational and self-regulated learning components of classroom academic

performance. Journal of Educational Psychology, 82, 33-40.

Pintrich, P. R., Smith, D., Garcia, T., & McKeachie, W. (1991). A manual for the use of the Motivated Strategies for Learning

Questionnaire (MSLQ). Ann Arbor, MI: The University of Michigan.

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated

Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801-803.

Puzziferro, M. (2008). Online technologies self-efficacy and self-regulated learning as predictors of final grade and satisfaction in

college-level online courses. The American Journal of Distance Education, 22, 72–89. doi:10.1080/08923640802039024

272

Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary

Educational Psychology, 25(1), 54–67.

Schunk, D. H., & Mullen, C. A. (2012). Self-efficacy as an engaged learner. In S. L. Christenson, A. L. Reschly, & C. Wylie

(Eds.), Handbook of research on student engagement (pp. 219–235). New York, NY: Springer US. doi:10.1007/978-1-4614-

2018-7_10

Schunk, D.H., Pintrich, P.R., & Meece, J. L. (2008). Motivation in education: Theory, research, and applications (3rd ed.).

Columbus, OH: Merrill.

Shea, P., & Bidjerano, T. (2010). Learning presence: Towards a theory of self-efficacy, self-regulation, and the development of a

communities of inquiry in online and blended learning environments. Computers & Education, 55, 1721–1731.

Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. New York, NY: John Wiley & Sons.

Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2008). A motivational perspective on engagement and disaffection:

Conceptualization and assessment of children’s behavioral and emotional participation in academic activities in the classroom.

Educational and Psychological Measurement, 69(3), 493-525. doi:10.1177/0013164408323233

Tucker, B. (2007). Laboratories of reform: Virtual high schools and innovation in public education. Education Sector Reports, 1-

19. Retrieved from http://jasonhuett.wiki.westga.edu/file/view/Virtual_schools.pdf/173365961/Virtual_schools.pdf

Turner, J. E., & Schallert, D. L. (2001). Expectancy–value relationships of shame reactions and shame resiliency. Journal of

Educational Psychology, 93(2), 320-329.

Watson, J., Murin, A., Vashaw, L., Gemin, B., & Rapp, C. (2011). Keeping pace with K-12 online learning: An annual review of

state-level policy and practice, 2011. Evergreen Education Group. Retrieved from http://files.eric.ed.gov/fulltext/ED535912.pdf

Wigfield, A., & Eccles, J. S. (2000). Expectancy–value theory of achievement motivation. Contemporary Educational

Psychology, 25, 68-81.

Wolters, C. A., & Pintrich, P. R. (1998). Contextual differences in student motivation and self-regulated learning in mathematics,

English, and social studies classrooms. Instructional Science, 26, 27-47.