ww.sciencedirect.com

j o u rn a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 4 ) 1e6

Available online at w

ScienceDirect

journal homepage: www.JournalofSurgicalResearch.com

Goal orientation in surgical residents: a study of themotivation behind learning

Rebecca L. Hoffman, MD,a,* Cristina Hudak-Rosander, MS,b

Jashodeep Datta, MD,a Jon B. Morris, MD,a and Rachel R. Kelz, MD, MSCEa

aDepartment of Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PennsylvaniabDepartment of Psychology, University of Utah, Salt Lake City, Utah

a r t i c l e i n f o

Article history:

Received 29 October 2013

Received in revised form

31 December 2013

Accepted 6 January 2014

Available online xxx

Keywords:

Goal orientation

Resident

Surgery

Education

Satisfaction

Burnout

Professionalism

* Corresponding author. Department of SurgeStreet, Philadelphia, PA 19104. Tel.: þ1 267 2

E-mail address: rebecca.hoffman@uphs.u0022-4804/$ e see front matter ª 2014 Elsevhttp://dx.doi.org/10.1016/j.jss.2014.01.005

a b s t r a c t

Background: The subconscious way in which an individual approaches learning, goal

orientation (GO), has been shown to influence job satisfaction, job performance, and

burnout in nonmedical cohorts. The aim of this study was to adapt and validate an in-

strument to assess GO in surgical residents, so that in the future, we can better understand

how differences in motivation affect professional development.

Materials and methods: Residents were recruited to complete a 17-item survey adapted from

the Patterns of Adaptive Learning Scales (PALS). The survey included three scales assessing

GO in residency-specific terms. Items were scored on a 5-point Likert scale, and the psy-

chometric properties of the adapted and original PALS were compared.

Results: Ninety-five percent of residents (61/64) participated. Median age was 30 y and 33%

were female. Mean (standard deviation) scale scores for the adapted PALS were: mastery

4.30 (0.48), performance approach (PAP) 3.17 (0.99), and performance avoid 2.75 (0.88). Mean

(standard deviation) scale scores for the original PALS items were: mastery 3.35 (1.02), PAP

2.76 (1.15), and performance avoid 2.41 (0.91). Cronbach alpha were a ¼ 0.89 and a ¼ 0.84 for

the adapted PAP and avoid scales, respectively, which were comparable with the original

scales. For the adapted mastery scale, a ¼ 0.54. Exploratory factor analysis revealed five

factors, and factor loadings for individual mastery items did not load consistently onto a

single factor.

Conclusions: This study represents the first steps in the development of a novel tool to

measure GO among surgical residents. Understanding motivational psychology in resi-

dents may facilitate improved education and professional development.

ª 2014 Elsevier Inc. All rights reserved.

1. Introduction attain competence; performance approach (PAP) orientation,

The subconscious way in which an individual approaches a

learning task, known as goal orientation (GO), has been shown

to influence job satisfaction, job performance, and burnout

in nonmedical cohorts [1e3]. There are three subtypes of

GO: mastery orientation (MG), in which learners strive to

ry, Perelman School of M75 3290; fax: þ1 215 662 7penn.edu (R.L. Hoffman).ier Inc. All rights reserved

in which learners strive to demonstrate competence; and

performance avoid (PAV) orientation, in which learners strive

to avoid demonstrating incompetence. For the PAV-oriented

person, the prospect of potential failure is likely to elicit

anxiety, encourage self-protective withdrawal, and disrupt

concentration and task involvement [4,5]. In addition,

edicine at the University of Pennsylvania, 4 Maloney 3400 Spruce983.

.

j o u r n a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 4 ) 1e62

neuroticism and emotional instability have been negatively

correlated with performance orientations [6].

GO can be used to understand and predict how people will

respond when faced with concrete achievement situations.

Because surgical residents face such situations daily, whether

in the operating room, taking a written or oral examination, or

engaging in feedback, studying GO may be particularly useful

in this population. For example, themastery-oriented resident

may view technical ability as something that can be improved

with effort and persistence, believe that effort leads to suc-

cess, and view feedback as useful information about how to

correct errors and develop competency. The performance-

oriented resident, however, may believe that the ability is

fixed and exert a tremendous amount of effort to compensate

for low ability. This resident may view specific negative

feedback as an overall judgment of one’s competency, which

can be particularly devastating and nonproductive [7].

Although both a mastery- and performance-oriented resi-

dent may exhibit competency, the notable difference may be

in the deleterious long-term effects on an individual’s

persistence, or lack thereof, in the face of failure and task

choice [4]. Further emphasizing the importance of the GO

construct, studies in both airline pilots and surgeons have

found that self-regulation and self-monitoring during com-

plex decision making rely on an understanding of one’s own

internal emotions juxtaposed with external environmental

factors [8,9]. Therefore, the ability of a resident to understand

his or her own GO might enhance their development of more

safe, patient oriented, and educational operative experiences.

Similarly, an increased awareness of and sensitivity to

different types of motivational behavior in the learning envi-

ronment by medical educators may help to enhance the

development of leadership, team building, self-assessment,

and job performance skills [10].

With the understanding that GO may be an important

construct to study among surgical residents, it is important to

identify a reliable and validmethod of assessing this construct

in medical cohorts. Relying on behavioral observations alone

is not sufficient, as both amastery- and performance-oriented

resident may exert the same degree of engagement to prove

task competency. In addition, individuals may manifest

various combinations of GO depending on the situation.

Because of this, the best way to study GO is to use self-report

measures. Although the measures of GO exist, they have not

yet been adapted and validated for use in medical cohorts.

To facilitate future study of the potential importance of GO

among surgical residents, we sought to adapt and validate a

well-studied GO instrument for use in the surgical resident

population to begin to understand the subconscious way in

which surgical residents approach tasks associated with

professional development.

2. Methods

2.1. Study population and setting

This study was deemed exempt from ongoing review by the

Institutional Review Board of the University of Pennsylvania.

After 2013American Board of Surgery In-Training Examination,

64 residents enrolled in a university-based surgical residency

programwere recruited to complete a 17-item survey designed

to test GO. The study population included both categorical and

preliminary surgical residents who were representative of 11

countries and 16 different states. Participants reviewed a

written study description, and inclusion was voluntary. Resi-

dents were made aware that their participation would not

affect their status within the program.

2.2. Educational setting

The sponsor hospital is a large (772-bed) urban, university-

based hospital with approximately 37,000 admissions annu-

ally, 1135 residents and fellows and 182 surgical faculty.

Educational conferences include twice weekly departmental

faculty-led didactics (one of which is a SCORE-based confer-

ence) [11], weekly morbidity and mortality conference,

monthly grand rounds and supplemental service-based con-

ferences. All residents (100%) complete a research fellowship

after the third clinical year.

2.3. Scale development

The Patterns of Adaptive Learning Scales (PALS) was devel-

oped by educational psychologists to examine the relation-

ship between the learning environment and students’

motivation, affect and behavior among school-aged children

[12]. Our survey instrument was adapted for use in graduate

surgical education using a subset of the PALS. The adapted

PALS included three scales assessing students’ personal

achievement GO (mastery, PAP, and PAV GO) in residency-

specific terms (Table 1). For instance, when the original PALS

referenced “class work,” “students,” and “teachers,” the

adapted PALS was translated into “situations,” “residents,”

and “attendings.”

2.4. Data collection

Study data were collected and managed using the Research

Electronic Data Capture tool hosted at the University of

Pennsylvania. Research Electronic Data Capture is a secure,

web-based application designed to support data capture for

research studies, providing: (1) an intuitive interface for

validated data entry; (2) audit trails for tracking data manip-

ulation and export procedures; (3) automated export pro-

cedures for seamless data downloads to common statistical

packages; and (4) procedures for importing data from external

sources [13].

2.5. Analytic strategy

Items were scored on a 5-point Likert scale (1 ¼ not at all true,

3 ¼ somewhat true, and 5 ¼ very true), and scale scores were

determined by averaging the scores for each item (Table 1). To

determine if the adapted PALS was a valid measure of GO

among surgical residents, the psychometric properties of the

adapted PALS were compared with those of the original PALS

using exploratory factor analysis and calculating Cronbach

alpha for each of the three subscales.

Table 1 eAdapted PALS GO questions showingmode, mean item score, overall scale scores, and values of Cronbach alphaaccording to type of GO compared with the original PALS (n [ 61).

PALS questions Mode Meanitemscore

AdaptedPALSscalescore

OriginalPALSscalescore

AdaptedPALS

Cronbachalpha

OriginalPALS

Cronbachalpha

MG oriented 4.30 3.35 0.54 0.86

1. I like situations that I will learn from even if I make mistakes 5 4.03

2. An important reason why I work so hard is because I enjoy it 5 4.07

3. An important reason why I work hard is to learn new things 5 4.39

4. I work so hard because I am interested in it 4,5 4.28

5. I like patient situations best when they really make me think 5 4.32

6. An important reason why I work so hard is because I want to

get better at my job

5 4.70

PAP oriented 3.17 2.76 0.89 0.86

7. I want to do better than other residents in my program 5 4.13

8. I would feel successful if I did better than most other residents 3 3.18

9. I would feel really good if I were the only resident in my class

who knew how to perform a task

1 2.75

10. Doing better than other residents is important to me 2 2.84

11. I would like to show my attending that I am more competent

than other residents in my program

3 2.95

PAV oriented 2.75 2.41 0.84 0.75

12. One reason I would not volunteer to do a new case is to avoid

looking stupid

1 1.95

13. The reason that I work so hard is so my attending does not

think I know less than other residents

3 2.51

14. An important reason why I work so hard is so that I do not

embarrass myself

4 3.11

15. The reason why I work so hard is so others won’t think Iam

incompetent/unintelligent

2 2.69

16. One of my main goals is to avoid looking like I can not do my

work

3 2.70

17. It is very important to me that I do not look incompetent

compared with my peers

4 3.51

MG ¼ mastery goal oriented.

j o u rn a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 4 ) 1e6 3

Exploratory factor analysis is a statistical tool that examines

patterns of correlations among questionnaire items to identify

and validate the factors, or subdomains, that are represented

by a set of items. Significant factors are identified by examining

eigenvalues, which are a reflection of the amount of variance

accounted for by each item. The sum of all the eigenvalues

equals the number of items in the instrument, and only factors

with a value>1 are retained by convention (because a value<1

indicates that a component accounts for less variance than a

single item). For the present study, the number of factors

emerging with eigenvalues >1 represents the number of sub-

types of GO represented within the questionnaire.

In this analysis, and by convention, factors were extracted

and then obliquely rotated to yield the simplest structure,

whereby each item loads cleanly onto as few factors as

possible (preferably only 1). Loading refers to the relative

weight with which each item correlates with a particular

factor. Loadings are consideredmeaningful if they exceed 0.40

or �0.40. Items, which fail to load substantially onto any one

factor may be removed from the analysis [14].

Cronbach alpha is a measure of internal consistency used

to convey the reliability of a psychometric test. It is used most

commonly as an adjunct to factor analysis to indicate the

degree to which a set of items measures a construct. In this

case, Cronbach alpha reflects the extent to which participants

answered consistently within the three different types of GO.

A Cronbach alpha of >0.9 is considered to have excellent

reliability, an a > 0.8 is considered good and a > 0.7 is

considered acceptable [15]. All data analysis was conducted

using STATA version 12.1 (StataCorp LP, College Station, TX.)

3. Results

Sixty-one residents (95%) completed the survey. The median

resident age was 30 y (interquartile range 28, 31), and 20 re-

spondents (33%) were female. Twenty-one residents (34%)

were in the first postgraduate year (PGY), 12 (20%) were PGY2,

and the remainder (46%) were PGY3-7. Informal, unsolicited

feedback from participants suggested that the survey content

was both novel and thought provoking. The mean (standard

deviation) scale scores for the adapted PALS were 4.30 (0.48)

for mastery items, 3.17 (0.99) for PAP items, and 2.75 (0.88) for

PAV items. The mean scale scores for the original PALS items

were 3.35 (1.02) for MG, 2.76 (1.15) for PAP, and 2.41 (0.91) for

PAV items.

Values of Cronbach alpha demonstrated good internal

consistency for the adapted PAP and PAV scales (a ¼ 0.89 and

a ¼ 0.84, respectively), which were comparable withmeasures

of internal consistency in the original scales (a ¼ 0.86 and

Table 3 e Summary of results.

� The adapted PALS survey was designed to evaluate three

domains of goal orientation: mastery, PAP, and PAV.

� The adapted PALS survey demonstrated good internal

consistency for the PAP and PAV orientations.

� Additional work is needed to refine items used to test

mastery goal orientation in the surgical resident population.

� In its current form, the adapted PALS can be used to inform

residents about their goal orientation.

j o u r n a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 4 ) 1e64

a ¼ 0.75, respectively). For the adapted MG scale, the value of

Cronbach alpha suggested poor internal consistency (a¼ 0.54),

in contrast to the original PALS MG scale (a ¼ 0.86).

Results of an exploratory factor analysis revealed five fac-

tors with Eigenvalues >1, in contrast to the three factors that

were represented in the original PALS. Examination of factor

loadings for individual items suggests that the MG scale items

did not consistently load on to a single factor. When items

were removed from the factor analysis which loaded onto

multiple factors or did not load onto the original three tested

factors, the overall value of Cronbach alpha did not change

dramatically (0.814 versus 0.819), and items meant to load

together in fact loaded together under the same factor

(Table 2). These final three factors accounted for 77% of the

total variance in the data. The two mastery-oriented ques-

tions of six that remained had good internal consistency

(a ¼ 0.87). All of the PAP items and three of six PAV remained

in the model and demonstrated a Cronbach alpha of a ¼ 0.79,

which is comparable with the original PALS survey (0.79). See

Table 3 for a summary of the results.

4. Discussion

Motivational theory is a novel construct in medical education

with practical applications in surgical education. The ability of

a program to understand its residents’ GOs could improve the

value of feedback for individual residents, facilitate appro-

priate goal setting on an individual basis, optimize the

educational value of the experience, and reduce burnout and

attrition. This study represents the first step toward the

development of an innovative tool to describe and understand

GO among surgical residents. In this study using factor anal-

ysis, we were able to successfully adapt and validate two of

three scales of GO (PAP and PAV) that comprise the PALS in-

strument for use in a population of surgical residents in a

Table 2 e Factor loadings in exploratory factor analysis.

Mastery goal oriented

An important reason why I work so hard is because

I enjoy it

0.94

I work so hard because I am interested in it 0.95

PAP oriented

I want to do better than other residents in my program 0.88

I would feel successful if I did better than most

other residents

0.75

I would feel really good if I were the only resident in my

class who knew how to perform a task

0.77

Doing better than other residents is important to me 0.82

I would like to show my attending that I am more

competent than other residents in my program

0.84

PAV oriented

An important reason why I work so hard is so that I do

not embarrass myself

0.89

The reason why I work so hard is so others won’t think

I am incompetent/unintelligent

0.82

It is very important to me that I do not look incompetent

compared with my peers

0.53*

* Also loaded weakly with PAP (0.41).

university-based training program. In addition, we found that

the adapted PALS instrument required minimal time to com-

plete and was of substantial interest to surgical residents.

Although the original PALS has been used to assess the GO of

school-age and college-age students, other methods of

assessing GO have been used in graduate school or other adult

populations [16e18]. In preparation for this study, we chose

the PALS because of its easily adaptable scale and brevity,

which made it appropriate for use in the field of surgical ed-

ucation. The low internal consistency and loading of the

mastery items onto additional factors in the PALS adaptation

suggests that the mastery scale needs additional modification

before wide-scale administration. It is likely that the sophis-

ticated nature of our study population, in terms of their life-

time educational experience, demands more detail in the

survey items to capture the mastery component of GO.

One of the important considerations in survey develop-

ment is the appropriateness of the measure to the population

being tested. Because surgical residents represent both a

unique student and professional population with substantial

demands on their time, surveys need to be succinct, and the

validity of their adaptation ascertained first in a pilot popu-

lation. In our adaptation, we have shown that, with the

exclusion of seven items from the original survey, we were

able to retain the three domains of GO in our adapted in-

strument and were able to preserve the overall validity of the

scale.

The Accreditation Council for Graduate Medical Education

Outcome Project in 1999 changed the accreditation process

from the assessment of a program’s potential to educate to the

evaluation of its actual accomplishments through an exami-

nation of program outcomes. The new competency-based

education is now centered on learner performance in reach-

ing specific objectives [19,20]. Interestingly, little effort has

focused on understanding the psychology of the adult learner

and their individual approaches to meet these new

competency-based objectives. Zweig and Webster [6] very

aptly note that with so many resources spent on developing

and delivering training content, little research goes into the

role trainees themselves play as recipients of this training.

With this innovative adaptation of the PALS for the surgical

resident population, we were able to identify at least three

motivational constructsdmastery, PAP, and PAV. That is to

say, surgical residents are not simplymotivated bymastery of

surgical skill in the traditional sense of the word. The moti-

vation for excellence in their chosen profession is complex,

and includes, in addition to the pure desire to attain and

demonstrate knowledge (mastery orientation), the somewhat

impure desire to outperform their peers and to avoid looking

j o u rn a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 4 ) 1e6 5

incompetent (performance orientation). In fact, of the three

constructs tested, it was the PAP and PAV GOs, which

demonstrated the highest internal consistency. This finding is

consistent with previous research on intraoperative decision

making, where surgeons acknowledged that competing pri-

orities related to specific sociocultural norms (the constant

need to demonstrate success and avoid looking incompetent)

versus the responsibility for patient safety and the provision of

quality care influenced their care [8,9].

Latham and Brown showed that the purposeful and indi-

vidualized creation of learning goals increased the satisfac-

tion of MBA students with their respective programs [21].

Furthermore, given current discourse regarding trainees

commitment to patients in the era of work hour reform, an

awareness of GO may help residents to acknowledge the

competing priorities that surgeons face during their careers

(of dedication to patients versus dedication to demonstrating

their identity as a “successful” surgeon) and to develop into

true professionals, aware of how what motivates them im-

pacts on patient care [8,9].

This pilot study has several limitations. Although the

response rate (95%) was excellent (low nonresponder bias), this

survey was performed with residents at a single institution,

somewhat limiting generalizability. As the survey was admin-

istered after the American Board of Surgery In-Training Exam-

ination, test fatigue, anxiety, and mood may have influenced

survey responses resulting inunmeasuredbias. However, given

the completeness of data collection and exceptional response

rate, test fatigue is unlikely to have influenced the study find-

ings. Moreover, as the purpose of the study was to validate the

instrument, the direction of any potential bias on the individual

items would be unlikely to alter the validity of the study find-

ings. Furthermore, capturing a larger number of residents may

have improved the factor analysis. Also, the fact that the

mastery items did not load onto a single consistent factor sug-

gests that the adaptation of the original scale for these items

mayhavebeen testingdifferent components ofmastery, or that

the itemswere not sensitive enough to test the construct in this

population. Nonetheless, the results demonstrate that the

survey tool was able to differentiate between the mastery and

performance motivational constructs, and that surgical resi-

dents aremotivated in their learning, at least in part, by each of

these intrinsic orientations.

The strength of this pilot study is its unique perspective on

surgical education, whereby the trainees themselves are the

subject of the research rather than the subject being a new

educational model or style. This is the first time that GO has

been studied in residents, which is surprising given that sur-

gical residents are recognized as one of the most highly

motivated groups of students. As such, the instrument needs

to bemodified to better capture residents’mastery orientation

and validated in a larger population. Once validation is ach-

ieved, combining the adapted PALS with other validated

questionnaires to measure burnout, job satisfaction, leader-

ship, feedback quality, and professionalism related to the

management of competing priorities may identify more spe-

cific areas of intervention. For instance, awareness of GO and

the implications of each style may allow faculty and trainees

to develop more individualized and attainable goals that

would promote satisfaction and improved performance.

Understanding GO in residents is an interesting and

important new direction in surgical education research given

its potential effect on important components of the training

experience and its impact on future professional develop-

ment. This pilot study represents the first promising step in

this new area of research.

5. Conclusions

Understanding GO in surgical residents, a highly motivated,

unique population of students, is important to improve the

delivery of education. The adapted survey demonstrated high

and comparable reliability in two of three constructs. In this

population, mastery GO is a much more complex and so-

phisticated construct than can be measured with the current

adaptation. This pilot represents the first step in the adapta-

tion of a tool to identify potentially maladaptive mechanisms

of achievement.

Acknowledgment

Author contributions: R.L.H. and C.H.-R. were responsible for

conception and design. R.L.H., C.H.-R., and R.R.K. were

responsible for analysis and interpretation. R.L.H., J.D., R.R.K.,

and J.B.M. were responsible for data collection. R.L.H. and

R.R.K. participated in article writing. C.H.-R., J.D., J.B.M., and

R.R.K. involved in critical appraisal and revision.

r e f e r e n c e s

[1] Cordes CL, Dougherty TW. A review and integration ofresearch on job burnout. Acad Manage Rev 1993;18:621.

[2] Janssen O, Van Yperen NW. Employees’ goal orientation,the quality of leader-member exchange, and the outcomesof job performance and job satisfaction. Acad Manag J 2004;47:368.

[3] Harris BS, Smith ML. The influence of motivational climateand goal orientation on burnout: an exploratory analysisamong division I collegiate student-athletes. Athletic Insight2009;11. http://www.athleticinsight.com/.

[4] Elliot AJ, Harackiewicz JM. Approach and avoidanceachievement goals and intrinsic motivation: a meditationalanalysis. J Pers Soc Psychol 1996;70:461.

[5] Dweck CS. Motivational processes affecting learning. AmPsychol 1986;41:1040.

[6] Zweig D, Webster J. What are we measuring? Anexamination of the relationships between the big-fivepersonality traits, goal orientation, and performanceintentions. Person Individ Diff 2004;36:1693.

[7] VandeWalle D. A goal orientation model of feedback-seekingbehavior. Hum Res Manag Rev 2003;13:581.

[8] Leung A, Luu S, Regehr G, et al. First, do no harm: balancingcompeting priorities in surgical practice. Acad Med 2012;87:1368.

[9] Epstein RM, Siegel DJ, Silberman J. Self-monitoring in clinicalpractice: a challenge for medical educators. J Contin EducHealth Prof 2008;28:5.

j o u r n a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 4 ) 1e66

[10] Payne SC, Youngcourt SS. A meta-analytic examination ofthe goal orientation Nomological Net. J Appl Psychol 2007;92:128.

[11] Surgical Council on Resident Education, SCORE. http://www.surgicalcore.org/index. Accessed 10/20/13.

[12] Midgley C, Maehr ML, Hruda LZ, et al. Manual for thepatterns of adaptive learning scales (PALS). Ann Arbor, MI:University of Michigan; 2000.

[13] Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG.Research electronic data capture (REDCap) - a metadata-driven methodology and workflow process for providingtranslational research informatics support. J Biomed Inform2009;42:377.

[14] Floyd FJ, Widaman KF. Factor analysis in the developmentand refinement of clinical assessment instruments. PsycholAssess 1995;7:286.

[15] Cortina JM. What is coefficient alpha? an examination oftheory and applications. J Appl Psychol 1993;78:98. 2007.

[16] Midgley C, Kaplan A, Middleton M, et al. Development andvalidation of scales assessing students’ achievement goalorientation. Contemp Educ Psychol 1998;23:113.

[17] Ryan AM, Hicks L, Midgley C. Social goals, academic goals,and avoiding seeking help in the classroom. J EarlyAdolescence 1997;17:152.

[18] Zweig DI, Webster J. Are goal orientation and self-efficacydifferent? A validation of scales. Paper presented at the 15th

annual meeting of the Society for Industrial andOrganizational Psychology, New Orleans, LA 2000.

[19] Swing S. The ACGME outcome project: a retrospective andprospective. Med Teach 2007;29:648.

[20] Joyce B. Introduction to competency-based residencyeducation: 2006 ACGME. http://ortho.ucsd.edu/educational/documents/CompetenciesforACGME.pdf. Accessed 5/6/13.

[21] Latham GP, Brown TC. The effect of learning vs. outcomegoals on self-efficacy, satisfaction and performance in anMBA program. App Psych Int Rev 2006;55:606.