6
Resident job satisfaction: Will 80 hours make a difference? Kirsten J. Lund, MD, * Ruben Alvero, MD, Stephanie B. Teal, MD, MPH Department of Obstetrics and Gynecology, University of Colorado Health Sciences Center, Denver, Colo KEY WORDS Duty hours Job satisfaction Quality of life Residency Objective: This study was undertaken to assess job satisfaction and quality of life among obstetrics and gynecology residents before the 80-hour work week. Study design: We administered a job satisfaction survey to residents before July 1, 2003, assessing satisfaction with residency training, indicators of current quality of life, and predictions for the effect of reduced work hours. Results: Residents were satisfied with training, with important outliers, including leisure time, ability to pursue educational reading, and surgical experience. We created job satisfaction facets that were generally reliable constructs and valid predictors for overall residency satisfaction. Residents predict more free time and a healthier lifestyle under the new requirements, but do not anticipate using additional time to study or teach. Conclusion: Job satisfaction facets for residents are proposed here and may be refined through further study. Lower scores for surgical experience are of concern in light of decreasing work hours. Educators must monitor self-directed learning efforts under new work hours. Ó 2004 Elsevier Inc. All rights reserved. On July 1, 2003, the Accreditation Council for Graduate Medical Education (ACGME) instituted duty hours restrictions for all its accredited residency training programs. 1 The ‘‘80-hour work week’’ had been in place for some specialties for more than 10 years but was recently made universal. Historical events in the state of New York, 2 the 1999 Institute of Medicine report publicizing the effect and number of medical errors, 3 and the filing of a petition by several groups, including the Public Citizen to the Occupational Safety and Health Administration 4 have provided impetus for change. There is also a sense among members of the public as well as medical educators that resident fatigue jeopardizes patient safety. 5 This hypothesis is supported by scientific evidence that sleep deprivation affects resident performance on tests of both cognitive and technical skill. 6,7 The intent of the duty hours restriction was to provide residents with adequate time away from work, to decrease resident fatigue, to minimize medical errors, and to thereby improve overall patient care. The concept of ‘‘patient care’’ is difficult to evaluate. The delivery of health care occurs in an increasingly cumbersome system, which by its very complexity thwarts most attempts to measure any isolated element of that system. Thus far, studies of patient care provided by residents under restricted working conditions have yielded conflicting results, and in fact preventable adverse events have been shown to be more common Presented at the 2004 CREOG and APGO Annual Meeting, Lake Buena Vista, Fla, March 3-6, 2004. * Reprint requests: Kirsten J. Lund, MD, 4200 E Ninth Ave, Box B-198, Denver CO 80262. E-mail: [email protected] 0002-9378/$ - see front matter Ó 2004 Elsevier Inc. All rights reserved. doi:10.1016/j.ajog.2004.07.072 American Journal of Obstetrics and Gynecology (2004) 191, 1805e10 www.ajog.org

Resident job satisfaction: Will 80 hours make a difference?

Embed Size (px)

Citation preview

Page 1: Resident job satisfaction: Will 80 hours make a difference?

American Journal of Obstetrics and Gynecology (2004) 191, 1805e10

www.ajog.org

Resident job satisfaction: Will 80 hoursmake a difference?

Kirsten J. Lund, MD,* Ruben Alvero, MD, Stephanie B. Teal, MD, MPH

Department of Obstetrics and Gynecology, University of Colorado Health Sciences Center, Denver, Colo

KEY WORDSDuty hours

Job satisfactionQuality of lifeResidency

Objective: This study was undertaken to assess job satisfaction and quality of life among

obstetrics and gynecology residents before the 80-hour work week.Study design: We administered a job satisfaction survey to residents before July 1, 2003, assessingsatisfaction with residency training, indicators of current quality of life, and predictions for the

effect of reduced work hours.Results: Residents were satisfied with training, with important outliers, including leisure time,ability to pursue educational reading, and surgical experience. We created job satisfaction facets

that were generally reliable constructs and valid predictors for overall residency satisfaction.Residents predict more free time and a healthier lifestyle under the new requirements, but do notanticipate using additional time to study or teach.

Conclusion: Job satisfaction facets for residents are proposed here and may be refined throughfurther study. Lower scores for surgical experience are of concern in light of decreasing workhours. Educators must monitor self-directed learning efforts under new work hours.� 2004 Elsevier Inc. All rights reserved.

On July 1, 2003, the Accreditation Council forGraduate Medical Education (ACGME) instituted dutyhours restrictions for all its accredited residency trainingprograms.1 The ‘‘80-hour work week’’ had been in placefor some specialties for more than 10 years but wasrecently made universal. Historical events in the state ofNew York,2 the 1999 Institute of Medicine reportpublicizing the effect and number of medical errors,3

and the filing of a petition by several groups, includingthe Public Citizen to the Occupational Safety andHealth Administration4 have provided impetus for

Presented at the 2004 CREOG and APGO Annual Meeting, Lake

Buena Vista, Fla, March 3-6, 2004.

* Reprint requests: Kirsten J. Lund, MD, 4200 E Ninth Ave, Box

B-198, Denver CO 80262.

E-mail: [email protected]

0002-9378/$ - see front matter � 2004 Elsevier Inc. All rights reserved.

doi:10.1016/j.ajog.2004.07.072

change. There is also a sense among members of thepublic as well as medical educators that resident fatiguejeopardizes patient safety.5 This hypothesis is supportedby scientific evidence that sleep deprivation affectsresident performance on tests of both cognitive andtechnical skill.6,7 The intent of the duty hours restrictionwas to provide residents with adequate time away fromwork, to decrease resident fatigue, to minimize medicalerrors, and to thereby improve overall patient care.

The concept of ‘‘patient care’’ is difficult to evaluate.The delivery of health care occurs in an increasinglycumbersome system, which by its very complexitythwarts most attempts to measure any isolated elementof that system. Thus far, studies of patient care providedby residents under restricted working conditions haveyielded conflicting results, and in fact preventableadverse events have been shown to be more common

Page 2: Resident job satisfaction: Will 80 hours make a difference?

1806 Lund, Alvero, and Teal

with ‘‘shift work’’ structures.8 There have also beenconcerns among resident educators regarding the effectof work restrictions on important aspects of residenttraining, including the quality and continuity of patientcare as well as the ability to teach to the same levels ofcompetence given fewer hours. Concern also exists as tothe impact of reduced resident work hours on non-resident providers, such as attending physicians, nurses,and other ancillary staff.

The assessment of physician job satisfaction hasbecome relevant given recent changes in the health careenvironment. Low physician job satisfaction has beenlinked to disability claims,9,10 physician turnover,11 andaltered prescribing practices12; high physician job satis-faction has been correlated with patient compliance.13

Survey instruments for measuring job satisfactionamong practicing physicians have been developed byusing ‘‘facets’’ or ‘‘dimensions’’ that predict work satis-faction, including autonomy, the work itself, professionalrelationships, personal time, educational preparation forthe job, teaching role, and compensation.14-16

Though studies of physicians in practice do not showa correlation between work hours and job satisfaction,17

the institution of work hours restrictions has been foundto correlate positively with residents’ perceived ‘‘qualityof life.’’18,19 The perceived lack of a good ‘‘quality oflife’’ has also been identified as a key reason fordecreasing interest in obstetric-gynecology (Ob-Gyn)training on the part of senior medical students.20

Given the importance of job satisfaction and qualityof life to practicing and training physicians, we un-dertook to assess resident job satisfaction before theinstitution of the ACGME duty hours in a university-based Ob-Gyn residency program, and to measure theirexpectations of the forthcoming changes.

Methods

Survey development

We developed a structured questionnaire containing 59items, divided into 4 segments: demographic data (5items), level of current satisfaction with the residencytraining experience (38 items), baseline assessment ofquality of life (5 items), and predictions as to the effectof the 80-hour work week on residency training (11items). All questions, except the demographic data, wereanswered on a 5-point Likert scale, where 1 = verydissatisfied and 5 = very satisfied or 1 = disagree and5 = agree as indicated in the tables.

Segments of this survey were adapted from work byother authors. Weaver et al21 developed a job satisfac-tion survey for family medicine residents that has beenvalidated and administered twice to more than 700residents in the state of Texas. The first 2 segments of

our survey were adapted from this work, with permis-sion. Some questions were changed as appropriate forOb-Gyn residency training. Bohil et al have presenteddata assessing the impact of work hours restrictions onresidency training from which the last segment wasadapted (Bohil, written personal communication,March 2004). The survey was designated as ‘‘exempt’’by the Colorado Multiple Institutional Review Board.

Survey administration

This survey was administered to all Ob-Gyn residents atthe University of Colorado in June 2003. Individualsurvey responses were anonymous and identified bya code number.

Statistical analysis

Survey responses were entered into an Excel spreadsheetand double checked for accuracy. Descriptive statistics, ttests, analysis of variance, and linear regression werecalculated using SPSS 12.0 for Windows (SPSS, Inc,Chicago, Ill). For comparison of multiple means, theBonferroni correction was used to adjust statisticalsignificance.

In keeping with previous survey development re-garding physician job satisfaction,14-16 survey questionswere grouped into ‘‘facets’’ of job satisfaction. Thesefacets included global satisfaction with training, faculty,academic educational opportunities, clinical trainingexperience, professional relationships, and lifestyle. Weassessed the reliability of job satisfaction facets, orwhether the items in each facet correlated with oneanother, using Cronbach’s alpha. This is a coefficient ofreliability that measures how well a set of items, orvariables, measures a single unidimensional construct.An acceptable reliability coefficient was defined as .7.22

To determine the validity of each facet, or whethera given facet was useful for predicting overall satisfac-tion with residency, we used multiple regression best-fitmodels, using the items in the facet against ‘‘overallsatisfaction with your Ob-Gyn residency program’’ asthe dependent variable.

Results

Population characteristics

Of 35 surveys, 33 were returned, a 94% response rate. Of9 surveys, all 9 were returned from the postgraduateyear 1 (PGY-1) and 2 classes, 8 of eight from the PGY-3, and seven of nine from the PGY-4 class. Tworesidents were male, 31 were female. The average ageof the residents was 33 years (range, 26-47). Seventeen(51%) were married, 16 single. Two residents hadchildren. The average work week in this residency,

Page 3: Resident job satisfaction: Will 80 hours make a difference?

Lund, Alvero, and Teal 1807

Table I Job satisfaction variables

Stem wording: ‘‘Please rate your satisfaction with the following aspects of your program’’:

Item no. Facet Question Mean SD

8 Global Overall satisfaction with Ob-Gyn as your chosen specialty 4.24 1.036 Global Overall satisfaction with your medical school education 4.06 0.797 Global Overall satisfaction with your Ob-Gyn residency program 3.79 0.8623 Global Overall learning environment in residency 3.55 0.6718 Faculty Quality of faculty overall 3.85 0.6213 Faculty Teaching from attending physicians 3.73 0.7220 Faculty Attending supervision overall 3.73 0.5732 Faculty Quality of didactic teaching 3.48 0.8015 Academic Amount of intellectual stimulation 4.03 0.5312 Academic Teaching from fellow residents 3.79 0.825 Academic Ability to research a given medical topic 3.48 0.9421 Academic Library facilities (including online) 3.44 0.841 Academic Ability to remain up-to-date with current medical developments 3.33 0.8917 Academic Personal participation in research 3.09 0.7736 Academic Educational leave 2.79 0.923 Academic Ability to pursue educational reading 2.42* 0.7925 Clinical/skill Obstetric experience 4.39y 0.7527 Clinical/skill Primary care experience 3.64 0.7422 Clinical/skill Exposure to subspecialties (MFM, GYN-ONC, RE/I, FPM/RS) 3.67 0.8211 Clinical/skill Gynecologic surgical experience 3.09y 0.9126 Clinical/system Patient diversity 4.27 0.672 Clinical/system Provide primary/preventive care 3.45 0.5630 Clinical/system Clinic facilities 3.36 0.7033 Clinical/system Ancillary hospital staff 3.36 0.904 Clinical/system Ability to provide cost-effective care 2.97 0.779 Prof relations Camaraderie with fellow residents 4.61 0.5610 Prof relations Level of respect from fellow residents 4.42 0.6635 Prof relations Quality of fellow residents 4.36 0.6538 Prof relations Social services/social workers 3.97 0.7337 Prof relations Relationship with faculty overall 3.85 0.8019 Prof relations Degree of autonomy 3.55 0.7914 Prof relations Quality of hospital nursing staff 3.45 0.5628 Prof relations Clinic staff and nursing 3.18 0.8524 Lifestyle Number of vacation days 3.91 1.0731 Lifestyle Salary and benefits package 3.64 0.8616 Lifestyle Call system setup 3.27 0.6329 Lifestyle Call frequency 3.27 0.8034 Lifestyle Amount of leisure time 2.52z 0.94

Items were scored on a 5-point scale, 1 = very dissatisfied, 5 = very satisfied. MFM, Maternal-Fetal Medicine; GYN-ONC, Gynecologic Oncology; RE/I,

Reproductive Endocrinology and Infertility; FPM/RS, Female Pelvic Medicine and Reconstructive Surgery.

* ‘‘Educational reading’’ lower than all other facet items (P ! .001) except ‘‘educational leave.’’y ‘‘Obstetric experience’’ higher than, and ‘‘Gynecologic surgical experience’’ lower than, primary care and subspecialty experience (P ! .01 for both).z ‘‘Amount of leisure time’’ lower than all other items in facet (P ! .001).

based on self-reporting in the spring of 2003, was 73hours per week.

Baseline satisfaction with residency

The overall results of residents’ satisfaction with aspectsof residency are summarized in Table I. Residents werequite satisfied with global measures, including satisfac-tion with medical school, residency, choice of specialty,and overall learning environment in residency. They

were generally satisfied with the teaching faculty andwith most measures of academic opportunity. However,residents were significantly less satisfied with theircurrent time available to pursue educational readingthan with other academic measures. Residents rankedobstetric experience significantly higher than, and gyne-cologic surgical experience significantly lower than,experience with primary care and with subspecialties(P ! .01). Professional relationships, particularly meas-ures of inter-resident relationships, were rated generally

Page 4: Resident job satisfaction: Will 80 hours make a difference?

1808 Lund, Alvero, and Teal

Table II Facets of job satisfaction

Validity (multiple regresson)*

Facet Number of items Reliability coefficient (Cronbach’s alpha) r P

Global 3 .753 d dFaculty 4 .783 .669 .002Academic 8 .676 .620 .261Clinical 9 .791 .633 .143Professional relations 8 .782 .775 .002Lifestyle 5 .789 .588 .034

* Dependent variable = overall satisfaction with residency. Validity of global facet not calculated, because the dependent variable is an item in this

facet.

Table III Predictions regarding the 80-hour work week

Stem wording: After the institution of the 80-hour workweek:

Type Question Mean SD

Lifestyle I will lead a healthier lifestyle. 3.97 1.06Lifestyle I will have more free time. 3.94 1.16Lifestyle I will be happier with my personal life. 3.78 1.18Training I will be happier with residency. 3.58 1.23Training Providing continuity of care for patients will become more difficult. 3.91 1.26Training The residency program will improve overall. 3.70 1.13Training I will be well trained in the practice of general Ob-Gyn. 3.62 .91Training I will have more interest in teaching. 3.59 1.13Training I will be able to give improved care to patients. 3.55 1.20Training I will use additional free time to study. 3.47* 1.14Training I will have more time to dedicate to teaching. 3.31* 1.09

Items were scored on a 5-point scale: 1 = disagree, 5 = agree.

* Lower than ‘‘I will lead a healthier lifestyle’’ (P = .069 for study and .015 for teaching).

highly. Among lifestyle items, vacation time was rankedmost highly, and ‘‘leisure time’’ significantly lower (P !.001) than any other lifestyle measure. Quality of thefaculty was the single factor most highly correlated withoverall satisfaction with residency (r= 0.645, P !.001).

The reliability and validity of the 6 facets of residentjob satisfaction are presented in Table II. These facetsare as follows: global satisfaction, faculty, academics,clinical, professional relationships, and lifestyle. ‘‘Over-all satisfaction with medical school education’’ did notcorrelate significantly with other global satisfactionmeasures and was dropped from the facet. Of the 6facets, 5 showed acceptable reliability coefficients, andof the 5 facets analyzed for validity, 4 showed significantcorrelation with overall residency satisfaction.

Current quality of life

Residents reported weekly mean hours spent sleeping(38.08, range 6-50), reading (2.86, range 0-10), weeklydays of exercise (2.15, range 0-6), minutes of exercise persession (44.59, range 0-90), and days per week in which 3discrete meals were eaten (3.167, range 0-7) No corre-lation was found between PGY level and amount oftime spent sleeping, reading, exercising, or reading.

Predictions for the 80-hour work week

Results of the final segment of the survey are summa-rized in Table III. Items are presented by questiontyped3 items directly address quality of life, 8 addressaspects of residency training. The highest level ofagreement was with the statement ‘‘I will lead a healthierlifestyle’’ (mean of 3.97). Anticipated increase in freetime (mean 3.94) was also high. Mean scores for ‘‘I willuse additional free time to study’’ (3.47) and ‘‘I will havemore time to dedicate to teaching’’ (3.31) were lowerthan the anticipation of leading a healthier lifestyle(P = .069 and .015, respectively).

Although residents tend to agree with statementsconcerning improvement in the residency (mean 3.70)and quality of residency training (mean 3.62) under thenew work week, residents are aware that patientcontinuity of care will be more difficult with the newsystem (mean 3.91).

Comment

To our knowledge, this study is the first to evaluate ina comprehensive fashion the job satisfaction of resi-dents in obstetrics and gynecology. These baseline dataprovide much insight into a large, university-based

Page 5: Resident job satisfaction: Will 80 hours make a difference?

Lund, Alvero, and Teal 1809

residency training program; perhaps more importantly,it provides a reference point for future studies of theactual effects of the 80-hour work week on multiplevariables. Residents in our program now work, after theimplementation of duty hours restrictions, an average of66 hours per week, or 7 hours per week less than theydid before July 1, 2003. The effects of this decrease inworkload on the various factors explored in this reportwill be an important area for future study. Extrapola-tion of our results to other residency programs may bereasonable, especially for programs with similar charac-teristics of size, clinical workload, and university affili-ation. Individual program results may differ, however,making a broader survey of residency under the newworkweek another important area of future study.

Our findings suggest that residents were quite satis-fied, in general, with their residency training before theinstitution of the ACGME duty hours restrictions.When measures of resident job satisfaction are brokendown into facets or dimensions, one can readily identifythose individual elements that impact satisfaction moststrongly, whether negatively or positively.

Our residency offers an abundance of obstetricexperience, including a large number of forceps andvacuum deliveries and direct involvement by residents inall phases of high-risk obstetric management. Theavailability of gynecologic surgery, although adequateto train competent residents, is not uniformly high forall surgical procedures. Residents correspondingly ratedsatisfaction with obstetric experience quite highly, andgynecologic surgical experience as the least satisfyingaspect of clinical training. As was noted previously,surgical experience and competence is a key concern ofresident educators nationally as programs adjust to the80-hour work week, and our study indicates that this isa concern for residents as well. It will be extremelyimportant for the specialty to monitor changes in casenumbers, as well as residents’ perceptions of theirsurgical training and competence.

Interpersonal relationships among the residents werevery highly rated, possibly reflecting the degree to whichresidents in a university-based, multihospital programrely on and socialize with one another. Although thebaseline measures of current quality of life are ofinterest, these measurements will be most useful forassessing change over time as the work week restrictionsare implemented.

In this survey, 6 facets of job satisfaction wereidentified. These facets are preliminary and were de-veloped using relatively few subjects. They show prom-ise, however, as predictors of job satisfaction, and futuredata collection will result in refinement of such facets.This may allow educators to identify residents who areat risk for attrition or disillusionment with training.

It is unclear whether the residents’ predictions aboutthe effects of the new work week will be accurate, but

certain parameters are worthy of comment. Residentsquite strongly expect that there will be more free timeafter the new work hours are implemented. This comple-ments the low satisfaction score given to the availabilityof leisure time under the current work model. Bycontrast, residents were less likely to anticipate thatthey would use additional free time to study, eventhough the ability to pursue educational reading underthe current system carried a notably low mean score;lower even than the availability of leisure time. Theseresponses suggest that residents, as a group, placea priority on quality of life during residency training,and that under a restricted work hour environment, thispriority may come at the expense of self-directedlearning.

As with surgical caseload, the acquisition of medicalknowledge is vulnerable under the work hours regula-tions. Faculty members will have limited time in whichto teach residents a wide range of medical information,as well as clinical and surgical skills. Of these, factualknowledge is the most amenable to learning outside ofofficial work hours. Given the results of this survey, itwill be important for resident educators to monitorresidents’ medical knowledge and efforts at self-directedlearning as the duty hours restrictions take effect.

References

1. ACGME duty hours standards fact sheet. Available at http://

www.acgme.org/. Accessed on February 22, 2004.

2. New York State Department of Health. Medical Staff in New

York Codes, Rules and Regulations, 10.405.4 (1998). Available at

http://www.health.state.ny.us/nysdoh/search/index.htm. Accessed

February 22, 2004.

3. Institute of Medicine. To err is human: building a safer health

system. September 1, 1999. Available at http://www.iom.edu/

reports.asp. Accessed on February 20, 2004.

4. Petition to the occupational safety and health administration

requesting that limits be placed on hours worked by medical

residents (HRG publication 1570). April 30, 2001. Available at

http://www.citizen.org/publications/release.cfm?ID=6771. Accessed

on February 22, 2004.

5. Philibert I, Friedmann P, Williams WT. New requirements for

resident duty hours. JAMA 2002;288:1112-4.

6. Samkoff JS, Jacques CH. A review of studies concerning effects of

sleep deprivation and fatigue on residents’ performance. Acad Med

1991;66:687-93.

7. Eastridge BJ, Hamilton EC, O’Keefe GE, Rege RV, Valentine RJ,

Jones DJ, et al. Effect of sleep deprivation on the performance of

simulated laparoscopic surgical skill. Am J Surg 2003;186:169-74.

8. Petersen LA, Brennan TA, O’Neil AC, Cook EF, Lee TH. Does

housestaff discontinuity of care increase the risk for preventable

adverse events?. Ann Intern Med 1994;121:866-72.

9. Fraunfelder FT, Fraunfelder N. Managed care and physician

disability. Manag Care Interface 1999;12:57-9.

10. Ainge D. Increased physician disability claims causing ‘‘crisis’’:

high utilization of disability insurance may leave physician groups

fewer options in the future. Med Group Manage J 1995;42:80-4 86.

11. Pathman DE, Konrad TR, Williams ES, Scheckler WE, Linzer M,

Douglas J. Physician job satisfaction, dissatisfaction, and turnover.

J Fam Pract 2002;51:593.

Page 6: Resident job satisfaction: Will 80 hours make a difference?

1810 Lund, Alvero, and Teal

12. Watkins C, Harvey I, Carthy P, Moore L, Robinson E, Brawn R.

Attitudes and behaviour of general practitioners and their pre-

scribing costs: a national cross-sectional survey. Qual Saf Health

Care 2003;12:29-34.

13. DiMatteo MR, Sherbourne CD, Hays RD, Ordway L,

Kravitz RL, McGLynn EA, et al. Physicians’ characteristics

influence patients’ adherence to medical treatment: results

from the Medical Outcomes Study. Health Psychol 1993;12:

93-102.

14. Williams ES, Konrad TR, Linzer M, McMurray J, Pathman DE,

Gerrity M, et al. Refining the measurement of physician job

satisfaction: results from the physician worklife survey. Med Care

1999;37:1140-54.

15. Coyle YM, Lu AA, Battles JB, Hynan LS. Measuring and

predicting academic generalists’ work satisfaction: implications

for retaining faculty. Acad Med 1999;74:1021-7.

16. Lichtenstein R. Measuring the job satisfaction of physicians in

organized settings. Med Care 1984;22:56-68.

17. Carr PL, Gareis KC, Barnett RC. Characteristics and outcomes

for women physicians who work reduced hours. J Womens Health

2003;12:399-405.

18. Whang EE, Mello MM, Ashley SW, Zinner MJ. Implementing

resident work hour limitations: lessons from the New York state

experience. Ann Surg 2003;237:449-55.

19. Barden CB, Specht MC, McCarter MD, Daly JM, Fahey TJ 3d.

Effects of limited work hours on surgical training. J Am Coll Surg

2002;195:531-8.

20. Liaison Committee on Obstetrics and Gynecology. Enhancing

student interest in the specialty of obstetrics and gynecology.

Washington (DC): American College of Obstetricians and Gyne-

cologists; 2003. p. 10-1.

21. Weaver SP, Mills TL, Passmore MA. Job satisfaction of family

practice residents. Fam Med 2001;33:678-82.

22. Santos JRA. Cronbach’s alpha: a tool for assessing the reliability

of scales. Texas agricultural extension service. Available at http://

www.joe.org/. Accessed on February 25, 2004.