3
Complete the program information form honestly and con- cisely. And make the site visitor feel welcome. THE TROUBLESOME ISSUES: SCHOLARLY ENVIRONMENT, CONTINUITY OF CARE, SUPERVISION, GRADED RESPONSIBILITY, FLEXIBILITY, AND EQUIVALENCY Arnold G. Diethelm, MD, UAB School of Medicine, Birming- ham, Alabama What is a scholarly environment? That is a difficult question to answer in a specific way. I suppose one could say that if you have substantial NIH funding and a number of research papers published yearly, including clinical studies, with faculty in general surgery that publish 15 or 20 papers a year-that is a scholarly environment. Occasionally, as a reviewer and a member of the RRC, I review a program with a faculty that does not have NIH grants and perhaps does not have many publications, and yet it is a very important rotation for the clinical training of the resident. I do not view that as a problem. It is the parent institution that I think should be judged on the question of scholarly environment. So I would think that if you have a strong parent institution with a scholarly environment, and there are other integrated or affiliated community hospitals with little or no research, then you have met the responsibility. Presently, we see the problem when the parent institution does not have grant support, does not publish papers, shows very little interest in academic matters, and has little in the way of facilities for the residents to participate in scholarly work. That would not be what one would consider an acceptable scholarly environment. Continuity of care is a subject we discussed earlier. I think it is quite straightforward. A resident should not just be available to go and scrub and “do a case.” Sometimes it is necessary to take care of an urgent clinical problem, and I understand that. But the resident should be involved in the preoperative evaluation, planning of the operation, and the postoperative care. Individual graded responsibility is the basis for the surgical training in the United States, and it has been around for many, many years. You begin as an intern, when you are well educated, but not at all trained. Then during a period of 4 or 5 years, you become both educated and trained. The responsibility needs to be accorded by the faculty. I do not think it is a good idea for an intern to put in a central line in the first week. Central lines can cause important prob- lems. On the other hand, in 4 months, thoroughly trained, that individual certainly could move ahead and place cen- tral lines. Dr. Pories addressed the equivalency of training and I have nothing in particular to add to this topic. When we look at the minimum numbers-let us say one person does 2 thy- roidectomies and another does 15-there is obviously a problem in the case distribution. Recognizing that when a resident is on a rotation at Christmas time, it is usually very light. If he or she was on a rotation doing a lot of endocrine surgery, the resident might miss elective operations at that time of the year. But it is the responsibility of those of us who are either program directors or on the faculty to help that individual readjust and gain some experience, perhaps later in the year. Finally, if a person does a part of an operation, that is really not the same as being the surgeon. I think we have to be careful that the faculty surgeon allows the resident surgeon to participate in the major part of the operation. That does not mean that you may not have to help at 1 or 2 specific points. If the resident does a proximal or a distal anastomoses, that is really not the same as being involved in the entire operation. All of us have had an increased demand for resident cov- erage. Training programs now include endoscopy. We have liver, kidney, cardiac , and lung transplantations. Where do we get the house staff to cover all this? We can’t. Many years ago, you just added more house staff. It was a pyramidal program. This is no longer acceptable. The hospital is now responsible for helping us provide manpower to cover these services. In our institution, we have no residents in the cardiac ICU. We have nurse clinicians who provide complete coverage. Our outpatient activities are covered in the transplant service by nurse clinicians. We do not have the surgical house staff to do that. You can have surgeons’ assistants, nurse practitioners, and nurse clinicians-- how- ever, you want to grade these in the hospital. This financial support must come from the hospital because there is no money in patient care revenue to provide this kind of support. It is very hard to find a good time for a conference. If it is the first thing in the morning, the residents cannot get to the operating room. You have it at 6 or 7 o’clock at night, and then the house staff goes to sleep. This problem is not unique to any specific hospital; it is true everywhere. In 30 years, I have tried to have conferences at all different times, and I think there is only one good time, and that is Saturday morning, when most people are not in the operating room. And if conferences are held early, the residents still have time to be off. Programs always need to be flexible. If we are interested in the resident and in allowing him or her to have the best possible educational experience, then flexibility is a must. RESIDENT PERFORMANCE ON THE ABS EXAMINATIONS: ABSITE, QUALIFYING, AND CERTIFYING EXAMINATIONS Josef Fischer, MD, University of Cincinnati Medical Center, Cincinnati, Ohio It is a pleasure to be asked to address this issue, because I think there are some concerns about it, and my sense is that the purpose of the in-training basic science examination may be misinterpreted by some as to what it really intends to accomplish. The in-training examination was organized in 1975, and the questions on it were largely those of clinical management. In 1990, the examination was changed so’ that it became the in-training surgical basic science examination; the split be- tween basic science, as applicable to surgical science, was 60% what was called basic science, and 40% clinical man- agement. After a I994 survey, carried out in large part by J. David Richardson, my successor as chairman of the board, the apportionment of questions between surgical basic sci- ence and clinical management was changed to 50% for each. CURRENT SURGERY 0 Volume 56 / Numbers l/2 . January/February 1999 93

Resident performance on the ABS examinations: ABSITE, qualifying, and certifying examinations

Embed Size (px)

Citation preview

Page 1: Resident performance on the ABS examinations: ABSITE, qualifying, and certifying examinations

Complete the program information form honestly and con- cisely. And make the site visitor feel welcome.

THE TROUBLESOME ISSUES: SCHOLARLY ENVIRONMENT, CONTINUITY OF CARE, SUPERVISION, GRADED RESPONSIBILITY, FLEXIBILITY, AND EQUIVALENCY

Arnold G. Diethelm, MD, UAB School of Medicine, Birming- ham, Alabama

What is a scholarly environment? That is a difficult question to answer in a specific way. I suppose one could say that if you have substantial NIH funding and a number of research papers published yearly, including clinical studies, with faculty in general surgery that publish 15 or 20 papers a year-that is a scholarly environment. Occasionally, as a reviewer and a member of the RRC, I review a program with a faculty that does not have NIH grants and perhaps does not have many publications, and yet it is a very important rotation for the clinical training of the resident. I do not view that as a problem. It is the parent institution that I think should be judged on the question of scholarly environment. So I would think that if you have a strong parent institution with a scholarly environment, and there are other integrated or affiliated community hospitals with little or no research, then you have met the responsibility. Presently, we see the problem when the parent institution does not have grant support, does not publish papers, shows very little interest in academic matters, and has little in the way of facilities for the residents to participate in scholarly work. That would not be what one would consider an acceptable scholarly environment.

Continuity of care is a subject we discussed earlier. I think it is quite straightforward. A resident should not just be available to go and scrub and “do a case.” Sometimes it is necessary to take care of an urgent clinical problem, and I understand that. But the resident should be involved in the preoperative evaluation, planning of the operation, and the postoperative care.

Individual graded responsibility is the basis for the surgical training in the United States, and it has been around for many, many years. You begin as an intern, when you are well educated, but not at all trained. Then during a period of 4 or 5 years, you become both educated and trained. The responsibility needs to be accorded by the faculty. I do not think it is a good idea for an intern to put in a central line in the first week. Central lines can cause important prob- lems. On the other hand, in 4 months, thoroughly trained, that individual certainly could move ahead and place cen- tral lines.

Dr. Pories addressed the equivalency of training and I have nothing in particular to add to this topic. When we look at the minimum numbers-let us say one person does 2 thy- roidectomies and another does 15-there is obviously a problem in the case distribution. Recognizing that when a resident is on a rotation at Christmas time, it is usually very light. If he or she was on a rotation doing a lot of endocrine surgery, the resident might miss elective operations at that time of the year. But it is the responsibility of those of us who are either program directors or on the faculty to help that individual readjust and gain some experience, perhaps later in the year. Finally, if a person does a part of an

operation, that is really not the same as being the surgeon. I think we have to be careful that the faculty surgeon allows the resident surgeon to participate in the major part of the operation. That does not mean that you may not have to help at 1 or 2 specific points. If the resident does a proximal or a distal anastomoses, that is really not the same as being involved in the entire operation.

All of us have had an increased demand for resident cov- erage. Training programs now include endoscopy. We have liver, kidney, cardiac , and lung transplantations. Where do we get the house staff to cover all this? We can’t. Many years ago, you just added more house staff. It was a pyramidal program. This is no longer acceptable. The hospital is now responsible for helping us provide manpower to cover these services. In our institution, we have no residents in the cardiac ICU. We have nurse clinicians who provide complete coverage. Our outpatient activities are covered in the transplant service by nurse clinicians. We do not have the surgical house staff to do that. You can have surgeons’ assistants, nurse practitioners, and nurse clinicians-- how- ever, you want to grade these in the hospital. This financial support must come from the hospital because there is no money in patient care revenue to provide this kind of support.

It is very hard to find a good time for a conference. If it is the first thing in the morning, the residents cannot get to the operating room. You have it at 6 or 7 o’clock at night, and then the house staff goes to sleep. This problem is not unique to any specific hospital; it is true everywhere. In 30 years, I have tried to have conferences at all different times, and I think there is only one good time, and that is Saturday morning, when most people are not in the operating room. And if conferences are held early, the residents still have time to be off.

Programs always need to be flexible. If we are interested in the resident and in allowing him or her to have the best possible educational experience, then flexibility is a must.

RESIDENT PERFORMANCE ON THE ABS EXAMINATIONS: ABSITE, QUALIFYING, AND CERTIFYING EXAMINATIONS

Josef Fischer, MD, University of Cincinnati Medical Center, Cincinnati, Ohio

It is a pleasure to be asked to address this issue, because I think there are some concerns about it, and my sense is that the purpose of the in-training basic science examination may be misinterpreted by some as to what it really intends to accomplish.

The in-training examination was organized in 1975, and the questions on it were largely those of clinical management. In 1990, the examination was changed so’ that it became the in-training surgical basic science examination; the split be- tween basic science, as applicable to surgical science, was 60% what was called basic science, and 40% clinical man- agement. After a I994 survey, carried out in large part by J. David Richardson, my successor as chairman of the board, the apportionment of questions between surgical basic sci- ence and clinical management was changed to 50% for each.

CURRENT SURGERY 0 Volume 56 / Numbers l/2 . January/February 1999 93

Page 2: Resident performance on the ABS examinations: ABSITE, qualifying, and certifying examinations

One of the things we can be proud of is that surgery is the only discipline that I am aware of that has a rigorous requirement for basic science in its curriculum. The ABS believes that as surgeons, rather than looking askance at this fact, we should be proud of this, especially with respect to other disciplines. As educators, we should enhance this requirement to better educate our trainees, because if we fail to introduce our trainees to the acquired basic science information that is central to their training, the surgeon really does become a cut on the dotted line, a surgical technician. The acquisition of fundamental knowledge is a Drerequisite for a well-educated surgeon, because today’s “basic science” becomes tomorrow’s surgical practice.

Clinical practice is evolving very rapidly at this point. For example, nitric oxide, which was the molecule of the year several years ago, was the subject of 1000 papers the first year after it was described. Only 2 years later, it is in clinical trials. If you missed the fact that this was something that was an important mediator, then your experience with nitric oxide in the ICU and in other situations really falls on deaf ears, and you become an ignorant surgeon and an ignorant technician.

Several years ago, proton pump inhibitors and p53 gene were jargon. Now they play a basic role in our nonsurgical treatment of ulcer disease. Some understanding of the neo- plastic process, which, although rudimentary at this point, will undoubtedly figure heavily into our surgical process. It almost goes without saying that a strong background in basic science, which only a surgical program can provide in an organized fashion, is essential to surgical training.

One thing that is widely misinterpreted is the purpose of the ITBSCE examination or the ABSITE. What is it? What is it intended to be? How is it to be used? Periodically the board gets a request from someone, however well intentioned, who wants to use the ABSITE examination as a evaluative tool of some aspect of surgery or some aspect of that particular resident. In fact, the board intends the ABSITE to be an educational rather than an evaluative tool; ABSITE is to be used to track the educational experience of residents and the ability of programs to educate residents. Then, if there is a problem with ABSITE scores, rightfully it is not that particular resident but the program that should be at risk. I am fully aware that despite the number of opportu- nities offered the various residents, they may not take ad- vantage of these opportunities-you can lead a horse to water, but you certainly can’t make it drink. Therefore, sometimes residents may not take advantage of the educa- tional opportunities or may not believe that there is suffi- cient emphasis placed on the educational opportunities. For the most part, most surgical residents in this country appear to have the intellectual capacity to do very well on the ARSITE. Therefore, failure to do so is basically the fault of the program for not conveying the information to the resi- dent in a manner in which he or she can absorb it,

Here is a quote from the latest annual evaluation that Tom Brewster, the assistant director of clinical evaluation, does for the ABS:

‘The in-training surgical basic science examination scores represent only one aspect of overall competence, and other data sources concerning residents’ knowledge, clinical skills, attitudes, habits, and technical expertise must be con- sidered in any summative assessment,

Such assessments are obviously and rightly quite complex and require the skilled judgment of program directors.

What about the question that has been asked in your par- ticular handout-about the dean’s daughter and scores of 32, 17, 10, and 5 over the past 4 years? The program director and the chair, if a different person, obviously flunked ge- netics. Intelligence is an inherited characteristic, and lack of intelligence is an inherited characteristic. In the case of the president, there may be a cognitive problem. But here you have confronted the program director with somebody who has, in fact, been doing poorly on the ABSITE examination for 4 years. I think there is only 1 conclusion: the program director or whoever is responsible for the cognitive side of Lhe education of this particular resident has been negligent. What should have been happening over that period of time? First of all, there has to be an assumption that the intell& gence level is adequate. The resident needs to be coun- seled. Some learning opportunities need to be structured with a mentor-in our program, each resident has an advi- sor. I spend a fair amount of time structuring a remedial type of situation for anyone who has not done well on the ABSITE. Because their intelligence is usually very high, intelligence is not the problem. You provide the SESAP. You review the keyword feedback as to what area they did not know, I am not sure they need to study that area or feedback, but it should be included in the remedial pro- gram. It has to be didactic. They should read, They do not have to read the JournaE of Clinica *Tnvest@ztion, but they need to read textbooks on a regular basis.

It is inreresting, at least in our program, the most dismal scores on the ABSITE examination are during the laboratory years, which are required. And the usual excuse is that residents were reading for their research. Since most of the research they do is in molecular biology, it is a little differ- ent. But they have not made time for textbook reading, and you can usually tell who is going to do that and who is going to do poorly because you do not see them at confer- ences. And the moment you put together a remedial pro- gram, that situation usually changes rapidly.

Suppose this individual continued on his or her merry way, and suppose the resident had an ABSITE score in the last year of 3. Then what alternative do you have as a program director? One alternative you do not have, and probably is in the contract of the resident with the hospital, is having gone on for 4 years, you have no alternative but to sign off on this individual. And were you not to do so, it would probably justify the resident in filing a complaint against JKXl.

One other thing that you are undoubtedly inrerested in, and one of the reasons you might not want to sign off, is what the chances are of this individual passing the qualifying examination. You might be surprised to know that the chances of such an individual passing the qualifying exam- ination are not too bad. These are the 1994 data, but in 1993 they were the same, and they are identical when 1993 through 1996 are taken into account and statistically eval- uated. Among the highest percentiie, there are always a few people who flunk. But more important, a substantial num- ber of people in the lowest categories-people you wouldn’t think had a prayer of passing-pass on the first attempt. The corrected correlation is 0.63. The uncorrected correlation is about 0.57. Why is this? First, you do not know whether the results are this low because of the basic science

94 CURRENT SURGERY 0 Volume 56 / Numbers l/2 * January/February 1999

Page 3: Resident performance on the ABS examinations: ABSITE, qualifying, and certifying examinations

component. Most of what the boards qualifying examina- tion emphasizes, probably to a slightly greater extent than the ITSBSE, is clinical management. Second, not everyone takes the boards immediately after they are finished. Al- though it is difficult to tell statistically, some of these people may seek further education before taking the board exam- ination. It is important to remember that the RRC is looking at the pass rate of the residents on the first time and that 60% is considered a satisfactory performance.

The board does not apologize for using the in-training surgical basic science examination to drive curriculum. For example, on one of the surveys we asked questions on statistics. One of the questions was, “What is life table analysis?” Life table analysis is essential to understanding

any basic journal article. Yet, only 38% of the trainees answered the question correctly. Pointing out the failure of statistical knowledge of many of our surgical trainees, par- ticularly when you begin to deal with outcomes, is some- thing that I think we are all going to have to be familiar with in our hospital programs. This may be a deficiency, and we really may be failing our trainees.

In summary, the in-training surgical basic science examina- tion is an educational not an evaluative tool. I think that most surgical residents have the intellectual apparatus to do well on it. If they don’t do well on it, rather than shooting the messenger we ought to appreciate the message. The program should put more effort into making certain that the residents do well on the examination.

CURRENT SURGERY . Volume 56 / Numbers l/2 l January/February 1999 95