Department of Graduate Medical Education Stanford University Medical Center PIFmanship 101 (or how to survive writing/reviewing program information forms)

Department of Graduate Medical Education

Stanford University Medical Center

PIFmanship 101(or how to survive writing/reviewing

program information forms)

Session Objectives

By the end of this session, you will be able to…Understand the basics of putting together a program information form (PIF)Improve outcomes by avoiding many of the PIFmanship “pitfalls” that lead to RRC citationsSave time when preparing/ reviewing PIFs for internal reviews and site visitsDecrease the fear and stress associated with completing a PIF

“I see a Site Visit coming in your future…”

“I see a Site Visit coming in your future…”

Gentle words of wisdom…

Your PIF is the site visitor’s first impression of your program, so...Be accurateAnswer the question that’s askedBe terse/tightHave documentation to back up your answersStart early – it takes months to write a good PIFThis is not something your mother, spouse, best friend, or admin asst. can do for you!

Common vs. Specialty

There are actually 2 PIFs to complete:The Common PIF

Addresses the program’s compliance with the Common Program Requirements (common to all GME programs)

The common PIF is completed online via the ACGME Web Accreditation Data System (WebADS)

The Specialty PIFAddresses compliance with the

specialty-specific program requirements

Depending on specialty, may be a Word document downloaded from the ACGME Site to be completed offline, or may be completed online via the ACGME Web Accreditation Data System (WebADS).

Common PIF

The common PIF contains questions regarding participating sites, faculty CVs, evaluation, and duty hours.

Most of the data should be updated annually by your coordinator.

To update the Common PIF: Go to: www.acgme.org/ads Use your ACGME assigned User

ID & PasswordGo to the PIF preparation section

(left-side menu) Once all of the data is entered

and VALIDATEDPrint as PDF

Common PIF – Page 1


Data is auto-populated based on entries elsewhere in WebADS

Original signatures required:Program DirectorCore Program Director

(subspecialty programs only)DIO



Previous citations are auto-populated

Remember, your citations and corrective actions live forever – respond with care.


“Major Changes” include:Changes in program format

e.g., Have you gone from 3 to 4 years?Changes in resident complement?Changes in program leadership?Changes in participating sites?

Only include changes since the last site visit!



Common PIF – Page 3-4

Sponsoring InstitutionAuto-populates from WebADSSingle/Limited Site Sponsor -

only sponsors one programParticipating Sites

Only list sites that provide a required one month full-time equivalent educational experienceRequired means all residents rotate

to that siteMake sure rotation lengths add

up to 12 months per year across all sites


Participating SitesBrief Educational Rationale

Residents are exposed to a wide array of patients with advanced diseases, larger indigent population, and many minority groups not as well represented in the other participating sites. There is an excellent exposure to the primary components of general surgery especially trauma and surgical critical care.


Participating SitesProgram Letter of Agreement (PLA)

is required unless under governance of sponsor!We’ve posted a PLA template to our GME

web siteNote that the PLA is an agreement

between the Program Director and the individual at the participating site charged with oversight of the residency program. As such, the PLA may be between you and yourself…e.g., at Stanford, the PM&R Program

Director is also the site director at the VA hospital



Program Director MUST be able to approve the selection of teaching faculty

The Program Director MUST evaluate and approve continuation of teaching faculty

Program Director MUST comply with the university’s or medical center’s policies on selection, evaluation, promotion, disciplinary action and supervisionRefer to the House Staff Policies and

Procedures document posted on our GME web site: http://med.stanford.edu/gme/policy/

Program Director MUST comply with ACGME and RC policies and proceduresInstitutionalCommonSpecialty-Specific



Physician Faculty RosterList Alphabetically and by SiteFaculty who spend at least 10

hrs per week in resident educationHours/week devoted to education

should be realistic! Board certification

If double boarded include both boards

If sub-specialty program director or faculty, ensure primary boards are included

Make sure the roster data matches CV data



Faculty CVsGreat potential for a HUGE

number of citationsAll fields must be completed (NO

BLANKS)Accurately list training history

including GMEEnsure certification is validEnsure licensure is current and

has not lapsed


Faculty CVsSelected Bibliography, Review

Articles, & ActivitiesStrict limit of 10“…from the last 5 years” excludes

any before 2006!Publications should not be “in press”

if submitted many years ago…If not ABMS certified, explain…


Be concise, but not THAT concise. Try instead:<Program Director> oversees the

operations of the entire program; supervises trainees during patient-care activities; mentors trainees’ research projects; leads lectures and seminars; monitors duty hour compliance; and coordinates evaluation of courses, rotations, trainees and faculty.



Non-Physician Roster and CVsAgain, accurately complete all

fields andObserve the 10 item / 5 year

limit



Number of PositionsNote: If you have a resident

making up a maternity leave, you must ask ACGME for an extra slot if you are over your quota

Actively Enrolled ResidentsOther than interns, everyone

should have prior years of GMEProgram Director MUST obtain

summative evaluation of previous experience for transfers




Transferred, Withdrawn or Dismissed ResidentsResidents who resign are NOT

dismissedMust provide competency-based

summative evaluation to new program for transfers

EvaluationYES, residents are evaluated

following each learning experience (i.e. rotation)

YES, evaluations are documented (state how)ElectronicallyResident Files


Methods of EvaluationAssessment Method

Direct ObservationSimulationOSCEEtc

Evaluator(s)Program DirectorFaculty“360” (required as of 2007)

NursesAncillary StaffPatientsOther



Answer the question that’s asked!


Describe how evaluators are educated to use the assessment methods listed above so that residents are evaluated fairly and consistently.The Program Director meets with

evaluators annually before new trainees start the program to review and discuss the core competencies, competency-based performance evaluations, and assessment methods to be used. The electronic assessment system and the rating scales are also reviewed and discussed during faculty meetings in order to ensure that evaluators are fully educated and up to date with the assessment methods and processes.


Describe how residents are informed of the performance criteria on which they will be evaluated.At the beginning of each academic

year, the Program Director conducts an orientation to address several key issues related to the residency including the performance criteria on which the residents will be evaluated. During this orientation session, the Program Director carefully details the specific evaluation methods to be used. Both the criteria and methods are also documented in the residency handbook. Additionally, at the beginning of each block rotation, the rotation director meets with the resident and carefully delineates the expectations and performance criteria on which the resident will be evaluated for that specific rotation.


Describe the system to ensure that faculty complete written evaluations of residents in a timely manner following each rotation or educational experience.At the end of each block rotation, our

online resident data collection and tracking system sends an automated reminder to the service attending(s) to evaluate the resident(s) on that particular rotation. Using this online system, the Program Coordinator tracks pending evaluations and follows up with faculty as needed to urge them to complete their evaluations on time. In the unlikely event that a faculty member is unresponsive to the coordinator's requests to complete evaluations, the Program Director contacts the faculty member and requests him/her to complete the evaluations.


Describe the process used to complete and document written semiannual resident evaluations, including the mechanism for reviewing results (e.g., who meets with the residents and how the results are documented in resident files).Resident performance is evaluated by the

teaching faculty at the conclusion of each rotation using an electronic evaluation form. Additionally, each faculty mentor meets with their resident advisee quarterly and documents a summary of the meeting to be placed in the resident’s file. The Program Director meets with each resident on an individual basis semiannually to review the accumulated performance evaluations and mentor notes, provide feedback, and update the resident’s learning plan as appropriate. A summary of these meetings is documented and placed in the resident’s file. The residents are free to review the contents of their records at any time.


Describe the system used by the residents to provide annual confidential written evaluations of the teaching faculty (have examples and forms available for review by site visitor).Residents annually confidentially

evaluate the teaching faculty using <system>. The electronically submitted evaluation forms are anonymous. Residents are sent e-mail reminders about completing evaluations in a timely manner by the Fellowship Coordinator. Paper copies of completed evaluations without personal identifiers are printed by the Program Coordinator, reviewed by the Program Director, and presented to individual faculty members for review and consideration.


Describe the program's (or Department's, if applicable) system for evaluating and providing feedback to the teaching faculty.Residents annually confidentially evaluate

the teaching faculty using <system>. The electronically submitted evaluation forms are anonymous. Residents are sent e-mail reminders about completing evaluations in a timely manner by the Fellowship Coordinator. Paper copies of completed evaluations without personal identifiers are printed by the Program Coordinator, reviewed by the Program Director, and presented to individual faculty members for review and consideration. If and when an evaluation reveals an issue with a particular member of the teaching faculty, the Program Director (and/or the Department Chair) meets with that faculty member more urgently to address the issue. Additionally, the Program Director meets with all faculty on an annual basis to review resident feedback and implement any necessary changes.


Describe the approach used for program evaluation.Residents and faculty annually evaluate

the program using the <system>. The electronically submitted evaluation forms are anonymous. Residents and faculty are also encouraged to provide feedback to the Program Director whenever any issue arises or as they see opportunities for improvement. Additionally, residents and faculty participate in an Annual Program Review Meeting led by the Program Director. Aggregated data including the most recent ACGME survey results and the resident/faculty program evaluation results are reviewed and used at this meeting to improve the program. The Program Coordinator keeps minutes during this annual meeting and documents any plans to address areas requiring improvement. Action plan progress is monitored and documented by the Program Director.


Describe one example how the program used the aggregated results of residents' performance and/or other program evaluation results to improve the program (have the written plan of action available for review by the site visitor).Over the past two years, the evaluations by the

faculty, as well as the results of the didactic EMG examinations, and resident performance on the SAE examination have all reflected an average to less than average performance by the residents in the area of electrodiagnosis. The educational committee discussed this area of concern, going over all of the evaluations, reviewing the examinations, and determined an action plan. Each of the participating sites exposing the residents to EMG increased the number of didactic sessions in this area, especially for the first-year residents. Noon conferences were instituted at Valley Medical Center and the VA emphasizing EMG education, which included hands-on sessions conducted by the faculty, with senior resident contribution. The EMG portion of the didactic curriculum was moved forward toward the beginning of the academic year (September) to expose the junior residents to the electrodiagnostic educational material earlier in their residency. The anatomy lab was also restructured to supplement the functional anatomy and EMG correlative didactics.


Describe the improvement efforts currently undertaken based on feedback from the ACGME Resident Survey.Program Director MUST:

review the results of the ACGME Resident Survey each year

discuss the results at your Annual Program Review Meeting

address each area of concern (more than 20%) or any negative Duty Hour response

While smaller programs do not receive their results annually, results aggregated over several years may be accessible – check in WebADS and/or with your DIO



Resident Duty HoursUse the summary report

generated by your tracking system


Briefly describe how the faculty provides appropriate supervision of residents in patient care activities.While the supervision of residents in the program

is designed to provide gradually increased responsibility and maturity in the performance of the skills, all residents are supervised at all times. The required Level of Supervision for specific tasks is assigned based on level of training unless by exception, the Program Director indicates that further training is required before such approval is granted for a given resident. The Levels of Supervision provided by faculty are defined as follows: Level-1: Direct Supervision - The supervising physician

is physically present with the resident and patient Level-2: Indirect Supervision -

A: Direct supervision immediately available – The supervising physician is physically within the confines of the site of patient care, and immediately available to provide Direct Supervision

B: Direct supervision available – The supervising physician is not physically present within the confines of the site of patient care, is immediately available via phone, and is available to provide Direct Supervision

Level-3: Oversight – The supervising physician is available to provide review of procedures/encounters with feedback provided after care is delivered

Residents are responsible for knowing the limits of their scope of authority. Whenever a question arises about resident competency to perform a procedure independently, the attending physician is consulted.

Specialty PIF

The specialty PIF is generally a Word document* and found on the ACGME web site: http://www.acgme.org/acWebsite/navPages/nav_comPIF.asp

The specialty PIF contains questions regarding the ACGME general competencies and may request a block diagram for your program, a narrative description of your program, documentation of scholarly activity, and/or case logs.

* For some programs, such as Medicine Specialties and Ophthalmology, the Specialty PIF is accessed and completed via WebADS, just like the Common PIF.

Specialty PIF – PBLI

Describe one learning activity in which residents engage to identify strengths, deficiencies, and limits in their knowledge and expertise (self-reflection and self-assessment); set learning and improvement goals; identify and perform appropriate learning activities to achieve self-identified goals (life-long learning).Describe one learning activity.e.g., programs may use a structured

process for reflection in which a faculty advisor guides the resident in using feedback , evaluations, and/or in-training exam scores to inform the self-assessment process.

Documentation of the semi-annual evaluation meetings in which this process is demonstrated would provide evidence that this requirement is being addressed.

Specialty PIF – PBLI

Describe one example of a learning activity in which residents engage to develop the skills needed to use information technology to locate, appraise, and assimilate evidence from scientific studies and apply it to their patients’ health problems. The description should include: a) locating information, b) using information technology, c) appraising information, d) assimilating evidence information (from scientific studies), e) applying information to patient care.Describe one example.An appropriate learning activity could

be a structured activity such as a journal club presentation, critical appraisal of a topic, or educational prescription with appropriate faculty oversight and formal assessment of skills.

Additional documentation would be the written goals and objectives for this learning activity and how residents are assessed.

Specialty PIF – Professionalism

Describe at least one learning activity, other than lecture, by which residents develop a commitment to carrying out professional responsibilities and an adherence to ethical principles.At intern retreat we devote an afternoon to

discussing the role of a pediatrician in the long term care of a chronically ill and ultimately terminally ill child. We use the film “Cameron’s Arc” to initiate a discussion of the crucial role the pediatrician played in the care of a child with Tay Sachs disease. Focusing not on the disease per se, but on the role of the pediatrician, we identify the important character traits that made the pediatrician so very crucial to this family and to this child: we discuss the sense of ownership, the process of bringing a family to an understanding of the disease they face, the essential simultaneous expression of empathy and control, the style of communication that adapted to each parent’s needs, the choice of timing of various conversations as the condition and circumstances change for this child and her family, and the holistic approach to a disease process that requires direction of difficult decisions and participation adjusted to the needs of a family at various points in the journey a disease may dictate. At the end of the exercise, we share and reflect as a group on what we each found so essential to the “meaning of being a doctor” that was so well portrayed by the pediatrician in this film.

Specialty PIF – Systems-Based Practice

Describe an activity that fulfills the requirement for experiential learning in identifying system errors. Important elements may include identified

faculty to guide the activity, mechanism to ensure active engagement by each resident, and evidence of experiential learning (not just passive presence at conferences and meetings) in which residents participate in identifying a system problem or error and contribute to a potential solution.

Additional documentation could include written goals and objectives for this learning activity and how residents are assessed. Aggregated resident outcomes may be in the form of percentage of residents that completed a patient safety or other Systems-Based Practice project by the end of training, annual list of improvements that resulted from such projects, etc.

Other PIF Questions

Describe how residents are informed about their assignments and duties during the residency.This should be in writing and

verbal. For example: All residents are given the

program training manual which describes their assignments and duties during orientation to the program. Each resident signs an attestation that they have received and read the manual. The manual is discussed during orientation with the residents as a group.

Other PIF Questions

Describe how the program handles complaints or concerns the residents raise.The House Staff Policies &

Procedures document (posted on our GME web site) includes documentation of “Dispute Resolution” (grievance) procedures. Simply cut and paste into the PIF.

Other PIF Questions

How are identified resident duty hour violations addressed?Residents are required to enter

their duty hours into our electronic tracking system weekly. Duty hours are closely monitored each week for compliance with the 80 hour, 1 day in 7, and 30/10 duty hour standards by the Program Coordinator. Any potential violations are reported to the Program Director for review and follow-up action if required. If a resident has reached the limit, his or her schedule is then adjusted if required and closely monitored to ensure that no violations occur over a four week period. The program takes duty hour monitoring and compliance very seriously.

Other PIF Questions

Describe how the program monitors for excessive service and modifies the program accordingly.The residency has several methods to monitor

for excessive service and modify the program if required: Formal rotation evaluations with alerts: Residents

complete monthly evaluations that are reviewed and communicated with the rotation directors.

Committee on Residency Training and Clinical Services: Each class has resident representatives meet with the Program Director monthly for a formal agenda to review and address issues in the residency program.

The rotation directors are sent the residents’ monthly feedback on a quarterly basis (to ensure confidentiality for the residents’ feedback)

Yearly, the residents have a retreat in which each aspect of the program is reviewed. Areas of concern are brought back to the Committee on Residency Training. The ACGME resident survey questions with respect to excessive service are reviewed.

Informal feedback: the residents are encouraged to let the Chief Residents, the Faculty, and the Program Director know immediately when either the service load or the educational activities are compromised.

Weekly meetings of the program director and chief residents to discuss the residents and experiences on individual rotations.

Computerized entry and monitoring of all duty hours.

Patient Care Data

Block Diagrams

General Information

Glossary of ACGME terms : http://www.acgme.org/acWebsite/a

bout/ab_ACGMEglossary.pdfPIF sometimes asks for…

Hospital Statistics613 Licensed Beds at SHC272 Beds at LPCH

Library Resourceshttp://lane.stanford.edu/about/physical-r

es.htmlhttp://lane.stanford.edu/about/digital-re

s.html

General Information

Major Participating Institutions (Affiliates):Children's Health Council [058191]Kaiser Permanente Medical Center

(Santa Clara) [050571]Kaiser Santa Teresa [058092]Lucile Salter Packard Children's

Hospital at Stanford [050572]San Mateo Medical Center

[050585]Santa Clara Valley Medical Center

[050438]Veterans Affairs Palo Alto Health

Care System [050273]

Table of Contents

Common Errors

Table of Contents InaccuratePages not numbered or not

numbered correctlyDocument not spell-checkedResponse does not answer the

question askedStatistics not added correctly% of time for faculty

inconsistent within the PIFType font does not match

The Land Mines

As you write your PIF, remember to read the questions carefully...One example does NOT mean several“How” does not mean “we do”No “the program will…” or “we plan to…”No “see attached” or “see below”

Putting It All Together

One complete PIF (common and specialty) is due to the site visitor at least 14 days before the site visit.

The DIO (Ann) must sign the PIF before it goes out …and will want to read it first!Submit your completed PIF 14-

30 days before it’s sent to the site visitor.

The DIO and GME staff will review the PIF and return to you with suggested revisions.

You really cannot have too many eyes look at one PIF!!

Have the following documentsavailable for the site visitor:

Policy for supervision of residents (addressing resident responsibilities for patient care, progressive responsibilities for patient management, and faculty responsibility for supervision) (CPR IV.A.4)

Program policies and procedures for residents’ duty hours and work environment (CPR II.A.j.4.; CPR VI.C.; IR II.D.4.i.; IR III.B. 3.)

Moonlighting policy (CPR II.A.4.j; CPR VI.F)


Documentation of internal review (date, participants’ titles, type of data collected, and date of review by the GMEC)

Overall educational goals for the program (CPR IV.A.1)

Competency-based goals and objectives for each assignment at each educational level (CPR IV.A.2)

Current Program Letters of Agreement (PLAs) (CPR I.B.1)


Files of current residents who have transferred into the program, if applicable (including documentation of previous experiences and summative competency-based performance evaluations) (CPR III.C.1)

Evaluations of residents at the completion of each assignment (CPR V.A.1.a)

Evaluations showing use of multiple evaluators (faculty, peers, patients, self, and other professional staff) (CPR V.A.1.b.(2))


Documentation of residents’ semiannual evaluations of performance with feedback (CPR II.A.4.g; V.A.1.b.(4))

Final (summative) evaluation of residents, documenting performance during the final period of education and verifying that the resident has demonstrated sufficient competence to enter practice without direct supervision (CPR V.A.2)

Completed annual written confidential evaluations of faculty by the residents (CPR V.B. 3)


Completed annual written confidential evaluations of the program by the residents (CPR V.C.1.d.(1))

Completed annual written confidential evaluations of the program by the faculty (CPR V.C.1.d.(1))

Documentation of program evaluation and written improvement plan (CPR V.C)

Documentation of resident duty hours (CPR II.A.4.j; VI.D.1-3)

Files of current residents and most recent program graduates

PIFmanship 101

Documents

Department of Graduate Medical Education Stanford University Medical Center PIFmanship 101 (or how to survive writing/reviewing program information forms)