Accreditation Program Oversight and Special Review

Each Autumn, the GME Accreditation Team systematically reviews each training program against an internal UW GME Accreditation Dashboard and UW GME Special Review Criteria. This review is central to our institutional oversight of the quality of our training programs, as required by ACGME. Data collected for each program within the Accreditation Dashboard includes the following:

Citation/s or Areas for Improvement (AFI) within the ACGME Letter of Notification
Certain accreditation statuses
Program Director and Program Administrator protected time (FTE)
ACGME Resident and Faculty Survey results
Clinical and Educational Work Hour submission and compliance reports
Resident and/or fellow well-being metrics

Relative to temporal trends and other UW GME accredited programs, each program’s data are also analyzed within the context of additional factors, including:

Timing within the accreditation cycle (i.e., past or upcoming ACGME Site Visit or Self-Study)
Program Director turnover
Timing of past UW GME action
GME internal review of the Annual Program Evaluation (APE)

For programs without significant markers of concern, the only action is continued annual monitoring. For programs demonstrating significant markers of concern, one of two other actions is recommended to the Graduate Medical Education Committee (GMEC). Impacted Program Directors will be provided with advance notice of this due process, as well as basic information regarding the observed domains of concern. A formal letter will follow GMEC proceedings.

Letter of Notice: A Letter of Notice is delivered when markers of concern are observed relative to other UW GME programs, but fall short of the bar for Special Review or b) where Special Review is deferred based on consideration of situational factors
Special Review: A Special Review is scheduled where significant markers of concern are observed relative to other UW GME programs. A Special Review is a one-time intervention by the GME Accreditation Team. It consists of interviews with program leadership and staff, trainees, faculty, and Division/Departmental leadership and culminates in a thematic report identifying improvement goals and a structure for accountability for corrective action/s. Please see the UW GME Special Review Policy for additional details.

Letters of Congratulations are delivered to programs where substantial improvement is noted following a Letter of Notice or Special Review. Where additional information arises, these decisions may be revisited within a given year.

The goals of a UW GME Special Review are to: 1) explore areas for improvement; 2) proactively enhance program compliance with ACGME requirements; 3) guide programs in corrective action where necessary and 4) advocate for learners and programs within the broader UW Medicine environment.

Previous Special Reviews have resulted in tangible outcomes and improvements including:

Physical infrastructure enhancements
Addition of non-physician clinical staff to address service needs
Support for addressing faculty concerns
Additional FTE dedicated to the training program

The GME Office views the program oversight process as a key opportunity for early understanding, support, partnership, and advocacy for all our programs, particularly those perceived to be at risk of adverse ACGME actions.

Additional Resource:

Special Review Protocol

Notification

Programs will receive an initial confirmation (via email with an attached PDF) from the GME Accreditation team notifying the Program Director (PD) and Program Administrator (PA) of a Special Review.

To Do (Group/Individual Responsible):

(Program) Notify all trainees and faculty (at a minimum, core faculty) that the program is receiving a Special Review.
(GME) Upload notification letter to MedHub under the Correspondence section (Program Accreditation>Correspondence)

Scheduling

GME Accreditation team will reach out to PD and PA to schedule Special Review meetings. A schedule template will be provided with specifics on groups and timing.
Required meetings include:

Residents/Fellows

Programs with 20 or fewer trainees: 60 minute interview of all trainees on duty on the day of the Special Review.
Programs with more than 20 trainees: 60 minute interview with a 10-15 trainees, ideally with equal representation for all years of training.
Programs with 1 trainee: 45 minute interview of trainee.
If in an ACGME training year, Chief Residents or Chief Fellows are welcome to participate in trainee interview.
Note: all trainees included should be ACGME trainees.

Teaching Faculty

60 minute interview with a minimum of 5 and maximum of 12 key faculty.
- Attendees should include directors of rotations at sites where trainees spend four months or more.
- Additional key faculty may include CCC members, PEC members or chairs.
- Please do not include the Department Chair, Division Head(s), Section Head(s), PD or Associate PD(s) in this meeting.

Program Leadership

60 minute initial meeting with PD, APD, and PA.
- A maximum of 1 APD should be included in the meeting.
Meeting with PA only available upon request.
30 minute concluding meeting with PD only.

Departmental Leadership

30 minute meeting with Department Chair or Division Head.
If applicable, 30 minute meeting with Vice Chair for Education.

To Do (Group/Individual Responsible):

(Program) In partnership with GME Accreditation team, identify and confirm a Special Review date.
(GME) In partnership with program, identify and confirm a Special Review date.
(Program) Follow the provided Special Review Scheduling Template to create a schedule for the GME Accreditation Team. See meeting attendee guidelines and restrictions in accordion menu to the left. (Due minimum of 12 days prior to Special Review date).
(Program) Block meeting time for residents/fellows to participate in confirmed meeting.
(GME) Send Outlook invite and Zoom link to Special Review participants (at least 7 days prior to Special Review day).

Data and Feedback Collection

As part of the preparation for the Special Review, the GME Accreditation Team collects program-specific data and solicits feedback for the program. Program-specific data and anonymous feedback is reviewed by the GME Accreditation Team to identify additional areas of underperformance to explore in Special Review meetings.

Program-specific data may include:

Most recent ACGME Accreditation Letter
Most recent ADS Update
Most recent Annual Program Evaluation (APE)
Most recent ACGME Resident and Faculty surveys
Specialty-specific ACGME Common Program Requirements

Feedback may include:

Anonymous feedback, including Mayo Leadership Inventory, from residents/fellows and faculty
Input on program from GME units
Input on program from RFPU-NW

To Do (Group/Individual Responsible):

(GME) Gather program-specific data.
(GME) Solicit input on program from GME units and RFPU-NW.
(Program/PD) Send template email and anonymous survey link (both provided by GME) to all core faculty members, as listed in ADS. In some instances, this step may have been completed before program received Special Review notification. (Minimum of 28 days prior to Special Review meeting).
(GME/Dr. Best) Send email and anonymous survey link to all ACGME-year residents/fellows in the program. In some instances, this step may have been completed before program received Special Review notification. (Minimum of 28 days prior to Special Review meeting.)

Special Review Preparation

Beyond the requested participation in anonymous surveys, there is no required preparation for either residents/fellows or faculty prior to the Special Review meetings.

To Do:

(Program) Submit completed Special Review Scheduling Template to GME Accreditation Team (at least 12 days prior to Special Review day).
Optional – (Program) Program may wish to redistribute results of ACGME resident and faculty surveys for review.

Special Review Day

GME Accreditation Team will conduct Special Review meetings virtually via Zoom.

Meetings will not be recorded.
De-identified notes will be taken throughout the Special Review meetings to assist in report writing.

Program participants are invited to provide 1:1 feedback outside of Special Review meetings with Dr. Jennifer Best.

To Do:

(GME) Solicit feedback around identified areas of underperformance and other relevant topics raised by participants.
(Program) As needed, assist meeting attendees with participation.

Special Review Report

After Special Review meetings have concluded, the program will receive a Special Review Report outlining quality improvement goals, suggested corrective actions and the process for GMEC monitoring of outcomes, including timelines

To Do:

(GME) Write and share drafted Special Review Report with PD. (5-7 weeks after Special Review day).
(PD) Review drafted Special Review Report and return to GME with substantive edits (1 week turnaround).
(GME) Assess PD’s edits, finalize Special Review Report and share with PD and PA. Department Chair or Division Head and, if applicable, Vice Chair for Education and Core program PD will be included in distribution. (6-8 weeks after Special Review day).
(GME) Compile Mayo Leadership Inventory data and share directly with PD for formative use only (6-8 weeks after Special Review day).
(GME) Upload Special Review schedule and report to MedHub.
(Program) Upon receipt, program leadership is strongly encouraged to share the contents of the Special Review report directly with residents/fellows and key faculty.

Special Review Response

After receiving the Special Review Report, the Program Director is responsible for providing an Action Plan and response to the Accreditation Team at 30 day and 6 month intervals (template provided).

To Do:

(GME) Provide reminder to PD and PA about 30-day written response due date.
(Program/PD) Provide written response to Special Review Report to Accreditation Team. Use provided Special Review Follow-Up and Action Plan template. (Due 30 days after report is received.)
(Program) Share Action Plan with residents/fellows and faculty.
(GME) Provide reminder to PD and PA about 6-month written response due date.
(Program/PD) Provide updated Action Plan to Accreditation Team. (Due 6 months after report is received.)

GMEC Monitoring

Following Special Review responses, program is included in ongoing GMEC monitoring, including review of ACGME Letters of Notification, ACGME Faculty/Resident survey results and Annual Program Evaluation.

Following a single annual accreditation cycle, if ongoing evidence of underperformance persists, PD is asked to deliver a progress report to GMEC in-person or in writing.

Following two annual accreditation cycles, if ongoing evidence of underperformance persists, Chair joins PD to deliver an in-person progress report to GMEC.

If following two annual accreditation cycles, ongoing evidence of underperformance persists, the program is formally referred to the DIO for additional interventions and the Special Review Protocol is completed.

FAQs

Who performs the Special Review?

The GME Accreditation team will conduct the Special Review, and may invite additional members. The Accreditation team includes:

Hadar Duman, Director of Accreditation
Jennifer Best, MD, Associate Dean for GME
Jenn Johal, Learning Environment Specialist

These individuals will participate on an ad hoc basis to ensure that no conflict of interest exists between the individuals and the program under review.

A required meeting participant cannot attend the scheduled date. What are the options?

For Resident/Fellow or faculty – Program should communicate to non-attendees that any feedback can be confidentially shared with Jenn Johal. Jenn will share deidentified information with the Accreditation Team. Alternatively, if the trainee or faculty prefers an interview, Jenn can coordinate an appropriate date.
For Program Director or Program Administrator – attendance is mandatory. Please work with Jenn to establish an agreeable date/time.

We are scheduled for a Special Review but just received a clean notification letter from the ACGME, what happens now?

If this occurs for your program the GME Accreditation team will review existing data and reserves the right to continue with or cancel the Special Review.

ACGME Accreditation Letters are an important segment of the Special Review Criteria but represent only a part of the holistic decision making process for Special Reviews.

When does a Special Review conclude?

A Special Review is a one-time intervention, with required responses at 30-days and 6-months after the intervention (see Special Review Response section above). After the Special Review intervention the program is included in the ongoing GMEC monitoring of program performance and will be notified (typically in late summer) if ongoing evidence of underperformance is detected (see GMEC Monitoring section above). For programs without evidence of underperformance no notification is provided.