Accreditation Program Oversight and Special Review

We share the same goal – a top-notch training experience for our learners.

Our GME Accreditation Team takes very seriously its responsibility for institutional oversight of training program quality, as required by ACGME. Each Autumn, we systematically review each accredited training program against an internal UW GME Accreditation Dashboard and UW GME Special Review Criteria.

Data collected for each program within this Accreditation Dashboard includes:

Accreditation status
Citation/s or Areas for Improvement (AFI) within the ACGME Letter of Notification
Program Director (PD) and Program Administrator protected time (FTE)
ACGME Resident and Faculty Survey results
Clinical and Educational Work Hour submission and compliance reports
Resident and/or fellow well-being metrics

Considering trends over time, each program’s data are also analyzed within the context of additional factors, including:

Timing within the accreditation cycle (i.e., past or upcoming ACGME Site Visit)
PD turnover
Timing of past UW GME action
GME internal review of the Annual Program Evaluation (APE)

Programs performing well on the dashboard are simply monitored on an annual basis.

For programs demonstrating significant markers of concern, a Special Review may be recommended to the Graduate Medical Education Committee (GMEC). A Special Review is an internal, one-time intervention by the GME Accreditation Team, consisting of interviews with program leadership and staff, trainees, faculty, and Division/Departmental leadership. It culminates in a thematic report identifying improvement goals and a structure for accountability for corrective action/s. (Please see the UW GME Special Review Policy for additional details)

Our team will provide impacted Program Directors with a formal letter following GMEC proceedings, including basic information regarding the observed domains of concern.

Over the past 3 years (2021 – 2024), an average of 7 programs per year are scheduled for a Special Review.

Programs who have had a Special Review and demonstrate residual concern may be identified for Escalation, which may consist of meetings between GME and departmental leadership and/or presentation to GMEC.

Programs with markers of concern not rising to the level of a Special Review are identified as Flagged, and programs may be approached to provide additional data. A Special Review may be pursued based on this supplemental information.

We also want to commend programs that are doing well. As such, a Letter of Congratulations will be delivered to programs where substantial improvement is noted following a Special Review.

UW GME training programs are of high quality.

Most of our residency and fellowship programs are in good standing with the ACGME – an undisputed result of the commitment, focus and excellence of our UW GME PDs, faculty, and administrative staff. Our robust program evaluation process is designed to ensure that we maintain this high institutional standard.

Data do not tell the whole story.

We know this, and also recognize that for programs with a small number of trainees, important ACGME survey data may not be available in real time. Where a UW GME Special Review is planned, goals include:

Exploration of areas for improvement
Proactive enhancement of program compliance with ACGME requirements
Guidance for programs in corrective action where necessary
Advocacy for learners and programs within the broader UW Medicine environment

The Special Review process works.

Previous Special Reviews have resulted in resources for programs and tangible improvements, including:

Physical infrastructure enhancements
Addition of non-physician clinical staff to address service needs
Support for addressing faculty concerns
Clarification of program policy
Additional FTE dedicated to the training program

We come as partners.

These annual notifications can provoke anxiety and questions for affected programs. We wish to reiterate that the institutional oversight processes are formative and used to achieve better outcomes. Special Reviews are not shared with the ACGME.

The GME Office views the program oversight process as a key opportunity for early detection, understanding, support, partnership, and advocacy for all our programs, particularly those that may be at risk for adverse ACGME actions. We welcome your feedback to improve our process.

Additional Resource:

Special Review Protocol

Notification

Programs will receive an initial confirmation (via email with an attached PDF) from the GME Accreditation team notifying the Program Director (PD) and Program Administrator (PA) of a Special Review.

To Do (Group/Individual Responsible):

(Program) Notify all trainees and faculty (at a minimum, core faculty) that the program is receiving a Special Review.
(GME) Upload notification letter to MedHub under the Correspondence section (Program Accreditation>Correspondence)

Scheduling

GME Accreditation team will reach out to PD and PA to schedule Special Review meetings. A schedule template will be provided with specifics on groups and timing.
Required meetings include:

Residents/Fellows

Programs with 20 or fewer trainees: 60 minute interview of all trainees on duty on the day of the Special Review.
Programs with more than 20 trainees: 60 minute interview with a 10-15 trainees, ideally with equal representation for all years of training.
Programs with 1 trainee: 45 minute interview of trainee.
If in an ACGME training year, Chief Residents or Chief Fellows are welcome to participate in trainee interview.
Note: all trainees included should be ACGME trainees.

Teaching Faculty

60 minute interview with a minimum of 5 and maximum of 12 key faculty.
- Attendees should include directors of rotations at sites where trainees spend four months or more.
- Additional key faculty may include CCC members, PEC members or chairs.
- Please do not include the Department Chair, Division Head(s), Section Head(s), PD or Associate PD(s) in this meeting.

Program Leadership

60 minute initial meeting with PD, APD, and PA.
- A maximum of 1 APD should be included in the meeting.
Meeting with PA only available upon request.
30 minute concluding meeting with PD only.

Departmental Leadership

30 minute meeting with Department Chair or Division Head.
If applicable, 30 minute meeting with Vice Chair for Education.

To Do (Group/Individual Responsible):

(Program) In partnership with GME Accreditation team, identify and confirm a Special Review date.
(GME) In partnership with program, identify and confirm a Special Review date.
(Program) Follow the provided Special Review Scheduling Template to create a schedule for the GME Accreditation Team. See meeting attendee guidelines and restrictions in accordion menu to the left. (Due minimum of 12 days prior to Special Review date).
(Program) Block meeting time for residents/fellows to participate in confirmed meeting.
(GME) Send Outlook invite and Zoom link to Special Review participants (at least 7 days prior to Special Review day).

Data and Feedback Collection

As part of the preparation for the Special Review, the GME Accreditation Team collects program-specific data and solicits feedback for the program. Program-specific data and anonymous feedback is reviewed by the GME Accreditation Team to identify additional areas of underperformance to explore in Special Review meetings.

Program-specific data may include:

Most recent ACGME Accreditation Letter
Most recent ADS Update
Most recent Annual Program Evaluation (APE)
Most recent ACGME Resident and Faculty surveys
Specialty-specific ACGME Common Program Requirements

Feedback may include:

Anonymous feedback, including Mayo Leadership Inventory, from residents/fellows and faculty
Input on program from GME units
Input on program from RFPU-NW

To Do (Group/Individual Responsible):

(GME) Gather program-specific data.
(GME) Solicit input on program from GME units and RFPU-NW.
(Program/PD) Send template email and anonymous survey link (both provided by GME) to all core faculty members, as listed in ADS. In some instances, this step may have been completed before program received Special Review notification. (Minimum of 28 days prior to Special Review meeting).
(GME/Dr. Best) Send email and anonymous survey link to all ACGME-year residents/fellows in the program. In some instances, this step may have been completed before program received Special Review notification. (Minimum of 28 days prior to Special Review meeting.)

Special Review Preparation

Beyond the requested participation in anonymous surveys, there is no required preparation for either residents/fellows or faculty prior to the Special Review meetings.

To Do:

(Program) Submit completed Special Review Scheduling Template to GME Accreditation Team (at least 12 days prior to Special Review day).
Optional – (Program) Program may wish to redistribute results of ACGME resident and faculty surveys for review.

Special Review Day

GME Accreditation Team will conduct Special Review meetings virtually via Zoom.

Meetings will not be recorded.
De-identified notes will be taken throughout the Special Review meetings to assist in report writing.

Program participants are invited to provide 1:1 feedback outside of Special Review meetings with Dr. Jennifer Best.

To Do:

(GME) Solicit feedback around identified areas of underperformance and other relevant topics raised by participants.
(Program) As needed, assist meeting attendees with participation.

Special Review Report

After Special Review meetings have concluded, the program will receive a Special Review Report outlining quality improvement goals, suggested corrective actions and the process for GMEC monitoring of outcomes, including timelines

To Do:

(GME) Write and share drafted Special Review Report with PD. (5-7 weeks after Special Review day).
(PD) Review drafted Special Review Report and return to GME with substantive edits (1 week turnaround).
(GME) Assess PD’s edits, finalize Special Review Report and share with PD and PA. Department Chair or Division Head and, if applicable, Vice Chair for Education and Core program PD will be included in distribution. (6-8 weeks after Special Review day).
(GME) Compile Mayo Leadership Inventory data and share directly with PD for formative use only (6-8 weeks after Special Review day).
(GME) Upload Special Review schedule and report to MedHub.
(Program) Upon receipt, program leadership is strongly encouraged to share the contents of the Special Review report directly with residents/fellows and key faculty.

Special Review Response

After receiving the Special Review Report, the Program Director is responsible for providing an Action Plan and response to the Accreditation Team at 30 day and 6 month intervals (template provided).

To Do:

(GME) Provide reminder to PD and PA about 30-day written response due date.
(Program/PD) Provide written response to Special Review Report to Accreditation Team. Use provided Special Review Follow-Up and Action Plan template. (Due 30 days after report is received.)
(Program) Share Action Plan with residents/fellows and faculty.
(GME) Provide reminder to PD and PA about 6-month written response due date.
(Program/PD) Provide updated Action Plan to Accreditation Team. (Due 6 months after report is received.)

GMEC Monitoring

Following Special Review responses, program is included in ongoing GMEC monitoring, including review of ACGME Letters of Notification, ACGME Faculty/Resident survey results and Annual Program Evaluation.

Following a single annual accreditation cycle, if ongoing evidence of underperformance persists, PD is asked to deliver a progress report to GMEC in-person or in writing.

Following two annual accreditation cycles, if ongoing evidence of underperformance persists, Chair joins PD to deliver an in-person progress report to GMEC.

If following two annual accreditation cycles, ongoing evidence of underperformance persists, the program is formally referred to the DIO for additional interventions and the Special Review Protocol is completed.

FAQs

Who performs the Special Review?

The GME Accreditation team will conduct the Special Review, and may invite additional members. The Accreditation team includes:

Hadar Duman, Director of Accreditation
Jennifer Best, MD, Associate Dean for GME
Jenn Johal, Learning Environment Specialist

These individuals will participate on an ad hoc basis to ensure that no conflict of interest exists between the individuals and the program under review.

A required meeting participant cannot attend the scheduled date. What are the options?

For Resident/Fellow or faculty – Program should communicate to non-attendees that any feedback can be confidentially shared with Jenn Johal. Jenn will share deidentified information with the Accreditation Team. Alternatively, if the trainee or faculty prefers an interview, Jenn can coordinate an appropriate date.
For Program Director or Program Administrator – attendance is mandatory. Please work with Jenn to establish an agreeable date/time.

We are scheduled for a Special Review but just received a clean notification letter from the ACGME, what happens now?

If this occurs for your program the GME Accreditation team will review existing data and reserves the right to continue with or cancel the Special Review.

ACGME Accreditation Letters are an important segment of the Special Review Criteria but represent only a part of the holistic decision making process for Special Reviews.

When does a Special Review conclude?

A Special Review is a one-time intervention, with required responses at 30-days and 6-months after the intervention (see Special Review Response section above). After the Special Review intervention the program is included in the ongoing GMEC monitoring of program performance and will be notified (typically in late summer) if ongoing evidence of underperformance is detected (see GMEC Monitoring section above). For programs without evidence of underperformance no notification is provided.