Population Health Management Curriculum
- Incorporating Principles of Population Health Management (PHM) into Curriculum (AAMC & Center for Excellence in Primary Care, UCSF)
- Kern DE, Thomas PA, Hughes MT. Curriculum Development for Medical Education: A Six-Step Approach. Baltimore: The Johns Hopkins University Press; 2009.
Goals and Objectives
Objectives: specific, measurable, achievable, realistic, and timely target of learning (how you get there)
The Objectives require five basic elements: Who? How much? How well? What? and By when? (e.g., the Gynecology curriculum for internal medicine residents: By the end of the gynecology curriculum, residents will be able to demonstrate, at least once, the appropriate technique, as defined on a check sheet, for obtaining a Pap smear and cervical cultures)
- Writing Goals and Objectives
- How to Write Well-Defined Learning Objectives
- Communicating Your Program’s Goals and Objectives
- 7-minute video on Writing Learning Objectives (UW CLIME)
Don’t forget!: When developing goals and objectives, carefully review the data from formative and summative evaluations as well as Milestones. A high frequency of deviation in these data suggests the need for curricular change/improvements that will trigger the need for writing new goals and objectives or revising the existing goals and objectives.
Types of learning objective: learner objectives (cognitive, affective, psychomotor domains) and process objectives. Sometimes the term “learner objectives” is used interchangeably with “outcome objectives.”
- Learner Objectives – Cognitive domain (“knowledge” objectives): There is a hierarchical level of knowledge, from simple factual knowledge (e.g., a definition of a certain term) to a higher level of cognitive functioning (e.g., problem-solving and clinical decision making). The faculty usually specify the highest-level objective expected of the learner. Bloom’s Taxonomy describes the hierarchy of mental skills.
- Learner Objectives – Affective domain (“attitudinal” objectives): Affective domain is referred to as attitudes, values, beliefs, biases, emotions, or role expectations that can affect a learner’s learning or performance. Affective objectives are implicit in most educational programs in GME.
- Learner Objectives – Psychomotor domain (“skill” or “behavioral” objectives): Psychomotor domain considers specific psychomotor tasks or actions that may involve hand or body movements, vision, hearing, speech, or the sense of touch. Learner objectives for the psychomotor domain must indicate whether learners are expected simply to achieve the ability to perform the skill or to incorporate the skill into their continuing behavior.
- Process Objectives: In addition to learner objectives, process objectives, which indicate the degree of participation expected from the learners, can be added. (e.g., By the end of this academic year, 90% of PGY-3 residents will have participated in a critical incident root cause analysis.)
Miller’s Pyramid helps faculty identify learner objectives’ domains and prerequisite knowledge to achieve higher order thinking skills and autonomous clinical practice.
Structured Didactic Activities (Educational Strategies)
Flipped Classroom (UW CLIME): One of the strategies that could be valuable for GME is “flipped classroom.” In flipped classroom, residents read the materials (e.g., books, watching the instructional videos) at home, in advance of class. During the class, trainees work through problems, engage in collaborative learning, and demonstrate skills. The core of flipped classroom is to maximize the learning resource, time.
Strategies for cognitive objectives: relevant for ACGME Competency-Medical Knowledge [IV.B.1.c)]
Educational methods to achieve cognitive objectives include, but are not limited to, lectures, readings, programmed learning (programmed textbook/software that present material in organized sequential fashion), team-based learning (a combination of reading, testing, discussion, and collaboration to achieve higher order cognitive learning objectives), learning projects, problem-based learning, discussion, small group learning, frequent feedback on performance, and artificial models and simulation.
Lectures can be designed to be engaging and interactive. A combination of mini-lecture, think-pair-share, problem-solving exercise, case discussion, group discussion, and instructional video clips can be incorporated into the lecture. For help with this strategy, see the 2.5 hour video, “The ways and hows of active learning in a large classroom setting” (UW CLIME)
Strategies for affective objectives: relevant for ACGME Competency-Professionalism [IV.B.1.a)]
Techniques that promote openness, introspection, and reflection can help attitudinal change. Reflection on experience and reading, feedback on performance, role models, role-plays, artificial models and simulation, real-life experiences, discussion, small group learning, team-based learning, and standardized patients can be used to achieve affective objectives. Role-model health professionals can help change attitudes by demonstrating successful approaches to a particular problem.
Strategies for psychomotor objectives: relevant for ACGME Competency-Patient Care and Procedural Skills [IV.B.1.b)]; Interpersonal and Communication Skills [IV.B.1.e]
Methods commonly used to achieve psychomotor objectives include supervised clinical experience, simulations, artificial models, role-plays, standardized patients, and audio/visual review of skills. In order to ensure psychomotor objectives are met, it is critical to create safe and supportive learning environment where (1) faculty-residents rapport is nurtured, (2) explicit recognition and reinforcement of the resident’s strengths are provided, and (3) feedback on deficiencies are provided in a factual, nonjudgmental, and positive manner. When employing role-playing, faculty must alleviate residents’ potential discomforts by establishing ground rules.
Strategies to promote Practice-Based Learning and Improvement [CPR VI.B.1.d]
PBLI competency can be promoted through independent learning projects, individualized learning plans, learning portfolios, role modeling, training in teaching skills, and metacognitive skills (monitoring their own learning).
PBLI encourages self-directed learning, which means learners are active. These active learners can learn things more efficiently, retain that knowledge better, and use it more effectively than passive learners. In order for successful self-directed learning to take place, there should be sufficient protected time for the activity, clear communications of expectations to the learner, required products (e.g., presentations, reports), and sufficient mentoring and supervision.
Strategies to promote Systems-Based Practice [CPR IV.B.1.f]
This competency overlaps with the Interpersonal and Communications Skills competency [IV.B.1.e]. Methods that can be used to help develop Systems-Based Practice include inclusion of other health professionals on health care teams, providing feedback on costs of care, case conferences focused on cost-effectiveness and quality of care, opportunities to work in disease management programs, redesigning teaching services toward multidisciplinary integration, and participation in quality improvement and safety teams.
*Kern DE, Thomas PA, Hughes MT. Curriculum Development for Medical Education: A Six-Step Approach. Baltimore: The Johns Hopkins University Press; 2009.
- Comprehensive Assessment of Struggling Learners Referred to a Graduate Medical Education Remediation Program (JGME)
- Guiding Principles for Resident Remediation (Academic Emergency Medicine)
- Twelve Tips for Developing and Maintaining a Remediation Program in Medical Education (Medical Teacher)
- Learner Deficits and Academic Outcomes of Medical Students, Residents, Fellows, and Attending Physicians Referred to a Remediation Program, 2006-2012 (Academic Medicine)
- Getting More Done: Strategies to Increase Scholarly Productivity
- Harvest the Low-Hanging Fruit: Strategies for Submitting Educational Innovations for Publication (including the list of outlets for dissemination)
- O’Brien BC, Balmer DF, Maggio LA. Finding Our Way Through Shades of Gray: 6 Virtues to Guide Researchers in Planning, Conducting, Writing up Research. J Gad Med Educ. 2017; 9(5): 555-559.
- Wright S, O’Brien BC, Nimmon L, Law M, Mylopoulos M. Research Design Considerations. J Gad Med Educ. 2016; 8(1): 97-98.
UW Statistical Consulting Services
A free consulting service, provided by Biostatistics and Statistics, is offered during UW academic quarters (not during Final Exam week). This service is part of a graduate course for Biostatistics and Statistics in which the graduate students are the primary consultants under the guidance of faculty instructors.
Learn more and request a consult on the Departments of Biostatistics and Statistics website.
Writing for Educational Journals
|Work-in Progress Checklist for Education Papers: Checklist to guide the development process for educational scholarship papers.|
|Writing Steps Checklist: Checklist to guide the actual writing of the paper from reading the author guidelines to formulating the abstract; developing the introduction, methods, results, and discussion sections.|
|Education Research Resources: Resources on the following topics: “Getting Started,” “Curriculum Development,” “Qualitative Approaches,” “Surveys,” Writing and Reviewing.”|
|Courses, Medical Education Websites, Journals, Timelines: On Line Courses for Reviewing Skills (not specific to medical education); Websites (i.e., BEME, MedEdPortal); Education Journals to Consider—A list of approximately 10 medical education journals (e.g., Academic Medicine, JGME, Medical Teacher); Calendar for Education Abstract Submissions|
Tables and Figures
|TABLE 1. Modified Newcastle-Ottawa Scale – for quantitative studies|
|TABLE 2. Medical Education Research Quality Instrument – for quantitative studies|
|FIGURE. Kirkpatrick’s Levels of Evaluation|
|TABLE 3. Best Evidence in Medical Education Global Scale|
|TABLE 4. Grid for Critical Appraisal of Qualitative Research Articles – for qualitative studies|
Educational Research and Writing
Research Toolkit: A toolkit for health research in partnership with practices and communities
|INSTRUMENTS: SURVEYS, INTERVIEWS, OTHERS|
|Sullivan GM. A Primer on the Validity of Assessment Instruments. J Grad Med Educ. 2011; 3(2):119-120.|
|Rickards G, Magee M, Artino Jr AR. You Can’t Fix by Analysis What You’ve Spoiled by Design: Developing Survey Instruments and Collecting Validity Evidence. J Grad Med Educ. 2012;4(4): 407-410.|
|Magee C, Rickards G, Byars LA, Artino Jr AR. Tracing the Steps of Survey Design: A Graduate Medical Education Research Example. J Grad Med Educ. 2013;5(1):1-5.|
|Willis GB, Artino Jr AR. What Do Our Respondents Think We’re Asking? Using Cognitive Interviewing to Improve Medical Education Surveys. Journal of Graduate Medical Education: J Grad Med Educ. 2013;5(3):353-356.|
|Artina AR. AM Last Page: Avoiding Five Common Pitfalls of Survey Design. Acad Med. 2011; 86(10): 1327.|
|Sullivan GM. Deconstructing Quality in Education Research. J Grad Med Educ. 2011; 3(2):121-124.|
|Sullivan GM. Getting Off the “Gold Standard”: Randomized Controlled Trials and Education Research. J Grad Med Educ. 2011; 3(3):285-289.|
|Sullivan GM, Sargeant J. Qualities of Qualitative Research: Part I. J Grad Med Educ. 2011; 3(4):449-452.|
|Sargeant J. Qualitative Research Part II: Participants, Analysis, and Quality Assurance. J Grad Med Educ. 2012; 4(1):1-3|
|Sullivan GM, Feinn R. Using Effect Size—or Why the P Value Is Not Enough. J Grad Med Educ. 2012; 4(3):279-282.|
|Sullivan GM. FAQs About Effect Size. J Grad Med Educ. 2012; 4(3):283-284.|
|INSTITUTIONAL REVIEW BOARD (IRB) REVIEW|
|Sullivan GM. Education Research and Human Subject Protection: Crossing the IRB Quagmire. J Grad Med Educ. 2011;3(1):1-4.|
|Sullivan GM. IRB 101. J Grad Med Educ. 2011;3(1):5-6.|
|ON WRITING AND PUBLISHING|
|Sullivan GM. Writing Education Studies for Publication. J Grad Med Educ. 2012; 4(2):133-137.|
|Lypson M, Philibert I. Residents and Authorship: Rights, Obligations, and Avoiding the Pitfalls. J Grad Med Educ. 2012; 4(2):138-139.|
|Sullivan GM. Publishing Your Education Work in the Journal of Graduate Medical Education. J Grad Med Educ. 2010;2(4):493-495.|
|Simpson D, Meurer L, Braza D. Meeting the Scholarly Project Requirement–Application of Scholarship Criteria beyond Research. J Grad Med Educ. 2012; 4(1): 111-112.|
|Meurer LN, Diehr S. Community-Engaged Scholarship: Meeting Scholarly Project Requirements While Advancing Community Health. J Grad Med Educ. 2012; 4(3): 385-386.|