The PAPERs Podcast

Episode host: Lara Varpio.

Listening to this episode would provide valuable insights into competency-based medical education (CBME) and the complexities surrounding it. The hosts discuss a paper that reports on the conversations in the literature about CBME. The history of those conversations and what they hold for the future of CBME is highlighted.

Episode notes

The following breakdown examines the key components of the study’s methodology and highlights the findings.

Background

I recognize that just a few episodes ago we did a paper that talked about the congruences and incongruences between our conceptualizations of CBME and professional identity formation. That conversation got a little spicy, and I got some emails from listeners about that discussion. Some of those emails were celebrating some of our comments. Others were a little less glowing. 

Looking across those emails and across the literature on CBME that I have in my files as “important” publications on the topic, I came to appreciate that the literature about CBME isn’t always consistently working off the same assumptions. Different authors in different contexts frame CBME in different ways. I decided that I wanted to find a paper that would help me to appreciate the diversity of the discourse in our field around CBME and to consider the future of CBME in light of that diversity.  

---

Reflection

What are some important turning points in the history of CBME that got us where we are today? What has gone well? What is misunderstood? What frustrates you?

Jason: “I’m really immersed in this subject! It’s my area of scholarly interest and I’ve been doing it for 30 years. That’s all my gray hair.” Jason complexities and challenges surrounding competency-based medical education (CBME) discussions. There is a the cacophony within these discussions, with the lack of agreement on fundamental definitions and topics even among professionals globally. This highlights a significant obstacle in advancing CBME initiatives effectively, as the absence of consensus can lead to misunderstandings, miscommunication, and fragmented efforts

Linda: recalls a pivotal moment in CBME history where a group of clinician educators convened to overhaul residency education, highlighting the enthusiasm followed by ups and downs in adoption. She compare the journey of CBME adoption to the Gartner hype cycle, noting phases of initial enthusiasm, followed by challenges and grudging acceptance. Acknowledging the ongoing frustrations and complexities in implementing CBME, they emphasize the difficulty of change and the need for continued refinement and evaluation.

Jon: Emphasizes the value and principles of competency-based medical education (CBME), having been part of the initial cohort pushing for its adoption. Jon talks about viewing CBME as a complex educational bundle rather than a singular intervention, highlighting the need for precision in defining its components. Reflecting on the past, they recall the problematic nature of the health system and how CBME emerged as a response to address issues of power, inflexibility, lack of accountability, and creativity. However, there is concerns about the implementation and evolution of CBME, suggesting a lack of attention to continuous improvement and outcome evaluation, and a tendency towards labeling rather than focusing on desired outcomes and refinement.

---

Today’s paper works from a simple premise: CBME is a concept that dates back to the 1970s (and so has a long history) and it has been broadly adopted in many different countries (and so there is a really massive literature on the topic). Within that literature, there are many conversations happening and not all of them are working from the same premises. Some conversations are dominant, others are ignored. Some conceputalizations are foregrounded despite the fact that other conceptualizations might be better. This paper argues that we need to understand the lines of conversation within the CBME literature so that we can see if our conversations are helping us achieve CBME’s goals. 

Method 

To identify, explore and map the conversations in the CBME literature, the authors conducted a scoping review, following the 5 steps of Arksey and O’Malley’s framework and incorporating the 6th / optional step suggested by Levac. 

Some methods highlights that I thought were pretty amazing: 

1. In keeping with the underpinning philosophies of scoping reviews, the authors cast a wide net to find a rich breadth of literature on CBME. They included the usual databases you’d expect (like Web of Science) but also some more obscure resources like Proquest News and Newspapers and Proquest Dissertation and Theses Global. And they did a hand search of the first 10 pages of hits in Google Scholar.  

2. They adhered to the PRISMA extension for scoping reviews 

3. They created heat maps to illustrate the frequency of different conversations over time.   

4. They told me their paradigmatic orientation. (love / smile emoji) 

Results/Findings

There were 387 articles in their corpus of included manuscripts.  

A few notable demographic pieces:  

• 74% of the articles in the corpus were published between 2010-2019. And 7% were published in the first few months of 2020. A hot topic in the recent years 
• In the corpus, 71% of the articles had a lead author from North America 

The authors identified 8 conversation themes across the corpus of articles. Please go to Table 2 in the paper that defines each theme AND gives examples of how the theme was manifested in the corpus. It is worth digging it up!!

The 8 themes are:

• Credibility: This conversation is about how CBME is aligned with educational theories, best practices. It is the literature that gives argues that CBME works.
• Application: This conversation is about pragmatically applying CBME in different contexts, the potential to assess it.
• Community influence: This is about how people in the local context shape CBME.
• Learner impact: You guess it, this is about how CBME impacts on learners’ attitudes and experiences.
• Assessment: The name says it all. This conversation is all about assessing CBME interventions.
• Educational developments:Yes, this is about the interventions needed to bring CBME to life in a curriculum.
• Organizational structures: This conversation discusses how institutions impact CBME implementation or how CBME impacts the institution.
• Societal impacts: This is about how society and patients might be impacted by CBME’s implementation.

Then the authors mapped those themes across time so we can see how conversations became more / less important overtime. AND they also divided the topics into perceived advantages, perceived disadvantages, and recommendations. They illustrated all that into a heat map — SO creative! 

In the early literature, pre 1990s, CBME was described has having real advantages for learners. It promised personalized learning with transparent expectations. By the 1990s, the conversations about CBME’s advantages for learners was a center stage item in the literature. It was about shared missions and common goals between faculty and learners. This continued into the 2000-2009 articles. The conversation about learner impact still focused  primarily on advantages, now including topics like how CBME supported more efficient training, with more possibilities for sub-specialty training and research experience. But this is when we get the first hint that there are disadvantages for learners, with conversations about how CBME might hinder professional identity formation for learners. When we hit the 2010-2019 literature, the conversation is shifting now to be about how learners and faculty have a shared responsibility for CBME’s success. And more concerns are being voiced, including for example, that the context in which the learner trains shapes how competency is defined and assessed. We have the first set of learner focused conversations around recommendations that learners need to develop a growth mindset for CBME to be effective. By 2020, a number of learner associated disadvantages are being raised, including the anxiety that learners can have about having to ask for many assessments or for feedback. 

It makes sense that learner perceptions would take time to be identified and then to have them inform the conversation about CBME. I think that this conversation is going to be increasingly important for CBME’s continued success. The conversation about learner impact is showing us that CBME has a dark side — while there are advantages for learners, there are disadvantages for them too. They need to have a growth mindset. We place them in a difficult power relation with faculty when they have to ask for evaluations. This conversation is going to become more and more important as we think about learner’s experiences of shame and of professional identity struggles. CBME is part of that. We’re gonna have to think of solutions to make sure that learners reap the advantages that CBME promises.  

Comments

So, what is the future of CBME? We’ve discussed some of the conversations that are interesting to us and where they might go. Are there other things you’d like to highlight about CBME? Will it ever live up to expectations? Is it doomed to never meet the grand aspirations we laid out for it? Should we keep investing in it? Or are we now at the point of diminishing returns? 

References

Articles mentioned or can be helpful:

Englander, R., Frank, J. R., Carraccio, C., Sherbino, J., Ross, S., Snell, L., & ICBME Collaborators (2017). Toward a shared language for competency-based medical education. Medical teacher, 39(6), 582–587.

One of original articles on CBME with definitions:

Frank, J. R., Snell, L. S., Cate, O. T., Holmboe, E. S., Carraccio, C., Swing, S. R., Harris, P., Glasgow, N. J., Campbell, C., Dath, D., Harden, R. M., Iobst, W., Long, D. M., Mungroo, R., Richardson, D. L., Sherbino, J., Silver, I., Taber, S., Talbot, M., & Harris, K. A. (2010). Competency-based medical education: theory to practice. Medical teacher, 32(8), 638–645.

The framework article that Jon mentions in the episode:

Van Melle, E., Frank, J. R., Holmboe, E. S., Dagnone, D., Stockley, D., Sherbino, J., & International Competency-based Medical Education Collaborators (2019). A Core Components Framework for Evaluating Implementation of Competency-Based Medical Education Programs. Academic medicine : journal of the Association of American Medical Colleges, 94(7), 1002–1009.