#15 Digitization & Despair: Two Cases of Complex Change 

Host: Jason Frank

Enjoy listening to us at your preferred podcast player.

Apple    Google    Spotify    Spreaker

Episode article

Humphrey-Murto, S., Makus, D., Moore, S., Watanabe Duffy, K., Maniate, J., Scowcroft, K., Buba, M., & Rangel, J. C. (2023). Training physicians and residents for the use of Electronic Health Records—A comparative case study between two hospitals. Medical Education, 57(4), 337–348. https://doi.org/10.1111/medu.14944

Openspotify

In this episode, Jason introduces a paper that uses case study methodology to look at complex educational interventions, and why so often they fail

Background 

What is it about big, complex, educational interventions that make them so often fail? Sometimes, I am obsessed with and perplexed by this problem. New curricula, new teaching, new content, new settings, new assessments…All these are often set up to fail.  

Another example of this is new technology, like a new e-portfolio or a new electronic health record (EHR). These two are notorious and dreaded phenomena.  

I have PTSD from my experience. I am still pondering the lessons learned from implementing the EHR education in our hospital, and what I would do differently for a future complex educational intervention.  

Enter, Dr. Humphrey-Murto et al. (Caveat: I work in the same institution as several of the authors and the first author is a friend…But she doesn’t know I am reviewing this paper…Yet.) 

Purpose 

The authors pitched that there is a gap in our understanding of how physicians are trained for EHR implementation, and this is holding back the world achieving EHR benefits. The authors set out to “describe and critically assess the interplay between educational activities and organizational factors that influenced EHR training and implementation across two different hospitals”.  

Methods

Have you ever used case study methodology? I had to take dive into the shallow end of my methods sources to look for descriptions of use and rigour (see text box).

The authors performed a comparative qualitative case study using Singh & Sittig’s Socio-technical model of studying health information technology in complex adaptive healthcare systems. The latter has 8 elements:

  1. Hardware & software 
  2. Clinical content
  3. Human-computer interface
  4. People 
  5. Workflows 
  6. Organizational  
  7. External rules 
  8. System monitoring 

Case study methodology

  1. Allows in-depth explorationsof complex phenomena in real life settings
  2. Aka “naturalistic designs
  3. Key authors Stake (1995) and Yin (2009)
  4. Intrinsic Case studies: a unique phenomenon
  5. Instrumental: a typical phonomenon
  6. Collective: multiple instances studied togehter
  7. Case studies can be used to explain, describe or explore a phenomenon in context
  8. Can help develop or refine theories
  9. Can be done through variety of epistemological lenses

They compared the implementation of the same EHR software at 2 teaching hospitals. The adult hospital chose “big bang” all-at-once implementation in 1 day, and the pediatric hospital chose “bit-by-bit” over 8 years(!). 

They performed 1:1 interviews remotely with a purposive sample of physicians, residents, students and administrators. They used standard methods of transcription, coding for themes, and group interpretation. They were not allowed to review proprietary training materials for the study.  

The study had research ethics board (REB) approval and they provided a table as their reflexivity statement.  

Results/Findings 

43 participants were interviewed until they “achieved sufficient information power”. There were no consistent differences between the 2 sites, so the data was analyzed in aggregate. 

Some notable highlights: 

  • Training provided was too general 
  • There was limited transferability to daily work 
  • Most learning was on the job and from peers, despite significant investment in corporate support (1200 trainers at the adult site!) 
  • Majority of physicians felt unprepared to work 
  • Majority of end-users felt strong negative emotional reactions (e.g. made to feel more incompetent) 
  • Clinical workflows were often completely disrupted, and users felt blind-sided by this 
  • Users employed workarounds 
  • Focus was on computers > patients 
  • Training was inadequate before, during, and after implementation
  • Corporate workforce transformation was needed.  

Conclusions 

The authors concluded that implementation of major complex projects like an EHR is not a software project, but a socio-technical organizational transformation. They provided recommendations on how to alleviate the significant impacts.  

Comments

This paper highlights both case study methods and has lessons for big complex educational interventions 

References

Crowe, S., Cresswell, K., Robertson, A., Huby, G., Avery, A., & Sheikh, A. (2011). The case study approach. BMC Medical Research Methodology, 11(1), 100. https://doi.org/10.1186/1471-2288-11-100

Malterud, K., Siersma, V. D., & Guassora, A. D. (2016). Sample Size in Qualitative Interview Studies: Guided by Information Power. Qualitative Health Research, 26(13), 1753–1760. https://doi.org/10.1177/1049732315617444

Singh, H., & Sittig, D. F. (2020). A Sociotechnical Framework for Safety-Related Electronic Health Record Research Reporting: The SAFER Reporting Framework. Annals of Internal Medicine, 172(11_Supplement), S92–S100. https://doi.org/10.7326/M19-0879

Stake, R. E. (1995). The art of case study research. Sage Publications.

0 comments

Leave a Comment

Related posts

3 latest episodes

Categories