#38 Feedback: One More Time
Episode Host: Jonathan Sherbino
Episode article
Ryan, A., Judd, T., Swanson, D., Larsen, D. P., Elliott, S., Tzanetos, K., & Kulasegaram, K. (2020). Beyond right or wrong: More effective feedback for formative multiple-choice tests. Perspectives on medical education, 9(5), 307–313. https://doi.org/10.1007/s40037-020-00606-z
Enjoy listening to us at your preferred podcast player.
Feedback convention (dogma?) suggests that it should be given immediately. This episode examines that premise. Via an elegant experiment, the authors examine the effect of feedback on learning and the optimal timing of feedback. Spoiler alert: it’s an experiment, so you’ll need to squint/stretch to see how/if the findings change your teaching practice.
Episode Notes
Background
I think I should start with an apology. I vividly recall a blood oath we (the hosts) took to never review a paper on feedback again. I mean, we need to fight Big Feedback (akin to Big Oil, Big pharm and other “Bigs” that skew the power balance of any ecosystem and enforce a hegemonic paradigm… see Vario, I got a thesaurus for Christmas). Put another way.. what more is there to say about feedback. Do it with respect, do it to benefit the learner, do it in immediately… wait… maybe that last bit is incorrect. Enter friends of the show… with a new take on an old topic.
What is retrieval practice? Actively recalling information (often informally via self-quizzing) from memory, a process that promotes learning.
What is test-enhanced learning? A special case of the above, where a formal test is used to enhance memory by recalling the necessary information or identifying associations necessary to complete a question.
What is the educational effect of tests? A number of phenomena that results from preparing for and taking a test, including: test preparation (studying), and feedback on performance to enhanced metacognitive skills, such as guided self-assessment, among others.
So, back to the opening assumption. Is the old adage feedback should be immediate clearly supported in the literature? There is conflicting evidence on whether immediate or delayed feedback is beneficial. Perhaps the phenomena is a function of experimental versus real-world scenarios OR a function of item response( delayed feedback enhances a correct response by facilitating retention and transfer; immediate feedback corrects an incorrect knowledge or behaviour).
Purpose
“We investigated the effects of immediate and delayed feedback on both near and far transfer. This study examined transfer and retention at both an immediate post-test assessment, and a follow-up test conducted one week later… It was hypothesized that immediate feedback would improve error correction on near transfer items; while delayed feedback would improve the retention of learned concepts, resulting in enhanced far transfer performance on the immediate post-test and the one-week follow up test.”
Ryan et al 2020
Methods
A mixed, between-within subjects experiment was conducted on 41 second year Australian medical students, using a 18 question MCQ test.
Intervention:
For half of the questions feedback was provided after each question. For the second half of the test, feedback was provided at the end.
Testing:
Immediately post intervention, a 36-question retention test was performed using items requiring near and far transfer of information provided by feedback from the intervention.
Near transfer items differed via clinically superficial details (e.g., gender, age), while far transfer items differed via a different clinical presentation or clinical decision that related to a common concept.
A two-way ANOVA was conducted to evaluate group differences. A mixed effects model was constructed (fixed effects: feedback timing, transfer type, correct parent item response; random effects: item difficulty)
Results/Findings
Summary of the findings:
- Parent items were answered correctly half the time, on average..
- After testing, average scores improved to 69% for immediate feedback and 67% for delayed feedback.
- Item difficulty is a major contributor to response variance.
- Transfer type impacted accuracy. (i.e., lower accuracy on far transfer.)
- Successfully answering the parent item (intervention question) predicted success in transfer.
- Interestingly, the timing of feedback (immediate or delayed) had no effect on performance.
TL: DR The hypothesis was NOT supported.
ALTERNATIVELY Feedback improves learning, but don’t get too worried about doing it “immediately”. Also, knowledge decay doesn’t happen at one week.
Conclusions
From the authors: “The findings of this study suggest that the timing of feedback delivery … does not influence the efficacy of [test-enhanced learning] TEL in this medical education setting. We therefore suggest that educators may consider practical factors when determining appropriate TEL feedback timing in their setting.”
Paperclips
The initial funding for this study was received in 2017. That’s a long time from start to finish (study published in 2023), and these are accomplished researchers. A reminder for all of us about the value of perseverance in our own projects.
0 comments