#79 – Equality for all in assessment
Episode Host: Lara Varpio.
How can we make assessments truly equitable? In this episode, Lara dives into a paper that explores fairness, inclusion, and justice as three distinct approaches to equity in assessment. Discover how these orientations can reshape our goals, strategies, and impact in education. This episode unpacks critical frameworks that empower educators to critically reflect on and reimagine assessment systems.
Episode 79 transcript. Enjoy PapersPodcast as a versatile learning resource the way you prefer- read, translate, and explore!
Episode article
Anderson, H. L. K., Govaerts, M., Abdulla, L., Balmer, D. F., Busari, J. O., & West, D. C. (2024). “Clarifying and expanding equity in assessment by considering three orientations: Fairness, inclusion and justice”. Medical Education, Advance online publication.
Episode notes
In this episode, Lara explores a paper that redefines the conversation about equity in assessment. The authors present three distinct orientations—fairness, inclusion, and justice—each offering unique perspectives on what equity in assessment entails and how it can be achieved. By examining these orientations, we uncover how the language and approach we use shape the goals we pursue and the strategies we implement in education.
Background
Equity in assessment is an important topic that is increasingly being recognized in health professions education’s assessment circles. But equity in assessment means different things from different perspectives. The metaphor the authors use is of a mountainous landscape—where you stand on that landscape really changes the way you see it. The way you see the topic of equity in assessment makes a significant difference in terms of what you see as the aim of your efforts and what you consider to be appropriate actions to achieve those aims.
Purpose
The article introduces three lenses for understanding equity in assessment:
- Fairness
- Inclusion
- Justice
1. Fairness, 2. Inclusion, 3. Justice (Infographic made by PapersPodcast-Team)
1. Fairness-oriented assessment
Focus: Ensuring assessment systems are unbiased and provide equal opportunities for learners.
- Advantages:
- Well-established models and frameworks exist to address unfairness.
- Psychometric tools help identify and mitigate biases in assessment data.
- Disadvantages:
- Educators may not see the same issues learners face.
- Focuses on psychometric improvements without addressing deeper systemic inequities.
This orientation works from the premise that a set of values should guide assessment and that fairness is one of those values. Given that orientation, assessment systems should provide all learners with equal and appropriate opportunities to be assessed with unbiased assessments. This orientation asserts that unfairness can creep into our assessments via several factors (e.g., interpersonal bias in the assessment environment, biased measurements, etc).
Using the fairness orientation, scholars typically rely on psychometric methods to measure construct irrelevant variance—i.e., the variability in quantitative or qualitative assessment data due to extraneous factors that are NOT related to the construct being assessed.
There are established organizational models and frameworks for identifying factors and forces that contribute to unfairness and there are ways of mitigating that unfairness that have been published.
Educators and leaders bear the responsibility ensuring fairness which means that, while they might see some problems, they likely don’t see the same ones as learners do. Also, a fair orientation foregrounds makes the assessment systems more psychometrically sound, but the systems themselves can still contain forces and factors that are unequal and unfair. This orientation won’t help you address that.
2. Assessment for inclusion
Focus: Individualized assessments that prioritize diverse learners and adapt to their needs.
- Advantages:
- Emphasizes co-design with learners to identify barriers and create solutions.
- Disadvantages:
- Risk of “minority tax” for those involved in the co-design process.
- This may raise concerns about fairness for learners who do not require accommodations.
Assessment for inclusion starts with the premise that all learners have the right to appropriate, individualized education that supports learning and growth. It prioritizes the inclusion of all learners in assessment—it prioritizes participation, aims to minimize discriminatory practices and argues for creating adaptable supports for learners to demonstrate their abilities in their assessments.
A typical approach to address inequities using an assessment for inclusion lens is to make adaptations to evaluation via accommodations.
The goal is to create a system that accommodates a diversity of learners. It often uses a co-design process where learners, faculty, and other users collaborate to identify barriers to learning and assessment.
It risks imposing a minority tax on those who do the work of representing learner voices in the co-design process. And it can raise questions about the fairness for learners who don’t get the accommodations that some learners need.
3. Justice-Oriented Assessment
Focus: Addressing historical, cultural, social, and political inequities embedded in assessment systems.
- Advantages:
- Deeply transformative; focuses on rebuilding systems with marginalized groups leading the process.
- Disadvantages:
- Requires significant time, skill, and effort.
- Demands engagement with broader societal systems beyond education.
This orientation asserts that assessment is foundational to a historical, cultural, social, and political problem that is perpetuated in our educational systems. Therefore, equity needs more than just psychometric considerations or institutional strategies. Instead, a justice orientation demands that we critique assessment paradigms, constructs, and practices from the point of view of marginalized groups.
It aims to dismantle assessment systems and then rebuild them, having users—especially minoritized individuals—co-lead and co-control the redesign process.
There are many educators and researchers in the education field who have a lot of knowledge about how to navigate this work.
This work takes a lot of time and a lot of skill. Deconstructing the current assessment paradigms and creating a more just one is not a simple task—i.e., it requires grappling with social, cultural, and political systems that exist outside of education contexts.
Conclusions
The authors propose three actionable strategies to advance equity in assessment:
- Create clarity about the orientation of your efforts. This has impact on the methods you use, the outcomes to expect, and the people to involve in the process.
- Be intentional when choosing an orientation. The authors recognize that using an orientation means that you also have to deal with the limitations, tensions, and assumptions of that orientation
- Expand your orientations. We use a fairness orientation often, but that orientation leaves a LOT of problems with equity still in place.
Transcript of Episode 79
This transcript is made by an autogenerated text tool and some manual editing by the Papers Podcast team. Read more under “Acknowledgment”.
Jason Frank, Lara Varpio, Linda Snell, Jonathan Sherbino.
Acknowledgment
This transcript was generated using machine transcription technology, followed by manual editing for accuracy and clarity. While we strive for precision, there may be minor discrepancies between the spoken content and the text. We appreciate your understanding and encourage you to refer to the original podcast for the most accurate context.
0 comments