Taking the Reins in Describing the Impact of Educational Scholarship

By: Simon Albon, PhD; Janet Cooley, PharmD, BCACP; Kristin Janke PhD

When proving the impact and value of our teaching and learning work, publications still seem to be the gold standard.  Initiatives like the #RxWritingChallenge have experienced phenomenal growth from pharmacy academics worldwide, as a means to support writing and publication.  While we wholeheartedly support the importance of becoming  better academic writers and disseminating our work as publications, there seem to be few alternatives for describing the impact of our educational work.  Traditional approaches to assessing impact may be limiting.  Therefore, we need to broaden our options for expressing the power and influence of educational scholarship.

Traditional Approaches to Impact
In educational scholarship, we have largely continued to examine contributions in the same way that traditional research has – where grants, publications, and citations are the predominant currency of impact and value. The journal impact factor and eigenfactor are journal-level scores that are seen as indicators of importance and prominence.  The h-index and g-index are author-level metrics based on the citation.  Promotion and Tenure committees may count publications, request author-level metrics, and weigh the relative prominence of journals in examining a faculty member’s work.  In addition, there is a growing sense that scholars need to be involved in the post-publication dissemination of their written works facilitated by social media and supported by altimetrics.¹ However, these are not the only measures of impact in educational scholarship.

Inherent in these approaches is an assumption that, if you publish, your work is being read.  Yet, research suggests this may not be the case. Meho for example, has suggested that up to 90% of papers published have never been read and 50% of papers have never been read by anyone other than their authors, referees, and journal editors.²  In their analysis of citation mistakes and modeling, Simkin and Roychowdhury found evidence that, even if your published work is cited, it does not mean the authors have read your paper.³  Yet, the number, weight and authorship order of publications is often emphasized.  To assist in understanding the broader use and influence of a scholar’s work, additional approaches beyond publication are needed.

Exploring Additional Forms of Impact

By definition, scholarship in teaching and learning must be made public and be open to review.⁴ However, there are various ways to meet these criteria.  Yes, publication is part of this picture, but what about the presentations we do within our pharmacy schools, or our  influence on the teaching practices of our colleagues? Clearly, publication alone does not adequately represent the scope of the impact of our educational work.  The academy (including promotion, tenure and career advancement policies) needs to recognize the value and breadth of processes and products involved in the dissemination of educational scholarship.  Figure 1 is a brainstorm of the ways our educational scholarship might impact the quality of our pharmacy programs.  What do you see? There are many paths and possible outcomes.

Figure 1. A Few Examples of the Possible Impact of Educational Scholarship

In addition, we need to collectively develop new ways of describing the impact of our work.   Developed by the Center for Teaching, Learning, and Technology at the University of British Columbia, the Teaching and Learning Impact Framework (TLIF) is one method for exploring impact.  It encourages us to examine the people, processes, and products related to educational inquiry, examining scale, scope, and contributions.

Taking the Reins as an Academy

Publication alone is not enough to represent the richness of our scholarship in education.  We need robust, and inclusive approaches to describing our impact that go beyond publication metrics.  As scholars, we bring unique talents and energies to our teaching and learning inquiry and the impact of this work is diverse.  We must recognize and continue to develop means of making our work publicly available and open to review by our peers.  We need models and frameworks for telling our stories of impact that include evidence to support our claims of impact.  We need methods to describe our work beyond our specialties.  Consider how powerful it would be to present your work as an impact story, where you describe your work’s impact with both quantitative and qualitative metrics, as well as how the work meets the mission of your organization or is developing into future projects.⁵   As an academy, we need to experiment with methods to represent impact, including visual mapping (Figure 1), and the best methods for sharing impact stories. This is no small task but we think the academy is up for it!  

So, go ahead, grab the reins and take control of your story and your impact.  Learn about the different metrics, consider the work that lies outside of these metrics and draft a paragraph that tells the story of your research. 

Disclosures

Kristin Janke is Executive Associate Editor for Currents in Pharmacy Teaching and Learning, an Editor and Peer Coach for CPTL Pulses, and an Editor for Innovations in Pharmacy where she is responsible for the education section.  Simon Albon and Janet Cooley are Editorial Board Members at Innovations in Pharmacy.

Acknowledgments

Ideas from this post were first presented as part of a presentation by the same name at the 2020 American Association of Colleges of Pharmacy Annual Meeting.

References

1. Green T. Maximizing dissemination and engaging readers: The other 50% of an author’s day: A case study. Learn Publ. 2019; 32: 395-405.
2. Meho L. The Rise and Rise of Citation Analysis. Phys. World. 2007; 20:1.
3. Simkin MV and Roychowdhury VP. Read Before You Cite! Complex Syst. 2003; 14: 269-274.
4. Shulman, LS. Taking Learning Seriously. Change. 1999; 31(4): 10-17.
5. Friesen F, Baker L, Ziegler C, Dionne A, Ng S. Approaching Impact Meaningfully in Medical Education Research. Acad Med. 2019; 94(7): 955-961.

---

Author Bio(s):

---

Pulses is a scholarly blog supported by Currents in Pharmacy Teaching and Learning