How Can We Make Program Evaluation Count?

By: Isabelle Lafleur, M.Sc.

In recent years, the evaluation of curricula and quality assurance have become increasingly important topics of interest.  In order to demonstrate that they reach the level of quality required, our programs generally go through two different quality processes: accreditation or institutional program evaluation or both.  However, is accreditation comparable to institutional program evaluation?  Even though the terms are often used interchangeably, there is a need to clarify both terms.    

Defining Quality

In order to answer the question of comparability, we must first define quality. The academic literature suggests that quality is not a unitary concept.¹  There are many significant challenges to defining quality since it is not static but rather dynamic.²  Defining quality is an important prerequisite for defining quality assurance.²  In education, it is more difficult to define quality than in industry sectors. This is mainly because the education process is very intricate and includes many elements that interrelate in a complex way, such as students, instructors, administrators, curriculum, teaching methods and assessment methods.¹  

Accreditation

The definition of accreditation is a process of quality assessment or quality audit resulting in a formal statement that the evaluated program or unit reaches or surpasses a certain level of quality.³  Accreditation is a review of the quality of higher education institutions and programs.⁴  An institution or program is granted accreditation for meeting minimum standards of quality.  The role of accrediting agencies addresses key issues of quality assurance in the accrediting review process.⁴  The standards are used by review committees as the basis for judgment and to make recommendations and decisions.⁴  The main objective of accreditation is to standardize the expectations of each program at the national level.  More recently, accrediting agencies have developed standards and procedures to guide institutions in the process of voluntary commitment to continuous improvement.⁴  

Like ACPE, the Canadian Council for Accreditation of Pharmacy Programs (CCAPP) grants accreditation to professional pharmacy programs that meet the standards set by CCAPP and to promote continued improvement of those educational programs.   CCAPP evaluates the programs on three major themes: academic program (i.e., educational outcomes, learning environment, and curriculum), resources (i.e., student services, human resources, practice site resources, financial resources, physical facilities and infrastructure as well as information resources),  as well as governance and program management (i.e. university structure and commitment, faculty organization and leadership, planning and evaluation, admissions, continuous quality assurance of the program).  The recommendations made by the Council relate to the achievement of the standards and tend towards national standardization.  Continuous improvement of the program is one of the standards in the accreditation process.  

Institutional Program Evaluation

Program evaluation examines information on an organizational unit or a study program in order to judge its quality.³  The evaluation process may involve various components, including evaluation of student learning or programmatic processes and investigation of the expected and unexpected outcomes.  Continuous improvement is central to institutional program evaluation.  

For example, at the Université de Montréal, all programs that are not targeted by an accreditation are required to go through an institutional program evaluation.  The program evaluation addresses five major themes: objectives and relevance of the program, student population (admission issues and student recruitment), academic program (program structure, training content, comprehensive approach to learning, teaching and assessment, faculty members and program management), resources (academic support, physical facilities and infrastructure, information resources and financial resources) and outcomes of the program (professional integration).  The recommendations made by the evaluation council are directly linked with the improvement of the program.  

How to make it count?

Accreditation and institutional program evaluation are two distinct ways to evaluate the quality of programs.  The aims of the two processes are different: standardization versus continuous improvement.  However, both approaches should lead to informed decisions based on regularly collected indicators.  Considering that these two processes require a large number of resources, commitment and energy, we must get data that is reliable, accurate and pertinent.  We need to build indicators that are useful not only to complete the accreditation or institutional program evaluation but, also, for decision making in our programs.  This is how we make program evaluation count.  In your school, have you developed innovative approaches to make program evaluation count?

Acknowledgements: Thank you to Dr. Ema Ferreira for her support and advice.

References:

1. Elassy N. The Concepts of quality, quality assurance and quality enhancement. Qual Assur Educ. 2015;23(3):250-261.

2. Welzant H, Schindler L, Puls-Elvidge S, Crawford L. Definitions of Quality in Higher Education: a synthesis of Literature. High Learn Res Commun. 2015;5(3):3-13.

3. Westerheijden DF. Walking towards a moving target: quality assurance in European Higner Education. Qual High Educ. 2005;2:52-71.

4. Ryan T. Quality Assurance in Higher Education : A Review of Literature. High Learn Res Commun. 2015;5(4):12. 

---

---

Pulses is a scholarly blog supported by Currents in Pharmacy Teaching and Learning