Teaching Excellence in Public Health | December 19th, 2018
Update | September 23, 2019
The University of Miami Department of Public Health Sciences sought to improve its peer review processes by setting out in 2018 to explore the state-of-the-art practices in accredited academic public health institutions. See the preliminary observations and sample measures below, as gathered in 2018 and validated in 2019. This research led to the finding that within schools of public health, individual departments use many different methods of peer evaluation and a wide variety of tools to measure their observations. And while some schools have a set of recommended instruments that they make available, faculty do not, necessarily, use the suggested tools.
As a result of this study, Miami adopted a new three-stage process for peer review that includes a:
- Pre-observation meeting
- Class observation
- Post-observation meeting.
The new form includes sections for these meetings together with the domains assessed and corresponding Likert scale. The form also includes guiding questions for each of the meetings.
The Miami public health program is small and the process is quite demanding for the participating faculty. Some find it a challenge to coordinate meeting times – so some of the pre-observation and post-observation meetings have occurred over the phone. Below is the new tool. Comments and ideas are welcomed to: Viviana Horigian, MD, Director of Public Health Education, University of Miami Department of Public Health Sciences, vhorigian@med.miami.edu.
Preliminary Observations and Sample Measures
Peer review of teaching is a process destined to improve instructional effectiveness of faculty, and constituted part of the instructional mentorship and development. In some instances, schools/programs use summative evaluations to inform personnel decisions. Broadly, the process usually entails a preliminary interview with the teaching faculty, class observation and a post observation meeting summarizing strengths and weaknesses. Class observations are directed to evaluate knowledge, instructional materials, class organization presentation form and substance, teacher and student interaction, student participation and assessment practices. Time and effort, potential bias and collegiality issues typically limit peer review of teaching. Nonetheless, this process remains one of the methods to improve the quality of instruction.
The following summary table presents information retrieved from ASPPH Academic Affairs members. Members were requested information on peer review of teaching, with a focus on both process and observation tools used. 12 schools/departments responded to the request. The table below summarizes processes and tools used as reported by these 12 members.
Preliminary Observations:
While some schools did not specify if the process was used only for formative purposes, data from the ASPPH report Innovations on Pedagogy survey (report here) indicates that 33 % of respondents (n=87) use peer reviews as a method for evaluation for promotion and tenure. This report also highlights that peer review of teaching is available in 47% of the schools/programs and reports that 37% of respondents find peer review very beneficial.
Colorado School of Public Health - Department of Epidemiology
Process used | 1. a. Pre-class review of syllabus for: appropriateness, clarity of goals, measurable learning objectives, planned activities and appropriate methods, cultural relevance and high expectations b. Pre-conference meeting to understand goals and expectations2. In-class observation3. Summary of observations – areas of strength, post-observation meeting and recommendation |
Tool used | N/S* |
Domains assessed as part of the observation | In-class observation
Form includes statements and invites for suggestions:
|
Final purpose | Class observation notes and post-observation go to the instructor and are shared with departmental chair or committee.
Importance of emphasizing the constructive nature of the observation |
Colorado School of Public Health - Department of Epidemiology (online courses)
Process used | Online evaluation form |
Tool used | N/S |
Domains assessed as part of the observation | Feasibility of finding and accessing the information (syllabus, calendar of assignments, courses, quizzes, examinations, discussion, requirements of synchronous or asynchronous courses) |
Final purpose | N/S |
Colorado School of Public Health – Department of Environmental Sciences
Process used | Peer observation |
Tool used | N/S |
Domains assessed as part of the observation | Open-ended evaluation on instructors command of material, clarity of presentation, apparent quality of preparation, effectiveness of their use of teaching aids (PowerPoint, whiteboard, overhead, etc.), encouragement and use of student feedback during lectures and discussions, additional suggestions/comments about the course and/or instructor |
Final purpose | N/S |
University of North Carolina Gillings School of Global Public Health
Process used | 3 step:
1. Pre-observation conference – with interview guide focused on goal and expectation of class, students, and teaching style 2. Classroom observation 3. Post-observation conference – with interview guide focused on reflections, including areas for improvement |
Tool used | Adapted from Sorcinelli Observation Guide |
Domains assessed as part of the observation | Open-ended questions
|
Final purpose | The forms are a guide and each department within the school uses different guides. The form adapted from Sorcinelli represents one of the forms in use.
Peer observation of teaching is a requirement for APT. |
Yale School of Public Health
Process used | In-class behavioral assessment of what the instructor and student are doing each minute from start to finish of the session |
Tool used | Modified version of Classroom Observation Protocol for Undergraduate STEM (COPUS) (Smith MK, Jones FHM, Gilbert SL, and Wieman CE. 2013. The Classroom Observation Protocol for Undergraduate STEM (COPUS): a New Instrument to Characterize University STEM Classroom Practices. CBE‐Life Sciences Education, Vol 12(4), pp. 618‐627) |
Domains assessed as part of the observation | Every two minutes student and instructor behaviors should be checked off under the following categories:
|
Final purpose | Class observation and subsequent consultation are shared only with the instructor for formative purposes. |
Emory University Rollins School of Public Health
Process used | In-class observation |
Tool used | Office of Evidence Based Learning, Department of Behavioral Sciences and Health Education, Rollins School of Public Health |
Domains assessed as part of the observation | 3 areas evaluated in a Likert scale (yes, somewhat, no , N/A)
|
Final purpose | The use of the form is not mandatory. It is used only upon request by teaching faculty who want to demonstrate professional development in the area of teaching as part of the APT packet. |
Penn State Department of Public Health Sciences (online)
Process used | Online assessment for online courses |
Tool used | Adapted from Penn State |
Domains assessed as part of the observation | Availability/evidence of good practices on: contact between students and faculty, reciprocity among students, active learning, prompt feedback, time to task, high expectations, diverse ways of learning |
Final purpose | N/S |
Penn State Department of Public Health Sciences
Process used | Peer Review Activity Guide on process and tools including a first phase for selection and scheduling of peer observations
1. Pre observation: lesson plan, teaching style, focus of observation instructor response to student evaluations 2. Review of student evals and syllabus 3. Class observation 4. Post observation 5. Evaluation: discussion and improvement plan |
Tool used | Penn State |
Domains assessed as part of the observation | Likert scale
|
Final purpose | Appears to affect promotion and tenure |
University of Arizona Mel and Enid Zuckerman College of Public Health
Process used | Observation of teaching
Observation tool Documentation assisted by the observation template |
Tool used | Adapted from Baskamp and University of Minnesota |
Domains assessed as part of the observation | Organized in 8 areas, but only 20 items within these areas are selected for observation
Likert scale on 4 points (very evident, evident mostly, evident during a portion of the class, not evident at all)
|
Final purpose | N/S |
University of Minnesota School of Public Health
Process used | 1. Observation of teaching
2. Peer review of class assignments and assessments: 4 p Likert scales 3. Peer review of examples of student performance: 4 p Likert scale (appears to be overall and includes grade distribution) 4. Peer review of syllabus |
Tool used | Peer Observation of Teaching protocol |
Domains assessed as part of the observation | Observation of Teaching – open-ended Context or Background – settingObservation areas:1. Instructor goals2. Significance of class activities3. Student engagement4. Examination of student achievement goalsBest practices: Assessments |
Final purpose | N/S |
Albany State University of New York School of Public Health
Process used | Peer observation checklist |
Tool used | Peer observation template |
Domains assessed as part of the observation | Observation – all based on instructor behavior
Check off if observed or not and provide comments on the following areas:
|
Final purpose | The form is only a guide suggested by the university’s teaching center. Departments use it at their discretion.
Peer observation of teaching is expected by departments and it is part of the promotion process. |
Thomas Jefferson University College of Population Health
Process used | Observation |
Tool used | Kent State College of Public Health – evaluation form |
Domains assessed as part of the observation | 5 point Likert scale (from excellent to poor)
|
Final purpose | N/S |
University of Maryland School of Public Health
Process used | Process:
1. Pre-class meeting – course goals, strategies, and questions to get feedback on 2. Classroom – at least one, but more if it has multiple components observation 3. Post-observation meeting – discussion to enhance teaching effectiveness student engagement, course efficiency, list of questions to guide the discussion (around strengths and weaknesses) 4. Synthesis and documentation both the instructor and the observer can prepare a summary to reflect on the three meetings |
Tool used | Peer Teaching Observation Guide |
Domains assessed as part of the observation | Class observation ranked in Likert scale (yes, mostly, somewhat, no)
|
Final purpose | The Peer Teaching Observation Guide is only a guide for evaluation of teaching.
Peer observation of teaching is a requirement for promotion/APT. |
University of Miami Department of Public Health Sciences
Process used | Observation, post-observation meeting, written summary |
Tool used | Teaching Observation Form |
Domains assessed as part of the observation | 6 point Likert scale (improvement necessary, effective, highly effective)
|
Final purpose | For teaching development only |
Stony Brook University Program in Public Health (no form provided)
Process used | 1. Student self-assessment of competency attainment (pre-course vs. post-course assessment)
2. Student end-of-semester course evaluation data 3. Student focus group data at the end of each course 4. Program director observation of teaching 5. Curriculum committee review of data collected (1-4 above) |
Tool used | N/S |
Domains assessed as part of the observation | Course content (meeting competencies)
|
Final purpose | 1. Data/feedback are shared with instructors to enhance/improve teaching
2. Program Director uses data to formulate comments about teaching for faculty promotion support letter |
*N/S: not specified
Compiled by Viviana E. Horigian, M.D. University of Miami, Department of Public Health Sciences.