Centre for Positive Psychology and Education

Professor Herb Marsh, University of Western Sydney & University of Oxford

Students' evaluations of teaching effectiveness (SETs) are of considerable interest, motivated in part by the traditional importance of teaching in universities. Interest also comes from an increasing emphasis on monitoring the quality of university teaching, and Assurance exercises in Higher Education.
It may be human nature to be suspicious on the tools used to evaluate one’s performances. Academic staff in particular are better trained to critique research than to teach. Hence, it is not surprising that SETs continue to be controversial. However, SETs are probably are the most thoroughly studied of all forms of personnel evaluation (with more than 10,000 published articles), and one of the best in terms of being supported by empirical research. Based on reviews of research by myself and others, SETs are:

• multidimensional;
• reliable and stable;
• primarily a function of the instructor who teaches a course rather than the course that is taught;
• relatively valid against a variety of indicators of effective teaching;
• relatively unaffected by a variety of variables hypothesized as potential biases, such as grading leniency, class size, workload and prior subject interest; and
• demonstrably useful in improving teaching effectiveness when coupled with appropriate consultation.

It is recommended that researchers reject a narrow criterion-related approach to validity and adopt a broad construct validation approach in which it is recognized that:

• effective teaching and SETs designed to reflect teaching effectiveness are multidimensional;
• no single criterion of effective teaching is sufficient; and
• tentative interpretations of relations with validity criteria and potential biases should be evaluated critically in different contexts, in relation to multiple criteria of effective teaching, and in relation to theory and existing knowledge.

Although SETs have a solid research base stemming largely from research conducted in the 1980s, it is surprising that research conducted in the last decade has not done more to address critical limitations previously identified and incorporate exciting methodological advances that are relevant to SET research. Perhaps the most damning observation is that most of the emphasis on the use of SETs is for personnel decisions rather than on improving teaching effectiveness. Why do universities continue to collect and disseminate potentially demoralising feedback to academics without more fully implementing programs to improve teaching effectiveness? Why is there not more SET research on how to enhance the usefulness of SETs as part of a program to improve university teaching? Why have there been so few intervention studies in the last decade that address the problems identified in reviews of this research conducted a decade ago? These, and other issues, are addressed in this Public Lecture.
 
Professor Herb Marsh (BA-Hons, Indiana; MA; PhD, UCLA; DSc, UWS; Aust Acad of Soc Sci; Brit Acad of Soc Sci) holds a joint appointment at the Centre for Positive Psychology and Education at the University of Western Sydney and at Oxford University. He is an “ISI highly cited researcher” (opens in a new window) with 842 publications 45,000+ citations, and a Google Scholar H-Index = 104. At UWS he has served as Dean of Graduate Research Studies, Pro-Vice-Chancellor of Research, and recipient of the inaugural Doctorate of Science and the inaugural Vice Chancellors award for PhD Supervision. He founded and Directs the SELF Research Centre that has 450 members and satellite centres at leading Universities around the world, and co-edits the SELF monograph series. He coined the phrase substantive-methodological research synergy which underpins his research efforts. His major Research/Scholarly interests include self-concept and motivational constructs; evaluations of teaching effectiveness; developmental psychology, quantitative analysis; value-added and contextual models; sports psychology; the peer review process; gender differences; peer support and anti-bullying.