Student Perceptions of High Course Workloads are Not Associated with Poor Student Evaluations of Instructor Performance
This study showed that student ratings of relative course workload were not correlated with student ratings of overall instructor performance in engineering, math and science, or humanities courses.
Some faculty believe that if a course is perceived by students to have a high workload, students rate that course (and the performance of the instructor) poorly. This belief persists even though previously-published studies (e.g., [1-3]) have shown little to no relationship between student ratings of course workload and instructor performance. Information about how students rate engineering courses in comparison to other types of courses, and about how students rate engineering courses at different types of institutions, might help faculty apply research findings to their own situations.
In this study, data from two different types of institutions (large, research-oriented; small, undergraduate education-oriented) and multiple types of courses (engineering, math and science, and humanities) were used to investigate whether student ratings of workload or of other course evaluation items were correlated with ratings of instructor performance.
Course-averaged evaluation scores for all classes offered at Rose-Hulman Institute of Technology during the 2004/2005 academic year were obtained from the Rose-Hulman Office of Institutional Research, Planning, and Assessment (490 engineering, 390 math and science, and 165 humanities courses). A second data set consisted of course-averaged evaluation scores for all classes offered through the Tulane University School of Engineering from the Fall of 1997 to the Fall of 2002 (823 courses). Nonparametric Spearman’s rho correlation coefficients were calculated for both data sets. Selected correlation coefficients were quantitatively compared using Fisher’s Z statistic [5]. Data from Rose-Hulman engineering courses were sorted into quartiles according to numerical ratings of overall instructor performance. The mean scores on each evaluation item from courses in the highest and lowest quartiles (e.g., viewed by students as the best- and worst-taught courses) were compared using the Mann-Whitney test [5].
Rose-Hulman student ratings of overall instructor performance were neither linearly nor quadratically related to ratings of course workload in relation to other courses of equal credit, in engineering, math and science, or humanities courses (rho values of 0.068, 0.054, and 0.079, respectively). Courses ranked in the lowest quartile (poorest ratings of overall instructor performance) received different (p < 0.05) mean ratings from highest-quartile courses on all evaluation items except for the items related to relative course workload and to the pace of the course material.
Rose-Hulman evaluation items most strongly correlated with overall instructor performance ratings tended to focus on the professor’s teaching/presentation methods, preparation, sensitivity to students and interest in the subject, and overall learning experience: for example, “The professor used teaching methods that helped me learn,” (rho = 0.894); “The professor met the stated course objectives,” (rho = 0.873); and “The professor generally was well-prepared for class” (rho = 0.797). Similar results were obtained from the Tulane data, in which items most strongly correlated with overall instructor performance ratings tended to focus on the professor’s teaching/presentation methods, interest in teaching and students, and overall learning experience. None of the correlation coefficients calculated using information from engineering courses were significantly (p < 0.05) different from coefficients calculated for the same evaluation item using information from math, science, and humanities courses.
The very small (rho = 0.068) correlation between student ratings of overall instructor performance and of engineering course workload was in agreement with previous investigations [1-3, 6], and is too small to be practically useful when determining reasons for poor evaluations or guiding efforts for improvement. Discriminating between work viewed by students as valuable to learning and work viewed as excessive or unneeded may yield different results [7]. This study tested and found no support for the simple hypothesis that students perceive high workloads as “bad” and low workloads as “good” and bias course/teaching evaluations accordingly. Faculty across academic disciplines and campus cultures seeking improved evaluations might instead focus on teaching methods that help students stay attentive and learn, on organization and preparation, and on teacher/student interactions.
[1] Marsh, H.W., "Students' Evaluations of University Teaching: Dimensionality, Reliability, Validity, Potential Biases, and Utility," Journal of Educational Psychology, Vol. 76, No. 5, 1984, pp. 707-754.
[2] Sixbury, G.R., and W.E. Cashin, "Description of Database for the Idea Diagnostic Form," IDEA Technical Report No. 9, 1995, Center for Faculty Evaluation & Development, Kansas State University, Manhattan, Kansas;
[3] Centra, J.A., "Will Teachers Receive Higher Student Evaluations by Giving Higher Grades and Less Course Work?," Research in Higher Education, Vol. 44, No. 5, 2003, pp. 495-518.
[4] Glantz, S.A., Primer of Biostatistics, 4th ed., New York, New York: McGraw-Hill Health Professions Division, 1997.
[5] Mickey, R.M., Dunn, O.J., and V.A. Clark, Applied Statistics: Analysis of Variance and Regression, 3rd ed., Hoboken, New Jersey: John Wiley & Sons, Inc., 2004.
[6] Dee, K.C., "Reducing the Workload in Your Class Won't "Buy" You Better Teaching Evaluation Scores: Re-Refutation of a Persistent Myth," Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition, Salt Lake City, Utah: American Society for Engineering Education, 2004, Session 1331.
[7] Marsh, H.W., "Distinguishing between Good (Useful) and Bad Workloads on Students' Evaluations of Teaching," American Educational Research Journal, Vol. 38, No. 1, 2001, pp. 183-212.
Author 1: Kay C. Dee dee@rose-hulman.edu Article Link: