Problem to be addressed
How to best spend lecture time to promote student learning in introductory courses is a topic that we often debate with ourselves and with our colleagues. Recent research suggests that there is more and deeper learning when students come to class with some basic exposure to the content so that they are prepared to learn from the experiences in lecture. Reading the textbook prior to coming to class is one way of providing the necessary basic exposure to content, but getting students to read the textbook is a difficult goal to achieve.
One possible solution is to use multimedia presentations to help students learn basic content prior to coming to class. There are three reasons why multimedia presentations might prove effective in this context. First, presenting basic physics content via asynchronous, web-based multimedia presentations can be monitored and enforced better than policing textbook reading. Second, multimedia modules can be designed with embedded assessments, and a modest amount of course credit can be assigned to viewing the materials (so long as some minimum time is spent on them) and to performance on the embedded assessments. The third reason why multimedia presentations of basic content might provide a viable solution to the textbook-irrelevance problem is that a considerable body of research exists on how to design multimedia materials to improve learning. There is, however, no research at the university level examining whether or not students learn basic physics content better from multi-media presentations compared to reading the textbook. Before any major effort is devoted to designing and implementing multimedia presentations in large enrollment courses, we need to know if they are at least as effective as reading a textbook in terms of imparting basic content knowledge.
Method
Four pilot multimedia learning modules (MLMs), were developed: 1) Coulomb’s Law, 2) Electric Fields, 3) Electric Flux, and 4) Gauss’ Law. Each MLM covered one lecture’s worth of course content, and was divided into approximately 10 scenes. Each scene was implemented as a Flash movie containing dynamic animations synchronized with an audio narration that was controlled by the student (pause, play, rewind, and position). Embedded formative assessments were included in two or three of the scenes for each module, which consist of questions that must be answered correctly before the student can move to the next scene. The average narration time for an MLM was 12 minutes; students typically took about 17 minutes to complete a single MLM including the embedded assessment.
The second, paper-presentation, mode consisted of verbatim material from a traditional textbook (heretofore referred to as the Textbook group) subject to the constraint that the same worked out examples covered in the MLMs had to also be covered in the textbook-based materials. We selected materials from the Tipler and Mosca textbook, largely because the material and worked-out examples in this textbook paralleled those in the MLMs. Each edited textbook unit was about 7 pages long and contained 8 figures and 2 examples.
Volunteers were solicited from students taking the first semester calculus-based mechanics course (Physics 211) at UIUC and offered compensation in exchange for participation in the study. Students were paid $15 for each of three 90-minute sessions, and a $20 study-completion bonus. Students were randomly assigned to groups, each receiving a different treatment (MLM, or Textbook).
The four units were delivered within two 90-minute sessions distributed over two days in the same week. Each 90-minute session consisted of two “lessons,” each lasting about 45 minutes. Each lesson began with a presentation (the MLM group sat at computers to receive the multimedia presentation; the other two groups received paper booklets containing either the MLM-script, or Textbook presentation) and was followed by a post-lesson assessment. Two weeks later students returned for a retention test, which lasted about 1 hour.
Results
The combined results from the Post-Lesson Assessments and associated error are given in the second column of the table above. The MLM group had an average score of 75% compared with the textbooks group’s score of 63%. The standard deviation of the Post-Lesson Assessments scores is 13.5%, which means students in the MLM group scored nearly a full standard deviation higher on the exams than their peers who used the textbook (effect size = 11.5/13.5 = 0.85). The Retention Test was given approximately two weeks after the lessons, and the raw results are shown in column 3 of the table. Once again the MLM group performed about 13% higher than the Textbook group, with an average score of 70% compared with the Textbook group’s average of 57%.
Post Lesson Retention
MLM (n = 16) 74.7+/-2.6 69.7+/-4.0
Textbook (n=16) 63.2+/-3.1 56.8+/-4.1
Our study has shown that learning of basic physics content by the current generation of undergraduates from a typical modern introductory textbook fared poorly when compared to learning from multimedia modules that were designed based on principles derived from research into multimedia learning. The effect sizes obtained in this study are consistent with those from numerous other multi-media learning studies. For example, Mayer and collaborators have performed a series of 39 experiments designed to compare the retention performance of groups using presentations that either were, or were not prepared according to the seven design principles they developed and they found positive effects in 33 of them with the average effect size being 0.8. We conclude that multimedia learning modules represent a viable, and more easily enforceable alternative means of pre-lecture preparation for introductory physics students.
Author 1: Timothy Stelzer; tstelzer@uiuc.edu