First Steps in Understanding Engineering Students' Growth of Conceptual and Procedural Knowledge in an Interactive Learning Context
This research is grounded in cognitive theory and focuses on how students mentally engage text-based and interactive materials while using instructional software. In pursuing this project, we have been most influenced by research on the development of skill and expertise in cognitive psychology and in engineering education.
This project began as NSF-sponsored research to study an instructional CD-ROM developed by E. E. Anderson for introductory thermodynamics. We were interested in exploring learning in this context because it represented a major trend in contemporary teaching using technology to provide students with engaging and evocative learning materials and aids. The primary goal was to assess the impact of the CD on learning and to identify and implement changes to improve the CD. Our main method of data collection was through computer-tracking—i.e., capturing, compiling, and analyzing every keystroke as students used the CD as part of their coursework. Those data showed students’ patterns of keyboard strokes but failed to reveal very much about students’ cognitive processing. Therefore, we turned to a well-established experimental method used in cognitive research—i.e., collecting verbal (or think-aloud) protocols. Under this method, students were interviewed individually for approximately one hour, and were asked to say what they were thinking as they worked through a section of the CD. We collected data from twenty-three engineering and science majors who had not yet taken a thermodynamics course, using introductory material (chapters 1 and 2) on the CD. The data were tape-recorded and subsequently transcribed. The transcriptions became the primary data in this study.
The major goal in the present paper was to analyze these data in a way that would yield insights into the general nature of these students’ cognitive processing, as well as the capacity of the think-aloud method to characterize individual differences between students. (See reflective essays by Ashlee Brown and Alli DeFinis for more details about the process of data analysis.) There were three major findings. First, students expressed significantly more cognitive activity on computer screens requiring interaction compared to text-based screens, based on the frequency of verbalizations. Second, there were striking individual differences in the extent to which students employed the materials. Third, verbalizations revealed that students applied predominantly lower-level cognitive processes when engaging these materials, and they failed to connect the conceptual and procedural knowledge that could be gained through through the CD in ways that would lead to deeper understanding.
Most of the higher-order cognitions occurred while students were working with screens that actively engaged them while performing virtual experiments or answering quiz questions. These activity types encouraged connecting to previous and outside knowledge, making inferences, applying mental mathematics, and drawing conclusions, more so than other types of activities. Unfortunately, the majority of the cognitions invoked by the students were of a low level, which is not inconsistent with the findings of other researchers such as Perry, King and Kitchener, Wolcott, and Pavelich for early college and engineering students. This finding is discouraging in view of the fact that the subjects were second and third-semester students in which one would hope to find more signs of higher-order cognitive development. But, as pointed out by research on the general college student population and on engineering student populations, this may be an unrealistic expectation.
A major achievement of this work was to provide a basis for distinguishing between students in terms of their depth of cognitive processing. We have now completed a follow-up study that involved engineering majors enrolled in introductory and advanced thermodynamics courses. The classification system and coding rubric in this study appear to be reliable, based on the follow-up study. The think-aloud methodology applied across a range of learning contexts (e.g., text-only, interactive, quiz) could potentially provide a productive paradigm for cognitive assessment across a broad range of engineering courses (See the reflection by Roman Taraban on Lessons Learned.) Author 1: Roman Taraban roman.taraban@ttu.edu Author 2: Edward E. Anderson ed.anderson@ttu.edu Author 3: Alli DeFinis alli.definis@ttu.edu Author 4: Ashlee G. Brown ashlee.g.brown@ttu.edu Author 5: Arne Weigold arne.weigold@ttu.edu Article Link: www.asee.org