Assessing Pedagogy in Bioengineering Classrooms: Quantifying Elements of the "How People Learn" Model Using the VaNTH Observation System (VOS)
Monica F. Cox & David S. Cordray
Purpose. The purpose of this study was to develop an index that would distinguish pedagogical practices in traditional, lecture-based courses and in innovative courses based upon principles of the “How People Learn” (HPL) framework (Bransford, Brown, & Cocking, 1999). Using data collected from the VaNTH Observation System (VOS), a direct observation system that allows trained observers to record data in code strings within four to six second intervals (Harris & Cox, 2003), this index parsed data into three categories (i.e., HPL-oriented instruction, traditional instruction, and classroom organization) representing 100% of the observed class time. The primary categories of interest within this paper are HPL-oriented instruction and traditional instruction, and comparisons of both types of instruction within HPL-oriented courses and in traditional courses are made.
Background. This study was conducted because of limitations in analyzing observation data using a previous index. Before the current study, data were collected using the Classroom Interaction Observation (CIO) portion of the VOS (Harris & Cox, 2003). The CIO parses data into code strings representing five categories- (1) who is initiating an action, (2) to whom the action is being initiated, (3) what classroom instructional practices are occurring, (4) the HPL framework dimensions that are present, and (5) the media that are being used within the classroom. When eleven content experts could not validate the previous categorization of code strings, which focused primarily on the presence of the HPL dimensions, a new method of analyzing CIO was proposed by the authors. This new method parsed all possible instructional code strings into one of 18 subcategories (i.e., eleven HPL instructional subcategories or seven traditional instruction subcategories) encompassing all CIO categories, resulting in each course reporting both a HPL subcategory sum and a traditional subcategory sum.
Method. The analyses using index are based on 182 classroom observations in twenty-eight undergraduate bioengineering courses across multiple levels at Vanderbilt University. Seventeen courses were designated as experimental (primarily implemented HPL-oriented pedagogical practices) classes, and eleven were designated as control (primarily used traditional, or nonHPL, engineering practices) classes. Data were collected by trained VOS observers during five academic semesters between 2002 and 2004. Some of the observers were aware of the designation of courses as either traditional or HPL-oriented, and some of the observers were not. Frequencies representing the seven traditional subcategories were summed to create a traditional pedagogical instruction percentage, and frequencies representing the eleven HPL subcategories were summed to create an HPL pedagogical instruction percentage. Individual class profiles of average percentages of traditional and HPL instruction and classroom organization were then created. Independent sample t-tests noting differences between the experimental and the control courses were then run. A Levene's test of equality of variances also was run.
Findings. Below are the following results from the paper:
- There were statistically significant differences between HPL-oriented and traditional courses for both the HPL Subcategory Sum and the Traditional Subcategory Sum.
- Compared to lecture-based courses, HPL-oriented courses reported a higher HPL Subcategory Sum, and compared to HPL-based courses, lecture-based courses reported a higher Traditional Subcategory Sum.
- Compared to traditional, lecture-based classes, HPL-designated classes reported higher occurrences for the following five HPL subcategories: guidance by the instructor, comments, praise, monitoring, and question and response.
- Compared to HPL-designated classes, lecture-based classes reported higher occurrences for the following three statistically significant traditional subcategory items: instruction by media, lecture, and no response. The fourth statistically significant traditional subcategory item, praise, had a higher occurrence in HPL-oriented courses than within traditional courses.
Conclusion. This newly created index distinguishes pedagogical practices in traditional and innovative courses. Such an index can be used to help faculty to reflect upon their teaching practices and to explore their teaching over time.
References
Bransford, J.D., A.L. Brown, and R.R. Cocking. 1999. How people learn: Brain, mind, experience, and school. Washington, DC: National Academies Press.
Harris, A.H., and M.F. Cox. 2003. Developing an observation system to capture instructional differences in engineering classrooms. Journal of Engineering Education 92 (4): 329–36.
Author 1: Monica F. Cox; mfc@purdue.eduAuthor 2: David S. Cordray; david.cordray@vanderbilt.edu
Article Link: http://soa.asee.org/paper/jee/paper-view.cfm?pdf=1001.pdf