Using More Than One Grader To Evaluate Student Class Participation: Controlled Experiments
Grant program: Teaching and Learning Development Grant (TLDG)
Grant recipient: Leyland Pitt, Beedie School of Business
Project team: Jan Kietzmann, Beedie School of Business, Shannon Bridson and Emily Treen, research assistants
Timeframe: February 2017 to April 2017
Funding: $6,000
Courses addressed:
- BUS 702 – MBA Marketing Management
- BUS 709 – Management Information Systems
Final report: View Leyland Pitt's final report (PDF)
Description: Student class participation, namely the extent to which a student participates in class discussion and analysis, is a common component of MBA courses at business schools and particularly in less quantitative courses such as marketing and management information systems (MIS). In most MBA programs it accounts for a significant proportion of the final grade for the majority of courses. Class participation, it can be argued, contributes significantly to the ability to work in a team structure and to communicate verbally with people inside and outside of an organization. Class participation allows students to practise developing an opinion, or a point of view, and then to articulate this point of view. Furthermore, an emphasis on active class participation cultivates more critical thinking and creativity. Quite simply, it is an excellent way of preparing students for the ‘real problems’ they will face in today's complex and global world.
Traditionally, and certainly in the AV business school environment, grading for class participation has only been done by the instructor. Students are informed well in advance of a course what is expected of them and how they will be evaluated. Instructors typically use a “mugshot list” – a class list with full student names as well as head-and-shoulders photographs of each student. Immediately after class the instructor goes through the list and recalls each student’s contribution or lack thereof and makes positive or negative marks against each name. At the end of the course this completed list enables the instructor to make a final assessment of each student’s participation and to award a final overall mark.
Skilled instructors become very effective at allocating class participation marks, with practice, and there are surprisingly few student complaints about the marks or the fairness with which they are awarded. However, in recent years some of the leading case-instruction business schools have resorted to the use of a “second-grader” in the MBA classroom. This has been driven both by a desire to drive out as much subjectivity as possible as it has been by the increase in class sizes. The second grader grades while class is taking place, and the instructor grades independently at the end of the class. Then these two grades for each student can be compared and reconciled.
Our plan for this TLDG grant is to employ the services of second class participation graders in the Full Time MBA Marketing Management (Bus 702) and Management Information Systems (Bus 709) courses as these courses are taught in tandem and the students also have other common assignments for them. We will employ the same second grader/assistant (a PhD student in business) for each course, who will sit at the back of the class and grade while the class is occurring. The instructor and the second grader will enter both grades for each student on the grades list. At the end of both courses this will facilitate comparisons of class participation grades to be made in a quasi-experimental situation. The results will inform on the consistency of grading by instructors and second graders.
We anticipate this approach to have a significant impact on student learning: Not only will students have the opportunity to receive regular feedback on their class performance if they so wish, we also expect that this feedback will lead to improved and enhanced class participation. This will range from simple logistics such as seating, always having a nameplate visible, making sure a hand is held high, to more articulate opinion raising, such as keeping the discussion on track, bringing appropriate numerical analysis to bear in discussions, and not merely repeating what has been said already, or what is in any case apparent from a case. Moreover this will also enable students to reflect on their learning and participation, and to think about what they have been doing prior to the feedback and what they will do in future classes.
Questions addressed:
- What is the difference between Instructor and Second Grader marks for class participation on the MBA marketing course (BUS 702)?
- What is the difference between Instructor and Second Grader marks for class participation on the MBA MIS course (BUS 709)?
- What is the difference between Instructor and Instructor marks for class participation on the MBA marketing course (BUS 702) and the MIS course (BUS 709)?
- What is the difference between Second Grader’s marks for class participation on the MBA marketing course (BUS 702) and the MIS course (BUS 709)?
- What value do students find in receiving feedback on their class participation from the second grader? Does student participation improve after receiving feedback?
- How do students feel about having a second grader in the class?
Knowledge sharing: An internal, informal presentation will be made to Beedie faculty; a more formal presentation will be made to Beedie faculty and staff at one of the regular “teaching day” occasions.
Pitt, L. (2017, May). Case Teaching and Case Writing. Special session at the Academy of Marketing Science Annual Conference, May 24-26, 2017, Academy of Marketing Science, Coronado Island, CA.
Keywords: case based teaching, second grader, student evaluation, participation grades, classroom discussion, feedback, open-ended questionnaire
View Leyland Pitt's ISTLD-funded projects:
“She Grabbed your What?” A Human Resources Management Case (G0160) - with David Hannah
Using More Than One Grader To Evaluate Student Class Participation: Controlled Experiments (G0199)
Exploring Case Writing as an Assessment and Teaching Tool in the MBA Classroom (G0249)
Lego Serious Play and the Power of the Story: Enhancing Student Creativity (G0395)
View Jan Kietzmann's ISTLD-funded projects:
Developing and Assessing Learning Outcomes for a Cohort-based Multidisciplinary Course with a Technical Component (G0013) - with Ted Kirkpatrick
“Casing” Entrepreneurship in Vancouver (G0131)
Teaching and Learning with Artificial Intelligence (G0198)
Using More Than One Grader To Evaluate Student Class Participation: Controlled Experiments (G0199) - with Leyland Pitt
Team Composition and Team Performance: An Experiment Using Artificial Intelligence (AI) (G0268)