Home

          Jennifer Hora
          Robbin Smith

The 2009 Teaching and Learning Conference’s Simulation and Role Play track held a successful three day workshop where a wide variety of ideas were exchanged by members.  Participants discussed simulations lasting as short as a few minutes of class time to semester long activities.  Conference participants presented a wide range of projects including papers addressing development of simulations in general, specific applications of simulations, and formal evaluations of student learning before and after simulations. 

In the Simulations and Role Play workshop, two subjects emerged as issues of primary concern for all instructors looking to utilize these activities successfully: the consequences of the design of the simulation and the importance of simulation assessment. 

Design
The TLC Simulation Track participants determined it is important for an instructor planning a simulation to consider five design elements before launching a simulation: target audience, amount of instructor control, duration of the event, outlining the goals, and planning a debriefing.  First, any instructor using a simulation must accommodate the audience.  Who are the participants?  What is their background?  What skills set do they possess and what substantive knowledge do they bring to the simulation.  Knowing the audience increases the chances of creating an exciting and stimulating simulation.  Second, the instructor must decide how much control to surrender to students.  Will the simulation have many or few parameters?  The nature of the instructor, the students, and the course play determining factors here. 

Third, when designing a simulation, instructors need to decide the duration of the simulation.  Class time is a valuable commodity; any time spent explaining and executing a simulation results in less time for lecture, group work, presentations, or other classroom activities.  Fourth, all simulations need clear goals, both for the instructor and the students.  Specifying these ahead of time and expressing them to the students (before, during, or after the simulation) increases the usefulness of the exercise.  And lastly, any instructor utilizing a simulation needs to build in class time for a debriefing after the exercise.  Students need a guided activity which allows for processing the material. 

Assessment
The Simulation participants’ second major focus was assessment and evaluation of experiential activities.  Track participants agreed that assessment of student work in a simulation is an essential part of a simulation activity and that a major issue they faced was determining the weight to be assigned to a simulation in the course grade breakdown.  The complexity of the simulation (consisting of the differing roles played by students, the demands on student participation, and the duration of the simulation) influenced the weight of the overall simulation grade, with track participants agreeing on the importance of a multi-pronged assessment of students’ work throughout the simulation.   The five modes of evaluation favored by track participants included:

• Self-Reflection Papers – Self-Reflection papers allow students to demonstrate a thorough understanding of course concepts and theories upon the conclusion of a simulation.  Moreover, self-reflection papers allow students to move beyond the content of the simulation and engage in larger debates about the utility of political science concepts they encountered in the simulation as well as the validity of the simulation construct itself.
• Peer Evaluation – For simulations and role playing that involve intensive group work, peer evaluations can successfully augment other assessment tools and provide the faculty member with a fuller picture of student participation.  Peer evaluations also allow students to identify individuals who acted as group leaders and to acknowledge individuals who played important, but largely unseen, roles during the simulation or role play.
• Portfolio Creation – Portfolios of material accumulated during a long term simulation not only allow students to demonstrate increased cognition of the subject matter as the activity progresses, but also permit students to revisit issues raised early on in the simulation process and analyze them from a more informed perspective that develops during the process.
• Post-Tests – Post-tests administered at the conclusion of a simulation or role play can be used to determine areas of student mastery of the subject matter.  Post-tests can also identify areas within the simulation that were not fully comprehended by the students.  Post-tests can be administered substantially after the simulation activity concludes in order to determine retention of materials as well as unintended lessons learned from the simulation.
• Participation – Clearly, the simplest assessment tool for a simulation and role playing activity is attendance and participation.  Measuring quality participation is a difficult task, and track participants agreed that attendance and participation should only be one aspect of a multifaceted simulation assessment model. 

TLC Simulation and Role Play track participants acknowledged that long-term assessment of simulations is crucial.  Although student evaluations generally reflect a positive view of simulations and role plays (with students often mentioning these activities as their favorite part of the course), little research has been done to determine the extent to which students retain the knowledge or skills garnered during a simulation.  As one participant questioned, does the simulation merely reduce “knowledge anxiety” or does it produce long-term cognitive outcomes?  While track participants would not ignore the “cool factor” of simulations for student participants, all agreed that more research must be done in this area in order to determine what it is that simulations really do. 

TLC track participants struggled with how to strike the balance between the two outcomes -affective learning and cognitive learning, particularly because the ability of faculty to accurately measure affective learning is so limited.  However, the possibility that a well-designed simulation could improve classroom dynamics and increase the willingness of students to embrace diverse perspectives is a promising one.  More research needs to be done in this area:  first, to develop accurate tools to measure affective learning, and second, to see if simulations substantially contribute to affective learning outcomes.