2022 Annual Summer Meeting

Project Name: Developing and testing a single session intervention to help students cope with fear of negative evaluation in active learning science courses

Proposing Team:

• Dr. Katelyn Cooper

• Jessica Schleider

Brief Description

This proposal aims to develop and test a Single-Session Intervention to help students cope with fear of negative evaluation, a common emotion experienced by science students in active learning classrooms. Aim 1 is to create and pilot an online SSI that leverages students’ interest in science to help students cope with fear of negative evaluation in the context of active learning biology courses. Aim 2 is to conduct a large, randomized controlled trial across multiple introductory science courses across multiple institutions to assess the impact of the SSI on (1) student fear of negative evaluation and other affective outcomes, (2) student performance outcomes, and (3) student persistence outcomes. Machine-learning will be used to identify for whom the intervention is most effective, with particular attention to examining the impact on students with underrepresented and underserved identities in science.

Q: How will this project engage with SEISMIC institutions and/or individuals in SEISMIC?

Seismic institutions would participate in the development and testing of the single-session intervention meant to help introductory science students cope with fear of negative evaluation. After a successful intervention is created, it would be made available to all SEISMIC institutions and individuals: We will design this SSI to be both scalable and accessible. The intervention will be easy to administer, independent of the instructor, freely available, and able to be completed anytime and at any location by the intended user.

Q: What does this project hope to achieve by Summer 2023?

We hope this project yields an effective, freely available SSI that has the power to help students cope with student fear of negative evaluation, which over 90% of undergraduate sciences majors report experiencing in the context of science courses. Equipping students with skills to cope with fear of negative evaluation via a free, self-guided online intervention that can be posted on a website for anyone to use has the potential to change who is participating in active learning classrooms and whose voices are heard in science. Finally, these coping skills will empower students across the sciences to meaningfully engage in not only the social evaluative situations they will face during their undergraduate years, but also the social evaluative situations they will encounter throughout their careers.

Project Name: It's time to leave high-stakes exams behind

Proposing Team

• Tim McKay

• Ben Koester

• Heather Rypkema

• and other colleagues at the UM Center for Research on Learning and Teaching

Brief Description

Even before SEISMIC started, it was recognized that the most inequitable introductory STEM courses were distinguished by their reliance on high-stakes, inauthentic (e.g. multiple choice) exams as the primary form of evaluation. During the first few years of SEISMIC, continued analysis has shown that performance gaps often emerge only in these exams, and not in other graded elements of these courses. The use of timed exams as the primary elements of student grades is quite rare in higher education, though it is common in large introductory STEM and some social science courses.

In most cases, neither instructors nor students believe that these exams are ideal. They often probe student understanding in superficial ways, and their reliance on right answers for scoring makes them vulnerable to cheating. Meanwhile, delivering high-stakes exams in large classes is extremely complex. They are often offered outside of ordinary class meetings (usually in the evening), need large and carefully coordinated teams of proctors, and regularly require special accommodations for large numbers of students. These exams and the pressures they create are also regularly implicated in concerns about student mental health.

We propose to thoroughly exam the schemes of evaluation in use in the SEISMIC General Courses List (https://www.seismicproject.org/about/overview/). For as many institutions as possible, we will combine this information about evaluative schemes with measurement working group analyses of course equity. Where possible, we will explore the contributions of exam scores to the observed inequities. We will contact the instructional teams for all of these classes to find out whether evaluative schemes shifted during COVID, and explore the impact of these changes on course outcome inequities.

With this information in hand, we will convene several meetings aimed at surfacing alternative approaches to large course evaluation which avoid the drawbacks of the current high-stakes timed exam system. These may include the use of more open ended written exams, mastery oriented grading, which allows students multiple opportunities to demonstrate their knowledge, and loosely scheduled online exams given over a period of 24 hours or more.

This effort is intended to launch a collaborative effort aimed at exploring practical alternatives to high stakes timed exams for large classes. The goal of moving away from this 20th century form of evaluation is to reduce course inequities, reduce the complex effort associated with exams, and improve inclusivity and overall student well-being.

Q: How will this project engage with SEISMIC institutions and/or individuals in SEISMIC?

The list of SEISMIC institutions we will include in this study will be determined by where we can find available collaborators. We will need at least one individual committed to gathering information about the evaluation schemes of each of the SEISMIC General Courses on their campus, along with the ability to access the institutional data needed to measure course inequities.

Q: What does this project hope to achieve by Summer 2023?

By Summer 2023, I expect we will have completed out census of course evaluation schemes, measured course inequities across this landscape, reported on the relation between evaluation schemes and inequity, and uncovered those courses which moved away from high-stakes exams during the COVID crisis. These results will then, we anticipate, be used to convince additional courses to make changes.