External Evaluation by CEDER
CEDER, also known as the Center for Education Design, Evaluation, and Research, is one of SEISMIC’s key evaluators. The evaluation mission of CEDER provides support to U-M central administration, campus units, individual researchers, and community-based educators for evaluation or technical assistance related to education programs and research.
This page represents a summary of CEDER’s evaluations of SEISMIC so that the general public and SEISMIC community can see their findings. We’ve been lucky enough to have CEDER evaluate us for multiple years, so information has been sorted by date.
Click on the hyperlinks to learn more!
Ongoing Activities
- Ongoing thought partnership and collaboration regarding program activities and planning
- Pre- and post-surveys surrounding all of the Summer Meetings and reports summarizing survey findings
2018 – 2019 Evaluation Activities
- 1. Established 2019 CEDER Evaluation Scope
- Guiding questions, timeline
- This served as an initial plan, but changes are made to the scopes of work as the year goes on and evaluation priorities are changed.
- 2. Initial Workshop analysis
- Focused on mentions of the SEISMIC themes across all institutions
- 3. Sticky Note Analysis
- Members of SEISMIC gathered to exchange ideas and create concept boards with sticky notes. These sticky notes were analyzed for the themes SEISMIC focuses on.
- Experiment Working Group
- Measurement Working Group
- Structures Working Group
- 4. ECoach Meeting Report
- ECoach members were offered opportunities to give feedback on the SEISMIC program
- 5. 2019 SEISMIC Summer Meeting Analysis
- A variety of evaluations were done on the 2019 Annual Summer Meeting
- Interview Report
- Survey Report | Survey Appendix
- 6. A SEISMIC Poster was created
2020 – 2021 Evaluation Activities
- 1. 2020 SEISMIC Summer Meeting Analysis
- Created Working Group Snapshots from the 2020 Summer Meeting Pre-Survey Report
- Created Charts based on responses from the 2020 Summer Meeting Post-Survey
- 2. Established 2021 CEDER Evaluation Scope
- This served as an initial plan, but changes are made to the scopes of work as the year goes on and evaluation priorities are changed.
- 3. Fall 2020 Interviews by CEDER
- CEDER conducted 15 interviews with SEISMIC members with varying roles and levels of engagement.
- Recommendations
- 4. 2021 SEISMIC Summer Meeting Analysis
- A plethora of data was collected throughout the annual summer meeting period and revealed many interesting patterns.
- Pre-Survey Report
- Chat Responses and Anonymous Feedback Report
- Post-Survey Report
- 5. Summary Presentation on SEISMIC Evaluation Activities
- Provides a great overview of all evaluations and analysis done during this time period
- Slides
2022 – 2023 Evaluation Activities
- 1. Established the 2022 CEDER Evaluation Scope
- 2. 2022 Summer Meeting Analysis
- 3. Created a Report of Focus Group Findings
- 4. 2022 Weeks of SEISMIC Analysis
SEISMIC Workshop Analysis
While linked above, we wanted to highlight the SEISMIC Workshop Analysis because it forms the foundation for so many of our projects.
The Center for Education, Design, Evaluation, and Research (CEDER) at the University of Michigan School of Education worked with SEISMIC staff to conduct a qualitative analysis of the sticky note exercise done in the partner workshops. The CEDER evaluation team received a total of 261 sticky notes from exercises done at 6 partner institutions (ASU, MSU, Purdue, UC Irvine, UCSB and UMich). During the workshop exercise, participants were asked to respond to the question” On this campus, what intro STEM equity or inclusion concerns do you want to explore?” SEISMIC staff gave some example issues, such as the likelihood of individuals enrolling in STEM courses is affected by social identity and group work in my chemistry lab class creates an unfriendly climate for some students. All sticky notes were collected and analyzed by pulling emergent themes from the data and then coding all sticky notes. The following results show the output from the qualitative analysis. There were a total of 11 emergent themes. The chart below displays the number of mentioned for each theme and has associated theme descriptions. An individual sticky note could be double coded for multiple themes, which is why the total number of mentions exceeds the number of stickies.
