Lessons from a Journal Club: Investigating Student Learning – Post 1: What is vs. what works?

Posted by Rhett Mcdaniel on Wednesday, August 18, 2021 in News, Resource.

by Leah Marion Roberts, Senior Graduate Teaching Fellow

Assessment of student learning is a familiar and essential component of teaching. As college instructors, we constantly ask ourselves questions like: “Are my students learning?” “What do they still not understand?” “Do students feel they can approach me with questions or concerns?”or, “That cool new thing I tried this semester — did it work?!”

What can systematic inquiry about teaching and learning teach us about our teaching?

Even though assessment is essential, instructors may not always be systematic about their inquiries into student learning and classroom experiences. What can systematic inquiry about teaching and learning teach us about our teaching? (So meta, yet so practical).

To explore this question, Dr. Cynthia Brame (Associate Director) and I developed and facilitated a journal club for faculty, postdocs, and graduate students. We read pairs of papers to illustrate types of questions instructors could ask in their teaching contexts, then we discussed and extrapolated the papers’ findings to our own classrooms. We drew from work in both SoTL (Scholarship of Teaching and Learning) and DBER (Discipline-Based Education Research) communities.

In our Journal Club, we wanted to explore the different types of questions that can be asked in teaching and learning contexts. In this blog series, I’ll share some highlights of our collective learning, organized thematically by the types of questions instructors could ask. Across three posts, I will explore the following themes: (1) exploring what already is vs. what works, (2) affective and cognitive assessments, (3) qualitative and quantitative research approaches (4) reasons for doing systematic inquiry (5) key takeaways from the journal club.

Theme 1: What works? vs What is?

As a starting point, we found it helpful to think about the difference between the following two general buckets of questions: (1) What works? (2) What is?

“What works?” is the type of question that, for many, will come to mind first when brainstorming research questions about teaching and learning. Instructors may want to know whether, for example, a new class activity improved students’ ability to articulate a core concept or perform a core skill. Or, maybe they want to know if their new approach of requiring every student to come to office hours at least once influenced how comfortable students felt asking the instructor questions throughout the semester. To get us started with an example of a research paper that explores a “what works” question, we read and discussed Optimizing the Efficacy of Learning Objectives through Pretests (Sana et al., 2020).

We then asked participants to brainstorm some “what works?” questions that were of particular interest to them. These questions are not final or formal research questions, but rather, example ideas of the types of questions in the “what works?” category.

“What is?” is a descriptive question about what is already happening. This type of question explores our current teaching practices, classroom environments, or student experiences. For example, an instructor may want to know about student perceptions of their course. Do they feel like they belong? Do they feel like they can disagree with the instructor? Do they think a particular activity helped them learn? Are there disparities in student experience or performance by social or identity groups? To demonstrate the value of asking “what is?” questions, we read the paper: Caution, Student Experience May Vary: Social Identities Impact a Student’s Experience in Peer Discussions (Eddy et al., 2017).

Our journal club participants came up with the following first-draft ideas for questions they could ask about “what is?” in their classrooms:

It’s very important to acknowledge our colleagues in this journal club, each of whom was a critical contributor to our experience. Thank you to all participants for awesome conversations and contributions to our collective learning!: Amanda Brockman, Sociology; Celina Callahan-Kapoor, Medicine Health and Society; Jessica Gilpin, Biological Sciences; Rachel Hanebutt, Human and Organizational Development; Brielle Harbin, Political Science; Sara Mayeux, Law; Danielle Picard, Medicine Health and Society; José Luis de Ramón Ruiz, Spanish and Portuguese; Cole Meier, Biological Sciences; Melanie Schuele, Hearing and Speech Sciences; James Sears, Biological Sciences; and Ben Yett, Electrical Engineering.