Educational Advancement Fund spotlight: “Measuring the Impact of Peer Review on Learning Gains”
Julie L. Johnson, professor of the Practice in Computer Science, recently told us about her grant from the Educational Advancement Fund, on measuring the impact of peer review on learning gains.
Can you tell us about your project and what inspired you to do it?
My project is to measure the learning gains from students interacting in peer review. Now I know there’s already a lot of research suggesting that students who engage in peer review become better writers. I learned about peer review within courses and the associated research results when I was in the Online Course Design Institute a couple of years ago. And I wholeheartedly agreed with it because I saw some small evidence of the benefits in my class, when I used it to allow students to earn extra credit. When my students posted to a public website called Stack Exchange, I was really impressed with how well they wrote. When they post it to that site, they wrote so well, and I wondered, why they didn’t write that well for me?
After going to the Online Course Design Institute, I decided to integrate peer review more fully into my course, and I started using peer review software (i.e. Peerceptiv, Kritik). I immediately noticed the difference in writing gains, which is a key skill in my computer science algorithms class, a class where students need to learn not only how to fix a problem but then how to explain their solutions to their peers, or to novices who might be working for them and so on.
I was very pleased with the writing gains, but I started to wonder if there were additional benefits. My students are problem solving and I wondered, “Is peer review helping them be better at solving problems?” So, for example, if students became better at explaining long division in third grade, do they also become better at doing long division as well?
It was that kind of a question I was asking myself, and that’s what inspired me to start the project. The project is to take four sections of my course, two taught by myself and two by my colleague Dan Arena, and randomly divide them, such that in two sections we’ll be doing peer review and two sections not.
And then we’re going to ask whether we see differences in students’ problem-solving ability on exam questions that are common to the four sections—not their explanations, which I expect to be better among the students who do peer review, but the actual mathematical problem-solving.
What is the anticipated impact of this project on students and the Vanderbilt community in general?
What I hope to see is a significant difference or statistical difference associated with using peer review in the course. If it bears out that in fact, the peer review is also helping with students’ learning gains around problem-solving, then not only will that inspire us to keep using this tool and share that information with our students to inspire them, but also to consider it in other courses. You know, in computer science, we don’t tend to think as much about writing and communication skills until seniors are getting ready for capstone, but would it also help first year or second year programmers to learn to better explain their code or present their solution? Again, we know it’s going to help with communication certainly, but does it actually help with learning that subject? If so, then we’re going to take a look at our own courses and then maybe share with colleagues outside of computer science.
Do you have any thoughts to share with others who might be thinking about applying for one of these teaching grants?
Yes, definitely. I would say one of the best things about the teaching grant is motivation. You know, I’ve been thinking about this project for about two and a half years. An NSF opportunity popped up where a group had a grant and they were looking for people to join them in a cohort that would walk through the next two years, doing empirical testing on teaching approaches in computer science (NFS-DEERS). And so that got me motivated to finally put pen to paper. But it can be hard to set aside your other duties to put together your IRB packet and then ultimately, you know, get results, analyze, and attempt to publish.
So if you’re thinking of just a small kind of side project, this might be that final motivation to push you toward doing it. It gets you to say, “I’ve committed to this. My name is out there on the Vanderbilt website. I’m going to follow through with this project.”
So it’s a real motivator.
In one or two sentences, how would you capture your teaching philosophy?
My teaching philosophy is to teach students to learn. I feel that’s so critical in computer science. It probably is in every subject, but it was really ingrained in me early in my career when our students would go off to internships and would come back and say, “We need to be learning language X. Every student should learn language X.” And as a young professor, I thought, “They’re right. We need to change our entire curriculum and learn language X.” No sooner did I think that than a year later everyone’s learning language Y. And so I needed to come up with a response for these eager and excited juniors or seniors who had just come back from industry and wanted to include this year’s trend in our program.
And I realized the key feature here isn’t language X or language Y. Because that’s going to change. And instead it’s, “How do I learn language X and how do I learn about learning language X?” And it’s never too soon to do that.
You know, we do teach our students a particular programming language. That’s our first intro classes, teaching them a programming language. How we do that is critical though. If we simply stick to the, the old-fashioned basics—you know, reading, highlighting, copying, reproducing—they will learn that language. And then they will need someone go through the same exercise with them again, when they’re out in industry, because they’ve got to learn a whole other language.
It just doesn’t work for us. And it probably doesn’t work for any subject. It’s just so much better for our students to be actively learning, in a flipped or moderately flipped classroom, where some of the learning is on them from the very beginning. It lets them start to take hold of the idea that they’re going to be doing this for the rest of their lives.
Leave a Response