SBG with voice feedback

This week I gave a presentation at the Physics Education Research Conference (PERC) on Standards-Based Grading (SBG) with Voice: Listening for Students’ Understanding. I was grateful for the opportunity to speak there and for the opportunity to gather my thoughts about what I’ve been up to, and to brainstorm changes for the future.

I’ve written before about how I taught my Theoretical Mechanics course last spring. What I want to do in this post is capture some of the great feedback I got from the educators at PERC.


When I told people what I was up to, some of them said things like “sounds good but it would never work for me – I have too many students.” This is a point I’ve been thinking about a lot as I’m really excited for this assessment technique and I’d love to use it in all my courses. The basic complaint is that it would be too hard to watch all the student screencasts in a large class.

At the end of my talk, Noah Finkelstein asked what the PERC community could research that would help me out and I mentioned this scaling problem right away. Specifically I asked whether the concept of peer review of these screencasts could be benefitial.

One idea people had was to check out Calibrated Peer Review. I have only scratched the surface at that site but I’m grateful for being pointed to it.

After thinking and talking about the idea of peer review for the past few days, I think I’ve decided to try some low-level research on this in my upcoming class (Advanced E&M) in the fall. I want each student to review a different student’s screencast once during the semester. At first I wondered how to motivate it and I realized I could do it the way I do everything: Make it a standard! Essentially the research question is whether performing peer review can aid in learning (from both perspectives). The standard will be something like “I can assess another students work using the common class scale.” I’ll assess their work by comparing their feedback to what I would give. If a students gets a 4 in that first try (meaning their feedback is similar in quality to mine), they can be done for the semester. If not, they can always try again.

I asked some students doing summer research a week ago about this plan and they immediately hated it. Their comments focused on the work they’d have to do for others. I wasn’t able to convince them of the value, to them, of doing the reviewing. I think I can couch it better in the course this fall. We’ll see.

Two passes through a problem

Another thing I asked the PERC community for help with was the notion of how students should prepare their assessments. The two main ways they do it are:

  1. Write up a solution on paper, scan it, and call up the pdf on their computers to screencast their commentary. They hover their mouse over the relevant steps to make sure I’m looking at what they want me to.
  2. Write up a solution with a pen tablet or smart pen, recording all the while.

I was talking with several people about the notion that the first method might lead to more learning. Effectively it forces the student to go over the material twice: once when writing it the first time, and once when they make the recording.

I think there’s something to that notion but it would be cool if someone could research it. For me, I’ll probably discuss it with my students (many of the students from the spring course are enrolled in the fall course) and see what they have to say about it.

General feedback

I feel that my talk went over reasonably well. Several people let me know that it got them thinking and a few said they’re planning on revisiting their fall syllabi with my talk in mind. I’m not sure if the SBG part or the “with voice” part was getting people the most excited. My guess is a little of both.

It was fun to be at the conference, as I had never been before. It was great to see how much setting “learning goals” or “learning outcomes” or, as I put it, “standards” is starting to focus a lot of the conversations around Physics Education Research. I’m excited for where that’s going in the future.

About Andy Rundquist

Professor of physics at Hamline University in St. Paul, MN
This entry was posted in sbar, sbg, teaching. Bookmark the permalink.

14 Responses to SBG with voice feedback

  1. Pingback: SBG with voice feedback | SuperFly Physics | My Blog

  2. First – I am curious how well this went over with the people at the conference. Whenever I talk about not grading homework, 95% of the people I talk to think I am crazy. Oh, they might be right – but I am not crazy because of no grades for homework.

    Second – I think the next time I teach in a large lecture, I am going to use some type of peer ranking to assist with reassessments. Maybe in order to come show me what you know, you have to first show it to two students that sign off on your work.

    Third – good job bringing attention to SBG.

    • Andy "SuperFly" Rundquist says:

      Hi, Rhett, thanks for the comment.

      1) I didn’t get a lot of “you’re crazy” but that’s because I had a 1/2 hour talk to convince them. Mostly they were supportive of the overall nature of sbg, based on their comments. It was interesting to notice how many posters were devoted to mapping out standards for various courses (though not necessarily advocating sbg).

      2) That’s a cool idea about having to go to peer reviewers before coming to you. I’m going to steal it without retribution, if you don’t mind.

      3) How much sbg do you do, I guess I don’t remember from your blog?

  3. Joss Ives says:

    Hi Andy,

    Since learning of SBG I have been really thinking that it is something that would be of interest to the PER community at large, so I’m thrilled that you had a chance to get out there and give a nice long talk. One fortunate thing is that a lot of people involved with PER and reforming courses/curricula have extra resources (grad students, post-docs, etc) to throw at their efforts so piquing their interest could lead to some nice systematic implementations with lots of the PER course assessment tools (student interviews/focus groups, conceptual diagnostics, longitudinal studies, attitude surveys, etc) looking directly at SBG in a higher ed environment.

    For the “two passes through a problem” discussion, I would be surprised if you would find any differences because with #2 the student will still have had to prepare the solution in advance before trying to write it out in real time. That’s my feeling anyway, but you have had a lot of experience with screencasting (yourself and evaluating those of students) so you might have some insight on some subtle differences between the two options that I don’t.

    • I’ve already had some people talk to me about how to leverage their resources to study SBG at the college level, sounds exciting!

      For the 2-passes issue, every once in a while a student with lots of confidence will bust out a 1-pass recording with really no prep. It’s rough, of course, but due to the self-selection they’re usually pretty good.

      On another note, Joss, you’ve got some fans out there. Your name came up a few times and people really liked your approach to things and your opinions. I got to say “and he’s my friend” a few times, beaming with pride.

      • Joss Ives says:

        Well it is certainly nice to be loved and/or respected (by both you and the PER community). I’m glad you got a chance to meet some folks from the PER community. I have found them very welcoming, thoughtful and down to party (a great combination for conferences). I was terribly sad that I couldn’t swing the resources and had to miss PERC this year.

        If you remember any of the names of folks that were talking about studying SBG at the college level, let me know. I would love to chat them up.

  4. Pingback: Raising the bar for an ‘A’—Capstones « Quantum Progress

  5. bretbenesh says:

    Hi Andy,

    I love the idea of peer feedback (I am doing it in my class this semester, and there is a chance I might end up using calibrated peer review), but I am nervous about including it as a standard. Might there be some pushback from students/administrators/professors that you are grading them on things that are not related to the content?

    I think you should do it, I am just nervous for you.

    • Andy "SuperFly" Rundquist says:

      Hi Bret,
      I’ve been thinking a little about this and I think I’ve convinced myself that this process of discerning what is a good presentation/scast/whatever from fellow students is something I can defend as core to the course. It’ll help them learn the material better and so I think it’ll work. I’ll lump it in with the holistic standards like “I can use Mathematica well” and the like.


      • bretbenesh says:

        Hi Andy,

        This works for me! Now I need to go and think about whether I should be using Calibrated Peer Review for my course (thanks again!).

  6. Pingback: CPR « Solvable by Radicals

  7. Pingback: SBG with voice round 2 | SuperFly Physics

  8. Pingback: Assessing with Student-Generated Videos « Solvable by Radicals

  9. Pingback: Grading Computer Programming with Voice - ProfHacker - The Chronicle of Higher Education

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s