SBG with voice revisions

This past semester I had my maiden voyage with Standards-Based Grading. I wrote a little about it here, herehere, and here. What I’d like to do in this post is discuss the lessons I learned and posit some changes I’m thinking about for the future.

What worked

I am fully convinced, now, of the incredible utility of having students turn in recordings of themselves attempting a standard assessment. Students would typically turn in a 5-10 minute screencast of themselves and I would then rate them on my four-point scale (doesn’t meet expectations, approaches expectations, meets expectations, and exceeds expectations). Here’s an example of a “meets” or 3/4. It’s a good example to show the difference between a turned-in Mathematica document and having the student walk me through it. Why not a 4? Because he “modeled an interesting ride” as the standard says but he didn’t do anything above and beyond. Another way to say it, I didn’t brag about this one.

What was a little unexpected for me was the range in scores I gave for very similar looking work. In the past, I would accept hand-written or Mathematica work as the finished product and I would grade what was there. Now I grade mostly based on what I hear. Do they show clear understanding of what they’re doing? Do they emphasize the more important/physical vs mathematical points? Do they note the physical meaning of, say, the cast-away solution of a quadratic equation?

I also didn’t get that many complaints about the assessment scores. This is partially due to the notion of reassessments being allowed all the way to the end of the term, but, I think, also partially due to how they couldn’t say “but I’ve got that written down.” Instead, they had to show me that they said it and emphasized it.

Another thing that worked well was the collective editing we did of the standards. I put the original document together at the beginning of the term, but once we got into a chapter we’d often have conversations about the value of one over another. Sometimes we’d combine two or three, and sometimes we’d split one into several. The students also quite easily accepted it when I would proclaim that the original standards were bunk and we needed to start from scratch for a particular chapter.

One last thing that worked well was our collaborative oral assessments. Three times during the semester we set aside several days in a row to do in-class oral assessments. Students would be randomly assigned a standard and would have ~10-15 minutes to put their work on the board. The magic happened when we would all discuss the appropriate score for the work.

What didn’t work

Talking with the students toward the end and reading their anonymous student evaluations, it was clear that the lack of structure was a large adjustment for them. All I mean by this was that I said you had to get all your standards assessments in by the end of the term. I kept us on the schedule as far as introducing and discussing new content but they were on their own regarding when to turn things in.

As you might expect, some students stayed relatively on top of things and managed several reassessments on most standards throughout the term. And of course some students didn’t. There was an amazing flurry of activity in the last couple of weeks and not everyone got to the point where they received the final course grade they were hoping for.

Having some students in the class who hadn’t really engaged with some of the material made some of the group problem solving activities we did in class a little flat. In the past, I’ve clearly had students who were behind, but typically a test would happen and they could wipe the slate clean. This year, there were times when people were effectively months behind.

Things to try

The number one thing I’m planning to change for my fall course is to have some sort of due dates for assessments. I know, I know, that kind of goes against the gospel of SBG but I think at least one of my plans might win your approval. Here are the possibilities I’ve kicked around in my head, with colleagues, and with the students (many of whom will take the fall class as well):

1. You must have an assessment turned in on a particular standard within two weeks of that standard becoming active
• “Becoming active” really means we’ve covered the material in class
• I don’t care what score you get on that assessment
• This will get the students “cracking the book” much earlier than the average this past semester. I look forward to coming into class two weeks later and saying something like “in your assessments it’s clear to me that many of you are misunderstanding X, Y, and/or Z. Let’s spend some more time on that today.”
2. All active standards must have one assessment in at 2 or 3 major points in the semester.
• These “major points” will likely correspond to the times when we do in-class oral assessments. What I like about that is it’ll make the orals that much better since they’ll be better prepared
• Again I don’t think I care what score you get. However, it might be interesting to say that you have to have at least a 3 by that point if you want to reassess later in the semester.
3. Students must turn in one assessment per class period.
• I set up a cool web page to do this in a way where each student would be assigned a random active standard at the end of each class period. I did that to give them a suggestion for what to do but I didn’t hold them to it.
• This would make sure that every student had forced reassessments for lots of standards, since, once they’re active, they stay active.

I see pros and cons to all three. I’m leaning toward “1″ right now but I’d love to get some feedback.

I’d also like to try to make my outside-of-class backchannel a little more vibrant. I used GroupMe.com, which acts as a texting version of a listserv this past semester and it was hit or miss on most days. If I seeded the conversation it tended to work better but I would have liked more. My buddy, Chad Topaz, at Macalester College uses Google Moderator to collect and crowdsource the priority of questions to be dealt with in class, but I like groupme’s ability to allow a true conversation before class. One thing I’m really interested in trying out is VoiceThread, where I can post screencasts (as this is also a flipped class) and students can add their own commentary, including questions. We’ll see.

Overall I was quite pleased with the course and I’m very excited to pursue SBG again this fall. There aren’t a lot of physics professors doing this sort of thing and it’ll be fun to talk to some this summer at a few different conferences that I’m going to. As usual, I’m extraordinarily thankful for the wonderful SBG community for giving me tips, encouragement, and a view of what’s possible.

Associate professor of physics at Hamline.
This entry was posted in sbar, sbg. Bookmark the permalink.

19 Responses to SBG with voice revisions

1. Mark Hammond says:

I don’t particularly like the second bullet of “2″, the part after “however.” This sounds like you get to a point where, if you don’t understand something, just forget about it. The “I never did understand ____, glad we’re past that” attitude should be discouraged, not encouraged.

That said, I know exactly where you’re coming from. We’re working on a more structured reassessment (student-initiated assessment) schedule to prevent the frantic, last minute surges. What I really hate about these surges (besides being frantic) is that the learning seems compromised. I had several students who left a large chunk of work for the two weeks before the first semester exam, and as a result, they were less facile with the first semester material all throughout the second semester. Further proof that cramming (even over the course of a week) doesn’t work so well.

• Andy "SuperFly" Rundquist says:

Thanks for the reply, Mark. Regarding the part after “However” I did something like that this past semester, only right at the end. As the end of the term was approaching students kept asking how late they could turn things in. I really wanted them to have put some effort on all the standards before the final (see the last link in the first paragraph). So, on the the fly, I decided in class one day that I only wanted to deal with reassessments during the finals week that were trying to go from a 3 to a 4. In other words, by the Friday before finals week they had to get things up to a 3 if they wanted to reassess further during finals week.

Effectively it just changed the end date (though I was careful to put that date in as the due date all semester long) and all the franticness happened the week before the final. Then, during finals week, it was kind of fun to watch a lot of screencasts that were trying to go from “solid” to “braggable”.

I do get what you mean about the notion of letting students learn things when they can learn it. But the impact on the classroom is the big motivator for me, here.

• Mark Hammond says:

I see what you’re saying. We did something like that, except that the last week before finals, we cut off all reassessment except for those students who still had not mastered “A” objectives (we have two flavors of objectives… A objectives are those that I can’t see someone passing the course if they haven’t mastered them… B objectives are higher level). This meant that well-meaning students were cut off from reassessing… at least for a week! We told them to go use that week to make sure they could show they had mastered all their objectives on the exam. THAT’s what the last week should be about. Preparing for the exam!

2. Guest says:

I’m taking this class in the fall, and I think this is a bunch of SHIT.

• Andy "SuperFly" Rundquist says:

Ok, I guess I deserved that. I was encouraging him to be more active in his web presence instead of lurking all the time.

3. Andy "SuperFly" Rundquist says:

I spoke about a lot of this with my wife (middle-, high-school teacher of english, spanish, and technolgy) this morning and realized that I left out some pretty important points: “or else WHAT?”

So let me give the “or else” for my 3 possibilities:
1. Or else you can’t assess that standard any more (harsh, I know, but I’m hoping it’s so harsh they’ll just buck up an do it).
2. Or else you can’t assess those standards any more (similar to the first one but it might seem less harsh, I’m not sure).
3. Or else you get a zero entered in the gradebook. I like this one from an SBG standpoint because it just means they’ll have to reassess.

Now comment, OR ELSE!

4. Stephen says:

Since I will be taking one of your standards-based grading classes in the fall, I would have to say that “1″ and “2″ seem like ones that would work the best. Number “3″ seems like setting us up to fail. We would have to do a reassessment for the random problem; and it that is the case, why would I even really try the first time around. I just don’t think it would get rid of the procrastination aspect. As the student, I would prefer “1″ the most. I think it provides a good compromise in the organizational structure. However, I’d like to see the ability to get back into the standard if at least a 2 was scored in the first two-week period. This way I have shown that I am attempting to master it (and am putting the effort forth), but I still have the chance to show you that I have fully mastered a topic.

I could live with “2″, but it just seems like there would be 2 or 3 frantic points instead of 1.

5. Joss Ives says:

Hi Andy. I’m constantly at war with myself on the issue of due dates and penalties for late things. My two major issues are (1) for something like homework it is very hard to accept it for full value after the due date if I have posted some sort of solutions at the due date and (2) I don’t like to give them rope to hang themselves with by accepting stuff at any point up until the final because they will often fizzle out at the end trying to get everything done.

My typical solution, which is pretty harsh, is to penalize late submissions by cutting the mark in half, but they can hand things in at any point between the due date and the final and still earn that half.

You could implement something like that in option 1 and give them something like 3 free extensions each worth a week of extra time beyond the regular two week window.

6. Chris Ludwig says:

My experience this first year of SBG was very similar to yours, with students all over the map in terms of their ability to manage their self-driven assessments (blog posts) in a timely manner. The “advice” videos students made at the end of the year for next year’s students are full of mentions of how the teacher (me) won’t be strictly watching over you and how you need to get your own work done. I am leaning towards your number 1 solution above, though, with some sort of deadlines other than the major marking periods so that students aren’t spending huge chunks of time in my class with no evidence of learning. I’m conflicted, though, on how exactly to implement deadlines since some students were working on really elaborate projects that just weren’t finished for several weeks. I want to give those super-creative students the time needed for great products at the same time I want to hold a candle to the feet of those students who don’t seem to do anything. I might start doing some sort of ‘check-ins’ with each student towards the end of each week or maybe every other week, which could also be used to help determine athletic eligibility. It might be something binary, like either you are progressing or you aren’t kind of thing.

• Andy "SuperFly" Rundquist says:

How are your students’ big projects worked into the standards, Chris? Are some elaborate ones meeting a standard in a similar way to a non-elaborate one? I’m just asking because I’m not sure how I would deal with that with any of my options. I like your “check-in” idea but would students be motivated to have something to share at the check in?
Thanks for the comment, it’s got me thinking, that’s for sure.

• Chris Ludwig says:

I’ve tried to push students to create a body of evidence that shows mastery of all the standards. Some of the bigger projects meet several standards while a smaller assignment might focus only on a particular content area standard. It really varied from one student to the next how they met all the standards. Some students would wait forever to post something, but when they did it was a massive compendium of several weeks of work. Other kids did short posts on a more regular basis. As long as both styles of work had the required elements, students had perfectly fine grades with either strategy. It really was a small subset of students that couldn’t get their act together to make sure that they were progressing at a decent pace.
For those kids, the check-in would help. The motivation for the check-in to have something done would be our school-wide extracurricular eligibility status, which they could lose for a week if the check-in was unsatisfactory.

• Joss Ives says:

Chris, I like this idea of “advice” videos to help future students succeed in the course.

If you do some sort of check-in system it might be worthwhile to have the student develop some sort of goals for the upcoming week to help them define what progress might look like and to have something against which to check.

7. Bret Benesh says:

Hi Andy,

Students were scrambling at the end of the semester—do you have any idea if they were scrambling significantly more that with the old grading scheme? And is their scrambling more or less visible with SBG with voice than under the old system?

Thanks!
Bret

• Andy "SuperFly" Rundquist says:

Hi Bret,
I think there was more scrambling this way because some students really didn’t do much throughout the semester. In the old way of doing things (for me) there would have been exams in the middle of the semester where students would have studied at least that material.
-Andy