One month in (SBG with voice)

I’m about one month into my Advanced Electromagnetism course and there’s a few things I want to note down before I forget about them.

I love voice!

I’ve been having some conversations with people lately about students having access to solutions (be it through instructor solution manuals, Cramster, or whatever). Some people are astonished that I have standards that are, effectively, fully worked out in the book. Take this one: “I can derive $\vec{\nabla}\cdot\frac{\hat{r}}{r^2}$.” This is a crucial piece to electrostatics, as the Dirac delta function is central to making Gauss’ law useful. The book does a wonderful job of deriving this and showing where we need the delta function. So how can I tell that a student really understands it, since, in principle, they just have to copy the book down? My answer? THEIR VOICE!

As I listen to my students in their screencasts or pencasts (or in person on oral assessment days, see below), I can really tell the difference between a student reading the book to me and a student who has internalized the concept.

Here’s an example. This is a snip of a screencast from a student assessment for “I can discuss the foundations of, usefulness of, and ramifications of the method of images.”

If you listen to him, right at the end he says something that isn’t correct, and, in fact doesn’t make any sense. He says that the field lines from the image charge “cancel out” the field lines from the real charge. What is so interesting to me is how his finished product, essentially the page at the end of his screencast, would have almost certainly earned him a high score back in the days when students turned in static documents to me.

To me, this shows the value of both dynamic documents and the student’s voice in those documents. I feel like I have a much better sense of how a student is internalizing a concept when I hear it from their own mouths with their own vocabulary.

Two week rule

My two week rule (more clearly explained here) is working out very well, though it still needs some tweaks. Students have to turn in an assessment for every standard within two weeks of that standard becoming active. If they don’t, they have to take a zero for that standard for the whole class. One month in and I haven’t had to give any zeros yet! Basically, they’re treating those deadlines pretty seriously.

One thing I really like about the current system is that students are more willing this time around to turn in what are, essentially, drafts of their assessments. They’ll turn in a pencast or screencast that they’ll admit is not complete but they’ll be careful to let me know where they need help.  For every assessment they turn in I generate a screencast with comments about what they’ve done correctly and tips for how to fix what’s not correct.

Oral assessments

This past week we dedicated all three days to oral assessments. Students were given a random active standard during the previous class period and they needed to bring a single page solution to the next class. Each person, on the day of the assessment, puts their sheet up on the document camera. I study it for up to one minute and then spend up to six minutes asking them questions about it. Note that they don’t “present” the standard, we simply don’t have the time for that this semester. Finally, the whole class spends up to two minutes critiquing the student’s performance, culminating in me proclaiming what the rubric score is.

At first I thought I wouldn’t like not being able to surprise them with a standard on the day of the assessments. I was worried that letting them have at least two days to prepare their sheets would allow them to bone up on something simply to present it, not learn it. I’m still not sure if learning is happening as deeply as I would like, however, I do like being able to ask deep, probing questions about the standard, knowing that they’ve got the basics in their head at that moment. That probing that I do is often very illuminating. I’ve found that some students really do understand these things at a deep level and I’ve found others who don’t peer deeply enough into the various steps in the book.

My friend Brad Martsberger has sat in on several class periods so far this year, one of which was this past week. I’m going to ask him to, at least, post in the comments his thoughts about it, but I’ll share two things he shared with me that he found interesting. First, he was interested to see how the students seemed comfortable taking low scores. He didn’t witness the grade grubbing he has seen in other courses. Second, he was interested in how honest the students were with each other. I chalk both of those up to the SBG notion of reassessments afterwards being allowed. But, even though I think I know why it’s happening, I’m really happy it is.

Synthesis (or, here you go, Joss)

My good friend Joss Ives has often kept me honest about synthesis. He’s pointed out to me the slippery slope of SBG where standards can become silos, where students don’t see the connections.

This year I’ve made a few adjustments to my approach to try to protect against that slippery slope. First, I have several standards like the one above with the phrase “discuss the foundations of, usefulness of, and ramifications of” that go along with standards asking students to actually calculate things. Essentially I’ve forced them to spread their knowledge about a concept over a couple of standards.

Second, I’ve got a couple of standards that act in support of several others. The best example is “I can derive all the relationships among the electric field, the electric potential, and the charge density (this is one of Griffiths’ triangles).” This “triangle” one, with the six possible connections, is used all the time by me in my conversations with students, and by them in their assessments.

Third, during oral assessments I ask all kinds of synthesis probing questions. Last year, the students and I decided that, when I did that, they would get a score for every standard that was affected. This year, I’m sticking with a single score for the standard they’ve been assigned for the oral assessment. One reason for this is that generally the students don’t do enough on the secondary standards to warrant a score. However, I like holding them accountable for all the connections that a particular standard might have.

More to come

I’m really having a good time teaching this course this year. I’m happy to use this blog as a journal of sorts, though I’d love to get some feedback as well. In the future I hope to blog more about the peer review my students are doing, and on some ideas I have to improve our class time together.

Professor of physics at Hamline University in St. Paul, MN
This entry was posted in physics, sbar. Bookmark the permalink.

16 Responses to One month in (SBG with voice)

1. That’s awesome Andy. I have always had students come in and have a discussion about topics that they are re-testing. Perhaps I should offer this as an alternative.

Do your students turn in one screencast per objective? How many hours do you spend watching student videos?

• Andy "SuperFly" Rundquist says:

The students can reassess as often as they like, though the bulk of the reassessments so far have been through this past week’s oral ones. I grade around 5-10 assessments per weekday (even though class only meets every other day) so that’s about an hour of work per day for me. There are only 14 students in this class. One nice thing I’ve recently discovered is that I can watch pencasts on the bus with the cool Livescribe ipod touch app. I watch them on the bus and come home to make my screencasts with comments.

2. Joss Ives says:

Hi Andy,

Always delightful to get a hat tip.

I like the idea of standards that act in support of other standards or are specifically about the connection between different standards. I think you would only need a small handful of this type of standard to help the students see that the individual standards are not silos.

Have you been able to make the synthesis part of the synthesis probing questions apparent to the students? I like your strategy of holding them accountable for the connections that a particular standard might have, without making them stress out about losing ground on those other standards should they not see the connections.

When reading this post my brain went back to the scaling issue. It seems in a large section, that a 24-student recitation section (1 hour per week led by a TA) would give students at least as much opportunity to have oral assessments as they get in your class. With your system of assigning the oral assessment questions ahead of time, it is reasonable that TAs and the instructor sit down to generate a list of questions for each assigned question (the same question being assigned once in each of multiple recitation sections). And then based on the presented solution, a handful of questions from the generated list would be asked.

At approximately 1 hour per week per 3 students for grading submitted standards, it is pretty costly in terms of person power required. Beyond peer review, I’m not sure what else you could do to make this seem feasible.

As always Andy, I’m very excited to follow along with you as you massage your system into increasingly good shape. Last week I had my first homework submissions via pencast, but haven’t marked them yet. I just realized that, as long as you use the same pen that they do, you can pencast your comments right over top of their written solutions on the dot paper. Neato!

• Andy "SuperFly" Rundquist says:

Hi Joss,
I’m hoping the peer review I’m doing a little of in this class can help the scaling issue. Certainly I’ve seen some limited evidence that students can give very constructive advice, but I don’t have enough data points yet.

I am pretty clear with them about how I’m probing for synthesis but the notion of having a handle on sythesis aspects of standards is not laid out as clearly as it could be in the rubric.

-Andy

Andy,

You hit two of the things that struck me during the portion I was able to sit in on. The students know exactly what it means to get a 3 or 4, they are honest about places where they didn’t meet those criteria, and, because they know they get another shot at it, take the grade for what it’s worth. It will be interesting to see what happens toward the end of the semester when a student re-submits a standard having addressed the comments from a previous submission but still doesn’t receive a 4. Will this student still be so accepting of a lower grade or will they feel like “what the hell, I did what you asked me to, why is that worth a ‘C’ and not full credit?”.

Another thing I really liked about the oral assessments (and maybe you get this from the pen/screencasts too) is that the students misconceptions, misunderstandings, errors, reasoning flaws, etc. are exposed pretty quickly. I think I was present for about 1 hour out of the 6 hours of oral assessment and easily identified a handful of things that I would make points of emphasis if I had a half a lecture to give.

Not so much related to SBG in particular, bot more related to the class. I notice that the students are sometimes proficient at cranking the math tools, but don’t always have a strong connection between the math and the physics. As examples, a density integrated over a volume should have a clear physical meaning; how to choose ‘good’ dipoles when doing the multipole expansion; when is it ok for a function to have a discontinuity and when is it not, what does it mean to put something “at infinity” in the real world, can it be done in a lab?

I’ll be interested to see how the semester goes. Especially how time spent grading goes when students are turning in re-assessments in addition to new ones.

• Andy "SuperFly" Rundquist says:

Thanks for posting this, I really appreciate it. I think you’ve hit the nail on the head with your comment about students coming later with reassessments and not liking their score as well. The problem is that I give some very pointed advice in my screencasts back to them, assuming in my mind that they get the notion that they still have to do well on the stuff I don’t talk about. Some focus solely on the things I mention and then fall down the rubric scale because the other stuff isn’t there and they start to get crabby about it. I’m really trying to get them to recognize what a “4” is in the oral assessments so that less of that happens, but we’ll see.

I liked how you noticed the math/physics disconnect that some of them have. You’re right, it’s surprising that they sometimes don’t recognize $\int_\text{all space} \rho\,d\tau$ as the total charge of the system, they just dutifully go about doing the integral. Again, I’m hoping detailed feedback and honesty from their colleagues can at least point this out to them, if not get them to learn it.

-Andy

4. bretbenesh says:

Hi Andy,

This only reinforces my hope that I can use VOICE with my elementary education students next semester. They are in the same boat: there are standards that they need to simply understand, not (necessarily) figure out from scratch. The VOICE (plus follow-up questions) should greatly help me figure out if they really understand why, say, you “invert and multiply” when dividing fractions.
Bret

• Andy "SuperFly" Rundquist says:

Hi Bret, thanks for the note. You just gave me a great idea! Instead of giving feedback indicating what could be better, sometimes I could just say “something’s wrong here” and let them have another crack at it with the same written document. Maybe that’ll get them to not be too crabby with the future reassessments. I think I’ll talk to them in class about this today! Thanks, Andy

5. bretbenesh says:

PS: You do have me a little scared about the workload, though. However, I will have a TA to help me with the grading, so I hope that it will be doable with 40-50 students.

6. Andy "SuperFly" Rundquist says:

I went back to figure out exactly what my work load has been. I’ve graded 11 standards for every student (there’s a few that aren’t technically past their 2-wk window yet). That comes out to just over 6 assessments graded every weekday. I don’t feel swamped very much this semester and I would guess that I could nearly double that output if I had to. That get’s me to a class in the mid 20’s. If peer review worked out, I think it could work without killing yourself.

7. Andy "SuperFly" Rundquist says:

update on student crabbiness: Brad wonders above about how willing students are to take a sub-perfect score after a few reassessments. In the past they start to get crabby. Collectively in class the other day we decided to try the following: If the assessment is close to a 4 with some mistakes, I’ll show where the mistakes are (on the page) but not discuss what they are or how to fix them. The student can then re-screencast their own work and tell me how to fix the mistakes. In other words, they won’t have to redo the whole problem. They seem pretty excited by it. What I like is giving that non-specific feedback. Hopefully it’ll force them to go back and listen to themselves. In fact, I just did one where all the math was correct but he said incorrect things. He’ll have to listen carefully to find the mistake.

8. Melissa says:

Andy– I continue to be intrigued by your SBG approach to courses, and using the screencasts with voice. I do have a question for you… have any of these courses had a significant laboratory component? Do you have thoughts about implementing this in a class that is primarily laboratory based?

• Andy "SuperFly" Rundquist says:

Hi Melissa,

I purposely chose these two courses because they didn’t have a lab, but only to simplify things. Next semester there’s a lab for a different course (Optics) and I think I’ll continue to use SBG. There are two ways I could go about it: 1) just do the lecture the sbg way like I have been doing and have them do lab reports like my old ways and then just break the grade up appropriately. 2) Create standards for the lab portion as well, like “I can propagate the error through my calculations” or “I can prove that generic Karo syrup isn’t optically active”. I’m leaning toward the second choice but I’d love to get others’ viewpoints as well.

-Andy

• Melissa says:

While I haven’t used SBG in my traditional classes, I’m thinking about testing the waters in our primarily lab-based electronics course. I can imagine providing some structured laboratory activities during lab time, but rather than evaluating students through formal write-ups or informal laboratory notebooks, the standards would involve hands-on competencies that students would have to exhibit by building/troubleshooting circuits. I like the idea that the assessment then centers on successfully using and applying the experimental skills that they have explored in lab. Since this is all new for me, though, I was interested in what the potential pitfalls might be.