Screencast vs live: assessment comparison

In my advanced electromagnetism class, students turn their voice in with every assessment. There are 14 students (junior and senior physics majors) in the class, and, with one week to go, they’ve turned in 475 videos (either screencasts using Jing or pencasts using Livescribe pens). They’ve also done 9 oral assessments each, with three just in the last week. In this post I wanted to compare and contrast those mechanisms of assessment.

Screencasts

For the screencasts, students pick one of the standards, and they either work an associated problem or discuss the relevant derivations. Typically, they write most, if not all, of it down first before hitting the record button, spending most of their time in the videos carefully explaining each step in the process.

Midway through the semester we enacted a new policy dubbed “improvable.” This meant that if I deemed that the majority of the work was solid but some was wrong or missing, I’d simply give them feedback (via Jing) saying that something was wrong or missing and they could turn in another video later only addressing the shortcomings. This policy has really helped morale in the course, as students don’t feel like they have to start over from scratch, especially on the long derivation standards.

Since students can find the solution to any of the problems in the text online, often what’s written on the page is not the problem. Rather, as I listen closely, I can usually discern which parts they’ve really mastered and which they’re simply repeating.

Live oral assessments

I’ve written about these before. Here the students come in with a single page on a standard I’ve randomly assigned to them the previous class day. They put the page up using a document camera, and I stare at it for about a minute. Then I spend the next 7-8 minutes grilling them on the material, using their written notes as context, though feeling free to venture beyond them as well. Then, in the last 2 minutes or so, we, as a class, discuss what happened and come to a consensus on the score (on my 4 point scale – see the syllabus linked at the top of this post).

With the three days we’ve spent this week doing this, I’ve started to do much more synthesis work with the students when they’re up at the board. For example, when a student was working on a standard from chapter one on the Helmholtz theorem (if you know the divergence and curl of a vector field, you basically know the vector field), I asked him how we’ve made use of the theorem in the rest of the class. This was a question I couldn’t ask back when we were working on chapter one. We also have some recent standards from chapter 7 that force the concept of synthesis. For example, the very last standard asks students to explain Maxwell’s equations. What’s cool is that they have to talk about how Griffiths’ two triangles (the relationships among charge density, the electric field, and the electric potential in one and the current density, magnetic field, and vector potential in the other) don’t quite cover all electromagnetic effects.

In the live assessments it’s also fun to make up the examples they need to do on the spot. One standard in chapter one asks the students to show that they can use the four fundamental theorems of calculus (1D, gradient, divergence (Gauss’ law), and curl (Stokes law)). In a video, the student makes up some situation and shows that the right- and left-hand sides of the various equations work out. Typically they do the type of fairly complicated examples worked in the book. But in the live assessments, I can do things like ask them to treat our classroom as the volume of interest and give them a relatively straightforward vector field inside the room. It’s really cool to watch them puzzle out the intricacies of the various theorems, working with math, and, more interestingly, their bodies as they mime the vector field.

Comparison

I think if I could figure out a way to do it, I’d do live assessments all the time. It’s really great to shift from being a summative evaluator (the video is completed before it comes to me) to being, I don’t know, something else. In the live cases, I’m really assessing how well they can think on their feet.

This week was great as a several of different times the whole class and I were trying to decide whether to penalize a student because they had written something wrong but they, thanks to my prodding, had that “oh s**t” moment and were able to explain what went wrong and fix it. Sometimes we concluded that the student still deserved a four (“exceeds expectations”) because of how they demonstrated mastery by thinking so quickly on their feet. Other times we backed off to a three (“meets expectations”) because the mistake should have never happened and it took them too long to see the problem. It’s tricky to discern how much help I’m giving with my prodding, but it’s nice that the rest of the class is there to help me do that.

The videos are nice as a record that the students can use as review materials. In the improvable cases, they can go back to watch themselves and they can watch my video feedback before embarking on the corrective video. Unfortunately, in those cases when the live assessment is judged as improvable, they don’t always remember exactly what needs to be improved (and, often, neither do I).

In the future I’m sure I’ll continue to use both of these assessment techniques. However, I’d really like to glean from both some best practices to refine my future syllabi. I’d love any help that you could provide as well.

About these ads

About Andy "SuperFly" Rundquist

Associate professor of physics at Hamline.
This entry was posted in physics, sbar, sbg. Bookmark the permalink.

3 Responses to Screencast vs live: assessment comparison

  1. Joss Ives says:

    Andy, let me preface the following by saying that I am really impressed with how you have managed to create what appears to be a very safe classroom environment that uses oral assessments as one of the primary forms of assessment.

    The one thing I don’t like about oral assessments is how it makes the thinking process visible to the point that the student sometimes/often feels vulnerable. On a written exam there’s time to sit back and really ponder or to stash the question in the back of your brain and then come back to it later. With the oral assessments this time buffer disappears.

    Of course there are lots of positives to this format. You can use some guided questioning to help the students see that they have all the pieces needed to answer the question, and perhaps just need a little help connecting those pieces. You have a way better window into their understanding than just looking at something on a written page.

    As somebody who is new to trying to use oral assessments, I’m wondering if you have any advise to offer in terms of dealing with the concerns that I have. In the weekly meeting I have planned for my Advanced Lab course, I’m planning to ask them pretty challenging questions based on their work each week, but the understanding we will have is that they can have the time (until the next weekly meeting) to ponder those questions and come back with some reasonable answers. I imagine that there will be many questions that will be beyond them in terms of what they have looked at so far and the purpose of those questions will be to motivate some of what I want them to do in the next week. Of course there will be a lot of questions that I would like them to be able to answer on the spot when I ask them, but I want them to feel that its more than OK for them to say “I don’t know, but I will figure it out” just like we do when they ask us really challenging questions.

    Like I said, this is quite new for me to be on the other side of this assessment technique and so far it has just been asking follow-up questions when they do presentations. I think oral assessments have tons going for them, but right now I tend to lob slow pitches at them because I don’t want them to feel frustrated and discouraged.

  2. Pingback: Mind-map final exam | SuperFly Physics

  3. Pingback: Innovation and Disruption in Everyday Education | LEARNINGANDPHYSICS

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s