Tuesday, April 29, 2008

Interesting (and Yet Painful) Grading

So I'm grading the papers from X class, a very theory-heavy course intended for advanced undergrads. Only 8 remain on the roster (this class is not for the faint of heart), and so I'm sitting here with their papers (when I should be doing work for my admin gig) and giving them lots of feedback.

Now, the paper assignment was a toughie and one that really pushed these students beyond what they normally must do in writing papers at this university. They were expected to write a 10-15 page paper in which they 1) made an argument about a primary (literary, film) source and 2) advance that argument through the integration of theory and criticism that 3) they synthesized with their own ideas about the primary text.

In other words, I set up an assignment that tried to force them to do a version of what they see critical and theoretical sources doing in their reading.

But so now I'm looking at the papers, and.... hmmm.... I'm not sure how I feel about them. On the one hand, I think it's fair to say that these are the most interesting papers as a group that I've read as a teacher. That said, they are also.... messy. The execution is off, whether because of how they're integrating sources or how they're organizing their ideas, or in some cases it's a matter of the writing feeling forced and tortured. And often its a combination of these things.

Now, as I'm going through, it's fair to say that every one of these papers would probably be head and shoulders above what most instructors at my university, in my department, see from their students. It's clear that they are really pushing themselves hard to stretch themselves intellectually and to attempt to think about things that are much more complicated than they typically think about in their writing. And I want to reward that. But on the other hand, I think it's important to take the messiness into account. Yes, this was the first experience with this sort of writing for them all, and I don't want to be a meanie, but at the same time, I want to give them real feedback. It's just BAD WRITING if every single sentence in a paragraph is passive, or if you use quotations to speak for you, or if you don't actually explain why you use the theory that you use. And so I feel like I'm in a weird position with all of this. So far I've just been commenting without thinking about how I'm going to grade the papers themselves. As I write this post, I'm thinking it could be really beneficial to assign grades for the different parts of the assignment. Yes. This is what I shall do. Because the paper is worth 30% of the grade, so I can divide the grade into 3, with 1/3 of the grade being for the proposal and following directions, 1/3 being for the technical/formal stuff with the writing, 1/3 being for the ideas and analysis. Yeah, I think that's the best approach. And that means that I can continue with the commenting as divorced from the grading, and just really engage with what they're writing, and then evaluate each later with the rubric that I'll design to arrive at the grade.

I so love writing here when it helps me figure stuff out. Now I think, however, that I should take a break for lunch.


k8 said...

If it helps any, writing research has shown that when people are struggling with ideas/concepts or writing in a new discipline or genre, writing skills sometimes regress. So, there is probably a very good reason why these writing issues are turning up in their papers.

life_of_a_fool said...

Your idea is what I was going to suggest. I do this with many of my assignments, with the different components weighted differently (which allows me to grade each part honestly, without worrying about an area that I think is important - but not THE MOST important - hurting them more than I think it should). Plus, it helps me evaluate two papers that are good, and bad, in different ways "against" one another, in a semi-structured way. I like this approach (though ideally you'd tell students in advance what the criteria are).

Dr. Virago said...

Ooh, ooh, I'm going to follow your 3-way split idea when I grade *my* stack of papers that asks them to do similar tasks and that I fear will be as messy. Thanks!

VirtualProf said...

I had to wonder -- Does this mean your students don't know how this paper will be graded until you give it back to them?

I like your assessment plan and have often used it with major researc papers but I think the students should have been provided clear assessment rubrics prior to submitting the paper.

A scheduled peer review prior to submitting the final paper (either required for a grade or optional for those who wish to take advantage of the opportunity) would help a lot with writing problems.

My grads and undergrads always peer review drafts for writing and organization before sending me the final paper for a grade. It also gives me a chance to look over the drafts and make suggestions if necessary. This means (1) they learn more as they revise to specifications; (2) their papers are much better; (3) I don't have to agonize over grading.

The_Myth said...

Like everyone else, I was also going to suggest exactly what you said you'll do. It's how I [and many of my own profs] graded essays of this sort.

I also wonder if what you're seeing is that the students didn't spend enough time proofreading to take out the "messiness." Having good ideas is wonderful, but form and function matter too. A lot of undergrads sometimes miss that important part because they are not being trained to think that all writers need more than a neat idea...they also need to be able to communicate clearly and appropriately to ensure readers understand that neat idea.

As an aside, it's been very gratifying [and a bit sad] to find that other people [in this case you] also have been at schools where there are low expectations of undergraduate writing. The students at the last place I taught openly resented that I downgraded them for messy writing...I think mostly because many of their other profs just graded them for their ideas [which I think was often just a projection of the prof's ideas onto the students' poor prose since the prose was often impenetrable and confusing].

Dr. Crazy said...

K8 - you know, I think I knew what you mentioned about writing skills regressing once, but I'd totally forgotten it, so yes, it does help :)

LoaF - exactly. See that's the thing: as I've been commenting, I'm noticing very different things across papers, and part of what I'm trying to do here is to come up with a mechanism for evaluating each paper for what it is as opposed to fitting them all into a one-size-fits all model that's just about how each paper falls in with the others. I'm thinking that what I've devised will allow me to give them a more individual sense of what their paper is doing. We'll see.

Dr. V - Always glad to help (even when I'm just figuring stuff out for myself :) )

VP - Does this mean that they don't know how the paper will be graded? Yes and no. I am actually really against handing out a rubric ahead of time for a few reasons. 1) I think that learning how to read an assignment and to follow it is important. It's not just about filling in blocks on a chart of points - it's about responding to the prompt and developing your own ideas in relation to it. I think that rubrics can forestall creativity in ways that are really counterproductive. 2) I like to keep my expectations somewhat flexible until I see what the students produce. I don't like to have my expectations so solid that there's no room for them to experiment without being penalized for it. 3) I think that emphasizing how an assignment will be graded in that particular way ultimately puts too much emphasis on the grade and not enough emphasis on producing a piece of writing (and original insights and knowledge) for its own sake. Yes, a grade is part of the equation - it's a way for me to communicate to them how they did - but I really don't think that the grade is the point of doing the assignment. Handing out a rubric ahead of time communicates that the grade really is the point. That said, I state very clearly on the assignment sheet what the expectations are, my expectations for writing in general are clearly outlined in my course policies, and students have a number of low-stakes assignments that build to the higher stakes "big paper" that give them a sense of how I grade and what I look for. They're not left in the dark at all, but at the same time, yes, I expect them to do some interpretive work. To me, that's a skill that it's important that I allow them to develop. Expectations for writing are not outlined in rubric form in most writing situations. The writer has to determine things like audience, appropriate style, organization, etc. on his/her own. If I don't give them practice with that in the classroom, I feel like I do them a disservice. And I commented on drafts for this paper two weeks ago and they had two days of structured peer review (not typical in my upper-level lit classes to do the peer review, but because of the singularity of this class, I felt like it was appropriate and so I moved the schedule around to allow for that). In other words, I'm not feeling like what I'm encountering is a matter of them not being prepared or clear about how they'd be assessed. I really think it's more a matter of what K8 says about them dealing with a writing situation that forced them out of their comfort zones and to deal with material that is difficult for them. And it's not that these papers are terrible by any stretch of the imagination - it's just that they're messy. And I think that's part of the learning they'll do in this course. To be honest, I'm totally psyched that not one of the papers I've seen is that bane of my existence: the boring but totally correct B paper. Give me messy over that any day of the week.

Myth: I don't think that this was a matter of them procrastinating or not taking adequate time. Again, these students have had working drafts for two weeks now. I really think the "messiness" is more about attempting to articulate ideas that are complex in language/structure that is more sophisticated than they're used to using. And you know, I think it's good that they're not comfortable. I think it means that they're getting it, even if their prose isn't the most enjoyable I've ever read. I'm seeing flashes of real brilliance - it's just not sustained brilliance, or it's embedded in confusing structure or syntax - that sort of thing.

k8 said...

The skills regression element is something that doesn't necessarily affect how I evaluate a piece of writing, but it does keep me from feeling too frustrated. I used to see it a lot when I worked in our writing center. I would work with a student throughout the semester and see this difference in skills based on whether the paper was for his or her home discipline or a different area.

It makes sense to me - the mind needs that space to process the issues. That's where showing students how you are evaluating the different aspects of the papers (like you are) really makes sense. It will also show students that the overall grade isn't necessarily reflecting just the quality of their ideas and analyses. It can soften the blow of a less-than-desired grade outcome.

Dr. Crazy said...

"It will also show students that the overall grade isn't necessarily reflecting just the quality of their ideas and analyses. It can soften the blow of a less-than-desired grade outcome."

That's exactly my hope. Because the thing is, as I'm making my way through these papers, the grades are skewing lower than these students will be used to receiving (not so much from me, but rather in their careers as students), and I want them to know it's not that their *ideas* are bad, but that the execution just didn't quite make it in some places. I mean, It's got to affect a student's grade (and their ability to communicate their ideas) when almost the entire paper is written in the passive voice. It's just got to. But that doesn't mean the student isn't brilliant - it just means that the insecurity is getting in the way of communicating that brilliance. (And yes, I've been on them all about the passive voice all semester, but I fight a losing battle.)

VirtualProf said...

I appreciate the further explanation. I like to use rubrics to show elements of grading, not necessarily specifics (hope that makes sense!). For example, I will use a category for organization and another for writing and another for credibility, and so forth. But I don't get majorly detailed with higher level undergrads.

You know, it used to be that teaching undergrads was different than teaching high school; now I wonder. I have been teaching grad Education and Communication courses for about six years now and I've seen the writing ability of graduate students decreasing alarmingly over those years.

It's frustrating to figure out how to assess assignments from students who just cannot write adequately at the college level. And what's even more frustrating is that I'm seeing less and less difference in the level of writing skills of undergrads and grads.

And yet I hear high school teachers complaining about the difficult they have with high schoolers' writing skills. They teach writing skills and for some reason, kids just are not getting it.

Thanks again!! I enjoy reading your blog.

life_of_a_fool said...

I don't think I've ever gotten a complaint using such a system. I like to think it feels less arbitrary to students - like, if their ideas are so brilliant, why aren't they getting an A? They can see why, a little more clearly (in addition to comments, of course, which I also give).

Anonymous said...

This is a random observation along the lines of k8's comment. I TA'd a writing course and in general the undergrads at my institution are pretty good. it's an elite institution, so they're generally pretty decent writers. but the writing course? Was awful. These were juniors and seniors but because it was a writing requirement course, they suddenly felt like they were now Writing. And they should say Something Important. And therefore produced the most unnatural, tortured prose in the process of trying to write like a grown up.

so in addition to the difficulty of articulating sophisticated ideas, sometimes I think they psych themselves out when they realize they're writing a real live paper.

Sisyphus said...

Yes ---- I would jump in to say that I value messiness when students are confronted with a difficult assignment. (I don't reward it, but I don't necessarily dock it per se.) I mean, think about it: a theoretical reading of a primary text incorporating criticism, that's what they ask for in grad seminars. Even here our profs don't ask for their upper-level majors to do all that in a single paper, except maybe the theory survey prof ---- _not_ that I am telling you not to assign this! I love it and think it's wonderful --- I'm just pointing out that the assignment is not just hard for undergrads, but hard, period. And they often appreciate it when you acknowledge that fact to them.

Dr. Crazy said...

Hey Sis,
Oh, I totally agree that it was a very challenging assignment for them. I don't typically do this sort of assignment even in upper-level classes. It's because it is actually a special topics theory course that this assignment was given (I've been circumspect about the course's topic while I'm teaching it), and to my mind, the only way to really engage with theory is to use it. The students have come such a long way this semester, and I've got to say, as much as this is a challenge for me to grade, I think it was good that I let them try their hand at the assignment. I'm actually thinking, now that I've been through a few more papers, that I may curve it up 5 points or something to acknowledge the difficulty, but I've still not decided about that. Whatever the case, even if I curve it, they'll see their "real" grade, too. I know I've got at least three in there who are considering grad school - I'm hoping that if they go that road, they'll be better prepared than I was for that work.

What's funny is that a couple of them are following me to my modernism class next semester - after this I suspect they will find the assignments in that course - even writing on Ulysses - easy. Heh.

Susan said...

Dr Crazy, I've been thinking something along the lines of your last comment. That is, what you gave your students was really like a grad student paper (or honors thesis type project). And it seems to me you want to say, somehow -- on the absolute scale this is where it is, but in terms of your work, WOW! It would be ironic if the most sophisticated work your students do gets a low grade precisely because they are stretching. So curving the grade upwards makes sense.

This is a great argument against grades, by the way -- what gets lost when we use shorthand to give our messages.

Oh, and the boring B paper? I'm totally with you. What AM I supposed to say?

Dr. Crazy said...

Let me just say that these students really have gone above and beyond in this paper, too. The papers are generally falling into the B-range, with no curve, just every single B is for a totally different reason (which is what makes the grading challenging). In other words, nobody is going to tank the course because of this assignment, and many would get an A in the course even without curving. I suppose I'm walking that fine line of really wanting to praise them, but wanting also to show them where they would go to improve from here.