New Kid asked in comments to my last post to explain what I do with the checklist for grading and how it's not a rubric, and I thought it might be worth doing an actual post about it if others might find it useful.
First, the reason I resist rubric-method of grading student writing is that it invites complaint. "My friend got four points on this section and I got five, and I think that we both did the same on that section." I hate dealing with that kind of crap. Also, I find that it can be difficult to justify why one person gets one "number" on a given section and another gets another. Grading writing IS subjective - not entirely quantifiable. Yes, I look for certain things, but there is no one recipe for an A paper, and I think a rubric can inspire students to look for that recipe rather than focusing on the quality of their ideas and claims.
That said, I do think that the valuable thing about a rubric is that students know that you were looking for certain things on the assignment, and that you were looking for all of those things on all students' papers. I also think that the rubric can eliminate the need to compose lengthy end-comments that repeat the same things from paper to paper, which when you teach a large number of students, can become INCREDIBLY time-consuming, even if you're copying and pasting in typed comments.
So, over the past couple of years, I developed a "checklist" method that is looser than a traditional rubric (doesn't assign point values to different things) but that has some of the benefits of a rubric model. Oh, and in my lit classes, I design the checklist so that it works for each of the two papers (which are basically the same assignment - two, 3-5 page topic-driven literary analysis, for example) and can work for a range of topics - it's not intended to deal with the specifics of the paper (which I do in the marginal comments) but to deal with my overall expectations for the assignment. Also since they've got two of these papers (with different topics assigned, of course), and since the first one is weighted less than the second, the checklist on the first works as a guide for them for the second paper.
Here's how I do it. I create two columns. On the left-hand side, I list between 10 and 15 things that I was looking for in the paper. This column is labeled "Aspects of the essay that worked particularly well" or something like that. Generally, for lit classes, the top 2/3 relate to issues surrounding what I wanted them to do with the literature. Things like, "Paper responds to the assignment" or "Paper offers insights into the texts under discussion in a critical and analytical way." You get the idea. The bottom 1/3 (or so) of the list has to do with mechanical sorts of things with their writing, and their conformity to MLA style, etc. I don't tell the students how these things are weighted, but they intuitively understand that the stuff at the top is more important than the stuff at the bottom. Then, on the right hand side of the page, I have another column that is labeled, "Aspects of the paper that could have been stronger." Basically, this checklist is the inversion of the checklist on the left. So if on the left it says, "Paper offers adequate context (historical, theoretical, and/or critical) for the claims that it makes about the text(s) under discussion," on the right it says, "Paper does not offer adequate context (etc.)."
You will notice that all of these comments emphasize what the PAPER does or does not do - not what the WRITER has chosen to do. In part I do this because I think it softens my more direct marginal comments. In the margins, I tend to address the writer as "you" and to respond to the ideas that are there as the writer's ideas - not as some abstract thing inside the paper. On the checklist, I like to take some of the personal quality out in order to show them that I wasn't only responding to "them" but to what I saw in the paper as distinct from them. Also, the checklist gives me the freedom not to comment on more global issues in the margins, so in the margins I can actually respond to more specific points in the essay.
Now, what I do when I grade is this. I'll use my current batch of papers as an example. They had five topics from which to choose. First, I organize them by topic. Once I have them organized in that way, I put them in reverse order (worst to best) based on the opening few sentences of the paper. (In my experience, the opening few sentences really tell me all I need to know to assign a grade. Sometimes there are exceptions, but those are rare.) The advantage of this method is that I don't read all of the "good" papers at once, but rather I get little rewards at different points in the grading. Then, I grade the each paper, writing marginal comments as I go. When I grade for lit classes, I will note grammatical/punctuation/etc. errors the first or second time I see them - if I see more, I write in the margin something like "transitions could be stronger throughout" and leave it at that. This allows the bulk of the marginal comments to respond to the content of the paper and not to overwhelm them with copy-editing type comments. Because, as I've indicated in a few different comments I've left on other blogs lately, teaching writing is NOT the primary aim of essays assigned in literature courses, nor is it my primary aim in teaching literature. IF I leave a summation comment at the end (which I don't always do as it isn't always necessary), it's in response to how they dealt with the topic that they chose, or in response to a particular point that they made. It is NOT an overarching comment that deals with all of the stuff on the checklist. That's what the checklist is for.
Once I've done all of the marginal commenting, I then turn to the checklist, and I check off what they've done well and what could have been stronger. Visually, this shows me pretty much what grade they should get. If the checks are about 50/50 on each side, that's usually about a C - average. If it's 70-80/30-20, then the paper's in the B-range. If it's 80-90/20-10, then it's in the A-range. Invert that, and then you get a sense of the D and F range. I think this helps to keep me honest with the grading, and to detach a bit from students whom I personally like but who may not have done very well on the paper, or students whom I don't personally like who really did a good job. Again, though, this is not outlined on a point system, which I would fudge anyway in order to give the grade that I thought the paper earned. It's just a rough guide for how grades in certain ranges look on the checklist, and so if students who got grades in the same range were to compare their checklists, the checklists look very similar. Once I've done all of the checking off, I attach the checklist page to the top of the document. The checklist page does not include the grade. One reason I do this is because I do comment a lot in the margins, and it can freak students out, even though I give as many or more positive and/or just-engaging-with-the-ideas comments as I do negative ones. Having every paper look the same when they get it back I think makes it easier for them to deal. I also don't put the grade on the checklist 1) because that keeps students' grades more private and 2) if they saw the grade on the top, they would probably just ignore the actual comments on the checklist. (I put the grade either on the first or last page of their actual paper.)
Another reason that I like this method is that it means I tend to comment as much on the great papers as I do on the less great ones. One thing that often happens to students who are strong writers is that they get almost no actual feedback on their writing. (This is what happened to me throughout my undergraduate education.) A "Great!" on top of a paper does not feedback make. That said, it can be really time-consuming to write an end comment with all of the ways in which the essay was successful. The checklist means that I'm not tempted to ignore those things that students did succeed at in the interest of time. Also, because the checklist handles all of the main things that I'm looking for in the essay, it means that I can respond to whatever interests me in the margins without feeling like I'm not "justifying the grade." I never think of grading in those terms anymore - I really think of it as something that allows me to interact with my students one-on-one, and that is actually something I like (even if grading still blows).
And you may be wondering, how does this all work out time-wise for the grading? Well, in the past, when I did the extended final commenting and didn't use checklists, it could take an hour per paper, which is outrageous for 3-5 page papers. I'd estimate that my current method means I come in at around 15-20 mins. a paper tops (and again, I comment EXTENSIVELY on ALL of them), which means for this batch of around 20 papers, I'll spend no more than 6 hours or so on grading. To give you a sense, last night I graded from around 11 PM to around 1:30 AM with frequent (and sometimes lengthy- like when I composed that blog post, or another few times when I commented on blogs or composed emails- breaks). In that time, I graded 7 papers. I really don't think it's possible to grade any more quickly than that.
12 years ago
26 comments:
I like this idea very much - thanks for the detail. I do something similar (i.e. no points system, just circling a number from 1 - outstanding - to 5 - "not in evidence" for each category). I think yours - with the two columns and the *kinds* of categories you use - sounds great. It may make me rethink my own little sheet, which needs it.
I'm glad you found it useful. I have used the 1-5 scale before, and I found it kind of cumbersome for papers that did some things very well and other things poorly. It's fine when everything on the paper is a 3, but when you've got numbers all over the map, I found it actually made deciding on the grade more difficult. Has that been your experience?
Jeez, Dr. C, I hate to ask you to talk more about this since you've produced so much text about it already, but... could you say a bit about how your students respond to the sheet? My fear with something like this would be that it seems a bit impersonal, but perhaps your extensive comments mitigate that? (I like your idea very much, I'm just thinking through it...) One of the advantages, I'd guess, is that it allows them see that there are objective criteria for writing assignments (to hold off the "English is totally subjective" critique).
No problem - I'm happy to respond. I think that the extensive marginal comments do mitigate it the impersonal quality of the sheet. Also, I think that sometimes they *don't* actually want you to respond personally - they just want to know what they need to work on and what they did well without dressing it up, if that makes sense. So while I might have a lot to say about how to develop their *individual* ideas more effectively in the margins - praising some parts while offering constructive criticism in others - I think that sometimes what they like about the checklist is that it's not so complicated - it's "your argument could be stronger" or "you're not analyzing your examples fully enough" without all of the touchy-feely mumbo jumbo that can make it less clear whether a professor actually likes what you did or not.
I do think, though, that if all you do is the checklist, with no personal commentary throughout the paper, that it would be going too far in the other direction. I think it's important to respond to them in more complex and personal ways in combination with the very straight-forward checklist.
I strongly disagree with you on this point. I've never found checklists particularly useful - the full rubric is always better, particularly when students ask what types of things they can do to improve their work. When dealing with argument formation and structure, I find that checklists do not always offer students enough concrete feedback as to HOW they need to improve their arguments.
I do agree, however, that scales are problematic for a variety of reasons, the largest of which is that most professors use a scalar approach as a rubric, when these are not actually *rubrics* per se. A true rubric has specific details for each point category from 1-5. Most professors do not take the time to develop these scales into rubrics, which is why the students perceive them as subjective – and they are absolutely correct to do so.
Rubrics CAN and DO work, but until professors are trained correctly as to how to construct and use them, they cannot be effective.
Hi Wiccachicky,
I didn't mean to imply that rubrics can't be effective, just I've not found them to work very well for me - just not my style, more than anything else. And I agree that a checklist alone can't offer students feedback that is concrete enough and specific enough for some issues - one must either provide extensive marginal comments or meet with students IN ADDITION to the checklist.
I think I got pretty good training in how to construct effective rubrics, and I do use rubrics for grading things like presentations - I just felt very hemmed in by that format in responding to student writing and ultimately I found the format ended up being more time-consuming than my current method, which has had good results for me and takes less time.
There is no one way to do this - every instructor is going to have his/her own preferences about whether rubrics are the way to go, whether just doing narrative comments is the way to go, or something in between (which is what I propose here). I posted my version of this because New Kid asked about it, and I figured that others might have difficulty meshing the rubric form with their personal grading/reading styles. This is just another way to go, if someone doesn't particularly like rubrics.
Oh, and one more thing. If students want particular advice about how to improve, I make them come and see me so we can discuss the paper. This is not to say that my marginal comments don't address specific things - they do - but if students want more, then they've got to take some initiative and ask for help. I think that's not very much to expect of them, and it also is useful for me in terms of getting to know them and coming up with strategies that respond to their unique writing styles.
I'm not saying that rubrics can't be one way to give students writing instruction - I'm just saying that they are not what I prefer, and because I don't prefer them, I don't do as good a job as a teacher the form because it doesn't feel comfortable for me.
I like the checklist idea -- I'm going to use it when I read my honors philosophy papers later this month... it will be their first longer paper and will give them some good ideas for the revision.
Thanks!
I really like this idea; thanks for sharing the details, Crazy. It seems like you've found a good middle ground in grading writing; the students get a clear "objective" explanation of what did and did not work yet also receive pointed and more personal feedback through the marginalia. I may have to adopt this idea for some papers coming to me later in the semester!
Thanks for posting this. I was going to ask about it too, but have been busy grading. I really need to cut down the time I spend grading (I can't work 60 hours a week and still do everything else I need to do, like course work for my PhD), and I find I'm writing the same end-comments over and over again, so this will help. Thank you.
Your welcome :)
Oh my god! I must go to sleep! I mean YOU'RE welcome! (Clearly I have no authority to say anything about how effectively to teach writing :) )
This is brilliant.
I learned the hard away about assigning points in a rubric. Now I use checks on a "slider" scale, with words like "excellent," "good," "work on improving this," etc. But your rubric is fabulous.
I mean, your CHECKLIST is fabulous.
I clearly need sleep, too!
Interesting post. I don't use a rubric, but your checklist idea is an interesting alternative.
Academicdave -
That is yet another option, and actually one I tried, though my problem was that I could never remember the keystrokes for particular comments. (It would be different if I taught sections of the same course semester after semester, I suppose, but generally I've got three very different preps per semester, all of which have differing assignments, and my brain can't hold 30-45 commands at the same time, all of which will change in another four weeks.)
Another thing, and I wonder if others can comment on this: Do you find if you do a lengthy "personal" comment that the students who need the most work don't actually read it or appreciate it? This was my experience in my first few semesters at my current institution - students who needed more assistance have tended to respond better to the more "bulleted" style of the checklist or to individual conferences, rather than a lengthy "personal" paragraph.
I like the idea of having them type the main three things at the top of the revision! That's something I think I'll try in my writing classes - maybe at first just with students to whom I assign required revisions as an experiment. Very cool idea!
Just to come back to this conversation late. You're right, Dr. Crazy, that sometimes when I have marks all over the map on a 1 to 5 scale, it makes it hard to quantify a final grade...this is why I'm loving your idea!
On the question of checklists or rubrics being impersonal...they don't have to be, at all. Besides marginal comments, I always add a blurb at the end of the checklist...a short (or, if there's a disaster, not so short) paragraph indicating what really worked and what didn't work, and specific areas for improvement. This personalizes it - but you still get the benefit of a sort of standardization to make the students happy, and ensure fairness.
Genius... thanks!
Thanks, Dr. C! I appreciate it (belatedly).
I've never worked very well with checklists or rubrics (though I've also never had any formal training in how to create a rubric, so I'm sure there are better models out there than what I've tried to cobble together), but the fault is in my versions, I think, rather than in the technique; I just always end up writing just as many comments as I did before, so they didn't save me any time. Which means probably that they weren't good rubrics, rather than that rubrics per se are bad.
I do like the comment above about bullet-pointing the three (or whatever) things students need to improve in their writing, though; I do worry about the comments becoming more something that justifies a grade than something that helps students improve their writing, and the bullet-pointing would help me focus on what will help them in the future.
That being said, I have heard comp people argue that it's impossible for a one-time review really to help students improve their writing - that to do so you really need to do developmental comments and allow them to revise, etc. etc. I find that a little depressing, though, because while I do incorporate revision at times, I don't in every course. Here perhaps there's a distinction between writing for a history class - which I think my comments on a history paper can help them with - and learning writing from scratch, if that makes sense. I also think my comments on history papers can, hopefully, help students improve as historians, even if not always as writers.
But I think I'm digressing here...
Oh, last thing: the audience in question makes a big difference, of course; the students I have here are very different, writing-wise, than students I've had elsewhere, and I find my comments are a little different.
(FWIW, once I've read a few papers to get a feel for them, I usually take 15-20 minutes for a ~5 pp. paper.)
Here's what I think about the developmental thing: I believe in that when I teach comp - in terms of always allowing revision and even requiring it - as that's the point of the course.
It's not that I don't "believe" in it when I teach lit, but I handle it differently. I do NOT let students revise (unless they really went off the deep end with an assignment - I'll assign a required revision rather than give an F outright, though students can't achieve better than a C with the revision) in lit classes. Rather, the "developmental" thing is something that I address through linking different assignments. For example, in my general studies sort of lit classes, I'll either have four papers with the same assignment (so theoretically they improve while writing on different things), or I'll have two papers that are topic-driven, with the first paper worth less than the second (again, to encourage them to apply the comments on the first). In upper-div. classes, I tend to assign different kinds of "response to lit" assignments, all of which link to the bigger research paper that is the one BIG writing assignment that they do in there. Because the assignments clearly relate, and I explain the relation, I hope that they do get some sense of the process of writing and of how to develop one's ideas over time.
One thing that occurs to me with your particular situation is that you may have luck with appending a kind of checklist _at the end_ of the paper, that ONLY addresses writing issues. You could use the same one for all assignments, and it would only take about 30 seconds more than what you currently do, but would allow you to give at least some feedback on writing that you might not normally spend your time giving. That might be a way of demonstrating the centrality of writing to what they do as novice-historians, without adding to your workload in a substantive way. Just something that occurred to me as I read your comment.
Dr. Crazy, I don't use rubrics, per se -- at least not the kind wiccachicky is talking about, because they simply aren't useful for all of the things I'm looking for. There are just too many non-quantifiable things in my assignments. And before anybody tells me it's because I don't get it, it would probably be good to mention that I spent three years on an assessment and outcomes -based writing across the curriculum project where we developed and used all kinds of instruments for assessment. One of the most important things I learned is that you have to develop the instruments and the assignments in some kind of coordinated fashion, and that the types of instruments often vary with one's own teaching and the type of assessments one uses.
Having said that, imagine your checklist as a matrix, and that's pretty much what I have. All the things I'm looking for (well, not particular historical facts that need mentioning -- I know people who do that, and I think it's awful) in a column on the left, boxes going across from F to A, with explanations of what those things mean -- 'fails to ... ' to 'exceptionally good job ...'. As I'm reading, I make comments on the paper. Then, I go through and mark the matrix. Since the boxes are fairly wide, I put the checkmarks in kind of relative positions in the matrix, so a high B on one thing is to the right of a low B on another thing. Then, I add up all the numbers, based on about where the chacks fall, and divide by criteria for an average score, and even weigh them when I divide essays into content and mechanics categories. Most of the time, the average score is pretty much what I thought when I was commenting, but having to look at each category separately means it's hard to game the system. The matrix also helps students because it's a graphic representation of their work. For me, it's a win-win thing.
Thank you for this, Dr. C., and thanks to your commenters for the additional suggestions and clarifications. I'm facing the greatest number of papers I've ever had to deal with in the next couple of weeks (though still nothing compared with what my comp-teaching compatriots have to manage), and I'll be using a version of your checklist in the hopes of both streamlining my process and providing more focused and useful feedback.
More and more, the academic-blog world is becoming the resource my sad little pegagogy seminar could never have hoped to be.
Great discussion, great ideas. I use a sheet that (always under revision) to show that there is some such thing as objective criteria and meeting the goals of the assignment, etc. The sheet presents characteristics of an "A" paper, a "B" paper and so forth. I highlight the sections that are present in the student paper under review.
I don't mark extensively on final drafts as I save that energy for the drafts in progress that I review. They do get some personal comments from me but not many.
I like the idea of identifying three areas of improvement.
I'd add more but I have to - gasp - grade papers.
3 sections of comp = 75 students. I've had one batch for almost 2 weeks.
Onward.
Regarding the reference to "the paper" rather than "you"...
I like to write about what "you" do well, but about what "the paper" lacks. Thus: "You use good evidence to make a case for claim X, but I don't find that the paper supports the conclusion."
Great detail - and I like your dissection of the rubric approach.
I've been using an add-in for Microsoft Word to handle the marginalia part...it's called "Annotate" (very original) and you can find it here: www.11trees.com
Post a Comment