Tuesday, September 25, 2007

Experiments in Teaching Research Skills: Update

As I was planning for the semester, I posted about an assignment that I developed for my junior-level class this fall, a group research project. Now the deadlines are staggered, so I'm not receiving all of the projects at once, but the early returns are in, and, well, it's not the fabulous success that I'd hoped.

As I see it, these are the complicating factors:

  1. As specific as the assignment is, students did not follow directions. As much as we've talked about my expectations, students did not follow directions. As much as I offered help and clarification, students did not follow directions.
  2. Students put the project off until the last minute, which meant that they could not effectively do the group portions of the assignment. This probably meant that they didn't have time to follow directions either.
  3. Students didn't take the help that I offered, and that was available through the library, for the project. And if they'd taken that help, perhaps they would have been able to follow directions?
So, is this a bad assignment? Well, it's too early to tell for sure, but I really don't think so. I think, actually, that the ways in which the students are screwing up are really illuminating some of the biggest problems that students have generally with research at an advanced level. Research beyond what one does early in one's college career requires attention to detail, careful focus, and well, that one play by the rules of the game (including meeting the requirements of the assignment, following directions, getting to work early on projects so that kinks have time to be worked out, etc.). These rules are often directly contradictory to the way that students handle their academic commitments when they are in college. And if they don't screw up sometimes, they're not going to reprogram themselves to do that kind of intense work. Why? Because they won't need to reprogram themselves for that.

Another thing that the difficulties students are having with the assignments illuminates is the fact that they don't know how to work collaboratively with other people. It strikes me that this is an important skill to insist that students master because in the "real world" we all need to work collaboratively with people. It's easy for students to see their academic lives as personal experiences; their successes and failures as either only affecting them or as reflecting personal vendettas that they imagine professors have against them. One reason that they can perceive their academic lives in this way is because of how assignments are structured: if all of your work is independent, and if the only interaction you get regarding your work is with the professor, it's difficult to see that in doing this work that you are participating in a broader conversation about ideas, concepts, and critical debates. They don't see the wider ranging implications of their academic work.

It's easy to think that research work is personal because so much of it requires solitude. But when I think about the contours of my job as a professor, really I do a great deal of collaborative work. From meeting commitments for committees on which I serve to dealing with editors and readers' reports to working collaboratively with students, most of what I do is not ultimately solitary. Sure, there are parts of what I do that are. But then all of that goes out into the world and has to be in line with the needs of others. I suspect this would be even more true if I worked in another profession. Most students won't become professors, and even if they do, they will need to have the skills necessary to play well with others. If we don't teach them those skills, then are we really giving them the education that they need?

And so, at this point, in spite of the early returns, I still feel like this is a good assignment. I'm hoping that future groups will learn from the mistakes of those who go before them (which they will have the opportunity to do, as I'm going to post the research projects on our course blackboard site, which they've known all along, and I've now decided that I will post them to the site with comments and necessary additions).

So I'm teaching this course again in the spring, and when I do, I do think that I'll make some changes to the assignment.

  1. I'm going to require each group to schedule one out-of-class meeting with me to discuss the project. I think that I've given them too much independence as the assignment now stands, and I think that requiring one meeting with me will be a good way of forcing them to get to work earlier and will enable me to address problems before we're at the grading stage.
  2. In combination with number one, I think that I will require that they have a draft of the project prepared for the meeting, so that I can provide comments.
  3. I will also be able to offer the next section of the class a model of the assignment because I'll have samples from this section. I had thought about doing a version of the assignment myself and giving it to them this semester, but the reality is that I just didn't have time to do it with the book and everything else.
So yes, there are kinks to be worked out on my end, too, with this unconventional research assignment. I don't want anybody to get the impression that I'm vilifying the students in this situation. I think part of the problem is that this sort of work is foreign to them.

But. I know with some of the students that the problem is not that the work is foreign to them (only) but also that they blew the assignment off. Because it is very straight-forward, they assumed that it was "easy." They assumed that it wouldn't require intense focus and time. About this, they were sadly mistaken.

You know, in thinking about this assignment, I'm wondering how I myself learned these skills. I think I had similar experiences as an undergrad with underestimating the level of time and commitment that certain assignments would take, but ultimately I was a "good" student, and that carried me through. The same is true for some of the students in this case. But I think part of my desire to break the process down for them has to do with the fact that in my own education, I could have used more help with this stuff.

Another thing I've been thinking about is the way that technology has transformed the way that students do research, in ways that aren't entirely positive. For example, I don't think that students tend to take many notes when they do research, something I still do as a leftover from the days before electronic sources and the days before I had a computer (for yes, I did not own a computer until I began my PhD program). I no longer use note cards, but I do take notes on paper, and at the top of a page of notes for a source, I put the correctly formatted bibliographic citation, just as I had to do back in the days before citation machine and easily accessible online databases and library catalogs. I don't think students have these sort of research habits, in large part because they're no longer essential to writing a paper. The problem is, if you don't master those sort of habits, then you're always half-assing it. You never really master the technical skills necessary to really solid and deep research. But how do we teach those kinds of habits? Because they aren't essential, if you require students to do this sort of thing, you're requiring an anachronism - they'll see it as busy work. They need to come to this stuff on their own, to some extent - even as I needed to come to it on my own during my freshman year of college.

Similarly, I'm not sure that students ever compose much on paper. Now, I no longer compose everything by hand, as I did back in the stone ages before I had a computer, but when I'm working on a really difficult passage, I find that writing longhand slows me down in ways that are really positive for the writing, and so I can go deeper than I tend to do when composing at the keyboard. (You may wonder why my blog posts often tend to be so long. One reason is that I type approximately 100 words a minute with approximately 98% accuracy - at least the last time that I was tested - and so I can pretty much compose at a keyboard as quickly as a person might speak and nearly as quickly as a person might think. Ah, there is always transcription typing to fall back on should the whole academic career not work out.) Again, though, I can't necessarily require that they compose long-hand, nor am I certain that this would solve some of the problems that I see.

For me, all of this stuff connects back to the fact that students often perceive academic work as something that happens outside of any real world context. Because they don't see these activities as having value - other than the value that is signified by the grade that is ultimately assigned - they don't necessarily invest the time and energy necessary to do their best work. And with my students, well, why would they? Most often they have other commitments that are more concrete than the commitment to their own education. I recognize that. And so I suppose what I'm trying to figure out is how to make the commitment to their education concrete and real in ways that it isn't for them naturally.


Notorious Ph.D. said...

The first part of this post sums up, I think, the core problem of assignment desigin: we set things up to get students to learn or exercise certain skills, but if they try to do it all at the last minute (and yes, I did this occasionally as an undergrad, too), the assignment is impossible, so the work they do doesn't accomplish the goals we spent so long planning. Sure, we can offer lots of help, but there is a line we cannot cross; that is, we can shift our assignments to accommodate less background knowledge than we'd like, but not to accommodate laziness. When it comes right down to it, the process itself is a large part of what we're after.

As for collaborative work, I have to admit that my first reaction was, "I hate collaborative work -- that's one of the reasons that I like my job." But reading further, I decided that you're right: the basis of our work is solitary, but without collaboration, we produce crap, and the best in our particular business are those who know how to work well with others -- presenting work, exchanging chapters, listening to and absorbing constructive critiques, then passing the favor on to the next generation of scholars.

litprof said...

Your point about note-taking resonates for me. Even if my computer is right in front of me, it never even occurs to me that my reading notes might be typed and not written longhand. I, too, put the ref. at the top of the page. The downfall of not having it electronically is that you do not have the benefit of the "find" function to look up key words, etc.

Your idea of meeting with the group ahead of time is a good one. Any kind of scaffolding is also pretty useful because it divides up the tasks and makes them seem manageable, builds up a logical sequence, and makes it obvious that the work will take time (hence, one step must build on the previous one). The checking-in types of things can be absolutely minimal and thus little work for you, but the deadline serves as a motivator for them.

Inside the Philosophy Factory said...

I stopped doing in-class presentations because I couldn't solve the slacker problem... could you explain how you structure the assignment so that a) one or two people can't slack and b) that poor work habits of one person don't pull down the grade of the others?

I'd like to go back to presentations because I think students learn a lot from the process, but only when the group ethic and expectations are such that they must work together.

Dr. Crazy said...

I see what you're saying about scaffolding, but one of the difficulties with building in more of that beyond one meeting with me is that the assignment really is so straightforward that more checking in seems just plain silly. Also, and this is huge for me here, this is an upper-level course. These students are *not* brand new to the research process, nor are they brand new to college. They should have ownership over their educations and the ability to do work independently, and if they have questions, they should use the resources available in order to get those questions answered. If they aren't given situations in which to practice doing that stuff, I don't know how they'll learn that they need to be able to do that stuff. And I really do think that some students need to screw up in order to learn those lessons. I'm certainly not saying here that everything on the assignment side is perfect as is, but I am reluctant to break the assignment down to the point that it doesn't have teeth anymore, if you get what I'm saying.

As for the structure of the assignment, here's how it works.

1) There is no presentation component. I've done group presentations and they generally just don't go very well. So when I have a presentation assignment, I make those presentations individual to solve that problem.

2) Only two parts of the grade are "group" related. The first requires the group to read, summarize, and analyze a theoretical selection (v. short). Now, it's true: if there are slackers, they can ride on the work of others for this one portion of the grade. BUT it isn't worth very much of the grade, and it will not produce a passing grade if you get full points on this part of the assignment. The second "group" grade comes from formatting. So yes, there has to be a point person to put everything into one word document in the proper font, but that person isn't responsible for what is in the other parts of the project - things like problems with MLA style, etc. will affect individual not group grades.

3) The rest of the project is individual, and it is clearly labeled as to who did what. And so if one person doesn't turn in their part of the project, I know who that is. If one person USES SOURCES ON THE SYLLABUS (which the assignment clearly says not to do, and which is entirely counterintuitive for a RESEARCH project but which yes, did happen) it's only going to hurt the person who made that choice. If a person doesn't use any scholarly sources (in spite of discussion of what those are and in spite of library instruction showing them where to find them) then it won't hurt the group grade - only their individual grade. Thus, in the grades for the first group, I anticipate grades ranging from A/B to F. Now, the F's won't be zeros, as they probably would have been were the assignment totally independent, and so in that way those people benefited from being in the group. But they are no way riding on the backs of more responsible group members to an easy A.

datamuse said...

Do you have them do a librarian-taught research session at the library? It doesn't solve all of those problems, but it does at least get them using the library, which in my experience some of them won't do otherwise. Those online databases are convenient but not necessarily easy to use, and from what I've seen most students don't know how to get the most out of them. I'm a librarian myself so I don't have anything to suggest for the rest of your quandary, unfortunately.

Dr. Crazy said...

Not only did we have an in-library instruction session with a librarian - tailored to the assignment and the needs of the course - but also this class has its own personal librarian assigned to it. In other words, the problem isn't lack of instruction or information at all. Something else is the block.

litprof said...

Scaffolding is about how you build new skills onto existing skills, not really about extensive checking or non-extensive checking. They can still take on individual responsibilities and learning when an assignment is well scaffolded. Something I have used in the past in a 200-level literature course comprised mostly of juniors and seniors is requiring them to turn in annotated bibliographies. Many students, even if they are in an upper-level course, will simply not realize *what kind of* work and effort is required for "research"/compiling "outside sources." The annotated bibs do not take a long time to grade (you scan to see that it's done correctly and that the annotated bib makes sense--it can be a "you did it" or "you didn't" grade), but they are immensely useful at making sure that students are actually finding proper sources and getting the meat from those sources.

I know it's really important to you that students learn to do the work independently. Coming up with an annotated bibliography (or similar assignment--doesn't have to be that) is something that requires them to put in time and effort on their own; it does not compromise independent work. And feedback time for you is minimal, but gives you a point of engagement in case there are problems.

I think in the end, scaffolding is really about explicitly structuring activities so that they build onto one another in a way that makes sense to students, and in a way that allows for them to get some help if they need it along the way. When people use scaffolding even in organizing graduate seminars, I can't see why it wouldn't be useful in an upper-level undergraduate course.

Dr. Crazy said...

I'm aware of what scaffolding is, and the thing is, this assignment is PART of a scaffold - the assignment itself is basically an annotated bibliography. The only thing that makes it a "group project" is the fact that they need to read something and provide one two-page summary/analysis and to submit all of their work in one document. This, ultimately, then provides the framework for the research that they'll need for their second essay in the course. And I've been very explicit about how the assignments fit together. So I don't want to give the impression that I don't believe in scaffolding or that I'm not a fan of it or that I don't use it in my classes. The point here is that I *do* and it doesn't necessarily make a difference for all students. I know the *theory* of it, and that theory often does translate into good practice and does often benefit students, which is why I *do* it. The point here is that you can put all of that in place and STILL the students do not do what is assigned.

Again, I do think that the next time I do the assignment that it will be a good idea to build in a required meeting with a draft due at that time, but really, I can't imagine what else I can do given the fact that the assignment is so basic as it is. Anything more to me really would make the assignment pointless, but perhaps I'm missing something?

To be clear: the assignment requires the summary/analysis thing (of an assigned text, which is available electronically so they don't even need to look for it) and then each individual person needs to find three primary sources and three scholarly sources that relate to the theme of the unit and annotate those sources. That is IT. We spent time talking about the difference between primary and secondary sources in class, we had library instruction so that they would know how to find these things, and they have their own personal librarian that they can consult, who checks in DAILY with a blackboard discussion thread set up specifically for this purpose and who gave them his email in case they wanted to consult with him privately. I've mentioned in class countless times that people should be meeting with me and I've opened up the floor for questions about the assignment during class time. Birds chirping, silence. If you have ideas about what else I should be doing, I'd be interested to hear them. (That sounds snarky, but I don't actually mean it that way - seriously, what besides one meeting with a required draft could I build in?)

litprof said...

Hmm. I'm sure we can speculate endlessly on what might work or what might not work (and such thinking is of course helpful and good), but I wonder if asking the students to reflect on why they did not successfully meet the tasks would be useful. I'm certain that laziness, procrastination, etc. would figure into some of the answers here (as well as being busy and other things in life taking precedence), but sometimes student feedback tells us things we aren't seeing. Especially if you are thinking about how to do this better in the future, and most definitely if you are wanting students to learn from this experience and to figure out how to do it "right" next time, a short self-reflective assignment on the work they did on this project could be very useful. It could be very specific--X was a goal/requirement of the assignment. Did your project meet this goal? Why did it (or didn't it)? What could you do differently? etc.