Tag Archives: grading

Rubrics: A Best Friend for Teachers & Students

Jessica Greenfield, PhD
Director of the Cooper International Learning Center and Lecturer in Italian
Contact at: jgreenfi@oberlin.edu

Rubrics are not a new-fangled teaching device designed by some evil assessment gurus who thought we didn’t have enough hoops to jump through as is. They’ve been around for years and, far from being a burden, are, in my opinion, a teacher’s best friend and a frequent lifeline.  As focus continues to grow on student evaluation of teaching effectiveness, and as we continue to reflect on our own teaching practice, it is important to ensure that the expectations we have for our students are clear and that we are evaluating them fairly and in a way that is explicit and obvious to them. Most of us do this as a matter of course; but rubrics are a fantastic tool to make this process as transparent as possible.  

www.stripgenerator.com

A few years ago, I became obsessed with assessment and feedback.  I noticed that my grading would often shift from lenient to harsh within a single class, leading me to grade and re-grade each assessment several times as I tried to be equitable.  I began to worry about fairness in my grading and clarity in my assignment design and explanations.  Furthermore, I discovered that I often was assessing too many elements within one formative activity.  And so, I began to search for better methods of assessment, which led me to the American Council for the Teaching of Foreign Languages, and the Integrated Performance Assessment (IPA).  While there are many elements of the IPA that I love, I was particularly enthralled with the rubrics provided in the guidebook, since they offered me a clear and compelling solution to the problems I had been experiencing.  I now use rubrics in a number of ways: as a tool to help students evaluate themselves, as a guide for designing and grading assignments, and as a method to bring students into the assessment process when we create rubrics together, specifically for summative assessments that are more creative in nature.  

Jennifer Gonzalez, Cult of Pedagogy: https://www.cultofpedagogy.com/holistic-analytic-single-point-rubrics/

While there are a lot of different rubric forms, I use two kinds most of all: ANALYTIC RUBRICS which are descriptive, finite, and exhaustive in the consideration of component parts, and HOLISTIC RUBRICS which use a single scale to apply an overall judgment in a specific area. I create “analytic” rubrics for each of my formative assignments, and “holistic” rubrics as a way of evaluating student participation or, often, for end-of-unit or end-of-course summative assessments.  One can also create a SINGLE POINT RUBRIC in which you have a given standard you are assessing, and the instructor then adds comments during the performance. For example, in an oral presentation single point rubric, the instructor might have a standard such as: student links the presentation to reading materials from the course syllabus. The instructor would then evaluate that one element by providing comments on where the student succeeded or failed in meeting that standard. To provide a clearer sense of these various kinds of rubrics, check out some examples and further definitions here.

Analytic Rubric Example

Analytic Rubric (Greenfield)

Holistic Rubric Example

Holistic Rubric (Greenfield)

Single-Point Rubric

Single Point Rubric: Jennifer Gonzalez: https://www.cultofpedagogy.com/holistic-analytic-single-point-rubrics/

The research is definitive in showing that rubrics not only provide a clear and fair framework for evaluation of student work, but also is conclusive that the most effective rubrics follow certain steps (considering learning objectives, deciding upon the elements to be evaluated, and then weighing each of those elements as it contributes to the overall evaluation). Personal experience reinforces the research: good rubrics are clear, descriptive, and avoid generalities.  They use such objective terms as “student uses…”; “student chooses…”; “student is successful in…”, rather than the more subjective, “I felt that…,” or, “it seems that…”, in order to provide as factual and objective feedback as possible.. The best rubrics are also context specific.  I never use the same rubric for two different kinds of assignments and I always make sure to review and adapt my rubrics from semester to semester to check for appropriateness.  Often, I will find some recurring area for improvement among student performance or an area that I did not describe well, which I update for the following semester.

Interpretative-Novice (Greenfield)

While the process of creating rubrics from scratch can be time consuming (and, rather than reinventing the wheel, consider building off of the many resources available online or from our own CTIE), the payoff is large; you can reduce time spent grading and confusion among students as to what you’re looking for, and you will also be providing more transparency for students throughout the coursework.  My teaching has definitely improved since adopting rubrics as my primary evaluation tool, mostly because I can spend more time focused on lesson preparation and less time reading and rereading student work. Some of the biggest improvements in my instruction since using rubrics include the following:

  1. My grading has become more uniform and fair.  I no longer need to worry about whether I’m being too lenient or too harsh; I write explicit descriptions that allow me to grade fairly.  It’s also cut down on grade inflation, which is always something at the back of my mind.
  2. In giving students the evaluation rubric at the same time that I introduce the assignment, I can be sure that students understand the elements within the assignment that I’m most interested in. I remember the day I discovered that the point values in rubrics don’t have to be equal across columns and rows, and that has allowed me to show students where the bulk of their energy should go in a given assignment. If I want to focus on vocabulary over style, I will give that column more points.
  3. Feedback is easy to give within the framework of a well-developed rubric as you are commenting on that specific aspect of the assignment.  Not only is evaluation time greatly reduced, but also the time an instructor spends writing comments and critiques.
  4. The very process of creating a rubric also helps me ensure that I’m clear on what the assignment is asking students to do. Developing a rubric is often a metacognitive process which prompts the instructor to reflect on the task that she will present to students, and encourages her to take the necessary time to determine if the instructions are clear and appropriate.

Finally, I’m so grateful for the rubrics tool in Blackboard.  Several years ago when I flipped my class, I discovered that I could recreate my rubrics right within the LMS and make them available to students.  You can also copy them from one semester to the next, so once you’ve done the work, it’s easy to utilize them again.  This feature also allows students to revisit the tool from the gradebook function and reflect on their performance before coming to you to discuss how they did on a certain task.

Composition with revisions (Greenfield)

If you are interested in making your own rubrics, my recommendation is to follow these steps for success:

  1. Reflect on the task and content for which you are creating the rubric.
  2. Create Student Learning Objectives for the assignment with an explicit set of expectations.
  3. Group and label the elements so that students can see what you will evaluate and you are clear on what elements you want to look for.
  4. Apply the rubric form and add descriptors in each box.  If you are applying points to the assignment, this is the time to weight the distribution of points as well.  
  5. Discuss with colleagues and/or students and make appropriate adjustments.

Some useful online resources:

Contract Improv – Three Approaches to Contract Grading

Steve Volk, March 27, 2016

Benin Plaque, c. 16- 17th century, brass “lost-wax” technique, British Museum Af1898,0115.38

Benin Plaque, c. 16- 17th century, brass “lost-wax” technique, British Museum Af1898,0115.38

Students in museums studies, more so than casual visitors, are frequently confronted with the question of how specific artifacts made their way from their point of origin into the museum where they are displayed for our enjoyment and edification. For some objects, the answer is relatively straightforward: the painting originally in, say, the French royal collection, was purchased by a dealer who sold it to a collector who donated it to the museum. For other artifacts, particularly if the museum in question is the British Museum (the end point of a vast collection of imperial booty), the origins of the artifact is more troubled. The catalog entry for this “Benin Plaque” (left), dating from the 16th-17th centuries, calmly notes that “following the British occupation of Benin City (Edo) in 1897 objects made of brass, ivory and wood were seized by British force from the royal quarters and various storerooms.”

But as this information doesn’t appear on the object’s label in the gallery, the viewer has little sense of the violent history — the imperial relations — that underwrote the trajectory of the plaque from Benin City to its current abode on Great Russell Street in London. Museologically and culturally speaking, that’s a problem. If museums are to represent (and not simply appropriate) objects from their colonial empires, the history of that displacement must be kept in sight.

This may seem an unusual way to begin an essay on grading, but I thought of the Benin Plaques and their absent labels as I prepared another set of grades for my students. Grading (as I’ve written many times before here and hardly need to remind you) is about as eagerly anticipated by teachers as a colonoscopy (and at least those are served up with propofol.) There are any number of reasons why this is the case, and at least some of the problems of grading do come with relatively straight-forward solutions. If you can’t bear reading the 27th paper on the role of the cottage in Frankenstein, then open your assignments to allow for a greater variety of responses. If the assignment essentially requires that students feed back to you what you’ve given to them, don’t expect to have an enjoyable experience reading them. Try completing your own assignments and if you find them boring or not conducive of learning, change them so that students can use the assignment to demonstrate both mastery and application.

Full Disclosure

Other issues involved in grading are more difficult to resolve, which brings us back to the Benin Plaques. What everyone knows, yet no label discloses, is that grades represent the ultimate power that faculty hold over our students. As much as our professional code of conduct requires – demands – that we grade fairly, objectively, and without regard to extraneous factors, there is no denying that we are humans and that, when it comes to grading, we are both shaped by, and must contend with, a variety of factors that make that difficult, if not impossible. These range from simple tiredness to complex issues of prejudice including racism and sexism. [See, for example, here (the impact of the teacher’s emotional state) and here (the impact of the stereotype threat) as examples.). Perhaps, just as the Benin Plaques should include on their label an indication of the nature of the power that brought them to the British Museum, so too should we include a label on all of our tests and assignments:

Warning: As much as I will try to grade your assignments objectively, fairly, and without prejudice, and as much as I will attempt to forget how annoyed I was with you when you [fill in appropriately]: didn’t do the reading/watched a Beyoncé video on your laptop instead of listening to what I was saying/left the class three times to go to the bathroom, I am only human, so caveat emptor!

When Life Is Young, British Library HMNTS 10413.dd.1

When Life Is Young, British Library HMNTS 10413.dd.1

Grading has a way of reversing the intent of teaching, not only closing off a formative process of dialog and reflection, but often contradicting what we have been insisting all semester: “It’s not about the grade.” Well, what if it is? And how do we tell students with a straight face not to worry so much about their grades when they know (as do we) that when all is said and done, the grade we give them can/will influence whether they get the fellowship they need to pursue their studies. I would venture that, for most of us, the problem is not that we feel pressured to give “C” work an “A” (although grade inflation, particularly at elite institutions, might suggest otherwise), but rather how we maintain a straight face when we suggest there is a clear and obvious difference between a “B” and a “B+,” between a “B+ and an “A-.” Particularly in the humanities and social studies, but likely in the sciences as well, we know full well that extraneous considerations (those extra trips to the bathroom!) can influence our decisions. There’s no way around the fact that a serious evaluation of our students’ work is so much more complex than can be expressed in that single letter, and giving a student a “B+/A-” really doesn’t resolve the problem.

What else is wrong with grades? Let me count the ways! As  Steven Mintz, Executive Director of the University of Texas System’s Institute for Transformational Learning and Professor of History at the University of Texas at Austin, pointed out,

  • A single, over-all grade conflates elements that need to be disentangled.
  • They tend to overly reward lower-order thinking skills (such as memorization and recall) rather than higher order skills (involving analysis, application, and synthesis).
  • Grades too often fail to accurately reflect student learning or mastery.
  • They are frequently de-motivating and discouraging.

Nor is this a recent finding: studies as early as 1912 questioned the utility (and validity) of grades, and research has fairly consistently underlined some of the main problems in grading practices.

Typical-student, British Library HMNTS 10413.dd.1

“Typical figure, showing tendency of student life,” British Library HMNTS 10413.dd.1

Does that mean that we should stop giving grades? Very few colleges or universities have abandoned the grading system altogether. Hampshire College, where faculty write narratives of their students’ learning rather than assigning grades, remains the exception. But even there, complex narratives probably won’t work in a class of 80 students, nor is Pass/Fail an option without draw-backs in a world in which grades are the norm. A “Pass” in Organic Chemistry might not help a student when she applies to med school.

Valen E. Johnson, professor and head of the department of statistics at Texas A&M University at College Station argues further that if we didn’t grade, “students probably wouldn’t work as much and wouldn’t do homework and wouldn’t study for exams”? While this is not universally the case, we can (and probably should) admit that we share at least some of the blame for not making learning more intrinsically motivating. But such an observation can only get us so far. Ultimately, we need to think about new ways of grading that can address some of the shortcomings of the current system. That’s where contract grading comes in.

CONTRACT GRADING: THREE OPTIONS

Contract grading, which actually was first used some decades ago, more recently has been gaining traction in higher ed. Briefly, contract grading attempts to reduce the subjectivity of the grading process for faculty and the induced passivity of students within the evaluation system in an attempt to arrive at a more integrative and meaningful process of assessment.

There are a variety of approaches to contract grading, each designed to meet an instructor’s intended outcomes, but all share the fundamental goal of clarifying the grading process for students so that they can make more informed decisions about their actions. While there are a number of different types of contract grading options, I’ve  summarized three different contract-grading approaches here. At the same time, I’d encourage you to talk to your colleagues about how they grade; raise the issue at department meetings: you’d be surprised how many have adopted this method of grading.

Contract grading as a means of negotiating authority

Songs of a Savoyard, British Library HMNTS 11651.k.42

Songs of a Savoyard, British Library HMNTS 11651.k.42

The essential factor in determining a grading approach, at least as I see it, is deciding what you hope the process of grading can achieve in the broadest terms. For some, revealing and addressing the nature of power relations within a classroom environment is the central element that a grading system can address. Ira Shore, for example, has written much about the importance of creating a democratic classroom in which power is both fully disclosed and openly negotiated with students. Similarly, Isabel Moreno Lopez argues that teachers should create a critical classroom in which “authority and responsibilities are shared between teacher and students, empowering all course members to become active, responsible participants of the learning process, not merely passive consumers.” For both, grading is a means by which power can be shared through a collectively negotiated contract co-constructed at the beginning of the semester.

Here, in a condensed version, is how Moreno Lopez describes her contract grading system:

The negotiation process starts at the beginning of the semester when the teacher presents the elements of a contract grading system to the students. In general terms, the grading system is based on the quality and quantity of work students are willing and capable of doing. That is, if a student signs a contract for an “A,” s/he will do more work in the course than the student who contracts for a “C.” The quality of work will also reflect the contracted grade. Students are permitted to rewrite the written assignments as many times as necessary to attain the contracted grade.

At the start of the semester, then, the teacher opens up class-time to discuss both the syllabus and the grading system. Then, s/he asks for questions, amendments, and comments on the original proposal. A debate follows, after which the students sign the contract, as amended by themselves, and keep a copy for their records. During the semester, the negotiation process continues, both in class discussions and in comments in the students’ journals. At the end of the semester, based on the contracts and their performance, students discuss with the teacher their final grades. This grade might be the same they contracted or might have varied depending on their performance and progress.

Moreno Lopez suggests that this negotiated grading system is valuable in two ways: it helps students see learning as a process and not an end, and it “encourages students to be active participants in their own learning process by allowing them to cooperate in what is usually considered the ultimate prerogative of the teacher: the assessment process.”

Shor, Moreno Lopez and others who engage in this form of critical pedagogy identify the classroom as a political arena where differences of power are necessarily, and properly, brought into the center of teaching where they are negotiated. In such a context, struggle and conflict is both inevitable and appropriate insofar as it is a reflection of the larger society, not a “bubble” separate from it.

Non-negotiated contract grading to improve learning

Spectroscope_British Library HMNTS 10027.ee

Spectroscope_British Library HMNTS 10027.ee

The grading contracts used by Jane Danielewicz and Peter Elbow in their composition courses are similar in some respects, but they are less concerned about using the classroom to negotiate authority than Shor or Moreno Lopez. Instead, they see their goal as creating “a classroom where both teachers and students get to give as much time and attention as possible to writing—not politics and culture. Of course political and cultural issues turn up in student writing, but our tendency is to discuss the effectiveness of the writing more than political and cultural issues themselves (not that one can ever completely separate the two).”

Danielewicz and Elbow present the grading contracts to students at the beginning of the semester rather than co-constructing them with student input. By using contracts, they seek “not only to help students learn more and function better as learners; we also want a grading system that encourages them to be the kind of persons our world needs; furthermore, we want to make our own teaching easier and more satisfying.” And they add, “That’s all.” Indeed, that would be plenty.

Here is a summary of the main elements of the Danielewicz-Elbow grading contract:

  1. Attend class regularly—not missing more than a week’s worth of classes.
  2. Meet due dates and writing criteria for all major assignments.
  3. Participate in all in-class exercises and activities.
  4. Complete all informal, low stakes writing assignments (e.g. journal writing or discussion-board writing).
  5. Give thoughtful peer feedback during class workshops and work faithfully with their group on other collaborative tasks (e.g., sharing papers, commenting on drafts, peer editing, on-line discussion boards, answering peer questions).
  6. Sustain effort and investment on each draft of all papers.
  7. Make substantive revisions when the assignment is to revise—extending or changing the thinking or organization—not just editing or touching up.
  8. Copy-edit all final revisions of main assignments until they conform to the conventions of edited, revised English.
  9. Attend conferences with the teacher to discuss drafts.
  10. Submit their mid term and final portfolio.

In other words, students get a “B” based solely on what they do, not on any evaluation of their work by the professor. Grades higher than a “B,” however, depend on the teacher’s evaluation of the quality of their writing. They will discuss in class what “exceptionally high quality” writing means, making the criteria as public and concrete as possible, but they don’t give students power over “high-grade” decisions.

Although they don’t evaluate the quality of their students’ writing up to a “B” grade, they also don’t withhold evaluation as they continue to provide students with feedback on the strengths and weaknesses in their work, both drafts and final version. But the evaluation (up to a “B”) is decoupled from grades. “As a result,” they write, “students don’t have to heed any of our judgments or advice when they revise their papers (though they must revise).” They want their students to feel that the evaluations they conduct are “from individual persons: yes, experts about writing, but individuals, nevertheless, who cannot pretend to be wholly impersonal or fair.”

Their article (“A Unilateral Grading Contract to Improve Learning and Teaching”) offers a fascinating discussion of how they came to the various elements of the contract, why, for example, they picked a “B” grade as the base-line mark for their contract (“Our original reasoning was merely timid—crassly negative and pragmatic: we were scared to ‘go all the way.’”), or whether their contract is actually less “fuzzy” than a standard grading system. “How can we defend ambiguous and arguable criteria like ‘conscientious effort,’ ‘thoughtful feedback,’ and ‘conscientious participation?’” they ask. “First, we don’t accuse someone of failing to meet one of these fuzzy criteria (‘no-effort,’ for example), unless the violation is grossly flagrant (e.g. drafts far short of the required length). Secondly, we’ll always take the students word for it.” In terms of a final, summative, grade they are persuaded that their decisions were relatively easy to make. Students who didn’t fulfill the contract (including some who were excellent writers), were disqualified. They then focused more closely on the remaining final portfolios that they found to be particularly strong.

Contract grading to encouraging active learning and community building

"The Life of George Barnwell; or, the London apprentice of the last century," British Library HMNTS 12621.dd.5.

“The Life of George Barnwell; or, the London apprentice of the last century,” British Library HMNTS 12621.dd.5.

The final example of contract-style grading is Asao B. Inoue’s community-based assessment approach. Similar to all contract models, Inoue, a writing instructor at Washington State University, moves away from teacher-centered assessment and evaluation while encouraging students to take more initiative. But, more than in the previous models, Inoue seeks to create a classroom in which “students take control of all writing assignments, their instructions, assessment criteria, and the practices and reflective activities that go along with their writing.” Such an approach, he maintains, “encourages a community of writers that are implicated in each others’ writing and assessment practices, and gets them to critically engage with these practices.”

Inoue’s model underscores the fact that assessment is a vital component in the act of writing. He spends considerable time discussing with students what they want out of their papers and how they should be read and assessed. It is a complex and recursive process that begins when the class collectively creates its first assessment rubric, a set of guidelines that everyone agrees to, and that they will use both as writers and assessors. This first rubric will be revised continually as the class moves from the early stages of writing (paragraph writing) to position papers and final essays.

Any student can suggest a rubric revision or raise a question about the rubric at any time. To test and revise the iterated rubric, class members write two separate paragraphs, each receiving three peer assessments that use the in-process rubric. The class — instructor and students alike — uses what it has learned from the paragraph assessments to revise the rubric, which becomes the new starting point for on-going assignments, and so on. Over a month, each student writes a position paper, receives responses and assessments from the entire class (both on paper and through class discussions), posts a revision of the position paper based on those discussions and input, gets a more formal peer-assessment of the revision by a few colleagues, writes an essay (often based on the position paper), and finally receives a formal peer-evaluation of the essay. The same process is repeated for a second paper. (The process is schematized in the illustration below.)

ChartWhen the students assess each others’ writing, they are not looking to identify an “A” paragraph or an “exemplary,” or “outstanding” one. Rather they use the rubrics to help them identify proficient paragraphs, ones that reach the proficiency markers they set out at the start of the process. If a paragraph hits these markers, then it has done its job.

Here, for example, is what the class came up as a “proficient” paragraph with after their discussions: A proficient and adequate paragraph will . . .

  • Contain a consistent claim
  • Support claim with appropriate evidence (when needed)
  • Elicit thought on the part of the audience
  • Adapt to or consider its audience
  • Use clear and concise language
  • Use appropriate language and grammar
  • Contain three or more sentences

They continue to refine this set of criteria over the course of the semester.

As Inoue explains,

I try simply to provide the structures for my students to create a rubric, re-think it, write from it, use it to assess each other, and, of course, reflect continually upon all these practices. I distribute guidelines, provide due dates, post weekly reflection prompts, and pose additional questions in class that facilitate assessment discussions on student writing. In short, I try to coach them toward sound assessment practices and active learning stances by making them do the hard work of assessment. I encourage them to voice disagreement, show agreement, and elaborate and qualify ideas. I act as a facilitator, questioner, and listener when we talk about each other’s writing. I try to keep us focused on our rubric in our assessment discussions, yet not be a guard to ivory towers… Our class writing isn’t about what I want — it’s about what the class can agree on they want and can justify in some way so that agreements can be made… My students must debate and decide on all the important decisions regarding their writing in the course from start to finish. The class is about them learning not me teaching.

The key to making assessment work pedagogically, according to Inoue, is periodic reflection on the assessment activities. He does it once a week based on open-ended prompts to point the students to the areas he wants them to reflect on. Community-based assessment pedagogy also offers ways to build a pragmatic sense of community that is active and purposeful.

If our purpose, as teachers, in assessing and evaluating student writing is to help students learn — if assessment is inherently a learning practice (which I think it is) — then the teacher shouldn’t control all of the process. And Inoue concludes:

Community-based assessment pedagogy, as described here, boils down to three classroom imperatives: (1) encourage active learning stances by allowing students to assess and evaluate their own and their colleagues’ writing practices, and make these assessments meaningful and purposeful, (2) situate assessment practices within a community of knowledge makers who construct assessment rubrics and define and justify assessment practices, i.e., encourage the class to work for one another as mutual agents working with and for each other’s benefit, writing for each other, and negotiating hard agreements together, and (3) give lots of opportunities to reflect on assessment that speaks to the larger class community, in order to theorize about writing, rhetorical conventions, assessment, and the judging of writing from specific criteria, i.e., what we say about what we are doing (or did) can help us do it better in the future. In my versions of this pedagogy, these imperatives rest on a framework of recursive, repeated writing and assessment activities.


 

As you will have noticed, none of these models makes grading “easy.” Contract grading is not the contemporary equivalent of throwing the papers down the steps and handing out marks depending on where they land. But, by bringing students into the assessment process, contract grading can help make assessment criteria clearer, remove some subjective aspects of grading, bolster student learning, and build community. And, by foregrounding the grading process as reflective of the inherent power of faculty (i.e., we may be their friends, but ultimately we will give them grades), contract grading provides a needed “label” for students and an invitation to faculty to re-imagine classroom practices.


Some additional bibliography:

Elbow, Peter and Jane Danielwicz. “A Unilateral Grading Contract to Improve Learning and Teaching.” English Department Faculty Publication Series. Paper 3.

Huot, B. “Toward a new discourse of assessment for the college writing classroom.” College English 65 (2002): 163–180.

Inoue, Asao B. “Community-based Assessment Pedagogy.” Assessing Writing 9 (2005) 208–238.

Kohn, Alfie. Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A’s, Praise, and Other Bribes. Boston, New York: Houghton Mifflin, 1993.

Moreno-Lopez, Isabel. “Sharing Power with Students: The Critical Language Classroom.” Radical Pedagogy 7:2 (2005).

Radican, Lynda S. “Contract Grades: An Agreement between Students and Their Teachers.” In Stephen Tchudi, ed. Alternatives to Grading Student Writing. Urbana, IL: NCTE, 1997, 285-290.

Shor, Ira. When Students Have Power: Negotiating Authority in a Critical Pedagogy. Chicago: University of Chicago Press, 1996.

Tchudi, Stephen, ed. Alternatives to Grading Student Writing. Urbana, IL: NCTE, 1997.

Grading: Fairer? Better? Utopia?

Steven Volk, November 8, 2015

Seth Anderson, "Violate This Parking Dibb," CC

Seth Anderson, “Violate This Parking Dibb,” CC

Clark Kerr, the former Chancellor of the University of California-Berkeley, in one of his many flashes of wit and wisdom, once observed, “I have sometimes thought of the modern university as a series of individual faculty entrepreneurs held together by a common grievance over parking.” The same could probably be said about grading. If there is one thing that we agree upon as faculty, it is an aversion to grading. I am no longer surprised when colleagues tell me that not having to grade papers is what finally convinced them to hang up their spurs and retire.

And it’s not just faculty who complain: grading is an equal opportunity grievance. Grading is not particularly high up on the students’ ten-best list of what they like about a college education. They certainly press us to explain what, precisely, it would take to turn a B+ into an A-. The “why-can’t-they-be-like-we-were” lobby outside of the academy see grade inflation as an indication of how faculty have caved to student demands. The media portray us as spineless, liberal wimps for not dolling out a hefty portion of C’s and D’s. And deans? Well, perhaps the day has passed when they attempted, subtly or not, to make a point by sending around a memo illustrating how the grades we gave compared with others in our department, division, and college.

Besides the fact that grading, if taken seriously, takes a huge amount of time, we struggle with it because (outside of multiple choice exams), grading is almost always more subjective than we’re comfortable with, and certainly more subjective than students expect it to be. What, precisely, would a student have to do to turn a B+ into an A-?. That’s a fair question to be asking, but, even if we grade with a rubric, we are still making fine shades of distinction that the starkness of the B+ simply can’t capture.

Image credit: John Tenniel‘s illustration for Lewis Carroll‘s poem “The Walrus and the Carpenter” (Wikipedia, Public Domain)

Image credit: John Tenniel‘s illustration for Lewis Carroll‘s poem “The Walrus and the Carpenter” (Wikipedia, Public Domain)

Nor does this take into account the fact that we’re humans, not machines. Are you a bit more generous with the first paper you read in the morning because you’re fresh and nicely caffeinated? Or are you a bit less charitable…because you’re fresh and nicely caffeinated? Our eyes cross by the 15th paper, and sometimes that will mean the student gets the A- (dear lord, anything to get me finished with this mountainous stack), and sometimes the B+ (don’t you have anything original to say? I’ve read the same idea 14 times already). It may all even out in the end, but try explaining that to the concerned student sitting across the desk from you.

Is There a Fairer Way?

We, of course, are a college that gives final grades, and even if we resist that by giving out Hampshire-College-style narratives rather than grades throughout the semester, we’re going to face the grading dilemma at the end of the semester. Is there any way to make that process fairer if not better?

David Gooblar, in his always-interesting “Pedagogy Unbound” column, recently looked at one aspect of fairness in grading: removing the “halo effect,” a specific kind of confirmation bias where favorable impressions in one area carry over to others. Those who worry about fairness in grading might wonder if, without their conscious knowledge, they are grading students higher if they did well on earlier work. B+ or A-? She got an “A” on her first two papers, so A- it is. Some would argue that we should be grading students’ work “blind,” with randomly assigned numbers replacing the names – much like an orchestra audition where the candidate performs from behind a screen. That way we won’t know what grades “33” or “7” got on their prior work.

There is some literature on “blind” grading and its impact on the halo effect as well as whether knowing the gender or race of the student influences grading, but it is hardly definitive. In a 2013 article in the Teaching of Psychology, John M. Malouff, Ashley J. Emmerton and Nicola S. Schutte, reported on a study of 126 instructors who were randomly assigned to grade a student giving a poor oral presentation or the same student giving a good oral presentation. All graders then assessed an unrelated piece of written work by the student. As hypothesized, the graders assigned significantly higher scores to written work following the better oral presentation.

On the other hand, studies looking for correlations between gender and grading have not been able to find much evidence to bolster their case (see here and here). Similar studies conducted in medical school also suggested that there was no widespread gender or racial bias in the grading of freshman medical students, although one can fairly question whether the same results would have obtained if the context were not medical school but rather a sophomore writing class or an 8th grade geometry setting. (And, to be clear, race has been shown to be a significant factor in the outcomes of standardized testing as bias is often built directly into the test.)

As Gooblar also points out, there also is much to be lost when we don’t know whose paper we are reading. Not only does it make it harder for students to come talk to you as they are working on their papers, but you have no way of knowing whether a particular student has made progress in a specific area you identified in some earlier work. And while much of our feedback is paper- specific, much is also person-specific, geared to issues that you have been discussing with the student.

"Skilled and unskilled laborers taking the TVA examination at the highschool building, Clinton, Tennessee." - NARA - 532813. Wikimedia public domain

“Skilled and unskilled laborers taking the TVA examination at the highschool building, Clinton, Tennessee.” – NARA – 532813. Wikimedia public domain

Is There a Better Way?

Certainly. I would recommend bringing a good 18-year old single malt scotch with you when you sit down to grade. Well, maybe not for those first papers in the morning. But, seriously, there are some things to keep in mind. What are we looking for in any grading system? The following 15 criteria are taken from Linda B. Nilson, Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time (Stylus Publishing, 2015). Nilson directs the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University. Grading systems, for Nilson, should embody the following criteria, although she will argue that they rarely do.

  1. Uphold high academic standards.
  2. Reflect student learning outcomes.
  3. Motivate students to learn.
  4. Motivate students to excel.
  5. Discourage cheating.
  6. Reduce student stress.
  7. Make students feel responsible for their grades.
  8. Minimize conflict between faculty and students.
  9. Save faculty time.
  10. Give students feedback they will use.
  11. Make expectations clear.
  12. Foster higher-order cognitive development and creativity.
  13. Assess authentically.
  14. Have high interrater agreement
  15. Be simple.

When reading the list the first time, I thought: Right, and why not add “bring peace to the Middle East” to the list? They all seem impossible tasks. But Nilson’s book sets out an argument for an alternative which, even if I’m not fully convinced at the end of the day, is worth a look. Nilson argues in favor of what she calls “specifications grading,” a type of grading that is similar to “contract grading” already implemented at Oberlin by a number of faculty. The basic idea is that in the class syllabus the instructor discusses precisely what students must do to get a particular grade, and that they can decide on this basis what specific grade, from an A to a D, they will be attempting to earn. Nilson discussed three central aspects of this grading system in an interview with Robert Talbert of the “Casting Out Nines” blog a year ago.

First, you grade all assignments and tests satisfactory/unsatisfactory, pass/fail, where you set “pass” at B or better work. Students earn full credit or no credit depending on whether their work meets the specs that you laid out for it. No partial credit. Think of the specs as a one-level, one-dimensional rubric, as simple as “completeness” – for instance, all the questions are answered or all the problems attempted in good faith, or the work satisfies the assignments (follows the directions) and meets a required length. Or the specs may be more complex – a description of, for example, the characteristics of a good literature review or the contents of each section of a proposal. You must write the specs very carefully and clearly. They must describe exactly what features in the work you are going to look for. You might include that the work be submitted on time. For the students, it’s all or nothing. No sliding by. No blowing off the directions. No betting on partial credit for sloppy, last-minute work.

Second, specs grading adds “second chances” and flexibility with a token system. Students start the course with 1, 2, or 3 virtual tokens that they can exchange to revise an unsatisfactory assignment or test or get a 24-hour extension on an assignment. […]

Third, specs grading trades in the point system for “bundles” of assignments and tests associated with final course grades. Students choose the grade they want to earn. To get above an F, they must complete all the assignments and tests in a given bundle at a satisfactory level. For higher grades, they complete bundles of more work, more challenging work, or both. In addition, each bundle marks the achievement of certain learning outcomes. The book contains many variations on bundles from real courses.

pencilSpecs grading, according to Nilson, assumes that there is no reason why students shouldn’t be able to achieve the outcome(s) the specs describe. Specifications are basically directions on how to produce a B-level-or-better work or the parameters within which students create a product. If students don’t understand them, they have to ask questions.

In a workshop for the faculty at the University of Pittsburgh, Nilson explained that the key to specifications grading is shifting the burden from grading to developing clear explanations for desired outcomes. Under specifications grading, students will (hopefully) fully understand faculty expectations, and the research confirms that when students know what faculty are looking for, they are more likely to do better.

Further, specification grading can also shift the way that faculty think about their students. Students taking a course in a non-major field “might decide they have better things to do this semester [than spend most of their time with that course]. What grade they wind up with says nothing about their capabilities, to me. It might say something about their time schedule,” Nilson added.

I’m not fully convinced that specifications grading is for me (and the faculty at the Pittsburgh workshop raised just the sort of questions I would have). But the points Nilson raises – the value of explaining clearly what students should be doing in an assignment, the determination of high standards, the placing of more responsibility in students’ hands – all of these points encourage me to study her approach further.

And those of you who use some form of “contract” or alternative grading? What has been your experience?

Commenting on Papers with iAnnotate

Steve Volk, September 16, 2013

I’m going to step onto some techie turf in today’s “Article of the Week”  – but don’t turn away just yet.

Here’s the question: How would you like to be able to comment on your students’ (or colleagues’) papers right on your iPad? Over the summer, I started experimenting with an app called “iAnnotate,” and am quite happy with it. The key thing for me is that it allows me to do something that I think will improve how I communicate with students on their papers. (By the way, I’m not being paid by Branchfire, the company that came out with the app and there are other applications, like PDF Expert, that do similar things. I’m also not a technology expert and haven’t reviewed many applications – I’m just saying what works for me.)

I’ll hold some of the technical blahblah for later, but consider this:

You open a student’s paper on your iPad (yes, this is for mobile devices only, either Apple or Android). You want to circle a word, highlight some text, underline a passage, add a comment in your handwriting or in type? You can do that. (Click on the photos to enlarge them.)

(You — or your students, can also use it to take notes on slides)

It gets better. Perhaps you want to insert an image into the paper, say a paper on Chile and you want to call the student’s attention to an iconic image that resonates with, corrects, or adds to what they have. Pictures are worth a 1,000 words they say! Just tap the camera icon and then touch the text where you want to insert the photo (which, of course, needs to be in your library), and boom! there it is.

OK, you can do all of that, and easily. It took me about 30 minutes to try out all the buttons and see which ones worked best for me. But the best is yet to come.

What if you get to a place in the paper and you really just want to say something to the student, where tone conveys more than text? You can do that…easily.  Just tap the little microphone icon at the point on the page where you want the recording to go, and you can record up to a minute. To play back, just hit the “go” arrow and you’ve done it. Is that cool, or what?

And here’s a 4-minute video that shows you how to do it: Using Voice Comments on iAnnotate

There’s lots more you can do with the app (read, manage, search, and share documents, take notes on lecture slides, signing documents, etc.). I have not seen any research to suggest whether students who get comments back via this kind of application pay more attention to them (i.e., learn better), but it does seem to have a lot of potential.

So, here’s the fine print – how you get it to work.

(1) You need an iPad or Android tablet – the point is that you’re “tapping” and “writing” on your device with a stylus or your finger, which you can’t do with a regular computer. (If you don’t have an iPad but would like to try this out, see me and I’ll help you arrange a loaner.)

(2) The app only works on a pdf file, which means you have to convert Word documents into a pdf (pretty simple to do – if you don’t convert on your own computer or ask your students to send you their papers in pdf format, when you tap a .doc file to open on your iPad and begin to work with it (e.g., make a comment), it will ask to save it in a pdf format and you’re good to go.

(3) You need to “transport” documents from your computer (or the student) to your tablet and back again. I use Dropbox, but you can use other cloud storage programs.

(4) It costs $9.99 – which is more than free, but less than $10!!

The very simple “how-to” for grading papers:

(1) A student sends you her paper either as a Word document or a pdf (you can ask them to convert), either directly to you via email attachment or by using Blackboard;

(2) You upload the document into your “cloud” device (e.g. Dropbox) and then open it in iAnnotate.

(3) Read the paper, make your comments, etc, save and return either directly to the student or to your own computer and then to the student with some instructions on how to use it: they don’t have to have an iPad but just double-click on, for example, the “loudspeaker” icon to hear your message or on the “paperclip” icon to see the photo.

I’m quite sure that I’ve got some of the technical details wrong, but, hey, I’m not paid for this and that’s why we have the wonderful folks at OCTET.

Here are links to some reviews of iAnnotate that appeared in the Chronicle of Higher Education and Faculty Focus.

Mark Sample, “Making the Most of iAnnotate on the iPad,” ProfHacker, Chronicle of Higher Education,” November 8, 2011;

Doug Ward, “Grading with Voice on an iPad,” June 19, 2012;

Dave Yearwood, “App of the Week Review: iAnnotate PDF,” Faculty Focus, June 8, 2012.