Tag Archives: grading

Contract Improv – Three Approaches to Contract Grading

Steve Volk, March 27, 2016

Benin Plaque, c. 16- 17th century, brass “lost-wax” technique, British Museum Af1898,0115.38

Benin Plaque, c. 16- 17th century, brass “lost-wax” technique, British Museum Af1898,0115.38

Students in museums studies, more so than casual visitors, are frequently confronted with the question of how specific artifacts made their way from their point of origin into the museum where they are displayed for our enjoyment and edification. For some objects, the answer is relatively straightforward: the painting originally in, say, the French royal collection, was purchased by a dealer who sold it to a collector who donated it to the museum. For other artifacts, particularly if the museum in question is the British Museum (the end point of a vast collection of imperial booty), the origins of the artifact is more troubled. The catalog entry for this “Benin Plaque” (left), dating from the 16th-17th centuries, calmly notes that “following the British occupation of Benin City (Edo) in 1897 objects made of brass, ivory and wood were seized by British force from the royal quarters and various storerooms.”

But as this information doesn’t appear on the object’s label in the gallery, the viewer has little sense of the violent history — the imperial relations — that underwrote the trajectory of the plaque from Benin City to its current abode on Great Russell Street in London. Museologically and culturally speaking, that’s a problem. If museums are to represent (and not simply appropriate) objects from their colonial empires, the history of that displacement must be kept in sight.

This may seem an unusual way to begin an essay on grading, but I thought of the Benin Plaques and their absent labels as I prepared another set of grades for my students. Grading (as I’ve written many times before here and hardly need to remind you) is about as eagerly anticipated by teachers as a colonoscopy (and at least those are served up with propofol.) There are any number of reasons why this is the case, and at least some of the problems of grading do come with relatively straight-forward solutions. If you can’t bear reading the 27th paper on the role of the cottage in Frankenstein, then open your assignments to allow for a greater variety of responses. If the assignment essentially requires that students feed back to you what you’ve given to them, don’t expect to have an enjoyable experience reading them. Try completing your own assignments and if you find them boring or not conducive of learning, change them so that students can use the assignment to demonstrate both mastery and application.

Full Disclosure

Other issues involved in grading are more difficult to resolve, which brings us back to the Benin Plaques. What everyone knows, yet no label discloses, is that grades represent the ultimate power that faculty hold over our students. As much as our professional code of conduct requires – demands – that we grade fairly, objectively, and without regard to extraneous factors, there is no denying that we are humans and that, when it comes to grading, we are both shaped by, and must contend with, a variety of factors that make that difficult, if not impossible. These range from simple tiredness to complex issues of prejudice including racism and sexism. [See, for example, here (the impact of the teacher’s emotional state) and here (the impact of the stereotype threat) as examples.). Perhaps, just as the Benin Plaques should include on their label an indication of the nature of the power that brought them to the British Museum, so too should we include a label on all of our tests and assignments:

Warning: As much as I will try to grade your assignments objectively, fairly, and without prejudice, and as much as I will attempt to forget how annoyed I was with you when you [fill in appropriately]: didn’t do the reading/watched a Beyoncé video on your laptop instead of listening to what I was saying/left the class three times to go to the bathroom, I am only human, so caveat emptor!

When Life Is Young, British Library HMNTS 10413.dd.1

When Life Is Young, British Library HMNTS 10413.dd.1

Grading has a way of reversing the intent of teaching, not only closing off a formative process of dialog and reflection, but often contradicting what we have been insisting all semester: “It’s not about the grade.” Well, what if it is? And how do we tell students with a straight face not to worry so much about their grades when they know (as do we) that when all is said and done, the grade we give them can/will influence whether they get the fellowship they need to pursue their studies. I would venture that, for most of us, the problem is not that we feel pressured to give “C” work an “A” (although grade inflation, particularly at elite institutions, might suggest otherwise), but rather how we maintain a straight face when we suggest there is a clear and obvious difference between a “B” and a “B+,” between a “B+ and an “A-.” Particularly in the humanities and social studies, but likely in the sciences as well, we know full well that extraneous considerations (those extra trips to the bathroom!) can influence our decisions. There’s no way around the fact that a serious evaluation of our students’ work is so much more complex than can be expressed in that single letter, and giving a student a “B+/A-” really doesn’t resolve the problem.

What else is wrong with grades? Let me count the ways! As  Steven Mintz, Executive Director of the University of Texas System’s Institute for Transformational Learning and Professor of History at the University of Texas at Austin, pointed out,

  • A single, over-all grade conflates elements that need to be disentangled.
  • They tend to overly reward lower-order thinking skills (such as memorization and recall) rather than higher order skills (involving analysis, application, and synthesis).
  • Grades too often fail to accurately reflect student learning or mastery.
  • They are frequently de-motivating and discouraging.

Nor is this a recent finding: studies as early as 1912 questioned the utility (and validity) of grades, and research has fairly consistently underlined some of the main problems in grading practices.

Typical-student, British Library HMNTS 10413.dd.1

“Typical figure, showing tendency of student life,” British Library HMNTS 10413.dd.1

Does that mean that we should stop giving grades? Very few colleges or universities have abandoned the grading system altogether. Hampshire College, where faculty write narratives of their students’ learning rather than assigning grades, remains the exception. But even there, complex narratives probably won’t work in a class of 80 students, nor is Pass/Fail an option without draw-backs in a world in which grades are the norm. A “Pass” in Organic Chemistry might not help a student when she applies to med school.

Valen E. Johnson, professor and head of the department of statistics at Texas A&M University at College Station argues further that if we didn’t grade, “students probably wouldn’t work as much and wouldn’t do homework and wouldn’t study for exams”? While this is not universally the case, we can (and probably should) admit that we share at least some of the blame for not making learning more intrinsically motivating. But such an observation can only get us so far. Ultimately, we need to think about new ways of grading that can address some of the shortcomings of the current system. That’s where contract grading comes in.

CONTRACT GRADING: THREE OPTIONS

Contract grading, which actually was first used some decades ago, more recently has been gaining traction in higher ed. Briefly, contract grading attempts to reduce the subjectivity of the grading process for faculty and the induced passivity of students within the evaluation system in an attempt to arrive at a more integrative and meaningful process of assessment.

There are a variety of approaches to contract grading, each designed to meet an instructor’s intended outcomes, but all share the fundamental goal of clarifying the grading process for students so that they can make more informed decisions about their actions. While there are a number of different types of contract grading options, I’ve  summarized three different contract-grading approaches here. At the same time, I’d encourage you to talk to your colleagues about how they grade; raise the issue at department meetings: you’d be surprised how many have adopted this method of grading.

Contract grading as a means of negotiating authority

Songs of a Savoyard, British Library HMNTS 11651.k.42

Songs of a Savoyard, British Library HMNTS 11651.k.42

The essential factor in determining a grading approach, at least as I see it, is deciding what you hope the process of grading can achieve in the broadest terms. For some, revealing and addressing the nature of power relations within a classroom environment is the central element that a grading system can address. Ira Shore, for example, has written much about the importance of creating a democratic classroom in which power is both fully disclosed and openly negotiated with students. Similarly, Isabel Moreno Lopez argues that teachers should create a critical classroom in which “authority and responsibilities are shared between teacher and students, empowering all course members to become active, responsible participants of the learning process, not merely passive consumers.” For both, grading is a means by which power can be shared through a collectively negotiated contract co-constructed at the beginning of the semester.

Here, in a condensed version, is how Moreno Lopez describes her contract grading system:

The negotiation process starts at the beginning of the semester when the teacher presents the elements of a contract grading system to the students. In general terms, the grading system is based on the quality and quantity of work students are willing and capable of doing. That is, if a student signs a contract for an “A,” s/he will do more work in the course than the student who contracts for a “C.” The quality of work will also reflect the contracted grade. Students are permitted to rewrite the written assignments as many times as necessary to attain the contracted grade.

At the start of the semester, then, the teacher opens up class-time to discuss both the syllabus and the grading system. Then, s/he asks for questions, amendments, and comments on the original proposal. A debate follows, after which the students sign the contract, as amended by themselves, and keep a copy for their records. During the semester, the negotiation process continues, both in class discussions and in comments in the students’ journals. At the end of the semester, based on the contracts and their performance, students discuss with the teacher their final grades. This grade might be the same they contracted or might have varied depending on their performance and progress.

Moreno Lopez suggests that this negotiated grading system is valuable in two ways: it helps students see learning as a process and not an end, and it “encourages students to be active participants in their own learning process by allowing them to cooperate in what is usually considered the ultimate prerogative of the teacher: the assessment process.”

Shor, Moreno Lopez and others who engage in this form of critical pedagogy identify the classroom as a political arena where differences of power are necessarily, and properly, brought into the center of teaching where they are negotiated. In such a context, struggle and conflict is both inevitable and appropriate insofar as it is a reflection of the larger society, not a “bubble” separate from it.

Non-negotiated contract grading to improve learning

Spectroscope_British Library HMNTS 10027.ee

Spectroscope_British Library HMNTS 10027.ee

The grading contracts used by Jane Danielewicz and Peter Elbow in their composition courses are similar in some respects, but they are less concerned about using the classroom to negotiate authority than Shor or Moreno Lopez. Instead, they see their goal as creating “a classroom where both teachers and students get to give as much time and attention as possible to writing—not politics and culture. Of course political and cultural issues turn up in student writing, but our tendency is to discuss the effectiveness of the writing more than political and cultural issues themselves (not that one can ever completely separate the two).”

Danielewicz and Elbow present the grading contracts to students at the beginning of the semester rather than co-constructing them with student input. By using contracts, they seek “not only to help students learn more and function better as learners; we also want a grading system that encourages them to be the kind of persons our world needs; furthermore, we want to make our own teaching easier and more satisfying.” And they add, “That’s all.” Indeed, that would be plenty.

Here is a summary of the main elements of the Danielewicz-Elbow grading contract:

  1. Attend class regularly—not missing more than a week’s worth of classes.
  2. Meet due dates and writing criteria for all major assignments.
  3. Participate in all in-class exercises and activities.
  4. Complete all informal, low stakes writing assignments (e.g. journal writing or discussion-board writing).
  5. Give thoughtful peer feedback during class workshops and work faithfully with their group on other collaborative tasks (e.g., sharing papers, commenting on drafts, peer editing, on-line discussion boards, answering peer questions).
  6. Sustain effort and investment on each draft of all papers.
  7. Make substantive revisions when the assignment is to revise—extending or changing the thinking or organization—not just editing or touching up.
  8. Copy-edit all final revisions of main assignments until they conform to the conventions of edited, revised English.
  9. Attend conferences with the teacher to discuss drafts.
  10. Submit their mid term and final portfolio.

In other words, students get a “B” based solely on what they do, not on any evaluation of their work by the professor. Grades higher than a “B,” however, depend on the teacher’s evaluation of the quality of their writing. They will discuss in class what “exceptionally high quality” writing means, making the criteria as public and concrete as possible, but they don’t give students power over “high-grade” decisions.

Although they don’t evaluate the quality of their students’ writing up to a “B” grade, they also don’t withhold evaluation as they continue to provide students with feedback on the strengths and weaknesses in their work, both drafts and final version. But the evaluation (up to a “B”) is decoupled from grades. “As a result,” they write, “students don’t have to heed any of our judgments or advice when they revise their papers (though they must revise).” They want their students to feel that the evaluations they conduct are “from individual persons: yes, experts about writing, but individuals, nevertheless, who cannot pretend to be wholly impersonal or fair.”

Their article (“A Unilateral Grading Contract to Improve Learning and Teaching”) offers a fascinating discussion of how they came to the various elements of the contract, why, for example, they picked a “B” grade as the base-line mark for their contract (“Our original reasoning was merely timid—crassly negative and pragmatic: we were scared to ‘go all the way.’”), or whether their contract is actually less “fuzzy” than a standard grading system. “How can we defend ambiguous and arguable criteria like ‘conscientious effort,’ ‘thoughtful feedback,’ and ‘conscientious participation?’” they ask. “First, we don’t accuse someone of failing to meet one of these fuzzy criteria (‘no-effort,’ for example), unless the violation is grossly flagrant (e.g. drafts far short of the required length). Secondly, we’ll always take the students word for it.” In terms of a final, summative, grade they are persuaded that their decisions were relatively easy to make. Students who didn’t fulfill the contract (including some who were excellent writers), were disqualified. They then focused more closely on the remaining final portfolios that they found to be particularly strong.

Contract grading to encouraging active learning and community building

"The Life of George Barnwell; or, the London apprentice of the last century," British Library HMNTS 12621.dd.5.

“The Life of George Barnwell; or, the London apprentice of the last century,” British Library HMNTS 12621.dd.5.

The final example of contract-style grading is Asao B. Inoue’s community-based assessment approach. Similar to all contract models, Inoue, a writing instructor at Washington State University, moves away from teacher-centered assessment and evaluation while encouraging students to take more initiative. But, more than in the previous models, Inoue seeks to create a classroom in which “students take control of all writing assignments, their instructions, assessment criteria, and the practices and reflective activities that go along with their writing.” Such an approach, he maintains, “encourages a community of writers that are implicated in each others’ writing and assessment practices, and gets them to critically engage with these practices.”

Inoue’s model underscores the fact that assessment is a vital component in the act of writing. He spends considerable time discussing with students what they want out of their papers and how they should be read and assessed. It is a complex and recursive process that begins when the class collectively creates its first assessment rubric, a set of guidelines that everyone agrees to, and that they will use both as writers and assessors. This first rubric will be revised continually as the class moves from the early stages of writing (paragraph writing) to position papers and final essays.

Any student can suggest a rubric revision or raise a question about the rubric at any time. To test and revise the iterated rubric, class members write two separate paragraphs, each receiving three peer assessments that use the in-process rubric. The class — instructor and students alike — uses what it has learned from the paragraph assessments to revise the rubric, which becomes the new starting point for on-going assignments, and so on. Over a month, each student writes a position paper, receives responses and assessments from the entire class (both on paper and through class discussions), posts a revision of the position paper based on those discussions and input, gets a more formal peer-assessment of the revision by a few colleagues, writes an essay (often based on the position paper), and finally receives a formal peer-evaluation of the essay. The same process is repeated for a second paper. (The process is schematized in the illustration below.)

ChartWhen the students assess each others’ writing, they are not looking to identify an “A” paragraph or an “exemplary,” or “outstanding” one. Rather they use the rubrics to help them identify proficient paragraphs, ones that reach the proficiency markers they set out at the start of the process. If a paragraph hits these markers, then it has done its job.

Here, for example, is what the class came up as a “proficient” paragraph with after their discussions: A proficient and adequate paragraph will . . .

  • Contain a consistent claim
  • Support claim with appropriate evidence (when needed)
  • Elicit thought on the part of the audience
  • Adapt to or consider its audience
  • Use clear and concise language
  • Use appropriate language and grammar
  • Contain three or more sentences

They continue to refine this set of criteria over the course of the semester.

As Inoue explains,

I try simply to provide the structures for my students to create a rubric, re-think it, write from it, use it to assess each other, and, of course, reflect continually upon all these practices. I distribute guidelines, provide due dates, post weekly reflection prompts, and pose additional questions in class that facilitate assessment discussions on student writing. In short, I try to coach them toward sound assessment practices and active learning stances by making them do the hard work of assessment. I encourage them to voice disagreement, show agreement, and elaborate and qualify ideas. I act as a facilitator, questioner, and listener when we talk about each other’s writing. I try to keep us focused on our rubric in our assessment discussions, yet not be a guard to ivory towers… Our class writing isn’t about what I want — it’s about what the class can agree on they want and can justify in some way so that agreements can be made… My students must debate and decide on all the important decisions regarding their writing in the course from start to finish. The class is about them learning not me teaching.

The key to making assessment work pedagogically, according to Inoue, is periodic reflection on the assessment activities. He does it once a week based on open-ended prompts to point the students to the areas he wants them to reflect on. Community-based assessment pedagogy also offers ways to build a pragmatic sense of community that is active and purposeful.

If our purpose, as teachers, in assessing and evaluating student writing is to help students learn — if assessment is inherently a learning practice (which I think it is) — then the teacher shouldn’t control all of the process. And Inoue concludes:

Community-based assessment pedagogy, as described here, boils down to three classroom imperatives: (1) encourage active learning stances by allowing students to assess and evaluate their own and their colleagues’ writing practices, and make these assessments meaningful and purposeful, (2) situate assessment practices within a community of knowledge makers who construct assessment rubrics and define and justify assessment practices, i.e., encourage the class to work for one another as mutual agents working with and for each other’s benefit, writing for each other, and negotiating hard agreements together, and (3) give lots of opportunities to reflect on assessment that speaks to the larger class community, in order to theorize about writing, rhetorical conventions, assessment, and the judging of writing from specific criteria, i.e., what we say about what we are doing (or did) can help us do it better in the future. In my versions of this pedagogy, these imperatives rest on a framework of recursive, repeated writing and assessment activities.


 

As you will have noticed, none of these models makes grading “easy.” Contract grading is not the contemporary equivalent of throwing the papers down the steps and handing out marks depending on where they land. But, by bringing students into the assessment process, contract grading can help make assessment criteria clearer, remove some subjective aspects of grading, bolster student learning, and build community. And, by foregrounding the grading process as reflective of the inherent power of faculty (i.e., we may be their friends, but ultimately we will give them grades), contract grading provides a needed “label” for students and an invitation to faculty to re-imagine classroom practices.


Some additional bibliography:

Elbow, Peter and Jane Danielwicz. “A Unilateral Grading Contract to Improve Learning and Teaching.” English Department Faculty Publication Series. Paper 3.

Huot, B. “Toward a new discourse of assessment for the college writing classroom.” College English 65 (2002): 163–180.

Inoue, Asao B. “Community-based Assessment Pedagogy.” Assessing Writing 9 (2005) 208–238.

Kohn, Alfie. Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A’s, Praise, and Other Bribes. Boston, New York: Houghton Mifflin, 1993.

Moreno-Lopez, Isabel. “Sharing Power with Students: The Critical Language Classroom.” Radical Pedagogy 7:2 (2005).

Radican, Lynda S. “Contract Grades: An Agreement between Students and Their Teachers.” In Stephen Tchudi, ed. Alternatives to Grading Student Writing. Urbana, IL: NCTE, 1997, 285-290.

Shor, Ira. When Students Have Power: Negotiating Authority in a Critical Pedagogy. Chicago: University of Chicago Press, 1996.

Tchudi, Stephen, ed. Alternatives to Grading Student Writing. Urbana, IL: NCTE, 1997.

Grading: Fairer? Better? Utopia?

Steven Volk, November 8, 2015

Seth Anderson, "Violate This Parking Dibb," CC

Seth Anderson, “Violate This Parking Dibb,” CC

Clark Kerr, the former Chancellor of the University of California-Berkeley, in one of his many flashes of wit and wisdom, once observed, “I have sometimes thought of the modern university as a series of individual faculty entrepreneurs held together by a common grievance over parking.” The same could probably be said about grading. If there is one thing that we agree upon as faculty, it is an aversion to grading. I am no longer surprised when colleagues tell me that not having to grade papers is what finally convinced them to hang up their spurs and retire.

And it’s not just faculty who complain: grading is an equal opportunity grievance. Grading is not particularly high up on the students’ ten-best list of what they like about a college education. They certainly press us to explain what, precisely, it would take to turn a B+ into an A-. The “why-can’t-they-be-like-we-were” lobby outside of the academy see grade inflation as an indication of how faculty have caved to student demands. The media portray us as spineless, liberal wimps for not dolling out a hefty portion of C’s and D’s. And deans? Well, perhaps the day has passed when they attempted, subtly or not, to make a point by sending around a memo illustrating how the grades we gave compared with others in our department, division, and college.

Besides the fact that grading, if taken seriously, takes a huge amount of time, we struggle with it because (outside of multiple choice exams), grading is almost always more subjective than we’re comfortable with, and certainly more subjective than students expect it to be. What, precisely, would a student have to do to turn a B+ into an A-?. That’s a fair question to be asking, but, even if we grade with a rubric, we are still making fine shades of distinction that the starkness of the B+ simply can’t capture.

Image credit: John Tenniel‘s illustration for Lewis Carroll‘s poem “The Walrus and the Carpenter” (Wikipedia, Public Domain)

Image credit: John Tenniel‘s illustration for Lewis Carroll‘s poem “The Walrus and the Carpenter” (Wikipedia, Public Domain)

Nor does this take into account the fact that we’re humans, not machines. Are you a bit more generous with the first paper you read in the morning because you’re fresh and nicely caffeinated? Or are you a bit less charitable…because you’re fresh and nicely caffeinated? Our eyes cross by the 15th paper, and sometimes that will mean the student gets the A- (dear lord, anything to get me finished with this mountainous stack), and sometimes the B+ (don’t you have anything original to say? I’ve read the same idea 14 times already). It may all even out in the end, but try explaining that to the concerned student sitting across the desk from you.

Is There a Fairer Way?

We, of course, are a college that gives final grades, and even if we resist that by giving out Hampshire-College-style narratives rather than grades throughout the semester, we’re going to face the grading dilemma at the end of the semester. Is there any way to make that process fairer if not better?

David Gooblar, in his always-interesting “Pedagogy Unbound” column, recently looked at one aspect of fairness in grading: removing the “halo effect,” a specific kind of confirmation bias where favorable impressions in one area carry over to others. Those who worry about fairness in grading might wonder if, without their conscious knowledge, they are grading students higher if they did well on earlier work. B+ or A-? She got an “A” on her first two papers, so A- it is. Some would argue that we should be grading students’ work “blind,” with randomly assigned numbers replacing the names – much like an orchestra audition where the candidate performs from behind a screen. That way we won’t know what grades “33” or “7” got on their prior work.

There is some literature on “blind” grading and its impact on the halo effect as well as whether knowing the gender or race of the student influences grading, but it is hardly definitive. In a 2013 article in the Teaching of Psychology, John M. Malouff, Ashley J. Emmerton and Nicola S. Schutte, reported on a study of 126 instructors who were randomly assigned to grade a student giving a poor oral presentation or the same student giving a good oral presentation. All graders then assessed an unrelated piece of written work by the student. As hypothesized, the graders assigned significantly higher scores to written work following the better oral presentation.

On the other hand, studies looking for correlations between gender and grading have not been able to find much evidence to bolster their case (see here and here). Similar studies conducted in medical school also suggested that there was no widespread gender or racial bias in the grading of freshman medical students, although one can fairly question whether the same results would have obtained if the context were not medical school but rather a sophomore writing class or an 8th grade geometry setting. (And, to be clear, race has been shown to be a significant factor in the outcomes of standardized testing as bias is often built directly into the test.)

As Gooblar also points out, there also is much to be lost when we don’t know whose paper we are reading. Not only does it make it harder for students to come talk to you as they are working on their papers, but you have no way of knowing whether a particular student has made progress in a specific area you identified in some earlier work. And while much of our feedback is paper- specific, much is also person-specific, geared to issues that you have been discussing with the student.

"Skilled and unskilled laborers taking the TVA examination at the highschool building, Clinton, Tennessee." - NARA - 532813. Wikimedia public domain

“Skilled and unskilled laborers taking the TVA examination at the highschool building, Clinton, Tennessee.” – NARA – 532813. Wikimedia public domain

Is There a Better Way?

Certainly. I would recommend bringing a good 18-year old single malt scotch with you when you sit down to grade. Well, maybe not for those first papers in the morning. But, seriously, there are some things to keep in mind. What are we looking for in any grading system? The following 15 criteria are taken from Linda B. Nilson, Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time (Stylus Publishing, 2015). Nilson directs the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University. Grading systems, for Nilson, should embody the following criteria, although she will argue that they rarely do.

  1. Uphold high academic standards.
  2. Reflect student learning outcomes.
  3. Motivate students to learn.
  4. Motivate students to excel.
  5. Discourage cheating.
  6. Reduce student stress.
  7. Make students feel responsible for their grades.
  8. Minimize conflict between faculty and students.
  9. Save faculty time.
  10. Give students feedback they will use.
  11. Make expectations clear.
  12. Foster higher-order cognitive development and creativity.
  13. Assess authentically.
  14. Have high interrater agreement
  15. Be simple.

When reading the list the first time, I thought: Right, and why not add “bring peace to the Middle East” to the list? They all seem impossible tasks. But Nilson’s book sets out an argument for an alternative which, even if I’m not fully convinced at the end of the day, is worth a look. Nilson argues in favor of what she calls “specifications grading,” a type of grading that is similar to “contract grading” already implemented at Oberlin by a number of faculty. The basic idea is that in the class syllabus the instructor discusses precisely what students must do to get a particular grade, and that they can decide on this basis what specific grade, from an A to a D, they will be attempting to earn. Nilson discussed three central aspects of this grading system in an interview with Robert Talbert of the “Casting Out Nines” blog a year ago.

First, you grade all assignments and tests satisfactory/unsatisfactory, pass/fail, where you set “pass” at B or better work. Students earn full credit or no credit depending on whether their work meets the specs that you laid out for it. No partial credit. Think of the specs as a one-level, one-dimensional rubric, as simple as “completeness” – for instance, all the questions are answered or all the problems attempted in good faith, or the work satisfies the assignments (follows the directions) and meets a required length. Or the specs may be more complex – a description of, for example, the characteristics of a good literature review or the contents of each section of a proposal. You must write the specs very carefully and clearly. They must describe exactly what features in the work you are going to look for. You might include that the work be submitted on time. For the students, it’s all or nothing. No sliding by. No blowing off the directions. No betting on partial credit for sloppy, last-minute work.

Second, specs grading adds “second chances” and flexibility with a token system. Students start the course with 1, 2, or 3 virtual tokens that they can exchange to revise an unsatisfactory assignment or test or get a 24-hour extension on an assignment. […]

Third, specs grading trades in the point system for “bundles” of assignments and tests associated with final course grades. Students choose the grade they want to earn. To get above an F, they must complete all the assignments and tests in a given bundle at a satisfactory level. For higher grades, they complete bundles of more work, more challenging work, or both. In addition, each bundle marks the achievement of certain learning outcomes. The book contains many variations on bundles from real courses.

pencilSpecs grading, according to Nilson, assumes that there is no reason why students shouldn’t be able to achieve the outcome(s) the specs describe. Specifications are basically directions on how to produce a B-level-or-better work or the parameters within which students create a product. If students don’t understand them, they have to ask questions.

In a workshop for the faculty at the University of Pittsburgh, Nilson explained that the key to specifications grading is shifting the burden from grading to developing clear explanations for desired outcomes. Under specifications grading, students will (hopefully) fully understand faculty expectations, and the research confirms that when students know what faculty are looking for, they are more likely to do better.

Further, specification grading can also shift the way that faculty think about their students. Students taking a course in a non-major field “might decide they have better things to do this semester [than spend most of their time with that course]. What grade they wind up with says nothing about their capabilities, to me. It might say something about their time schedule,” Nilson added.

I’m not fully convinced that specifications grading is for me (and the faculty at the Pittsburgh workshop raised just the sort of questions I would have). But the points Nilson raises – the value of explaining clearly what students should be doing in an assignment, the determination of high standards, the placing of more responsibility in students’ hands – all of these points encourage me to study her approach further.

And those of you who use some form of “contract” or alternative grading? What has been your experience?

Commenting on Papers with iAnnotate

Steve Volk, September 16, 2013

I’m going to step onto some techie turf in today’s “Article of the Week”  – but don’t turn away just yet.

Here’s the question: How would you like to be able to comment on your students’ (or colleagues’) papers right on your iPad? Over the summer, I started experimenting with an app called “iAnnotate,” and am quite happy with it. The key thing for me is that it allows me to do something that I think will improve how I communicate with students on their papers. (By the way, I’m not being paid by Branchfire, the company that came out with the app and there are other applications, like PDF Expert, that do similar things. I’m also not a technology expert and haven’t reviewed many applications – I’m just saying what works for me.)

I’ll hold some of the technical blahblah for later, but consider this:

You open a student’s paper on your iPad (yes, this is for mobile devices only, either Apple or Android). You want to circle a word, highlight some text, underline a passage, add a comment in your handwriting or in type? You can do that. (Click on the photos to enlarge them.)

(You — or your students, can also use it to take notes on slides)

It gets better. Perhaps you want to insert an image into the paper, say a paper on Chile and you want to call the student’s attention to an iconic image that resonates with, corrects, or adds to what they have. Pictures are worth a 1,000 words they say! Just tap the camera icon and then touch the text where you want to insert the photo (which, of course, needs to be in your library), and boom! there it is.

OK, you can do all of that, and easily. It took me about 30 minutes to try out all the buttons and see which ones worked best for me. But the best is yet to come.

What if you get to a place in the paper and you really just want to say something to the student, where tone conveys more than text? You can do that…easily.  Just tap the little microphone icon at the point on the page where you want the recording to go, and you can record up to a minute. To play back, just hit the “go” arrow and you’ve done it. Is that cool, or what?

And here’s a 4-minute video that shows you how to do it: Using Voice Comments on iAnnotate

There’s lots more you can do with the app (read, manage, search, and share documents, take notes on lecture slides, signing documents, etc.). I have not seen any research to suggest whether students who get comments back via this kind of application pay more attention to them (i.e., learn better), but it does seem to have a lot of potential.

So, here’s the fine print – how you get it to work.

(1) You need an iPad or Android tablet – the point is that you’re “tapping” and “writing” on your device with a stylus or your finger, which you can’t do with a regular computer. (If you don’t have an iPad but would like to try this out, see me and I’ll help you arrange a loaner.)

(2) The app only works on a pdf file, which means you have to convert Word documents into a pdf (pretty simple to do – if you don’t convert on your own computer or ask your students to send you their papers in pdf format, when you tap a .doc file to open on your iPad and begin to work with it (e.g., make a comment), it will ask to save it in a pdf format and you’re good to go.

(3) You need to “transport” documents from your computer (or the student) to your tablet and back again. I use Dropbox, but you can use other cloud storage programs.

(4) It costs $9.99 – which is more than free, but less than $10!!

The very simple “how-to” for grading papers:

(1) A student sends you her paper either as a Word document or a pdf (you can ask them to convert), either directly to you via email attachment or by using Blackboard;

(2) You upload the document into your “cloud” device (e.g. Dropbox) and then open it in iAnnotate.

(3) Read the paper, make your comments, etc, save and return either directly to the student or to your own computer and then to the student with some instructions on how to use it: they don’t have to have an iPad but just double-click on, for example, the “loudspeaker” icon to hear your message or on the “paperclip” icon to see the photo.

I’m quite sure that I’ve got some of the technical details wrong, but, hey, I’m not paid for this and that’s why we have the wonderful folks at OCTET.

Here are links to some reviews of iAnnotate that appeared in the Chronicle of Higher Education and Faculty Focus.

Mark Sample, “Making the Most of iAnnotate on the iPad,” ProfHacker, Chronicle of Higher Education,” November 8, 2011;

Doug Ward, “Grading with Voice on an iPad,” June 19, 2012;

Dave Yearwood, “App of the Week Review: iAnnotate PDF,” Faculty Focus, June 8, 2012.