Student Evaluations of Teaching: Once More into the Debate

Steve Volk (September 21, 2014)

A slight detour this week from the daily business of the semester to a look towards its end. This Article of the Week was spurred by an article which appeared in the Chronicle of Higher Education this past week. In “Scholars Take Aim at Student Evaluations’ ‘Air of Objectivity,’” Dan Berrett reports that a new examination of end-of-semester student evaluations has found that they “are often misused statistically and shed little light on the quality of teaching.” Other than that, they’re probably OK. (That’s just me being snarky, so disregard.) More seriously, the draft study by Philip B. Stark, a professor of statistics at UC Berkeley, and Richard Freishtat, senior consultant at Berkeley’s Center for Teaching and Learning, repeats some of the critiques that have been leveled against Student Evaluations of Teaching (SETs) for a long time and raises some new ones. [NOTE: You can access the Chronicle via the library's website; other articles will be posted to CTIE's Blackboard site.]

I can’t comment on the research design or reliability of these studies, and there are certainly arguments in favor of SETs, but the following data has been reported over the years:

  • Some research found a correlation between  SET scores and students’ grade expectations although revenge (“I’m going to get back at that teacher”) was not found to be an important element.
  • Effectiveness scores (faculty rated high for being “effective” teachers and enjoyment scores are related.
  • Students’ ratings of instructors can be predicted from the students’ reactions to 30 seconds of silent video of the instructor; first impressions may dictate end-of-course evaluation scores; and essentially represent nothing more than a snap judgment by students. [Pamela Ann Hayward, "Students Initial Impression of Teaching Effectiveness," PhD dissertation, University of Illinois, Champaign-Urbana, 2000).]
  • Gender, ethnicity, and the instructor’s age and physical attractiveness matter in SET ratings.
  • SETs don’t tell us anything about the quality of the teaching.
  • They can be used by some students to get back at faculty for not teaching the course that they, the students, wanted.
  • SET scores correlate with the lecturer’s charisma and other factors unrelated to teaching.
From Ruth Krauss and Maurice Sendak, Open House for Butterflies (Harper Collins 2001)

From Ruth Krauss and Maurice Sendak, Open House for Butterflies (Harper Collins 2001)

The Berkeley authors add a new critique based on the how the numbers generated by SETs are understood. For one thing, response rates can seriously skew the data. Has anyone else felt that same sinking feeling when, on the day you hand out teaching evaluations, the three students who you knew were loving the class were absent? Raise your hands. Non-responders are not the same as responders and the higher the rate of non-response on the day student teaching evaluations are distributed, the less useful is the final average – and yet that’s the number that just stands there nakedly, without further clothing or explanation.

 

 

 

It almost goes without saying that the averages of small samples are more susceptible to what the authors call the luck of the draw. If two students out of 8 are missing on evaluation day, does the final average tell us anything? And this goes along with the fact that anonymity is almost always lost in classes under 6-7, making the findings much less reliable. At the very least, we need a policy for collecting SETs in courses of 10 or under.

The authors also point to some common statistical errors that are made when the one number we focus on is the average.  “Academic personnel review processes often invite comparing an instructor’s average scores to the departmental average,” they write. “Such averages and comparisons make no sense, as a matter of statistics. They presume that the difference between 3/7 [a score of 3 on a 7 point scale] and 4/7 means the same thing as the difference between 6/7 and 7/7. They presume that the difference between 3/7 and 4/7 means the same thing to different students. They presume that 5/7 means the same thing to different students and to students in different courses. They presume that a 3/7 ‘balances’ a 7/7 to make two 5/7s. For teaching evaluations, there is no reason any of those things should be true.”

Now, I’ll leave it to my colleagues in statistics to fully evaluate the case the authors make, although it seems right to me on a basic level. Using the Oberlin 5-point scale, if an instructor received one “5” and one “1,” would we have learned that he was a “3”? Or, to put it a different way, what would we have learned? (I can’t resist repeating the authors’ statistics joke: Three statisticians go deer hunting. The first shoots and misses a yard to the left; the second shoots and misses a yard to the right; the third yells “We got it!” Is this what statisticians do when they get together???)

Anyway, there are more issues to consider. A lot of teaching is incommensurable in a quantifiable way. Teaching a “service-oriented” introductory course is not the same as teaching an upper-level seminar; teaching at 9:00 AM is not the same as teaching at 1:30 PM, etc. These are common issues. None of them are meant to suggest that we shouldn’t be doing our best in all of these courses or that there is absolutely no way to figure out what any single instructor is doing since we’re all different. We should and we can…but SETs are a relatively thin thread on which to hang the evaluation of teaching, something we hold to be massively important.

We have done a number of things to improve Student Evaluations of Teaching at Oberlin. Some years ago we put every department on the same 1-5 scale and made sure the “description” for each number is uniform across campus. We removed those questions that students aren’t competent to answer (e.g. does the instructor know the subject matter?), have made the questions as straight-forward and clear as possible so that when students answer them we know what question it is that they are answering (validity). And we designed our SETs based on research that has found which areas of inquiry are likely to produce reliable data (i.e., do different students – say a first-year and a junior – give the same instructor similar marks; would the same student give the same instructor the same mark later, etc. In short, do students often agree.).

Still, the literature in this area is abundantly clear, and this new study out of Berkeley only makes the point more persuasively. SETs can tell you certain things. They can tell you: (1) about student engagement with the class and whether engagement has changed over time; (2) who are the outliers in different categories – who seems to be consistently at the top or toward the bottom of the ratings; and (3) some information from a careful reading of written student comments  – but such comments are usually not comparable across classes.

What they don’t tell you is the effectiveness of the instructor as a teacher or whether the students are learning. (Grades and exam scores don’t do that either since we don’t know if exams are “hard” or “easy,” let alone the value added by the course to the students’ learning.)

So what do we know about evaluating teaching? We know that:

  • SETs, as stated above, can provide valuable information about student engagement and student evaluation of their own learning … but only in a few areas, and that it is more useful to read the comments and see changes over time than to generate a set of averages, let alone – please, no! – combining all the numbers into one big “average” for the instructor: He scored 3.8 in that History class.
  • There is no such thing as a perfect measurement of teaching effectiveness, but that shouldn’t stop us from putting more effort into what can produce better information that can help us understand more about the kind of teaching that is going on in our classes.
  • We need to be using multiple measures of teaching effectiveness, particularly when talking about high-stakes evaluations (reappointment, tenure, promotion in rank). These include:
  1. SETs, when used as suggested above, from which we can learn whether the instructor is engaging students in class in ways that are important, not just entertaining.
  1. Peer observations of teaching by trained observers, not just colleagues who know the subject matter. These observations need to be made uniform across departments and programs, including training for the observers, a pre-observation interview to get a better sense of what the instructor is intending to get at in the class that will be observed, and a post-observation interview to allow for further clarification of what went on. Here’s one example of this process from the teaching center at the University of Southern California.
  1. A “forensic” examination of course syllabi, usually by colleagues in similar fields outside of the college to determine whether the instructor’s teaching is keeping up with the field.
  1. Finally, and most importantly, I would argue, is a teaching portfolio which allows the instructor to discuss her approach to teaching and how it has developed over time. A teaching portfolio would include examples of teaching materials and samples of student work and would indicate how an instructor has used earlier critiques to refashion and rethink her teaching. Teaching portfolios are geared to helping the instructor and those reviewing the portfolio see the dynamics of teaching and how/if the teacher is looking for ways to incorporate valid critiques of teaching or new information and research about pedagogy into her courses. [Peter Seldin, The Teaching Portfolio: A Practical Guide to Improved Performance and Promotion/Tenure Decisions, 4th Ed. (Jossey-Bass, 2010).]

Stark and Freishtat pose the following questions, which I think are good ones to ask. When you come right down to it, we need to have a system of teaching evaluation that can help us answer them.

Is this a good and dedicated teacher? Is she engaged in her teaching? Is she following pedagogical practices found to work in the discipline? Is she available to students? Is she putting in appropriate effort? Is she creating new materials, new courses, or new pedagogical approaches? Is she revising, refreshing, and reworking existing courses based on feedback and on-going investigation? Is she helping keep the curriculum in the department up to date? Is she trying to improve? Is she improving? Is she contributing to the college’s teaching mission in a serious way? Is she supervising undergraduates for research, internships, and honors theses? Is she advising and mentoring students? Do her students do well when they graduate?

 

 

Evaluations of teaching that can help us answer these questions would, indeed, be valuable.

Using Small-Group Discussions Effectively

Steve Volk, September 14, 2014

Compared to our colleagues at most universities, our classes are blissfully small. Computer Science 61A enrolls nearly 1,100 students at Berkeley; Economics 10 topped out at more than 800 at Harvard. Still, many of the classes we teach are beyond the comfortable-discussion size of 10-15 students regardless of what our faculty-student ratio may indicate. That doesn’t mean we should abandon small-group discussions as a pedagogic strategy, but it does require some planning, especially in the critical step of socializing the information gained in smaller groups among the whole class. How can we use small discussion sections most effectively in classes that enroll 30, 50, or more students?

Large-Class

 

Why Discuss?

This post is more of a “how to” than a “why to,” but it’s still important to touch on the importance of discussion in student learning. A constructivist notion of learning, simply put, holds that understanding is gained by experience and reflection. When we encounter something new, we have to reconcile it with previously held ideas or experiences, figuring out how to make sense of the new knowledge. In that process we become the active creators of our own knowledge rather than sponges just absorbing what others tell us. While our students can (usually) reproduce what we tell them, learning is not the process of hearing-remembering-repeating, even though repeating and remembering may well be a part of ultimate learning. To learn means to ask questions, challenge ideas, explore unfamiliar territory, come to clarity in our own terms. As Ruth Tringham, a Berkeley anthropology professor put it, traditional teaching models are like banking, “where you pour knowledge into a student and hope to get some interest back,” whereas what we really want is for students to come “to grips with the questions themselves and learn to evaluate information.”

There are a lot of ways create an environment in which students can do this, not just one “best” practice. Most of these approaches involve active learning techniques such as experimentation, problem solving, and, of course, discussions. We all probably have experienced students who express (usually on their evaluation forms) a dislike of small group discussions. Since instructors “know” the answers, we should be doing the talking, not their peers who only read the stuff (or not!!) last night.

Round-table-discussionDiscussions are valuable for us in a number of ways: we can hear what questions (or answers) students are posing and propose ways for them to restate questions more usefully, They can help us suggest more fruitful questions, draw out important conclusions, underline misconceptions. Discussions can also be valuable ways to help students become more careful listeners, note takers, or observers, as well as learning to engage productively in a conversation with their peers.

Setting Up Discussion Groups

OK, enough of the theory; time for practice. Having decided on the utility of discussion in class and having decided to break the class into smaller sections rather than to have a faculty-directed “discussion of the whole,”  what are some ways to go about it? Some faculty have decided to divide up their larger classes and meet in smaller groups one day a week; that can work, but, like labs in the sciences, it requires a lot of time on the part of the instructor.

Alternatively, many divide the class into smaller discussion groups during the class session, sending the groups to different parts of the room. This obviously works best in classrooms with flexible seating where chairs/tables are easily movable (speaking of which:  Shouldn’t we have more of those?) Arranging small-group discussion in fixed seating amphitheaters often turns into an exercise in gymnastics.

Dividing the students up

For a very quick discussion (3-5 min), it’s a lot easier just to have students turn their chairs to those sitting next to them. For longer discussions, it’s better to mix them up since our restless students most often will sit in the same seat every class. Try counting off with all the #1’s in one part of the class; #2’s in another, etc., is an easy way to go. There are others.

You also need to decide whether all the groups will be discussing the same question or if each group will have a different question or problem to address. Decide on any rules or goals for the discussion: everyone must speak; there will be one note-taker, a different person must report back, each student is responsible for thinking about a different aspect of the question, etc. I’ve generally found it useful to have all the students write up a summary and some conclusions to their discussion both as a way of consolidating their learning from the discussion and as a way of helping them listen to their colleagues and take their ideas seriously.

Socializing Learning from Small Group Discussions

For me, the question of how you socialize the discussion that has been generated in the groups is both the most important aspect of small-group discussions and logistically the most persistent challenge. To get the most out of a discussion, the information gained in each of the groups needs to be shared. That’s the way that instructors can, in good constructivist fashion, help students restate questions in useful ways, underline relevant concepts, point to useful avenues for further research. You can sit in on each of the groups, but that often upsets its dynamic and, particularly when you have a lot of groups, you’ll only hear a small part of what’s going on. Here are some other suggestions:

  • The most obvious point is to make sure you leave enough time for the whole-group discussion; how often we simply run out of time before bringing the groups together!
  • Ask one person in each group to report out the group’s discussion;
  • Have one person from each group write/draw some conclusions on the board as the discussions are coming to an end. As they do this, you can look at what is being written, find commonalities, differences, points you want to take up. You can also take a picture of the board with your phone and post it to Blackboard, which will allow you to refer back to what was written or start the next class with slides from the board.

Dictatorship

Google Docs to the Rescue [Click on this link for short video on how to make and share a Google Doc]

I’ve found one other way to be extremely useful, although it requires some modest technology and that some students in your class have brought their laptops to class. You can generally count on this in large classes. (If you don’t permit laptops as a policy, you’ll have to plan ahead and ask some students to bring theirs. As a sidebar – if you want something useful on laptop use in the classroom, check out this article in the Chronicle of Higher Education from a few weeks ago).

  • Before class, open a Google Doc, give it a title (Discussion Sept. 21, 2014), and create as many columns as you will have groups. You can just label the columns 1, 2, 3… or you can give them a short description if each group will be talking about a different topic (#1-State; #2-Religion; #3-Civil Society…)

Google-Doc

Sample-Doc

  • Break the students up into groups, making sure that the groups contain at least one student with a laptop. Invite one student in each group, the one who has the laptop, into the Google doc you have created. This can be done quickly since they are all registered to Oberlin.edu addresses and once you type in a few letters of their email addresses, you will find them. They should soon be able to see the Google Doc that you have prepared.

Share

  • Assign that student with the laptop to make note of either the group’s main points or its main conclusions as the discussion develops.
  • Make sure your own laptop connected to the classroom projector and is displaying the Google doc that you have created.
  • As the discussion unfolds in each group, you will see what is happening in each of the groups.

Full-Doc

  • When you call the class back together, you can synthesize their discussion and conclusions since all the groups are writing on the same Google Doc (although only in the column they have been assigned). Because you can see the discussions “develop” in real time, you can quickly find commonalities and differences, points that were missed, misunderstandings and fruitful avenues to pursue.
  • Finally, you can save the document, which the whole class has produced, and make it available to the students (via Blackboard). You can have the students think about it the same way that you have begun in class. You can bring up the document at the start of the next class, or just archive it for the next time you offer that class.

In short, it’s a great way of socializing the learning that’s going on in the different groups, particularly as it allows you to “sit in” on each of them without changing their dynamic at all.

[Again, if you need a short video on how to create and share a Google Doc with your students, just click on this link.]

 

Mindsets: “I’m not really good at that…”

By Steve Volk (September 7, 2014)

My mother (who taught Spanish and French), my sister (quite competent in French), and I (Spanish) used to tease my father mercilessly about his inability to speak a language other than English. We drove around Mexico when I was young and laughed with great zest when, after each meal, he would try to ask, in Spanish, for the check (“La cuenta, por favor”). What emerged from his mouth were strange sounds that had quite literally become lost in the translation. The server would look at him in puzzlement until one of us stepped in to the rescue.

Can I Have the Bill

For my own part, I still remember the “D” I got on my drawing of an American eagle in the 4th grade from Mrs. Simmons, who (I thought) was a lovely teacher and was just pointing out a reality: I couldn’t draw, never could, still can’t. My father’s problem was that he just couldn’t learn another language. (He often told the story of how, when he was a student at the University of Wisconsin – he became a lawyer; no slouch, he – his Latin teacher gave him a “C” instead of failing him if he promised never to take a foreign language again.)

So, where are these familial stories going? To the mindset research of Carol Dweck, a psychology professor at Stanford, which is the subject of today’s “Article of the Week.” Those in psychology or neuroscience will surely know her work. To boil it down drastically: through decades of research, Dweck (and her co-investigators) came to the conclusion that most people have two very different understandings about intellectual abilities and where they come from. Some think that people are just naturally talented in certain areas (foreign languages, art, math, music, etc.), and if you weren’t born with those abilities, there’s not much you can do to change that. Others think that intellectual abilities can be cultivated and developed if you apply yourself to the challenges at hand. It’s not that people don’t differ in their current skill levels, nor that with hard work everyone can be a Serena Williams, a Yo Yo Ma, or an Albert Einstein, but this second group believes that they can improve their underlying abilities if they work at it. (Interestingly, Einstein once wrote, “It’s not that I’m so smart, it’s that I stay with problems longer.”) Dweck called these approaches “mindsets,” and labeled the former a “Fixed Mindset,” and the latter a “Growth Mindset.”

fixedmindsetgrowthmindset

Fixed vs. Growth Mindsets

Now, for those who are about to tune away thinking, “why is he peddling this pop motivational claptrap?”, don’t leave just yet. The research is solid (Dweck is a renowned scholar), but more importantly, her research contains implications for how we teach and interact with our students. Consider this. According to “mindset” research, those with a fixed mindset see evaluations (grades, comments written on papers or spoken in class, etc.) as appraisals of their intelligence, how “smart” they are, not as an indication of how much work they put into the paper, or how much reading informed their question in class, or what they learned. To get a “B” on an exam essentially means that they not smart enough to be “A” students. (Or that we, the teachers, got it wrong!)

gifted-talented-NYorker

“Zach is in the gifted-and-talented-and-you’re-not class.”

The repercussions of this kind of thinking are serious. Most critical, in my mind, is that students who show the characteristics of the “fixed” mindset will try to avoid challenges. Think of it this way – and this is particularly true for our students who have done well throughout their pre-college careers. Receiving a “B” in a course proves that the student is not as intelligent as the “A” student. Now that’s the last thing students want to disclose, so they will avoid challenges and stick to areas where they are (more) convinced of their abilities. Those with a growth mindset, on the other hand, have a greater tendency to take on challenges, persist in the face of adversity, and learn from criticism, largely because (at some level), they have accepted that their brains are just another muscle that needs to be exercised in order to develop.

Dweck summarizes this in the following way. If you have a fixed mindset:

  • Your goal in the classroom is to not look dumb.
  • Having to exert effort makes you feel dumb.
  • If you have a setback, you really feel dumb.

But if you have a growth mindset, then:

  • Your goal in the classroom is to learn.
  • Having to exert effort makes you feel like you’re learning.
  • If you have a setback, you see it as a learning opportunity.

Some years ago, Dweck brought some students into the brain-wave lab at Columbia University to study how their brains behaved as they answered difficult questions and received feedback. She found that those she tagged as having a fixed mindset actually were tuning out information that could help them learn and improve. They were only interested in hearing feedback that reflected directly on their present ability, and didn’t even show an interest in hearing the right answer when they had gotten a question wrong, because they had already failed and it just proved they weren’t able to answer those questions. Those with a growth mindset, on the other hand, were quite attentive to information that could help them expand their existing knowledge and skills, regardless of whether they’d gotten the question right or wrong — in other words, their priority was learning, not the binary of success and failure.

Dweck-MindsetThis should raise the question of whether anything can be done to address mindsets that are already “set in place.” I will never be a Picasso, but will I be able to think of myself as someone who can, with practice, draw? I’m reminded of the famous Stanford marshmallow experiment that showed that young children who could delay gratification (by not eating the marshmallow when the tester left the room) would do markedly better in myriad ways later in life than those with low self-control who gobbled it down after a few seconds. (A new study has suggested other variables are equally critical.) Anyway, I was always left thinking that the poor 4-year old who wolfed down the marshmallow was doomed for life and nothing could be done about it.

Actually, that’s not the case with self-control (“Once you realize that will power is just a matter of learning how to control your attention and thoughts, you can really begin to increase it,” says Walter Mischel, who devised the experiment), and it’s not the case with mindsets. And that’s where we come in.

What We Can Do

1. With advisees: Mindset research suggests that we need to encourage students to think of learning as a challenge they can take on in areas they are convinced they are “not really good at.” How do we respond when an advisee says that she’s really bad at math so will try to satisfy her QFR requirement some other way? Or when a student says that he’s just not a good writer and so will try to avoid classes that have a lot of writing? We could say that these courses are “good for you,” or that you should go beyond your comfort zone, but that doesn’t give the student any sense of why it’s “good” to go beyond one’s comfort zone since they haven’t succeeded in math or English in the past and, not succeeding again will only confirm that they’re dumb, which is the last thing they want to be revealed about them. What if we talk about Dweck’s research when these moments come up, which can give them a better idea of why challenges aren’t just “good” for you, but are at the basis of learning?

2. With students in our classes: We need to give students a sense that doing well is not a question of having an innate ability in the subject, but of hard work. When we talk about feedback and grades, we should insist on high standards and at the same time give assurances that our students can do it if they apply themselves, although there’s a caveat here. We can help students with advice on effective study and test-taking strategies, but, as Dweck pointed out, “Study skills and learning skills are inert until they’re powered by an active ingredient.” Students may know how to study, but won’t want to if they believe their efforts will be futile, that they are not good at biology and the class they are in will just prove how dumb they are. You need, as well, to “target that belief,” as Dweck puts it, and then you will “see more benefit than you have any reason to hope for.”

Think about adopting some of the following strategies in your classes:

  • Talk about how learning occurs (a little bit of brain science never hurt), and what the research has shown about different mindsets. You can tell stories about former students who thought they would never learn the subject but who, with persistence and effort, ended up being successful in the course or in their later careers.
  • Talk about what it will take to learn the course material effectively —make explicit your expectations for the amount of time they should be putting in and the types of activities they should be engaging in outside of class.
  • Emphasize that “fast” learning, getting assignments or exams done quickly, is not the same as “deep” learning. Often students who take longer to “get it” learn the material more deeply. Faster can at times produce the right answer, but often at the expense of learning.
  • Break difficult or complex tasks down into their component parts so that students will see for themselves their own skills building up over time.
  • Think of how you respond to students in class. For students who answer correctly (or with good insight), just delivering praise (“Great answer, Alexa!”) doesn’t give them or those who didn’t have the right answer any sense of where that great answer came from or where to go from there. Perhaps this might help more: “Good answer from Alexa…she was able to look at the evidence from the last experiment [filling in what that was] and apply it to the new circumstance. Where else can we go with this? What’s still missing?”

3. In terms of our curriculum: Think, for a moment, of the structure of our curriculum. Its premise is the same as that of all liberal arts curricula: depth (major) plus breadth (“general education” requirements). But why the breadth? We have a new “Curriculum Exploration Requirement” that requires students to “complete a number of courses distributed across the curriculum,” and “engage the curriculum broadly from the time they arrive at Oberlin.” We do this “to encourage students to become familiar with a range of scholarly approaches in different subject areas by exploring the curricula in each of the three broad divisions of the College (arts and humanities, social and behavior sciences, and natural sciences and mathematics).” This is not a bad goal, but what if we were to think about it in terms of what mindset research tells us. The point is not just or only exposure to different scholarly, disciplinary or epistemological approaches for the sake of exposure, but an insistence that students challenge themselves in order to build the growth mindset that will serve them well later on. If we require students to take courses in areas they feel “dumb” in, they will often try to satisfy the requirements by essentially avoiding them (“what can I do to satisfy QFR without taking math?”), or, if they do enroll, will not really engage in them (pass/fail?) since to do so and do poorly will only prove how stupid they are. (And, perhaps we enable that: take that philosophy class pass/fail if you’re worried about the grade!) Instead, how do we build challenges into the curriculum?  How do we provide a curriculum that encourages students to realize that they can learn in areas that previously they had closed off? How, to return to Einstein’s quote, can we construct a curriculum that helps all our students stay with problems longer? If we think about our curriculum through Dweck’s research, it suggests that the basis of curricular decisions should not be on breadth for the sake of breadth, but challenge for the sake of challenge. We can do that by engaging the various divisions of the College and Conservatory, but why we do that now becomes clearer.

It should also be clear that everything about mindset research that applies to students in the classroom, applies equally to what they do outside of the classroom: in athletics, leadership, artistic endeavors, etc.

A final point on teaching via mindset research. We set high standards, expect our students to meet them, and will provide them guidance for doing just that. But are we inadvertently introducing cues into the classroom that needlessly (and quite likely unintentionally) tell some students that they don’t belong? We need to think of the examples we use, the posters on our walls, the pronouns we employ. Will they tell some students that math, or computer science, or creative writing is just not for them? I think of a quote from Justice Sonya Sotomayor, in her recent biography.  She said that when she reached college she “felt like a visitor landing in an alien land.” “I have spent my years since [college],” she continued, “while at law school, and in my various professional jobs, not feeling completely a part of the worlds I inhabit.” What can we do to insure that all our students not only feel that they belong, but understand that, if they work hard, they can grow.

OK, then. Back to my drawing of the American eagle!

how-to-draw-an-eagle-0000

Some additional resources:

Carol Dweck’s homepage at Stanford links to more than 40 pdf’s of her articles.

Mindset: The New Psychology of Success (Ballantine Books, 2007).

Self-theories: Their Role in Motivation, Personality, and Development (Essays in Social Psychology) (Psychology Press, 2000).

Marina Krakovsky, “The Effort Effect,” Stanford Alumni Magazine (March/April 2007).

Maria Popova, “Fixed vs. Growth: The Two Basic Mindsets that Shape Our Lives,” Brain Pickings (Jan. 29, 2014).

“Mindset and Math/Science Achievement (2008),” National Numeracy (Nov. 21, 2013). The article links to Carol S. Dweck, “Mindsets and Math/Science Achievement.”

Critical Thinking in the Classroom: Some Questions for the Summer

Steve Volk (May 4, 2014)

If you did a search for the “learning goals” of liberal arts colleges, you probably wouldn’t  find a single one that didn’t emphasize “critical thinking.” In fact, critical thinking as a desired educational outcome only makes headlines when some group decides that it’s not what schools should be teaching, which brings us to the 2012 platform of the Texas Republican Party:

We oppose the teaching of Higher Order Thinking Skills (HOTS) (values clarification), critical thinking skills and similar programs that are simply a relabeling of Outcome-Based Education (OBE) (mastery learning) which focus on behavior modification and have the purpose of challenging the student’s fixed beliefs and undermining parental authority” (p. 12).

Ah, yes. Education should not challenge fixed beliefs or parental authority! Still, I’m not interested in pursuing that line of thought at the moment (as tempting as that might be), but rather want to consider what we mean when we talk about “critical thinking.” And, while I’m at it, I’d like to raise some questions for us to think about over the summer months which are visible right on the horizon: Are we doing what we should to foster critical thinking skills in the classroom? What more could we be doing? What kind of support do we need to create classroom pedagogies that foreground critical thinking? What challenges are we likely to face?

Walking to the Horizon (by I-am-Avalon). Creative Commons

While there is some discussion as to what, precisely, we mean by “critical thinking,” most cognitive or developmental psychologists would be content with the description provided by the American Association of Colleges & Universities (AAC&U) in one of its “VALUE” rubrics: “Critical thinking is a habit of mind characterized by the comprehensive exploration of issues, ideas, artifacts, and events before accepting or formulating an opinion or conclusion.” Those who create classroom strategies based around Bloom’s (2001 revised) taxonomy will recognize critical thinking as central to higher order thinking skills: analyzing, evaluating, and creating. (Bloom’s Taxonomy is a standard means of categorizing cognitive tasks by complexity, with the simplest at the bottom and the most complicated at the top.)

Bloom's Taxonomy (Revised)

Daniel T. Willingham, a cognitive scientist at the University of Virginia, provides his own layperson’s definition: “Critical thinking,” he writes, “consists of seeing both sides of an issue, being open to new evidence that disconfirms your ideas, reasoning dispassionately, demanding that claims be backed by evidence, deducing and inferring conclusions from available facts, solving problems, and so forth.”

The November 19, 2012 “Article of the Week” was Willingham’s “Critical Thinking: Why Is It So Hard to Teach?” In it he discussed the complications of actually teaching critical thinking, and you can review it again to see his arguments.

But I remain confident that as we explore the various components of critical thinking, we can develop teaching strategies which foreground it as an essential learning outcome for each class we teach. Perhaps what is so difficult about teaching critical thinking is that it is just one cognitive competency that is at stake within broad pedagogical contexts that require the development of specific abilities (e.g., the ability to take multiple perspectives, to layer relationships, etc.) and dispositions (including risk taking, task persistence, the ownership of learning, and perceptions of accomplishment)  (Perkins, 1994).

(Public Domain)

Shari Tishman, David Perkins and others (Tishman et al, 1993) have been examining what they call “thinking dispositions” for many years, developing useful approaches to help teachers develop informed pedagogies. Their list of seven dispositions that normally describe productive intellectual behavior includes:

  • The disposition to be broad and adventurous – open minded; explore alternative views; being alert to narrow thinking; the ability to generate multiple options.
  • The disposition toward sustained intellectual curiosity: to wonder, probe, find problems, observe closely and formulate questions; a zest for inquiry, alertness for anomalies.
  • The disposition to clarify and seek understanding: a desire to understand clearly, to seek connections and explanations; an alertness to muddiness, an appreciation of the need for focus; an ability to build conceptualizations.
  • The disposition to be “planful” and strategic: the drive and ability to set goals, make and execute plans, envision outcomes; an alertness to a lack of direction.
  • The disposition to be intellectually careful: the urge for precision, organization, thoroughness; an alertness to possible error or inaccuracy; the ability to process information precisely.
  • The disposition to seek and evaluate reasons: the tendency to question the given, to demand justification; an alertness to the need for evidence; the ability to weigh and assess reasons.
  • The disposition to be metacognitive: the tendency to be aware of and monitor the flow of one’s own thinking situations; ability to exercise control of mental processes and to be reflective.

Much like the broad field of critical thinking, it isn’t easy to teach these dispositions. As we rush to cover the content areas that our students need in any particular class, we often quietly shove any focus on these learning dispositions out the window.

Matthäus Merian, Defenestration of Prague, 1618 (Creative Commons)

But even if it’s not a question of sacrificing dispositions to content coverage, foregrounding learning dispositions can be challenging for a number of reasons. For one, we often find that students, in their own cognitive development, are embedded in a “multiplist/subjectivist” phase of thinking, as Patty deWinstanley has pointed out in her valuable discussions with faculty preparing to teach first year seminars. [The best known proponent of a theory of intellectual and cognitive development among college age students is William G. Perry (Perry, 1970).] Whatever a student in that stage thinks is right, is right; and in areas where the “right” answer isn’t known, a multiplicity of views is right. Our challenge then becomes how we move students from this phase to a more “evaluativist” position.

In many ways, grappling with critical thinking and creating thinking dispositions in our classes can turn them into what Mary Louise Pratt described as a “contact zone” – a social space where “cultures meet, clash, and grapple with each other…” If difficult, that “zone” is also where, Dewey argued (1910), real thinking occurs. Dewey talked about some experiences as “educative” because they enhanced the making of further experiences, and others as “mis-educative” because they had no further influence upon later experiences (Dewey, 1947). Thinking occurs, he argued, when the “normal flow” is interrupted, when our common assumptions and perceptions are challenged, when problems and conflict arise. In other words, learning occurs at this point of “clash.”

Our challenge as teachers, then, is to encourage the emergence of a “contact zone” classroom, what I would call the “uncomfortable classroom,” at the same time that we use the discomfort created to open it to real dialogue (Freire, 1970). When we have figured that out, we will be far on our way to promoting critical thinking in the classroom.

Still, as I suggested in the opening questions, creating such pedagogies often requires considerable support, so I encourage you to use the summer months to think about what you could use to help create challenging and dialogic classrooms. What seems perfectly clear and sounds impeccably logical on paper can feel quite different in the heat of the moment in a classroom discussion.

Some bibliography

John Dewey, Experience and Education [1947] (Free Press, 1997).

_______ , How We Think [1910] (CreateSpace Independent Publishing Platform (February 10, 2013).

Paulo Freire, Pedagogy of the Oppressed [1970] (Bloomsbury Academic; 30th Anniversary edition (September 1, 2000).

See David N. Perkins, The Intelligent Eye. Learning to Think by Looking at Art (LA: The Getty Center for Education in the Arts, 1994).

William G. Perry, Forms of Intellectual and Ethical Development in the College Years: A Scheme (New York: Holt, Rinehart, and Winston, 1970.)

Shari Tishman, Eileen Jay, David N. Perkins, “Teaching Thinking Dispositions: From Transmission to Enculturation,” Theory into Practice 32:3 (Summer 1993): 147-153.]

Daniel T. Willingham, “Critical Thinking: Why Is It So Hard to Teach?” American Educator (Summer 2007): 8-19.

Closing Time: Managing the End of the Semester

Steve Volk (April 28, 2014)

(Note: This is a revised and updated version of “Topics in Teaching and Learning” written on May 9, 2011).

“Every new beginning comes from some other beginning’s end, yeah,” Semisonic

The end of the semester, like the first week, poses specific challenges to teachers. For most of us, it feels like we’re in a head-long rush to complete the syllabus, hand out evaluations [Check back to the “Article of the Week” from December 8, 2010 for tips on how to read your Student Evaluations], and prepare students for final exams or papers, all the while trying to achieve closure on the semester. It’s also a time when both student and faculty energy levels have bottomed out, even more so in the spring semester. It probably goes without saying that the best way to end the semester is the way that works for you. But here are some suggestions that have come up over the years from my own practice and some that I’ve taken from other teaching and learning centers.

Icky List - Wait But Why

  • Revisit the course goals written in your syllabus with your class. This is a good time to synthesize the main points covered in your course by way of a discussion of the goals you established at the start of the semester and what you, in fact, were able to accomplish. It’s yet one more way to help students reflect on the design of your course, why you structured it as you did, and how the assignments they have completed (along with the final assignment) were there to help the students accomplish the course objectives. The review allows students to step back somewhat from the course content in order to examine what they have accomplished on a broader level.
  • After you have revisited the syllabus and the course goals with students, you can open time for student reflection and self-assessment, encouraging them to think about how they have achieved the course’s goals, what they still need to do before taking a final exam or writing their last paper. You can extend this by asking students to write a short (anonymous) self-evaluation. This allows them to reflect on their performance and behavior in the class. Such an exercise goes substantially beyond the self-assessment questions on the Student Evaluation of Teaching (SET) forms which they will be getting, and hopefully will help them think about their own learning and the next classes they want to take.  One instructor (Ted Panitz, a math teacher at Cape Cod Community College) asks his students the following:
  • Has your approach to math changed during this course or compared to previous courses? How?
  • Have your attitudes or feelings about math changed?
  • How do you feel you performed in this course?
  • What would you do differently if you had a chance to do this all over again?

Information Overload - Creative Commons (http://www.principals.com.au)

One question you might want to consider, particularly for a class in which there has been a substantial amount of discussion, is to ask students to reflect on their own participation in the discussions and whether they thought they intervened in a way that supported (everyone’s) learning in the classroom or whether it had the effect of isolating or silencing other students.

If you want, you can also add sections which encourage students to suggest ways you can improve class procedures or ask how they feel about particular teaching approaches you have used that semester and would like to hear specific feedback.

  • Have students create a concept map of the course they are completing. (For tips, see the “Article of the Week” from March 14, 2011.)
  • Student presentations often occur in the last few weeks of the semester. I know of one instructor who has her students present a short lesson for the class on the issue, topic, or theme that they found most difficult or challenging during the semester. It is an excellent way for students to prepare for exams, since we all know that teaching a subject is the best way to learn it.
  • Encourage your students to revisit earlier writing (or other) assignments in the course as a way to measure their own learning in the class, to assess what they have learned and the areas in which they still feel unconsolidated. One way to do this is to ask students to bring their papers to class and then break them into smaller groups where they can discuss their papers with peers.
  • In a similar fashion, you can have students in small groups discuss how their thinking has changed over the course of the semester. They can take notes for themselves (and/or for you).

This can include new appreciations for the content covered, for their own strengths and weaknesses, or for meta issues as they reflect on their own learning.

  • Students can be encouraged to discuss what they consider to be the critical moments in the course: insights they have had; content that they have found most surprising; highlights in the course.
  • By way of course review for exams, you can group students to collaborate on one or two typical exam questions involving analysis, synthesis, application, etc.

Learning from the Semester

In an “Article of the Week” from November 25, 2013, I offered three ways for faculty to look back and learn from the semester that just ended. Here (again) are some questions to think about:

  • What do you feel was the strongest part of your teaching (and student learning) this semester?
  • Why do you think that happened? Link outcomes to your teaching methods.
  • Do you think you achieved your learning goals for the course? This, of course, should lead you back to your learning objectives, help you think about them again, and consider whether you can actually answer this question.
  • What do you think basically didn’t work in the course? What do you feel least pleased, or most uneasy, about?  What left you thinking: next time, I just won’t do that?
  • As above: Why did you (or didn’t you) reach your learning objectives? Link outcomes to your teaching approach.
  • Getting concrete: what do you want to at least think about doing differently next time?
  • Very briefly: If you are not sure what to do to change the results, who are the people and what are the resources that can help?

    Procrastination - Creative Common images

Stress and Anxiety

While we all know this at some basic level, it is useful to keep in mind how stressful the end of the semester can be for students – and for faculty. We all have a lot to do, and there are many crunch-time challenges. In terms of students, we all notice a general increase in their tiredness and, often, illness. But we should also be aware of times when stress turns into anxiety and when our usual techniques for helping students regain their footing and confidence could use some extra support. Don’t forget the help that can be provided by the Dean of Students’ office, the Counseling Center, the Dean of Studies office or Student Academic Services. If you are not sure whom you should be talking to, always start with the class deans. According to the Anxiety and Depression Association of America, some 40 million college students suffer from an anxiety disorder and 75% will have an anxiety episode by age 22. For more on coping with student stress, see my “Article of the Week” for September 2, 2013.

Saying Good-bye…

.… can be a lot harder than you imagine, and it’s not unusual to feel a sense of loss as the semester (the year and, for your seniors, a college career) come to an end. Even after many years teaching, I’m often still amazed at how hard this can be.

So, don’t be afraid to offer your students some parting thoughts even though this might sound really cheesy. If you mean them, your students will appreciate them.

I often tell my students that, once they have graduated, I’m happy to have them as “friends” on Facebook and that it actually means a lot to me that they keep in touch, let me know how they are doing and what they’re up to.

And, of course, this is the time for any end-of-semester ritual that you may have developed (donuts, sing-a-longs, poetry reading, etc.).

Finally

From Josh Eyler’s blog (“A Lifetime’s Training: Thoughts on Teaching and Learning in Higher Education”), a post that I included at the end of the Fall 2013 semester, “The Final Class of the Semester”:

On every final exam I have ever given, I have written a note expressing my thanks to students for their work over the course of the semester.  In this note, I also include quotes from two movies.  The first is from Back to the Future, one of my favorite comedies.  Towards the end of the film and after many hijinks have ensued, Marty McFly says to the 1955 versions of his parents, “It’s been…educational.”  I use this quote to inject a little levity into the generally high-stressed atmosphere of the exam and hopefully also to emphasize the ways in which we have all learned from each other.

The other quote that I use is from Dead Poets Society.  Love it or hate it, the movie has some powerful things to say.  The quote that I borrow from the film is not the over-used ‘Carpe Diem,’ but instead the line that follows it:  ‘Make your lives extraordinary.’

Most of all, this is what I hope for each of my students, and I wish them all the very best.

And two end-of-semester haikus from the blogger at “Confessions of a Community College Dean” (Inside Higher Ed.com):

temper tantrums fly

yet are mercifully brief

who has energy?

amazed at colleagues

miracles on a shoestring

take a bow, people

The Last Five Minutes: Class Endings and Student Learning

Steve Volk (Director, Center for Teaching Innovation and Excellence), April 20, 2014

A recent article by David Gooblar in the Chronicle of Higher Education’s on-line “Pedagogy Unbound” section reminded me how important it is to end a class well, both individual classes (today’s topic), and the semester as a whole (which I’ll turn to soon). We spend a fair amount of time thinking about how we start a class: perhaps summarizing material from the past class, highlighting written responses to the readings that students have posted, offering a snapshot of what the day’s class will cover. But the ending is often less planned, particularly as we rush to get through the topics we had intended to cover that day.

Why is that? Probably a lot of factors are to blame, but the most common one I’ve encountered is that faculty try to put too much into the 50 or 75 minutes we have in a regular class session; we try to cover too much. Many, myself included, particularly when I was a bit newer to the game, are worried that we will run out of things to say before the clock signals the end of the class. As a result, we over-prepare … just to be sure. Of course, we never actually run out of things to say. Rather, we run out of time in which to say them. Now, when we look up at the clock, we find that there are five minutes left and we have 15 minutes worth of “stuff” still to deliver. What to do?

Usually we try one of these strategies:

  • We talk faster in an attempt to squeeze it in;
  • We continue to talk even as the students put on their coats, anxiously lean towards the door, and we spy next instructor assigned to the same classroom peaking into the room;
  • We carry over a the “untaught” material from that class on to the next (which only seems to exacerbate the problem if it happens every day);
  • We drop that part of the class where the students are asked to synthesize and share the main conclusions from the small-group discussions they had just finished.

Bud Collyer as host of "Beat the Clock" (1958) - Wikimedia Commons

None of these strategies, obviously, is optimal for student learning, but I’ve employed each of them enough times (and observed others doing so as well), that it seems reasonable to offer a few suggestions from my perch at the back of the room. Now, some students may be so deeply engaged in a class that they don’t notice the time…but that usually isn’t the case. Because they need to be in another class in 10 minutes or simply expect the class to be ending, any observer can see them closing down, as it were. They start to pack their bags, put on their coats, close their computers, or just look at the door. The key point is that they are not listening (or learning) any more, and going past the time the class is to end by more than 1-2 minutes only makes them more anxious and less able to hear. We may feel that we were able to squeeze in everything we wanted to say in those last few minutes…but they probably didn’t get it. Production but no reception.

Too Much Johnson, William Gillette, 1895 (Public Domain).

This can be even more of an issue when teachers sacrifice some of the most important lessons of the class because they have run out of time. This will often happen when students have broken into smaller discussion groups and you had planned to bring them back together both to share their observations and to have them (or you) synthesize the key points you wanted to cover. Not every small group discussion has to be shared or synthesized, and you can develop other ways to do this without re-forming into a group-of-the-whole (see Tips for Capturing Small-Group Discussions below), but if you count on that moment to raise the cognitive level of the class and you have to forgo it because of lack of time, you’re passing up an important moment of student learning. There’s no question that this will happen from time to time: issues might come up in discussion that are just too good to sidetrack. And you can always start the next class with a summary of the last discussion (although, usually, students aren’t as able to return to that discussion two days later as you would like them to be).

Why Are We Doing It This Way?

As I suggested above, we often over-prepare a class for fear of being caught with our critical pants down: we have come to the end of our useful knowledge for that day and have nothing more to say. Besides the fact that I think we can always find something to say or to have the students do (and it does get easier as the years go by), I think the tendency to put too much into a class often reflects the fact that we haven’t thought enough about what our specific goals are for that class session. As a history teacher, I know that my “goal” was often only to cover a given chronology. If I was talking about the origins of the Cuban Revolution, I knew I wanted to end up in 1959; 1956 just wouldn’t do, so I’d speak faster to get it all in before class ended. But as I continued to think about what I was doing and what the students were getting out of it, I realized that more is not always more – quite often it is less. You think the students have understood something that you squeezed in at the end, but they haven’t.  As I tried increasingly to get at the central analytic issues involved in any particular class, I realized that I could plan a class that didn’t have to cover all the material I was delivering (after all, they have readings and other resources), but could focus on a few exemplary moments to help them work through the central concepts (in this case problematizing the question of what revolutions are, what we mean by “revolutionary origins,” what was it about the specific history of Cuba that gave rise to the events of 1959). This extra planning hasn’t meant that I never run out of time in my classes, but at least the main part of the learning that I want to happen occurs sometime before the last few minutes of the class.

Che Guevara and Fidel Castro enter Havana, January 1959

How to Make Use of those Last Minutes?

There is an additional benefit to class planning: you can now use the final minutes of class in an activity that not only can hold the students’ attention, but can help you significantly in understanding what they got out of the class and how you might want to begin the next class. Use the last 2-3 minutes of class time to have them write. They can write a “muddy-point” commentary, noting something that they didn’t understand or would like further discussed in the next class. They can focus on the 1-3 points that they learned from that class. Or they can look ahead: what are their preconceptions of the next class. (I have posted a very short article on this topic, “The Final Three Minutes with 100 Undergraduates,” by Robert Hampel, on CTIE’s Blackboard site. It appeared in the most recent edition of College Teaching 62 (2014): 77–78.) These short exercises have the advantage of focusing student attention on a very defined task for the last few minutes of class rather than on wondering when you will stop talking, giving you an important idea of what they learned (or didn’t learn) in the class, and allowing those who have more to say (and who have the time to stay) to spend a minute or two more than the others with their comments. (Of course, there’s still the next instructor assigned to the class looking through the door wondering when she’ll be able to get in!)

Tips for Capturing Small-Group Discussions

Often, in discussions, I use one of two techniques as a way to help students develop and hold onto their conclusions. If I have 6-7 different small groups working at the same time, either on the same or a different set of questions, I have them to come up to the board towards the end of the time I have set aside for discussion and write down the conclusions, answers, or questions their group arrived at. If there is no time to bring these different “conversations” together, I take a picture of the board with my phone and post it to Blackboard. The students can refer to the image after class and I can start the next class with the image of the last class’s board projected on a screen. (You can do the same thing with post-in notes, having them write on the notes in their groups and then sticking them under appropriate headings.)

Partial image of chalk board after discussion (Steve Volk)

Arranging "post-it" notes (Steve Volk)

The other technique I use is to make sure that someone with a laptop is a part of each group. Before class, I have prepared a Google Doc with the same number of columns as there are small groups (add more columns if you’re not sure of the number), assign numbers to each of the discussion groups that has formed up, enter the email addresses of the student with the laptop into an “invitation” to join the document (all of which takes, literally, about 2 minutes), and then have them write conclusions, answers, or questions in their assigned column as they are discussing the material. I project that Google Doc onto the screen and can see each group’s discussion develop in real time. Again, if there is time to pull everything together, we do that at the end of class when we all look at the document that is projected; if we have run out of time, I save the document, post it to Blackboard for them to read, and (if I want) pull it up at the start of the next class.

Google Doc example from class (Steve Volk)

Google Doc example from class (Steve Volk)

Multidisciplinarity: Reflections on a Molecular Microbiologist’s Visit to a Literature Course

Marcelo Vinces leads the newly established Center for Learning, Education, and Research in the Sciences (CLEAR). He helps coordinate on-campus undergraduate research, trains peer mentors, and coordinates workshops for faculty. Vinces earned a BA in biology at Cornell University and a PhD in molecular microbiology at Tufts University. Prior to Oberlin, Vinces worked in Washington, D.C., as an American Association for the Advancement of Science (AAAS) Science and Technology policy fellow at the National Science Foundation.

Kelly Bezio is a visiting assistant professor of English. Her research is in communicable disorders, particularly how American authors used such illnesses to imagine—paradoxically—community formation. This research has allowed her to explore, among other topics, the mysteries and miracles of smallpox inoculation, the dank boredom of foreign quarantine, and awe-inspiring scourges such as cholera.  She is currently co-editing an essay collection titled Religion and Medicine in America’s Secular Age, and aspires to write a second monograph on how literary depictions of democratic liberal subjectivity drew on developments in modern chemistry—creating what we might call a chemical aesthetics. She received her Ph.D. in English from the University of North Carolina at Chapel Hill.

Marcelo’s observations:

Kelly and I met for the first time when we sat next to each other at one of our students’ presentation for a biology class. The student, Nicole Le, gave an oral presentation on the nightingale for her final class project that was squarely at the intersection of literature, neuroscience and evolutionary biology. From our initial meeting and subsequent conversations, Kelly ended up inviting me to her class.  So, on February 18, early in the semester, I took part in ENGL 351, Literature, Medicine, Culture, a course that explores what stories about doctors and medicine, from Mary Shelley’s Frankenstein to Fox TV’s House, tell us about our culture, our history, and the experience of being human. The topic of the day’s class was Nature vs. Nurture (Mad Scientists.)

Students had read two short stories by Nathaniel Hawthorne, “Rappaccinni’s Daughter” and “The Birth-Mark,” as well as selections from Mary Shelley’s Frankenstein. Additionally, Kelly had asked if I had any suggested readings, and I recommended an essay by H. Davies, “Can Mary Shelley’s Frankenstein be read as an early research ethics text?” I had previously used this as a recommended reading to summer research students in conjunction with a research ethics session the Office of Undergraduate Research had organized and an unrelated screening of Bride of Frankenstein that I had organized.

Nathaniel Hawthorne (Creative Commons)

Kelly shared her planning notes for the class with me which called for a very interactive class between students talking about “doing science” and the students talking about the stories they had read for class. Some of the main questions to be covered included: What is the nature/nurture debate? How have you related to the nature/nurture debate in the past? How do the readings give us a new perspective on this debate? In other words, what would our scientist characters say about this debate? In contrast, what would our authors say about this debate? How does the “mad scientist” (as a stock character) complicate this debate?

Reflections on the Discussion

I introduced myself and gave a brief chalkboard chat on the research I did for my PhD and for my postdoc. As I was explaining the science, it occurred to me to raise the point of why I chose to study what I did. It’s easy to explain why the questions I pursued were interesting scientifically, but were there other underlying motivations? For example, what made the topics exciting and sexy to me? Was there an aesthetic quality I was not consciously aware of that made me gravitate towards such questions? Did I feel love towards my research? I think I was conscious of these notions only because of the context of the class and the readings we had just done. It would otherwise never have come up in conversation or even in my thoughts within the context of doing science research and being among science colleagues.

Marcelo Vinces Chalkboard Image

The students did not have many questions for me as the start. I felt they were still a bit shy or reserved or didn’t yet know what to make of me. But one good question I did get asked me if, “as a scientist, was there a conflict in trying to get to answers and at the same time trying to get publications?” I absolutely loved this question because it is a fundamental conflict in science. While we are in it to get answers to questions, there are the realities that also put demands on us, sometimes in line with our need for selfless inquiry, but other times in conflict: publish or perish, aiming for high profile journals, needing to get tenure, patents. These don’t always lend themselves to pure scientific inquiry for the sake of truth and knowledge. For example, I noted to the student, we often have interesting but negative results that hide for eternity in lab journals that would be of interest for other scientists to know, but that no journal would want to publish. Vast amounts of good and useful data may lie in lab notebooks and computers around the world that we may never know about. This is due to the current structures that scaffold the scientific enterprise.

The discussion moved into the readings, focusing on what we could call an “irrational spark” as the motivation for doing science, which, after all, is usually seen as the most rational form of inquiry that exists. And yet the scientists in these stories, driven by love or fame, used science for what many would say were not just irrational purposes, but immoral ones too.

It was difficult to discern what Hawthorne was getting at in these stories, if he had something definitive to say about scientists or science. With Shelley we had the help of the reading I suggested, which takes the view that Shelley was not an anti-science romantic, but rather saw the promise of science but also the very human drives that compromised it.

The questions and discussions, very nicely facilitated by Kelly, included, what motivates scientists? How has science changed our definition of evil? Are Hawthorne and Shelley offering cautionary tales? Mourning the imperfectability of the scientific method? Something else?  Can science actually make things more evil than nature? Can literature?

Missed opportunities? More Reflections

We sadly ran out of time just as the discussion was reaching a crescendo of ideas, thoughts and energy. In retrospect, Kelly or I could have offered more prompts for students designed to take advantage of a scientist in the room. As we discussed the above questions, what would a scientist have to say? Do scientists think of these questions? What does a scientist feel and think as he or she reads about these mad scientists? Are there mad scientists? What exists to make sure mad scientists don’t run amok? What could the scientist in the room learn from these readings and subsequent discussion?

I am curious to hear from Kelly’s point of view how this guest appearance worked for her course, how engaged were the students, and what she would do differently in the future.

Future ideas?

I for one feel the session opened up pathways of inquiry for me that are not usually ones available in the scientific context. For example, what is the psychology at work that drives a scientist to spend hours alone, at night and weekends, to seek answers to sometimes very obscure questions? What are the misunderstandings that exist among non-scientists about the scientific endeavor? What place can literature serve in educating scientists of about research ethics, motivation, and science history? Ultimately, I feel this is a perfect illustration of how we need more of these intersectional events that bring together people from very distinct disciplines that may nevertheless share some important interests in common.

——

Kelly’s observations:

The students were enthusiastic about having Marcelo come in. They hadn’t heard of CLEAR, and it struck me as a good opportunity to introduce them to the many ways interdisciplinarity was happening on campus.

My intention was for Marcelo to come in and talk about the research he conducted as a scientist in order to provide students with a modern counterpoint to the stories about scientists they were reading by Nathaniel Hawthorne. Listening to Marcelo’s comments, I was noticing some common themes: interest becoming obsession, the delights of observation, the power that comes from doing research, the role love plays in doing research, to name a few.

Mary Shelley by Reginald Easton[Public domain, via Wikimedia Commons

I framed the visit to the students (an approach Marcelo and I agreed on ahead of time) as an opportunity to learn what Marcelo had to teach us about doing scientific research and an opportunity to teach him, in turn, something about the stories we had read.

A couple of interventions may have made this frame more successful, in terms of empowering the students. On the one hand, it may have helped to tell them ahead of time to come up with discussion questions. That is to say, teaching is often about asking questions, not delivering information. (The next day in class, the students told me they would have liked to have heard more of Marcelo’s perspective on the stories we all read. This could have happened if they or I had thought to ask him his perspective on particular moments in the text or about the readings in general. The students reported that they were worried about offending him, since the stories we read are often taken as critical of the scientific enterprise.) On the other hand, it may have helped to have students break up into groups after Marcelo’s comments and come up with the “lessons” or “questions” they wanted to put to him, rather than jumping right in to discussion.

That being said, we had a really wonderful discussion that day, and I speculate that it was because they were imagining Marcelo’s potentially-offended response to the critique of science, and so pushed beyond that interpretation of the stories. (Maybe). Over the course of the discussion, we came up with several excellent questions:

  • How is science changing the definition of evil?
  • How is literature changing the definition of evil?
  • What motivates science? (If we want to continue our literary corollary, what motivates literature?)
  • What do we want science’s motivations to be?
  • What do we want literature’s motivations to be?
  • Can humans make things that are more evil than Nature would create?

Ultimately, given the opportunity to do this kind of collaboration again—which I would love to do—I think I might try to build conversation between the class and the guest speaker by asking questions like, “As a scientist, how did you respond to the stories?” and “As a literature student, how did you respond to the stories?”

Follow Up:

Marcelo and I met on March 20 to discuss the joint session [see above], and I was really taken with the photograph he took of the whiteboard from our class, which was covered with his drawings of his research, questions we came up with as a class during our discussion, and a student’s note about a scientist he studied in another class.

What struck me about photograph was how well it represents the benefit of this kind of multidisciplinary day in the classroom. It collects the topics we discussed, all of which came from specific disciplinary perspectives, and the new questions that arose when those topics were brought up together.

I noticed also during our conversation that Marcelo and I came up with different disciplinary inspirations post-class. I was interested in how talking about “mad scientists” with a scientist helped us to ask new literary questions of Nathaniel Hawthorne. For instance, at the end of class, one of the students raised the idea that Hawthorne may have been interested in working through cultural expectations of literature and what it was able to do—or should do, ethically speaking. In other words, the stories we read may be as much about literary ethics as they are about scientific ethics. On the other hand, Marcelo was thinking about how literature might be used to enliven the study of ethics in a STEM context.

I think in conclusion that multidisciplinarity produces different ways of being disciplinary. This is not new (of course), but I was intrigued by the material results of this abstract truism.

Only Connect? What is the Future of Higher Education

Last November, our colleague, David Orr, delivered the Presidential Lecture entitled: “Only Connect: A 4th Gyre.” (If you missed it, you can catch up with it on YouTube).  David borrowed E.M. Forster’s phrase, “only connect” to underline what is needed to “turn vicious cycles into virtuous cycles that eventually transform our politics, economy, cities, buildings, infrastructure, landscapes, transportation, agriculture, technologies, and our manner of thinking.”

Forster’s injunction, from Howards End, spoke to the importance of bringing all the parts of one’s life together: “Only connect the prose and the passion, and both will be exalted, and human love will be seen at its height. Live in fragments no longer.”

EM Forster, Howards End

This charge, to “only connect,” ricocheted around my brain as I read a new study produced by the Gallup organization for the Lumina Foundation.  “What America Needs to Know About Higher Education Redesign,” argues that “the importance of post-secondary education in preparing and connecting people with a good job” should be at the center of a redesign of higher education. By stressing that single purpose, the study stands Forster on his head, arguing in favor of a life in fragments, one in which students are disconnected from community and purpose, to say nothing of passion and prose.

Now, Lord knows, I have nothing against our graduates getting “good jobs.” Indeed, we would do well to think about this more. But the study so assiduously frames its findings within a master narrative of an education system that has failed in its mandate to prepare workers for the labor market that it ignores not just the larger purposes of a higher education, but its own data. To cite just one example, the poll highlights the finding that 95% of the public judge the quality of the college or university on the ability of its graduates to “get a good job.” 95%! That’s hard to argue with – the U.S. public want a higher ed sector that “only connects” graduates and jobs.

And yet an even higher percentage of respondents (98%) judged the quality of a college or university not on its connection to the job market, but on the qualifications of the faculty, and a substantial majority (86%) found the percentage of students who pursue further education to be of significant importance when determining the worth of colleges and universities. These points weren’t included in Lumina’s bullet points or picked up by the press; neither particularly fit into their seemingly preferred degree-to-job narrative.

Even more disturbing was the second part of the study which polled “business leaders” (also defined as “thought leaders”) in executive positions (but not further described). Nearly 70% of those polled said they would consider hiring employees without a post-secondary degree or certification, and only 16% of them agreed that “most jobs at my business require a post-secondary degree or credential to be successful.” Thought leaders? A myriad of other polls have underscored the importance of a post-secondary education for future success in the labor market(See, for example, results from the American Association of Colleges and Universities and here and Northeastern University). But, evidently, not those questioned by Lumina/Gallup.

Let me be clear. We do want our graduates to earn a living, to be able to find meaningful employment. If the employers at Google or Genetech or Ford express a dissatisfaction with college graduates, shouldn’t we pay attention? Yes, at some level, we should. But not, it would appear, from Lumina’s particular group of 623 business leaders. A further data point from the study (one that, again, didn’t make the highlight reel) discloses that 68% of the employers surveyed said that it was somewhat to very likely that employees at their businesses who earned a post-secondary degree while working for them would leave their current jobs for employment at other firms.

Student working in Statistics Machine Room, 1964, London School of Economics

I’m no wizard at statistics, but what we seem to have here is a poll taken of “thought leaders” who (in significant percentages) don’t think post-secondary education is important for hiring at the present time, don’t think that post-secondary education will be needed for a large percentage of their firms’ jobs in the future, and recognize that most of their employees would abandon ship if they actually did manage to earn a post-secondary degree. Yet the study headlines the fact that only 11% of those polled strongly agree with the statement that higher education is actually graduating students to meet their needs. Huh?

As troubling as I find this instrumentalist view of higher education, I think the Lumina/Gallup data is useful in three respects. (1) It encourages those of us in the very rarified atmosphere of liberal arts colleges (only 4% of the total higher education sector) to think about how we relate to the sector as a whole and, more to the point, what we believe higher education should do for all who pass through its doors. (2) It draws attention to the fact that employment is what will be in the future for almost all our students, and while that employment will not be as narrowly defined as Lumina suggests, we do need to think about the connection between the education we provide and preparation for that future. (3) Finally, and importantly, it begs the question of what it is that we do in higher education.  I’ll only address the last point here by circling back to the charge to “only connect.”

David Orr was not the only writer on higher education to look back to Forster. In 1998, another environmentalist, the historian William Cronon, published a lovely article in The American Scholar titled, “‘Only Connect…’ The Goals of a Liberal Education.” (You might remember Cronon from 2011 when the Republican Party of Wisconsin subpoenaed his emails in an attempt to intimidate environmentalists at the University of Wisconsin.) As opposed to the Lumina Foundation, whose study suggests that higher education is solely about preparing graduates for their first, entry-level (and apparently dead-end) jobs, Cronon takes a broader view of what it is we do, one that is so engaging I hope you will read his article, which I’m offering as this week’s “Article of the Week.” (You can find it here, and in CTIE’s Blackboard site: CTIE>Article of the Week>March 3, 2014.)

What do we mean by a “liberal education,” Cronon asks? He points out that “liberal,” is derived from the Latin (liberalis) meaning “free.” But it’s also rooted in the Old English word lēodan (“to grow”) and lēod (“people), not to mention the Greek word eleutheros (“free”) and even the Sanskrit word rodhati, meaning “one climbs,” “one grows.” As he sums up: “Freedom and growth: here, surely, are values that lie at the very core of what we mean when we speak of a liberal education…it aspires to nurture the growth of human talent in the service of human freedom” (74).

Lieut. F. Schwatka, Wonderland; or, Alaska and the Inland Passage (Chicago: Rand McNally, 1886), 13.

In his essay, Cronon considers what should go into a liberal education curriculum. But, in the end, he remarks that it’s not really about the items in the curriculum, the boxes you check off, so much as the values we seek to instill in the students who share that educational process with us. To this end, he describes ten such values. I’ll list them here, but recommend that you read the article and discuss the points raised with colleagues.

How does one recognize liberally educated people, Cronon asks?

1. They listen and they hear.

2. They read and they understand.

3. They can talk with anyone.

4. The can write clearly and persuasively and movingly.

5. They can solve a wide variety of puzzles and problems.

6. They respect rigor not so much for its own sake but as a way of seeking truth.

7. They practice humility, tolerance, and self-criticism.

8. They understand how to get things done in the world.

9. They nurture and empower the people around them.

And, of course,

10. They follow E. M. Forster’s injunction from Howard’s End: “Only connect…”

If I had to choose between Lumina’s idea of higher education and Bill Cronon’s…well, no contest.

I’d like to periodically return to these points and see how we can develop a discussion around each of them. Are these the characteristics we want to see in our graduates? Are there others? How are they reflected in our curriculum and our graduation requirements?  Much to think about.

Tapping the Potential of Digital Scholarship

“We have inherited a cyber-infrastructure of systems, data, and services that arose from and is optimized for research in science and engineering. As a result, humanists have access to technology but are in search of questions: What scholarship becomes possible when, from their desktops, scholars can access vast stores of admittedly highly heterogeneous data together with powerful capabilities for analysis and presentation?” This is the question that Amy Friedlander, Senior Advisor in the Office of the Assistant Director of the Directorate for Social, Behavioral & Economic Sciences at the US National Science Foundation, raised in a 2008 paper, “Asking Questions and Building a Research Agenda for Digital Scholarship.”

It’s a great question, and gets at the heart of what has been called the “digital humanities,” or, more recently, “digital scholarship.” Digital scholarship, if I had to summarize it in one sentence, is about directing the massive computational power we now have at the mountains of data that have been digitized in recent years (texts, artifacts, images, sounds, etc.) to ask questions that have previously been inconceivable and to generate insights that can be truly astounding. The potential of digital scholarship to unearth new knowledge is breathtaking. I first became aware of what I would call the “low-hanging fruit” of digital scholarship — database searching — some years ago when, in the midst of another project, I came across a curious early 19th century neuroscientist (at least, that’s how I began to think of him). Alexander Walker studied the brain and the nervous system in Scotland and practiced medicine in England in the early decades of the 19th century. But he barely creased the historical record there. Quite by accident, after I had come upon a newly digitized collection of 19th century U.S. newspapers, I popped his name into the search box and discovered that he actually had quite a following in the U.S. South. (He seems to have been adopted by those seeking to advance a “scientific” authority for racism.) In any case, searchable databases opened potentials for research that would have been inconceivable previously. And this was just the beginning of the data and text mining possibilities of digital scholarship.

From searching digital sources, the field of digital scholarship has expanded widely, suggesting yet another way that liberal arts colleges can leverage its strengths to produce new knowledge and draw students into significant research projects, especially in the humanities, arts, and social sciences. By linking our computational scientists and information specialists with colleagues from other field (and other colleges), we are perfectly situated to take advantage of this potential.What can be done? Among the what I assume to be thousands of on-going projects in the U.S. and elsewhere, here are just a few examples drawn from Oberlin as well as other colleges and universities.

* Historians of science who specialize on the Iberian world have long explored the nature of scientific inquiry in Spain and the Americas, in the 17th and 18th century. A critical study on the topic was published in Barcelona in 1983 (Diccionario Historico de la ciencia moderna en España, 2 vols.). The dictionary provided information on 360 individuals active in scientific fields in Spain during this period, and was quite useful when any of these individuals crossed your radar screen. But what if you wanted to see how all of them interacted, how their interests changed over time, where and when they were active? A project out of Stanford (Mapping the Republic of Letters) has produced a remarkable database, “An Intellectual Map of Science in the Spanish Empire, 1600-1810.”  Scholars working on this project created a database with 18 different entry points which, when examined, disclosed that, as just one example, Jesuits dominated the mathematical and geographical sciences in the seventeenth century, but naval officers took that position in the eighteenth century.

* The end of slavery in the U.S. Civil War came not simply through presidential or legislative decree but through the actions of enslaved people and soldiers in the field as well.  These interactions and policies developed unevenly over time and space.  By examining the maps in Visualizing Emancipation and the primary sources found there, scholars have uncovered (and students can examine) the factors that enabled men and women to escape slavery and the geographic patterns that marked emancipation.

Fugutive Slaves, Runaway; Freedman's Colony, NY, 1864

* During the summer of 1919, a delegation under the leadership of Oberlin College President Henry Churchill King and Chicago businessman Charles R. Crane traveled to areas of the former Ottoman territories. Even as the fate of these lands was being decided at the Paris Peace Conference, The King-Crane Commission, as it became known, met delegations and invited written petitions from various religious and political groups in order to find out what their wishes were in this regard. A digital collection of the records of the King-Crane commission was recently compiled under the direction of Oberlin Archivist Ken Grossi, Visiting Assistant Professor of Politics, Maren Milligan, and Ted Waddelow ’12.

Group portrait of the King-Crane Commission and other persons unknown, in Damascus, 1919

* The Grinnell Beowulf is a translation and teaching edition of the Old English poem. Six students worked with a faculty member to translate Beowulf into readable and poetic modern English. Planned as what Grinnell calls a MAP (Mentored Advanced Project), Beowulf provided an opportunity for the students to produce an edition designed for an undergraduate audience, which includes 165 annotations as well as introductions to the poem and the translation process.

Some Digital Scholarship Research and Projects

Here are just a few sources on digital scholarship to give you a greater sense of its potential. There are many, many more.

Council on Library and Information Resources, Promoting Digital Scholarship, “Formulating Research Challenges in the Humanities, Social Sciences and Computation.”

Cheminformatics.

Edward L. Ayers, “Does Digital Scholarship Have a Future?” EDUCAUSE Review, Aug. 5, 2013.

Lisa Spiro, “Doing Things with Text,” 2013 and “Creating Timelines”.

Digital Scholarship at Oberlin and the Ohio 5

The field of digital scholarship is rapidly emerging as can be seen by its institutional emergence in many colleges and universities. At Oberlin, we are fortunate to share the expertise of Jacob Heil, the Mellon Digital Scholar shared by the Ohio 5 Colleges. You can read more about the digital collections  at Ohio 5 schools here, and the on-going digital scholarship projects here.

Digital Scholarship Workshop at Oberlin: Wednesday, Feb. 12 (4:30-6:30), CTIE

Jacob Heil will be join a workshop on digital scholarship co-sponsored by the library and CTIE on Wednesday, Feb. 12. The purpose of the workshop is to introduce those less familiar with digital scholarship to full potential, to help faculty and staff begin to formulate their own DS projects, and to acquaint those who have been formulating DS projects to the resources we have to advance those projects. The workshop will also showcase DS projects by Oberlin faculty.

Beginning…again. Start of the Semester “Expectations Reflection Paper”

Steve Volk, January 31, 2014

One of the things that I most enjoy about a life in the academy is the bi-annual prospect it provides to start anew. Whether we follow through on them or not, the resolutions we make at the start of each new semester offer an opportunity to reflect on what went well and what went pear-shaped in the last semester, as well as a chance to institute some changes to address the shortcomings.

There’s a boat-load of hopefulness built into this bi-annual reset button, and I was reminded of the importance of this once again when reading the obituary of Pete Seeger, a personal hero who visited Oberlin many times during his long career. “The key to the future of the world,” he said in 1994, “is finding the optimistic stories and letting them be known.”

Nora Borges, Norah Borges: Obra grafica, 1920-1930

So, how to begin…again? CTIE has posted “first-week-of-the-semester” advice at the start of most semesters, and you can find the entries on CTIE’s Blackboard page. Here’s one additional suggestion of something to do at the start of the semester: Hand out an “Expectations Reflection Paper.” Many of us do something like this as a way to help students articulate what they hope to get out of the course and as a way that we can learn something more about our students and what they think they have signed up for. (It’s always a sobering experience to realize that their expectations and your syllabus don’t particularly align.)

You can set aside some time in the class for students to respond to the handout, although you can also have them work on the assignment outside of class as long as they know they need to turn it in at the start of the next class. Some faculty require that students put their names on the assignment, others explicitly don’t, and a some make it optional. Generally speaking, if you want to use the expectations paper as a way to get to know your students, you’ll need to know who authored them.

What to ask? Here are some starting ideas:

  • Why are you interested in this subject; or what prompted you to take this class? (Don’t be afraid of saying that you need it to fulfill a requirement of some sort.)
  • Have you taken other courses in this area or have you had other experiences (in classes or outside of class) that you think are relevant?
  • What content knowledge do you hope to gain by taking this class?
  • What skills do you hope to gain by taking this class?
  • How do you learn the best? Formal lectures, class discussions, small group discussions, readings, assignments, practical engagement, group work, etc.?
  • Do you know of anything that might get in the way of your full participation in this class? (Is this a particularly busy semester for you? Health issues? Family issues? Part-time work? Lack of particular skills? Shyness? Please list anything that you feel comfortable listing, anything you think I can help with or should be paying attention to.)
  • Do you have any particular worries about this class that derive from your own concerns, the reputation of the class, what you have heard from other students?
  • Is your workload for this semester high, low, or normal?
  • What can you tell me that can help me remember you? (E.g., “I’m the one with pink hair who loves Bach and always sits in the back row.”)
  • In this class, do you expect to work more, less, or the same amount as in other courses?
  • Are you doing anything else this semester (or in general) that relates to or corresponds with the subject of this course (either in terms of what you are studying or things that you are doing outside of your classes)?
  • What one thing can I do to help your learning in this class?
  • What one thing can YOU do to help your learning in this class?

Other areas to include? Send them along and I’ll compile them.

Nora Borges, Norah Borges: Obra grafica, 1920-1930