*Side note: My blog had been rather neglected for the past 2 years. I always have good intentions, but they rarely translate to finding the time to write here. This summer, for whatever reason, I found the motivation to regularly open up my file of “future blog posts” and sit down and write them. In almost exactly three months (5/25 to today, 8/26), I wrote 42 posts, 27 posted for three months, twice a week, and another 15 scheduled to post through 10/16 (this one I’m writing right now for 10/18 makes 43). That’s a lot of writing time, and one might argue that I could have written one or more manuscripts in the time I spent writing blog posts. But it’s debatable whether I would have used the time as effectively if it was manuscript-focused time. And, at some point in your career, you can take a bit more freedom to make choices, such as being generative in different areas than the traditional ones. I recognize, though, that I cannot write nearly as frequently during the academic year. So, perhaps my posts will continue beyond mid-October, but if there’s another hiatus, thanks for coming along for the ride and I hope you found something useful.
And now, student evaluations of teaching. I will refer to them as SETs which is what they are called at UConn.
The first time I received student evaluations of my teaching I was a graduate student TA for a section of research methods. I called a friend and told her I was sad because I had a really negative review. She said, really, I had great reviews! And after we talked for a while, we figured out that both of us had mostly positive feedback with 1-2 negative people in the pile, it was just that I perseverated on the negative one and couldn’t remember the positive ones, and she ignored the negative and focused on the positive.
I still find it hard not to perseverate on negative feedback in my SETs. I think it’s because the most negative ones are generally the most strongly emotionally valenced, and so they are the most memorable. At least to me. I really want to take student evaluations into account when I revise my course each year. But I find that if I’m not careful, those highly emotionally valenced ones may sway my view. I have even at times changed a course in response to such a review, having then the next year more students complain about the new format.
SETs don’t have the same meaning for me now that I am post-tenure, post-full professor. Previously it at times felt like SETs were linked to job security. Now, to be honest, as long as I don’t get scores low enough to raise eyebrows in the Provost’s Office, no one is going to care much about my SETs.
All that said, I do want to learn what students did and didn’t like about my courses, and figure out how to improve them. I had a colleague who prided himself on never looking at his teaching evaluations. I’ve had other colleagues who, when I was Undergrad Director and met with them to discuss lower ratings, would blame it all on the students, such as, “this was such a bad bunch this semester.” I can’t quite understand that. I don’t think we should bend over backwards to please students all the time, but I do think we need to consider students’ perspective on the course.
When I was Undergrad Program Director, I often had to help other faculty interpret their SETSs and figure out how to respond to them in course design. During this process, I developed a way to summarize the SETs for others, and I now use that technique with my own SETs. If you teach a small graduate course, you may find this technique less useful. But if you have a large course like mine, then summarizing them may help you find patterns.
Essentially what I am doing is coding the responses. I pull up the open-ended responses, open an Excel file, and read through them one at a time. For each one, I try to code or categorize any unique point that person says. I keep a running tally of each point, so that the second time someone says “she was so enthusiastic!” I don’t make a new line, I just add a count to the line “enthusiastic/energetic/engaging.” I do separate summaries for “Most positive aspects” and “What can instructor do to change.” Once I’m done coding, I sort them by # of responses in descending order, and I can much more easily see the patterns and get a sense of the overall student perspective, rather than only remembering the most strongly worded ones.
This year, when I first received my SETs for my 290 student class at the end of the semester, I skimmed them but didn’t take the time to read them. The sense I came away with from them was that they overall liked me and thought I was enthusiastic, but that students also were unhappy that I made them do group assignments because the class was a mix of freshmen – seniors; unhappy that instructor seemed unsure of herself; and unhappy that I relied on my notes too much. But when I tabulated, those last three points were literally each from one person – in a class of 290. That’s what I remembered the most, though. So instead, when I tabulated them, I could much more easily see the trends. Here are all of the answers to the question about “most positive aspects” that 5 or more students made:
And now, student evaluations of teaching. I will refer to them as SETs which is what they are called at UConn.
The first time I received student evaluations of my teaching I was a graduate student TA for a section of research methods. I called a friend and told her I was sad because I had a really negative review. She said, really, I had great reviews! And after we talked for a while, we figured out that both of us had mostly positive feedback with 1-2 negative people in the pile, it was just that I perseverated on the negative one and couldn’t remember the positive ones, and she ignored the negative and focused on the positive.
I still find it hard not to perseverate on negative feedback in my SETs. I think it’s because the most negative ones are generally the most strongly emotionally valenced, and so they are the most memorable. At least to me. I really want to take student evaluations into account when I revise my course each year. But I find that if I’m not careful, those highly emotionally valenced ones may sway my view. I have even at times changed a course in response to such a review, having then the next year more students complain about the new format.
SETs don’t have the same meaning for me now that I am post-tenure, post-full professor. Previously it at times felt like SETs were linked to job security. Now, to be honest, as long as I don’t get scores low enough to raise eyebrows in the Provost’s Office, no one is going to care much about my SETs.
All that said, I do want to learn what students did and didn’t like about my courses, and figure out how to improve them. I had a colleague who prided himself on never looking at his teaching evaluations. I’ve had other colleagues who, when I was Undergrad Director and met with them to discuss lower ratings, would blame it all on the students, such as, “this was such a bad bunch this semester.” I can’t quite understand that. I don’t think we should bend over backwards to please students all the time, but I do think we need to consider students’ perspective on the course.
When I was Undergrad Program Director, I often had to help other faculty interpret their SETSs and figure out how to respond to them in course design. During this process, I developed a way to summarize the SETs for others, and I now use that technique with my own SETs. If you teach a small graduate course, you may find this technique less useful. But if you have a large course like mine, then summarizing them may help you find patterns.
Essentially what I am doing is coding the responses. I pull up the open-ended responses, open an Excel file, and read through them one at a time. For each one, I try to code or categorize any unique point that person says. I keep a running tally of each point, so that the second time someone says “she was so enthusiastic!” I don’t make a new line, I just add a count to the line “enthusiastic/energetic/engaging.” I do separate summaries for “Most positive aspects” and “What can instructor do to change.” Once I’m done coding, I sort them by # of responses in descending order, and I can much more easily see the patterns and get a sense of the overall student perspective, rather than only remembering the most strongly worded ones.
This year, when I first received my SETs for my 290 student class at the end of the semester, I skimmed them but didn’t take the time to read them. The sense I came away with from them was that they overall liked me and thought I was enthusiastic, but that students also were unhappy that I made them do group assignments because the class was a mix of freshmen – seniors; unhappy that instructor seemed unsure of herself; and unhappy that I relied on my notes too much. But when I tabulated, those last three points were literally each from one person – in a class of 290. That’s what I remembered the most, though. So instead, when I tabulated them, I could much more easily see the trends. Here are all of the answers to the question about “most positive aspects” that 5 or more students made:
`So I get a clear sense that students like that I gave lots of personal examples, and real world examples, and students found me engaging/enthusiastic/passionate. All of these positives even won out over showing videos, so I take that as a strong win.
Toward the bottom I can see the things that only one student mentioned as a positive:
Toward the bottom I can see the things that only one student mentioned as a positive:
These are things that might make me feel good in some cases, but I don’t have to spend a lot of time thinking about them, either. Now, here are all of the things that 5 or more students said I could do to improve the course:
I’m feeling quite good that the most common response was for students to say “nothing” or “N/A” or “I can’t think of anything.” After that, I’ve highlighted for myself the next most common things that students said. Lots of students would like me to have more detail in my notes. That’s something that I’m not willing to change, because there is research on learning and memory to support my decision to have PP slides without a lot of detail, forcing students to pay attention and take better notes. In fact, 9 students said they liked the brevity in my slides. My best guess is that most of the students who asked for more detail in my slides were among the students who attended class infrequently, and they want me to post notes on Blackboard that have enough information that they didn’t miss anything. I’m not going to do that, but I can spend more time on the first day of class this semester explaining why my notes are the way they are.
Some students thought my (open-notes, open-book) exams were tricky. And some students thought I should be more clear about what exams would be like and what would be on exams. So I can probably spend more time in class pointing out some of the concepts that students find tricky, and talking about how to use the weekly review questions I post. A lot of the things I got out of this section of my SET’s are about ways I can help the students understand what is happening in the course, rather than changing my policies per se. But there definitely have been times that I have read something in my SET’s that has made me make some more radical changes in my course.
This technique has really helped me take the emotion out of reading my SETs, so that I can concentrate on what students do and don’t appreciate about the course, and respond appropriately. How do you deal with student evaluations of teaching?
“How to gather useful information from student evaluations of teaching first appeared on Eva Lefkowitz’s blog on October 18, 2018.”
Some students thought my (open-notes, open-book) exams were tricky. And some students thought I should be more clear about what exams would be like and what would be on exams. So I can probably spend more time in class pointing out some of the concepts that students find tricky, and talking about how to use the weekly review questions I post. A lot of the things I got out of this section of my SET’s are about ways I can help the students understand what is happening in the course, rather than changing my policies per se. But there definitely have been times that I have read something in my SET’s that has made me make some more radical changes in my course.
This technique has really helped me take the emotion out of reading my SETs, so that I can concentrate on what students do and don’t appreciate about the course, and respond appropriately. How do you deal with student evaluations of teaching?
“How to gather useful information from student evaluations of teaching first appeared on Eva Lefkowitz’s blog on October 18, 2018.”
RSS Feed
