i teach

A few weeks after grades are submitted, evaluation reports become accessible to Washington University faculty. Reviewing the reports and interpreting the results can help you understand students’ responses to your course, gain insights into student learning, and identify aspects of your teaching that you can improve.

To generate a course-evaluation report for your course, log-in at http://evals.wustl.edu and click on “Evaluation Reports.” First select the semester of the course, then check the boxes to indicate which questions you would like to be included in the report.

As you review each report, look for the following information: the numerical data, information about students, and the written comments.

1.  The Numerical Data

• Student Response Rate
  A higher response rate ensures that the evaluations system has gathered the widest possible range of responses to the course and your teaching. For tips on how to increase the response rate the next time you teach, click here.

• Scores
  Most of the questions on the evaluations ask students to answer with a numerical rating on a range of 1 to 7. In each case, the meaning of each end of the scale is provided (e.g. 1 = poor, 7 = excellent).  Both an average score and the distribution is shown for each item.

First review the scores on the first two questions under the first section, Overall Evaluation: “Give an overall rating for the course” and “Give an overall rating of the instructor’s teaching.” Both of these items have consistently been shown to have a positive correlation with student learning, measured by exam scores and course grades. See Research on Course Evaluations.

• For each item, review the distribution of scores, as well as the average score, marked in bold. Below your average score for the item, the report also includes the System’s Average (S) and Department’s Average (D) for all instructors assigned the same template (e.g. Lecture Class, Seminar Class, Laboratory Class, etc.). For more information about templates, see Customizing Online Evaluations.
• Looking at the System’s Average and Department’s Average scores for the template can give you a rough idea of how your own average scores compare with those of other instructors. However, keep in mind that there are many variables that can affect average scores and that can therefore make such comparisons difficult. For example, two different courses within a single department might be assigned the Lecture Class template, but one might be a small, upper-division, elective course, while the other is a large, introductory required course. The latter types of courses tend to be rated lower than the former types of courses. Comparing courses from different departments (i.e. comparing your average score on an item with the system-average score on the template) will be even more difficult, given that evaluation ratings in some disciplines (e.g. humanities) tend to be higher than in others (e.g. the physical sciences). For more information about how these variables, and others, can affect student ratings of instruction, see Research on Course Evaluations.

 2.  Information about Students

In the section titled “Other Considerations –Not Instructor Specific,” you will see responses to questions about each student’s expected grade, class level, and gender, as well as questions on whether the student was taking the course to fulfill a requirement or because it was part of a cluster. This information is useful in at least two ways:

• It can help you to determine whether the course attracted the students you expected and planned for, and whether you might make future adjustments to ensure that the course is designed for the students who typically enroll.

• It can help you put average scores in context. For example, as discussed in the article Research on Course Evaluations, moderate positive correlations have been found between student ratings and course grades.  

3.       Written Comments

• When you review the open-ended questions at the end of the evaluation report, keep in mind that the most useful comments are those that are specific and relevant. Sometimes the most glowing comments (“I loved the professor” or “the professor was awesome”) make you feel good, but do not contain any information that you can use to identify your strengths as an instructor. By the same token, you will sometimes receive negative comments that are vague or seem personal (“The homework assignments were stupid” or “This class was a waste of time”). The anonymity of the online evaluations may encourage some students to write harsh comments or to not take the process seriously. For tips on encouraging your students to write specific comments, see Increasing the Number and Usefulness of Student Responses to Online Evaluations.

  Rather than dwell on comments that are either highly positive or highly negative, look for the comments that help you identify specific aspects of the course that worked well and specific areas you could improve upon. In addition, try to find the rationale behind the students’ responses. For example, if they did not like the papers, why not? Did they find the topics too difficult to tackle? Or did they need additional guidance on what was expected on each paper? Or, if they liked your teaching, did they appreciate your engaging lecture style, the time you took to give them thoughtful feedback on their work, or the in-class activities you designed?

• If you are teaching a large class, consider grouping comments by topic to help you understand their significance. For example, group all comments about course organization together so that you can understand the range of responses to this specific aspect of the course and your teaching (Lewis 2001).
• Use the numerical scores to help you further contextualize the written comments. For example, you may have a few students who use the open-ended questions to point out that the course was not well organized, while the numerical scores on organization of the course and your instruction show you that the majority of students found the course well organized. To understand variables which can affect the numerical ratings, read Research on Course Evaluations.
• You can also use the number assigned to each respondent as a means of connecting all comments written by a specific student. (To find the number, look to the left of each response.) This strategy will help you better understand a specific student’s responses. For example, one student who identifies the discussions as the aspect of the course that she or he liked the least might have indicated that she or he loved the lectures, “because I learn better when I can take notes on a lecture and think about the material later; I am not comfortable answering questions in class.”
• Keep an open mind as you read the comments. Be prepared for the possibility that your students may perceive your teaching differently than you do, and consider why that may be the case. For example, you might be surprised when students write something like “I didn’t understand what was expected on the papers,” when you handed out what you thought was extensive written guidance on the assignment. However, it may be that there are other steps you can take to clarify expectations, such as asking students to submit questions about a major assignment, discussing the assignment in detail, distributing a grading rubric, or sharing samples of past student work. Writing clear prompts for student writing can be difficult. For assistance with your prompts, contact Gina Frey, Director of the Teaching Center, or register for The Teaching Center’s workshops on Teaching with Writing.
• Think about what the written comments might reveal about student learning. Hodges and Stanton (2007) point out that comments that seem off-base to instructors might actually suggest that students are struggling to learn the more complex ways of thinking required in university-level courses. For example, in a quantitative course, comments such as “problems on the exam were nothing like those in class/problem sets” may reflect a common approach of novice-learners to problem solving: to try to memorize types of problems and their solutions (Hodges and Stanton, 2007; p. 281). It may be that the student who wrote the comment had not yet developed a conceptual approach to problem solving that would have allowed that student to see conceptual connections among problems that appear may appear, in the details, to be dissimilar. You can help your students develop conceptual-thinking skills by integrating into your class opportunities for students to reflect on and to explain the process and rationale of the steps taken in solving problems.  

  Another useful example discussed by Hodges and Stanton is a common type of comment in courses that utilize active-learning activities. Sometimes students react negatively to active learning and write comments on course evaluations such as “I didn’t come to college to teach myself.” This type of comment may reveal that the extent to which the active-learning activities fell outside of the student’s expectations and experiences (Hodges and Stanton, 2007; p. 284). When introducing such activities into your course, it is essential to shape students’ expectations at the start of the semester, explaining not only what each student will be expected to do, but also the rationale and purpose behind the activities. It is also essential to help students build the skills that they need to engage in these activities, such as collaboration, communication, and organizational skills.

To develop additional insights, make an appointment for a consultation at The Teaching Center. All instructors can benefit from discussing course-evaluation results and thinking about how to use the results to improve teaching and make adjustments in a course.

Faculty who would like to discuss their own course evaluations may contact Gina Frey to set up an appointment. Graduate students and postdoctoral instructors may contact Beth Fisher.

References

Hodges, L. and K. Stanton (2007). Translating comments on student evaluations into the language of learning. Innovative Higher Education 31. 279-286.

Lewis, K. G. (2001). Making sense of student written comments. New Directions for Teaching and Learning 97 (Fall): 25-32.