There were a number of significant problems with the administration of student evaluations during the Spring 2021 semester.  Issues regarding sections in which student evaluations were supposed to be given but were not, as well as sections in which tenured faculty did not request that student evaluations be given but were given, have been addressed (see the bottom section of the post on the MSCA website).

Regarding sections in which student evaluations were appropriately administered, there was a prevalent issue of low student response rates.  These, of course, were outside the control of faculty.

As a result, we would like to point language in the collective bargaining agreement that addresses low response rates, and which you may want to consider referencing in your narrative.

“Student evaluations shall be used giving due consideration to the number of courses and sections that were taught during the review period and for which student evaluations are available, as well as the number of students by which such evaluations have been made.” (2017-2020 CBA, page 112)

Moreover, ETS reported in May 2014 “Studies consistently report that online [student evaluation of their teachers (SET)] yield significantly lower response rates than paper-based SETs.”

The report also notes that there is “significant differences between in-class versus online evaluations, with average student ratings on online surveys significantly lower than average student ratings provided in class.”

It also states “if the interrater reliability of the online SIR II is lower than that of the paper-based SIR II, then use of the online SIR II would be more difficult to justify.”

It is clear, that low response rates – which are outside the control of faculty – are a significant problem regarding the reliability of the results.

Educational Testing Services (ETS) discontinued processing its Student Instructional Report II (SIR II) prior to the 2019-2020 academic year.  Since then, the state universities have employed Scantron to produce and compile results of the questions that were on the SIR II forms.

As a result, since the 2019-2020 academic year, any comparative norms reported have been meaningless.  Any comparative norm that may have been reported does not take into account class size, discipline, course level, or a number of other criteria that ETS used to use.

In fact, ETS would not produce a class report if fewer than five students responded.  “The number of students responding can affect the results when the class is very small (fewer than 10) […] For this reason, a Class Report will not be produced when fewer than five students respond.” (ETS SIR II report, page 4)

In addition, the overuse of student evaluations is also something that ETS has warned against (see ETS’s Guideline for the Use of Results of the Student Instructional Report II (SIR II)).

Given all these issues, we encourage faculty to address any such concerns they have about the SIRII results in their narrative.

I hope this information is helpful.

CJ O’Donnell
MSCA President