What’s the difference between a 10-point and a six-point performance rating scale, besides four points?
A lot, if you’re a woman being rated. A new study looking at student ratings of university teaching performance shows that a substantial gender gap under a 10-point system disappears when the system used only has six points.
“We were somewhat surprised at the magnitude of the effect,” says András Tilcsik, an associate professor of strategic management, who holds the Canada Research Chair in Strategy, Organizations, and Society, at the University of Toronto’s Rotman School of Management. He co-authored the study with associate professor Lauren A. Rivera at Northwestern University.
The researchers looked at what happened when a professional school at a large North American university switched from a 10-point to a six-point instructor rating scale. The school had a minority of female instructors, but their numbers dipped to less than one-fifth for half of the school’s major subject areas.
Under the old system, male instructors garnered perfect 10 ratings 31.4 percent of the time in subjects with the highest numbers of male teachers; female instructors got the top score in only 19.5 percent of cases. It was a different story though under the six-point scale. There, both sexes garnered top ranking almost equally; 41.2 percent for men and 41.7 percent for women.
The findings were replicated in a separate experiment where students were asked to read a transcript of a faculty lecture and then rate the lecture, with random variations made to the lecturer’s gender and the maximum allowable points.
Altering the ratings scale does not eliminate bias, the researchers caution. But the right one can limit the expression of pre-existing biases, such as stereotypes that associate brilliant performance with men more than women.
“Gender biases are particularly strong when people are making very fine distinctions between brilliant versus very good performance,” says Prof. Tilcsik. “With a six-point scale, we are basically taking away the ability to make those fine distinctions.”
Making performance rating systems as fair as possible is important, says the study, because professional advancement, from salary to seniority, can be significantly influenced by them. Any workplace concerned with eliminating gender inequalities should therefore pay attention to the design of the tools it uses to evaluate performance and merit, the study suggests.
