Anyone who has worked at a large organization, especially if they have been in charge of a department or section, will have encountered the dreaded metrics. Someone in upper management decides that they need to measure precisely how effective each part of the organization is functioning and so they develop some sort of metric that is sent out which section heads are supposed to periodically fill in and return.
The problem is that unless you are dealing with highly tangible and easily measurable entities, like the number of widgets that are produced per day, metrics can turn out to be extremely frustrating to fill out and even counter-productive, as Jerry Z. Muller explains.
The key components of metric fixation are the belief that it is possible – and desirable – to replace professional judgment (acquired through personal experience and talent) with numerical indicators of comparative performance based upon standardised data (metrics); and that the best way to motivate people within these organisations is by attaching rewards and penalties to their measured performance.
The rewards can be monetary, in the form of pay for performance, say, or reputational, in the form of college rankings, hospital ratings, surgical report cards and so on. But the most dramatic negative effect of metric fixation is its propensity to incentivise gaming: that is, encouraging professionals to maximise the metrics in ways that are at odds with the larger purpose of the organisation. If the rate of major crimes in a district becomes the metric according to which police officers are promoted, then some officers will respond by simply not recording crimes or downgrading them from major offences to misdemeanours. Or take the case of surgeons. When the metrics of success and failure are made public – affecting their reputation and income – some surgeons will improve their metric scores by refusing to operate on patients with more complex problems, whose surgical outcomes are more likely to be negative. Who suffers? The patients who don’t get operated upon.
To the debit side of the ledger must also be added the transactional costs of metrics: the expenditure of employee time by those tasked with compiling and processing the metrics in the first place – not to mention the time required to actually read them. As the heterodox management consultants Yves Morieux and Peter Tollman note in Six Simple Rules (2014), employees end up working longer and harder at activities that add little to the real productiveness of their organisation, while sapping their enthusiasm. In an attempt to staunch the flow of faulty metrics through gaming, cheating and goal diversion, organisations often institute a cascade of rules, even as complying with them further slows down the institution’s functioning and diminishes its efficiency.
As head of a center in a large university, I would periodically get metrics to fill out about how effectively the center was functioning. But the metrics were generic ones meant for all the units, all of which have very diverse missions. My center’s mission was to help faculty improve their teaching but the boxes that I was supposed to fill out had little relevance to that goal. My suggestions that they needed to customize the metrics fell on deaf ears. That may be because it was too much work and the person issuing the metrics was being evaluated using criteria that did not require that level of detail, the kind of problem Muller points out. In my case, I decided that my time was better spent trying to improve teaching than agonizing over the metrics and filled in the boxes somewhat cavalierly. Ultimately, I felt that whether my center was considered effective or not would be determined by the faculty grapevine, what they said about how helpful we were to them. If the faculty were grumbling to the top management that my center was of no use, no glowing metric numbers could counter that.
The kind of gaming that Muller speaks about was evident in my seminar class as well. In such classes, participation is important. Since I discuss with the students what would be the best grading system, I asked them what we should do about assigning a participation grade. Should I create a metric that counted the number of times they spoke as well as the length and quality of their comments and keep track during class? The students overwhelmingly rejected that idea. They said that in the classes where that was the policy, they would speak just to meet the metric goals even if they had nothing they really wanted to say. They considered it a waste of time that actually diminished the classroom experience. They told me that they trusted my judgment about the level of their participation and to let them know if I felt that they needed to improve. That was what I did and there was no problem. It saved me from the tedious task of keeping track and the class could focus on having interesting discussions.