The Modern Metrics Fetish: WTF?

 The New York Times had an article recently about school testing and Common Core. Along the way they said that the challenge of comparing children’s progress, state to state, and state and country to country, is "one of the most stubborn problems in education." To which an alert reader/parent said basically: WTF?

As she described it: "I could easily list 500 priorities for my child’s education. But the question of how Massachusetts test scores compare with test scores in Minnesota, in Mississippi, anywhere, wouldn’t make the cut ... Somehow, though, this need to compare is so all-consuming that it has reshaped children’s entire school experiences."

I get what she is saying. When did metrics and measurement become so important that they're worth sacrificing all our other goals for? Why this fetish for metrics? When did the desire to measure become to intense that it gets in the way of actually moving forward with things?

I've got nothing against the gathering of information, but for a lot of things, measurement just doesn't work that well. Some things are hard to quantify, and even when you can quantify them, it's hard to know what data is relevant. Even when you know what to look for, the relevant data is hard to find, gather, and organize. Things keep getting in the way -- like how do you know whether some schools are becoming better by squeezing out the bad students? There are huge problems.

The crazy thing is that everyone knows all of this. It's no secret that some things are hard to quantify and that even when you can quantify you can't get the info that's relevant. It's no secret that what happens if you keep moving forward is that you're measuring the measurable things -- like money or test scores -- and then acting like those are the things you were interested in in the first place, which is ridiculous. It's no secret that people who do this end up by silencing their critics with "There is no alternative! You don't understand: the challenge of measurement is "one of the most stubborn problems in education!"

So if we know all of this why does it keep happening? Why do we keep creating metrics that measure the things we're not interested in and then using those metrics to make plans? Why is it so hard to have a reasoning process that involves people making informed judgments based on things they know and also consultation with the relevant parties?

My guesses, from most obvious to most obscure, with remarks:

1. Maybe the "data" approach just fits with the outcomes desired by the powers-that-be -- so the "data" is just window dressing on decisions made for other reasons. This is the classic "stupid or evil?" debate, and I don't have anything new to add.

2. Maybe the data give you the illusion of having an "answer," where the alternative involves vagueness, uncertainty, and judgment. As I've said before, some people seem to prefer having an answer they know is wrong to not having a clear answer -- which seems to me to make no sense.

3. Maybe underlying 1. and 2. is some misguided idea that judgment involves partiality while metrics and data are objective and neutral. News flash: as has been discussed again and again at least since Weber's work in the early 20th century, every way of using concepts in measurement requires making value-laden judgments about significance. "Metrics" often just makes the values harder to see, where "judgments" would put them right in plain view, where you could, you know, talk about and debate them.

4. Here I'm getting really speculative and into overgeneralizing cultural diagnosis territory, but I feel like somewhere along the way of the last hundred years or so, western culture got lost and started thinking that everything that isn't SCIENCE is somehow RELIGION. For example, I'm still irritated by how this law article refers to non-utilitarian ethical reasoning as "faith-based": as if utiltiarianism is somehow value-free and not just another set of values, and as if anything appealing to values is somehow off-limits for respectable inquiry.

This last is so bizarre I don't even know where to start. If you give up religion or faith, what you get is basically what you always have: people with various values, preferences, opinions, and hopes and dreams for the world. Often these willl be different and often they'll conflict. The great task of living together is figuring out what to do about that. It's a tough task, and it's not one that the little elves of "measurement" are somehow going to do for us.