The Pernicious Illusion Of User-Generated Neutrality

I feel like this there's this dream in late-capitalist societies that somehow you can write a neutral set of rules for human interaction -- that is, a way of setting up a set of rules or procedures that doesn't rely on contentious and messy stuff like what matters and why, or what is or isn't harmful, or what people do or don't have the right to do.

Of course that isn't true. There's no such thing as a neutral set of rules for human interaction. If you're going to talk about how people should treat one another, you're going to have to base that on all kinds of value-laden beliefs.

But there's this tenacious implicit idea or hope that somehow, if you just wrote the right rules, and got an effective enforcement mechanism, you'd be good to go. On the internet, this often takes the form of letting users determine everything. The dream is we can introduce a ratings system, step back, and let it all sort it self out.

I feel like it's getting more and more difficult to ignore the fact that this dream is an illusion. You can't make things work without some dialogue and agreement about what's important and what's sort of OK and what isn't OK.

It's long been known that the sharing economy has a race problem. But I was reminded of this issue when I read this recent article on "Broadly," about discrimination and the gig economy. It starts with the story of a trans woman, who wants to rent a room on AirBnB. When she informs her potential host she is trans, she is refused because the host doesn't want her son to feel uncomfortable.

The article also has this other story, where a young woman hires a cleaner through a gig economy site. When he shows up, he becomes aggressive and angry and tells her off for how messy her place is. She can't get him to leave, and eventually gives him an extra 40 dollars to go home.

Then there's also discussion of how, when women let out rooms to men, they end up feeling uncomfortable in their homes -- not threatened, just uneasy, because of the way the men take up a lot of space, put their stuff everywhere, and help themselves to the music collection.

The focus of the article is on women, people of color, and others being "safe" in the gig economy, but it also touches briefly on the deeper problem: that having an app "ratings" system -- effectively punishing bad guys -- really just doesn't cut it. Proper resolution of all of these cases requires not "ratings" from customers but rather informed judgment about what is and isn't appropriate in the way we treat one another.

It's complicated. People get to feel comfortable and not threatened. But sometimes -- as in the trans case -- if a person feels unsafe or uncomfortable it's their problem. Other times -- as in the cleaner case -- if a person feels unsafe or uncomfortable it's the other person's problem. Still others -- as in the case of guys "taking up space" -- I'm really not sure whose problem it is.

To say that everyone can rent to whoever they want and that people can go ahead and "rate" one another really doesn't address the relevant difficulties. You actually have to make judgments about what's fair, not fair, OK, not OK. And then you have to have some system for putting those judgments into practice.

It's the same thing for social media. You can't distinguish protected speech from abusive speech without making value judgments about what kinds of things are OK and not OK. For example, did you read about how Facebook banned a plus-sized model who was advertising a thing about positive and healthy body-image? 

Though the reversed the decision later, Facebook banned it for showing "body parts in an undesirable manner."

Explaining their decision, Facebook wrote: "Ads may not depict a state of health or body weight as being perfect or extremely undesirable. Ads like these are not allowed since they make viewers feel bad about themselves. Instead, we recommend using an image of a relevant activity, such as running or riding a bike."

Leaving aside all the other baffling questions this passage raises -- like, you're going to ban all ads that make people feel bad about themselves? WTF? -- obviously it's a value judgment to say that making people feel "bad" is worth banning and it's a value judgment to say when making people feel bad is OK and when it isn't. Anti-smoking campaigns make people feel bad too. So what?

In reversing the decision, Facebook said that the policy was meant to guard against the promotion of anorexia and eating disorders. A worthy goal, but again, not a factual one, not a value-neutral one, and not an obvious one to put into practice -- as the kerfuffle itself shows.

At the end of the Broadly article, an expert on the gig economy is quoted as saying,

"It's not really fashionable to be in favour of bureaucracy and rules, but equal pay for equal work, minimum wage laws, employment standards that limit employers' right to fire at will, and anti-discrimination laws were the results of years of struggle by feminists, unionists, and anti-racism groups," he says. "I don't think they should be thrown away just because a new app has a rating system."

I think this is spot-on. I've never understood the depth of antagonism to bureaucracy. If you're falsely accused, it's bureaucracy that's going to save you. If you want safe drinking water, it's bureaucracy that's there for you. And if you want fairness -- in education, in employment, or even just in your gig economy -- it's bureaucracy that's going to do that for you.

But it's not because they're value-neutral that bureaucracies do this. It's because, at least when they're working well, they encode values that we care about, and they put into place systems for making things happen. Systems that epitomize what tech companies seem to be trying to get away from.

An user-generated ratings system does neither of these things. It just allows everyone to put into concrete practice all the crappy, racist, sexist, transphobic, hate and phobia that they're carrying around. It's not "user-generated neutrality." It's like an amplifier for all of our worst qualities.