Geeks With Blogs
Ulterior Motive Lounge UML Comics and more from Martin L. Shoemaker (The UML Guy),
Offering UML Instruction and Consulting for your projects and teams.

On a mailing list where I hang out, a participant recently said (paraphrased): “He believes that popularity proves quality. I believe that there is almost no correlation between quality and popularity.”

We hear this sort of thing all the time. There’s an implication among self-appointed elites that “the masses” — i.e., everyone who’s not them — just can’t recognize quality. It’s assumed that “popular” is proof that something is bad. You see this attitude in film snobs who insist that an Oscar nomination for The Return of the King is some sort of travesty, because the film is a popular fantasy and not some art house flick or some historical epic. To be fair: you see it in Lord of the Rings fans who for years have been telling others who didn’t like the books that they just didn’t appreciate great literature. And before the films, you could see it among the literati who snubbed Lord of the Rings because it’s a popular fantasy rather than a dreary, post-modern, self-referential, obscurantist yawn. You see it in opera buffs who assume the rest of us are subintelligent because we don’t share their passion for opera. You see it in young rebels who look down on the lives of the conformist “sheeple” and who demonstrate that they are individuals and not “sheeple” — by all dressing and talking and acting and piercing alike. And you even see it in gourmets who extol the virtues of French food over more pedestrian fare like food from McDonald’s.

But the truth is: they’re wrong, every single one of them. They proceed from two clearly false assumptions: that there is one clear, objective, inarguable standard of quality; and that of all human beings, they somehow have been born with/been granted/achieved the unique ability to pronounce what the standard is.

But the fact is just the opposite. If I can avoid butchering the Latin too poorly, de gustibus non disputandum: with taste there can be no dispute. Or in the modern vernacular: there’s no accounting for taste. When someone tries to tell you that his tastes are objectively correct, he’s demonstrating how self-centered he is or how shallow his thinking is.

Does that mean there are no things that are objectively better than some other things? Can’t we all agree that Shakespeare is better than “The Simpsons”? Nope: I could gather up quite a debate on both sides of that issue; and the pro-Simpsons side would be every bit as educated and erudite as the pro-Shakespeare side.

Can’t we all agree that French food is better than McDonald’s? No, for multiple reasons: many people dislike new tastes, and prefer comfort and familiarity; not everyone likes the spices in French food; and if you grew up with French food every day, you might see it as “normal” and McDonald’s as a new experience, where novelty makes it attractive.

And so on, and so on, and so on. If you take any “objective” measure of general quality (as opposed to quality for a particular purpose, which may be assessed much more precisely) and examine it all the way down to its roots, you find personal tastes, past experiences, biases, and other responses that aren’t objective at all. There’s no objective measure of quality.

Except one. See, like many things that are immeasurable in the small, quality is measurable in the large, through statistics. No one person can absolutely proclaim that a certain thing is a quality product; but we can measure with reasonable precision how many people accept and endorse the quality of a product, by virtue of their purchases. In other words, the list writer I paraphrased has it exactly wrong: the closest thing we have to an objective measure of quality is popularity. If a significant number of people enjoy a product, then the odds that you will like it are higher. We’re all individuals, not ruled by statistics; but statistics are a useful piece of information to help you find products to try. Quantity purchased is a valid measure of quality. The market identifies products that the largest number of people accept as quality products.

And before anyone chimes in about betamax, QWERTY keyboards, CDs vs. albums, Microsoft, or any other oft-cited “evidence” that the market can produce the “wrong” answer: go reread my post, because you still missed the point. Don’t force me to go haul out the evidence that shows the conventional wisdom is wrong in every one of these cases, or I’ll produce so much it crashes the server.

Posted on Saturday, November 15, 2008 6:20 PM Code is not Enough | Back to top

Comments on this post: Quantity IS Quality

# re: Quantity IS Quality
Requesting Gravatar...
Wow, very interesting. Have you read "Zen and the Art of Motorcycle Maintenance?" It goes on a very long, thought-provoking tour of quality especially as it relates to the subjectivity or objectivity of a quality statement. (i.e. do things have quality, or do people assign quality to things? or is quality a third dimension altogether, acting as a lens between people and things?)

Back to reality though, my counter-argument to popularity as a measure of quality is in the definition of popularity, and the many other factors involved in popular consensus besides quality. You might be thinking of a certain specific application where quality is the only factor, but I'll use your own examples:

Assuming McDonald's is the highest-popularity food chain, what would a worldwide survey of "highest quality food" return? Probably not fast food. Or, assume that the Ford Focus is the highest-selling car of 2008-- does that mean it's the highest-quality car of 2008? Obviously not, it might just be the cheapest car available or maybe their marketing campaign was really good. (Are we counting price and marketing in the evaluation of an item's quality?)

To accurately use statistics, a tight definition of terms and expert analysis of the results is needed. Also, it's problematic to use economics to make a claim.
Left by Will on Nov 15, 2008 11:19 PM

# re: Quantity IS Quality
Requesting Gravatar...
To follow up (I don't want to seem like flame bait) if you define "popularity" as the "popular consensus of quality" then yes I would agree that it would be the closest thing we have to an objective decision on quality. But if "popularity" is "widespread usage or high demand" then I would agree with your opponent that the two are largely unrelated.

The varying definitions of the word might be of use to the debate:
Left by Will on Nov 15, 2008 11:26 PM

# re: Quantity IS Quality
Requesting Gravatar...

You raise good points, but perhaps my point was muddied. My real concern is people -- especially developers -- deciding that their measure of quality is THE measure of quality, then blaming "the masses" when the market disagrees.

Quality is an n-dimensional space, where n is roughly proportional to the number of people assessing the quality. Every single person has a personal set of values that define quality; and if their scales don't align well with yours, that's an objective fact that can't be argued away.

Railing that "the masses have no taste" may make someone feel superior, but it's a lousy way to develop and market code (unless your target audience is you). It's far smarter to either adjust to their measure or at least recognize their measure if you want to change it.

I do have to add that a worldwide survey of "highest quality food" would likely be meaningless unless carefully designed. That's a loaded question, the sort people are prone to answering the way they think they should, rather than truthfully. Assessing what they really value is hard; assessing their purchasing preferences is (relatively) easier. If I want to write successful software, I need to write what they'll buy, not what I think they should buy. Sometimes this is a hard lesson for creators. Heck, sometimes for ME: the market wants what it wants; I can give it choices, but I can't make it want those choices.

And thanks for the very first comments on my new blog!
Left by Martin L. Shoemaker on Nov 15, 2008 11:46 PM

# re: Quantity IS Quality
Requesting Gravatar...
I agree with you that the idea of "quality" can be subjective, and that different people define different standards.

One thing I would suggest that we need to identify are the base standards that everyone can agree on. With the restaurant metaphor, regardless of whether its McDonalds or Wolfgang Puck, I expect:
- Safe food
- A clean restaurant
- Good service

Software should be the same way: we need to have some base standards that we can then move up from to hash out our discussions with.

Of course, I suspect any discussion over what those standards should be will cause the same sort of disagreement we've already seen. ;)

Left by D'Arcy from Winnipeg on Nov 16, 2008 2:27 AM

# re: Quantity IS Quality
Requesting Gravatar...
Actually, D'Arcy, I think you're absolutely right. I think that we need to define our quality standards up front, in an open discussion with the team. I think McConnell discusses that in Code Complete, might've been Rapid Development. If we hammer out the standards up front -- if we define "what is quality" and "what is exceptional quality" -- then good devs will code to those standards. If we don't define them, each dev "knows" the "right" answers -- his or her own standards -- and will code to those. Then disagreement WILL be there. Better that it be put on the table up front than revealed in late deliveries.

And even if the team reaches a consensus, that doesn't make it the market's consensus (though I hope the team's consensus is informed by market input). It's a natural human reaction -- especially when you've poured sweat and time and self into a project -- to ask, "Why can't they see how good this is?" But success lies in asking, "How did I miss their target? Now that I know what their target is, can I redirect to hit it?"
Left by Martin L. Shoemaker on Nov 16, 2008 1:26 PM

Your comment:
 (will show your gravatar)

Copyright © Martin L. Shoemaker | Powered by: