Wine Ratings and the Nature of 1+1=2
I rarely take room on this blog to correct wine writers or to respond to them. But let's file this under, "I can't resist".
In a recent article on the Huffington Post, importer, distributor and writer David Duman attempted to refute those who take time to defend the 100 point rating system. Among the things he focused upon was what he called the "It's Not Objective Fallacy".
Duman claims that those defenders of the 100 point system who assert that a numerical rating that comes along with a review of the wine is a subjective claim are wrong and they in fact ARE implying objectivity with their number score. Duman writes:
"While perhaps theoretically true [that the number is a subjective evaluation of the wine], in practice there is of course an implied objectivity in any quantitative rating system: that, within that system, a wine receiving more points (or stars, or hats or whatever) is of course better than a wine with fewer."
To this I need to remind Mr. Duman that anyone who doesn't understand the difference between "1+1=2" and "I Believe" has bigger problems than tryng to figure out the 100 Point wine rating system.
When a writer offers a written review of the wine and attaches a "91" point rating to that review it can ONLY be a subjective measure for the simple and obvious reason that there is no agreed upon and unalterable criteria for evaluating pleasure, let alone wine. The 91 points (along with the review) is one person's (or a panel's) view of the wine. The fact that I might have a different view of that wine that does nothing to alter the substance or nature of the wine in question proves sine dubio that the score is a subjective evaluation of the wine.
There is nothing objective and no implication of objectivity when saying, "Wine X is 5 points better than Wine Y". All you have is a personal evaluation of the wine. Whether one uses numbers, stars, thumbs or even just words to describe their evaluation, that evaluation is subjective, not objective, and any wine writer who has also used equal, plus and minus signs in their lives understands this because they understand the difference between "objective" and "subjective".
The fact is, a review of a wine that states, "This Cabernet is the best I have tasted this year" is little different than placing a 95 point score next to a wine. It's merely that a different set of vocabulary has been used to describe the writer's experience with the wine.
There may in fact be solid arguments to be made against using the 100 point rating system. However, the idea that this system implies objectivity is not one of them.
Finally, Duman asserts that the application of the 100 point rating by consumers also indicates the scores' implied objectivity. We know this, he says, because wines with higher scores are in greater demand and cost more. He explains:
"The critic can protest all he wants that that [implied objectivity] might not be his intention, but as any student of Post-Modernism would tell you, the writer's intention has nothing to do with it and we can only assess value in its application."
Putting Derrida and Foucault aside, I think it's fair to say that the consumer, in creating more demand for a high scoring wine, is simply taking the subjective advice of a critic. They believe the critic when they assert through a score and review that Wine X is better than Wine Y and act on that belief. It is a reaction no different than a moviegoer running out to see a film that they read Roger Ebert loved, despite the fact that Leonard Maltin said the movie stunk. I can tell you from experience that a Roger Ebert-endorsed film does not guarantee I will like it, despite the fact that I ran out and saw it. I can furthermore tell you that despite having bought wines because they got high scores, it turned out I did not like them. I did not question my personal evaluation of the wine any more than I questioned my personal evaluation of the movie just because a critic I admire liked the film. Put another way, I defied any pretense to objectivity in the critics review.
Demand for high scoring wines do not prove consumers view those scores as objective. It proves they view them as reliable evaluations. There is a difference so large as to require those who think otherwise to go back and study the meaning of "1 + 1 = 2".
Mr. Duman and those who also believe there is a problem of implied objectivity with the 100 point system need to go back to the drawing board and rework their formula.
I think your argument works here for a five point scale or a ten point scale. The wine can be rated a ten subjectively just as a beautiful woman. The problem comes when you expand that to 100 points implying that wine can be judged and rated with that precision. The mere fact that a critic distinguishes and scores one wine an 85 and another an 86 and a different one a 92 and another a 93 certainly implies a precision that anyone knows anything about wine tasting, knows that it doesn’t exist.
When you imply precision, you imply accuracy and along with that comes an assumption that there is something of substance behind the score.
Congratulations on getting references to Derrida and Foucault into a discussion of wine! You are of course right that wine scores do not imply objectivity. But they do imply precision, clarity, and the assumption that qualities in wine can be meaningfully quantified. Controversies about wine scores are about what they mean, not about whether they are independent of individual preferences.
Morton:
How is it impossible for such a precision exist?
Edible:
The score does not imply the assumption that qualities can be meaningfully quantified. It implies that qualities and experience together can be ranked against other qualities and experience based on personal criteria.
Et tu Thomas? Et tu?!
Si, et moi.
Tom,
You wrote: “The score does not imply the assumption that qualities can be meaningfully quantified. It implies that qualities and experience together can be ranked against other qualities and experience based on personal criteria”.
What I had in mind is this. The “qualities and experiences” of one wine can be ranked against others. But the most plausible kind of ranking would be an ordinal ranking-one wine is better than another but worse than a third, etc., without stating how much better or how much worse.
But the 100-point ranking system implies a cardinal ranking. That is to say, the difference between a wine scored 94 and a wine scored 95 is the same as the difference between a wine scored 90 and one scored 89, namely 1 point. This suggests that there is some quantifiable quality that is being measured. Plausibly, the quality being measured is “overall satisfaction” or “overall pleasure”, or something along those lines. There are deep problems in claiming that pleasure can be quantified, but lets put those aside for the moment and suppose that it can. Matters get more complicated if that pleasure being measured is a function of the quality of the wine (subjectively understood as you note.)
What qualities? Well, things like balance, structure, concentration, typicity (or region and terroir), uniqueness, personality, and the like. But if “overall satisfaction” is a quantifiable notion and a function of these qualities, then these qualities must be quantifiable as well. Perhaps they are but it is not obvious. Perhaps wine critics have a way of quantifying them, but if they do it isn’t apparent.
To the extent that people recognize that wine scores are capable only of indicating an ordinal ranking there is no problem. But the scoring system suggests something more.
To my mind, this does not make wine scores useless for some purposes–when a customer with limited knowledge confronts a shelf full of wines it helps to have something to go on. But they are not a stand-in for a comprehensive evaluation.
Edible Arts:
Perfect.
Tom,
You also said this:
“They believe the critic when they assert through a score and review that Wine X is better than Wine Y and act on that belief.”
If the assertion is obviously subjective, why would anyone believe it as truth?
The only logical answer to that question is that the consumer must believe that the critic has engaged in a quantifiable, objective evaluation. Either that, or consumers are collectively stupid.
This is a subject I try to stay out of because I use the 100 point scale to rate wines I review, and people have every right to ignore it if they so choose. However if anyone thinks that the next generation of consumers finds no value in scores, replace each occurence of the word ‘wine’ with ‘iPhone application’ and references of wine magazines to computer magazines, in the following excerpt from the author’s post:
And fundamentally, the quantitative rating of wine damages the effort to broaden consumers’ palates and hampers the introduction of a broader range of wines from all over the world into the American wine market. It reduces wine discourse to numbers and stars and pits wines from certain parts of the world, particularly those that were not established in America when Wine Advocate and Wine Spectator began issuing scores, against others.
As someone who writes about wine, I appreciate that how I do that brings value to my readers (consumers). I know a lot more about wine than I do iPhone apps and use the star rating system to find the top examples without having to buy them blind.
Thomas,
You and I seem to have this discussion once per year and I always enjoy it.
It’s not as though consumers believe a Robert Parker 95 point cabernet is objectively better than a Robert Parker 89 the way they believe that 1+!=2 is objectively the truth. Rather, they trust Robert Parker’s opinion that the one wine is better than the other.
The key is that they are willing to be convinced otherwise. On the other hand, they are not willing to be convinced otherwise about the equation 1+!=2.
Tom,
You need to spend some time selling wine to consumers.;)
I imagine that the trust of which you speak is an extension of a belief that the critic has a method of determining which is better and why it is better. Again, if the consumer does not believe in a method behind the proclamation, why trust or believe the critic at all? Why even care about the evaluation?
It wouldn’t be the first time that a profession or an industry has been built on an illusion–or maybe it’s an allusion instead…
Thomas,
I think you may be overestimating the degree to which consumers need or want an understanding of how a critic approaches their evaluation. I believe what’s going on is that they simply trust their expertise and that alone is enough.
I think you are not taking into account the profound gullibility of the general wine buying public. Those point scales are magical and, combined with the look of the label, are the only things they have to go on. They don’t necessarily know that the score is one person’s opinion. The people who take the time to read wine magazines and blogs, the people you are right to assume know the point scale does not imply objectivity, are different from the much larger group who do not do those things.
It does not good to discuss and debate the mathematics of the 100 point scale, for one simple reason: there is no 100 point scale. Instead, there are lots of them, some more glorified than others (RP, WS), some so inconsequential that we hardly know they exist (John/Jane Blogger), and some that are abjectly contrived (JustWinePoints.com, retailers that make up their own scores). They are all appear to quack alike, but the DIFFERENCES in methodology and valuation are significant… and invisible to all but the closest of observers. The net result is as if we are all claim to be talking about the same thing — “the” 100 point scale — but we are all talking in slightly different dialects about a ginormous object that we are each only viewing from a limited angle/viewpoint.
Given wine’s inherent subjective vs. objective dynamic, plus the passion and knowledge borught to the conversation by those (mostly wine pros and geeks) who care enough to engage in discussion at length, the conversation is doomed to spin on and on and on.
THat does not mean we should stop talking about it or expect the debate to end anytime soon. I take great comfort in the knowledge that in dealing with consumers of all levels of experience, I am convinced that the actual influence of the ratings themselves is lessening every year. If and when someone can come up with a better paradigm, all the 90-pt gnashing will fade even more quickly.
Tom,
You will exasperate me if you keep this up.
“I believe what’s going on is that they simply trust their expertise and that alone is enough.”
Trust what expertise? What evidence do they have that the critic is an expert beyond the fact that we are all experts in our own subjectivity?
The way i see it, the potential fact that a consumer might consider someone an expert probably leads to the illusion that there is a viable, considered method behind the evaluation.
First, sorry it took me so long to comment on this thread. Second, wow, I’m extraordinarily happy that what I wrote spawned such a thoughtful discussion. Edible Arts, Thomas Pellechia and others have already articulated much of my counter argument.
We’re having a semantic debate here. Tom, you seemed to think that I was advocating for the scale’s objectiveness in the “universal” sense that 1+1=2 or that if I drop an egg out of my window it hits the ground and has a very strong likelihood of breaking.
In fact, I was making the argument that within the system (critics’ respective 100 point systems), there is the implied objectivity that a wine that a critic scores a 91 is inherently, objectively better than a wine that he or she scores a 90 based on whatever subjective criteria that critic uses. Multiply that by innumerable score givers and the use of scores in marketing wine and you have situation in which we have a 100 point rating system that implies some varying degree of objectivity for consumers.
But in fact, you can create a subjective reality in which 1+1 does not equal 2 but, within the rules created, it can be objectively proven that 1+1=3 or even 1+1=banana.
And effectively, wine critics who are proponents of ANY form of quantitative scoring of wine have created a world in which 1+1=banana. It is a world in which a 100-point wine is a “perfect” wine, which is as believable to me as asserting that a 100 point wine is a parthenocarpic Musa cultivar.
(Forgive me if that was too relativist.)
I reject that we need to replace one rating system with another as we already have the best form of wine criticism available: thoughtful, subjective narrative writing. Why wine writers, even very good ones, keep arguing for a system that wants to boil their writing and tasting notes down to a number is beyond me.
(And a last side note to Doug Wilder, star ratings of iPhone apps in the App Store are crowd-sourced from many users of the app and, while far from perfect, represents an aggregation of the assessments of dozens, hundreds, or even thousands of users and, as such, are fundamentally different than an individual critic’s assessment.)
Thanks for a great debate. If anyone is still following this thread, let me know your thoughts.
you need to reread the first quote you quoted and sit and think about it.
he clearly says that while theoretically what you reiterated is right practically most wine consumers are dumb and use numbers as an objective standard. he is not the person you should be trying to convince. the vast majority of wine drinkers in the us is.
your argument against his argument is the argument that he already argued against. you yourself need to rethink the formula and come up with a different argument as he already argued against yours.