Vodka Research

Posted June 1, 2006 at 12:14 am

Perhaps you saw the Internet meme about vodka filtration that was circulating last year. An enterprising group of young “scientists” purchased a bottle of extremely cheap vodka and a Brita filtration pitcher, and after pouring the vodka through the charcoal filter several times, they claimed that the result was indistinguishable from expensive “top shelf” vodka.

I admired the ingenuity of these researchers, but I found their experimental methods somewhat suspect. They began by tasting the cheap vodka, which they all thought was horrible. Next they filtered it once and drank some more. “Much better,” they agreed. They ran it through the filter again and found that it tasted even better! I’m sure you realize that this experimental design has certain confounding factors.

So naturally, I was skeptical of the findings; after all, if all that were required to produce good vodka was plenty of filtration, it’s hard to believe that cheap vodka would be so repugnant. Even so, if their results were borne out by further study, the potential gains would be staggering. Instead of wasting my money on Chopin or Grey Goose, I could purchase a $9 plastic handle of Vladimir vodka, run it though a filter, and mix up deliciously smooth martinis at a fraction of the expense.

So, in true scientific spirit, I replicated the vodka filtration study at our weekly Computer Science Department social event, but under revised conditions that I believe produced more reliable results.

First Experiment

We began by filtering a bottle of “Pavlova,” a foul-smelling but extraordinarily inexpensive brand of vodka (cost: $8 per liter). We decided to compare the filtered Pavlova to Ketel One, a Dutch vodka that is generally very highly regarded (cost: $27 per liter).

We set out two pitchers of vodka labeled A and B along with small cups for tasting. Our subjects sampled the two varieties, wrote down their preferences on small sheets of paper, and cast their votes in a ballot box.

This initial vodka filtration experiment seemed a success. Of the 24 people who participated in the blind taste test, two-thirds preferred the inexpensive filtered vodka (Pavlova) over the expensive premium vodka (Ketel One). I myself preferred the Ketel One; I wasn’t sure whether to be pleased with my refined taste in vodka, or disappointed that I couldn’t use the filtration trick to reduce my monthly martini budget.

Second Experiment

Although our preliminary results were encouraging, we decided that further experimentation was required before drawing any definitive conclusions. In our next experiment, conducted several months later, we investigated two additional factors:

1. The first experiment failed to establish that people would choose the more expensive vodka in the absence of filtration. In the second experiment, we ran an initial baseline in which participants sampled unfiltered versions of both vodkas, to ensure that the taste preference could be attributed to the filtration.
2. It is possible that the filtration process actually removes alcohol from the vodka. This would certainly account for the improved taste, but it would make the procedure much less valuable. In the second experiment, we used an alcoholmeter (a modified version of a hydrometer that measures percentage content of alcohol) to see if the filtration removed alcohol from the liquor.

Our second experiment used four pitchers, labeled A through D. Pitchers A and B contained unfiltered Pavlova and Ketel One, respectively. Pitchers C and D contained Ketel One and filtered Pavlova. In the first experimental condition, subjects compared the vodka in pitchers A and B, rated each vodka on a 5-point Likert scale, and indicated which vodka they preferred. In the second condition, the same set of subjects compared pitchers C and D, marking their preferences the same way.

We recruited 26 subjects for the second experiment. Our results were as follows:

First Comparison Subjects preferring A to B: 12 Subjects preferring B to A: 12 Subjects with no preference: 2

Second Comparison Subjects preferring C to D: 11 Subjects preferring D to C: 13 Subjects with no preference: 2

Our hydrometer readings showed significant differences in alcohol concentration between the three varieties of vodka: Ketel One measured 88 proof, unfiltered Pavlova 82 proof, and filtered Pavlova 78 proof. We suspect that the reduced alcohol content in the filtered vodka was not actually a result of the filtration, but rather evaporation during the filtration process, as it was repeatedly poured from container to container.

Ketel One: 88 Proof

Filtered Pavlova: 78 Proof

Although our second experiment still demonstrated a minor benefit from filtration, the effect was far less pronounced than in the first experiment, and in fact the difference may be attributed to the lower alcohol content of the filtered vodka. In the first experiment we filtered the vodka the night before and left it in a pitcher overnight, which may have resulted in even greater alcohol evaporation, accounting for the more pronounced differences.

There are a variety of ways in which our data could be interpreted, but our general analysis is that most people can’t tell the difference between expensive vodka and cheap vodka, regardless of whether or not it has been filtered. Of the 12 subjects who preferred Ketel One in the first trial, only 7 preferred it in the second trial; meanwhile, 4 of the 12 subjects who preferred unfiltered Pavlova in the first trial decided they preferred Ketel One to filtered Pavlova during the second trial. This seemingly haphazard set of preferences would be consistent with the hypothesis that our subjects were in general unable to discriminate between the vodkas in either condition.

Despite this confusion between vodkas, our findings imply that Ketel One is superior to Pavlova, at least by one metric. If we assume that a simple way to make vodka taste smoother is to reduce its alcohol content, then we would expect weaker vodkas to come out ahead in taste tests. The fact that many people preferred Ketel One during both trials, despite its significantly higher alcohol concentration, suggests that it is of generally higher quality.

Conclusions

To summarize our findings,

1. Given a particular brand of vodka, people prefer its taste after it has been filtered, but this is most likely because filtration reduces the alcohol content.
2. Most people can’t tell the difference between an expensive vodka with high alcohol content and a cheaper vodka with lower alcohol content.

Our second experiment demonstrated approximately equal preferences for Pavlova and Ketel One. Although Pavlova contains 3-5% less alcohol by volume than Ketel One, it is also 70% cheaper, so it would seem a clear winner.

Do our results indicate that you should always buy cheap vodka for your parties? Not necessarily. Vodka distribution at parties is rarely administered in a double-blind fashion. In situations in which the taster is aware of which brand of vodka he is drinking, his preconceived notions of its quality will likely provide a strong influence on his perception of its taste.

Future Work

In the second experiment, we chose two side-by-side comparisons rather than a single three-way comparison because we felt it would be too difficult for subjects to rank three alternatives. Unfortunately this meant that our second experiment never gave us a direct comparison between filtered and unfiltered Pavlova. There was no significant difference between the average Likert scale ratings for filtered and unfiltered Pavlova (2.85 for filtered and 2.87 for unfiltered), but a more thorough experiment might add an additional condition in which the two were compared.

We observed differences between the alcohol contents of the three vodkas, and we assume that alcohol concentration is closely connected to taste preference. In our next experiment, we may attempt to control for alcohol concentration by diluting the stronger vodkas with water. We would also like to control for the size of each vodka sample; in our experiments we allowed subjects to pour their own samples, and a large sip of vodka may be perceived as tasting worse than a small sip of the same vodka.

We also hope to investigate the influence of preconceived notions of quality on perceived taste. In a future study, we plan to pour the same variety of vodka into two different bottles, one labeled with an expensive brand like Ketel One, and one labeled with a cheap brand like Pavlova. By ensuring that the bottle labels and price tags were visible during the taste test, we could measure the effect of brand reputation on perceived quality.

Our experiment complete, Collin polishes off the remaining vodka.