Vodka Research

Posted June 1, 2006 at 12:14 am 31 Comments

Perhaps you saw the Internet meme about vodka filtration that was circulating last year. An enterprising group of young “scientists” purchased a bottle of extremely cheap vodka and a Brita filtration pitcher, and after pouring the vodka through the charcoal filter several times, they claimed that the result was indistinguishable from expensive “top shelf” vodka.

Original Vodka Experiment

I admired the ingenuity of these researchers, but I found their experimental methods somewhat suspect. They began by tasting the cheap vodka, which they all thought was horrible. Next they filtered it once and drank some more. “Much better,” they agreed. They ran it through the filter again and found that it tasted even better! I’m sure you realize that this experimental design has certain confounding factors.

So naturally, I was skeptical of the findings; after all, if all that were required to produce good vodka was plenty of filtration, it’s hard to believe that cheap vodka would be so repugnant. Even so, if their results were borne out by further study, the potential gains would be staggering. Instead of wasting my money on Chopin or Grey Goose, I could purchase a $9 plastic handle of Vladimir vodka, run it though a filter, and mix up deliciously smooth martinis at a fraction of the expense.

So, in true scientific spirit, I replicated the vodka filtration study at our weekly Computer Science Department social event, but under revised conditions that I believe produced more reliable results.

First Experiment

We began by filtering a bottle of “Pavlova,” a foul-smelling but extraordinarily inexpensive brand of vodka (cost: $8 per liter). We decided to compare the filtered Pavlova to Ketel One, a Dutch vodka that is generally very highly regarded (cost: $27 per liter).

Pavlova and Ketel One

We set out two pitchers of vodka labeled A and B along with small cups for tasting. Our subjects sampled the two varieties, wrote down their preferences on small sheets of paper, and cast their votes in a ballot box.

This initial vodka filtration experiment seemed a success. Of the 24 people who participated in the blind taste test, two-thirds preferred the inexpensive filtered vodka (Pavlova) over the expensive premium vodka (Ketel One). I myself preferred the Ketel One; I wasn’t sure whether to be pleased with my refined taste in vodka, or disappointed that I couldn’t use the filtration trick to reduce my monthly martini budget.

Experiment 1 Chart

Second Experiment

Although our preliminary results were encouraging, we decided that further experimentation was required before drawing any definitive conclusions. In our next experiment, conducted several months later, we investigated two additional factors:

  1. The first experiment failed to establish that people would choose the more expensive vodka in the absence of filtration. In the second experiment, we ran an initial baseline in which participants sampled unfiltered versions of both vodkas, to ensure that the taste preference could be attributed to the filtration.
  2. It is possible that the filtration process actually removes alcohol from the vodka. This would certainly account for the improved taste, but it would make the procedure much less valuable. In the second experiment, we used an alcoholmeter (a modified version of a hydrometer that measures percentage content of alcohol) to see if the filtration removed alcohol from the liquor.

Our second experiment used four pitchers, labeled A through D. Pitchers A and B contained unfiltered Pavlova and Ketel One, respectively. Pitchers C and D contained Ketel One and filtered Pavlova. In the first experimental condition, subjects compared the vodka in pitchers A and B, rated each vodka on a 5-point Likert scale, and indicated which vodka they preferred. In the second condition, the same set of subjects compared pitchers C and D, marking their preferences the same way.

We recruited 26 subjects for the second experiment. Our results were as follows:

First Comparison
Subjects preferring A to B: 12
Subjects preferring B to A: 12
Subjects with no preference: 2
Second Comparison
Subjects preferring C to D: 11
Subjects preferring D to C: 13
Subjects with no preference: 2

Our hydrometer readings showed significant differences in alcohol concentration between the three varieties of vodka: Ketel One measured 88 proof, unfiltered Pavlova 82 proof, and filtered Pavlova 78 proof. We suspect that the reduced alcohol content in the filtered vodka was not actually a result of the filtration, but rather evaporation during the filtration process, as it was repeatedly poured from container to container.

Ketel One

Ketel One: 88 Proof

Filtered Pavlova

Filtered Pavlova: 78 Proof

Although our second experiment still demonstrated a minor benefit from filtration, the effect was far less pronounced than in the first experiment, and in fact the difference may be attributed to the lower alcohol content of the filtered vodka. In the first experiment we filtered the vodka the night before and left it in a pitcher overnight, which may have resulted in even greater alcohol evaporation, accounting for the more pronounced differences.

Experiment 2 Chart

There are a variety of ways in which our data could be interpreted, but our general analysis is that most people can’t tell the difference between expensive vodka and cheap vodka, regardless of whether or not it has been filtered. Of the 12 subjects who preferred Ketel One in the first trial, only 7 preferred it in the second trial; meanwhile, 4 of the 12 subjects who preferred unfiltered Pavlova in the first trial decided they preferred Ketel One to filtered Pavlova during the second trial. This seemingly haphazard set of preferences would be consistent with the hypothesis that our subjects were in general unable to discriminate between the vodkas in either condition.

Despite this confusion between vodkas, our findings imply that Ketel One is superior to Pavlova, at least by one metric. If we assume that a simple way to make vodka taste smoother is to reduce its alcohol content, then we would expect weaker vodkas to come out ahead in taste tests. The fact that many people preferred Ketel One during both trials, despite its significantly higher alcohol concentration, suggests that it is of generally higher quality.

Conclusions

To summarize our findings,

  1. Given a particular brand of vodka, people prefer its taste after it has been filtered, but this is most likely because filtration reduces the alcohol content.
  2. Most people can’t tell the difference between an expensive vodka with high alcohol content and a cheaper vodka with lower alcohol content.

Our second experiment demonstrated approximately equal preferences for Pavlova and Ketel One. Although Pavlova contains 3-5% less alcohol by volume than Ketel One, it is also 70% cheaper, so it would seem a clear winner.

Do our results indicate that you should always buy cheap vodka for your parties? Not necessarily. Vodka distribution at parties is rarely administered in a double-blind fashion. In situations in which the taster is aware of which brand of vodka he is drinking, his preconceived notions of its quality will likely provide a strong influence on his perception of its taste.

Future Work

In the second experiment, we chose two side-by-side comparisons rather than a single three-way comparison because we felt it would be too difficult for subjects to rank three alternatives. Unfortunately this meant that our second experiment never gave us a direct comparison between filtered and unfiltered Pavlova. There was no significant difference between the average Likert scale ratings for filtered and unfiltered Pavlova (2.85 for filtered and 2.87 for unfiltered), but a more thorough experiment might add an additional condition in which the two were compared.

We observed differences between the alcohol contents of the three vodkas, and we assume that alcohol concentration is closely connected to taste preference. In our next experiment, we may attempt to control for alcohol concentration by diluting the stronger vodkas with water. We would also like to control for the size of each vodka sample; in our experiments we allowed subjects to pour their own samples, and a large sip of vodka may be perceived as tasting worse than a small sip of the same vodka.

We also hope to investigate the influence of preconceived notions of quality on perceived taste. In a future study, we plan to pour the same variety of vodka into two different bottles, one labeled with an expensive brand like Ketel One, and one labeled with a cheap brand like Pavlova. By ensuring that the bottle labels and price tags were visible during the taste test, we could measure the effect of brand reputation on perceived quality.

Collin finishes the vodka
Our experiment complete, Collin polishes off the remaining vodka.

31 Comments »

RSS feed for comments on this post. TrackBack URI

  1. OHHHH! I tried this, only with grey goose and bartons and not so in depth! the grey goose won, but i ws dissappointd that my friends cant tell the difference between good vodka and cheapass sucky vodka.

    Comment by Heather — June 12, 2006 #

  2. I think one of the points Monzy alluded to, which may be overlooked by readers… especially Heather… is: although Pavlova may be cheapass, it’s not necessarily sucky.

    Just because Ketel One has a prettier bottle and a higher pricetag, many drinkers (myself included) wouldn’t be able to tell the difference in “quality” if they didn’t already know the reputation of the drink.

    I look forward to reading his results of the third experiment. I have a feeling it will shed some light on peoples’ perceptions of Grey Goose, Chopin, et al.

    Comment by Ryan — June 19, 2006 #

  3. dan-
    it makes me smile to know that you did an experiment comparing shitty and tasty vodka.

    it’s late right now (my flight to boston got cancelled, i’m all giddy on ritalin and can’t fall asleep), i’m gonna try and get rest (soon), but in the mean time i’m gonna read more of your blog.

    can i link it on mine? oh right, you should check out mine if you want (www.ashamarie.com). i started it a few days before i went to volunteer in new orleans–mid february. word. hope you’re well.

    asha

    Comment by asha — June 20, 2006 #

  4. i agree that most people can’t tell the diff. esp. cs nerds who don’t drink enough to flunk cs like me. that’s why i’m in law school! i’ll prosecute your patent claims soon enough!

    Comment by Contributor X — August 8, 2006 #

  5. Hmmmm,
    Maybe try a version in which the participants include ‘regular’ vodka drinkers and non-vodka drinkers.

    Comment by dragyourbone — August 11, 2006 #

  6. I admire your frugality and your adherence to the scientific method.
    However, real martinis are made with gin.

    Vodka + vermouth + olives in a martini glass is properly called a “vodkatini”.

    Keep up the good work.

    Comment by jacob — September 2, 2006 #

  7. Try Burnetts Vodka. Its cheap and better than
    Absolut, Smirnoff and many other well-known brands.

    Comment by Mario Crociati — September 3, 2006 #

  8. One condition that seems widely overlooked is the filtered version of expensive vodka. Although only useful for the purposes of the experiment, I’m curious to know whether filtration has an effect on an already supposedly high quality vodka.

    Comment by Sparky — September 7, 2006 #

  9. Knock knock, my vodka tasting chums. Vodka’s meant to be tasteless. If you can taste it, they’ve either added flavouring (bleurgh) or their filtration process sucks.

    Comment by Vodka King — September 15, 2006 #

  10. Monzy, do you have any more statistical data on this? I’m doing a project on linear equations (my math class gets lame projects), and I thought this would be interesting to try.

    Email me at the.snowlion@gmail.com if you can.

    — Joe

    Comment by Joe — November 12, 2006 #

  11. Doesn’t matter. Ketel One is the vodka for pimps. And Ketel One & tonic is the drink of pimps.

    Comment by goshone — December 13, 2006 #

  12. I have longed believed that certain vodkas were
    over-rated. I have found a great inexpensive
    vodka labled: Burnetts. It also comes flavored.
    M Crociati

    Comment by Mario Crociati — December 14, 2006 #

  13. This is compelling work. What about the conventional wisdom that cheap vodka gives you a worse hangover (purportedly due to “impurities”)? I’d love to see some research on that.

    Comment by TasterSpoon — December 22, 2006 #

  14. real vodca drinkers know! period

    Comment by raw — December 24, 2006 #

  15. My local liquor dealer told be that Skyy and Royal Gate were the same product just sold in different bottles. So I bought two little bottles and did a semi-blind taste test. That means I marked the glasses but drank them in the dark (in my Home Theater while watching a movie). In fact there was no comparison. The cheap Royal Gate smelled like finger nail polish remover. Skyy had no smell at all. I don’t know Pavlova. Try Royal Gate. I taste tested Ketel One and Absolut and a bunch of others years ago. Smirnoff was best.

    Comment by Pat — February 22, 2007 #

  16. If I might suggest, you should also consider doing a blind test where individuals have a set amount of both vodkas over the course of an evening, drinking nothing else, followed by feedback the next day as to how they are feeling, rated 1-10, whether they have any body aches, headaches, stomach aches etc.

    The point being, filtration may alter initial flavor, but might not filter out impurities, etc. that potentially cause a “day after” effect.

    If impurities in vodka, etc. aren’t part of the problem with cheap alcohol, then you’d expect Ketal drinkers to have the worst aftereffects, as they would’ve consumed more alcohol, as their vodka was of higher proof.

    Comment by Mark Kraft — March 2, 2007 #

  17. this was done on mythbusters and proven to not be true. They had professional tasters in who always picked the top range vodka over the one that had been filtered (filtered 6 times in some cases). They came to the conclusion that filtering does make the cheap vodka slightly better but still nowhere near the quality of the good stuff

    Comment by Richie — March 9, 2007 #

  18. Is this filtering process so expensive that it is just not worth the trouble for the distiller?

    Comment by MJ99 PHP Scripts — March 13, 2007 #

  19. Richie is wrong. That’s not what MythBusters found at all. They declared the myth busted but in fact they proved that filtering worked. Their expert taster in fact was able to perfectly distinguish how many times each vodka had been filtered. That is to say he could tell the 6X from the 5X from the 4X etc… He could also distinguish the “top shelf” vodka which I believe was Gray Goose. He said that was the one he preferred. So thay busted the myth that you could produce a vodka indistinguishable from Gray Goose but proved that filtering works. BTW Gray Goose has a light but distintive flavor as does Stoli. You may prefer those flavors or you may prefer a more flavorless vodka like Skyy or Smirnoff. Obviously you can’t filter a flavor in.

    Comment by pat — March 24, 2007 #

  20. Um, this experiment was done by Myth Busters, and they had a “professional” Russian vodka taste tester who worked for an unnamed vodka distillery back in Russia. I recommend you check out the episode to see the result.

    Comment by Vip3rousmango — June 20, 2007 #

  21. Well this is all fine and dandy, but they do not say anywhere how many times they filtered it through the brita filter. Not very respectful of the scientific process. Personally i am a little dissappointed.

    Comment by Devin — June 20, 2007 #

  22. Why not use grain alcohol to raise the alcohol level of the cheap stuff to the same as the more expensive and see what the results are?

    Comment by Netopia — June 25, 2007 #

  23. Please don’t forget to use new filters for vodka filtration. Ethanol is used to clean carbon filters; if you use an old filter that’s been cleaning your water, the vodka will be unusually gross after filtration. I ran similar experiments at a vodka tasting party with a lot of scientists, and blind taste tests where people directly compared filtered to unfiltered skol revealed that about 3/4 of participants could tell the difference. However, they had been comparing the 25 vodkas available, so may have been sensitized to the various nuances in advance. But while it was improved by removing the strong off-flavors, it still lacked the smoothness and positive flavors that better brands had. You might try the following brands for different aspects:
    Svedka: Probably the cheapest good vodka you can get, at around $14/750ml for a reasonably smooth and clean flavor.
    3 vodka: the most flavorless and smooth vodka I know of. Preferred by people who don’t like liquor.
    Russian standard or Hangar One: The two best vodkas available for overall combination of smoothness and flavor. Significantly better than grey goose.

    Comment by Jon — November 23, 2007 #

  24. Obviously I have come in late, but found this discussion interesting.

    Jon reported on blind taste tests comparing the 25 vodkas available.

    I think there is probably a chemical equivalent of the Heisenberg Uncertainty Principle. Actually, the inverse: the observer is influenced by the observation.

    The findings of a group of individual who tested 25 different vodkas are, IMHO, somwhat suspect.

    Nazdrovye

    Izzy

    Comment by Iskandar B — October 22, 2008 #

  25. OK – here is my 2 cents. Vodka is distilled from any of the common grain cereals. It is a grain neutral spirit – it is made originally to a high proof – usually 190 proof or 95% alcohol and then cut with filtered water. It is not impossible that each type of permitted grain – has esters that are discernable to the nose. The water used to cut the proof is also contributing something to the taste, is it hard or soft or mineralized ??Bowman Vodka a very fine and inexpensive product formerly made upon ground now known as Reston VA – is now produced near Harrisonburg VA. It is filtered 4 times, as is Grey Goose. My Virginia based VA ABC store manager tells me 2 things about his preferences and Grey Goose in particular. One – at slightly higher cost per bottle .10 to .50 cents each he orders everything he can bottled in glass. Plastic is slowly killing all of us. Second – as to the Goose. It was for many years a 4th shelf – or bottom shelf Vodka. Then in the 1990’s they spent a fortune on advertising and “created the myth”. Burnetts – mentioned by another writer is a fine Vodka and a fine Gin too – referred to as a bartenders gin sometimes, because it is inoffensive and mixes well. Anyone who orders a Grey Goose Martini Dirty or with any known dead vegtable in it, and presumes they can tell Grey Goose from anything else is simply untutored and wasting a lot of money. My parting thought – last I knew the rail gin at Chiles at Tyson’s Corner was Tanquery and that is no booze to sneer at. My point ? Get your smelling nose and your fixation with image onto the same page. Chasing mere image is going to raise the cost of your life by 50% or more for no good purpose. My applause to the testors !!

    Comment by J Dorsey — December 12, 2008 #

  26. I do feel the need to point out that you didn’t take a statistically valid number of data points, so your conclusions can’t be considered sound. Generally at least 30 data points must be collected to be considered statistically valid… IE one person would have to do 30 double-blind taste tests before you could really conclude whether or not they couldn’t tell the difference between the vodkas over the course of the trials. I would recommend redoing the study with a much greater number of data samples. You could also eliminate the evaporation hypothesis by subjecting the unfiltered vodka to the same type znd number of physical manipulations as the filtered vodka, or, even better, constructing an airtight experimental vessel within which to perform the filtering and store both vodka samples.

    Comment by Mike Kupietz, FileMaker Pro Consultant — May 31, 2009 #

  27. Hmmm. Suspect data? Anyone who drinks straight alcohol (vodka, gin, burbon, etc.) knows, or should know, that only the first sniff and taste are reliable. The more one sniffs and tastes the less sensitive become your taster and smeller. After five or six sniffs and tastes I would be surprised if anyone could actually tell the difference between premuium vodka and sterno! Rule one – when testing any alcohol (at room temperature) only SNIFF ONE SAMPLE the first pass, and wait or clense your olafactiries before sniffing the second sample. The esters and residual oils will be evident only then. Then have a cracker or bit of cheese before the second sample. Rule two – buy and drink the alcohol of your choice regardless of the price or label!

    Comment by john loberg — November 19, 2009 #

  28. Despite I don’t like vodka, I’m proud you mentioned Chopin as the ‘expensive and good’ one. My polish proud just popped into the air :D

    Comment by assd — December 4, 2009 #

  29. I work in advertising and one of my clients is a young batch distillery producing vodka, rum, gin, and eventually whiskey. I learned the process of distillation and the simple science and regulation behind vodka. Basically vodka is pure ethanol with water mixed back into it. In countries like poland and rusa perhaps, the regs arent so stringent and other elements might be mixed into the drink but in the u.s. the concetration of other impurities MUST BE very very small as to make them negligible in order to sell it as vodka (to sell it at all). Its something like .093 percent othr junk allowed.

    When tested, vodkas like grey goose and ketel one and ones like yours in the experiment and gibleys or house brand crap are in no way distinguished by impurity. Grey goose may have more gunk than gibleys and ketel one may have les than both of the others. This stuff is all fact and simple fact at that. Any vodka producer (not their marketing dept.) Can tell u this and prove it to you.

    So your experiment is awesome. A great way to bring science into the realm of taste and perception and out of the lab where people obviously have no interest in being.

    It might seem funny at first glance that even after your findings report the first comment is made by someone calling inexpensive vodka the cheap crap. But the, dont we learn this again and again? The porsche cayenne and the volkswagon taureg are the same car. A cheap chinese gucci bag and the real thing are indistinguishable even by the makers of gucci bags. Of course there are instances all over the place where quality is hugely important and proves to be more lasting, effective, etc. It is that fact alone that allows us to be suckered again and again when its not.

    My conclusion? Star belied sneetches.

    Nice report.

    Comment by Chad — November 6, 2011 #

  30. Here is a way to test perception vs. reality. Fill a Gray Goose bottle with cheap vodka and serve it to your friends. I tried this recently at a poker game where many of my friends are vodka snobs. Nobody noticed. Of course when I revealed it to them at the end of the night there were a few who claimed “they knew it”.

    Comment by Jason — January 31, 2013 #

  31. Why the hell did you stop blogging? I demand to be entertained once more by your wrecklessly irresponsible lifestyle.

    Comment by C. M. Yoo — April 13, 2013 #

Leave a comment

XHTML: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Powered by WordPress with Pool theme design by Borja Fernandez.
Entries and comments feeds.