26 Jun The Criteria For, And Problems With, Judging And Rating Indiana Breweries
by Mark E. Lasbury for Indiana On Tap
Very recently, Indianapolis Monthly released their current list of best new breweries. I applaud this action unreservedly. Every time a publication of the substantial circulation of Indianapolis Monthly devotes a story and substantial online work to a story about Indiana craft beer, I am all for it.
For everyone not in Indianapolis on a regular basis, let’s discuss why this list is a big deal. The metropolitan area of Indianapolis now boasts fifty-one breweries, with at least three more set to open some time this year (Kirk Haus Brewing, Happy Brewing, and Guggman Haus Brewing). Some breweries are old enough to be called venerable (Broad Ripple Brewpub, Oaken Barrel Brewing), while an entire slew or slew-and-a-half of breweries have opened in the last three to four years.
There are Brettanomyces breweries, Belgian breweries, experimental and high gravity breweries, and even primarily German lager breweries. Indianapolis breweries have won a bucketful of medals at state, national, and international competitions, and there will definitely be more to come. Indianapolis breweries are, as are several in Indiana, invited to pour at major festivals and receive rave reviews when they do.
Paraphrasing Richard Powers from his novel Orfeo, “Eventually, as with all things we love, we end up turning them into a competition.” Indie craft beer is no different than anything else, and this is why rating apps are so popular and why best of lists garner so many page views. You choose to either argue or praise, based on whether your favorites are there or not. However, rating and ranking are difficult tasks, and “the wisdom of crowds” set aside, they can be faulty, or at least present less than the full story.
The 2016 top breweries as ranked by BA. You have to look carefully to not e that the rankings were based on sales. This raises the question – just how should one rank different breweries? image credit: Brewers Association
Because of the diversity of brewery styles, brewery ages, and patron loyalty based on location, putting together a best brewery list is difficult. Are you judging breweries on how close they come to the styles of beer they say they are making? Do you judge them on the technical brilliance or flaws of their beer? Do you judge them on their location or the experience you had last time you were in? Are beer names, branding, and label art important to you – seems silly, but there are those people for whom these items are paramount.
These are the same problems that people run into when rating beers and breweries on Untappd, BeerAdvocate and the like. Are you scoring a beer based on whether you like the style, or according to how well the beer was made in that style? Are you rating a brewery based on the quality and ingenuity of its beers, or by how you liked your food, whether you had a bad workday, or if the people around you were too loud? Unless you drink a fair number of beers from a brewery and approach them objectively, should you really be rating them (or berating them, as the case may be) publicly?
Notice the changes in yelp stars from 2005 to 2014. Rants (1 star) are way up, meaning people trying to sink businesses. Ones are more numerous than twos OR threes. The increase in 5 stars is probably a friends and family affect. image credit: Small Business Trends
A friend who recently opened a brewery told me the other day that he is so glad that he did this at age fifty instead of at age thirty. A younger version of him would have been devastated by some of the feedback that every brewery gets. Despite knowing in his heart that many times these opinions are not well supported or have little to do with the quality of the beer – it can be soul crushing to hear horrible, non-constructive criticism of your efforts.
On the other hand, a best NEW brewery list or set of ratings should be slightly easier to put together. It may have all the same problems of putting together a best brewery list, but you have fewer breweries to choose from so having to pick across styles and categories is less likely. In addition, new breweries are more likely to all be on the same footing. Many have owners/brewers that might have full time jobs and are doing this as a second gig until it gets going. Most very new breweries are making the mistakes that all the older breweries made, but have learned from and don’t have to worry about now.
Unfortunately, there are problems with rating or picking a “best of” list that are unique to new breweries. Primarily, the younger they are, the more rapidly they are changing. Just as Mike Tyson said that everybody has a plan for a fight until they get hit in the face, breweries may have a production schedule, a business plan, and a list of reliable recipes – until they open their brewery on what is most likely a very different brewing system than they are used to.
This problem is especially pernicious for home brewers that have decided to go pro – now very popular path to commercial brewing. Switching from direct fire to steam jet, or electric to gas, or even just ramping up size and adjusting for the time it takes water to heat up and cool down can bring changes to tried and true beers. It takes time to grow into a new brew house.
Grand openings for breweries rarely occur when they actually open. They need a good amount of time to work into their system, so soft openings are longer for breweries than for most restaurants. image credit: Unknown Brewing
A brewery less than 4-6 months old should be considered an infant – lots of potential, but there’s little way of knowing how things will turn out. A brewery that has six months under its belt has now started to learn its system, and Walter and I have often seen breweries make a strong turn in a positive direction around this time. A brewery a year old or older is now responsible for what they put out; they have had enough time to rework recipes if needed and to gather enough credible feedback on what they are doing well and what needs a tweak.
So while exceptions do exist, a brewery more than a year old has basically told us who they are. Brewers can change, philosophies can change, and there can be slow learners, so none of this is set in stone, but in general, breweries less than 6-8 months old are the ones who should be given a lot of leeway. Heck, I would advocate that all Untappd, Yelp, Beeradvocate, and Facebook ratings and rants should self-destruct once a brewery hits eight months of age and they should start again with a blank slate.
Walter is very cognizant of this fact, and always reminds me to take a first offering from a brewery with a very open mind. In a few cases, breweries hit the ground running – they offer mature, well-rounded beers right out of the gate (Round Town, Deviate are a couple that come to mind). But in most cases, a new brew house means a steep learning curve.
And here is where there might be a slight hitch in Indianapolis Monthly’s list. Their most recent list of best new breweries was put out in September of 2014 – so to them, any brewery which opened after their list came out is considered new, sort of. They included ten breweries, and didn’t try to rank them (thank goodness); you can read about the story here.
Central State Brewing is on Indianapolis Monthly’s list of best new breweries. According to Walter and I, no one can argue with that, and Central State is so good that one could make an argument to have rankings instead of just a list. See how easy it is to fall into the list and rankings trap? image credit: Central state Brewing
I like all the choices, but not for most of the reasons they stated. For instance, Central State Brewing was a needed niche brewery when they opened, and the niche helped them get established. But they have gone far beyond just being a brett-based brewery. Their community involvement, innovative and far-reaching collaborations, and award-winning taproom make them a player no matter what their age.
Likewise, the list contained Cannon Ball Brewing and praised their food and socially forward-looking choice of location in their write up. But my reason would be the fantastic progress that Mark Swartz made on his new system since his opening just seven short months ago, and most of that while he was holding down another full time job. His beers have gotten so good, so fast that Cannon Ball is much deserving of praise.
Other choices on the list were equally deserving for various reasons. Round Town has much going for it – Jerry Sutherlin is an uber-experienced brewer and Max and Eric “round” out an A+ team to run a brewery and taproom. Look for cans of Round Town out soon. Finally, Deviate Brewing has carved itself a niche that no other brewery in Indianapolis, or Indiana for that matter, chose to fill. They have a good amount of high gravity beer, loads of ABVs and big mouth feels, along with an experimental attitude and a philosophy that serving beer in their taproom is the pinnacle of their experience.
This all being said, I do have to wonder why some other breweries weren’t included on Indianapolis Monthly’s list. Flix Brewhouse in Carmel opened in 2015, well after the previous best new brewery list was published, yet they were not included. Chris Knott and Josh Miller have won no fewer than seventeen medals at state and national competitions since opening the Carmel franchise of Flix, and Chris has now has been named executive brewer for the entire Flix Brewhouse company.
GABF, FOBAB, WBC, NABA, IBC – there are medals out there to be won for indie craft beer. If the medal includes a bottle opener…. all the better. image credit: pintrest
If you want to rank or list top breweries, perhaps medals are one way to go. The competitions are judged by trained individuals not affected by the social experience they are having while in a brewery or craft beer bar. True, not every brewery will choose to enter competitions, will have the money or beer to enter competitions, or will have beer that fits easily into competition categories – but on a purely objective basis, a brewery that has entered competitions and won medals is likely to be a stronger brewery than one which has entered contests and not won medals. Now – can you figure who did choose to enter?
So, with all the medals Flix Brewhouse has won, why were they ignored for the recent best new brewery list? I sat down with Josh and Chris to talk it through. I was miffed at the exclusion, but they talked me down and started dealing in logic. I was of the opinion that Flix as a movie theatre was still overpowering Flix as a brewery in many people’s minds, but Josh thought that that (yes, I wrote “that that”) was an opinion which was being overcome.
Chris thinks that that they were overlooked because they are part of a corporation, with breweries in several cities. Hmmm… Chris has input into the beer for all the Flix brewhouses, and no beer is shipped from Flix to Flix. All the beer served in Carmel is made by Chris and Josh, so if this was the reason for exclusion, it is misplaced criticism. Tin Man has a separate brew house in Kokomo and Sun King has a separate brew house in Fishers – are they corporate brewers?
Yet Nicholas Landers, brewer at Gordon Biersch – Louisville, had exactly the same opinion. The widely held feeling that he is not part of the local beer community is something he says that he must deal with many times each year. Any time a listing of best local breweries or breweries of note comes out, Gordon Biersch is overlooked because they have breweries in many states. The fact that they are completely separate breweries seems to make no difference. I guess Chris at Flix isn’t as paranoid as some might think.
So, is Indianapolis Monthly taking more than beer into account when listing best new breweries. Is it about food? Is it about atmosphere, is it about personal experience on the one or two visits you make? If so, then list them under restaurants or experiences – brewery descriptions should be about beer. Enough said.
It’s true, there is more to a brewery than just beer. Big Lug Canteen’s industrial design matches its innovative beer offerings. Owner Eddie Sahm has put together a great team, including brewer Scott Ellis, designers from CODO Design, and a comforting food menu to build a nice experience. Why it would be left off the list is hard to imagine. A complete list of new breweries would have added only four or five more entries, so perhaps it might have been prudent to include all the breweries in the list.
Keefe Pietri’s and George Garrison’s brewery in Fishers has made significant strides in the year and a half that they have been open. They are a good argument why breweries shouldn’t be judged until they are about a year old. image credit: Heady Hollow Brewing
Black Circle is one of the newest breweries on the block, but just as Round Town had great beer from the get go, so did Dan Gayle’s and Jesse Rice’s establishment just off of Keystone at 46th Street. The gose and its variants have been amazing, but the NE IPA was equally impressive. As equal parts beer bar, brewery, and music venue, Black Circle offers Indianapolis a new type of experience.
If the newness of Black Circle was a factor in the decision, it makes one wonder why an amazing brewery that has the track record of Chilly Water was included on the other end. Just ten days after the best new brewery list was published (June 20), Chilly Water celebrates(ed) its third anniversary (July 1). The differences between a seven-month old brewery like Cannon Ball or Black Circle and a three-year brewpub like Chilly Water make me question why they would be pitted against one another.
All three breweries are fantastic in their way, though Chilly Water’s maturity necessarily means that they have explored beer styles and have done barrel-aged beers that the others have not had a chance to do yet. The fact that Dan and Skip at Chilly Water perform these advanced skills very well is undoubtedly why they were included on the list.
So what have I learned from this free association writing? One, it is hard to compare breweries through time or across styles. Two, people often judge beers and breweries by what I consider to be the wrong parameters. The fact that you don’t like porters is no reason to rate a porter a 1.0 on Untappd. Three, constructive criticism is always better than a rant. Fourth and perhaps most importantly, breweries can be great for very different reasons – look for the good everywhere you go.
No Comments