WHY RESTAURANT REVIEWS NEED TO IGNORE THE CRITICS AND THE MASSES — AND FOLLOW THE DATA


Originally published 02/02/2016

By Bo Peabody
Renzell Founder

As an owner and connoisseur of high-end restaurants, I’m surprised that within the range of restaurant ratings and reviews—from the user-generated to the professional—no one has a clear and consistent methodology rooted in modern data analysis practices. I created a new company, Renzell, to change that.

Restaurants hold a unique place in our city’s and country’s landscape, above and beyond the service of food. They are an art form, a source of cultural inspiration, and a force for economic change.

Critiquing any art form is a delicate balance of context, appreciation, history, and bias, and restaurants are no different. It’s true for professional reviewers at places like The New York Times or The Michelin Guide—for which anonymity, subjectivity, and natural human biases create a whole set of unique challenges—as well as for the casual diners who are enabled to post reviews on Yelp and Google—for which frequency, familiarity, and context make people uniquely unqualified to critique.

Finding a middle ground—between a single subjective review and the screed of the masses—means applying a statistical methodology in how critics are chosen, how data is collected, and how that data is analyzed. It requires curating small group of people, but diverse enough to account for a range of biases and large enough to supply a meaningful data set. And it requires collecting a uniform and cross-sectional data set to create an accurate assessment about how any individual restaurant performs on a regular basis.

Paul Meehl, one of the 20th century’s most prominent American psychology professors, published a short but highly influential book that helped define his prominence in 1970 called Clinical vs. Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. Meehl, who taught at The University of Minnesota (and was, for a time, the President of the American Psychology Association), analyzed the success of predictions by comparing the subjective observations of trained experts versus those made by systematically analyzing a structured set of data points.

Dr. Meehl analyzed everything from predicting criminal recidivism, diagnosing psychological maladies, and the future academic performance of students. In sixty percent of the cases, data analysis produced a more accurate assessment than the predictions of trained experts. (In the other forty percent the comparisons resulted in a draw, which is equivalent to a win for the data as it is more efficient to analyze data than to do subjective analysis.)

It has become almost commonplace these days for companies and industries to use data science and analysis to determine the best approaches and to find the right answers. So why not restaurants? At Renzell, we take exactly this data-driven approach. We take no stock in supposed experts and even less in the wisdom of the crowd. We believe that regular, consistent patrons of a restaurant who provide a consistent data set can create the right feedback we need to produce the most accurate ratings. These patrons are our selected members (we accept applications through renzell.com) and they tell us about each and every experience they have at the restaurants we cover.

Where the traditional reviews and rating system put the bulk of their focus on the food, Renzell’s system takes the totality of a diners experience, from the way they were greeted to the comfort of the seats to the temperature of the food to the presentation of each dish. Renzell looks at eight attributes—cocktails, design, food, hospitality, service, value, vibe, and wine/sake—and asks diners to weigh in about every aspect. Roughly a third of your time in a restaurant is spent eating, the rest (how you were seated, the pacing between courses, your request for a new napkin) is filled with nuanced experiences that are equally as important.

Renzell collects thousands of data points in order to obtain the full scope of the experience, and how it compares to other experiences. Not simply whether the halibut was too cold, the maitre d' was impolite, or the soundtrack was too loud. Restaurants, after all, like any artistic venture, are made up of many attributes, and we want to understand the entirety of the experience. Renzell’s system delivers a holistic assessment of what dining out can and should be for anyone on any night. Not what it was like for one person on one night.

With a uniform data set and a constantly tuning algorithm to analyze the data, Renzell can do for restaurants what Dr. Meehl did for his psychological study: to accurately predict which restaurants consistently deliver on their promise and how likely they are to perform in any given category for the next guest.