Understanding the Bayesian average is one thing. Understanding how to calculate it is something different. Understanding how to apply it is something in a whole other league. So here’s a quick and simple case study regarding product feedback and comparing aggregate product ratings using Bayesian statistics.
Situation
Your widget factory produces three different widgets and targets two different segments of the market. For the sake of simplicity, we won’t go in to the actual definitions of either your product or the market, just assume that they are well suited for one another. In order to cope with changing market conditions, you’re asked to slim down production and cut one of these three widgets from the market.
This is an important decision because the company will need to support the construction of a new facility based on the sales of these two remaining widgets. Cut the wrong product, and you might not make it to the new production season.
Vital Statistics
Here’s a table showing how each market segment rates your products. Assume we’re working on a scale from 0 (I wouldn’t use this if you paid me to) to 5 (I can’t live without this product):
| Product A | Product B | Product C | |
| Segment 1 | 4.2 | 3.5 | 1.0 |
| Segment 2 | 3.7 | 4.0 | 4.8 |
| Segment 3 | 2.4 | 4.2 | 2.1 |
Also, you should know that Segment 1 has 5,000 people, Segment 2 has 25,000, and Segment 3 has 1,000 people.
First Glance
On a first glance, you can easily determine the simple average for each product (average of all three ratings independent of the segment size). Product A has a rating of 3.43, Product B has a rating of 3.90, and Product C has a rating of 2.63. According to the simple average you should drop Product C from your offering for the time being and focus more on Products A and B.
Unfortunately, Product C was favored the most among Segment 2. Remember that Segment 2 was the largest segment of all three. Let’s recalculate using a weighted average. In this case, Product A now has a rating of 3.73, Product B has a rating of 3.93, and Product C has a rating of 4.10. Now our numbers are telling us to drop Product A and focus instead on marketing Products B and C. This is an entirely different situation!
Bayesian Average
Using Bayesian statistics, we look at the entire segment at once. This moves beyond merely taking an average rating for each product and assumes that consumers are capable of rating products in relation to one another as well. It’s still an imperfect solution, but if we assume someone who likes Product C says so without thinking about Product A it would be naive. In the real world, many consumers will be aware of your other product offerings and will rate in context rather than in isolation.
To determine the the true product rating for each product we first need the Bayesian average for each segment, adjusting our product ratings so they are comparable. Here is the same table adjusted using Bayesian mathematics:
| Product A | Product B | Product C | |
| Segment 1 | 3.23 | 3.05 | 2.43 |
| Segment 2 | 4.05 | 4.13 | 4.33 |
| Segment 3 | 2.78 | 3.23 | 2.70 |
Now it’s a matter of re-applying our weighted average to determine the properly adjusted rating for each product category. In this case, the rating for Product A is 4.01, the rating for Product B is 3.93, and the rating for Product C is 3.97. Now that we’ve looked at a more holistic overview, we realize it would be best to drop Product B from our offering and focus instead on Products A and C.
Summary
We looked at three different analytical tools that gave us three very different answers. First we considered a simple average, but determined this was insufficient because it failed to take segment size into account. Second we considered a weighted average, but determined this data was inaccurate because it failed to account for consumer’s knowledge of the entire product line.
Finally we considered a weighted average of the same ratings with a Bayesian filter. This method took both segment size and consumer knowledge into account, giving us the most accurate means possible with which to judge our three different products. According to this model, Product B is the least popular (although not by much). Dropping Product B from our line offering would upset our bottom line the least.
Keep in mind that this is an academic treatment of the situation. In any real world scenario, there would be considerable risk with dropping any product line. This is but on analytical tool a savvy management team could use to determine long-term fiscal and brand equity impacts of the decision.
