New Coke was the result of the largest marketing / consumer research project ever. The conclusion of this research was to change Coke's formula. Coca-Cola Chairman Roberto Goizueta claimed that the decision was “one of the easiest we have ever made”. Coca-Cola thought this way for two reasons:
#1 Research was done with a sample of over 200,000 customers.
#2 Coca-Cola’s researchers triangulated the validity of their data with a mixed-method approach. They used focus groups, various surveys, and individual interviews.
All these data & research did the opposite of what they were supposed to. Their large sample size gave them lots of useless data. Data triangulation—which was supposed to safe guard them—did the opposite: it convinced them that their useless data, were useful.
Why does this happen?
In an unnatural system, variance is unbounded. As your data set grows, the unbounded variance grows nonlinearly compared to the valid data. As variance increases, deviations grow larger, and happen more frequently. Spurious relationships grow much faster than authentic ones. The noise becomes the signal.
The New Coke flop is a perfect example of this.
New Coke was the result of the largest marketing / consumer research project ever. The conclusion of this research was to change Coke's formula. Coca-Cola Chairman Roberto Goizueta claimed that the decision was “one of the easiest we have ever made”. Coca-Cola thought this way for two reasons:
#1 Research was done with a sample of over 200,000 customers.
#2 Coca-Cola’s researchers triangulated the validity of their data with a mixed-method approach. They used focus groups, various surveys, and individual interviews.
All these data & research did the opposite of what they were supposed to. Their large sample size gave them lots of useless data. Data triangulation—which was supposed to safe guard them—did the opposite: it convinced them that their useless data, were useful.
Why does this happen?
In an unnatural system, variance is unbounded. As your data set grows, the unbounded variance grows nonlinearly compared to the valid data. As variance increases, deviations grow larger, and happen more frequently. Spurious relationships grow much faster than authentic ones. The noise becomes the signal.