Case study · controlled test with a known answer
Range reviews and launch forecasts usually start from a consumer decision tree: brand first, then flavor, then size, or some other order. We fitted every possible tree to the same conjoint survey of a 60-product category and compared them with Chosŏn, which takes no tree at all. The shoppers were simulated, so the true answers are known.
Result: every tree sent 2.7 to 4 times as many of a cut product's buyers to the wrong place as Chosŏn did (25% to 37% against 9%). And whichever tree was drawn, one of the brand owner's questions came out wrong: either where buyers go, or what a launch adds to profit.
Where a dropped product's buyers go, and where a new product's buyers come from, is the core of ranging and launch decisions. The usual way to bring that into a conjoint simulator is to draw a consumer decision tree first and make the model respect it: shoppers are assumed to switch within the same branch before they leave it. In models this is a nested logit, where each branch of the tree is a "nest".
A tree has one order. If shoppers stay with their flavor and with their brand at the same time, a tree can put only one of them at the top. Chosŏn's correlated-error model doesn't need an order: it estimates from the survey's choices how much shoppers switch within flavor, within brand and within pack, all at once.
| approach | to the same flavor (truth 45%) | to the same brand (truth 36%) | buyers sent to the wrong product |
|---|---|---|---|
| Chosŏn, no tree | 47% | 39% | 9% |
| The usual tree: brand, then flavor, then size | 26% to 29% | 35% to 36% | 32% to 34% |
| The best tree: flavor, then brand, then size | 40% to 42% | 23% to 28% | 25% to 27% |
| The other four trees | 13% to 36% | 12% to 35% | 27% to 37% |
| No tree, standard HB logit | 19% | 17% | 37% |
Each brand launched its new product. How much of the launch's volume came from the brand's own products, and how much profit did it add?
| approach | launch volume from the brand's own products (truth 39%) | error in the launch's added profit | brand cuts 3 of 10: forecast error |
|---|---|---|---|
| Chosŏn, no tree | 41% | 7% | 2.3% |
| The usual tree: brand first | 39% | 10% to 19% | 8% to 10% |
| The best tree: flavor first | 26% to 32% | 18% to 28% too high | 7.5% to 7.7% |
| No tree, standard HB logit | 19% | 47% too high | 6.7% |
The two trees trade places. The brand-first tree gets cannibalization right, because it keeps a brand's buyers inside the brand, but it sends a third of cut products' buyers to the wrong place. The tree the data prefers gets where buyers go closer, but it under-counts how much a launch takes from its own brand, so the launch looks 18% to 28% more profitable than it is. Chosŏn gets both, because it doesn't have to choose.
Some categories do follow a tree. We built one: shoppers who switch stay first with their flavor, then with their brand within the flavor. Three more surveys, the same comparison.
| approach | buyers sent to the wrong product | launch volume from own brand (truth 29%) | brand cuts: forecast error |
|---|---|---|---|
| Chosŏn, no tree | 14% | 30% | 2.4% |
| The right tree: flavor, then brand | 13% to 14% | 26% to 32% | 8.7% to 8.9% |
| The usual tree: brand first | 24% to 25% | 34% to 39% | 10% to 12% |
Drawn correctly, the tree matches Chosŏn on where buyers go. Chosŏn reached the same answer without being told the tree, and its forecasts stayed closer. Drawing the usual tree instead doubled the misplaced buyers.