Course · 3. Menu-based choice (MBC)
When the menu model helps, and when it doesn't
As with CBC, every claim here comes from tests on simulated data where the truth is known.
Where it helps
- One model at every menu size. Coding every combination of items grows as 2^k, so standard practice codes small menus whole, splits mid-size menus into sections treated as unrelated, and switches method for large menus, with no fixed point at which to switch. Chosŏn uses one model from a handful of items to 30 or more, with every pair's link estimated. Against menus split into sections it made far fewer wrong calls (28 against 146 of 240 at 6-12 items; 50 against 207 at 20-30); against the method used for large menus (each item's choice predicted from the other items' prices) it tied at 20-30 items. Between 6 and 12 items, where an analyst has to choose a method, that comparison is untested. (Partly shown.)
- Questions that span visits. Who comes back for what, and how many different customers a range reaches over a month, against models that treat every visit as a new person: about 0.7-0.8 points closer on next-visit pairs in our tests. Standard hierarchical Bayes logits with a separate model per item also give each person their own tastes and tie with ours on this; ours is closer on same-visit pairs by 0.2-0.3 points. These tests simulated data from our model's own family; on data simulated from a logit, the two tie.
Where it makes little difference
- Cross-price effects. Ours are estimated the standard way; they are not why to choose this model.
- Which set of items reaches the most customers (TURF): a standard method, in chapter 5.
- Single-visit questions (buy rates, same-visit pairs) against the standard multivariate probit: the two share these by construction.
Not yet
- Bundles that were not in the survey (a discount on a combination nobody was shown). Bundles tested as their own menu item are supported.
- Switching when an item is taken off the menu (only switching through prices is measured).
- Menus that hide some items, and respondents who saw different numbers of menus.
Try it in the tutorial: the answer
End of chapter 3. Next: MaxDiff
Tutorial links open the example project in the app; it is free and needs a free account.