Course · 4. MaxDiff
Reading the MaxDiff results
Overview
The main findings in words: what ranks first and whether that is settled, what ranks last, an item that ranks middling on average but high for a group of people, and how well the fit predicts the held-out sets.
Items
- Share of preference: each respondent's chance of picking the item as best if shown every item, averaged over respondents. It sums to 100%. With 16 items, equal liking would give 6.25% each.
- Rank, with its 95% range. Items with overlapping ranks are not clearly ordered: in the tutorial the top item could rank 1st or 2nd, and the second 1st to 3rd.
- In the top 3: the share of respondents with the item among their 3 highest scores. Two items with the same share of preference can differ here: one liked moderately by everyone, the other loved by some and disliked by others.
- Score: the average centred score (logit scale).
- Picked best / picked worst: raw counts, as a share of the times the item was shown. They need no model.
Simulator
Untick items to see how preference splits among a shortlist: share of preference, and first choice (the share of respondents for whom the item is the best of the shortlist). Shares are within the shortlist: they say which of these people prefer, not how many would want any.
Reach (TURF)
The TURF search finds the set of items that reaches the most people. Since MaxDiff does not say who wants an item at all, Chosŏn counts a person as reached when one of the set's items is among their top m of all items (3 by default). This rule is a convention: change m and the reach changes. Read it as "the set that covers the most people's favourites", and compare sets with each other, not with a market size.
Diagnostics
The holdout check scores each respondent's held-out last set: how often the item the model rates highest is the one picked best, and the lowest the one picked worst (from the rest). Both models are shown, with the difference and its 95% range, next to chance. One set per respondent is little data, so the ranges are wide.
Try it in the tutorial: the items
Next: What to expect
Tutorial links open the example project in the app; it is free and needs a free account.