The Multiple-Choice Minefield

Here’s the blunt truth: most “easy” surveys fail on question quality, option design, sampling, mobile UX, and data/consent hygiene. Keep this list handy.

Top 5 DON’Ts

  1. Don’t write bad questions. No double-barrelled (“How satisfied and how often…”), leading/loaded (“How awful is…”), or vague (“regularly”, “often”) wording.
  2. Don’t botch multiple choice. Overlapping ranges (18–25, 25–34), missing “None/Not applicable/Don’t know”, or forcing single-select when multi-select is correct.
  3. Don’t assume convenience samples generalise. A link on your socials ≠ representative. State the frame and limits; don’t oversell findings.
  4. Don’t ship grid-heavy forms to mobile. Matrix/rank items, long pages, tiny tap targets, and free-text marathons tank completion and data quality.
  5. Don’t hoard personal data. Auto-collecting emails, asking for postcodes/DoB “just in case”, or burying consent. Minimise, disclose purpose, retention, and who sees it.

Top 5 DOs

  1. Decide what you’ll do with each answer before you ask it. If you can’t name the decision it informs, cut it.
  2. Pilot hard. 5–10 target users; watch on real phones. Do quick cognitive interviews (“What does this question mean to you?”). Fix and re-test.
  3. Make options MECE. Mutually exclusive, collectively exhaustive. Always add None, Not applicable, Don’t know, and Other (specify) when relevant. Randomise option/order where appropriate; anchor scales clearly (e.g., 1=Very poor, 5=Excellent).
  4. Use logic to shorten. Branching/required rules so people only see relevant items. One question per screen on mobile; set input types (email, number, date) and add server-side validation.
  5. Plan sampling & fieldwork. Define your population, channels, targets (e.g., n≥100 for a directional read; ≥30 per subgroup), incentives, and monitoring (drop-off points, item nonresponse).

Multiple-choice: common traps (and fixes)

  • Ambiguous categories: “Small/Medium/Large” means nothing—define (e.g., “<10 staff”, “10–49”…).
  • Overlaps/gaps: Use half-open ranges (e.g., 0–4, 5–9…).
  • Wrong select type: If more than one answer can be true, use multi-select and say so.
  • Missing escape hatches: Include None, N/A, Don’t know, Other (specify) where logically needed.
  • Unbalanced scales: Keep equal positives/negatives and neutral centred; avoid mixing frequency and agreement on one scale.

Mobile UX: quick checks

  • One question per page; large hit areas; minimal typing.
  • Avoid matrices, drag-and-drop, and ranking on phones.
  • Keep it short (aim ≤5–7 minutes, ~10–15 well-designed items).
  • Test on iOS/Android, small screens, low bandwidth.
  • Provide progress, clear back/next, and save/resume if long.

Data & operations hygiene

  • Consent/notice: Plain purpose, legal basis, retention, contact; avoid collecting identifiers unless essential.
  • Quality controls: Use dedupe rules (tokens/sign-in if appropriate), soft attention checks sparingly, and post-fieldwork cleaning (outliers, contradictory answers).
  • Version discipline: If you edit mid-field, record the change and analyse versions separately.
  • Accessibility: Labels tied to inputs, good contrast, no colour-only cues, screen-reader friendly.
  • Language: Plain, culture-neutral.

By @ourlocality

@OurLocality Free Community Publishing and News Since 2010 ... get your local news here: https://ourlocality.org/news/

**Feed Your Community Not the Algorithm!