The Survey Playbook — Design Surveys That Drive Decisions
By Saroj Jha, AAJ · Pairs with Survey Studio.
Most surveys fail because they start with questions instead of a decision. This playbook installs a 5D framework — Define → Design → Distribute → Dig in → Decide — for building surveys that produce decisions, not just data. It pairs with Survey Studio, AAJ's free survey builder, so you can move from outline to a fielded survey in the same hour.
Who this playbook is for
- Teams about to run a survey but who can't yet name the decision it should inform.
- Marketers whose past surveys produced data, slides, and no action — with response rates sliding.
- Founders and operators who need repeatable templates (CSAT, NPS, PMF, onboarding, churn) they can run again next quarter.
Why most surveys fail
The root cause is structural, not creative. Teams skip the decision step, write double-barreled or leading questions, run them at the wrong audience, then read averages instead of segments. The result is a deck no one acts on and a list that's slower to respond next time. The fix is a short, opinionated operating system applied every time.
The 5D framework
1. Define — start from the decision
Finish the sentence: "After this survey, we will decide ___ based on ___." Name one primary decision, one primary metric (NPS, CSAT, % choosing X, a ranking), and 2–4 secondary questions. If you can't name the decision, don't run the survey yet.
2. Design — audience, sample, questions
Pick the exact audience (prospects, power users, churned accounts — not "everyone"). Set a response target: 30–50 for a directional read, 100–150 per segment for confident comparisons, 300+ for quantitative claims. Keep the survey to 5–7 focused questions; every question maps back to the decision.
3. Distribute — channel and timing
Match channel to audience. Web link for universal reach, email for engaged lists (personalize and state the time cost), SMS for transactional moments, embed for in-product or post-interaction feedback. Time the ask: post-interaction surveys outperform cold blasts. Send one reminder after 3–4 days — it typically adds 30–50% more responses.
4. Dig in — read in order
Response and completion rate first (low completion flags a bad question), then the primary metric vs. benchmark, then segments (slice by the profiling question — the split usually is the story), then open text tagged into 3 themes covering ~80% of comments.
5. Decide — make the call and close the loop
Write the decision in one paragraph, ship the change, and tell respondents what you heard and what you're doing about it. Closing the loop is the single biggest lever on next quarter's response rate.
Metric recipes
| Metric | Question | Scale | How to read it |
|---|---|---|---|
| NPS (loyalty) | How likely are you to recommend us to a friend or colleague? | 1–10 | % Promoters (9–10) − % Detractors (1–6) |
| CSAT (satisfaction) | How satisfied were you with ___? | 1–5 stars | % choosing 4–5 |
| CES (effort) | How easy was it to ___? | 1–7 | Average; lower effort predicts loyalty |
| PMF (Sean Ellis) | How would you feel if you could no longer use this product? | Very / Somewhat / Not disappointed | % "very disappointed" — 40%+ is a strong signal |
Eight survey templates
Customer satisfaction (CSAT) — post-interaction
Rate the experience (1–5), what worked, what to improve, how easy it was to get what you needed (CES 1–7).
Net Promoter (NPS) — relationship
0–10 recommend, main reason for the score, one thing to improve.
Product-market fit (Sean Ellis test)
How disappointed would you be without this product, who benefits most, main benefit, how to improve it for you.
Onboarding feedback (day 7)
Rate onboarding (1–5), where you got stuck (checkboxes), what almost stopped you from getting started.
Churn / cancellation
Main reason for leaving (MC), what could have kept you, likelihood to return (1–5).
Feature prioritization
Which would be most valuable (MC), which would you use weekly (checkboxes), what problem does that solve.
Lead qualification (top-of-funnel)
Biggest marketing challenge, company stage, what success looks like in 90 days, work email.
Event / content feedback
How useful (1–5), most valuable takeaway, what topic to cover next.
Response-rate benchmarks
- Email to an engaged list — 20–30%
- Email to a cold or large list — 5–15%
- In-product / post-interaction prompt — 25–40%+
- SMS at a transactional moment — 20–35%
- Public social link — 1–5%
Seven mistakes to avoid
- No decision behind the survey. Start with the decision sentence.
- Leading or loaded wording. Run every question through the seven question rules.
- Double-barreled questions. Two ideas, one box — uninterpretable. Split them.
- Too long. Completion and quality both fall after ~7 questions.
- Wrong or mixed audience. Representative beats large.
- Reading averages only. Segment, or you'll miss the story.
- No follow-up. Results that don't change anything train people to ignore the next survey.
Quick-start checklist
- Wrote the decision + primary metric.
- Defined the exact audience and a response target.
- 5–7 questions, types match, each tied to the decision.
- Built and previewed it in Survey Studio.
- Picked the right channel; stated the time cost.
- Planned one reminder.
- Set a date to read results and make the call.
- Planned the loop-closing message to respondents.
Put it to work
Open Survey Studio and outline your next survey using the 5D framework. When you're ready to wire responses into a central database with dashboards, book a strategy call.
Related: Analytics & Attribution Playbook — Customer Onboarding & Retention Playbook — Website & CRO Playbook — ← All Playbooks
Frequently Asked Questions
How do I write a survey that actually produces decisions?
Start from the decision, not the questions. Finish the sentence "After this survey, we will decide ___ based on ___." Define one primary decision, one primary metric (NPS, CSAT, a ranking, a %), and 2–4 secondary questions. If you can't name the decision, don't run the survey yet.
How many responses do I need?
For directional B2B reads on a small audience, 30–50 responses are usually enough. For confident segment comparisons, aim for 100–150 per segment. For quantitative claims or public stats, target 300+. Plan for response rate — if email converts ~25% and you need 100 responses, send to ~400 well-matched people.
How long should a survey be?
Five to seven focused questions is the sweet spot. Completion and quality both drop after about seven questions. Every question should map back to your decision — cut everything else.
What's a realistic survey response rate?
It varies by channel: email to an engaged list runs 20–30%, email to a cold list 5–15%, in-product or post-interaction prompts 25–40%+, SMS at a transactional moment 20–35%, and a public social link just 1–5%. Short surveys, a clear ask, good timing, and one reminder are the biggest levers.
When should I use NPS vs CSAT vs PMF?
Use NPS (1–10 recommend score) to track loyalty over time. Use CSAT (1–5 satisfaction) right after an interaction or release. Use the Sean Ellis PMF question ("How would you feel if you could no longer use this product?") to gauge product-market fit — 40%+ "very disappointed" is a strong signal.
Should surveys be anonymous?
Keep them anonymous when you want candor — especially for churn, brand, or employee feedback. When you need to follow up or segment, ask for identifying info only at the end and explain why. Always close the loop with respondents: "You said X, we're doing Y." It raises future response rates.