Home / Module 3
Module 3

User Research & Evidence

📖 2 Lessons⏱ 60–90 min 🧪 Lab: Persona Building with Evidence Scoring✅ Quiz: 6 questions

Learning Objectives

Lesson 3.1
Evidence Quality Framework

Not all evidence is equal. PMs must distinguish strong signals from weak ones before building on them.

Evidence TypeStrengthExample
Behavioral data🟢 Strong"62% of users drop off at step 3"
Observed behavior in interview🟢 Strong"I watched her struggle with the API key field for 10 minutes"
Self-reported behavior🟡 Medium"I usually spend 3 hours on setup"
Stated preference🔴 Weak"I wish there was a better way"
Feature request🔴 Weakest"Can you add dark mode?"
⚠️
Rule: Never build on feature requests alone. Always trace back to the underlying behavior or problem. A feature request is a proposed solution — you need to understand the problem before evaluating any solution.
Lesson 3.2
Building Personas from Evidence

A persona is a research-based archetype representing a user segment.

✓
Good personas are:
  • Grounded in observed behavior, not assumptions
  • Specific about goals and frustrations
  • Actionable (a designer can make decisions from them)
✗
Bad personas are:
  • Demographic descriptions with no behavioral insight
  • So generic they apply to anyone
  • Based on assumptions rather than research

What Every Evidence-Based Persona Must Include

Lab 3

Persona Building with Evidence Scoring

Scenario: Using the 5 interviews from Lab 2 plus the quantitative data below, build evidence-based personas for FlowScale.

Quantitative Data — FlowScale Analytics (Last 30 Days)

New signups: 1,247
Completed onboarding (all 5 steps): 473 (38%)
Drop-off by step:
  - Step 1 (Create account): 95% completion
  - Step 2 (Verify email): 88% completion
  - Step 3 (Connect data source): 41% completion ← MAJOR DROP
  - Step 4 (Configure workspace): 78% (of those who reach it)
  - Step 5 (Create first workflow): 67% (of those who reach it)

Time to first workflow (completers): avg 47 minutes
Users who create first workflow within 24h: 3x higher 30-day retention
Support tickets (onboarding): 312/month, top issue: "can't connect data source"
Seats purchased vs active: avg 200 purchased, 60 active (70% inactive)
Step 1

Create prompts/persona-builder.md with these requirements: 2–3 personas, each citing interview quote or data point for every frustration, flagging any claim as [ASSUMPTION — needs validation] if evidence is insufficient, and rating evidence strength (Strong/Medium/Weak).

Step 2 & 3

Run the persona builder with all data loaded. Then complete an evidence audit for each persona:

## Persona: [Name]
| Claim | Evidence Source | Strength | Gap? |
|-------|----------------|----------|------|
| "Struggles with API key setup" | Lisa: "I don't know what an API key is" | Strong | No |
| "Most users are non-technical" | Only 1 of 5 interviews states this | Weak | Yes — need broader survey |
Step 4

Write a research gap memo at memory/research-gaps/W03-gaps.md. Identify at least 3 gaps. For each: what you need to know, current evidence, research needed, and priority.

Deliverables

  • Persona builder prompt
  • Raw AI output (personas)
  • Evidence scorecard for each persona
  • Research gap memo (minimum 3 gaps identified)

How to Verify Completion

  • Each persona has at least 3 frustrations, each with an evidence source and strength rating
  • At least one frustration is marked [ASSUMPTION] if it lacks strong evidence
  • Your evidence scorecard explicitly checks quantitative vs qualitative evidence balance
  • Research gap memo identifies specific data needed (not "more research") — e.g., "survey to determine % of non-technical users"
Done when: A new PM joining the team could read your personas and research gap memo to understand both what we know about users and what critical questions remain unanswered.
Quiz

Module 3 — Knowledge Check

0 / 6 answered
Question 1 of 6
A data point says "62% of users drop off at step 3." What evidence strength is this?
A Strong — it's behavioral data from product analytics
B Medium — it's a number but doesn't explain the cause
C Weak — quantitative data is less reliable than interviews
D Unknown — we need more context
Correct. Behavioral data is the strongest form of evidence because it reflects what users actually do, not what they say they do. Analytics showing 62% drop-off at step 3 is a strong signal. It doesn't explain the cause — that's what interviews add — but as evidence of a problem, it's strong.
Question 2 of 6
What makes a persona "actionable" as opposed to just descriptive?
A It includes a photo and a name to make it memorable
B It lists the user's demographics in detail
C It enables a designer or engineer to make a product decision without needing to ask "what exactly do I build?"
D It is approved by the sales team who talk to customers regularly
Correct. Actionability is the test: can the team use this persona to make decisions? A persona that says "Sarah is frustrated with onboarding" is not actionable. "Sarah cannot connect data sources without IT help, causing 2-day delays, which causes her to build manual workarounds" — that's actionable.
Question 3 of 6
When should a persona claim be marked [ASSUMPTION — needs validation]?
A When the PM disagrees with the claim
B When the claim is not supported by direct quotes or data — it's inferred or assumed
C When only one interview supports it
D When the claim seems too obvious to need validation
Correct. [ASSUMPTION — needs validation] flags claims that go beyond the available evidence. Even a claim from one interview can be factual if it's a direct quote — it's just low-confidence. A claim with no evidence at all is an assumption and must be explicitly flagged to prevent it from being treated as fact.
Question 4 of 6
FlowScale analytics show 200 seats purchased with only 60 active (70% inactive). Which persona does this data point most directly support?
A Lisa (Operations Coordinator) — she was confused by API keys
B Marcus (VP Operations) — he specifically mentioned 200 seats with only 60 active
C Priya (Senior Ops Manager) — she built workflows from scratch
D James (Operations Director) — he needs permissions
Correct. Marcus explicitly said "We pay for 200 seats but only 60 are active" — the analytics data directly validates his self-reported numbers, upgrading his claim from Medium (self-reported) to Strong (confirmed by behavioral data).
Question 5 of 6
What is the purpose of a research gap memo?
A To summarize everything the research found
B To explain why more budget is needed for user research
C To document which customers should be interviewed next
D To document what critical questions remain unanswered and what specific research would answer them, with priority
Correct. A research gap memo is about what you don't know yet — not a summary of what you do know. It should specify each gap, why it matters, what evidence is currently available (even if weak), and what research would close it.
Question 6 of 6
Why should personas not represent less than 10% of the user base?
A Because 10% is a statistical threshold for significance
B Because building for very small segments consumes engineering capacity without meaningful user impact on the main business metrics
C Because users in smaller segments are typically less valuable customers
D Because AI cannot build reliable personas for small segments
Correct. Personas guide product decisions. If you build a feature for a persona representing 5% of users, you're using significant engineering capacity for minimal activation/retention impact. Personas should represent segments large enough to meaningfully move the metrics you care about.
Module 3 Score