Measuring Success
The Content Scorecard helps you decide whether a piece of player education is ready to publish or needs revision. This module explains the NPS question, five semantic pairs, and the three-survey decision tree. You will learn to read the results and identify what to improve.
Operator note: Replace
{{PROGRAM_NAME}}with your program name throughout. The NPS question wording, the five semantic pairs, the scoring thresholds, and the ship-or-revise decision tree are part of the brand measurement system and should not be modified — they’re what keeps scorecards comparable across operators. What you do customize is how you deploy the survey (embed widget, Google Forms, Typeform) to fit your stack. Narrator scripts should be recorded in the Playbook Tier 1 voice.
Program: Brand Foundations | Duration: ~20 min | Prerequisites: Customer Journey
Quick-scan index
| Section | What it covers | Time |
|---|---|---|
| Learning objectives | What you’ll know after this module | — |
| Section 1: The Content Scorecard | NPS anchor metric and five semantic pairs | ~10 min |
| Section 2: Running a Scorecard | Three survey instruments and the decision tree | ~7 min |
| Module test | 5 graded questions, 80% to pass | ~3 min |
| Key takeaways | Summary | — |
Learning objectives
After completing this module, you will be able to:
- Build a Content Scorecard and distinguish reader feedback from evidence of behavior change
- Pick the right survey instrument — Pulse, Full, or Recall — for the stage you’re testing
- Run the ship-or-revise decision tree against a real scorecard, including the edge cases
Section 1: The Content Scorecard
Reading
The Content Scorecard collects readers’ views of a piece. Use it to decide what to investigate or revise. It does not establish that the piece changed behavior or reduced harm.
The NPS question asks: “How likely would you be to recommend this material to a friend who gambles?” Responses run from 0 to 10. Calculate the percentage answering 9–10 (Promoters), then subtract the percentage answering 0–6 (Detractors). Responses of 7–8 are Passives; they remain in the denominator.
For example, with 100 completed responses, 55 Promoters, 25 Passives and 20 Detractors give NPS +35. The result measures reported recommendation intent, not actual sharing.
The course uses these internal planning bands:
| NPS | Course interpretation |
|---|---|
| 50 or above | Exceptional within this planning framework |
| 30–49 | Strong |
| 0–29 | Below the publication target; investigate |
| Below 0 | More Detractors than Promoters; revise and retest |
These are not verified industry benchmarks or proof of effectiveness. Record who responded, the sample size, recruitment method and exposure to the piece before comparing results.
Five semantic pairs describe perceived qualities. Respondents use a 1–7 scale, with the more favorable term at 7.
| Pair | What the response describes |
|---|---|
| Forgettable–Memorable | How memorable the piece feels now; test later recall separately |
| Boring–Engaging | Perceived interest |
| Preachy–Respectful | Perceived respect and tone |
| Confusing–Clear | Perceived clarity; test actual understanding separately |
| Generic–Made for me | Perceived relevance |
A pair average of 5.5 or above is strong in this framework; 4.0–5.4 meets the working threshold. Below 4.0 calls for investigation; below 3.0 is a serious concern. Read the comments and the spread of responses as well as the average.
The Brand DNA alarm overrides the other scores. If Preachy–Respectful is below 4.0, hold publication, review the comments and revise the voice. A high NPS does not cancel that concern. This is a Playbook review rule, not a validated diagnostic threshold.
Narrator Script
Narrator: [transition slide] You A/B test email subject lines. You measure campaign net promoter score. Your {{PROGRAM_NAME}} material should get the same treatment. Because Tier 1 material is a product, not a pamphlet.
[pause 3s] {{PROGRAM_NAME}} material competes for the same player attention as your sportsbook promos, casino loyalty emails, and brand campaigns. If you would measure those with net promoter score, measure this with net promoter score. The quality bar is commercial-grade, not good enough for compliance.
[show nps benchmark scale] The anchor metric is one question. How likely would you recommend this material to a friend who gambles. Scored zero to ten, calculated as percent Promoters, which are nines and tens, minus percent Detractors, which are zeros through sixes. Here are the benchmarks. Fifty or above is Exceptional. Players are sharing the material organically. Thirty to forty-nine is Strong, on par with gaming and sportsbook apps. Zero to twenty-nine is positive but below target. Run diagnostics. Below zero means more detractors than promoters. Revise before shipping.
For perspective, streaming apps like Netflix and Spotify score fifty to seventy. Gaming and sportsbook apps score thirty to fifty. Traditional compliance material is unmeasured. Nobody even asks players what they think. {{PROGRAM_NAME}} targets the same range as the products it sits alongside.
[show five-pairs grid] Net promoter score tells you whether material is working. The semantic pairs tell you why, or why not. There are five pairs, each scored one to seven. Forgettable versus Memorable measures stickiness. Boring versus Engaging measures entertainment value. Preachy versus Respectful measures voice register, and this is the critical pair. Confusing versus Clear measures comprehension. Generic versus Made for me measures cultural resonance.
[pause 5s] If Preachy versus Respectful scores below four point zero, stop. Your material may be lecturing players. This is the Brand DNA alarm. It overrides an otherwise strong scorecard. Revise against the voice principles from Voice & Tone before shipping.
[pause 3s] Hold onto these two pieces — the net promoter score bands and the five-pair thresholds. In the next section they snap together into a single decision tree, and you’ll run that tree on six real scorecards yourself.
Section 2: Running a Scorecard
Reading
Choose an instrument for the question you need to answer.
| Instrument | What it asks | When to use it |
|---|---|---|
| Pulse | NPS, Forgettable–Memorable and one open-ended improvement question | Initial feedback after a launch or bounded pilot |
| Full | NPS, all five semantic pairs and open-ended feedback | Diagnose a concern or review an important piece in more depth |
| Recall | Free recall, knowledge questions and self-reported follow-through | Follow up 7–14 days after exposure |
The working response targets are 30 for Pulse or Full, 30 per variant for an exploratory A/B test, and 25 for Recall. These are planning targets, not sample-size calculations or guarantees of reliability. Recruitment, missing responses, variation and the decision’s stakes affect what you can infer. Self-reported follow-through does not prove that the material caused a behavior change.
Apply the decision rules to the measures you actually collected:
- Revise and retest if NPS is below 0. Also hold and revise the voice if a measured Preachy–Respectful score is below 4.0; below 3.0 indicates a particularly serious concern.
- Diagnose if NPS is 0–29 or another measured pair is below 4.0. Use the Full survey and comments to decide what to change.
- Ship or continue the tested use if NPS is at least 30 and every pair you measured is at least 4.0, with no other unresolved content, accessibility or safety concern. Schedule Recall for important material.
A Pulse only measures Memorable. It cannot certify the other four pairs. If you already have unfavorable Full results, a later or higher NPS does not erase them.
Two worked examples: A fictional Pulse with 100 responses gives NPS +38 and Memorable 5.8. It meets the course’s initial thresholds, so continue the tested use and arrange a follow-up. If another Pulse gives NPS +14 and Memorable 4.2, run Full diagnostics. A subsequent Preachy–Respectful result of 3.6 triggers a hold and voice revision; a Generic–Made for me result of 3.4 also warrants closer attention to relevance. The scores suggest where to look; comments and inspection identify the actual changes.
Keep an identifiable version of the piece with each result. Record the instrument, exposure date, response count, recruitment and any missing values. Recheck high-traffic material when the audience, product or wording changes; a quarterly review is a useful planning cadence.
Allow roughly 3–4 hours across two weeks as an initial team estimate, then use your own experience to plan. Survey setup, recruitment and analysis can take longer. Use the survey widget or your approved survey platform, and store comparable results in one place under your privacy and retention policies.
Narrator Script
Narrator: [transition slide] Three surveys. One decision tree. The entire process takes three to four hours spread across two weeks.
[show three-instruments card] The Pulse is your default. Three items, under a minute, target thirty responses. It gives you the net promoter score, the Forgettable versus Memorable pair, and one open-ended question. Send it after every launch.
The Full is your diagnostic. Seven items, about two minutes, target thirty responses or sixty if you are A/B testing two versions. It adds all five semantic pairs so you can pinpoint which dimension needs work. Run it when the Pulse flags a problem.
The Recall goes out seven to fourteen days after players saw the material. It tests three things the other surveys cannot measure. Free recall, which is simply asking what do you remember. True-or-false knowledge checks. And behavioral follow-through, which asks did you actually do anything differently. Material that scores well on Pulse but fails on Recall is engaging but not memorable.
[show decision tree] The decision tree is straightforward. Step one, send the Pulse. Step two-A, if net promoter score is thirty or above and all pairs are four point zero or above, ship it. Schedule the Recall survey in seven to fourteen days. Step two-B, if net promoter score is zero to twenty-nine or any pair is below four, run the Full to diagnose. Step two-C, if net promoter score is below zero or if Preachy versus Respectful is below three point zero, stop shipping and revise.
[pause 3s] Sample sizes matter. Target thirty responses for reliable results. If you are comparing two versions of the same material, you need sixty, which is thirty per variant.
[show deploy-options card] {{PROGRAM_NAME}} ships a ready-to-use survey widget that auto-calculates scores with color-coded results. You can embed it as an iframe in about five minutes. Or use Google Forms in about ten minutes with manual scoring. Or use Typeform or SurveyMonkey in about fifteen minutes for a polished experience. Whichever you pick, make it the team default so every scorecard stays comparable.
[transition to exercise] Now put the tree to work. Six real scorecards are waiting below — for each one, make the call: ship, diagnose, or revise. The thresholds you just learned decide every one of them, no gut feel allowed. Work through them, then continue.
Exercise: Ship, Diagnose, or Revise?
Type: scenario
Instructions: Ship, Diagnose or Revise? Apply the course’s internal thresholds to each fictional scorecard. Unasked pairs are unknown. Scores describe this sample, not behavior change.
| Item | Correct Category | Feedback Correct | Feedback Incorrect |
|---|---|---|---|
| What is the next step for this Pulse result? | Ship | NPS +35 and Memorable 5.7 meet the initial thresholds. Continue the tested use, keep other release checks in place and schedule Recall. | NPS +35 clears 30 and Memorable 5.7, the only pair asked, clears 4.0, so the call is Ship. |
| What is the next step for this Pulse result? | Diagnose | NPS +21 is below the target of 30. Run the Full survey and inspect comments before deciding what to revise. | A positive NPS is not enough for this course’s target. The 0–29 band calls for Full diagnostics. |
| What is the next step for this Pulse result? | Revise | NPS −8 is below 0: more Detractors than Promoters. Revise and retest; comments and inspection show what to change. | The negative NPS triggers revision in this framework. Memorable 5.1 does not override it. |
| What is the next step for this Full result? | Revise | Preachy–Respectful 2.8 is below the hold-and-revise threshold of 4.0, despite NPS +44. Review the comments and revise the voice. | Preachy–Respectful 2.8 is below 4.0, so hold and revise; NPS +44 can’t override it. The comments, not the average, show what to fix. |
| What is the next step for this Pulse result? | Diagnose | Memorable 3.8 is below 4.0, though NPS +33 clears 30. Run the Full survey. Memorable is a perception score, not a delayed recall test. | Ship needs every measured pair to reach 4.0 as well as NPS 30. Memorable 3.8 calls for diagnostics. |
| What is the next step for this Pulse result? | Ship | NPS +52 and Memorable 6.1 meet the initial thresholds. The NPS records recommendation intent, not observed sharing. Schedule the follow-up. | NPS +52 clears 30 and Memorable 6.1, the only pair asked, clears 4.0, so the call is Ship. |
Module Test
Narrator: [transition slide] Time to test what you have learned. Five questions on the Scorecard framework — the net promoter score bands, the five semantic pairs, the three instruments, and the decision tree. You need eighty percent to mark this module complete. Take your time. If you do not pass on the first try, review the material and retake the test.
Question 1
Assesses: Learning objective 1
Stem: What does the NPS question in the Content Scorecard ask players?
| Option | Text |
|---|---|
| A | ”Did you find this material useful?” |
| B | ”How likely would you be to recommend this material to a friend who gambles?” |
| C | ”Would you read this material again?” |
| D | ”How satisfied are you with our responsible gambling program?” |
Correct: B
Explanation: NPS is a recommendation question, the same metric commercial products use. Tier 1 material is a product too.
Source: Research Methodology
Question 2
Assesses: Learning objective 3
Stem: What NPS threshold does this course use for the initial Ship decision, provided every measured pair also meets its threshold and no other concern remains?
| Option | Text |
|---|---|
| A | 10 |
| B | 20 |
| C | 30 |
| D | 50 |
Correct: C
Explanation: NPS 30 is the course’s initial publication target, where its Strong band starts. These are internal planning rules, not verified industry benchmarks.
Source: Research Methodology
Question 3
Assesses: Learning objective 1
Stem: Which semantic pair triggers the “Brand DNA alarm” when it scores below 4.0?
| Option | Text |
|---|---|
| A | Forgettable — Memorable |
| B | Boring — Engaging |
| C | Preachy — Respectful |
| D | Generic — Made for me |
Correct: C
Explanation: Under this framework, Preachy–Respectful below 4.0 triggers a hold and voice revision. Comments and wording, not the average, show what to fix.
Source: Brand Personality
Question 4
Assesses: Learning objective 3
Stem: What is the next step for this Pulse scorecard?
| Option | Text |
|---|---|
| A | Ship it — 22 is a positive NPS |
| B | Run the Full survey to diagnose which dimensions need work |
| C | Send the Recall survey to check retention |
| D | Revise the material immediately — this is a failing score |
Correct: B
Explanation: NPS +22 is below the course’s +30 threshold, not negative: run the Full survey. Scores show where to look, not what caused a response.
Source: Research Methodology
Question 5
Assesses: Learning objective 2
Stem: What does the Recall survey measure that Pulse and Full surveys do not?
| Option | Text |
|---|---|
| A | Whether players enjoyed the material in the moment |
| B | Which semantic pairs scored lowest |
| C | Delayed recall, knowledge and self-reported follow-through 7–14 days later |
| D | How many players completed the survey |
Correct: C
Explanation: Pulse and Full record current reactions; only Recall checks memory and knowledge later. Self-reported follow-through doesn’t prove the material caused it.
Source: Research Methodology
Key Takeaways
- The scorecard measures reported reactions to content. It does not establish changes in knowledge, behavior or harm.
- Calculate NPS as the percentage of Promoters minus the percentage of Detractors, keeping Passives in the denominator.
- The five semantic pairs show where to investigate. Read the comments and inspect the material before deciding what to change.
- A measured Preachy–Respectful score below 4.0 requires a hold and revision, regardless of NPS.
- Use Pulse for initial feedback, Full for closer diagnosis and Recall for a later check. A Pulse cannot certify pairs it did not measure.
- Apply the course’s internal thresholds alongside other publication checks: NPS 30+ and measured pairs 4.0+ support an initial Ship decision; NPS 0–29 or another low pair calls for diagnosis; negative NPS or Respectful below 4.0 requires revision.
- Plan recruitment and analysis for the decision you need to make. A fixed response count does not guarantee reliable or representative findings.
Source basis
You don’t need to read these but they’re published research related to the scorecard — in case a stakeholder asks.
- Osgood, C.E., Suci, G.J., & Tannenbaum, P.H. (1957). The Measurement of Meaning. University of Illinois Press. Why it’s here: the original semantic differential methodology — the academic basis for using bipolar 1-7 pairs (Preachy-Respectful, Boring-Engaging, etc.) to measure perceived qualities of a stimulus.
- Reichheld, F.F. (2003). The one number you need to grow. Harvard Business Review, 81(12), 46-54. Why it’s here: the foundational NPS paper — the methodology and the recommendation-to-a-friend question wording the scorecard borrows directly.
- Dillard, J.P., Shen, L., & Vail, R.G. (2007). Does perceived message effectiveness cause persuasion or vice versa? 17 consistent answers. Human Communication Research, 33(4), 467-488. Why it’s here: research on the relationship between perceived message effectiveness and persuasion. It does not validate these five custom pairs or establish that this scorecard predicts gambling behavior or harm outcomes.
References
| Resource | What to use it for |
|---|---|
| Research Methodology | Full NPS and semantic pair methodology, survey templates |
| Brand Personality | Voice principles for revising Preachy-Respectful failures |
| Survey Widget | Embeddable scorecard survey with auto-scoring |