A reproducible pricing experiments lab: ready‑to‑run A/B templates for trials, offers and promotions

Most studios run pricing "tests" that aren't tests at all. You drop your trial from $99 to $49 for a month, enrollments look decent, so you keep it. Next quarter someone convinces you to try a free week instead, that also seems fine, so now you're running that. Six months later nobody can actually tell you which offer converts better, retains longer, or attracts the students who stick around past belt three.

The problem isn't that owners don't experiment. It's that the experiments aren't reproducible. No fixed measurement window, no guardrail on what counts as a win, no threshold that separates a real signal from a good week. So you end up making permanent pricing decisions off noise.

This is a walkthrough of how to set up a small, boring, repeatable pricing experiments lab for a martial arts studio — the kind you can run every quarter without a data scientist. We'll cover the exact templates (trial length, discount depth, onboarding touch), how long to run each one, the guardrails that stop you from fooling yourself, and the statistical thresholds that actually apply when your monthly sign-up volume is small.

Why studio pricing tests usually fail

The core issue is sample size. A martial arts studio isn't Netflix. You might get 30–60 new leads a month, and only a fraction convert. When someone runs a two-week test on a $30 price change and sees "12 sign-ups vs 9," they treat it as a result. It's not. With numbers that small, a difference of three sign-ups falls easily inside random variation. Run the same test again and it might flip.

The second issue is contaminated comparisons. A studio changes the trial price and redesigns the intro email and the head instructor happens to be teaching more that month. When enrollments go up, which lever moved the needle? Nobody knows, because you changed three things at once.

The third — and this one's sneaky — is measuring the wrong window. A lot of owners judge an offer by how many people sign the trial. But the offer that pulls the most trials is very often the one that pulls the worst students. A free two-week trial fills your mat with people who never intended to pay. If your success metric stops at "trials started," you'll happily pick the offer that quietly wrecks your retention.

A real pricing experiment for a martial arts studio has to account for all three: small volume, single-variable changes, and a measurement window that runs long enough to see who actually pays and stays.

The three things worth testing (and what to leave alone)

You have limited experiment slots per year because each one needs weeks to run. Don't waste them on cosmetic stuff. Three levers tend to move outcomes enough to be worth a formal test:

1. 1
   Trial length. 7 days vs 14 days vs 30 days vs a fixed "4-class" trial. Length changes both who signs up and how committed they feel by the time you ask for money.
2. 2
   Discount depth. The size and shape of your intro offer — $99/first month vs 50% off two months vs a flat $149 "6-week starter" package. Depth affects conversion but also affects how anchored people are to the discounted price when the real rate kicks in.
3. 3
   Onboarding touch. How much human contact happens during the trial — automated reminders only, vs one instructor check-in call, vs a structured first-30-days sequence. This is the highest-leverage variable most studios never test, and it interacts heavily with the other two.

Leave the following alone as formal experiments: your logo, website colors, small copy tweaks in emails. These matter, but the volume required to measure them properly is far beyond what your studio produces. Test them informally if you want, but don't burn a real experiment slot on them.

The core template: one variable, two arms, fixed window

Every experiment in this lab uses the same skeleton so results stay comparable quarter to quarter. It sounds almost too simple, but the structure is the whole point — without it, you're just watching numbers and telling yourself a story.

1. 1
   Pick ONE variable. Trial length, discount depth, or onboarding touch. Not two.
2. 2
   Define exactly two arms — the control (what you run now) and one challenger. No three-way tests until you have the volume; they split your already-thin numbers into useless slices.
3. 3
   Assign leads by alternation, not by choice. New lead one goes to control, lead two to challenger, lead three to control, and so on. Never let staff pick which offer to "pitch" — that quietly stacks the deck.
4. 4
   Set the enrollment window before you start. You collect new leads for a fixed number of weeks (usually 6–8), then stop adding people.
5. 5
   Set the measurement window separately. You then wait until every enrolled person has hit the same downstream checkpoint — typically paid their first full month.
6. 6
   Only read the result at the end. No peeking and stopping early because one arm "looks good." Early peeking is how studios convince themselves of results that aren't there.

The reason for alternating assignment matters more than it looks. If you let the front desk offer the free trial to warmer leads and the paid trial to price-sensitive ones, your challenger arm gets an easier population and wins for reasons that have nothing to do with the offer.

Alternation removes that staff-driven selection bias by construction, which is why it's a non-negotiable part of the lab.

Trial length template

Trial length is worth testing early because the difference between a time-based free trial and a class-count trial is pretty significant in practice — not just in conversion rates, but in the type of student who shows up.

The pattern with trial length: time-based free trials get more starts but more ghosts — people who take one class in week one and disappear. A fixed-class trial forces attendance, which builds the habit that actually predicts conversion. Studios that switch to a class-count trial usually see slightly fewer trial starts but higher conversion, because the students self-select for commitment.

Your guardrail exists so you don't celebrate a higher conversion rate that comes from a collapsed number of trials. If the class-count trial converts at 45% but only 8 people started it, versus 30% on 40 free trials, the free trial still produced more paying members. Rate alone lies. Always read rate and raw count together.

Discount depth template

Depth is where studios overspend by reflex. The instinct is "go deeper, get more people." But deeper discounts often attract exactly the students most likely to churn the moment they see the full rate, and they anchor everyone to a number far below your real price.

1. 1
   Control

   $99 first month, then standard rate.

2. 2
   Challenger

   Standard first month with a $75 "gear + belt" bonus included.

Notice the challenger doesn't discount the membership at all — it adds value instead. This is worth testing because value-adds don't anchor people to a low price the way a raw discount does. When their second month hits full price, there's no "wait, it doubled" shock, because the price never dropped in the first place.

Run this over an 8-week enrollment window, then measure conversion to a second full-price month — not the first. First-month retention on a discount tells you almost nothing. The truth shows up at month two, when the real price lands.

Onboarding touch template

This is the one most likely to produce a genuine, durable win, and it's usually cheaper than any discount. The variable is human contact during the trial.

1. 1
   Control

   automated reminder texts only.

2. 2
   Challenger

   same automation plus one scheduled 5-minute instructor check-in after the student's second class.

“

Make the check-in a scheduled calendar event so it happens reliably rather than depending on memory.

That single call does something no email does — it makes a beginner feel seen before they've decided whether martial arts is "for them." The check-in also surfaces quiet quitters: the student who's embarrassed they can't remember the warm-up sequence, or who felt lost in a class above their level. You catch that in a two-minute conversation and save a member who'd have silently disappeared.

If you're formalizing this, it pairs directly with the sequence in the first-30-days onboarding workflow, since the check-in call becomes a fixed step rather than something that happens when an instructor remembers. The whole point of an experiment is that the touch happens reliably for the challenger arm — one skipped call per week and you're no longer measuring what you think you're measuring.

Measurement windows: enrollment vs outcome

These are two different clocks and mixing them up ruins results.

The enrollment window is how long you accept new people into the test. Keep it fixed — 6 to 8 weeks for most studios. This gives each arm a comparable batch of leads under comparable seasonal conditions.

The outcome window is how long you wait after the last person enrolls before you read results. For a trial conversion test, that's until the last enrolled trial has ended and had a payment decision. For a retention test, it's until everyone has reached day 60 or your chosen mark.

Here's a simple visual of how the two clocks relate.

A worked example. Say you enroll leads from March 1 to April 26 (8 weeks). The last person to start a 14-day trial does so on April 26, so their trial ends around May 10, and their second-month payment decision lands mid-June. You don't read the experiment until mid-June. If you'd peeked in early May, half your challenger arm hadn't even finished their trial yet — you'd have been comparing incomplete data against incomplete data.

Studios rush this constantly. The pressure to "just make a call" is real, but reading a pricing experiment early is worse than running no experiment, because it gives you false confidence in a wrong answer.

Guardrails: the metrics that override a "win"

A guardrail is a metric that, if it moves the wrong way, cancels the win no matter how good the primary result looks. Set these before the test starts — not after you see the numbers.

1. 1
   Trial volume floor

   if the challenger cuts trial starts by more than ~25%, a higher conversion rate doesn't count as a win.

2. 2
   Second-month retention

   if the challenger converts more first months but loses more people at month two, the offer failed — it just moved the churn downstream.

3. 3
   Refund / cancellation spikes

   if either arm produces a noticeable jump in early cancellations or refund requests, flag it, because you may be attracting the wrong students.

4. 4
   Instructor load

   if a challenger (usually an onboarding-touch arm) can't actually be delivered consistently by your staff at full enrollment, it's not a real option even if it wins on paper.

That last guardrail is the one people forget. An onboarding check-in that works beautifully when you have 12 trial students may quietly fall apart at 40, because nobody has time to make the calls. If the win depends on a touch you can't sustain, it isn't really a win.

Statistical thresholds when your numbers are small

You will not get statistically clean results at studio volume, and you should stop pretending otherwise. What you can do is set honest thresholds that keep you from acting on noise.

1. 1
   Minimum per arm

   don't make a permanent decision on fewer than ~30 people per arm. Below that, differences are mostly random.

2. 2
   Meaningful gap

   treat a conversion difference under about 8–10 percentage points as "no clear signal" when your arms are in the 30–50 range. A 32% vs 36% result is a shrug, not a decision.

3. 3
   Repeat before you commit

   if a challenger wins by a real margin, run it again the next quarter before making it permanent. A result that survives twice is far more trustworthy than one big number.

4. 4
   Direction over precision

   you're not proving significance to a journal. You're asking "does this consistently push things the right way?" Two matching quarters in the same direction beats one dramatic quarter every time.

The honest framing: your lab isn't producing p-values, it's producing repeatable directional evidence. That's enough to make better pricing decisions than the vast majority of studios, who make them on gut feel.

A real scenario

A single-location BJJ and kids' program studio was running a 30-day free trial and converting around 28% to paid, with roughly 40 new trials a month. Retention past month two was weak — a lot of members drifted after the discount honeymoon.

They ran two sequential experiments off the templates above. First, a trial-length test: 30-day free trial vs a 6-class trial. The class-count arm pulled slightly fewer starts (about a 15% drop, inside the guardrail) but converted noticeably better — low-to-mid 30s vs high 20s. They kept it, then re-ran it the next quarter to confirm. It held.

Second, an onboarding-touch test on the winning trial format: automation only vs automation plus a second-class check-in call. The check-in arm converted better and its month-two retention was clearly stronger, because the calls caught frustrated beginners early.

Net effect after both experiments settled in over about a quarter: conversion moved from ~28% into the mid-30s, and month-two retention improved enough that the owner estimated somewhere in the range of $2k–$3k in additional monthly recurring revenue — without deepening a single discount. The wins came from structure and contact, not price cuts.

When this makes sense — and when it doesn't

Run this lab if you have at least ~25–30 new leads a month, you're currently changing offers on gut feel, or you suspect your discounts are attracting churners. The method pays off fastest for studios that have been guessing.

Hold off if you're getting under ~10 leads a month. At that volume no test will produce a readable result for months, and your energy is better spent on lead generation and the fundamentals in your referral and promotion operations first. Fix the top of the funnel before you optimize the offer inside it.

Don't do this if you can't commit to the discipline — fixed windows, no early peeking, single-variable changes, alternating assignment. A sloppy experiment is worse than none, because it produces confident wrong answers. If you're going to eyeball it and stop early the moment one arm looks good, save yourself the effort and just keep your current offer.

Where software quietly helps

None of this requires special tools, but two parts get error-prone by hand: alternating lead assignment and holding to the measurement window without peeking.

If your studio management platform can tag each new lead into a test arm automatically and only surface results once the outcome window closes, you remove the two most common ways these experiments get contaminated — staff steering leads, and owners reading results too early. That's really all the automation you need here: consistent assignment, a locked window, and clean reporting at the end. The thinking still has to be yours.

One last thing

The value of a pricing experiments lab isn't any single winning offer. It's that you stop making permanent decisions off a good week or a persuasive staff member.

Once you're running one clean, single-variable test per quarter — with a fixed window, real guardrails, and a repeat before you commit — your pricing stops being a series of guesses and starts compounding. Each quarter you know a little more than the last, and unlike a discount, that knowledge doesn't cost you margin.

Once you're running one clean, single-variable test per quarter — with a fixed window, real guardrails, and a repeat before you commit — your pricing stops being a series of guesses and starts compounding. Each quarter you know a little more than the last, and unlike a discount, that knowledge doesn't cost you margin.