“Can we just test a small batch first?”. The “sensible” question that isn’t so sensible
It sounds careful. Responsible. Low risk.
Like you’re doing marketing properly. But most of the time, it’s not caution.
It’s discomfort dressed up as logic.
When a Trial Isn’t Really a Trial
I was recently asked: “Can we trial 50 letters and see if it works?”
On the surface, fair. But the framing is off.
Because 50 isn’t a test.
It’s a guess with a spreadsheet attached.
Why Sample Size Matters More Than You Think
Here’s the bit people don’t like: Small samples don’t give answers A small sample size doesn’t tell you if something works.
It tells you what happened. Once.
In a very specific set of conditions you don’t control.
That’s not insight. That’s noise with a story wrapped around it.
Lessons From Real-World Sales
Long before job titles, I was knocking doors on 100% commission selling home improvements.
No base salary. Just effort and outcomes. Some days we’d be out all day and get nothing. Other days, same hours, same streets, bam. Five leads!
- Same pitch.
- Same energy.
- Same me.
The difference wasn’t mindset. And it definitely wasn’t how many beers I’d had the night before. It was people.
- Right people.
- Right timing.
- Right moment for that offer to make sense.
I could knock 100 houses and get nothing with the exact same message. Then walk onto the next street and get three in a row.
If I’d judged that based on the first 50 doors, I’d have said it didn’t work.
I’d have been wrong.
Understanding the Role of Timing and Audience
The variable isn’t always you. That experience strips things back.
Sometimes it’s not your message or your channel. Sometimes you just haven’t reached enough of the right people yet.
And if your sample doesn’t include enough of those people, you’re dead in the water.
Why Small Campaigns Produce Misleading Results
Why do 50 handwritten letters tell you almost nothing? You send 50 handwritten style letters. You get no response.
You draw the conclusion that “It doesn’t work.”
But what actually happened? You didn’t reach enough of the right people, at the right time, in that tiny window.
That’s not failure. That’s absence of data.
Applying the Same Logic Across Marketing Channels
We already understand this just not consistently.
If you put £50 into Google Ads, would you expect clarity? Of course not.
You already know that’s not enough to generate meaningful data.
Yet in other channels, suddenly 50 feels like a “test”.
Same logic. Different disguise.
What Successful Marketers Know
The people getting results aren’t guessing and the companies quietly winning with Google Ads aren’t dabbling. They’re spending £1k, £5k, £10k+ a month.
Not because they enjoy it. Because they understand something simple:
Volume reveals patterns.
Tiny samples don’t.
The Difference Between Activity and Evidence
A proper test doesn’t ask: “Did something happen?”
It asks: “Is there a pattern here we can trust?”
That requires:
- Enough people
- Enough time
- Enough repetition
Enough to absorb randomness.
Why Good Ideas Get Abandoned Too Early
The line I’ve heard for 20 years: “If it works, we’ll do more.” It sounds logical. But it quietly kills good ideas.
Because early results rarely look convincing. They look uneven. Just like knocking doors did.
- Some days nothing.
- Some days everything.
That’s not failure.
That’s the early stage of something real.
Testing Versus Taking a Chance
There’s a difference between testing and gambling small numbers over time? Fine. You’ll eventually build a picture.
But calling 50 a “test” and expecting a decision?
That’s not testing, it’s gambling.
And we don’t gamble with people’s money.
What Counts as a Meaningful Sample Size?
So what does “enough” actually look like?
- A warm Audience, people who know you: 100- 500 can be a reasonable starting point.
- If it’s cold audience, you’re typically looking at 1,000–5,000 (market dependent) before you see anything reliable.
Not because bigger sounds better, but because that’s what it takes to reach enough of the right people.
Marketing Is a Numbers Game
The real job isn’t sending. It’s finding. Marketing isn’t about hoping something lands. It’s about reaching enough people for it to inevitably land.
Fifty doesn’t give you that. It gives you a coin flip.
And calling it a test doesn’t make it one.
A Recent Campaign That Proves the Point
We had a client invest £2,500 into 5,000 handwritten compliment slips.
- These weren’t posted.
- They were hand-delivered.
Around 4,000 have landed so far and still more to go. Yesterday, this came through:
- 17 calls.
- 14 quotes.
- 9 sales.
Product value sits around £2–3k. So yes – it’s working. But more importantly, it’s proving the point. This didn’t work because of one perfectly timed message. It worked because we reached enough of the right people for it to land.
Why Data Quality Matters More Than Early Results
The part most people miss.
If we’d “tested” this with 50 – there’s a very real chance none of those 9 sales happen. In fact, I would best my most favourite pair of Irregular Choice Shoes on it!
Not because the idea was wrong. Because the sample was.
- Numbers matter.
- Representative samples matter.
- And most importantly – decisions based on incomplete data are expensive.
Usually more expensive than doing it properly in the first place. If you are interested in discussing handwritten compliment slips or letters – feel free to reach out. Its my passion subject 😉
Daniella
Founder of Pen Written Post
