I help companies scale high-impact experimentation programs that actually drive growth (not just more tests or prettier dashboards).

Over the past decade, I’ve led data and experimentation teams at some of the world’s most product-led companies including Meta, Canva, and Booking.com. I’ve helped teams design better experiments, make smarter decisions, and build the platforms and cultures that support real velocity.

These days, I work as an independent consultant out of Sydney, helping mid-to-large APAC companies turn experimentation into a growth engine. I bring a mix of hands-on support, strategic guidance, and proven frameworks developed from my experience like NAV (a practical lens to measure and improve the Number, Accuracy, and Value of your experiments).

My services include:

🧠 Experimentation strategy sprints
🧪 Fractional leadership and advisory
📈 Workshops & other custom projects

If you're trying to scale experimentation and want an experienced partner who's done it at the highest level, let’s talk.

➡️ Learn more at drsimonj.com

Simon Jackson

Dr Simon Jackson

Building in Experimentation 🚀 Ex-Meta, Canva, Booking.com

Baby Yoda meme with top panel, "Third significant AB test in a row!" Lower panel, "Finding your AA test is always significant too."

The biggest shift happening in Experimentation right now 👇

How things were:
- Running tests was done mostly by tech/digital-first companies
- Experimentation teams had hyper-specialised and technical skills

What I see now:
- Experimentation is being rapidly adopted by every decently-sized company with a website, in every industry
- It’s increasingly being done by people with zero experimentation or stats training.

Why it’s happening:
The business case has become clear: The biggest and best companies have used experiment to outlearn, outpace, and outgrow competition.
The barriers to modifying code are rapidly dropping thanks to AI, and experimentation is becoming more accessible with it.

What leaders should do:
1.  Build organisational competence around metrics, decision-making, and learning loops
2.  Put in place the systems, training, and culture that empower everyone to experiment responsibly

The shift is already happening: experimentation is scaling far beyond the experts.
The leaders who prepare their people for this reality will move faster, learn smarter, and win bigger.

It took me 10 years and thousands of experiments to realise this 👇

The belief I had:
To make a decision based on an experiment, I needed to examine all the data.

What changed it:
I'd constantly find myself in endless debates, never-ending follow up analyses, and almost never making a better decision.
But over time I learned from world-class how to do better...

The new belief:
The best and fastest decisions happen when you front-load the hard thinking.
Build out your metrics and decision criteria without data.
The rest is easy.

If you're constantly debating experiment results and feeling like data just creates more questions than answers:
Stop looking at results until you/your team has agreed on the clear "if-then" statement for acting on the data.
You meet to align on the rules or execute the rules. Never both.

A/A testing is a must.

Here's how it totally changed an old team of mine 👇

Before:
- Experimentation bugs would be flagged randomly, sometimes days or weeks too late.
- Data and engineering teams would waste cycles, scrambling to figure out issues.
- Lots of data, and great experiments, had to be scrapped.

After:
- System bugs were identified within 24 hours at most.
- Teams could isolate why and deploy fixes, fast.
- Major reduction in lost data and experiments.

How we got there:
1. **Ongoing A/A Test:** Constantly run a test where Control and Treatment are identical to check for unexpected metric separation (especially core metrics).
2. **Treat Failure as a P0 Bug:** If the A/A test starts failing seriously, pause testing immediately and fix the underlying issues. Don’t let it run.
3. **Report on failure rates:** Regularly report your A/A failure rates to build confidence and trust in experiments and your team.

If you’re in the “before” state, start by running just one A/A test!
They're easy to setup and you won't regret it.

AB Tasty and VWO are joining forces!

What stands out to me isn’t the deal mechanics, but the signal 👇

Money continues to flow into experimentation.

This is another data point in a growing pattern and, in my view,
it’s not despite AI, but because of it.

As teams ship faster, generate more ideas, and make bigger changes with less friction, the need to:

→ Understand what’s actually working
→ Learn quickly and reliably
→ Detect unintended harm

only increases.

Experimentation isn’t being diminished by AI.
It’s becoming more critical to making good decisions at scale.

And the flow of capital is starting to reflect that.

Curious how others are reading this announcement?

***
🗞️ Announcement link in comments

If you’re the team that runs experiments…

I’ve got bad news for you 👇

World-class product companies don’t treat experimentation as a team.

They treat it as a way of working.

⬇️

In these companies, you won’t hear:
“Speak to the experimentation team.”

You’ll hear:
“Speak to the onboarding team.”
(who run experiments.)

This distinction matters.

⬇️
🎯 First, what teams optimise for.

Less mature teams feel pressure to show motion.
So they optimise for activity.
More experiments. More dashboards. More updates.

World-class teams optimise for coverage.

They run the fewest experiments needed to meaningfully reduce risk and unlock opportunity across the product, not just one surface.

⬇️
📚 Next, how learning is treated.

Less mature teams rarely get challenged.
A “win” is taken at face value.

World-class teams slow this part down.

They invest heavily in:
→ Metric design
→ Guardrails
→ Clear decision rules

So every learning is trustworthy and directional, not just statistically significant.

⬇️
💥 Finally, how impact is defined.

Less mature teams feel constant pressure to justify themselves.
So impact gets reduced to short-term revenue lift.

World-class teams look broader.

They measure how experimentation:
→ Shapes the roadmap
→ Improves strategic clarity
→ Reduces harm
→ Improves long-term decision quality

⬇️
Here’s the real point:

Competitive advantage doesn't just come from running tests.

It’s embedding a system for reliable, high-quality learning, faster than everyone else.

That kind of culture is very hard to copy.

⬇️
So if you’re THE team running experiments in a company where you want everyone to experiment, start here:

👉 Teach another team to experiment.

You can’t build a culture of experimentation if you’re the only ones doing it.

Simple text image: "You can’t build a culture of experimentation if you’re the only ones doing it."

“Marketing should own landing pages. Not product.”

I heard a CRO lead say this recently.

I get why people say this.
I also think it’s one of the most damaging ideas in experimentation.

Because it replaces the most powerful idea we have — letting data decide — with politics and boundaries.

⬇️
❌ What goes wrong

In strong experimentation cultures, ownership isn’t about codebases.
It’s about outcomes.

Yes, teams need clear responsibility for maintaining systems.

But when “ownership” turns into exclusive rights to make changes, you get:

👉 Teams optimising what they’re allowed to touch
👉 Not what actually moves the metric

⬇️
🎯 Outcomes don’t respect org charts 

If a team’s goal is acquisition, they’ll care about:

→ Landing pages
→ Ads and creative
→ Sign-up flows
→ Onboarding
→ First activation
→ Sharing mechanics

Putting hard borders around these surfaces doesn’t create focus.
It creates local optimisation and global underperformance.

⬇️
🚀 What high-performing teams do instead

In the most innovative companies I’ve worked with:

✔ Almost anyone can test changes almost anywhere
✔ Clear guardrails exist
✔ Data, not hierarchy, decides

Not because chaos is good.
But because speed and evidence beat hierarchy every time.

⬇️
🧭 So what should ownership look like? 

Marketing shouldn’t “own landing pages”.
Marketing should own creating demand.

Product shouldn’t “own the funnel”.
Product should own building great experiences.

And experimentation exists precisely to let teams cross boundaries...
guided by outcomes, not job titles.

If this makes you uncomfortable, that’s probably the point.

Simple text image: "Experimentation lets teams cross boundaries guided by outcomes, not org charts."

Most growth/CRO teams *say* they're data-driven...

But very few understand what "pre-registration" actually means 👇

What pre-registration means: 
- It's a commitment to *which* metrics matter *before* a test starts.
- It covers the primary metric, key secondary metrics, and crucial guardrails (like technical stability or fraud).
- It formalises the decision rule: when will we ship, iterate, or kill the test?

Why it matters:
- It eliminates confirmation and hindsight bias, preventing you from cherry-picking metrics after the results are in.
- It forces clarity and alignment across product, data, and leadership *before* resources are committed.

Common failure modes:
- Teams register the primary metric but forget about others (e.g., shipping a win that secretly broke the platform).
- Teams register too many metrics, making the post-test story muddy and confusing (you need focus).

What to do instead:
1.  Include primary, key secondary, and guardrail metrics in your experiment planning template. If it's not documented, it doesn't count.
3.  Keep your criteria clear and tight. Usually, primary should improve, secondaries to move as expected, guardrails not negative.

If a metric isn't in the plan, it's irrelevant to the decision

Customer insights and analytics director with 20+ years of experience in data-driven strategies to enhance customer experience and drive sales. A curious, critical thinker with a customer-first mindset, blending the leadership, strategic and technical skills to deliver actionable insights from analytics, reporting, research and personalisation.

My left brain gets it's kicks out of numbers, data, code and analysis. Whilst my right brain loves the synthesis and strategy. This blend of right and left brain, EQ and IQ enables me to 'connect the dots' to drive understanding, action and value.

taplio

The Battle

Simon Jackson

Nicola Shawyer

Followers

Posts

Engagements

Likes

Performance ScoreSimon Jackson's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Simon Jackson

Nicola Shawyer

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Simon Jackson's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreSimon Jackson's LinkedIn Performance