Ben Labay

CEO at Speero | Experimentation & Research Program Leader

CEO at Speero. Recovering Research Scientist. Experimentation Obsessive.

I help SaaS and Ecommerce brands turn data into a competitive advantage. Over the last 8 years, I’ve led research and testing programs for Miro, P&G, ADP, and MongoDB—helping them bridge the gap between complex statistics and better customer experiences.

My mission? To help organizations use experimentation as a superior way to grow.

Areas of Expertise: ◽️ Experimentation Strategy & Scaling ◽️ UX & Usability Research ◽️ Agency & Brand Growth Marketing

Looking for help with your testing program? Drop me a line at ben@speero.com or follow my journey here on LinkedIn.

CXL, and by extension Speero, is an inspired workplace, constantly pushing and questioning services and products and what works and what doesn't. The agency pushes on experimentation and customer experience research for SaaS, Ecommerce, and Lead Gen clients around the world. 

These organizations and teams have business and customer strategies, we test them. Ben works within Speero to scope research and test program strategy for companies like Proctor & Gamble, ADP, Codecademy, MongoDB, and Toast. https://speero.com

Two charts making the rounds right now tell completely opposite stories about AI and jobs.
 
Chart on left: Anthropic analyzed millions of Claude conversations and mapped them against jobs. The gap between what AI could theoretically do and what people actually use it for is massive. Even in Computer & Math, observed usage covers about a third of theoretical capability. Ofc displacement/disruption potential is enormous, we're early.
 
Chart on right: Citadel Securities pulled Indeed data showing software engineer postings are up 11% YoY. Rising. In the exact sector most exposed to AI.
 
So AI can theoretically do a huge chunk of software work. People are actively using it for that. And demand for engineers is going up.
 
This is 'Jevons paradox'. (I'd expect to hear this term more and more)
 
In the 1860s, Jevons noticed better steam engines didn't reduce coal use. They increased it. Efficiency made coal-powered stuff cheaper, so people just did more of it.
 
Same thing here. AI makes building software faster and cheaper. You'd think fewer developers. But what actually happens is the scope of what gets built expands. More companies build custom tooling. More workflows get automated. More MVPs get tested. People don't do less. They do more.
 
In experimentation and SaaS (my world), same dynamic. AI compresses the cost of standing up a test. Naive read: fewer people. What's actually happening: more tests. The bottleneck moves from "can we build this" to "do we have the strategic surface area to deploy against."
 
The teams pulling ahead right now aren't shrinking. They're expanding scope because AI made learning cheaper. Same people, way more velocity.
 
Jevons, not displacement. (At least for now)
 
This is exactly the kind of thing we're digging into at 🎪 #CircusLondon (May 11-12, speero.com/circus).

Emil Palikot from Northeastern is presenting 

"Better Together: Quantifying the Benefits of AI-Assisted Recruitment" ... a randomized experiment with 37,000 applicants showing AI-assisted hiring actually increases interview pass rates and job placement. AI selecting different candidates than humans would, using different signals. Not replacing recruiters. Changing what recruiting looks like.
 
More Jevons. More complement than substitute. Come see it. 

If you're in-house, and esp a leader/decision maker, DM me for a ticket.

CEO @ Speero | Experimentation for growing SaaS, Ecommerce, Lead Gen

Towards being AI-native for CRO/testing teams. 
My favorite piece I've written in a while, and most important.

This article is my thoughts and strategy for how experimentation teams (and my team and clients at Speero) transition to AI-native workflows, and buil

Open article: An experimentation team's AI workflow architecture (what I'm building for Speero) by Ben Labay • 7 min read

An experimentation team's AI workflow architecture (what I'm building for Speero)

Experimenting with Paper.design as the canvas layer for a fully agent-driven experimentation design workflow. 

> Cursor, Claude Code, and a human working on the same Paper file simultaneously. Three agents (two AI, one human), one canvas, real-time.
 
Why does this matter for experimentation programs?
 
The bottleneck in most test design workflows isn't the idea. It's the handoff. Strategist writes a brief. Designer interprets it. Developer builds it. Each translation step loses signal and adds latency. The velocity killer isn't bad ideas, it's the number of hops between "what if we tried X" and a buildable variant sitting in front of stakeholders.
 
Paper is an HTML/CSS editor at its core, which means the gap between what's on the canvas and what an LLM can read/write is almost zero. Minimal translation. That's the key insight. Figma's MCP is mostly read-only (11 tools). Paper's MCP exposes 21 tools, and most of them are write tools. Create artboards, write HTML, update styles, delete nodes, rename layers... the AI isn't just looking at the design, it's building on the canvas alongside you.
 
So the workflow becomes: strategist describes the variant, coding agent builds it directly on the design canvas, designer refines it in real-time, and there's no export/import/handoff step. The feedback loop compresses from days to minutes.
 
Paper is still in alpha and it's rough in places. But the architecture is very promising. I think this is a signal of where design tooling is heading for experimentation teams specifically. The tools that win won't be the ones with the most features. They'll be the ones with the least translation between human intent and machine execution.
 
We're still early on this. But if your experimentation program is bottlenecked on variant production (and most are, even if they don't name it that way), the design-to-code layer is worth paying attention to.
 
Anyone else exploring agent-driven design workflows for experimentation? Curious what tools are working.

h/t Laszlo Zagyva leading this 💪

What makes #EXL, now #CircusLondon quite difference is the thought we put behind networking. 

The event is half conference, half networking event. And in London, it will cover how we are adapting Experimentation with AI. 

One way we make this work is by doing massive 'wedding planning' efforts to get 200+ people in the right tables with the right people. Again and again, round after round. It's not perfect, but it's honestly where the magic happens. And the 'forced' networking is where the real value come from. 

The rest of it, the keynotes, great food, parties, workshops are all there to grease the wheels of the experience.  

Shout out to who make these table sessions flow, our moderators. Here's our current list, and check the diversity of titles!! 

• Evie Thompson : Group Head of Data Engineering and AI, Benifex
• Laura Haczewski : Strategic Product Manager, ODEON Cinemas Group
• Steve Quinlan : Digital Experience Manager, NatWest Group
• Surjit Panda : Group Head of Data Engineering and AI, American Express
• Tony Him : Senior Product Manager, Nando's UK & IRE
• Vasileios Fasoulas : Head of Engineering, meshx
• Vignesh Lokanathan : Experimentation | Optimisation | CRO, BBC
• Melanie Kyrklund : Global Head of Experimentation, Specsavers
• Peter Gostev : Head of AI, LMArena
• Theo Gough: Data & Analytics Head of Strategy, Governance & Experimentation Enablement, Just Eat Takeaway.com
• Marco Cordioli  Head of Marketing Innovation & GenAI, VELUX
• Jeffrey John Baker : Partner, Head of AI Strategy, AI Forge
• Julian Kaljuvee : Generative AI Engineer, Microsoft
• Gareth Holder PhD : Head of Data Science and Analytics, Matalan
• Jamie Martin : Director of Product Analytics & Experimentation, NBCUniversal
• Matt Mulvey : Experimentation Lead - Commercial, Sky
• Divya Isaiah : Head of Experimentation, BBC
• Bhavik Patel : Data Director, Huel
• Dr Simon Jackson : Experimentation Data Scientist & Consultant, DrSimonJ
• Vasant Kumar : Senior Product Manager, Growth, Mozilla
• Stewart Ehoff : Growth, RS Group
• Luiza de Lange : Head of CRO

This is an article on the convergence of:

- Experimentation programs
- Pre-registration protocols
- Strategic finance

The short version: most experimentation programs say they're running experiments to move metrics, but they're not actually set up to take bets. I think this is the difference between 'experimentation' and 'growth experimentation'.

This is an article on the convergence of: Experimentation programs, pre-registration protocols, and strategic finance I've been circling around an ide

Open article: Growth Experimentation: What it means to run your testing program like a portfolio of bets by Ben Labay • 10 min read

Growth Experimentation: What it means to run your testing program like a portfolio of bets

Eddie Aguilar and I interviewed four companies switching to Amplitude Experiment for unified experimentation. Huckleberry Labs, Zwift, FanDuel, BigCommerce.

The pattern was the same every time.

Not a tooling problem. A system problem.

When your A/B testing engine, your analytics, and your identity layer are all separate systems, you spend most of your energy validating the data (instead of validating the hypothesis). 

You get what Huckleberry called "the law of most surprises" (not a compliment). Conflicting results across tools. Polluted samples. Analysts manually stitching data in a "very manually intensive process."

It's the Frankenstein Stack. And it's everywhere.

The companies that switched didn't just change tools. They changed the operating model.

Here's what actually unlocked:

1. Velocity. FanDuel more than doubled experiment velocity in year one. Not because they got smarter. Because context switching disappeared. Same platform for insights and experiments. Engineers stopped fielding one-off requests. Product teams stopped waiting.

2. Trust. Zwift now tests in production. They created an internal "Employees" cohort that lets engineers and PMs hop on bikes (their product), test live features, and find bugs before rollout. That only happens when your targeting is trustworthy enough to contain experiences. Trust in the data is the precondition for everything else.

3. Identity resolution. This one is underrated. Most tools assign variants at the device level. Huckleberry had accounts used by mothers, fathers, nannies... across multiple devices. Device-level assignment was polluting every test. Moving to user-level resolution fixed a problem they didn't know how to fix, and it required no extra engineering lift.

The meta-insight isn't really about Amplitude. It's about the compounding cost of silos.

When your data lives in disconnected systems, the friction compounds. Each test requires extra reconciliation. Each stakeholder meeting starts with distrust. And eventually the program stagnates... not from lack of ideas, but from lack of confidence in the output.

𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗲 𝘁𝗵𝗲 𝘀𝘆𝘀𝘁𝗲𝗺. 𝗡𝗼𝘁 𝘁𝗵𝗲 𝘁𝗼𝗼𝗹.

Wrote this up as a whitepaper (in partnership with Amplitude) with Jonny Longden and Martin P. who synthesized the full narrative. Worth a read if you're in growth or product experimentation.

Link in comments. Curious what patterns others are seeing in their stacks.

Amplitude Whitepaper FINAL

We built 🎪 #CircusLondon  because we couldn't find the EU event we wanted to attend.

250+ in-house experimentation leaders. Two days. The whole thing is designed around one question: what happens when AI meets rigorous experimentation culture?

May11-12, London

Three rings of topics:
1. > AI for experimentation (the practitioner layer)
2. > Experimenting on AI (the builder layer: testing non-deterministic systems, LLM-powered features, black box models)
3. > Experimenting with AI (the leadership layer: governance, ROI, team structure)

Keynotes from Etsy, Zalando, Northeastern, and Leah Tharin! Plus 100+ roundtable sessions matching you with the people and problems you actually care about. 

LightningTalks and Posters on AI+Experimentation applications (lots of synth user stuff!)

Workshop day on Day 1 includes hands-on Claude Code sessions (building customer insight systems and AI workflows from scratch).

Also... we designed the networking to be genuinely useful. 80% in-house attendance.  Facilitated tables. Parties that aren't afterthoughts.

Code earlybird30 gets you 30% off at checkout — but it expires end of February. Only a few days left.

speero.com/circus

Who's coming?

Open article: Circus London 2026 by speero.com

Claude skills will be yesterday's news soon? It will all be plugins and plugin marketplaces next. (and not browser plugs btw). And then what's an agent?

These terms get used interchangeably and that's fine until you're trying to actually build something. So here's how I think about them.
 
𝗔 𝘀𝗸𝗶𝗹𝗹 𝗶𝘀 𝗽𝗮𝗰𝗸𝗮𝗴𝗲𝗱 𝗲𝘅𝗽𝗲𝗿𝘁𝗶𝘀𝗲. It's the instructions, context, and knowledge you give an AI so it can do a specific thing well, consistently, without you re-explaining from scratch every time. It lives inside the AI. At Speero, a skill is what teaches Claude how to run an audit, how to write in our voice, how to structure a presentation. The skill is the thing we're building first. It's the foundation.
 
𝗔 𝗽𝗹𝘂𝗴𝗶𝗻 (𝗼𝗿 𝘁𝗼𝗼𝗹, 𝗼𝗿 𝗶𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻, 𝘁𝗵𝗲 𝗻𝗮𝗺𝗲𝘀 𝗮𝗿𝗲 𝗶𝗻𝘁𝗲𝗿𝗰𝗵𝗮𝗻𝗴𝗲𝗮𝗯𝗹𝗲) is how you give the AI a hand that can reach into the world. On its own, Claude can only work with what you paste into the chat. A plugin changes that. Now it can read a Google Doc, query a database, pull from your CRM, post to Slack. It extends what the AI can act on, not just think about.
 
𝗔𝗻 𝗮𝗴𝗲𝗻𝘁 𝗶𝘀 𝘁𝗵𝗲 𝘁𝗵𝗶𝗻𝗴 𝘁𝗵𝗮𝘁 𝘂𝘀𝗲𝘀 𝗯𝗼𝘁𝗵. It's an AI system that can pursue a goal over multiple steps, making decisions along the way without you orchestrating every move. The key difference is autonomy. It doesn't just answer a prompt. It works toward an outcome.
 
Simple version: a skill is expertise, a plugin is a hand, an agent is a mind deciding what to do with both.
 
𝗙𝗼𝗿 𝗦𝗽𝗲𝗲𝗿𝗼 𝗶𝗻 𝗽𝗿𝗮𝗰𝘁𝗶𝗰𝗲: the audit skill teaches Claude the how. A Google Drive plugin would let it read client survey data directly. An agent would be Claude pulling that data, scoring it against the benchmark, and generating the deck... without you touching it between steps. 

That's the direction we're building toward. #speerotroupe

taplio

Ben Labay's Linkedin Analytics

Ben Labay

Performance ScoreBen Labay's LinkedIn Performance

What is Ben talking about?

Ben Labay's Best Posts (last 30 days)

Ready to grow your LinkedIn presence?

taplio

.css-6ytdur{background-image:linear-gradient(284deg, rgba(39, 43, 72, 1) 0%, rgba(33, 47, 183, 1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Ben Labay's Linkedin Analytics

Ben Labay

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Ben Labay's LinkedIn Performance

What is Ben talking about?

Ben Labay's Best Posts .css-1emoo84{font-size:var(--chakra-fontSizes-md);font-weight:400;color:var(--chakra-colors-gray-600);}(last 30 days)

Ready to grow your LinkedIn presence?

Ben Labay's Linkedin Analytics

Performance ScoreBen Labay's LinkedIn Performance

Ben Labay's Best Posts (last 30 days)