I was early Marketing at Uber spending 5 years building Brand, Performance, and Lifecycle Marketing Data Science teams. I got to work with & learn from world-class marketing, data science, and martech leaders. Over the course of my career there I've measured over $1Bn in Marketing Spend and estimate that I've helped drive $200M+ in incremental revenue and cost savings . 

Then I moved on to a few early stage startups and scaleups (both as a FTE and Consultant) where I learned how to be even scrappier when building Marketing. 

Now I help Marketers access those same playbooks top Consumer Tech Marketers use to prove, optimize, and scale ROI by sharing frameworks, processes, and analyses. 

I'm doing this because I love the Marketing stories and people behind it in a world where Product seems to get most of the credit. There's so much magic in Marketing and I want to share more about it. 

Check it out here → https://experimental.beehiiv.com/

Outside of my newsletter and podcast, I love spending time with my 2 beautiful kids and my incredible wife in Amsterdam. We used to love traveling and eating out but you know... kids ...  so now I love cooking and exploring all the fun things little kids can do!

Sundar Swaminathan

CEO: We're using 7-day click attribution. 

New CMO: What's your average purchase cycle?

CEO: Probably 30 days from first awareness to purchase.

CMO: Then our attribution window is cutting off 75% of the customer journey.

CEO: Wait, what?

CMO: We're only giving credit to touchpoints in the last 7 days. Everything that happened before that gets ignored.

CEO: So we should use a 30-day window?

CMO: Maybe.

CEO: This is confusing. How do we pick the right window?

CMO: There is no right window. There's the window that matches your business reality, but more importantly reflects incrementality. 

CEO: What does that mean?

CMO: If you're selling impulse purchases, 1-day windows probably work. If you're selling considered purchases, you need longer windows but also need to acknowledge the attribution is fuzzy.

CEO: So what should we do?

CMO: Test different windows and see how channel performance changes. If Facebook looks amazing at 7 days but terrible at 28 days, that tells you something.

CEO: What does it tell me?

CMO: That Facebook is getting credit for conversions close to the purchase, but it might not be the thing that started the journey.

CEO: How do other companies handle this?

CMO: The sophisticated ones stop relying on attribution windows alone and run incrementality tests instead.

CEO: So attribution is useless?

CMO: It's useful for directional insights. Just don't make million-dollar decisions based on which arbitrary window you picked. 

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to your CEO later

Former Head of Marketing Science at Uber who optimized $1Bn+ across Brand, Performance, and Lifecycle → Sharing weekly playbooks used by top Marketers to measure and scale ROI.

CEO: Need to cut Brand Marketing budget by 30%.

CMO: Why Brand Marketing?

CEO: We need to preserve cash now. Brand Marketing feels like the safest place to cut.

CMO: Why does it feel safe?

CEO: Because we can turn it back on when things improve. Engineering headcount we can't.

CMO: You're right that we can turn it back on. But you're wrong that it's safe.

CEO: Explain.

CMO: When you cut Perf Marketing, you see the impact immediately. Revenue drops in weeks. And when you cut Brand Marketing? You won't feel it for months, but it'll come.

CEO: So we should cut Performance instead of Brand?

CMO: I'm saying both have consequences. Performance cuts hit revenue fast. Brand cuts hit revenue slowly, which makes them feel safer but they're actually riskier.

CEO: How is slow worse?

CMO: Because by the time you realize brand cuts hurt, your competitors have filled the void. Customers forgot about you. And it takes twice as long to rebuild than it did to build initially.

CEO: We're not cutting all of marketing. Just 30%.

CMO: Where's the 30% coming from?

CEO: I was hoping you'd tell me.

CMO: Well, it's not coming from me and it's not what I'd recommend, but I know I'm not going to win this fight, but I do have a recommendation. 

CEO: What do you recommend?

CMO: Cut performance by 15% and brand by 15%. Spread the pain. Don't kill either completely.

CEO: Why not just cut the one that's less important?

CMO: Because Brand Marketing is not less important! They work together. Brand makes performance cheaper. Performance capitalizes on brand equity. Kill one and the other suffers too.

CEO: How do other companies handle this?

CMO: The smart ones actually do the opposite of what you're saying. They protect Brand Marketing during downturns because that's when competitors are cutting. You can gain share cheaply. The desperate ones cut everything and wonder why recovery takes years.

CEO: That's a tough call.

CMO: It is. But cutting Marketing always feels safe because the damage is delayed. That's what makes it dangerous. 

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to your CEO later

CEO: Brand campaign doesn't look it's working! 

CMO: What metrics are you looking at?

CEO: The usual. New customer acquisition, conversion rate, revenue growth. All flat.

CMO: Good :) 

CEO: What do you mean? You said this campaign would drive growth.

CMO: It will. In two to three months. 

CEO: Two or three months? Why so long?

CMO: Because of how our customers actually buy. Walk me through how you decided to buy those shoes you're wearing. 

CEO: Hmm...I saw ads for it, ignored them for a while, then a colleague mentioned it, then I researched it, then maybe a month later I finally bought it.

CMO: Why'd you buy it a month later?

CEO: I needed shoes and I saw another ad for them which .... OH! I get it! 

CMO: Exactly! That's what's happening here. Our product isn't something people impulse buy. We cant snap our fingers and make people want our product.

CEO: So we're just... waiting for them to be ready?

CMO: We're making sure that when they're ready, they think of us first instead of our competitors.

CEO: But how do we know it's working if we can't see results for months?

CMO: We track leading indicators like search volume for our brand terms. Those move first. Revenue moves later.

CEO: What are those showing right now?

CMO: Search volume for our name is climbing. 

CEO: Why didn't you lead with that?

CMO: Because you asked about revenue. But revenue is a lagging indicator. It's like planting a tree and asking why there's no fruit after six weeks.

CEO: That's a good analogy.

CMO: Thanks. Now, stop texting me and go back to bed. It's 3:30 AM.

--
🔔 Follow me for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save so you don't lose it

CMO: Our MMM is crazy. It changes all the time. 
.
Me: How often are you refreshing the model?

CMO: Monthly. We want the latest insights.

Me: That's your problem.

CMO: Why? Shouldn't we use the most recent data?

Me: MMM needs long time horizons to detect patterns. Monthly refreshes are too frequent. You're picking up noise, not signal.

CMO: How long should we wait?

Me: Quarterly at minimum. Ideally every six months. You need enough data to see through seasonal fluctuations and campaign variability.

CMO: Okay, what else could be causing the instability?

Me: Are you changing variables between runs?

CMO: Sometimes. We add new channels or remove ones we stopped spending on.

Me: Every time you change the model structure, you're asking a different question. The answers will change because the question changed.

CMO: So we need to keep the structure consistent?

Me: As consistent as possible. If you must add variables, retrain the full history with the new structure. Don't just append.

CMO: This is frustrating. I thought MMM would give us definitive answers.

Me: MMM gives you directional guidance, not precision. It's better at telling you "brand is underinvested" than "spend exactly $47K more on brand."

CMO: Then why did we spend six figures on this?

Me: Because it's still valuable for strategic allocation decisions. Just don't expect it to be as precise as an A/B test.

CMO: How should we be using it then?

Me: Use incrementality tests for precision. MMM for strategy. Attribution for daily steering. 

--
🔔 Follow me for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save it so you don't lose it

CEO: Our ROAS is at 8x! Best it's ever been.

New CMO: Why do you celebrate ROAS?

CEO: Because it's a good metric! 

CMO: But ROAS doesn't tell you if you're growing. It tells you how efficient your spending is.

CEO: Aren't those the same thing?

CMO: Not even close. We could have 20x ROAS and be shrinking as a business.

CEO: Give me an example.

CMO: We're spending less on acquisition and only targeting our highest-converting audiences. 
ROAS looks amazing because we're cherry-picking easy wins.

CEO: But we're being more efficient.

CMO: We're being more efficient at reaching fewer people. Our total customer volume is down because we've narrowed our focus so much.

CEO: So high ROAS is bad?

CMO: Not always, but ROAS alone doesn't always tell the whole story and if we optimize to just ROAS, we might be missing the picture. 

CEO: What should we be optimizing for?

CMO: Well like most things in Marketing, it's triangulating across a few pictures:
1. Total new customers
2. ROAS (ideally incremental if we have it)
3. LTV / CAC 

CEO: Why do all the dashboards show ROAS then?

CMO: Because platforms want you focused on efficiency, not growth. It makes their reporting look good even when your business isn't growing. ROAS is useful for comparing high level channel efficiency, but it shouldn't be the only goal.

CEO: How do we fix this?

CMO: Stop celebrating ROAS improvements in isolation. Always pair it with growth metrics. If ROAS is up but then new customer revenue  is down, we're headed in the wrong direction. It just means we likely cut ad spend or perhaps focused on retargeting already warm leads. 

CEO: That's counterintuitive.

CMO: Most good marketing metrics are. 

--
🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save it so you don't lose it

CMO: Test showed zero lift. Let's cut the channel.

Me: Hold on. Let me see the test setup first.

CMO: We turned off the ads for a month. Revenue didn't change. Seems pretty clear.

Me: Did you account for delayed conversions?

CMO: What do you mean?

Me: Buyers don't convert immediately. They hear about you, research later, maybe buy weeks after exposure.

CMO: We ran the test for a full month though.

Me: Right, but if your average time-to-conversion from exposure is 6 weeks, a one-month test won't capture the full effect.

CMO: So we should've run it longer?

Me: At least 8 weeks for a product like yours, but really 12 at the minimum. Better would be me to actually use sample sizes. 

CMO: So the test is useless?

Me: Not useless, but potentially not representative. We should also check if the impact shows up in brand search lift instead of direct conversions.

CMO: Why would it show up there?

Me: Because that channel can build awareness. People hear about you, then search for you later. If branded search went up during the test period, that's a signal ads are working even if direct conversions didn't move.

CMO: We didn't track that.

Me: Then we're not capturing the full impact. Let me redesign this test with the right metrics and timeframe before we kill the channel.

CMO: How much will that cost us?

Me: Way less than cutting a working channel because we measured it wrong. 

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1 Billion across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CEO: Just saw that CAC dropped 25%! Nice!

CMO: Meh, I am not happy. 

CEO: Why? Lower CAC is always better.

CMO: Not if we're acquiring fewer customers overall.

CEO: Wait, are we?

CMO: New customer volume is down 15% versus last quarter.

CEO: But our CAC improved.

CMO: Right. We got more efficient at a smaller pool. It's like bragging about catching fish faster while fishing in a shrinking pond.

CEO: I don't understand. How is lower CAC bad?

CMO: CAC is an efficiency metric. It tells you how well you're converting. It doesn't tell you if you're growing.

CEO: So what should I be looking at?

CMO: CAC and total new customers. I want new customer growth to outpace CAC growth. If CAC drops but we're acquiring fewer customers, we're optimizing ourselves into irrelevance.

CEO: What's driving this?

CMO: We tightened our targeting so much that we're only reaching people ready to buy right now. We're ignoring the 95% of the market that isn't in-market yet.

CEO: But those people have terrible conversion rates.

CMO: Today, yes. But if we don't reach them now, our competitors will. And when they're ready to buy, we won't be on their radar.

CEO: So, what are we going to do?

CMO: We need to balance efficiency and reach. Some campaigns for high-intent, low CAC conversions. Others for building future pipeline, even if CAC looks worse.

CEO: How do other companies think about this?

CMO: The smart ones track both CAC and total customer growth. If both are improving, you're winning. If only CAC improves while volume drops, you're in trouble.

CEO: Got it. Thanks for explaining that. I feel much better. 

CMO: Good. Now stop reading reports and slacking me at 2:30 AM.

--
🔔 Follow me for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save it so you don't lose it

CEO: We need to be sending more emails! 

New CMO: Why? What data are you seeing?

CEO: Email open rates correlate perfectly with revenue.

CMO: I believe you, but that doesn't mean email is driving revenue.

CEO: What else would be driving it?

CMO: Could be anything. Seasonality, product improvements, word of mouth, other marketing channels.

CEO: But they move together. That has to mean something.

CMO: Likely, but that might not be what's happening. 

CEO: So what's actually happening with email?

CMO: My guess? When people are already interested in buying, they open more emails. Email isn't creating demand. It's capturing attention from people who already have intent.

CEO: How do we know for sure?

CMO: Run a holdout test. Stop sending email to a random part of the audience for a month. Then we can split it by segment and If revenue drops in that segment, email is causal. If it doesn't, it's just correlated.

CEO: What if revenue does drop?

CMO: Then we know email matters and we can invest more confidently.

CEO: And if it doesn't drop?

CMO: Then we just saved a ton of money on a channel that wasn't actually working. We can reinvest those resources into more testing and be more aggressive at iterating on the campaigns we thought were working. 

CEO: This seems like a lot of work to validate something that looks obvious.

CMO: The expensive mistakes are the ones that look obvious but aren't true. Would you rather spend $500K scaling email based on a correlation or spend $20K on a test to know for sure?

CEO: When you put it that way, the test seems cheap.

CMO: It is cheap. Way cheaper than scaling the wrong thing. 

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to your CEO later

CEO: What's our marketing efficiency?

CMO: What do you mean by efficiency?

CEO: You know, how well is marketing performing?

CMO: That depends. Are you asking about ROAS? CAC payback? LTV ratios?

CEO: Aren't those all the same thing?

CMO: Not even close. They measure completely different aspects of marketing performance.

CEO: Can't you just give me one number that tells me if marketing is efficient?

CMO: I could, but it would be meaningless.

CEO: Why?

CMO: Because different channels work on different timelines. Performance marketing pays back in weeks. Brand takes months. Combining them into one number hides everything useful.

CEO: So what should I be asking for?

CMO: Depends on what decision we're trying to make.

CEO: I want to know if we should increase marketing budget.

CMO: Then you need incremental ROAS. 

CEO: Why not use Blended ROAS? 

CMO: Blended ROAS might be 5x. Looks great. But if we're maxed out on our best channels, incremental ROAS might only be 2x. Adding budget would actually hurt efficiency.

CEO: So I can't just ask "what's our marketing efficiency?"

CMO: You can. But I need to know what you're trying to decide first. Then I'll give you the right metric.

CEO: This is way more complicated than I thought.

CMO: Marketing measurement is. That's why one-number answers are usually wrong. 

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to your CEO later

CEO: Finance says you drove $2M. You say $2.3M?

CMO: We're both right. Here's why...

CEO: It's revenue. How can there be different #s?

CMO: Finance measures when cash hits the bank. I measure when the customer converts.

CEO: Why does that matter?

CMO: I bring in signups, and often they buy the same day. But sometimes, they buy that week and sometimes a few week laters, but the revenue is attributed to my signups.

CEO: Can't you just align on one definition?

CMO: We could, but then one of us is optimizing for the wrong thing.

CEO: Explain.

CMO: If I optimize for recognized revenue, I'll ignore user behaviors where they are still valuable but might purchase later. We'd be optimizing towards same day purchase events. 

CEO: And if Finance uses your number?

CMO: They count based on dates so signups that happened this week but haven't shown revenue yet will not count for them. Their job is different than mine.

CEO: So we just accept having two different numbers?

CMO: We accept that different functions need different metrics. The key is we both understand what the other is measuring and why. 
--
🔔 Follow me for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save it so you don't lose it

CEO: AI is my #1 priority!

New CMO: Yeah... it's like my 7th...

CEO: What?! We need to be AI first and fast! 

CMO: Here's whats more important for me :
1. Our Marketing data is an insane mess
2. We have ZERO organic social presence
3. Our landing pages tell a disjointed story
4. I have no idea what channels are incremental
5. Our users don't know about half of what we do
6. Oh and retention is 30% below where it should be

CEO: Okay, great, but our competitors are using AI! 

CMO: Let them. I know for a fact they're building on a foundation of sh*t. It'll come tumbling down. I promise you that I will make AI my H2 priority but you need to let me focus on H1 on getting things right. And that's an accelerated timeline. I'm not even adding the fact that I just joined and need to ramp up.

CEO: Okay, I see your 6 months timeline and I raise you 2.

CMO: No! Give me 6 months. Let me do what you hired me for.

CEO: Okay, 4 months?!

CMO: ** closes Zoom meeting **

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CEO: Do we need an MMM?

CMO: That's not the question we should be asking...

CEO: Ohhh, what is?!

CMO: Are we ready for one?

CEO: What do you mean?

CMO: First, an MMM is not a holy grail. It's one part of a holistic measurement strategy.

CEO: That doesn't sound so bad...

CMO: Agree. The problem is that it might tell us something we weren't expecting. That channel A isn't contributing as much. 

CEO: Okay... still not seeing an issue...

CMO: Will you let me finish!

CEO: Haha oops... too excited.

CMO: Okay, so it might tell us something we don't know and we have to be prepared to then test that and validate it. That could mean we turn off 50% or even 100% of our "best" channel.

CEO: Oh boy... really?!

CMO: Yes. That's why the question we should be asking is "Are we ready for one?". Are we culturally and operationally ready to act on one or are you just asking because you heard that's what we need.

CEO: Definitely the latter, but this is really helpful.

CMO: Good. Don't worry, you'll ask me this 10x more times and I'll say the same thing 😞 

--
🔔 Follow me for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save it so you don't lose it

Marketing alone should NOT own CAC. 

That's the equivalent of saying:

- Product alone owns LTV 
- Ops alone owns conversion rate
- Finance alone owns payback period 

Sure, it's easy to silo these metrics and "assign" an owner but we do that to create an illusion of control. 

The only time CAC should be considered a Marketing metric is when all of the levers to drive CAC fall under Marketing.

This means:
1. Activation experience
2. Landing pages
3. Signup flow
4. Pricing

Otherwise, Marketing can really only influence CPM, CTR, and traffic which hamstrings their impact and influence. 

There's no perfect solution but the best orgs understand 3 things:

1. The customer doesn't care who owns what
2. You can't keep top talent this way 
3. Silos just lead to inefficiencies

It's why so many Marketers get frustrated when they're asked to reduce CAC but hit a ceiling in terms of what they can deliver.

If you want Marketing to own a metric, give them ownership.

It's pretty straightforward. 

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

My next guest raised $6M to fix Marketing Measurement after being frustrated with the tools he had available at Dropbox, Pilot, and Adobe 🤑  

Pranav Piyush, CEO & Founder at Paramark (and Reforge's course instructor on Marketing Measurement) joined experiMENTAL Newsletter & Podcast to break down why marketing feels like a "foreign language" to the C-suite and how to fix it.

We chatted in depth about:
⮑ Why MMM is now table takes
⮑ How to make marketing metrics CFO-friendly 
⮑ Why brand and performance teams are dysfunctional 

My favorite discussions:
⭐ Most marketing leaders feel anxious before board meetings because there's a massive communication gap between CMOs and the C-suite. It feels like they're speaking a different language.

⭐ If you're doing $10-50M in ads stop wasting time on holdout tests. You likely don't have enough data and should focus on pulse-up testing and new channels. 

⭐ Pranav wishes he could remove all decimals from marketing dashboards. Saying "17% incrementality" creates false precision. The reality is that It's probably somewhere between 13-19%, and that's okay.

If you're a marketing leader struggling to prove ROI to your CFO or CEO, or you're tired of attribution tools that don't actually help you make better decisions, this conversation is for you!

Pranav brings an engineer's rigor and a marketer's empathy to one of the most broken parts of modern marketing. 

Tune in wherever you love listening to podcasts using the links in the comment!

CEO: Are we measuring ROI of our Super Bowl Ads?

CMO: No, for 3 important reasons.

CEO: Oh boy, I'm nervous. 

CMO: Haha I figured, but trust that it's not so bad:
1. We can't measure ROI in a traditional sense

It's a national TV campaign and the US is our largest market so there's no real opportunity of a holdout

2. If we try to measure business impact around the time of our ad, we'll be concentrating all the "impact" into a short window and it'll make the ROI look poor

3. All other proxy metrics like search volume, traffic, etc. are going to be metrics you think are "soft" and don't represent business value.

CEO: Okay all good points, but we're spending a sh*tload

CMO: Yes, I agree, and to be clear there is an ROI. I just won't see it for months and so I can't share the ROI with you.

CEO: Okay, fair. What ROI are you expecting.

CMO: Great question. Let's get into this!

1. It's a launch of our new product. We're getting in front of hundreds of millions of our target demographic. Many of them won't be ready to buy today but they will be in a few months and we'll be there because...

2. It's the launch of our new campaign which is rolling out across 8 different channels over the next few months. Everyone is going to see us everywhere!

3. It's a competitive bet too. Our competitors are also showing up and obviously I think our team is better than theirs so people will compare us and we're going to win. Plus if we don't show up, that's just another void they'll fill 

For the campaigns we do rollout over the next couple months, we will be using our standard measurement framework of Attribution + Incrementality tests + MMM to triangulate ROI. 

Overall, there's an ROI and I believe it's huge but it'll take time to materialize

CEO: Okay. We have trusted you this far and you have not led us astray. Forgive me. I was wrong to despair.

CMO: Did you just quote Lord of the Rings?

CEO: Did we just become besties?!

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to your CEO later

CEO: CAC keeps climbing. Should we shift to brand?

CMO: It's not an either/or decision.

CEO: But, we have a fixed budget. If we spend more on brand, we need to spend less on performance.

CMO: True. But the real tradeoff isn't "performance" versus "brand". We need to decide on whether we prioritize today's revenue or tomorrow's efficiency.

CEO: Explain.

CMO: Performance marketing captures current demand. Brand Marketing creates it and at higher volumes. When we only do Performance, we're competing for the same pool of high-intent customers as everyone else.

CEO: Why is that bad?

CMO: Well, you run out of high intent customers eventually. By layering in Brand, we increase the pool of higher intent customers.

CEO: But brand takes FOREVER to show results.

CMO: Six to twelve months to show business results but much sooner to show that we're moving in the right direction. But once it kicks in, our performance channels get more efficient without changing anything.

CEO: So Brand makes performance cheaper?

CMO: Exactly. Strong brand means higher consideration, which means more people searching for you, which means lower CPCs and better conversion rates.

CEO: So we need both?

CMO: You need both working together. Performance capitalizes on brand equity. Brand reduces performance costs over time. They're complementary, not competitive.

CEO: How do other companies balance this?

CMO: The good ones don't see it as balance. They see it as sequencing and investing. 
Scale performance → Build Brand → Scale performance → Build Brand in ongoing loop. 

Eventually there's a cap, but we're not even close. 

But by then we'll have much better unit economics.

A rare few can survive on just performance marketing but they'll have insane AOVs and LTV / CACs which is not realistic. Also, they're in a market where they might not be looking to win market share and are happy with their current growth rates. 

CEO: So, we don't want to make the same mistake most companies do? 

CMO: Right. They wait until performance stops working to start Brand Marketing. By then, they're desperate and impatient. Brand Marketing needs time. We should start before we need it. 

--
🔔 Follow Sundar for Marketing Strategy & Science
👉 Join 2.6K+ Marketers learning how to measure and scale ROI: https://lnkd.in/gxR5c4Xd

CMO: Hey, engagement is up but revenue is flat. 

Me: What's your hypothesis on why?

CMO: Honestly? I'm stumped. The funnel looks healthy. Everything we're tracking is improving.

Me: Can I ask you a different question? What drives someone to purchase from you?

CMO: Product quality, pricing, trust in the brand. The usual stuff.

Me: Right. Now, how closely do your engagement metrics correlate with any of those?

CMO: Hmmm... not sure. 

Me: That's the issue. We're measuring what's easy to track, not what predicts outcomes.

CMO: So what should we be doing differently?

Me: Start by working backwards. What's the customer action that leads directly to revenue for you?

CMO: Completing their first transaction. Once someone buys our retention is strong.

Me: Perfect. So what predicts first transaction? Not correlates. Predicts. 

CMO: How quickly they experience value in the product. And whether they trust us enough to enter payment info.

Me: Now we're getting somewhere. Are any of your current metrics measuring speed to value or trust?

CMO: No. We're measuring clicks and time spent.

Me: Which is why engagement is up but revenue isn't. Those metrics aren't connected to what drives purchasing behavior.

CMO: This makes sense, but how do we figure out what actually matters?

Me: If you've got the data, then look at it and see what it says. If not, then another test you can do is change your optimization metric. Play around with what happens if you use X instead of Y to optimize. 

CMO: This is going to be uncomfortable. The team loves celebrating engagement wins.

Me: They'll love celebrating revenue wins more. And once you show them the connection between the right metrics and business impact, they'll never go back.

CMO: Alright. Let's do this. Where do we start?

Me: Make a list of every metric you track. Then next to each one, write down why you believe it predicts revenue. If you can't articulate it, cut it.

--
🔔 Follow me for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save so you don't lose it

In 2025, I wrote 38 articles on Marketing. 

Here are my top 10 by open rate (60%+):

Oh and they're all free to read 😃 

1. How to write a measurement brief 
https://lnkd.in/gPrr-tAF

2. How I build Uber’s Brand Science Team
https://lnkd.in/evJDr2wc

3. Why analyzing Marketing feels impossible
https://lnkd.in/gqNagEBr

4. How to calculate LTV
https://lnkd.in/gxUqcxcw

5. 7 Brutal truths about CAC
https://lnkd.in/gVjh-YRK

6. How to measure when you can’t A/B test
https://lnkd.in/gcHsSpqy

7. How to build better Marketing targets
https://lnkd.in/gmH8VV8T

8. How to create more impactful analyses
https://lnkd.in/esmGEWmk

9. Why startups shouldn’t spend a $ on Paid Ads
https://lnkd.in/e8kFYYU5

10. How Data Scientist should use AI
https://lnkd.in/expdRfFZ

Let me know which one was your favorite!

Every startup at every stage needs a data leader, but the engagement model evolves:

Pre-product market fit → Advisor

Especially if you're a marketplace or E-com, they can help you with what to analyze, track, and prioritize.  

Even just an hour or two a week is huge! 

Scaling → fractional head of data 

Assuming you have 1-4 data folks (engineers, analysts etc), a Fractional data leader can be a game changer. 

You’ll need someone to mentor your data team and help the executive team scale. 

Scaling is about building systems and having the right data leader will help you build that.

Beyond → Coach / Consultant

Once you’re bigger, you’ll likely have a full time Head of Data but the truth is they’ll need help. 

Going from scale to “corporate” has different demands and most data leaders aren’t prepared for it. 

Bring in advisors and coaches to help your leader scale. 

Everything from how to ask for budget to how to promote people are fundamental skills that can be coached. 

Follow Sundar for Marketing Strategy & Science

li_at

JSESSIONID

LinkedIn App Challenge

Talks about #solopreneurship and #experimentation

CEO: I need to present marketing ROI to the board. 

CMO: How are you defining ROI? 

CEO: The ROI. How much revenue are we generating per dollar spent?

CMO: On which channels? Which timeframe? Are we including brand or just performance?

CEO: All of it. Just give me one number.

CMO: I can't do that responsibly.

CEO: Why not? Finance gives me one number for profitability.

CMO: Did Finance give you the ROI for Product?

CEO: Well... no...

CMO: Okay, exactly. My job is to help you here but it's important that we're aligned on the right story and narrative. Storytelling is kind of my job. What does the board want?

CEO: The board wants to know if Marketing is working.

CMO: That's not specific enough. Are they asking if we should increase budget? Cut budget? Shift budget somewhere else?

CEO: Probably whether we should increase it.

CMO: Then you'll want to show incremental ROI not blended ROI.

CEO: Do we know our incremental ROI?

CMO: On performance channels, yes. On our Brand Marketing efforts, we'd need to run tests to find out.

CEO: The board isn't going to like "we need to run tests."

CMO: Then we give them blended ROI with a massive caveat that it doesn't predict what happens if we scale. Their choice.

CEO: Why is this so complicated?

CMO: Because you're asking for simple answers to complex questions. Marketing isn't accounting. 

CEO: What do other CMOs do?

CMO: The good ones educate their board on what's measurable versus what's theater. The desperate ones give clean numbers that look good but don't mean anything.

CEO: I don't want to be desperate. Help me educate them. 

CMO: Let's set up a session tomorrow so I can help you understand it better first. Also, read some of the articles from the newsletter that I DM'ed you about. 

CEO: experiMENTAL? Done. 

--
🔔 Follow me for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save it so you don't lose it

I help companies scale high-impact experimentation programs that actually drive growth (not just more tests or prettier dashboards).

Over the past decade, I’ve led data and experimentation teams at some of the world’s most product-led companies including Meta, Canva, and Booking.com. I’ve helped teams design better experiments, make smarter decisions, and build the platforms and cultures that support real velocity.

These days, I work as an independent consultant out of Sydney, helping mid-to-large APAC companies turn experimentation into a growth engine. I bring a mix of hands-on support, strategic guidance, and proven frameworks developed from my experience like NAV (a practical lens to measure and improve the Number, Accuracy, and Value of your experiments).

My services include:

🧠 Experimentation strategy sprints
🧪 Fractional leadership and advisory
📈 Workshops & other custom projects

If you're trying to scale experimentation and want an experienced partner who's done it at the highest level, let’s talk.

➡️ Learn more at drsimonj.com

Simon Jackson

Dr Simon Jackson

Building in Experimentation 🚀 Ex-Meta, Canva, Booking.com

Baby Yoda meme with top panel, "Third significant AB test in a row!" Lower panel, "Finding your AA test is always significant too."

The biggest shift happening in Experimentation right now 👇

How things were:
- Running tests was done mostly by tech/digital-first companies
- Experimentation teams had hyper-specialised and technical skills

What I see now:
- Experimentation is being rapidly adopted by every decently-sized company with a website, in every industry
- It’s increasingly being done by people with zero experimentation or stats training.

Why it’s happening:
The business case has become clear: The biggest and best companies have used experiment to outlearn, outpace, and outgrow competition.
The barriers to modifying code are rapidly dropping thanks to AI, and experimentation is becoming more accessible with it.

What leaders should do:
1.  Build organisational competence around metrics, decision-making, and learning loops
2.  Put in place the systems, training, and culture that empower everyone to experiment responsibly

The shift is already happening: experimentation is scaling far beyond the experts.
The leaders who prepare their people for this reality will move faster, learn smarter, and win bigger.

It took me 10 years and thousands of experiments to realise this 👇

The belief I had:
To make a decision based on an experiment, I needed to examine all the data.

What changed it:
I'd constantly find myself in endless debates, never-ending follow up analyses, and almost never making a better decision.
But over time I learned from world-class how to do better...

The new belief:
The best and fastest decisions happen when you front-load the hard thinking.
Build out your metrics and decision criteria without data.
The rest is easy.

If you're constantly debating experiment results and feeling like data just creates more questions than answers:
Stop looking at results until you/your team has agreed on the clear "if-then" statement for acting on the data.
You meet to align on the rules or execute the rules. Never both.

A/A testing is a must.

Here's how it totally changed an old team of mine 👇

Before:
- Experimentation bugs would be flagged randomly, sometimes days or weeks too late.
- Data and engineering teams would waste cycles, scrambling to figure out issues.
- Lots of data, and great experiments, had to be scrapped.

After:
- System bugs were identified within 24 hours at most.
- Teams could isolate why and deploy fixes, fast.
- Major reduction in lost data and experiments.

How we got there:
1. **Ongoing A/A Test:** Constantly run a test where Control and Treatment are identical to check for unexpected metric separation (especially core metrics).
2. **Treat Failure as a P0 Bug:** If the A/A test starts failing seriously, pause testing immediately and fix the underlying issues. Don’t let it run.
3. **Report on failure rates:** Regularly report your A/A failure rates to build confidence and trust in experiments and your team.

If you’re in the “before” state, start by running just one A/A test!
They're easy to setup and you won't regret it.

AB Tasty and VWO are joining forces!

What stands out to me isn’t the deal mechanics, but the signal 👇

Money continues to flow into experimentation.

This is another data point in a growing pattern and, in my view,
it’s not despite AI, but because of it.

As teams ship faster, generate more ideas, and make bigger changes with less friction, the need to:

→ Understand what’s actually working
→ Learn quickly and reliably
→ Detect unintended harm

only increases.

Experimentation isn’t being diminished by AI.
It’s becoming more critical to making good decisions at scale.

And the flow of capital is starting to reflect that.

Curious how others are reading this announcement?

***
🗞️ Announcement link in comments

If you’re the team that runs experiments…

I’ve got bad news for you 👇

World-class product companies don’t treat experimentation as a team.

They treat it as a way of working.

⬇️

In these companies, you won’t hear:
“Speak to the experimentation team.”

You’ll hear:
“Speak to the onboarding team.”
(who run experiments.)

This distinction matters.

⬇️
🎯 First, what teams optimise for.

Less mature teams feel pressure to show motion.
So they optimise for activity.
More experiments. More dashboards. More updates.

World-class teams optimise for coverage.

They run the fewest experiments needed to meaningfully reduce risk and unlock opportunity across the product, not just one surface.

⬇️
📚 Next, how learning is treated.

Less mature teams rarely get challenged.
A “win” is taken at face value.

World-class teams slow this part down.

They invest heavily in:
→ Metric design
→ Guardrails
→ Clear decision rules

So every learning is trustworthy and directional, not just statistically significant.

⬇️
💥 Finally, how impact is defined.

Less mature teams feel constant pressure to justify themselves.
So impact gets reduced to short-term revenue lift.

World-class teams look broader.

They measure how experimentation:
→ Shapes the roadmap
→ Improves strategic clarity
→ Reduces harm
→ Improves long-term decision quality

⬇️
Here’s the real point:

Competitive advantage doesn't just come from running tests.

It’s embedding a system for reliable, high-quality learning, faster than everyone else.

That kind of culture is very hard to copy.

⬇️
So if you’re THE team running experiments in a company where you want everyone to experiment, start here:

👉 Teach another team to experiment.

You can’t build a culture of experimentation if you’re the only ones doing it.

Simple text image: "You can’t build a culture of experimentation if you’re the only ones doing it."

“Marketing should own landing pages. Not product.”

I heard a CRO lead say this recently.

I get why people say this.
I also think it’s one of the most damaging ideas in experimentation.

Because it replaces the most powerful idea we have — letting data decide — with politics and boundaries.

⬇️
❌ What goes wrong

In strong experimentation cultures, ownership isn’t about codebases.
It’s about outcomes.

Yes, teams need clear responsibility for maintaining systems.

But when “ownership” turns into exclusive rights to make changes, you get:

👉 Teams optimising what they’re allowed to touch
👉 Not what actually moves the metric

⬇️
🎯 Outcomes don’t respect org charts 

If a team’s goal is acquisition, they’ll care about:

→ Landing pages
→ Ads and creative
→ Sign-up flows
→ Onboarding
→ First activation
→ Sharing mechanics

Putting hard borders around these surfaces doesn’t create focus.
It creates local optimisation and global underperformance.

⬇️
🚀 What high-performing teams do instead

In the most innovative companies I’ve worked with:

✔ Almost anyone can test changes almost anywhere
✔ Clear guardrails exist
✔ Data, not hierarchy, decides

Not because chaos is good.
But because speed and evidence beat hierarchy every time.

⬇️
🧭 So what should ownership look like? 

Marketing shouldn’t “own landing pages”.
Marketing should own creating demand.

Product shouldn’t “own the funnel”.
Product should own building great experiences.

And experimentation exists precisely to let teams cross boundaries...
guided by outcomes, not job titles.

If this makes you uncomfortable, that’s probably the point.

Simple text image: "Experimentation lets teams cross boundaries guided by outcomes, not org charts."

Most growth/CRO teams *say* they're data-driven...

But very few understand what "pre-registration" actually means 👇

What pre-registration means: 
- It's a commitment to *which* metrics matter *before* a test starts.
- It covers the primary metric, key secondary metrics, and crucial guardrails (like technical stability or fraud).
- It formalises the decision rule: when will we ship, iterate, or kill the test?

Why it matters:
- It eliminates confirmation and hindsight bias, preventing you from cherry-picking metrics after the results are in.
- It forces clarity and alignment across product, data, and leadership *before* resources are committed.

Common failure modes:
- Teams register the primary metric but forget about others (e.g., shipping a win that secretly broke the platform).
- Teams register too many metrics, making the post-test story muddy and confusing (you need focus).

What to do instead:
1.  Include primary, key secondary, and guardrail metrics in your experiment planning template. If it's not documented, it doesn't count.
3.  Keep your criteria clear and tight. Usually, primary should improve, secondaries to move as expected, guardrails not negative.

If a metric isn't in the plan, it's irrelevant to the decision

taplio

The Battle

Sundar Swaminathan

Simon Jackson

Followers

Posts

Engagements

Likes

Performance ScoreSundar Swaminathan's LinkedIn Performance

Performance ScoreSimon Jackson's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Sundar Swaminathan

Simon Jackson

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Sundar Swaminathan's LinkedIn Performance

Performance ScoreSimon Jackson's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreSundar Swaminathan's LinkedIn Performance