eCom Unity #3 - Measuring retention with incrementality testing

Deep-Dive | Measuring the real impact of retention tactics

Last week, I was trying to show you that looking at the average CLV is misleading. As in, it doesn’t really show how much extra lifetime value you generate from an incremental second purchase.

This week, I want to show you the one thing you need to have in place as an eCom brand if you want to take your lifetime value seriously:

Incrementality testing.

Here is what I mean and how you can easily implement it:

1️⃣ What is incrementality testing?

With incrementality testing you can measure the additional revenue/profit you generate from taking a certain action.

2️⃣ Why should you care about it?

There are two main reasons why incrementality testing is important (especially when it comes to retention).

1. Selection Bias - all the things that you do to boost retention like email, WhatsApp, an app, a loyalty program are all things that your best customer are more inclined to use in the first place. So it won't be a surprise that the CLV of customers who downloaded the app, will be higher.

2. CLV is a lagging indicator - a higher CLV is the outcome of good retention and therefore makes it tough to analyse what led to it.

3️⃣ How do you implement it?

Quite simple.

Use customer tags to group your customers randomly into different groups.

E.g. 0-9 based on the last digit in their customer ID (this can be done automatically via Shopify flows for example).

Then whenever you want to implement something that increases retention, you only expose a sub-segment of your customers to this approach.

E.g. You only invite people with the group 1+2 to download the app over the course of a month but leave out the remaining groups.

Then you have a clear, randomly assigned split, of which customers were asked to download the app.

You’ll find a more detailed explanation of how to set this up for your shop in the next section below 👇️ 

4️⃣ How does it help you?

With this split, you can clearly measure the incremental impact of asking people to download the app.

Simply use cohort reports in Klar to cluster your customers based on these customer tags and select the date range during which you ran the split.

Then you can compare the repeat rate and customer lifetime value of these customers.

In the example in the image, we can see that groups 1+2 have a repeat rate of around 40% whereas the remaining groups are at around 35%. An uplift of 15%.

And CLV is also higher at €160 vs €125. An uplift of 28%.

Since customers are placed into the groups completely at random, the only difference between them is whether or not you asked them to download your app.

Thus, you can be sure that there is a clear incremental uplift.

In this case €35 per customer in contribution margin.

At around 20.000 customers this store is acquiring per month, results in a total additional contribution margin of €700.000

Now to be honest, this is not rocket science.

Yet, I can count the number of brands doing is on one, maybe two hands.

In times, where profitability is priority number one, two, and three, every investment needs to prove it's worth it.

And randomly created cohort incrementality tests are the best way to do just that.

Deep Dive | How to set up incrementality testing

So how do I automatically set up these tags I was talking about in the previous section in my store?

With Klar, this is super simple to do if you have the right customer tags in place, where each customer is randomly assigned to a group, and only a specific group(s) is exposed to this new tactic.

You can then use the Cohort report to see how these customers behave relative to everyone else and since this group is created randomly, the difference in CLV will be the incremental impact.

Now, back to the original questions.

How do I set up these tags?

Simple. You just need to do two things:

1️⃣ Set Up a Shopify Flow for new customers

Build a new flow in Shopify where new customers get automatically assigned a tag based on the last ID in their customer profile.

This only takes 5 minutes.

2️⃣ Upload tags for old customer

For existing customers, you unfortunately need to do a manual upload.

But again, this is super simple.

Just download a list of your customers.

Copy this Excel formula to create a new tag based on their customer ID:

=IF(T2="","Incrementality Test - "&RIGHT(A2,1),T2&", Incrementatlity Test - "&RIGHT(A2,1))

And then just reupload your customers and all tags will be added.

And that’s it.

Takes around 15 minutes to do, but will give you everything you need to get started with incrementality testing.

There’s also this full step-by-step video guide on how to set this up:

Brand Breakdown | Pip Decks’ impressive creative collection

You might not know Pip Decks. But they are the hidden champions of paid social ads scaling to mid 8-figures with “just” card decks. Card decks for over $100.

What stands out in their creative strategy?

Diverse Creatives & Personas. Platform-Native Content. Benefit-Driven Approach.

I created an overview of their 32 top-performing ads that are getting the most reach (meaning the most spend) in a Miro board and went over them in a deep-dive video.

You can get access here: https://getklar.typeform.com/pipdecks

Upcoming Events | “Spice up your eCom game”

🇬🇧 For all London-based eCommerce leaders: we’re hosting a cooking class next week where you will collaborate with professional chefs and at the same time gain insights from other eCommerce experts on their growth strategies.

Tue, October 8, 7 pm. Apply here: https://www.getklar.com/ecomunity-events/spice-up-your-ecom-game-ecommerce-leaders-cooking-class

🇩🇪 Same thing, but in Hamburg: an exclusive cooking class for eCommerce leaders to exchange, cook together, of course, and listen to TEVEO sharing insights in their growth formula.

Thu, October 24, 7 pm. Apply here: https://www.getklar.com/ecomunity-events/the-secret-sauce-ecommerce-leaders-cooking-class

Thank you for reading :)

I am on a mission to help eCom brands build better, more profitable businesses.

I have built one 8-figure and one 9-figure eCom brand in the past and with Klar, I work with over 700 eCom brands today.

If you have specific questions you would like me to cover next, just reply to this email.

Have a great week.

Max ✌️

Not part of the eCom Unity yet? (Brands only)

Click here to directly exchange with hundreds of other eCom decision-makers from the coolest brands such as Kapten & Son, Purelei, KoRo, Junglück, Blackroll, and many more in our closed Slack community.

No agencies. No sales pitches. High-class content & events. 100% free.