Retention: which metrics to watch and which to ignore

There are plenty of retention metrics, and each one has its fans. An early team needs only a few, calculated correctly. Most mistakes here aren’t about picking the wrong metric. They’re about definitions: what counts as active, which date you count from, and what window you look through.

Define “active” first

Opening the app isn’t activity. Someone may have tapped a notification and closed it a second later. Activity is the action the product exists for: sent an invoice, finished a workout, published a post, ordered a delivery. Pick one such action and measure retention against it.

The second definition is the window. A product for daily tasks should be measured in days or weeks. A service people naturally use once a month will look dead on daily metrics even when it’s perfectly healthy. The window should match the natural frequency of the task, not whatever makes the chart easier to build.

Cohorts

A single “monthly active users” number mixes people who arrived yesterday with people who arrived a year ago. New signups hide old users leaving, and the other way round. That’s why retention is read by cohort: groups that arrived in the same week or month.

Useful things to do with cohorts:

• compare neighboring cohorts: if later ones retain better than earlier ones, the product is improving;
• split cohorts by source: people from different channels often behave differently;
• mark releases and onboarding changes on the chart so you can see what affected what.

One cohort is an observation. A trend across several is a conclusion.

Activation

Many people leave without ever understanding why they came. Activation is the moment someone first gets value: sees their first report, takes their first order, invites a colleague and works with them. Find that moment and you have a metric that predicts retention before retention can be measured.

How to find it: compare people who were still around after a month with those who left, and look at what the first group did in their first days that the second didn’t. That’s correlation, not cause, so you then need to test it: nudge new users toward that action and see whether retention moves.

Keep two numbers in view: the share of new users who reach activation, and how long it takes them. Both can usually be improved without touching the core product, through onboarding, hints and the first few emails.

Frequency

Retention answers “did they come back?” Frequency answers “how often?” Someone who shows up once a month and someone who shows up every day look identical in a cohort table, and mean very different things for the product.

A simple way to see frequency is the distribution of users by number of active days per week or per month. If there’s a visible group using the product almost daily, that’s your core. Look at who they are, where they came from and what they do differently. Those are the people to find more of.

For paid products, add revenue retention: how much a cohort brings in after six months compared with its first month. If customers move up to more expensive plans, a cohort’s revenue can grow even after some of its people have left.

What to ignore

Numbers that grow on their own and tell you nothing:

• cumulative signups, which by definition can’t go down;
• downloads and installs with no action afterwards;
• page views and average session time, where “long” sometimes means “couldn’t find the button”;
• social media followers;
• “active users” with no definition of active.

These metrics are nice to show, but nobody makes decisions with them. If a change in a number wouldn’t tell you to do anything differently, you can stop looking at it.

In short

One key action, a window that matches natural frequency, cohorts instead of totals, activation as an early signal and frequency as a measure of depth. That’s enough to see what’s actually happening with the product.

SUBXCAT can help set up these metrics in your product and build a report the team actually reads.