Why Metrics Get Worse With Scale

Conventional wisdom suggests that the most important metrics for a startup – such as unit economics, cost of acquisition, lifetime value, churn rates – typically get better with time. I hear this asserted frequently by entrepreneurs who confidently project their businesses with increasingly improving metrics as they scale into the future.

The topic of scaling startups is one that I enjoy thinking, living and writing about (most recently, Scaling the Chasm).  In the class I teach at Harvard Business School, the first module of the course is dedicated to examining startups when they are pre-product market and struggling to find product-market fit while the second module is dedicated to what the challenges of scale post product-market fit.

One of the themes I explore in the class is the tough reality that many metrics can actually get worse over time for a startup. Take growth rate as a simple one.  The law of large numbers suggests it is easier to double in size when you are doing $1 million in revenue as compared to when you are doing $10 million, never mind $100 million. Thus, more mature companies naturally have slower growth rates than younger ones. Here are a few other key metrics that are hard to scale:

Customer acquisition. Most of the marketing techniques that look good in the early days cannot be scaled 10x, never mind 100x. For example, PR doesn’t scale. It seems like such an amazingly efficient source of customers, yet ask any marketing communications or PR professional to acquire 10x the number of customers that they did last year and they’ll look at you as if you have 10 heads.  Search engine marketing (SEM) and app store optimization (ASO) exploit arbitrage opportunities in keywords and placement, but those arbitrage opportunities are effective only for a moment in time and for a certain level of spend. When you spend more, you risk losing that edge. Similarly, if you try to scale email too much, you quickly risk fatiguing your list and spending money acquiring less valuable customers when compared to your core segment.

Customer acquisition is like drilling for oil.  A particularly successful tactic allows you to find a gusher, which you can take advantage of for a while, but eventually the well dries out and you have to find another well.  One of my CEOs pointed out to me at a board meeting last week:

“Our average customer acquisition cost (CAC) is irrelevant for the future. It is the marginal CAC that matters the most – that is, what does it cost to acquire the next incremental set of customers?”

Word of mouth, referrals, virality – these are all amazingly powerful customer acquisition techniques that hold the promise of scale, but they require you to have a great product, not just a great marketing plan, and a product that is elegantly design for virality.

Churn rates are another metric that can get harder with scale. When you expand your market, the next market segment may not be as perfect a “bullseye” market fit as the early segments and early customers. Even as the product matures, the customers that are recently acquired that represent newer segments can be less dedicated. A new battle for product-market fit must be waged – something that never ends – particularly as you expand into new customer segments and verticals.

Monetization can get harder with scale as well. Monetizing the initial user base – who is your most dedicated and often organically acquired – is easier than the more marginal users who you are spending incrementally more money to acquire from indirect channels that may not produce as loyal customers as the initial channels. Even a company as amazing and well run as TripAdvisor (who I once claimed had a better business model than anyone outside the mob) has seen average revenue per user (ARPU) decline over the years, from $14.10 in 2009 to $11.80 in 2012. During that period of time, their monthly uniques grew over 2.5x, from 25m to 65m. More recently, with the shift to mobile and the growth in emerging markets, this ARPU decline has become even more dramatic as mobile visitors and international visitors monetize at a lower rate than their earlier segments of online, US visitors.

The trick to keeping your metrics steady during growth, if not improving over time, is to find a series of techniques and keep improving on them as you go.  That’s why so many great entrepreneurs obsess over the details of landing page wording, button placement and color on a page, creative copy, etc. They know that being able to scale 10x from where they are has no silver bullet, but rather a series of tactics that need to be executed against. And they recognize that often times, as you are scaling 10x and 100x, your metrics may erode on the margin.

If your core metrics only erode 10-20% while you are scaling fast, like TripAdvisor’s ARPU, you are in pretty good shape. If they erode 50-75%, you are in deep trouble. Just remember, don’t project to investors that every metric is going to get better over time. Otherwise, you will be dismissed as naïve, out of touch, overly optimistic, insane or all of the above. Never a good combination.

4 thoughts on “Why Metrics Get Worse With Scale

  1. This analysis does not take into cognisance the type of Customer Aquisition and Frequency and value of the purchase by the acquired customer during a financial year in the short term / medium terms and long term horizon. So if a customer once acquired remains loyal and makes frequent purchases , then the CAC over time horizons will come down. So the categorisation or acquired customers must be analysed over a period of time to understand the quality of the customers acquired. Today , startups are burning money of investors purely on existentialistic basis and have no commitments to trhe company’s future.Investors must ensure that revellers of the party on investors money are seggregated from the serious entreprenueurs .
    Nirmal Kumar Bhardwaj
    ceo@thebengaluruangels.com

    Like

  2. We explicitly account for metric erosion, which keeps our growth and unit economics measures on track. I think that the whole point of raising venture capital is so that you can invest in minimizing metric erosion. For example, if at the point where you are growing at 3x per year, you can maintain 85% of that growth rate for 5 years, you’re going to have a terrific IPO – and you get that by investing in product, brand, ecosystem and other moats.

    Like

  3. Great article. So few startup projections reflect even a facsimile of the real world. Flattening growth curves should be a sign of success – provide you reach a certain scale first of course!

    Like

  4. Awesome article Jeff. Agree 100%. Very few forward-looking projections account for metric erosion, which is why most of them are absurd beyond 12 months.

    Like

Leave a comment