Forecasting and detecting anomalies in active users

If you’re a product manager, then you know that tracking active users is one of the most important things you can do to measure your product’s success. But what happens when your active user numbers start trending down or suddenly spike up? How do you know if it’s just a normal variation, or if there’s actually something wrong with your product?

In this blog post, we’ll show you how to use forecasting and anomaly detection insights from Stormly to keep tabs on your active users metric and detect any unexpected changes. Stay ahead of any potential problems before they happen!

Before you dive in, you might want an introduction to measuring daily active users and stickiness ratios.

Why daily active users matter

Some say that daily active users (DAU) is just a vanity metric. Why is it then quoted often in stories about the hottest tech IPOs or quarterly results of publicly traded companies?

When the number of daily active users is growing, it’s usually an indication that the company is doing something right.

Growth in active users can come from two sources:

1. Newly acquired users, by investing in user acquisition.
2. Existing users you already have, that keep coming back, which is what we call the stickiness of a product.

So, DAU can be seen as a way to gauge the potential of growth for a product. And for many companies – especially the ones dependent on advertising as the main source of revenue, or the ones with networking effects – the more users are active, the more useful the product. And this leads to more revenue.

Daily active users are thus seen by many industry people, analysts and product owners as an indirect measure of stickiness and (potential) growth.

It’s no surprise that when the biggest social network of our time announced that their DAU was declining for the first time in its history, it led to the single largest daily decline of any stock in history. It’s a clear indication that growth has plateaued, and for a product with network effects, it usually means a decline into becoming obsolete.

So it’s probably stating the obvious, but it’s crucial to keep an eye on DAU growth and future trends. In the next section, we’ll show you how easy it is with Stormly to forecast and get the trend of DAU growth.

In the second section, we’ll show you how you can keep tabs on DAU growth by using anomaly detection to alert you whenever DAU significantly drops or increases.

Forecasting and trending daily active user growth

Forecasting DAU is really simple using Stormly’s forecasting Insight. Once you’ve opened the insight, you get a few options for product related metrics that you can forecast. In our case, we choose “active users”.

Let’s decide we want to forecast the next two weeks. So instead of 7, we select 14 days to forecast. It also lets us define what it means to be an active user. In this case, we’re working on a music streaming app, and consider a user active only if they play or share a song. We simply pick those two actions, and we’re ready to go.

Before we run our forecast, we’d like to tap into Stormly’s super-powers. These are the filtering and segmentation options that are available at the top of any Insight. These allow us to compare the forecast for different countries or specific behavior, for example.

For now, we’re only interested in forecasting users that came from the US, so that’s what we’re going to use in our insight filter.

After we run our Insight, we get a clear picture of the forecasted and expected future trend for our daily active users that play or share a song, and are from the US.

We can see from our chart that there is a weekly ebb and flow of active users, where the peak starts on Thursdays and then slowly declines again towards Wednesdays.

This can also be seen in our forecasted active users, which are visible on the chart showing the lower and upper forecast bands.

If we look at the day-of-week effect, we can confirm that Thursdays indeed have relatively more active users, and they slowly decline in the days after, with the biggest drop on Wednesdays.

Let’s look back at our forecast chart above from earlier. On the right side, we can see a smoothed trend-line with weekly fluctuations taken out. As we’ve seen before, daily fluctuations is something we definitely have in our music streaming app. As we can see, the historical trend (purple part), as well as the forecasted trend-line (green part), indicate that the number of active users from the US is expected to increase.

So we can be assured that we’re doing something right, either in terms of new user acquisition or existing users coming back.

Detecting anomalies in daily active users

The next and final step in being fully on top of active user growth, is to be alerted whenever a positive or negative anomaly occurs.

A sudden drop in active users can indicate different types of underlying problems. One you want to be on top of as soon as they happen, are technical issues.

If your website or app is malfunctioning for whatever reason, the number of active users will take a nosedive for sure as soon as this happens.

To start setting up your active user anomaly detection, open the “Anomaly Detection” Insight. Now select the option “Active Users” to detect anomalies in that metric.

The next step is to set up what defines an active user. We’ll set it up similar to how we did for forecasting, so only users who play or share a song are considered active.

All other options relate to anomaly detection itself. For example, we can decide to also get alerted when there are positive anomalies. That is, when there are more active users than expected.

Once we run our insight, it will show if there are any anomalies in our historical data as well. As we can see below, we currently have no negative anomalies in the amount of active users.

But our active users can always drop unexpectedly. So we want to make sure we save this insight. Once it’s saved, we’ll be automatically notified when the number of active users drops.

That’s all we need to do to keep an eye on forecasted and trends in our active users, as well as getting notified when the number of active users changes significantly!

By using anomaly detection on your active users metric with Stormly, you can keep tabs on any sudden changes and take proactive steps to prevent them from becoming bigger problems. Not only that, but you can also use Stormly’s forecasting capabilities to predict future growth and plan for it accordingly.

Sign up today and get started improving your product in seconds!