Build a loyalty points anomaly detector using Amazon Lookout for Metrics

by
Amazon AWS

Today, gaining customer loyalty cannot be a one-off thing. A brand needs a focused and integrated plan to retain its best customers—put simply, it needs a customer loyalty program. Earn and burn programs are one of the main paradigms. A typical earn and burn program rewards customers after a certain number of visits or spend.

For example, a fast food chain has launched its earn and burn loyalty pilot program in some locations. They are looking to use the loyalty program to make their customer experience more personal. Upon testing, they want to expand it to more locations across different countries in the future. The program allows customers to earn points for every dollar that they spend. They can redeem the points toward different rewards options. To attract new customers, they also give points to new customers. They test the redeem pattern every month to check the performance of the loyalty program at different locations. Identifying redeem pattern anomalies is crucial in order to take corrective action in time and ensure the overall success of the program. Customers have different earn and redeem patterns at different locations based on their spend and choice of food. Therefore, the process of identifying an anomaly and quickly diagnosing the root cause is difficult, costly, and error-prone.

This post shows you how to use an integrated solution with Amazon Lookout for Metrics to break these barriers by quickly and easily detecting anomalies in the key performance indicators (KPIs) of your interest.

Lookout for Metrics automatically detects and diagnoses anomalies (outliers from the norm) in business and operational data. You don’t need ML experience to use Lookout for Metrics. It’s a fully managed machine learning (ML) service that uses specialized ML models to detect anomalies based on the characteristics of your data. For example, trends and seasonality are two characteristics of time series metrics in which threshold-based anomaly detection doesn’t work. Trends are continuous variations (increases or decreases) in a metric’s value. On the other hand, seasonality is periodic patterns that occur in a system, usually rising above a baseline and then decreasing again.

In this post, we demonstrate a common loyalty points earn and burn scenario, in which we detect anomalies in the customer’s earn and redeem pattern. We show you how to use these managed services from AWS to help find anomalies. You can apply this solution to other use cases such as detecting anomalies in air quality, traffic patterns, and power consumption patterns, to name a few.

Solution overview

This post demonstrates how you can set up anomaly detection on a loyalty points earn and redeem pattern using Lookout for Metrics. The solution allows you to download relevant datasets and set up anomaly detection to detect earn and redeem patterns.

Let’s see how a loyalty program typically works, as shown in the following diagram.

Customers earn points for the money they spend on the purchase. They can redeem the accumulated points in exchange for discounts, rewards, or incentives.

Building this system requires three simple steps:

  1. Create an Amazon Simple Storage Service (Amazon S3) bucket and upload your sample dataset.
  2. Create a detector for Lookout for Metrics.
  3. Add a dataset and activate the detector to detect anomalies on historical data.

Then you can review and analyze the results.

Create an S3 bucket and upload your sample dataset

Download the file loyalty.csv and save it locally. Then continue through the following steps:

  1. On the Amazon S3 console, create an S3 bucket to upload the loyalty.csv file.

This bucket needs to be unique and in the same Region where you’re using Lookout for Metrics.

  1. Open the bucket you created.
  2. Choose Upload.

  1. Choose Add files and choose the loyalty.csv file.
  2. Choose Upload.

Create a detector

A detector is a Lookout for Metrics resource that monitors a dataset and identifies anomalies at a predefined frequency. Detectors use ML to find patterns in data and distinguish between expected variations in data and legitimate anomalies. To improve its performance, a detector learns more about your data over time.

In our use case, the detector analyzes daily data. To create the detector, complete the following steps:

  1. On the Lookout for Metrics console, choose Create detector.
  2. Enter a name and optional description for the detector.
  3. For Interval, choose 1 day intervals.
  4. Choose Create.

Your data is encrypted by default with a key that AWS owns and manages for you. You can also configure if you want to use a different encryption key from the one that is used by default.

Now let’s point this detector to the data that you want it to run anomaly detection on.

Create a dataset

A dataset tells the detector where to find your data and which metrics to analyze for anomalies. To create a dataset, complete the following steps:

  1. On the Lookout for Metrics console, navigate to your detector.
  2. Choose Add a dataset.

  1. For Name, enter a name (for example, loyalty-point-anomaly-dataset).
  2. For Timezone, choose as applicable.
  3. For Datasource, choose your data source (for this post, Amazon S3).
  4. For Detector mode, select your mode (for this post, Backtest).

With Amazon S3, you can create a detector in two modes:

  • Backtest – This mode is used to find anomalies in historical data. It needs all records to be consolidated in a single file. We use this mode with our use case because we want to detect anomalies in a customer’s historical loyalty points redeem pattern in different locations.
  • Continuous – This mode is used to detect anomalies in live data.
  1. Enter the S3 path for the live S3 folder and path pattern.
  2. Choose Detect format settings.
  3. Leave all default format settings as is and choose Next.

Configure measures, dimensions, and timestamps

Measures define KPIs that you want to track anomalies for. You can add up to five measures per detector. The fields that are used to create KPIs from your source data must be of numeric format. The KPIs can be currently defined by aggregating records within the time interval by doing a SUM or AVERAGE.

Dimensions give you the ability to slice and dice your data by defining categories or segments. This allows you to track anomalies for a subset of the whole set of data for which a particular measure is applicable.

In our use case, we add two measures, which calculate the sum of the objects seen in the 1-day interval, and have one dimension, for which earned and redeemed points are measured.

Every record in the dataset must have a timestamp. The following configuration allows you to choose the field that represents the timestamp value and also the format of the timestamp.

The next page allows you to review all the details you added and then choose Save and activate to create the detector.

The detector then begins learning the data inthe data source. At this stage, the status of the detector changes to Initializing.

It’s important to note the minimum amount of data that is required before Lookout for Metrics can start detecting anomalies. For more information about requirements and limits, see Lookout for Metrics quotas.

With minimal configuration, you have created your detector, pointed it at a dataset, and defined the metrics that you want Lookout for Metrics to find anomalies in.

Review and analyze the results

When the backtesting job is complete, you can see all the anomalies that Lookout for Metrics detected in the last 30% of your historical data. From here, you can begin to unpack the kinds of results you will see from Lookout for Metrics in the future when you start getting the new data.

Lookout for Metrics provides a rich UI experience for users who want to use the AWS Management Console to analyze the anomalies being detected. It also provides the capability to query the anomalies via APIs.

Let’s look at an example anomaly detected from our loyalty points anomaly detector use case. The following screenshot shows an anomaly detected in loyalty points redemption at a specific location on the designated time and date with a severity score of 91.

It also shows the percentage contribution of the dimension towards the anomaly. In this case, 100% contribution comes from the location ID A-1002 dimension.

Clean up

To avoid incurring ongoing charges, delete the following resources created in this post:

  • Detector
  • S3 bucket
  • IAM role

Conclusion

In this post, we showed you how to use Lookout for Metrics to remove the undifferentiated heavy lifting involved in managing the end-to-end lifecycle of building ML-powered anomaly detection applications. This solution can help you accelerate your ability to find anomalies in key business metrics and allow you focus your efforts on growing and improving your business.

We encourage you to learn more by visiting the Amazon Lookout for Metrics Developer Guide and trying out the end-to-end solution enabled by these services with a dataset relevant to your business KPIs.

About the Author

Dhiraj Thakur is a Solutions Architect with Amazon Web Services. He works with AWS customers and partners to provide guidance on enterprise cloud adoption, migration, and strategy. He is passionate about technology and enjoys building and experimenting in the analytics and AI/ML space.

Read More