Amazon Personalize now supports dynamic filters for applying business rules to your recommendations on the fly

by
Amazon AWS

We’re excited to announce dynamic filters in Amazon Personalize, which allow you to apply business rules to your recommendations on the fly, without any extra cost. Dynamic filters create better user experiences by allowing you to tailor your business rules for each user when you generate recommendations. They save you time by removing the need to define all possible permutations of your business rules in advance and enable you to use your most recent information to filter recommendations. You control the recommendations for your users in real time while responding to their individual needs, preferences, and changing behavior to improve engagement and conversion.

For online retail use cases, you can use dynamic filters in Amazon Personalize to generate recommendations within the criteria specified by the shopper, such as brand choices, shipping speed, or ratings. For video-on-demand users, you can make sure you include movies or television shows from their favorite directors or actors based on each individual user’s preferences. When users subscribe to premium services, or their subscription expires, you can ensure that their content recommendations are based on their subscription status in real time.

Based on over 20 years of personalization experience, Amazon Personalize enables you to improve customer engagement by powering personalized product and content recommendations and targeted marketing promotions. Amazon Personalize uses machine learning (ML) to create higher-quality recommendations for your websites and applications. You can get started without any prior ML experience, using simple APIs to easily build sophisticated personalization capabilities in just a few clicks. All your data is encrypted to be private and secure, and is only used to create recommendations for your users.

Setting up and applying filters to your recommendations is simple; it only takes only a few minutes to define and deploy your business rules. You can use the Amazon Personalize console or API to create a filter with your logic using the Amazon Personalize DSL (Domain Specific Language).

To create a filter that is customizable in real-time, you define your filter expression criteria using a placeholder parameter instead of a fixed value. This allows you to choose the values to filter by applying a filter to a recommendation request, rather than when you create your expression. You provide a filter when you call the GetRecommendations or GetPersonalizedRanking API operations, or as a part of your input data when generating recommendations in batch mode through a batch inference job.

This post walks you through the process of defining and applying item and user metadata-based recommendation filters with statically or dynamically defined filter values in Amazon Personalize.

Prerequisites

To define and apply filters, you first need to set up some Amazon Personalize resources on the Amazon Personalize console. For full instructions, see Getting Started (Console).

  1. Create a dataset group.
  2. Create an Interactions dataset using the following schema : 
    {
    "type": "record",
    "name": "Interactions",
    "namespace": "com.amazonaws.personalize.schema",
    "fields": [
        {
            "name": "USER_ID",
            "type": "string"
        },
        {
            "name": "ITEM_ID",
            "type": "string"
        },
        {
            "name": "EVENT_TYPE",
            "type": "string"
        },
        {
            "name": "EVENT_VALUE",
            "type": ["null","float"]
        },
        {
            "name": "IMPRESSION",
            "type": [“null”,"string"]
        },
        {
            "name": "TIMESTAMP",
            "type": "long"
        }
    ],
    "version": "1.0"
}

  3. Import the data using the following data file.
  4. Create an Items dataset using the following schema: 
    {
	"type": "record",
	"name": "Items",
	"namespace": "com.amazonaws.personalize.schema",
	"fields": [
		{
			"name": "ITEM_ID",
			"type": "string"
		},
		{
			"name": “TITLE”,
			"type": "string"
		},
		{
			"name": "GENRE",
			"type": [
				"null",
				"string"
			],
			"categorical": true
		},
		{
            		"name": "CREATION_TIMESTAMP",
            		"type": "long"
        		}
	],
	"version": "1.0"
}

  5. Import data using the following data file.
  6. Create a Users dataset using the following schema: 
    {
    "type": "record",
    "name": "Users",
    "namespace": "com.amazonaws.personalize.schema",
    "fields": [
        {
            "name": "USER_ID",
            "type": "string"
        },
        {
            "name": "AGE",
            "type": ["null", "int"]
        },
        {
            "name": “SUBSCRIPTION”,
            "type": [“null”,”string”]
        }
    ],
    "version": "1.0"
}

  1. Import the data using the following data file.
  2. Create a solution using any recipe. In this post, we use the aws-user-personalization
  3. Create a campaign.

Creating your filter

Now that you have set up your Amazon Personalize resources, you can define and test your custom filters.

Filter expression language

Amazon Personalize uses its own DSL called filter expressions to determine which items to exclude or include in a set of recommendations. Filter expressions can only filter recommendations based on datasets in the same dataset group, and you can only use them to filter results for solution versions (an Amazon Personalize model trained using your datasets in the dataset group) or campaigns (a deployed solution version for real-time recommendations). Amazon Personalize can filter items based on user-item interactions, item metadata, or user metadata datasets. For filter expression values, you can specify fixed values or add placeholder parameters, which allow you to set the filter criteria when you get recommendations from Amazon Personalize.

Dynamically passing values in filters is only supported for IN and = operations. For range queries, you need to continue to use static filters. Range queries use the following operations: NOT IN, <, >, <=, and >=.

The following are some examples of filter expressions.

Filtering by item

To remove all items in a genre chosen when you make your inference call with a filter applied, use the following filter expression:

EXCLUDE ItemId WHERE items.genre in ($GENRE)

To remove all items in the Comedy genre, use the following filter expression:

EXCLUDE ItemId WHERE items.genre in ("Comedy")

To include items with a number of downloads less than 20, use the following filter expression:

INCLUDE ItemId WHERE items.number_of_downloads < 20

This last filter expression includes a range query (items.number_of_downloads < 20). To perform this query in an Amazon Personalize recommendation filter, it needs to be predefined as a static filter.

Filtering by interaction

To remove items that have been clicked or streamed by a user, use the following filter expression:

EXCLUDE ItemId WHERE interactions.event_type in (“click”, “stream”)

To include items that a user has interacted with in any way, use the following filter expression:

INCLUDE ItemId WHERE interactions.event_type in ("*")

Filtering by item based on user properties

To exclude items where the number of downloads is less than 20 if the current user’s age is greater than 18 but less than 30, use the following filter expression:

EXCLUDE ItemId WHERE items.number_of_downloads < 20 IF CurrentUser.age > 18 AND CurrentUser.age < 30

You can also combine multiple expressions together using a pipe ( | ) to separate them. Each expression is evaluated independently and the recommendations you receive is the union of those expressions.

The following filter expression includes two expressions. The first expression includes items in the Comedy genre; the second expression excludes items with a description of classic, and the results are the union of the results of both filters.

INCLUDE Item.ID WHERE items.genre IN (“Comedy”) | EXCLUDE ItemID WHERE items.description IN ("classic”)

Filters can also use multiple filter expressions with dynamically passed values to be applied to your recommendations. The first expression includes a genre defined in the request for recommendations with the $GENRE value, and second expression excludes a description defined in the request for recommendations with the $DESC value. The result of the filter is the union of the results of both expressions that includes the $GENRE but excludes items with a description defined in the request for recommendations with the $DESC value:

INCLUDE Item.ID WHERE items.GENRE IN ($GENRE) | EXCLUDE ItemID WHERE item.DESCRIPTION IN ("$DESC”)

For more information, see Datasets and Schemas. For further details on filter definition DSL, see our documentation.

Creating a filter on the Amazon Personalize console

You can use the preceding DSL to create a customizable filter on the Amazon Personalize console. Complete the following steps:

  1. On the Amazon Personalize console, on the Filters page, choose Create filter.
  2. For Filter name, enter the name for your filter.
  3. Select Build expression or add your expression manually to create your custom filter.
  4. Enter a value of $ plus a parameter name that is similar to your property name and easy to remember (for example, $GENRE).
  5. Optionally, to chain additional expressions with your filter choose, +.
  6. To add additional filter expressions, choose Add expression.
  7. Choose Finish.

Creating a filter takes you to a page containing detailed information about your filter. Here you can view more information about your filter, including the filter ARN and the corresponding filter expression you created (see the following screenshot). You can also delete filters on this page or create more filters from the summary page.

You can also create filters via the createFilter API in Amazon Personalize. For more information, see CreateFilter.

Range queries are not supported when dynamically passing values to recommendations filters. For example, filters excluding items with less than 20 views in the items metadata must be defined as static filters.

Applying your filter via the Amazon Personalize console

The Amazon Personalize console allows you to spot-check real-time recommendations from the Campaigns page. On this page, you can test your filters while retrieving recommendations for a specific user on demand. To do so, navigate to the Campaign tab; this should be in the same dataset group that you used to create the filter. You can then test the impact of applying the filter on the recommendations.

The following screenshot shows results when you pass the value Action as the parameter to a filter based on the items’ genre.

When applying filters to your recommendations in real time via the console, the Filter parameters section auto-populates with the filter parameter and expects a value to be passed to the filter when you choose Get recommendations button.

Applying your filter via the SDK

You can also apply filters to recommendations that are served through your SDK or APIs by supplying the filterArn as an additional and optional parameter to your GetRecommendations calls. Use filterArn as the parameter key and supply the filterArn as a string for the value. filterArn is a unique identifying key that the CreateFilter API call returns. You can also find a filter’s ARN on the filter’s detailed information page.

The following example code is a request body for GetRecommendations API that applies a filter to a recommendation:

{
    "campaignArn": "arn:aws:personalize:us-west-2:000000000000:campaign/test-campaign",
    "userId": "1",
    "itemId": "1",
    "numResults": 5,
    "filterArn": "arn:aws:personalize:us-west-2:000000000000:filter/test-filter"
    "filter-values": ‘[{
	"GENRE": ““ACTION”, “HORROR””
	}]

Summary

Recommendation filters in Amazon Personalize allow you to further customize your recommendations for each user in real time to provide an even more tailored experience that improves customer engagement and conversion. For more information about optimizing your user experience with Amazon Personalize, see What Is Amazon Personalize?

 

About the Author

Vaibhav Sethi is the Product Manager for Amazon Personalize. He focuses on delivering products that make it easier to build machine learning solutions. In his spare time, he enjoys hiking and reading.

 

 

 

 Samuel Ashman is a Technical Writer for Amazon Personalize. His goal is to make it easier for developers to use machine learning and AI in their applications. His studies in computer science allow him to understand the challenges developers face. In his free time, Samuel plays guitar and exercises.

 

 

 

Parth Pooniwala is a Senior Software Engineer with Amazon Personalize focused on building AI-powered recommender systems at scale. Outside of work he is a bibliophile, film enthusiast, and occasional armchair philosopher.

Read More