Today, we’re excited to announce intelligent user segmentation powered by machine learning (ML) in Amazon Personalize, a new way to deliver personalized experiences to your users and run more effective campaigns through your marketing channels.
Traditionally, user segmentation depends on demographic or psychographic information to sort users into predefined audiences. More advanced techniques look to identify common behavioral patterns in the customer journey (such as frequent site visits, recent purchases, or cart abandonment) using business rules to derive users’ intent. These techniques rely on assumptions about the users’ preferences and intentions that limit their scalability, don’t automatically learn from changing user behaviors, and don’t offer user experiences personalized for each user. User segmentation in Amazon Personalize uses ML techniques, developed and perfected at Amazon, to learn what is relevant to users. Amazon Personalize automatically identifies high propensity users without the need to develop and maintain an extensive and brittle catalog of rules. This means you can create more effective user segments that scale with your catalog and learn from your users’ changing behavior to deliver what matters to them.
Amazon Personalize enables developers to build personalized user experiences with the same ML technology used by Amazon with no ML expertise required. We make it easy for developers to build applications capable of delivering a wide array of personalization experiences. You can start creating user segments quickly with the Amazon Personalize API or AWS Management Console and only pay for what you use, with no minimum fees or upfront commitments. All data is encrypted to be private and secure, and is only used to create your user segments.
This post walks you through how to use Amazon Personalize to segment your users based on preferences for grocery products using an Amazon Prime Pantry dataset.
Overview of solution
We’re introducing two new recipes that segment your users based on their interest in different product categories, brands, and more. Our item affinity recipe (aws-item-affinity) identifies users based on their interest in the individual items in your catalog, such as a movie, song, or product. The item attribute affinity recipe (aws-item-attribute) identifies users based on the attributes of items in your catalog, such as genre or brand. This allows you to better engage users with your marketing campaigns and improve retention through targeted messaging.
The notebook that accompanies this post demonstrates how to use the aws-item-affinity and aws-item-attribute recipe to create user segments based on their preferences for grocery products in an Amazon Prime Pantry dataset. We use one dataset group that contains user-item interaction data and item metadata. We use these datasets to train solutions using the two recipes and create user segments in batch.
To test the performance of the solution, we split the interactions data into a training set and test set. The Amazon Prime Pantry dataset has approximately 18 years of interaction data from August 9, 2000, to October 5, 2018, with approximately 1.7 million interactions. We hold out 5% of the most recent interactions and train on the remaining 95%. This results in a split where we use interactions from August 9, 2000, through February 1, 2018, to train the solution and use the remaining 8 months of interactions to simulate future activity as ground truth.
Results
When reproducing these tests in the notebook, your results may vary slightly. This is because when training, the solution the parameters of the underlying models are randomly initialized.
Let’s first review the results by looking at a few examples. We ran queries on three items, and identified 10 users that have a high propensity to engage with the items. We then look at the users’ shopping histories to assess if they would likely be interested in the queried product.
The following table shows the results of a segmentation query on gingerbread coffee, an item we might want to promote for the holiday season. Each row in the table shows the last three purchases of the 10 users returned from the query. Most of the users we identified are clearly coffee drinkers, having recently purchased coffee and coffee creamers. Interestingly, the item we queried on is a whole bean coffee, not a ground coffee. We see in the item histories that, where the information is available, the users have recently purchased whole bean coffee.
| Gingerbread Coffee, 1 lb Whole Bean FlavorSeal Vacuum Bag: Bite into a freshly baked Gingerbread Coffee | |||
|---|---|---|---|
| USER_ID | Last Three Purchases | ||
| A1H3ATRIQ098I7 | Brew La La Red Velvet Cupcake Coffee | Ola’s Exotic Super Premium Coffee Organic Uganda B | Coffee Masters Gourmet Coffee |
| ANEDXRFDZDL18 | Pepperidge Farm Goldfish Crackers | Boston Baked Beans (1) 5.3 Oz Theater Box Sizecont | Boost Simply Complete Nutritional Drink |
| APHFL4MDJRGWB | Dunkin’ Donuts Original Blend Ground Coffee | Coffee-Mate Coffee Mix | Folgers Gourmet Selections Coconut Cream Pie Flavo |
| ANX42D33MNOVP | The Coffee Fool Fool’s House American | Don Francisco’s Hawaiian Hazelnut | Don Francisco’s French Roast Coffee |
| A2NLJJVA0IEK2S | Coffee Masters Flavored Coffee | Lays 15pk Hickory Sticks Original (47g / 1.6oz per | Albanese Confectionery Sugar Free Gummy Bears |
| A1GDEQIGFPRBNO | Christopher Bean Coffee Flavored Ground Coffee | Cameron’s French Vanilla Almond Whole Bean Coffee | Cameron’s Coffee Roasted Whole Bean Coffee |
| A1MDO8RZCZ40B0 | Master Chef Ground Coffee | New England Ground Coffee | Maxwell House Wake Up Roast Medium Coffee |
| A2LK2DENORQI8S | The Bean Coffee Company Organic Holiday Bean (Vani | Lola Savannah Angel Dust Ground | New England Coffee Blueberry Cobbler |
| AGW1F5N8HV3AS | New England Coffee Colombian | Kirkland Signature chicken breast | Lola Savannah Banana Nut Whole Bean |
| A13YHYM6FA6VJO | Lola Savannah Triple Vanilla Whole Bean | Lola Savannah Vanilla Cinnamon Pecan Whole Bean | Pecan Maple Nut |
The next table shows a segmentation query on hickory liquid smoke, a seasoning used for barbecuing and curing bacon. We see a number of different grocery products that might accompany barbecue in the users’ recent purchases: barbecue sauces, seasonings, and hot sauce. Two of the users recently purchased Prague Powder No. 1 Pink Curing Salt, a product also used for curing bacon. We may have missed these two users if we had relied on rules to identify people interested in grilling.
| Wright’s Natural Hickory Seasoning Liquid Smoke, 128 Ounce This seasoning is produced by burning fresh cut hickory chips, then condensing the smoke into a liquid form. | |||
|---|---|---|---|
| USER_ID | Last Three Purchases | ||
| A1MHK19QSCV8SY | Hoosier Hill Farm Prague Powder No.1 Pink Curing S | APPLE CIDER VINEGAR | Fleischmann’s Instant Dry Yeast 1lb bagDry Yeast.M |
| A3G5P0SU1AW2DO | Wright’s Natural Hickory Seasoning Liquid Smoke | Eight O’Clock Whole Bean Coffee | Kitchen Bouquet Browning and Seasoning Sauce |
| A2WW9T8EEI8NU4 | Hidden Valley Dips Mix Creamy Dill .9 oz Packets ( | Frontier Garlic Powder | Wolf Chili Without Beans |
| A2TEJ1S0SK7ZT | Black Tai Salt Co’s – (Food Grade) Himalayan Cryst | Marukan Genuine Brewed Rice Vinegar Unseasoned | Cheddar Cheese Powder |
| A3MPY3AGRMPCZL | Wright’s Natural Hickory Seasoning Liquid Smoke | San Francisco Bay OneCup Fog Chaser (120 Count) Si | Kikkoman Soy Sauce |
| A2U77Z3Z7DC9T9 | Food to Live Yellow Mustard Seeds (Kosher) 5 Pound | 100 Sheets (6.7oz) Dried Kelp Seaweed Nori Raw Uns | SB Oriental Hot Mustard Powder |
| A2IPDJISO5T6AX | Angel Brand Oyster Sauce | Bullhead Barbecue Sauce | ONE ORGANIC Sushi Nori Premium Roasted Organic Sea |
| A3NDGGX7CWV8RT | Frontier Mustard Seed | Da Bomb Ghost Pepper HOT SaucesWe infused our hot | Starwest Botanicals Organic Rosemary Leaf Whole |
| A3F7NO1Q3RQ9Y0 | Yankee Traders Brand Whole Allspice | Aji No Moto Ajinomoto Monosodium Glutamate Umami S | Hoosier Hill Farm Prague Powder No.1 Pink Curing S |
| A3JKI7AWYSTILO | Lalah’s Heated Indian Curry Powder 3 Lb LargeCurry | Ducal Beans Black Beans with Cheese | Emerald Nuts Whole Cashews |
Our third example shows a segmentation query on a decoration used to top cakes. We see that the users identified are not only bakers, but are also clearly interested in decorating their baked goods. We see recent purchases like other cake toppers, edible decorations, and fondant (an icing used to sculpt cakes).
| Letter C – Swarovski Crystal Monogram Wedding Cake Topper Letter, Jazz up your cakes with a sparkling monogram from our Sparkling collection! These single letter monograms are silver plated covered in crystal rhinestones and come in several sizes for your convenience. | |||
|---|---|---|---|
| USER_ID | Last Three Purchases | ||
| A3RLEN577P4E3M | The Republic Of Tea | Alyssa’s Gluten Free Oatmeal Cookies – Pack of 4. | Double Honey Filled Candies |
| AOZ0D3AGVROT5 | Sea Green Disco Glitter Dust | Christmas Green Disco Glitter Dust | Baby Green Disco Glitter Dust |
| AC7O52PQ4HPYR | Rhinestone Cake Topper Number 7 by otherThis delic | Rhinestone Cake Topper Number 5This delicate and h | Rhinestone Cake Topper Number 8 by otherThis delic |
| ALXKY9T83C4Z6 | Heart Language of Love Bride and Groom White Weddi | Bliss Cake Topper by Lenox (836473)It’s a gift tha | First Dance Bride and Groom Wedding Cake TopperRom |
| A2XERDJ6I2K38U | Egyptian Gold Luster Dust | Kellogg’s Rice Krispies Treats | Wilton Decorator Preferred Green Fondant |
| A1474SH2RB49MP | Assorted Snowflake Sugar Decorations Disney Movie | Darice VL3L Mirror Acrylic Initial Letter Cake Top | Edible Snowflakes Sugar Decorations (15 pc). |
| A24E9YGY3V94N8 | TOOGOO(R) Double-Heart Cake Topper Decoration for | Custom Personalized Mr Mrs Wedding Cake Topper Wit | Jacobs Twiglets 6 Pack Jacobs Twiglets are one of |
| A385P0YAW6U5J3 | Tinksky Wedding Cake Topper God Gave Me You Sparkl | Sweet Sixteen Cake Topper 16th Birthday Cake Toppe | Catching the Big One DecoSet Cake DecorationReel i |
| A3QW120I2BY1MU | Golda’s Kitchen Acetate Cake Collars – 4. | Twinings of London English Breakfast Tea K-Cups fo | Chefmaster by US Cake Supply 9-Ounce Airbrush Clea |
| A3DCP979LU7CTE | DecoPac Heading for The Green DecoSet Cake TopperL | Rhinestne Cake Topper Number 90This delicate and h | Rhinestone Cake Topper Letter KThis delicate and h |
These three examples make sense based on our editorial judgement, but to truly assess the performance of the recipe, we need to analyze more of the results. To do this broader assessment, we run the aws-item-affinity solution on 500 randomly selected items that appear in the test set to query a list of 2,262 users (1% of the users in the dataset). We then use the test set to assess how frequently the 2,262 users purchased the items during the test period. For comparison, we also assess how frequently 2,262 of the most active users purchased the items during the test period. The following table shows that the aws-item-affinity solution is four times better at identifying users that would purchase a given item.
| Test Metrics | ||
| Hits | Recall | |
| Personalize – Item Affinity | 0.2880 | 0.1297 |
| Active User Baseline | 0.0720 | 0.0320 |
Although these results are informative, they’re not a perfect reflection of the performance of the recipe because the user segmentation wasn’t used to promote the items which users later interacted with. The best way to measure performance is an online A/B test—running a marketing campaign on a list of users derived from the aws-item-affinity solution alongside a set of the most active users to measure the difference in engagement.
Conclusion
Amazon Personalize now makes it easy to run more intelligent user segmentation at scale, without having to maintain complex sets of rules or relying on broad assumptions about the preferences of your users. This allows you to better engage users with your marketing campaigns and improve retention through targeted messaging.
To learn more about Amazon Personalize, visit the product page.
About the Authors
Daniel Foley is a Senior Product Manager for Amazon Personalize. He is focused on building applications that leverage artificial intelligence to solve our customers’ largest challenges. Outside of work, Dan is an avid skier and hiker.
Debarshi Raha is a Senior Software Engineer for Amazon Personalize. He is passionate about building AI-based personalization systems at scale. In his spare time, he enjoys traveling and photography.
Ge Liu is an Applied Scientist at AWS AI Labs working on developing next generation recommender system for Amazon Personalize. Her research interests include Recommender System, Deep Learning, and Reinforcement Learning.
Haizhou Fu is a senior software engineer on the Amazon Personalize team working on designing and building recommendation systems and solutions for different industries. Outside of his work, he loves playing soccer, basketball and watching movies, reading and learning about physics, especially theories related to time and space.