Index your Microsoft Exchange content using the Exchange connector for Amazon Kendra

Amazon Kendra is a highly accurate and simple-to-use intelligent search service powered by machine learning (ML). Amazon Kendra offers a suite of data source connectors to simplify the process of ingesting and indexing your content, wherever it resides.

Valuable data in organizations is stored in both structured and unstructured repositories. An enterprise search solution should be able to pull together data across several structured and unstructured repositories to index and search on.

One such unstructured data repository is Microsoft Exchange. Email conversations contain important messages exchanged between various parties over time. Users often attach documents containing valuable information in the context of that email. In addition to emails, an Exchange account gives access to other valuable sources of information like calendar entries, OneNote notebooks, and contacts.

We’re excited to announce that you can now use the Amazon Kendra connector for Microsoft Exchange to search information stored in your Exchange account. In this post, we show how to index information stored in Exchange and use the Amazon Kendra intelligent search function. In addition, the ML-powered intelligent search can accurately find information from unstructured documents having natural language narrative content, for which keyword search is not very effective.

Solution overview

With Amazon Kendra, you can configure multiple data sources to provide a central place to search across your document repository. For our solution, we demonstrate how to index a Exchange repository or folder using the Amazon Kendra connector for Exchange. The solution consists of the following steps:

1. Configure an app on Exchange and get the connection details.
2. Store the details in AWS Secrets Manager.
3. Create an Exchange data source via the Amazon Kendra console.
4. Index the data in the Exchange repository.
5. Run a sample query to test the solution.

Prerequisites

To try out the Amazon Kendra connector for Exchange, you need the following:

• An Exchange account.
• An AWS account with privileges to create AWS Identity and Access Management (IAM) roles and policies. For more information, see Overview of access management: Permissions and policies.
• Basic knowledge of AWS.

Configure an Exchange app and gather connection details

Before we set up the Exchange data source, we need a few details about your Exchange repository. Let’s gather those in advance.

1. Log in to the Azure portal using your global admin user account and choose Next.
   
2. Enter your password and choose Sign in.
   
3. On the Azure welcome page, choose App registrations.
   
4. Choose New registration.
   
5. Enter a name for the app (for example, my-exchange-app) and choose Register.
   
6. Note down the tenant ID (you need it when setting up the data source for Amazon Kendra).
7. Under Client credentials, choose Add a certificate or secret.
   
8. Choose New client secret.
   
9. Enter a description (for example, my exchange secret).
10. Choose an expiration period (for this post, 6 months).
11. Choose Add.
    
12. Note the secret ID and value to use later when setting up the data source.
    
13. In the navigation pane, choose API permissions.

This is where you can add or remove admin permissions.

14. For this post, leave the defaults as is.
    

Store Exchange credentials in Secrets Manager

To store your Exchange credentials in Secrets Manager, compete the following steps:

1. On the Secrets Manager console, choose Store a new secret.
2. Select Other type of secret.
3. Create two key-value pairs for clientid and clientsecret and enter the values saved from Exchange.
4. Choose Next.
   
5. For Secret name, enter a name (for example, AmazonKendra-my-exchange-secret).
6. Enter an optional description.
7. Choose Next.
   
8. In the Configure rotation section, keep all settings at their defaults and choose Next.
   
9. On the Review page, choose Store.
   

Configure the Amazon Kendra connector for Exchange

To configure the Amazon Kendra connector, complete the following steps:

1. On the Amazon Kendra console, choose Create an Index.
   
2. For Index name, enter a name for the index (for example, my-exchange-index).
3. Enter an optional description.
4. For Role name, enter an IAM role name.
5. Configure optional encryption settings and tags.
6. Choose Next.
   
7. For Specify provisioning, select Developer edition and choose Next.
   
8. In the Configure user access control section, leave the settings at their defaults and choose Next.
9. On the review page, choose Create.

This creates and propagates the IAM role and then creates the Amazon Kendra index, which can take up to 30 minutes.

Create an Exchange data source

Complete the following steps to create your data source:

1. On the Amazon Kendra console, choose Data sources in the navigation pane.
2. Under Microsoft Exchange, choose Add connector.
   
3. For Data source name, enter a name (for example, my-exchange-data-source).
4. Enter an optional description.
5. Choose Next.
   
6. For Tenant ID, choose the tenant ID you collected earlier.
7. For AWS Secrets Manager secret, choose the secret you created earlier.
8. For IAM role, choose Create a new role.
9. For Role name, enter a name (for example, AmazonKendra-myexchange-datasource-role).
10. Choose Next.
    
11. For User email ID, you can enter a list of email IDs. To capture content from all users, leave the field blank.

We have kept the default selections, but you can fine-tune your selection of content as needed.

12. For Sync mode, select Full sync (this is the first time and we need to import all content).
13. For Frequency, choose Run on demand.
14. Choose Next.
    
15. Set any optional field mappings and choose Next.
16. Choose Review and Create and choose Add data source.
17. Choose Sync now.
    
18. Wait for the sync to complete.
    

Test the solution

Now that you have ingested the content from your Exchange account into your Amazon Kendra index, you can test some queries.

1. Go to your index and choose Search indexed content.
2. Enter a sample search query and test out your search results (your query will vary based on the contents of your account).
   

The Exchange connector also crawls local identity information from Exchange. You can use this feature to narrow down your query by user.

3. To use this feature, go back to the search results page.
4. Expand Test query with user name or groups and choose Apply user name or groups.
   

For Microsoft Exchange, we don’t import groups, we just import user names. User names are email IDs in this case.

5. Enter the user ID (email) of your user and choose Apply.
   
6. Rerun your search query.

This brings you a filtered set of results based on your criteria.

7. Go back to the search page and enter the name of a user who doesn’t have access to this content, then choose Apply.
   
8. Run the same query again.
   

When fronting Amazon Kendra with an application such as an application built using Experience Builder, you can pass the user identity (in the form of the email ID) to Amazon Kendra to ensure that each user only sees content specific to their user ID. Alternately, you can use AWS IAM Identity Center (successor to AWS Single Sign-On) to control user context being passed to Amazon Kendra to limit queries by user.

Congratulations! You have successfully used Amazon Kendra to surface answers and insights based on the content indexed from your Exchange account.

Limitations

This solution has the following limitations:

• Multiple domain emails are not supported.
• Sticky notes are not supported.
• Incremental updates are valid only for a specific period (7 days) before the client application needs to run a full synchronization again.
• Exchange Online has rate limits that govern the speed of ingestion. For more information, refer to Exchange Online limits.

Clean up

To avoid incurring future costs, clean up the resources you created as part of this solution. If you created a new Amazon Kendra index while testing this solution, delete it. If you only added a new data source using the Amazon Kendra connector for Exchange, delete that data source.

Conclusion

With the Microsoft Exchange connector for Amazon Kendra, organizations can tap into the repository of information stored in their account securely using intelligent search powered by Amazon Kendra.

To learn about these possibilities and more, refer to the Amazon Kendra Developer Guide. For more information on how you can create, modify, or delete metadata and content when ingesting your data from Exchange, refer to Enriching your documents during ingestion and Enrich your content and metadata to enhance your search experience with custom document enrichment in Amazon Kendra.

About the author

Ashish Lagwankar is a Senior Enterprise Solutions Architect at AWS. His core interests include AI/ML, serverless, and container technologies. Ashish is based in the Boston, MA, area and enjoys reading, outdoors, and spending time with his family.

Read More