Offline RL Made Easier: No TD Learning, Advantage Reweighting, or Transformers

Offline RL Made Easier: No TD Learning, Advantage Reweighting, or Transformers


A demonstration of the RvS policy we learn with just supervised learning and a depth-two MLP. It uses no TD learning, advantage reweighting, or Transformers!

Offline reinforcement learning (RL) is conventionally approached using value-based methods based on temporal difference (TD) learning. However, many recent algorithms reframe RL as a supervised learning problem. These algorithms learn conditional policies by conditioning on goal states (Lynch et al., 2019; Ghosh et al., 2021), reward-to-go (Kumar et al., 2019; Chen et al., 2021), or language descriptions of the task (Lynch and Sermanet, 2021).

We find the simplicity of these methods quite appealing. If supervised learning is enough to solve RL problems, then offline RL could become widely accessible and (relatively) easy to implement. Whereas TD learning must delicately balance an actor policy with an ensemble of critics, these supervised learning methods train just one (conditional) policy, and nothing else!

Offline RL Made Easier: No TD Learning, Advantage Reweighting, or Transformers



A demonstration of the RvS policy we learn with just supervised learning and a depth-two MLP. It uses no TD learning, advantage reweighting, or Transformers!

Offline reinforcement learning (RL) is conventionally approached using value-based methods based on temporal difference (TD) learning. However, many recent algorithms reframe RL as a supervised learning problem. These algorithms learn conditional policies by conditioning on goal states (Lynch et al., 2019; Ghosh et al., 2021), reward-to-go (Kumar et al., 2019; Chen et al., 2021), or language descriptions of the task (Lynch and Sermanet, 2021).

We find the simplicity of these methods quite appealing. If supervised learning is enough to solve RL problems, then offline RL could become widely accessible and (relatively) easy to implement. Whereas TD learning must delicately balance an actor policy with an ensemble of critics, these supervised learning methods train just one (conditional) policy, and nothing else!

Search for knowledge in Quip documents with intelligent search using the Quip connector for Amazon Kendra

Organizations use collaborative document authoring solutions like Salesforce Quip to embed real-time, collaborative documents inside Salesforce records. Quip is Salesforce’s productivity platform that transforms the way enterprises work together, delivering modern collaboration securely and simply across any device. A Quip repository captures invaluable organizational knowledge in the form of collaborative documents and workflows. However, finding this organizational knowledge easily and securely along with other document repositories, such as Box or Amazon Simple Storage Service (Amazon S3), can be challenging. Additionally, the conversational nature of collaborative workflows renders the traditional keyword-based approach to search ineffective due to having fragmented, dispersed information in multiple places.

We’re excited to announce that you can now use the Amazon Kendra connector for Quip to search messages and documents in your Quip repository. In this post, we show you how to find the information you need in your Quip repository using the intelligent search function of Amazon Kendra, powered by machine learning.

Solution overview

With Amazon Kendra, you can configure multiple data sources to provide a central place to search across your document repository. For our solution, we demonstrate how to configure a Quip repository as a data source of a search index using the Amazon Kendra connector for Quip.

The following screenshot shows an example Quip repository.

The workspace in this example has a private folder that is not shared. That folder has a subfolder that is used to keep expense receipts. Another folder called example.com is shared with others and used to collaborate with the team. This folder has five subfolders that hold documentation for development.

To configure the Quip connector, we first note the domain name, folder IDs, and access token of the Quip repository. Then we simply create the Amazon Kendra index and add Quip as a data source.

Prerequisites

To get started using the Quip connector for Amazon Kendra, you must have a Quip repository.

Gather information from Quip

Before we set up the Quip data source, we need a few details about your repository. Let’s gather those in advance.

Domain name

Find out the domain name. For example , for the Quip URL https://example-com.quip.com/browse, the domain name is quip. Depending on how single sign-on (SSO) is set up in your organization, the domain name may vary. Save this domain name to use later.

Folder IDs

Folders in Quip have a unique ID associated with them. We need to configure the Quip connector to access the right folders by supplying the correct folder IDs. For this post, we index the folder example.com.

To find the ID of the folder, choose the folder. The URL changes to show the folder ID.

The folder ID in this case is xj1vOyaCGB3u. Make a list of the folder IDs to scan; we use these IDs when configuring the connector.

Access token

Log in to Quip and open https://{subdomain.domain}/dev/token in a web browser. In the following example, we navigate to https://example-com.quip.com/dev/token. Then choose Get Personal Access Token.

Copy the token to use in a later step.

We now have the information we need to configure the data source.

Create an Amazon Kendra index

To set up your Amazon Kendra index, complete the following steps:

  1. Sign in to the AWS Management Console and open the Amazon Kendra console.

If you’re using Amazon Kendra for the first time, you should see the following screenshot.

  1. Choose Create an index.
  2. For Index name, enter my-quip-example-index.
  3. For Description, enter an optional description.
  4. For IAM role, use an existing role or create a new one.
  5. Choose Next.
  6. Under Access control settings, select No to make all indexed content available to all users.
  7. For User-group expansion, select None.
  8. Choose Next.

For Provisioning editions, you can choose from two options depending on the volume of the content and frequency of access.

  1. For this post, select Developer edition.
  2. Choose Create.

Role creation takes approximately 30 seconds; index creation can take up to 30 minutes. When complete, you can view your index on the Amazon Kendra console.

Add Quip as a data source

Now let’s add Quip as a data source to the index.

  1. On the Amazon Kendra console, under Data management in the navigation pane, choose Data sources.
  2. Choose Add connector under Quip.
  3. For Data source name, enter my-quip-data-source.
  4. For Description, enter an optional description.
  5. Choose Next.
  6. Enter the Quip domain name that you saved earlier.
  7. Under Secrets, choose Create and add a new Secrets Manager secret.
  8. For Secret name, enter the name of your secret.
  9. For Quip token, enter the access token you saved earlier.
  10. Choose Save and add secret.
  11. Under IAM role, choose a role or create a new one.
  12. Choose Next.
  13. Under Sync scope, for Add folder IDs to crawl, enter the folder IDs you saved earlier.
  14. Under Sync run schedule¸ for Frequency, select Run on demand.
  15. Choose Next.

The Quip connector lets you capture additional fields like authors, categories, and folder names (and even rename as needed).

  1. For this post, we don’t configure any field mappings.
  2. Choose Next.
  3. Confirm all the options and add the data source.

Your data source is ready in a few minutes.

  1. When your data source is ready, choose Sync now.

Depending on the size of the data in the Quip repository, this process can take a few minutes to a few hours. Syncing is a two-step process. First, the documents are crawled to determine the ones to index. Then the selected documents are indexed. Some factors that affect sync speed include repository throughput and throttling, network bandwidth, and the size of documents.

The sync status shows as successful when the sync is complete. Your Quip repository is now connected.

Run a search in Amazon Kendra

Let’s test the connector by running a few searches.

  1. On the Amazon Kendra console, under Data management in the navigation pane, choose Search indexed content.
  2. Enter your search in the search field. For this post, we search for EC2 on Linux.

The following screenshot shows our results.

Limitations

There are some known limitations for the data source ingestion. Some limitations are due to the need for admin access for accessing some of the content, others due to specific implementation details. They are as follows:

  • Only full crawls are supported. If you want the connector to support changelog crawls, admin API access is required, and you have to enable the admin API on the Quip website.
  • Only shared folders are crawled. Even if we use the personal access token of an admin user, we can’t crawl data in the private folders of other users.
  • The solution doesn’t support specifying file types for inclusion and exclusion, because Quip doesn’t store the file type extension, just the file name.
  • Real-time events require a subscription and admin API access.

Conclusion

The Amazon Kendra connector for Quip enables organizations to make the invaluable information stored in Quip documents available to their users securely using intelligent search powered by Amazon Kendra. The connector also provides facets for Quip repository attributes such as authors, file type, source URI, creation dates, parent files, and category so users can interactively refine the search results based on what they’re looking for.

For more information on how you can create, modify, and delete data and metadata using custom document enrichment as content is ingested from the Quip repository, refer to Customizing document metadata during the ingestion process and Enrich your content and metadata to enhance your search experience with custom document enrichment in Amazon Kendra.

About the Authors

Ashish Lagwankar is a Senior Enterprise Solutions Architect at AWS. His core interests include AI/ML, serverless, and container technologies. Ashish is based in the Boston, MA, area and enjoys reading, outdoors, and spending time with his family.

Vikas Shah is an Enterprise Solutions Architect at Amazon web services. He is a technology enthusiast who enjoys helping customers find innovative solutions to complex business challenges. His areas of interest are ML, IoT, robotics and storage. In his spare time, Vikas enjoys building robots, hiking, and traveling.

Read More

Learning to Prompt for Continual Learning

Posted by Zifeng Wang, Student Researcher, and Zizhao Zhang, Software Engineer, Google Research

Supervised learning is a common approach to machine learning (ML) in which the model is trained using data that is labeled appropriately for the task at hand. Ordinary supervised learning trains on independent and identically distributed (IID) data, where all training examples are sampled from a fixed set of classes, and the model has access to these examples throughout the entire training phase. In contrast, continual learning tackles the problem of training a single model on changing data distributions where different classification tasks are presented sequentially. This is particularly important, for example, to enable autonomous agents to process and interpret continuous streams of information in real-world scenarios.

To illustrate the difference between supervised and continual learning, consider two tasks: (1) classify cats vs. dogs and (2) classify pandas vs. koalas. In supervised learning, which uses IID, the model is given training data from both tasks and treats it as a single 4-class classification problem. However, in continual learning, these two tasks arrive sequentially, and the model only has access to the training data of the current task. As a result, such models tend to suffer from performance degradation on the previous tasks, a phenomenon called catastrophic forgetting.

Mainstream solutions try to address catastrophic forgetting by buffering past data in a “rehearsal buffer” and mixing it with current data to train the model. However, the performance of these solutions depends heavily on the size of the buffer and, in some cases, may not be possible at all due to data privacy concerns. Another branch of work designs task-specific components to avoid interference between tasks. But these methods often assume that the task at test time is known, which is not always true, and they require a large number of parameters. The limitations of these approaches raise critical questions for continual learning: (1) Is it possible to have a more effective and compact memory system that goes beyond buffering past data? (2) Can one automatically select relevant knowledge components for an arbitrary sample without knowing its task identity?

In “Learning to Prompt for Continual Learning”, presented at CVPR2022, we attempt to answer these questions. Drawing inspiration from prompting techniques in natural language processing, we propose a novel continual learning framework called Learning to Prompt (L2P). Instead of continually re-learning all the model weights for each sequential task, we instead provide learnable task-relevant “instructions” (i.e., prompts) to guide pre-trained backbone models through sequential training via a pool of learnable prompt parameters. L2P is applicable to various challenging continual learning settings and outperforms previous state-of-the-art methods consistently on all benchmarks. It achieves competitive results against rehearsal-based methods while also being more memory efficient. Most importantly, L2P is the first to introduce the idea of prompting in the field of continual learning.

Compared with typical methods that adapt entire or partial model weights to tasks sequentially using a rehearsal buffer, L2P uses a single frozen backbone model and learns a prompt pool to conditionally instruct the model. “Model 0” indicates that the backbone model is fixed at the beginning.

<!–

Compared with typical methods that adapt entire or partial model weights to tasks sequentially using a rehearsal buffer, L2P uses a single frozen backbone model and learns a prompt pool to conditionally instruct the model. “Model 0” indicates that the backbone model is fixed at the beginning.

–>

Prompt Pool and Instance-Wise Query
Given a pre-trained Transformer model, “prompt-based learning” modifies the original input using a fixed template. Imagine a sentiment analysis task is given the input “I like this cat”. A prompt-based method will transform the input to “I like this cat. It looks X”, where the “X” is an empty slot to be predicted (e.g., “nice”, “cute”, etc.) and “It looks X” is the so-called prompt. By adding prompts to the input, one can condition the pre-trained models to solve many downstream tasks. While designing fixed prompts requires prior knowledge along with trial and error, prompt tuning prepends a set of learnable prompts to the input embedding to instruct the pre-trained backbone to learn a single downstream task, under the transfer learning setting.

In the continual learning scenario, L2P maintains a learnable prompt pool, where prompts can be flexibly grouped as subsets to work jointly. Specifically, each prompt is associated with a key that is learned by reducing the cosine similarity loss between matched input query features. These keys are then utilized by a query function to dynamically look up a subset of task-relevant prompts based on the input features. At test time, inputs are mapped by the query function to the top-N closest keys in the prompt pool, and the associated prompt embeddings are then fed to the rest of the model to generate the output prediction. At training, we optimize the prompt pool and the classification head via the cross-entropy loss.

Illustration of L2P at test time. First, L2P selects a subset of prompts from a key-value paired prompt pool based on our proposed instance-wise query mechanism. Then, L2P prepends the selected prompts to the input tokens. Finally, L2P feeds the extended tokens to the model for prediction.

Intuitively, similar input examples tend to choose similar sets of prompts and vice versa. Thus, prompts that are frequently shared encode more generic knowledge while other prompts encode more task-specific knowledge. Moreover, prompts store high-level instructions and keep lower-level pre-trained representations frozen, thus catastrophic forgetting is mitigated even without the necessity of a rehearsal buffer. The instance-wise query mechanism removes the necessity of knowing the task identity or boundaries, enabling this approach to address the under-investigated challenge of task-agnostic continual learning.

Effectiveness of L2P
We evaluate the effectiveness of L2P in different baseline methods using an ImageNet pre-trained Vision Transformer (ViT) on representative benchmarks. The naïve baseline, called Sequential in the graphs below, refers to training a single model sequentially on all tasks. The EWC model adds a regularization term to mitigate forgetting and the Rehearsal model saves past examples to a buffer for mixed training with current data. To measure the overall continual learning performance, we measure both the accuracy and the average difference between the best accuracy achieved during training and the final accuracy for all tasks (except the last task), which we call forgetting. We find that L2P outperforms the Sequential and EWC methods significantly in both metrics. Notably, L2P even surpasses the Rehearsal approach, which uses an additional buffer to save past data. Because the L2P approach is orthogonal to Rehearsal, its performance could be further improved if it, too, used a rehearsal buffer.

L2P outperforms baseline methods in both accuracy (top) and forgetting (bottom). Accuracy refers to the average accuracy for all tasks and forgetting is defined as the average difference between the best accuracy achieved during training and the final accuracy for all tasks (except the last task).

We also visualize the prompt selection result from our instance-wise query strategy on two different benchmarks, where one has similar tasks and the other has varied tasks. The results indicate that L2P promotes more knowledge sharing between similar tasks by having more shared prompts, and less knowledge sharing between varied tasks by having more task-specific prompts.

Prompt selection histograms for benchmarks of similar tasks (left) and varied tasks (right). The left benchmark has higher intra-task similarity, thus sharing prompts between tasks results in good performance, while the right benchmark favors more task-specific prompts.

Conclusion
In this work, we present L2P to address key challenges in continual learning from a new perspective. L2P does not require a rehearsal buffer or known task identity at test time to achieve high performance. Further, it can handle various complex continual learning scenarios, including the challenging task-agnostic setting. Because large-scale pre-trained models are widely used in the machine learning community for their robust performance on real-world problems, we believe that L2P opens a new learning paradigm towards practical continual learning applications.

Acknowledgements
We gratefully acknowledge the contributions of other co-authors, including Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas Pfister. We would also like to thank Chun-Liang Li, Jeremy Martin Kubica, Sayna Ebrahimi, Stratis Ioannidis, Nan Hua, and Emmanouil Koukoumidis, for their valuable discussions and feedback, and Tom Small for figure creation.

Read More

Integrate ServiceNow with Amazon Lex chatbot for ticket processing

Conversational interfaces (or chatbots) can provide an intuitive interface for processes such as creating and monitoring tickets. Let’s consider a situation in which a recent hire on your team is required to cut tickets for office equipment. To do so, they have to interact with a ticketing software that the organization uses. This often requires accessing the ticketing system, knowing which ticket to open, and then tracking the ticket manually through the process until completion. In this post, we show you how to integrate an Amazon Lex chatbot with ServiceNow . The bot will make it easier for creation and tracking of tickets for day-to-day activities such as issuing new office equipment for new hires. You can also integrate the experience into a customer support call to seamlessly create tickets for the callers.

Solution overview

The following diagram illustrates the solution workflow.

The solution includes the following steps:

  1. A user sends a message to create a ticket or get pending tickets in the queue through a Slack app.
  2. Slack forwards the message to be processed by Amazon Lex.
  3. Amazon Lex invokes fulfillment Lambda function:
    1. Amazon Lex sends the event to the fulfillment AWS Lambda function.
    2. AWS Lambda function processes the message and makes HTTP requests to the backend ServiceNow instance.
  4. Response is sent to the user:
    1. The ServiceNow instance returns a response to the fulfillment Lambda function.
    2. Fulfillment Lambda function returns the response to Amazon Lex bot based on Sentiment.
    3. Amazon Lex returns the response to the user through Slack bot.
    4. The user is able to see the response on the Slack bot and reply with another query.

To implement this architecture, you create the following:

  • A ServiceNow instance
  • The fulfillment Lambda function
  • An Amazon Lex bot
  • A Slack app

Prerequisites

Before getting started, make sure you have the following prerequisites:

Create the ServiceNow developer instance

To create your ServiceNow instance, complete the following steps:

  1. Sign up for a ServiceNow developer instance.

You receive an email with a personal sandbox environment in the format devNNNNN.service-now.com.

This step sends a verification email to the email that you used during the signup process.

  1. After you’re verified, you can sign in to your account.
  2. Enter your email and choose Next.

You’re asked if you need a developer oriented IDE or a guided experience.

  1. For this post, choose I need a guided experience.
  2. Select the check box to agree to the terms of service and choose Finish Setup.

You’re redirected to a page where you should be able to see that the instance is being set up.

When the instance is ready, you should be able to see the instance details.

  1. Note the instance URL, user name, and password, which you use in the following steps.

You need to log in as the system administrator user so you can view the ServiceNow incidents.

  1. Navigate to the following URL (replace https://devNNNNN.service-now.com with your own instance URL that you noted earlier): https://devNNNNN.service-now.com/nav_to.do?uri=change_request_list.do.
  2. Log in using the user name admin and the password you noted earlier.

You’re redirected to the ServiceNow console.

  1. Choose Incidents in the navigation pane.

The default search criteria should show you a sample incident.

  1. If you remove all the search criteria and choose Run, you should be able to see all the ServiceNow incidents available.

The following screenshot shows the search with no filters and the sample ServiceNow incidents.

Create the Lambda function

Now that you have set up a ServiceNow instance and logged in to check out the incidents, you’re ready to set up the solution. The first step is to create the Lambda function and configure environment variables for this function for storing the ServiceNow instance URL and credentials in a secure manner, and for the function to use the ServiceNow instance account.

Create the fulfillment Lambda function

In this step, you create a Lambda function that helps the Amazon Lex bot communicate with ServiceNow to create or describe the incidents, and have some logic to frame a response to Amazon Lex based on the sentiment analysis that Amazon Lex forwards to Lambda. To create your function, complete the following steps:

  1. On the Lambda console, choose Functions in the navigation pane.
  2. Choose Create function.
  3. Select Author from scratch.
  4. For Function name, enter a name (for this post, ChatBotLambda).
  5. For Runtime, choose Node.js 14x.

We use the latest Node.js runtime (as of this writing), but you can use your preferred runtime.

  1. For the function permissions, select Create a new role with basic Lambda permissions.
  2. Use the policy AWSLambdaBasicExecutionRole.

This execution role should be sufficient for this post. For more information, see AWS Lambda execution role.

  1. Choose Create function.
  2. After you create the function, you can use the inline editor to edit the code for index.js.

The following is sample code for the function that you’re using as the compute layer for our logic:

var https = require('https');
exports.handler = async (event, context) => {
    console.log('Received event:',JSON.stringify(event, null,2));
    var intent = event.sessionState.intent.name;
    var slots = event.sessionState.intent.slots;
    var ticketType = slots.ticketType.value.interpretedValue.toLowerCase();
    
    return new Promise((resolve, reject) => {
      if (intent == 'GetTicket') {
        for (var i=0; i<event.interpretations.length; i++){
          if (event.interpretations[i].intent.name == intent){
            var sentimentResponse = event.interpretations[i].sentimentResponse;
            if (sentimentResponse.sentiment != null) {
            var sentimentLabel = sentimentResponse.sentiment;
            }
          }
        }
        
        var ticketCount = slots.ticketCount.value.interpretedValue;
        getTickets(ticketType, ticketCount, sentimentLabel, resolve, reject);       // Get the records
      }
      else if (intent == 'LogTicket') {
        var shortDesc = slots.shortDesc.value.interpretedValue;
        logTicket(ticketType, shortDesc, resolve, reject);
      }
    });
};
// Get tickets from ServiceNow
//
function getTickets(recType, count, sentimentLabel, resolve, reject) {
  var snowInstance = process.env.SERVICENOW_HOST;
  console.log("sentimentLabel:-", sentimentLabel);
  var options = {
      hostname: snowInstance,
      port: 443,
      path: '/api/now/table/' + recType + '?sysparm_query=ORDERBYDESCsys_updated_on&sysparm_limit='+count,
      method: 'get',
      headers: {
        'Content-Type': 'application/json',
        Accept: 'application/json',
        Authorization: 'Basic ' + Buffer.from(process.env.SERVICENOW_USERNAME + ":" + process.env.SERVICENOW_PASSWORD).toString('base64'),
      }
  };
  var request = https.request(options, function(response) {
      var returnData = '';
      response.on('data', chunk => returnData += chunk);
      response.on('end', function() {
        var responseObj = JSON.parse(returnData);
        var speechText = "";
        if(responseObj.result){
          if (sentimentLabel == "NEGATIVE") {
            speechText =  "I am sorry you are having a bad day. Here are the " + count + " most recent incidents: ";
          }
          else {
            speechText =  "Here are the " + count + " most recent incidents: "; 
          }
          for (let i = 0; i < count; i++) {
            var rec_number = i + 1;
            speechText += "Record " + rec_number + " " + responseObj.result[i].short_description + ". ";
          }
          speechText += "End of tickets.";
          var retMsg = {
            "sessionState": {
              "dialogAction": {
                "type": "Close"
              },
              "intent": {
                "confirmationState": "Confirmed",
                "name": "GetTicket",
                "state": "Fulfilled",
              },
            },
            "messages": [
              {
                "contentType": "PlainText",
                "content": speechText,
                
              }
            ]
          };
          resolve(retMsg);
        }
        else{
          reject(JSON.parse('{"Error": "No tickets Found"}'));
        }
      });
      response.on('error', e => context.fail('error:' + e.message));
    });
    request.end();
}
function logTicket(recType, shortDesc, resolve, request) {
  var requestData = {
        "short_description": shortDesc,
        "created_by": 'me',
        "caller_id": 'me'
  };
  var postData = JSON.stringify(requestData);
  var options = {
        host: process.env.SERVICENOW_HOST,
        port: '443',
        path: '/api/now/table/' + recType,
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'Accept': 'application/json',
            'Authorization': 'Basic ' + Buffer.from(process.env.SERVICENOW_USERNAME + ":" + process.env.SERVICENOW_PASSWORD).toString('base64'),
            'Content-Length': Buffer.byteLength(postData)
        }
    };
    var request = https.request(options, function (res) {
        console.log("res:", res);
        var body = '';
        res.on('data', chunk => body += chunk);
        res.on('end', function() {
        var responseObj = JSON.parse(body);
        console.log("responseObj:", responseObj);
        var ticketNumber = responseObj.result.number;
        var ticketType = responseObj.result.sys_class_name;
          var retMsg = {
            "sessionState": {
              "dialogAction": {
                "type": "Close"
              },
              "intent": {
                "confirmationState": "Confirmed",
                "name": "LogTicket",
                "state": "Fulfilled",
              },
            },
            "messages": [
              {
                "contentType": "PlainText",
                "content": "Done! I've opened an " + ticketType + " ticket for you in ServiceNow. Your ticket number is: " + ticketNumber + "."
              }
            ]
          };  
          resolve(retMsg);
        });
        res.on('error', e => context.fail('error:' + e.message));
    });
    request.write(postData);
    request.end();
}

Before moving on to the next step, don’t forget to choose Deploy to deploy this code to the $LATEST version of the Lambda function.

Configure the fulfillment Lambda function

Next, you create the following environment variables with appropriate values. You use these variables to securely store the ServiceNow instance URL and credentials that the function uses to connect to the ServiceNow instance. Every time the user sends a message through the Amazon Lex bot to create or get incident tickets, this Lambda function is invoked to make a request to the ServiceNow instance to create or get the incidents. Therefore, it needs the instance URL and credentials in order to connect to the instance.

  • SERVICENOW_HOST – The domain name for the ServiceNow instance that you created earlier
  • SERVICENOW_USERNAME – The user name for the system administrator role (admin)
  • SERVICENOW_PASSWORD – The password that you received earlier

These variables are available on the Configuration tab, as shown in the following screenshot.

Create the Amazon Lex chatbot

Now that you have created the Lambda function, you create the conversational interface (the chatbot) using Amazon Lex. For this post, you build the chatbot IncidentBot to communicate with ServiceNow and read or create incident tickets to process the events. This type of bot can be created for organizations or businesses that have multiple interfaces to internal systems, ranging from HR to travel to support, which employees must memorize for bookmarks. The chatbot also performs sentiment analysis on the users’ messages sent via the bot and returns a response based on the sentiment detected.

You create two intents:

  • GetTicket – Gets the existing tickets from ServiceNow
  • LogTicket – Submits a new ticket, which creates a ServiceNow incident in our instance

This post uses the following conversation to model a bot:

  • User: Create an incident ticket to order a new laptop.
  • IncidentBot: Done! I’ve opened an incident ticket for you in ServiceNow. Your ticket number is: INC0010006.
  • User: List top 2 incident.
  • IncidentBot: Here are the 2 most recent incidents: Record 1 order a new laptop. Record 2 request access to ServiceNow. End of tickets.

The Lambda function that you set up earlier only works with Lex V2. If you’re using the V1 console, choose Try the new Lex V2 Console as shown in the following screenshot, or choose Switch to the new Lex V2 Console in the navigation pane.

Complete the following steps to create your chatbot:

  1. Download the file IncidentBot.zip.
  2. On the Amazon Lex console, choose Bots in the navigation pane.
  3. On the Action menu, choose Import.
  4. For Bot name, enter IncidentBot.
  5. For Input file¸ choose Browse file and choose the .zip file you downloaded.
  6. Select Create a role with basic Amazon Lex permissions.

This creates a new IAM role that the chatbot uses to make requests to other AWS services.

  1. In the section Children’s Online Privacy Protection Act (COPPA), select No (COPPA doesn’t apply to this example).
  2. Keep the remaining fields at their default and choose Create bot.
  3. When the bot is available, choose Aliases in the navigation pane to see the alias created for this bot.
  4. Choose the alias TestBotAlias to see the alias details.

As shown in the following screenshot, this chatbot just uses the language English (US).

To have an effective conversation, it’s important to understand the sentiment and respond appropriately. In a conversation, a simple acknowledgment when talking to an unhappy user might be helpful, such as, “I am sorry you are having a bad day.”

To achieve such a conversational flow with a bot, you have to detect the sentiment expressed by the user and react appropriately. Previously, you had to build a custom integration by using Amazon Comprehend APIs. As of this writing, you can determine the sentiment natively in Amazon Lex.

You can enable sentiment analysis on the Lex V2 bot by editing the alias.

  1. On the alias details page, choose Edit.
  2. Select Enable sentiment analysis and choose Confirm.

For this post, you analyze the messages that you receive from end-users in order to understand their mood and return the appropriate response, which is governed by the Lambda logic that uses the detected sentiment to change the response text accordingly.

  1. To add the function to the alias, on the alias details page, choose English (US).
  2. For Source, choose ChatBotLamba.
  3. For Lambda function version or alias, chose $LATEST.
  4. Choose Save.

You’re now ready to build the intent.

  1. In the navigation pane, choose Bot versions.
  2. Choose the draft version of your bot to see its details.
  3. Choose Intents in the navigation pane to explore the intents you created.
  4. To build the bot, choose Build.

Test the Amazon Lex bot

We test the following scenarios:

  • The user sends a message to create a new ServiceNow incident using the example utterance “create an incident ticket with request access to ServiceNow.”
  • The user retrieves the existing ServiceNow incidents using the utterance “list top 2 incident tickets.”
  • The user can also show negative sentiment in the message and retrieve the response accordingly using the utterance “what are the top 2 bad incident tickets.”

To test the bot, on the Intents page, choose Test.

As shown in the following screenshot, you created two incident tickets using the following utterances:

  • create an incident ticket with request access to service now
  • create an incident ticket with order a new laptop

This creates two tickets in the ServiceNow instance.

Now let’s retrieve the last two tickets using the utterance “list top 2 incident tickets.”

You can test sentiment analysis as shown in the following screenshot, in which the bot responds to a negative sentiment.

Create a Slack app and integrate Slack with the bot

You can integrate the Amazon Lex bot with various web or mobile applications and client-side codes, in addition to popular platforms like Facebook Messenger, Slack, Kik, and Twilio SMS. For this post, you create a Slack app and integrate your Amazon Lex bot with Slack. For instructions, see Integrating an Amazon Lex Bot with Slack.

An added benefit is that the chabot can determine the sentiment of the user and respond accordingly. The real-time sentiment analysis gives the supervisors the feedback that they need in an organic and automated way without requiring any separate process for feedback collection. The sentiment analysis can be used by supervisors to track negative sentiment on tickets created by users and can also be used to return the response accordingly from our Lambda function– for example different responses for negative sentiment v/s positive or neutral sentiment.

You should now be able to use your Slack app to send messages to the Amazon Lex bot and retrieve the same responses as you tested earlier. The following screenshot shows the same messages tested on the Slack app, with the same results.

Congratulations! You just built an incident bot using Amazon Lex with sentiment analysis that integrates with ServiceNow.

Clean up

To avoid incurring future charges, delete the resources that you created and clean up your account.

You can clean up the AWS environment using the following steps:

  1. On the Lex V2 console, choose Bots in the navigation pane to see a list of all your Lex V2 bots.
  2. Select the bot you created and on the Actions menu, choose Delete.
  3. On the Lambda console, choose Functions in the navigation pane.
  4. Select the function you created and on the Actions menu, choose Delete.

Conclusion

This post showed how you can integrate Amazon Lex bot with ServiceNow incident management and a Slack app.  You can integrate the same experience  to create and manage tickets as part of your customer support calls. For more information about incorporating these techniques into your bots, see the Lex V2 Developer Guide.

About the Authors

Chanki Nathani is a Cloud Application Architect for AWS Professional Services. As an architect, he supports customers with architecting, designing, automating and building new applications, as well as migrating existing applications to AWS. He is passionate about Cloud and Serverless Technologies. In his spare time, he enjoys traveling and blogging about food from different places.

Vaibhav Chaddha is a Machine Learning Engineer with AWS Professional Services. He spends his time helping customers design and implement solutions using Amazon ML services, to address their business challenges.

Read More

Don’t let data drift derail edge compute machine learning models

Diagram showing Ekya’s architecture. Video data flows from a series of cameras into specialized, lightweight inference models and shared resource pools before reaching the edge.

Edge computing has come of age, with deployments enabling many applications that process data from IoT sensors and cameras. In 2017, we identified the symbiotic relationship between edge computing and video analytics in an article, noting that live video analytics is the “killer app” for edge computing. Edge devices come in various shapes and sizes but are inherently resource-constrained relative to the cloud. 

These resource constraints necessitate lightweight machine learning (ML) models at the edge. Using techniques for model specialization and compression, the community has obtained edge models whose compute and memory footprints are substantially lower (by 96x for object detector models). Such models are super amenable to deploy at the edge. 

Smooth going so far, but the villain in the story is data drift! This is the phenomenon where the live data in the field diverges significantly from the initial training data. We achieved the phenomenally low compute footprints for edge models only because we specialized the models to be specific to the camera streams. But in the bargain, they lost their ability to generalize much beyond what they have seen during training. This lack of generality comes back to bite us when data drifts and accuracy of the models drop – by as much as 22% – when they are deployed in the field. 

Ekya is a solution, developed with collaborators at University of California, Berkeley and University of Chicago, that addresses the problem of data drift on the edge compute box. Instead of sending video data to the cloud for periodic retraining of models, which is costly in its bandwidth usage and can raise privacy questions, Ekya enables both retraining and inference to co-exist on the edge box. For more details, take a look at our paper: Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers, which has been published at NSDI 2022. We are excited to release the code for Ekya as well. 

Read the paper
Download the code

Not only can you use the code to reproduce all experiments in our paper, we also hope that the code can help you easily build a continuous learning system for your edge deployment. Oh, and one more thing—we are also pointing to the raw video datasets released by the City of Bellevue. This includes 101 hours of video from five traffic intersections, all of which have also been labeled with our golden YOLOv3 model. We hope that the videos from the City of Bellevue as well as the other datasets included in the repository will aid in the building of new edge models as well as improving our pre-trained specialized models to significantly advance the state of the art.

Please reach out to Ganesh Ananthanarayanan with any questions.

Explore More

  • Video

    Video Analytics for Smart Cities

    Microsoft Research has an on-going pilot in Bellevue, Washington for active traffic monitoring of traffic intersections live 24X7. This project is focused on is video streams from cameras at traffic intersections. Traffic-related accidents are among the top 10 reasons […]

The post Don’t let data drift derail edge compute machine learning models appeared first on Microsoft Research.

Read More

Welcome ‘In the NVIDIA Studio’: A Weekly Celebration of Extraordinary Artists, Their Inspiring Art and Innovative Techniques

Creating content is no longer tethered to using paint and stone as mediums, nor being in massive studios. Visual art can now be created anywhere, anytime.

But being creative is still challenging and time-consuming. NVIDIA is making artistic workflows easier and faster by giving creators tools that enable them to remain in their flow state.

That’s what NVIDIA Studio is — an ecosystem of creative app optimizations, GPU-accelerated features and AI-powered apps, powered by NVIDIA RTX GPUs and backed by world-class Studio Drivers.

Our new ‘In the NVIDIA Studio,’ blog series celebrates creativity everywhere by spotlighting 3D animators, video editors, photographers and more, every week. We’ll showcase their inspirational and thought-provoking work, and detail how creators are using NVIDIA GPUs to go from concept to completion, faster than ever.

The series kicks off with 3D artist Jasmin Habezai-Fekri. Check out her work below, created with Unreal Engine, Adobe Substance 3D and Blender, accelerated by her GeForce RTX 2070 GPU.

Habezai-Fekri Dreams in 3D

‘Old Forgotten Library’ and ‘Shiba Statue’ highlight Habezai-Fekri’s use of vivid colors.

Based in Germany, Habezai-Fekri works in gaming as a 3D environment artist, making props and environments with hand-painted and stylized physically based rendering textures. She revels in creating fantasy and nature-themed scenes, accentuated by big, bold colors.


 

Habezai-Fekri’s passion is creating artwork with whimsical charm, piquing the interest of her audiences while creating a sense of immersion, rounding out her unique flair.

 

One such piece is Bird House — a creative fusion of styles and imagination.

With this piece, Habezai-Fekri was learning the ins and outs of using Unreal Engine, while trying to replicate “something very 2D-esque in a 3D space, giving it all a very painterly yet next-gen feeling.” Through iteration, she developed her foundational skills and found that having a set art direction and visual style gave it her own signature.

Prop development for ‘Bird House.’

Habezai-Fekri uses Blender software for modeling and employs Zbrush for her highpoly sculpts to help bring stylized details into textures and models. The fine details are critical for invoking the real-life emotions she hopes to cultivate. “Creating immersiveness is a huge aspect for me when making my art,” Habezai-Fekri said.

Hand-painted, stylized wood textures and details in ‘Bird House.’

Looking closer reveals Habezai-Fekri’s personal touches in the textures in Bird House — she hand-painted them in Adobe Substance 3D Painter. RTX GPU-accelerated light and ambient occlusion in Substance 3D helps speed up her process by outputting new textures in mere seconds.

 

“Having a hand-painted pass on my textures really enhances the assets and lets me channel that artistic side throughout a heavily technical process,” she said.

“In our industry, with new tools being released so frequently, it’s inevitable to constantly learn and expand your skill set. Being open to that from the start really helps to be more receptive to it.”

Habezai-Fekri’s work often uses vivid colors. To make it look inviting and friendly, she purposely saturates colors, even if the subject matter is not colorful by nature.

Habezai-Fekri also finds inspiration in trying new tools and workflows, particularly when she sees other artists and creatives doing amazing work.

By partnering with creative app developers, the NVIDIA Studio ecosystem regularly gives Habezai-Fekri new tools that help her create faster. For example, RTX-accelerated OptiX ray tracing in Blender’s viewport enables her to enjoy interactive, photorealistic rendering in real time.

RTX GPUs also deliver rendering speeds up to 2.5x faster with Blender Cycles 3.0. This means a lot less waiting and a lot more creating.

Everything comes together for Habezai-Fekri with the application of final textures and colors in Unreal Engine. NVIDIA RTX GPUs feature advanced capabilities like DLSS, which enhances interactivity of the viewport in Unreal Engine by using AI to upscale frames rendered at lower resolution, while still retaining detail.

Habezai-Fekri works for Airship Syndicate. Previously, she has been an artist at Square Enix and ArtStation. View her work on ArtStation, including a new learning course providing project insights.

NVIDIA Studio Resources

Habezai-Fekri is one of the artists spotlighted in the latest Studio Standouts video, “Stunning Art From Incredible Women Artists.”

See more amazing digital art in the video from Yulia Sokolova, Nourhan Ishmai, Ecem Okumus and Laura Escoin.

Learn more about texturing in Substance 3D Painter by exploring artist and Adobe Creative Director Vladimir Petkovic’s series, “From Texturing to Final Render in Adobe Substance Painter.”

Join the growing number of 3D artists collaborating around the globe in real time, and working in multiple apps simultaneously, with NVIDIA Omniverse.

Check back In the NVIDIA Studio every week to discover new featured artists, creative tips and tricks, and the latest NVIDIA Studio news. Follow NVIDIA Studio on Facebook, Twitter and Instagram, subscribe to the Studio YouTube channel and get updates directly in your inbox by joining the NVIDIA Studio newsletter.

The post Welcome ‘In the NVIDIA Studio’: A Weekly Celebration of Extraordinary Artists, Their Inspiring Art and Innovative Techniques appeared first on NVIDIA Blog.

Read More