Interact with an Amazon Lex V2 bot with the AWS CLI, AWS SDK for Python (Boto3), and AWS SDK for DotNet

by Amazon AWS

Amazon Lex is a service for building conversational interfaces into any application. The new Amazon Lex V2 console and APIs make it easier to build, deploy, and manage bots. The Amazon Lex V2 console and APIs provide a simple information architecture in which the bot intents and slot types are scoped to a specific language. Versioning is performed at the bot level so that resources such as intents and slot types don’t have to be versioned individually. All the resources associated with the bot (language, intents, and slot types) are archived as part of the created bot version. This new methodology makes it easier to manage your bots. You can find more information in the Amazon Lex V2 API reference.

The Amazon Lex V2 console and APIs introduce a new method to build and interact with your bots. This post illustrates different methods to interact with a Amazon Lex V2 chatbot programmatically using the:

For this post, we use the MakeAppointment bot, which simulates the workflow of setting up an appointment with a dentist.

Obtain ID values for your bot

To get started, you must collect the following information:

  • Your bot ID
  • Your alias ID
  • Your locale ID (language code)

Obtain your bot ID

You can find your bot ID on the Amazon Lex V2 console.

  1. On the Amazon Lex V2 console, on the Bots page, choose the bot you want to interact with.

For this post, we use MakeAppointmentBot.

  1. Choose the bot and in the Bot details section, you can find the ID.

For this post, our ID is UFIDGBA6DE.

Obtain your alias ID

To obtain your bot’s alias ID, complete the following steps:

  1. On the Amazon Lex V2 console, navigate to your bot.
  2. In the navigation pane, under Bot versions, under Deployment, choose Aliases.
  3. Choose your alias.

For this post, we use TestBotAlias.

  1. In the Details section, you can find the alias ID.

For this post, our alias ID is TSTALIASID.

Obtain your locale ID

Your locale ID indicates which language your bot uses.

  1. On the Amazon Lex V2 console, in the navigation pane, choose Bot versions.

  1. Choose the version of your bot you want to use.

For this post, we use the version called Draft version.

  1. In the navigation pane, under your bot version, choose All languages.

You can see a list of the available languages for your chatbot.

  1. Choose your preferred language.

For this post, we use English (US), which is en_US.

Interacting via the AWS CLI

The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

To demonstrate the AWS CLI, you can use AWS CloudShell. AWS CloudShell is a browser-based shell that makes it easy to securely manage, explore, and interact with your AWS resources. CloudShell is pre-authenticated with your console credentials.

There are some cases where AWS CloudShell don’t have the latest version of the AWS CLI installed. If you receive an error “Invalid choice: ‘lexv2-runtime’”, follow the instructions contained in this guide to update the AWS CLI on AWS CloudShell.

Invalid choice: 'lexv2-runtime', maybe you meant:

  * lex-runtime
[cloudshell-user@ip-10-1-148-180 ~]$ aws --version
aws-cli/2.1.16 Python/3.7.3 Linux/4.14.209-160.339.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off

In my case after updating, I got the following version:

[cloudshell-user@ip-10-1-148-180 ~]$ aws --version
aws-cli/2.1.28 Python/3.8.8 Linux/4.14.209-160.339.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off

And this time when I run the “aws lexv2-runtime” command it is recognized:

[cloudshell-user@ip-10-1-148-180 ~]$ aws lexv2-runtime

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help

aws: error: the following arguments are required: operation

After collecting the required information, enter your bot’s parameters (please keep in mind your IDs will be different):

bot-id='UFIDGBA6DE'
bot-alias-id='TSTALIASID'
localeId='en_US'
session-id='test-session'

To get started, you will submit the text “I would like to see the dentist” by using the following command:

aws lexv2-runtime recognize-text --bot-id 'UFIDGBA6DE' --bot-alias-id 'TSTALIASID' --locale-id 'en_US' --session-id 'test_session' --text 'I would like to see the dentist'

You will receive the following response:

{
    "messages": [
        {
            "content": "What type of appointment would you like to schedule?",
            "contentType": "PlainText"
        }
    ],
    "sessionState": {
        "dialogAction": {
            "type": "ElicitSlot",
            "slotToElicit": "AppointmentType"
        },
        "intent": {
            "name": "MakeAppointment",
            "slots": {
                "AppointmentType": null,
                "Date": null,
                "Time": null
            },
            "state": "InProgress",
            "confirmationState": "None"
        },
        "originatingRequestId": "85f22c97-b5d3-4a74-9e3d-95446768ecaa"
    },
    "interpretations": [
        {
            "nluConfidence": {
                "score": 0.84
            },
            "intent": {
                "name": "MakeAppointment",
                "slots": {
                    "AppointmentType": null,
                    "Date": null,
                    "Time": null
                },
                "state": "InProgress",
                "confirmationState": "None"
            }
        },
        {
            "intent": {
                "name": "FallbackIntent",
                "slots": {}
            }
        }
    ],
    "sessionId": "test_session"
}

Diving deep into the response you will notice the dialog state is ElicitSlot, or in other words, the chatbot is requiring the user to provide a value for the slot type ‘AppointmentType’.

 "sessionState": {
        "dialogAction": {
            "type": "ElicitSlot",
            "slotToElicit": "AppointmentType"
        }

Also, you will notice the messages:

 "messages": [
        {
            "content": "What type of appointment would you like to schedule?",
            "contentType": "PlainText"
        }

In this case you will state “Root canal” as the appointment type:

aws lexv2-runtime recognize-text --bot-id 'UFIDGBA6DE' --bot-alias-id 'TSTALIASID' --locale-id 'en_US' --session-id 'test_session' --text 'Root canal'

The response you will receive will look similar to this:

{
    "messages": [
        {
            "content": "When should I schedule your appointment?",
            "contentType": "PlainText"
        }
    ],
    "sessionState": {
        "dialogAction": {
            "type": "ElicitSlot",
            "slotToElicit": "Date"
        },
        "intent": {
            "name": "MakeAppointment",
            "slots": {
                "AppointmentType": {
                    "value": {
                        "originalValue": "Root canal",
                        "interpretedValue": "Root canal",
                        "resolvedValues": [
                            "root canal"
                        ]
                    }
                },
                "Date": null,
                "Time": null
            },
            "state": "InProgress",
            "confirmationState": "None"
        },
        "originatingRequestId": "85f22c97-b5d3-4a74-9e3d-95446768ecaa"
    },
    "interpretations": [
        {
            "nluConfidence": {
                "score": 1.0
            },
            "intent": {
                "name": "MakeAppointment",
                "slots": {
                    "AppointmentType": {
                        "value": {
                            "originalValue": "Root canal",
                            "interpretedValue": "Root canal",
                            "resolvedValues": [
                                "root canal"
                            ]
                        }
                    },
                    "Date": null,
                    "Time": null
                },
                "state": "InProgress",
                "confirmationState": "None"
            }
        },
        {
            "intent": {
                "name": "FallbackIntent",
                "slots": {}
            }
        }
    ],
    "sessionId": "test_session"
}

Check the current dialogAction:

"sessionState": {
        "dialogAction": {
            "type": "ElicitSlot",
            "slotToElicit": "Date"
        }

Also, the messages:

"messages": [
        {
            "content": "When should I schedule your appointment?",
            "contentType": "PlainText"
        }

This time you will provide the pickup date:

aws lexv2-runtime recognize-text --bot-id 'UFIDGBA6DE' --bot-alias-id 'TSTALIASID' --locale-id 'en_US' --session-id 'test_session' --text 'Next Wednesday'

This time the response will look similar to this:

{
    "messages": [
        {
            "content": "At what time should I schedule your appointment?",
            "contentType": "PlainText"
        }
    ],
    "sessionState": {
        "dialogAction": {
            "type": "ElicitSlot",
            "slotToElicit": "Time"
        },
        "intent": {
            "name": "MakeAppointment",
            "slots": {
                "AppointmentType": {
                    "value": {
                        "originalValue": "Root canal",
                        "interpretedValue": "Root canal",
                        "resolvedValues": [
                            "root canal"
                        ]
                    }
                },
                "Date": {
                    "value": {
                        "originalValue": "Next Wednesday",
                        "interpretedValue": "2021-03-17",
                        "resolvedValues": [
                            "2021-03-17"
                        ]
                    }
                },
                "Time": null
            },
            "state": "InProgress",
            "confirmationState": "None"
        },
        "originatingRequestId": "85f22c97-b5d3-4a74-9e3d-95446768ecaa"
    },
    "interpretations": [
        {
            "nluConfidence": {
                "score": 1.0
            },
            "intent": {
                "name": "MakeAppointment",
                "slots": {
                    "AppointmentType": {
                        "value": {
                            "originalValue": "Root canal",
                            "interpretedValue": "Root canal",
                            "resolvedValues": [
                                "root canal"
                            ]
                        }
                    },
                    "Date": {
                        "value": {
                            "originalValue": "Next Wednesday",
                            "interpretedValue": "2021-03-17",
                            "resolvedValues": [
                                "2021-03-17"
                            ]
                        }
                    },
                    "Time": null
                },
                "state": "InProgress",
                "confirmationState": "None"
            }
        },
        {
            "intent": {
                "name": "FallbackIntent",
                "slots": {}
            }
        }
    ],
    "sessionId": "test_session"
}

At time of writing this blog “next Wednesday” is March 17, 2021, which is what your chatbot interpreted:

chatbot interpreted:
 "Date": {
                        "value": {
                            "originalValue": "Next Wednesday",
                            "interpretedValue": "2021-03-17",
                            "resolvedValues": [
                                "2021-03-17"
                            ]
                        }

And the messages:

    "messages": [
        {
            "content": "At what time should I schedule your appointment?",
            "contentType": "PlainText"
        }

Next step is to define the Time:

aws lexv2-runtime recognize-text --bot-id 'UFIDGBA6DE' --bot-alias-id 'TSTALIASID' --locale-id 'en_US' --session-id 'test_session' --text '9 am'

The response will be similar to this:

{
    "messages": [
        {
            "content": "09:00 is available, should I go ahead and book your appointment?",
            "contentType": "PlainText"
        }
    ],
    "sessionState": {
        "dialogAction": {
            "type": "ConfirmIntent"
        },
        "intent": {
            "name": "MakeAppointment",
            "slots": {
                "AppointmentType": {
                    "value": {
                        "originalValue": "Root canal",
                        "interpretedValue": "Root canal",
                        "resolvedValues": [
                            "root canal"
                        ]
                    }
                },
                "Date": {
                    "value": {
                        "originalValue": "Next Wednesday",
                        "interpretedValue": "2021-03-17",
                        "resolvedValues": [
                            "2021-03-17"
                        ]
                    }
                },
                "Time": {
                    "value": {
                        "originalValue": "9 am",
                        "interpretedValue": "09:00",
                        "resolvedValues": [
                            "09:00"
                        ]
                    }
                }
            },
            "state": "InProgress",
            "confirmationState": "None"
        },
        "originatingRequestId": "85f22c97-b5d3-4a74-9e3d-95446768ecaa"
    },
    "interpretations": [
        {
            "nluConfidence": {
                "score": 1.0
            },
            "intent": {
                "name": "MakeAppointment",
                "slots": {
                    "AppointmentType": {
                        "value": {
                            "originalValue": "Root canal",
                            "interpretedValue": "Root canal",
                            "resolvedValues": [
                                "root canal"
                            ]
                        }
                    },
                    "Date": {
                        "value": {
                            "originalValue": "Next Wednesday",
                            "interpretedValue": "2021-03-17",
                            "resolvedValues": [
                                "2021-03-17"
                            ]
                        }
                    },
                    "Time": {
                        "value": {
                            "originalValue": "9 am",
                            "interpretedValue": "09:00",
                            "resolvedValues": [
                                "09:00"
                            ]
                        }
                    }
                },
                "state": "InProgress",
                "confirmationState": "None"
            }
        },
        {
            "intent": {
                "name": "FallbackIntent",
                "slots": {}
            }
        }
    ],
    "sessionId": "test_session"
}

This time your chatbot interpreted the value as 09:00 and used it as the value for the slot Time:

 "Time": {
                    "value": {
                        "originalValue": "9 am",
                        "interpretedValue": "09:00",
                        "resolvedValues": [
                            "09:00"
                        ]
                    }
                }

Because the slots have been set now, the sessionState dialogAction is ConfirmIntent.

'sessionState': {
        'dialogAction': {
            'type': 'ConfirmIntent'
        }

This time, messages contains the details contained in the slots.

    "messages": [
        {
            "content": "09:00 is available, should I go ahead and book your appointment?",
            "contentType": "PlainText"
        }
    ]

You will submit the confirmation:

aws lexv2-runtime recognize-text --bot-id 'UFIDGBA6DE' --bot-alias-id 'TSTALIASID' --locale-id 'en_US' --session-id 'test_session' --text 'yes'

Response will be similar to:

{
    "sessionState": {
        "dialogAction": {
            "type": "Close"
        },
        "intent": {
            "name": "MakeAppointment",
            "slots": {
                "AppointmentType": {
                    "value": {
                        "originalValue": "Root canal",
                        "interpretedValue": "Root canal",
                        "resolvedValues": [
                            "root canal"
                        ]
                    }
                },
                "Date": {
                    "value": {
                        "originalValue": "Next Wednesday",
                        "interpretedValue": "2021-03-17",
                        "resolvedValues": [
                            "2021-03-17"
                        ]
                    }
                },
                "Time": {
                    "value": {
                        "originalValue": "9 am",
                        "interpretedValue": "09:00",
                        "resolvedValues": [
                            "09:00"
                        ]
                    }
                }
            },
            "state": "ReadyForFulfillment",
            "confirmationState": "Confirmed"
        },
        "originatingRequestId": "85f22c97-b5d3-4a74-9e3d-95446768ecaa"
    },
    "interpretations": [
        {
            "nluConfidence": {
                "score": 1.0
            },
            "intent": {
                "name": "MakeAppointment",
                "slots": {
                    "AppointmentType": {
                        "value": {
                            "originalValue": "Root canal",
                            "interpretedValue": "Root canal",
                            "resolvedValues": [
                                "root canal"
                            ]
                        }
                    },
                    "Date": {
                        "value": {
                            "originalValue": "Next Wednesday",
                            "interpretedValue": "2021-03-17",
                            "resolvedValues": [
                                "2021-03-17"
                            ]
                        }
                    },
                    "Time": {
                        "value": {
                            "originalValue": "9 am",
                            "interpretedValue": "09:00",
                            "resolvedValues": [
                                "09:00"
                            ]
                        }
                    }
                },
                "state": "ReadyForFulfillment",
                "confirmationState": "Confirmed"
            }
        },
        {
            "intent": {
                "name": "FallbackIntent",
                "slots": {}
            }
        }
    ],
    "sessionId": "test_session"
}

Also the intent sate is “ReadyForFulfillment” and “Confirmed”

'state': 'ReadyForFulfillment',
'confirmationState': 'Confirmed'

Interact programmatically with Amazon Lex V2 via Boto3

Boto is the AWS SDK for Python. It enables Python developers to create, configure, and manage AWS services, such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3). Boto provides an easy-to-use, object-oriented API, as well as low-level access to AWS services.

Support for Amazon Lex V2 APIs is introduced in version 1.16.59.

To interact with your Amazon Lex bot using the Boto3 lexv2-runtime, you need to use the values that you collected earlier for botID, AliasId, and localeID.

Boto3 code example

After I collected the required information, I have the following parameters (these will be different for your account):

botId='UFIDGBA6DE'
botAliasId='TSTALIASID'
localeId='en_US'

The first message you send to your bot is “I would like to see the dentist “:

import boto3
# LexV2 client uses 'lexv2-runtime'
client = boto3.client('lexv2-runtime')

# Submit the text 'I would like to see a dentist'
response = client.recognize_text(
    botId='UFIDGBA6DE',
    botAliasId='TSTALIASID',
    localeId='en_US',
    sessionId="test_session",
    text='I would like to see the dentist')

You get a response similar to the following:

{
    'ResponseMetadata': {
        'RequestId': 'e998ce7f-3381-4501-9973-970b221504a5',
        'HTTPStatusCode': 200,
        'HTTPHeaders': {
            'x-amzn-requestid': 'e998ce7f-3381-4501-9973-970b221504a5',
            'date': 'Tue,
            9Mar202123: 38: 01GMT',
            'content-type': 'application/json',
            'content-length': '666'
        },
        'RetryAttempts': 0
    },
    'messages': [
        {
            'content': 'What type of appointment would you like to schedule?',
            'contentType': 'PlainText'
        }
    ],
    'sessionState': {
        'dialogAction': {
            'type': 'ElicitSlot',
            'slotToElicit': 'AppointmentType'
        },
        'intent': {
            'name': 'MakeAppointment',
            'slots': {
                'AppointmentType': None,
                'Date': None,
                'Time': None
            },
            'state': 'InProgress',
            'confirmationState': 'None'
        },
        'originatingRequestId': '85f22c97-b5d3-4a74-9e3d-95446768ecaa'
    },
    'interpretations': [
        {
            'nluConfidence': {
                'score': 0.82
            },
            'intent': {
                'name': 'MakeAppointment',
                'slots': {
                    'AppointmentType': None,
                    'Date': None,
                    'Time': None
                },
                'state': 'InProgress',
                'confirmationState': 'None'
            }
        },
        {
            'intent': {
                'name': 'FallbackIntent',
                'slots': {
                    
                }
            }
        }
    ],
    'sessionId': 'test_session'
}

We can dive deeper into the response to see the dialog state is ElicitSlot, meaning the chatbot is requiring the user to provide a value for the slot type AppointmentType:

'sessionState': {
        'dialogAction': {
            'type': 'ElicitSlot',
            'slotToElicit': 'AppointmentType'
        }

In this case, you enter Root Canal:

response = client.recognize_text(
    botId='UFIDGBA6DE',
    botAliasId='TSTALIASID',
    localeId='en_US',
    sessionId="test_session",
    text='Root Canal')

This time, your response is the following:


{
    'ResponseMetadata': {
        'RequestId': 'fae988d3-c956-4ea4-b5f8-2646d914adf4',
        'HTTPStatusCode': 200,
        'HTTPHeaders': {
            'x-amzn-requestid': 'fae988d3-c956-4ea4-b5f8-2646d914adf4',
            'date': 'Tue,
            9Mar202123: 40: 03GMT',
            'content-type': 'application/json',
            'content-length': '842'
        },
        'RetryAttempts': 0
    },
    'messages': [
        {
            'content': 'When should I schedule your appointment?',
            'contentType': 'PlainText'
        }
    ],
    'sessionState': {
        'dialogAction': {
            'type': 'ElicitSlot',
            'slotToElicit': 'Date'
        },
        'intent': {
            'name': 'MakeAppointment',
            'slots': {
                'AppointmentType': {
                    'value': {
                        'originalValue': 'RootCanal',
                        'interpretedValue': 'RootCanal',
                        'resolvedValues': [
                            'rootcanal'
                        ]
                    }
                },
                'Date': None,
                'Time': None
            },
            'state': 'InProgress',
            'confirmationState': 'None'
        },
        'originatingRequestId': '85f22c97-b5d3-4a74-9e3d-95446768ecaa'
    },
    'interpretations': [
        {
            'nluConfidence': {
                'score': 1.0
            },
            'intent': {
                'name': 'MakeAppointment',
                'slots': {
                    'AppointmentType': {
                        'value': {
                            'originalValue': 'RootCanal',
                            'interpretedValue': 'RootCanal',
                            'resolvedValues': [
                                'rootcanal'
                            ]
                        }
                    },
                    'Date': None,
                    'Time': None
                },
                'state': 'InProgress',
                'confirmationState': 'None'
            }
        },
        {
            'intent': {
                'name': 'FallbackIntent',
                'slots': {
                    
                }
            }
        }
    ],
    'sessionId': 'test_session'
}

The chatbot interpreted that the value for the slot AppointmentType is root canal:

'slots': {
                'AppointmentType': {
                    'value': {
                        'originalValue': 'Root Canal',
                        'interpretedValue': 'Root Canal',
                        'resolvedValues': [
                            'root canal'
                        ]
                    }
                },
                'Date': None,
                'Time': None
            }

You can also check the current dialogAction:

'sessionState': {
        'dialogAction': {
            'type': 'ElicitSlot',
            'slotToElicit': 'Date'
        }

In the messages section, you can see the following:

    'messages': [
        {
            'content': 'When should I schedule your appointment?',
            'contentType': 'PlainText'
        }
    ]

Provide appointment date:

response = client.recognize_text(
    botId='UFIDGBA6DE',
    botAliasId='TSTALIASID',
    localeId='en_US',
    sessionId="test_session",
    text='Next Wednesday')

You get a response similar to the following:

{
    'ResponseMetadata': {
        'RequestId': '8639e4dd-bbfe-42a3-aa9e-96391fd821fd',
        'HTTPStatusCode': 200,
        'HTTPHeaders': {
            'x-amzn-requestid': '8639e4dd-bbfe-42a3-aa9e-96391fd821fd',
            'date': 'Tue,
            9Mar202123: 41: 51GMT',
            'content-type': 'application/json',
            'content-length': '1058'
        },
        'RetryAttempts': 0
    },
    'messages': [
        {
            'content': 'At what time should I schedule your appointment?',
            'contentType': 'PlainText'
        }
    ],
    'sessionState': {
        'dialogAction': {
            'type': 'ElicitSlot',
            'slotToElicit': 'Time'
        },
        'intent': {
            'name': 'MakeAppointment',
            'slots': {
                'AppointmentType': {
                    'value': {
                        'originalValue': 'Root Canal',
                        'interpretedValue': 'Root Canal',
                        'resolvedValues': [
                            'root canal'
                        ]
                    }
                },
                'Date': {
                    'value': {
                        'originalValue': 'Next Wednesday',
                        'interpretedValue': '2021-03-17',
                        'resolvedValues': [
                            '2021-03-17'
                        ]
                    }
                },
                'Time': None
            },
            'state': 'InProgress',
            'confirmationState': 'None'
        },
        'originatingRequestId': '85f22c97-b5d3-4a74-9e3d-95446768ecaa'
    },
    'interpretations': [
        {
            'nluConfidence': {
                'score': 1.0
            },
            'intent': {
                'name': 'MakeAppointment',
                'slots': {
                    'AppointmentType': {
                        'value': {
                            'originalValue': 'RootCanal',
                            'interpretedValue': 'RootCanal',
                            'resolvedValues': [
                                'rootcanal'
                            ]
                        }
                    },
                    'Date': {
                        'value': {
                            'originalValue': 'Next Wednesday',
                            'interpretedValue': '2021-03-17',
                            'resolvedValues': [
                                '2021-03-17'
                            ]
                        }
                    },
                    'Time': None
                },
                'state': 'InProgress',
                'confirmationState': 'None'
            }
        },
        {
            'intent': {
                'name': 'FallbackIntent',
                'slots': {
                    
                }
            }
        }
    ],
    'sessionId': 'test_session'
}

At the time of this writing, “next Wednesday” is March 17, 2021, which is exactly what your chatbot interpreted:

 'Date': {
                        'value': {
                            'originalValue': 'Next Wednesday',
                            'interpretedValue': '2021-03-17',
                            'resolvedValues': [
                                '2021-03-17'
                            ]
                        }

You can see the following in the messages section:

'messages': [
        {
            'content': 'At what time should I schedule your appointment?',
            'contentType': 'PlainText'
        }
    ]

The next step is to define the PickupTime:

'sessionState': {
        'dialogAction': {
            'type': 'ElicitSlot',
            'slotToElicit': 'Time'
        }

Use the following code:

response = client.recognize_text(
    botId='UFIDGBA6DE',
    botAliasId='TSTALIASID',
    localeId='en_US',
    sessionId="test_session",
    text='9 am')

You receive a response similar to the following:

{
    'ResponseMetadata': {
        'RequestId': 'e8c06004-960b-442f-bf0d-0d995ec366bf',
        'HTTPStatusCode': 200,
        'HTTPHeaders': {
            'x-amzn-requestid': 'e8c06004-960b-442f-bf0d-0d995ec366bf',
            'date': 'Tue,
            9Mar202123: 44: 39GMT',
            'content-type': 'application/json',
            'content-length': '1223'
        },
        'RetryAttempts': 0
    },
    'messages': [
        {
            'content': '09:00 is available, should I go ahead and book your appointment?',
            'contentType': 'PlainText'
        }
    ],
    'sessionState': {
        'dialogAction': {
            'type': 'ConfirmIntent'
        },
        'intent': {
            'name': 'MakeAppointment',
            'slots': {
                'AppointmentType': {
                    'value': {
                        'originalValue': 'Root Canal',
                        'interpretedValue': 'Root Canal',
                        'resolvedValues': [
                            'root canal'
                        ]
                    }
                },
                'Date': {
                    'value': {
                        'originalValue': 'Next Wednesday',
                        'interpretedValue': '2021-03-17',
                        'resolvedValues': [
                            '2021-03-17'
                        ]
                    }
                },
                'Time': {
                    'value': {
                        'originalValue': '9 am',
                        'interpretedValue': '09:00',
                        'resolvedValues': [
                            '09:00'
                        ]
                    }
                }
            },
            'state': 'InProgress',
            'confirmationState': 'None'
        },
        'originatingRequestId': '85f22c97-b5d3-4a74-9e3d-95446768ecaa'
    },
    'interpretations': [
        {
            'nluConfidence': {
                'score': 1.0
            },
            'intent': {
                'name': 'MakeAppointment',
                'slots': {
                    'AppointmentType': {
                        'value': {
                            'originalValue': 'Root Canal',
                            'interpretedValue': 'Root Canal',
                            'resolvedValues': [
                                'rootcanal'
                            ]
                        }
                    },
                    'Date': {
                        'value': {
                            'originalValue': 'Next Wednesday',
                            'interpretedValue': '2021-03-17',
                            'resolvedValues': [
                                '2021-03-17'
                            ]
                        }
                    },
                    'Time': {
                        'value': {
                            'originalValue': '9 am',
                            'interpretedValue': '09:00',
                            'resolvedValues': [
                                '09:00'
                            ]
                        }
                    }
                },
                'state': 'InProgress',
                'confirmationState': 'None'
            }
        },
        {
            'intent': {
                'name': 'FallbackIntent',
                'slots': {
                    
                }
            }
        }
    ],
    'sessionId': 'test_session'
}

The chatbot set your appointment Time for 9:00 AM:

 'Time': {
                        'value': {
                            'originalValue': '9 am',
                            'interpretedValue': '09:00',
                            'resolvedValues': [
                                '09:00'
                            ]
                        }

Because the slots have been set, the sessionState dialogAction is now ConfirmIntent:

'sessionState': {
        'dialogAction': {
            'type': 'ConfirmIntent'
        }

This time, the messages section contains the details contained in the slots:

'messages': [
        {
            'content': '09:00 is available, should I go ahead and book your appointment?',
            'contentType': 'PlainText'
        }

You submit the confirmation:

response = client.recognize_text(
    botId='4HKK09CVHD',
    botAliasId='TSTALIASID',
    localeId='en_US',
    sessionId="test_session",
    text='yes')

The response looks like the following:

{
    'ResponseMetadata': {
        'RequestId': '362d4478-caea-49cf-8b1b-b0ab035e324f',
        'HTTPStatusCode': 200,
        'HTTPHeaders': {
            'x-amzn-requestid': '362d4478-caea-49cf-8b1b-b0ab035e324f',
            'date': 'Tue,
            9Mar202123: 46: 48GMT',
            'content-type': 'application/json',
            'content-length': '1125'
        },
        'RetryAttempts': 0
    },
    'sessionState': {
        'dialogAction': {
            'type': 'Close'
        },
        'intent': {
            'name': 'MakeAppointment',
            'slots': {
                'AppointmentType': {
                    'value': {
                        'originalValue': 'Root Canal',
                        'interpretedValue': 'Root Canal',
                        'resolvedValues': [
                            'root canal'
                        ]
                    }
                },
                'Date': {
                    'value': {
                        'originalValue': 'Next Wednesday',
                        'interpretedValue': '2021-03-17',
                        'resolvedValues': [
                            '2021-03-17'
                        ]
                    }
                },
                'Time': {
                    'value': {
                        'originalValue': '9 am',
                        'interpretedValue': '09:00',
                        'resolvedValues': [
                            '09:00'
                        ]
                    }
                }
            },
            'state': 'ReadyForFulfillment',
            'confirmationState': 'Confirmed'
        },
        'originatingRequestId': '85f22c97-b5d3-4a74-9e3d-95446768ecaa'
    },
    'interpretations': [
        {
            'nluConfidence': {
                'score': 1.0
            },
            'intent': {
                'name': 'MakeAppointment',
                'slots': {
                    'AppointmentType': {
                        'value': {
                            'originalValue': 'Root Canal',
                            'interpretedValue': 'Root Canal',
                            'resolvedValues': [
                                'root canal'
                            ]
                        }
                    },
                    'Date': {
                        'value': {
                            'originalValue': 'Next Wednesday',
                            'interpretedValue': '2021-03-17',
                            'resolvedValues': [
                                '2021-03-17'
                            ]
                        }
                    },
                    'Time': {
                        'value': {
                            'originalValue': '9 am',
                            'interpretedValue': '09:00',
                            'resolvedValues': [
                                '09:00'
                            ]
                        }
                    }
                },
                'state': 'ReadyForFulfillment',
                'confirmationState': 'Confirmed'
            }
        },
        {
            'intent': {
                'name': 'FallbackIntent',
                'slots': {
                    
                }
            }
        }
    ],
    'sessionId': 'test_session'
}

The sessionState dialogAction this time is Close:

'sessionState': {
        'dialogAction': {
            'type': 'Close'
        }

Also, the intent state is ReadyForFulfillment and Confirmed:

'state': 'ReadyForFulfillment',
'confirmationState': 'Confirmed'

About DotNet SDK

The AWS SDK for DotNet provides an interface that you can use from your DotNet applications to interact with your bots. You can use the SDK for DotNet to build client applications for users.

The following application interacts with the . It uses the AmazonLexRuntimeV2Client from the SDK for DotNet to call the RecognizeText operation to conduct a conversation with the bot.

To run the application, you need the following values:

  • botId –The identifier assigned to the bot when you created it. You can see the bot ID on the Amazon Lex console on the Bot details
  • botAliasId – The identifier assigned to the bot alias when you created it. You can see the bot alias ID on the Amazon Lex console on the Aliases If you can’t see the alias ID in the list, choose the gear icon on the upper right and turn on Alias ID.
  • localeId – The identifier of the locale that you used for your bot. For a list of locales and language codes, see Languages and locales supported by Amazon Lex.
  • accessKey and secretKey – The authentication keys for your account. If you don’t have a set of keys, create them using the AWS Identity and Access Management (IAM) console.
  • sessionId – An identifier for the session with the Amazon Lex bot. In this case, the code uses a random UUID.

Additionally, if your bot isn’t in the US East (N. Virginia) AWS Region, make sure that you change the region value.

Let’s look at the following sample code to understand how we can easily interact with Amazon Lex V2 from the DotNet framework.

First, we create an AmazonLexRuntimeV2Client object with the access key and secret:

            // create a LexV2 runtime client
            AmazonLexRuntimeV2Client client = 
            new AmazonLexRuntimeV2Client(access_key
                                        , secret_key
                                        , Amazon.RegionEndpoint.GetBySystemName(preferred_region));

Provide the following information:

// provide the parameters
// Randomly generated GUID unique per session
string sessionId = Guid.NewGuid().ToString();

// provide the unique BotId
string botId = "XXXXX";
//provide the unique BotAliasId 
string botAliasId = "XXXXX";
// refer to the list of supported locales 
string localeId = "en_US";

The request uses the required URI parameters botAliasId, botId, localeId, and sessionId.

The user initiates the conversation by providing the intent:

string userInput1 = ""

Next, we call the GetRecongizeTextV2Response method by passing the required parameters and the AmazonLexRuntimeV2Client client object:

var response1 = GetRecongizeTextV2Response(client, botId, botAliasId, localeId, sessionId, userInput1)

Inside the GetRecongizeTextV2Response method, we create a RecognizeTextRequest object, containing the parameters to the RecognizeText operation:

RecognizeTextRequest request = new RecognizeTextRequest
{
    BotAliasId = botAliasId,
    BotId = botId,
    LocaleId = localeId,
    SessionId = sessionId,
    Text = userInput
};

The RecognizeTextRequest in JSON format looks like the following:

{
    "BotAliasId": "YOURBOTALIASID",
    "BotId": "YOURBOTID",
    "LocaleId": "en_US",
    "RequestAttributes": {
        
    },
    "SessionId": "cb447544-1920-432d-aa41-4ff2237a0a62",
    "SessionStateValue": null,
    "Text": "I would like to see the dentist"
}

Then we send the user input to Amazon Lex. Client applications can use this API to send requests to Amazon Lex at runtime. You can send requests synchronously or asynchronously. For this post, we send the request using the RecognizeTextAsync method asynchronously. Amazon Lex interprets the user input using the machine learning model that it built for the bot. See the following code:

response = client.RecognizeTextAsync(request).GetAwaiter().GetResult();

In response, Amazon Lex returns the next message to convey to the user and an optional response card to display:

{
    "Interpretations": [
        {
            "Intent": {
                "ConfirmationState": {
                    "Value": "None"
                },
                "Name": "MakeAppointment",
                "Slots": {
                    "AppointmentType": null,
                    "Date": null,
                    "Time": null
                },
                "State": {
                    "Value": "InProgress"
                }
            },
            "NluConfidence": {
                "Score": 0.84
            },
            "SentimentResponse": null
        },
        {
            "Intent": {
                "ConfirmationState": null,
                "Name": "FallbackIntent",
                "Slots": {
                    
                },
                "State": null
            },
            "NluConfidence": null,
            "SentimentResponse": null
        }
    ],
    "Messages": [
        {
            "Content": "What type of appointment would you like to schedule?",
            "ContentType": {
                "Value": "PlainText"
            },
            "ImageResponseCard": null
        }
    ],
    "RequestAttributes": {
        
    },
    "SessionId": "cb447544-1920-432d-aa41-4ff2237a0a62",
    "SessionStateValue": {
        "ActiveContexts": [
            
        ],
        "DialogAction": {
            "SlotToElicit": "AppointmentType",
            "Type": {
                "Value": "ElicitSlot"
            }
        },
        "Intent": {
            "ConfirmationState": {
                "Value": "None"
            },
            "Name": "MakeAppointment",
            "Slots": {
                "AppointmentType": null,
                "Date": null,
                "Time": null
            },
            "State": {
                "Value": "InProgress"
            }
        },
        "OriginatingRequestId": "36e1cb54-1861-45c9-8d67-898e632daf31",
        "SessionAttributes": {
            
        }
    },
    "ResponseMetadata": {
        "RequestId": "36e1cb54-1861-45c9-8d67-898e632daf31",
        "Metadata": {
            
        }
    },
    "ContentLength": 690,
    "HttpStatusCode": 200
}

As you can see, Amazon Lex returns the next message inside the Content as a plain text, with HttpStatusCode as 200.

DotNet SDK code example

The following is the full source code:

using System;
using Amazon.LexModelsV2;
using Amazon.LexRuntimeV2;
using Amazon.LexRuntimeV2.Model;

// This is a sample application to interact with a bot using RecognizeText API using DotNet.
namespace OrderFlowerV2
{
    class OrderFlowers
    {
      
        const string access_key = "XXXXXXXXXX"; // provide you access key
        const string secret_key = "XXXXXXXXXX"; // provide the secret
        const string preferred_region = "XXXXX"; // provide an appropriate region


        
        static void Main(string[] args)
        {
            
            // create a LexV2 runtime client
            AmazonLexRuntimeV2Client client = new AmazonLexRuntimeV2Client(access_key,
            secret_key, Amazon.RegionEndpoint.GetBySystemName(preferred_region));
            
            string sessionId = Guid.NewGuid().ToString(); // Randomly generated GUID unique per session.
            string botId = "XXXXX"; // provide the unique BotId
            string botAliasId = "XXXXX"; //provide the unique BotAliasId
            string localeId = "en_US"; // refer to the list of supported locales

            // utterance 1
            string userInput1 = "I would like to order flowers";
            var response1 = GetRecongizeTextV2Response(client
                                                        , botId, botAliasId, localeId, sessionId, userInput1);
            
            Console.WriteLine("User - "+userInput1);
            foreach (var message in response1.Messages)
            {
                Console.WriteLine("Bot - "+message.Content);
            }

            // utterance 2
            string userInput2 = "1 rose";
            var response2 = GetRecongizeTextV2Response(client
                                                       , botId, botAliasId, localeId, sessionId, userInput2);
            
            Console.WriteLine("User - "+userInput2);
            foreach (var message in response2.Messages)
            {
                Console.WriteLine("Bot - "+message.Content);
            }

            // utterance 3
            string userInput3 = "next Monday";
            var response3 = GetRecongizeTextV2Response(client, botId, botAliasId, localeId, sessionId, userInput3);
            
            Console.WriteLine("User - "+userInput3);
            foreach (var message in response3.Messages)
            {
                Console.WriteLine("Bot - "+message.Content);
            }

            // utterance 4
            string userInput4 = "7 in the evening";
            var response4 = GetRecongizeTextV2Response(client, botId, botAliasId, localeId, sessionId, userInput4);
            
            Console.WriteLine("User - "+userInput4);
            foreach (var message in response4.Messages)
            {
                Console.WriteLine("Bot - "+message.Content);
            }

            // utterance 5
            string userInput5 = "Yes";
            var response5 = GetRecongizeTextV2Response(client, botId, botAliasId, localeId, sessionId, userInput5);
            
            Console.WriteLine("User - "+userInput5);
            foreach (var message in response5.Messages)
            {
                Console.WriteLine("Bot - "+message.Content);
            }
        }
        /// <summary>
        /// Call Amazon LexV2 RecognizeTextAsync method to get the response from the Bot
        /// </summary>
         /// <param name="client"></param>
        /// <param name="botId"></param>
        /// <param name="botAliasId"></param>
        /// <param name="localeId"></param>
        /// <param name="sessionId"></param>
        /// <param name="userInput"></param>
        /// <returns></returns>
        private static RecognizeTextResponse GetRecongizeTextV2Response(AmazonLexRuntimeV2Client client, string botId, string botAliasId, string localeId, string sessionId, string userInput)
        {
            // create a RecognizeTextResponse object
            RecognizeTextResponse response = new RecognizeTextResponse();
            try
            {
                // create RecognizeTextRequest object with the required parameters
                RecognizeTextRequest request = new RecognizeTextRequest
                {
                    BotAliasId = botAliasId,
                    BotId = botId,
                    LocaleId = localeId,
                    SessionId = sessionId,
                    Text = userInput
                };
                // call the RecognizeText async method to get the response 
                response = client.RecognizeTextAsync(request).GetAwaiter().GetResult();
                
            }catch(AmazonLexModelsV2Exception exception)
            {
                Console.WriteLine("AmazonLexModelsV2Exception - "+exception.Message);
            }
            catch(AmazonLexRuntimeV2Exception exception)
            {
                Console.WriteLine("AmazonLexRuntimeV2Exception - "+exception.Message);
            }
            catch(Exception exception)
            {
                Console.WriteLine("Exception - "+exception.Message);
            }
            return response;
        }
    }
}

Conclusion

Now that you have learned how to interact with your Amazon Lex v2 chatbot programmatically with the AWS CLI, Boto3, and, the AWS SDK for DotNet, you can integrate your bots with your applications or with different platforms. If you want to learn more about Amazon Lex v2, you can explore building a banking bot with support for English and Spanish. You can also dive deep into Amazon Lex through the Developer Guide.

About the Authors

Juan Bustos is an AI Services Specialist Solutions Architect at Amazon Web Services, based in Dallas, TX. Outside of work, he loves spending time writing and playing music as well as trying random restaurants with his family.

 

 

 

Wrick Talukdar is a Senior Solutions Architect with Amazon Web Services and is based in Calgary, Canada. Wrick works with enterprise AWS customers to transform their business through innovative use of cloud technologies. Outside work he enjoys reading and photography.

 

 

 

Esther Lee is a Product Manager for AWS Language AI Services. She is passionate about the intersection of technology and education. Out of the office, Esther enjoys long walks along the beach, dinners with friends and friendly rounds of Mahjong.

Read More

DeepLearning.AI, Coursera, and AWS launch the new Practical Data Science Specialization with Amazon SageMaker

by Amazon AWS

Amazon Web Services (AWS), Coursera, and DeepLearning.AI are excited to announce Practical Data Science, a three-course, 10-week, hands-on specialization designed for data professionals to quickly learn the essentials of machine learning (ML) in the AWS Cloud. DeepLearning.AI was founded in 2017 by Andrew Ng, an ML and education pioneer, to fill a need for world-class AI education. DeepLearning.AI teamed up with an all-female team of instructors including Amazon ML Solutions Architects and Developer Advocates to develop and deliver the three-course specialization on Coursera’s education platform. Sign up for the Practical Data Science Specialization today on Coursera.

Moving data science projects from idea to production requires a new set of skills to address the scale and operational efficiencies required by today’s ML problems. This specialization addresses common challenges we hear from our customers and teaches you the practical knowledge needed to efficiently deploy your data science projects at scale in the AWS Cloud.

Specialization overview

The Practical Data Science Specialization is designed for data-focused developers, scientists, and analysts familiar with Python to learn how to build, train, and deploy scalable, end-to-end ML pipelines—both automated and human-in-the-loop—in the AWS Cloud. Each of the 10 weeks features a comprehensive, hands-on lab developed specifically for this specialization and hosted by AWS Partner Vocareum. The labs provide hands-on experience with state-of-the-art algorithms for natural language processing (NLP) and natural language understanding (NLU) using Amazon SageMaker and Hugging Face’s highly-optimized implementation of the BERT algorithm.

In the first course, you learn foundational concepts for exploratory data analysis (EDA), automated machine learning (AutoML), and text classification algorithms. With Amazon SageMaker Clarify and Amazon SageMaker Data Wrangler, you analyze a dataset for statistical bias, transform the dataset into machine-readable features, and select the most important features to train a multi-class text classifier. You then perform AutoML to automatically train, tune, and deploy the best text classification algorithm for the given dataset using Amazon SageMaker Autopilot. Next, you work with Amazon SageMaker BlazingText, a highly optimized and scalable implementation of the popular FastText algorithm, to train a text classifier with very little code.

In the second course, you learn to automate an NLP task by building an end-to-end ML pipeline using BERT with Amazon SageMaker Pipelines. Your pipeline first transforms the dataset into BERT-readable features and stores the features in the Amazon SageMaker Feature Store. It then fine-tunes a text classification model to the dataset using a Hugging Face pre-trained model that has learned to understand human language from millions of Wikipedia documents. Finally, your pipeline evaluates the model’s accuracy and only deploys the model if the accuracy exceeds a given threshold.

In the third course, you learn a series of performance-improvement and cost-reduction techniques to automatically tune model accuracy, compare prediction performance, and generate new training data with human intelligence. After tuning your text classifier using hyperparameter tuning, you deploy two model candidates into an A/B test to compare their real-time prediction performance and automatically scale the winning model using Amazon SageMaker Hosting. Lastly, you set up a human-in-the-loop pipeline to fix misclassified predictions and generate new training data using Amazon Augmented AI (Amazon A2I) and Amazon SageMaker Ground Truth.

“The field of data science is constantly evolving with new tools, technologies, and methods,” says Betty Vandenbosch, Chief Content Officer at Coursera. “We’re excited to expand our collaboration with DeepLearning.AI and AWS to help data scientists around the world keep up with the many tools at their disposal. Through hands-on learning, cutting-edge technology, and expert instruction, this new content will help learners acquire the latest job-relevant data science skills.”

Register today

The Practical Data Science Specialization from DeepLearning.AI, AWS, and Coursera is a great way to learn AI and ML essentials in the cloud. The three-course specialization is a great resource to start building and operationalizing data science projects efficiently with the depth and breadth of Amazon ML services. Improve your data science skills by signing up for the Practical Data Science Specialization today at Coursera!

About the Authors

 Antje Barth is a Senior Developer Advocate for AI and Machine Learning at Amazon Web Services (AWS). She is co-author of the O’Reilly book – Data Science on AWS. Antje frequently speaks at AI / ML conferences, events, and meetups around the world. Previously, Antje worked in technical evangelism and solutions engineering at Cisco and MapR, focused on data center technologies, big data, and AI applications. Antje co-founded the Düsseldorf chapter of Women in Big Data.

 

Chris Fregly is a Principal Developer Advocate for AI and Machine Learning at Amazon Web Services (AWS). He is a co-author of the O’Reilly book – Data Science on AWS. Chris has founded multiple global meetups focused on Apache Spark, TensorFlow, and Kubeflow. He regularly speaks at AI / ML conferences worldwide, including O’Reilly AI & Strata, Open Data Science Conference (ODSC), and GPU Technology Conference (GTC). Previously, Chris founded PipelineAI, where he worked with many AI-first startups and enterprises to continuously deploy ML/AI Pipelines using Apache Spark ML, Kubernetes, TensorFlow, Kubeflow, Amazon EKS, and Amazon SageMaker.

 

Shelbee EigenbrodeShelbee Eigenbrode is a Principal AI and Machine Learning Specialist Solutions Architect at Amazon Web Services (AWS). She holds 6 AWS certifications and has been in technology for 23 years spanning multiple industries, technologies, and roles. She is currently focusing on combining her DevOps and ML background to deliver and manage ML workloads at scale. With over 35 patents granted across various technology domains, she has a passion for continuous innovation and using data to drive business outcomes. Shelbee co-founded the Denver chapter of Women in Big Data.

 

Sireesha Muppala is an Enterprise Principal SA, AI/ML at Amazon Web Services (AWS) who guides customers on architecting and implementing machine learning solutions at scale. She received her Ph.D. in Computer Science from the University of Colorado, Colorado Springs, and has authored several research papers, whitepapers, blog articles. Sireesha frequently speaks at industry conferences, events, and meetups. She co-founded the Denver chapter of Women in Big Data.

Read More

Use Amazon Translate in Amazon SageMaker Notebooks

by Amazon AWS

Amazon Translate is a neural machine translation service that delivers fast, high-quality, and affordable language translation in 71 languages and 4,970 language pairs. Amazon Translate is great for performing batch translation when you have large quantities of pre-existing text to translate and real-time translation when you want to deliver on-demand translations of content as a feature of your applications. It can also handle documents that are written in multiple languages.

Document automation is a common use case where machine learning (ML) can be applied to simplify storing, managing, and extracting insights from documents. In this post, we look at how to run batch translation jobs using the Boto3 Python library as run from an Amazon SageMaker notebook instance. You can also generalize this process to run batch translation jobs from other AWS compute services.

Roles and permissions

We start by creating an AWS Identity and Access Management (IAM) role and access policy to allow SageMaker to run batch translation jobs. If you’re using a simple text translation (such as under 5,000 bytes), the job is synchronous and the data is passed to Amazon Translate as bytes, However, when run as a batch translation job where files are accessed from an Amazon Simple Storage Service (Amazon S3) bucket, the data is read directly by Amazon Translate instead of being passed as bytes by the code run in the SageMaker notebook (in case of shorter text strings).

This section creates the permissions need to allow Amazon Translate access the S3 files.

  1. On the IAM console, choose Roles.
  2. Choose Create a role.
  3. Choose AWS service as your trusted entity.
  4. For Common use cases, choose EC2 or Lambda (for this post, we choose Lambda).

  1. Choose Next: Permissions.

For this post, we create a policy that’s not too open.

  1. Choose Create policy.

  1. On the JSON tab, enter the following policy code, which for this post is named policy-rk-read-write (also provide the name of the bucket containing the translated files):
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::your-bucket"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::your-bucket/*"
            ]
        }
    ]
}

  1. On the Create role page, attach your new policy to the role.

  1. For Role name, enter a name (for this post, we name it translates3access2).
  2. Choose Create role.

So far everything you have done is a common workflow; now we make a change that allows Amazon Translate to have that trust relationship.

  1. On the IAM console, choose the role you just created.

  1. On the Trust relationships tab, choose Edit trust relationship.

  1. In the Service section, replace the service name with translate.

For example, the following screenshot shows the code with Service defined as lambda.amazonaws.com.

The following screenshot shows the updated code as translate.amazonaws.com.

  1. Choose Update Trust Policy.

Use a SageMaker notebook with Boto3

We can now run a Jupyter notebook on SageMaker. Every notebook instance has an execution role, which we use to grant permissions for Amazon Translate. If you’re performing a synchronous translation with a short text, all you need to do is provide TranslateFullAccess to this role. In production, you can narrow down the permissions with granular Amazon Translate access.

  1. On the SageMaker console, choose the notebook instance you created.
  2. In the Permissions and encryption section, choose the role.

  1. Choose Attach policies.

  1. Search for and choose TranslateFullAccess.

If you haven’t already configured this role to have access to Amazon S3, you can do so following the same steps.

You can also choose to give access to all S3 buckets or specific S3 buckets when you create a SageMaker notebook instance and create a new role.

For this post, we attach the AmazonS3FullAccess policy to the role.

Run an Amazon Translate synchronous call

You can now run a simple synchronous Amazon Translation job on your SageMaker notebook.

Run an Amazon Translate asynchronous call

If you try to run a batch translation job using Boto3 as in the following screenshot, you have a parameter called DataAccessRoleArn. This is the SageMaker execution role we identified earlier; we need to be able to pass this role to Amazon Translate, thereby allowing Amazon Translate to access data in the S3 bucket. We can configure this on the console, wherein the role is directly passed to Amazon Translate instead of through code run from a SageMaker notebook.

You first need to locate your role ARN.

  1. On the IAM console, choose the role you created (translates3access2).
  2. On the Summary page, copy the role ARN.

  1. Create a new policy (for this post, we call it IAMPassPolicyTranslate).
  2. Enter the following JSON code (provide your role ARN): 
    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TranslateAsyncPass",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::XXXXXXXXXX:role/translates3access2"
            
        }
    ]
}

  3. Choose Next.
  4. You can skip the tags section and choose Next
  5. Provide a name for the policy (for this post, we name it IAMPassPolicyTranslate).

This policy can now pass the translates3access2 role.

The next step is to attach this policy to the SageMaker execution role.

  1. Choose the execution role.
  2. Choose Attach policies.

  1. Attach the policy you just created (IAMPassPolicyTranslate).

You can now run the code in the SageMaker notebook instance.

Conclusion

You have seen how to run batch jobs using Amazon Translate in a SageMaker notebook. You can easily apply the same process to running the code using Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Compute Cloud (Amazon EC2), or other services. You can also as a next step combine services like Amazon Comprehend, Amazon Transcribe, or Amazon Kendra to automate managing, searching, and adding metadata to your documents or textual data.

For more information about Amazon Translate, see Amazon Translate resources.

About the Authors

Raj Kadiyala is an AI/ML Tech Business Development Manager in AWS WWPS Partner Organization. Raj has over 12 years of experience in Machine Learning and likes to spend his free time exploring machine learning for practical every day solutions and staying active in the great outdoors of Colorado.

 

 

Watson G. Srivathsan is the Sr. Product Manager for Amazon Translate, the AWS natural language processing service. On weekends you will find him exploring the outdoors in the Pacific Northwest.

Read More

Build reusable, serverless inference functions for your Amazon SageMaker models using AWS Lambda layers and containers

by Amazon AWS

In AWS, you can host a trained model multiple ways, such as via Amazon SageMaker deployment, deploying to an Amazon Elastic Compute Cloud (Amazon EC2) instance (running a Flask + NGINX, for example), AWS Fargate, Amazon Elastic Kubernetes Service (Amazon EKS), or AWS Lambda.

SageMaker provides convenient model hosting services for model deployment, and provides an HTTPS endpoint where your machine learning (ML) model is available to provide inferences. This lets you focus on your deployment options such as instance type, automatic scaling policies, model versions, inference pipelines, and other features that make deployment easy and effective for handling production workloads. The other deployment options we mentioned require additional heavy lifting, such as launching a cluster or an instance, maintaining Docker containers with the inference code, or even creating your own APIs to simplify operations.

This post shows you how to use AWS Lambda to host an ML model for inference and explores several options to build layers and containers, including manually packaging and uploading a layer, and using AWS CloudFormation, AWS Serverless Application Model (AWS SAM), and containers.

Using Lambda for ML inference is an excellent alternative for some use cases for the following reasons:

  • Lambda lets you run code without provisioning or managing servers.
  • You pay only for the compute time you consume—there is no charge when you’re not doing inference.
  • Lambda automatically scales by running code in response to each trigger (or in this case, an inference call from a client application for making a prediction using the trained model). Your code runs in parallel and processes each trigger individually, scaling with the size of the workload.
  • You can limit the number of concurrent calls to an account-level default of 1,000, or request an appropriate limit increase.
  • The inference code in this case is just the Lambda code, which you can edit directly on the Lambda console or using AWS Cloud9.
  • You can store the model in the Lambda package or container, or pulled down from Amazon Simple Storage Service (Amazon S3). The latter method introduces additional latency, but it’s very low for small models.
  • You can trigger Lambda via various services internally, or via Amazon API Gateway.

One limitation of this approach when using Lambda layers is that only small models can be accommodated (50 MB zipped layer size limit for Lambda), but with SageMaker Neo, you can potentially obtain a 10x reduction in the amount of memory required by the framework to run a model. The model and framework are compiled into a single executable that can be deployed in production to make fast, low-latency predictions. Additionally, the recently launched container image support allows you to use up to a 10 GB size container for Lambda tasks. Later in this post, we discuss how to overcome some of the limitations on size. Let’s get started by looking at Lambda layers first!

Inference using Lambda layers

A Lambda layer is a .zip archive that contains libraries, a custom runtime, or other dependencies. With layers, you can use libraries in your function without needing to include them in your deployment package.

Layers let you keep your deployment package small, which makes development easier. You can avoid errors that can occur when you install and package dependencies with your function code. For Node.js, Python, and Ruby functions, you can develop your function code on the Lambda console as long as you keep your deployment package under 3 MB. A function can use up to five layers at a time. The total unzipped size of the function and all layers can’t exceed the unzipped deployment package size limit of 250 MB. For more information, see Lambda quotas.

Building a common ML Lambda layer that can be used with multiple inference functions reduces effort and streamlines the process of deployment. In the next section, we describe how to build a layer for scikit-learn, a small yet powerful ML framework.

Build a scikit-learn ML layer

The purpose of this section is to explore the process of manually building a layer step by step. In production, you will likely use AWS SAM or another option such as AWS Cloud Development Kit (AWS CDK), AWS CloudFormation, or your own container build pipeline to do the same. After we go through these steps manually, you may be able to appreciate how some of the other tools like AWS SAM simplify and automate these steps.

To ensure that you have a smooth and reliable experience building a custom layer, we recommend that you log in to an EC2 instance running Amazon Linux to build this layer. For instructions, see Connect to your Linux instance.

When you’re are logged in to your EC2 instance, follow these steps to build a sklearn layer:

Step 1 – Upgrade pip and awscli

Enter the following code to upgrade pip and awscli:

pip install --upgrade pip
pip install awscli --upgrade

Step 2 – Install pipenv and create a new Python environment

Install pipnv and create a new Python environment with the following code:

pip install pipenv
pipenv --python 3.6

Step 3 – Install your ML framework

To install your preferred ML framework (for this post, sklearn), enter the following code:

pipenv install sklearn

Step 4 – Create a build folder with the installed package and dependencies

Create a build folder with the installed package and dependencies with the following code:

ls $VIRTUAL_ENV
PY_DIR='build/python/lib/python3.6/site-packages'
mkdir -p $PY_DIR
pipenv lock -r > requirements.txt
pip install -r requirements.txt -t $PY_DIR

Step 5 – Reduce the size of the deployment package

You reduce the size of the deployment package by stripping symbols from compiled binaries and removing data files required only for training:

cd build/
find . -name "*.so" | xargs strip
find . -name '*.dat' -delete
find . -name '*.npz' -delete

Step 6 – Add a model file to the layer

If applicable, add your model file (usually a pickle (.pkl) file, joblib file, or model.tar.gz file) to the build folder. As mentioned earlier, you can also pull your model down from Amazon S3 within the Lambda function before performing inference.

Step 7 – Use 7z or zip to compress the build folder

You have two options for compressing your folder. One option is the following code:

7z a -mm=Deflate -mfb=258 -mpass=15 -r ../sklearn_layer.zip *

Alternatively, enter the following:

7z a -tzip -mx=9 -mfb=258 -mpass=20 -r ../sklearn_layer.zip *

Step 8 – Push the newly created layer to Lambda

Push your new layer to Lambda with the following code:

cd ..
rm -r build/


aws lambda publish-layer-version --layer-name sklearn_layer --zip-file fileb://sklearn_layer.zip

Step 9 – Use the newly created layer for inference

To use your new layer for inference, complete the following steps:

  1. On the Lambda console, navigate to an existing function.
  2. In the Layers section, choose Add a layer.

  1. Select Select from list of runtime compatible layers.
  2. Choose the layer you just uploaded (the sklearn layer).

You can also provide the layer’s ARN.

  1. Choose Add.

Step 10 – Add inference code to load the model and make predictions

Within the Lambda function, add some code to import the sklearn library and perform inference. We provide two examples: one using a model stored in Amazon S3 and the pickle library, and another using a locally stored model and the joblib library.

1.	from sklearn.externals import joblib  
2.	import boto3  
3.	import json  
4.	import pickle  
5.	  
6.	s3_client = boto3.client("s3")  
7.	  
8.	def lambda_handler(event, context):  
9.	  
10.	     #Using Pickle + load model from s3  
11.	     filename = "pickled_model.pkl"  
12.	     s3_client.download_file('bucket-withmodels', filename, '/tmp/' + filename)  
13.	       loaded_model = pickle.load(open('/tmp/' + filename, 'rb'))  
14.	       result = loaded_model.predict(X_test)  
15.	  
16.	       # Using Joblib + load the model from local storage  
17.	       loaded_model = joblib.load(“filename.joblib”)  
18.	       result = loaded_model.score(X_test, Y_test)  
19.	       print(result)  
20.	       return {'statusCode': 200, 'body': json.dumps(result)}

Package the ML Lambda layer code as a shell script

Alternatively, you can run a shell script with only 10 lines of code to create your Lambda layer .zip file (without all the manual steps we described).

  1. Create a shell script (.sh file) with the following code:
createlayer.sh
#!/bin/bash

if [ "$1" != "" ] || [$# -gt 1]; then
	echo "Creating layer compatible with python version $1"
	docker run -v "$PWD":/var/task "lambci/lambda:build-python$1" /bin/sh -c "pip install -r requirements.txt -t python/lib/python$1/site-packages/; exit"
	zip -r layer.zip python > /dev/null
	rm -r python
	echo "Done creating layer!"
	ls -lah layer.zip

else
	echo "Enter python version as argument - ./createlayer.sh 3.6"
fi
  1. Name the file createlayer.sh and save it.

The script requires an argument for the Python version that you want to use for the layer; the script checks for this argument and requires the following:

  • If you’re using a local machine, EC2 instance, or a laptop, you need to install Docker. When using an SageMaker notebook instance terminal window or an AWS Cloud9 terminal, Docker is already installed.
  • You need a requirements.txt file that is in the same path as the createlayer.sh script that you created and has the packages that you need to install. For more information about creating this file, see https://pip.pypa.io/en/stable/user_guide/#requirements-files.

For this example, our requirements.txt file has a single line, and looks like the following:

scikit-learn
  1. Add any other packages you may need, with version numbers with one package name per line.
  2. Make sure that your createlayer.sh script is executable; on Linux or macOS terminal window, navigate to where you saved the createlayer.sh file and enter the following:
chmod +x createlayer.sh

Now you’re ready to create a layer.

  1. In the terminal, enter the following:
./createlayer.sh 3.6

This command pulls the container that matches the Lambda runtime (which ensures that your layer is compatible by default), creates the layer using packages specified in the requirements.txt file, and saves a layer.zip that you can upload to a Lambda function.

The following code shows example logs when running this script to create a Lambda-compatible sklearn layer:

./createlayer.sh 3.6
Creating layer compatible with python version 3.6
Unable to find image 'lambci/lambda:build-python3.6' locally
build-python3.6: Pulling from lambci/lambda
d7ca5f5e6604: Pull complete 
5e23dc432ea7: Pull complete 
fd755da454b3: Pull complete 
c81981d73e17: Pull complete 
Digest: sha256:059229f10b177349539cd14d4e148b45becf01070afbba8b3a8647a8bd57371e
Status: Downloaded newer image for lambci/lambda:build-python3.6
Collecting scikit-learn
  Downloading scikit_learn-0.22.1-cp36-cp36m-manylinux1_x86_64.whl (7.0 MB)
Collecting joblib>=0.11
  Downloading joblib-0.14.1-py2.py3-none-any.whl (294 kB)
Collecting scipy>=0.17.0
  Downloading scipy-1.4.1-cp36-cp36m-manylinux1_x86_64.whl (26.1 MB)
Collecting numpy>=1.11.0
  Downloading numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl (20.1 MB)
Installing collected packages: joblib, numpy, scipy, scikit-learn
Successfully installed joblib-0.14.1 numpy-1.18.1 scikit-learn-0.22.1 scipy-1.4.1
Done creating layer!
-rw-r--r--  1 user  ANTDomain Users    60M Feb 23 21:53 layer.zip

Managing ML Lambda layers using the AWS SAM CLI

AWS SAM is an open-source framework that you can use to build serverless applications on AWS, including Lambda functions, event sources, and other resources that work together to perform tasks. Because AWS SAM is an extension of AWS CloudFormation, you get the reliable deployment capabilities of AWS CloudFormation. In this post, we focus on how to use AWS SAM to build layers for your Python functions. For more information about getting started with AWS SAM, see the AWS SAM Developer Guide.

  1. Make sure you have the AWS SAM CLI installed by running the following code:
sam --version
SAM CLI, version 1.20.0
  1. Then, assume you have the following file structure:
./
├── my_layer
│   ├── makefile
│   └── requirements.txt
└── template.yml

Let’s look at files inside the my_layer folder individually:

  • template.yml – Defines the layer resource and compatible runtimes, and points to a makefile:
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
  MyLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      ContentUri: my_layer
      CompatibleRuntimes:
        - python3.8
    Metadata:
      BuildMethod: makefile
  • makefile – Defines build instructions (uses the requirements.txt file to install specific libraries in the layer):
build-MyLayer:
    mkdir -p "$(ARTIFACTS_DIR)/python"
    python -m pip install -r requirements.txt -t "$(ARTIFACTS_DIR)/python"
  • requirements.txt – Contains packages (with optional version numbers):
sklearn

You can also clone this example and modify as required. For more information, see https://github.com/aws-samples/aws-lambda-layer-create-script

  1. Run sam build:
sam build
Building layer 'MyLayer'
Running CustomMakeBuilder:CopySource
Running CustomMakeBuilder:MakeBuild
Current Artifacts Directory : /Users/path/to/samexample/.aws-sam/build/MyLayer

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided
  1. Run sam deploy –guided:
sam deploy --guided
Configuring SAM deploy
======================

	Looking for config file [samconfig.toml] :  Not found

	Setting default arguments for 'sam deploy'
	=========================================
	Stack Name [sam-app]: 
	AWS Region [us-east-1]: 
	#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
	Confirm changes before deploy [y/N]: y
	#SAM needs permission to be able to create roles to connect to the resources in your template
	Allow SAM CLI IAM role creation [Y/n]: y
	Save arguments to configuration file [Y/n]: y
	SAM configuration file [samconfig.toml]: 
	SAM configuration environment [default]: 

	Looking for resources needed for deployment: Not found.
	Creating the required resources...
	Successfully created!

		Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-18scin0trolbw
		A different default S3 bucket can be set in samconfig.toml

	Saved arguments to config file
	Running 'sam deploy' for future deployments will use the parameters saved above.
	The above parameters can be changed by modifying samconfig.toml
	Learn more about samconfig.toml syntax at 
	https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
Initiating deployment
=====================
Uploading to sam-app/1061dc436524b10ad192d1306d2ab001.template  366 / 366  (100.00%)

Waiting for changeset to be created..

CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------------------------------------------
Operation                          LogicalResourceId                  ResourceType                       Replacement                      
-----------------------------------------------------------------------------------------------------------------------------------------
+ Add                              MyLayer3fa5e96c85                  AWS::Lambda::LayerVersion          N/A                              
-----------------------------------------------------------------------------------------------------------------------------------------

Changeset created successfully. arn:aws:cloudformation:us-east-1:497456752804:changeSet/samcli-deploy1615226109/ec665854-7440-42b7-8a9c-4c604ff565cb


Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y

2021-03-08 12:55:49 - Waiting for stack create/update to complete

CloudFormation events from changeset
-----------------------------------------------------------------------------------------------------------------------------------------
ResourceStatus                     ResourceType                       LogicalResourceId                  ResourceStatusReason             
-----------------------------------------------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS                 AWS::Lambda::LayerVersion          MyLayer3fa5e96c85                  -                                
CREATE_COMPLETE                    AWS::Lambda::LayerVersion          MyLayer3fa5e96c85                  -                                
CREATE_IN_PROGRESS                 AWS::Lambda::LayerVersion          MyLayer3fa5e96c85                  Resource creation Initiated      
CREATE_COMPLETE                    AWS::CloudFormation::Stack         sam-app                            -                                
----------------------------------------------------------------------------------------------------------------------------------------- 

Now you can view these updates to your stack on the AWS CloudFormation console.

You can also view the created Lambda layer on the Lambda console.

 

Package the ML Lambda layer and Lambda function creation as CloudFormation templates

To automate and reuse already built layers, it’s useful to have a set of CloudFormation templates. In this section, we describe two templates that build several different ML Lambda layers and launch a Lambda function within a selected layer.

Build multiple ML layers for Lambda

When building and maintaining a standard set of layers, and when the preferred route is to work directly with AWS CloudFormation, this section may be interesting to you. We present two stacks to do the following:

  • Build all layers specified in the yaml file using AWS CodeBuild
  • Create a new Lambda function with an appropriate layer attached

Typically, you run the first stack infrequently and run the second stack whenever you need to create a new Lambda function with a layer attached.

Make sure you either use Serverless-ML-1 (default name) in Step 1, or change the stack name to be used from Step 1 within the CloudFormation stack in Step 2.

Step 1 – Launch the first stack

To launch the first CloudFormation stack, choose Launch Stack:

The following diagram shows the architecture of the resources that the stack builds. We can see that multiple layers (MXNet, GluonNLP, GuonCV, Pillow, SciPy and SkLearn) are being built and created as versions. In general, you would use only one of these layers in your ML inference function. If you have a layer that uses multiple libraries, consider building a single layer that contains all the libraries you need.

Step 2 – Create a Lambda function with an existing layer

Every time you want to set up a Lambda function with the appropriate ML layer attached, you can launch the following CloudFormation stack:

The following diagram shows the new resources that the stack builds.

Inference using containers on Lambda

When dealing with the limitations introduced while using layers, such as size limitations, and when you’re invested in container-based tooling, it may be useful to use containers for building Lambda functions. Lambda functions built as container images can be as large as 10 GB, and can comfortably fit most, if not all, popular ML frameworks. Lambda functions deployed as container images benefit from the same operational simplicity, automatic scaling, high availability, and native integrations with many services. For ML frameworks to work with Lambda, these images must implement the Lambda runtime API. However, it’s still important to keep your inference container size small, so that overall latency is minimized; using large ML frameworks such as PyTorch and TensorFlow may result in larger container sizes and higher overall latencies. To make it easier to build your own base images, the Lambda team released Lambda runtime interface clients, which we use to create a sample TensorFlow container for inference. You can also follow these steps using the accompanying notebook.

Step 1 – Train your model using TensorFlow

Train your model with the following code:

model.fit(train_set,
                    steps_per_epoch=int(0.75 * dataset_size / batch_size),
                    validation_data=valid_set,
                    validation_steps=int(0.15 * dataset_size / batch_size),
                    epochs=5)

Step 2 – Save your model as a H5 file

Save the model as an H5 file with the following code:

model.save('model/1/model.h5') #saving the model

Step 3 – Build and push the Dockerfile to Amazon ECR

We start with a base TensorFlow image, enter the inference code and model file, and add the runtime interface client and emulator:

FROM tensorflow/tensorflow

ARG FUNCTION_DIR="/function"
# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}
COPY app/* ${FUNCTION_DIR}

# Copy our model folder to the container
COPY model/1 /opt/ml/model/1

RUN pip3 install --target ${FUNCTION_DIR} awslambdaric

ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/bin/aws-lambda-rie
RUN chmod 755 /usr/bin/aws-lambda-rie
COPY entry.sh /
ENTRYPOINT [ "/entry.sh" ]
CMD [ "app.handler" ]

You can use the script included in the notebook to build and push the container to Amazon Elastic Container Registry (Amazon ECR). For this post, we add the model directly to the container. For production use cases, consider downloading the latest model you want to use from Amazon S3, from within the handler function.

Step 4 – Create a function using the container on Lambda

To create a new Lambda function using the container, complete the following steps:

  1. On the Lambda console, choose Functions.
  2. Choose Create function.
  3. Select Container image.
  4. For Function name, enter a name for your function.
  5. For Container image URI, enter the URI of your container image.

Step 5 – Create a test event and test your Lambda function

On the Test tab of the function, choose Invoke to create a test event and test your function. Use the sample payload provided in the notebook.

Conclusion

In this post, we showed how to use Lambda layers and containers to load an ML framework like scikit-learn and TensorFlow for inference. You can use the same procedure to create functions for other frameworks like PyTorch and MXNet. Larger frameworks like TensorFlow and PyTorch may not fit into the current size limit for a Lambda deployment package, so it’s beneficial to use the newly launched container options for Lambda. Another workaround is to use a model format exchange framework like ONNX to convert your model to another format before using it in a layer or in a deployment package.

Now that you know how to create an ML Lambda layer and container, you can, for example, build a serverless model exchange function using ONNX in a layer. Also consider using the Amazon SageMaker Neo runtime, treelite, or similar light versions of ML runtimes to place in your Lambda layer. Consider using a framework like SageMaker Neo to help compress your models for use with specific instance types with a dedicated runtime (called deep learning runtime or DLR).

Cost is also an important consideration when deciding what option to use (layers or containers), and this is related to the overall latency. For example, the cost of running inferences at 1 TPS for an entire month on Lambda at an average latency per inference of 50 milliseconds is about $7 [(0.0000000500*50 + 0.20/1e6) *60*60*24*30* TPS ~ $7)]. Latency depends on various factors, such as function configuration (memory, vCPUs, layers, containers used), model size, framework size, input size, additional pre- and postprocessing, and more. To save on costs and have an end-to-end ML training, tuning, monitoring and deployment solution, check out other SageMaker features, including multi-model endpoints to host and dynamically load and unload multiple models within a single endpoint.

Additionally, consider disabling the model cache in multi-model endpoints on Amazon SageMaker when you have a large number of models that are called infrequently—this allows for a higher TPS than the default mode. For a fully managed set of APIs around model deployment, see Deploy a Model in Amazon SageMaker.

Finally, the ability to work with and load larger models and frameworks from Amazon Elastic File System (Amazon EFS) volumes attached to your Lambda function can help certain use cases. For more information, see Using Amazon EFS for AWS Lambda in your serverless applications.

About the Authors

Shreyas Subramanian is a AI/ML specialist Solutions Architect, and helps customers by using Machine Learning to solve their business challenges on the AWS platform.

 

 

 

Andrea Morandi is an AI/ML specialist solutions architect in the Strategic Specialist team. He helps customers to deliver and optimize ML applications on AWS. Andrea holds a Ph.D. in Astrophysics from the University of Bologna (Italy), he lives with his wife in the Bay area, and in his free time he likes hiking.

Read More