Oracle Digital Assistant Techexchange

Oracle Digital Assistant TechExchange

Article.

How to Build a Webhook to Connect Your Oracle Digital Assistant Chatbot to Google Home Michael Wootton, June 2019

Let's assume you built a chatbot using Oracle Digital Assistant and you've exposed it on various messengers and Alexa too. A next step then could be to connect your chatbot to Google Home.

I die research this topic quite a lot but couldn't find anything complete describing how to do this using Google Actions on Google API version 2. So, what has started as a research, eventually has become an opportunity for me to write a comprehensive guide to integrate Oracle Digital Assistant with Google Home.

Following this article, you learn how to build a webhook, how to create an Action on Google, how to create a channel in Oracle Digital Assistant and how to use Goggle sign-in (account linking) to access user information from Google.

Note that because I live in Brazil, a requirement was to build chatbots in more than one language (English, Spanish and Portuguese). Therefore I built the webhook to be able to call two different channels, each calling a different language optimized skill.

Oracle Digital Assistant – TechExchange

BACKGROUND ...... 3 1. CREATE GOOGLE HOME – ACTIONS ON GOOGLE ...... 3 1.1. CREATE ACTIONS ON GOOGLE ACCOUNT ...... 3 1.2. CREATE ACTIONS FOR YOUR GOOGLE PROJECT ...... 4 1.3. Project Details – Setup ...... 6 1.4. DIALOGFLOW ...... 8 1.4.1. Update “Default Welcome Intent” ...... 8 1.4.2. Update “Default Fallback Intent” ...... 10 1.4.3. Create a “SIGN_IN” Intent ...... 12 1.4.4. Create “Cancel” Intent ...... 13 1.4.5. Define Fulfillment ...... 15 1.4.6. Execute Integrations ...... 16 1.4.7. Adding More Languages ...... 19 1.5. BACK TO ACTIONS ON GOOGLE - DIALOGFLOW ...... 21 1.5.1. Update Invocation ...... 21 1.5.2. Setup Account Linking ...... 21 2. CREATE YOUR CHANNELS IN ODA ...... 22 2.1. CREATE NEW CHANNELS IN ODA ...... 22 3. UNDERSTANDING THE WEBHOOK CODE...... 23 3.1. EXPLAINING BY LINE RANGES: ...... 23 3.1.1. Line 10 ...... 23 3.1.2. Line 21 to 58 ...... 23 3.1.3. Line 65 to 77 ...... 24 3.1.4. Line 78 to 116 ...... 25 3.1.5. Line 118 to 210 ...... 28 3.1.6. Line 255 to 327 ...... 31 3.1.7. Line 330 to 414 ...... 31 3.1.8. Line 417 to 419 ...... 32 3.1.9. If you only have on Chatbot ...... 32 4. CONCLUSION...... 32

Oracle Digital Assistant - TechExchange

Background Google Home supports multiple languages, including English, Spanish and Portuguese. Therefore, Google Home was my option of choice for building a chatbot that presents information from my SaaS (EPM EPBCS) in Portuguese. However, I also needed support for Spanish (for LAD) and English. For this, I needed to create a webhook that could connect to two chatbots I built using Oracle Digital Assistant. Note: I created two chatbots, one for Spanish and another to handle Portuguese and English. It was my first experience in building chatbots and today I know that I could have used a single chatbot supporting multiple languages using resource bundles for Portuguese, Spanish and English. However, the approach I took is not wrong at all, just another implementation of multi-language support. In this tutorial I am going to show you how you create a webhook that works with Google Home and that uses account linking to get user information (Google user id, email, user name) form the user's Google accounts. Note: If you simply connect to Google Home, the user Id you get is a generated random identifier. So each user – bot session uses its own user Id, making it difficult to save user information for later use. In my case I created a chatbot that can connect to any EPBCS Instance so at first connection I ask for the EPM server it will connect and also for the user name for future use if the chatbot is approached by the same user. For this I needed a user Id that does not change.

Below are the individual steps covered in this article 1. Create Google Home – Actions on Google 2. Create your Channels in Oracle Digital Assistant 3. Understanding the webhook Code

1. Create Google Home – Actions on Google

You use Actions on Google to create skills, intents and the webhook connection. Actions can use the following options to connect to external services: Actions SDK and Dialogflow. For this article I decided to Dialogflow. 1.1. Create Actions on Google Account • You must have a Gmail account for creating Actions on Google. • Navigate to https://developers.google.com/actions/ • Click on the “Sign In" link on the top right and authenticate with your Google credentials

Oracle Digital Assistant - TechExchange

• After successful login, click the GO TO GOOGLE ACTIONS CONSOLE link on the top right (as shown in the image below).

1.2. Create Actions for Your Google Project • In the Console page, click on + Add/Import Project to create your new Actions on Google Skill

Oracle Digital Assistant - TechExchange

• In the pop-up window titled “New Project”: • Provide a project name, like planning in my case. • Choose a language for the Skill. Note that you can add other languages later. • Choose a country or region. I, for example, chose Brazil. • Click on Create Project (as shown in the image below)

• Next, you will see a Welcome to your New Project Screen. Use this screen to choose the Google Actions template to build with. • I chose: "Conversational – Build a customized conversational experience”. This option uses the Dialogflow option.

Oracle Digital Assistant - TechExchange

• Choosing a template creates the skill and navigates you to the Actions on Google page for your Action ("planning" in my example). The name of the Action is displayed on the top right of the page next to the “Actions on Google” title.

1.3. Project Details – Setup

In this section you will: • Set the Google Home invocation name for your Actions • Define the Intents (Actions) to use • Select functionality used by an Action • Google sign-in (Account Linking) • Use of Webhook • Phrases to associate with your Actions • Test your Action on Google in the different languages • Configure the Webhook endpoint

Oracle Digital Assistant - TechExchange

Note: The steps listed below must be done for each language you need to support. 1.3.1. Invocation

• In this screen you choose how your Action is called when talking to Google. Commonly, you choose a name that consists of 2 words. • You also choose the voice to be used (Male, Female). For ease of use and demonstration, you may want to use the opposite voice of what you have selected to be the default for your Google Home. • Ensure you press SAVE on the top of the page after changing options. 1.3.2. Actions • You will have no Actions created so click on the ADD YOUR FIRST ACTION option

Oracle Digital Assistant - TechExchange

• On the Create Actions Screen presented click on “Custom Intent” already highlighted and Click Build.

1.4. DialogFlow • Pressing the "Build" button in the previous section, lead you to the dialog flow page, opened in a separate tab. Before the page opened you needed to grant permission to various areas of your Google account.

• Choose the Default Language again, will normally bring “English-en” as default, but choose your language, as selected in the creation of the ACTION • Also choose your Default Time Zone, so that your actions and request use this time zone. • Notice the GOOGLE PROJECT presented just below, it is showing you that this DIALOGFLOW is linked to your Google Actions Project, in this case it shows you the CODE (my example is: planning-c0e24). • Then click CREATE in the DIALOGFLOW screen

1.4.1. Update “Default Welcome Intent” • Once the CREATE is done it will present you a new screen with TWO Intents already created:

Oracle Digital Assistant - TechExchange

➢ Default Fallback Intent ➢ Default Welcome Intent • First, we will update the “Default Welcome Intent” • This Intent is used if you call your Action on Google without any command. Example: If you say in English (if you had this language already set) “OK Google talk to planning” • In my case I wanted that this kind of Interaction isn't sent to the WEBHOOK and that those are handled by Google instead • Configure the two intents such that they don't require a call to an external service and instead get served from a default configuration. Click on the DEFAULT WELCOME INTENT

• Note that it already brings in the Section EVENTS the event called “Welcome”, so leave it this way • Also, in the Training Phrases, there can be already some phrases in it, in the language you are working on, don´t worry about it, you can leave as they are, they won´t be used. • In the Action and Parameters section it will bring already – “input.welcome”. Also leave it as it is.

Oracle Digital Assistant - TechExchange

• In Responses section – Default tab - it can bring already some responses. These responses are chosen randomly each time the Intent is called. In my case, I chose to leave just one Answer, so I could easily know it was my Welcome answer and that everything was right. You can leave them as they are, or delete all except the one you want to be there or delete all of them and add the one you want. There must be at least one ANSWER, choose accordingly to the language you are updating. • In Responses section – Google Assistant tab. Select the option: “Use response from the DEFAULT tab as the first response”, so that when using Google Assistant, it uses the same responses that you set in the Default Tab. • In Fulfillment Section – Leave things as they are, which means that no webhook will be called and the DIALOGFLOW will answer the Default Welcome Intent from configured responses. • Click Save. • Click on the Intents Selection on the LEFT tabs to show the Intents again.

1.4.2. Update “Default Fallback Intent” • The second Intent to update is the “DEFAULT FALLBACK INTENT” • This Intent is called whenever no other intent handles the user message. • It will be called for example if we first call the Actions on Google with a command or other word: ➢ If you say in English (if you had this language already set) - ➢ “OK Google talk to planning Consult Sales” Google on Actions will try to find an Intent that has “Consult Sales” or “Consult” or “Sales” as Training Phrases (here we are talking about Google on Actions and not your chatbot) and as it doesn´t find any, it will call the “Default Fallback Intent” and consider “Consult Sales” as the text being received. If the responds to a message sent from the chatbot, then the user message should be sent to the webhook, and then the chatbot. • Click on the “Default Fallback Intent”

Oracle Digital Assistant - TechExchange

• Note that the EVENTS section (not shown in the image above) is empty. • Also, Training Phrases are empty. • In the Action and Parameters section it will bring already – “input.unknown”. Also leave it as it is. • In Responses section – Default tab, it already has some responses defined. These responses are here because the default behavior of an Intent is to answer to the user without sending anything to the webhook. And because the Default Fallback Intent is a Built-in Intent, it already has some responses defined. • In our case however, we don´t want to use the defined responses because we will send the incoming content to the chatbot using a webhook. Therefore, you can delete all pre-built responses and leaving the Default tab empty. • In Responses section – Google Assistant tab. The option: “Use response from the DEFAULT tab as the first response”, will be NOT SELECTED, leave as it is, because the chatbot is expected to respond to it. • In Fulfillment Section – Check the option: Enable webhook call for this Intent. Here we are telling this Intent to: a. Send whatever comes as response to our webhook b. Our webhook will process it: • understand the layout of the Google Incoming Message and Intent • Reshape the message to the CHATBOT layout • Send the formatted message to the Chatbot c. The Chatbot will receive the message and process it, as a new message or as a following answer to a previous message sent to the Google Home Assistant

Oracle Digital Assistant - TechExchange

d. Send the answer to the WEBHOOK e. And the WEBHOOK will: • Receive the message from the CHATBOT • Reformat to the Google Assistant (Actions on Google – DIALOGFLOW) format • Send back the response to Google, this can be a message awaiting a response from the user or an END CONVERSATION message to Google. • Click Save. • Click on the Intents Selection on the LEFT tabs to show the Intents again. Now we need to create some more Intents as follows.

1.4.3. Create a “SIGN_IN” Intent • First, we are going to create a new Intent called SIGN_IN • This Intent is used when we ask the user to LINK ACCOUNT or give us permission to get his Google Account Information. This Intent is triggered by a setup we will do later in the process. When the LINK ACCOUNT process is started and after it ends it sends the “SIGN_IN” Intent from the “link Account” to the WEBHOOK and we can treat it to see if the user answered YES to let us get his information or said NO to this permission. • By my experience, this intent is only called if you use a command to call the LINK ACCOUNT process (with the command “conv.ask(new SignIn(‘text to user’)”). • In our case, for this Intent, yes, we want to send this information to our webhook, so it can process the Intent, checking if he answered YES or NO to it and send him a message accordingly, so I: • Click on the “CREATE INTENT” button at the Top Middle.

• In the Intent name put “SIGN_IN”

Oracle Digital Assistant - TechExchange

• In the Section EVENTS, expand the section and click “ADD EVENT”, then in the add event box type “sign”, then you will see an option of “Google Assistant Sign in”, choose this option. • Also, in the Training Phrases, expand the section and click “ADD TRAINING PHRASES”, then in the “add user expression” box type “sign in”, then press enter to have the phrase added. • In the Action and Parameters section it will be empty. Leave it as it is. • In Responses section – Default tab. it will be empty. Leave it as it is. • In Fulfillment Section – Check the option: Enable webhook call for this Intent. • Click Save. • Click on the Intents Selection on the LEFT tabs to show the Intents again. Now, we are going to create a new Intent called Cancel

1.4.4. Create “Cancel” Intent

• This Intent is used when we want to end the conversation. As you may know if the Chatbot is expecting a response in the middle of your conversation and you don´t answer, and later that day or even another day you restart the conversation with a new call to the Chatbot, the chatbot will consider (if you are getting the UserId from the google account and not a random userId created when you don´t have the users google information) that this is the continuation of the last unfinished conversation, and will respond this way. Some integrations like the one to Messenger seem to have a time out implemented, in the webhook so that if you come back to the conversation after some time it considers a new conversation. I don´t know how to implement this in a simple way, so I created this intent to be able to interpret a cancel command (this will be a custom Intent, Google Home doesn´t have a default Cancel Intent) and send to the webhook a ‘cancel’ text.

Oracle Digital Assistant - TechExchange

On the other hand, in my chatbot code, I expect always a ‘cancel’ response and redirect in the Dialog workflow in ODA to a Goodbye routine that ends the conversation (with a RETURN DONE command) and sends back to the user the goodbye information and my webhook sends this message to the Google Assistant with a “conv,close” statement that also tells to Google Assistant that this conversation is over. So, I had to implement this way because not only a “conv.close” from the webhook to Google Assistant is needed, but also that the Chatbot considers the conversation ended, and returns with a return statement. Here you can read how Google Assistant treats end of conversations: https://developers.google.com/actions/assistant/conversation-exits • In our, case for this Intent, yes, we want to send this information to our webhook, so it can process the Intent, send it to the Chatbot so it is processes accordingly, so I: • Click on the “CREATE INTENT” button at the Top Middle.

• In the Intent name put “Cancel” • In the Section EVENTS, expand the section and click “ADD EVENT”, then in the add event box type “actions_intent_CANCEL”, this will work as a custom Intent. • Also, in the Training Phrases, expand the section and click “ADD TRAINING PHRASES”, then in the “add user expression” box type “Cancel”, then press enter to have the phrase added. You can add more training phrases so that the Intent is selected when used. Remember you will do this in all the languages you will use so in each language use the correct word for that language. • In the Action and Parameters section it will be empty. Leave it as it is. • In Responses section – Default tab. it will be empty. Leave it as it is. • In the Responses Section – Check on the option Set this Intent as end of conversation.

Oracle Digital Assistant - TechExchange

• In Fulfillment Section – Check the option: Enable webhook call for this Intent. • Click Save. • Click on the Intents Selection on the LEFT tabs to show the Intents again.

1.4.5. Define Fulfillment • Now, we must set the Endpoints to be used by the Actions on Google for our planning Skill. • On the left tab, Click on the “Fulfillment’ button • Check the WEBHOOK Option to Enable it. • Then you will see some new fields to be filled, URL BASIC AUTH.

• In the URL you will put the webhook URL, in my case I decided to host my webhook in HEROKU, here you have many options where to host your webhook. Use your URL and append it with the endpoint that your webhook will respond to messages sent from Google Assistant (in my case this is “/fulfillment”) So, my URL looks like this:

URL* https://xxxxxxxxxxx.herokuapp.com/fulfillment

Oracle Digital Assistant - TechExchange

Where: https://xxxxxxxxxxx.herokuapp.com is your URL to access your webhook and /fulfillment is the endpoint that will respond to the Google Assistant messages. • All the other fields are not needed, so leave them in Blank • Click SAVE

1.4.6. Execute Integrations • The last point in Dialogflow is to send to Actions on Google all the things we setup here, remember that DIALOGLOW and Actions on Google are separate tools, but they have Integration and you update Actions on Google using the INTEGRATIONS option from Dialogflow Every time you update something in Dialogflow you must run again the Integration. 1.4.6.1. Dialogflow and Actions on Google pairing Dialogflow knows that this specific dialoglow is integrated to you Actions on Google (planning) skill because you came to it from Actions on Google project and this way it created the integration between the two, you can check this clicking on the gear icon just beside the planning name under the logo Dialogflow, it will show you your Dialogflow information. Here you can see in the GENERAL tab:

Project ID – your Actions on Google skill code (planning-c0e24 in my case). API Version: V2 API – The version of Actions on Google that you are using, don´t change this. 1.4.6.2. Integration Setup and Execution

Oracle Digital Assistant - TechExchange

• On the left tab, Click on the “Integrations’ button • Under Google Assistant click on INTEGRATION SETTINGS

• You will see a new pop-up screen that brings an Explicit Invocation already set, the Default Welcome Intent, we must tell here that we want to integrate also the other Intents we created so, under Implicit Invocation section, click in the Add Intent Box:

a. It will show you the other Intents you created, choose “SIGN_IN” and repeat the operation and choose also “Cancel”

Oracle Digital Assistant - TechExchange

b. With this you will now have the three Intents appearing in the screen.

c. Notice that on the side of each Intent we have a check box with “Sign In required”, this option is to tell Actions on Google in which Intent he will trigger the Account Linking functionality and ask the user to link accounts and you then be able to get his userId and extra information from his google account. In some documents they consider only doing this after you are certain that the user will interact with your Chatbot, and not use it and the Welcome Intent (where the user only called the chatbot but didn’t send any information to it yet), see this link to a better explanation: https://developers.google.com/actions/identity/google-sign-in In our case as my Chatbot is only for demo purposes I am using “Google Sign In” account linking, but you may need something stronger and maybe related to user being authenticated in your ID system, I have seen many other papers around explaining how to do this, but it´s not my case, so I went on with Google Sign In. d. Click on “Sign in Required” for the Default Welcome Intent.

➢ If it is “required” as we are setting it, the user MUST Accept the Account Linking to be able to go on with the chatbot, if he refuses he will receive a message and the conversation will END.

Oracle Digital Assistant - TechExchange

➢ Another way of doing this, is to NOT Check any “Sign in Required” and call the” Sign In” process by yourself, in this case the user is not obligated to accept the account linking, and you can set the userId, or name or email as you desire for an anonymous user. Although I said anonymous, Google home even without account linking is consistent in setting a userId, as we can read at the end of this document: https://developers.google.com/actions/identity/user-info In the code I am providing you can set as an option the “Default Welcome Intent” in Dialogflow as a Webhook Intent, this will send the welcome Intent to your webhook and you can call the command (conv.ask (new SignIn(‘text’)) ) to invoke the intent helper that will invoke the account linking process, I did this call on the webhook code, in the “Default Welcome Intent” part, that is only called if you set the Dialogflow for this Intent to call the webhook, Also the SIGN_IN Intent is called just after the account linking is completed (with YES or No from the user) so that you can give him a specific message in either case. (I will explain the code of the webhook later) e. Click on the TEST button, this will integrate and take the Intents to Actions on Google and if everything is ok will also open a new window taking you to the TEST part of Actions on Google.

f. Also, if you already created more languages, then after clicking here it will also create the languages in Google Actions. On the first time you Integrate, it will show you a screen telling that it will integrate with your Actions on Google. Also let the Auto-preview changes checked so further changes will let you integrate again. Click Continue.

1.4.7. Adding More Languages • If you want to create more languages that your Chatbot and webhook can accept, now it’s time to do it. • Go back to the DialogFlow screen, should be opened yet

Oracle Digital Assistant - TechExchange

• Click on the gear icon just at the side of you Dialogflow application name

• Click on the Languages Tab just at the side of the General tab you should see. • You should have already a language here, and a DEFAULT at the right side of it, saying that this is your default language. • One line bellow there is space to choose more languages, I chose Spanish-es as an example.

• After you finished choosing the languages you want, click on SAVE on top of the page. • You will see now that beside the language name on the left bar menu, you now have all the languages you created.

• All the Intents created for your first language already are there for the other languages, but you must update some things, like: • Training phrases, Responses (deleting or creating new items there. • After everything is set, do the previous Step ”o” again, to send this new settings to Actions on Google, if in this case, if any of your languages are not ok the Actions on Google new screen won´t open and a message will appear in Dialogflow, this means that something is missing.(usually SIGN_IN and Cancel Intents (as they are not default Intents) needs new training phrases).

Oracle Digital Assistant - TechExchange

1.5. Back to Actions on Google - DialogFlow

1.5.1. Update Invocation • When Actions on Google Opens the new Window, you will have to update the Invocation name and male or female voice used, for the new languages.

1.5.2. Setup Account Linking • Well now that we are back to Actions on Google and set the SIGN_IN Intent to be used on the Default Welcome Intent, we need to turn on the Account Linking functionality in Actions on Google. For this we go to the new Actions on Google Screen and click on the left panel on “Account Linking”

Oracle Digital Assistant - TechExchange

• On the Add Account Linking page, click on “Yes, allow users to sign up for new account via voice” and click NEXT • On Linking Type choose: Google Sign In and click NEXT • The system will present you with a Client ID issued by Google to your Actions, keep this information because you will need it to put in your webhook app. • Click SAVE

2. Create your Channels in ODA Go to ODA and open the Channels definitions. 2.1. Create new Channels in ODA 2.1.1. Give it a name, I called mine “GoogleHome_PT” 2.1.2. Description as you desire 2.1.3. Type – Webhook 2.1.4. Platform Version – 1.1 Conversation Model 2.1.5. Outgoing URL – in my case as I had the webhook in HEROKU I put https://xxxxxxxxxx.herokuapp.com plus the endpoint used by my webhook to receive the CHATBOT Messages – “/bot/message/pt” where “pt” is to indicate to my webhook that it is receiving form the CHATBOT with the Portuguese version. So the final URL is https://xxxxxxxxxx.herokuapp.com/bot/message/pt When I created the Channel for my Spanish Version I ended this URL with “es” and in my webhook I tested the end of the endpoint to be able to identify from which Channel and Chatbot version I received the message.

Oracle Digital Assistant - TechExchange

SO the final version for Spanish Channel was : https://xxxxxxxxxx.herokuapp.com/bot/message/es 2.1.6. After saving you will get the Webhook URL and the secret code for this channel or more than one if you created for other language versions. We will use this webhook urls and secret codes in the webhook code, later. 2.1.7. Choose the Skill (Chatbot) that will correspond to your channel.

3. Understanding the Webhook Code. • The code provided for the service.js uses some functions that I didn´t know myself before this (It was my first experience programming in Java, that’s why I will also apologize for any mistakes or not optimal usage, or even redundancy of code in some parts). • I had special difficulty in understanding the Pubsub function, when to use resolve(), and many other things, hhahahahaha. I had a lot of help from “Mireille Duroussaud” and “Matt Vander Vliet” in this. • The code uses Oracle´s “@Oracle/bots-node-sdk” although I copied and pasted some of the code to be able to present messages in more than one language to the user. • My webhook mainly treats text and actions coming from the Chatbot to format to the Google Home text format. These where issued mainly by the “System.CommonResponse”, “System.List” and “System.Text” components of the Skill. So, if you have more complex answers you may have to work a little more on the conversion to text and other cards or options. • Something to understand also is that Actions on Google version 2 API is very different from version 1, and that DialogFlow treats message formats from Google Home in a different format that Actions SDK does. Also, Google Assistant message process is Synchronous and the calls to ODA are not, so in this pubsub helped a lot, even more when you have a promise in one place of the code and when the Chatbot answers back the response automatically goes to another point (app.post) • So, let’s start with points I think should me explained. 3.1. Explaining by Line ranges:

3.1.1. Line 10 const assistant = dialogflow({debug: true, clientId:'4819215140- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.apps.googleusercontent.com',});

Here is where we use the “Client Id” you got when setting the Account Linking in Actions on Google (Step “1.4.2”, Setup Account Linking - AoG), internally is telling that the check of Account Linking must be done against this ClientId (your Actions on Google app)

3.1.2. Line 21 to 58

Oracle Digital Assistant - TechExchange

Setting the webhook Client and getting the url and secret from the channel that sent the message from ODA to the webhook depending on the params, that in this case are the two final letters of the endpoint (giving me the locale information)

In this snippet in the URL part, I put the beginning as the NGROK address for my environment plus the URL for the channels, the localhost:8800 ngrok address.

3.1.3. Line 65 to 77

Here I set the app.post to receive the message form the CHATBOT and set that after “/bot/message/” is the params received, that I want to have as a “locale” variable. The “/bot/message/” and locale is indicated in the Output URL of the Channel

Oracle Digital Assistant - TechExchange

Also, I publish with pubsub (line 75), what means that from here the code will go to the function pointed in the pubsub Subscribe. The subscribe happened just before the SEND command to the CHATBOT (using @oracle-bots-node-sdk) on line 194 on “Default Fallback Intent” and on line 308 on the “Cancel Intent”. In this “Pubsub.Subscribe” I use the “userId” as the key for the subscription and point to the callback function when anything is published to this user (in this case is the “userId”). PubSub.subscribe(userId, treatandsendtoGoogle) in line 192 and line 316. So, after line 75 the next command executed is on the “treatandsendtoGoogle” variable function, that is where I receive the message, format to Google format and sent to the Assistant. This is one point where I Duplicated the code of this function as I defined the variable in the “Default Fallback Intent” and “Cancel Intent”, the only difference in both codes, is that in: • Default Fallback Intent line: 183 to 188, I use a “conv.ask” (that doesn´t end the conversation) or “conv.close” (that sends the message and closes the conversation with the Assistant). • Cancel intent line: 310 I use “conv.close” (that sends the message and closes the conversation with the Assistant)

3.1.4. Line 78 to 116

Oracle Digital Assistant - TechExchange

In this Section is where we treat the Intent Type of “Default Welcome Intent”, this part of the code is only used if you had previously set:

• Dialogflow Intent “Default Welcome Intent” as using the Webhook

Also, if using the webhook for this Intent, important in this case to not set the “Sign In Required”, leave it unchecked for all Intents when doing the “Integrations” part of setup in Dialogflow.

When there is no “Sign In Required” you can ask for the “SIGN IN” or Account linking at any time best suits for you, in my case I set it to the Default Welcome Intent, but in this case the user is not forced to accept to go on with the Chatbot use.

Let me explain a little better how it works:

1. When you set “Sign In Required” for let’s say “Default Welcome Intent” and don´t use the webhook:

a. This Intent is called when the users activates the Google Actions without any other phrase………like? “OK GOOGLE talk to planning” b. As soon as Dialogflow receives the message from the Google Assistant it already starts the Account Linking process. During this process, no other Intent is called c. If you answer “No” (in the language your user is using) to the questions about Account Linking (or creating and Account with them). The conversation is ended, and you don´t go further on the Chatbot conversation. • The next time you try to use the Skill you receive another try for the Account Linking d. If you answer “Yes” to both questions then the Default Welcome Intent gets your account data and goes on answering the Intent with the predefined Responses, if there are more than one, then it randomly chooses one.

Oracle Digital Assistant - TechExchange

From here on it uses the other Intents defined till the end of the conversation and beginning of Another.

As the Account Linking is done, the Next time you begin the Chatbot no question about Account Linking is made and your information is passed to the Webhook and so on.

This situation only changes if you UNLINK your account to the Action on Google Skill, this can be done on your HOME app on the cellphone, or in the Actions on Google Console where we can TEST the Skill.

2. When you don´t set “Sign in Required” for any Intent. a. Then you can ask for an ACCOUNT LINKING in any Intent you decide to, by using the same kind of code that I used for the “Default Welcome Intent”. • Here we need one point of Attention: After you execute the command like on line 90, 96 and 102:

conv.ask(new SignIn('To get you Google account details, like name and email, answer YES'));

You can´t go on with any other command that maybe sends message to the CHATBOT via the Channel, because it should end the processing after this command and not go on doing more things or asking your Chatbot to treat what was said before you initiated the Account Linking with this command.,

This is because Google Assistant will process the Account Linking request and Ignore what your Chatbot would answer and you may get a problem with your conversation flow.

So, it’s better to treat only the Account Linking and after it ends, with or without the users permission, give back the control to the user so he can go on and start form there.

b. Let’s explain this process.

I first check if “conv.user.profile.payload” is undefined, because this payload only exists if the user granted permission to get his data. And as every default welcome Intent message comes to this Intent, he may just be beginning the conversation but can already have granted permission and I have already what I need and just respond with “Hi” in each locale and wait for the next message. Locale is also part of the conversation information.

1. If it is defined, then the user already granted access to his user data and I can go on and user his information and correct userId.

2. If it is undefined means this user is entering the Skill for the first time, or in the last time he didn’t grant access to the Account Linking.

I also ask if “conv.user.storage.userId” is undefined The “conv,user,storage” is an area you have in the conversation to save whatever information you want and get it later in the same conversation, while you go on with the conversation flow. So, if during a conversation the user doesn´t grant me access to his information I set the “userId” in this area with the random user I will generate for him for this conversation.

➢ If it is defined, I don´t ask for Account Linking again and go on answering the “Hi” in each locale.

➢ If it is undefined means it is the first time in this conversation and in this case I: o Will ask again for the Account Linking Process to Initiate.

Oracle Digital Assistant - TechExchange

o I also check the locale to give a personal message before the default message the Account Linking sends. o When the Account Linking ends, with Yes or No Another Intent is called automatically, the “SIGN_IN” Intent.

This Intent is defined in lines 216 to 253 and receives the parameters from the result of Account Linking, basically with the Status of the Account Linking OK is access granted.

If granted I get userId from the right variable and Say “Hi, username, what can I do for you?” in each locale and send to Google Assistant.

If Not granted, advise that I may have to ask for some more information in the middle of the CHATBOT if needed, this because in my chatbot I keep information regarding user’s name and EPM instance he wants to access. “Hi, as you did not let me access your details, during the process I will have to ask for some information, what can I do for you?” in each locale and send to Google Assistant

3.1.5. Line 118 to 210 This is the “Default Fallback Intent” processing, as we set the Dialogflow Every normal phrase said will come in this intent. The only exception is when the user answers “Cancel” or something similar in each language, but cancel works in this case, then the “Cancel” Intent will be called. All other answers will come to this Intent. A point here about languages in Actions on Google and languages that user can choose in this Cellphone in Google Home app, and about Resource Bundles. The Languages available in Google on Actions as for today (May 2019): Danish (da), German (de), English, (en), French (fr), Hindi (hi), Indonesian (id), Italian (it), Japanese (ja), Korean (South Korea) (ko), Dutch (nl), Norwegian (no), Polish (pl), Portuguese (European) pt, Portuguese (Brasil) pt-BR, Russian (ru), Spanish (es-es), Swedish (sv), Thai (th), Turkish (tr), Ukrainian (uk), Chinese (simplified) (zh-cn), Chinese (Hong Kong) (zh-hk), Chinese (Traditional) (zh-tw)

When you choose to create a language in Google on Actions you can choose Individual Locales, as Spanish (Europe) or Spanish (Latin America) or even the broader Spanish that includes both Locales (Europe) and (Latin America). But on the other hand, with your Google Home App, in your cellphone you can have more languages available, with more details, here only some examples: Spanish (Argentina) (es-AR), Spanish (Mexico) (es-MX), Spanish (Spain) (es-ES)

Oracle Digital Assistant - TechExchange

And when you define you Resource Bundles in ODA you can give them the codes you want, like: es-419 (Spanish Mexico), or es-Es (Spanish Europe), but you define the code, and if it has Capital letters or not etc. • If the bot can’t match the language set for the Digital Assistant locale, or any other sending a message to it, with a language tag defined in the bundle, it defaults to a less-specific tag (if one is available). For example, it uses fr (a subtag) if the bundle has no entry for fr-CA. If none of the entries match the browser’s language, the bot uses the default entry, English. For more information on this fallback to the most generic entry, see Resource Bundle Entry Resolution So be careful on how you define these three codes and if you have to convert what comes as locale from the Digital Assistant to be able to match what was defined in Resource Bundles. Now about the code: • First, I get Locale from the message to be able to set different: ➢ Channel URL and Secret for each locale ➢ Messages in the right locale when they are sent by the Webhook itself ➢ In my Chatbot I defined an es-439 locale, so every Spanish locale transform in es-439 • I test if I have “conv.user.profile.payload” yes means I have users info from google Account, “No” means I have the random userId in the storage area of conversation. • I set channels info depending on the locale. • Set a promise. ➢ Set layout of Chatbot message o Create additionalProperties, to send locale to chatbot, chatbot will receive this information in "${profile.locale}" and "${profile.clientType}". ➢ Pubsub.Subscribe with callback function to be able to work when message from chatbot returns. PubSub.subscribe(userId, treatandsendtoGoogle). ➢ Execute send command with optional parameter, channel information. ➢ IF error occurs, send message of error and do a “PubSub.unsubscribe” • When message returns from Chatbot, it goes straight to line 69 – app.post('/bot/message/:locale' and with “PubSub.publish” will return to line 171 and execute “treatandsendtoGoogle” where message is formatted in Google layout and sent to Google ➢ Here is important to explain that, if we send to Google with “conv.ask” we are telling Google that we await a response and Google Assistant will not end the conversation and sending a “conv.close” it will consider the conversation ended.

Oracle Digital Assistant - TechExchange

➢ To be able to know if the user is in normal conversation flow or ending the conversation you must treat this in your ODA flow normally by having a command: transitions: return: action: "done" But doing this says nothing to Google Assistant, so we must send any other information to the webhook, so it can recognize that you want to end the conversation, and then be able to send a “conv.close”, instead of “conv.ask”. The way I found to do this is using the “System.CommonResponse” instead of the “System.Output” in ODA flow, and use the “channelCustomProperties:” option of “CommonResponse”. With this command only used when the flow sends message that ends the conversation (this usually is at a specific point in your ODA flow), you can control when to tell Google Assistant to close the conversation. This option is controlled by ChannelType, so you can only send to “webhook” type ODA Channels. The snippet kept like this in my ODA Flow:

clickexit: component: "System.CommonResponse" properties: processUserMessage: false keepTurn: false metadata: responseItems: - type: "text" text: "${rb('ClickExit')}" channelCustomProperties: - channel: "webhook" properties: end_conversation: "true" transitions: return: action: "done"

When this option is used your JSON message received by the webhook will have a special field, called channelExtensions.

"body”: {"messagePayload":{"text":"Thanks for being here!!!","type":"text","channelExtensions":{"end_conversation":"true"}},"userId":" 113258672069157831413"},

So, I just must test if “channelExtensions” exists, if YES then send a “conv.close”, if not send a “conv.ask”

Oracle Digital Assistant - TechExchange

• “PubSub.unsubscribe”

3.1.6. Line 255 to 327 This is the “Cancel Intent” processing. I could have used an Intent that responds to the user with a goodbye phrase and ends the conversation, but in my testing I found out that the CHATBOT needs to send the correct end command, so that if it receives a next conversation from the same user, treats it as a new conversation, and not as a continuation of an old conversation, as commented in the section of this document where I explain the creation of the Cancel Intent in the Dialogflow. As the cancel is only used after the initial conversation begun, somethings are already set. • First, I get Locale from the message to be able to set different: ➢ Channel URL and Secret for each locale ➢ Messages in the right locale when they are sent by the Webhook itself • I test if I have “conv.user.profile.payload” yes means I have users info from google Account, “No” means I have the random userId in the storage area of conversation. • I set channels info depending on the locale. • Set a promise. ➢ Set layout of Chatbot message o Create additionalProperties, to send locale to chatbot, chatbot will receive this information in "${profile.locale}" and "${profile.clientType}". ➢ Pubsub.Subscribe with callback function to be able to work when message from chatbot returns. PubSub.subscribe(userId, treatandsendtoGoogle). ➢ Execute send command with optional parameter, channel information. ➢ IF error occurs, send message of error and do a “PubSub.unsubscribe” • When message returns from Chatbot, it goes straight to line 69 – app.post('/bot/message/:locale' and with “PubSub.publish” will return to line 299 and execute “treatandsendtoGoogle” where message is formatted in Google layout and sent to Google via a “conv.close” command that ends the conversation in Google Assistant • “PubSub.unsubscribe”

3.1.7. Line 330 to 414 Functions copied from @oracle-bots-node-sdk to Transform the parts of messages received from the bot to the Text format, needed to send to Google Home.

Oracle Digital Assistant - TechExchange

3.1.8. Line 417 to 419 app.use('/fulfillment',bodyParser.json(),assistant); app.post('/fulfillment', assistant ); App.use and app.post for the messages received from Google Home. The “/fulfillment” is the webhook address indicated in the Dialogflow Fulfillment Section

3.1.9. If you only have on Chatbot • In lines 28 to 58 – Change the code to: (it is commented in the code) const webhook = new WebhookClient({ channel: { url: 'http://2b2d3e3d.ngrok.io/connectors/v1/tenants/chatbot- tenant/listeners/webhook/channels/291868e7-1eeb-490d-9fe5-c84362f34492', secret: 'BpZMnlY64tzVoBZHRtcgNvvs90ZE8lN6', } }); • Remove parts where you set URL and Secret based on Locale: ➢ Lines 139 to 157 ➢ Lines 269 to 287 • Change lines that use “webhook.send”: Remove the “channeloc” optional attribute: “webhook.send(message)” ➢ Lines: 193 and 315

4. Conclusion. So finally, you should have your Chatbot integrated to Google Home Assistant, working also on Android Cellphones. If you don't have more than one Chatbot, you don't need the parts where I, based on the locale, chose the URL and Secret of the Channel.

Oracle Digital Assistant - TechExchange