Enabling multiple languages
1. Introduction
Floatchat provides a number of capabilities to assist you in building and managing your bots in multiple languages as well as just for non-English languages. These capabilities span across the various key components of the platform, namely FAQs and Intents which drive the NLP behaviour as well as the paths or the conversational flows.
Some of the key aspects of these capabilities are –
- Tagging FAQs for NLP: You can tag and associate FAQs and Intents to a particular language to better manage them as well as use the NLP models specific to the language. The user’s queries are also resolved to keep the language factor in mind.
- Localize Messages in Conversation Flows (Paths): Building different paths for different languages is not needed at all since now you can just build a single path/flow and add the strings for the different languages you are supporting.
- Localize Settings Messages: Similar to the user-facing messages for the respective languages, even the settings, and configurations would be managed by the language properties files. This includes, among others, the bot’s name itself, various messages for configurations like user feedback, other fixed messages like writing a message, etc. This will allow the user the option to customize these as well as have different messages in different languages.
- Synonyms & Stopwords for Languages: You can also have separate sets of synonyms and stopwords based on language which are picked up appropriately by the system.
2. Using Localization in Floatchat
To understand the various aspects of these capabilities, we will create a simple chatbot that will welcome a user and let them explore the chatbot in their choice of language. For this, we will use two languages English and Arabic where English is the default language and Arabic is the secondary language.
2.1 Availability
Multilingual capability is present in our Professional and above plans.
2.2 Enabling Localization for your Chatbot
The first step to enable this feature, go to configure section present on the left-hand side, select languages and click on Enable.
Once this feature is enabled, properties files to manage the user-facing messages in the paths are generated for all the paths of the bot as well as FAQs are set up to support this functionality.
After the setup is completed, you can see the screen where the languages are available. Also, the strings for the various paths and settings of the bots are managed in this section.
The bot has a default language that it uses to fall back for missing strings for other languages as well as for the FAQs/Intent lookups. The default language initially would be set as English which can be changed. We will understand the significance of the default language in more detail in a later section.
Add the other languages which you want to support in the bot using the Add New button.
2.3 Properties Files: Managing localized messages
All Messages from the Conversation Flow Modeler or various Bot Configurations, are identified & referred to as Keys.
The keys and corresponding values for all messages can be downloaded as this properties file. This helps in easy management of the messages as well as getting them translated and updated since all the messages are in one place. The file contains all the user-facing messages from all the paths which are set up in the bot at that point. Whenever a path is added, the keys are also updated to include the new ones. This file also contains messages corresponding to various settings that can be localized.
The keys/messages in the file are sectioned by the path in which those messages are present and then grouped by nodes. Comments are added in the file for each section to help you navigate the file easily. It is recommended to manage the strings after you have completed the bot flow design and all the paths are set up. This would lead to less frequent updating of the strings.
Once the file is downloaded, you can include the translated messages in the file here corresponding to each key/line in this file. You can use the default language file as a reference as well to help you get the context. Once this has been done Upload the key file into the new language from the same section. Please ensure that no messages/strings are missed in the properties files.
2.4 FAQs & Intent
In the earlier section, we saw how to set up the language feature as well as manage the messages for the paths and settings by using the keys in the properties file. Now we will focus on the FAQs and Intents.
As highlighted earlier, FAQs can be associated with a language. Hence, you can have multiple sets of FAQs, each corresponding to the languages you are supporting.
In the following steps, we are going to learn how to add FAQs for the bot which supports two languages.
-
Go to the FAQ workflow from the Build section.
-
Click on ADD FAQ for adding the queries and their variations.
-
Select the language you want to associate the particular FAQ set with.
-
Enter the query including the variations and their response and click on ADD. The FAQ will be added and tagged to the language.
When a user asks a query, it will look up the FAQs stored in the user’s selected language (which is set via the ‘Select Language node’, explained later). It will also additionally look up the FAQs that are associated with the bot’s default locale (if they are different).
2.5 Control the user flow
After the translated files are set up for the paths and the FAQs are added in the respective languages, we will now learn how the user can change the language or how to determine and set the user’s language.
The Select Language node is provided to facilitate the setting of a language in the path.
Define a flow and use the node that allows the user to switch to their desired language.
Once this node is encountered, all the nodes/paths after this node would be displayed in a particular language which is selected here. Also, the NLP engine would consider it as the set language for the user which it would use to determine the response.
Alternative Method for setting the language
Other than the Select language node, there are other ways in which the language can be set for a user. It can be set from the website chat widget script by setting the user.language with the appropriate ISO 639-1 code for the language. A use-case where this would come in handy is when the website is aware of the user’s language preference and that can be passed on to the bot’s context as well.
The same attribute can be set via the Set Attributes node, the Script node as well as the JSON API node similar to setting any other standard attribute. The behaviour in each of these would be the same as using the Select Language node and would switch the language of the bot with the relevant messages being displayed as per the newly set language.
2.6 Default Language
The default language is the language which your bot initially displays. By default, it is set to English. The bot at any time would have one language which is selected as the default. This is used as a fallback language as well in case the user’s language is not set or defined.
Changing the default language
This default language can be changed by toggling the radio button for a particular language. There are two modes that are supported for changing the default language with different behaviours for each.
The first option is where you can associate the existing FAQs of your bot to the new default language and copy over the missing strings of the original default language to the new one.
The other mode doesn’t do any additional processing of the FAQs or strings and keeps them as is.
Notes –
- This essentially becomes migrating from the old to the new default language and is recommended to be used especially for cases where the localization is set up for the first time and the language needs to be updated to the new one.
You should make sure that all the necessary strings of your bot are added/updated for the default language as it may result in users seeing the keys in place of missing values.
Additional things to know regarding Localization
- Business plan users who are using the Stopwords and Synonyms feature can segregate these into the supported languages and they will be picked up appropriately based on the user and bot settings.
- There would be no changes in language for the entities, they will continue to be language-agnostic and can be managed by having multiple values in the same entity or creating multiple entities for each language and associating them with the relevant language FAQs.
- The FAQ upload via CSV would also support a new column that would specify the language association of the FAQs. The values of the language fields should be the ISO639-1 code for the language.
- The Train workflow would also support specifying the language. The only exception would be when selecting from close matches or via search, the language of the original FAQ trained would continue to be applicable. Also, interaction history would show the language from which the query was matched as well via tags.
3. Best Practices for building multi-lingual bots
3.1 Easily change the language
Make it super easy for the bot user to get to the options for changing languages. If you are very sure of their configurations, you can use the Select Language node for doing it as well as control it via the attributes.
If you are not sure of their preferences, present the options for selecting language upfront. You should also have FAQs set up to enable changing/switching languages in all the languages you are supporting, especially the default language since that would be looked up to always.
3.2 Multi-language instructions
Make sure to add the messages for changing the language in all the languages you are supporting. This fact goes for any greeting or introductory messages as well which are meant to be generic/language-agnostic. Also, at least the most common keywords/actions, FAQs should be ideally supported for all the languages for a seamless experience for everyone.
3.3 No missing values
Make sure to review all the strings/messages for all the language you have added to check if anything is missing and if everything is correct, including the settings and default values. The user experience would be severely affected if the end-user sees keys instead of messages from the bot
3.4 Avoid using special characters
Special characters like *, “”,^.$,# throw an error for some languages. Adding them to the key will lead to an error while localizing a bot. Also while creating attributes n needs to add ‘{{‘ in the key, again it will give erroneous output.
3.5 Language not supported
If the language you are supporting isn’t on the list of supported languages, you can select the closest one similar to the language and add your FAQs and path messages. You can choose to leave it as English as well and the default matching rules engine would be applicable in that case.
3.6 Languages Supported
The following is the list of 52 languages we support at present
1. Arabic | 14. Finnish | 27. Kannada | 40. Romanian |
2. Bengali | 15. French | 28. Korean | 41. Russian |
3. Bulgarian | 16. German | 29. Malay | 42. Slovak |
4. Burmese | 17. Greek | 30. Malayalam | 43. Somali |
5. Chinese | 18. Gujarati | 31. Marathi | 44. Spanish |
6. Catalan | 19. Hebrew | 32. Mongolian | 45. Swedish |
7. Croatian | 20. Hindi | 33. Nepali | 46. Tamil |
8. Czech | 21. Hungarian | 34. Norwegian | 47. Telugu |
9. Danish | 22. Indonesian | 35. Persian | 48. Thai |
10. Dutch | 23. Italian | 36. Polish | 49. Turkish |
11. English | 24. Javanese | 37. Portuguese | 50. Ukrainian |
12. Estonian | 25. Japanese | 38. Punjabi | 51. Urdu |
13. Filipino | 26. Kazakh | 39. Pushto | 52. Vietnamese |
- Arabic
- Bengali
- Bulgarian
- Burmese
- Chinese
- Catalan
- Croatian
- Czech
- Danish
- Dutch
- English
- Estonian
- Filipino
- Finnish
- French
- German
- Greek
- Gujarati
- Hebrew
- Hindi
- Hungarian
- Indonesian
- Italian
- Javanese
- Japanese
- Kazakh
27. Kannada
28. Korean
29. Malay
30. Malayalam
31. Marathi
32. Mongolian
33. Nepali
34. Norwegian
35. Persian
36. Polish
37. Portuguese
38. Punjabi
39. Pushto
40. Romanian
41. Russian
42. Slovak
43. Somali
44. Spanish
45. Swedish
46. Tamil
47. Telugu
48. Thai
49. Turkish
50. Ukrainian
51. Urdu
52. Vietnamese
To understand the various aspects of these capabilities, we will create a simple chatbot that will welcome a user and let them explore the chatbot in their choice of language. For this, we will use two languages English and Arabic where English is the default language and Arabic is the secondary language.
References
- NLP: Learn more about how to train your bot using Floatchat proprietary NLP Engine
- Conversation Flow Modeler: Learn more about how to build conversation flows
- Synonyms & Stopwords: Learn more about how Flaotchat handles synonyms and stopwords as well as options for you to update these for your bot
To understand the various aspects of these capabilities, we will create a simple chatbot that will welcome a user and let them explore the chatbot in their choice of language. For this, we will use two languages English and Arabic where English is the default language and Arabic is the secondary language.
2.1 Availability
Multilingual capability is present in our Professional and above plans.
2.2 Enabling Localization for your Chatbot
The first step to enable this feature, go to configure section present on the left-hand side, select languages and click on Enable.
Once this feature is enabled, properties files to manage the user-facing messages in the paths are generated for all the paths of the bot as well as FAQs are set up to support this functionality.
After the setup is completed, you can see the screen where the languages are available. Also, the strings for the various paths and settings of the bots are managed in this section.
The bot has a default language that it uses to fall back for missing strings for other languages as well as for the FAQs/Intent lookups. The default language initially would be set as English which can be changed. We will understand the significance of the default language in more detail in a later section.
Add the other languages which you want to support in the bot using the Add New button.
2.3 Properties Files: Managing localized messages
All Messages from the Conversation Flow Modeler or various Bot Configurations, are identified & referred to as Keys.
The keys and corresponding values for all messages can be downloaded as this properties file. This helps in easy management of the messages as well as getting them translated and updated since all the messages are in one place. The file contains all the user-facing messages from all the paths which are set up in the bot at that point. Whenever a path is added, the keys are also updated to include the new ones. This file also contains the messages corresponding to various settings that can be localized.
The keys/messages in the file are sectioned by the path in which those messages are present and then grouped by nodes. Comments are added in the file for each section to help you navigate the file easily. It is recommended to manage the strings after you have completed the bot flow design and all the paths are set up. This would lead to less frequent updating of the strings.
Once the file is downloaded, you can include the translated messages in the file here corresponding to each key/line in this file. You can use the default language file as a reference as well to help you get the context. Once this has been done Upload the key file into the new language from the same section. Please ensure that no messages/strings are missed in the properties files.
2.4 FAQs & Intent
In the earlier section, we saw how to set up the language feature as well as manage the messages for the paths and settings by using the keys in the properties file. Now we will focus on the FAQs and Intents.
As highlighted earlier, FAQs can be associated with a language. Hence, you can have multiple sets of FAQs, each corresponding to the languages you are supporting.
In the following steps, we are going to learn how to add FAQs for the bot which supports two languages.
- Go to the FAQ workflow from the Build section.
- Click on ADD FAQ for adding the queries and their variations
- Select the language you want to associate the particular FAQ set with.
Enter the query including the variations and their response and click on ADD. The FAQ will be added and tagged to the language.
Bot behaviour for resolving queriesWhen a user asks a query, it will look up the FAQs stored in the user’s selected language (which is set via the ‘Select Language node’, explained later). It will also additionally look up the FAQs that are associated with the bot’s default locale (if they are different).
Bot behaviour for resolving queries
When a user asks a query, it will look up the FAQs stored in the user’s selected language (which is set via the ‘Select Language node’, explained later). It will also additionally look up the FAQs that are associated with the bot’s default locale (if they are different).
2.5 Control the user flow
After the translated files are set up for the paths and the FAQs are added in the respective languages, we will now learn how the user can change the language or how to determine and set the user’s language.
The Select Language node is provided to facilitate the setting of a language in the path.
Define a flow and use the node that allows the user to switch to their desired language.
Once this node is encountered, all the nodes/paths after this node would be displayed in a particular language which is selected here. Also, the NLP engine would consider it as the set language for the user which it would use to determine the response.
Alternative Method for setting the language
Other than the Select language node, there are other ways in which the language can be set for a user. It can be set from the website chat widget script by setting the user.language with the appropriate ISO 639-1 code for the language. A use-case where this would come in handy is when the website is aware of the user’s language preference and that can be passed on to the bot’s context as well.
The same attribute can be set via the Set Attributes node, the Script node as well as the JSON API node similar to setting any other standard attribute. The behaviour in each of these would be the same as using the Select Language node and would switch the language of the bot with the relevant messages being displayed as per the newly set language.
2.6 Default Language
The default language is the language which your bot initially displays. By default, it is set to English. The bot at any time would have one language which is selected as the default. This is used as a fallback language as well in case the user’s language is not set or defined.
Changing the default language
This default language can be changed by toggling the radio button for a particular language. There are two modes that are supported for changing the default language with different behaviours for each.
The first option is where you can associate the existing FAQs of your bot to the new default language and copy over the missing strings of the original default language to the new one.
The other mode doesn’t do any additional processing of the FAQs or strings and keeps them as is.
Notes –
- This essentially becomes migrating from the old to the new default language and is recommended to be used especially for cases where the localization is set up for the first time and the language needs to be updated to the new one.
You should make sure that all the necessary strings of your bot are added/updated for the default language as it may result in users seeing the keys in place of missing values.
Additional things to know regarding Localization
- Business plan users who are using the Stopwords and Synonyms feature can segregate these into the supported languages and they will be picked up appropriately based on the user and bot settings
- There would be no changes in language for the entities, they will continue to be language-agnostic and can be managed by having multiple values in the same entity or creating multiple entities for each language and associating them with the relevant language FAQs.
- The FAQ upload via CSV would also support a new column that would specify the language association of the FAQs. The values of the language fields should be the ISO639-1 code for the language.
- The Train workflow would also support specifying the language. The only exception would be when selecting from close matches or via search, the language of the original FAQ trained would continue to be applicable. Also, interaction history would show the language from which the query was matched as well via tags.
3. Best Practices for building multi-lingual bots
3.1 Easily change the language
Make it super easy for the bot user to get to the options for changing languages. If you are very sure of their configurations, you can use the Select Language node for doing it as well as control it via the attributes.
If you are not sure of their preferences, present the options for selecting language upfront. You should also have FAQs set up to enable changing/switching languages in all the languages you are supporting, especially the default language since that would be looked up to always.
3.2 Multi-language instructions
Make sure to add the messages for changing the language in all the languages you are supporting. This fact goes for any greeting or introductory messages as well which are meant to be generic/language-agnostic. Also, at least the most common keywords/actions, FAQs should be ideally supported for all the languages for a seamless experience for everyone.
3.3 No missing values
Make sure to review all the strings/messages for all the language you have added to check if anything is missing and if everything is correct, including the settings and default values. The user experience would be severely affected if the end-user sees keys instead of messages from the bot
3.4 Avoid using special characters
Special characters like *, “”,^.$,# throw an error for some languages. Adding them to the key will lead to an error while localizing a bot. Also while creating attributes n needs to add ‘{{‘ in the key, again it will give erroneous output.
3.5 Language not supported
If the language you are supporting isn’t on the list of supported languages, you can select the closest one similar to the language and add your FAQs and path messages. You can choose to leave it as English as well and the default matching rules engine would be applicable in that case.
3.6 Languages Supported
The following is the list of 52 languages we support at present
- Arabic
- Bengali
- Bulgarian
- Burmese
- Chinese
- Catalan
- Croatian
- Czech
- Danish
- Dutch
- English
- Estonian
- Filipino
- Finnish
- French
- German
- Greek
- Gujarati
- Hebrew
- Hindi
- Hungarian
- Indonesian
- Italian
- Javanese
- Japanese
- Kazakh
27. Kannada
28. Korean
29. Malay
30. Malayalam
31. Marathi
32. Mongolian
33. Nepali
34. Norwegian
35. Persian
36. Polish
37. Portuguese
38. Punjabi
39. Pushto
40. Romanian
41. Russian
42. Slovak
43. Somali
44. Spanish
45. Swedish
46. Tamil
47. Telugu
48. Thai
49. Turkish
50. Ukrainian
51. Urdu
52. Vietnamese
References
- NLP: Learn more about how to train your bot using Floatchat proprietary NLP Engine
- Conversation Flow Modeler: Learn more about how to build conversation flows
- Synonyms & Stopwords: Learn more about how Flaotchat handles synonyms and stopwords as well as options for you to update these for your bot