Contact Us

If you still have questions or prefer to get help directly from an agent, please submit a request.
We’ll get back to you as soon as possible.

Please fill out the contact form below and we will reply as soon as possible.

  • Go to Haptik Website
  • Contact Us
  • Home
  • Bot Building
  • Testing

How to make use of Training Suggestions?

Written by Medha Anand

Updated on April 8th, 2022

Contact Us

If you still have questions or prefer to get help directly from an agent, please submit a request.
We’ll get back to you as soon as possible.

Please fill out the contact form below and we will reply as soon as possible.

  • Getting Started
    Build Deploy Analyse Manage Account Bot Deactivation
  • Bot Building
    Essentials Smart Skills Steps User Messages Bot Responses Entities Connections Customisations User feedback collection Testing Whatsapp Bots NLU Bot Maintenance
  • Smart Agent Chat
    Set up Admin Settings MyChats Section (Agent Inbox) Live Traffic Section Teams Section Archives Section Analytics Plans on Smart Agent Chat
  • Conversation Design
    Design Basics Design Guides Designing for Platforms Designing WhatsApp Bots
  • Developer Guides
    Code Step Integration Static Step Integration Shopify Integration SETU Integration Exotel Integration CIBIL integration Freshdesk KMS Integration PayU Integration Zendesk Guide Integration Twilio Integration Razorpay Integration LeadSquared Integration USU(Unymira) Integration Helo(VivaConnect) Integration Salesforce KMS Integration Stripe Integration PayPal Integration CleverTap Integration Fynd Integration HubSpot Integration Magento Integration WooCommerce Integration Microsoft Dynamics 365 Integration
  • Deployment
    Web SDK WhatsApp Facebook Instagram Sunshine Conversation LINE Google Business Messages Telegram MS Teams Bot as an API iOS SDK Android SDK
  • External Agent Tool Setup
    Zendesk Chat Salesforce Service Cloud Freshchat Zoho NICE CXOne Gorgias
  • Analytics & Reporting
    Intelligent Analytics
  • Notifications
    SMS Notifications Success Measurement
  • Commerce Plus
    Catalog Integration Bot Building Guide Channel Deployments Unified ML Pipeline Documentation
  • Troubleshooting Guides
    Error Messages FAQs
  • Release Notes
+ More

The Training Suggestions mail helps Bot Builders discover methods by which a bot can be improved and brought to a state where it is utilizing the capabilities of the ML Model to its maximum potential. It aims to give feedback to Bot Builders on how well the bot has been trained from the "User Messages" and "Steps" created by the Bot Builder. However, before we delve into the details of the Training Suggestions, it’s essential to understand the expectations of the ML Model from the Bot Builder, so that improvements can be made accordingly.

What does the ML Model expect to perform at its fullest?

  • 10 variants on each Start Step.
  • 10 variants on each Connection response including -
    • Start Step to Dependent Step.
    •  Dependent Step to  Dependent Step.
    • OutputStep to Static Step.
    • Output Step to Dependent Step.
  • If more than 10 Connection responses are not present, the performance of the ML Model, even on Start Steps would be affected negatively.
  • Similar or exact same “User Messages” should not be present in multiple steps.
  • Steps should not be too similar in the goal they aim to achieve, else the bot may not be able to differentiate, and decide the correct Step. The image below shows two similar steps, and due to the high similarity between these Steps, a minor variation in the User Message prevented a one-shot response from the bot and triggered a Disambiguation Message.
  • The quality of the “User Messages” should be as per User Messages Guidelines.

The Training Suggestions reaches the Conversation Studio as part of the Training Completion Email. It is divided into certain scenarios, and each scenario has a list of issues with Training Data present in the bot. If either of the scenarios have more than 30 entries, the section would be attached as a CSV to the email.

The Bot Builder should go through each of the scenarios, and make sure they fix the issues if there are any so that the bot functions properly.

Delete

The Training Suggestions only considers Start Steps for evaluation and display purposes.

1. Standard Scenario

From: Niko@haptik.co
To: Bot training, User
Subject Line: <bot name> training is completed. Here are some suggestions 
Body:
Hi <first name>, 
Your <bot name> training is completed. Good job - before you take it live, we want to make sure it’s intelligent enough to woo your customers. 
Here are some quick checks we ran, just to be double sure. If the below tables are empty, pat yourself on the back - your bot is perfectly trained! 

Will your bot get confused between two similar user says statements added at multiple steps?
User says statement
Step containing 
user says
Similar user says statement
Step containing similar User Says




Delete

How to fix it? 

If your bot gets confused between two similar user says statements added at multiple steps, then you can fix it by,

  • Testing out different variations of User Message statements for your bot, wherein you can frame the above-mentioned User Messages in different ways and ask your bot. If the bot gets confused in taking the next step or answers the query incorrectly, you can then add new statements variations in your preferred Step.
  • Make sure that similar User Message statements are not present in multiple steps.
Will your bot fail to interpret a question if framed differently?
User says statement
Step containing 
user says


Delete

How to fix it? 

If your bot fails to interpret a question, when framed differently, then you can fix it by,

  • Testing out different variations of the above user message statements for your bot. wherein you can frame the above-mentioned user messages in different ways and ask your bot. If it gets confused in taking the next step or answers the query incorrectly, you can also add the new variations of statements in the same Step. 
  • Make sure there are 10 or more User Message statements added for every Step.
  • Make sure there are 10 or more User Message statements added for every Connection.
Will your bot get confused between any two similar steps? 
Step 1 
No. of user says
User Says causing overlap in step
Step 2
No. of user says
User Says causing overlap in step
Degree of similarity








Delete

How to fix it?

If your bot gets confused between two similar steps, then you can fix it by,

  • Testing out mixed variations of the above User Message statements for your bot. You can also mix the words from variations of similar User Messages and then ask your bot. If it gets confused in taking the next step or answers the query incorrectly, you can add new statements variations in your preferred Step.
  • Merging two Steps that are very similar, or cater to similar use-cases. 
  • Make sure there are 10 or more variations of User Messages added for every Step and Connection.

If you have any questions, please reach out to support@haptik.ai. After you’ve resolved all the above issues, your bot is good to go! 

Cheers <cheers emoji>, 
Niko, your Platform Guide

2. Scenario - Feedback is too large to display

From: Niko@haptik.co
To: Bot training, User
Subject Line: <bot name> training is completed. Here are some suggestions
Body: 
Hi <first name>, 
Your <bot name> training is completed. Good job - before you take it live, we wanted to make sure it’s intelligent enough to woo your customers. 
Here are some quick checks we ran, just to be double sure.  If the below tables are empty, pat yourself on the back - your bot is perfectly trained!

Will your bot get confused between two similar answers for any particular question?

The report is attached as similar_user_says.csv
No. of entries - 123

Delete

How to fix it?

If your bot gets confused between two similar answers for any particular question, then you can fix it by,

  • Testing out different variations of User Message statements for your bot, and then framing the above-mentioned User Messages in different ways, and asking your bot. If it gets confused in taking the next step or answers the query incorrectly, add the new statements variations in your preferred Step.
  • Making sure similar User Messages are not present in multiple Steps.
Will your bot fail to interpret a question if framed differently?
The report is attached as bot_has_not_learnt.csv
No. of entries - 123
Delete

How to fix it?

If your bot fails to interpret a question if it has been framed differently, then you can fix this by,

  • Testing out different variations of the above User Message statements for your bot, and then framing the above-mentioned User Messages in different ways and asking your bot. If it gets confused in taking the next step or answers the query incorrectly, add the new variations of statements in the same Step.
  • Make sure there are 10 or more variations of User Messages added for every Step.
  • Make sure there are 10 or more variations of User Messages added for every Connection because without this a Step might not get initiated.
Will your bot get confused between any two similar steps? 

The report is attached as similarity_between_intents.csv
No. of entries - 123

Delete

How to fix it?

If your bot gets confused between two similar Steps, then you can fix it by,

  • Testing out mixed variations of the above User Message statements for your bot. Mixing the words from variations of similar User Messages and then asking your bot. If it gets confused in taking the next step or answers the query incorrectly, add the new statements variations in your preferred Step.
  • Merging two Steps that are very similar or cater to similar use-cases.
  • Make sure there are 10 or more variations of User Messages added for every Step and Connection. 

If you have any questions, please reach out to support@haptik.ai. After you’ve resolved all the above issues, your bot is good to go! 

Cheers <cheers emoji>,
Niko, your Platform Guide

3. Scenario - There is no feedback

From: Niko@haptik.co
To: Bot training, User
Subject Line: <bot name> training is completed. Here are some suggestions 
Body: 
Hi <first name>, 
Your <bot name> training is completed. Good job - before you take it live, we wanted to make sure it’s intelligent enough to woo your customers. 
Here are some quick checks we ran, just to be double sure, and looks like your bot passed all the tests! 

Will your bot get confused between two similar answers for any particular question?
No errors found!
Will your bot fail to interpret a question if framed differently?
No errors found!
Will your bot get confused between any two similar steps? 
No errors found!

Your bot is good to go! 

Cheers <cheers emoji>,
Niko, your Platform Guide

FAQs

1. Is it compulsory to add 10 User Messages to all independent responses and Connection User Messages?

Yes, because if you add less than 10 User Messages, then the ML model becomes confident giving rise to a lot of issues in the Training email.

2. Can we initially add more than 10 user messages as independent responses before working on the issue of the Training Suggestions email?

Yes, because if you add less than 10 User Messages, then the ML model becomes underconfident, giving rise to a lot of issues in the Training email.

3. Is it advisable to add duplicate User Messages?

No, you can reshuffle the words.

4. Should we remove the User Messages in order to solve the issue of the Training email?

No, you can test the variation if it is going to the wrong Step, and then add that User Messages to the correct Step, and add more similar variations on the same Step.

Was this article helpful?

Yes
No
Give feedback about this article

Related Articles

  • How to test the bot?
  • How to use bot QA tool?
  • What is bot functional testing?
  • How to use the debug logs?

Platform

  • Conversation Studio
  • Smart Skills
  • Advanced NLU
  • Intelligent Analytics
  • Omnichannel
  • Smart Agent Chat
  • Enterprise Security
  • Integrations

Solutions

  • Conversational Commerce
  • Lead Generation
  • Customer Care
  • WhatsApp
  • Conversational IVR
  • Google Business Messages

Industries

  • Retail/ E-Commerce
  • Financial Services
  • Travel & Hospitality
  • Telecom

Knowledge

  • ROI Calculator
  • Reports & Research
  • Case Studies
  • Webinars
  • ISAT
  • Tech Blog
  • Business Blog
  • Resources
  • Haptik v/s Yellow
  • Haptik v/s Liveperson
  • Haptik v/s IBM Watson
  • Haptik v/s Verloop
  • Conversations on AI

Company

  • Why Haptik
  • About Us
  • Careers
  • News & Media
  • Awards & Recognition
  • Contact Us
  • Partnerships
  • Investor Relations

Subscribe

Sign up to recieve the latest updates

Find us on

  • Twitter-footer
  • Linkedin-footer
  • YT-footer
  • Insta-footer
  • G2-footer
  • Facebook-footer

Knowledge Base Software powered by Helpjuice

Copyright © jio Haptik Technology Limited 2021 | Data Security & Privacy Policy | GDPR

North America | Asia Pacific | Africa | enterprise@haptik.ai

Definition by Author

0
0