Training None Intent in LUIS

Question

I have a travel bot with following intents:-

1. BookAFlight (trained with 20 utterances)

2. GetTicketCopy (trained with 20 utterances)

3. CancelTicket (trained with 20 utterances)

4. None (default) (currently not trained)

MS documentation suggests that I train None for atleast 1-2 utterances for every 10 utterances added to other intents. Which means I would need say 6-12 utterances to train None.

My query is what kind of utterances should be used to train None?

1. Everything under the sun apart from what is relevant to my bot ( for e.g. I want to order a pizza, How is the weather today, Who is the president of USA? etc.)

2. All negative statements corresponding to the utterances used to train my other intents (for e.g. I don't want to book a flight ticket, I don't want to take a print out, I don't want to cancel my ticket etc.)

3. All utterances that corresponds to intents currently not covered in my scope but which users could still ask to a travel bot (for e.g. I want to Book a Cab to the airport, What is the status of my flight)

Long story short, I am trying to identify what kind of utterances should go into my None intent . Is None the right place to handle "Negative" variations of valid utterances.

Thoth

Answer 1

None intent is not made for "Negative variations" but to tag everything that is not managed by your other intents.

So you should add utterances corresponding to cases that your bot cannot handle but linked to your context (your 3rd idea).

For example in one of my projects, None intent is trained with other use-cases of my customer than the one I will treat with my bot, based on my customer logs. And it's helping avoiding to throw an intent in a bad case

Brotherinlaw

Thanks Nicolas. Any thoughts on what would be the best way to handle negative variations. Currently my app returns a very high score for "BookAFlight" when the user input is "I don't want to Book a flight". There is another thread where developers have used None to handle negative variations while some others have suggested a more sophisticated approach like a sentiment analysis before pushing the input to LUIS models. #38498433 – Thoth

Try to add these negative examples to None yes and check on the tester. The problem with sentiment analysis is the additional cost of the call for a few cases – Brotherinlaw

Tried another approach. Created an new intent (TraveBot_Negative_Intents) just to handle Negative variations . Trained it with 1-2 negative utterances per functional area ( I don't want to book a ticket, don't tell me how to cancel a ticket etc.). That way I have a sure way of recognizing all negative variations and respond to them appropriately. None can be now left alone to handle areas which are currently out of scope. Results are looking promising. Happy to hear if this works for others in the community as well. Share your thoughts if it does/doesn't. – Thoth

It's also a valid approach, it only depends on your business case behind – Brotherinlaw

This answer is kind of the opposite of what MS states here, it says "Fill it with utterances that are outside of your domain." – Antiparallel

Answer 2

The LUIS docs suggest you should use COMPLETELY off-topic utterances for the None intent: "Start with something specific that your bot shouldn't answer such [as] 'What kind of dinosaur has blue teeth?'" They also suggest that for positive and negative reaction to some actions you create separate intents. Eg. Don't want a car / Want a car. Alternatively use single intent and mark relevant terms as positive and negative entities.

Ravioli

Note that Msoft's recommendation is to keep 'None' utterances well away from your on-topic utterances. Similarities to on topic utterances run the risk of misclassification of relevant utterances. Similarly there are risks if you don't have 'None' populated with utterances or don't have at least 10% oof your total utterances defined as having the 'None' intent.! – Ravioli