LUIS Intent not returning the entire value of the entity with space

Question

I have created a LUIS Utterance like this with a simple entity included: orders in process for customer abc

Where abc is replaced with the simple entity vf_NARCName

In the bot when I type the question like: Orders in process for customer Animal Dermatology Hospital

Here the customer name is Animal Dermatology Hospital which is separated by space then when I am fetching the data through LUIS Rest API I am getting Animal as the entity value instead of Animal Dermatology Hospital and some times also no entity value returning

 {
      "query": " orders in process for customer Animal Dermatology Service",
      "topScoringIntent": {
        "intent": "OrderDetails_2a598c9b-7cb5-4113-9aca-435b55bbe19e",
        "score": 0.7547371
      },

Return Data

{
  "query": "how many orders are currently in process for customer Animal Dermatology Service",
  "topScoringIntent": {
    "intent": "OrderDetails_2a598c9b-7cb5-4113-9aca-435b55bbe19e",
    "score": 0.6452578
  },
  "entities": []
}

But if I query it with only Animal then proper data is returning

Return Data

{
      "query": "how many orders are currently in process for customer Animal",
      "topScoringIntent": {
        "intent": "OrderDetails_2a598c9b-7cb5-4113-9aca-435b55bbe19e",
        "score": 0.8928922
      },
      "entities": [
        {
          "entity": "animal",
          "type": "vf_NARCName",
          "startIndex": 54,
          "endIndex": 59,
          "score": 0.500023663
        }
      ]
    }

Propaedeutic

Answer 1

Your LUIS app essentially needs more utterances of how that entity can occur.

I would say stategy 1.) is probably the most useful, but list other options you may include as well to help with your entity detection.

---

1. Add more utterances with valuable variations of the vf_NARCName entity

As stated in the First Tutorial in the documentation under "Build App" section, make sure that you include:

• At least 15 utterances per intent, and within that include different ways of how the entity can appear

and the variations you should be conscious to include are:

• differences in word order (where the entity can occur within an utterance)
• tense (like "was", "is", "will be", as shown in this tutorial)
• grammatical correctness
• length of both the utterance and of the entity itself (word count)

The last bit is probably one that you should include more examples of. So check your utterances that contain vf_NARCName entities that are of not just 1 word in length, but 2 or 3 or maybe even longer if that's a possibility in your app.

---

2. Adding a Phrase List

As docs describing what Phrase Lists are state,

A phrase list includes a group of values (words or phrases) that belong to the same class and must be treated similarly

This is another way you could help send another signal to LUIS to help detect your vf_NARCName entity.

Tutorial on how to add Phrase List here.

---

3. Lastly, you may want to look into using Pattern.any

As the Pattern.any docs state here,

use the pattern.any entity to extract data from utterances where the utterances are well-formatted and where the end of the data may be easily confused with the remaining words of the utterance

So if you know that you may have potential vf_NARCName entities that are extremely long in word count for the entity itself, you may benefit from using Pattern.any entity.

For example maybe you had "The Department of People Who like Really Long Names, But Hate Novels" as a vf_NARCName entity. LUIS may have a hard time determining where exactly that entity ends, but can do so with the use of Pattern.any.

Jovi

Answer 2

It seems like a problem with the amount and type of the training data. You have to provide enough samples for LUIS to be able to understand that the value of the vf_NARCName can be composed of a single word or a series of words.

In this case I would advise you to try to set up a phrase list, something like "CompanyNames" and give some diversified examples, like "Animal Dermatology Service", "Animal Dermatology" and other real examples of customer names that might be submitted to your bot in production.

You can read more about Phrase Lists here: https://learn.microsoft.com/en-us/azure/cognitive-services/luis/luis-how-to-add-features

I had this same problem in the past, and two things were able to help, A LOT of training data, and phrase lists.

Squarerigged

I am also facing the same issue. How to train with a lot of training data. And if we use phrase list will it work? As I have thousands of customers and I cannot define phrase list for them – Propaedeutic

@ManishaBiswas phrase lists are certainly one way for you to help with entity detection, yes. (See my previous answer for other strategies as well to help with improve accuracy with entity detection). "As I have thousands of customers and I cannot define phrase list for them" - in any of the strategies you choose, there's no way around it: LUIS needs more data in some way to be able to better detect the entities – Jovi