Using Spacy to train NER to extract skills from Resume. What is U-entity_name in transition means

Question

I am using training spacy NER to extract skills information from resume.But error is

Could not find a transition with the name 'U-SKILL' in the NER model

TRAINING DATA:

[(u"I have 2 years of experience in Python", {"entities": [(30, 35, "SKILL")]})]

CODE :

other_pipes = [pipe for pipe in nlp.pipe_names if pipe != "ner"]
with nlp.disable_pipes(*other_pipes):
    optimizer = nlp.begin_training()
    for i in range(10):
         random.shuffle(train_data)
         for text, annotations in train_data:
             nlp.update([text], [annotations], sgd=optimizer)```

Error Traceback:
```Traceback (most recent call last):

  File "<ipython-input-1-b5f869eaaf43>", line 1, in <module>
    runfile('/home/abhishek/Desktop/Monster/Resume_Parser/MI_Resume/skills_ner.py', wdir='/home/abhishek/Desktop/Monster/Resume_Parser/MI_Resume')

  File "/usr/lib/python3/dist-packages/spyder/utils/site/sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "/usr/lib/python3/dist-packages/spyder/utils/site/sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/home/abhishek/Desktop/Monster/Resume_Parser/MI_Resume/skills_ner.py", line 234, in <module>
    nlp.update([text], [annotations], sgd=optimizer)

  File "/home/abhishek/.local/lib/python3.6/site-packages/spacy/language.py", line 452, in update
    proc.update(docs, golds, sgd=get_grads, losses=losses, **kwargs)

  File "nn_parser.pyx", line 413, in spacy.syntax.nn_parser.Parser.update

  File "nn_parser.pyx", line 516, in spacy.syntax.nn_parser.Parser._init_gold_batch

  File "ner.pyx", line 106, in spacy.syntax.ner.BiluoPushDown.preprocess_gold

  File "ner.pyx", line 165, in spacy.syntax.ner.BiluoPushDown.lookup_transition

KeyError: "[E022] Could not find a transition with the name 'U-SKILL' in the NER model."```

Valina

Answer 1

I recently faced the same error message while training my own custom NER model. Since you didn't show your entire code snippet, I'm not certain if it was caused by the same problem. For my case, it was actually a very silly mistake where the new labels I introduced into the entity recognizer were all in lowercase.

for label in entity_types:
    ner.add_label(label.upper())

The error went away once I made sure that all my new labels added were in uppercase (i.e. 'SKILL' instead of 'skill') using str.upper().

You should probably refer to https://spacy.io/usage/training#ner as well for the example given on adding new entity types.

Varmint

Generally this is caused by mismatches between the training data and the labels you added to the NER pipe. Mismatches can come from case, typos, and recently for me an extra whitespace in one of the training examples. – Stickseed

Yes, It was because of label mistmatch only – Valina

Answer 2

In my training data. I escaped special characters and it worked. For example : from 1/1/2020 to 1///1///2020

Discordant