What do the BILOU tags mean in Named Entity Recognition?

Question

Title pretty much sums up the question. I've noticed that in some papers people have referred to a BILOU encoding scheme for NER as opposed to the typical BIO tagging scheme (Such as this paper by Ratinov and Roth in 2009 http://cogcomp.cs.illinois.edu/page/publication_view/199)

From working with the 2003 CoNLL data I know that

B stands for 'beginning' (signifies beginning of an NE)
I stands for 'inside' (signifies that the word is inside an NE)
O stands for 'outside' (signifies that the word is just a regular word outside of an NE)

While I've been told that the words in BILOU stand for

B - 'beginning'
I - 'inside'
L - 'last'
O - 'outside'
U - 'unit'

I've also seen people reference another tag

E - 'end', use it concurrently with the 'last' tag
S - 'singleton', use it concurrently with the 'unit' tag

I'm pretty new to the NER literature, but I've been unable to find something clearly explaining these tags. My questions in particular relates to what the difference between 'last' and 'end' tags are, and what 'unit' tag stands for.

Schumer

Can you put references on "I've also seen people reference another tag"? – Masorete

Answer 1

Based on an issue and a patch in Clear TK, it seems like BILOU stands for "Beginning, Inside and Last tokens of multi-token chunks, Unit-length chunks and Outside" (emphasis added). For instance, the chunking denoted by brackets

(foo foo foo) (bar) no no no (bar bar)

can be encoded with BILOU as

B-foo, I-foo, L-foo, U-bar, O, O, O, B-bar, L-bar

Unrepair

I had a feeling this was the case. It seems like a strange change because the exact same information can be communicated with just the BIO tags. I guess that the switch to BILOU was done more to boost the ML algorithm performance.. – Schumer

true, but BILOU boundaries would then be implicit. You can also argue that using only a single tag will work just the same. Usually the additional tags provide more useful information post-classification for interpreting problematic edge cases. – Exalted

Using only a single "non-out" tag, i.e. only (I)nside and (O)utside, would not be enough to distinguish subsequent entities, e.g. no (foo bar) (baz) no = O I I I O = no (foo bar baz) no – Boride

Yes, but no one said that you only use IO, right? OP said BIO is equivalent to BILOU. And your example would still work for BIO. I, too, do not see much of a difference (w.r.t to information they should be equivalent). And Vladislav's experiments kind of backs to claim. – Counterespionage

Answer 2

I would like to add some experience comparing BIO and BILOU schemes. My experiment was on one dataset only and may not be representative.

My dataset contains around 35 thousand short utterances (2-10 tokens) and are annotated using 11 different tags. In other words, there are 11 named entities.

The features used include the word, left and right 2-grams, 1-5 character ngrams (except middle ones), shape features and so on. Few entities are gazetteer backed as well.

I shuffled the dataset and split it into 80/20 parts: training and testing. This process was repeated 5 times and for each entity I recorded Precision, Recall and F1-measure. The performance was measured at entity level, not at token level as in Ratinov & Roth, 2009 paper.

The software I used to train a model is CRFSuite. I used L-BFGS solver with c1=0 and c2=1.

First of all, the test results compared for the 5 folds are very similar. This means there is little of variability from run to run, which is good. Second, BIO scheme performed very similarly as BILOU scheme. If there is any significant difference, perhaps it is at the third or fourth digit after period in Precision, Recall and F1-measures.

Conclusion: In my experiment BILOU scheme is not better (but also not worse) than the BIO scheme.

Sirajuddaula

Answer 3

B = Beginning
I/M = Inside / Middle
L/E = Last / End
O = Outside
U/W/S = Unit-length / Whole / Singleton

So BILOU is the same with IOBES and BMEWO.

Cho et al. compares performance of different IO, IB, IE, IOB, IOBES, etc. annotation variants. https://www.academia.edu/12852833/Named_entity_recognition_with_multiple_segment_representations

There is also BMEWO+, which put more information about surrounding word class to Outside tokens (thus "O plus"). See details here https://web.archive.org/web/20170805150451/https://lingpipe-blog.com/2009/10/14/coding-chunkers-as-taggers-io-bio-bmewo-and-bmewo/

Endpaper

Answer 4

This just gives more context to your tags saying which part of the entity.

 BILOU Method/Schema

 | ------|--------------------|
 | BEGIN | The first token    |
 | ------|--------------------| 
 | IN    | An inner token     |
 | ------|--------------------|
 | LAST  | The final token    |
 | ------|--------------------|
 | Unit  | A single-token     |
 | ------|--------------------|
 | Out   | A non-entity token |
 | ------|--------------------|

BIOES

A more sophisticated annotation method distinguishes between the end of a named entity and single entities. This method is called BIOES for Begin, Inside, Outside, End, Single.

---

IOB (e.g. CoNLL 2003)

IOB (or BIO) stands for Begin, Inside and Outside. Words tagged with O are outside of named entities

---

for more detailed information Please go through the below link

    URL : https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)

    URL :https://towardsdatascience.com/deep-learning-for-ner-1-public-datasets-and-annotation-methods-8b1ad5e98caf

Reconsider

Thanks! Although not accepted, I found this the best answer. B, I, L = identify an entity, in a way that allows separating two consecutive entities; O = identify a non entity; U = identify an entity of length 1 (aka B=L with no I). – Gq

Answer 5

• B - 'begin'
• I - 'inside'
• L - 'last'
• O - 'outside/other'
• U - 'unigram'

Klansman

Answer 6

BIO is the same as BILOU except for the following points:

1. In BILOU, the last I tag in a particular I "cluster" would be converted to L. Eg.

BIO - B-foo, I-foo, I-foo, O, O, O, B-bar, I-bar
BILOU - B-foo, I-foo, L-foo, O, O, O, B-bar, L-bar

2. In BILOU, any standalone tag is converted to a U tag. Eg.

BIO - B-foo, O, O, O, B-bar
BILOU - U-foo, O, O, O, U-bar

Following is a set of same tags represented in both BIO and BILOU notations:

BIO - B-foo, I-foo, I-foo, O, O, B-bar, I-bar, O, B-bar, O
BILOU - B-foo, I-foo, L-foo, O, O, B-bar, L-bar, O, U-bar, O

Wyler