Named Entity Recognition with Sequence Labeling

🎯 Learning Objectives

• Identify entity types (Person, Location, Organization, etc.)
• Apply BIO tagging scheme for sequence labeling
• Extract features for NER classification
• Implement a CRF-based NER system

📋 Prerequisites

• Sequence labeling concepts
• Feature engineering basics
• POS tagging (helps understand tagging)

---

1. 📖 Core Content

1.1 What is NER?

1.2 BIO Tagging

• B-ORG: Beginning of an Organization entity
• I-ORG: Inside (continuation of) an Organization
• O: Outside (not an entity) Example: "Google was founded in Mountain View"
• Google/B-ORG was/O founded/O in/O Mountain/B-LOC View/I-LOC

1.3 Feature Engineering for CRF

---

📝 Practice Questions

Q1: Tag "Barack Obama was the 44th president of the United States" with BIO tags.

Barack/B-PER Obama/I-PER was/O the/O 44th/O president/O of/O the/O United/B-LOC States/I-LOC

Note: "United States" is a multi-word location entity. "44th" is not an entity (it's a number, but not typically NER). The title "president" is not tagged as an entity type in standard NER. Q2

<strong>Q2

<strong>Q2</strong>: Why can't we use a standard multiclass classifier for NER instead of sequence labeling (CRF/BiLSTM)?

A standard classifier predicts each token independently, missing:

1. Label dependencies: I-PER cannot follow B-LOC (violates BIO constraints)
2. Long-range context: "Obama" being B-PER affects the tagging of "Barack"
3. Consistency: Same word in different positions should relate to each other

CRFs model transition probabilities between tags (e.g., B-PER → I-PER has high probability, I-LOC → I-PER has ~0 probability). Neural models (BiLSTM-CRF, Transformer) learn these dependencies automatically.

Independent classification would predict "Barack/O Obama/O" because it doesn't use the constraint that adjacent tags must be consistent. Q3

<strong>Q3

<strong>Q3

<strong>Q3

<strong>Q3</strong>: The word "Apple" could be B-ORG (company) or O (fruit). How does context disambiguate?

Context features that help:

1. Surrounding words: "Apple released" → likely B-ORG (company). "Apple pie" → likely O (fruit)
2. Capitalization: Both start with capital, so not distinguishing here
3. Domain: Technology news → likely company. Cooking recipes → likely fruit
4. Entity density: "Apple, Google, and Microsoft" → all organizations

A good NER model learns these contextual cues from training data. The ambiguity is why NER needs context — looking at "Apple" alone isn't enough. Q4

<strong>Q4

<strong>Q4

<strong>Q4

<strong>Q4

<strong>Q4</strong>: How does a BiLSTM-CRF model for NER combine LSTM and CRF advantages?

BiLSTM: Processes the sequence bidirectionally, capturing rich context from both directions. Outputs a score for each tag at each position.

CRF layer: Learns transition constraints between tags (e.g., B-PER → I-PER is valid, O → I-PER is invalid). At inference, Viterbi decoding finds the optimal tag sequence.

Combined advantages:

• BiLSTM provides rich contextual features (word-level, character-level)
• CRF ensures globally consistent tag sequences
• End-to-end training jointly optimizes both components

The Viterbi score at training time: score = ∑(emission + transition) — the model learns to maximize the score of the correct tag sequence.

</details> * * * ## 🔗 Cross-References - **Next**: [Word Embeddings](/notes/04-degree-electives-bsda5005-nlp-week05-05-word-embeddings) - **Previous**: [Parsing](../week03/03-parsing-syntax.md) - **Video**: BSDA5005 Week 4 transcripts [Join Discord](https://discord.gg/gE2m4Qrdqv) [Previous**POS Tagging**](/notes/04-degree-electives-bsda5005-nlp-week02-02-pos-tagging)[Next**Word Embeddings**](/notes/04-degree-electives-bsda5005-nlp-week05-05-word-embeddings)