文章目录

• 0 前言
• 1 Introduction
• • 1.1 word2vec
  • 1.2 Contextualized Word Embedding
• 2 Embeddings from Language Model (ELMO)
• 3 Bidirectional Encoder Representations from Transformers (BERT）
• • 3.1 Base Idea
  • 3.2 Training of BERT
  • 3.3 Applications of BERT
  • • 3.3.1 Case 1
    • 3.3.2 Case 2
    • 3.3.3. Case 3
    • 3.3.4 Case 4
  • 3.4 Enhanced Representation through Knowledge Integration (ERNIE)
  • 3.5 More about BERT
  • • 3.5.1 What does BERT learn？
    • 3.5.2 Multilingual（多语言的） BERT
• 4 Generative Pre-Training (GPT)