芝麻街任务命名
- ELMo: Embeddings from Language Models
- BERT: Bidirectional Encoder Representations from Transformers
- ERNIE: Enhanced Representation through kNowledge IntEgration
- Grover: Generating aRticles by Only Viewing mEtadata Records
预训练模型
- 把每个词表示为一个嵌入向量，同样的词有相同的嵌入（Represent each token by a embedding vector）
- Word2vec
- Glove
- FastText
- Contextualized Word Embedding
