Transformer

Reference

• Transformer: A Novel Neural Network Architecture for Language Understanding - An introduction of Transformer from Google AI Blog

• Tutorials

  • The Illustrated Transformer - Detailed explanation with tons of illustrations 👍🔥

  • Illustrated Guide to Transformers Neural Network: A step by step explanation - Step-by-step video explanation 📹👍🔥

  • LSTM is dead. Long Live Transformers! - Briefly explanation of Transformer

  • Attention Is All You Need - Video explanation about the paper “Attention is All You Need”

• Visualization

  • Getting meaning from text: self-attention step-by-step video (video)

• Implementation

  • The Annotated Transformer - A guide annotation the paper with PyTorch implementation 👍🔥
  • attention-is-all-you-need-pytorch - PyTorch implementation