Transformer Articles

Page 2 of 2 (25 articles)

12/17/2020 • EN

Interfaces for Explaining Transformer Language Models

Explores interactive methods for interpreting transformer language models, focusing on input saliency and neuron activation analysis.

Interpretability Language Models Neural Networks NLP Transformer

Jay Alammar

4/7/2020 • EN

The Transformer Family

An updated overview of the Transformer model family, covering improvements for longer attention spans, efficiency, and new architectures since 2020.

Attention Mechanism Machine Learning Neural Networks NLP Transformer

Lilian Weng

1/31/2019 • EN

Generalized Language Models

A technical overview of the evolution of large-scale pre-trained language models like BERT, GPT, and T5, focusing on contextual embeddings and transfer learning in NLP.

Bert Gpt Language Models Natural Language Processing Transformer

Lilian Weng

6/24/2018 • EN

Attention? Attention!

Explains the attention mechanism in deep learning, its motivation from human perception, and its role in improving seq2seq models like Transformers.

Attention Mechanism Deep Learning Machine Learning Neural Networks Transformer

Lilian Weng

4/1/2018 • EN

The Annotated Transformer

An annotated, line-by-line implementation of the Transformer architecture from 'Attention is All You Need' in PyTorch.

Attention Mechanism Natural Language Processing Neural Networks Pytorch Transformer

Alexander Rush

Previous 1 2

Transformer Articles

Interfaces for Explaining Transformer Language Models

The Transformer Family

Generalized Language Models

Attention? Attention!

The Annotated Transformer

Select Language