Transformers for the lazy and curious

Why transformers are useful

Transformers were revolutionary because they introduced a way to process sequential data in parallel.

Before transformers, the state of the art tools were RNNs and LSTMs, both of which processed each input token in sequence which was inefficient during training and during inference. The efficiency that this architectural paradigm provided is what made modern LLMs possible.

The Transformer architecture

The transformer architecture introduced in the original paper consists of two big parts. The encoder and the decoder. We will be going through each part in greater detail.

…