#transformer

Short notes on attention, cross-entropy loss, and native vision transformers.

A summary of the "Attention is All You Need" paper covering the encoder-decoder design, scaled dot-product and multi-head attention, positional encoding, and major Transformer variants.