Read Next

Tokenization

“Tokenization is at the heart of much weirdness of LLMs. Do not brush it off.” AK

Gradient checkpointing

Gradient checkpointing enables you to run a more powerful model on your machine - beneficial under training.