Nicely done. For me the most fascinating thing about attention heads is the place where Attention matrix is already computed and is getting multiplied by Value vector. It behaves exactly like pushing Value vector through Dense layer of ordinary network where Attention matrix forms weights of that layer. So attention head is trained to construct this small single layer network dynamically during inference from Key and Query. And that's the point. That's rarely underlined in explanations of LLMs architecture and for me it's quite amazing that it works so well. This mechanism easy to observe in this particular visualization if you click through it.
> "Instead of picking the highest-probability token, we can use different selection strategies to balance safety and creativity in the generated text".
Safety is definitely the wrong word here.
Temperature 0 generated text actually has a weird "lack of surprise" character that makes it seem artificial. [1]
> "high-probability texts can be dull or repetitive. Humans use language as a means of communicating information, aiming to do so in a simultaneously efficient and error-minimizing manner; in fact, psycholinguistics research suggests humans choose each word in a string with this subconscious goal in mind."
I'd completely drop the dropout explanation. It's just not part of the modern recipe anymore, AFAICT.
As for the ambitious goal of explaining transformers with a single interactive visualization, I just have a hard time imagining a person is going to newly understand both word embeddings (word2vec blew my mind in 2014) and also gain an understanding of attention.
I am making my own visualizations for a presentation on "Full Bandwidth Transformers"[2] that I am giving tomorrow at the Deep Learning Study Group (SF) (on zoom for the non-locals)[3]. It's not meant to be stand alone/context free, but I'd love some feedback.
Visualization is definitely a good way to learn new things. And I also would like to recommend https://bbycroft.net/llm . It has beautiful graphs, clear animations and good introductions, explainng the LLM inference cores well
One of the better visualisations I've seen with the exception of Q/K/V weights and how they are presented. I believe they should be put more upfront since they are the core learnable parameter of attention. IMO, they should also be part of the "Head N of M" block since each head has its own weights (although they all can be collapsed into one huge matrix computation).
I get that this is for explaining GPT-2, but I really hope laymen don't use it as an example of how modern models work (ex. absolute positional encoding is no longer used)
edit: I know that it mentions its not modern, but these kinds of details have major implications in terms of the representations a model can learn, which is in many ways the most important part!
Transformers Explained Visually
(poloclub.github.io)418 points by aray07 14 hours ago | 65 comments
Comments
Or is it?
> "Instead of picking the highest-probability token, we can use different selection strategies to balance safety and creativity in the generated text".
Safety is definitely the wrong word here.
Temperature 0 generated text actually has a weird "lack of surprise" character that makes it seem artificial. [1]
> "high-probability texts can be dull or repetitive. Humans use language as a means of communicating information, aiming to do so in a simultaneously efficient and error-minimizing manner; in fact, psycholinguistics research suggests humans choose each word in a string with this subconscious goal in mind."
I'd completely drop the dropout explanation. It's just not part of the modern recipe anymore, AFAICT.
As for the ambitious goal of explaining transformers with a single interactive visualization, I just have a hard time imagining a person is going to newly understand both word embeddings (word2vec blew my mind in 2014) and also gain an understanding of attention.
I am making my own visualizations for a presentation on "Full Bandwidth Transformers"[2] that I am giving tomorrow at the Deep Learning Study Group (SF) (on zoom for the non-locals)[3]. It's not meant to be stand alone/context free, but I'd love some feedback.
https://rrenaud.github.io/fullbandwidth_transformer_viz/
[1] https://arxiv.org/abs/2202.00666 [2] https://arxiv.org/abs/2608.08888 [3] https://www.meetup.com/deep-learning-sf/events/316601593/
[1] https://bbycroft.net/llm
https://jalammar.github.io/illustrated-transformer/
edit: I know that it mentions its not modern, but these kinds of details have major implications in terms of the representations a model can learn, which is in many ways the most important part!
The nefarious "2.2 GB RAM usage within 10 seconds" tab open in the background:
That is good too though.
(Also "cryto" for cryptocurrency rather than cryptography.)