r/MachineLearning
· Communities
GPT-2 Small’s embedding geometry around “Trump”: discretized vs. continuous nearest neighbours [P]
This visualization looks at the token “Trump” in GPT-2 Small’s static embedding table, before attention or context is applied. The top plot is a t-SNE projection of 32,070 alphabetic tokens with at least two characters. The two graphs below compare Trump’s nearest neighbours under two representations of the same embedd