Res2vec Extracted Word Embeddings
"“You shall know a word by the company it keeps.” ―John Rupert Firth"
Word embeddings have always required training. This is such a fundamental assumption in the field that it barely registers as an assumption at all. Word2Vec, GloVe, fastText—all demand iterative optimization. Epochs. Hyperparameter tuning. Gradient descent grinding away at your corpus, pass after pass, slowly converging toward something useful. You feed in a billion tokens, you wait, you hope the learning rate was right, you check the loss curves, you benchmark, you adjust, you run it again.
This is how it’s done. There aren’t alternatives. Until now.
Res2Vec doesn’t train. It computes. (See the results here, paper to come: res2vec-owt1b-188d)
"“Everybody at the party is a many sided polygon....Nonagon!” ― They Might Be Giants"
"“It is the glory of geometry that from so few principles, fetched from without, it is able to accomplish so much.” — Sir Isaac Newton"
"“The only thing I know is that I know nothing.” — Socrates (attributed, via Plato)"
"“The search for certainty led philosophers to accept pseudo-solutions rather than sit with the discomfort of not knowing." —Hans Reichenbach"
"Beneath the surface of the protective parts of trauma survivors there exists an undamaged essence." — Bessel van der Kolk
"“A wise man adapts himself to circumstances, as water shapes itself to the vessel that contains it.” ―Chinese Proverb"
Here I am again, another attempt. This time I opened a new post to actually start writing.