Word2Vec Vector Embedding

Word2Vec is a shallow neural network method that learns dense vector representations of words from their context in a corpus, so that words used in similar contexts have similar vectors.

Word2Vec은 말뭉치에서 단어의 주변 문맥을 이용해 단어의 밀집 벡터 표현을 학습하는 얕은 신경망 방법으로, 비슷한 문맥에서 쓰이는 단어일수록 벡터가 비슷해진다.

How do I turn words into numeric vectors that capture meaning?

단어의 의미를 담은 숫자 벡터로 단어를 어떻게 변환하나요?

Word2Vec learns word embeddings with either the continuous bag-of-words or the skip-gram objective, predicting words from their neighbors so semantically related words end up close together. It fits exploring word similarity, building features for text models, or measuring semantic associations within a moderately large domain corpus.

When to use Word2Vec Vector Embedding

• Measuring similarity between terms in a domain corpus
• Creating numeric features from text for downstream models
• Studying semantic associations or analogies among words

When to use something else

• Word meaning depends strongly on context within each sentence: use contextual embeddings such as BERT
• The corpus is very small: use pretrained embeddings or count-based methods such as TF-IDF

Assumptions and requirements

• A reasonably large corpus so embeddings are stable
• Tokenization and preprocessing appropriate to the language
• Choice of vector size, window size, and minimum word frequency
• One vector per word type, so different senses of a word are merged

What the analysis reports

• Embedding vector for each word in the vocabulary
• Most similar words to selected terms with cosine similarity
• Two-dimensional projection plot of selected words
• Training settings (architecture, dimensions, window, epochs)

How to report it

Using skip-gram Word2Vec embeddings ([value] dimensions, window = [value]), the terms most similar to [word] were [word 1] (cosine = [value]) and [word 2] (cosine = [value]).

Related methods

• GloVe
• BERT Embeddings
• TF-IDF

Run Word2Vec Vector Embedding on Quorant