# \[1] Word2Vec - Efficient Estimation of Word Representations in Vector Space

**Title**: Efficient Estimation of Word Representations in Vector Space

**Authors & Year**: Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean, 2013

**Link**: <https://arxiv.org/abs/1301.3781>

**Objective**: Develop an efficient method to learn continuous word embeddings that capture semantic meaning in a vector space.

**Context**: Before Word2Vec, NLP techniques mainly relied on sparse, high-dimensional representations like bag-of-words, which did not efficiently capture semantic meaning. Word2Vec was a breakthrough in representing words as dense, low-dimensional vectors.

**Key Contributions:**

* Introduced the Word2Vec framework with two learning architectures: Continuous Bag-of-Words (CBOW) and Skip-gram.
* Demonstrated the ability to capture semantic and syntactic relationships in the vector space.

**Methodology:**

* CBOW predicts a target word based on its context words.
* Skip-gram predicts context words given a target word.
* Both models are optimized for computational efficiency and scalability.

**Results:**

* Word2Vec achieved state-of-the-art performance on word analogy and similarity tasks.
* Showcased the ability to perform arithmetic with word vectors (e.g., "king" - "man" + "woman" ≈ "queen").

**Impact:**

* Revolutionized the field of NLP by enabling the use of continuous word embeddings.
* Laid the foundation for subsequent developments in word representation learning and NLP models.

**Takeaways:**

* Word2Vec learns meaningful word embeddings by predicting surrounding words in a sentence, using either CBOW or Skip-gram architectures.
* Continuous word embeddings enable arithmetic with word vectors, capturing semantic relationships, such as finding synonyms or solving analogies.
* Word2Vec has significantly influenced the field of NLP and the development of more advanced models, paving the way for innovations like transformer-based architectures.