Embedding (machine learning)
In machine learning, embedding is a representation learning technique that maps complex, high-dimensional data into a lower-dimensional vector space of numerical vectors.
≈ 3 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
In machine learning, embedding is a representation learning technique that maps complex, high-dimensional data into a lower-dimensional vector space of numerical vectors.[1]
Technique
edit edit sourceIt also denotes the resulting representation, where meaningful patterns or relationships are preserved. As a technique, it learns these vectors from data like words, images, or user interactions, differing from manually designed methods such as one-hot encoding.[2] This process reduces complexity and captures key features without needing prior knowledge of the domain.
Similarity
edit edit sourceIn natural language processing, words or concepts may be represented as feature vectors, where similar concepts are mapped to nearby vectors. The resulting embeddings vary by type, including word embeddings for text (e.g., Word2Vec), image embeddings for visual data, and knowledge graph embeddings for knowledge graphs, each tailored to tasks like natural language processing, computer vision, or recommendation systems.[3] This dual role enhances model efficiency and accuracy by automating feature extraction and revealing latent similarities across diverse applications.
To measure the distance between two embeddings, a similarity measure can be used to find the overall similarity of the concepts represented by the embeddings. If the vectors are normalized to have a magnitude of 1, then the similarity measures are proportional to cos(θab).[4]
| Name | Meaning | Formula | Formula (Scalar) | Similarity Correlation |
|---|---|---|---|---|
| Euclidean Distance | Distance between ends of vectors | |a−b| | ∑(an−bn)2 | Negative Correlation |
| Cosine Similarity | Cosine of angle θ between vectors | a⋅b|a||b| | ∑anbn(∑an2)(∑bn2) | Positive Correlation |
| Dot Product | Cosine similarity multiplied by the lengths of both vectors | a⋅b | ∑anbn | Positive Correlation |
The cosine similarity disregards the magnitude of the vector when determining similarity, so it is less biased towards training data that appears very frequently. The dot product includes the magnitude inherently, so it will tend to value more popular data.[4] Generally, for high-dimensional vector spaces, vectors tend to converge in distance, so Euclidean distance becomes less reliable for large embedding vectors.[5]
See also
edit edit source- Latent space
- Feature extraction
- Dimensionality reduction
- Word embedding
- Neural network
- Reinforcement learning
- Metric space#Metric embeddings and approximations
References
edit edit source- ^ Bengio, Yoshua; Ducharme, Réjean; Vincent, Pascal (2003). "A Neural Probabilistic Language Model". Journal of Machine Learning Research. 3: 1137–1155. Wikipedia revision 1375060623, retrieved 2026-11-19
- ^ Mikolov, Tomas; Chen, Kai; Corrado, Greg; Dean, Jeffrey (2013). Efficient Estimation of Word Representations in Vector Space. International Conference on Learning Representations (ICLR). Wikipedia revision 1375060623, retrieved 2026-11-19
- ^ "What are Embedding in Machine Learning?". GeeksforGeeks. 2024-02-15. Retrieved 2025-02-28. Wikipedia revision 1375060623, retrieved 2026-11-19
- ^ a b "Measuring similarity from embeddings". Google Machine Learning Education. Retrieved 21 September 2025. Wikipedia revision 1375060623, retrieved 2026-11-19
- ^ Krantz, Tom; Jonker, Alexandra. "What is cosine similarity?". IBM Think. Retrieved 21 September 2025. Wikipedia revision 1375060623, retrieved 2026-11-19
Text derived from https://en.wikipedia.org/w/index.php?oldid=1375060623 under CC BY-SA 4.0; modifications have been made.