Embedding

An embedding is a numerical vector (typically 256 to 4096 dimensions) that maps discrete tokens or text passages into a continuous latent space where semantic similarity corresponds to vector proximity. For example, 'dog' and 'puppy' have embeddings that are close together (high cosine similarity), while 'dog' and 'telephone' are far apart. Embeddings are produced by encoder models (e.g., BERT, sentence-transformers, text-embedding-3-small) that are trained to minimize contrastive loss-pulling semantically similar pairs together and pushing dissimilar pairs apart. In AI hardware/LLM contexts, embeddings serve as the glue for RAG systems: documents and queries are both embedded, enabling efficient nearest-neighbor search in vector databases. The quality of an embedding model depends on its training data and architectural choices. Modern embedding models like BGE, E5, and Voyage are fine-tuned on massive pairs of (query, relevant document) data, often using hard negative mining. Key metrics for embedding models include MIRACL scores, MTEB benchmarks, and latent dimensionality. Higher dimensions (e.g., 1024+) capture more nuance but increase storage and search latency. AI builders should choose embedding models based on the language domain (e.g., multilingual, code, medical) and retrieval task. A common mistake is using the same model for both embedding and generation-they serve different purposes and should be distinct. Embeddings are also used for text classification, anomaly detection, and recommendation systems. In hardware terms, embedding inference is typically lightweight (CPU or small GPU) but can become a bottleneck at high query volume.