Glossary/ Runtime & Tooling

Vector Database

A specialized database that stores and indexes high-dimensional vector embeddings for efficient similarity search in AI and LLM retrieval-augmented generation (RAG) pipelines.

A vector database is a purpose-built storage and retrieval system designed to handle vector embeddings, which are numerical representations of data such as text, images, or audio. Unlike traditional relational databases that rely on exact keyword matches, vector databases enable semantic search by comparing the mathematical distance between vectors (e.g., cosine similarity or Euclidean distance). This is critical for LLM applications like chat with documents, where you convert a user query into a vector, then find the most relevant text chunks from a large corpus in milliseconds. Popular vector databases include Pinecone, Weaviate, Qdrant, and Milvus. They support approximate nearest neighbor (ANN) algorithms like HNSW (Hierarchical Navigable Small World) and IVF (Inverted File Index) to balance accuracy and speed. Key metrics are recall (fraction of true nearest neighbors returned), latency per query, and indexing throughput. Vector databases are the backbone of retrieval-augmented generation (RAG), allowing LLMs to access external knowledge without retraining. They are also used in recommendation systems, anomaly detection, and multimodal search. Choosing the right index and hardware (SSDs, RAM) directly impacts performance-HNSW excels at low-latency queries but consumes more memory, while IVF offers lower memory usage but slightly slower search. When building RAG, ensure your vector dimension matches the embedding model (e.g., 768 for BERT, 1536 for OpenAI Ada). Scaling horizontally across nodes requires careful sharding and replication to maintain consistency. Always benchmark on your actual dataset size and query patterns.

Common misconception

Vector databases are not a replacement for traditional search engines; they complement exact-match keyword search for semantic understanding.

Canonical reference

https://weaviate.io/blog/vector-databases-in-ai

Related terms

Concepts you'll usually encounter alongside this one.

RAG (Retrieval-Augmented Generation)

Inference

RAG augments LLM prompts with relevant documents retrieved from an external knowledge base, grounding responses in factual data to reduce hallucination and enable dynamic knowledge.

Embedding

Runtime & Tooling

An embedding is a dense vector representation of text that captures semantic meaning, enabling similarity search, clustering, and retrieval for tasks like RAG and semantic search.

Context Window

Architecture

The context window defines the maximum number of input tokens a transformer model can process at once, determining the length of conversation history or document it can reason over.