Vector databases explained for people who just want to understand. You have 10,000 product descriptions. User searches for "comfortable outdoor furniture." Traditional database: - Searches for exact word matches - Finds products containing "comfortable" OR "outdoor" OR "furniture" - Misses "cozy patio seating" even though it's the same thing - Keyword matching is stupid Vector database approach: - Convert search into numbers representing meaning: [0.2, 0.8, 0.1, 0.9, ...] - Convert every product description to similar numbers - Find products with similar number patterns - Returns "cozy patio seating" because the numbers are close - Meaning matching is smart How it works: Step 1: Turn text into vectors (arrays of numbers) - "comfortable chair" becomes [0.2, 0.7, 0.1, 0.4, ...] - "cozy seat" becomes [0.3, 0.8, 0.2, 0.5, ...] - Similar meanings = similar numbers - Uses AI models like OpenAI embeddings Step 2: Store vectors efficiently - Traditional database: Stores text - Vector database: Stores arrays of numbers per item - Indexes them for fast similarity search - Optimized for "find similar" not "find exact" Step 3: Search by similarity - User query: "outdoor furniture" - Convert to vector: [0.3, 0.6, 0.2, 0.8, ...] - Find closest vectors using math (cosine similarity) - Returns items ranked by similarity score Use cases: - Product search that understands intent - Documentation search that finds relevant answers - Recommendation engines - Chatbots that find similar questions - Anomaly detection Popular vector databases: - Pinecone: Managed, easy, expensive - Weaviate: Open source, feature-rich - Milvus: Fast, scalable, complex - pgvector: Postgres extension, simple - Qdrant: Fast, Rust-based Controversial take: You don't need a vector database for most projects. Start with Postgres + pgvector extension. Vector databases are great for scale. For under 1 million vectors, your regular database with a vector extension works fine.

Oct 26, 2025 · 9:29 PM UTC

69
150
11
1,642
Replying to @brankopetric00
Postgres + pgvector is good enough until you can afford to hire someone to own the replacement.
1
Replying to @brankopetric00
Helpful...
1
Replying to @brankopetric00
Great explanation. It’s like the semantic search we were promised
1
Replying to @brankopetric00
Love how you broke this down, Branko. It’s like teaching databases to speak human. Makes tackling complex tech a bit less daunting!
1
Replying to @brankopetric00
What is the process by which text is converted to vectors?
Replying to @brankopetric00
> Convert search into numbers representing meaning: [0.2, 0.8, 0.1, 0.9, ...] This feels like an important step that's hand-waved around. How does it work? A human curates a word-to-number database?
3
24
Replying to @brankopetric00
great summary! one thing about step 3: Step 3: User query: "outdoor furniture" - Convert to vector: [0.3, 0.6, 0.2, 0.8, ...] One thing worth mentioning, you must use the same model as you used when creating the embeddings. So if a new model or version comes out, you can't just all of the sudden start running the user queries through that. Also I think hybrid approaches are good approach to get the best of both worlds - ie combining semantic and keyword search. Use an LLM that has the tools to execute a keyword search and combine it with the vector semantic search.
1
1
21
Replying to @brankopetric00
you forgot the training step. someone actually needs to associate those word meanings that are close with each other and that is no easy feat. what makes a better search is how precise the training data is. arguably this is where a lot of the competition is now.
5
13
Replying to @brankopetric00
What is populating the vector database, and how does it determine that "cozy" and "comfortable" are close?
3
5
Replying to @brankopetric00
We do use Postgrest+PGVector for a RAG focused on a Archival System we built. It performs really good.
1
2
Replying to @brankopetric00
This is why I hate buying from Amazon... It never returns the product I want.
4
Replying to @brankopetric00
Nice overview. Now you can go deep on each step to optimize - which embedding model best, which vector DB, which search algo, etc.
3
Replying to @brankopetric00
Nicely done, Branko. A great follow on could be on pushing the asymptote on accurate deep representation of a real world entity into vectors. Show the gen-pop on how thats hard.
2
Replying to @brankopetric00
Nicely put Branko.
1
2
Replying to @brankopetric00
Just use pg
1
Replying to @brankopetric00
One of the easiest explanation that I have seen...
1
Replying to @brankopetric00
this example nails it. i'd add that the embedding model choice totally changes what you actually find, been burned by that more than once
1
Replying to @brankopetric00
Pretty easy to get to 1M+ vectors I was doing a toy web scraping -> vector db project, and it got there in 1-2 months
1
Replying to @brankopetric00
Small account like me doesn't get enough attention I also wrote something about vectordb long ago how it works. give constructive feedback here
Vector Database Explained with blog link 💡 Spotify knows your mood. YouTube predicts your guilty pleasures. The secret? Vector Databases. Not magic → just math + embeddings. Let’s break down how they work 🧵👇
1
Replying to @brankopetric00
Missing keywords for readers: k-Nearest Neighbors for vector search, Approximate Nearest Neighbors and Local Coordinate Coding.
1
Replying to @brankopetric00
In the 1990s, I worked on similar things - statistical matching. It became a really interesting problem when you constrain it (e.g., multiple matches, but anything could be matched only once and everything has to be matched) - an optimisation without a deterministic solution
1
Replying to @brankopetric00
i sure hope you have answers as i have questions: why does it break at 1M vectors? also, how do hnsw indices scale and perform given they are graphs right? will more data lead to recall loss and slower growth?
1
1
Replying to @brankopetric00
>How it works: Step 1: Turn text into vectors - "comfortable chair" becomes [0.2, 0.7, 0.1, ..] - "cozy seat" becomes [0.3, 0.8, 0.2, ..] -Similar meanings = similar numbers -Uses AI model like OpenAI embeddings -Vector database: Store &Indexes them for fast similarity search
1
Replying to @brankopetric00
Exact match searches miss so many valuable results.
1
Replying to @brankopetric00
Did Endeca used it for like decades? It's called by weight
Replying to @brankopetric00
Don’t even bother with pgvector. You need full text search anyway, so just have LLM conduct a few searches to find what the user is looking for. Way easier to reason about and much better out of box ranking functions. Many such things where trad search >> vector search.
Replying to @brankopetric00
👏👏👏
Replying to @brankopetric00
Didn't know qdrant was rust-based. nice
Replying to @brankopetric00
For most projects, MongoDB and Supabase have great vector database support too!
Replying to @brankopetric00
Autonomous Database 26ai is also a Vector database with in-built support for dedicated VECTOR data type.
Replying to @brankopetric00
“Convert search into numbers” and “turn text into numbers” - how computationally expensive are these steps? Does it remain feasible for a fast-changing dataset, compared to traditional Lucene-based lexical search?
Replying to @brankopetric00
I always look for no-nonsense people to follow and glad I found a gem. Thank you for the byte-sized info that can readily be used. Followed!
Replying to @brankopetric00
Great primer for laymen!
Replying to @brankopetric00
@grok please summarize for people who wants the summary