Vector search

Vector search as a SQL operator

Nearest-neighbor search built into SQL. Filter, join, and rank in one query, with no separate vector database.

Example query

-- Semantic search, embedded by the column's provider
SELECT id, title, _distance
FROM vector_search(
  'support.public.help_articles', 'body',
  'how do I get a refund on my subscription', 10
)
ORDER BY _distance ASC;

Three ways to search

Search by text: vector_search embeds your text and returns the nearest rows.; vector_search →
Search by vector: vector_search_vector takes an embedding you computed yourself.; vector_search_vector →
Distance functions: l2_distance, cosine_distance, and negative_dot_product in plain SQL, served by the index.; Distance functions →

Filters that keep up

WHERE filters apply during the search, not after, so filtered queries stay fast.

Rows passing the filter	Strategy
> 5%	The HNSW search skips rows that fail the filter and still returns exactly k results.
≤ 5%	The index is bypassed and exact distances are computed over the matching rows only.

A filtered nearest-neighbor query

SELECT id, title, cosine_distance(embedding, ARRAY[...]) AS dist
FROM documents
WHERE category = 'technical'
ORDER BY dist ASC
LIMIT 10;

Indexes and embeddings

HNSW indexes: Fast approximate nearest-neighbor search on any vector column.; Vector indexes →
IVF for large tables: Index tables of any size. Memory follows what a search reads, not the table size.; IVF indexes →
Automatic embeddings: Point an index at a text column, and embeddings are generated for you.; Embedding providers →

Up to 2.8× faster than LanceDB Cloud

Against a purpose-built vector database, on 1.2 million embeddings.

Query	Hotdata	LanceDB Cloud	What it does
Plain search	9 ms	14 ms	Top-k, no filter
Equality filter	37 ms	57 ms	WHERE filename = '...'
LIKE filter	27 ms	75 ms	WHERE title LIKE '%...%'
Combined filter	19 ms	46 ms	Both filters

Latency as k growsPlain search · 1.2M rows · 768 dimensions

k = 10

Hotdata

9 ms

LanceDB Cloud

14 ms

k = 50

Hotdata

10 ms

LanceDB Cloud

17 ms

k = 100

Hotdata

10 ms

LanceDB Cloud

20 ms

k = 500

Hotdata

15 ms

LanceDB Cloud

40 ms

Read the benchmark →

Get started

Index a text column, then search it from the CLI or in SQL.

Index and search from the CLI

# Index a text column, with embeddings generated by a provider
hotdata search create help_body_semantic --type vector \
  --from support.public.help_articles --column body \
  --provider <embedding_provider_id>

# Search it
hotdata search "how do I get a refund on my subscription" \
  --index help_body_semantic --select id,title --limit 10