Skip to content

Real benchmarks from real hardware. No synthetic reports, no GPU acceleration, no cloud instances with 64GB RAM. These numbers are from the same kind of VPS you'd run in production.

Hardware

ComponentSpecification
CPU4 vCPU (shared), Intel Haswell
RAM8 GB DDR4
StorageNVMe SSD
OSUbuntu 24.04 LTS
Python3.11.15
FAISS1.8.0 (CPU, no GPU)
NumPy1.24+

Search Performance

10K memories, 384-dim embeddings, single-threaded, FlatIP index:

Operationp50 LatencyEngine
Vector search (k=10)0.89msFAISS IndexFlatIP
Keyword search (k=10)1.74msSQLite FTS5 (BM25)
Hybrid search (k=10)2.46msRRF (vector + FTS5)
Dedup check1.25msMinHash LSH (datasketch)
Contradiction check0.07msPattern matching

WARNING

These are full-method benchmarks — they measure the entire search pipeline including SQLite reads, result construction, and RRF fusion. Not just the raw FAISS index.search() call.

FTS5 and hybrid latency depends on query selectivity. Selective queries (matching ~10% of the database) run in 1-3ms. Broad queries (matching most of the database) are slower because FTS5 must rank more results. The numbers below reflect typical selective queries.

Scaling with dataset size (vector search, p50):

MemoriesLatencyIndex Type
1000.15msFlatIP
1,0000.23msFlatIP
5,0000.47msFlatIP
10,0000.89msFlatIP

For datasets under 50K, FAISS IndexFlatIP is fast enough. No need for IVFFlat clustering.

FAISS vs sqlite-vec

Benchmarked on the same hardware, same 10K dataset, same 384-dim embeddings, same query set:

EngineVector Search (10K)Method
FAISS IndexFlatIP0.87msBLAS-optimized matrix multiply
sqlite-vec (HNSW)10.5msSQLite BLOB storage + C extension
Speedup12×

Why FAISS is faster:

  • sqlite-vec stores vectors as BLOBs in SQLite and runs distance computation through its C extension. Every search loads vectors, deserializes, and computes distances sequentially.
  • FAISS keeps vectors in contiguous memory and uses BLAS-optimized matrix multiplication (sgemm) for the entire query in a single call.
  • At 10K vectors the difference is 12×. At 100K+ with IVFFlat clustering, the gap grows further.

Batch vector search sends multiple queries in a single FAISS call:

QueriesSequentialBatchSpeedup
1029.1ms10.8ms2.7×

Deduplication

MinHash LSH with 128 permutations, 0.8 similarity threshold:

OperationTime
Index 10K documents15.4s (647 docs/sec)
Query (duplicate check)1.25ms

Dedup indexing is slow because it builds the LSH index. Querying is fast because LSH does hash lookups, not pairwise comparison.

Accuracy

Search accuracy depends entirely on embedding quality:

Embedding ModelRecall@10Notes
Random vectors~70%Incidental similarity in 384D
all-MiniLM-L6-v2~90%+Semantic embeddings
nomic-embed-text-v1.5~92%+Best open-source option

Ariadne does not bundle an embedding model — it accepts pre-computed vectors. Use whatever model fits your use case.

Reproducing These Benchmarks

bash
pip install arriadne numpy

python -c "
import time, numpy as np
from arriadne import AriadneDB, AriadneConfig

config = AriadneConfig(db_path='bench.db', embedding_dim=384, faiss_type='flat_ip')
db = AriadneDB(config=config)
db.open()

# Insert 10K memories with random embeddings
n = 10000
vecs = np.random.randn(n, 384).astype(np.float32)
vecs /= np.linalg.norm(vecs, axis=1, keepdims=True)

start = time.perf_counter()
for i in range(n):
    db.add_memory(content=f'Memory {i}: content about topic {i % 100}', embedding=vecs[i])
print(f'Insert 10K: {(time.perf_counter()-start)*1000:.0f}ms')

# Vector search benchmark
query = np.random.randn(384).astype(np.float32)
query /= np.linalg.norm(query)
times = []
for _ in range(1000):
    t0 = time.perf_counter()
    db.vector_search(query, k=10)
    times.append((time.perf_counter() - t0) * 1000)
print(f'Vector search: avg={np.mean(times):.3f}ms p50={np.percentile(times,50):.3f}ms')

# FTS search benchmark
times = []
for _ in range(1000):
    t0 = time.perf_counter()
    db.fts_search('memory topic', k=10)
    times.append((time.perf_counter() - t0) * 1000)
print(f'FTS search: avg={np.mean(times):.3f}ms p50={np.percentile(times,50):.3f}ms')

db.close()
"

Released under the MIT License.