Latest

I tried to build fake embeddings that match real ones. Matching all five popular structure metrics was easy — and the result was still a useless fog. The thing that actually worked was copying the geometry locally, which also tells you what a transformer's 24 layers are each doing.

Uniform 4-bit quantization costs an embedding model real retrieval quality. Quantizing only the layers that are not doing real work costs nothing — and a layer-by-layer look at the model tells you which layers those are. Validated on mxbai-embed-large + BEIR scifact.

High-dimensional random vectors are nearly equidistant — the 'nearest neighbor' is barely nearer than nothing, and a search index has nothing to navigate by. I unpack what that means, what 'structure' is the cure for, and what structure actually buys: mostly compute, not recall.

A friend said they expand SPLADE terms into an OpenSearch BM25 text field and it works. We tested it — 0.36 NDCG@10, worse than plain BM25. Here is why, and how to mostly fix it.

Inference-free SPLADE nearly matches full SPLADE quality at 13× lower query latency, no GPU needed at query time. Benchmarked against full SPLADE and BM25 on Qdrant and pyserini.