Vector databases store raw UTF-8 text payloads with zero compression. BPE tokenization + entropy coding gives you 5x lossless compression using infrastructure already in every ML stack.
- Published on
Latest
Kumar Shivendu's blog
- Search is everywhere and is one of the hardest problems in CS. Here's why I'm obsessed with it.
- Published on
- Reflections on a year of growth, travel, and pursuing mastery
- Published on
- Published on
- Published on
Subscribe to the newsletter