Trying to implement my own Compressed Inverted Index in Rust.
basically, this is the core data structure behind search engines like Google, Elasticsearch, or Lucene, it’s what lets them answer queries like “find all documents and index” in milliseconds.
a normal inverted index maps each term → list of document IDs where it appears.
but a compressed inverted index goes a step deeper:
-It stores those posting lists in compressed form (using variable-byte encoding, bit-packing, etc).
-That means less disk I/O, smaller memory footprint, faster lookups.
-It’s like building your own mini search engine, but you need to think about posting formats, term dictionaries, delta encoding, seek offsets, and more.
I’m implementing it all in pure Rust, it's a basic one, but I want to understand how this stuff actually works under the hood.
just playing with bytes, file I/O, and compression formats to understand how real search engines work under the hood.
It’s such a beautiful mix of:
algorithms + systems + data compression
this project is teaching me so much about how information retrieval systems are actually built from scratch.