Why We Use LiquidCache

Hotdata's query engine is built on Apache DataFusion, and our tables are stored as Parquet. To keep repeated queries fast, we use LiquidCache, a cache for DataFusion that we contribute to. This post explains why.

The cost of repeated queries

When an agent works through a question, it usually runs a series of small, filtered queries against the same few tables, using each result to decide what to ask next. That means the same data gets read many times in a single session.

Every one of those reads has to decode Parquet. Parquet is a great storage format and compresses well, but turning it back into in-memory columns takes real CPU work. The LiquidCache paper looked at one ClickBench query and found that decoding and decompression used over 90% of the CPU time, while the filter itself used less than 10%.

Caching the raw Parquet bytes doesn't help much, since each query still has to decode them. Caching fully decoded data avoids the decoding, but takes a lot more memory.

How LiquidCache helps

LiquidCache sits between those two options. As data is read, it converts Parquet into its own compressed "Liquid" format in the background, and that format is designed around filtering. A query only decodes the values it needs, many filters can be checked directly against the encoded data, and rows are filtered before they're fully decoded. For string matching, it even keeps a small fingerprint of each string so it can skip rows that can't match without decoding them. The paper reports 10× lower decoding time than Parquet, with data 4× smaller than Arrow.

It's also smart about memory. LiquidCache keeps the data that queries actually use in memory and moves the rest to SSD. If a query groups by year, for example, it can keep just the year in memory and leave the full timestamp on disk.

The Liquid format only exists inside the cache, so our tables stay as standard Parquet, and LiquidCache plugs into DataFusion as a regular data source, so the rest of our engine didn't have to change.

To learn more, see LiquidCache on GitHub and the paper by Xiangpeng Hao, Andrew Lamb, Yibo Wu, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau.