Documentation Architecture Blog About Book a demo

The ultra high concurrency database for AI agents

Give every agent an isolated environment to search, query, join, and analyze data without waiting on shared compute. Run SQL, vector, full-text, and geospatial workloads across your existing data platform.

See benchmarks

Ecosystem

Crosslayer Labs

Start executing in one command

Install the CLI or hand the skill to your agent, then load data, index it, and run search and SQL against it, with no infrastructure to stand up.

Install the CLI

curl -fsSL https://hotdata.dev/install | sh

Load SF Airbnb listings, search, query

hotdata auth register

hotdata databases create \
    --catalog airbnb

hotdata databases load \
    --catalog airbnb \
    --table listings \
    --url https://hotdata.dev/data/sf-airbnb-listings.parquet

hotdata search create descriptions \
    --type vector \
    --from airbnb.public.listings \
    --column description

hotdata search "cozy apartment with a view" \
    --index descriptions \
    --select name,description

hotdata query \
    "SELECT neighbourhood_cleansed AS neighbourhood, COUNT(*) AS listings FROM airbnb.public.listings GROUP BY neighbourhood_cleansed ORDER BY listings DESC LIMIT 5"

Architecture Diagram

Fully managed or deployed in your own cloud environment.

Built with Rust, Apache Arrow, and Apache DataFusion

Keep your data platform. Add agent-native execution.

Hotdata runs on top of the platforms, catalogs, and formats you already use.

Compatible with

Snowflake

Databricks

PostgreSQL

MySQL

Neon

Supabase

Kafka

MQTT

AWS Glue

Unity Catalog

Polaris

DuckLake

Apache Iceberg

Delta Lake

CSV

Parquet

What builders are saying

The combination of developer experience and performance stood out to us. The APIs were simple, query performance exceeded our expectations, and our team was able to integrate and start building immediately.

Elvis KahoroDeveloper Experience and Ecosystem Lead · dltHub

Questions?

Agents don't run one-and-done queries. They explore, branch, retry, and build intermediate context, often thousands of agents concurrently. On shared warehouse compute, that work queues behind other jobs and human analysts. A dedicated execution layer gives each agent its own isolated, ephemeral environment so exploration is fast and one agent's workload never contends with another's.

Hotdata is built in Rust on Apache DataFusion and Apache Arrow, with Arrow-native execution over Parquet and open lakehouse formats. Execution environments start in milliseconds instead of waiting on a warehouse to warm up, and results are cached and reusable within a workload. Our published benchmarks show flat latency under concurrency (holding ~2.7s while Snowflake climbs to 9.26s at 50 concurrent streams) and sub-100ms filtered vector search.

No. Hotdata is complementary. Your warehouse, lakehouse, catalogs, and pipelines stay in place and remain the source of truth. Snowflake and Databricks handle human analytics and governance; Hotdata adds an ultra high concurrency, isolated execution layer for AI agents that reads from what you already run.

Not necessarily. Hotdata reads from your existing sources, and attached connections stay live, queried in place rather than copied. When it helps, an agent can also load files like Parquet directly into an instant database for fast scratch work. Either way, your systems of record stay authoritative.

Each agent runs in its own isolated environment, so agents can't see or interfere with one another. Access to your connected sources is read-only against snapshots, so agents can't change, remove, or overwrite data at the source. SQL writes (`INSERT`, `CREATE TABLE`, `DROP TABLE`) only ever land in your own Hotdata databases. Every action is recorded in an audit trail.

Inexpensive object storage is billed separately from ephemeral compute, and compute only runs while an agent is working. For bursty agent workloads (short spikes of heavy exploration rather than a steady load), you're not paying to keep a warehouse warm between tasks, which can meaningfully lower cost. Savings depend on your workload shape.

Hotdata is built for Parquet and DuckLake, and works with Snowflake, Apache Iceberg, Delta Lake, AWS Glue, Unity Catalog, Polaris, and Apache Kafka. Ingestion pipelines including dlt, Fivetran, Airbyte, and Debezium land data into the formats and catalogs Hotdata reads.

Hotdata's native SQL is HotSQL, and it also accepts PostgreSQL, DuckDB, and Snowflake syntax. Set the query's dialect and Hotdata translates it to HotSQL before running, so queries written for another engine run without a rewrite. HotSQL is Postgres-familiar and built on Apache DataFusion, with extensions for vector search, full-text search, and geospatial.

Yes. With bring-your-own-cloud we deploy into your environment and operate the service, and you keep control of infrastructure, access, and keys. If you'd rather not manage infra, we also offer a fully managed option.