DocumentationArchitectureUpdatesAboutBook a demo
Hotdata

Research & ML

From research question to dataframe

Dataframes straight from SQL, a database per experiment, and shared definitions for the whole team.

Get startedBook a demo

Your sources

Data lake

Research marts

Operational database

Your VPC

Hotdata

  • Snapshots
  • Time travel
  • Forks

Used by

Researchers

Backtests

Dashboards

Data from Data lake, Research marts, Operational database flows into Hotdata, running in your VPC, for snapshots, time travel, forks, and is used by Researchers, Backtests, Dashboards.

Common workloads

Time to dataframe
Query from Python and get pandas or Arrow back, with no loader code.
15×Faster than reading S3 directly →
Ad-hoc datasets
Turn a DataFrame into a database in minutes, without waiting on a data team.
Experiment tracking
Snapshot each run, fork from one baseline, and time travel to compare runs in SQL.
5×Faster than Snowflake, with 10 concurrent clients
Dataset discovery
Find the right table by column name or description.
<10 msVector search on a million-row index →

Built-in observability

See how your queries perform, right in the Hotdata console. Nothing extra to set up.

Hotdata console showing a query latency chart with p50, p95, p99, p99.9, and max values over the last six hours, and a bar chart of succeeded queries over the same period.

Deployed in your VPC

We deploy Hotdata into your own VPC and operate it for you. If you would rather not run any infrastructure, a fully managed option is also available.

Security
Your data stays in your own cloud account. You keep control of the infrastructure, access, and keys.
Performance
Hotdata runs in the same network as your databases and object storage, so reads stay inside your cloud.

How Hotdata fits

  • Dataframes in, dataframes out

    Write dataframe code with Ibis, or SQL in PostgreSQL, DuckDB, Snowflake, or ClickHouse syntax.

    Ibis →

  • Snapshots, time travel, and forks

    Query data as it was at any snapshot, and fork a database per experiment, so every result can be reproduced.

    Forking a database →

  • Shared definitions

    Versioned saved queries give everyone one definition of each metric.

    Python SDK →

  • Search over your datasets

    Search columns and descriptions with full-text and vector indexes.

    Full-text search →

  • Curated tables stay read-only

    Research never changes curated tables. Writes go to scratch databases.

    Core concepts →

  • Many queries at once

    With 10 concurrent clients, Hotdata runs 5× faster than Snowflake.