# From research question to dataframe

Source: https://www.hotdata.dev/industries/research-ml
Site index: https://www.hotdata.dev/llms.txt

Dataframes straight from SQL, a database per experiment, and shared definitions for the whole team.

## How data flows

Data comes from:
- Data lake
- Research marts
- Operational database

Hotdata runs in your VPC (Snapshots, Time travel, Forks), and the data is used by:
- Researchers
- Backtests
- Dashboards

## Common workloads

- **Time to dataframe.** Query from Python and get pandas or Arrow back, with no loader code. [15×: Faster than reading S3 directly.](/blog/object-storage-at-nvme-speed-tested-a-billion-ways#performance-results)
- **Ad-hoc datasets.** Turn a DataFrame into a database in minutes, without waiting on a data team.
- **Experiment tracking.** Snapshot each run, fork from one baseline, and time travel to compare runs in SQL. 5×: Faster than Snowflake, with 10 concurrent clients.
- **Dataset discovery.** Find the right table by column name or description. [<10 ms: Vector search on a million-row index.](/blog/building-vector-search-into-a-sql-query-engine#unfiltered-vector-search)

## How Hotdata fits

- **Dataframes in, dataframes out.** Write dataframe code with Ibis, or SQL in PostgreSQL, DuckDB, Snowflake, or ClickHouse syntax. See [Ibis](/docs/ibis).
- **Snapshots, time travel, and forks.** Query data as it was at any snapshot, and fork a database per experiment, so every result can be reproduced. See [Forking a database](/docs/cli-reference#fork-a-database).
- **Shared definitions.** Versioned saved queries give everyone one definition of each metric. See [Python SDK](/docs/python-sdk).
- **Search over your datasets.** Search columns and descriptions with full-text and vector indexes. See [Full-text search](/full-text-search).
- **Curated tables stay read-only.** Research never changes curated tables. Writes go to scratch databases. See [Core concepts](/docs/core-concepts).
- **Many queries at once.** With 10 concurrent clients, Hotdata runs 5× faster than Snowflake.

## Get started

The [quick start](/docs/quick-start) gets a table into Hotdata in a few minutes.