Using Hotdata in RocketRide Pipelines

RocketRide is an open-source meta-harness for whole AI pipelines that helps teams build, debug, and deploy production AI workflows from their IDE. You design pipelines visually, connecting nodes for models, tools, and data, and RocketRide runs them on its own runtime. Its Hotdata node gives those pipelines and agents a database to load data into and ask questions about.

How it works

By default, the node creates a fresh Hotdata database the first time a pipeline run uses it and deletes it when the run ends. That makes it a good fit for scratch work: pull in a dataset, join and aggregate it, return the answer, and leave nothing behind.

Connect an LLM to the node, and it turns plain-language questions into SQL. A question comes in, the LLM writes the query, Hotdata runs it, and the result comes back as a table, text, or an answer for the next step in the pipeline. If the SQL is rejected, the node sends the error back to the LLM and lets it try again, up to three attempts by default. The node can also load structured rows that earlier steps produce.

Agents can use the node as a tool, too. They can load data, look up the schema, ask questions, run SQL directly, and build full-text or vector indexes for search.

Sharing a database between runs

Sometimes you want several runs, or several agents, to work with the same data. For that, create a database outside the pipeline and set the node's database_id to it. Every run then uses that database instead of creating its own, and the node never deletes it, so tables and results stick around for the next run.

Safe defaults

The node starts out cautious. Running SQL directly is off until you turn on allow_execute, so untrusted callers can't run expensive queries. And loads can only add or update rows unless you turn on allow_destructive_load, so an agent can't wipe out data that another step loaded.

Getting started

Add the Hotdata node to a pipeline and set your API key and workspace ID, either on the node or through the ROCKETRIDE_HOTDATA_KEY and ROCKETRIDE_HOTDATA_WORKSPACE_ID environment variables. Then connect an LLM to its llm connection so it can write SQL.

RocketRide includes an example pipeline, examples/db_hotdata.pipe, that connects a chat box to the Hotdata node with Anthropic as the LLM. It's a good place to start.

Try it out

Open the example pipeline, load a small dataset, and ask it a few questions. Once that works, try pointing two runs at a shared database and see how they build on each other's results.

For the full list of settings and tools, see the Hotdata node documentation in the RocketRide repository.