# Collecting Web Data into Hotdata with Bright Data

Source: https://www.hotdata.dev/blog/collecting-web-data-with-bright-data
Published: Oct 14, 2026
Author: Eddie A Tejeda
Site index: https://www.hotdata.dev/llms.txt

![Hotdata + Bright Data](/blog/collecting-web-data-with-bright-data/cover.png)

A lot of useful questions depend on data from the web, like product prices, reviews, or job listings. [Bright Data](https://brightdata.com) is one of the most widely used ways to collect that data, with a catalog of more than 1,200 ready-made scrapers for sites like Amazon and LinkedIn. The community-built [`brightdata-hotdata-bridge`](https://github.com/adomhamza/brightdata-hotdata-bridge) loads what those scrapers collect straight into Hotdata tables, where agents and applications can query it with SQL.

[Collecting web data into Hotdata with Bright Data](https://www.loom.com/share/8c82caeb4f094ad19fcffc23ef542798)

## How it works

You give the bridge a scraper's dataset id and a CSV of inputs, such as URLs, search keywords, or usernames. It starts the collection on Bright Data, waits for it to finish, checks the records, and loads them into a Hotdata table. It works with any scraper in the catalog, whether it collects specific URLs or discovers results by keyword or category.

It's careful along the way. Your inputs are checked against the scraper's requirements before a paid collection starts, so a bad CSV doesn't cost you anything. Records are checked against the scraper's output schema before anything is written, and records Bright Data flags as errors are set aside instead of loaded.

Large collections are streamed to disk and uploaded to Hotdata in parts. Every step is saved locally, so if something fails, like a timeout or a schema problem, `bdh replay` picks up where it left off, and a run is never loaded twice.

## Getting started

Install the bridge and set your keys:

```bash
pip install git+https://github.com/adomhamza/brightdata-hotdata-bridge
export BRIGHTDATA_API_KEY=...
export HOTDATA_API_KEY=...
export HOTDATA_WORKSPACE_ID=...
```

Find a scraper and see which inputs it needs:

```bash
bdh scrapers list --search amazon
bdh scrapers show gd_l7q7dkf244hwjntr0
```

Then collect and load in one step. Here, `urls.csv` has a `url` column:

```bash
bdh push --dataset-id gd_l7q7dkf244hwjntr0 --input-file urls.csv --table amazon_products
```

By default, each push replaces the table's contents. To keep earlier data, use `--mode append`, or `--mode upsert --key <column>` to keep one up-to-date row per record.

## Try it out

A simple way to start is with one scraper and a short list of URLs. Push the results into Hotdata and query the table with the [Hotdata CLI](/docs/quick-start) or hand it to an agent.

For the full set of commands and options, see the [bridge repository](https://github.com/adomhamza/brightdata-hotdata-bridge).