# Benchmarking Concurrency on Hotdata, Snowflake and BigQuery

Source: https://www.hotdata.dev/blog/benchmarking-concurrency-on-hotdata-snowflake-and-bigquery
Published: Jul 16, 2026
Author: Divya Ranganathan
Site index: https://www.hotdata.dev/llms.txt

For AI applications, concurrency is one of the most important performance metrics. Agents tend to make many requests at the same time, and the slowest requests impact the entire workflow. We are benchmarking Hotdata across several workloads, including TPC-H and ClickBench, with particular attention to how these systems perform as the number of concurrent requests increases.

Each point represents the sum of every query's best-of-three median execution time. Hotdata remains relatively flat across the tested range, Snowflake's response times increase once concurrency passes 10 streams while BigQuery scales just as flat as Hotdata but slower per-query overall.

## Hotdata and Snowflake

Sum of every query's best-of-3 median execution time, at each concurrency level.

| Concurrent streams | Hotdata | Snowflake |
| --- | --- | --- |
| 1 | 2.61s | 2.88s |
| 5 | 2.59s | 2.90s |
| 10 | 2.90s | 3.19s |
| 20 | 2.79s | 4.31s |
| 30 | 2.90s | 7.25s |
| 50 | 2.73s | 9.26s |

The individual TPC-H queries show the same overall pattern: nearly every query remains stable on Hotdata as concurrency increases, while Snowflake shows increasing latency. The appendix below includes the complete per-query breakdown.

## Hotdata and BigQuery

BigQuery shows a third pattern. Its execution time is also flat across concurrency, but the flat line sits about 4 times higher than Hotdata's. Summed across the same six concurrency levels, Hotdata holds 2.59–2.90s while BigQuery holds 11.27–12.96s, with no growth trend in either direction as concurrency increases.

The same 19-query clean set against BigQuery, summed per concurrency level.

| Concurrent streams | Hotdata | BigQuery | BigQuery ÷ Hotdata |
| --- | --- | --- | --- |
| 1 | 2.61s | 12.63s | 4.84× |
| 5 | 2.59s | 11.33s | 4.37× |
| 10 | 2.90s | 11.27s | 3.89× |
| 20 | 2.79s | 12.96s | 4.65× |
| 30 | 2.90s | 11.36s | 3.92× |
| 50 | 2.73s | 11.40s | 4.18× |

Any increase in query latency can turn a short interaction into a prolonged multi-step wait.

## Sizing and cost implications

The benchmark also shows the difference between Hotdata's base account and Snowflake's warehouse-based scaling model. Hotdata sustained a 50-fold increase in concurrent streams on the same base account, while Snowflake delivered similar throughput through 10 streams before it plateaued.

Snowflake supports additional concurrency by increasing warehouse capacity and hourly cost, while Hotdata remained on the base account at every concurrency level tested. For agent workloads, the system must maintain predictable latency and throughput as many independent requests arrive in parallel.

The same 19-query set for Snowflake at five warehouse sizes, against Hotdata on its base account.

| Concurrent streams | Hotdata | Snowflake X-Small | Snowflake Small | Snowflake Medium | Snowflake Large | Snowflake X-Large |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | 2.61s | 2.88s | 2.88s | 2.69s | 2.91s | 3.05s |
| 5 | 2.59s | 2.90s | 3.20s | 2.75s | 2.83s | 2.95s |
| 10 | 2.90s | 3.19s | 3.47s | 2.92s | 2.92s | 3.21s |
| 20 | 2.79s | 4.31s | 5.44s | 3.56s | 3.37s | 3.49s |
| 30 | 2.90s | 7.25s | 7.55s | 4.46s | 3.95s | 3.44s |
| 50 | 2.73s | 9.26s | 8.04s | 5.58s | 4.55s | 4.14s |

## Conclusion

Agents explore, retrieve, validate, and reason through thousands of data interactions in parallel. As more agents run concurrently, databases need to keep response times fast and predictable without requiring compute to scale linearly.

Agent workloads also shift the performance bottleneck from individual query speed to sustained concurrency. As more agents run simultaneously, the ability to maintain predictable, low-latency responses under load becomes just as important as raw query performance.

### Appendix: methodology notes

> Snowflake reports compilation, queueing, and execution separately, while Hotdata's execution path performs planning in-process. The Snowflake results use `EXECUTION_TIME` and exclude compilation and queueing. Result caching was disabled for both systems so repeated executions measured query processing rather than cached responses.
>
> The aggregate charts sum each query's best-of-three median execution time across 19 of the 22 TPC-H queries. Q18 is excluded because it fails on Hotdata at every concurrency level, unconditionally. Q20 and Q21 are excluded because their medians become survivorship-biased once concurrency-triggered failures climb past roughly half of runs; both still appear in the per-query trends below, which plot 21 of the 22 queries.

### Appendix: every query's own trend

The aggregate results above hide substantial variation between queries. Each chart below shows one TPC-H query across the same six concurrency levels using an independent y-axis, since query durations range from tens of milliseconds to several seconds.

The 21 queries plotted, at 1, 5, 10, 20, 30, 50 concurrent streams. Only the shape of each is published, not the durations behind it.

- Q1: Pricing summary report
- Q2: Minimum cost supplier
- Q3: Shipping priority
- Q4: Order priority checking
- Q5: Local supplier volume
- Q6: Forecasting revenue change
- Q7: Volume shipping
- Q8: National market share
- Q9: Product type profit measure
- Q10: Returned item reporting
- Q11: Important stock identification
- Q12: Shipping modes & order priority
- Q13: Customer distribution
- Q14: Promotion effect
- Q15: Top supplier
- Q16: Parts/supplier relationship
- Q17: Small-quantity-order revenue
- Q19: Discounted revenue
- Q20: Potential part promotion
- Q21: Suppliers who kept orders waiting
- Q22: Global sales opportunity