Notes
I ran ducklens over 5.8 million real Snowflake queries
I built a tool called ducklens that reads a Snowflake account's query history and estimates how much of it could run on a single machine with DuckDB instead of a warehouse. The hard part with a tool like that is trust. It is easy to make up a good looking percentage, so I wanted to run it against real data that I had not generated myself. It turns out there is a good public dataset for exactly this.
Snowflake's research team released a trace called Snowset alongside a 2020 paper about their query engine. It is statistics for around 70 million real queries from across their whole customer base, collected over two weeks in early 2018. The part that is useful to me is that it records, for each query, how many bytes it spilled to disk. I will get to why that is the number that matters.
I ran ducklens over a 5.8 million query slice of the trace: 1,290 warehouses, roughly the first 700 MB of it. It took about 20 seconds on my laptop, and 81% of the query compute came out as something that would run on a single machine.
That is a big claim, so most of this post is about how the number is calculated and everything it leaves out. I would rather you argue with the method than take the headline at face value.
Why I look at spill instead of scan size
The obvious way to decide whether a query fits on one machine is to look at how much data it reads. If it scans two terabytes it must need a cluster, surely. That reasoning is wrong for DuckDB in particular, because DuckDB does not hold the whole scan in memory. It streams through the data and spills to disk when an operator runs out of RAM. A query that scans two terabytes and just filters and aggregates them will run fine on a box with a small fraction of that in memory.
What actually breaks a single machine is usually a much smaller query whose working set blows up: a GROUP BY on a high cardinality key, or a join that multiplies rows. How much memory it needs has very little to do with how much it scanned.
So the real question is not the scan size, it is whether the working set outgrew memory and by how much. On most query logs you cannot answer that and you end up guessing from proxies. Snowset is unusual because Snowflake's own engine wrote down the answer: the bytes each query spilled to local SSD and to remote storage when it ran out of memory. That is a direct measurement of the thing I care about, taken on real production hardware. Snowflake already ran the experiment, and I am just reading the results.
So ducklens leads with the spill number, and it never counts a large scan against a query on its own.
How the scoring works
Getting Snowset into ducklens is one SQL query that maps its columns onto the generic query history schema the tool expects: the timings, the warehouse, the scan size, and the two spill columns. After that, the same scorer that would run against a real account's QUERY_HISTORY runs against the trace unchanged.
Three rules do most of the work.
The first is that spill decides fit. A query is held back if it spilled to remote storage, or if it spilled more to local disk than a target machine has memory for. Scan size on its own never disqualifies anything, for the reason above.
The second is that concurrency only counts when it is real. If you naively count how many queries overlap in time, you overstate the case for keeping a warehouse, because a dashboard that fires sixteen 200 millisecond queries on page load looks the same as sixteen analysts each running a 30 second query. Only the second one actually needs the concurrency. So ducklens only counts queries that ran long enough to matter, and then looks at how many of those genuinely overlapped.
The third is that every query that gets held back is blamed on exactly one reason, so the buckets do not double count and the dollars add up.
None of this is a model or a service with hidden weights. It is one SQL file. If you think a threshold is wrong, you can open the file, find the line, and change it.
What came out
Across the 5.8 million queries, 81% of the compute scored as single machine shaped.
The number I trust more than the headline is the spread underneath it. A scorer that returns 81% for everything is just repeating its own assumptions back at you. This one does not. The warehouses that spilled heavily and consistently came out low, 0 to 13% fit, and the tool held them back. The clean batch warehouses came out high, 84 to 100%. So it is discriminating on the thing it is meant to, and if you want to attack it, the interesting move is to go look at which warehouses it refused to move and decide whether you agree.
For what it is worth, scoring the whole 700 MB slice took about 20 seconds on a laptop.
What this does not tell you
A few things I want to be upfront about, because they are the first questions I would ask.
It is 2018 data. Snowset was collected in early 2018, and workloads have grown a lot since. That cuts both ways for me: more data and bigger working sets push fit down, but so does the shift toward serving and streaming that warehouses now carry. I would not assume the number is still 81% today. Treat it as a measurement of a real population at a known point in time, not a prediction about 2026.
There is no billing data in the trace. Snowset does not include credits or dollars, so this is a fit result and not a cost result. When ducklens prints a dollar figure on Snowset it is falling back to a rough size model, and I treat those numbers as illustration only. On a real account the dollars come from that account's own metering and invoices, which is the whole reason to run it there.
It only sees the shape of the work, not the SQL. The trace has no query text, so the scorer is working from resource behaviour: timing, spill, concurrency. It cannot see the things a real audit checks for, like a Snowflake specific function or a semi structured access pattern that would not port. Fitting by shape is necessary but it is not sufficient.
And it is a sample, not the whole trace. I scored the leading part of the trace, about 5.8 million of the roughly 70 million queries. That is a deterministic slice, not a random one. If you think the front of the trace is not representative, downloading more of it and re-running is cheap.
What it means if you are paying a real Snowflake bill
Fit is only the first question. Whether a migration is actually worth doing is an economic one, and the arithmetic is short. Moving work off Snowflake pays off when the compute you can move costs more per year than what you would move it to, which is a reasonably large machine plus object storage, somewhere around $10,000 to $16,000 a year. From running the tool across a range of account profiles, the win becomes worth the trouble once the movable spend is past roughly $25,000 to $30,000 a year, and below about a $40,000 a year total bill the machine costs more than you save. ducklens says stay in that case, and that is the right answer. The accounts where this gets interesting are in the $80,000 to $500,000 a year range with batch and analytics heavy workloads.
There are workloads where the answer is no regardless of the score, and they are the ones the tool holds back: warehouses fronting a lot of concurrent dashboard users, anything taking sustained concurrent writes, multi petabyte estates, and heavily governed data with masking, row policies, and audited shares. Those are what the 0 to 13% warehouses in this run look like in the data.
One use for the number involves migrating nothing at all. Snowflake's renewal pricing has been climbing, on the order of 15 to 22% a year for existing customers depending on size, going by SpendHound's buyer data. Their Adaptive Compute, which went generally available in June 2026, bills per query and helps with over provisioning, but the per credit price did not change. Walking into a renewal with a scored, itemised account of which fraction of your bill has a cheap alternative is leverage, whether or not you ever act on it. There was a Hacker News thread in June 2026 where someone described a company spending $2 million a year on Snowflake where only 2 of its 1,200 clients had data large enough to need distributed scale. That is one person's anecdote and not data, but it is the same shape the trace shows: most of the compute on these platforms does not need the thing the platform charges a premium for.
Running it yourself
There are two ways to do it, one with a download and one without.
For the real trace: get the leading parquet files of Snowset from the repository (about 700 MB reproduces my slice), run the one mapping query that renames its columns into the schema ducklens reads, and run the audit. The exact commands are in the README. Budget about 20 seconds for the scoring.
Without a download: scripts/tpch_to_history.py runs the TPC-H benchmark on your machine and turns it into a synthetic account that ducklens then audits like any other. Read the result with the right caution: the execution times and scan sizes are real measurements from the run, but the billing and the spill are modelled. It is there so you can watch the whole thing work end to end, not to prove anything about Snowflake.
Or point it at your own account's query history, which is the only version of the number that is actually about you.
ducklens is on GitHub, MIT licensed, and free to run. The scoring is one SQL file, so you do not have to trust me about any of this. If you want the version reconciled to your real invoice, with a written recommendation on what to move and what to keep, that is the paid assessment I do, and it says keep it on Snowflake whenever the numbers say so.