# Software Engineering Daily: turbopuffer

2025-09-30 · Recording: https://softwareengineeringdaily.com/2025/09/30/turbopuffer-with-simon-horup-eskildsen/ · archived copy: https://archive.sirupsen.com/recordings/software-engineering-daily-podcast.mp3

_Lightly edited automatic transcript: speech-to-text, cleaned up by an LLM. It may contain errors; the recording is the source._

## Introduction

<!-- t=0 -->
Vector search has become a foundational technology for AI applications, enabling everything from semantic code search to contextual retrieval for large language models. However, a major challenge with vector databases has been the cost as data storage scales. turbopuffer is a vector database that focuses on speed, cost and scalability. It was created by Simon Eskildsen and Justine Li in 2023 and has seen adoption from high-profile companies such as Cursor and Notion. Simon joins the podcast with Gregor Vand to discuss the origin of turbopuffer, its unique technical design, the economics of vector storage and more.

<!-- t=41 -->
Gregor Vand is a CTO and founder, currently working at the intersection of communication, security and AI, and is based in Singapore. His latest venture, Wintik.ai, reimagines what email can be in the AI era. For more on Gregor, find him at vand.hk or on LinkedIn.

<!-- t=73 -->
**Gregor:** Hello, welcome to Software Engineering Daily. My guest today is Simon Eskildsen.

<!-- t=79 -->
**Simon:** Thank you for having me, Gregor.

<!-- t=80 -->
**Gregor:** Great to have you here, Simon. We're here to talk all about turbopuffer, a company some of our audience may have heard of, and quite a few haven't yet. You're probably also using this product behind the scenes without realizing it, so we'll get into that as well. To begin with, you've got a really interesting backstory, and we always dive into those with our guests. Could you walk us through, very briefly, how you got to turbopuffer? I'll call out that you spent quite a chunk of time at Shopify, and it would be interesting to understand how that helped lead into what turbopuffer is today.

## Background at Shopify

<!-- t=121 -->
**Simon:** That's right. I started my career at Shopify and moved to Canada, where Shopify was built, as a result. I moved from Denmark, where I grew up, and worked on infrastructure at Shopify for almost a decade. When I joined in 2013, it was in the hundreds of requests per second. When I left, we regularly saw peaks north of 1 million requests per second. As part of that, I worked on more or less every aspect of the infrastructure. Generally, the bottlenecks at that kind of scale tend to appear in the data layer, and those are the most persistent ones. So I spent almost the entire time working on the layer between the Rails app and the databases, sometimes inside the databases themselves, but mostly on top of them. Almost 10 years working on every aspect of database scalability at Shopify.

<!-- t=166 -->
After that, I spent about two years working in small increments at my friends' companies on whatever infrastructure problems they had. It turns out that in 2022 and 2023, the persistent infrastructure problem was mostly tuning Postgres autovacuum. It was through that work that I discovered a new type of database was ready to be built. A bunch of things had changed that allowed a new database architecture. It felt like lots of companies were going to want to connect a lot of data to AI, and that a new search engine on this new storage architecture was ready to be built. So I started hacking on the first version of turbopuffer in the summer of 2023.

<!-- t=217 -->
**Gregor:** Nice. Sticking on that for a second, you left Shopify almost two years before you started on turbopuffer. Why not launch straight into something else? I guess you were waiting for a problem to solve. How would you speak to the choice to go and help other people with their problems for a little while?

<!-- t=237 -->
**Simon:** I went to Shopify right after high school, so I'd spent my entire career inside one company. I couldn't develop high conviction about joining one particular company because I'd only been in one my whole life. I did a little bit of work for another startup during high school, but other than that it was a decade with one company. I had a bunch of friends with interesting infrastructure challenges, so I wanted to bop around, take small contracts, solve actual problems at SaaS companies, and over-deliver on some infrastructure promise. That was a really good way for me to see a bunch of companies, and that was the primary concern. I took summers off for two years, which was nice and gave me space to get in the best shape of my life, among other things.

<!-- t=277 -->
I didn't leave with the intention of founding a company. I didn't leave with the intention of anything, other than that I was ready for a change, and this felt like the natural thing. I also spent a lot of time on my napkin math, a series of blog posts where I do first-principles estimates: the memory bandwidth is this much, so you should be able to do the task in this long. I spent a bunch of time writing that and building out the numbers. Then at some point this problem started staring me in the eye and I couldn't stop thinking about it.

## The Readwise origin story

<!-- t=313 -->
**Gregor:** Awesome. Let's get into turbopuffer, starting with the origins. You've written a few blog posts, and one of the first is the origin story. I believe you were helping friends at Readwise. Can you talk us through where this came from?

<!-- t=326 -->
**Simon:** Readwise is run by some of my friends. Their first product took every highlight you made on Kindle and sent you an email every day or every week with some of them. I love the product and have always been close with the founders. At the time they were launching a Reader product, a read-it-later app: you take an article, it's nicely formatted, it's on your phone, and you can listen to it. For that, they just needed their Postgres database up to snuff, and I was helping with that, again tuning autovacuum. It's the biggest problem we have. I felt a little gaslit by the Orange Site, having spent a decade thinking MySQL was a worse database than Postgres. It turns out they're just two different databases, and one of the biggest challenges with Postgres is tuning autovacuum, which I spent some time doing for them.

<!-- t=369 -->
One of the other things we were working on was in the fall of 2022. Everyone remembers what they were doing when ChatGPT dropped, and for me it was in the middle of this database tuning. We decided to build a bunch of AI functionality to help people comprehend what's going on in an article. At the time the context windows were pretty small, so we started using vectors very early on to pull the right parts of the article in. I also played around with a little recommendation engine: we would find one article and recommend another that was similar. It was clear there was a foundational layer brewing.

<!-- t=406 -->
Once we had something working, I ran the back-of-the-envelope math on what it would cost to take all the articles in Readwise Reader, even at that pre-launch stage, and put them into a vector index on one of the databases around then that could do it. Their Postgres database was costing them a couple thousand dollars, around 3K a month. The math told me that on a reputable vector database we thought could do the workload, it would cost 30 grand a month. Even if the company could afford that, it clearly detracted from launching the feature. We weren't going to invest in something an order of magnitude more expensive just to store a subset of the data for powering recommendations and some context into an LLM. We put it in the bucket of "token costs for LLMs are coming down, so presumably someone's going to figure out how to make this cheaper as well." I finished the engagement, got their database up to snuff, and they had a very successful launch.

<!-- t=469 -->
But I couldn't stop thinking about how we were going to make this vector storage cheaper. My memories of running the search clusters at Shopify were coming back to me. They were some of the worst clusters to get woken up by. It quickly became obvious that all of the incumbents were storing everything in memory.

## Why vector storage is expensive

<!-- t=488 -->
**Simon:** Say you have a kilobyte of text, as you might have in an article (often there are multiple kilobytes). To turn that into vector embeddings, you put it into chunks that are semantically meaningful. Maybe a kilobyte of text has four paragraphs, so four chunks, and you turn each one into a vector. Those vectors might be six kilobytes each, so now you have 20 to 30 kilobytes of vector data from one kilobyte of text. That's a 20 to 30x amplification. For a normal full-text search index, the amplification to build all the structures is around two, maybe three on a bad day depending on the data distribution, but somewhere between 1.8 and 3. For vector indexes we're talking about 30x just to store the vectors, let alone build an index. That's why the costs were so prohibitive, and on top of that all of these databases had to store everything in DRAM, which on a cloud provider costs somewhere between $2 and $5 per gigabyte. It was clear that at Readwise the economics didn't line up.

## NVMe SSDs and object storage as prerequisites

<!-- t=558 -->
**Gregor:** There was also a timing element here, hardware-wise, with NVMe SSDs, non-volatile memory express. Could you speak to that back-end piece?

<!-- t=569 -->
**Simon:** Exactly. To build a good database, you need two things: a new workload and a new storage architecture. The new workload here is that there's a new way to power recommendations and pull context into an LLM. The new storage architecture started to become clear to me in early 2023. NVMe SSDs are very fast. I think they got faster than anyone really expected. But they weren't generally available in the clouds until the late 2010s, around 2017 or 2018 on AWS, so databases hadn't been built around them. The phenomenal thing about NVMe SSDs is that per gigabyte they're about 100 times cheaper than DRAM, but in terms of bandwidth, in gigabytes per second, they're only about an order of magnitude slower than DRAM, maybe even only 5x. That's an amazing set of properties, but most databases haven't been built to get all that bandwidth through. You have to bypass the Linux page cache to get the maximum, and you have to build your storage engine so that it gets a lot of data on every round trip.

<!-- t=635 -->
That was the first thing that had to be true. The second was that object storage like S3 or GCS, depending on your cloud, is consistent. You put an object, and when you read it, it's the same object. S3 only became consistent in 2020, which is remarkably late, but it's a very nice property when you're building a database. You can build around it by creating new files and so on, but it's a prerequisite.

<!-- t=659 -->
The other primitive turbopuffer was waiting for was only released, and we're recording in July of 2025, at re:Invent last year, so about seven months ago. That's the final API required for a turbopuffer-like database: compare-and-swap. It lets you put a file on S3, read it, modify it, and write it back only if it hasn't been changed in the meantime. With that simple primitive you can build a database like turbopuffer that has no dependencies other than object storage, S3 or GCS. This API was available first in GCS on GCP, which is why turbopuffer started on GCP. We developed strong conviction that it was going to be a ubiquitous feature, because every other object storage implementation other than S3 had it. Finally they released it last year, and a month later we went into AWS; until then we'd run workloads across clouds.

<!-- t=714 -->
So these were the three things we needed to build turbopuffer, and they became available around that time. Suddenly we had the two prerequisites for a great database: a new workload and a new storage architecture. A new storage architecture meant there was differentiation from tacking this onto existing databases.

## Vector indexes: graphs versus clusters

<!-- t=737 -->
**Gregor:** In terms of the technical challenges in making S3-like storage performant, were you aware of them ahead of time, or did you come across them while solving the problem? What would you say those were?

<!-- t=754 -->
**Simon:** I started working on turbopuffer in May of 2023. The main challenge was this. Let's talk about vector indexes for a second, because that's relevant to how you put this on S3. There are two broad categories of how you build a vector index.

<!-- t=771 -->
Say you're Spotify and you want to recommend music. You take every song and pump it through some kind of model, and it places the song in vector space. Songs that are adjacent in vector space, as you plot them in this massive coordinate system (imagine it in two dimensions), would also be songs that might be similar. Clusters form naturally: a rock cluster, a pop cluster, a rap cluster, as the semantic relationships unfold in this space. Models got very good in 2022 at taking a piece of data and plotting it into the coordinate system, but not so much at storing the coordinate system and querying it. A vector index is essentially just a coordinate system in many dimensions, though you should visualize it in two. When I go on an e-commerce site and search for "red dress" and they have a burgundy skirt, those two points would be very close in vector space. That's a very nice problem to have solved in such a simple way.

<!-- t=829 -->
But searching for a query across everything you have, maybe the hundreds of millions of songs on Spotify, maybe the billion things in an e-commerce catalog, the only way to get the exact vectors closest in that coordinate system is to look at every single vector and compare it with the query vector. That's the only way you're guaranteed to find the top 10 closest vectors, but it's incredibly slow. A million vectors can easily be gigabytes, and searching gigabytes in memory, if you max out the node, takes maybe a couple hundred milliseconds. That's maybe not bad in itself, but every node that's completely maxed out is doing five requests per second, and that costs thousands of dollars per month depending on node sizes and so on.

<!-- t=883 -->
Instead, we use approximate algorithms. We're okay with maybe 95% or 99% accuracy over what we retrieve. In the vast majority of cases we get the exact results, but in some outlier cases we don't. There are two ways to build these approximate nearest neighbor indexes, or ANN for short. There are others, but these are the two people have productionized.

<!-- t=903 -->
The first is graph-based indexes. As you add things to the coordinate system, you can use heuristics to connect vectors that are adjacent, and then you walk the graph: you drop yourself in the middle and continuously move to the nodes closest to the query to find good results. This works really well, and it was by far the most common method of vector search in 2023. Almost every production use case used it.

<!-- t=938 -->
The second is the cluster-based approach. This is actually the classical method, the first thing people came up with and productionized. You take all the vectors and create clusters. Using the music example, there might be a rock cluster, a pop cluster and a rap cluster, and these natural clusters form. In every cluster you take the average of everything that belongs to it, and that's the centroid. When you search for songs close to a given song, you find it's closest to the pop cluster, so you search only the pop cluster. That way you're only searching, say, 33% of everything. In reality you can cut the space so that you only have to search a small percentage of the entire dataset. It's a very nice way to visualize it, and there's even some semantic meaning in what the clusters mean.

## Why clusters fit object storage

<!-- t=993 -->
**Simon:** In 2023 it seemed like graphs were going to win. But the problem with graphs is that they're very fast in memory, where you're dropped into the center and every read is 100 nanoseconds. Land in the center, 100 nanoseconds; next part of the graph, 100 nanoseconds; and so on. In aggregate that's still very fast, because memory is fairly fast and the CPU cache works well for these random reads. But on disk and on S3 this doesn't work. If you're reading a couple of kilobytes off S3, your P90 is in the hundreds of milliseconds. You land in the middle of the graph: 100 milliseconds. You go to the next node: 100 or 200 milliseconds. And the next, and the next, and every single time it's 200 milliseconds, because you can't predict which jumps you'll make. You can make a bunch of heuristics, but you can't do much.

<!-- t=1043 -->
You can try to shrink the diameter of the graph, to make it denser, so there are fewer jumps. That can work, but on large graphs it's very difficult to get below maybe nine or ten round trips on millions or hundreds of millions of vectors. It also becomes very expensive to write, because every time you update one of these nodes you have to update a lot of things around it. That's not a big deal in memory, but it's a big deal on disk, which doesn't like lots of random reads, and especially on S3, where you'd be editing a lot of random files. So graphs just don't work that well there. They're amazing in memory, and it's almost impossible to beat their performance in memory. But when things are on disk or S3, you have round-trip latencies in the hundreds of milliseconds for S3 and hundreds of microseconds for disk, and neither works well with lots of dependent jumps.

<!-- t=1092 -->
If we go back to the clustering method, which seems crude and primitive, you can just download all those centroids: here's the rap centroid, here's the pop centroid, and whatever other genres. You download a centroids.bin, a nicely packed binary file, search through all the centroids, and find the closest few clusters. Then you do another round trip to S3 and download just those clusters. You've gone to S3 twice and fetched a lot of data, but S3 is fine with that; you can download a lot of data in 100 to 200 milliseconds. This works really well for disk too: you get the chunk of centroids in one big page, and then the individual clusters, again in two round trips.

<!-- t=1132 -->
So we built the storage engine around minimizing the number of round trips. We use a cluster-based index because it minimizes round trips, which gives really good performance on storage media that are so much cheaper than memory. Memory is about $2 to $5 per gigabyte, S3 is $0.02 per gigabyte, and disks are about $0.08 to $0.10 per gigabyte.

## Cold and warm queries

<!-- t=1158 -->
**Gregor:** We'll get on to how we measure any of this. I'm visualizing what you've described, though maybe some listeners still find it hard, so I'll come back to that. You've written that turbopuffer should take a maximum of three round trips for what you call sub-second cold latency. Could we talk about cold and warm here? I think many in our audience are familiar with cold and warm data. How does that come into it?

<!-- t=1207 -->
**Simon:** The canonical source of truth for all data in turbopuffer is object storage. When you do a write and we return success, it's been committed to the most durable systems on Earth: S3, GCS and friends. When you do a query, we send it to a node. If that node has the data in memory cache, we use that, which is the fastest. If not, we go to disk, and if not, we go to object storage. If we haven't seen a query for, say, a week, it's not going to be in cache anymore, and we have to go directly to object storage.

<!-- t=1240 -->
In the first round trip, we get a bunch of metadata files: the schema, the most recent index, and other metadata about the namespace. Then we get the centroids and anything else relevant to the filtering you're doing. Then we get the clusters, and exactly what we need from them to satisfy the query. Those are the fundamental three round trips. There are situations where we do more. If you're fetching a lot of information and we can't decide that fetching it all in the third round trip is the best for performance, the query planner has to decide whether to do another round trip or to fetch more data.

<!-- t=1277 -->
That's the scenario for a cold query. Depending on their size and the mood of S3 that day, these round trips take about 200 milliseconds each, so we get cold query performance just short of a second, around 600 to 800 milliseconds. S3 also has caches, so it depends, but that's generally what we see. In fact, when you query S3 the variance is so high that once we cross the 98th percentile of the latency we generally see for something of that size, we send off a second request to minimize the variance in query latencies.

<!-- t=1311 -->
That's a cold query, in the high hundreds of milliseconds. A warm query can go to memory or disk, and at that point there's no reason it can't be as fast as any traditional storage architecture. It's only the cold query that happens once in a blue moon. Some of our customers do pre-flight queries: when you open the Q&A dialog in Notion, it sends a query to hint to turbopuffer that we should start warming that index. We do that in the order that reduces latency as fast as possible to minimize the impact for the user. So generally this is not an issue we see, especially with the cost savings we get from having only a single copy.

<!-- t=1350 -->
This is also where the name comes from. The puffer fish is fully deflated when the data is only on object storage, and we can inflate it all the way into DRAM or even CPU caches. The more you query the namespace, the faster turbopuffer gets.

## Visualizing data in turbopuffer

<!-- t=1373 -->
**Gregor:** Got it, that makes sense. A sidebar question, jumping into the product itself: if you're a developer, how can you visualize the data through turbopuffer? I've used other vector store products and seen how they represent this data. Some of it worked and some didn't make a lot of sense to me. How has turbopuffer approached this?

<!-- t=1395 -->
**Simon:** Do you mean visualizing it to the user?

<!-- t=1397 -->
**Gregor:** Correct, through a GUI or so forth.

<!-- t=1401 -->
**Simon:** turbopuffer's console is still fairly simple and operational. I would love for there to be more of a playground. What we see our customers do is generally export the namespace locally and then try to visualize it. I don't know how many of our customers visualize the data and how many just start writing evals against it. I would love to help with tooling, but we've been very focused on the database itself.

<!-- t=1423 -->
**Gregor:** That's helpful, and it probably speaks to where turbopuffer has been focusing, versus other products that try to hit all the points at once and end up with a more general approach.

## Measuring recall

<!-- t=1435 -->
**Gregor:** Let's go to performance. In vector stores the term is recall, the trade-off between latency and accuracy. I believe turbopuffer samples at least 1% of all queries internally, and you've presented that data anonymized in blog posts. Talk to us about how you measure and why.

<!-- t=1463 -->
**Simon:** Recall is extremely important, because if you're building a search pipeline, you don't want to think in your evals about whether your search engine is inaccurate. That should be the job of the vendor or the database you've chosen. You should not have to manually tune it or manually run recall against it. As we iterate on our ANN algorithm, we run it against various internal benchmarks, including academic ones, to make sure it performs. But nothing beats real-world performance. In the real world, someone is going to insert the same song a million times into a cluster with lots of other songs. They're going to put that burgundy skirt in there 20 million times, not realize they have the bug, and still expect accuracy to be high. So accuracy is very important to us, and we consider it our job.

<!-- t=1521 -->
Let me briefly explain what recall is. Going back to my original example, the only way to get the absolute true result of the top 10 closest vector embeddings to a query is to look at the entire dataset, O(N). With the ANN, you issue a query against the approximate nearest neighbor index and compare the top 10 approximate results with the exact results. If you have a recall of 90%, nine of the 10 results were correct in the ANN result. If it's 100%, everything overlaps. We find our customers are very happy somewhere between 90 and 100%, erring on around 95% recall averaged across everything. Generally anything above 90% is really good.

<!-- t=1572 -->
Recall is a tricky metric. You could search for "banana" in a dataset that only knows about songs, and it's going to give you a result, because in some way banana is close to something. The vector distance might be very long, and in a clustered index, the further the clusters are from the query vector, the worse the results get. So it's a flawed metric, but it's the best flawed metric we've got, and the one we're all measuring against.

<!-- t=1604 -->
Very early in turbopuffer's history we decided to take this on as our problem. As you mentioned, a percentage of production queries are sampled. For a random percentage, somewhere around 1% but scaling with how many queries the user does, we send the query to a different worker fleet, which evaluates the exact result against the approximate result, compares them, and reports the number back to our instrumentation in Datadog. We have a dashboard where we look at every single customer's recall, and we get big red dots if someone's recall is below 90%. Production is the only thing that tells the true story; you can't just evaluate against academic benchmarks. Being on call for almost 10 years at Shopify taught me that nothing matters other than production. It's the same for accuracy: the only way we'll trust our recall is if we know that on all the production datasets it's above 90%.

## Native filtering

<!-- t=1666 -->
**Gregor:** Got it. I believe native filtering also comes into this. Could you talk about that?

<!-- t=1673 -->
**Simon:** Filtering is the most important thing for recall, because this is where it gets tricky, and where just tapping a vector index onto an existing database is not quite enough to ensure high recall. Let's work through some simple examples to explain what pre-filter, post-filter and in-place filtering mean.

<!-- t=1694 -->
We'll keep using an e-commerce example on a very large dataset, and say we want to filter out any products that are not public. Say 99% of all products are public, and 1% are held back because people are iterating on them, haven't released them yet, or they've sold out. If only 1% of the data doesn't match the filter, it's probably fine to over-fetch by around 1% and filter afterward. That's a post-filter, and if you evaluate recall (the only accurate way is against everything), you'll get very high recall with a post-filter.

<!-- t=1742 -->
But imagine a filter that matches only 10% of the dataset, say everything that's the color blue. If I've over-fetched 100 items, I'm not going to get 100% recall; the math doesn't work out for the precision you need. In that case, since it's only 10% of the dataset, maybe we can read another index, like a traditional B-tree or bitmap index for "blue," and then evaluate all those vectors. That works okay for something like 10%. It will be a little slow, because that's a lot of vectors to look at, but it might be okay.

<!-- t=1775 -->
The trickiest queries are the ones in between, the ones that filter out maybe 50% of the dataset. Imagine you're in Singapore, and everything that ships to Singapore is maybe 50% of the catalog. Do you post-filter? You'd have to over-fetch by a lot to make sure you have the right recall, because in a clustered index the filter might completely eliminate some clusters. Maybe food items never ship to Singapore, so when you search for a banana, almost nothing matches, and the search moves into the next cluster, like clothing, that might ship. When you think about it in the clustered sense, the query planner really has to be aware of how much of each cluster matches the filter and how much that cuts off. Then you need heuristics about how many vectors you have to look at, in order of distance from the query vector, to guarantee high recall.

<!-- t=1828 -->
I realize this is hard to parse, but what I'm trying to impress is that the query planner, which decides how much data the query needs to look at to get high recall, needs to be very aware of both the vector index and the filter index. Most databases that have just slapped a vector index onto an existing database will only pre-filter or post-filter, and often the user has to choose. You have to know about the selectivity of your dataset, and it's not a trivial thing to choose.

## SPFresh and maintaining cluster indexes

<!-- t=1860 -->
**Gregor:** That leads nicely into vector indexes. I believe it's SPFresh that's used at turbopuffer. Most of our audience is probably familiar with the concept of an index at a high level: you tell the database, "these are the kinds of queries we'll be making regularly, so we need an index across these three columns on this table." That's a relational database index. How does that look in the vector context, and why SPFresh?

<!-- t=1902 -->
**Simon:** We talked about graph-based and cluster-based indexes. Graph-based indexes are really nice because you can add things to them, they fit neatly into the graph, and you can keep adding. You don't have to worry that much about recall, because recall on a graph-based index is usually phenomenal. That's why they've been so popular. But we chose a cluster-based index for the reasons I mentioned before. With a cluster index, someone could start adding products in a category or cluster that didn't exist when you created the initial clusters. A clustering algorithm generally has to look at all the data and then decide what the clusters are, and when you get into millions, hundreds of millions or even billions of vectors, that can easily take days. With a state-of-the-art algorithm it can take days on a very large machine. You can start using GPUs to do it faster, but it's still very difficult.

<!-- t=1955 -->
So basically these clusters can't be incrementally maintained the way a B-tree can. In a traditional database index, you just insert into the tree and it nicely balances, and there are lots of properties that let you add to it incrementally, like a graph. To maintain a clustered index incrementally, you need a bunch of heuristics.

<!-- t=1980 -->
Continuing with the e-commerce example, say you have a shoe cluster, and this customer keeps adding shoes. At some point the shoe cluster has maybe 1,000 items in it, and that's a lot of data to search every time you search for something relevant to shoes. So we have to split the cluster. Every time you write, it's "this is a shoe, this is a shoe," until it reaches some terminal size and we split it. Maybe there's now a sneaker cluster and a leather shoe cluster. You can imagine incrementally maintaining the clusters like this: they reach some size and we split them, and once in a while you remove enough items that we have to merge clusters. Every time we do this, we have to recompute the centroids. It's much more complicated than that, but that's the general idea behind SPFresh, and with enough of these heuristics you can make it work. This is probably the most complicated part of the entire turbopuffer codebase, to make it work at very large scale, with recall, and with filters. That's how SPFresh works, and it scales very well.

## Namespaces and scaling

<!-- t=2060 -->
**Gregor:** Moving to scaling generally. My experience in this space is in email, and there namespacing was a hard requirement. You've got thousands of users, and you never want someone searching and someone else's email turning up. That would be catastrophic. So namespacing, the idea that every user's data is in a siloed environment, was incredibly important to us. I'm aware that namespacing is part of turbopuffer, and I believe it scales to something like 14 million namespaces... sorry, 40 million. And I imagine your Shopify experience, with its huge scaling challenges, fed into how you approached this.

<!-- t=2107 -->
**Simon:** The only way you can scale anything is to shard it. So we decided to make namespacing a core sharding primitive that we expose to the user. If you can give us your sharding key in a simple way, we can scale really far with you. In turbopuffer, a namespace maps directly to one shard, and a namespace is just a prefix on object storage. If you have "gregor-email," that's prefix number one, literally "gregor-email/" on S3, and all your files are in there. Then you have "simon-email/" and so on. That way we're only constrained by how many prefixes S3 can have. S3 can have a lot of them. We have yet to see any limit, and there's no documented limit on how big this can be. turbopuffer has more than 100 million of these namespaces.

<!-- t=2162 -->
This works great because it also means we can encrypt every namespace differently. You might decide you want to encrypt the gregor-email prefix on S3 with a key that you have access to, and I want to do that with mine, which adds another layer of defense that's basically as good as having the data in your own bucket. There's no logical difference. You can rotate or revoke the key at any time.

<!-- t=2190 -->
Namespacing is a core tenet of turbopuffer. Thinking back to my Shopify experience, most of the problems we solved were by sharding on the shop, since shops didn't have to talk to each other other than in some edge cases. That's exactly what we did at turbopuffer, building it in as a core primitive. In the future a namespace will probably map to multiple shards, but it will be abstracted behind this single layer of completely horizontally scalable search.

<!-- t=2219 -->
**Gregor:** Nice. As I say, it was a core primitive I was looking for. Surprisingly, other people in the space take slightly odd approaches to it, or virtually no approach at all, so the audience should look out for that.

## Customers: Cursor and Notion

<!-- t=2235 -->
**Gregor:** Let's turn to who's actually using this. There are huge names using turbopuffer. Notion and Cursor are probably two of the biggest, and most of our audience will be familiar with them and can start to imagine how the technology works under the hood. How is that translated into what it means for them as a user each day? Could we talk about both cases?

<!-- t=2265 -->
**Simon:** Let's start with Cursor. I got to know the Cursor team in 2023. When they saw the first announcement of turbopuffer, knowing the team as well as I do now, it's clear they just had a conversation around the dinner table at some point thinking, "Why hasn't anyone built it like this?" They're big fans of S3, just like I am, so it slotted right into their mental model. We went back and forth over email with a bunch of bullet-point lists, spent some time with the team, and were completely aligned on what needed to be built. There were a couple of missing features, so we built those out for them, and then they migrated.

<!-- t=2296 -->
For them, the storage architecture of turbopuffer just made sense. When you open a codebase in Cursor, it indexes the codebase, and part of that indexing is embedding it. This powers a lot of the agents and functionality inside Cursor that can do semantic search on the codebase. I use it all the time to ask where it does this or that, in plain language. Cursor embeds the entire codebase with their own trained embedding models and then uses that as a tool call in everything else. For them, the previous solution, having everything in memory, just didn't make a lot of sense. You don't need every codebase that ever opened Cursor in memory at all times. That's incredibly expensive, and the per-user economics weren't sustainable. They needed a way to bring the per-user cost to something that made sense. The turbopuffer model of inflating the puffer fish when you're querying the codebase fit: only some percentage of codebases are active at once, but they're still valuable to keep around. It was such a clear fit for the storage architecture. Their first bill was reduced by 95% moving to turbopuffer. They've been amazing partners and have inspired a lot of how we've built turbopuffer since.

<!-- t=2373 -->
Notion is another customer, and it's a very similar story. They were using a vector database that also kept things in memory, and again the per-user economics didn't make sense. Again, only a percentage of workspaces are active at once. That really appealed to them as they wanted to connect a lot of this data into their LLMs.

<!-- t=2397 -->
I don't want to give the impression that turbopuffer is only good for these use cases. Fundamentally, the cheapest way to store data in the cloud is to put it in object storage at $0.02 per gigabyte and cache it on a compute node only when it's actually being used. In a traditional storage architecture you have to replicate the data to three nodes. In turbopuffer's case you have it in object storage at $0.02 per gigabyte, and then on disk and in memory at a blended cost of maybe around $0.10 per gigabyte. In a traditional storage architecture with memory and disk and all of that, you easily run into dollars per gigabyte. That's what allows this pricing, and it allows a truly serverless environment where you can just push in the vectors. This was great for both Notion and Cursor because they have a lot of writes: every time you edit files in Cursor or edit in Notion, it issues a write to update the vectors in turbopuffer.

<!-- t=2448 -->
**Gregor:** That's interesting. I've noticed how fast Cursor is at indexing. I love using Cursor workspaces, dumping, say, three different codebases into a workspace and having it figure things out across all three. The indexing has always impressed me, so now I know why. Credit to Cursor as well, but it's great to understand the technology behind the scenes. I was at a Notion event not long ago, and they're doing a big push on their AI offering. A lot of it is, "we're going to index across all these data types," like Google Drive, and I believe Gmail is coming soon, and Slack, well, probably Slack, given that announcement about the API restrictions, but I'm sure they're figuring that out. At the end of the day, it's a lot of data that needs to be indexed and then searched with AI RAG. So it's super interesting that turbopuffer is behind the scenes on that.

<!-- t=2503 -->
**Simon:** Every company has a lot of data they want to turn into context. If you have a lot of data, the economics of adopting another data store, rather than perhaps using a vector index in your relational database, can start to make sense when you get into the tens of millions of documents that need to be indexed. It's the same reason we've always moved search workloads out after some point for full-text search. After a certain threshold, the downsides of having another data store start to make sense. When you want to connect a lot of data to AI, we think the storage architecture makes a lot of sense.

## Pricing and no free plan

<!-- t=2533 -->
**Gregor:** You touched on pricing with the Cursor and Notion examples. turbopuffer doesn't offer any kind of free plan, and you've been vocal about why. I fully support that, just to be clear, but it'd be great to hear the reasoning, as much for developers' understanding for their own products as for people who might come to try turbopuffer and be disappointed there's no free plan.

<!-- t=2558 -->
**Simon:** First of all, if you're not having a good time in the first 30 days and you've tried the product earnestly, you have the right to cancel. We think the companies that benefit the most from turbopuffer won't find this price point scary. For small companies, it might make sense to use a small index tacked onto the existing relational database. But if you have the ambition to index tens of millions of vectors, it can make sense to start on turbopuffer. In a lot of cases it's really about wanting to give people a really good experience, and that's easier with a commercial-only offering. We can put the necessary staff behind supporting people when they have questions and give them a really good experience, so that you get the feeling we're part of your team.

## Team size and roadmap

<!-- t=2600 -->
**Gregor:** Absolutely. You've announced quite recently that you're GA, general availability. What does turbopuffer as an org look like now? I haven't done the research, and I always find it interesting: are you two people? 200?

<!-- t=2612 -->
**Simon:** We're fewer than 20 people, and it's a very engineering-heavy team. We've hired a bunch of database engineers, who are the majority of the team, working on the database. Now we're starting to hire other types of roles to expand the team, especially to support our customers. We really want a dense, P99 engineering environment, so we try to hold the standards high on the team that we put in front of our customers and that is developing this product.

<!-- t=2642 -->
**Gregor:** That's a very nice number. I think there's been a bit of over-reporting on the one-person unicorn and all that nonsense, so it's good to call out that some of the best companies coming out at the moment are maybe 10, 20, or a bit more. From what you can share, where is turbopuffer going over the next six months to a year? What's top of mind for where the product needs to go?

<!-- t=2667 -->
**Simon:** One of the biggest frontiers we're pushing right now is more database features. For example, we just launched conditional writes: the ability to say "I only want to update this document if it's actually newer than the document that's in the database." There's also a lot on the full-text search side. turbopuffer doesn't just do vector search; it also does full-text search, and we see more and more of our customers doing that. There are a lot of expectations about what you can do in full-text search, so we have to build that on the storage engine we built from the ground up. We have a very solid, generic LSM storage engine underpinning all of this that we've matured. Now it's really about features, and about even more performance than we deliver now.

## The visual language

<!-- t=2714 -->
**Gregor:** We're coming up on time. One final question, completely unrelated to databases specifically: the visual language and front-facing site. It's very fun, sort of pixel art, if you want to give it a name. Where did the idea come from, and who leads that on the team?

<!-- t=2730 -->
**Simon:** I think it's neo-retro pixel art or something; I don't know what the aesthetic is called. This came from when I was back home in Denmark and some friends were visiting. I was talking to my friend, who's a designer, about the idea of turbopuffer. He said later that he had no idea what I was talking about, but I sounded really excited and he was so encouraging. In my mind, lateral to exactly how to lay out the bytes on disk and build the storage and ANN index, was this aesthetic I wanted. We riffed back and forth on what it would look like and created some sample sites. It was very bare-bones. At Shopify I spent 10 years evaluating databases, so I was on the buy side for the majority of my career. Every time I went to a database website, I wanted to know what it costs, what the trade-offs are, who the customers are and what they're using it for. Those are the only questions I really care about, and most database websites are filled with so many other things. Any database has trade-offs, and I want to know whether this is the right set of trade-offs for me. What's the architecture? What are the guarantees? What's the consistency model? "Show, don't tell" was very important to me.

<!-- t=2803 -->
Then we wanted to breathe a little fun into it. turbopuffer is a whimsical name in itself, and this aesthetic appeared over many, many iterations that you can see on the Internet Archive. We're really fond of it, and we'll continue to iterate on it because it makes us happy when we go to the website.

<!-- t=2820 -->
**Gregor:** As you've called out, if you go to the site you'll see a whole bunch of sliders and so on, so you get into the meat of it very quickly. Having this fun aesthetic around it definitely sets the tone for who you are as a company. It reminds me, in a different form, of TigerBeetle, the financial database. They have a very specific type of database, but they also bring fun visuals to it, and obviously the name, so it's very memorable. I remember seeing something about turbopuffer on Hacker News a little while ago. When this episode was suggested I knew the name straight away and could visualize the website before I even came back to it. So I think it's very smart.

## Wrap-up

<!-- t=2861 -->
**Gregor:** Thanks so much for coming on. Where can people find you, and where's the best place to get acquainted with turbopuffer?

<!-- t=2867 -->
**Simon:** turbopuffer.com is a great starting point. We post on all the social media platforms, and you'll find the links there. I'm sirupsen on X, where these days I mostly tweet when I go out for runs for some reason, but that's where you'll find me and the company.

<!-- t=2879 -->
**Gregor:** Awesome. Thank you so much, Simon. A very deep technical episode today, which I think a lot of the audience will love. turbopuffer, I think you're doing very interesting things, and I look forward to following along. Thanks for coming on.

<!-- t=2894 -->
**Simon:** Thank you so much for inviting me on.

<!-- t=2894 -->
**Gregor:** Thank you so much for joining us.
