Neki is now available in platform preview.
Neki is built from lessons we’ve learned over eight years of running some of the largest sharded MySQL clusters in the world. Thousands of production workloads with millions of queries per second for companies where even a few seconds of downtime is a very public event. We know what it means to power the world’s biggest tier 0 workloads.
When we released PlanetScale Postgres a year and a half ago, we knew we needed to do more. In that time we’ve onboarded several thousands of customers on PlanetScale, some of them rivaling the size of our largest MySQL customers. Time and time again, we watched teams approach the ceiling of a single machine with Postgres. Metal bought them time, but with customers hitting the upper limit of what a single machine is capable of, we found there was no good option to hand them. Enter Neki.
Why Neki?
You already know the problems that come with fast-growing Postgres databases: tables too large to vacuum or index without affecting traffic, backups taking hours, connection limits, maintenance windows for schema changes, transaction wraparound and so much more.
You can move to a bigger instance, but eventually you run out of big enough machines, and the problems don’t scale linearly as you add more cores and IOPS.
The existing answers each ask you to give something up. Application-level sharding pushes routing into your code. Postgres-”compatible” distributed databases hide the shard key from you, take away your extensions, and add complexity and latency which becomes difficult to handle and debug.
So we built Neki with a few principles, the biggest one being: stick to Postgres, don’t work around it, fake it, or turn away from it.
How does Neki work?
We architected Neki from first principles for Postgres, with real Postgres on every shard. There are four moving parts.
Neki routers
Your application first connects to a Neki router. The router speaks the Postgres wire protocol so your existing drivers and ORMs keep working with a single connection string. A router has a full Postgres query parser, a distributed query planner, query buffering and more. It parses your query, builds a plan that decides which shards should run it, sends the work out, and combines the results back into one stream. Routers can scale vertically and horizontally, so no single router becomes the bottleneck.
Sharding and shard groups
Every shard in Neki is real Postgres with 1 primary and at least 2 replicas, spread across availability zones. There is no modified storage engine. Extensions, SQL support, and performance behave the way Postgres behaves, because it is Postgres.
Shards are organized into shard groups, so different tables or workloads can live on different sets of shards. Each shard uses a configuration profile that defines its instance size, replica count, storage, Postgres parameters, and extensions, so you can size each group for its own traffic.
Connection pooling
Sidecars run alongside every Postgres instance. This is the piece that makes Neki's connection handling meaningfully better than just sticking PgBouncer in front of a database. Because Neki controls both ends of the connection, the router side and the Postgres side, it can size pools to what each instance can actually serve instead of estimating from outside the process.
Control plane
The control plane tracks the health of every node, runs planned switchovers and unplanned failovers, and coordinates the workflows that reshard data, apply schema changes, and perform version upgrades.
Data topology
Tying it together is the data topology, a JSON configuration that maps your logical tables onto physical shards. You define shard indexes, which specify the column Neki routes on and how that value gets hashed, and shard groups, which control how many shards a set of tables spreads across and which shards those are. Routers cache the topology and consult it on every plan.
What you get beyond sharding
Everything you would normally schedule a maintenance window for runs as a built-in workflow in Neki. Workflows provision new target nodes, catch them up with replication, switch traffic with a __neki metafunction, and retire the old nodes. All through the same psql connection your application uses.
This online operations model covers schema changes, version upgrades, planned and unplanned failovers, imports, and resharding.
Neki also includes all of the features you’ve come to rely on with PlanetScale: Insights, schema recommendations, branching, MCP, and more.
You can also run Neki unsharded, as a single primary with replicas. You get the improved connection pooling, online DDL, zero downtime upgrades, and health monitoring before you need to shard. When you do, resharding is a workflow you run against the cluster you already have.
What is a platform preview?
We wanted to get Neki into your hands as soon as possible. You should not run production workloads on Neki during the platform preview. The product is still changing, and some of those changes will be breaking.
If you have any feedback, questions, or face any issues during the platform preview, please let us know. Fill out a support ticket or join our Discord
Try Neki today
Sign in to PlanetScale, opt in to the platform preview, and create a Neki cluster. Read the Neki docs for more information about Neki's architecture, how to shard, and more.
If you have a large Postgres cluster and are curious whether Neki is a good fit, get in touch. We would love to do a private demo for your team, dig into your schema and query patterns, and give you real suggestions on how to shard.