What is a Neki router?

The core challenge in operating a sharded database is deciding which shard(s) should execute a query. Each query passes through two plans.

The Neki router builds the first plan using the data topology to decide which shard(s) receive the work and how results from multiple shards should be handled. Postgres then builds the plan you already know how to read, which decides how each selected shard executes its work.

A fleet of routers

One of Postgres's fundamental scaling constraints is its process-per-connection architecture. Every direct connection requires a backend process on the Postgres instance it reaches. Those processes consume memory, and large process counts add scheduling overhead.

This becomes a bottleneck for applications that want thousands of client connections.

Neki changes this relationship by putting a fleet of routers between the application and Postgres. Routers handle client connections within their own process instead of requiring a separate Postgres backend process for each connection. On one side, the application connects with a single connection string. On the other is a database that can span anywhere from a single shard to thousands, each backed by a Postgres primary and its replicas.

Neki routers are operationally stateless. They do not store durable application data, and their cached data topology, table definitions, and query plans can be rebuilt. This makes it easy to add, resize, or even remove routers as workloads change. If a router fails, the sessions connected to it are lost. Clients can reconnect to a different healthy router.

This gives a Neki database two different scaling knobs. Shards scale data and the core database engine (Postgres). Routers scale distributed query processing and client connection handling.

The router speaks Postgres

To an application, a Neki router provides one dedicated Postgres connection to one database. Existing drivers, frameworks, and tools connect normally.

Behind that connection, the router may coordinate work over shared connections to many Postgres instances.

Making this work requires more than speaking the Postgres protocol. With Neki, the router becomes the application's Postgres endpoint, removing the need for a separate PgBouncer layer. However, it does more than either a conventional connection pooler or a transparent TCP proxy. It understands Postgres protocol messages, parses the SQL it receives, and can break statements into work for individual shards and coordinate the results. This matters most for complex joins and aggregations, where the SQL sent to each shard can look very different from the original query. It also owns and preserves the client session state. When needed, it applies the relevant session state to work sent to Postgres.

Each Postgres instance has a Neki sidecar running beside it. Once the router has planned a statement, it sends work over gRPC to a sidecar for each selected shard. The sidecar does not parse or plan the SQL. It forwards the work to Postgres over a pooled connection and streams the response back to the router.

One query, two plans

Let’s follow one query through its Neki plan and Postgres plan.

Say we have an orders table sharded on user_id using xxhash.

SELECT id, total_cents
FROM orders
WHERE user_id = 42;

The Neki plan decides where the query runs

First, we can ask the router to show us its Neki plan:

EXPLAIN (NEKI_PLAN, FORMAT TEXT, COSTS FALSE)
SELECT id, total_cents
FROM orders
WHERE user_id = 42;

Route [EqualUnique]
  Query: SELECT id, total_cents FROM public.orders WHERE user_id = $1
  ShardGroup: user_data
  Values: $1

What does this mean, though?

EqualUnique confirms that the router can route this query to one shard.

Values: $1 marks the shard key as a bind parameter instead of the literal 42. The router caches this plan and reuses it for any user_id, substituting the real value into xxhash each time it runs.

ShardGroup: user_data identifies the shard group containing orders.

Query is the SQL sent to Postgres. Here, the router added the public schema qualifier and replaced 42 with $1. Because the entire plan is one Route, the router isn't aggregating or reordering rows on the way back. It gets one Postgres response and streams it straight to the client.

The Postgres plan decides how the shard runs it

After the router builds the Neki plan, it sends the Query to Postgres, where the regular Postgres plan is created. A regular EXPLAIN shows how Postgres plans the query on that shard:

EXPLAIN (FORMAT TEXT, COSTS FALSE)
SELECT id, total_cents
FROM orders
WHERE user_id = 42;

Index Scan using orders_pkey on orders
  Index Cond: (user_id = 42)

Postgres chose an index scan using orders_pkey. Index Cond shows that it uses user_id = 42 to search the index instead of scanning the whole table.

How Neki plans a scatter-gather query

The first query gave the router user_id = 42, which was narrow enough to send the query on to a single Postgres shard. Now, let’s see what happens if we ask for up to 100 paid orders without specifying a user_id:

SELECT id, total_cents
FROM orders
WHERE status = 'paid'
LIMIT 100;

The router cannot use the status = 'paid' predicate to select a shard, so it falls back to scattering the query across the shard group. We can add NEKI_PG_PLAN to include a representative Postgres plan inside the Neki plan:

EXPLAIN (NEKI_PLAN, NEKI_PG_PLAN, FORMAT TEXT, COSTS FALSE)
SELECT id, total_cents
FROM orders
WHERE status = 'paid'
LIMIT 100;

Collapse
└── Limit
    └── Route [Scatter]
          Query: SELECT id, total_cents FROM public.orders WHERE status = $1 LIMIT 100
          ShardGroup: user_data
          PostgresPlan: Limit
              ->  Seq Scan on orders
                    Filter: (status = 'paid'::text)

This is a scatter-gather query, and its Neki plan has three operators.

Route [Scatter] sends Query to every shard in user_data. There is no Values field because the query does not provide a value the router can use with the table's shard index.

Limit caps the combined result at 100 rows. Neki also pushes LIMIT 100 into Query, so each shard can return up to 100 rows, while the application receives no more than 100 rows total.

Collapse is the gather side of the operation. It collapses the multiple shard streams into a single output stream to the application, collecting all returned rows and summing them into one CommandComplete tag.

PostgresPlan is the plan returned by one representative shard for that route. That shard applies its own limit above a sequential scan of orders, using status = 'paid' as a filter because the table has no index on status.

Routers let Neki scale client connection handling and distributed query processing beyond the limits of any one Postgres instance. Shards scale data storage and write capacity, while the data topology tells routers how that data is distributed.

Together, Neki can scale its router and shard fleets independently as the workload grows.

Request access to Neki.