This guide reshards an orders table from two Neki shards onto two new ones and moves traffic in two stages: replica reads first, then writes. __neki.workflow_switch_reads moves only non-primary reads, the ones a session sends with __neki.target = 'REPLICA'. Primary reads and all writes stay on the source until __neki.workflow_switch_writes. The guide proves which shards serve which reads by pausing the workflow's replication for a moment.
What you will build
reshard_create(by_customer on sh2, sh3 -> by_customer_v2 on sh4, sh5) -> copy + stream
workflow_switch_reads -> replica reads: sh4, sh5 primary reads + writes: sh2, sh3
workflow_switch_writes -> everything: sh4, sh5
workflow_complete(drop_source_data)
AI Prompt
Using the PlanetScale CLI (pscale 0.344.0) and psql, reshard a Neki table onto new shards and switch reads to the new shards before writes.
Requirements:
- Create the Neki database with --engine neki, --cluster-size PS-10-AWS-ARM-NEKI and --replicas 2, add two shards, shard the table, load it with COPY, then add two more shards for the target. Wait until every shard answers a direct SELECT 1 with SET __neki.shard. Use one database role for every step.
- Create the workflow with __neki.reshard_create(workflow, database, source_shard_group_uid, target_shard_group_json, target_shard_group_uid, options) and wait for phase streaming in __neki.list_workflows().
- __neki.workflow_switch_reads(workflow) moves only non-primary reads. A session reads from replicas with SET __neki.target = 'REPLICA'; the default __neki.target is PRIMARY, and those reads stay on the source until writes switch.
- Confirm which shards serve replica reads: __neki.workflow_stop(workflow), insert a row through the router, read it with __neki.target PRIMARY and REPLICA, then __neki.workflow_start(workflow).
- Then __neki.workflow_switch_writes(workflow). Reversing a write switch is not a documented rollback path in Platform Preview.
- Complete with __neki.workflow_complete(workflow, '{"drop_source_data": true}').
- Keep a writer running and log every result.
- Run the verification step below before finishing.
Prerequisites
A PlanetScale organization with access to the Neki Platform Preview. Create an account if you do not have one, and see Neki availability and access for how an organization administrator joins the preview.
pscale0.344.0 with a service token that can create databases,psql18.6,jq, andpython3:
export PLANETSCALE_SERVICE_TOKEN_ID="<token id>"
export PLANETSCALE_SERVICE_TOKEN="<token>"
export PLANETSCALE_ORG="<organization name>"
pscale version
- A Neki database with three shards and an
orderstable holding 200,000 rows, sharded bycustomer_idacrosssh2andsh3. Create them with these commands:
pscale database create reviews --engine neki --org "$PLANETSCALE_ORG" \
--region us-east --cluster-size PS-10-AWS-ARM-NEKI --replicas 2 --wait
pscale role create reviews main admin --inherited-roles postgres --org "$PLANETSCALE_ORG" --format json > role.json
export PGHOST=$(jq -r .access_host_url role.json) PGUSER=$(jq -r .username role.json) \
PGPASSWORD=$(jq -r .password role.json) PGPORT=5432 PGDATABASE=postgres PGSSLMODE=require
rm role.json
until psql -At -c "SELECT 1" >/dev/null 2>&1; do sleep 5; done
pscale branch shard create reviews main --config-profile default --count 2 --org "$PLANETSCALE_ORG"
until [ "$(psql -At -c 'SELECT count(*) FROM __neki.list_shards()')" = "3" ]; do sleep 10; done
for u in $(psql -At -c "SELECT uid FROM __neki.list_shards()"); do
until psql -At -c "SET __neki.shard='$u';" -c "SELECT 1" >/dev/null 2>&1; do sleep 5; done
done
psql -q -v ON_ERROR_STOP=1 \
-c "CREATE TABLE orders (customer_id bigint NOT NULL, order_id bigint NOT NULL, total_cents bigint NOT NULL, PRIMARY KEY (customer_id, order_id))" \
2>&1 | grep -v NOTICE
SHARD_1=$(psql -At -c "SELECT uid FROM __neki.list_shards() WHERE name='sh1'")
SHARD_2=$(psql -At -c "SELECT uid FROM __neki.list_shards() WHERE name='sh2'")
SHARD_3=$(psql -At -c "SELECT uid FROM __neki.list_shards() WHERE name='sh3'")
cat > data-topology.json <<JSON
{
"shard_indexes": { "xxhash_customer_id": { "type": "xxhash", "columns": ["customer_id"] } },
"shard_groups": [
{ "uid": "$SHARD_1", "key_ranges": [ { "shard_uid": "$SHARD_1" } ] },
{ "uid": "by_customer", "default_shard_index": "xxhash_customer_id",
"key_ranges": [ { "shard_uid": "$SHARD_2", "end": "80" }, { "shard_uid": "$SHARD_3", "start": "80" } ] }
],
"databases": { "postgres": { "schemas": { "public": { "tables": {
"orders": { "shard_group": "by_customer" }
} } } } },
"default_shard_group": "$SHARD_1",
"authoritative_shard_group": "$SHARD_1"
}
JSON
pscale branch data-topology update reviews main --org "$PLANETSCALE_ORG" --format json < data-topology.json > /dev/null
sleep 10
python3 -c "
for i in range(1, 200001): print(f'{(i*7919)%5000+1},{i},{(i*31)%100000}')" > orders.csv
psql -c "\copy orders (customer_id, order_id, total_cents) FROM 'orders.csv' WITH (FORMAT csv)"
The COPY prints COPY 200000.
Note: Output samples are trimmed to the relevant lines. Each shard bills as a three-node cluster.
PS-10-AWS-ARM-NEKIlists at $30 per month per shard, prorated to the millisecond, and this guide runs five shards. Current rates are on the Neki pricing page. Delete the database when you finish withpscale database delete reviews --force --org "$PLANETSCALE_ORG".
1. Add the target shards
pscale branch shard create reviews main --config-profile default --count 2 --org "$PLANETSCALE_ORG" > /dev/null
until [ "$(psql -At -c 'SELECT count(*) FROM __neki.list_shards()')" = "5" ]; do sleep 10; done
for u in $(psql -At -c "SELECT uid FROM __neki.list_shards()"); do
until psql -At -c "SET __neki.shard='$u';" -c "SELECT 1" >/dev/null 2>&1; do sleep 5; done
done
until [ "$(pscale branch config-profile list reviews main --org "$PLANETSCALE_ORG" --format json | jq -r '.[0].state')" = "ready" ]; do sleep 15; done
SHARD_4=$(psql -At -c "SELECT uid FROM __neki.list_shards() WHERE name='sh4'")
SHARD_5=$(psql -At -c "SELECT uid FROM __neki.list_shards() WHERE name='sh5'")
psql -At -c "SELECT name || ' ' || uid FROM __neki.list_shards() ORDER BY name"
Real output (shard ids differ per database):
sh1 shtdf543fse5gx
sh2 sh98umy6dzl8v7
sh3 sh3wene3sychw6
sh4 shawtgnmb0acsf
sh5 shppk3n6szyfns
2. Start a writer and create the workflow
cat > writer.sh <<'EOF'
#!/bin/bash
n=1000000
while [ ! -f stop.flag ]; do
n=$((n+1))
out=$(psql -At -c "INSERT INTO orders VALUES ($((n % 5000 + 1)), $n, 100)" 2>&1 | head -1)
echo "$(date -u +%T) $n $out" >> writer.log
sleep 0.2
done
EOF
chmod +x writer.sh
rm -f stop.flag writer.log
nohup ./writer.sh >/dev/null 2>&1 &
psql -At <<SQL
SELECT * FROM __neki.reshard_create('move_orders', 'postgres', 'by_customer',
'{"default_shard_index":"xxhash_customer_id","key_ranges":[{"shard_uid":"$SHARD_4","end":"80"},{"shard_uid":"$SHARD_5","start":"80"}]}',
'by_customer_v2', '{}');
SQL
until [ "$(psql -At -c "SELECT phase FROM __neki.list_workflows() WHERE workflow = 'move_orders'")" = "streaming" ]; do sleep 10; done
psql -At -c "SELECT workflow, type, status, phase, traffic_state, rows_copied, streams_error FROM __neki.list_workflows() WHERE workflow = 'move_orders'"
Real output:
move_orders
move_orders|reshard|running|streaming|not_switched|200004|0
3. Switch reads
psql -x -c "SELECT * FROM __neki.workflow_switch_reads('move_orders')"
Real output:
-[ RECORD 1 ]-+----------------------------------------
workflow | move_orders
message | Reads switched for workflow move_orders
traffic_state | reads_switched
4. Confirm which shards serve each kind of read
Pause the workflow's replication, insert a row through the router, and read it back both ways:
psql -At -c "SELECT * FROM __neki.workflow_stop('move_orders')" > /dev/null; sleep 5
psql -At -c "INSERT INTO orders VALUES (7, 9000001, 1)"
sleep 10
echo "primary read: $(psql -At -c "SELECT count(*) FROM orders WHERE customer_id = 7 AND order_id = 9000001")"
echo "replica read: $(psql -At -c "SET __neki.target = 'REPLICA'" -c "SELECT count(*) FROM orders WHERE customer_id = 7 AND order_id = 9000001" | tail -1)"
psql -At -c "SELECT * FROM __neki.workflow_start('move_orders')" > /dev/null; sleep 10
echo "replica read after restart: $(psql -At -c "SET __neki.target = 'REPLICA'" -c "SELECT count(*) FROM orders WHERE customer_id = 7 AND order_id = 9000001" | tail -1)"
until [ "$(psql -At -c "SELECT phase FROM __neki.list_workflows() WHERE workflow = 'move_orders'")" = "streaming" ]; do sleep 5; done
Real output:
INSERT 0 1
primary read: 1
replica read: 0
replica read after restart: 1
The write went to the source, and the primary read, also served by the source, found it. The replica read did not find it while replication was paused, ten seconds after the insert, because replica reads were coming from the new shards. Once replication resumed, the row reached them.
5. Switch writes
date -u +%T
psql -x -c "SELECT * FROM __neki.workflow_switch_writes('move_orders')"
date -u +%T
sleep 10
psql -At -c "SELECT workflow, traffic_state FROM __neki.list_workflows() WHERE workflow = 'move_orders'"
echo "writer: ok=$(grep -c 'INSERT 0 1' writer.log) failed=$(grep -vc 'INSERT 0 1' writer.log)"
Real output. The switch took about 4 seconds, and the writer, inserting throughout, had no failures:
22:15:38
-[ RECORD 1 ]-+-----------------------------------------
workflow | move_orders
message | Writes switched for workflow move_orders
traffic_state | reads_and_writes_switched
22:15:42
move_orders|reads_and_writes_switched
writer: ok=66 failed=0
6. Complete the workflow
touch stop.flag; sleep 2
psql -At -c "SELECT * FROM __neki.workflow_complete('move_orders', '{\"drop_source_data\": true}')"
for n in sh2 sh3 sh4 sh5; do
u=$(psql -At -c "SELECT uid FROM __neki.list_shards() WHERE name='$n'")
echo -n "$n: "
psql -At -c "SET __neki.shard='$u';" -c "SELECT count(*) FROM orders" | tail -1
done
psql -At -c "SELECT count(*) FROM orders"
Real output:
sh2: 0
sh3: 0
sh4: 98231
sh5: 101837
200068
Verify the result
echo "$(( 200000 + 1 + $(grep -c 'INSERT 0 1' writer.log) ))"
psql -At -c "SELECT count(*) FROM orders"
Expected: both numbers are equal (200068 in this run): the 200,000 loaded rows, the one row from step 4, and every successful writer insert are all on the new shards.
How it works
A Reshard workflow copies every table in the source shard group to the target shards and then streams later changes to them. Routing is switched separately for two kinds of traffic. workflow_switch_reads points non-primary reads, those a session sends with __neki.target set to REPLICA or RDONLY, at the target shards; primary reads, the default, follow writes. workflow_switch_writes stops new writes on the source, lets the streams catch up, and routes writes and primary reads to the target. The staged switch is described on the Neki data migration page.
Common issues
Default reads do not move at the read switch
A connection that never sets __neki.target reads with PRIMARY and keeps reading the source until writes switch. In testing, 40 inserts each followed by an immediate default read all found their row after the read switch, because both went to the source. Point the read-only part of the application at replicas with SET __neki.target = 'REPLICA' (or the options startup parameter) to have it served by the new shards first.
Replica reads can lag writes
During the read switch, replica reads are served by the target, which receives writes through the workflow's streams. With replication paused, a row written ten seconds earlier was missing from replica reads. Use replica reads only for queries that tolerate that delay.
Rolling back after writes switch
workflow_reverse_reads returns replica reads to the source. Moving writes back after workflow_switch_writes is not a documented rollback path in Platform Preview, so check the target under replica reads before switching writes.
The old shard group stays in the topology
After completion, the tables point at by_customer_v2, and by_customer remains in the data topology with the emptied source shards. Remove it with a data topology update before deleting those shards.
Next steps
- Delete the database with
pscale database delete reviews --force --org "$PLANETSCALE_ORG". - Read the Neki data migration guide for the full list of workflow functions.
verification:
status: verified
tested_at: "2026-10-28"
product_version: "pscale 0.344.0, Neki PostgreSQL 18.6 (Platform Preview)"
command: "SELECT count(*) FROM orders, compared with 200000 + 1 + successful writer inserts"
expected_result: "Equal; every row is on the new shards and the source shards hold none."