# Agent setup prompt Source: https://planetscale.com/docs/agent-setup/prompt PlanetScale instructions for AI agents to install skills, configure MCP, and authenticate the CLI. Vitess and Postgres Use PlanetScale with your AI agent Paste this into your agent to fetch the official PlanetScale setup instructions: PlanetScale instructions for setting up AI agents with PlanetScale Vitess/MySQL, Neki, and Postgres. Agents with the PlanetScale CLI installed can also fetch setup instructions in JSON: ```bash theme={null} pscale agent-guide --format json ``` Commands on this page require `pscale` 0.292.0 or later. Install or upgrade the CLI in the prerequisites below before running the setup steps. Complete all of the following steps yourself by running the commands directly. Do not ask the user to run setup commands unless a step requires browser approval they must perform. *** ## Install PlanetScale Skills and MCP Use the section for your agent below. Skills cover operational workflows (assessment, safety review, Insights, schema recommendations). The hosted MCP server gives MCP clients access to PlanetScale account data. ### Prerequisites Install the PlanetScale CLI if it is not already available (**0.292.0 or later**): ```bash theme={null} brew install pscale brew upgrade pscale ``` Verify the CLI version and agent automation support: ```bash theme={null} pscale version pscale agent-guide --format json pscale auth check --format json ``` If `agent-guide` is unknown or `auth check --format json` prints nothing, upgrade with `brew upgrade pscale`. If `auth check` returns `"status": "action_required"`, follow `issues` and `next_steps` in the JSON (typically `pscale auth login --format json` or service-token flags for CI). Always pass `--format json` on `pscale` commands in automation. Put `--org` on resource subcommands (`database`, `branch`, `sql`, …), not on root `pscale`. *** ### Install skills (all agents) **Setup script (recommended):** ```bash theme={null} git clone https://github.com/planetscale/skills.git cd skills && script/setup ``` The script installs to every agent it finds (`~/.cursor/skills/`, `~/.claude/skills/`, `~/.agents/skills/`). For other agents, pass the skills directory explicitly: ```bash theme={null} script/setup ~/.codex/skills ``` **Skills CLI:** ```bash theme={null} npx skills add planetscale/skills -g -y ``` After installing, load skill **`planetscale-pscale-cli-automation`** for CLI conventions and **`planetscale-safe-orchestrator`** when the user asks for a full PlanetScale assessment. See [github.com/planetscale/skills](https://github.com/planetscale/skills). *** ### Cursor, GitHub Copilot, and VS Code **MCP** — add to `.cursor/mcp.json`, `.vscode/mcp.json`, or your agent's MCP config under `"mcpServers"`: ```json theme={null} "planetscale": { "url": "https://mcp.pscale.dev/mcp/planetscale" } ``` OAuth triggers automatically on first PlanetScale MCP tool use. See [PlanetScale MCP server](/docs/mcp-server) for details. **Skills** — use `script/setup` above or install into `/.cursor/skills/` for project-scoped workflows. *** ### Claude Code Hosted MCP (recommended): ```bash theme={null} claude mcp add --transport http "planetscale" https://mcp.pscale.dev/mcp/planetscale ``` Install skills with `npx skills add planetscale/skills -g -y` or `script/setup`. *** ### Zed Add the hosted MCP server to your Zed settings, using the URL above. See [PlanetScale MCP server](/docs/mcp-server) for the current setup steps. Install skills with `script/setup` or `npx skills add planetscale/skills -g -y`. *** ### Headless / CI (no browser) Use a [PlanetScale service token](/docs/cli/service-tokens) instead of OAuth login. Pass credentials on each command: ```bash theme={null} pscale auth check --format json \ --service-token-id "$PLANETSCALE_SERVICE_TOKEN_ID" \ --service-token "$PLANETSCALE_SERVICE_TOKEN" ``` **Do not retry** these under service-token auth. The failure is by design. See the [service token automation matrix](/docs/cli#service-token-automation) for full guidance. | Command | Exact error | Use instead | | :- | :- | :- | | `pscale org show` | `not authenticated yet. Please run 'pscale auth login'` | `pscale org list --format json`, then `PLANETSCALE_ORG` or `--org` | | `pscale service-token list` | `pscale service-token list is unavailable when authenticated with a service token` | `pscale api organizations//service-tokens --format json` | | `pscale service-token show-access` | `pscale service-token show-access is unavailable when authenticated with a service token` | `pscale api organizations//service-tokens/ --format json` | *** ## Project context (application repositories) In the user's application repo, add or update a **project** `AGENTS.md` with: * PlanetScale organization, database, branch, and engine (Vitess, Neki, or Postgres) * Whether agents may run write SQL or only read-only queries * Approval rules for schema changes, Traffic Control, webhooks, and credentials Do not edit project `AGENTS.md` without user approval. See skill `planetscale-mcp-agent-operating-model` in [planetscale/skills](https://github.com/planetscale/skills). *** ## Verify setup Confirm **`pscale` 0.292.0+** and JSON auth work, then run: ```bash theme={null} pscale agent-guide --format json pscale auth check --format json pscale org list --format json ``` When setup is complete, tell the user: ```text theme={null} PlanetScale Agent Setup Complete ✓ Skills installed (planetscale/skills) ✓ MCP https://mcp.pscale.dev/mcp/planetscale ✓ CLI pscale auth check --format json ⚡ Restart your agent to load MCP servers ``` *** ## Resources | Resource | URL | | - | - | | This prompt (Markdown) | `https://planetscale.com/docs/agent-setup/prompt.md` | | CLI agent guide | `pscale agent-guide --format json` or [CLI agent-guide](/docs/cli/agent-guide) | | PlanetScale skills | [github.com/planetscale/skills](https://github.com/planetscale/skills) | | MCP setup | [PlanetScale MCP server](/docs/mcp-server) | | AI & automation overview | [AI-enhanced development](/docs/ai-tooling) | | `pscale sql` (non-interactive SQL) | [CLI sql](/docs/cli/sql) | These instructions are published at `https://planetscale.com/docs/agent-setup/prompt.md` so you can re-verify their authenticity at any time. # Agents Source: https://planetscale.com/docs/agents Use agents with PlanetScale to answer database questions, investigate performance, and prepare fixes. Connect an agent to PlanetScale to get help with your databases. The hosted MCP server provides access to database information and tools, while PlanetScale skills provide guidance for investigating queries and planning changes. ## PlanetScale Bot for Grok Bot Get started with PlanetScale, ask questions about your database, and investigate slow or failing queries. Connect a codebase to trace database issues back to your code and help fix them via Cloud Agents, or set up webhooks to follow database events. > **[Use the PlanetScale Bot template](https://x.ai/bot/0hgwu8KbM0FIY-idg0x6L)** > > Preview PlanetScale Bot by PlanetScale and add a copy to Grok Bot. > **[Set up Grok Bot](/docs/agents/grok-bot)** > > Install Grok Bot, connect PlanetScale, and configure optional codebase and webhook access. ## Use your own agent > **[Agent setup prompt](/docs/agent-setup/prompt)** > > Set up PlanetScale skills, MCP, and the CLI in your coding agent. > **[PlanetScale MCP server](/docs/mcp-server)** > > Connect an MCP-compatible agent to your PlanetScale account. > **[AI-enhanced development](/docs/ai-tooling)** > > Explore PlanetScale plugins and skills for your development tools. > **[Self-improving database](/docs/self-improving-database)** > > Configure recurring database reviews with a coding agent. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Grok Bot Source: https://planetscale.com/docs/agents/grok-bot Set up PlanetScale Bot to answer database questions, investigate performance, and follow webhook events. PlanetScale Bot helps you get started with PlanetScale, understand database performance, and fix slow or failing queries using its [MCP server](/docs/mcp-server) and installed skills. > **[Add PlanetScale Bot](https://x.ai/bot/0hgwu8KbM0FIY-idg0x6L)** > > Open the PlanetScale Bot by PlanetScale template, then choose Add to Grok Bot. > **[Connect the PlanetScale plugin](https://x.ai/bot/plugin/741)** > > Add PlanetScale to Grok Bot and authorize access to your databases. ## Before you begin You need a PlanetScale account and access to the databases you want to investigate. If you are new to Grok Bot, start with these resources: * [Download Grok Bot](https://x.ai/bot) and [follow the getting-started guide](https://docs.x.ai/grok-bot/get-started). * [Set up Grok Bot on mobile](https://docs.x.ai/grok-bot/mobile). * [Connect plugins](https://cursor.com/help/grok-bot/connect-plugins). ## Add PlanetScale Bot and connect it to PlanetScale **1. Review and add the bot** Open the [PlanetScale Bot template](https://x.ai/bot/0hgwu8KbM0FIY-idg0x6L). You should see PlanetScale Bot by PlanetScale. ![PlanetScale Bot template details in Grok Bot, showing Context, PlanetScale Webhook Triage, and the PlanetScale integration](https://mintcdn.com/planetscale-2/ogPtjo9qMB1g5dXJ/agents/images/grok-bot-template-light.png?fit=max\&auto=format\&n=ogPtjo9qMB1g5dXJ\&q=85\&s=72ee6c4537f6c21ff3710f258c696156) **2. Let the agent set itself up** PlanetScale Bot creates its memories, skills, and **PlanetScale Webhook Triage** routine, then prompts you to install the PlanetScale plugin. The routine starts paused. ![PlanetScale Bot setting itself up and creating the PlanetScale Webhook Triage routine](https://mintcdn.com/planetscale-2/ogPtjo9qMB1g5dXJ/agents/images/grok-bot-setup-light.png?fit=max\&auto=format\&n=ogPtjo9qMB1g5dXJ\&q=85\&s=905f2681d12d818da86d737df969f384) **3. Install the PlanetScale plugin** Accept the PlanetScale plugin installation prompt. If you don't see it, open the [PlanetScale plugin](https://x.ai/bot/plugin/741) and choose **Add**. ![PlanetScale Bot confirming setup is complete and showing the accepted PlanetScale plugin installation prompt](https://mintcdn.com/planetscale-2/ogPtjo9qMB1g5dXJ/agents/images/grok-bot-install-light.png?fit=max\&auto=format\&n=ogPtjo9qMB1g5dXJ\&q=85\&s=8462e22410850199d6b56c0be21e6d9c) **4. Authorize PlanetScale** Choose **Authorize** on the PlanetScale card and follow the PlanetScale authorization flow in your browser. ![PlanetScale installed with an Authorize button on its connection card](https://mintcdn.com/planetscale-2/ogPtjo9qMB1g5dXJ/agents/images/grok-bot-authorize-light.png?fit=max\&auto=format\&n=ogPtjo9qMB1g5dXJ\&q=85\&s=4c79c057156265b298964d6171ce3550) **5. Choose a database** Tell the bot which organization, database, and branch to use. Ask it to verify access with a read-only check before investigating. For example: ```text theme={null} Use PlanetScale organization , database , branch . Confirm you can access it, then explain which queries account for the most execution time in Insights. Show your evidence and suggested next steps. Do not change the database. ``` ## Connect a codebase (optional) With access to your repository, PlanetScale Bot can trace database issues back to your code and help fix them via [Cursor Cloud Agents](https://cursor.com/docs/cloud-agent). Tell the bot which repository to work on. If it is not already connected to Cursor, connect your source control account through [Cursor Integrations](https://cursor.com/dashboard/integrations). See [Work with Grok Bot](https://cursor.com/docs/grok-bot/work) for more. ## Setting up the webhook routine Webhooks are optional. They let PlanetScale Bot investigate events as they arrive, check whether database changes completed, and flag what needs your attention. Without webhooks, the bot responds when you ask. **1. Configure and enable the routine** Ask the bot to configure **PlanetScale Webhook Triage** for your chosen organizations and databases, then turn it on. Open the routine to find its **Webhook URL** and **Authorization header**. ![PlanetScale Webhook Triage routine settings showing the active toggle and webhook endpoint, key, and header with values obscured](https://mintcdn.com/planetscale-2/0gogNjLFt8syD9n_/agents/images/grok-bot-webhook-settings-light.png?fit=max\&auto=format\&n=0gogNjLFt8syD9n_\&q=85\&s=9af534edae64e00d9f157d8d2933b91e) Webhook credentials are hidden in this screenshot. **2. Add the webhook in PlanetScale** In your database's **Settings → Webhooks → Add webhook**, paste the routine's Webhook URL into **URL** and the header value into **Authorization header**, including the scheme, such as `Bearer `, without the `Authorization:` label. ![PlanetScale Add webhook form with URL, Authorization header, and event selection fields](https://mintcdn.com/planetscale-2/Qv8bmPHOBgofISPU/agents/images/grok-bot-add-webhook-light.png?fit=max\&auto=format\&n=Qv8bmPHOBgofISPU\&q=85\&s=60e2c37b6fe9aa35fabb6917aeebe60f) **3. Choose events and test delivery** Choose the [events you want the bot to follow](/docs/api/webhook-events), save the webhook, and use **Test webhook**. Confirm delivery in PlanetScale and acknowledgment in Grok Bot. Webhooks cover supported events; they don't notify the bot about every slow query or CPU spike. See [Setting up webhooks](/docs/api/webhooks) for more details. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # AI-enhanced development Source: https://planetscale.com/docs/ai-tooling Improve your database development experience with AI-powered tooling specifically designed for PlanetScale Vitess and Postgres ## Agents Use the [PlanetScale Bot for Grok Bot](/docs/agents/grok-bot) to answer database questions, investigate performance, and follow webhook events. See [Agents](/docs/agents) for the template and other ways to connect an agent to PlanetScale. ## Agent setup prompt Official bootstrap instructions for AI agents — install skills, configure MCP, and authenticate the CLI. Fetch as Markdown for agent sessions: > **[Agent setup prompt](/docs/agent-setup/prompt)** > > Copy or fetch the [Markdown export](/docs/agent-setup/prompt.md) (`https://planetscale.com/docs/agent-setup/prompt.md` on production) If the PlanetScale CLI is installed (**`pscale` 0.292.0+**), agents can also run: ```bash theme={null} pscale agent-guide --format json pscale auth check --format json ``` Upgrade with `brew upgrade pscale` if `agent-guide` is unknown. See [CLI agent-guide](/docs/cli/agent-guide) for the JSON response fields. ## PlanetScale MCP Server Authenticated MCP servers allow your AI tools to access information specific to your PlanetScale account. Access your PlanetScale organizations, databases, branches, schema, and Insights data from within your development tools. The hosted MCP server can be installed in development tools as well as other MCP-accessible tools like Claude in the web browser and desktop app. > **[PlanetScale MCP Server](/docs/mcp-server)** > > Install in Claude Code, Cursor, VS Code and more ## Automated Database Optimization Use the MCP server with AI coding agents to continuously optimize your database. Agents can review production Insights data and Schema Recommendations, then open pull requests with performance improvements — on a recurring schedule. > **[Self-improving database](/docs/self-improving-database)** > > Set up automated agents that optimize your database daily ## PlanetScale Skills Operational workflows for agents — read-only assessment, safety review, Insights analysis, schema recommendations, Traffic Control, and change gates — live in the public skills pack: > **[planetscale/skills](https://github.com/planetscale/skills)** > > Install assessment and automation skills for Cursor, Claude Code, and more **Setup script (recommended):** ```bash theme={null} git clone https://github.com/planetscale/skills.git cd skills && script/setup ``` **Skills CLI:** ```bash theme={null} npx skills add planetscale/skills -g -y ``` Load **`planetscale-pscale-cli-automation`** for CLI conventions and **`planetscale-safe-orchestrator`** for a full PlanetScale assessment. See the [skills README](https://github.com/planetscale/skills) for usage and the Class A–E safety model. ## Database Skills (MySQL / Vitess query patterns) For MySQL- and Vitess-specific query and schema guidance (separate from the operational skills pack above): ```bash theme={null} npx skills add planetscale/database-skills ``` Check out the [Database Skills](https://database-skills.com/) website for more information. ## Claude Code Plugin The [PlanetScale Claude Code Plugin](https://github.com/planetscale/claude-plugin) is a convenient way to install the PlanetScale MCP server into Claude Code. In Claude Code, add this GitHub repository as a marketplace, then install the plugin: ```bash theme={null} /plugin marketplace add planetscale/claude-plugin /plugin install planetscale@planetscale-plugins ``` You may need to restart your Claude Code session before the MCP server is available. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # OpenAPI Spec Source: https://planetscale.com/docs/api/openapi-spec Download the PlanetScale API OpenAPI 3.0 specification The following specification is for `https://api.planetscale.com/v1`. > **[Download OpenAPI Spec](/docs/openapi.yaml)** > > Download the PlanetScale API OpenAPI 3.0 specification in YAML format. The spec is updated weekly to ensure it is up to date with the latest API changes. # PlanetScale API and OAuth applications Source: https://planetscale.com/docs/api/planetscale-api-oauth-applications The PlanetScale API allows you to manage your PlanetScale databases programmatically. ## PlanetScale API overview The API supports many actions you can take in the PlanetScale web app or CLI and can be used to integrate PlanetScale into your existing workflows and tools. The PlanetScale API does **not** include direct access to the data in the database. Some endpoints will consist of database schema information or connection information. You will still need to use a [client library, object–relational mapping tool (ORM)](/docs/vitess/tutorials/connect-any-application), or the [PlanetScale serverless driver](/docs/vitess/tutorials/planetscale-serverless-driver) to read from and write data to your PlanetScale database. Learn more about the API and how to use it in the [PlanetScale API documentation](/docs/api/reference/getting-started-with-planetscale-api). ## Use cases Some examples of what you can do with the PlanetScale API: * Automatically [create and delete database branches](/docs/api/reference/create_branch) from CI/CD pipelines or data migration tooling * Programmatically build out new environments that connect to PlanetScale database branches for testing * Get information about a PlanetScale user, database, branch, organization, and deploy request * [Check the status of deploy requests](/docs/api/reference/get_deploy_request) in the deploy queue * Automate [creating and deleting database connection strings](/docs/api/reference/create_password) for internal users or tools * [Create, update, approve, deploy, and delete deploy requests](/docs/api/reference/create_deploy_request) programmatically from tooling outside of PlanetScale Anywhere that can programmatically use an HTTP API can be integrated with PlanetScale. This includes CLI tools, build scripts, desktop applications, and more. ## OAuth applications The PlanetScale platform also allows you to create OAuth applications. Creating an OAuth application within PlanetScale allows your application to access your users’ PlanetScale accounts. With PlanetScale OAuth applications, you can choose what access your application needs, and a user will allow (or deny) your application those accesses on their PlanetScale account. The organization that you create the OAuth application in is the "owner" of the application. By using PlanetScale OAuth as a PlanetScale partner or developer, your organization is agreeing to prominently display a privacy policy and obtain consent to your terms of use from all users of your products and services. You can read more about creating an OAuth application in PlanetScale in the [PlanetScale OAuth documentation](/docs/api/reference/oauth). If you build something you would like to share with us, please email us at `education (at) planetscale.com`. We would love to hear about your experience building the application, and we may even feature your application in future blog posts, videos, or social media. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Add a member to a team Source: https://planetscale.com/docs/api/reference/add_organization_team_member post /organizations/{organization}/teams/{team}/members ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_teams` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_organization` | # Assign shards to a configuration profile Source: https://planetscale.com/docs/api/reference/assign_shard_configuration_profile_shards patch /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/shards ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Add shards to a configuration profile Source: https://planetscale.com/docs/api/reference/bulk_create_shard_configuration_profile_shards post /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/shards/bulk ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Cancel a change request for an admin Source: https://planetscale.com/docs/api/reference/cancel_admin_change_request delete /organizations/{organization}/databases/{database}/branches/{branch}/admin/changes/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Cancel a resize request Source: https://planetscale.com/docs/api/reference/cancel_bouncer_resize_request delete /organizations/{organization}/databases/{database}/branches/{branch}/bouncers/{bouncer}/resizes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Cancel a change request Source: https://planetscale.com/docs/api/reference/cancel_branch_change_request delete /organizations/{organization}/databases/{database}/branches/{branch}/resizes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Cancel a queued deploy request Source: https://planetscale.com/docs/api/reference/cancel_deploy_request post /organizations/{organization}/databases/{database}/deploy-requests/{number}/cancel ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Cancel a change request for a branch Source: https://planetscale.com/docs/api/reference/cancel_neki_change_request delete /organizations/{organization}/databases/{database}/branches/{branch}/neki-changes/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Cancel an invitation Source: https://planetscale.com/docs/api/reference/cancel_organization_invitation delete /organizations/{organization}/invitations/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_members` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_members` | # Cancel a change request for a router Source: https://planetscale.com/docs/api/reference/cancel_router_change_request delete /organizations/{organization}/databases/{database}/branches/{branch}/routers/{router}/changes/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Cancel a change request for a configuration profile Source: https://planetscale.com/docs/api/reference/cancel_shard_configuration_profile_change_request delete /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/changes/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Check deploy request storage Source: https://planetscale.com/docs/api/reference/check_deploy_request_storage get /organizations/{organization}/databases/{database}/deploy-requests/{number}/storage-check Checks whether the deploy request's target branch cluster has enough storage to safely deploy the schema changes. ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_deploy_requests` | | Database | `read_deploy_requests` | # Close a deploy request Source: https://planetscale.com/docs/api/reference/close_deploy_request patch /organizations/{organization}/databases/{database}/deploy-requests/{number} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_deploy_requests` | | Database | `write_deploy_requests` | # Complete an errored deploy Source: https://planetscale.com/docs/api/reference/complete_errored_deploy post /organizations/{organization}/databases/{database}/deploy-requests/{number}/complete-deploy ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Complete a gated deploy request Source: https://planetscale.com/docs/api/reference/complete_gated_deploy_request post /organizations/{organization}/databases/{database}/deploy-requests/{number}/apply-deploy ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Complete a revert Source: https://planetscale.com/docs/api/reference/complete_revert post /organizations/{organization}/databases/{database}/deploy-requests/{number}/revert ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Create a backup Source: https://planetscale.com/docs/api/reference/create_backup post /organizations/{organization}/databases/{database}/branches/{branch}/backups ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_backups` | | Database | `write_backups` | | Branch | `write_backups` | Vitess and Postgres Use `get_backup` to poll the current state of a backup. To restore a backup, call `create_branch` with the `backup_id` and a required `cluster_size` (discover valid sizes via `list_cluster_size_skus`). # Create a backup policy Source: https://planetscale.com/docs/api/reference/create_backup_policy post /organizations/{organization}/databases/{database}/backup-policies ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_backups` | | Database | `write_backups` | Vitess and Postgres # Create a bouncer Source: https://planetscale.com/docs/api/reference/create_bouncer post /organizations/{organization}/databases/{database}/branches/{branch}/bouncers ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Create a branch Source: https://planetscale.com/docs/api/reference/create_branch post /organizations/{organization}/databases/{database}/branches ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_branch`, `restore_production_branch_backup`, `restore_backup` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_branches`, `restore_production_branch_backups`, `restore_backups` | | Database | `write_branches`, `restore_production_branch_backups`, `restore_backups` | | Branch | `restore_backups` | Vitess and Postgres This endpoint also facilitates backup restores: pass a `backup_id` (from a backup in the `success` state) together with a required `cluster_size` to restore that backup's schema and data into the new branch. There is no separate restore endpoint. When `backup_id` is set, `region` is ignored — restored branches use the backup's region. Do not hard-code cluster size SKU names — discover valid, enabled sizes with `list_cluster_size_skus`. After creation, poll `get_branch` until `ready` is `true` before connecting, and — for branches restored from a backup — before deleting the source backup or any scratch resources. # Create a database Source: https://planetscale.com/docs/api/reference/create_database post /organizations/{organization}/databases ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_databases` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `create_databases` | Vitess and Postgres Do not hard-code cluster size SKU names — discover them with `list_cluster_size_skus` (filter by the matching `engine` and `enabled: true`; the first enabled non-metal SKU, typically `PS_10`, is a safe default for temporary databases). `cluster_size` is **required for MySQL/Vitess** databases and optional for PostgreSQL. Discover valid `region` slugs with `list_regions_for_organization` and only pick regions where `enabled` is `true`. After creation, poll `get_database` until `ready` is `true` before creating branches or backups. # Create an IP restriction entry Source: https://planetscale.com/docs/api/reference/create_database_postgres_cidr post /organizations/{organization}/databases/{database}/cidrs ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Create a dedicated read replica Source: https://planetscale.com/docs/api/reference/create_dedicated_read_replica post /organizations/{organization}/databases/{database}/branches/{branch}/dedicated-read-replicas ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Create a deploy request Source: https://planetscale.com/docs/api/reference/create_deploy_request post /organizations/{organization}/databases/{database}/deploy-requests ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_deploy_requests` | | Database | `write_deploy_requests` | # Create a keyspace Source: https://planetscale.com/docs/api/reference/create_keyspace post /organizations/{organization}/databases/{database}/branches/{branch}/keyspaces ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `create_branch` # Create or renew an OAuth token Source: https://planetscale.com/docs/api/reference/create_oauth_token post /organizations/{organization}/oauth-applications/{id}/token Create an OAuth token from an authorization grant code, or refresh an OAuth token from a refresh token ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `write_oauth_tokens` > [!WARNING] > > **This endpoint is deprecated.** Please use the new standard OAuth 2.0 token endpoint at `POST https://auth.planetscale.com/oauth/token` instead. See [Create OAuth token](/docs/api/reference/oauth/create-token) for updated documentation. # Send an invitation Source: https://planetscale.com/docs/api/reference/create_organization_invitation post /organizations/{organization}/invitations ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_members` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_members` | # Create an organization team Source: https://planetscale.com/docs/api/reference/create_organization_team post /organizations/{organization}/teams ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_teams` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_organization` | # Create a password Source: https://planetscale.com/docs/api/reference/create_password post /organizations/{organization}/databases/{database}/branches/{branch}/passwords ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | > [!NOTE] > > This endpoint creates a **Vitess/MySQL branch password**. Branch passwords are > the credential system for PlanetScale **Vitess/MySQL** databases. If your > database is **Postgres**, create [role credentials](/docs/api/reference/create_role) > instead — Postgres databases do not have branch passwords. > [!WARNING] > > The `plain_text` password returned in the response is shown **only once** and > cannot be retrieved later. Record it securely as soon as the password is > created. If it is lost, you must [create a new > password](/docs/api/reference/create_password) — renewing only extends expiration and > does not reissue the secret. **Cleanup:** delete a password with [Delete a password](/docs/api/reference/delete_password). Deleting a password immediately disconnects any clients using it. # Create a new query patterns report Source: https://planetscale.com/docs/api/reference/create_query_patterns_report post /organizations/{organization}/databases/{database}/branches/{branch}/query-patterns ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Create role credentials Source: https://planetscale.com/docs/api/reference/create_role post /organizations/{organization}/databases/{database}/branches/{branch}/roles ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_production_branch_password`, `create_production_read_only_branch_password`, `create_branch_password` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | > [!NOTE] > > This endpoint creates **Postgres role credentials**. Roles are the credential > system for PlanetScale **Postgres** databases. If your database is > **Vitess/MySQL**, create a [branch password](/docs/api/reference/create_password) > instead — Vitess/MySQL databases do not have roles. > [!WARNING] > > The `password` returned in the response is shown **only once** and cannot be > retrieved later. Record it securely as soon as the role is created. If it is > lost, rotate it with [Reset role credentials](/docs/api/reference/reset_role). **Cleanup:** delete a role with [Delete a role](/docs/api/reference/delete_role). If the role owns objects, has extra permissions, or created other roles, you must designate a successor — see [Managing roles](/docs/postgres/connecting/roles#deleting-a-role). # Create a router for a branch Source: https://planetscale.com/docs/api/reference/create_router post /organizations/{organization}/databases/{database}/branches/{branch}/routers ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Create a service token Source: https://planetscale.com/docs/api/reference/create_service_token post /organizations/{organization}/service-tokens Create a new service token for the organization. ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `write_service_tokens` # Add shards to a branch Source: https://planetscale.com/docs/api/reference/create_shard post /organizations/{organization}/databases/{database}/branches/{branch}/shards ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Create a configuration profile for a branch Source: https://planetscale.com/docs/api/reference/create_shard_configuration_profile post /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Add a shard to a configuration profile Source: https://planetscale.com/docs/api/reference/create_shard_configuration_profile_shard post /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/shards ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Create a switchover Source: https://planetscale.com/docs/api/reference/create_switchover post /organizations/{organization}/databases/{database}/branches/{branch}/switchovers Moves the primary of a Postgres branch. On a branch with replicas the primary steps down and a replica is promoted in its place. A branch running a single instance has nothing to promote, so that instance is restarted where it stands and the branch is unreachable while it comes back. Read `method` on the response to see which one the branch got. A branch accepts one switchover at a time. A switchover that ends in the `failed` state has an unconfirmed outcome: the primary may still have moved, and nothing is rolled back. ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Create a traffic budget Source: https://planetscale.com/docs/api/reference/create_traffic_budget post /organizations/{organization}/databases/{database}/branches/{branch}/traffic/budgets ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Create a traffic rule Source: https://planetscale.com/docs/api/reference/create_traffic_rule post /organizations/{organization}/databases/{database}/branches/{branch}/traffic/budgets/{budget_id}/rules ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Create a webhook Source: https://planetscale.com/docs/api/reference/create_webhook post /organizations/{organization}/databases/{database}/webhooks ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | Vitess and Postgres # Create workflow Source: https://planetscale.com/docs/api/reference/create_workflow post /organizations/{organization}/databases/{database}/workflows > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Delete a backup Source: https://planetscale.com/docs/api/reference/delete_backup delete /organizations/{organization}/databases/{database}/branches/{branch}/backups/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `delete_backups`, `delete_production_branch_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `delete_backups`, `delete_production_branch_backups` | | Database | `delete_backups`, `delete_production_branch_backups` | | Branch | `delete_backups` | Vitess and Postgres # Delete a backup policy Source: https://planetscale.com/docs/api/reference/delete_backup_policy delete /organizations/{organization}/databases/{database}/backup-policies/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_backups` | | Database | `write_backups` | Vitess and Postgres # Delete a bouncer Source: https://planetscale.com/docs/api/reference/delete_bouncer delete /organizations/{organization}/databases/{database}/branches/{branch}/bouncers/{bouncer} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Delete a branch Source: https://planetscale.com/docs/api/reference/delete_branch delete /organizations/{organization}/databases/{database}/branches/{branch} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `delete_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `delete_branches`, `delete_production_branches` | | Database | `delete_branches`, `delete_production_branches` | | Branch | `delete_branch` | Vitess and Postgres # Delete a database Source: https://planetscale.com/docs/api/reference/delete_database delete /organizations/{organization}/databases/{database} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `delete_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `delete_databases` | | Database | `delete_database` | Vitess and Postgres # Delete an IP restriction entry Source: https://planetscale.com/docs/api/reference/delete_database_postgres_cidr delete /organizations/{organization}/databases/{database}/cidrs/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Delete a dedicated read replica Source: https://planetscale.com/docs/api/reference/delete_dedicated_read_replica delete /organizations/{organization}/databases/{database}/branches/{branch}/dedicated-read-replicas/{dedicated_read_replica} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Delete a keyspace Source: https://planetscale.com/docs/api/reference/delete_keyspace delete /organizations/{organization}/databases/{database}/branches/{branch}/keyspaces/{keyspace} ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `delete_branch`, `delete_production_branch` # Delete an OAuth token Source: https://planetscale.com/docs/api/reference/delete_oauth_token delete /organizations/{organization}/oauth-applications/{application_id}/tokens/{token_id} ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `delete_oauth_tokens` > [!WARNING] > > **This endpoint is deprecated.** Please use the new standard OAuth 2.0 revocation endpoint at `POST https://auth.planetscale.com/oauth/revoke` instead. See [Revoke OAuth token](/docs/api/reference/oauth/revoke-token) for updated documentation. # Delete the current billing payment method Source: https://planetscale.com/docs/api/reference/delete_organization_billing_payment_method delete /organizations/{organization}/billing/payment-method Delete the current payment method for an organization ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_payment_method` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_payment_method` | # Delete an organization team Source: https://planetscale.com/docs/api/reference/delete_organization_team delete /organizations/{organization}/teams/{team} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_teams` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_organization` | # Delete a password Source: https://planetscale.com/docs/api/reference/delete_password delete /organizations/{organization}/databases/{database}/branches/{branch}/passwords/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `delete_production_branch_password`, `delete_production_read_only_branch_password`, `delete_branch_password` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | > [!NOTE] > > This deletes a **Vitess/MySQL** branch password created with [Create a > password](/docs/api/reference/create_password), and immediately disconnects any > clients using it. To delete a **Postgres** role, use [Delete a > role](/docs/api/reference/delete_role) instead. # Delete a query patterns report Source: https://planetscale.com/docs/api/reference/delete_query_patterns_report delete /organizations/{organization}/databases/{database}/branches/{branch}/query-patterns/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Delete role credentials Source: https://planetscale.com/docs/api/reference/delete_role delete /organizations/{organization}/databases/{database}/branches/{branch}/roles/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `delete_production_branch_password`, `delete_production_read_only_branch_password`, `delete_branch_password` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | > [!NOTE] > > This deletes a **Postgres role**. To delete a **Vitess/MySQL** branch password, > use [Delete a password](/docs/api/reference/delete_password) instead. Roles created > with [Create role credentials](/docs/api/reference/create_role) that own objects, > have extra permissions, or created other roles require a successor — see > [Deleting a role](/docs/postgres/connecting/roles#deleting-a-role). # Delete a router from a branch Source: https://planetscale.com/docs/api/reference/delete_router delete /organizations/{organization}/databases/{database}/branches/{branch}/routers/{router} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Delete a service token Source: https://planetscale.com/docs/api/reference/delete_service_token delete /organizations/{organization}/service-tokens/{id} Delete a service token from the organization. ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `delete_service_tokens` # Remove a shard from a branch Source: https://planetscale.com/docs/api/reference/delete_shard delete /organizations/{organization}/databases/{database}/branches/{branch}/shards/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Delete a configuration profile for a branch Source: https://planetscale.com/docs/api/reference/delete_shard_configuration_profile delete /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Remove a shard from a configuration profile Source: https://planetscale.com/docs/api/reference/delete_shard_configuration_profile_shard delete /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/shards/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Remove shards from a configuration profile Source: https://planetscale.com/docs/api/reference/delete_shard_configuration_profile_shards delete /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/shards ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Delete a traffic budget Source: https://planetscale.com/docs/api/reference/delete_traffic_budget delete /organizations/{organization}/databases/{database}/branches/{branch}/traffic/budgets/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Delete a traffic rule Source: https://planetscale.com/docs/api/reference/delete_traffic_rule delete /organizations/{organization}/databases/{database}/branches/{branch}/traffic/budgets/{budget_id}/rules/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Delete a webhook Source: https://planetscale.com/docs/api/reference/delete_webhook delete /organizations/{organization}/databases/{database}/webhooks/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | Vitess and Postgres # Demote a branch Source: https://planetscale.com/docs/api/reference/demote_branch post /organizations/{organization}/databases/{database}/branches/{branch}/demote Demotes a branch from production to development ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `connect_production_branch`, `demote_branches` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `demote_branches` | | Database | `demote_branches` | # Disable safe migrations for a branch Source: https://planetscale.com/docs/api/reference/disable_safe_migrations delete /organizations/{organization}/databases/{database}/branches/{branch}/safe-migrations # Dismiss a schema recommendation Source: https://planetscale.com/docs/api/reference/dismiss_schema_recommendation post /organizations/{organization}/databases/{database}/schema-recommendations/{number}/dismiss ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | Vitess and Postgres # Enable safe migrations for a branch Source: https://planetscale.com/docs/api/reference/enable_safe_migrations post /organizations/{organization}/databases/{database}/branches/{branch}/safe-migrations # Enable force cutover for a deploy request Source: https://planetscale.com/docs/api/reference/force_cutover_deploy_request post /organizations/{organization}/databases/{database}/deploy-requests/{number}/force-cutover ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Get the admin for a branch Source: https://planetscale.com/docs/api/reference/get_admin get /organizations/{organization}/databases/{database}/branches/{branch}/admin ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a change request for an admin Source: https://planetscale.com/docs/api/reference/get_admin_change_request get /organizations/{organization}/databases/{database}/branches/{branch}/admin/changes/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a backup Source: https://planetscale.com/docs/api/reference/get_backup get /organizations/{organization}/databases/{database}/branches/{branch}/backups/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_backups` | | Database | `read_backups` | | Branch | `read_backups` | Vitess and Postgres Poll this endpoint after `create_backup` until `state` is `success` before restoring. `pending` and `running` are transient; `failed`, `canceled`, and `ignored` are terminal — stop retrying. # Get a backup policy Source: https://planetscale.com/docs/api/reference/get_backup_policy get /organizations/{organization}/databases/{database}/backup-policies/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_backups` | | Database | `read_backups` | Vitess and Postgres # Get a bouncer Source: https://planetscale.com/docs/api/reference/get_bouncer get /organizations/{organization}/databases/{database}/branches/{branch}/bouncers/{bouncer} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a branch Source: https://planetscale.com/docs/api/reference/get_branch get /organizations/{organization}/databases/{database}/branches/{branch} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Retrieve a branch anomaly Source: https://planetscale.com/docs/api/reference/get_branch_anomaly get /organizations/{organization}/databases/{database}/branches/{branch}/insights/anomalies/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # Get a branch change request Source: https://planetscale.com/docs/api/reference/get_branch_change_request get /organizations/{organization}/databases/{database}/branches/{branch}/changes/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get table storage metrics by keyspace Source: https://planetscale.com/docs/api/reference/get_branch_keyspace_table_metrics get /organizations/{organization}/databases/{database}/branches/{branch}/metrics/keyspace-tables ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get time-series metrics Source: https://planetscale.com/docs/api/reference/get_branch_metrics get /organizations/{organization}/databases/{database}/branches/{branch}/metrics Retrieve time-series metrics for a database branch. ```sh curl --get \ --header "Authorization: $PLANETSCALE_SERVICE_TOKEN" \ --data-urlencode "metrics=queries" \ --data-urlencode "period=15m" \ "https://api.planetscale.com/v1/organizations/acme/databases/app-db/branches/main/metrics" ``` Each series contains its metric name, display label, identifying labels, and sampled `[Unix timestamp, value]` points: ```json { "type": "MetricSeries", "start_date": "2026-09-09T17:00:00Z", "end_date": "2026-09-09T17:15:00Z", "interval": 60, "series": [ { "type": "TimeSeries", "metric": "queries", "label": "Queries", "labels": { "tablet_type": "primary" }, "points": [[1788973200, 42.0]] } ] } ``` ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Retrieve a branch query Source: https://planetscale.com/docs/api/reference/get_branch_query get /organizations/{organization}/databases/{database}/branches/{branch}/insights/queries/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # List queries for an error fingerprint Source: https://planetscale.com/docs/api/reference/get_branch_query_error get /organizations/{organization}/databases/{database}/branches/{branch}/insights/errors/{fingerprint} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # Get time-series metrics for SQL queries Source: https://planetscale.com/docs/api/reference/get_branch_query_metrics get /organizations/{organization}/databases/{database}/branches/{branch}/metrics/query ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Retrieve a query tag Source: https://planetscale.com/docs/api/reference/get_branch_query_tag get /organizations/{organization}/databases/{database}/branches/{branch}/insights/tags/{tag} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # Get a branch schema Source: https://planetscale.com/docs/api/reference/get_branch_schema get /organizations/{organization}/databases/{database}/branches/{branch}/schema ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Get table storage metrics Source: https://planetscale.com/docs/api/reference/get_branch_table_metrics get /organizations/{organization}/databases/{database}/branches/{branch}/metrics/tables ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Get time-series metrics grouped by query tags Source: https://planetscale.com/docs/api/reference/get_branch_tag_metrics get /organizations/{organization}/databases/{database}/branches/{branch}/metrics/tag ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Get current user Source: https://planetscale.com/docs/api/reference/get_current_user get /user Get the user associated with this service token ### Authorization A OAuth token must have at least one of the following scopes in order to use this API endpoint: **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | User | `read_user` | # Get the data topology for a branch Source: https://planetscale.com/docs/api/reference/get_data_topology get /organizations/{organization}/databases/{database}/branches/{branch}/data-topology ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a database Source: https://planetscale.com/docs/api/reference/get_database get /organizations/{organization}/databases/{database} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database`, `delete_database`, `write_database`, `read_branch`, `delete_branch`, `create_branch`, `promote_branches`, `demote_branches`, `delete_production_branch`, `connect_branch`, `connect_production_branch`, `connect_production_read_only_branch`, `delete_branch_password`, `delete_production_branch_password`, `delete_production_read_only_branch_password`, `read_deploy_request`, `create_deploy_request`, `approve_deploy_request`, `read_comment`, `create_comment`, `restore_backup`, `restore_production_branch_backup`, `read_backups`, `write_backups`, `delete_backups`, `delete_production_branch_backups`, `write_branch_vschema`, `write_production_branch_vschema`, `write_branch_topology`, `write_production_branch_topology`, `read_workflow`, `write_workflow`, `delete_workflow` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # Get an IP restriction entry Source: https://planetscale.com/docs/api/reference/get_database_postgres_cidr get /organizations/{organization}/databases/{database}/cidrs/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database`, `read_databases` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | # Get database throttler configurations Source: https://planetscale.com/docs/api/reference/get_database_throttler get /organizations/{organization}/databases/{database}/throttler ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_deploy_request`, `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | # Get a dedicated read replica Source: https://planetscale.com/docs/api/reference/get_dedicated_read_replica get /organizations/{organization}/databases/{database}/branches/{branch}/dedicated-read-replicas/{dedicated_read_replica} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get the default postgres role Source: https://planetscale.com/docs/api/reference/get_default_role get /organizations/{organization}/databases/{database}/branches/{branch}/roles/default ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # Get the default configuration profile for a branch Source: https://planetscale.com/docs/api/reference/get_default_shard_configuration_profile get /organizations/{organization}/databases/{database}/branches/{branch}/default-configuration-profile ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get the deploy queue Source: https://planetscale.com/docs/api/reference/get_deploy_queue get /organizations/{organization}/databases/{database}/deploy-queue The deploy queue returns the current list of deploy requests in the order they will be deployed. # Get a deploy request Source: https://planetscale.com/docs/api/reference/get_deploy_request get /organizations/{organization}/databases/{database}/deploy-requests/{number} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_deploy_requests` | | Database | `read_deploy_requests` | # Get deploy request throttler configurations Source: https://planetscale.com/docs/api/reference/get_deploy_request_throttler get /organizations/{organization}/databases/{database}/deploy-requests/{number}/throttler ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_deploy_requests` | | Database | `read_deploy_requests` | # Get a deployment Source: https://planetscale.com/docs/api/reference/get_deployment get /organizations/{organization}/databases/{database}/deploy-requests/{number}/deployment Get the deployment for a deploy request ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_deploy_requests` | | Database | `read_deploy_requests` | # Get current metric values Source: https://planetscale.com/docs/api/reference/get_instant_branch_metrics get /organizations/{organization}/databases/{database}/branches/{branch}/metrics/instant ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Get current tablet metrics Source: https://planetscale.com/docs/api/reference/get_instant_tablet_metrics get /organizations/{organization}/databases/{database}/branches/{branch}/metrics/tablets-instant ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get an invoice Source: https://planetscale.com/docs/api/reference/get_invoice get /organizations/{organization}/invoices/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_invoices` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_invoices` | # Get invoice line items Source: https://planetscale.com/docs/api/reference/get_invoice_line_items get /organizations/{organization}/invoices/{id}/line-items Get the line items for an invoice ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_invoices` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_invoices` | # Get a keyspace Source: https://planetscale.com/docs/api/reference/get_keyspace get /organizations/{organization}/databases/{database}/branches/{branch}/keyspaces/{keyspace} ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `read_branch` # Get keyspace rollout status Source: https://planetscale.com/docs/api/reference/get_keyspace_rollout_status get /organizations/{organization}/databases/{database}/branches/{branch}/keyspaces/{keyspace}/rollout-status ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `read_branch` # Get the VSchema for the keyspace Source: https://planetscale.com/docs/api/reference/get_keyspace_vschema get /organizations/{organization}/databases/{database}/branches/{branch}/keyspaces/{keyspace}/vschema ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` # Get a maintenance schedule Source: https://planetscale.com/docs/api/reference/get_maintenance_schedule get /organizations/{organization}/databases/{database}/maintenance-schedules/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | # Get a change request for a branch Source: https://planetscale.com/docs/api/reference/get_neki_change_request get /organizations/{organization}/databases/{database}/branches/{branch}/neki-changes/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get an OAuth application Source: https://planetscale.com/docs/api/reference/get_oauth_application get /organizations/{organization}/oauth-applications/{application_id} ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `read_oauth_applications` # Get an OAuth token Source: https://planetscale.com/docs/api/reference/get_oauth_token get /organizations/{organization}/oauth-applications/{application_id}/tokens/{token_id} ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `read_oauth_tokens` > [!WARNING] > > **This endpoint is deprecated.** Please use the new standard OAuth 2.0 token info endpoint at `GET https://auth.planetscale.com/oauth/token/info` instead. See [Get OAuth token info](/docs/api/reference/oauth/token-info) for updated documentation. # Get an organization Source: https://planetscale.com/docs/api/reference/get_organization get /organizations/{organization} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | User | `read_organizations` | | Organization | `read_organization` | # Get the current billing payment method Source: https://planetscale.com/docs/api/reference/get_organization_billing_payment_method get /organizations/{organization}/billing/payment-method Get the current payment method for an organization ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_payment_method` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_payment_method` | # Get an organization member Source: https://planetscale.com/docs/api/reference/get_organization_membership get /organizations/{organization}/members/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_organization` | # Get an organization team Source: https://planetscale.com/docs/api/reference/get_organization_team get /organizations/{organization}/teams/{team} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_organization` | # Get a team member Source: https://planetscale.com/docs/api/reference/get_organization_team_member get /organizations/{organization}/teams/{team}/members/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_organization` | # Get a password Source: https://planetscale.com/docs/api/reference/get_password get /organizations/{organization}/databases/{database}/branches/{branch}/passwords/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # Download a finished query patterns report Source: https://planetscale.com/docs/api/reference/get_query_patterns_report get /organizations/{organization}/databases/{database}/branches/{branch}/query-patterns/{id}/download ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Show the status of a query patterns report Source: https://planetscale.com/docs/api/reference/get_query_patterns_report_status get /organizations/{organization}/databases/{database}/branches/{branch}/query-patterns/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Retrieve query statistics Source: https://planetscale.com/docs/api/reference/get_query_statistics get /organizations/{organization}/databases/{database}/branches/{branch}/insights/{fingerprint} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # Retrieve a summary of query statistics Source: https://planetscale.com/docs/api/reference/get_query_summary get /organizations/{organization}/databases/{database}/branches/{branch}/insights/{fingerprint}/summary ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # Get a role Source: https://planetscale.com/docs/api/reference/get_role get /organizations/{organization}/databases/{database}/branches/{branch}/roles/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # Get a router for a branch Source: https://planetscale.com/docs/api/reference/get_router get /organizations/{organization}/databases/{database}/branches/{branch}/routers/{router} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a change request for a router Source: https://planetscale.com/docs/api/reference/get_router_change_request get /organizations/{organization}/databases/{database}/branches/{branch}/routers/{router}/changes/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a schema recommendation Source: https://planetscale.com/docs/api/reference/get_schema_recommendation get /organizations/{organization}/databases/{database}/schema-recommendations/{number} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # Get a service token Source: https://planetscale.com/docs/api/reference/get_service_token get /organizations/{organization}/service-tokens/{id} Get information about a service token. ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `read_service_tokens` # Get a shard for a branch Source: https://planetscale.com/docs/api/reference/get_shard get /organizations/{organization}/databases/{database}/branches/{branch}/shards/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a configuration profile for a branch Source: https://planetscale.com/docs/api/reference/get_shard_configuration_profile get /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a change request for a configuration profile Source: https://planetscale.com/docs/api/reference/get_shard_configuration_profile_change_request get /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/changes/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a shard in a configuration profile Source: https://planetscale.com/docs/api/reference/get_shard_configuration_profile_shard get /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/shards/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a switchover Source: https://planetscale.com/docs/api/reference/get_switchover get /organizations/{organization}/databases/{database}/branches/{branch}/switchovers/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get time-series tablet metrics Source: https://planetscale.com/docs/api/reference/get_tablet_metrics get /organizations/{organization}/databases/{database}/branches/{branch}/metrics/tablets ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a traffic budget Source: https://planetscale.com/docs/api/reference/get_traffic_budget get /organizations/{organization}/databases/{database}/branches/{branch}/traffic/budgets/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get a webhook Source: https://planetscale.com/docs/api/reference/get_webhook get /organizations/{organization}/databases/{database}/webhooks/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # Get workflow Source: https://planetscale.com/docs/api/reference/get_workflow get /organizations/{organization}/databases/{database}/workflows/{number} > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. Workflows created with MoveTables don't appear in this endpoint's responses. # Getting started with PlanetScale API Source: https://planetscale.com/docs/api/reference/getting-started-with-planetscale-api Learn how to start using the PlanetScale API. ## Overview You can use the PlanetScale API to manage your PlanetScale databases programmatically. The PlanetScale API does **not** include direct access to the data in the database. Some endpoints will consist of database schema information or connection information. ## Authentication Before making your first API call, set up the proper authentication for the PlanetScale API using the `Authorization` header. There are two API authentication types: **Service tokens** and **OAuth**. ### Service tokens Most endpoints only need a service token for authentication, but some organization-specific endpoints also need OAuth. Each endpoint will state what types of authentication are allowed. See the [Service tokens documentation](/docs/api/reference/service-tokens) for creating a service token and making your first API call with the PlanetScale API. ### OAuth applications All OAuth applications have a comprehensive list of scopes that the application can request from the PlanetScale user. See the [OAuth documentation](/docs/api/reference/oauth) for more info. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Lint a branch schema Source: https://planetscale.com/docs/api/reference/lint_branch_schema get /organizations/{organization}/databases/{database}/branches/{branch}/schema/lint ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List change requests for an admin Source: https://planetscale.com/docs/api/reference/list_admin_change_requests get /organizations/{organization}/databases/{database}/branches/{branch}/admin/changes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List parameters for an admin Source: https://planetscale.com/docs/api/reference/list_admin_parameters get /organizations/{organization}/databases/{database}/branches/{branch}/admin/parameters ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List admin size SKUs for a branch Source: https://planetscale.com/docs/api/reference/list_admin_size_skus get /organizations/{organization}/databases/{database}/branches/{branch}/admin-size-skus ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List audit logs Source: https://planetscale.com/docs/api/reference/list_audit_logs get /organizations/{organization}/audit-log ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_audit_logs` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_audit_logs` | # List backup policies Source: https://planetscale.com/docs/api/reference/list_backup_policies get /organizations/{organization}/databases/{database}/backup-policies ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_backups` | | Database | `read_backups` | Vitess and Postgres # List backups Source: https://planetscale.com/docs/api/reference/list_backups get /organizations/{organization}/databases/{database}/branches/{branch}/backups ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_backups` | | Database | `read_backups` | | Branch | `read_backups` | Vitess and Postgres # Get bouncer resize requests Source: https://planetscale.com/docs/api/reference/list_bouncer_resize_requests get /organizations/{organization}/databases/{database}/branches/{branch}/bouncers/{bouncer}/resizes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List bouncers Source: https://planetscale.com/docs/api/reference/list_bouncers get /organizations/{organization}/databases/{database}/branches/{branch}/bouncers ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List branch anomalies Source: https://planetscale.com/docs/api/reference/list_branch_anomalies get /organizations/{organization}/databases/{database}/branches/{branch}/insights/anomalies ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # Get bouncer resize requests Source: https://planetscale.com/docs/api/reference/list_branch_bouncer_resize_requests get /organizations/{organization}/databases/{database}/branches/{branch}/bouncer-resizes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # Get branch change requests Source: https://planetscale.com/docs/api/reference/list_branch_change_requests get /organizations/{organization}/databases/{database}/branches/{branch}/changes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List branch queries Source: https://planetscale.com/docs/api/reference/list_branch_queries get /organizations/{organization}/databases/{database}/branches/{branch}/insights ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # List branch query errors Source: https://planetscale.com/docs/api/reference/list_branch_query_errors get /organizations/{organization}/databases/{database}/branches/{branch}/insights/errors ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # List query statistics grouped by tag Source: https://planetscale.com/docs/api/reference/list_branch_query_tag_summaries get /organizations/{organization}/databases/{database}/branches/{branch}/insights/tags/summaries ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # List query tags Source: https://planetscale.com/docs/api/reference/list_branch_query_tags get /organizations/{organization}/databases/{database}/branches/{branch}/insights/tags ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_databases`, `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # List branches Source: https://planetscale.com/docs/api/reference/list_branches get /organizations/{organization}/databases/{database}/branches ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # List available cluster sizes Source: https://planetscale.com/docs/api/reference/list_cluster_size_skus get /organizations/{organization}/cluster-size-skus List available cluster sizes for an organization ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | User | `read_organizations` | | Organization | `read_organization` | Vitess and Postgres SKU names are not a closed enum and vary by organization, engine, and region. Always discover them from this endpoint rather than hard-coding values. # List IP restriction entries Source: https://planetscale.com/docs/api/reference/list_database_postgres_cidrs get /organizations/{organization}/databases/{database}/cidrs ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database`, `read_databases` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | # List database regions Source: https://planetscale.com/docs/api/reference/list_database_regions get /organizations/{organization}/databases/{database}/regions ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database`, `delete_database`, `write_database`, `read_branch`, `delete_branch`, `create_branch`, `promote_branches`, `demote_branches`, `delete_production_branch`, `connect_branch`, `connect_production_branch`, `connect_production_read_only_branch`, `delete_branch_password`, `delete_production_branch_password`, `delete_production_read_only_branch_password`, `read_deploy_request`, `create_deploy_request`, `approve_deploy_request`, `read_comment`, `create_comment`, `restore_backup`, `restore_production_branch_backup`, `read_backups`, `write_backups`, `delete_backups`, `delete_production_branch_backups`, `write_branch_vschema`, `write_production_branch_vschema`, `write_branch_topology`, `write_production_branch_topology`, `read_workflow`, `write_workflow`, `delete_workflow` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres Results are limited to regions on the same cloud provider as the database's default branch. Use the `slug` of a region where `enabled` is `true` as the `region` parameter when creating branches; if omitted, the database's own region is used. # List databases Source: https://planetscale.com/docs/api/reference/list_databases get /organizations/{organization}/databases ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database`, `delete_database`, `write_database`, `read_branch`, `delete_branch`, `create_branch`, `promote_branches`, `demote_branches`, `delete_production_branch`, `connect_branch`, `connect_production_branch`, `connect_production_read_only_branch`, `delete_branch_password`, `delete_production_branch_password`, `delete_production_read_only_branch_password`, `read_deploy_request`, `create_deploy_request`, `approve_deploy_request`, `read_comment`, `create_comment`, `restore_backup`, `restore_production_branch_backup`, `read_backups`, `write_backups`, `delete_backups`, `delete_production_branch_backups`, `write_branch_vschema`, `write_production_branch_vschema`, `write_branch_topology`, `write_production_branch_topology`, `read_workflow`, `write_workflow`, `delete_workflow` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | Vitess and Postgres # Get dedicated read replica change requests Source: https://planetscale.com/docs/api/reference/list_dedicated_read_replica_change_requests get /organizations/{organization}/databases/{database}/branches/{branch}/dedicated-read-replica-changes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List dedicated read replicas Source: https://planetscale.com/docs/api/reference/list_dedicated_read_replicas get /organizations/{organization}/databases/{database}/branches/{branch}/dedicated-read-replicas ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List deploy operations Source: https://planetscale.com/docs/api/reference/list_deploy_operations get /organizations/{organization}/databases/{database}/deploy-requests/{number}/operations List deploy operations for a deploy request ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_deploy_requests` | | Database | `read_deploy_requests` | # List deploy request reviews Source: https://planetscale.com/docs/api/reference/list_deploy_request_reviews get /organizations/{organization}/databases/{database}/deploy-requests/{number}/reviews ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_deploy_requests` | | Database | `read_deploy_requests` | # List deploy requests Source: https://planetscale.com/docs/api/reference/list_deploy_requests get /organizations/{organization}/databases/{database}/deploy-requests List deploy requests for a database ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_deploy_requests` | | Database | `read_deploy_requests` | # List cluster extensions Source: https://planetscale.com/docs/api/reference/list_extensions get /organizations/{organization}/databases/{database}/branches/{branch}/extensions ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List generated query patterns reports Source: https://planetscale.com/docs/api/reference/list_generated_query_patterns_reports get /organizations/{organization}/databases/{database}/branches/{branch}/query-patterns ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | Vitess and Postgres # Get invoices Source: https://planetscale.com/docs/api/reference/list_invoices get /organizations/{organization}/invoices Get the invoices for an organization ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_invoices` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_invoices` | # Get keyspaces Source: https://planetscale.com/docs/api/reference/list_keyspaces get /organizations/{organization}/databases/{database}/branches/{branch}/keyspaces ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `read_branch` # List maintenance schedules Source: https://planetscale.com/docs/api/reference/list_maintenance_schedules get /organizations/{organization}/databases/{database}/maintenance-schedules ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | # List maintenance windows Source: https://planetscale.com/docs/api/reference/list_maintenance_windows get /organizations/{organization}/databases/{database}/maintenance-schedules/{id}/windows ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | # List change requests for a branch Source: https://planetscale.com/docs/api/reference/list_neki_change_requests get /organizations/{organization}/databases/{database}/branches/{branch}/neki-changes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List OAuth applications Source: https://planetscale.com/docs/api/reference/list_oauth_applications get /organizations/{organization}/oauth-applications ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `read_oauth_applications` # List pending invitations Source: https://planetscale.com/docs/api/reference/list_organization_invitations get /organizations/{organization}/invitations ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_members` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_members` | # List organization members Source: https://planetscale.com/docs/api/reference/list_organization_members get /organizations/{organization}/members ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_organization` | # List team members Source: https://planetscale.com/docs/api/reference/list_organization_team_members get /organizations/{organization}/teams/{team}/members ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_organization` | # List teams in an organization Source: https://planetscale.com/docs/api/reference/list_organization_teams get /organizations/{organization}/teams ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_organization` | # List organizations Source: https://planetscale.com/docs/api/reference/list_organizations get /organizations When using a service token, returns the list of organizations the service token has access to. When using an OAuth token, returns the list of organizations the user has access to. ### Authorization A OAuth token must have at least one of the following scopes in order to use this API endpoint: **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | User | `read_organizations` | # List cluster parameters Source: https://planetscale.com/docs/api/reference/list_parameters get /organizations/{organization}/databases/{database}/branches/{branch}/parameters Returns the parameters for a branch. To update the parameters, use the "update_branch_change_request" endpoint. ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List passwords Source: https://planetscale.com/docs/api/reference/list_passwords get /organizations/{organization}/databases/{database}/branches/{branch}/passwords ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # List public regions Source: https://planetscale.com/docs/api/reference/list_public_regions get /regions Endpoint is available without authentication. # List read-only regions Source: https://planetscale.com/docs/api/reference/list_read_only_regions get /organizations/{organization}/databases/{database}/read-only-regions List read-only regions for the database's default branch ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database`, `delete_database`, `write_database`, `read_branch`, `delete_branch`, `create_branch`, `promote_branches`, `demote_branches`, `delete_production_branch`, `connect_branch`, `connect_production_branch`, `connect_production_read_only_branch`, `delete_branch_password`, `delete_production_branch_password`, `delete_production_read_only_branch_password`, `read_deploy_request`, `create_deploy_request`, `approve_deploy_request`, `read_comment`, `create_comment`, `restore_backup`, `restore_production_branch_backup`, `read_backups`, `write_backups`, `delete_backups`, `delete_production_branch_backups`, `write_branch_vschema`, `write_production_branch_vschema`, `write_branch_topology`, `write_production_branch_topology`, `read_workflow`, `write_workflow`, `delete_workflow` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | # List regions for an organization Source: https://planetscale.com/docs/api/reference/list_regions_for_organization get /organizations/{organization}/regions ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | User | `read_organizations` | | Organization | `read_organization` | Call this before `create_database` to discover valid `region` slugs. Pick a region where `enabled` is `true` and where the field matching your engine (`mysql_supported` or `postgresql_supported`) is `true`. The entry with `current_default: true` is the organization default used when `region` is omitted. # List roles Source: https://planetscale.com/docs/api/reference/list_roles get /organizations/{organization}/databases/{database}/branches/{branch}/roles ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # List change requests for a router Source: https://planetscale.com/docs/api/reference/list_router_change_requests get /organizations/{organization}/databases/{database}/branches/{branch}/routers/{router}/changes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List parameters for a router Source: https://planetscale.com/docs/api/reference/list_router_parameters get /organizations/{organization}/databases/{database}/branches/{branch}/routers/{router}/parameters ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List router size SKUs for a branch Source: https://planetscale.com/docs/api/reference/list_router_size_skus get /organizations/{organization}/databases/{database}/branches/{branch}/router-size-skus ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List routers for a branch Source: https://planetscale.com/docs/api/reference/list_routers get /organizations/{organization}/databases/{database}/branches/{branch}/routers ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List schema recommendations Source: https://planetscale.com/docs/api/reference/list_schema_recommendations get /organizations/{organization}/databases/{database}/schema-recommendations ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # List service tokens Source: https://planetscale.com/docs/api/reference/list_service_tokens get /organizations/{organization}/service-tokens List service tokens for an organization. ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `read_service_tokens` # List change requests for a configuration profile Source: https://planetscale.com/docs/api/reference/list_shard_configuration_profile_change_requests get /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/changes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List extensions for a configuration profile Source: https://planetscale.com/docs/api/reference/list_shard_configuration_profile_extensions get /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/extensions ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List parameters for a configuration profile Source: https://planetscale.com/docs/api/reference/list_shard_configuration_profile_parameters get /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/parameters ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List shards for a configuration profile Source: https://planetscale.com/docs/api/reference/list_shard_configuration_profile_shards get /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/shards ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List configuration profiles for a branch Source: https://planetscale.com/docs/api/reference/list_shard_configuration_profiles get /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List shards for a branch Source: https://planetscale.com/docs/api/reference/list_shards get /organizations/{organization}/databases/{database}/branches/{branch}/shards ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List switchovers Source: https://planetscale.com/docs/api/reference/list_switchovers get /organizations/{organization}/databases/{database}/branches/{branch}/switchovers ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List traffic budgets Source: https://planetscale.com/docs/api/reference/list_traffic_budgets get /organizations/{organization}/databases/{database}/branches/{branch}/traffic/budgets ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List traffic budgets affecting a query fingerprint Source: https://planetscale.com/docs/api/reference/list_traffic_budgets_for_query get /organizations/{organization}/databases/{database}/branches/{branch}/insights/{fingerprint}/traffic/budgets ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_branch`, `delete_branch`, `create_branch`, `connect_production_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_branches` | | Database | `read_branches` | | Branch | `read_branch` | # List webhooks Source: https://planetscale.com/docs/api/reference/list_webhooks get /organizations/{organization}/databases/{database}/webhooks List webhooks for a database ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `read_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `read_databases` | | Database | `read_database` | Vitess and Postgres # List workflows Source: https://planetscale.com/docs/api/reference/list_workflows get /organizations/{organization}/databases/{database}/workflows > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. Workflows created with MoveTables don't appear in this endpoint's responses. # OAuth Source: https://planetscale.com/docs/api/reference/oauth Using PlanetScale OAuth enables your users to connect their accounts to PlanetScale. ## Overview Creating an OAuth application within PlanetScale allows your application to access your users’ PlanetScale accounts. With PlanetScale OAuth applications, you can choose what access your application needs, and a user will allow (or deny) your application those accesses on their PlanetScale account. The organization that you create the OAuth application in is the "owner" of the application. ## Getting started ### 1. Creating an OAuth application in PlanetScale 1. To create a new OAuth application, log into your organization and click **Settings > OAuth applications**. 2. Create a new OAuth application by clicking **Create new application**. 3. You will need to fill out the following fields: * **Name**: A user-friendly name for your OAuth application. * **Domain**: The full URL to your application's domain. * **Redirect URI**: The full URL PlanetScale should redirect users on completion of the authorization flow, also known as the callback URL. It must have the same domain as the domain above. * **Avatar**: An image that represents your OAuth application. (Optional but recommended.) > [!WARNING] > > You will also be agreeing, on behalf of your organization, to prominently display a privacy policy and obtain consent to your organization's terms of use from all users of your products and services. ### 2. Credentials to copy to your application code Once you have created your OAuth application in PlanetScale, you will need the following credentials to use the OAuth authorization flow: * **ID**: Your OAuth application's ID. * **Client ID**: Your OAuth application's client ID. * **Redirect URL**: The full URL PlanetScale should redirect users on completion of the authorization flow, also known as the callback URL. * **Client secret**: Your OAuth application's client secret, used to exchange authorization codes for access tokens. (This will only be shown once, make sure to save it!) Later in this document, we will go through how you use each of these credentials. We recommend saving them as environment variables. ### 3. OAuth application access scopes Every OAuth application in PlanetScale will request from its users a specific set of permissions in the users' databases. We call these permissions "access scopes." They are broken into: * User access * Organization access * Database access * Branch access Access is scoped to a resource. For example, selecting `write_branches` on an organization allows you to write branches across all databases in organizations the user gives permission to, while `write_branches` on a database enables you to only write branches in databases the user gives permission to. The API reference for each endpoint will say what scope is needed. In this step, select the access scopes you think your application will need on a user's account and click the **Save access scopes** button. ![OAuth access scopes selection interface](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/reference/31ea0ef-CleanShot_2024-02-01_at_15.32.27.jpg?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=dc6795cb7acae69e9110f5dc4e083ffc) View OAuth access scopes documentation [Open](/docs/api/reference/oauth-access-scopes) This is only a partial list of the OAuth access scopes. For a full list of scopes, see the OAuth access scopes documentation. ## OAuth Authorization Flow PlanetScale's OAuth implementation supports the [Authorization Code grant type](https://oauth.net/2/grant-types/authorization-code/). The following diagram walks through the flow. ![OAuth authorization flow diagram](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/reference/c46b041-oauth_diagram.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=5a79e6b26ef6501413ca162638a3cfc6) OAuth authorization flow diagram ### 0. Prerequisites You must have created an OAuth application in PlanetScale (see step 1 above) and saved your: * **Client ID**: Your OAuth application's client ID * **Client Secret**: Your OAuth application's client secret * **Redirect URI**: The callback URL for your application These credentials will be used to exchange authorization codes for access tokens. ### 1. User authorizes your OAuth application on their account Your application should direct your users to the PlanetScale authorization page (see URL below) so that they can grant your application access to their PlanetScale account: ```text Text theme={null} https://auth.planetscale.com/oauth/authorize?client_id=CLIENT_ID&redirect_uri=REDIRECT_URI&state=STATE ``` ### Query parameters: **client\_id** `string` *required* Your OAuth application's client id **redirect\_uri** `string` *required* The full URL PlanetScale should redirect users on completion of the authorization flow, also known as the callback URL **state** `string` You may also optionally pass a state parameter, which exists to prevent third-party attacks. Pass a random string, and PlanetScale will return it in step 2. Compare to ensure the request came from your application. ## 2. The authorization code returns to your application Upon authorization, PlanetScale will redirect the user to your `redirect_uri` with an authorization code in the query parameters. The authorization code is only good for one use. It will look like the following URL: ``` https://my-redirect-uri.com?code=AUTHORIZATION_CODE&state=STATE ``` ### Query parameters: **code** `string` An authorization code to be exchanged for an access token **state** `string` Compare with the original `state` parameter to ensure they match. Abort the process if they do not because the request may have come from a third party. Only present if you provided one in step 1. ### 3a. Exchange authorization code for an OAuth token Your application can now exchange the authorization code for an access token. **POST** ``` https://auth.planetscale.com/oauth/token?client_id=CLIENT_ID&client_secret=CLIENT_SECRET&code=CODE&grant_type=authorization_code&redirect_uri=REDIRECT_URI ``` The `POST` request will need the following: ### Query parameters: **client\_id** `string` *required* Your OAuth application's client ID **client\_secret** `string` *required* Your OAuth application's client secret **code** `string` *required* The code located in the query parameters of the previous step **grant\_type** `string` *required* Set to `authorization_code` **redirect\_uri** `string` *required* The full URL PlanetScale should redirect users on completion of the authorization flow, also known as the callback URL The response will look similar to the following: ```json JSON theme={null} { "access_token": "pscale_oauth_8zO_rNQCct1Uj8zkTWLh3kgwAqg8UabGIc43D2eINvo", "token_type": "Bearer", "expires_in": 2592000, "refresh_token": "pscale_oauth_refresh_W_zjmZ1a14sczj15bxJdsW_kiv063OrHG4CBh0IXR9M", "scope": "read_user read_databases" } ``` **access\_token** `string` Your OAuth access token. Use this in the `Authorization: Bearer ` header to make API calls on behalf of the user. **token\_type** `string` Will always be "Bearer" **expires\_in** `integer` The number of seconds until the access token expires **refresh\_token** `string` A refresh token that can refresh your access token when it expires. See step 3b for more info. **scope** `string` Space-separated list of scopes granted to this token You will need the `access_token` to make API calls on behalf of the user using Bearer authentication. ### 3b. Refreshing an OAuth token When an OAuth token expires, you can refresh it: **POST** ``` https://auth.planetscale.com/oauth/token?client_id=CLIENT_ID&client_secret=CLIENT_SECRET&grant_type=refresh_token&refresh_token=REFRESH_TOKEN ``` The `POST` request will need the following: #### Query parameters: **client\_id** `string` *required* Your OAuth application's client ID **client\_secret** `string` *required* Your OAuth application's client secret **refresh\_token** `string` *required* The refresh token sent in the initial token response in step 3a **grant\_type** `string` *required* Set to `refresh_token` The response will look similar to the response in 3a. ### 4. Using the OAuth token Now your application can make requests to the PlanetScale API on behalf of the user. To make requests to the API, add the `access_token` in the `Authorization` header as a Bearer token in your HTTP API request using the following format: ```text Text theme={null} Authorization: Bearer ``` Here is a cURL example: ```curl cURL theme={null} curl --request GET 'https://api.planetscale.com/v1/organizations' \ --header 'Authorization: Bearer ' ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # OAuth access scopes Source: https://planetscale.com/docs/api/reference/oauth-access-scopes The full list of OAuth application access scopes. See the [OAuth documentation](/docs/api/reference/oauth) for more information on creating OAuth applications on PlanetScale. ## User access | Access permissions | Description | | :- | :- | | read\_user | Read user | | write\_user | Write user | | read\_organizations | Read a user's organizations | ## Organization access | Access permissions | Description | | :- | :- | | write\_organization | Write organization | | read\_organization | Read organization | | read\_invoices | Read organization invoices | | delete\_organization | Delete organization | | read\_databases | Read organization databases | | create\_databases | Create organization databases | | write\_databases | Write organization databases | | delete\_databases | Delete organization databases | | write\_members | Write members | | read\_members | Read members | | delete\_members | Delete members | | read\_branches | Read database branches | | write\_branches | Write database branches | | delete\_branches | Delete database branches | | promote\_branches | Promote database branches | | delete\_production\_branches | Delete a production database branch | | manage\_production\_branch\_passwords | Read, write, and delete production branch passwords | | write\_deploy\_requests | Create and update deploy requests in a database | | read\_deploy\_requests | Read deploy requests in a database | | deploy\_deploy\_requests | Deploy deploy requests in a database | | approve\_deploy\_requests | Approve deploy requests in a database | | write\_comments | Create deploy request comments in a database | | read\_comments | Read deploy request comments in a database | | manage\_passwords | Read, write, and delete branch passwords | | write\_backups | Create and update backups | | read\_backups | Read backups | | delete\_backups | Delete backups | | delete\_production\_branch\_backups | Delete production backups | | restore\_backups | Restore this branch's backups to new branches | | restore\_production\_branch\_backups | Restore production branch backups to new | ## Database access | Access permissions | Description | | :- | :- | | write\_members | Write members | | read\_members | Read members | | delete\_members | Delete members | | read\_branches | Read database branches | | write\_branches | Write database branches | | delete\_branches | Delete database branches | | promote\_branches | Promote database branches | | delete\_production\_branches | Delete a production database branch | | manage\_production\_branch\_passwords | Read, write, and delete production branch passwords | | write\_deploy requests | Create and update deploy requests in a database | | read\_deploy\_requests | Read deploy requests in a database | | deploy\_deploy\_requests | Deploy deploy requests in a database | | approve\_deploy\_requests | Approve deploy requests in a database | | write\_comments | Create deploy request comments in a database | | read\_comments | Read deploy request comments in a database | | read\_database | Read database information | | delete\_database | Delete a database | | write\_database | Write database | | manage\_passwords | Read, write, and delete branch passwords | | write\_backups | Create and update backups | | read\_backups | Read backups | | delete\_backups | Delete backups | | delete\_production\_branch\_backups | Delete production backups | | restore\_backups | Restore this branch's backups to new branches | | restore\_production\_branch\_backups | Restore production branch backups to new branches | ## Branch access | Access permissions | Description | | :- | :- | | manage\_passwords | Read, write, and delete branch passwords | | write\_branch | Write a database branch | | read\_branch | Read a database branch | | delete\_branch | Delete a database branch | | write\_backups | Create and update backups | | read\_backups | Read backups | | delete\_backups | Delete backups | | restore\_backups | Restore this branch's backups to new branches | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Create or refresh OAuth token Source: https://planetscale.com/docs/api/reference/oauth/create-token Exchange an authorization code for an access token or refresh an expired token ## Endpoint ``` POST https://auth.planetscale.com/oauth/token ``` This is the standard OAuth 2.0 token endpoint for creating and refreshing access tokens. ## Request Body The request body should be sent as `application/x-www-form-urlencoded`. ### For authorization code exchange **grant\_type** `string` *required* Must be `authorization_code` **code** `string` *required* The authorization code received from the authorization flow **redirect\_uri** `string` *required* The redirect URI used in the authorization request **client\_id** `string` *required* Your OAuth application's client ID **client\_secret** `string` *required* Your OAuth application's client secret ### For token refresh **grant\_type** `string` *required* Must be `refresh_token` **refresh\_token** `string` *required* The refresh token from a previous token response **client\_id** `string` *required* Your OAuth application's client ID **client\_secret** `string` *required* Your OAuth application's client secret ## Response ### Success Response (200 OK) ```json theme={null} { "access_token": "pscale_oauth_8zO_rNQCct1Uj8zkTWLh3kgwAqg8UabGIc43D2eINvo", "token_type": "Bearer", "expires_in": 2592000, "refresh_token": "pscale_oauth_refresh_W_zjmZ1a14sczj15bxJdsW_kiv063OrHG4CBh0IXR9M", "scope": "read_user read_databases" } ``` **access\_token** `string` The OAuth access token to use for API requests **token\_type** `string` Will always be "Bearer" **expires\_in** `integer` Number of seconds until the access token expires **refresh\_token** `string` Token to use for refreshing the access token when it expires **scope** `string` Space-separated list of scopes granted to this token ## Example ```bash theme={null} # Exchange authorization code for access token curl -X POST https://auth.planetscale.com/oauth/token \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "grant_type=authorization_code" \ -d "code=YOUR_AUTHORIZATION_CODE" \ -d "redirect_uri=https://your-app.com/callback" \ -d "client_id=YOUR_CLIENT_ID" \ -d "client_secret=YOUR_CLIENT_SECRET" # Refresh an access token curl -X POST https://auth.planetscale.com/oauth/token \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "grant_type=refresh_token" \ -d "refresh_token=YOUR_REFRESH_TOKEN" \ -d "client_id=YOUR_CLIENT_ID" \ -d "client_secret=YOUR_CLIENT_SECRET" ``` ## Error Responses ### 400 Bad Request Invalid request parameters (e.g., missing required fields, invalid grant\_type). ### 401 Unauthorized Invalid client credentials (client\_id or client\_secret is incorrect). ### 400 Invalid Grant The authorization code or refresh token is invalid, expired, or already used. # Revoke OAuth token Source: https://planetscale.com/docs/api/reference/oauth/revoke-token Revoke an OAuth access token or refresh token ## Endpoint ``` POST https://auth.planetscale.com/oauth/revoke ``` This is the standard OAuth 2.0 token revocation endpoint. Once revoked, a token can no longer be used to access the API. ## Request Body The request body should be sent as `application/x-www-form-urlencoded`. **token** `string` *required* The access token or refresh token to revoke **client\_id** `string` *required* Your OAuth application's client ID **client\_secret** `string` *required* Your OAuth application's client secret ## Response ### Success Response (200 OK) Returns an empty response with status 200 when the token is successfully revoked. ```json theme={null} {} ``` ## Example ```bash theme={null} curl -X POST https://auth.planetscale.com/oauth/revoke \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "token=YOUR_ACCESS_TOKEN" \ -d "client_id=YOUR_CLIENT_ID" \ -d "client_secret=YOUR_CLIENT_SECRET" ``` ## Error Responses ### 401 Unauthorized Invalid client credentials (client\_id or client\_secret is incorrect). ## Notes * Revoking an access token does not automatically revoke its associated refresh token * You can revoke either access tokens or refresh tokens using this endpoint * Once revoked, the token cannot be used again and cannot be un-revoked # Get OAuth token info Source: https://planetscale.com/docs/api/reference/oauth/token-info Retrieve information about the current OAuth access token ## Endpoint ``` GET https://auth.planetscale.com/oauth/token/info ``` Returns information about the currently authenticated access token, including whether it's active, scopes, and expiration. ## Authentication This endpoint requires a valid OAuth access token in the Authorization header: ``` Authorization: Bearer YOUR_ACCESS_TOKEN ``` ## Response ### Success Response (200 OK) - Active Token ```json theme={null} { "active": true, "scope": "read_user read_databases", "client_id": "pscale_app_abc123", "token_type": "Bearer", "exp": 1709078400, "iat": 1706486400, "sub": "user_xyz789" } ``` **active** `boolean` Whether the token is currently active (not expired or revoked) **scope** `string` Space-separated list of scopes granted to this token **client\_id** `string` The OAuth application's client ID **token\_type** `string` Will always be "Bearer" **exp** `integer` Unix timestamp when the token expires **iat** `integer` Unix timestamp when the token was issued (created) **sub** `string` Subject - the ID of the user who authorized the token ### Success Response (200 OK) - Inactive Token If the token is expired, revoked, or invalid: ```json theme={null} { "active": false } ``` ## Example ```bash theme={null} curl -X GET https://auth.planetscale.com/oauth/token/info \ -H "Authorization: Bearer YOUR_ACCESS_TOKEN" ``` ## Use Cases This endpoint is useful for: * Validating that a token is still active * Checking which scopes a token has access to * Determining when a token will expire * Identifying which user authorized the token * Token introspection for security auditing # Pagination Source: https://planetscale.com/docs/api/reference/pagination How to paginate API responses ## Page-based pagination The majority of our API endpoints use standard page-based pagination. * `page`: the page number * `per_page`: the number of items returned per page (default: 25, max: 100) To set these values, pass them as query parameters in your request. For example, [in a list organizations `GET` request](/docs/api/reference/list_organizations), to return the first page at 50 items per page: ```curl cURL theme={null} curl --request GET \ --url 'https://api.planetscale.com/v1/organizations?page=1&per_page=50' \ --header 'Authorization: TOKEN_ID:TOKEN_SECRET' \ --header 'accept: application/json' ``` ## Cursor-based pagination For API endpoints with large result sets, we use cursor-based pagination. Cursor endpoints will return the following in their payload. * `cursor_start`: The ID of the first item returned. * `cursor_end`: The ID of the last item returned. * `has_next`: Whether there is a next page of results. * `has_prev`: Whether there is a previous page of results. For example: ```json json theme={null} { "type": "list", "has_next": true, "has_prev": false, "cursor_start": "b34zxm9mkz7g", "cursor_end": "eeq8f2lwrlum", "data": [] } ``` ### Cursor-based query parameters To pagination records, the following query parameters are available: * `starting_after`: The public\_id of the last item in the previous page. * `ending_before`: The public\_id of the first item in the next page. * `limit`: The number of items to return. Default DEFAULT\_LIMIT. For example, use the following to retrieve to items after `eeq8f2lwrlum`. ```curl cURL theme={null} curl --request GET \ --url 'https://api.planetscale.com/v1/organizations/my-org/audit-logs?starting_after=eeq8f2lwrlum&limit=50' \ --header 'Authorization: TOKEN_ID:TOKEN_SECRET' \ --header 'accept: application/json' ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Promote a branch Source: https://planetscale.com/docs/api/reference/promote_branch post /organizations/{organization}/databases/{database}/branches/{branch}/promote Promotes a branch from development to production ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `connect_production_branch`, `promote_branches` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `promote_branches` | | Database | `promote_branches` | # Queue a deploy request Source: https://planetscale.com/docs/api/reference/queue_deploy_request post /organizations/{organization}/databases/{database}/deploy-requests/{number}/deploy ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Rate limits Source: https://planetscale.com/docs/api/reference/rate-limits The current rate limit on the PlanetScale API is 600 requests per minute. This rate limit is subject to change. You will see a **429 error response code** if you hit the rate limit. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Reassign objects owned by one role to another role Source: https://planetscale.com/docs/api/reference/reassign_role_objects post /organizations/{organization}/databases/{database}/branches/{branch}/roles/{id}/reassign ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `delete_production_branch_password`, `delete_production_read_only_branch_password`, `delete_branch_password` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # Remove a member from an organization Source: https://planetscale.com/docs/api/reference/remove_organization_member delete /organizations/{organization}/members/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_organization` | # Remove a member from a team Source: https://planetscale.com/docs/api/reference/remove_organization_team_member delete /organizations/{organization}/teams/{team}/members/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_teams` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_organization` | # Renew a password Source: https://planetscale.com/docs/api/reference/renew_password post /organizations/{organization}/databases/{database}/branches/{branch}/passwords/{id}/renew ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | > [!NOTE] > > This extends the expiration of a **Vitess/MySQL** branch password created with > [Create a password](/docs/api/reference/create_password). The Postgres equivalent is > [Renew role expiration](/docs/api/reference/renew_role). > [!WARNING] > > Renewing does **not** issue a new password — the response's `plain_text` field is > null. If you have lost the password, [create a new > one](/docs/api/reference/create_password) instead. # Renew role expiration Source: https://planetscale.com/docs/api/reference/renew_role post /organizations/{organization}/databases/{database}/branches/{branch}/roles/{id}/renew ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_production_branch_password`, `create_production_read_only_branch_password`, `create_branch_password` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # Reset default credentials Source: https://planetscale.com/docs/api/reference/reset_default_role post /organizations/{organization}/databases/{database}/branches/{branch}/roles/reset-default ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch`, `delete_production_branch_password`, `delete_production_read_only_branch_password`, `delete_branch_password` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # Reset a role's password Source: https://planetscale.com/docs/api/reference/reset_role post /organizations/{organization}/databases/{database}/branches/{branch}/roles/{id}/reset ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch`, `delete_production_branch_password`, `delete_production_read_only_branch_password`, `delete_branch_password` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | > [!NOTE] > > This rotates the credentials of a **Postgres role** created with [Create role > credentials](/docs/api/reference/create_role) — it returns a new `password`. > Vitess/MySQL branch passwords have no in-place secret rotation; [create a new > password](/docs/api/reference/create_password) instead. > [!WARNING] > > The new `password` in the response is shown **only once**. Record it securely; > existing connections using the old password will stop working. # Review a deploy request Source: https://planetscale.com/docs/api/reference/review_deploy_request post /organizations/{organization}/databases/{database}/deploy-requests/{number}/reviews Review a deploy request by either approving or commenting on the deploy request ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `approve_deploy_request`, `review_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `approve_deploy_requests` | | Database | `approve_deploy_requests` | # Run maintenance for a configuration profile Source: https://planetscale.com/docs/api/reference/run_shard_configuration_profile_maintenance post /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/maintenance ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Run maintenance for selected configuration profiles Source: https://planetscale.com/docs/api/reference/run_shard_configuration_profiles_maintenance post /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/maintenance ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Service tokens Source: https://planetscale.com/docs/api/reference/service-tokens Using service tokens with the PlanetScale API. ## Overview This document will show you how to create a service token for the API in PlanetScale. All PlanetScale API endpoints require one. You can read more about using service tokens outside of the API in the [PlanetScale service token documentation](/docs/api/reference/service-tokens). ## Authorization header To make requests to the API, add the service token values in the `Authorization` header in your HTTP API request using the following format: ```text Text theme={null} Authorization: : ``` Here is a cURL example: ```curl cURL theme={null} curl --request GET 'https://api.planetscale.com/v1/organizations' \ --header 'Authorization: :' ``` ## Creating a service token To create a service token using the dashboard, log into your organization and click **Settings > Service tokens > New service token**. Give the token a descriptive name (this is used for your reference only) and click **Create service token**. ### Service token ID Copy this value to use as `SERVICE_TOKEN_ID`. The ID is also visible on the service token page after you continue to token permissions. ### Service token The token is generated immediately after the service token is created. Copy this value to use as `SERVICE_TOKEN`. ![Modal showing service token ID and token secret](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/reference/ef1a137-new-service-token.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=107e7a31cb40468829705d77578a6a54) Modal showing service token ID and token secret ## Access permissions You can access your specific service token page from your organization's **Settings > Service token** page. Service tokens are configured with granular permissions for both organizations and databases access. On the page for your specific service token you can add one or many of the following permissions. ![Showing the UI for organization and database access with buttons to add what access permissions](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/reference/858081d-service-token-accesses.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=212e3bfb2ce7c8ec931826464010b8cf) Showing the UI for organization and database access with buttons to add what access permissions. Please note that you may only add service token accesses that you are also authorized to do. For example, as an organization member, you can't create a service token with `create_databases` access. ### Organization access permissions A service token can have granular permissions across an organization with one or multiple of the following access permissions. Check the box next to each permission option you need to grant. Once you are done, click **Save permissions**. | Access permissions | Description | | :- | :- | | read\_organization | Get information about an organization | | read\_invoices | Get invoices for an organization | | read\_audit\_logs | Get audit logs for an organization | | read\_databases | Get information about an organization's databases | | create\_databases | Create organization databases | | delete\_databases | Delete organization databases | | read\_oauth\_applications | Get information about an organization's OAuth applications and their tokens | | read\_oauth\_tokens | Get OAuth tokens for an OAuth application | | write\_oauth\_tokens | Create and refresh OAuth grants and tokens for an OAuth application | | delete\_oauth\_tokens | Delete OAuth tokens for an OAuth application | | read\_service\_tokens | Get information about service tokens | | write\_service\_tokens | Create and update service tokens | | delete\_service\_tokens | Delete service tokens | | write\_teams | Create and update teams | | read\_metrics\_endpoints | Get information about branch metrics endpoints | ### Database access permissions A service token can have granular permissions across a database with one or multiple of the following access permissions. Select the database you want to grant access for and check the box next to each permission option you need to grant. Once you are done, click **Save permissions**. | Access permissions | Description | | :- | :- | | read\_database | Get information about a database | | write\_database | Update information about a database | | delete\_database | Delete a database | | create\_branch | Create a database branch | | read\_branch | Read a database branch | | delete\_branch | Delete a database branch | | delete\_production\_branch | Delete a production database branch | | connect\_branch | Connect to a database branch | | connect\_production\_branch | Connect to a production database branch | | delete\_branch\_password | Delete a password for a non-production branch | | delete\_production\_branch\_password | Delete a password for a production branch | | create\_deploy\_request | Create a deploy request | | read\_deploy\_request | Read a deploy request | | approve\_deploy\_request | Approve a deploy request | | create\_comment | Create a deploy request comment | | read\_comment | Read a deploy request comment | | read\_backups | List backups | | write\_backups | Create and update backups | | delete\_backups | Delete development branch backups | | delete\_production\_branch\_backups | Delete production branch backups | | restore\_backup | Restore a development branch backup | | restore\_production\_branch\_backup | Restore a production branch backup | | write\_branch\_vschema | Update the VSchema for a database branch | | write\_production\_branch\_vschema | Update the VSchema for a production database branch | > [!TIP] > > When a service token with `create_databases` organization access creates a database, it is automatically granted database access permissions for that database. See [Creating databases with service tokens](/docs/api/service-tokens#creating-databases-with-service-tokens) for more information. ## Service tokens and deploy requests approvals When a database requires administrator approval for deploy requests (located in your database's Settings page), a service token cannot approve a deploy request created by the same service token. Also, users can't approve a deploy request created by a service token that they created. ## Example ### Creating a PlanetScale branch with service tokens Creating a database branch is one of the many API endpoints documented in our [PlanetScale API docs](/docs/api/reference/create_branch). The following steps are required to make a successful API request: **Step 1** Create a service token as described at the top of this page. Make sure to copy and paste the service token ID and service token somewhere safe. **Step 2** You need to make sure your service token has the correct access permissions. Each endpoint documentation describes the service token access permissions it needs in the `Authorization` section. For example, in the create a branch endpoint you will need to grant the service token the `create_branch` access for your database: ![Service token access permissions for create\_branch](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/reference/675ebf8-H0I9wF2.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=05690326dc352864b43cfb62a955672f) **Step 3** After your permissions are set, you can now use the service token in a cURL request: ```curl cURL theme={null} curl --request POST \ --url https://api.planetscale.com/v1/organizations/ORGANIZATION_NAME/databases/DATABASE_NAME/branches \ --header 'Authorization: SERVICE_TOKEN_ID:SERVICE_TOKEN' \ --header 'accept: application/json' \ --header 'content-type: application/json' --data '{"name": "BRANCH_NAME","parent_branch": "PARENT_BRANCH_NAME"}' ``` Make sure to fill in the `ORGANIZATION_NAME`, `DATABASE_NAME`, `BRANCH_NAME`, `PARENT_BRANCH_NAME`, `SERVICE_TOKEN_ID`, and `SERVICE_TOKEN` variables. **Step 4** You will get a response with the following data, as described in the specific API endpoints documentation: ```json JSON expandable theme={null} { "id": "xsvewamrebwz", "type": "Branch", "name": "new-feature", "created_at": "2023-01-10T23:31:01.573Z", "updated_at": "2023-01-10T23:31:01.573Z", "restore_checklist_completed_at": null, "access_host_url": "xsvewamrebwz.us-east-1.psdb.cloud", "schema_last_updated_at": "2023-01-10T23:31:01.573Z", "mysql_address": "us-east.connect.psdb.cloud", "mysql_provider_address": "aws.connect.psdb.cloud", "initial_restore_id": null, "mysql_edge_address": "aws.connect.psdb.cloud", "ready": false, "production": false, "sharded": false, "shard_count": 0, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "Taylor Barnett", "avatar_url": "https://www.gravatar.com/avatar/2a44999e8816311f19eea3d7516d9204?d=https%3A%2F%2Fapp.planetscale.com%2Fgravatar-fallback.png&s=64" }, "restored_from_branch": null, "html_url": "https://app.planetscale.com/taylorhackyplace/example-test/new-feature", "url": "https://api.planetscale.com/v1/organizations/taylorhackyplace/databases/example-test/branches/new-feature", "region": { "id": "kc0e1ij8juzp", "type": "Region", "provider": "AWS", "enabled": true, "public_ip_addresses": [ "23.23.187.137", "52.6.141.108", "52.70.2.89", "50.17.188.76", "52.2.251.189", "52.72.234.74", "35.174.68.24", "52.5.253.172", "54.156.81.4", "34.200.24.255", "35.174.79.154", "44.199.177.24" ], "display_name": "AWS us-east-1", "location": "Northern Virginia", "slug": "us-east" }, "multiple_admins_required_for_demotion": false, "parent_branch": "main" } ``` ### Potential errors If you get a `{"code":"not_found","message":"Not Found"}` response, it is likely you either did not change the variables in the example cURL request or you did not set the access permissions correctly for the service token. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Set the default configuration profile for a branch Source: https://planetscale.com/docs/api/reference/set_default_shard_configuration_profile put /organizations/{organization}/databases/{database}/branches/{branch}/default-configuration-profile ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Skip revert period Source: https://planetscale.com/docs/api/reference/skip_revert_period post /organizations/{organization}/databases/{database}/deploy-requests/{number}/skip-revert Skips the revert period for a deploy request ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Test a webhook Source: https://planetscale.com/docs/api/reference/test_webhook post /organizations/{organization}/databases/{database}/webhooks/{id}/test Sends a test event to the webhook ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | Vitess and Postgres # Update the admin for a branch Source: https://planetscale.com/docs/api/reference/update_admin patch /organizations/{organization}/databases/{database}/branches/{branch}/admin ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Update auto-apply for deploy request Source: https://planetscale.com/docs/api/reference/update_auto_apply put /organizations/{organization}/databases/{database}/deploy-requests/{number}/auto-apply Enables or disabled the auto-apply setting for a deploy request ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Update auto-delete branch for deploy request Source: https://planetscale.com/docs/api/reference/update_auto_delete_branch put /organizations/{organization}/databases/{database}/deploy-requests/{number}/auto-delete-branch Enables or disabled the auto-delete branch setting for a deploy request ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Update a backup Source: https://planetscale.com/docs/api/reference/update_backup patch /organizations/{organization}/databases/{database}/branches/{branch}/backups/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_backups` | | Database | `write_backups` | | Branch | `write_backups` | Vitess and Postgres # Update a backup policy Source: https://planetscale.com/docs/api/reference/update_backup_policy patch /organizations/{organization}/databases/{database}/backup-policies/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_backups` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_backups` | | Database | `write_backups` | Vitess and Postgres # Upsert a bouncer resize request Source: https://planetscale.com/docs/api/reference/update_bouncer_resize_request patch /organizations/{organization}/databases/{database}/branches/{branch}/bouncers/{bouncer}/resizes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Update a branch Source: https://planetscale.com/docs/api/reference/update_branch patch /organizations/{organization}/databases/{database}/branches/{branch} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_branches`, `write_databases` | | Database | `write_branches`, `write_database` | | Branch | `write_branch` | Vitess and Postgres # Upsert a change request for cluster size, replicas, storage, or parameters Source: https://planetscale.com/docs/api/reference/update_branch_change_request patch /organizations/{organization}/databases/{database}/branches/{branch}/changes ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Change a branch cluster configuration Source: https://planetscale.com/docs/api/reference/update_branch_cluster_config patch /organizations/{organization}/databases/{database}/branches/{branch}/cluster ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `write_database` # Set the data topology for a branch Source: https://planetscale.com/docs/api/reference/update_data_topology put /organizations/{organization}/databases/{database}/branches/{branch}/data-topology ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_databases` | | Branch | `write_branch` | # Update an IP restriction entry Source: https://planetscale.com/docs/api/reference/update_database_postgres_cidr patch /organizations/{organization}/databases/{database}/cidrs/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Update database settings Source: https://planetscale.com/docs/api/reference/update_database_settings patch /organizations/{organization}/databases/{database} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | Vitess and Postgres # Update database throttler configurations Source: https://planetscale.com/docs/api/reference/update_database_throttler patch /organizations/{organization}/databases/{database}/throttler ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Update a dedicated read replica Source: https://planetscale.com/docs/api/reference/update_dedicated_read_replica patch /organizations/{organization}/databases/{database}/branches/{branch}/dedicated-read-replicas/{dedicated_read_replica} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Update deploy request throttler configurations Source: https://planetscale.com/docs/api/reference/update_deploy_request_throttler patch /organizations/{organization}/databases/{database}/deploy-requests/{number}/throttler ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_deploy_request` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `deploy_deploy_requests` | | Database | `deploy_deploy_requests` | # Configure keyspace settings Source: https://planetscale.com/docs/api/reference/update_keyspace patch /organizations/{organization}/databases/{database}/branches/{branch}/keyspaces/{keyspace} ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `create_branch` # Update the VSchema for the keyspace Source: https://planetscale.com/docs/api/reference/update_keyspace_vschema patch /organizations/{organization}/databases/{database}/branches/{branch}/keyspaces/{keyspace}/vschema ### Authorization A service token must have at least one of the following access in order to use this API endpoint: **Service Token Accesses** `write_production_branch_vschema`, `write_branch_vschema` # Update an organization Source: https://planetscale.com/docs/api/reference/update_organization patch /organizations/{organization} ### Authorization A OAuth token must have at least one of the following scopes in order to use this API endpoint: **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_organization` | # Update organization member role Source: https://planetscale.com/docs/api/reference/update_organization_membership patch /organizations/{organization}/members/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_organization` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_organization` | # Update an organization team Source: https://planetscale.com/docs/api/reference/update_organization_team patch /organizations/{organization}/teams/{team} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_teams` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_organization` | # Update a password Source: https://planetscale.com/docs/api/reference/update_password patch /organizations/{organization}/databases/{database}/branches/{branch}/passwords/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `connect_production_branch`, `connect_production_read_only_branch`, `connect_branch` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # Update role name Source: https://planetscale.com/docs/api/reference/update_role patch /organizations/{organization}/databases/{database}/branches/{branch}/roles/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `create_production_branch_password`, `create_production_read_only_branch_password`, `create_branch_password` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Database | `manage_passwords`, `manage_production_branch_passwords`, `manage_read_only_passwords`, `manage_production_read_only_passwords` | | Branch | `manage_passwords`, `manage_read_only_passwords` | # Update a router for a branch Source: https://planetscale.com/docs/api/reference/update_router patch /organizations/{organization}/databases/{database}/branches/{branch}/routers/{router} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Update a shard for a branch Source: https://planetscale.com/docs/api/reference/update_shard patch /organizations/{organization}/databases/{database}/branches/{branch}/shards/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Update a configuration profile for a branch Source: https://planetscale.com/docs/api/reference/update_shard_configuration_profile patch /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Enable or disable an extension for a configuration profile Source: https://planetscale.com/docs/api/reference/update_shard_configuration_profile_extension patch /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/extensions/{extension} # Update a shard in a configuration profile Source: https://planetscale.com/docs/api/reference/update_shard_configuration_profile_shard patch /organizations/{organization}/databases/{database}/branches/{branch}/configuration-profiles/{configuration_profile}/shards/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Update a traffic budget Source: https://planetscale.com/docs/api/reference/update_traffic_budget patch /organizations/{organization}/databases/{database}/branches/{branch}/traffic/budgets/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | # Update a webhook Source: https://planetscale.com/docs/api/reference/update_webhook patch /organizations/{organization}/databases/{database}/webhooks/{id} ### Authorization A service token or OAuth token must have at least one of the following access or scopes in order to use this API endpoint: **Service Token Accesses** `write_database` **OAuth Scopes** | Resource | Scopes | | :------- | :---------- | | Organization | `write_databases` | | Database | `write_database` | Vitess and Postgres # Verify workflow Source: https://planetscale.com/docs/api/reference/verify_workflow patch /organizations/{organization}/databases/{database}/workflows/{number}/verify-data > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Workflow cancel Source: https://planetscale.com/docs/api/reference/workflow_cancel delete /organizations/{organization}/databases/{database}/workflows/{number} > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Workflow complete Source: https://planetscale.com/docs/api/reference/workflow_complete patch /organizations/{organization}/databases/{database}/workflows/{number}/complete > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Workflow cutover Source: https://planetscale.com/docs/api/reference/workflow_cutover patch /organizations/{organization}/databases/{database}/workflows/{number}/cutover > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Workflow retry Source: https://planetscale.com/docs/api/reference/workflow_retry patch /organizations/{organization}/databases/{database}/workflows/{number}/retry > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Workflow reverse cutover Source: https://planetscale.com/docs/api/reference/workflow_reverse_cutover patch /organizations/{organization}/databases/{database}/workflows/{number}/reverse-cutover > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Workflow reverse traffic Source: https://planetscale.com/docs/api/reference/workflow_reverse_traffic patch /organizations/{organization}/databases/{database}/workflows/{number}/reverse-traffic > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Workflow switch primaries Source: https://planetscale.com/docs/api/reference/workflow_switch_primaries patch /organizations/{organization}/databases/{database}/workflows/{number}/switch-primaries > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Workflow switch replicas Source: https://planetscale.com/docs/api/reference/workflow_switch_replicas patch /organizations/{organization}/databases/{database}/workflows/{number}/switch-replicas > [!WARNING] > > This endpoint is deprecated. Use the MoveTables endpoints under `/organizations/{organization}/databases/{database}/branches/{branch}/move-tables/workflows`, or [`pscale branch vtctl move-tables`](/docs/cli/move-tables), instead. # Service tokens Source: https://planetscale.com/docs/api/service-tokens PlanetScale provides the ability to create service tokens for your PlanetScale organization via the CLI or directly in the UI. ## Overview Service tokens provide an alternate authentication method to be used with the PlanetScale CLI and API. They are typically used in automated scenarios where `pscale auth login` cannot be used. Service tokens are also required for any calls to the API, as well as minting OAuth tokens for API use. ## Create service tokens using the PlanetScale dashboard To create a service token using the dashboard, log into your organization, go to the [**"Settings"** > **"Service tokens"**](https://app.planetscale.com/~/settings/service-tokens) page, and click the **"New service token"** button. Give the token a name (this is used for your reference only) and click **"Create service token"**. The modal will update, displaying your service token where the Name field was. Copy the ID and token values as you'll need them moving forward. Click **"Edit token permissions"** to proceed. > [!TIP] > > Be sure to copy the service token after you create it. There's no way to retrieve the token value once you leave this page. ![Service token detail page](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/service-tokens/modal-with-service-token-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=1f18f07b836f9395cc035223cfc9e437) ## Assign service token permissions Service tokens are configured with granular permissions, both for the organization that owns them as well as on a per database level. Before you can use a service token, these permissions must be added. ### Add organization permissions Organization permissions are required when performing operations that are specific to the organization and not for an individual database. To enable a service token for performing these operations, locate the **Organization access** section and click **"Add organization permissions"**. In the **Organization access permissions** modal, check the box next to each of the permission scopes that you want to assign to the token. Click **"Save permissions"** once finished. For a full list of organization access permissions, see the [API documentation for service tokens](/docs/api/reference/service-tokens#organization-access-permissions). ### Add database permissions In order to perform operations specific to a database, permissions can be assigned per-database. To do this, locate the section titled **Database access** and click **"Add database access"** to open the **Database access permissions** modal. Select the database you want to grant access to and check the box next to each permission option you need to grant. Once you are done, click **"Save permissions"**. For a full list of database access permissions, see the [API documentation for service tokens](/docs/api/reference/service-tokens#database-access-permissions). ![The Database access permissions modal.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/service-tokens/db-access-permissions-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=2f2a5fae0c095305dc848397de6962a5) ### Add permissions for all databases Organization admins can grant database permissions to all current and future databases. This is available in the **Permissions for all databases** section. Any permission added to all databases will not be able to be disabled on an individual basis. ## Creating databases with service tokens A service token with the `create_databases` organization permission can create new databases in your organization. When a service token creates a database, it is automatically granted [database access permissions](/docs/api/reference/service-tokens#database-access-permissions) for that database. You do not need to add database permissions separately for each database the token creates. Only organization administrators can grant the `create_databases` permission to a service token. Auto-granted permissions apply only to databases the token creates, not to existing databases in the organization. To grant this permission from the dashboard, open the service token's **Organization access** section and click **"Add organization permissions"**. In the **Organization access permissions** modal, check the box for **Create organization databases** and click **"Save permissions"**. You can also grant organization access using the CLI: ```bash theme={null} pscale service-token add-access create_databases ``` Once the `create_databases` permission is in place, you can create databases with the token using the CLI or API. Refer to the [Create a database](/docs/api/reference/create_database) API reference for all available parameters. ## Service tokens and deploy requests approvals When a database requires administrator approval for deploy requests (located in your database's **Settings** page), a service token cannot approve a deploy request created by the same service token. Also, users can't approve a deploy request created by a service token that they created. ## Use a service token with the PlanetScale CLI To use service tokens with the PlanetScale CLI, set the following environment variables in your terminal: ```bash theme={null} export PLANETSCALE_SERVICE_TOKEN= export PLANETSCALE_SERVICE_TOKEN_ID= ``` When you execute commands using the PlanetScale CLI, it will automatically parse those values and use them to access the service. However, you’ll also need to pass in your organization name using the `--org` flag like so: ```bash theme={null} pscale branch create --org ``` If you don’t want to set environment variables, you may also pass in the Service Token and Service Token ID by using the [`--service-token` and `--service-token-id` flags](/docs/cli/service-token) respectively: ```bash theme={null} pscale branch create --org --service-token --service-token-id ``` ## Use a service token with the PlanetScale API In order to execute a request to the PlanetScale API, you'll need a service token to execute requests directly or for minting OAuth tokens. Both the ID and token are required in the `Authorization` header without a scheme. Below is an example of how to use a service token to list details about the organizations the token can access: ```bash theme={null} curl --request GET \ --url 'https://api.planetscale.com/v1/organizations' \ --header 'Authorization: :' ``` Refer to the [API docs](/docs/api/reference/getting-started-with-planetscale-api) for more details on how to use the API. ## Modify service token permissions If you want to modify the permissions granted to a service token, start by opening the service token from the settings pane. Select the three dots next to the organization or database name permissions you want to modify and click **"Edit permissions"**. ![The location of the Edit permissions option for organization permissions.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/service-tokens/edit-org-perms-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=2ff592a3bac8993002f800c5b41ad4b0) This will open a modal that allows you to modify the permissions the service token has to access that organization. ## Delete a service token You can delete a service token at any time from the service token detail page. Simply click the **"Delete service token"** button. ![Delete service token.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/service-tokens/delete-service-token-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=81a02e008e79a793ba15464b09e3b759) ## Manage service tokens using the PlanetScale CLI Service tokens can also be created and managed directly from the [PlanetScale CLI](/docs/cli/service-token). ### Create a new service token Use the following command to create a service token: ```bash theme={null} pscale service-token create ``` This command will return a service token ID and value for your use. ### Add database access permissions You can add database access permissions to your service token for each database in your organization. To add database access permissions, use the command: ```bash theme={null} pscale service-token add-access --database ``` For example, to give a service token the ability to create, read, and delete branches on a specific database, use the following command: ```bash theme={null} pscale service-token add-access read_branch delete_branch create_branch --database ``` A complete list of service token access permissions can be found in the [PlanetScale API documentation](/docs/api/reference/service-tokens#access-permissions). ### Remove database access permissions You can also remove database access permissions for a service token. Use the following command to remove one or more permissions: ```bash theme={null} pscale service-token delete-access --database ``` ### Delete a service token To delete a service token, run the following command: ```bash theme={null} pscale service-token delete ``` ## Service token automation Running `pscale` in CI or an AI agent uses a [service token](/docs/cli/service-tokens) instead of `pscale auth login`. Provide the token as environment variables (recommended) or as flags on each command; both are equivalent: ```bash theme={null} # Environment variables (recommended for CI and agents) export PLANETSCALE_SERVICE_TOKEN_ID="" export PLANETSCALE_SERVICE_TOKEN="" # Or per-command flags pscale database list --org \ --service-token-id "" \ --service-token "" ``` A service token has **no active organization**, so resource commands need the org supplied explicitly. Set it with `PLANETSCALE_ORG`, the `--org` flag on the subcommand, or `pscale org switch ` (which works with a service token): ```bash theme={null} export PLANETSCALE_ORG="" pscale database list --format json # uses PLANETSCALE_ORG pscale database list --org --format json # explicit flag ``` > [!WARNING] > > `--org` goes on the **resource subcommand** (`database`, `branch`, `sql`, `api`, …), never on root `pscale`. `pscale --org database list` fails with `unknown flag: --org`. Per-command-family matrices (env-var auth, `--service-token` flag, required `--org`, Postgres/Vitess, `--format json`, API equivalent): [`org`](/docs/cli/org#service-token-automation-org) · [`service-token`](/docs/cli/service-token#service-token-automation-service-token) · [`database`](/docs/cli/database#service-token-automation-database) · [`branch`](/docs/cli/branch#service-token-automation-branch) · [`role`](/docs/cli/role#service-token-automation-role) · [`password`](/docs/cli/password#service-token-automation-password). ### Commands to avoid under a service token Do not retry these with service-token auth. The failure is by design, not transient. This is the most common cause of agent retry loops. | Command | Why | Do this instead | Exact error | | :- | :- | :- | :- | | `pscale org show` | A service token has no "current" organization | `pscale org list --format json`, then pass `--org` or `PLANETSCALE_ORG` | `not authenticated yet. Please run 'pscale auth login'` | | `pscale service-token list` | Token management is blocked when authenticated with a token | `pscale api organizations//service-tokens --format json` | `pscale service-token list is unavailable when authenticated with a service token` | | `pscale service-token show-access` | Same as `list` | `pscale api organizations//service-tokens/ --format json` | `pscale service-token show-access is unavailable when authenticated with a service token` | | `pscale service-token` (other sub-commands) | Create/manage tokens blocked under service-token auth | `pscale auth login`, then CLI; or [Service tokens API](/docs/api/reference/service-tokens) | `pscale service-token is unavailable when authenticated with a service token` | | `pscale auth login` / `logout` | Not needed; the token is the credential | Set the token env vars; run `pscale auth check --format json` to confirm | requires an interactive browser device flow | | `pscale shell` / `connect` | Interactive sessions | Use [`pscale sql`](/docs/cli/sql) for non-interactive queries | opens an interactive session | > [!WARNING] > > `pscale org show` prints `not authenticated yet. Please run 'pscale auth login'` under a valid service token. This is the **no-current-org** state, not an auth failure. Do **not** respond by running `pscale auth login`. Discover the org with `pscale org list` and pass it with `--org` or `PLANETSCALE_ORG`. `pscale auth check --format json` confirms a token is wired up. Expect `"authenticated": true`, `"auth_method": "service_token"`, and (until an org is set) an `action_required` status with a `NO_ORG` issue and a non-zero exit code. Resolve it by passing `--org`/`PLANETSCALE_ORG` on your commands. ### `pscale api` fallback for hard-to-parse output When a command's human output is hard to parse, prefer `--format json`, or drop to the raw API with `pscale api`, which returns the API response verbatim and uses the same token: ```bash theme={null} # org list pscale api organizations --format json # database list / show pscale api organizations//databases --format json pscale api organizations//databases/ --format json # branch list / show pscale api organizations//databases//branches --format json pscale api organizations//databases//branches/ --format json # service-token list / show-access (CLI blocked under service-token auth) pscale api organizations//service-tokens --format json pscale api organizations//service-tokens/ --format json ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Webhook events reference Source: https://planetscale.com/docs/api/webhook-events Webhooks in PlanetScale allow you to trigger an HTTP POST callback to a configured URL when specific events happen within your PlanetScale organization. Vitess, Neki, and Postgres ## Overview Webhooks can be used to build integrations, such as notifications, and automate workflows. See the [webhooks documentation](/docs/api/webhooks) for more information. ## Webhook headers All webhooks from PlanetScale will have an `X-PlanetScale-Signature` header. This header is a SHA-256 HMAC hex digest of the request body, using your webhook secret as the key. You can use this header to verify that the webhook payload was sent by PlanetScale. See the documentation on [validating a webhook signature](/docs/api/webhooks#validating-a-webhook-signature) for more information. If configured, webhook deliveries include your Authorization header value unchanged. See [authenticating webhook deliveries](/docs/api/webhooks#authenticating-webhook-deliveries) for more information. ## Webhook request body parameters | Parameter | Type | Description | | :- | :- | :- | | `timestamp` | integer | Unix epoch time. | | `event` | string | Name of the webhook event, see [webhook events](#webhook-events) for more info about each event. | | `resource` | object | Information about the event, uses the same response body as the API responses for the same resource. See examples below. | ## Webhook events | Webhook event | `event` parameter | Trigger | Engines | | :- | :- | :- | :- | | [Branch anomaly](#branch-anomaly) | `branch.anomaly` | The branch has a new anomaly in insights. | Vitess, Neki, Postgres | | [Branch primary promoted](#branch-primary-promoted) | `branch.primary_promoted` | A new primary has been promoted for the branch. | Postgres | | [Branch out of memory](#branch-out-of-memory) | `branch.out_of_memory` | A Postgres database experienced an out of memory event. | Postgres | | [Branch ready](#branch-ready) | `branch.ready` | The branch is created and ready to connect. | Vitess, Neki, Postgres | | [Branch schema recommendation](#branch-schema-recommendation) | `branch.schema_recommendation` | A new schema recommendation has been created. | Vitess, Neki, Postgres | | [Branch sleeping](#branch-sleeping) | `branch.sleeping` | The branch is now sleeping. | Vitess, Neki, Postgres | | [Branch start maintenance](#branch-start-maintenance) | `branch.start_maintenance` | A production branch is about to start maintenance. | Vitess, Neki, Postgres | | [Backup failed](#backup-failed) | `backup.failed` | A backup has failed. | Vitess, Neki, Postgres | | [Backup succeeded](#backup-succeeded) | `backup.succeeded` | A backup has completed successfully. | Vitess, Neki, Postgres | | [Cluster storage](#cluster-storage) | `cluster.storage` | A Postgres database has reached a storage threshold (60%, 75%, 85%, 90%, 95%). | Postgres | | [Database access request](#database-access-request) | `database.access_request` | PlanetScale staff opens an access request. | Vitess, Neki, Postgres | | [Deploy request opened](#deploy-request-opened) | `deploy_request.opened` | The deploy request has been opened. | Vitess | | [Deploy request queued](#deploy-request-queued) | `deploy_request.queued` | The deploy request has been added to the deploy queue. | Vitess | | [Deploy request in progress](#deploy-request-in-progress) | `deploy_request.in_progress` | The deploy request has started running. | Vitess | | [Deploy request pending cutover](#deploy-request-pending-cutover) | `deploy_request.pending_cutover` | The deploy request is ready to cutover and waiting on the user. | Vitess | | [Deploy request schema applied](#deploy-request-schema-applied) | `deploy_request.schema_applied` | The deploy request has finished applying the schema. | Vitess | | [Deploy request errored](#deploy-request-errored) | `deploy_request.errored` | The deploy request has stopped due to an error. | Vitess | | [Deploy request reverted](#deploy-request-reverted) | `deploy_request.reverted` | The deploy request has been reverted. | Vitess | | [Deploy request closed](#deploy-request-closed) | `deploy_request.closed` | The deploy request has been closed. | Vitess | | [Keyspace storage](#keyspace-storage) | `keyspace.storage` | A Vitess keyspace has reached a storage threshold (60%, 75%, 85%, 90%, 95%). | Vitess | | [Webhook test](#webhook-test) | `webhook.test` | A webhook test is triggered. | Vitess, Neki, Postgres | > [!NOTE] > > If there is an event you want to use that is not included in this list, please [contact us](https://planetscale.com/contact) and let us know what event you want to trigger a webhook on. ### Branch anomaly The branch has a new anomaly event in PlanetScale Insights. The `branch.anomaly` event uses the same response body as a `200` response from the [Get a branch](/docs/api/reference/get_branch) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1698252879, "event": "branch.anomaly", "organization": "myorg", "database": "example_database", "resource": { "id": "ecrmjy2f4a5o", "type": "Branch", "name": "dev", "created_at": "2023-10-25T16:54:12.879Z", "updated_at": "2023-10-25T16:54:39.820Z", "restore_checklist_completed_at": null, "access_host_url": "ecrmjy2f4a5o.us-east-4.psdb.cloud", "schema_last_updated_at": "2023-10-25T16:54:12.879Z", "mysql_address": "us-east.connect.psdb.cloud", "mysql_provider_address": "aws.connect.psdb.cloud", "initial_restore_id": null, "schema_ready": true, "state": "ready", "cluster_name": "PS_DEV", "mysql_edge_address": "aws.connect.psdb.cloud", "ready": true, "production": false, "safe_migrations": false, "sharded": false, "shard_count": 0, "stale_schema": false, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "restored_from_branch": null, "private_connectivity": false, "private_edge_connectivity": false, "html_url": "https://app.planetscale.com/demo-db/example_database/dev", "url": "https://api.planetscale.com/v1/organizations/demo-db/databases/example_database/branches/dev", "region": { "id": "kc0e1ij8juzp", "type": "Region", "provider": "AWS", "enabled": true, "public_ip_addresses": [ "23.23.187.137", "52.6.141.108", "52.70.2.89", "50.17.188.76", "52.2.251.189", "52.72.234.74", "35.174.68.24", "52.5.253.172", "54.156.81.4", "34.200.24.255", "35.174.79.154", "44.199.177.24", "35.173.174.19", "44.212.228.57", "44.216.88.45" ], "display_name": "AWS us-east-1", "location": "N. Virginia", "slug": "us-east", "current_default": true }, "parent_branch": "main" } } ``` ### Branch primary promoted A new primary has been promoted for the Postgres database. The `branch.primary_promoted` event uses the same response body as a `200` response from the [Get a branch](/docs/api/reference/get_branch) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1698252879, "event": "branch.primary_promoted", "organization": "myorg", "database": "example_database", "resource": { "id": "q9x7ylb3hcmx", "type": "Branch", "name": "main", "cluster_architecture": "x86_64", "cluster_name": "PS_20_AWS_X86", "cluster_display_name": "PS-20", "cluster_iops": null, "restore_checklist_completed_at": null, "schema_last_updated_at": "2025-09-17T20:49:14.983Z", "state": "ready", "created_at": "2025-07-15T12:38:22.664Z", "updated_at": "2025-09-24T21:00:35.717Z", "actor": { "id": "u2dr4sbhz6ag", "type": "User", "display_name": "Mike Coutermarsh", "avatar_url": "https://www.gravatar.com/avatar/236f8e1435745283319e2bcfeed75072?d=https%3A%2F%2Fapp.planetscale.com%2Fgravatar-fallback.png&s=64" }, "restored_from_branch": null, "frozen": true, "safe_migrations": false, "default": true, "enabling_vectors": false, "metal": false, "production": true, "ready": true, "stale_schema": false, "parent_branch": null, "private_edge_connectivity": false, "replicas": 2, "read_only_reason": null, "read_only_at": null, "minimum_storage_bytes": 21474836480, "maximum_storage_bytes": 3324304687104, "storage_autoscaling": true, "storage_shrinking": true, "storage_type": "gp3", "storage_iops": 4000, "storage_throughput_mibs": 300, "has_replicas": true, "has_read_only_replicas": false, "has_roles": true, "region": { "id": "ri0pbcmdkjsh", "type": "Region", "provider": "AWS", "enabled": true, "public_ip_addresses": [ "18.117.23.127", "3.131.243.164", "3.132.168.252", "3.131.252.213", "3.132.182.173", "3.15.49.114", "3.209.149.66", "3.215.97.46", "34.193.111.15" ], "display_name": "AWS us-east-2", "location": "Ohio", "slug": "aws-us-east-2", "current_default": false }, "kind": "postgresql" } } ``` ### Branch out of memory A Postgres database experienced an out of memory (OOM) event and had to restart. OOM events cause the database to restart, drop all active connections, and roll back any in-progress transactions. This webhook will send at most once every 30 minutes for databases that are experiencing continuous OOM events. See the [out of memory documentation](/docs/postgres/troubleshooting/out-of-memory) for information on why OOM events occur and how to prevent them. The `branch.out_of_memory` event uses the same response body as a `200` response from the [Get a branch](/docs/api/reference/get_branch) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1698252879, "event": "branch.out_of_memory", "organization": "myorg", "database": "example_database", "resource": { "id": "q9x7ylb3hcmx", "type": "Branch", "name": "main", "cluster_architecture": "x86_64", "cluster_name": "PS_20_AWS_X86", "cluster_display_name": "PS-20", "cluster_iops": null, "restore_checklist_completed_at": null, "schema_last_updated_at": "2025-09-17T20:49:14.983Z", "state": "ready", "created_at": "2025-07-15T12:38:22.664Z", "updated_at": "2025-09-24T21:00:35.717Z", "actor": { "id": "u2dr4sbhz6ag", "type": "User", "display_name": "Mike Coutermarsh", "avatar_url": "https://www.gravatar.com/avatar/236f8e1435745283319e2bcfeed75072?d=https%3A%2F%2Fapp.planetscale.com%2Fgravatar-fallback.png&s=64" }, "restored_from_branch": null, "frozen": true, "safe_migrations": false, "default": true, "enabling_vectors": false, "metal": false, "production": true, "ready": true, "stale_schema": false, "parent_branch": null, "private_edge_connectivity": false, "replicas": 2, "read_only_reason": null, "read_only_at": null, "minimum_storage_bytes": 21474836480, "maximum_storage_bytes": 3324304687104, "storage_autoscaling": true, "storage_shrinking": true, "storage_type": "gp3", "storage_iops": 4000, "storage_throughput_mibs": 300, "has_replicas": true, "has_read_only_replicas": false, "has_roles": true, "region": { "id": "ri0pbcmdkjsh", "type": "Region", "provider": "AWS", "enabled": true, "public_ip_addresses": [ "18.117.23.127", "3.131.243.164", "3.132.168.252", "3.131.252.213", "3.132.182.173", "3.15.49.114", "3.209.149.66", "3.215.97.46", "34.193.111.15" ], "display_name": "AWS us-east-2", "location": "Ohio", "slug": "aws-us-east-2", "current_default": false }, "kind": "postgresql" } } ``` ### Branch ready The branch is created and ready to connect. The `branch.ready` event uses the same response body as a `200` response from the [Get a branch](/docs/api/reference/get_branch) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1698252879, "event": "branch.ready", "organization": "myorg", "database": "example_database", "resource": { "id": "ecrmjy2f4a5o", "type": "Branch", "name": "dev", "created_at": "2023-10-25T16:54:12.879Z", "updated_at": "2023-10-25T16:54:39.820Z", "restore_checklist_completed_at": null, "access_host_url": "ecrmjy2f4a5o.us-east-4.psdb.cloud", "schema_last_updated_at": "2023-10-25T16:54:12.879Z", "mysql_address": "us-east.connect.psdb.cloud", "mysql_provider_address": "aws.connect.psdb.cloud", "initial_restore_id": null, "schema_ready": true, "state": "ready", "cluster_name": "PS_DEV", "mysql_edge_address": "aws.connect.psdb.cloud", "ready": true, "production": false, "safe_migrations": false, "sharded": false, "shard_count": 0, "stale_schema": false, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "restored_from_branch": null, "private_connectivity": false, "private_edge_connectivity": false, "html_url": "https://app.planetscale.com/demo-db/example_database/dev", "url": "https://api.planetscale.com/v1/organizations/demo-db/databases/example_database/branches/dev", "region": { "id": "kc0e1ij8juzp", "type": "Region", "provider": "AWS", "enabled": true, "public_ip_addresses": [ "23.23.187.137", "52.6.141.108", "52.70.2.89", "50.17.188.76", "52.2.251.189", "52.72.234.74", "35.174.68.24", "52.5.253.172", "54.156.81.4", "34.200.24.255", "35.174.79.154", "44.199.177.24", "35.173.174.19", "44.212.228.57", "44.216.88.45" ], "display_name": "AWS us-east-1", "location": "N. Virginia", "slug": "us-east", "current_default": true }, "parent_branch": "main" } } ``` ### Branch schema recommendation A new schema recommendation has been created. This event is sent for Vitess, Neki, and Postgres databases. The `branch.schema_recommendation` event uses the same response body as a `200` response from the [Get a schema recommendation](/docs/api/reference/get_schema_recommendation) API endpoint. Set an [Authorization header](/docs/api/webhooks#authenticating-webhook-deliveries) on the webhook to send a token with each delivery. A receiving service can use that header to start a Cursor or Devin agent, such as a Cursor Automation, a background agent, or a Devin session, when a new schema recommendation is created. **Example:** ```json expandable theme={null} { "timestamp": 1698252879, "event": "branch.schema_recommendation", "organization": "myorg", "database": "example_database", "resource": { "id": "sr8k2n4q1w0p", "html_url": "https://app.planetscale.com/myorg/example_database/insights/recommendations/12", "title": "Add an index to users", "table_name": "users", "keyspace": "example_database", "ddl_statement": "CREATE INDEX users_email_idx ON users (email)", "number": 12, "state": "open", "recommendation_type": "new_index", "created_at": "2026-10-01T19:00:00.000Z", "updated_at": "2026-10-01T19:00:00.000Z", "applied_at": null, "dismissed_at": null, "closed_by_deploy_request": null, "dismissed_by": null } } ``` ### Branch sleeping The branch is now sleeping. The `branch.sleeping` event uses the same response body as a `200` response from the [Get a branch](/docs/api/reference/get_branch) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1697739653, "event": "branch.sleeping", "organization": "myorg", "database": "example_database", "resource": { "id": "bffzv8jfk9gc", "type": "Branch", "name": "dev", "created_at": "2023-10-14T18:17:42.998Z", "updated_at": "2023-10-19T18:17:43.105Z", "restore_checklist_completed_at": null, "access_host_url": "bffzv8jfk9gc.us-east-4.psdb.cloud", "schema_last_updated_at": "2023-10-19T18:07:57.623Z", "mysql_address": "us-east.connect.psdb.cloud", "mysql_provider_address": "aws.connect.psdb.cloud", "initial_restore_id": null, "schema_ready": true, "state": "ready", "cluster_name": "PS_DEV", "mysql_edge_address": "aws.connect.psdb.cloud", "ready": true, "production": false, "safe_migrations": false, "sharded": false, "shard_count": 0, "stale_schema": false, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "restored_from_branch": null, "private_connectivity": false, "private_edge_connectivity": false, "html_url": "https://app.planetscale.com/demo-db/example_database/dev", "url": "https://api.planetscale.com/v1/organizations/demo-db/databases/example_database/branches/dev", "region": { "id": "kc0e1ij8juzp", "type": "Region", "provider": "AWS", "enabled": true, "public_ip_addresses": [ "23.23.187.137", "52.6.141.108", "52.70.2.89", "50.17.188.76", "52.2.251.189", "52.72.234.74", "35.174.68.24", "52.5.253.172", "54.156.81.4", "34.200.24.255", "35.174.79.154", "44.199.177.24", "35.173.174.19", "44.212.228.57", "44.216.88.45" ], "display_name": "AWS us-east-1", "location": "N. Virginia", "slug": "us-east", "current_default": true }, "parent_branch": "main" } } ``` ### Branch start maintenance A production branch is about to start maintenance. This webhook is available for Vitess, Neki, and Postgres databases, with different behaviors: **Vitess** Vitess maintenance webhooks are available for enterprise customers with maintenance windows configured. During maintenance, Vitess and security upgrades are rolled out. Any queued cluster configuration changes are applied. **Neki** Neki maintenance webhooks are available for production branches with a weekly maintenance window. Maintenance can restart cluster components and close active connections. See the [Neki maintenance windows documentation](/docs/neki/cluster-configuration/maintenance-windows) for more information. **Postgres** Postgres maintenance webhooks are available for all customers. During maintenance, security patches and upgrades are rolled out. Your database cluster might restart, causing connections to be temporarily disconnected. See the [Postgres maintenance windows documentation](/docs/postgres/cluster-configuration/maintenance-windows) for more information. By subscribing to this webhook, you will receive a notification right before maintenance for your production branch is about to start. The `branch.start_maintenance` event uses the same response body as a `200` response from the [Get a branch](/docs/api/reference/get_branch) API endpoint. **Example:** ```json expandable theme={null} { "timestamp": 1698252879, "event": "branch.start_maintenance", "organization": "myorg", "database": "example_database", "resource": { "id": "q9x7ylb3hcmx", "type": "Branch", "name": "main", "cluster_architecture": "x86_64", "cluster_name": "PS_20_AWS_X86", "cluster_display_name": "PS-20", "cluster_iops": null, "restore_checklist_completed_at": null, "schema_last_updated_at": "2025-09-17T20:49:14.983Z", "state": "ready", "created_at": "2025-07-15T12:38:22.664Z", "updated_at": "2025-09-24T21:00:35.717Z", "actor": { "id": "u2dr4sbhz6ag", "type": "User", "display_name": "Mike Coutermarsh", "avatar_url": "https://www.gravatar.com/avatar/236f8e1435745283319e2bcfeed75072?d=https%3A%2F%2Fapp.planetscale.com%2Fgravatar-fallback.png&s=64" }, "restored_from_branch": null, "frozen": true, "safe_migrations": false, "default": true, "enabling_vectors": false, "metal": false, "production": true, "ready": true, "stale_schema": false, "parent_branch": null, "private_edge_connectivity": false, "replicas": 2, "read_only_reason": null, "read_only_at": null, "minimum_storage_bytes": 21474836480, "maximum_storage_bytes": 3324304687104, "storage_autoscaling": true, "storage_shrinking": true, "storage_type": "gp3", "storage_iops": 4000, "storage_throughput_mibs": 300, "has_replicas": true, "has_read_only_replicas": false, "has_roles": true, "region": { "id": "ri0pbcmdkjsh", "type": "Region", "provider": "AWS", "enabled": true, "public_ip_addresses": [ "18.117.23.127", "3.131.243.164", "3.132.168.252", "3.131.252.213", "3.132.182.173", "3.15.49.114", "3.209.149.66", "3.215.97.46", "34.193.111.15" ], "display_name": "AWS us-east-2", "location": "Ohio", "slug": "aws-us-east-2", "current_default": false }, "kind": "postgresql" } } ``` ### Backup failed A backup has failed. This event is sent for Vitess, Neki, and Postgres databases. The `backup.failed` event uses the same response body as a `200` response from the [Get a backup](/docs/api/reference/get_backup) API endpoint. **Example:** ```json expandable theme={null} { "timestamp": 1698252879, "event": "backup.failed", "organization": "myorg", "database": "example_database", "resource": { "id": "k2n8pqw4x1yz", "type": "Backup", "name": "2026.08.27 15:04:05.123", "state": "failed", "size": 0, "estimated_storage_cost": 0, "created_at": "2026-08-27T15:04:05.123Z", "updated_at": "2026-08-27T15:12:44.890Z", "started_at": "2026-08-27T15:04:06.001Z", "expires_at": "2026-09-26T15:04:05.123Z", "completed_at": "2026-08-27T15:12:44.890Z", "deleted_at": null, "pvc_size": 0, "uncompressed_size": 0, "protected": false, "required": true, "restored_branches": [], "actor": null, "backup_policy": { "id": "p0licy12abcd", "type": "BackupPolicy", "name": "production", "target": "production", "retention_value": 2, "retention_unit": "day", "frequency_value": 12, "frequency_unit": "hour", "schedule_time": "00:00", "created_at": "2026-01-15T12:00:00.000Z", "updated_at": "2026-01-15T12:00:00.000Z" }, "schema_snapshot": null, "database_branch": { "id": "q9x7ylb3hcmx", "type": "Branch", "name": "main" } } } ``` ### Backup succeeded A backup has completed successfully. This event is sent for Vitess, Neki, and Postgres databases. The `backup.succeeded` event uses the same response body as a `200` response from the [Get a backup](/docs/api/reference/get_backup) API endpoint. **Example:** ```json expandable theme={null} { "timestamp": 1698252879, "event": "backup.succeeded", "organization": "myorg", "database": "example_database", "resource": { "id": "b4ckup98wxyz", "type": "Backup", "name": "2026.08.27 03:00:00.000", "state": "success", "size": 2147483648, "estimated_storage_cost": 0.14, "created_at": "2026-08-27T03:00:00.000Z", "updated_at": "2026-08-27T03:18:22.441Z", "started_at": "2026-08-27T03:00:01.102Z", "expires_at": "2026-09-26T03:00:00.000Z", "completed_at": "2026-08-27T03:18:22.441Z", "deleted_at": null, "pvc_size": 0, "uncompressed_size": 5368709120, "protected": false, "required": true, "restored_branches": [], "actor": null, "backup_policy": { "id": "p0licy12abcd", "type": "BackupPolicy", "name": "production", "target": "production", "retention_value": 2, "retention_unit": "day", "frequency_value": 12, "frequency_unit": "hour", "schedule_time": "00:00", "created_at": "2026-01-15T12:00:00.000Z", "updated_at": "2026-01-15T12:00:00.000Z" }, "schema_snapshot": null, "database_branch": { "id": "q9x7ylb3hcmx", "type": "Branch", "name": "main" } } } ``` ### Cluster storage A storage threshold (60%, 75%, 85%, 90%, 95%) has been crossed for a Postgres database. ```json theme={null} { "timestamp": 1745526125, "event": "cluster.storage", "organization": "myorg", "database": "example_database", "resource": { "branch_name": "main", "storage_percentage": 44, "storage": [ { "bytes_used": 208470605824, "bytes_total_capacity": 472424366080, "bytes_available": 263953760256, "storage_percentage": 44.1 } ] } } ``` ### Database access request PlanetScale staff opens an access request to a database. The `database.access_request` event is triggered when PlanetScale staff initiates an access request to your organization for support purposes. **Example:** ```json expandable theme={null} { "timestamp": 1745617110, "event": "database.access_request", "organization": "planetscale", "database": "planetscale", "resource": { "type": "PlanetScaleAdminAccessRequest", "id": "23wl4nynt06x", "actor": { "id": "abc123", "type": "User", "display_name": "Mike Coutermarsh", "avatar_url": "https://www.gravatar.com/avatar/236f8e1435745283319e2bcfeed75072?d=https%3A%2F%2Fapp.planetscale.com%2Fgravatar-fallback.png&s=64" }, "revoked_by": null, "responder": null, "access_target": { "id": "abc123", "type": "Organization", "name": "planetscale", "created_at": "2021-02-17T20:03:56.393Z", "deleted_at": null, "updated_at": "2023-06-02T14:13:01.679Z" }, "organization": { "id": "abc123", "type": "Organization", "name": "planetscale", "created_at": "2021-02-17T20:03:56.393Z", "deleted_at": null, "updated_at": "2023-06-02T14:13:01.679Z" }, "state": "pending", "access_reason": "This is a test.", "denial_reason": null, "responded_at": null, "created_at": "2025-04-25T21:38:30.088Z", "cancelled_at": null, "expires_at": null, "duration": 24, "duration_units": "hour" } } ``` ### Deploy request opened The deploy request has been opened. The `deploy_request.opened` event uses the same response body as a `200` response from the [Get a deploy request](/docs/api/reference/get_deploy_request) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1698252899, "event": "deploy_request.opened", "organization": "myorg", "database": "example_database", "resource": { "id": "4xsz0ql82y4n", "type": "DeployRequest", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "closed_by": null, "branch": "dev", "branch_deleted": false, "branch_deleted_by": null, "branch_deleted_at": null, "into_branch": "main", "into_branch_sharded": false, "into_branch_shard_count": 0, "approved": false, "state": "open", "deployment_state": "pending", "num_comments": 0, "deployment": { "id": "uvkd7injje2f", "type": "Deployment", "into_branch": "main", "deploy_request_number": 5, "actor": { "id": "boo", "type": "User", "display_name": "Ghost", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cutover_actor": null, "cancelled_actor": null, "schema_last_updated_at": "2023-10-25T16:54:59.728Z", "preceding_deployments": [], "deploy_operations": [], "deploy_operation_summaries": [], "deployable": false, "cutover_expiring": false, "lint_errors": [], "deployment_revert_request": null, "auto_cutover": true, "created_at": "2023-10-25T16:54:59.863Z", "cutover_at": null, "deploy_check_errors": null, "finished_at": null, "queued_at": null, "ready_to_cutover_at": null, "started_at": null, "state": "pending", "submitted_at": null, "updated_at": "2023-10-25T16:54:59.863Z" }, "html_url": "https://app.planetscale.com/demo-db/example_database/deploy-requests/5", "number": 5, "notes": "", "html_body": "", "created_at": "2023-10-25T16:54:59.797Z", "updated_at": "2023-10-25T16:54:59.797Z", "closed_at": null, "deployed_at": null } } ``` ### Deploy request queued The deploy request has been added to the deploy queue. The `deploy_request.queued` event uses the same response body as a `200` response from the [Get a deploy request](/docs/api/reference/get_deploy_request) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1698252953, "event": "deploy_request.queued", "organization": "myorg", "database": "example_database", "resource": { "id": "4xsz0ql82y4n", "type": "DeployRequest", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "closed_by": null, "branch": "dev", "branch_deleted": false, "branch_deleted_by": null, "branch_deleted_at": null, "into_branch": "main", "into_branch_sharded": false, "into_branch_shard_count": 0, "approved": false, "state": "open", "deployment_state": "queued", "num_comments": 0, "deployment": { "id": "uvkd7injje2f", "type": "Deployment", "into_branch": "main", "deploy_request_number": 5, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cutover_actor": null, "cancelled_actor": null, "schema_last_updated_at": "2023-10-25T16:55:00.288Z", "preceding_deployments": [], "deploy_operations": [ { "id": "wo1m619ufrpc", "type": "DeployOperation", "state": "pending", "keyspace_name": "example_database", "table_name": "Persons", "operation_name": "ALTER", "eta_seconds": null, "progress_percentage": null, "deploy_error_docs_url": null, "ddl_statement": "ALTER TABLE `Persons` DROP COLUMN `Address`", "syntax_highlighted_ddl": "
ALTER TABLE `Persons`
DROP COLUMN `Address`
", "created_at": "2023-10-25T16:55:26.562Z", "updated_at": "2023-10-25T16:55:26.562Z", "can_drop_data": true, "table_recently_used": false, "table_recently_used_at": null, "deploy_errors": null } ], "deploy_operation_summaries": [], "deployable": true, "cutover_expiring": false, "lint_errors": [], "deployment_revert_request": null, "auto_cutover": false, "created_at": "2023-10-25T16:54:59.863Z", "cutover_at": null, "deploy_check_errors": null, "finished_at": null, "queued_at": "2023-10-25T16:55:53.543Z", "ready_to_cutover_at": null, "started_at": null, "state": "queued", "submitted_at": null, "updated_at": "2023-10-25T16:55:53.552Z" }, "html_url": "https://app.planetscale.com/demo-db/example_database/deploy-requests/5", "number": 5, "notes": "", "html_body": "", "created_at": "2023-10-25T16:54:59.797Z", "updated_at": "2023-10-25T16:55:53.556Z", "closed_at": null, "deployed_at": null } } ``` ### Deploy request in progress The deploy request has started running. The `deploy_request_in_progress` event uses the same response body as a `200` response from the [Get a deploy request](/docs/api/reference/get_deploy_request) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1698252961, "event": "deploy_request.in_progress", "organization": "myorg", "database": "example_database", "resource": { "id": "4xsz0ql82y4n", "type": "DeployRequest", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "closed_by": null, "branch": "dev", "branch_deleted": false, "branch_deleted_by": null, "branch_deleted_at": null, "into_branch": "main", "into_branch_sharded": false, "into_branch_shard_count": 0, "approved": false, "state": "open", "deployment_state": "in_progress", "num_comments": 0, "deployment": { "id": "uvkd7injje2f", "type": "Deployment", "into_branch": "main", "deploy_request_number": 5, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cutover_actor": null, "cancelled_actor": null, "schema_last_updated_at": "2023-10-25T16:55:00.288Z", "preceding_deployments": [], "deploy_operations": [ { "id": "wo1m619ufrpc", "type": "DeployOperation", "state": "pending", "keyspace_name": "example_database", "table_name": "Persons", "operation_name": "ALTER", "eta_seconds": null, "progress_percentage": null, "deploy_error_docs_url": null, "ddl_statement": "ALTER TABLE `Persons` DROP COLUMN `Address`", "syntax_highlighted_ddl": "
ALTER TABLE `Persons`
DROP COLUMN `Address`
", "created_at": "2023-10-25T16:55:26.562Z", "updated_at": "2023-10-25T16:55:26.562Z", "can_drop_data": true, "table_recently_used": false, "table_recently_used_at": null, "deploy_errors": null } ], "deploy_operation_summaries": [], "deployable": true, "cutover_expiring": false, "lint_errors": [], "deployment_revert_request": null, "auto_cutover": false, "created_at": "2023-10-25T16:54:59.863Z", "cutover_at": null, "deploy_check_errors": null, "finished_at": null, "queued_at": "2023-10-25T16:55:53.543Z", "ready_to_cutover_at": null, "started_at": "2023-10-25T16:56:01.035Z", "state": "in_progress", "submitted_at": "2023-10-25T16:55:53.702Z", "updated_at": "2023-10-25T16:56:01.057Z" }, "html_url": "https://app.planetscale.com/demo-db/example_database/deploy-requests/5", "number": 5, "notes": "", "html_body": "", "created_at": "2023-10-25T16:54:59.797Z", "updated_at": "2023-10-25T16:56:01.063Z", "closed_at": null, "deployed_at": null } } ``` ### Deploy request errored The deploy request has stopped due to an error. The `deploy_request.errored` event uses the same response body as a `200` response from the [Get a deploy request](/docs/api/reference/get_deploy_request) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1697736651, "event": "deploy_request.errored", "organization": "myorg", "database": "example_database", "resource": { "id": "jwdnj3q31jd6", "type": "DeployRequest", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "closed_by": null, "branch": "dev", "branch_deleted": false, "branch_deleted_by": null, "branch_deleted_at": null, "into_branch": "main", "into_branch_sharded": false, "into_branch_shard_count": 0, "approved": false, "state": "open", "deployment_state": "error", "num_comments": 0, "deployment": { "id": "xqvfpq4yllwc", "type": "Deployment", "into_branch": "main", "deploy_request_number": 1, "actor": { "id": "boo", "type": "User", "display_name": "Ghost", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cutover_actor": null, "cancelled_actor": null, "schema_last_updated_at": "2023-10-19T17:29:49.430Z", "preceding_deployments": [], "deploy_operations": [], "deploy_operation_summaries": [], "deployable": false, "cutover_expiring": false, "lint_errors": [ { "type": "DeploymentLintError", "lint_error": "NO_UNIQUE_KEY", "subject_type": "table_error", "statement": "", "keyspace_name": "example_database", "table_name": "Persons", "error_description": "table \"Persons\" has no unique key: all tables must have at least one unique, not-null key without using text / blob columns or partial indexes.", "column_name": "", "foreign_key_column_names": [], "auto_increment_column_names": [], "charset_name": "", "engine_name": "", "vindex_name": null, "json_path": null, "schema_identifier": "SOURCE", "conflict_columns": null, "conflict_indexes": null, "conflict_constraints": null, "conflict_partitions": null, "conflict_error": "CONFLICT_UNSPECIFIED", "conflict_ddl_verb": "DDL_VERB_UNSPECIFIED", "check_constraint_name": "", "enum_value": "", "partitioning_type": "", "partition_name": "", "view_name": "", "boost_query": null } ], "deployment_revert_request": null, "auto_cutover": true, "created_at": "2023-10-19T17:29:49.032Z", "cutover_at": null, "deploy_check_errors": "", "finished_at": null, "queued_at": null, "ready_to_cutover_at": null, "started_at": null, "state": "error", "submitted_at": null, "updated_at": "2023-10-19T17:29:54.166Z" }, "html_url": "https://app.planetscale.com/demo-db/example_database/deploy-requests/1", "number": 1, "notes": "", "html_body": "", "created_at": "2023-10-19T17:29:48.993Z", "updated_at": "2023-10-19T17:29:54.197Z", "closed_at": null, "deployed_at": null } } ``` ### Deploy request pending cutover The deploy request is ready to apply the schema and is waiting on the user to confirm. The `deploy_request.pending_cutover` event uses the same response body as a `200` response from the [Get a deploy request](/docs/api/reference/get_deploy_request) API endpoint. The link includes a detailed description of each field in the API reference. **Example:** ```json expandable theme={null} { "timestamp": 1698252961, "event": "deploy_request.pending_cutover", "organization": "myorg", "database": "example_database", "resource": { "id": "4xsz0ql82y4n", "type": "DeployRequest", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "closed_by": null, "branch": "dev", "branch_deleted": false, "branch_deleted_by": null, "branch_deleted_at": null, "into_branch": "main", "into_branch_sharded": false, "into_branch_shard_count": 0, "approved": false, "state": "open", "deployment_state": "pending_cutover", "num_comments": 0, "deployment": { "id": "uvkd7injje2f", "type": "Deployment", "into_branch": "main", "deploy_request_number": 5, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cutover_actor": null, "cancelled_actor": null, "schema_last_updated_at": "2023-10-25T16:55:00.288Z", "preceding_deployments": [], "deploy_operations": [ { "id": "wo1m619ufrpc", "type": "DeployOperation", "state": "in_progress", "keyspace_name": "example_database", "table_name": "Persons", "operation_name": "ALTER", "eta_seconds": null, "progress_percentage": 100, "deploy_error_docs_url": null, "ddl_statement": "ALTER TABLE `Persons` DROP COLUMN `Address`", "syntax_highlighted_ddl": "
ALTER TABLE `Persons`
DROP COLUMN `Address`
", "created_at": "2023-10-25T16:55:26.562Z", "updated_at": "2023-10-25T16:55:26.562Z", "can_drop_data": true, "table_recently_used": false, "table_recently_used_at": null, "deploy_errors": null } ], "deploy_operation_summaries": [], "deployable": true, "cutover_expiring": false, "lint_errors": [], "deployment_revert_request": { "id": "11ljwej314iy", "type": "DeploymentRevert", "actor": { "id": "boo", "type": "User", "display_name": "Ghost", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cancelled_at": null, "finished_at": null, "waiting_period_end_at": null, "deploy_revert_operations": [] }, "auto_cutover": false, "created_at": "2023-10-25T16:54:59.863Z", "cutover_at": null, "deploy_check_errors": null, "finished_at": null, "queued_at": "2023-10-25T16:55:53.543Z", "ready_to_cutover_at": null, "started_at": "2023-10-25T16:56:01.035Z", "state": "in_progress", "submitted_at": "2023-10-25T16:55:53.702Z", "updated_at": "2023-10-25T16:56:01.057Z" }, "html_url": "https://app.planetscale.com/demo-db/example_database/deploy-requests/5", "number": 5, "notes": "", "html_body": "", "created_at": "2023-10-25T16:54:59.797Z", "updated_at": "2023-10-25T16:56:01.063Z", "closed_at": null, "deployed_at": null } } ``` ### Deploy request schema applied The deploy request has finished applying the schema. The `deploy_request.schema_applied` event uses the same response body as a `200` response from the [Get a deploy request](/docs/api/reference/get_deploy_request) API endpoint. The link includes a detailed description of each field in the API reference. ```json expandable theme={null} { "timestamp": 1698252989, "event": "deploy_request.schema_applied", "organization": "myorg", "database": "example_database", "resource": { "id": "4xsz0ql82y4n", "type": "DeployRequest", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "closed_by": null, "branch": "dev", "branch_deleted": false, "branch_deleted_by": null, "branch_deleted_at": null, "into_branch": "main", "into_branch_sharded": false, "into_branch_shard_count": 0, "approved": false, "state": "open", "deployment_state": "complete_pending_revert", "num_comments": 0, "deployment": { "id": "uvkd7injje2f", "type": "Deployment", "into_branch": "main", "deploy_request_number": 5, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cutover_actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cancelled_actor": { "id": "boo", "type": "User", "display_name": "Ghost", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "schema_last_updated_at": "2023-10-25T16:55:00.288Z", "preceding_deployments": [], "deploy_operations": [ { "id": "krosesxjzl1p", "type": "DeployOperation", "state": "complete", "keyspace_name": "example_database", "table_name": "Persons", "operation_name": "ALTER", "eta_seconds": 0, "progress_percentage": 100, "deploy_error_docs_url": null, "ddl_statement": "ALTER TABLE `Persons` DROP COLUMN `Address`", "syntax_highlighted_ddl": "
ALTER TABLE `Persons`
DROP COLUMN `Address`
", "created_at": "2023-10-25T16:56:01.399Z", "updated_at": "2023-10-25T16:56:28.745Z", "can_drop_data": true, "table_recently_used": false, "table_recently_used_at": null, "deploy_errors": "" } ], "deploy_operation_summaries": [], "deployable": true, "cutover_expiring": false, "lint_errors": [], "deployment_revert_request": { "id": "1cio8bfvx0pp", "type": "DeploymentRevert", "actor": { "id": "boo", "type": "User", "display_name": "Ghost", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cancelled_at": null, "finished_at": null, "waiting_period_end_at": "2023-10-25T17:26:29.153Z", "deploy_revert_operations": [ { "id": "p3r4o6t4wr7x", "type": "DeployRevertOperation", "ddl_statement": "ALTER TABLE `Persons` DROP COLUMN `Address`", "operation_name": "ALTER", "state": "submit", "table_name": "Persons", "created_at": "2023-10-25T16:56:28.833Z", "updated_at": "2023-10-25T16:56:29.067Z", "revert_errors": null } ] }, "auto_cutover": false, "created_at": "2023-10-25T16:54:59.863Z", "cutover_at": "2023-10-25T16:56:22.312Z", "deploy_check_errors": null, "finished_at": "2023-10-25T16:56:29.121Z", "queued_at": "2023-10-25T16:55:53.543Z", "ready_to_cutover_at": "2023-10-25T16:56:01.426Z", "started_at": "2023-10-25T16:56:01.035Z", "state": "complete_pending_revert", "submitted_at": "2023-10-25T16:55:53.702Z", "updated_at": "2023-10-25T16:56:29.121Z" }, "html_url": "https://app.planetscale.com/demo-db/example_database/deploy-requests/5", "number": 5, "notes": "", "html_body": "", "created_at": "2023-10-25T16:54:59.797Z", "updated_at": "2023-10-25T16:56:29.128Z", "closed_at": null, "deployed_at": "2023-10-25T16:56:22.312Z" } } ``` ### Deploy request reverted The deploy request has been reverted. The `deploy_request.reverted` event uses the same response body as a `200` response from the [Get a deploy request](/docs/api/reference/get_deploy_request) API endpoint. The link includes a detailed description of each field in the API reference. ```json expandable theme={null} { "timestamp": 1698253029, "event": "deploy_request.reverted", "organization": "myorg", "database": "example_database", "resource": { "id": "4xsz0ql82y4n", "type": "DeployRequest", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "closed_by": null, "branch": "dev", "branch_deleted": false, "branch_deleted_by": null, "branch_deleted_at": null, "into_branch": "main", "into_branch_sharded": false, "into_branch_shard_count": 0, "approved": false, "state": "open", "deployment_state": "in_progress_revert", "num_comments": 0, "deployment": { "id": "uvkd7injje2f", "type": "Deployment", "into_branch": "main", "deploy_request_number": 5, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cutover_actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cancelled_actor": { "id": "boo", "type": "User", "display_name": "Ghost", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "schema_last_updated_at": "2023-10-25T16:55:00.288Z", "preceding_deployments": [], "deploy_operations": [ { "id": "krosesxjzl1p", "type": "DeployOperation", "state": "complete", "keyspace_name": "example_database", "table_name": "Persons", "operation_name": "ALTER", "eta_seconds": 0, "progress_percentage": 100, "deploy_error_docs_url": null, "ddl_statement": "ALTER TABLE `Persons` DROP COLUMN `Address`", "syntax_highlighted_ddl": "
ALTER TABLE `Persons`
DROP COLUMN `Address`
", "created_at": "2023-10-25T16:56:01.399Z", "updated_at": "2023-10-25T16:56:28.745Z", "can_drop_data": true, "table_recently_used": false, "table_recently_used_at": null, "deploy_errors": "" } ], "deploy_operation_summaries": [], "deployable": true, "cutover_expiring": false, "lint_errors": [], "deployment_revert_request": { "id": "1cio8bfvx0pp", "type": "DeploymentRevert", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cancelled_at": null, "finished_at": "2023-10-25T16:57:09.472Z", "waiting_period_end_at": "2023-10-25T17:26:29.153Z", "deploy_revert_operations": [ { "id": "p3r4o6t4wr7x", "type": "DeployRevertOperation", "ddl_statement": "ALTER TABLE `Persons` DROP COLUMN `Address`", "operation_name": "ALTER", "state": "complete", "table_name": "Persons", "created_at": "2023-10-25T16:56:28.833Z", "updated_at": "2023-10-25T16:57:09.437Z", "revert_errors": "" } ] }, "auto_cutover": false, "created_at": "2023-10-25T16:54:59.863Z", "cutover_at": "2023-10-25T16:56:22.312Z", "deploy_check_errors": null, "finished_at": "2023-10-25T16:56:29.121Z", "queued_at": "2023-10-25T16:55:53.543Z", "ready_to_cutover_at": "2023-10-25T16:56:01.426Z", "started_at": "2023-10-25T16:56:01.035Z", "state": "in_progress_revert", "submitted_at": "2023-10-25T16:55:53.702Z", "updated_at": "2023-10-25T16:57:02.319Z" }, "html_url": "https://app.planetscale.com/demo-db/example_database/deploy-requests/5", "number": 5, "notes": "", "html_body": "", "created_at": "2023-10-25T16:54:59.797Z", "updated_at": "2023-10-25T16:57:02.327Z", "closed_at": null, "deployed_at": "2023-10-25T16:56:22.312Z" } } ``` ### Deploy request closed The deploy request has been closed. The `deploy_request.closed` event uses the same response body as a `200` response from the [Get a deploy request](/docs/api/reference/get_deploy_request) API endpoint. The link includes a detailed description of each field in the API reference. ```json expandable theme={null} { "timestamp": 1698253030, "event": "deploy_request.closed", "organization": "myorg", "database": "example_database", "resource": { "id": "4xsz0ql82y4n", "type": "DeployRequest", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "closed_by": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "branch": "dev", "branch_deleted": false, "branch_deleted_by": null, "branch_deleted_at": null, "into_branch": "main", "into_branch_sharded": false, "into_branch_shard_count": 0, "approved": false, "state": "closed", "deployment_state": "complete_revert", "num_comments": 0, "deployment": { "id": "uvkd7injje2f", "type": "Deployment", "into_branch": "main", "deploy_request_number": 5, "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cutover_actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cancelled_actor": { "id": "boo", "type": "User", "display_name": "Ghost", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "schema_last_updated_at": "2023-10-25T16:55:00.288Z", "preceding_deployments": [], "deploy_operations": [ { "id": "krosesxjzl1p", "type": "DeployOperation", "state": "complete", "keyspace_name": "example_database", "table_name": "Persons", "operation_name": "ALTER", "eta_seconds": 0, "progress_percentage": 100, "deploy_error_docs_url": null, "ddl_statement": "ALTER TABLE `Persons` DROP COLUMN `Address`", "syntax_highlighted_ddl": "
ALTER TABLE `Persons`
DROP COLUMN `Address`
", "created_at": "2023-10-25T16:56:01.399Z", "updated_at": "2023-10-25T16:56:28.745Z", "can_drop_data": true, "table_recently_used": false, "table_recently_used_at": null, "deploy_errors": "" } ], "deploy_operation_summaries": [], "deployable": true, "cutover_expiring": false, "lint_errors": [], "deployment_revert_request": { "id": "1cio8bfvx0pp", "type": "DeploymentRevert", "actor": { "id": "g2dr4sbhz6ag", "type": "User", "display_name": "PlanetScale Bot", "avatar_url": "https://app.planetscale.com/gravatar-fallback.png" }, "cancelled_at": null, "finished_at": "2023-10-25T16:57:09.472Z", "waiting_period_end_at": "2023-10-25T17:26:29.153Z", "deploy_revert_operations": [ { "id": "p3r4o6t4wr7x", "type": "DeployRevertOperation", "ddl_statement": "ALTER TABLE `Persons` DROP COLUMN `Address`", "operation_name": "ALTER", "state": "complete", "table_name": "Persons", "created_at": "2023-10-25T16:56:28.833Z", "updated_at": "2023-10-25T16:57:09.437Z", "revert_errors": "" } ] }, "auto_cutover": false, "created_at": "2023-10-25T16:54:59.863Z", "cutover_at": "2023-10-25T16:56:22.312Z", "deploy_check_errors": null, "finished_at": "2023-10-25T16:56:29.121Z", "queued_at": "2023-10-25T16:55:53.543Z", "ready_to_cutover_at": "2023-10-25T16:56:01.426Z", "started_at": "2023-10-25T16:56:01.035Z", "state": "complete_revert", "submitted_at": "2023-10-25T16:55:53.702Z", "updated_at": "2023-10-25T16:57:09.836Z" }, "html_url": "https://app.planetscale.com/demo-db/example_database/deploy-requests/5", "number": 5, "notes": "", "html_body": "", "created_at": "2023-10-25T16:54:59.797Z", "updated_at": "2023-10-25T16:57:10.046Z", "closed_at": "2023-10-25T16:57:09.995Z", "deployed_at": "2023-10-25T16:56:22.312Z" } } ``` ### Keyspace storage A storage threshold (60%, 75%, 85%, 90%, 95%) has been crossed for a keyspace. > [!NOTE] > > These webhooks are only sent for Vitess databases on PlanetScale Metal instances. ```json theme={null} { "timestamp": 1745526125, "event": "keyspace.storage", "organization": "myorg", "database": "example_database", "resource": { "branch_name": "main", "keyspace_name": "example_database", "storage_percentage": 44, "shards": [ { "shard": "x-x", "bytes_used": 208470605824, "bytes_total_capacity": 472424366080, "bytes_available": 263953760256, "storage_percentage": 44.1 } ] } } ``` ### Webhook test A webhook test is triggered. You can only send one webhook test every 20 seconds. **Example:** ```json expandable theme={null} { "timestamp": 1697738828, "event": "webhook.test", "organization": "myorg", "database": "example_database", "resource": { "id": "34rbgzmvgb9m", "type": "Database", "region": { "id": "kc0e1ij8juzp", "type": "Region", "provider": "AWS", "enabled": true, "public_ip_addresses": [ "23.23.187.137", "52.6.141.108", "52.70.2.89", "50.17.188.76", "52.2.251.189", "52.72.234.74", "35.174.68.24", "52.5.253.172", "54.156.81.4", "34.200.24.255", "35.174.79.154", "44.199.177.24", "35.173.174.19", "44.212.228.57", "44.216.88.45" ], "display_name": "AWS us-east-1", "location": "N. Virginia", "slug": "us-east", "current_default": true }, "name": "example_database", "notes": "", "state": "ready", "created_at": "2023-10-18T22:16:54.071Z", "updated_at": "2023-10-19T17:25:40.577Z", "ready": true, "sharded": false, "html_url": "https://app.planetscale.com/demo-db/example_database", "url": "https://api.planetscale.com/v1/organizations/demo-db/databases/example_database", "branches_url": "https://api.planetscale.com/v1/organizations/demo-db/databases/example_database/branches", "plan": "scaler_pro" } } ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Webhooks Source: https://planetscale.com/docs/api/webhooks Vitess and Postgres ## Webhooks in PlanetScale With a webhook in PlanetScale, you can trigger an HTTP POST callback to a configured URL when a specific event occurs within your PlanetScale organization. The callback payload will contain useful data related to the event. With this data, webhooks can be used to build custom integrations, notifications, and automate other workflows. Webhooks in PlanetScale for Vitess are not like MySQL triggers and cannot be triggered on database table events like `INSERT`, `UPDATE`, or `DELETE`. Instead, think of events for your database cluster's life cycle such as: | Webhook event | Trigger | | :- | :- | | Branch ready | The branch is created and ready to connect. | | Deploy request opened | The deploy request has been opened. | | Deploy request queued | The deploy request has been added to the deploy queue. | | Deploy request closed | The deploy request has been closed. | | Keyspace storage | A keyspace has reached a storage threshold (60%, 75%, 85%, 90%, 95%). | For more information about the events you can trigger a webhook with in PlanetScale, including example payloads, and supported database engines, see the [webhook event reference documentation](/docs/api/webhook-events). `branch.schema_recommendation` fires when PlanetScale creates a new schema recommendation. Vitess, Neki, and Postgres databases support this event. With an [Authorization header](#authenticating-webhook-deliveries), a delivery can start a Cursor or Devin agent, such as a Cursor Automation, a background agent, or a Devin session. See the [schema recommendation event](/docs/api/webhook-events#branch-schema-recommendation). ## Common webhook use cases There are various scenarios where webhooks can be useful, some of them include: * Creating notifications in Slack, Microsoft Teams, GitHub, and other tools * Integrating with CI/CD processes for the automation of schema changes * Updating external issue trackers like Jira * Starting a Cursor or Devin agent when a new schema recommendation is created ## Managing webhooks You can manage webhooks in PlanetScale using three different methods: * **Dashboard** — Create and manage webhooks through the web interface (instructions below) * **CLI** — Use the [`pscale webhook`](/docs/cli/webhook) command to manage webhooks from your terminal * **API** — Use the [webhook API endpoints](/docs/api/reference/list_webhooks) for managing webhooks in your applications ## Setting up a webhook using the dashboard You must be a [database administrator](/docs/security/access-control#database-administrator) to create a webhook for a database. **Step 1** From the PlanetScale organization dashboard, select your database **Step 2** Click on **Settings** from the menu on the left **Step 3** Navigate to the **Webhooks** page from the sub-menu on the left under **Settings** **Step 4** Click **"Add webhook"**. **Step 5** Add an unique URL for the webhook to send events to. The URL must: * Use HTTPS. You can use sites like [https://webhook.site](https://webhook.site/) or tools like [ngrok](https://ngrok.com/docs/getting-started/) to create a HTTPS URL to test webhooks locally. * Be reachable from the internet, no local hosts. * Be able to handle incoming HTTP callbacks. **Step 6** Optional: Add an **Authorization header** if the receiving service requires authentication. Enter the complete value, including the authentication scheme. For example, `Bearer `. PlanetScale sends this value unchanged. This supports webhook triggered Cursor and Devin automations. **Step 7** Select the events you want to trigger the webhook. See the [webhook events reference](/docs/api/webhook-events) for more information on the events and example response bodies. **Step 8** Click **"Save webhook"**. **Step 9** Select the **...** ellipsis and click **"Test webhook"** to send a test event to the webhook. You should receive a POST request with the event type `webhook.test`. **Step 10** On the **"Webhooks"** page, you can see the status of the last sent webhook under each webhook you set up. ## Handling deliveries There are a few things to remember when receiving a webhook from PlanetScale: * You must receive events with an HTTPS server. * Your server must [quickly respond](#responding-to-webhooks-quickly) with a `2xx` successful status code to indicate that the webhook was received successfully. * PlanetScale will not follow any redirects from the server. * PlanetScale's webhooks will originate from one of the following IP addresses: ``` 3.209.149.66 3.215.97.46 34.193.111.15 ``` * We recommend [validating all webhook signatures](#validating-a-webhook-signature) to ensure the webhook came from PlanetScale, not from a third party, and was not tampered with. ### Responding to webhooks quickly To protect your service from being overloaded by webhook deliveries, we recommend responding to the webhook request immediately and handling the processing of the webhook asynchronously. PlanetScale will wait a maximum of 2 seconds for a response from your server before marking the webhook as failed. ### Limits * Each database can have up to **5 webhooks**. If you need more webhooks per database, please [contact us](https://planetscale.com/contact). * You can only send one webhook test **every 20 seconds**. * Webhooks that repeatedly fail will be disabled. ### Authenticating webhook deliveries If the receiving service requires authentication, configure an Authorization header when you create or update the webhook. Enter the complete value, including the authentication scheme—for example, `Bearer `. PlanetScale sends this value unchanged: ```http theme={null} Authorization: Bearer ``` The Authorization header value is write-only. API responses and the dashboard edit form indicate whether a header is configured without revealing its value. When editing a webhook in the dashboard, leave the configured header unchanged to preserve it, choose **Replace** to enter a new value, or choose **Remove** to clear it when you save. In the CLI, omit `--authorization-header` to preserve the configured value, or use `pscale webhook update --clear-authorization-header` to clear it. The Authorization header authenticates PlanetScale to the receiving service. It does not replace the `X-PlanetScale-Signature` header, which lets your own endpoint verify the payload's origin and integrity. ### Validating a webhook signature To ensure that your server only processes webhook deliveries sent by PlanetScale, we recommend validating the webhook signature before processing the delivery further. All webhooks from PlanetScale will have an `X-PlanetScale-Signature` header. This header is a SHA-256 HMAC hex digest of the request body, using your webhook secret as the key. You can use this header to verify that the webhook payload was sent by PlanetScale. To do this, you need to: 1. Retrieve your webhook secret from PlanetScale. Go to your database's settings page > **"Webhooks"** page. Click the **...** ellipsis next to the webhook you want to validate and click **"Show secret"**. 2. Copy and securely store your webhook secret on your server. Never hard code the secret into your application or check it into source control. Follow the best practices for your deployment provider and framework for storing secrets. 3. Validate incoming webhook payloads against the secret to verify that the payload was sent by PlanetScale. You should calculate a hash using the JSON payload and the webhook secret. Then, compare the calculated hash to the `X-PlanetScale-Signature` header. If the two hashes match, the webhook payload is valid. ### Example webhook signature validation The following are examples of validating a webhook signature that uses a SHA-256 HMAC hex digest of the request body. #### JavaScript ```javascript theme={null} const crypto = require('crypto') const WEBHOOK_SECRET = process.env.WEBHOOK_SECRET const verify_signature = (req) => { const signature = crypto.createHmac('sha256', WEBHOOK_SECRET).update(JSON.stringify(req.body)).digest('hex') const trusted = Buffer.from(signature, 'ascii') const header = req.headers.get('x-planetscale-signature') if (header === undefined) { return false } const untrusted = Buffer.from(header) return crypto.timingSafeEqual(trusted, untrusted) } ``` You can then call `verifySignature` in any JavaScript environment when you receive a webhook. > [!WARNING] > > You must create the digest using the *exact* body string sent in the POST request. > If you are using [Express](https://expressjs.com/), you need to ensure that you grab the raw body string. > You'll want to use `bodyParser.raw` instead of `express.json` for getting the POST request body: > > ```javascript theme={null} > const app = express() > app.use(bodyParser.raw({ inflate: true, type: 'application/json' })) > ``` Then you can access the body with `req.body` as shown above. #### TypeScript ```typescript theme={null} import crypto from 'crypto' const WEBHOOK_SECRET: string = process.env.WEBHOOK_SECRET const verify_signature = (req: Request): boolean => { const signature = crypto.createHmac('sha256', WEBHOOK_SECRET).update(JSON.stringify(req.body)).digest('hex') const trusted = Buffer.from(signature, 'ascii') const header = req.headers.get('x-planetscale-signature') if (header === undefined) { return false } const untrusted = Buffer.from(header) return crypto.timingSafeEqual(trusted, untrusted) } ``` You can then call `verify_signature` when you receive a webhook. ```typescript theme={null} const handleWebhook = (req: Request, res: Response) => { if (!verify_signature(req)) { res.status(401).send('Unauthorized') return } // The rest of your logic here } ``` #### Python This example shows how to validate the webhook in a Flask app. ```python expandable theme={null} from flask import Flask, request from hashlib import sha256 import hmac app = Flask(__name__) # Use the PLANETSCALE_WEBHOOK_SECRET environment variable to set the secret token SECRET_TOKEN = os.environ.get('PLANETSCALE_WEBHOOK_SECRET', 'default_secret_token') @app.route('/webhook', methods=['POST']) def webhook(): payload_body = request.data signature_header = request.headers.get('x-planetscale-signature') try: verify_signature(payload_body, SECRET_TOKEN, signature_header) # The signature is valid, you can process the payload here return "Signature is valid." except Exception as e: # Handle the verification failure here return str(e), 403 def verify_signature(payload_body, secret_token, signature_header): if not signature_header: raise Exception("x-planetscale-signature header is missing!") hash_object = hmac.new(secret_token.encode('utf-8'), msg=payload_body, digestmod=sha256) expected_signature = hash_object.hexdigest() if not hmac.compare_digest(expected_signature, signature_header): raise Exception("Request signatures didn't match!") if __name__ == '__main__': app.run(debug=True) ``` #### Ruby on Rails ```ruby theme={null} SECRET = Rails.application.credentials.planetscale.fetch(:webhook_secret) # Private method in your controller def verify_signature body = request.body.read signature = request.headers["X-PlanetScale-Signature"] calculated_signature = OpenSSL::HMAC.hexdigest(OpenSSL::Digest.new("sha256"), SECRET, body) unless ActiveSupport::SecurityUtils.secure_compare(signature, calculated_signature) render(json: { message: "Unauthorized" }, status: :unauthorized) end end ``` You can then call `verify_signature` on each request to validate the webhook. ```ruby theme={null} before_action :verify_signature, only: [:create] ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Billing Source: https://planetscale.com/docs/billing PlanetScale applies billing plans at the **database level**. You can create several databases under one Organization. ## Overview Usage charges are scoped to each database, which you can find all together in the [billing section of your Organization](/docs/billing#organization-usage-and-billing-page). Each plan is further broken down in the [PlanetScale plans documentation](/docs/planetscale-plans). ## Organization usage and billing page Each organization has its own billing page available for administrators, from which you can: * View your current and previous usage per database * Enter/update your credit card information * Activate coupons * Add/update your business address, Vat ID, and other information shown on invoices * Download current and previous invoices **To find your billing page:** **Step 1** Go to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select the organization whose billing page you want to view **Step 3** Click on "Settings" in the top nav **Step 4** Click on "Billing" in the side nav ### PlanetScale invoice details Invoices provide line items for both usage and discounts received. Depending on the configuration of your database and the features you use, you may see line items for: * Your primary database branch * Development branch usage * Storage usage per GB (for non-Metal databases) * Read-only region nodes and storage (for Vitess databases) * Prorated discounts, if the branch existed for a smaller time period than the billing period Storage is prorated by a percentage equal to the existence of a branch's hours/billing period in hours. ### Download an invoice To download an invoice, go to [your billing page](#organization-usage-and-billing-page) (`Organization > Settings > Billing`). You'll see a table of current and previous monthly invoices. You can download an invoice by month by clicking the "**Download**" button. This will send you to a Stripe invoice page, where you'll have the option to download the complete invoice in PDF format, see invoice details, or download your receipt. To see more details about your billing from the PlanetScale dashboard, click the "**View details**" button on the Billing page next to the month you want to view. This will show you an overview of the charges for all of the databases in your organization. ## Payment methods All plans require that your organization has a valid payment method on file. You can use a debit or credit card, but we do not accept pre-paid cards. It's possible to use the same card for multiple organizations, but only after the card has been used to successfully pay for a database in the first organization. You can also show, update, or delete the current card from the [CLI](/docs/cli/billing) or update it from the [MCP server](/docs/mcp-server). Updating always happens in Stripe Checkout; PlanetScale never collects the card number. ## Spend management You have the option to set spend alerts from your organization's billing page. Once on the billing page, click the checkbox on the right next to "Enable spend alerts", type in the max budget, and click "Save". Organization administrators will receive an email when the organization hits 75% and 100% of its maximum monthly spend. ## Using coupons You can redeem a coupon in the PlanetScale dashboard. To redeem a coupon, you must first enter your credit card information. Once you have a credit card on file, go to your Organization Settings page, click "Billing", click "Redeem a coupon" on the right, enter your coupon, and click "Redeem coupon". > [!NOTE] > > You can only have one active coupon at a time. Coupons are not stackable. > [!NOTE] > > You may incur additional costs if your usage continues beyond the period, dollar amount, or any other metrics specified in the coupon terms. Additional costs will be charged to your card on file. If you have any questions about the terms of the coupon, please reach out to [our Support team](/docs/support). ### How do coupons affect invoices? You will see your coupon reflected in your monthly invoice. Go to "Settings" > "Billing" and select the invoice for the month(s) where your coupon was active. You will see a note at the top of the invoice similar to this: `Amount reflects your $xx.xx discount with code YOURCOUPONCODE` ![PlanetScale dashboard - Example coupon factored into invoice](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/billing/coupons-in-invoices.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=6754b22374f3ef1a825edbbcc0f0e01b) ## Canceling your plan Deleting a database will end its plan and prorate the plan fee on your current invoice. You can delete a database from its settings page. ## Why do I see a pre-authorization charge on my card? If you added a new billing method to your account or created a new database afterwards, you may see a temporary hold on your credit card. This is a pre-authorization that we use to verify that your card is valid, but you will not be charged the amount. This pre-authorization is automatically cancelled after verification, but it may take a few days for your bank to update your account statement to show this cancellation. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI Source: https://planetscale.com/docs/cli To interact with PlanetScale and manage your databases, you can use the `pscale` CLI to do the following: * Create, delete and list your Vitess, Neki, and Postgres databases and branches * Run non-interactive SQL for agents and scripts (`pscale sql`) * Open a secure MySQL or PostgreSQL shell instance, including Neki router and replica routing * Manage Neki configuration profiles, shards, routers, sidecars, and data topology * Manage your deploy requests * Bootstrap AI agents (`pscale agent-guide`) * ...and more! > **[Install pscale](/docs/cli/planetscale-environment-setup)** > > Install or upgrade the PlanetScale CLI for macOS, Linux, or Windows. Agents and automation should start with the [Agent setup prompt](/docs/agent-setup/prompt), `pscale --skill`, or `pscale agent-guide --format json`. Agent automation commands require `pscale` 0.292.0 or later; run `brew upgrade pscale` if `agent-guide` is unknown. Always pass `--format json` in automation. > [!NOTE] > > `pscale` can use the MySQL command-line client to quickly open an interactive shell for a database branch. Optional instructions for installing the MySQL client can be found for each platform below. ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## Available Commands Use `pscale [command] [command]` to start up the `pscale` CLI in your terminal. | **Command** | **Subcommands/Options** | **Flags** | **Product** | **Description** | | :- | :- | :- | :- | :- | | [`agent-guide`](/docs/cli/agent-guide) | | `--format json`, `--skill` | Vitess, Postgres, Neki | Bootstrap JSON for AI agents or print the installable agent skill | | [`api`](/docs/cli/api) | | `--help`, `--org string`, `--database string`, `--branch string`, `--field key=value`, `--header stringArray`, `--input string`, `--method string`, `--query key=value` | Vitess, Postgres, Neki | Performs authenticated calls against the PlanetScale API and prints the response to stdout. | | [`audit-log`](/docs/cli/audit-log) | `list`, `auth-attempts download` | `--help`, `--org string` | Vitess, Postgres, Neki | List all [audit logs](/docs/security/audit-log#review-your-organization-audit-log) and download authentication-attempt exports | | [`auth`](/docs/cli/auth) | `login`, `logout`, `check` | `--help`, `--format json` | Vitess, Postgres, Neki | Authenticate via console or JSON device login for agents | | [`backup`](/docs/cli/backup) | `create`, `delete`, `list`, `policy`, `restore`, `restore show`, `show`, `update` | `--help`, `--org string` | Vitess, Postgres, Neki | Manage [Vitess](/docs/vitess/backups), [Postgres](/docs/postgres/backups), and [Neki](/docs/neki/backups) branch backups | | [`billing`](/docs/cli/billing) | `payment-method` | `--help`, `--org string` | Vitess, Postgres, Neki | Show, update, or delete the organization [payment method](/docs/billing#payment-methods) | | [`branch`](/docs/cli/branch) | `admin`, `config-profile`, `connections`, `create`, `data-topology`, `delete`, `demote`, `diff`, `extensions list`, `lint`, `list`, `maintenance`, `parameters list`, `promote`, `query-patterns`, `refresh-schema`, `resize`, `router`, `routing-rules`, `safe-migrations`, `schema`, `shard`, `show`, `sidecar`, `switch`, `switchover`, `update`, `vtctld`, `vtgate` | `--help`, `--org string` | Vitess, Postgres, Neki | Manage [Vitess](/docs/vitess/schema-changes/branching), [Postgres](/docs/postgres/branching), and [Neki](/docs/neki/branching) branches, including [Neki configuration](/docs/neki/cluster-configuration) | | [`completion`](/docs/cli/completion) | `bash`, `zsh`, `fish`, `powershell` | `--help` | Vitess, Postgres, Neki | Generate completion script for specified shell | | [`connect`](/docs/cli/connect) | `` `` | `--execute string`, `--execute-env-url string`, `--execute-protocol string`, `--help`, `--host string`, `--org string`, `--port string`, `--remote-addr string`, `--role string` | Vitess | Create a [secure connection](/docs/vitess/tutorials/connect-any-application#option-2-connect-using-the-planetscale-proxy) to the given database and branch | | [`database`](/docs/cli/database) | `aggressive-cutover`, `create`, `delete`, `dump`, `ip-restriction`, `list`, `read-only-regions`, `regions`, `restore-dump`, `show`, `throttler`, `update` | `--help` | Vitess, Postgres, Neki | Manage databases | | `dedicated-read-replica` | `create`, `delete`, `list`, `show`, `update` | `--help`, `--org string`, `--region string`, `--replicas int`, `--cluster-size string`, `--parameters stringArray`, `--force` | Postgres | Manage dedicated read replicas for a Postgres branch | | [`deploy-request`](/docs/cli/deploy-request) | `apply`, `cancel`, `close`, `create`, `deploy`, `deployment`, `diff`, `force-cutover`, `list`, `operations`, `queue`, `retry`, `revert`, `review`, `reviews`, `show`, `skip-revert`, `storage-check`, `throttler`, `unblock`, `update` | `--help` | Vitess | Manage [deploy requests](/docs/vitess/schema-changes/deploy-requests#create-a-deploy-request) including [gated deployments](/docs/vitess/schema-changes/deploy-requests#gated-deployments) | | `help` | `agents`, `agent-guide`, `api`, `audit-log`, `auth`, `backup`, `billing`, `branch`, `completion`, `connect`, `database`, `dedicated-read-replica`, `deploy-request`, `help`, `import`, `insights`, `inspect`, `keyspace`, `logs`, `maintenance`, `metrics`, `org`, `password`, `pgbouncer`, `ping`, `region`, `role`, `service-token`, `shell`, `signup`, `size`, `sql`, `traffic-control`, `webhook`, `workflow` | `--help` | Vitess, Postgres, Neki | View help for any command or the agent automation help topic | | [`import`](/docs/cli/import) | `d1 doctor`, `d1 lint`, `d1 convert-schema`, `d1 start`, `d1 verify`, `d1 status`, `d1 complete` | `--help`, `--org string` | Postgres | Import external databases ([Cloudflare D1](/docs/postgres/imports/postgres-imports)) into PlanetScale Postgres | | [`insights`](/docs/cli/insights) | `queries`, `queries samples`, `queries show`, `queries summary`, `queries traffic-budgets`, `errors`, `errors show`, `anomalies`, `anomalies show`, `tags`, `tags show`, `tags summaries`, `recommendations`, `recommendations show`, `recommendations dismiss` | `--help`, `--org string`, `--format json` | Vitess, Postgres, Neki | Server-side query insights, anomalies, and schema recommendations from production traffic | | [`inspect`](/docs/cli/inspect) | `all`, `table-sizes`, `index-sizes`, `unused-indexes`, `redundant-indexes`, `invalid-indexes`, `seq-scans`, `long-running-queries`, `locks`, `outliers`, `calls`, `bloat`, `vacuum-stats`, `replication-slots`, `subscriptions` | `--help`, `--org string`, `--format json`, `--keyspace string`, `--dbname string`, `--role string`, `--replica` | Vitess, Postgres, Neki | Live, read-only diagnostic checks over a direct database connection | | [`logs`](/docs/cli/logs) | `` `` | `--help`, `--org string`, `--query string`, `--period string`, `--from string`, `--to string`, `--level string`, `--server string`, `--shard string`, `--limit int`, `--page int` | Postgres, Neki | Query recent [Postgres](/docs/postgres/monitoring/logs) or [Neki](/docs/neki/monitoring/logs) branch logs | | [`keyspace`](/docs/cli/keyspace) | `create`, `create-external`, `delete`, `list`, `parameters`, `read-only-regions`, `resize`, `rollout-status`, `settings`, `show`, `update-settings`, `vschema` | `--help`, `--org string` | Vitess | Manage [keyspaces](/docs/vitess/sharding/keyspaces) within a database branch, including [external keyspaces](/docs/cli/keyspace#create-an-external-keyspace) | | [`maintenance`](/docs/cli/maintenance) | `list`, `show`, `windows` | `--help`, `--org string` | Vitess | View planned maintenance schedules and windows for a database | | [`org`](/docs/cli/org) | `list`, `member`, `show`, `sso`, `switch`, `team`, `update` | `--help` | Vitess, Postgres, Neki | Manage and switch [organizations](/docs/security/access-control), including members, teams, and SSO | | [`metrics`](/docs/cli/metrics) | `instant`, `keyspace-tables`, `queries`, `report`, `show`, `tables`, `tablets`, `tags` | `--help`, `--org string`, `--format json`, `--metric string`, `--period string`, `--from string`, `--to string`, `--steps int` | Vitess, Postgres, Neki | Query historical and current branch metrics from PlanetScale's metrics service | | [`move-tables`](/docs/cli/move-tables) | `list`, `create`, `show`, `status`, `start`, `stop`, `switch-traffic`, `reverse-traffic`, `complete`, `cancel` | `--help`, `--org string` | Vitess | Move tables between keyspaces and [import external MySQL databases](/docs/vitess/imports/database-imports) with [Vitess MoveTables](/docs/cli/move-tables) (`pscale branch vtctl move-tables`) | | [`password`](/docs/cli/password) | `create`, `delete`, `list`, `renew`, `show`, `update` | `--help`, `--org string`, `--name string`, `--new-name string`, `--cidrs strings`, `--status string` | Vitess | Manage [branch credentials](/docs/vitess/connecting/connection-strings) | | [`pgbouncer`](/docs/cli/pgbouncer) | `create`, `delete`, `list`, `resize`, `show` | `--help`, `--org string` | Postgres | Manage dedicated [PgBouncers](/docs/postgres/connecting/pgbouncer) for a Postgres branch | | [`ping`](/docs/cli/ping) | | `--help`, `--count, -n int`, `--concurrency int`, `--provider, -p string` `--timeout duration` | Vitess, Postgres, Neki | Check [latency](/docs/vitess/connecting/network-latency) between your machine and PlanetScale's public regions | | [`region`](/docs/cli/region) | `list` | `--org string` | Vitess, Postgres, Neki | View available [regions](/docs/vitess/regions) | | [`role`](/docs/cli/role) | `create`, `default`, `delete`, `get`, `list`, `reassign`, `renew`, `reset`, `reset-default`, `update` | `--help`, `--org string`, `--inherited-roles string`, `--ttl duration`, `--force`, `--successor string`, `--name string`, `--web` | Postgres, Neki | Manage [Postgres](/docs/postgres/connecting/roles) and [Neki](/docs/neki/connecting/roles) roles | | [`service-token`](/docs/cli/service-token) | `add-access`, `create`, `delete`, `delete-access`, `list`, `show-access` | `--help`, `--org string` | Vitess, Postgres, Neki | Manage access of [service tokens](/docs/api/reference/service-tokens) | | [`size`](/docs/cli/size) | `cluster list` | `--help`, `--org string`, `--engine string`, `--region string`, `--metal`, `--external` | Vitess, Postgres, Neki | View available [cluster sizes](/docs/planetscale-plans), including [external keyspace sizes](/docs/cli/size#list-external-keyspace-cluster-sizes) | | [`traffic-control`](/docs/cli/traffic-control) | `budget`, `rule` | `--help`, `--org string` | Postgres | Manage [Database Traffic Control](/docs/postgres/traffic-control) budgets and rules for a Postgres database branch | | [`shell`](/docs/cli/shell) | `` `[branch_name]` `[db_name]` | `--help`, `--db-name string`, `--local-addr string`, `--org string`, `--remote-addr string`, `--role string`, `--replica`, `--router` | Vitess, Postgres, Neki | Open an interactive shell to the specified database and branch | | [`signup`](/docs/cli/signup) | | `--help` | Vitess, Postgres, Neki | Sign up for a new PlanetScale account | | [`sql`](/docs/cli/sql) | `` `` | `--org string`, `--query string`, `--role string`, `--replica`, `--dbname string`, `--keyspace string`, `--force`, `--vertical`, `--format json` | Vitess, Postgres, Neki | Execute a SQL query without an interactive shell (agents/scripts) | | [`webhook`](/docs/cli/webhook) | `create`, `delete`, `list`, `show`, `test`, `update` | `--help`, `--org string`, `--events string`, `--url string`, `--enabled` | Vitess, Postgres, Neki | Manage [webhooks](/docs/api/webhooks) for databases | | [`workflow`](/docs/cli/workflow) | `cancel`, `complete`, `create`, `cutover`, `list`, `retry`, `reverse-cutover`, `reverse-traffic`, `show`, `switch-traffic`, `verify-data` | `--help`, `--org string` | Vitess | Deprecated. Use [`move-tables`](/docs/cli/move-tables) instead | ## Flags You may use the following flags with the PlanetScale CLI commands. | **Flag** | **Description** | | :- | :- | | `--api-token string` | The API token to use for authenticating against the PlanetScale API | | `--api-url string` | The base URL for the PlanetScale API. (default "[https://api.planetscale.com/](https://api.planetscale.com/)") | | `--config string` | Config file *(default: `$HOME/.config/planetscale/pscale.yml`)* | | `--debug` | Enable debug mode | | `-f, --format string` | Show output in specific format. Possible values: *\[human, json, csv] (default: "human")* | | `-h, --help` | Get more information about a command | | `--no-color` | Disable color output | | `--service-token string` | Service Token for authenticating | | `--service-token-id string` | The Service Token ID for authenticating | | `--skill` | Print the installable agent skill file and exit | | `--version` | Show pscale version | ## Service tokens permissions A complete list of access permissions available for use with service tokens can be found in the [PlanetScale API documentation](/docs/api/reference/service-tokens#access-permissions). ## Service token automation Running `pscale` in CI or an AI agent uses a [service token](/docs/cli/service-tokens) instead of `pscale auth login`. Provide the token as environment variables (recommended) or as flags on each command; both are equivalent: ```bash theme={null} # Environment variables (recommended for CI and agents) export PLANETSCALE_SERVICE_TOKEN_ID="" export PLANETSCALE_SERVICE_TOKEN="" # Or per-command flags pscale database list --org \ --service-token-id "" \ --service-token "" ``` A service token has **no active organization**, so resource commands need the org supplied explicitly. Set it with `PLANETSCALE_ORG`, the `--org` flag on the subcommand, or `pscale org switch ` (which works with a service token): ```bash theme={null} export PLANETSCALE_ORG="" pscale database list --format json # uses PLANETSCALE_ORG pscale database list --org --format json # explicit flag ``` > [!WARNING] > > `--org` goes on the **resource subcommand** (`database`, `branch`, `sql`, `api`, …), never on root `pscale`. `pscale --org database list` fails with `unknown flag: --org`. Per-command-family matrices (env-var auth, `--service-token` flag, required `--org`, Postgres/Vitess, `--format json`, API equivalent): [`org`](/docs/cli/org#service-token-automation-org) · [`service-token`](/docs/cli/service-token#service-token-automation-service-token) · [`database`](/docs/cli/database#service-token-automation-database) · [`branch`](/docs/cli/branch#service-token-automation-branch) · [`role`](/docs/cli/role#service-token-automation-role) · [`password`](/docs/cli/password#service-token-automation-password). ### Commands to avoid under a service token Do not retry these with service-token auth. The failure is by design, not transient. This is the most common cause of agent retry loops. | Command | Why | Do this instead | Exact error | | :- | :- | :- | :- | | `pscale org show` | A service token has no "current" organization | `pscale org list --format json`, then pass `--org` or `PLANETSCALE_ORG` | `not authenticated yet. Please run 'pscale auth login'` | | `pscale service-token list` | Token management is blocked when authenticated with a token | `pscale api organizations//service-tokens --format json` | `pscale service-token list is unavailable when authenticated with a service token` | | `pscale service-token show-access` | Same as `list` | `pscale api organizations//service-tokens/ --format json` | `pscale service-token show-access is unavailable when authenticated with a service token` | | `pscale service-token` (other sub-commands) | Create/manage tokens blocked under service-token auth | `pscale auth login`, then CLI; or [Service tokens API](/docs/api/reference/service-tokens) | `pscale service-token is unavailable when authenticated with a service token` | | `pscale auth login` / `logout` | Not needed; the token is the credential | Set the token env vars; run `pscale auth check --format json` to confirm | requires an interactive browser device flow | | `pscale shell` / `connect` | Interactive sessions | Use [`pscale sql`](/docs/cli/sql) for non-interactive queries | opens an interactive session | > [!WARNING] > > `pscale org show` prints `not authenticated yet. Please run 'pscale auth login'` under a valid service token. This is the **no-current-org** state, not an auth failure. Do **not** respond by running `pscale auth login`. Discover the org with `pscale org list` and pass it with `--org` or `PLANETSCALE_ORG`. `pscale auth check --format json` confirms a token is wired up. Expect `"authenticated": true`, `"auth_method": "service_token"`, and (until an org is set) an `action_required` status with a `NO_ORG` issue and a non-zero exit code. Resolve it by passing `--org`/`PLANETSCALE_ORG` on your commands. ### `pscale api` fallback for hard-to-parse output When a command's human output is hard to parse, prefer `--format json`, or drop to the raw API with `pscale api`, which returns the API response verbatim and uses the same token: ```bash theme={null} # org list pscale api organizations --format json # database list / show pscale api organizations//databases --format json pscale api organizations//databases/ --format json # branch list / show pscale api organizations//databases//branches --format json pscale api organizations//databases//branches/ --format json # service-token list / show-access (CLI blocked under service-token auth) pscale api organizations//service-tokens --format json pscale api organizations//service-tokens/ --format json ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: agent-guide Source: https://planetscale.com/docs/cli/agent-guide Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `agent-guide` command Shows guidance for AI agents and automation using `pscale`. Use **`--format json`** for a machine-readable bootstrap response with conventions, hosted MCP details, skills install commands, and suggested next steps. Use **`--skill`** to print the same guide as an installable skill file. `pscale --skill` on the root command prints that skill file and exits. > [!NOTE] > > **Requires `pscale` 0.292.0 or later.** Install or upgrade with `brew install pscale` or `brew upgrade pscale`. For a copy-paste agent setup prompt, see [Agent setup prompt](/docs/agent-setup/prompt) ([Markdown export](/docs/agent-setup/prompt.md)). **Usage:** ```bash theme={null} pscale agent-guide --format json ``` ### JSON response fields | Field | Description | | - | - | | `status` | `"ok"` when the guide loaded successfully | | `guide` | Full CLI agent guide text (same content as the CLI repo `AGENTS.md`) | | `first_command` | Suggested first command (`pscale auth check --format json`) | | `agent_guide_command` | Command to re-fetch this guide | | `hosted_mcp_url` | Hosted PlanetScale MCP server URL | | `mcp_docs_url` | Link to MCP documentation | | `skills_repo_url` | [planetscale/skills](https://github.com/planetscale/skills) repository | | `skills_setup_command` | Clone + `script/setup` one-liner | | `skills_npx_command` | `npx skills add planetscale/skills -g -y` | | `skills_cli_automation_skill` | Skill id for CLI conventions (`14-pscale-cli-automation`) | | `supported_engines` | `mysql`, `postgresql` | | `conventions` | Short list of CLI rules for agents | | `next_steps` | Commands to run after reading the guide | `--skill` writes a YAML-frontmatter skill file to stdout and skips this JSON object. Use `pscale agent-guide --skill` or `pscale --skill` when you need the installable skill rather than the bootstrap response. ### Available flags | **Flag** | **Description** | | - | - | | `--skill` | Print the guide as an installable agent skill file and exit | | `-f`, `--format json` | **Required for automation** — JSON on stdout | | `--api-url ` | Non-production API base URL | | `--config ` | Config file (default `$HOME/.config/planetscale/pscale.yml`) | | `-h`, `--help` | Help for `agent-guide` | ## Examples **Human-readable output (default):** ```bash theme={null} pscale agent-guide ``` **Installable skill file:** ```bash theme={null} pscale agent-guide --skill pscale --skill ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: api Source: https://planetscale.com/docs/cli/api Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `api` command Performs authenticated calls against the PlanetScale API and prints the response to stdout. This is a helper for constructing any call to [the PlanetScale API](/docs/api/reference). It can invoke any endpoint, whether support in equivalent commands of the CLI has been added or not. For these reasons, the `api` command is useful in cases such as scripting or where you might use `curl` against the API. **Usage:** ```bash theme={null} pscale api ``` ### Endpoint The `` should be a path for the v1 PlanetScale API, not including the `v1` prefix. You can use placeholders such as `{org}`, `{db}` and `{branch}` in the endpoint, and these will be replaced with the currently selected organization, and the project's database and branch. If invoked from outside a project, `{db}` and `{branch}` yield an empty string. These values can also be specified by the `--org`, `--database`, and `--branch` flags. Refer to the [API documentation](/docs/api/reference) for available endpoints. ### Query parameters Query parameters, such as for pagination, can be added with the `--query`/`-Q` flag, in `key=value` format. ### HTTP methods By default, all HTTP requests use the GET method, unless fields for the body are passed via `--field`/`-F`, in which case a POST method is used. If another method is required, use the `--method`/`-X` flag to specify which one to use. ### Headers Use the `--header`/`-H` flag to specify headers manually. Default headers are already included, such as `User-Agent`, `Content-Type`, `Accept` and `Authorization`. ### Request body If a body is required for the endpoint you're targeting, the `--field`/`-F` flag allows you to specify values at specific paths of a JSON document. Repeat the flag to set multiple values. All bodies sent are JSON objects; it isn't possible to send non-JSON content, or JSON entities of non-object type as the root of the body. If you wish to pass the content of a file as the body, use the `--input`/`-I` flag. If the body should be read from stdin, use `--input="-"`. Fields specified with `--field`/`-F` will be updated in the content of `--input`/`-I` if the content is JSON. If not, an error will be returned. ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for the `api` command. | | `-F`, `--field` key=value | HTTP body to send with the request, in `key=value` format where value is a JSON entity, unless value starts with a `@` in which case, the string after `@` represents a file that will be read. Nested types are represented as `root.depth1.depth2=value`. | | `-H`, `--header` stringArray | HTTP headers to add to the request. | | `-I`, `--input` string | HTTP body to send with the request, as a file that will be read and then sent. Use `-` to read from stdin. | | `-X`, `--method` string | HTTP method to use for the request. Defaults to GET for requests without a body, or POST when a body is specified with `--field`/`-F` or `--input`/`-I`. | | `-Q`, `--query` key=value | Query to append to the URL path, in `key=value` format. | | `--org ` | The organization for the current user. | | `--database ` | The database this project is using. | | `--branch ` | The branch this project is using. | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### To get the current user: **Command:** ```bash theme={null} pscale api user ``` **Output:** The body of the HTTP response. ### To list an organization's databases: **Command:** ```bash theme={null} pscale api organizations/{org}/databases ``` **Output:** The body of the HTTP response. ### To create a database: **Command:** ```bash theme={null} pscale api organizations/{org}/databases -F 'name="my-database"' ``` **Output:** The body of the HTTP response. ### To get a database: **Command:** ```bash theme={null} pscale api organizations/{org}/databases/my-database ``` **Output:** The body of the HTTP response. ### To delete a database: **Command:** ```bash theme={null} pscale api -X DELETE organizations/{org}/databases/my-database ``` **Output:** The body of the HTTP response. ### To add a note on a database from the content of a file: **Command:** ```bash theme={null} pscale api -X PATCH organizations/{org}/databases/{db} -F 'notes=@mynotes.txt' ``` **Output:** The body of the HTTP response. ### To create a database branch from a file: **Command:** ```bash theme={null} pscale api organizations/{org}/databases/{db}/branches --input=spec.json ``` **Output:** The body of the HTTP response. ### To create a database branch from stdin, override the name: **Command:** ```bash theme={null} pscale api organizations/{org}/databases/{db}/branches --input=- -F 'name="my-name"' ``` **Output:** The body of the HTTP response. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: audit log Source: https://planetscale.com/docs/cli/audit-log Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `audit log` command Lists all [audit logs](/docs/security/audit-log) in an organization. The user running the command must have [Organization-level permissions](/docs/security/access-control), specifically `list_organization_audit_logs`. **Usage:** ```bash theme={null} pscale audit-log ``` ### Available sub-commands | **Sub-command** | **Description** | **Product** | | :- | :- | :- | | `list` | List all audit logs in an organization | Postgres, Vitess, Neki | | `auth-attempts download` | Download a report of database authentication attempts | Postgres | ### The `list` sub-command flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `audit-log` command | | `--action` | Filter based on action type | | `--limit` int | The number of events to return. Min: 1, Max: 100 | | `--starting-after` string | The ID of the audit log to start after (for pagination) | | `--org ` | The organization for the current user | ## The `auth-attempts download` sub-command Generates a report of authentication attempts against your Postgres databases and downloads it as a ZIP archive. Each authentication attempt includes information such as the source IP address, credential, targeted database and branch, outcome, and failure details. Available to [Organization Administrators](/docs/security/access-control#organization-administrator) only. Service tokens cannot generate these reports. **Usage:** ```bash theme={null} pscale audit-log auth-attempts download [flags] ``` The command generates the report, waits for it to be ready, and writes the archive to disk. The archive contains two files: the data (`auth-attempts.csv`, `.jsonl`, or `.parquet`) and a `manifest.json` recording the exact window and filters used. ### Time window flags Specify either `--since` or `--start-at`. Named values and timestamps without a zone use your local timezone; requests are sent in UTC. | **Flag** | **Description** | | :- | :- | | `--since` | Window ending now, as a positive duration: `24h`, `7d`, `2w` | | `--start-at` | Inclusive start: `today`, `yesterday`, `YYYY-MM-DD`, local ISO, or RFC3339 | | `--end-at` | Exclusive end: `now`, or any format `--start-at` accepts. Defaults to now | The window is inclusive of `start-at` and exclusive of `end-at`. ### Filter flags Filters combine with AND. Repeat a flag to match any of several values. | **Flag** | **Description** | | :- | :- | | `--source-ip` | Source IPv4 address or CIDR range (repeat or comma-separate) | | `--branch` | Branch public ID or `database/branch` name (repeat or comma-separate) | | `--username` | Authentication username (repeatable; commas are part of the name, not separators) | | `--startup-database` | Startup database name (repeatable) | | `--outcome` | `allow` or `deny` | | `--failure-reason` | `bad_password`, `unknown_user`, `authorization_failed`, `ip_not_allowed`, or `other` | | `--backend-route` | `postgres`, `pgbouncer`, or `unknown` | ### Output flags | **Flag** | **Description** | | :- | :- | | `--export-format` | `csv`, `jsonl`, or `parquet`. Defaults from `--format`: `human` and `csv` produce CSV, `json` produces JSONL. `parquet` must be requested explicitly | | `--output` | Output file name, or `-` to write raw ZIP bytes to stdout | | `--org` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `list` sub-command with `--org` flag **Command:** ```bash theme={null} pscale audit-log list --org ``` **Output:** ```bash theme={null} ID (25) ACTOR (25) ACTION EVENT REMOTE IP LOCATION CREATED AT ------------- ----------- ------------------------ ------------------------ --------------- ---------------- ------------ xxxxxxxxxx Name Open_web_console main branch.open_web_console xxx.xxx.xxx.x Los Angeles, CA 1 day ago ``` ### Pagination Use the ID from the last result and pass it as the `--starting-after` to retrieve the next page of results. ```bash theme={null} pscale audit-log list --limit 5 --starting-after ``` ### Download authentication attempts Export the last 24 hours: ```bash theme={null} pscale audit-log auth-attempts download --since 24h --output auth-attempts.zip ``` **Output:** ```bash theme={null} Generating authentication-attempt export for ... Successfully downloaded auth attempts (csv) to auth-attempts.zip ``` Filter by a source IP address or range: ```bash theme={null} pscale audit-log auth-attempts download --since 7d \ --source-ip 203.0.113.10 --outcome deny --output suspect-ip.zip ``` Filter by authentication username: ```bash theme={null} pscale audit-log auth-attempts download --since 7d \ --username app_readonly --outcome deny --output credential.zip ``` Export a fixed window as Parquet: ```bash theme={null} pscale audit-log auth-attempts download \ --start-at 2026-07-29T00:00:00Z --end-at 2026-07-30T00:00:00Z \ --export-format parquet --output window.zip ``` Write the archive to stdout instead of directly to a file: ```bash theme={null} pscale audit-log auth-attempts download --since 24h --output - > report.zip ``` See [Authentication attempts](/docs/postgres/monitoring/authentication-attempts) for what the report contains and how to work with it. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: auth Source: https://planetscale.com/docs/cli/auth Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `auth` command This command allows you to login, logout, and refresh your authentication. Auth tokens generated by `pscale` are valid for one month. **Usage:** ```bash theme={null} pscale auth ``` ### Available sub-commands | **Sub-Command** | **Description** | **Product** | | :- | :- | :- | | `login` | Authenticate with the PlanetScale API | Postgres, Vitess, Neki | | `logout` | Log out of the PlanetScale API | Postgres, Vitess, Neki | | `check` | Check if you are authenticated | Postgres, Vitess, Neki | > [!NOTE] > > Agents and automation should use **`--format json`** on `auth check` and `auth login`. Start with [`pscale agent-guide --format json`](/docs/cli/agent-guide) or the [Agent setup prompt](/docs/agent-setup/prompt). ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `auth` command | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `check` sub-command **Command:** ```bash theme={null} pscale auth check ``` **Output:** ```bash theme={null} You are authenticated. ``` If you are not authenticated, exit code 1 will be returned. **JSON mode (recommended for agents):** ```bash theme={null} pscale auth check --format json ``` Returns a structured response with `status`, `authenticated`, `auth_method`, `organization`, `issues`, and `next_steps`. | `status` | Meaning | | - | - | | `ok` | Authenticated and ready (may include non-blocking notes such as missing org) | | `action_required` | Exit code 2 — follow `issues` and `next_steps` (login, org switch, or fix credentials) | Service tokens in CI: ```bash theme={null} pscale auth check --format json \ --service-token-id "$PLANETSCALE_SERVICE_TOKEN_ID" \ --service-token "$PLANETSCALE_SERVICE_TOKEN" ``` ### The `login` sub-command (JSON for agents) **Command:** ```bash theme={null} pscale auth login --format json ``` Pending authorization JSON is written to **stderr** while waiting for browser approval. **Stdout** contains a single final JSON object when login completes (`status: ok` or `action_required` if organization setup fails after credentials are saved). Fields include `verification_url`, `user_code`, and `browser_opened`. Open `verification_url` manually if the browser does not open. Do not retry login in a loop without browser access. ### The `login` sub-command (interactive) ```bash theme={null} pscale auth login ``` **Output:** A new browser tab will open and ask you to sign in via browser if you're not already signed in. Next, you'll be asked to confirm the Device confirmation code displayed in your terminal: ```bash theme={null} Confirmation Code: XXXXXXX ``` If they match, click "Confirm code", and you'll be signed in to the CLI. ### The `logout` sub-command **Command:** ```bash theme={null} pscale auth logout ``` **Output:** ```bash theme={null} Press Enter to log out of the PlanetScale API. ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: backup Source: https://planetscale.com/docs/cli/backup Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `backup` command This command allows you to create, list, show, update, restore, and delete branch backups for Vitess, Neki, and Postgres databases, and manage scheduled backup policies. **Usage:** ```bash theme={null} pscale backup ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Description** | **Product** | | :- | :- | :- | :- | | `create ` | `--name` | Backup a branch's data and schema | Postgres, Vitess, Neki | | `delete ` | `--force` | Delete a branch backup | Postgres, Vitess, Neki | | `list ` | | List all backups of a branch | Postgres, Vitess, Neki | | `policy list ` | | List backup policies for a database | Postgres, Vitess, Neki | | `policy show ` | | Show a backup policy | Postgres, Vitess, Neki | | `policy create ` | `--target`\*, `--retention-value`\*, `--retention-unit`\*, `--frequency-value`\*, `--frequency-unit`\*, `--schedule-time`\*, `--name`, `--schedule-day`, `--schedule-week` | Create a backup policy | Postgres, Vitess, Neki | | `policy update ` | `--name`, `--target`, `--retention-value`, `--retention-unit`, `--frequency-value`, `--frequency-unit`, `--schedule-time`, `--schedule-day`, `--schedule-week` | Update a backup policy | Postgres, Vitess, Neki | | `policy delete ` | `--force` | Delete a backup policy | Postgres, Vitess, Neki | | `restore ` | `--cluster-size`, `--replicas`, `--config-profile`, `--router` | Restore a backup to a new branch | Postgres, Vitess, Neki | | `restore show ` | | Preview the Neki configuration-profile and router sizes a restore will use if you do not override them | Neki | | `show ` | | Show a specific backup of a branch | Postgres, Vitess, Neki | | `update ` | `--protected`\* | Update a backup's protected status | Postgres, Vitess, Neki | > \* *Flag is required* #### Sub-command flag descriptions | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--name` | Optional name for an on-demand backup or backup policy. | `create`, `policy create`, `policy update` | | `--cluster-size` | Cluster size for the restored branch. Optional for Neki; omit it to keep the source default configuration-profile size. | `restore` | | `--replicas` | Number of additional replicas for a Postgres or Neki restore. `0` creates a single-node Postgres branch; Neki requires a highly available configuration. Omit to use the target cluster size default. Not used for Vitess. | `restore` | | `--config-profile` | Neki restore override: `name=[,cluster-size=][,replicas=]`. Repeatable. Omitted profiles inherit the source. | `restore` | | `--router` | Neki restore override: `name=[,size=][,replicas-per-cell=]`. Repeatable. Omitted routers inherit the source. | `restore` | | `--target` | Branch target: `production` or `development`. | `policy create`, `policy update` | | `--retention-value` | Retention period value. | `policy create`, `policy update` | | `--retention-unit` | Retention unit: `hour`, `day`, `week`, `month`, or `year`. | `policy create`, `policy update` | | `--frequency-value` | Frequency value. | `policy create`, `policy update` | | `--frequency-unit` | Frequency unit: `hour`, `day`, `week`, or `month`. | `policy create`, `policy update` | | `--schedule-time` | Schedule time of day in `HH:MM` format. | `policy create`, `policy update` | | `--schedule-day` | Day of week (`0`=Sunday … `6`=Saturday); used for weekly/monthly schedules. | `policy create`, `policy update` | | `--schedule-week` | Week of month (`0`=first … `3`=fourth); used for monthly schedules. | `policy create`, `policy update` | | `--force` | Delete a backup or backup policy without confirmation. | `delete`, `policy delete` | | `--protected` | Protect the backup from deletion (`--protected=false` to disable). Required on `update`. | `update` | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `backup` command | | `--org ` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `list` sub-command with `--org` flag **Command:** ```bash theme={null} pscale backup list --org ``` **Output:** ```bash theme={null} ID NAME STATE SIZE CREATED AT UPDATED AT STARTED AT EXPIRES AT COMPLETED AT -------------- --------------------- --------- ------- ------------- ------------- ------------- ------------------- -------------- xxxxxxxx 2022.02.11 16:01:03 success 24.1M 3 hours ago 3 hours ago 3 hours ago 1 day from now 3 hours ago xxxxxxxx 2022.02.10 16:01:03 success 23.2M 1 day ago 1 day ago 1 day ago 20 hours from now 1 day ago ``` ### The `show` sub-command **Command:** ```bash theme={null} pscale backup show ``` You can find the `` by running the `pscale backup list ` command. **Output:** ```bash theme={null} ID NAME STATE SIZE CREATED AT UPDATED AT STARTED AT EXPIRES AT COMPLETED AT -------------- --------------------- --------- ------- ------------- ------------- ------------- ------------------- -------------- xxxxxxxx 2022.02.11 16:01:03 success 24.1M 3 hours ago 3 hours ago 3 hours ago 1 day from now 3 hours ago ``` ### Manage backup policies Backup policies define automatic backup frequency, schedule, and retention for production or development branches. This is separate from one-off backups created with `pscale backup create`. ```bash theme={null} pscale backup policy list pscale backup policy create \ --target production \ --retention-value 7 \ --retention-unit day \ --frequency-value 1 \ --frequency-unit day \ --schedule-time 02:00 pscale backup policy update --retention-value 14 pscale backup policy delete ``` ### Restore a backup to a new branch `` is the branch to create. It must not already exist. Identify the backup by ID; the source branch is not a restore argument. ```bash theme={null} pscale backup restore ``` For Neki, omitted configuration-profile and router sizes inherit the live source branch. Preview those defaults first: ```bash theme={null} pscale backup restore show ``` Override individual Neki configuration profiles or routers. `name` is required; size and replica count are optional per entry. ```bash theme={null} pscale backup restore \ --config-profile name=default,cluster-size=PS_40,replicas=2 \ --config-profile name=analytics,replicas=2 \ --router name=default,size=NKR-20,replicas-per-cell=1 ``` `--config-profile` and `--router` are Neki-only. Sidecar, admin, and parameter settings are not restored. You can also restore with [`pscale branch create --restore`](/docs/cli/branch). ### Protect a backup from deletion `--protected` is required. Use `--protected=false` to turn protection off. ```bash theme={null} pscale backup update --protected pscale backup update --protected=false ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: billing Source: https://planetscale.com/docs/cli/billing Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `billing` command This command manages the organization's [payment method](/docs/billing#payment-methods) and invoices. There is one current card. Updating it opens Stripe Checkout in a browser; the CLI never collects the card number. **Usage:** ```bash theme={null} pscale billing ``` `--org` is required. ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Product** | **Description** | | :- | :- | :- | :- | | `invoice list` | `--page `, `--per-page ` | Postgres, Vitess, Neki | List invoices for an organization | | `invoice show ` | | Postgres, Vitess, Neki | Show an invoice | | `invoice line-items ` | `--page `, `--per-page ` | Postgres, Vitess, Neki | List line items for an invoice | | `payment-method show` | | Postgres, Vitess, Neki | Show the current card | | `payment-method update` | | Postgres, Vitess, Neki | Create a Stripe Checkout session, open it when possible, and wait until the card is verified and saved | | `payment-method status ` | | Postgres, Vitess, Neki | Show a Checkout setup started by `update`. Pass the setup id, not the card id from `show` | | `payment-method delete` | `--force` | Postgres, Vitess, Neki | Delete the current card | #### Sub-command flag descriptions | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--force` | Delete the payment method without a confirmation prompt | `payment-method delete` | | `--page ` | Page of results to fetch. Default is `1` for invoices. | `invoice list`, `invoice line-items` | | `--per-page ` | Number of results per page. Default is `25`. | `invoice list`, `invoice line-items` | ### Available flags | **Flag** | **Description** | | - | - | | `-h`, `--help` | View help for `billing` command | | `--org ` | The organization for the current user | `--org` is required. ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. Local override inside a Git repository is `$CWD/.pscale.yml` in the project's root. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | Service tokens need `read_payment_method` to show the card and `write_payment_method` to update or delete it. ## Examples ### Show the current card **Command:** ```bash theme={null} pscale billing payment-method show --org ``` ### Update the card **Command:** ```bash theme={null} pscale billing payment-method update --org ``` This creates a Checkout session and waits until you finish in the browser. Leave the command running. If it is interrupted, use `status` with the setup id it printed. Do not run `update` again while that setup is still pending. Agents should pass `--format json`. Pending details are written to stderr; a final JSON object is written to stdout when Checkout completes. ### Check a setup after an interrupted update **Command:** ```bash theme={null} pscale billing payment-method status --org ``` `status` does not poll. If the setup is still `pending`, wait and run the same command again. ### Delete the current card **Command:** ```bash theme={null} pscale billing payment-method delete --org ``` In JSON mode, pass `--force` after the user has approved the deletion. ### List invoices Invoice list and line-item commands are paginated. Request the next page with `--page` when you need it; do not walk every page automatically. ```bash theme={null} pscale billing invoice list --org pscale billing invoice list --org --page 2 --per-page 25 pscale billing invoice show --org pscale billing invoice line-items --org ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: branch Source: https://planetscale.com/docs/cli/branch Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `branch` command This command creates and manages Vitess, Postgres, and [Neki branches](/docs/neki/branching). It also contains the Neki configuration, shard, router, and data-topology commands. **Usage:** ```bash theme={null} pscale branch ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Description** | **Product** | | :- | :- | :- | :- | | `admin ` | `show`, `update`, `parameters`, `sizes`, `changes` (`list`, `show`, `cancel`) | Manage the Admin configuration for a Neki branch | Neki | | `connections ` | `show`, `top`, `kill`, `kill-transaction` | Inspect and act on live branch connections. See the [`connections` reference](/docs/cli/connections). | Postgres, Vitess | | `config-profile ` | `create`, `default`, `delete`, `list`, `set-default`, `show`, `update`, `parameters`, `extensions` (`enable`, `disable`), `maintenance`, `changes` (`list`, `show`, `cancel`) | Manage configuration profiles for a Neki branch | Neki | | `create ` | `--from `, `--region `, `--restore `, `--restore-point `, `--seed-data`, `--cluster-size `, `--replicas `, `--major-version `, `--min-storage `, `--max-storage `, `--config-profile`, `--router`, `--wait` | Create a new branch on the specified database | Postgres, Vitess, Neki | | `data-topology ` | `get`, `ls` (`--shards`), `update` | View or update a Neki branch's data topology | Neki | | `delete ` | `--force`, `--delete-descendants` | Delete the specified branch from a database | Postgres, Vitess, Neki | | `demote ` | | Demote a production branch to development | Vitess | | `diff ` | `--web` | Show the diff of the specified branch against the parent branch. | Vitess | | `extensions list ` | | List the extensions available on a Postgres branch's cluster image | Postgres | | `lint ` | | Lint the schema of a branch | Vitess | | `list ` | `--web` | List all branches of a database | Postgres, Vitess, Neki | | `maintenance run ` | `--update-postgres-minor-version` (Postgres only) | Run maintenance for a Postgres or Neki branch | Postgres, Neki | | `parameters list ` | `--namespace `, `--extension`, `--internal` | List the configuration parameters of a Postgres branch | Postgres | | `promote ` | | Promote a database branch to production | Vitess | | `query-patterns list ` | `--limit `, `--starting-after ` | List query pattern reports for a branch. See the [`query-patterns` reference](/docs/cli/query-patterns). | Postgres, Vitess | | `query-patterns show ` | | Show a query pattern report. See the [`query-patterns` reference](/docs/cli/query-patterns). | Postgres, Vitess | | `query-patterns delete ` | `--force` | Delete a query pattern report. See the [`query-patterns` reference](/docs/cli/query-patterns). | Postgres, Vitess | | `query-patterns download ` | `--output ` | Generate and download a CSV report of branch query patterns. See the [`query-patterns` reference](/docs/cli/query-patterns). | Postgres, Vitess | | `refresh-schema ` | | Refresh the schema for a database branch | Vitess | | `resize ` | `--cluster-size `, `--replicas `, `--parameters `, `--wait`, `--wait-timeout ` | Change a Postgres branch's cluster size, replica count, or configuration parameters | Postgres | | `resize status ` | | Show the latest change request for a Postgres branch | Postgres | | `resize cancel ` | | Cancel the queued change request for a Postgres branch | Postgres | | `routing-rules get ` | | Show the routing rules of a branch | Vitess | | `routing-rules update ` | `--routing-rules `\* | Replace the routing rules of a branch | Vitess | | `safe-migrations enable ` | | Enables safe migrations for a database branch | Vitess | | `safe-migrations disable ` | | Disables safe migrations for a database branch | Vitess | | `router ` | `create`, `delete`, `list`, `show`, `update`, `sizes`, `changes` (`list`, `show`, `cancel`) | Manage router groups for a Neki branch | Neki | | `schema ` | `--web`, `--keyspace`, `--namespace` | Show the schema of a branch | Postgres, Vitess, Neki | | `shard ` | `assign`, `create`, `delete`, `list`, `show`, `update` | Manage shards and their configuration-profile assignments | Neki | | `show ` | `--web` | Show a specific branch | Postgres, Vitess, Neki | | `sidecar ` | `list`, `show`, `update`, `parameters`, `changes` (`list`, `show`, `cancel`) | Manage connection-pool sidecars for a Neki branch | Neki | | `switch ` | `--database `\*, `--create`, `--parent-branch `, `--wait` | Switch to the specified branch | Postgres, Vitess, Neki | | `switchover ` | `--candidate ` | Move the primary of a Postgres branch to a replica. See [Switchovers](/docs/postgres/troubleshooting/switchovers). | Postgres | | `switchover list ` | `--page `, `--per-page ` | List switchovers for a Postgres branch | Postgres | | `switchover show ` | | Show a Postgres switchover | Postgres | | `update ` | `--new-name `, `--deletion-protected` | Rename a branch or change its deletion protection | Postgres, Vitess, Neki | | `vtgate show ` | | Show the current VTGate configuration for a Vitess branch | Vitess | | `vtgate resize ` | `--vtgate-size `, `--vtgate-count `, `--vtgate-max-count `, `--vtgate-autoscaling`, `--vtgate-target-cpu-utilization ` | Resize VTGates for a Vitess production branch | Vitess | | `vtgate resize status ` | | Show the latest VTGate resize request for a Vitess branch | Vitess | | `vtgate resize cancel ` | | Cancel a queued VTGate resize for a Vitess branch | Vitess | | `vtctl move-tables ` | `list`, `create`, `show`, `status`, `start`, `stop`, `switch-traffic`, `reverse-traffic`, `complete`, `cancel` | Run Vitess MoveTables workflows. `vtctld` also works as an alias. See the [`move-tables` reference](/docs/cli/move-tables). | Vitess | ### Service token automation: `branch` Legend: ✅ supported · 🚫 unavailable · 👤 interactive login only. All sub-commands require `--org` or `PLANETSCALE_ORG`. | Sub-command | Env-var auth | `--service-token` flag | Requires `--org` | Postgres / Vitess | `--format json` | API equivalent | | :- | :-: | :-: | :-: | :- | :-: | :- | | `list ` | ✅ | ✅ | ✅ | Both | ✅ | `pscale api organizations//databases//branches --format json` | | `show ` | ✅ | ✅ | ✅ | Both | ✅ | `pscale api organizations//databases//branches/ --format json` | | `create` / `delete` | ✅ | ✅ | ✅ | Both | ✅ | `POST pscale api organizations//databases//branches --format json` · `DELETE pscale api organizations//databases//branches/ --format json` | | `schema ` | ✅ | ✅ | ✅ | Both | ✅ | `pscale api organizations//databases//branches//schema --format json` | | `diff` / `promote` | ✅ | ✅ | ✅ | Vitess | ✅ | `pscale api organizations//databases//branches//schema/lint --format json` · `POST pscale api organizations//databases//branches//promote --format json` | | `connections` | ✅ | ✅ | ✅ | Both | ✅ | `pscale branch connections top --org --format json` | ```bash theme={null} export PLANETSCALE_ORG="" pscale branch list --format json pscale branch show --format json ``` Setup and commands to avoid: [CLI overview](/docs/cli) · [Service tokens](/docs/api/service-tokens#service-token-automation) > \* *Flag is required* #### Sub-command flag descriptions Some of the sub-commands have additional flags unique to the sub-command. This section covers what each of those does. See the above table for which context. | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--from ` | Parent branch that you want to create a new branch off of | `create` | | `--region ` | Region where database should be created | `create` | | `--restore ` | Create a new branch from a backup ID. For Neki restore sizing, see `--config-profile` and `--router`. | `create` | | `--restore-point ` | For Postgres or Neki, restore to a point-in-time recovery timestamp (for example `2023-01-01T00:00:00Z`). Requires `--restore` or `--from`. Cannot be used with `--seed-data`. | `create` | | `--seed-data` | Create a new branch and seed data using the [Data Branching® feature](/docs/vitess/schema-changes/data-branching) | `create` | | `--web` | Perform the action in your web browser | `diff`, `list`, `schema`, `show` | | `--wait` | Wait until the branch is ready (`create`) or the change request completes (`resize`) | `create`, `resize` | | `--major-version` | PostgreSQL major version for a Postgres or Neki branch. Defaults to the parent branch or the database default. Ignored when restoring from a backup. | `create` | | `--min-storage ` | Minimum storage size in bytes for a Postgres or Neki branch, or for shards in a Neki configuration profile. Not available for Vitess. | `create`, `config-profile create`, `config-profile update` | | `--max-storage ` | Maximum storage size in bytes for storage autoscaling on a Postgres or Neki branch, or for shards in a Neki configuration profile. Not available for Vitess. | `create`, `config-profile create`, `config-profile update` | | `--storage-autoscaling` | Enable storage autoscaling for shards in a Neki configuration profile (`--storage-autoscaling=false` to disable). | `config-profile create`, `config-profile update` | | `--storage-iops ` | Storage IOPS for shards in a Neki configuration profile | `config-profile create`, `config-profile update` | | `--storage-throughput ` | Storage throughput in MiB/s for shards in a Neki configuration profile | `config-profile create`, `config-profile update` | | `--update-postgres-minor-version` | Also upgrade a Postgres branch to the latest PostgreSQL minor version during the maintenance run. | `maintenance run` | | `--database ` | Specify the database name | `switch` | | `--create` | Create a new branch if it does not exist | `switch` | | `--parent-branch ` | If a new branch is being created, use this to specify a parent branch. Default is `main`. | `switch` | | `--delete-descendants` | Recursively delete all descendant branches when deleting a branch | `delete` | | `--limit ` | Number of query pattern reports to return, up to 100. | `query-patterns list` | | `--starting-after ` | Fetch the next page of query pattern reports after a report ID. | `query-patterns list` | | `--output ` | Write the query patterns CSV report to a specific file path. | `query-patterns download` | | `--candidate ` | The replica to promote during a switchover, as returned by `pscale branch infra`. Omit to select automatically. | `switchover` | | `--page ` | Page of results to fetch. | `switchover list` | | `--per-page ` | Number of results per page. Default is `100`. | `switchover list` | | `--new-name ` | New name for the branch | `update` | | `--deletion-protected` | Protect the branch from deletion (`--deletion-protected=false` to disable) | `update` | | `--routing-rules ` | JSON file with the routing rules to set on the branch | `routing-rules update` | | `--cluster-size ` | Cluster size for a new branch, a Postgres branch resize, or shards in a Neki configuration profile. For Neki `create`, omit this flag to use the database default. Use `pscale size cluster list` (`--engine neki` for Neki) or the [cluster sizing](/docs/neki/cluster-configuration/cluster-sizing) menu. | `create`, `resize`, `config-profile create`, `config-profile update` | | `--replicas ` | Additional replica count for a Postgres branch resize, a Postgres or Neki backup restore, or shards in a Neki configuration profile. | `create`, `resize`, `config-profile create`, `config-profile update` | | `--read-only-replicas ` | [Read-only replica](/docs/neki/read-only-replicas) count for shards in a Neki configuration profile, from 0 to 8. | `config-profile create`, `config-profile update` | | `--parameters ` | Set a configuration parameter (e.g. `pgconf.max_connections=200`). Repeat the flag to set multiple parameters. Use `pscale branch parameters list --internal=false` to find parameters you can change for Postgres, or `config-profile parameters`, `admin parameters`, or `sidecar parameters` for Neki. | `resize`, `config-profile update`, `admin update`, `router update`, `sidecar update` | | `--wait-timeout ` | Maximum time to wait for the change request to complete with `--wait`. Default is `10m`. | `resize` | | `--namespace ` | Only show parameters in this namespace (e.g. `pgconf`, `pgbouncer`, `patroni`) | `parameters list` | | `--extension` | Only show parameters that configure an extension (`--extension=false` hides them) | `parameters list` | | `--internal` | Only show internal parameters, which cannot be changed (`--internal=false` hides them) | `parameters list` | | `--vtgate-size ` | VTGate size SKU (e.g. `VTG_320`, `VTG_1280`). Hyphens are accepted and converted to underscores. | `vtgate resize` | | `--vtgate-count ` | Number of VTGates per availability zone (minimum when autoscaling is enabled) | `vtgate resize` | | `--vtgate-max-count ` | Maximum VTGates per availability zone when autoscaling is enabled | `vtgate resize` | | `--vtgate-autoscaling` | Enable or disable VTGate autoscaling (`--vtgate-autoscaling=false` to disable) | `vtgate resize` | | `--vtgate-target-cpu-utilization ` | Target CPU utilization percent when autoscaling is enabled | `vtgate resize` | | `--config-profile ` | Configuration profile that hosts new shards, or the profile to assign or filter by. On `create --restore`, repeat `name=[,cluster-size=][,replicas=]` to override Neki restore sizes. | `create`, `shard create`, `shard assign`, `shard list` | | `--exclude-config-profile ` | Exclude shards assigned to this configuration profile | `shard list` | | `--count ` | Number of shards to create. Defaults to `1`. | `shard create` | | `--display-name ` | Shard display name. Pass an empty string to clear it. | `shard update` | | `--query ` | Search shards by name, display name, or configuration profile | `shard list` | | `--size ` | Router size SKU (for example `NKR-5`) or Admin size (for example `NKA-0`). Hyphens are accepted and converted to underscores. Use `router sizes` or `admin sizes` to list valid SKUs. | `router create`, `router update`, `admin update` | | `--replicas-per-cell ` | Number of router replicas in each cell | `router create`, `router update` | | `--autoscaling` | Enable horizontal router scaling within each cell | `router update` | | `--max-replicas-per-cell ` | Maximum router replicas in each cell when autoscaling | `router update` | | `--target-cpu-utilization ` | Target average CPU utilization when router autoscaling is enabled | `router update` | | `--name ` | New name for a configuration profile | `config-profile update` | | `--postgres-major-version` | PostgreSQL major version for a configuration profile | `config-profile create`, `config-profile update` | | `--postgres-minor-version` | PostgreSQL minor version. Requires `--postgres-major-version`. | `config-profile create`, `config-profile update` | | `--router` | On `create --restore`, repeat `name=[,size=][,replicas-per-cell=]` to override Neki restore router sizes. | `create` | | `--shards` | Group data-topology output by the physical shards that host the data | `data-topology ls` | | `--period ` | Only show changes from this period | `config-profile changes list`, `admin changes list`, `router changes list`, `sidecar changes list` | | `--completed-at ` | Only show changes completed at this time | `config-profile changes list`, `admin changes list`, `router changes list`, `sidecar changes list` | > [!NOTE] > > You can use `--region` with `--restore` to restore a backup into another region. `--restore-point` is available for Postgres and Neki. When you pass `--restore-point` without `--restore`, `--from` is required so the CLI can look up a backup that covers that timestamp. ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for branch command | | `--org ` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `connections` sub-command **Command:** ```bash theme={null} pscale branch connections top ``` Opens a live view of the branch's connection activity. It works for Postgres and Vitess branches; for Vitess, pass `--keyspace` and `--shard` to target a tablet, or run interactively to select them. See the [`connections` reference](/docs/cli/connections), [Inspect live Postgres connections](/docs/postgres/monitoring/connections), and [Inspect live Vitess connections](/docs/vitess/monitoring/connections) for the full workflow. ### Neki configuration commands Neki infrastructure commands are nested under `pscale branch`. See [Neki cluster configuration](/docs/neki/cluster-configuration) and [Neki data topology](/docs/neki/data-topology) for the workflows behind these commands. #### Configuration profiles ```bash theme={null} pscale branch config-profile list pscale branch config-profile default pscale branch config-profile create \ --cluster-size --replicas 2 --postgres-major-version 18 pscale branch config-profile update \ --cluster-size --parameters pgconf.max_connections=200 pscale branch config-profile set-default pscale branch config-profile parameters --namespace pgconf pscale branch config-profile extensions pscale branch config-profile extensions enable pscale branch config-profile extensions disable pscale branch config-profile maintenance pscale branch config-profile changes list pscale branch config-profile changes show pscale branch config-profile changes cancel pscale branch config-profile delete ``` `create` and `update` accept `--cluster-size`, `--replicas`, `--postgres-major-version`, `--postgres-minor-version`, `--min-storage`, `--max-storage`, `--storage-autoscaling`, `--storage-iops`, and `--storage-throughput`. `--postgres-major-version` is required when you pass `--postgres-minor-version`. `update` also accepts `--name` and repeatable `--parameters namespace.name=value`. `parameters` accepts `--namespace`, `--extension`, and `--internal`. `extensions enable` and `extensions disable` toggle an extension that the catalog marks as enablable. `maintenance` upgrades one or more profiles to the latest image and returns immediately; use `pscale branch maintenance run` to maintain every profile on the branch. Change lists accept `--period`, `--completed-at`, `--page`, and `--per-page`. #### Shards ```bash theme={null} pscale branch shard create --config-profile --count 2 pscale branch shard list --config-profile pscale branch shard show pscale branch shard assign --config-profile pscale branch shard update --display-name "tenant-a" pscale branch shard delete ``` `--config-profile` is required on `create` and `assign`. `list` and `show` include whether the shard is the authoritative copy from the branch data topology. `list` accepts `--config-profile`, `--exclude-config-profile`, `--query`, `--page`, and `--per-page`. `update --display-name ""` clears a display name. `delete` accepts `--force`. #### Routers ```bash theme={null} pscale branch router create --size NKR-5 --replicas-per-cell 2 pscale branch router list pscale branch router show pscale branch router sizes pscale branch router update \ --autoscaling --max-replicas-per-cell 8 --target-cpu-utilization 50 pscale branch router changes list pscale branch router changes show pscale branch router changes cancel pscale branch router delete ``` `create` accepts `--size` and `--replicas-per-cell`. `update` also accepts `--autoscaling`, `--max-replicas-per-cell`, `--target-cpu-utilization`, and repeatable `--parameters`. `sizes` lists valid router SKUs. Change lists accept `--period`, `--completed-at`, `--page`, and `--per-page`. `delete` accepts `--force`. Connect through a named router with [`pscale shell --router`](/docs/cli/shell). #### Admin and sidecars Each Neki branch has one Admin. Each configuration profile has one sidecar. Both are created and deleted with the cluster or profile. ```bash theme={null} pscale branch admin show pscale branch admin sizes pscale branch admin parameters pscale branch admin update --size NKA-0 pscale branch admin changes list pscale branch sidecar list pscale branch sidecar show pscale branch sidecar update \ --parameters pgbouncer.default_pool_size=50 pscale branch sidecar changes list ``` `` is the sidecar ID from `sidecar list`, or the configuration profile name. Admin `update` accepts `--size` and `--parameters`. `admin sizes` lists valid Admin SKUs. Sidecar `update` accepts `--parameters`. Change lists include a **Changes** column with the previous and requested size or parameter values, and accept `--period`, `--completed-at`, `--page`, and `--per-page`. #### Data topology and maintenance ```bash theme={null} pscale branch data-topology get pscale branch data-topology ls pscale branch data-topology ls --shards --format json pscale branch data-topology update < data-topology.json pscale branch maintenance run ``` `data-topology update` reads a JSON object from standard input. `ls --shards` groups the topology by the physical shards that host the data. `maintenance run` updates every profile on the branch to the latest available image. #### Restore a Neki backup `pscale branch create --restore` creates a new branch from a backup, the same as [`pscale backup restore`](/docs/cli/backup). For Neki, omit `--cluster-size` to keep the source default profile size. Override individual profiles and routers with repeatable `--config-profile` and `--router` flags. ```bash theme={null} pscale branch create --restore pscale branch create --restore \ --config-profile name=default,cluster-size=PS_40,replicas=2 \ --router name=default,size=NKR-20,replicas-per-cell=1 ``` Preview the live source sizes first with `pscale backup restore show `. ### The `query-patterns` sub-command **Command:** ```bash theme={null} pscale branch query-patterns list --org pscale branch query-patterns show --org pscale branch query-patterns delete --org pscale branch query-patterns download --org ``` List, inspect, and delete existing Query Insights reports for a branch. The `download` command creates a new report, waits for it to finish generating, and downloads the CSV file. See the [`query-patterns` reference](/docs/cli/query-patterns) for the full workflow. ### The `update` sub-command **Command:** ```bash theme={null} pscale branch update --new-name pscale branch update --deletion-protected pscale branch update --deletion-protected=false ``` Renames a branch or changes its deletion protection. Only the flags you pass are sent, and at least one is required. ### The `extensions` sub-command **Command:** ```bash theme={null} pscale branch extensions list ``` Lists the extensions available on a Postgres branch's cluster image. This is the catalog the image can load, not the result of `CREATE EXTENSION`. Preload libraries are configured with `pscale branch resize --parameters`. See [Postgres extensions](/docs/postgres/extensions). ### The `switchover` sub-command **Command:** ```bash theme={null} pscale branch switchover --candidate ``` Moves the primary of a Postgres branch to a replica. With `--candidate`, the named replica is promoted; without it, an eligible replica is selected automatically. On a branch without replicas, the single instance is restarted in place instead. See [Switchovers](/docs/postgres/troubleshooting/switchovers) for the full workflow. List past switchovers or show one by id. `switchover list` supports `--page` and `--per-page`. ```bash theme={null} pscale branch switchover list pscale branch switchover show ``` ### The `list` sub-command with `--web` flag **Command:** ```bash theme={null} pscale branch list --web ``` **Output:** Opens the Branches page, ``, in browser. ### The `parameters` sub-command **Command:** ```bash theme={null} pscale branch parameters list --namespace pgconf ``` Lists the configuration parameters of a Postgres branch, including their current values, defaults, allowed ranges, and whether changing them requires a restart. Use `--namespace` to limit the output to a single namespace such as `pgconf`, `pgbouncer`, or `patroni`. By default, the list includes both internal parameters and parameters you can change. Use `--internal=false` to list only the parameters you can change: ```bash theme={null} pscale branch parameters list --internal=false ``` ### The `resize` sub-command **Command:** ```bash theme={null} pscale branch resize --parameters pgconf.max_connections=200 --parameters pgconf.work_mem=8MB --wait ``` Creates a change request for a Postgres branch. A single change request can update the cluster size (`--cluster-size`), the number of replicas (`--replicas`), and configuration parameters (`--parameters`, repeatable) together. With `--wait`, the command polls until the change request completes; without it, the command returns immediately and you can follow up with `resize status`. The CLI validates `--parameters` against the parameters returned by `parameters list --internal=false`. Parameters that do not exist or cannot be changed are rejected before a change request is submitted. Some parameters require a database restart to take effect; the command tells you which ones before the change is applied. **Command:** ```bash theme={null} pscale branch resize status ``` Shows the latest change request for the branch, including its state (`queued`, `pending`, `resizing`, `completed`, or `canceled`) and what it changes. **Command:** ```bash theme={null} pscale branch resize cancel ``` Cancels the queued change request for the branch. Only change requests that have not started being applied can be canceled. ### The `vtgate` sub-command **Command:** ```bash theme={null} pscale branch vtgate show ``` Shows the current VTGate size, count, and autoscaling settings for a Vitess branch. **Command:** ```bash theme={null} pscale branch vtgate resize \ --vtgate-size VTG_320 \ --vtgate-autoscaling \ --vtgate-count 2 \ --vtgate-max-count 8 \ --vtgate-target-cpu-utilization 50 ``` Queues a VTGate resize for a Vitess production branch. Pass at least one of `--vtgate-size`, `--vtgate-count`, `--vtgate-max-count`, `--vtgate-autoscaling`, or `--vtgate-target-cpu-utilization`. Development branches cannot be resized. See the [VTGate documentation](/docs/vitess/scaling/vtgates) for size defaults, autoscaling limits, and pricing. **Command:** ```bash theme={null} pscale branch vtgate resize status ``` Shows the latest VTGate resize request for the branch. **Command:** ```bash theme={null} pscale branch vtgate resize cancel ``` Cancels a queued VTGate resize. Only resize requests that have not started being applied can be canceled. ### The `vtctl move-tables` sub-command **Command:** ```bash theme={null} pscale branch vtctl move-tables list ``` Lists Vitess MoveTables workflows across every keyspace on a branch. Pass `--target-keyspace` to list one keyspace. The output includes a `next_steps` field with the command to run next. See the [`move-tables` reference](/docs/cli/move-tables). ### The `diff` sub-command **Command:** ```bash theme={null} pscale branch diff ``` **Output:** ```sql theme={null} -- users -- +CREATE TABLE `users` ( + `id` bigint unsigned NOT NULL AUTO_INCREMENT, + `name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL, + PRIMARY KEY (`id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; ``` This will return the diff against the parent branch. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: completion Source: https://planetscale.com/docs/cli/completion ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `completion` command This command allows you to generate a completion script for the specified shell. **Usage:** ```bash theme={null} pscale completion ``` ### Available sub-commands | **Sub-command** | **Description** | **Product** | | :- | :- | :- | | `bash` | Generated completion for `bash` shell | All | | `zsh` | Generated completion for `zsh` shell | All | | `fish` | Generated completion for `fish` shell | All | | `powershell` | Generated completion for `powershell` shell | All | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `completion` command | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## How to load completions ### Bash ```bash theme={null} source <(pscale completion bash) # To load completions for each session, execute once: # Linux: pscale completion bash > /etc/bash_completion.d/pscale # macOS: pscale completion bash > /usr/local/etc/bash_completion.d/pscale ``` ### Zsh ```bash theme={null} # If shell completion is not already enabled in your environment, # you will need to enable it. You can execute the following once: echo "autoload -U compinit; compinit" >> ~/.zshrc # To load completions for each session, execute once: pscale completion zsh > "${fpath[1]}/_yourprogram" # You will need to start a new shell for this setup to take effect. ``` ### fish ```bash theme={null} pscale completion fish | source # To load completions for each session, execute once: pscale completion fish > ~/.config/fish/completions/pscale.fish ``` ### PowerShell ```bash theme={null} pscale completion powershell | Out-String | Invoke-Expression # To load completions for every new session, run: pscale completion powershell > pscale.ps1 # and source this file from your PowerShell profile. ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: connect Source: https://planetscale.com/docs/cli/connect ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). ## The `connect` command This command creates a secure connection to a database branch for a local client. It is supported for **Vitess/MySQL databases only**; to connect to a Postgres database, use a [connection string](/docs/postgres/connecting/quickstart) with a [role](/docs/postgres/connecting/roles). **Usage:** ```bash theme={null} pscale connect ``` If there is only one branch available on the database, you can leave off ``. ### Available flags | **Flag** | **Description** | | :- | :- | | `--execute ` | Run the specified command after successfully connecting to the database. | | `--execute-env-url ` | Environment variable name that contains the exposed Database URL. Default is `DATABASE_URL`. | | `--execute-protocol ` | Protocol for the exposed URL (by default `DATABASE_URL`) value in execute. Default is `mysql2`. | | `-h`, `--help` | Help with the `connect` command. | | `--host ` | Local host to bind and listen for connections. Default is `127.0.0.1`. | | `--org ` | The organization of the database you want to connect to. | | `--port ` | Local port to bind and listen for connections. Default is `3306`. | | `--remote-addr
` | PlanetScale Database remote network address. By default, the remote address is automatically populated from the PlanetScale API. | | `--role ` | Define the access level [with a role](/docs/vitess/security/password-roles) | Available roles for the `--role` flag are: * `reader` * `writer` * `readwriter` * `admin` ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `connect` command with `--execute` flag **Command:** ```bash theme={null} pscale connect --execute 'node app.js' ``` **Output:** This command connects to the specified PlanetScale branch and runs the `node app.js` command. Since no `--execute-env-url` flag was passed, it uses the default `DATABASE_URL` environment variable. You can find a full example of this in our [Node quickstart](/docs/vitess/tutorials/connect-nodejs-app). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: connections Source: https://planetscale.com/docs/cli/connections Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `connections` command Inspect and manage live connection activity for a database branch. Use it to monitor active sessions in real time, find queries that are blocking others, and cancel a query or terminate a connection without opening a SQL shell. It works for Postgres, Vitess, and Neki branches; the command detects the branch's engine and adapts what it shows and what actions are available. There are two ways to use it: * `top` launches an interactive terminal UI to watch activity live and act from the keyboard. * `show` and the action subcommands provide one-shot, scriptable output that works in pipelines and for AI agents. Pair `show --format json` with `kill` and `kill-transaction`. For a walkthrough of the interactive workflow, see [Inspect live Postgres connections](/docs/postgres/monitoring/connections) for Postgres branches, [Inspect live Vitess connections](/docs/vitess/monitoring/connections) for Vitess branches, or [Inspect live Neki connections](/docs/neki/monitoring/connections) for Neki branches. Access depends on your database role and whether the branch is a production branch. It also requires a signed-in user (authenticate with `pscale auth login`) rather than a service token. For the full breakdown, see Permissions for [Postgres](/docs/postgres/monitoring/connections#permissions), [Vitess](/docs/vitess/monitoring/connections#permissions), or [Neki](/docs/neki/monitoring/connections#permissions) branches. Because it runs over a reserved administrative connection, `connections` keeps working even when the database has exhausted its normal connections, so you can still see what's happening and cancel or terminate a connection to recover. ### Database engine support The command surface is the same for both engines, but the engine determines how you target a branch and which actions apply. | | **Postgres** | **Vitess** | **Neki** | | :- | :- | :- | :- | | Targeting flags | `--instance`, `--role` | `--keyspace`, `--shard` | None | | Scope | All instances, including replicas (`--role replica`) | One primary tablet | Client sessions on all routers | | `kill ` | Yes (`pg_terminate_backend`) | Yes (`KILL`) | Yes (terminates the router session) | | `kill --query` | Yes (`pg_cancel_backend`) | Yes (`KILL QUERY`) | Yes (cancels the router session's query) | | `kill-transaction` | Yes | No | Yes | | Blocking/lock detail in `top` | Yes | No | No | Neki `show` and `top` only show router sessions. See [Inspect live Neki connections](/docs/neki/monitoring/connections) for what cancelling and terminating a router session does. Vitess `show` and `top` read a single primary tablet. If a branch has more than one keyspace, or a keyspace is sharded, pass `--keyspace` and `--shard` to pick the tablet. An interactive `top` prompts you to choose when a target is missing; non-interactive runs return an error that lists the available keyspaces or shards. There is no cluster-wide or cross-shard view, and replica connections are not surfaced. **Usage:** ```bash theme={null} pscale branch connections [sub-command] ``` ### Available sub-commands | **Sub-command** | **Product** | **Description** | | :- | :- | :- | | `show` | Postgres, Vitess, Neki | List current live connections once | | `top` | Postgres, Vitess, Neki | Show live connection activity (interactive) | | `kill` | Postgres, Vitess, Neki | Terminate a connection, or cancel its current query with `--query` | | `kill-transaction` | Postgres, Neki | Terminate the connection running a transaction, by `transaction_id` | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `connections` command | | `--org string` | The organization for the current user | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | ## Examples ### The `show` sub-command List the current live connections for a branch once and exit. This is the command to reach for in scripts and automation. Human output uses vertical records (one field per line, like `psql`'s `\x` mode) so query text and other fields are never truncated. Use `--format json` when a script or agent needs to read the `query_id`, `transaction_id`, and `connection_id` fields and act on them. JSON output is an envelope: ```json theme={null} { "database_kind": "postgresql", "captured_at": "2026-06-01T17:10:46Z", "instances": [{ "id": "...", "role": "primary" }], "connections": [ { "pid": 12345, "instance": "...", "instance_role": "primary", "state": "active", "duration_ms": 8200, "wait_event_type": "Lock", "wait_event": "transactionid", "username": "app", "application_name": "checkout", "client_addr": "10.0.0.5", "query_text": "UPDATE orders SET ...", "blocked_by": [], "query_id": "...", "transaction_id": "...", "connection_id": "..." } ] } ``` `connection_id` is always present. `transaction_id` is `null` when the connection has no open transaction, and `query_id` is `null` when it has no current query. Pass the value you want to act on to the corresponding action sub-command. For Vitess branches, the envelope reports `"database_kind": "mysql"` and a `topology` object with the `keyspace`, `shard`, and `tablet` the rows came from instead of an `instances` list. Each row's `connection_id` and `query_id` are tablet-qualified IDs (for example `zone1-1001-101`); blocking and wait-event fields are empty, and `transaction_id` is `null`. For Neki branches, the envelope reports `"database_kind": "neki"`. Each row is a router session. `instance` is the router group, and rows add `router_name`, `router_cell`, `router_uid`, `availability_zone`, `backend_start`, and a `sidecar_backends` list of the shard backends serving the session. PIDs are unique per router, so two rows can share a PID. If some routers couldn't be reached, a `warnings` list says the results are incomplete. Blocking and wait-event fields are empty. **Usage:** ```bash theme={null} pscale branch connections show [flags] ``` **Available flags:** * `--instance string` - For Postgres, filter the list to a single instance, by `id` from the `show` response. * `--role string` - For Postgres, filter the list to rows whose instance role is `primary` or `replica`. Cannot be combined with `--instance`. * `--keyspace string` - For Vitess, target a specific keyspace. * `--shard string` - For Vitess, target a specific shard. **Example:** ```bash theme={null} pscale branch connections show --format json ``` ### The `top` sub-command Show live connection activity. Run it in an interactive terminal to launch the TUI, where you can sort, inspect a connection, and act on connections by keyboard. For Postgres branches you can also drill into a connection's blockers. See [Inspect live Postgres connections](/docs/postgres/monitoring/connections), [Inspect live Vitess connections](/docs/vitess/monitoring/connections), or [Inspect live Neki connections](/docs/neki/monitoring/connections) for a full tour of the interface. When output is piped or redirected, `top` runs headlessly and records snapshots to a trace file. `--capture` is required in this mode. You can replay a captured trace later with `--replay`. For a Vitess branch with more than one keyspace or a sharded keyspace, pass `--keyspace` and `--shard`, or run interactively and `top` will prompt you to select a target. **Usage:** ```bash theme={null} pscale branch connections top [database] [branch] [flags] ``` If you omit the branch, pscale uses the database's branch automatically when there's only one, and prompts you to choose when there are several. **Available flags:** * `--interval duration` - Refresh interval. Default is `1s`. * `--capture string` - Write captured samples to a trace file. Required in headless mode. * `--replay string` - Replay a previously captured trace file in the TUI. Mutually exclusive with `--capture`. * `--duration duration` - Run for this duration. Default is to run until interrupted. * `--instance string` - For Postgres, filter the live view to a single instance, by `id` from the snapshot response. * `--role string` - For Postgres, filter the live view to rows whose instance role is `primary` or `replica`. Cannot be combined with `--instance`. * `--keyspace string` - For Vitess, target a specific keyspace. * `--shard string` - For Vitess, target a specific shard. **Example:** ```bash theme={null} pscale branch connections top ``` Target a specific Vitess shard: ```bash theme={null} pscale branch connections top --keyspace commerce --shard -80 ``` Capture an hour of activity to a trace file, then replay it: ```bash theme={null} pscale branch connections top --capture trace.jsonl --duration 1h pscale branch connections top --replay trace.jsonl ``` ### The `kill` sub-command Terminate a connection by its `connection_id`, or pass `--query` to cancel only the running query by its `query_id`. For Postgres, terminating a connection runs `pg_terminate_backend` and cancelling a query runs `pg_cancel_backend`, which stops the in-flight query but leaves the connection open. For Vitess, these map to `KILL` and `KILL QUERY`; pass the tablet-qualified ID from `show`. Both forms are guarded against stale snapshots. Cancelling a query only takes effect if the connection is still running that same query. Terminating a connection verifies it is still the same connection you observed in `show` (matched by its backend start time), so it cannot act on a different connection that has reused the process ID. If the target has already moved on, the command has no effect. > [!WARNING] > > This command takes effect immediately. There is no confirmation prompt. Run `show` first, confirm you have the correct > target ID, and run `show` again afterward to verify the result. **Usage:** ```bash theme={null} pscale branch connections kill pscale branch connections kill --query ``` **Available flags:** * `--query` - Cancel the `query_id` instead of terminating the `connection_id`. * `--keyspace string` - For Vitess, target a specific keyspace. * `--shard string` - For Vitess, target a specific shard. ### The `kill-transaction` sub-command Terminate the connection running a specific transaction, identified by its `transaction_id`. This runs `pg_terminate_backend`, but only if the connection is still running the same transaction you observed in `show`. If the transaction has already finished or moved on, the action is rejected so you do not terminate the wrong work. This is the safer of the two terminate paths, and the right tool for an idle-in-transaction session that is holding a lock. This sub-command works for Postgres and Neki branches. On Neki, it terminates the router session if it still has the same open transaction, and the transaction is rolled back. Vitess branches do not expose transaction state; use `kill` to terminate the connection or `kill --query` to cancel its current query. > [!WARNING] > > This command is destructive and takes effect immediately, with no confirmation prompt. It terminates a database > connection. Run `show` first to confirm the `transaction_id`, and run `show` again afterward to verify. **Usage:** ```bash theme={null} pscale branch connections kill-transaction ``` ## Related documentation > **[Inspect live Postgres connections](/docs/postgres/monitoring/connections)** > **[Inspect live Vitess connections](/docs/vitess/monitoring/connections)** > **[PlanetScale CLI environment setup](/docs/cli/planetscale-environment-setup)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: database Source: https://planetscale.com/docs/cli/database Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `database` command This command allows you to create, read, update, and delete Vitess, Neki, and Postgres databases. It also provides engine-specific dump, restore, networking, and migration controls. **Usage:** ```bash theme={null} pscale database ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Description** | **Product** | | :- | :- | :- | :- | | `aggressive-cutover show ` | | Show whether aggressive cutover is enabled for future deploy requests | Vitess | | `aggressive-cutover enable ` | | Enable [aggressive cutover](/docs/vitess/schema-changes/aggressive-cutover) for a database | Vitess | | `aggressive-cutover disable ` | | Disable aggressive cutover for a database | Vitess | | `create ` | `--engine `, `--region `, `--cluster-size `, `--major-version `, `--replicas `, `--min-storage `, `--max-storage `, `--wait` | Create a database with the specified name | Postgres, Vitess, Neki | | `delete ` | `--force` | Delete the specified database | Postgres, Vitess, Neki | | `dump ` | `--local-addr
`, `--output `, `--tables `, `--columns `, `--threads (defaults to 16)` | Backup and dump the specified database | Vitess | | `ip-restriction list ` | | List IP restriction entries | Postgres | | `ip-restriction show ` | | Show an IP restriction entry | Postgres | | `ip-restriction create ` | `--cidrs `\*, `--schema `, `--role `, `--description ` | Create an IP restriction entry | Postgres | | `ip-restriction update ` | `--cidrs `, `--schema `, `--role `, `--description ` | Update an IP restriction entry | Postgres | | `ip-restriction delete ` | `--force` | Delete an IP restriction entry | Postgres | | `list` | | List all databases in the current org | Postgres, Vitess, Neki | | `read-only-regions list ` | `--page `, `--per-page ` | List Vitess read-only regions for the database's default branch | Vitess | | `regions list ` | `--page `, `--per-page ` | List regions available to a database for its engine | Postgres, Vitess, Neki | | `restore-dump ` | `--dir `\*, `--local-addr
`, `--overwrite-tables`, `--threads (defaults to 1)`, `--allow-different-destination`, `--show-details`, `--schema-only`, `--data-only`, `--starting-table `, `--ending-table ` | Restore the specified database from a local dump directory | Vitess | | `show ` | `--web` | Retrieve information about a database | Postgres, Vitess, Neki | | `throttler show ` | | Show database-level Vitess migration throttler configuration | Vitess | | `throttler update ` | `--ratio `, `--configuration ` | Update database-level Vitess migration throttler configuration | Vitess | | `update ` | `--new-name`, `--default-branch`, `--restrict-branch-region`, `--production-branch-web-console`, `--insights-raw-queries`, `--require-approval-for-deploy`, `--allow-data-branching`, `--allow-foreign-key-constraints`, `--automatic-migrations`, `--migration-framework`, `--migration-table-name` | Update database settings | Postgres, Vitess, Neki | ### Service token automation: `database` Legend: ✅ supported · 🚫 unavailable · 👤 interactive login only. All sub-commands require `--org` or `PLANETSCALE_ORG`. | Sub-command | Env-var auth | `--service-token` flag | Requires `--org` | Postgres / Vitess | `--format json` | API equivalent | | :- | :-: | :-: | :-: | :- | :-: | :- | | `list` | ✅ | ✅ | ✅ | Both | ✅ | `pscale api organizations//databases --format json` | | `show ` | ✅ | ✅ | ✅ | Both | ✅ | `pscale api organizations//databases/ --format json` | | `create ` | ✅ | ✅ | ✅ | Both | ✅ | `POST pscale api organizations//databases --format json` | | `delete ` | ✅ | ✅ | ✅ | Both | ✅ | `DELETE pscale api organizations//databases/ --format json` | | `dump` | ✅ | ✅ | ✅ | Vitess | N/A | N/A | | `restore-dump` | ✅ | ✅ | ✅ | Both | N/A | N/A | ```bash theme={null} export PLANETSCALE_ORG="" pscale database list --format json pscale database show --format json ``` Setup and commands to avoid: [CLI overview](/docs/cli) · [Service tokens](/docs/api/service-tokens#service-token-automation) > \* *Flag is required* #### Sub-command flag descriptions Some of the sub-commands have additional flags unique to the sub-command. This section covers what each of those does. See the above table for which context. | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--engine ` | Database engine: `mysql` for Vitess, `postgresql` for Postgres (`postgres` is accepted), or `neki` for Neki. Defaults to `mysql`. | `create` | | `--region` | Specify the [region](/docs/vitess/regions) of the new database. When omitted, the API selects the default region. | `create` | | `--cluster-size` | Cluster size for the new database. Use `pscale size cluster list` for Vitess, Neki, and Postgres (`--engine neki`) sizes. See also [Neki cluster sizing](/docs/neki/cluster-configuration/cluster-sizing). | `create` | | `--major-version` | PostgreSQL major version for a Postgres or Neki database. Defaults to the latest available major version. | `create` | | `--replicas` | Number of replicas for a Postgres or Neki database. Neki production databases require at least two replicas during Platform Preview. | `create` | | `--min-storage` | Minimum storage size in bytes for a Postgres or Neki database. Not available for Vitess. | `create` | | `--max-storage` | Maximum storage size in bytes for storage autoscaling on a Postgres or Neki database. Not available for Vitess. | `create` | | `--wait` | Wait until the database is ready. | `create` | | `--force` | Skip confirmation for destructive actions. | `delete`, `ip-restriction delete` | | `--page ` | Page of results to fetch. | `regions list`, `read-only-regions list` | | `--per-page ` | Number of results per page. | `regions list`, `read-only-regions list` | | `--cidrs ` | IPv4 CIDR ranges to allow. Repeatable or comma-separated. Required on create; replaces the existing list on update. | `ip-restriction create`, `ip-restriction update` | | `--schema ` | Postgres schema to restrict. Omit (or empty on update) for all schemas. | `ip-restriction create`, `ip-restriction update` | | `--role ` | Postgres role to restrict. Omit (or empty on update) for all roles. | `ip-restriction create`, `ip-restriction update` | | `--description ` | Optional description for the IP restriction entry. | `ip-restriction create`, `ip-restriction update` | | `--new-name ` | Rename the database. | `update` | | `--default-branch ` | Set the default branch. | `update` | | `--restrict-branch-region` | Limit branch creation to the database region. Boolean; set explicitly with `=true` or `=false`. | `update` | | `--production-branch-web-console` | Allow the web console on the production branch. Boolean; set explicitly with `=true` or `=false`. | `update` | | `--insights-raw-queries` | Collect full SQL for Insights (Vitess only). Boolean; set explicitly with `=true` or `=false`. | `update` | | `--require-approval-for-deploy` | Require admin approval for deploy requests (Vitess only). Boolean; set explicitly with `=true` or `=false`. | `update` | | `--allow-data-branching` | Allow seeding branches with data (Vitess only). Boolean; set explicitly with `=true` or `=false`. | `update` | | `--allow-foreign-key-constraints` | Allow foreign key constraints (Vitess only). Boolean; set explicitly with `=true` or `=false`. | `update` | | `--automatic-migrations` | Copy migration data to new branches and deploy requests (Vitess only). Boolean; set explicitly with `=true` or `=false`. | `update` | | `--migration-framework ` | Migration framework for the database (Vitess only). | `update` | | `--migration-table-name ` | Migration table name for the database (Vitess only). | `update` | | `--ratio ` | Throttler ratio between 0 and 95 applied to all eligible keyspaces. 0 disables throttling; 95 slows migrations the most. | `throttler update` | | `--configuration ` | Per-keyspace throttler ratio as `keyspace=ratio` (repeatable). Use instead of `--ratio`, not together. | `throttler update` | | `--local-addr
` | Local address to bind and listen for connections. By default the proxy binds to 127.0.0.1 with a random port. | `dump`, `restore-dump` | | `--threads` | Number of concurrent threads to use | `dump`, `restore-dump` | | `--output ` | Output directory of the dump. By default the dump is saved to a folder in the current directory. | `dump` | | `--tables ` | Comma separated string of tables to dump. By default, all tables are dumped. | `dump` | | `--columns ` | Specify which columns to include when dumping specific tables. Use format `table:col1,col2`. Can be specified multiple times. | `dump` | | `--wheres string` | Comma separated string of WHERE clauses to filter the tables to dump. | `dump` | | `--replica` | Dump from a replica tablet in the primary region (if available; will fail if not). | `dump` | | `--rdonly` | Dump from a rdonly tablet in the primary region (if available; will fail if not). Not for separate read-only regions. Use `--read-only-region` instead. | `dump` | | `--read-only-region ` | Dump from a Vitess [read-only region](/docs/vitess/scaling/read-only-regions) (region slug, display name, or id). List regions with `pscale keyspace read-only-regions`. Cannot be combined with `--replica` or `--rdonly`. | `dump` | | `--keyspace ` | Optionally target a specific keyspace to be dumped. Useful for sharded databases. | `dump` | | `--shard ` | Optional shard to target, must be used with keyspace | `dump` | | `--output-format ` | Output format for the dump. Options: `sql` (default), `json`, `csv`. | `dump` | | `--dir ` | Directory containing the files to be used for the restore. | `restore-dump` | | `--overwrite-tables` | If true, will attempt to DROP TABLE before restoring. | `restore-dump` | | `--allow-different-destination` | If true, will allow you to restore the files to a database with a different name without needing to rename the existing dump's files. | `restore-dump` | | `--show-details` | If true, will add extra output during the restore process. | `restore-dump` | | `--schema-only` | If true, will only restore the schema files during the restore process. | `restore-dump` | | `--data-only` | If true, will only restore the data files during the restore process. | `restore-dump` | | `--starting-table ` | Table to start from for the restore (useful for restarting from a certain point) | `restore-dump` | | `--ending-table ` | Table to end at for the restore (useful for stopping restore at a certain point) | `restore-dump` | | `--web` | Perform the action in your web browser | `show` | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | Get help with the `database` command | | `--org ` | Specify the organization for the database you're acting upon | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | > [!NOTE] > > The `--format` flag does not apply to the database dump files created by the `dump` subcommand. However, you can control the output format using the `--output-format` flag, which supports SQL (default), JSON, and CSV formats. When using SQL format, dumps are compatible with [mydumper](https://github.com/mydumper/mydumper). ## Examples ### Create a new `scaler_pro` database **Command:** ```bash theme={null} pscale database create new-database --region --cluster-size PS_80 ``` **Output:** Database `new-database` was successfully created. ### Create a Postgres or Neki database with storage size settings Use `--min-storage` and `--max-storage` to set the initial and maximum network-attached storage size in bytes when creating a Postgres or Neki database. These flags are not available for Vitess. **Command:** ```bash theme={null} pscale database create new-postgres-db --engine postgres --region --min-storage 10737418240 --max-storage 21474836480 ``` In this example, the database starts at 10 GiB (`10737418240` bytes) and can scale up to 20 GiB (`21474836480` bytes). The same flags work with `--engine neki`. ### Create a Neki database Use `--engine neki` to create a Neki database. `--wait` keeps the command open until the database is ready. ```bash theme={null} pscale database create new-neki-db \ --engine neki \ --region \ --cluster-size \ --replicas 2 \ --min-storage 10737418240 \ --max-storage 21474836480 \ --wait ``` Choose `` with `pscale size cluster list --engine neki` or from the [Neki cluster sizing](/docs/neki/cluster-configuration/cluster-sizing) menu. ### Create a dump of an existing branch: This command is only available for Vitess databases. For Postgres and Neki databases, use the [pg\_dump](https://www.postgresql.org/docs/current/app-pgdump.html) command instead. **Command:** ```bash theme={null} pscale database dump --output="" --org="" ``` **Output:** A local export of your database will be generated within the current directory by default but since we are providing an `--output` location above that will be used instead. ### Export data in different formats: You can specify the output format when dumping your Vitess database using the `--output-format` flag: **Export as JSON:** ```bash theme={null} pscale database dump --output-format=json ``` **Export as CSV:** ```bash theme={null} pscale database dump --output-format=csv ``` **Export as SQL (default):** ```bash theme={null} pscale database dump --output-format=sql ``` ### Dump specific columns from tables: You can use the `--columns` flag to export only specific columns from your tables. This is useful when you need a partial export or want to exclude sensitive data: ```bash theme={null} # Export only specific columns from multiple tables pscale database dump --columns 'users:id,name' --columns 'orders:id,total' ``` The schema dump will include the complete table schema, but the data files will only contain the specified columns. ### Dump from a read-only region: For Vitess databases with [read-only regions](/docs/vitess/scaling/read-only-regions), dump from a specific region instead of the primary: ```bash theme={null} pscale database dump --read-only-region eu-west ``` Pass a region slug, display name, or id. List configured regions with `pscale database read-only-regions list ` or `pscale keyspace read-only-regions `. ### List available regions `database regions list` returns regions available to that database for its engine. `database read-only-regions list` returns configured Vitess read-only regions for the database's default branch and is rejected for Postgres and Neki databases. ```bash theme={null} pscale database regions list pscale database read-only-regions list ``` ### Update database settings Only flags you pass are sent to the API. Boolean flags must be set explicitly (`=true` or `=false`). Flags marked Vitess-only are rejected for Postgres and Neki databases. ```bash theme={null} pscale database update --default-branch main --restrict-branch-region=true pscale database update --require-approval-for-deploy=true ``` ### Manage Postgres IP restrictions Aliases: `pscale database cidr` and `pscale database cidrs`. Postgres only. ```bash theme={null} pscale database ip-restriction list pscale database ip-restriction create --cidrs 203.0.113.0/24 --description office pscale database ip-restriction update --cidrs 203.0.113.0/24,198.51.100.10/32 pscale database ip-restriction delete ``` ### Database-level Vitess migration throttler Sets the default throttler for future deploy requests on the database. This is not the [per-deploy-request throttler](/docs/cli/deploy-request#the-deploy-request-command-with-throttler-update-subcommand) and not the tablet/vtctld throttler. Vitess only. Use `--ratio` for one ratio across all eligible keyspaces, or `--configuration` for per-keyspace ratios (not both). ```bash theme={null} pscale database throttler show pscale database throttler update --ratio 25 pscale database throttler update --configuration main=10 --configuration sharded=40 ``` ### Database-level aggressive cutover Sets [aggressive cutover](/docs/vitess/schema-changes/aggressive-cutover) for future deploy requests on the database. This is not [`deploy-request force-cutover`](/docs/cli/deploy-request), which forces the cutover of a single migration that is already delayed. To opt in a single deploy request ahead of time, use [`deploy-request create --aggressive-cutover`](/docs/cli/deploy-request). Vitess only. ```bash theme={null} pscale database aggressive-cutover show pscale database aggressive-cutover enable pscale database aggressive-cutover disable ``` ### Restore a backup to an existing branch: **Command:** ```bash theme={null} pscale database restore-dump --dir="" --org="" ``` **Output:** You should receive output indicating the restore is progressing until it completes successfully. The command above will allow you to restore an existing backup to another branch located either within the same organization/database as the original, or within a completely different organization/database. As of `pscale` v0.218.0 or newer the `--allow-different-destination` flag is now available. If this flag is provided it will make the steps below about renaming the files unnecessary. If you opt to import into a database with a different name you will have to make sure you rename the files from your backup beforehand. For example, the files will be named something like this: ```bash theme={null} .-schema.sql ``` And you will want to rename all of the files in the dump folder to have the new database name if it is not the same as the existing one: ```bash theme={null} .-schema.sql ``` If importing into a branch that already contains table definitions that you want to overwrite, you may also be required to pass in the optional `--overwrite-tables` flag. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: deploy-request Source: https://planetscale.com/docs/cli/deploy-request ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `deploy-request` command This command allows you to create, review, diff, and manage deploy requests for your Vitess clusters. This command is not currently available for Postgres database clusters. **Usage:** ```bash theme={null} pscale deploy-request ``` > [!NOTE] > > Your database must have a production branch with [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled before you can create a deploy request. ### Available sub-commands | Sub-command | Sub-command flags | Description | **Product** | | :- | :- | :- | :- | | `apply ` | | Trigger a deploy request to swap over to the new schema. | Vitess | | `cancel ` | | Cancel a deploy request. | Vitess | | `close ` | | Close the specified deploy request. | Vitess | | `create ` | `--into `, `--notes `, `--enable-auto-apply`, `--disable-auto-apply`, `--aggressive-cutover` | Create a new deploy request. | Vitess | | `deploy ` | `--instant`, `--strategy ` | Deploy the specified deploy request. | Vitess | | `deployment ` | | Show the deployment for a deploy request. | Vitess | | `diff ` | `--web` | Show the diff of the specified deploy request. | Vitess | | `edit ` | `--enable-auto-apply`, `--disable-auto-apply`, `--auto-delete-branch`, `--aggressive-cutover` | Alias for `update`. | Vitess | | `force-cutover ` | `--force` | Force cutover when a migration is delayed by a table lock. | Vitess | | `list ` | `--web` | List all deploy requests for a database. | Vitess | | `operations ` | | List deploy operations for a deploy request. | Vitess | | `queue ` | | Show the deploy queue for a database. | Vitess | | `retry ` | | Retry failed tables on a partially failed deploy. | Vitess | | `revert ` | | Revert a deployed deploy request. | Vitess | | `review ` | `--web`, `--approve`, `--comment ` | Approve or comment on a deploy request. | Vitess | | `reviews ` | | List reviews for a deploy request. | Vitess | | `show ` | `--web` | Show the specified deploy request. | Vitess | | `skip-revert ` | | Skip and close a pending deploy request revert. | Vitess | | `storage-check ` | | Check storage readiness for a deploy request. | Vitess | | `throttler show ` | | Show throttler configuration for a deploy request. | Vitess | | `throttler update ` | `--ratio `, `--configuration ` | Update throttler configuration for a deploy request. | Vitess | | `unblock ` | | Unblock the deploy queue after a failed deploy or revert. | Vitess | | `update ` | `--enable-auto-apply`, `--disable-auto-apply`, `--auto-delete-branch`, `--aggressive-cutover` | Update settings on a deploy request. | Vitess | > \* *Flag is required* The value `` represents the deploy request number (not to be confused with `id`). To see a deploy request number, run `pscale deploy-request list `. You can also find the number in the PlanetScale dashboard in the URL of the specified deploy request: `https://app.planetscale.com///deploy-requests/`. #### Sub-command flag descriptions Some of the sub-commands have additional flags unique to the sub-command. This section covers what each of those does. See the above table for which context. | Sub-command flag | Description | Applicable sub-commands | | :- | :- | :- | | `--into ` | Specify that the new deploy request deploy to a specified branch. Default is `main`. | `create` | | `--notes ` | A note describing the deploy request. Acts as the first comment. | `create` | | `--enable-auto-apply` | Enable auto-apply for this deploy request. When enabled, the deploy request will swap over to the new schema once ready. | `create`, `update` | | `--disable-auto-apply` | Disable auto-apply for this deploy request. If neither flag is provided, the setting is inherited from the previous deploy request. | `create`, `update` | | `--auto-delete-branch` | Delete the source branch after the deploy request completes (`--auto-delete-branch=false` to keep it). | `create`, `update` | | `--aggressive-cutover` | Opt this deploy request into [aggressive cutover](/docs/vitess/schema-changes/aggressive-cutover): Vitess kills blocking queries and transactions on the first cutover attempt instead of retrying for up to an hour. Does not change the database setting or other deploy requests. On `update`, pass `--aggressive-cutover=false` to turn it off. | `create`, `update` | | `--web` | Perform the action in your web browser | `diff`, `list`, `show` | | `--approve` | Approve a deploy request | `review` | | `--comment ` | Leave a comment on a deploy request | `review` | | `--instant` | Deploy a deploy request using MySQL's built-in ALGORITHM=INSTANT option. Deployment will be faster, but cannot be reverted. | `deploy` | | `--strategy ` | Deployment strategy. `serial` (default) joins the deploy queue. `parallel` runs alongside the queue. See [Parallel deployments](/docs/vitess/schema-changes/deploy-requests#parallel-deployments). | `deploy` | | `--force` | Skip the confirmation prompt and force cutover now. | `force-cutover` | | `--ratio ` | Throttler ratio between 0 and 95 applied to all eligible keyspaces. 0 disables throttling; 95 slows migrations the most. | `throttler update` | | `--configuration ` | Per-keyspace throttler ratio as `keyspace=ratio` (repeatable). Use instead of `--ratio`, not together. | `throttler update` | ### Available flags | Flag | Description | | :- | :- | | `-h`, `--help` | Get help with the `deploy-request` command | | `--org ` | Specify the organization for the deploy request you're acting upon | ### Global flags | Command | Description | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `deploy-request` command with `review` subcommand and `--comment` flag **Command:** ```bash theme={null} pscale deploy-request review 1 --comment 'Lets wait on this.' ``` **Output:** A comment is added to the deploy request ``/1. ### The `deploy-request` command with `force-cutover` subcommand The final step of a migration requires a brief table lock. Long-running transactions can block that lock and delay completion. PlanetScale retries for up to 1 hour, then forces cutover automatically. Use this command to skip the wait. It kills long-running transactions that are blocking the table lock so the migration can finish. Only allowed when the deployment state is `in_progress_cutover`. See [Aggressive cutover](/docs/vitess/schema-changes/aggressive-cutover). **Command:** ```bash theme={null} pscale deploy-request force-cutover ``` **Output:** Successfully requested force cutover for deploy request ``/``. Vitess will attempt again momentarily. ### The `deploy-request` command with the `--aggressive-cutover` flag Opt a single deploy request into [aggressive cutover](/docs/vitess/schema-changes/aggressive-cutover) ahead of time, so you don't have to wait for the delay and run `force-cutover`. This is per deploy request, not the [database-level setting](/docs/cli/database#database-level-aggressive-cutover), and other deploy requests are unaffected. You can change it until the deploy request is deployed. **Command:** ```bash theme={null} pscale deploy-request create --aggressive-cutover pscale deploy-request update --aggressive-cutover pscale deploy-request update --aggressive-cutover=false ``` **Output:** `show` and `queue` include an `AGGRESSIVE CUTOVER` column. With `--format json`, check `deployment.aggressive_cutover`. > [!WARNING] > > Aggressive cutover terminates active queries and transactions holding locks on the table. Make sure your application retries killed queries. ### The `deploy-request` command with `retry` subcommand When an online schema deploy fails on some tables while others are still copying or ready, the deploy stays in progress. The deploy page shows that it has partially failed and cannot continue. Retry will restart the migration for only the failed tables. Tables that already healthy will keep their existing progress. **Command:** ```bash theme={null} pscale deploy-request retry ``` ### The `deploy-request` command with `throttler update` subcommand Adjust the migration throttler for a single deploy request. This is per deploy request, not the [database-level throttler](/docs/cli/database#database-level-vitess-migration-throttler). Use `--ratio` for one ratio across all eligible keyspaces, or `--configuration` for per-keyspace ratios. **Command:** ```bash theme={null} pscale deploy-request throttler update --ratio 25 ``` **Output:** The throttler configuration for deploy request ``/`` is updated. ### The `deploy-request` command with `unblock` subcommand When a deployment or revert errors, PlanetScale blocks the deploy queue as a precaution. This is the same action as **Unblock deploy queue** in the dashboard. It does not apply a gated deploy, which is [`deploy-request apply`](/docs/cli/deploy-request), and it does not fix a schema that failed deploy checks. **Command:** ```bash theme={null} pscale deploy-request unblock ``` ### The `deploy-request` command with `update` subcommand Changes settings on an existing deploy request. At least one flag is required, and flags you leave off are not sent. `edit` is an alias for this command. **Command:** ```bash theme={null} pscale deploy-request update --enable-auto-apply pscale deploy-request update --auto-delete-branch pscale deploy-request update --auto-delete-branch=false pscale deploy-request update --aggressive-cutover ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: import Source: https://planetscale.com/docs/cli/import ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `import` command This command imports external databases into PlanetScale Postgres. Today it supports **Cloudflare D1** (SQLite) through the `d1` sub-command group. For other migration paths, see the [Postgres imports overview](/docs/postgres/imports/postgres-imports). **Usage:** ```bash theme={null} pscale import d1 ``` Branch-scoped sub-commands take the positional form ` [BRANCH_NAME]`, defaulting to the default branch. The organization comes from your pscale config (`pscale org`) or the `--org` flag. > [!NOTE] > > `pscale import d1` supports the `human` (default) and `json` output formats. The `csv` format is not supported. ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Product** | **Description** | | :- | :- | :- | :- | | `d1 doctor` | | Postgres | Check that pgloader, psql, and other prerequisites are installed | | `d1 lint` | `--input ` | Postgres | Analyze a D1 SQL export for migration issues | | `d1 convert-schema` | `--input `, `--output ` | Postgres | Convert SQLite schema in a D1 export to PostgreSQL DDL | | `d1 start [BRANCH_NAME]` | `--input `, `--dry-run`, `--migration-id `, `--method `, `--dbname `, `--force` | Postgres | Start importing a D1 export (lint + plan, then load) | | `d1 verify [BRANCH_NAME]` | `--migration-id `, `--input `, `--sqlite `, `--dbname ` | Postgres | Verify the import: row counts, sequences, coercion, and content checks | | `d1 status [BRANCH_NAME]` | `--migration-id ` | Postgres | Show local migration state | | `d1 complete [BRANCH_NAME]` | `--migration-id `, `--force` | Postgres | Mark a D1 migration as complete in local state | #### Sub-command flag descriptions Some of the sub-commands have additional flags unique to the sub-command. This section covers what each of those does. See the above table for which context. | **Sub-command flag** | **Type** | **Description** | **Applicable sub-commands** | | :- | :- | :- | :- | | `--input ` | string | Path to the D1 SQL export produced by `wrangler d1 export` (required for `lint`, `convert-schema`, and `start`) | `lint`, `convert-schema`, `start`, `verify` | | `--output ` | string | Output PostgreSQL schema file (default `.postgres.sql`) | `convert-schema` | | `--dry-run` | boolean | Lint and build the import plan without loading Postgres; prints a migration ID for the real run | `start` | | `--migration-id ` | string | Migration ID from a prior `start --dry-run` (required for `verify`, `status`, and `complete`) | `start`, `verify`, `status`, `complete` | | `--method ` | string | Import method: `pgloader` (exports ≥ 1 GB) or `psql` (exports \< 1 GB; schema via psql, data via pgloader). Chosen automatically by export size when omitted | `start` | | `--dbname ` | string | Destination PostgreSQL database name (default `postgres`) | `start`, `verify` | | `--sqlite ` | string | Path to a local SQLite file to use for source row counts | `verify` | | `--force` | boolean | Skip the confirmation prompt | `start`, `complete` | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for import command | | `--org ` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: insights Source: https://planetscale.com/docs/cli/insights Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `insights` command Surface PlanetScale's server-side analysis of a database: aggregated query statistics, failing query patterns, resource anomalies, and schema recommendations, all computed from production traffic. For live, connection-level diagnostics (table sizes, locks, running queries), use [`pscale inspect`](/docs/cli/inspect) instead. Query insights require Query Insights to be enabled for the database. See [Postgres](/docs/postgres/monitoring/query-insights), [Vitess](/docs/vitess/monitoring/query-insights), or [Neki](/docs/neki/monitoring/query-insights). **Usage:** ```bash theme={null} pscale insights [] --org ``` Place **positional arguments first**, then flags. **`--org` is required.** ### Available sub-commands | **Sub-command** | **Product** | **Description** | | :- | :- | :- | | `queries` | Postgres, Vitess, Neki | List top queries ranked by a performance metric | | `queries samples` | Postgres, Vitess, Neki | List recent executions for a query fingerprint | | `queries show` | Postgres, Vitess, Neki | Show one query execution by sample ID | | `queries summary` | Postgres, Vitess, Neki | Show aggregate statistics for a query fingerprint | | `queries traffic-budgets` | Postgres, Vitess | List traffic budgets that affect a query fingerprint | | `errors` | Postgres, Vitess, Neki | List queries that are failing with errors | | `errors show` | Postgres, Vitess, Neki | List the individual query executions that failed with an error fingerprint | | `anomalies` | Postgres, Vitess, Neki | List detected resource anomalies (CPU, memory, IOPS, rows read/written) | | `anomalies show` | Postgres, Vitess, Neki | Show one anomaly and the queries correlated with it | | `tags` | Postgres, Vitess, Neki | List query tag keys (sqlcommenter / system) | | `tags show` | Postgres, Vitess, Neki | Show a query tag key and its values | | `tags summaries` | Postgres, Vitess, Neki | List query statistics grouped by tag keys | | `recommendations` | Postgres, Vitess, Neki | List schema recommendations with ready-to-apply DDL | | `recommendations show` | Postgres, Vitess, Neki | Show one schema recommendation and its full DDL | | `recommendations dismiss` | Postgres, Vitess, Neki | Dismiss a schema recommendation | ### Available flags | **Flag** | **Description** | | - | - | | `--org ` | Organization name **(required)** | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `-h`, `--help` | Help for `insights` | ## Examples ### The `queries` sub-command List the top queries for a branch, ranked by cumulative execution time (default). **Usage:** ```bash theme={null} pscale insights queries --org ``` **Available flags:** | **Flag** | **Description** | | - | - | | `--sort ` | Metric to rank by. One of: `totalTime`, `count`, `errorCount`, `rowsRead`, `rowsReturned`, `rowsAffected`, `rowsReadPerReturned`, `p50Latency`, `p99Latency`, `maxLatency`, `cpuTime`, `ioTime`, `lastRun`. Default: `totalTime`. | | `--dir ` | Sort direction: `asc` or `desc`. Default: `desc`. | | `--limit ` | Number of queries to return. Default: `15`. | | `--period ` | Time period to aggregate over (for example `1h`, `24h`). | **Examples:** ```bash theme={null} pscale insights queries --org pscale insights queries --org --sort rowsReadPerReturned pscale insights queries --org --sort p99Latency --period 1h --format json ``` ### The `queries samples` sub-command List recent executions for a specific query fingerprint. Use the fingerprint from `pscale insights queries`. `--keyspace` is required. **Usage:** ```bash theme={null} pscale insights queries samples --org --keyspace ``` **Available flags:** | **Flag** | **Description** | | - | - | | `--keyspace ` | Keyspace for the fingerprint (required; from `insights queries`) | | `--limit ` | Number of samples to return. Default: `25`. | | `--period ` | Time period to look back (for example `1h`, `1d`). | **Example:** ```bash theme={null} pscale insights queries samples --org --keyspace --format json ``` `queries show` takes an individual execution/sample `id` from `queries samples`. `queries samples` and `queries summary` take a query `fingerprint`. These identifiers are not interchangeable. ### The `queries show` sub-command Show one query execution using an ID returned by `pscale insights queries samples`. ```bash theme={null} pscale insights queries show --org ``` ### The `queries summary` sub-command Show aggregate statistics for a fingerprint from `pscale insights queries`. `--keyspace` is required (use the keyspace column from the queries list). **Available flags:** | **Flag** | **Description** | | - | - | | `--keyspace ` | Keyspace for the fingerprint (required; from `insights queries`) | | `--period ` | Named time period to summarize (for example `1h`, `12h`, or `1d`) | | `--from ` | Start of a custom time range as an ISO 8601 timestamp (use with `--to`) | | `--to ` | End of a custom time range as an ISO 8601 timestamp (use with `--from`) | `--period` cannot be combined with `--from` and `--to`. Both range flags must be set together. ```bash theme={null} pscale insights queries summary --org --keyspace pscale insights queries summary --org --keyspace --period 1h ``` ### The `queries traffic-budgets` sub-command List traffic budgets that affect a query fingerprint. Pass `--keyspace` from the queries list. The command is paginated; pass `--page` for the next page instead of walking every page automatically. **Available flags:** | **Flag** | **Description** | | - | - | | `--keyspace ` | Keyspace for the fingerprint (from `insights queries`) | | `--page ` | Page of results to fetch. | | `--per-page ` | Number of results per page. Default: `25`. | ```bash theme={null} pscale insights queries traffic-budgets --org --keyspace ``` ### The `errors` sub-command List failing query patterns with error messages. **Usage:** ```bash theme={null} pscale insights errors --org ``` **Available flags:** | **Flag** | **Description** | | - | - | | `--limit ` | Number of errors to return. Default: `15`. | | `--period ` | Time period to aggregate over (for example `1h`, `24h`). | **Example:** ```bash theme={null} pscale insights errors --org --format json ``` #### `errors show` List the individual executions that failed with an error fingerprint, including the users, keyspaces, and statements involved. Use the fingerprint column from `insights errors` (`error_fingerprint` in JSON), not the truncated id. **Available flags:** | **Flag** | **Description** | | - | - | | `--limit ` | Number of queries to return. Default: `25`. | | `--period ` | Time period to look back (for example `1h`, `1d`). | ```bash theme={null} pscale insights errors show --org pscale insights errors show --org --period 1h --limit 50 ``` ### The `anomalies` sub-command List detected resource anomalies for a branch. **Usage:** ```bash theme={null} pscale insights anomalies --org ``` #### `anomalies show` Show one anomaly from `insights anomalies`, along with the queries whose activity correlates with it. ```bash theme={null} pscale insights anomalies show --org ``` ### The `tags` sub-command List query tag keys from sqlcommenter / system tags on a branch. **Usage:** ```bash theme={null} pscale insights tags --org ``` **Available flags:** | **Flag** | **Description** | | - | - | | `--period ` | Time period to look back (for example `1h`, `1d`). | | `--fingerprint ` | Only tags seen on this query fingerprint. | | `--keyspace ` | Filter tags to a keyspace. | | `--limit ` | Number of top values to show in human output. Default: `3`. | **Example:** ```bash theme={null} pscale insights tags --org --format json ``` #### `tags show` Show a query tag key and its values. ```bash theme={null} pscale insights tags show --org ``` #### `tags summaries` List query statistics grouped by one or more tag keys. `--tags` is required and repeatable. ```bash theme={null} pscale insights tags summaries --org --tags username --tags app --format json ``` ### The `recommendations` sub-command List schema recommendations for a database: unused tables and indexes, duplicate indexes, bloated tables and indexes, missing indexes derived from production query patterns, and sequence overflow risks. Each recommendation includes ready-to-apply DDL in JSON output. This sub-command takes a database name only (not a branch). **Usage:** ```bash theme={null} pscale insights recommendations --org ``` **Example:** ```bash theme={null} pscale insights recommendations --org --format json ``` #### `recommendations show` Show a single schema recommendation, including the full ready-to-apply DDL. `` is the recommendation sequence number from `pscale insights recommendations`, the same value used by `recommendations dismiss`. ```bash theme={null} pscale insights recommendations show --org ``` #### `recommendations dismiss` Dismiss a schema recommendation. Use the recommendation number from `pscale insights recommendations`. Interactive confirmation is required unless `--force` is passed. ```bash theme={null} pscale insights recommendations dismiss --org --force --reason "not applicable" ``` If the database or branch is not found, or Query Insights is not enabled, the command returns an error explaining both possible causes. ## Related documentation > **[pscale inspect](/docs/cli/inspect)** > **[Query Insights (Postgres)](/docs/postgres/monitoring/query-insights)** > **[Query Insights (Vitess)](/docs/vitess/monitoring/query-insights)** > **[Query Insights (Neki)](/docs/neki/monitoring/query-insights)** > **[Anomalies (Neki)](/docs/neki/monitoring/anomalies)** > **[Schema recommendations (Postgres)](/docs/postgres/monitoring/schema-recommendations)** > **[Schema recommendations (Neki)](/docs/neki/monitoring/schema-recommendations)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: inspect Source: https://planetscale.com/docs/cli/inspect Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `inspect` command Run read-only diagnostic checks against a database branch. Each check is a single bounded query against the engine's statistics tables. Nothing is written to the database. `inspect` uses the same ephemeral credential model as [`pscale sql`](/docs/cli/sql): always read-only, always bounded. For server-side, traffic-aware analysis (slow queries, schema recommendations, anomalies), use [`pscale insights`](/docs/cli/insights) instead. The two surfaces cross-reference each other in human output and JSON `next_steps`, so you can move from live state to traffic-aware history (or the reverse) without leaving the CLI. **Usage:** ```bash theme={null} pscale inspect --org ``` Place **positional arguments first**, then flags. **`--org` is required.** ### Database engine support Checks adapt to the database engine. MySQL (Vitess) checks read `information_schema`, `mysql`, and `sys`. Postgres and Neki checks read `pg_catalog` and `pg_stat` views. Checks that do not apply to an engine explain what to use instead (often a matching `pscale insights` command). | | **Postgres and Neki** | **Vitess** | | :- | :- | :- | | Targeting flags | `--dbname`, `--role` | `--keyspace` | | Scope | One PostgreSQL database per run | One shard's MySQL instance per run | | Replica checks | `--replica` for Postgres. For Neki, use [`pscale shell --replica`](/docs/cli/shell). | `--replica` (via `--keyspace` tablet type) | On sharded Vitess databases, statistics reflect one shard's MySQL instance per run. Pass `--keyspace` to pick the keyspace, or target an exact shard with `--keyspace 'mykeyspace/-80'` (enumerate shards with `SHOW VITESS_SHARDS` via `pscale sql`). Databases can have hundreds of shards, so no check fans out across shards automatically. On Postgres and Neki, pass `--dbname` to target the PostgreSQL database your application uses (defaults to `postgres`). The reader role may lack `CONNECT` on non-default databases; use `--role admin` if connecting with `--dbname` fails. Neki runs these checks through its router; use Query Insights for traffic-aware, cross-shard analysis. ### Available checks | **Check** | **Postgres and Neki** | **Vitess** | **Description** | | :- | :- | :- | :- | | `table-sizes` | Yes | Yes | Tables by total size, largest first | | `index-sizes` | Yes | Yes | Indexes by size, largest first | | `unused-indexes` | Yes | Yes | Indexes with little or no use — removal candidates | | `redundant-indexes` | Via `insights recommendations` | Yes | Indexes made redundant by another index | | `invalid-indexes` | Yes | No | Invalid indexes left over from failed concurrent index builds | | `seq-scans` | Yes | Yes | Tables receiving full-table scans | | `long-running-queries` | Yes | Yes | Queries running longer than 5 minutes | | `locks` | Yes | Yes | Blocking locks and the sessions stuck behind them | | `outliers` | Yes\* | Via `insights queries` | Queries by cumulative execution time | | `calls` | Yes\* | Via `insights queries` | Most frequently called queries | | `bloat` | Yes | Yes | Wasted space: estimated bloat (PostgreSQL) or fragmentation (MySQL) | | `vacuum-stats` | Yes | Via `inspect bloat` | Autovacuum and autoanalyze health | | `replication-slots` | Yes | Via `branch vtctl move-tables list` | Replication slots: status, WAL retention, and lag | | `subscriptions` | Yes | Via `branch vtctl move-tables list` | Per-table logical replication progress on this subscriber | | `all` | Yes | Yes | Run every applicable check and print a combined report | \* `outliers` and `calls` require the `pg_stat_statements` extension on PostgreSQL. If it is not installed, the check is skipped and points you at the matching `pscale insights` command. ### Available flags | **Flag** | **Description** | | - | - | | `--org ` | Organization name **(required)** | | `--keyspace ` | Vitess keyspace to inspect, optionally with a shard and tablet type (for example `mykeyspace`, `mykeyspace/-80`, `mykeyspace/-80@replica`). Defaults to `@primary`. | | `--dbname ` | Postgres or Neki database name to inspect. Default: `postgres`. | | `--role ` | Access role for ephemeral credentials: `reader`, `writer`, `readwriter`, or `admin`. Default: `reader`. | | `--replica` | Run checks against a replica instead of the primary for PlanetScale Postgres or Vitess. For Neki, use [`pscale shell --replica`](/docs/cli/shell), which sets `__neki.target=REPLICA`. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. `csv` is supported for individual checks only, not `inspect all`. | | `-h`, `--help` | Help for `inspect` | `pscale inspect --replica` uses the same Postgres `|replica` username suffix as `pscale sql`. That is not how Neki routes replica traffic. Do not add `|replica` to a Neki username. ## Examples Run a single check: ```bash theme={null} pscale inspect table-sizes --org pscale inspect locks --org --format json pscale inspect unused-indexes --org --format csv ``` Run every applicable check: ```bash theme={null} pscale inspect all --org pscale inspect all --org --format json ``` Target a specific Vitess shard: ```bash theme={null} pscale inspect table-sizes --org --keyspace commerce/-80 ``` Inspect a non-default PostgreSQL database: ```bash theme={null} pscale inspect table-sizes --org --dbname myapp --role admin ``` ### JSON output Individual checks return a `CheckResult` object with `check`, `database`, `branch`, `columns`, `rows`, `row_count`, and optional `skipped` and `next_steps` fields. `inspect all` returns a `Report` with a `results` array (one entry per check) and top-level `next_steps` pointing at the complementary `pscale insights` commands. ## Related documentation > **[pscale insights](/docs/cli/insights)** > **[pscale sql](/docs/cli/sql)** > **[PlanetScale CLI environment setup](/docs/cli/planetscale-environment-setup)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: keyspace Source: https://planetscale.com/docs/cli/keyspace ## Getting started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `keyspace` command This command allows you to view your keyspaces, create internal or [external keyspaces](#create-an-external-keyspace), resize them, [configure disk scaling](#configure-disk-scaling), and view or update your Vitess VSchemas. This command is not available for Postgres databases. **Usage:** ```bash theme={null} pscale keyspace ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Product** | **Description** | | :- | :- | :- | :- | | `create ` | `--cluster-size `, `--additional-replicas `, `--shards ` | Vitess | Create a new keyspace within a database branch. | | `create-external ` | `--host `\*, `--source-database `\*, `--username `\*, `--password `\*, `--ssl-mode `\*, `--port `, `--cluster-size `, `--ssl-certificate-authority `, `--ssl-client-certificate `, `--ssl-client-key `, `--ssl-server-name `, `--min-tls-version `, `--tablet-cell `, `--skip-lint-errors`, `--dry-run`, `--wait` | Vitess | Create an external keyspace by connecting a production branch to an existing MySQL database. | | `delete ` | `--force` | Vitess | Delete a keyspace from a database branch. | | `list ` | | Vitess | List all keyspaces within a database branch. | | `parameters list ` | `--namespace ` | Vitess | List a keyspace's VTTablet and MySQL [parameters](/docs/vitess/cluster-configuration/parameters) with their current and default values. | | `parameters set ` | `--parameters `, `--reset ` | Vitess | Change VTTablet and MySQL parameters on a keyspace, or set them back to their defaults. | | `parameters changes list ` | `--page `, `--per-page ` | Vitess | List parameter changes to a keyspace and their state. | | `parameters changes show ` | | Vitess | Show a parameter change. | | `parameters changes cancel ` | | Vitess | Cancel a parameter change that hasn't started applying. | | `show ` | | Vitess | Show a specific keyspace within a database branch. | | `read-only-regions ` | | Vitess | List [read-only regions](/docs/vitess/scaling/read-only-regions) for a keyspace. | | `resize ` | `--cluster-size `, `--additional-replicas ` | Vitess | Resize a keyspace, including external keyspaces. | | `resize cancel ` | | Vitess | Cancel an ongoing keyspace resize. | | `resize status ` | | Vitess | Show the status of the keyspace's last resize. | | `settings ` | | Vitess | Show the settings for a keyspace. | | `update-settings ` | `-i`, `--interactive`, `--disk-scaling-strategy `, `--max-rollout `, `--max-storage `, `--replication-durability-constraints-strategy `, `--storage `, `--throttler-enabled`, `--throttler-threshold `, `--vreplication-batch-replication-events`, `--vreplication-enable-noblob-binlog-mode`, `--vreplication-optimize-inserts` | Vitess | Update the settings for a keyspace. | | `vschema show ` | | Vitess | Show the VSchema for a sharded keyspace. Empty on non-sharded keyspaces. | | `vschema update ` | `--vschema `\* | Vitess | Update a VSchema of a keyspace. | | `rollout-status ` | | Vitess | Check the status of a keyspace resize request. | > \* *Flag is required* #### Sub-command flag descriptions | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--additional-replicas ` | `` is the number of replicas to add to the keyspace. By default, production branches include 2 replicas. Do not use this flag for external keyspaces. | `create`, `resize` | | `--cluster-size ` | `` is the size of the database cluster. Optional for `create-external`; when omitted, PlanetScale chooses a size from the source storage. For `resize`, pass a size from `pscale size cluster list --external`. | `create`, `create-external`, `resize` | | `--dry-run` | Check connectivity and schema compatibility without creating the keyspace. | `create-external` | | `--host ` | Hostname of the external MySQL server. | `create-external` | | `--min-tls-version ` | Minimum TLS version for the external connection. | `create-external` | | `--password ` | Password for the external MySQL user. | `create-external` | | `--port ` | Port of the external MySQL server. Default is `3306`. | `create-external` | | `--skip-lint-errors` | Create even if datasource lint reports errors, when the organization allows it. | `create-external` | | `--source-database ` | Name of the database on the external MySQL server, not the PlanetScale database. | `create-external` | | `--ssl-certificate-authority ` | CA certificate chain, or a path to a PEM file. | `create-external` | | `--ssl-client-certificate ` | Client certificate, or a path to a PEM file. | `create-external` | | `--ssl-client-key ` | Client private key, or a path to a PEM file. | `create-external` | | `--ssl-mode ` | SSL verification mode: `disabled`, `preferred`, `required`, `verify_ca`, or `verify_identity`. | `create-external` | | `--ssl-server-name ` | SSL server name override. | `create-external` | | `--tablet-cell ` | Cell where the external tablet runs. | `create-external` | | `--username ` | Username to connect to the external MySQL server. | `create-external` | | `--wait` | Wait until the keyspace is ready. | `create-external` | | `--force` | Delete the keyspace without a confirmation prompt. Required in non-interactive or non-`human` output modes. | `delete` | | `--disk-scaling-strategy ` | How disk size is managed: `grow` automatically increases disk size as needed, up to `--max-storage`. `disable` turns off automatic disk growth. `shrink` resizes disks to `--storage` and then turns off automatic disk growth. | `update-settings` | | `-i, --interactive` | Run the command in interactive mode. | `update-settings` | | `--namespace ` | Only show parameters in this namespace: `vttablet` or `mysqld`. | `parameters list` | | `--parameters ` | Set a parameter, where `` is `vttablet` or `mysqld`, for example `vttablet.vreplication-parallel-insert-workers=4`. Repeatable. | `parameters set` | | `--reset ` | Set a parameter back to its default, for example `vttablet.vreplication-parallel-insert-workers`. Repeatable. | `parameters set` | | `--page `, `--per-page ` | Page through parameter changes. `--per-page` defaults to 25. | `parameters changes list` | | `--max-rollout ` | Maximum number of shards to update concurrently during a keyspace rollout. Accepts values from 1 to 32. | `update-settings` | | `--max-storage ` | Maximum size disks can automatically grow to, in bytes or with a unit such as `200GiB` or `1TiB`. Must be at least the size of the largest current disk and no more than the organization's storage limit (4 TB by default). | `update-settings` | | `--replication-durability-constraints-strategy ` | Replication strategy to use. Options: maximum, dynamic, minimum (default "maximum"). | `update-settings` | | `--throttler-enabled` | Pause schema migrations and VReplication workflows when replication lag is above the threshold. Pass `--throttler-enabled=false` to turn the throttler off. | `update-settings` | | `--throttler-threshold ` | Replication lag in seconds above which migrations and workflows are paused. Default is 5. | `update-settings` | | `--storage ` | Disk size to shrink to, in bytes or with a unit such as `200GiB`. Must be a multiple of 1 GiB and no more than the organization's storage limit. Only accepted when the disk scaling strategy is `shrink`, either passed with `--disk-scaling-strategy` or already set on the keyspace. Rejected if any disk is using more than 80% of the requested size. | `update-settings` | | `--shards ` | Number of shards in the keyspace (default 1). | `create` | | `--vreplication-batch-replication-events` | When enabled, sends fewer queries to MySQL to improve performance. | `update-settings` | | `--vreplication-enable-noblob-binlog-mode` | When enabled, omits changed BLOB and TEXT columns from replication events, which reduces binlog sizes. (default true) | `update-settings` | | `--vreplication-optimize-inserts` | When enabled, skips sending INSERT events for rows that have yet to be replicated. (default true) | `update-settings` | | `--vschema ` | `` is the path to the updated VSchema file. | `vschema update` | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for auth command | | `--org ` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### Create an external keyspace Attach an existing MySQL database as an external keyspace on a **production** branch. To [import its data into PlanetScale](/docs/vitess/imports/database-imports), create a [MoveTables workflow](/docs/cli/move-tables#import-from-an-external-keyspace) with the external keyspace as the source. `--source-database` is the remote MySQL database name. The `--username` user must have the [import user permissions](/docs/vitess/imports/import-tool-user-requirements), and your database must allow connections from [PlanetScale's IP addresses](/docs/vitess/imports/import-tool-migration-addresses). ```bash theme={null} pscale keyspace create-external \ --host db.example.com \ --source-database commerce \ --username import \ --password secret \ --ssl-mode required \ --wait ``` Check compatibility without creating the keyspace: ```bash theme={null} pscale keyspace create-external \ --host db.example.com \ --source-database commerce \ --username import \ --password secret \ --ssl-mode required \ --dry-run ``` `--dry-run` reports connection failures, server setting and user grant errors, and schema lint errors. Connection, server setting, and grant errors stop the keyspace from being created. Table-level schema errors don't: fix those tables or leave them out of the import with `--exclude-tables`. ### Resize an external keyspace Use the same `resize` command as an internal keyspace. Do not pass `--additional-replicas`. ```bash theme={null} pscale keyspace resize --cluster-size pscale keyspace resize status ``` You can also create and resize external keyspaces from the [Clusters](/docs/vitess/cluster-configuration) page in the dashboard. ### Delete an external keyspace Deleting an external keyspace disconnects PlanetScale from your external database and removes its stored credentials. If you imported from it, complete the MoveTables workflow first. ```bash theme={null} pscale keyspace delete ``` ### Update tablet throttler settings ```bash theme={null} pscale keyspace settings pscale keyspace update-settings --throttler-enabled=false pscale keyspace update-settings --throttler-threshold=10 ``` ### Update shard rollout concurrency Set the maximum number of shards updated concurrently during keyspace configuration rollouts: ```bash theme={null} pscale keyspace settings pscale keyspace update-settings --max-rollout=4 ``` `--max-rollout` accepts values from 1 to 32. If the API value is not set, `pscale keyspace settings` displays `not set`, and PlanetScale uses the default of 1. ### Configure disk scaling Show the keyspace's current disk scaling strategy and storage sizes: ```bash theme={null} pscale keyspace settings ``` Let disks grow automatically, up to a limit: ```bash theme={null} pscale keyspace update-settings \ --disk-scaling-strategy grow \ --max-storage 1TiB ``` Turn off automatic disk growth: ```bash theme={null} pscale keyspace update-settings --disk-scaling-strategy disable ``` Shrink disks to a smaller size. Automatic disk growth is turned off afterwards: ```bash theme={null} pscale keyspace update-settings \ --disk-scaling-strategy shrink \ --storage 200GiB ``` The shrink is rejected if any disk is using more than 80% of the requested size. Settings you don't pass keep their current values. See [Disk autoscaling](/docs/vitess/cluster-configuration/disk-autoscaling) for how each option works. ### Change keyspace parameters List a keyspace's VTTablet and MySQL [parameters](/docs/vitess/cluster-configuration/parameters), then change them. Changes for both namespaces are submitted together and roll out to the keyspace: ```bash theme={null} pscale keyspace parameters list --namespace vttablet pscale keyspace parameters set \ --parameters vttablet.vreplication-parallel-insert-workers=4 \ --parameters mysqld.wait_timeout=600 pscale keyspace parameters changes list ``` A change is `applying` while it rolls out and `completed` once every shard has it. A keyspace can have one unfinished change per namespace at a time. To set parameters back to their defaults, pass `--reset`: ```bash theme={null} pscale keyspace parameters set \ --reset vttablet.vreplication-parallel-insert-workers \ --reset mysqld.wait_timeout ``` ### List read-only regions for a keyspace ```bash theme={null} pscale keyspace read-only-regions ``` Output includes each region's slug, cluster size, and replica count. Use a region slug with `pscale password create ... --read-only-region` or `pscale database dump ... --read-only-region`. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: logs Source: https://planetscale.com/docs/cli/logs Postgres and Neki ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `logs` command Query recent logs for a Postgres or Neki branch. By default, the command returns up to 100 entries from the primary server over the last hour, newest first. **Usage:** ```bash theme={null} pscale logs --org ``` ### Available flags | **Flag** | **Description** | | - | - | | `--org ` | Organization name **(required)** | | `--query ` | Text or LogsQL expression to search for | | `--period ` | Time period to query: `15m`, `1h`, `3h`, `6h`, `12h`, `1d`, `7d`, or `8d`. Defaults to `1h`. | | `--from ` | Start of a custom ISO 8601 time range. Requires `--to`. | | `--to ` | End of a custom ISO 8601 time range. Requires `--from`. | | `--level ` | Include `DEBUG`, `INFO`, `WARNING`, or `ERROR` entries. Repeat the flag or pass a comma-separated list. | | `--server ` | Include the `primary` role or a named server. Repeat the flag or pass a comma-separated list. Defaults to `primary`. | | `--shard ` | Include a Neki shard. Repeat the flag or pass a comma-separated list. | | `--limit ` | Maximum number of entries to return. Defaults to `100`. | | `--page ` | Page to fetch. Defaults to `1`. | | `-h`, `--help` | Help for `logs` | `--period` cannot be combined with `--from` and `--to`. When you use a custom range, provide both timestamps. ### Global flags | **Flag** | **Description** | | - | - | | `-f`, `--format ` | Output format: `human` (default), `json`, or `csv` | | `--api-token`, `--service-token-id`, `--service-token` | Authentication | | `--api-url ` | API base URL | | `--config ` | Config file | | `--debug` | Enable debug mode | | `--no-color` | Disable color output | ## Examples Show recent primary logs: ```bash theme={null} pscale logs mydatabase main --org myorganization ``` Show errors from the last six hours as JSON: ```bash theme={null} pscale logs mydatabase main --org myorganization \ --period 6h --level ERROR --format json ``` Filter a Neki branch to selected shards: ```bash theme={null} pscale logs mydatabase main --org myorganization \ --shard shard-a --shard shard-b ``` Query a custom time range: ```bash theme={null} pscale logs mydatabase main --org myorganization \ --from 2026-08-20T16:00:00Z \ --to 2026-08-20T18:00:00Z ``` Log output can contain query text and application data. Review it before sharing or attaching it to a support request. ## Related documentation > **[Neki logs](/docs/neki/monitoring/logs)** > **[Postgres logs](/docs/postgres/monitoring/logs)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: maintenance Source: https://planetscale.com/docs/cli/maintenance ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `maintenance` command List and show [maintenance schedules](/docs/plans/managed/maintenance-schedules) for a Vitess database on an Enterprise plan. Maintenance schedules define when PlanetScale can perform planned maintenance (for example Vitess or MySQL version updates). This command is read-only. **`--org` is required.** **Usage:** ```bash theme={null} pscale maintenance ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Description** | **Product** | | :- | :- | :- | :- | | `list ` | | List maintenance schedules for a database | Vitess | | `show ` | | Show a maintenance schedule | Vitess | | `windows ` | | List maintenance windows for a schedule | Vitess | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for the `maintenance` command | | `--org ` | The organization for the current user (required) | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### List schedules ```bash theme={null} pscale maintenance list --org ``` ### Show a schedule ```bash theme={null} pscale maintenance show --org ``` ### List windows for a schedule ```bash theme={null} pscale maintenance windows --org ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: metrics Source: https://planetscale.com/docs/cli/metrics Vitess and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `metrics` command Query historical and current branch metrics from PlanetScale's metrics service. Human output summarizes time series with latest, min, average, max, and a sparkline trend. JSON preserves the API response for automation. CSV emits one row per sample or current value for scripts and analysis tools. For server-side query analysis (slow queries, anomalies, schema recommendations), use [`pscale insights`](/docs/cli/insights). For live connection-level diagnostics, use [`pscale inspect`](/docs/cli/inspect). To ask an AI assistant for the same metrics, use [`planetscale_get_branch_metrics`](/docs/mcp-server). > [!NOTE] > > **Requires `pscale` 0.321.0 or later.** **Usage:** ```bash theme={null} pscale metrics --org ``` Place **positional arguments first**, then flags. **`--org` is required.** ### Available sub-commands | **Sub-command** | **Product** | **Description** | | :- | :- | :- | | `show` | Vitess, Postgres | Historical metric time series over a named period or custom range | | `instant` | Vitess, Postgres | Current metric values | | `report` | Vitess, Postgres | Engine-aware grouped performance report (MySQL and PostgreSQL use different sections) | | `queries` | Vitess, Postgres | Historical metrics for selected SQL query patterns | | `tables` | Vitess, Postgres | Table storage metrics | | `keyspace-tables` | Vitess | Table storage metrics grouped by keyspace | | `tablets` | Vitess | Historical tablet metric series | | `tablets instant` | Vitess | Current tablet metric values | | `tags` | Vitess, Postgres | Historical metrics grouped by query tag sets | ### Available flags | **Flag** | **Description** | | - | - | | `--org ` | Organization name **(required)** | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `-h`, `--help` | Help for `metrics` | ## Examples ### The `show` sub-command Query historical metric series for a branch. **`--metric` is required** and may be repeated or comma-separated. **Usage:** ```bash theme={null} pscale metrics show --org --metric ``` **Available flags:** | **Flag** | **Description** | | - | - | | `--metric ` | Metric to query (repeat or comma-separate) **(required)** | | `--period ` | Named time period (for example `1h`, `12h`, or `1d`; defaults to `12h`) | | `--from ` | Start of a custom time range as an ISO 8601 timestamp (use with `--to`) | | `--to ` | End of a custom time range as an ISO 8601 timestamp (use with `--from`) | | `--steps ` | Requested number of data points | | `--tablet-type ` | Filter by tablet type (Vitess) | | `--keyspace ` | Filter by keyspace (Vitess) | | `--shard ` | Filter by shard | | `--role ` | Filter by Postgres role | | `--container ` | Filter by container | | `--pod ` | Filter by one pod | | `--pods ` | Filter by pods (repeat or comma-separate) | | `--query-id ` | Filter by query pattern ID (repeat or comma-separate) | | `--fingerprint ` | Filter by query fingerprint | | `--budget-id ` | Filter by traffic budget ID | | `--rule-id ` | Filter by traffic rule ID | | `-q`, `--search ` | Filter by search terms | `--period` cannot be combined with `--from` and `--to`. Both range flags must be set together. **Examples:** ```bash theme={null} pscale metrics show --org --metric queries --metric latency_p99 --period 1h pscale metrics show --org --metric queries --period 1h --format csv pscale metrics show --org --metric queries --period 1h --format json ``` ### The `instant` sub-command Show current metric values. **`--metric` is required** and may be repeated or comma-separated. **Usage:** ```bash theme={null} pscale metrics instant --org --metric ``` **Available flags:** | **Flag** | **Description** | | - | - | | `--metric ` | Metric to query (repeat or comma-separate) **(required)** | | `--role ` | Filter by Postgres role | | `--shard ` | Filter by shard | | `--container ` | Filter by container | | `--pod ` | Filter by pod | **Examples:** ```bash theme={null} pscale metrics instant --org --metric planetscale_volume_usage_percentage pscale metrics instant --org --metric planetscale_volume_usage_percentage --format json ``` ### The `report` sub-command Produce a curated performance report for a branch. The database engine is detected automatically. MySQL (Vitess) and PostgreSQL reports include different metric sections, including current-value sections where applicable. Section headings are bold in human output and plain text with `--no-color`. **Usage:** ```bash theme={null} pscale metrics report --org ``` **Available flags:** | **Flag** | **Description** | | - | - | | `--period ` | Named report period: `15m`, `1h`, `3h`, `6h`, `12h`, `1d`, `2d`, `7d`, or `8d` (default `1d`) | | `--from ` | Start of a custom time range as an ISO 8601 timestamp (use with `--to`) | | `--to ` | End of a custom time range as an ISO 8601 timestamp (use with `--from`) | | `--steps ` | Requested number of historical data points | **Examples:** ```bash theme={null} pscale metrics report --org --period 1d pscale metrics report --org --period 7d --no-color pscale metrics report --org --period 1d --format json ``` #### Report sections **Vitess (MySQL)** reports include: * Workload, errors, and traffic control * Latency and execution time * Query efficiency and fan-out * Buffer and block activity * Network traffic * VTGate utilization by availability zone * Storage by table **PostgreSQL** reports include the sections above (with Postgres-specific network and connection metrics) plus: * Edge network traffic * Connections and connection pooling * CPU, memory utilization, and IOPS * PostgreSQL memory composition * Storage utilization * Transactions, replication, and WAL * Pod health * Current connection capacity (instant values) * Current storage capacity (instant values) * Backup activity (instant values) ### The `queries` sub-command Show metrics for selected SQL query patterns. **`--metric` is required.** Select queries with `--fingerprint` and `--keyspace` together, or with `--query-id` as `-`. The short `id` from `insights queries` is not a query pattern ID. `--query-id` cannot be combined with `--fingerprint` or `--keyspace`. Filters that match nothing return zero-filled series rather than an empty response, so check the point values, not the series count. ```bash theme={null} pscale metrics queries --org --metric latency_p99 --fingerprint --keyspace --period 1h ``` ### The `tables` and `keyspace-tables` sub-commands Show storage metrics for tables. JSON preserves the untyped storage-metrics API response. ```bash theme={null} pscale metrics tables --org pscale metrics keyspace-tables --org ``` ### The `tablets` sub-command Show tablet metric series. **`--metric` is required.** `--workflow` applies only to `--metric vreplication_lag`. The CLI does not check that the workflow exists locally. ```bash theme={null} pscale metrics tablets --org --metric replication_lag --period 1h pscale metrics tablets instant --org --metric replication_lag ``` ### The `tags` sub-command Show metrics grouped by query tags. **`--metric` is required**, and at least one `--tag-set` is required. Repeat `--tag-set` for independent series; comma-separate keys inside one set (`Busername=alice,Senv=production`). Tag keys need the Insights type prefix from `insights tags`. `--budget-id` and `--rule-id` apply only to the `traffic_control_warnings` and `traffic_control_throttled` metrics. ```bash theme={null} pscale metrics tags --org --metric queries --tag-set Busername=alice --tag-set Busername=bob --period 1h ``` ### JSON and CSV output Use `--format json` for automation. **Historical (`show`):** `start_date`, `end_date`, `interval`, and `series`. Each series contains `metric`, `label`, `labels`, and `[Unix timestamp, value]` points. **Instant (`instant`):** current values grouped by metric and dimensions. **Report (`report`):** composite JSON with `engine`, `period` (or `from`/`to`), and `sections` containing the underlying series or instant results. CSV includes a `section` column on each row. ## Metric names Pass metric identifiers to `--metric`. Common examples include `queries`, `latency_p99`, `connections`, and `planetscale_volume_usage_percentage`. For the full list accepted by the API, see [Get time-series metrics](/docs/api/reference/get_branch_metrics) and [Get instant branch metrics](/docs/api/reference/get_instant_branch_metrics). Prometheus metric names for scraping are documented separately in the [Vitess Prometheus metrics reference](/docs/vitess/integrations/prometheus-metrics) and [Postgres Prometheus metrics](/docs/postgres/monitoring/prometheus-metrics-postgres). ## Related documentation > **[pscale insights](/docs/cli/insights)** > **[pscale inspect](/docs/cli/inspect)** > **[Cluster metrics (Postgres)](/docs/postgres/monitoring/metrics)** > **[Metrics API reference](/docs/api/reference/get_branch_metrics)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: move-tables Source: https://planetscale.com/docs/cli/move-tables ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `move-tables` command Run [Vitess MoveTables](https://vitess.io/docs/user-guides/migration/move-tables/) workflows on a Vitess database branch: copy tables from one keyspace to another, check copy and traffic state, switch reads then writes, and complete or cancel the workflow. Use it to [shard tables](/docs/vitess/sharding/sharding-quickstart), [change the number of shards](/docs/vitess/sharding/sharding-a-sharded-keyspace), and [import an external MySQL database](/docs/vitess/imports/database-imports) from an external keyspace. `move-tables` prints JSON. The output includes a `next_steps` field with a suggested next command. Use it to see what to run after `list` or `status`. Pass `--format json` when you script `move-tables`, so progress messages don't mix with the JSON. **Usage:** ```bash theme={null} pscale branch vtctl move-tables ``` `pscale branch vtctld move-tables` also works. > [!NOTE] > > You must be an [Organization Administrator](/docs/security/access-control#organization-administrator) or a [Database Administrator](/docs/security/access-control#database-administrator) of the database to create and change workflows. [Service tokens](/docs/api/service-tokens) need `read_workflow` to list and view workflows, `write_workflow` to create, start, stop, switch traffic, and complete them, and `delete_workflow` to cancel them. ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Description** | | :- | :- | :- | | `list ` | `--target-keyspace ` | List MoveTables workflows on the branch | | `create ` | `--workflow `\*, `--source-keyspace `\*, `--target-keyspace `\*, `--tables `†, `--all-tables`†, `--exclude-tables `, `--auto-start`, `--stop-after-copy`, `--defer-secondary-keys`, `--on-ddl `, `--sharded-auto-increment-handling `, `--global-keyspace `, `--source-time-zone `, `--tenant-id `, `--cells `, `--tablet-types `, `--atomic-copy` | Create a MoveTables workflow | | `show ` | `--workflow `\*, `--target-keyspace `\* | Show a workflow | | `status ` | `--workflow `\*, `--target-keyspace `\* | Show copy progress and traffic state | | `start ` | `--workflow `\*, `--target-keyspace `\* | Start or resume a stopped workflow | | `stop ` | `--workflow `\*, `--target-keyspace `\* | Stop a running workflow | | `switch-traffic ` | `--workflow `\*, `--target-keyspace `\*, `--tablet-types `\*, `--dry-run`, `--initialize-target-sequences`, `--max-replication-lag-allowed ` | Switch traffic to the target keyspace | | `reverse-traffic ` | `--workflow `\*, `--target-keyspace `\*, `--tablet-types `, `--dry-run`, `--max-replication-lag-allowed ` | Switch traffic back to the source keyspace | | `complete ` | `--workflow `\*, `--target-keyspace `\*, `--keep-data`\*, `--keep-routing-rules`\*, `--rename-tables`, `--dry-run` | Finish the workflow and clean up | | `cancel ` | `--workflow `\*, `--target-keyspace `\*, `--keep-data`\*, `--keep-routing-rules`\* | Cancel a workflow that is still in progress | > \* *Flag is required* > > † *Pass either `--tables` or `--all-tables`* `list` also accepts the alias `ls`. #### Sub-command flag descriptions | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--workflow ` | Workflow name. Letters, numbers, `_`, and `-` only | `create`, `show`, `status`, `start`, `stop`, `switch-traffic`, `reverse-traffic`, `complete`, `cancel` | | `--source-keyspace ` | Keyspace to copy tables from. For an import, this is the [external keyspace](/docs/cli/keyspace#create-an-external-keyspace) | `create` | | `--target-keyspace ` | Keyspace to copy tables to. On `list`, filters to one keyspace. `list` shows workflows from every keyspace when you omit it | `list`, `create`, `show`, `status`, `start`, `stop`, `switch-traffic`, `reverse-traffic`, `complete`, `cancel` | | `--tables ` | Tables to move (comma-separated). Cannot be combined with `--all-tables` or `--exclude-tables` | `create` | | `--all-tables` | Move all tables from the source keyspace | `create` | | `--exclude-tables ` | Tables to skip when moving all tables (comma-separated) | `create` | | `--auto-start` | Start the workflow after creation (default `true`). Pass `--auto-start=false` and run `start` later to start it yourself | `create` | | `--stop-after-copy` | Stop the workflow after the copy phase | `create` | | `--defer-secondary-keys` | Create secondary indexes after the copy finishes instead of during it. Only sent when you set it explicitly; pass `--defer-secondary-keys=false` to keep indexes in place during the copy. See the warning below. | `create` | | `--on-ddl ` | What to do when a schema change runs on the source during the workflow: `STOP` (default), `IGNORE`, `EXEC`, `EXEC_IGNORE` | `create` | | `--sharded-auto-increment-handling ` | `AUTO_INCREMENT` handling for sharded targets: `REMOVE` (default) removes it, `REPLACE` replaces it with Vitess sequences, `LEAVE` keeps it | `create` | | `--global-keyspace ` | Unsharded keyspace in which to create sequence tables when `--sharded-auto-increment-handling` is `REPLACE` | `create` | | `--source-time-zone ` | Convert `DATETIME` values from this timezone to UTC | `create` | | `--tenant-id ` | Tenant ID for multi-tenant MoveTables | `create` | | `--cells ` | Cells to restrict the workflow to (comma-separated) | `create` | | `--tablet-types ` | Tablet types for the workflow or traffic switch (comma-separated). Use `REPLICA,RDONLY` for reads and `PRIMARY` for writes | `create`, `switch-traffic`, `reverse-traffic` | | `--atomic-copy` | Copy all tables in a single consistent copy phase. Use it when the source has foreign key constraints | `create` | | `--dry-run` | Show what would happen without applying it | `switch-traffic`, `reverse-traffic`, `complete` | | `--initialize-target-sequences` | Initialize target sequence tables when switching primary traffic, so new IDs start above the existing ones | `switch-traffic` | | `--max-replication-lag-allowed ` | Maximum replication lag allowed before switching | `switch-traffic`, `reverse-traffic` | | `--keep-data` | On `complete`: keep the source tables (`true`) or drop them (`false`). Must be `true` when the source is an external keyspace. On `cancel`: keep the data copied into the target keyspace (`true`) or delete it (`false`) | `complete`, `cancel` | | `--keep-routing-rules` | Keep routing rules (`true`) or remove them (`false`) | `complete`, `cancel` | | `--rename-tables` | Rename source tables instead of dropping them. Not supported when the source is an external keyspace | `complete` | > [!WARNING] > > `--defer-secondary-keys` copies target tables without secondary indexes. After each table finishes copying, Vitess re-adds them with `ALTER TABLE ... ADD KEY`. Copies are often much faster. > > On large tables with many indexes, the post-copy `ALTER` can: > > * take a long time > * add significant disk and I/O load > * require enough binlog retention on the source > * delay the workflow from reaching the replicating, cutover-ready state > > Cancelling or deleting the workflow can hang while that `ALTER` is running. Don't use it with foreign key constraints. > > To keep indexes in place during the copy, pass `--defer-secondary-keys=false`: the copy is slower, but there's no long rebuild at the end. Write boolean flags with an `=`, such as `--keep-data=false`. A space-separated value such as `--keep-data false` is read as `--keep-data=true`. ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `move-tables` | | `--org ` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Lifecycle A typical MoveTables run looks like this: 1. **Create** the workflow and wait until copy finishes and streams are `Running`. 2. **Verify** with `pscale branch vtctl vdiff create`, then `vdiff show` until it completes with no mismatches. You can skip VDiff and switch replica traffic directly. 3. **Switch replica traffic** with `--tablet-types REPLICA,RDONLY`. 4. **Switch primary traffic** with `--tablet-types PRIMARY`. 5. **Preview cleanup** with `complete --dry-run`, then complete after you review the result. Use `status` between steps. Each stream's state is `Copying`, `Running`, `Lagging`, `Stopped`, or `Error`. `traffic_state` values are: * `Reads Not Switched. Writes Not Switched` * `All Reads Switched. Writes Not Switched` * `Reads Not Switched. Writes Switched` * `All Reads Switched. Writes Switched` Use `stop` to pause a workflow and `start` to resume it, for example after it stops on a schema change. Until you complete a workflow, you can switch traffic back with `reverse-traffic` or `cancel` it. You can also follow each workflow on the **Workflows** page in the dashboard. The page is read-only. ## Examples List MoveTables workflows and get the next command to run: ```bash theme={null} pscale branch vtctl move-tables list mydb main --org acme --format json ``` Create a workflow: ```bash theme={null} pscale branch vtctl move-tables create mydb main \ --org acme \ --workflow commerce2customer \ --source-keyspace commerce \ --target-keyspace customer \ --tables customers,orders \ --format json ``` Check copy and traffic state: ```bash theme={null} pscale branch vtctl move-tables status mydb main \ --org acme \ --workflow commerce2customer \ --target-keyspace customer \ --format json ``` Example `status` JSON after replica traffic has switched: ```json theme={null} { "traffic_state": "All Reads Switched. Writes Not Switched", "next_steps": [ { "command": "pscale branch vtctld move-tables switch-traffic mydb main --org acme --workflow commerce2customer --target-keyspace customer --tablet-types PRIMARY --format json", "reason": "Switch primary traffic after validating replica traffic" } ] } ``` Switch replica traffic, then primary traffic: ```bash theme={null} pscale branch vtctl move-tables switch-traffic mydb main \ --org acme \ --workflow commerce2customer \ --target-keyspace customer \ --tablet-types REPLICA,RDONLY \ --format json pscale branch vtctl move-tables switch-traffic mydb main \ --org acme \ --workflow commerce2customer \ --target-keyspace customer \ --tablet-types PRIMARY \ --format json ``` Verify with VDiff before switching traffic: ```bash theme={null} pscale branch vtctl vdiff create mydb main \ --org acme \ --workflow commerce2customer \ --target-keyspace customer \ --format json pscale branch vtctl vdiff show mydb main \ --org acme \ --workflow commerce2customer \ --target-keyspace customer \ --uuid \ --format json ``` Preview cleanup, then complete: ```bash theme={null} pscale branch vtctl move-tables complete mydb main \ --org acme \ --workflow commerce2customer \ --target-keyspace customer \ --keep-data=false \ --keep-routing-rules=false \ --dry-run \ --format json ``` ### Import from an external keyspace After you [create an external keyspace](/docs/cli/keyspace#create-an-external-keyspace), move its tables into your PlanetScale keyspace: ```bash theme={null} pscale branch vtctl move-tables create mydb main \ --org acme \ --workflow import_commerce \ --source-keyspace commerce_source \ --target-keyspace mydb \ --all-tables \ --defer-secondary-keys \ --format json ``` When you complete an import, `--keep-data` must be `true`, so the tables in your source database are never dropped: ```bash theme={null} pscale branch vtctl move-tables complete mydb main \ --org acme \ --workflow import_commerce \ --target-keyspace mydb \ --keep-data=true \ --keep-routing-rules=false \ --format json ``` See [Database imports](/docs/vitess/imports/database-imports) for the full import walkthrough. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: org Source: https://planetscale.com/docs/cli/org Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `org` command This command allows you to list, show, switch, and update [organizations](/docs/security/access-control#organization-member), and to manage organization members, teams, and [SSO](/docs/security/sso). **Usage:** ```bash theme={null} pscale org ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Product** | **Description** | | :- | :- | :- | :- | | `list` | | Postgres, Vitess, Neki | List all currently active organizations with timestamps | | `member list` | `--query `, `--page `, `--per-page ` | Postgres, Vitess, Neki | List the members of an organization | | `member show ` | | Postgres, Vitess, Neki | Show an organization member | | `member update ` | `--role `\* | Postgres, Vitess, Neki | Change another member's organization role | | `member remove ` | `--force`, `--delete-passwords`, `--delete-service-tokens` | Postgres, Vitess, Neki | Remove a member from an organization | | `show` | | Postgres, Vitess, Neki | Display the currently active organization | | `sso show` | | Postgres, Vitess, Neki | Show organization SSO status | | `sso enable` | | Postgres, Vitess, Neki | Enable organization SSO | | `sso disable` | `--force` | Postgres, Vitess, Neki | Disable organization SSO | | `sso configure` | | Postgres, Vitess, Neki | Open the identity provider setup portal | | `sso directory enable` | | Postgres, Vitess, Neki | Open the directory sync setup portal | | `sso directory disable` | `--force` | Postgres, Vitess, Neki | Disable directory sync | | `sso domain list` | | Postgres, Vitess, Neki | List SSO email domains | | `sso domain show ` | | Postgres, Vitess, Neki | Show an SSO email domain | | `sso domain verify` | `--wait`, `--wait-timeout ` | Postgres, Vitess, Neki | Open the domain verification portal | | `sso domain delete ` | `--force` | Postgres, Vitess, Neki | Delete an SSO email domain | | `switch ` | `--save-config ` | Postgres, Vitess, Neki | Switch the currently active organization | | `team create` | `--name `\*, `--description ` | Postgres, Vitess, Neki | Create a team | | `team delete ` | `--force` | Postgres, Vitess, Neki | Delete a team | | `team list` | `--query `, `--page `, `--per-page ` | Postgres, Vitess, Neki | List teams in an organization | | `team member add ` | | Postgres, Vitess, Neki | Add an organization member to a team | | `team member list ` | `--page `, `--per-page ` | Postgres, Vitess, Neki | List members of a team | | `team member remove ` | `--force`, `--delete-passwords` | Postgres, Vitess, Neki | Remove a member from a team | | `team show ` | | Postgres, Vitess, Neki | Show a team | | `team update ` | `--name `, `--description ` | Postgres, Vitess, Neki | Update a team's name or description | | `update` | `--billing-email `, `--idp-managed-roles`, `--idp-sso-managed-roles`, `--spend-alert`, `--spend-alert-amount ` | Postgres, Vitess, Neki | Update organization settings | > \* *Flag is required* ### Service token automation: `org` Legend: ✅ supported · 🚫 unavailable with a service token · 👤 interactive login only. | Sub-command | Env-var auth | `--service-token` flag | Requires `--org` | Postgres / Vitess | `--format json` | API equivalent | | :- | :-: | :-: | :-: | :- | :-: | :- | | `org list` | ✅ | ✅ | No | Both | ✅ | `pscale api organizations --format json` | | `org show` | 🚫 | 🚫 | No | Both | ✅ | N/A | | `org switch ` | ✅ | ✅ | No | Both | ✅ | N/A | | `org update` | ✅ | ✅ | Yes | Both | ✅ | `pscale api --method PATCH organizations/ --format json` | | `org sso show` | ✅ | ✅ | Yes | Both | ✅ | `pscale api organizations//sso --format json` | | `org sso enable` | ✅ | ✅ | Yes | Both | ✅ | `pscale api --method POST organizations//sso --format json` | | `org sso configure` | ✅ | ✅ | Yes | Both | ✅ | `pscale api --method POST organizations//sso/configure --format json` | | `org sso domain list` | ✅ | ✅ | Yes | Both | ✅ | `pscale api organizations//sso/domains --format json` | | `org sso domain show ` | ✅ | ✅ | Yes | Both | ✅ | `pscale api organizations//sso/domains/ --format json` | | `org sso domain verify` | ✅ | ✅ | Yes | Both | ✅ | `pscale api --method POST organizations//sso/domains --format json` | | `org sso domain delete ` | ✅ | ✅ | Yes | Both | ✅ | `pscale api --method DELETE organizations//sso/domains/ --format json` | | `org sso directory enable` | ✅ | ✅ | Yes | Both | ✅ | `pscale api --method POST organizations//sso/directory --format json` | | `org sso directory disable` | ✅ | ✅ | Yes | Both | ✅ | `pscale api --method DELETE organizations//sso/directory --format json` | | `org sso disable` | ✅ | ✅ | Yes | Both | ✅ | `pscale api --method DELETE organizations//sso --format json` | `org show` fails with `not authenticated yet. Please run 'pscale auth login'` under a valid service token. That is the **no-current-org** state. Do **not** run `pscale auth login`. Use `org list --format json`, then set `PLANETSCALE_ORG` or pass `--org` on resource commands. SSO commands require `manage_sso` on the service token. `org update` requires `write_organization`. Setup and commands to avoid: [CLI overview](/docs/cli) · [Service tokens](/docs/api/service-tokens#service-token-automation) #### Sub-command flag descriptions Some of the sub-commands have additional flags unique to the sub-command. This section covers what each of those does. See the above table for which context. | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--save-config ` | Path to store the organization. By default, the configuration is automatically deduced based on where `pscale` is executed. | `switch` | | `--query ` | Filter members by a name or email prefix, or filter teams by name. | `member list`, `team list` | | `--page ` | Page of results to fetch. | `member list`, `team list`, `team member list` | | `--per-page ` | Number of results per page. Default is `100`. | `member list`, `team list`, `team member list` | | `--role ` | Organization role to assign: `admin`, `member`, or `analyst`. See [access control](/docs/security/access-control). | `member update` | | `--force` | Skip confirmation for destructive actions. | `member remove`, `team delete`, `team member remove`, `sso disable`, `sso directory disable`, `sso domain delete` | | `--delete-passwords` | Also delete passwords created by the member, or passwords created through the team. Cannot be used when removing yourself from the organization. | `member remove`, `team member remove` | | `--delete-service-tokens` | Also delete service tokens created by the member. Cannot be used when removing yourself. | `member remove` | | `--name ` | Team name. Required on create. | `team create`, `team update` | | `--description ` | Team description. | `team create`, `team update` | | `--billing-email ` | Billing email for the organization. | `update` | | `--idp-managed-roles` | Whether the identity provider manages organization roles through directory sync. Boolean; set explicitly with `=true` or `=false`. Enabling this turns off `--idp-sso-managed-roles`. | `update` | | `--idp-sso-managed-roles` | Whether the identity provider manages organization roles through SSO. Boolean; set explicitly with `=true` or `=false`. Enabling this turns off `--idp-managed-roles`. | `update` | | `--spend-alert` | Enable or disable billing spend alerts. Boolean; set explicitly with `=true` or `=false`. | `update` | | `--spend-alert-amount ` | Monthly spend amount that triggers spend alerts. Required when enabling spend alerts. | `update` | | `--wait` | After opening the domain verification portal, wait until a domain is verified or failed. | `sso domain verify` | | `--wait-timeout ` | Maximum time to wait with `--wait`. Default is `10m`. | `sso domain verify` | ### Available flags | **Flag** | **Description** | | - | - | | `-h`, `--help` | View help for `org` command | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. Local override inside a Git repository is `$CWD/.pscale.yml` in the project's root. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `org` command with `switch` sub-command **Command:** ```bash theme={null} pscale org switch ``` **Output:** Successfully switched to organization `` (using file: `/Users/name/.config/planetscale/pscale.yml`) ### The `org` command with `member` sub-commands **Command:** ```bash theme={null} pscale org member list --org pscale org member show --org pscale org member update --role member --org pscale org member remove --org ``` Members can be identified by email or by the `USER_ID` shown by `org member list`, which is paginated at 100 members per page. Changing another member's role or removing them requires the organization admin role, and you cannot change your own role or remove the last admin. You can remove yourself without being an admin. ### Update organization settings `--org` is required. Only flags you pass are sent to the API. ```bash theme={null} pscale org update --org --billing-email billing@example.com pscale org update --org --idp-managed-roles=false pscale org update --org --idp-sso-managed-roles=true pscale org update --org --spend-alert=true --spend-alert-amount 2500 pscale org update --org --spend-alert=false ``` `--spend-alert-amount` cannot be combined with `--spend-alert=false`. You cannot enable both `--idp-managed-roles` and `--idp-sso-managed-roles`. Enabling one turns the other off. ### Manage organization SSO `--org` is required. Organization administrators can enable SSO from the CLI. Service tokens need `manage_sso` access. See [Single sign-on](/docs/security/sso) for what enabling SSO does to membership. `sso domain verify` returns a portal URL and tries to open it. Domain records appear after you finish verification in the portal. With `--wait`, the command prints the URL, then waits until a new domain is verified or failed. ```bash theme={null} pscale org sso show --org pscale org sso enable --org pscale org sso domain verify --org pscale org sso domain verify --org --wait pscale org sso domain list --org pscale org sso domain show --org pscale org sso configure --org pscale org sso directory enable --org pscale org sso disable --org ``` In JSON mode, `disable`, `directory disable`, and `domain delete` require `--force`. ### Manage organization teams Teams can be identified by ID, name, or slug. `--org` is required. List and member-list commands are paginated; pass `--page` for the next page instead of walking every page automatically. ```bash theme={null} pscale org team list --org pscale org team show --org pscale org team create --name --description --org pscale org team update --name --org pscale org team delete --org pscale org team member list --org pscale org team member add --org pscale org team member remove --org ``` SSO or directory-managed teams cannot be updated, deleted, or have members added or removed through the API. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: password Source: https://planetscale.com/docs/cli/password ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). ## The `password` command Manage database passwords for a Vitess database branch. This command is only supported for Vitess databases. To manage a role for Postgres, see the [role command](/docs/cli/role). **Usage:** ```bash theme={null} pscale password ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Product** | **Description** | | :- | :- | :- | :- | | `create ` | `--ttl`, `--role`, `--replica`, `--read-only-region` | Vitess | Create new credentials to access a branch's data | | `delete ` | `--force` | Vitess | Delete the specified branch credentials | | `list ` | `--name`, `--status`, `--page`, `--per-page`, `--web` | Vitess | List all credentials of a database | | `renew ` | | Vitess | Renew the specified branch credentials | | `show []` | `--name` | Vitess | Show the metadata of one set of branch credentials | | `update []` | `--name`, `--new-name`, `--cidrs` | Vitess | Rename credentials or replace their IP restrictions | ### Service token automation: `password` Legend: ✅ supported · Vitess only. All sub-commands require `--org` or `PLANETSCALE_ORG`. | Sub-command | Env-var auth | `--service-token` flag | Requires `--org` | Postgres / Vitess | `--format json` | API equivalent | | :- | :-: | :-: | :-: | :- | :-: | :- | | `list ` | ✅ | ✅ | ✅ | Vitess | ✅ | `pscale api organizations//databases//branches//passwords --format json` | | `create` | ✅ | ✅ | ✅ | Vitess | ✅ | `POST pscale api organizations//databases//branches//passwords --format json` | | `delete` | ✅ | ✅ | ✅ | Vitess | ✅ | `DELETE pscale api organizations//databases//branches//passwords/ --format json` | ```bash theme={null} export PLANETSCALE_ORG="" pscale password list --format json ``` Setup and commands to avoid: [CLI overview](/docs/cli) · [Service tokens](/docs/api/service-tokens#service-token-automation) The value `` represents the ID number of the set of credentials. To find all available credentials and their IDs, run `pscale list `. #### Sub-command flag descriptions Some of the sub-commands have additional flags unique to the sub-command. This section covers what each of those does. See the above table for which context. | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--ttl` | TTL defines the time to live for the password. Durations such as `30m` or `24h`, or bare seconds like `3600`, are accepted. Default is `0` (never expires). | `create` | | `--role ` | Add a [role to a password](/docs/vitess/security/password-roles). Defaults to `reader` for `--replica` and `--read-only-region`, otherwise `admin`. | `create` | | `--replica` | Route reads to the branch's primary-region replicas and all [read-only regions](/docs/vitess/scaling/read-only-regions) (nearest replica). Cannot be combined with `--read-only-region`. | `create` | | `--read-only-region ` | Scope the password to a Vitess [read-only region](/docs/vitess/scaling/read-only-regions). Accepts a region slug, display name, or id. List regions with `pscale keyspace read-only-regions`. Cannot be combined with `--replica`. | `create` | | `--force` | Delete a password without confirmation. | `delete` | | `--name ` | Select the password by name instead of ID (`show`, `update`), or filter the list by a name substring (`list`). | `list`, `show`, `update` | | `--new-name ` | New name for the password. | `update` | | `--cidrs ` | Replace the IP addresses and CIDR ranges allowed to use this password. Pass an empty value to remove all restrictions. | `update` | | `--status ` | Filter passwords by status: `active`, `renewable`, or `expired`. | `list` | | `--page ` | Page of results to fetch. | `list` | | `--per-page ` | Number of results per page. Default is `100`. | `list` | | `--web` | Perform the action in your web browser | `list` | Available roles for the `--role` flag are: * `reader` * `writer` * `readwriter` * `admin` ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `password` command | | `--org ` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### Create a read-only region password Create credentials that always connect to a specific Vitess read-only region: ```bash theme={null} pscale password create --read-only-region eu-west ``` List configured regions for a keyspace first: ```bash theme={null} pscale keyspace read-only-regions ``` ### The `show` sub-command Show the metadata of one set of credentials, by ID or by name: ```bash theme={null} pscale password show pscale password show --name ``` Only metadata is returned. The password itself is shown once, when it is created or renewed, and cannot be retrieved afterwards. ### The `update` sub-command Rename credentials or replace the IP addresses allowed to use them, without rotating the password: ```bash theme={null} pscale password update --new-name reporting pscale password update --name reporting --cidrs 10.0.0.0/8,192.168.1.1/32 pscale password update --name reporting --cidrs "" ``` An empty `--cidrs` value clears all IP restrictions. To rotate a password, create a new one and delete the old one. ### The `list` sub-command with filters Filter credentials by a name substring and by status: ```bash theme={null} pscale password list --name prod --status active ``` ### The `delete` sub-command Delete a password from a database branch: **Command:** ```bash theme={null} pscale password delete ``` **Output:** ```bash theme={null} ? Please type // to confirm: Password was successfully deleted from . ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: pgbouncer Source: https://planetscale.com/docs/cli/pgbouncer ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `pgbouncer` command Manage [dedicated PgBouncers](/docs/postgres/connecting/pgbouncer) for a Postgres database branch. Dedicated PgBouncers run separately from the database nodes and provide connection pooling for primary or replica traffic. This is distinct from the local PgBouncer on port 6432 and from `pgbouncer.*` parameters on `pscale branch resize`. This command is only available for PostgreSQL databases. **`--org` is required.** **Usage:** ```bash theme={null} pscale pgbouncer ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Description** | **Product** | | :- | :- | :- | :- | | `list ` | | List dedicated PgBouncers for a branch | Postgres | | `show ` | | Show a dedicated PgBouncer | Postgres | | `create ` | `--target`\*, `--name`, `--size`, `--replicas-per-cell` | Create a dedicated PgBouncer | Postgres | | `resize ` | `--size`, `--replicas-per-cell`, `--target`, `--parameters`, `--wait`, `--wait-timeout` | Change size, replicas, target, or parameters | Postgres | | `resize status ` | | Show the latest resize request | Postgres | | `resize cancel ` | | Cancel unfinished resize requests | Postgres | | `delete ` | `--force` | Delete a dedicated PgBouncer | Postgres | > \* *Flag is required* #### Sub-command flag descriptions | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--name` | Name for the PgBouncer. Optional on create; auto-generated if omitted. | `create` | | `--target` | Traffic target: `primary`, `replica`, or `replica_az_affinity`. Required on create. | `create`, `resize` | | `--size` | PgBouncer size SKU (for example `PGB_10`). | `create`, `resize` | | `--replicas-per-cell` | Number of PgBouncer replica servers per cell. | `create`, `resize` | | `--parameters` | Set a PgBouncer parameter as `namespace.name=value` (for example `pgbouncer.default_pool_size=100`). Repeatable. | `resize` | | `--wait` | Wait for the resize request to complete before returning. | `resize` | | `--wait-timeout` | Maximum time to wait with `--wait`. Default: `10m`. | `resize` | | `--force` | Delete a PgBouncer without confirmation. | `delete` | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `pgbouncer` command | | `--org ` | The organization for the current user (**required**) | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ```bash theme={null} pscale pgbouncer list --org pscale pgbouncer create --org --target replica --size PGB_10 pscale pgbouncer show --org pscale pgbouncer resize --org --size PGB_20 pscale pgbouncer resize status --org pscale pgbouncer delete --org --force ``` ## Related documentation > **[Dedicated PgBouncers](/docs/postgres/connecting/pgbouncer)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: ping Source: https://planetscale.com/docs/cli/ping Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `ping` command This command allows you to see the [latency](/docs/vitess/connecting/network-latency) between your machine and PlanetScale's public regions. **Usage:** ```bash theme={null} pscale ping ``` ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `ping` command | | `-n`, `--count` | Number of total pings. (default 10) | | `-p`, `--provider` | Only ping endpoints for the specified infrastructure provider. Possible values: `aws`, `gcp`. | | `--concurrency` | Number of concurrent pings. (default 8) | | `--timeout` | Timeout for a ping to succeed. (default 5s) | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples **Command:** ```bash theme={null} pscale ping ``` **Output:** ```shell theme={null} NAME (17) LATENCY ENDPOINT TYPE ----------------------------- --------- ------------------------------------------------ ----------- AWS us-west-2 34.6ms aws.connect.psdb.cloud optimized AWS us-west-2 34.8ms us-west.connect.psdb.cloud direct GCP us-central1 57.5ms gcp.connect.psdb.cloud optimized GCP us-central1 57.9ms gcp-us-central1.connect.psdb.cloud direct AWS us-east-2 60.5ms aws-us-east-2.connect.psdb.cloud direct GCP us-east4 69.2ms gcp-us-east4.connect.psdb.cloud direct AWS us-east-1 70.2ms us-east.connect.psdb.cloud direct GCP northamerica-northeast1 80.9ms gcp-northamerica-northeast1.connect.psdb.cloud direct AWS ap-northeast-1 116.6ms ap-northeast.connect.psdb.cloud direct GCP asia-northeast3 149.5ms gcp-asia-northeast3.connect.psdb.cloud direct AWS ap-southeast-2 150.9ms aws-ap-southeast-2.connect.psdb.cloud direct AWS eu-central-1 154.4ms eu-central.connect.psdb.cloud direct AWS eu-west-1 157.2ms eu-west.connect.psdb.cloud direct AWS sa-east-1 179.5ms aws-sa-east-1.connect.psdb.cloud direct AWS ap-southeast-1 189.4ms ap-southeast.connect.psdb.cloud direct AWS ap-south-1 243.1ms ap-south.connect.psdb.cloud direct AWS eu-west-2 670.5ms aws-eu-west-2.connect.psdb.cloud direct ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale environment set up Source: https://planetscale.com/docs/cli/planetscale-environment-setup ## Table of Contents > **[macOS installation](#macos-instructions)** > **[Linux installation](#linux-instructions)** > **[Windows installation](#windows-instructions)** > **[Manual setup (any OS)](#manual-setup-any-os)** > **[Using the PlanetScale CLI](#using-the-planetscale-cli)** ## Setup overview ### macOS instructions To install the PlanetScale CLI on macOS, we recommend using Homebrew. How to install or verify Homebrew is on your computer: **Step 1** Open Terminal. **Step 2** Check if you have Homebrew installed by running the following command: ```bash theme={null} brew -v ``` **Step 3** If you don't see "Homebrew" and a version number, then download and [install Homebrew](https://brew.sh/). **Step 4** Once you've installed Homebrew, repeat *Step 2* to verify. **Installing via Homebrew** * Run the following command: ```bash theme={null} brew install pscale ``` Agent automation commands (`agent-guide`, JSON `auth check`, hosted `mcp install`, `pscale sql`) require **`pscale` 0.292.0 or later**. Verify with `pscale agent-guide --format json`. * To install the MySQL command-line client: ```bash theme={null} brew install mysql-client ``` * To install the PostgreSQL command-line client: ```bash theme={null} brew install postgresql@17 ``` To upgrade to the latest version: ```bash theme={null} brew upgrade pscale ``` ### Linux instructions `pscale` is available as downloadable binaries from the [PlanetScale releases](https://github.com/planetscale/cli/releases/latest) page. Download the .deb or .rpm from the [releases](https://github.com/planetscale/cli/releases/latest) page and install with `sudo dpkg -i` and `sudo rpm -i` respectively. The MySQL and PostgreSQL command-line clients can be installed via your package manager. **MySQL client:** * Debian-based distributions: ```bash theme={null} sudo apt-get install mysql-client ``` * RPM-based distributions: ```bash theme={null} sudo yum install community-mysql ``` **PostgreSQL client:** * Debian-based distributions: ```bash theme={null} sudo apt-get install postgresql-client ``` * RPM-based distributions: ```bash theme={null} sudo yum install postgresql ``` ### Windows instructions On Windows, `pscale` is available via [scoop](https://scoop.sh/), and as a downloadable binary from the [PlanetScale releases](https://github.com/planetscale/cli/releases/latest) page: ```bash theme={null} scoop bucket add pscale https://github.com/planetscale/scoop-bucket.git scoop install pscale mysql postgresql ``` To upgrade to the latest version: ```bash theme={null} scoop update pscale ``` **Installation via binary** Download the latest [Windows release](https://github.com/planetscale/cli/releases/latest) and unzip the `pscale.exe` file into the folder of your choice. Then, run it from PowerShell or whatever terminal you regularly use. **MySQL client setup:** The MySQL command-line client is available in the [Windows MySQL Installer](https://dev.mysql.com/doc/refman/8.0/en/windows-installation.html). To launch `pscale shell` you will need to have the `mysql.exe` executable's directory in your shell's PATH. In PowerShell, add that directory to your current shell's PATH: ```powershell theme={null} $env:path += ";C:\Program Files\MySQL\MySQL Server 8.0\bin" ``` **PostgreSQL client setup:** The PostgreSQL command-line client is available from the [PostgreSQL downloads page](https://www.postgresql.org/download/windows/). After installation, you'll need to add the PostgreSQL bin directory to your PATH to use `psql`: ```powershell theme={null} $env:path += ";C:\Program Files\PostgreSQL\17\bin" ``` ## Manual setup (any OS) If you prefer to manually install the `pscale` binary for your operating system, the following two methods may be used. ### Download the binary Download the pre-compiled binaries from the [PlanetScale releases](https://github.com/planetscale/cli/releases/latest) page and download the binary for your operating system to the desired location. The binary may be run using the terminal of your choice from that location. ### Install using `bin` [bin](https://github.com/marcosnils/bin) is a cross-platform tool to manage binary files. You can install the `pscale` CLI using `bin` with the following command: ```bash theme={null} bin install https://github.com/planetscale/cli ``` ### Install the MySQL or PostgreSQL Client In either case, the MySQL or PostgreSQL client will need to be installed separately as well. **MySQL client:** Refer to the [official MySQL documentation](https://dev.mysql.com/doc/refman/8.0/en/installing.html) and select the operating system you are working with. **PostgreSQL client:** Refer to the [official PostgreSQL documentation](https://www.postgresql.org/download/) and select the operating system you are working with. ## Using the PlanetScale CLI Authenticate with your PlanetScale account: ```bash theme={null} pscale auth login ``` This opens a browser window to complete sign-in. See the [`auth` command reference](/docs/cli/auth) for details. See all available commands by running: ```bash theme={null} pscale --help ``` Verify that you're using the latest version: ```bash theme={null} pscale version ``` You're all set! Check out our [CLI reference page](/docs/cli) to explore all that's possible with the PlanetScale CLI. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: query-patterns Source: https://planetscale.com/docs/cli/query-patterns Vitess and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `query-patterns` command List, inspect, delete, and download query pattern reports for a database branch. The `download` command creates a Query Insights report, waits for it to finish generating, and writes the CSV file locally. Query pattern reports require Query Insights to be enabled for the database. Each CSV report includes up to 25,000 query patterns. **Usage:** ```bash theme={null} pscale branch query-patterns --org ``` ### Available sub-commands | **Sub-command** | **Product** | **Description** | | :- | :- | :- | | `list` | Postgres, Vitess | List query pattern reports for a branch | | `show` | Postgres, Vitess | Show the status and details of a query pattern report | | `delete` | Postgres, Vitess | Delete a query pattern report | | `download` | Postgres, Vitess | Generate and download a CSV report of branch query patterns | ### Available flags | **Flag** | **Description** | | - | - | | `--limit ` | Number of reports to return, up to 100. Defaults to 25. Available for `list`. | | `--starting-after ` | Fetch the next page after a report ID. Available for `list`. | | `--force` | Delete a report without confirmation. Available for `delete`. | | `--output ` | Output file for the CSV report. Defaults to `query-patterns----.csv` in the current directory. Available for `download`. | | `--org ` | Organization name | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `-h`, `--help` | Help for `query-patterns` | ## Examples List reports for a branch: ```bash theme={null} pscale branch query-patterns list --org ``` Show a report: ```bash theme={null} pscale branch query-patterns show --org ``` Delete a report: ```bash theme={null} pscale branch query-patterns delete --org ``` Generate and download a report with the default file name: ```bash theme={null} pscale branch query-patterns download --org ``` Generate and download a report to a specific path: ```bash theme={null} pscale branch query-patterns download --org --output ./query-patterns.csv ``` Use JSON output for automation: ```bash theme={null} pscale branch query-patterns download --org --format json ``` The JSON response includes the report ID, final state, and local file path: ```json theme={null} { "id": "report1", "state": "completed", "file": "./query-patterns.csv" } ``` If the branch is not found, or Query Insights is not enabled for the database, the command returns an error explaining both possible causes. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: region Source: https://planetscale.com/docs/cli/region Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `region` command This command lists all available [PlanetScale regions](/docs/vitess/regions). **Usage:** ```bash theme={null} pscale region ``` ### Available sub-commands | **Sub-command** | **Product** | **Description** | | :- | :- | :- | | `list` | Postgres, Vitess, Neki | List all regions | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `region` command | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `region` command with `list` sub-command **Command:** ```bash theme={null} pscale region list ``` **Output:** ```bash theme={null} NAME SLUG ENABLED ------------------------------------------------- ---------------------------- --------- AWS us-east-2 (Ohio) aws-us-east-2 Yes AWS us-east-1 (N. Virginia) us-east Yes AWS us-west-2 (Oregon) us-west Yes AWS eu-west-1 (Dublin) eu-west Yes AWS ap-south-1 (Mumbai) ap-south Yes AWS ap-southeast-1 (Singapore) ap-southeast Yes AWS ap-northeast-1 (Tokyo) ap-northeast Yes AWS eu-central-1 (Frankfurt) eu-central Yes AWS ap-southeast-2 (Sydney) aws-ap-southeast-2 Yes AWS sa-east-1 (Sao Paulo) aws-sa-east-1 Yes GCP us-central1 (Council Bluffs, Iowa) gcp-us-central1 Yes AWS eu-west-2 (London) aws-eu-west-2 Yes GCP us-east4 (Ashburn, Virginia) gcp-us-east4 Yes GCP northamerica-northeast1 (Montréal, Québec) gcp-northamerica-northeast1 Yes GCP asia-northeast3 (Seoul, South Korea) gcp-asia-northeast3 Yes GCP europe-west1 (St Ghislain, Belgium) gcp-europe-west1 Yes GCP us-east1 (Moncks Corner, South Carolina) gcp-us-east1 Yes GCP europe-west4 (Eemshaven, Netherlands) gcp-europe-west4 Yes AWS ca-central-1 (Montreal) aws-ca-central-1 Yes ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: role Source: https://planetscale.com/docs/cli/role Postgres and Neki ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). ## The `role` command Manage database roles for a Postgres or Neki database branch. **Usage:** ```bash theme={null} pscale role [command] ``` ### Available sub-commands | **Sub-Command** | **Product** | **Description** | | :- | :- | :- | | `create` | Postgres, Neki | Create a new role for a database branch | | `default` | Postgres, Neki | Show the default role without rotating its credentials | | `delete` | Postgres, Neki | Delete a role | | `get` | Postgres, Neki | Retrieve information about a specific role | | `list` | Postgres, Neki | List all roles for a database branch | | `reassign` | Postgres, Neki | Reassign objects owned by a role to another role | | `renew` | Postgres, Neki | Renew a role's expiration | | `reset` | Postgres, Neki | Reset a role's password | | `reset-default` | Postgres, Neki | Reset the credentials for the default `postgres` role | | `update` | Postgres, Neki | Update a role's name | ### Service token automation: `role` Legend: ✅ supported · Postgres only. All sub-commands require `--org` or `PLANETSCALE_ORG`. | Sub-command | Env-var auth | `--service-token` flag | Requires `--org` | Postgres / Vitess | `--format json` | API equivalent | | :- | :-: | :-: | :-: | :- | :-: | :- | | `list ` | ✅ | ✅ | ✅ | Postgres | ✅ | `pscale api organizations//databases//branches//roles --format json` | | `get ` | ✅ | ✅ | ✅ | Postgres | ✅ | `pscale api organizations//databases//branches//roles/ --format json` | | `create` / `delete` / `update` / `renew` / `reset` | ✅ | ✅ | ✅ | Postgres | ✅ | `POST pscale api organizations//databases//branches//roles --format json` · `DELETE pscale api organizations//databases//branches//roles/ --format json` · `PATCH pscale api organizations//databases//branches//roles/ --format json` · `POST pscale api organizations//databases//branches//roles//renew --format json` · `POST pscale api organizations//databases//branches//roles//reset --format json` | ```bash theme={null} export PLANETSCALE_ORG="" pscale role list --format json ``` Setup and commands to avoid: [CLI overview](/docs/cli) · [Service tokens](/docs/api/service-tokens#service-token-automation) ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `role` command | | `--org string` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `create` sub-command Create a new role for a Postgres or Neki database branch: **Usage:** ```bash theme={null} pscale role create [flags] ``` **Available flags:** * `--inherited-roles string` - Comma-separated list of role names to inherit privileges from. Common values are 'pg\_read\_all\_data' for read access, 'pg\_write\_all\_data' for write access, and 'postgres' for admin access. * `--ttl duration` - TTL defines the time to live for the role. Durations such as "30m", "24h", or bare integers such as "3600" (seconds) are accepted. The default TTL is 0s, which means the role will never expire. * `--with-replication` - Grant the `REPLICATION` privilege for logical replication. Requires `--inherited-roles postgres`. **Example:** ```bash theme={null} pscale role create my-database main api-user --inherited-roles pg_read_all_data --ttl 24h ``` ### The `default` sub-command Show the default role for a Postgres or Neki database branch without rotating its credentials: **Usage:** ```bash theme={null} pscale role default [flags] ``` **Example:** ```bash theme={null} pscale role default my-database main ``` ### The `delete` sub-command Delete a role: **Usage:** ```bash theme={null} pscale role delete [flags] ``` **Available flags:** * `--force` - Delete a role without confirmation * `--successor string` - Role to transfer ownership to before deletion. Usually 'postgres'. **Aliases:** `delete`, `rm` **Example:** ```bash theme={null} pscale role delete my-database main role-123 --successor postgres ``` ### The `get` sub-command Retrieve information about a specific role: **Usage:** ```bash theme={null} pscale role get [flags] ``` **Available flags:** * `--replica` - Return connection details for a branch replica. * `--dedicated-read-replica ` - Return connection details for a dedicated read replica. Postgres only. * `--bouncer ` - Return connection details for a named PgBouncer on Postgres or Neki. These connection-target flags are mutually exclusive. **Example:** ```bash theme={null} pscale role get pscale role get --replica ``` ### The `list` sub-command List all roles for a Postgres or Neki database branch: **Usage:** ```bash theme={null} pscale role list [flags] ``` **Available flags:** * `-w`, `--web` - List roles in your web browser. **Aliases:** `list`, `ls` **Example:** ```bash theme={null} pscale role list my-database main ``` ### The `reassign` sub-command Reassign objects owned by one role to any other role: > [!WARNING] > > Be careful with this command. Reassigning objects like databases, tables, or schemas will change who is able to write to them, alter them, or delete them. **Usage:** ```bash theme={null} pscale role reassign --successor ``` **Available flags:** * `--force` - Force reset without confirmation **Example:** ```bash theme={null} pscale role reassign my-database main role-123 --successor postgres ``` ### The `renew` sub-command Renew a role's expiration: **Usage:** ```bash theme={null} pscale role renew [flags] ``` **Example:** ```bash theme={null} pscale role renew my-database main role-123 ``` ### The `reset` sub-command Reset the credentials for any API-created role: > [!WARNING] > > Be careful with this command. If you are currently using the affected role's credentials for connecting to your database, running this command will reset the password, and new connections using the old password will not work. **Usage:** ```bash theme={null} pscale role reset [flags] ``` **Available flags:** * `--force` - Force reset without confirmation **Example:** ```bash theme={null} pscale role reset my-database main role-123 ``` ### The `reset-default` sub-command Reset the credentials for the default `postgres` role: > [!WARNING] > > Be careful with this command. If you are currently using the default `postgres` role credentials for connecting to your database, running this command will reset the password, and new connections using the old password will not work. **Usage:** ```bash theme={null} pscale role reset-default [flags] ``` **Available flags:** * `--force` - Force reset without confirmation **Example:** ```bash theme={null} pscale role reset-default my-database main ``` ### The `update` sub-command Update a role's name: **Usage:** ```bash theme={null} pscale role update [flags] ``` **Available flags:** * `--name string` - New name for the role **Example:** ```bash theme={null} pscale role update my-database main role-123 --name new-role-name ``` ## Related documentation > **[Managing Postgres roles](/docs/postgres/connecting/roles)** > **[Managing Neki roles](/docs/neki/connecting/roles)** > **[Postgres roles API documentation](/docs/api/reference/list_roles)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: service-token Source: https://planetscale.com/docs/cli/service-token Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `service-token` command This command allows you to create, list, and manage access for service tokens. **Usage:** ```bash theme={null} pscale service-token ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Product** | **Description** | | :- | :- | :- | :- | | `add-access ` | `--database ` | Postgres, Vitess, Neki | Add individual permissions to the specified service token in the organization | | `create` | `--name `, `--ttl ` | Postgres, Vitess, Neki | Create a service token for the organization | | `delete ` | | Postgres, Vitess, Neki | Delete an entire service token in the organization | | `delete-access ` | `--database ` | Postgres, Vitess, Neki | Delete individual permissions granted to a service token in the organization | | `list` | | Postgres, Vitess, Neki | List the IDs, names, and expiration times of an organization's service tokens | | `show ` | | Postgres, Vitess, Neki | Show a service token in the organization | | `show-access` | | Postgres, Vitess, Neki | Fetch a service token and its accesses | ### Service token automation: `service-token` Legend: 🚫 unavailable with a service token (all sub-commands). All `service-token` sub-commands are **unavailable when authenticated with a service token**. Exact error: `pscale service-token is unavailable when authenticated with a service token`. Do not retry the CLI. Use the alternatives below. | Sub-command | Env-var auth | `--service-token` flag | Requires `--org` | Postgres / Vitess | `--format json` | API equivalent | | :- | :-: | :-: | :-: | :- | :-: | :- | | `list` | 🚫 | 🚫 | ✅ | Both | ✅ | `pscale api organizations//service-tokens --format json` | | `show-access ` | 🚫 | 🚫 | ✅ | Both | ✅ | `pscale api organizations//service-tokens/ --format json` | | `create` | 🚫 | 🚫 | ✅ | Both | ✅ | `pscale auth login`, then CLI; or [Service tokens API](/docs/api/reference/service-tokens) | | `add-access` / `delete-access` / `delete` | 🚫 | 🚫 | ✅ | Both | ✅ | `pscale auth login`, then CLI; or [Service tokens API](/docs/api/reference/service-tokens) | Read operations (`list`, `show-access`): use `pscale api` with the same service token. Create and manage operations: run under `pscale auth login`; there is no service-token CLI path for those sub-commands. Setup and commands to avoid: [CLI overview](/docs/cli) · [Service tokens](/docs/api/service-tokens#service-token-automation) #### Sub-command flag descriptions Some of the sub-commands have additional flags unique to the sub-command. This section covers what each of those does. See the above table for which context. | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--database ` | The database this project is using. | `add-access`, `delete-access` | | `--name ` | The name for the service token. | `create` | | `--ttl ` | Time to live (in seconds) for the service token. The token will be invalid when the TTL has passed. | `create` | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `service-token` command | | `--org ` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### Adding service token access for a database **Command:** ```bash theme={null} pscale service-token add-access read_branch delete_branch create_branch --database ``` This will add the following permissions to the specified access token: `read_branch`, `delete_branch`, `create_branch`. The output will show the updated permissions on all databases. You can find a list of all permissions in the [PlanetScale API documentation](/docs/api/reference/service-tokens#access-permissions). **Output:** | RESOURCE NAME | RESOURCE TYPE | ACCESSES | | :- | :- | :- | | dbname | Database | read\_branch, delete\_branch, create\_branch | | dbname2 | Database | create\_branch | | my-org | Organization | create\_databases, delete\_databases | ### Adding service token access for an organization **Command:** ```bash theme={null} pscale service-token add-access read_organization ``` This will grant the service token the `read_organization` access on the organization. **Output:** | RESOURCE NAME | RESOURCE TYPE | ACCESSES | | :- | :- | :- | | my-org | Organization | read\_organization | ### Show a service token ```bash theme={null} pscale service-token show ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Service tokens Source: https://planetscale.com/docs/cli/service-tokens PlanetScale provides the ability to create service tokens for your PlanetScale organization via the CLI or directly in the UI. ## Overview Service tokens provide an alternate authentication method to be used with the PlanetScale CLI and API. They are typically used in automated scenarios where `pscale auth login` cannot be used. Service tokens are also required for any calls to the API, as well as minting OAuth tokens for API use. ## Create service tokens using the PlanetScale dashboard To create a service token using the dashboard, log into your organization, go to the [**"Settings"** > **"Service tokens"**](https://app.planetscale.com/~/settings/service-tokens) page, and click the **"New service token"** button. Give the token a name (this is used for your reference only). You can also optionally set an expiration date for the token — once the expiration date is reached, the token will stop working. Click **"Create service token"** to proceed. The modal will update, displaying your service token where the Name field was. Copy the ID and token values as you'll need them moving forward. Click **"Edit token permissions"** to proceed. > [!TIP] > > Be sure to copy the service token after you create it. There's no way to retrieve the token value once you leave this page. ![Service token detail page](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/service-tokens/modal-with-service-token-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=1f18f07b836f9395cc035223cfc9e437) ## Assign service token permissions Service tokens are configured with granular permissions, both for the organization that owns them as well as on a per database level. Before you can use a service token, these permissions must be added. ### Add organization permissions Organization permissions are required when performing operations that are specific to the organization and not for an individual database. To enable a service token for performing these operations, locate the **Organization access** section and click **"Add organization permissions"**. In the **Organization access permissions** modal, check the box next to each of the permission scopes that you want to assign to the token. Click **"Save permissions"** once finished. For a full list of organization access permissions, see the [API documentation for service tokens](https://planetscale.com/docs/api/reference/service-tokens#organization-access-permissions). ### Add database permissions In order to perform operations specific to a database, permissions can be assigned per-database. To do this, locate the section titled **Database access** and click **"Add database access"** to open the **Database access permissions** modal. Select the database you want to grant access to and check the box next to each permission option you need to grant. Once you are done, click **"Save permissions"**. For a full list of database access permissions, see the [API documentation for service tokens](https://planetscale.com/docs/api/reference/service-tokens#database-access-permissions). ![The Database access permissions modal.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/service-tokens/db-access-permissions-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=2f2a5fae0c095305dc848397de6962a5) ### Add permissions for all databases Organization admins can grant database permissions to all current and future databases. This is available in the **Permissions for all databases** section. Any permission added to all databases will not be able to be disabled on an individual basis. ## Service tokens and deploy requests approvals When a database requires administrator approval for deploy requests (located in your database's **Settings** page), a service token cannot approve a deploy request created by the same service token. Also, users can't approve a deploy request created by a service token that they created. ## Use a service token with the PlanetScale CLI To use service tokens with the PlanetScale CLI, set the following environment variables in your terminal: ```bash theme={null} export PLANETSCALE_SERVICE_TOKEN= export PLANETSCALE_SERVICE_TOKEN_ID= ``` When you execute commands using the PlanetScale CLI, it will automatically parse those values and use them to access the service. However, you’ll also need to pass in your organization name using the `--org` flag like so: ```bash theme={null} pscale branch create --org ``` If you don’t want to set environment variables, you may also pass in the Service Token and Service Token ID by using the [`--service-token` and `--service-token-id` flags](/docs/cli/service-token) respectively: ```bash theme={null} pscale branch create --org --service-token --service-token-id ``` ## Use a service token with the PlanetScale API In order to execute a request to the PlanetScale API, you'll need a service token to execute requests directly or for minting OAuth tokens. Both the ID and token are required in the `Authorization` header without a scheme. Below is an example of how to use a service token to list details about the organizations the token can access: ```bash theme={null} curl --request GET \ --url 'https://api.planetscale.com/v1/organizations' \ --header 'Authorization: :' ``` Refer to the [API docs](/docs/api/reference/getting-started-with-planetscale-api) for more details on how to use the API. ## Modify service token permissions If you want to modify the permissions granted to a service token, start by opening the service token from the settings pane. Select the three dots next to the organization or database name permissions you want to modify and click **"Edit permissions"**. ![The location of the Edit permissions option for organization permissions.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/service-tokens/edit-org-perms-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=2ff592a3bac8993002f800c5b41ad4b0) This will open a modal that allows you to modify the permissions the service token has to access that organization. ## Delete a service token You can delete a service token at any time from the service token detail page. Simply click the **"Delete service token"** button. ![Delete service token.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/service-tokens/delete-service-token-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=81a02e008e79a793ba15464b09e3b759) ## Manage service tokens using the PlanetScale CLI Service tokens can also be created and managed directly from the [PlanetScale CLI](/docs/cli/service-token). ### Create a new service token Use the following command to create a service token: ```bash theme={null} pscale service-token create ``` You can optionally specify a name for the service token using the `--name` flag: ```bash theme={null} pscale service-token create --name "My Token" ``` You can also specify a time to live (TTL) in seconds using the `--ttl` flag. The token will be invalid once the TTL has passed: ```bash theme={null} pscale service-token create --name "My Token" --ttl 3600 ``` This command will return a service token ID, value, and expiration time (if TTL was specified) for your use. ### Add database access permissions You can add database access permissions to your service token for each database in your organization. To add database access permissions, use the command: ```bash theme={null} pscale service-token add-access --database ``` For example, to give a service token the ability to create, read, and delete branches on a specific database, use the following command: ```bash theme={null} pscale service-token add-access read_branch delete_branch create_branch --database ``` A complete list of service token access permissions can be found in the [PlanetScale API documentation](/docs/api/reference/service-tokens#access-permissions). ### Remove database access permissions You can also remove database access permissions for a service token. Use the following command to remove one or more permissions: ```bash theme={null} pscale service-token delete-access --database ``` ### Delete a service token To delete a service token, run the following command: ```bash theme={null} pscale service-token delete ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: shell Source: https://planetscale.com/docs/cli/shell Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `shell` command This command opens a secure shell instance to your database so that you can manipulate it from the command line. For Vitess databases, it uses the MySQL command-line client (`mysql`). For Postgres and Neki databases, it uses the PostgreSQL command-line client (`psql`). The appropriate client [must be installed](/docs/cli/planetscale-environment-setup) prior to use. > [!NOTE] > > If your Managed cluster has public connectivity disabled, you must connect over PrivateLink. Set up the VPC endpoint using the service name we provide, make sure your DNS resolves your PlanetScale hostnames to that PrivateLink endpoint, and then run `pscale shell mydatabase mybranch`. > > If you use a VPN such as Tailscale, do not override `psdb.cloud` DNS in your VPN configuration. Split‑DNS misconfiguration can point hostnames at the wrong place and break shell connectivity. **Usage:** ```bash theme={null} pscale shell [BRANCH_NAME] [DB_NAME] ``` By default, if no branch names are given and there is only one branch, it automatically opens a shell to that branch. If there are multiple branches for the given database, you'll be prompted to choose one. ### Available flags | **Flag** | **Description** | | | :- | :- | - | | `-h`, `--help` | Help for shell command | | | `--local-addr
` | Local address where the Vitess proxy binds and listens. By default it binds to `127.0.0.1` with a random port. Not supported for Postgres or Neki. | | | `--org ` | The organization for the current user | | | `--remote-addr
` | PlanetScale Database remote network address. By default the remote address is populated automatically from the PlanetScale API | | | `--replica` | Connect to a replica. The CLI adds the \` | replica`username suffix for Vitess and Postgres. For Neki, it instead sets`PGOPTIONS`to`-c \_\_neki.target=REPLICA\`. | | `--role ` | Role defines the access level, allowed values are: reader, writer, readwriter, admin. Defaults to 'reader' for replica passwords, otherwise defaults to 'admin' | | | `--router ` | Connect through the named router group. Neki only. | | | `--db-name ` | PostgreSQL database name to connect to (default `postgres`). Also accepted as a third positional argument. Postgres and Neki only. | | Available roles for the `--role` flag are: * `reader` - Read-only access * `writer` - Write-only access * `readwriter` - Read and write access * `admin` - Full administrative access For replica connections (`--replica` flag), the default role is `reader`. For regular connections, the default role is `admin`. ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API | | `--api-url ` | The base URL for the PlanetScale API (default `https://api.planetscale.com/`) | | `--config ` | Config file (default is `$HOME/.config/planetscale/pscale.yml`) | | `--debug` | Enable debug mode | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv` | | `--no-color` | Disable color output | | `--service-token ` | Service Token for authenticating | | `--service-token-id ` | The Service Token ID for authenticating | ## Examples ### Basic shell usage **Open a shell to a database (auto-selects branch if only one exists):** ```bash theme={null} pscale shell mydatabase ``` **Open a shell to a specific branch:** ```bash theme={null} pscale shell mydatabase mybranch ``` **Open a Postgres or Neki shell to a specific database name:** ```bash theme={null} pscale shell --db-name pscale shell ``` Once the shell is opened, you can run SQL as expected. **Example MySQL session:** ```bash theme={null} DATABASE_NAME/BRANCH_NAME > DATABASE_NAME/BRANCH_NAME > show tables; +---------------+ | Tables_in_db | +---------------+ | users | +---------------+ DATABASE_NAME/BRANCH_NAME > exit; ``` **Example Postgres session:** ```bash theme={null} psql-17 (17.5 (Homebrew)) SSL connection (protocol: TLSv1.3, cipher: TLS_AES_128_GCM_SHA256, compression: off, ALPN: postgresql) Type "help" for help. pg/|⚠ main ⚠|> \dt List of relations Schema | Name | Type | Owner --------+-------------+-------+------------------------- public | users | table | pscale_api_2e2o0t28kd0v (1 row) pg/|⚠ main ⚠|> \q ``` Type `exit` (MySQL) or `\q` (Postgres and Neki) to exit the shell. ### Using replica connections **Connect to a read-only replica:** ```bash theme={null} pscale shell mydatabase mybranch --replica ``` Replica connections route reads to replicas and default to the `reader` role. For Neki, `--replica` sets the session's Neki target to `REPLICA`. You can combine it with `--router` to select both a router group and a replica target. ### Connect through a Neki router group ```bash theme={null} pscale shell mydatabase mybranch --router analytics pscale shell mydatabase mybranch --router analytics --replica ``` ### Using specific roles **Connect with a specific role:** ```bash theme={null} pscale shell mydatabase mybranch --role reader ``` ### Import an existing .sql file using the `shell` command **Command:** The following example assumes you have already ran the `pscale shell` command and you have the shell open to run a MySQL command. To import an existing `.sql` file you may have available you would want to use the MySQL `source` command and provide it the path to your file: ```bash theme={null} DATABASE_NAME/BRANCH_NAME > source .sql; ``` **Output:** Your file should be imported as expected. > [!NOTE] > > When importing `.sql` dump files there are a few caveats to be aware of as sometimes the `.sql` file may have everything wrapped in a `START TRANSACTION;` / `COMMIT;` transaction which will result in the import timing out if it takes more than 20 seconds to complete due to our 20 second transaction timeout limit so you will want to make sure those are removed prior to beginning an import of the file. > > Additionally, if your current schema requires our [Vitess foreign key constraints](/docs/vitess/foreign-key-constraints) support you may need to ensure it has been enabled within your database Settings area first before proceeding with your import. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: signup Source: https://planetscale.com/docs/cli/signup ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `signup` command This command allows you to sign up for a PlanetScale account straight from the command line. You'll be prompted to register with an email and password. **Usage:** ```bash theme={null} pscale signup ``` ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `signup` command | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### The `signup` command **Command:** ```bash theme={null} pscale signup ``` **Output:** ```bash theme={null} You are registering a new PlanetScale account. ? What is your e-mail? me@example.com ? Please type your password ****************** ? Please confirm your password ****************** You've successfully signed up for PlanetScale! Please check your email for a confirmation link and then get started with `pscale auth login`. ``` The password must be a minimum of 10 characters and include: 1 uppercase, digit, or special character. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: size Source: https://planetscale.com/docs/cli/size Vitess, Neki, and Postgres Lists the sizes for various components within PlanetScale. ## Usage ```bash theme={null} pscale size [command] ``` ## Available commands | **Command** | **Product** | **Description** | | :- | :- | :- | | `cluster` | Postgres, Vitess, Neki | List the sizes for PlanetScale databases | ## Flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | Help for size | | `--org string` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token string` | The API token to use for authenticating against the PlanetScale API | | `--api-url string` | The base URL for the PlanetScale API. (default "\<[https://api.planetscale.com/>](https://api.planetscale.com/>)") | | `--config string` | Config file (default is `$HOME/.config/planetscale/pscale.yml`) | | `--debug` | Enable debug mode | | `-f`, `--format string` | Show output in a specific format. Possible values: \[human, json, csv] (default "human") | | `--no-color` | Disable color output | | `--service-token string` | The service token for authenticating | | `--service-token-id string` | The service token ID for authenticating | ## The `cluster` sub-command List the sizes for PlanetScale databases. **Usage:** ```bash theme={null} pscale size cluster [command] ``` **Alias:** `clusters` ### Available sub-commands | **Command** | **Product** | **Description** | | :- | :- | :- | | `list` | Postgres, Vitess, Neki | List the sizes that are available for a PlanetScale database | ## The `list` sub-command List the sizes that are available for a PlanetScale database. Use `--engine` to specify the database engine type. **Usage:** ```bash theme={null} pscale size cluster list [flags] ``` **Alias:** `ls` ### Available flags | **Flag** | **Description** | | :- | :- | | `--engine string` | Filter cluster sizes by database engine. Supported values: `mysql`, `postgresql`, `neki`. Omit to list all engines. | | `--external` | View cluster sizes for external Vitess keyspaces. Mutually exclusive with `--metal`. Only valid with the MySQL engine. | | `-h`, `--help` | Help for list | | `--metal` | View cluster sizes and rates for clusters with metal storage | | `--region string` | View cluster sizes and rates for a specific region | ## Examples ### List all available cluster sizes ```bash theme={null} pscale size cluster list ``` Omitting `--engine` lists Vitess, Neki, and Postgres sizes. ### List Vitess cluster sizes explicitly ```bash theme={null} pscale size cluster list --engine mysql ``` ### List PostgreSQL cluster sizes ```bash theme={null} pscale size cluster list --engine postgresql ``` ### List Neki cluster sizes ```bash theme={null} pscale size cluster list --engine neki ``` Neki admin and router SKUs are listed separately with `pscale branch admin sizes` and `pscale branch router sizes`. See [Neki cluster sizing](/docs/neki/cluster-configuration/cluster-sizing). ### List cluster sizes for a specific organization ```bash theme={null} pscale size cluster list --org ``` ### List cluster sizes for a specific region ```bash theme={null} pscale size cluster list --region ``` ### List PostgreSQL cluster sizes for a specific region ```bash theme={null} pscale size cluster list --engine postgresql --region us-east ``` ### List Metal cluster sizes ```bash theme={null} pscale size cluster list --metal ``` For Postgres or Neki Metal clusters, use: ```bash theme={null} pscale size cluster list --engine postgresql --metal pscale size cluster list --engine neki --metal ``` ### List external keyspace cluster sizes ```bash theme={null} pscale size cluster list --external ``` `--cluster-size` is optional on `create-external`. Use these sizes with `pscale keyspace create-external --cluster-size` and `pscale keyspace resize --cluster-size` for [external keyspaces](/docs/cli/keyspace#create-an-external-keyspace). `--external` is MySQL/Vitess only. For more information about PlanetScale cluster sizes and pricing, see: * [PlanetScale Vitess plans documentation](/docs/planetscale-plans) * [Neki pricing](/docs/neki/pricing) * [PlanetScale Postgres pricing](/docs/postgres/pricing) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: sql Source: https://planetscale.com/docs/cli/sql Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `sql` command Execute a single SQL query against a database branch **without** an interactive shell. Intended for agents and scripts. For interactive sessions, use [`pscale shell`](/docs/cli/shell) instead. > [!NOTE] > > **Requires `pscale` 0.292.0 or later.** **Usage:** ```bash theme={null} pscale sql --org --query "" ``` Place **positional arguments first**, then flags. **`--org` is required.** ### Available flags | **Flag** | **Description** | | - | - | | `--query ` | SQL query to execute **(required)** | | `--org ` | Organization name **(required)** | | `--role ` | Access level: `reader` (default), `writer`, `readwriter`, `admin` | | `--replica` | Route reads to a replica for PlanetScale Postgres or Vitess. For Neki, use [`pscale shell --replica`](/docs/cli/shell) instead, which sets `__neki.target=REPLICA`. | | `--dbname ` | Postgres or Neki database name (default `postgres`) | | `--keyspace ` | MySQL/Vitess keyspace (default `@primary`, same as `pscale shell`) | | `--force` | Allow destructive SQL after explicit user approval (see below) | | `--vertical` | Print each row vertically, one column per line. Ending a query with `\G` has the same effect. | | `-h`, `--help` | Help for `sql` | Unlike [`pscale shell`](/docs/cli/shell), the default `--role` is **`reader`**. Pass `--role admin` (or `writer` / `readwriter`) for writes. ### Destructive SQL Queries containing **`DELETE`**, **`DROP`**, or **`TRUNCATE`** as statement keywords are blocked unless `--force` is passed. With `--format json`, blocked queries return `"status": "action_required"` and `"query_kind": "destructive"`. Agents must **ask the user to approve**, then re-run with `--force`. Never use `--force` without explicit user approval. ### JSON output Use `--format json` for automation. **Success:** `status`, `database`, `branch`, `kind` (`mysql`, `postgresql`, or `neki`), `role`, `row_count`, `columns`, `rows`, and `next_steps`; `rows_affected` when applicable and `replica` when `--replica` was used. Neki databases use the PostgreSQL query path. **Errors:** one JSON object on stdout with `status: "error"`, `error`, `issues`, and `next_steps`. **Destructive guard:** `status: "action_required"`, `query_kind: "destructive"`, `message`, `issues`, and `next_steps` (includes a `--force` retry command). ### Global flags | **Flag** | **Description** | | - | - | | `-f`, `--format json` | JSON on stdout (recommended for agents) | | `--api-token`, `--service-token-id`, `--service-token` | Authentication | | `--api-url ` | API base URL | | `--config ` | Config file | | `--debug` | Debug mode | | `--no-color` | Disable color output | ## Examples **Read query (default reader role):** ```bash theme={null} pscale sql --org --format json --query "SELECT 1" ``` **Read from a PlanetScale Postgres or Vitess replica:** ```bash theme={null} pscale sql --org --format json --replica --query "SELECT COUNT(*) FROM users" ``` `pscale sql --replica` appends the Postgres `|replica` username suffix. That is not how Neki routes replica traffic. For Neki replica reads, use [`pscale shell --replica`](/docs/cli/shell). Do not add `|replica` to a Neki username. **Postgres or Neki with an explicit database name:** ```bash theme={null} pscale sql --org --format json --dbname postgres --query "SELECT version()" ``` **MySQL multi-keyspace:** ```bash theme={null} pscale sql --org --format json --keyspace --query "SELECT 1" ``` **Vertical output for wide rows:** ```bash theme={null} pscale sql --org --replica --query "SHOW REPLICA STATUS\G" ``` **Destructive SQL (after user approval):** ```bash theme={null} pscale sql --org --format json --force --query "DELETE FROM staging_rows WHERE id = 1" ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: traffic-control Source: https://planetscale.com/docs/cli/traffic-control ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `traffic-control` command Manage [Database Traffic Control](/docs/postgres/traffic-control) budgets and rules for a branch. This command is only supported for Postgres databases. **Usage:** ```bash theme={null} pscale traffic-control [command] ``` ### Available sub-commands | **Sub-command** | **Description** | | :- | :- | | `budget` | Manage traffic budgets | | `rule` | Manage traffic rules | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `traffic-control` command | | `--org string` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | *** ## The `budget` sub-command Manage traffic budgets. **Usage:** ```bash theme={null} pscale traffic-control budget [command] ``` ### Available sub-commands | **Sub-command** | **Description** | | :- | :- | | `create` | Create a traffic budget | | `delete` | Delete a traffic budget | | `list` | List traffic budgets | | `show` | Show a traffic budget | | `update` | Update a traffic budget | ### The `budget list` sub-command List the traffic budgets for a branch, including their IDs. **Usage:** ```bash theme={null} pscale traffic-control budget list [flags] ``` **Available flags:** | **Flag** | **Description** | | :- | :- | | `--fingerprint string` | Only list budgets with a rule for this query fingerprint | ### The `budget create` sub-command Create a traffic budget. **Usage:** ```bash theme={null} pscale traffic-control budget create [flags] ``` **Available flags:** | **Flag** | **Description** | | :- | :- | | `--name string` | Name of the traffic budget (required) | | `--mode string` | Mode of the budget: `enforce`, `warn`, or `off` (default `warn`) | | `--rate int` | Rate at which capacity refills, as a percentage of server resources (0-100). Unlimited when not set. | | `--capacity int` | Maximum capacity that can be banked (0-6000). Unlimited when not set. | | `--burst int` | Maximum capacity a single query can consume (0-6000). Unlimited when not set. | | `--concurrency int` | Percentage of available worker processes (0-100). Unlimited when not set. | ### The `budget update` sub-command Update a traffic budget. **Usage:** ```bash theme={null} pscale traffic-control budget update [flags] ``` A budget's `` can be retrieved by running the `show`, `create`, or `list` commands. **Available flags:** Same flags as [`budget create`](#the-budget-create-sub-command). ### The `budget show` sub-command Show a traffic budget. **Usage:** ```bash theme={null} pscale traffic-control budget show [flags] ``` ### The `budget delete` sub-command Delete a traffic budget. **Usage:** ```bash theme={null} pscale traffic-control budget delete [flags] ``` **Available flags:** | **Flag** | **Description** | | :- | :- | | `--force` | Delete a budget without confirmation | **Aliases:** `delete`, `rm` *** ## The `rule` sub-command Manage traffic rules. **Usage:** ```bash theme={null} pscale traffic-control rule [command] ``` ### Available sub-commands | **Sub-command** | **Description** | | :- | :- | | `create` | Create a traffic rule on a budget | | `delete` | Delete a traffic rule from a budget | ### The `rule create` sub-command Create a traffic rule on a budget. **Usage:** ```bash theme={null} pscale traffic-control rule create [flags] ``` This command adds a rule to the budget. Rules are combined via OR, tags within a rule are combined via AND. Create multiple rules to get the desired AND/OR behavior. **Available flags:** | **Flag** | **Description** | | :- | :- | | `--fingerprint string` | SQL fingerprint to match | | `--keyspace string` | Keyspace to match | | `--kind string` | Kind of rule (default `match`) | | `--tag stringArray` | Tag in the format `key=,value=,source=` (repeatable) | ### The `rule delete` sub-command Delete a traffic rule from a budget. **Usage:** ```bash theme={null} pscale traffic-control rule delete [flags] ``` **Available flags:** | **Flag** | **Description** | | :- | :- | | `--force` | Delete a rule without confirmation | **Aliases:** `delete`, `rm` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: webhook Source: https://planetscale.com/docs/cli/webhook Vitess, Neki, and Postgres ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `webhook` command This command allows you to create, list, update, test, and delete [webhooks](/docs/api/webhooks) for your database. **Usage:** ```bash theme={null} pscale webhook ``` ### Available sub-commands | **Sub-command** | **Description** | **Product** | | :- | :- | :- | | `create ` | Create a new webhook for a database | Vitess, Postgres, Neki | | `delete ` | Delete a webhook from a database | Vitess, Postgres, Neki | | `list ` | List all webhooks for a database | Vitess, Postgres, Neki | | `show ` | Show details for a specific webhook | Vitess, Postgres, Neki | | `test ` | Send a test event to a webhook | Vitess, Postgres, Neki | | `update ` | Update an existing webhook | Vitess, Postgres, Neki | #### Sub-command flags | **Sub-command flag** | **Description** | **Applicable sub-commands** | | :- | :- | :- | | `--authorization-header ` | Complete Authorization header value, including the authentication scheme (for example, `Bearer `). Sent unchanged. | `create`, `update` | | `--clear-authorization-header` | Remove the configured Authorization header. Cannot be used with `--authorization-header`. | `update` | | `--events ` | Comma-separated list of events to trigger the webhook. See [webhook events](/docs/api/webhook-events) for available events. | `create`, `update` | | `--url ` | The HTTPS URL where webhook events will be sent | `create`, `update` | | `--enabled` | Enable or disable the webhook. Use `--enabled` or `--enabled=true` to enable, `--enabled=false` to disable. Webhooks are disabled by default when created. | `update` | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for `webhook` command | | `--org ` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Examples ### List webhooks for a database **Command:** ```bash theme={null} pscale webhook list --org ``` This lists all webhooks configured for the specified database. **Output:** ```bash theme={null} No webhooks exist in database . ``` Or if webhooks exist: ```bash theme={null} ID URL EVENTS ENABLED CREATED AT UPDATED AT -------------- ------------------------------------- ------------------------------- --------- ------------ ------------ abc123xyz https://example.com/webhook branch.ready, branch.sleeping Yes 2 days ago 1 day ago ``` ### Create a webhook **Command:** ```bash theme={null} pscale webhook create --org \ --events "branch.ready,branch.sleeping" \ --url https://example.com/webhook \ --authorization-header "Bearer $AUTOMATION_TOKEN" ``` This creates a new webhook for the specified database with the selected events. The example sends the complete Authorization header value, `Bearer $AUTOMATION_TOKEN`, with the token supplied by the environment variable. PlanetScale sends the value unchanged. The header value is write-only and will not appear in command output. The webhook will be disabled by default until you enable it with the `update` command. **Output:** ```bash theme={null} ID URL SECRET EVENTS ENABLED CREATED AT UPDATED AT -------------- ------------------------------------- ------------------------------------------------------------------ ------------------------------- --------- ------------ ------------ abc123xyz https://example.com/webhook 8e46bd50ca092655b1efdfca329f0d79eb976714030a8bfa031397eb0d1cb433 branch.ready, branch.sleeping No now now ``` > [!NOTE] > > When you create a webhook, a secret is generated and displayed **only once** in the output. Store this secret securely as you'll need it to [validate webhook signatures](/docs/api/webhooks#validating-a-webhook-signature). You can also view the secret later from the database settings page in the dashboard. ### Show webhook details **Command:** ```bash theme={null} pscale webhook show --org ``` This displays detailed information about a specific webhook, including its ID, URL, secret, events, enabled status, and timestamps. **Output:** ```bash theme={null} ID URL SECRET EVENTS ENABLED CREATED AT UPDATED AT -------------- ------------------------------------- ------------------------------------------------------------------ ------------------------------- --------- ---------------- ---------------- abc123xyz https://example.com/webhook b4c29e6ae54a6456496cec7dcbfad7ace6e973a694802de2978b4d6e001fca6e branch.ready, branch.sleeping No 25 seconds ago 25 seconds ago ``` > [!NOTE] > > The `show` command displays the webhook secret, which is useful if you need to retrieve it after creation. Store this secret securely as you'll need it to [validate webhook signatures](/docs/api/webhooks#validating-a-webhook-signature). ### Update a webhook **Command:** ```bash theme={null} pscale webhook update --org --enabled ``` This enables an existing webhook. To disable a webhook, use `--enabled=false`. You can also update the events or URL, or set an Authorization header with `--authorization-header`. Supply the complete header value, including the authentication scheme—for example, `--authorization-header "Bearer $AUTOMATION_TOKEN"`. PlanetScale sends the value unchanged. Omitting `--authorization-header` preserves the configured value. Webhook output indicates whether an Authorization header is configured without revealing its value. To remove a configured header, run: ```bash theme={null} pscale webhook update --org \ --clear-authorization-header ``` **Output:** ```bash theme={null} ID URL EVENTS ENABLED CREATED AT UPDATED AT -------------- ------------------------------------- ------------------------------- --------- -------------- ------------ abc123xyz https://example.com/webhook branch.ready, branch.sleeping Yes 58 seconds ago now ``` ### Test a webhook **Command:** ```bash theme={null} pscale webhook test --org ``` This sends a test event to the webhook URL to verify it's configured correctly. You can only send one test event every 20 seconds per webhook. **Output:** ```bash theme={null} Test event was successfully sent to webhook . ``` ### Delete a webhook **Command:** ```bash theme={null} pscale webhook delete --org ``` This permanently deletes the webhook from the database. **Output:** ```bash theme={null} Webhook was successfully deleted from . ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale CLI commands: workflow Source: https://planetscale.com/docs/cli/workflow ## Getting Started Make sure to first [set up your PlanetScale developer environment](/docs/cli/planetscale-environment-setup). Once you've installed the `pscale` CLI, you can interact with PlanetScale and manage your databases straight from the command line. ## The `workflow` command > [!WARNING] > > `pscale workflow` is deprecated. To move tables between keyspaces, use [`pscale branch vtctl move-tables`](/docs/cli/move-tables). To import an external MySQL database, create an [external keyspace and move its tables with MoveTables](/docs/vitess/imports/database-imports). > > `pscale workflow` only lists workflows it created. Workflows created with `pscale branch vtctl move-tables` do not appear in `pscale workflow list`. This command manages workflows that were created with `pscale workflow create` or the earlier dashboard workflow pages: listing them, switching traffic, and cutting over. This command is not supported for Postgres databases. **Usage:** ```bash theme={null} pscale workflow ``` ### Available sub-commands | **Sub-command** | **Sub-command flags** | **Product** | **Description** | | :- | :- | :- | :- | | `cancel ` | `--force` | Vitess | Cancel a workflow that is in progress | | `complete ` | | Vitess | Mark a workflow as complete | | `create ` | `--defer-secondary-keys`, `--global-keyspace `, `-i`, `--interactive`, `--name `, `--on-ddl `, `--source-keyspace `, `--tables `, `--target-keyspace ` | Vitess | Create a new workflow within a branch | | `cutover ` | `--force` | Vitess | Completes the workflow, cutting over all traffic to the target keyspace | | `list ` | | Vitess | List all of the workflows for a PlanetScale database | | `retry ` | | Vitess | Retry a workflow that has been stopped or failed | | `reverse-cutover ` | | Vitess | Reverse the cutover of a workflow back to the source keyspace | | `reverse-traffic ` | | Vitess | Route queries back to the source keyspace for a specific workflow | | `show ` | | Vitess | Show a specific workflow for a PlanetScale database | | `switch-traffic ` | `--force`, `--replicas-only` | Vitess | Route queries to the target keyspace for a specific workflow | | `verify-data ` | | Vitess | Verify data consistency for a specific workflow | #### Sub-command flag descriptions Some of the sub-commands have additional flags unique to the sub-command. This section covers what each of those does. See the above table for which context. | **Sub-command flag** | **Type** | **Description** | **Applicable sub-commands** | | :- | :- | :- | :- | | `--defer-secondary-keys` | boolean | Don't create secondary indexes for tables until they've been copied (default true) | `create` | | `--force` | boolean | Force the operation without prompting for confirmation | `cancel`, `cutover`, `switch-traffic` | | `--global-keyspace ` | string | Choose an unsharded keyspace where sequence tables will be created for any workflow table that contains AUTO\_INCREMENT | `create` | | `-i, --interactive` | boolean | Create the workflow in interactive mode | `create` | | `--name ` | string | Name of the workflow | `create` | | `--on-ddl ` | string | Action to take when a DDL statement is encountered during a running workflow. Options: EXEC, EXEC\_IGNORE, STOP, IGNORE (default "STOP") | `create` | | `--replicas-only` | boolean | Route read queries from the replica and read-only tablets to the target keyspace | `switch-traffic` | | `--source-keyspace ` | string | Keyspace where the tables will be copied from | `create` | | `--tables ` | strings | Tables to migrate to the target keyspace | `create` | | `--target-keyspace ` | string | Keyspace where the tables will be copied to | `create` | ### Available flags | **Flag** | **Description** | | :- | :- | | `-h`, `--help` | View help for workflow command | | `--org ` | The organization for the current user | ### Global flags | **Command** | **Description** | | :- | :- | | `--api-token ` | The API token to use for authenticating against the PlanetScale API. | | `--api-url ` | The base URL for the PlanetScale API. Default is `https://api.planetscale.com/`. | | `--config ` | Config file. Default is `$HOME/.config/planetscale/pscale.yml`. | | `--debug` | Enable debug mode. | | `-f`, `--format ` | Show output in a specific format. Possible values: `human` (default), `json`, `csv`. | | `--no-color` | Disable color output. | | `--service-token ` | The service token for authenticating. | | `--service-token-id ` | The service token ID for authenticating. | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cloudflare Source: https://planetscale.com/docs/connect/cloudflare Create and pay for PlanetScale Postgres or MySQL databases from the Cloudflare dashboard without adding billing details to PlanetScale. Vitess and Postgres You can create PlanetScale Postgres or MySQL databases directly from the Cloudflare dashboard and pay for usage through your [Cloudflare account](https://developers.cloudflare.com/fundamentals/setup/account/create-account/). Resources are provisioned in your PlanetScale account with normal dashboard access. Your billing page in the PlanetScale dashboard will indicate that some billing is handled through Cloudflare. ## When to use this Use this flow when you want a PlanetScale database for [Cloudflare Workers](https://developers.cloudflare.com/workers/) and [Hyperdrive](https://developers.cloudflare.com/hyperdrive/) with billing unified in Cloudflare. If you already have a PlanetScale database, or prefer to manage billing on PlanetScale directly, see [Connect an existing database](#connect-an-existing-database) below. ## Before you begin Before you can create a Cloudflare-billed PlanetScale database, you'll need: * A [Cloudflare account](https://developers.cloudflare.com/fundamentals/setup/account/create-account/) * A PlanetScale account (you can create one during setup) ## Create a database from Cloudflare **Step 1** Log in to the Cloudflare dashboard and navigate to **Storage & databases** > **Postgres & MySQL (Hyperdrive)**. ![Create a PlanetScale database from the Cloudflare dashboard](https://mintcdn.com/planetscale-2/G873q0u1kC24Q7We/connect/assets/cf_create_planetscale_db-light.png?fit=max\&auto=format\&n=G873q0u1kC24Q7We\&q=85\&s=1a44f9fcffb3dfe41122ff3e0e63b001) **Step 2** Click **Create PlanetScale database**. **Step 3** Under **New PlanetScale database**, select your PlanetScale organization. ![Select your PlanetScale organization in the Cloudflare dashboard](https://mintcdn.com/planetscale-2/G873q0u1kC24Q7We/connect/assets/cf_select_planetscale_db-light.png?fit=max\&auto=format\&n=G873q0u1kC24Q7We\&q=85\&s=d51eddc2f1e834399cc34e02cf6df5d5) **Step 4** Click **Create database** to be redirected to the PlanetScale dashboard to finish configuration. ## Finish setup on PlanetScale After Cloudflare redirects you to PlanetScale, configure your database: * Database name * Engine (Postgres or MySQL/Vitess) * Region * Cluster size When you create the database, PlanetScale automatically associates it with your Cloudflare account. You have the same access to PlanetScale features as normal, including the MCP server, branching, query insights, roles, and backups. ## Billing PlanetScale usage for databases created through Cloudflare is billed at [standard PlanetScale pricing](https://planetscale.com/pricing) through your Cloudflare account. To confirm billing is routed through Cloudflare, go to your PlanetScale organization **Settings** > **Billing**. You'll see a banner indicating that some billing is handled through Cloudflare, with a link to your Cloudflare billing page. ![PlanetScale billing page showing some billing through Cloudflare](https://mintcdn.com/planetscale-2/G873q0u1kC24Q7We/connect/assets/ps_paid_via_cloudflare-light.png?fit=max\&auto=format\&n=G873q0u1kC24Q7We\&q=85\&s=f86a8415704451b4c27b9a0bca3976db) ## Connect Hyperdrive Once your database is ready, a banner at the top of the PlanetScale dashboard directs you back to the Cloudflare dashboard to set up Hyperdrive. ![PlanetScale dashboard banner directing to Cloudflare Hyperdrive setup](https://mintcdn.com/planetscale-2/G873q0u1kC24Q7We/connect/assets/ps_hyperdrive_banner-light.png?fit=max\&auto=format\&n=G873q0u1kC24Q7We\&q=85\&s=ed5cdcc4e668e6e2a03461068a8a7bc1) Hyperdrive connects your Worker to PlanetScale with connection pooling and query caching. For step-by-step Hyperdrive setup instructions, see: * [PlanetScale Postgres with Cloudflare Workers](/docs/postgres/tutorials/planetscale-postgres-cloudflare-workers) * [Cloudflare Workers database integration (Vitess/MySQL)](/docs/vitess/integrations/cloudflare-workers) For Cloudflare's own documentation, see [PlanetScale on Cloudflare Workers](https://developers.cloudflare.com/workers/databases/third-party-integrations/planetscale/). ## Connect an existing database You can still connect an existing PlanetScale database to Hyperdrive without using Cloudflare billing. This is the right path if you have already created a database on PlanetScale and want to link it to a Hyperdrive configuration. In the Cloudflare Hyperdrive wizard, choose **Connect to PlanetScale database** and sign in to your existing PlanetScale account. See the integration guides for full setup instructions: * [PlanetScale Postgres with Cloudflare Workers](/docs/postgres/tutorials/planetscale-postgres-cloudflare-workers) * [Cloudflare Workers database integration (Vitess/MySQL)](/docs/vitess/integrations/cloudflare-workers) ## Further reading * [PlanetScale for Cloudflare users](https://planetscale.com/cloudflare) — overview of the PlanetScale + Cloudflare stack * [Cloudflare Hyperdrive documentation](https://developers.cloudflare.com/hyperdrive/) * [Faster PlanetScale Postgres connections with Cloudflare Hyperdrive](https://planetscale.com/blog/cloudflare-hyperdrive-real-time) — build a real-time application with PlanetScale and the Cloudflare global network ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Stripe Projects Source: https://planetscale.com/docs/connect/stripe-projects Stripe Projects allows you to provision and manage PlanetScale Postgres and MySQL/Vitess databases directly from the Stripe CLI with the Stripe Projects developer preview. Vitess and Postgres Resources are provisioned in your PlanetScale account with normal access and dashboards. Your billing page in the PlanetScale dashboard will indicate that you are connected to the Stripe app and payments are made through Stripe. > [!NOTE] > > [Stripe Projects](https://docs.stripe.com/projects) is currently in **developer preview**. You can request access to try it out [here](https://projects.dev/). ## Before you begin Before you can use PlanetScale with the Stripe CLI, you'll need: * [Access](https://projects.dev/) to the developer preview * A Stripe account * A PlanetScale account (or Stripe Projects can create one for you) * The [Stripe CLI](https://docs.stripe.com/stripe-cli) installed and up to date * The Projects plugin installed: ```bash theme={null} stripe plugin install projects ``` ## Quickstart Initialize Stripe Projects in your app directory, add a PlanetScale database, and sync credentials to your local environment: ```bash theme={null} # Enter into your app directory cd my-app # Initialize a new Stripe project stripe projects init # Add a PlanetScale Postgres database stripe projects add planetscale/postgresql # Or add a PlanetScale MySQL database stripe projects add planetscale/mysql # Sync credentials to your local .env file stripe projects env --sync ``` After syncing, your `.env` file will be updated with your PlanetScale database connection string. ## Add a database When you run `stripe projects add`, you can sign in to your existing PlanetScale account or let Stripe Projects create a new one. You'll be prompted to configure: * Database name * Cluster size * Region * Number of replicas Once confirmed, your database is provisioned and credentials are returned in an agent-readable format. ## Manage credentials List your project's environment variables (values are not shown): ```bash theme={null} stripe projects env list ``` Sync credentials to your local `.env` file: ```bash theme={null} stripe projects env --sync ``` Rotate credentials for your PlanetScale database: ```bash theme={null} stripe projects rotate planetscale/postgresql stripe projects rotate planetscale/mysql ``` ## Check project status Review your project's linked providers, provisioned services, and health: ```bash theme={null} stripe projects status ``` ## Open your PlanetScale dashboard ```bash theme={null} stripe projects open planetscale ``` ## Remove a database > [!WARNING] > > This is a destructive operation. Running this command will **delete your PlanetScale database**. ```bash theme={null} stripe projects remove planetscale/postgresql stripe projects remove planetscale/mysql ``` ## Non-interactive and agent use All commands support flags for use in CI/CD pipelines, scripts, and AI agents: | Flag | Description | | - | - | | `--json` | Return output as structured JSON | | `--no-interactive` | Disable prompts; commands fail if input is missing | | `--auto-confirm` | Accept confirmation prompts automatically | | `--quiet` | Suppress non-essential output | Example — add a database with no prompts: ```bash theme={null} stripe projects add planetscale/postgresql \ --no-interactive \ --json ``` ## Further reading For full documentation on Stripe Projects, including billing, LLM context generation, the complete command reference, and regional availability, see the [Stripe Projects documentation](https://docs.stripe.com/stripe-projects). You can also [view PlanetScale in the Stripe Marketplace](https://marketplace.stripe.com/apps/planetscale) to get started. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # How PlanetScale uses AI Source: https://planetscale.com/docs/how-we-use-ai Which PlanetScale features use generative AI, and how we ensure customer data stays secure and private. Vitess and Postgres ## Overview PlanetScale uses generative AI to power two [Insights](/docs/postgres/monitoring/query-insights) features. This overview explains those features and how PlanetScale ensures safety and privacy of customer data. ## Which features use generative AI Two features of [Insights](/docs/postgres/monitoring/query-insights) use generative AI: * Postgres [index recommendations](/docs/postgres/monitoring/schema-recommendations) use a large language model to interpret query patterns and identify potential indexes that could improve performance of those patterns. The prompt sent to the LLM includes one or more query patterns and one or more table schemas. Query patterns and table schemas include the names of tables and columns, but not any actual row data. This process runs automatically on each database each night. * Query summaries for both [Postgres](/docs/postgres/monitoring/query-insights#query-deep-dive) and [Vitess](/docs/vitess/monitoring/query-insights#query-deep-dive) use a large language model to convert a query pattern to a one-sentence description. The prompt sent to the LLM contains a query pattern. This feature is invoked only when a user clicks on the "Summarize query" button on an Insights query-details page. ## Privacy and security All AI use is covered by PlanetScale's existing [terms of service](https://planetscale.com/legal/agreement) and [subprocessor list](https://planetscale.com/legal/subprocessors). Our LLM provider, [Google Gemini](https://cloud.google.com/gemini/docs/discover/data-governance), encrypts all communication in transit, does not use prompts for training, and retains prompts only long enough to screen for abuse. ## Opting out We understand that not every customer wants to use AI. You can disable the LLM-based features for all the databases in your account or organization on the organization settings page. # PlanetScale Source: https://planetscale.com/docs/index PlanetScale is a relational database platform that brings you scale, performance, and reliability — without sacrificing developer experience. We offer [Vitess](/docs/vitess), [Neki](/docs/neki), and [PostgreSQL](/docs/postgres) clusters, powered by locally-attached NVMe drives that deliver unlimited IOPS and ultra-low latency. > **[Vitess](/docs/vitess)** > > Fully-managed Vitess clusters with unlimited scalability: sharding, branching, deploy requests, query insights, and more. > **[Neki](/docs/neki)** > > Horizontal sharding for Postgres: data topology, query routing, online schema changes, and query insights. > **[Postgres](/docs/postgres)** > > Fully-managed PostgreSQL: high availability, query insights, branching, and more. All clusters are deployed with one primary and a minimum of 2 replicas across 3 availability zones for high availability. Automated failovers, managed version upgrades, and a team of database experts holding the pager means you can spend your time shipping instead of worrying about your database. For large-scale applications that are hitting the limits of vertical scale, [Vitess](/docs/vitess) provides horizontal sharding for MySQL-compatible workloads, and [Neki](/docs/neki) provides it for Postgres. You get query routing, built-in connection pooling, and a path to grow beyond a single primary. ![PlanetScale dashboard](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/planetscale-dashboard-darkmode-BDgbDVqL.avif?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=d29993b92a1c6293f3406147f31a21c3) ## PlanetScale plans > **[PlanetScale plans](/docs/planetscale-plans)** > > What's the difference between the Base plan and Enterprise? > **[Deployment options](/docs/plans/deployment-options)** > > PlanetScale offers multi-tenant and single-tenant deployment options. > **[Bring your own cloud](/docs/plans/managed)** > > Learn about our Enterprise offering — PlanetScale Managed. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Model Context Protocol (MCP) Source: https://planetscale.com/docs/mcp-server Connect Claude, Cursor, Notion, and other MCP-compatible tools to your PlanetScale databases and Insights Vitess and Postgres ## What is the PlanetScale MCP server? * A hosted MCP server that accesses your PlanetScale organizations, databases, branches, schema, Insights data, and the organization payment method. * Authenticated via OAuth. [Service tokens](/docs/mcp/service-token) are also supported for CI and headless clients. * Accessible from any MCP client that supports HTTP-hosted servers. [The PlanetScale MCP server: AI for your database](https://www.youtube-nocookie.com/embed/d3bc3PryOfY?rel=0) ## Quick start > **[Cursor](https://cursor.com/en-US/install-mcp?name=PlanetScale\&config=eyJ1cmwiOiJodHRwczovL21jcC5wc2NhbGUuZGV2L21jcC9wbGFuZXRzY2FsZSJ9)** > > One-click install > **[VS Code](vscode:mcp/install?%7B%22url%22%3A%22https%3A%2F%2Fmcp.pscale.dev%2Fmcp%2Fplanetscale%22%2C%22name%22%3A%22PlanetScale%22%2C%22type%22%3A%22http%22%7D)** > > One-click install > **[Claude Code](#claude-code)** > > See instructions > **[OpenCode](#opencode)** > > See instructions > **[Codex](#codex-cli)** > > See instructions > **[Notion](#notion)** > > See instructions > **[Other clients](#installation-instructions)** > > See all options ## Authentication The hosted MCP server uses **OAuth** so you can authorize access to PlanetScale organizations, databases, branches, Insights data, and the organization payment method directly in your client via the MCP server. * Each client (for example, Claude Code, Cursor, or other MCP-compatible tools) registers as an OAuth application with PlanetScale. * When you connect the MCP server from your client, you're redirected to PlanetScale to sign in and grant access. **Server URL:** ``` https://mcp.pscale.dev/mcp/planetscale ``` > [!TIP] > > If you only need access to Insights data and Schema Recommendations (no query execution), use the insights-only server: `https://mcp.pscale.dev/mcp/planetscale-insights-only`. This server excludes the `planetscale_execute_read_query` and `planetscale_execute_write_query` tools. For CI, headless agents, or any client that can send a custom HTTP header, see [MCP service tokens](/docs/mcp/service-token). ## Security and credentials * Each query uses short-lived, ephemeral credentials that are created on demand and deleted immediately after execution. > [!WARNING] > > We advise caution when giving LLMs *write* access to any production database. Always carefully review queries before execution. ### Scopes and permissions OAuth permissions are controlled through scopes: * Scopes define which organizations, databases, branches, or features the MCP server can see. * You choose whether the MCP server has no access, read-only access, or full access to databases at the organization or per-database level. * Payment method access is separate: no access, read-only (view the saved card), or full access (start Checkout and confirm the saved card). Full access is required to update the card. Organization administrators can further limit query access for everyone in the organization. In organization **Settings**, set **MCP query access**: * **No query access.** Members cannot grant read or write query tools. * **Read-only queries.** Members can grant read queries, but not write queries. * **Read and write queries.** No extra limit. This is the default. A member's role still limits which permissions they can grant. The setting applies to new authorizations. MCP clients that were already authorized keep the access they were granted until you revoke it. Schema, Insights, billing, and other non-query tools are not affected. When a query permission is unavailable, the authorization page explains why. This setting does not change [service token](/docs/mcp/service-token) permissions. Most MCP clients provide a way to re-authenticate the MCP server if you need to update your permissions. Service token access is not chosen in the OAuth prompt. Grant it on the token. See [MCP service tokens](/docs/mcp/service-token). ## Installation instructions ### Cursor > **[Cursor](https://cursor.com/en-US/install-mcp?name=PlanetScale\&config=eyJ1cmwiOiJodHRwczovL21jcC5wc2NhbGUuZGV2L21jcC9wbGFuZXRzY2FsZSJ9)** > > Click to install the MCP server configuration for Cursor. Manual installation: 1. Open the command palette and type "Cursor Settings" 2. Under "Tools & MCP" click "New MCP Server" 3. Paste the following JSON into the configuration file that opens ```json .cursor/mcp.json theme={null} { "mcpServers": { "planetscale": { "url": "https://mcp.pscale.dev/mcp/planetscale" } } } ``` 4. Once this configuration is saved, Cursor will attempt to authenticate and show a login prompt. Select this prompt to grant Cursor access to your PlanetScale account. 5. You may need to restart Cursor to load the new configuration. ### VS Code > **[VS Code](vscode:mcp/install?%7B%22name%22%3A%22PlanetScale%22%2C%22type%22%3A%22http%22%2C%22url%22%3A%22https%3A%2F%2Fmcp.pscale.dev%2Fmcp%2Fplanetscale%22%7D)** > > Click to install the MCP server configuration for VS Code. Use the button above to launch VS Code and configure the PlanetScale MCP server automatically. Alternatively, follow the steps below to do it manually: 1. Open the Command Palette (`Ctrl+Shift+P` on Windows/Linux or `Cmd+Shift+P` on macOS) 2. Type "MCP: Add Server" 3. Choose "HTTP" 4. Enter the following details: * URL: [https://mcp.pscale.dev/mcp/planetscale](https://mcp.pscale.dev/mcp/planetscale) * Name: "PlanetScale" 5. Click "Add" **Authorization** After adding the PlanetScale MCP server you will need to start the server and authorize: 1. Open the Command Palette (`Ctrl+Shift+P` on Windows/Linux or `Cmd+Shift+P` on macOS) 2. Type "MCP: List Servers" 3. Select "PlanetScale" 4. Click "Start Server" 5. When the dialog appears saying "The MCP Server Definition 'PlanetScale' wants to authenticate to PlanetScale MCP," click "Allow" 6. A popup will ask "Do you want Code to open the external website?" — click "Cancel" 7. You'll see a message: "Having trouble authenticating to 'PlanetScale MCP'? Would you like to try a different way? (URL Handler)" 8. Click "Yes" 9. Click "Open" and complete the PlanetScale sign-in flow to connect to PlanetScale MCP ### OpenCode OpenCode supports automatic authentication for OAuth-based remote MCP servers. 1. Ensure [OpenCode](https://opencode.ai/) is installed and available in your terminal ```bash theme={null} opencode --version ``` 2. Add the PlanetScale MCP server to your OpenCode configuration file. ```json ~/.config/opencode/opencode.json theme={null} { "$schema": "https://opencode.ai/config.json", "mcp": { "planetscale": { "type": "remote", "url": "https://mcp.pscale.dev/mcp/planetscale" } } } ``` OpenCode configuration files can be stored inside a project or globally, see the [OpenCode configuration file](https://opencode.ai/docs/config/#precedence-order) documentation for more information. 3. Enter the following to authenticate with the PlanetScale MCP server ```bash theme={null} opencode mcp auth planetscale ``` 4. Enter OpenCode ```bash theme={null} opencode ``` 5. List the available MCP servers with the `/mcp` slash command. ```bash theme={null} /mcp ``` You should see the PlanetScale MCP server listed and connected. ```bash theme={null} planetscale connected ``` ### Claude.ai and Claude for desktop The PlanetScale MCP server is available as a connector in the [Claude Connectors Directory](https://claude.com/connectors/planetscale). 1. Navigate to **Settings > Connectors** 2. Click **Browse connectors** 3. Search for "PlanetScale" 4. Click **+** 5. Follow the prompts to authenticate with your PlanetScale account Alternatively, install it as a custom connector: 1. Click **Add custom connector** at the bottom of the section 2. Enter the PlanetScale MCP server URL: `https://mcp.pscale.dev/mcp/planetscale` 3. Click **Add** to save the connector 4. Follow the prompts to authenticate with your PlanetScale account > [!NOTE] > > Custom connectors using remote MCP are not available on all plans. See the [Claude Support page](https://support.claude.com/en/articles/11175166-getting-started-with-custom-connectors-using-remote-mcp) for more information. To use the connector in a conversation: 1. Click the **+** button in the lower left of your chat interface 2. Select **Connectors** 3. Toggle the PlanetScale connector on for that conversation ### Claude Code The PlanetScale MCP server is available as a [Claude Code plugin](https://github.com/planetscale/claude-plugin). You must first add the PlanetScale marketplace, and then install the plugin. ``` /plugin marketplace add planetscale/claude-plugin /plugin install planetscale@planetscale ``` Alternatively, you can add the MCP server directly to your Claude Code configuration: 1. Ensure [Claude Code](https://code.claude.com/docs/en/overview) is available in your terminal ```bash theme={null} claude --version ``` 2. Run the following command to add the PlanetScale MCP server to your Claude Code ```bash theme={null} claude mcp add --transport http "planetscale" https://mcp.pscale.dev/mcp/planetscale ``` 3. Enter Claude Code ```bash theme={null} claude ``` 4. List the available MCP servers with the `/mcp` slash command. ```bash theme={null} /mcp ``` You should see the PlanetScale MCP server listed in need of authentication. Select it, press "Enter" and follow the instructions in the browser to authenticate. ```bash theme={null} ❯ planetscale · △ needs authentication ``` You should see the following message once authenticated: ```bash theme={null} ❯ /mcp ⎿ Authentication successful. Connected to planetscale. ``` ### Claude Managed Agents Connect the PlanetScale MCP server to [Claude Managed Agents](https://platform.claude.com) using a credential vault. 1. Go to [platform.claude.com](https://platform.claude.com) 2. Open **Credential vaults** → **Create vault** → **Add credential** 3. Select **MCP OAuth** 4. Enter the server URL: `https://mcp.pscale.dev/mcp/planetscale` 5. Leave **Access token** and **OAuth client credentials** blank 6. Click **Connect** 7. Complete the PlanetScale sign-in flow when prompted ### Gemini CLI 1. Ensure [Gemini CLI](https://geminicli.com/) is available in your terminal ```bash theme={null} gemini --version ``` 2. Run the following command to add the PlanetScale MCP server to your Gemini CLI ```bash theme={null} gemini mcp add --transport http "planetscale" https://mcp.pscale.dev/mcp/planetscale ``` 3. Enter Gemini CLI ```bash theme={null} gemini ``` 4. List the available MCP servers with the `/mcp` slash command and select "List" ```bash theme={null} /mcp list ``` You should see the PlanetScale MCP server listed in need of authentication. ```bash theme={null} 🔴 planetscale - Disconnected (OAuth not authenticated) ``` Now run the following command to authenticate: ```bash theme={null} /mcp auth planetscale ``` You should see the following message once authenticated: ```bash theme={null} ℹ ✅ Successfully authenticated with MCP server 'planetscale'! ``` ### Codex CLI 1. Ensure [Codex CLI](https://developers.openai.com/codex/cli/) is available in your terminal ```bash theme={null} codex --version ``` 2. Run the following command to add the PlanetScale MCP server to your Codex CLI ```bash theme={null} codex mcp add planetscale --url https://mcp.pscale.dev/mcp/planetscale ``` You should be immediately prompted to authenticate. Follow the instructions in the browser to authenticate. 3. Enter Codex CLI ```bash theme={null} codex ``` 4. List the available MCP servers with the `/mcp` slash command ```bash theme={null} /mcp ``` You should see the PlanetScale MCP server listed, enabled and authenticated. ```bash theme={null} • planetscale • Status: enabled • Auth: OAuth ``` ### Amp CLI 1. Ensure [Amp CLI](https://ampcode.com/) is available in your terminal ```bash theme={null} amp --version ``` 2. Run the following command to add the PlanetScale MCP server to your Amp CLI ```bash theme={null} amp mcp add planetscale https://mcp.pscale.dev/mcp/planetscale ``` 3. Enter Amp CLI ```bash theme={null} amp ``` 4. You should be immediately prompted to authenticate. Follow the instructions in the browser. 5. List the available MCP servers with the `/mcp list tools` command. ```bash theme={null} /mcp list tools ``` You should see the PlanetScale MCP server listed and its available tools. ### Notion You can access the PlanetScale MCP server from inside a Notion Custom Agent. 1. Create a new Notion Agent or open an existing one 2. In the Custom Agent open "Settings" and scroll to "Tools and access" 3. Click "Add connection" 4. Insert the MCP server URL: `https://mcp.pscale.dev/mcp/planetscale` 5. Ensure Authentication is set to OAuth 6. Click "Connect" 7. Complete the authentication process to give the Notion Custom Agent access to specific organizations and databases ## Help your agent target your project's database By default, the MCP server is likely to scan through all your organizations and databases to find those relevant to your query. If you are working within a project that targets a single database, it's a good idea to include this information in your `AGENTS.md` file to skip these tool calls. For example: ```markdown AGENTS.md theme={null} ## PlanetScale Database - **Organization:** `my-organization` - **Database:** `my-database` - **Branch:** `main` ``` ## Query handling All queries execute over HTTP via the hosted MCP server. This makes the MCP server useful in environments that normally cannot run database queries directly, such as browser-based tools or sandboxed AI agents. The PlanetScale MCP server includes several built-in behaviors to help ensure safe, observable, and performant query execution. ### Read queries * Read queries route to a replica when your branch has replicas configured, reducing load on the primary. This is the default behavior. * Set `use_replica` to `false` on `planetscale_execute_read_query` to run against the primary instead. Works for Vitess, Neki, and Postgres databases. * All queries include a `source=planetscale-mcp` comment, making them easy to identify and track in [Insights](/docs/what-is-planetscale#insights). #### Postgres row-level security For Postgres databases, read queries run with a short-lived role that has `pg_read_all_data`. This role does not bypass row-level security (RLS). When a read query returns zero rows or a zero count (such as `SELECT COUNT(*)` returning 0), the MCP server checks whether RLS is active on tables you can access. If so, the response includes a `warnings` array with code `postgres_rls_active`, listing the affected relations. A zero-row result does not necessarily mean the table is empty — RLS may be hiding rows from the MCP role. ### Write query safeguards To help prevent accidental data loss, the MCP server blocks certain operations: * `UPDATE` or `DELETE` statements without a `WHERE` clause are blocked. * `TRUNCATE` statements are blocked. * DDL statements (`CREATE`, `DROP`, `ALTER`, etc.) prompt the LLM to request human confirmation before proceeding. ### Postgres database targeting For PlanetScale Postgres databases, the MCP server connects to the default `postgres` database by default. If you have created additional databases in the same cluster using `CREATE DATABASE`, you can specify which database to query using the `postgres_database_name` parameter. This parameter is available on both the `planetscale_execute_read_query` and `planetscale_execute_write_query` tools. When omitted, queries run against the default `postgres` database for your branch. ## Example workflows Once installed, you can ask your MCP-enabled editor or agent to: * "Show me all databases in my PlanetScale organization and highlight anything running on PlanetScale Metal." * "List the branches for my production database and summarize their differences." * "Look at my slowest queries over the last day and suggest index or query changes." * "Which route tag is spending the most database time?" * "Check whether the CPU and memory profile for my database is appropriate for the current workload." * "Is my main branch full, lagging, or close to its connection limit?" * "Explain what changed between yesterday's and today's query patterns in Insights." You can also run these workflows on a recurring schedule using [Cursor Automations](https://cursor.com/docs/cloud-agent/automations). See [Self-improving database](/docs/self-improving-database) for ready-to-use prompts. ## Available tools The hosted PlanetScale MCP server includes a curated set of tools designed for day-to-day database exploration, debugging, and insights. The available tools are: 1. `planetscale_list_organizations` - List all PlanetScale organizations you have access to. 2. `planetscale_list_databases` - List all databases within an organization. 3. `planetscale_list_branches` - List all branches within a database. 4. `planetscale_get_organization` - Get details about a specific organization. 5. `planetscale_get_database` - Get details about a specific database. 6. `planetscale_get_branch` - Get details about a database branch. 7. `planetscale_get_branch_schema` - Get the schema for a database branch. 8. `planetscale_execute_read_query` - Execute a read-only SQL query (SELECT, SHOW, DESCRIBE, EXPLAIN) against a PlanetScale database. Optionally accepts `use_replica` to route to a replica (default) or primary. For Postgres, optionally accepts `postgres_database_name` to target a non-default database. Returns RLS warnings when policies may filter results. 9. `planetscale_execute_write_query` - Execute a write SQL query (INSERT, UPDATE, DELETE, or DDL) against a PlanetScale database. For Postgres, optionally accepts `postgres_database_name` to target a non-default database. 10. `planetscale_get_insights` - Get query performance insights for a PlanetScale database branch. 11. `planetscale_list_regions_for_organization` - List all available regions for an organization. 12. `planetscale_list_cluster_size_skus` - List all available cluster size SKUs. 13. `planetscale_list_invoices` - List all invoices for an organization. 14. `planetscale_get_invoice_line_items` - Get all line items for an invoice, with prorated costs broken down by database branch. 15. `planetscale_get_organization_billing_payment_method` - Show the organization's current payment method. 16. `planetscale_update_payment_method` - Start Stripe Checkout to add or replace the organization's card. Returns a `checkout_url` and `setup_id`. Requires Payment method full access. 17. `planetscale_get_payment_method_setup` - Check a Checkout setup by `setup_id`. Does not poll; do not start a second update while one is pending. 18. `planetscale_search_documentation` - Search the PlanetScale documentation. 19. `planetscale_list_schema_recommendations` - List schema recommendations for a database, including suggestions for adding indexes, removing redundant indexes, and preventing primary key exhaustion. 20. `planetscale_list_query_error_patterns` - List failing queries for a database branch, aggregated by error fingerprint, with the error message, occurrence count, last run, and total and average duration. Optionally search by error text, sort, and filter by tablet type. 21. `planetscale_list_query_error_executions` - List the individual failed executions behind an `error_fingerprint` from `planetscale_list_query_error_patterns`, including the normalized SQL, tables, keyspace, user, row counts, duration, and query tags. 22. `planetscale_get_postgres_logs` - Fetch server logs for a Postgres database branch, newest first. Filter by log level, time range, server role (primary or replica), and pod name, or pass a raw LogsQL query for advanced filtering. Postgres-only; not available for Vitess databases. 23. `planetscale_list_query_tags` - List the [query tags](/docs/postgres/monitoring/query-tags) seen on a database branch's queries, with their values and query counts. Optionally filter by tag name, fingerprint, keyspace, or tablet type. 24. `planetscale_get_query_tag` - Get a single query tag and its values, using a tag ID from `planetscale_list_query_tags`. 25. `planetscale_list_query_tag_summaries` - Group query statistics by the values of one or more query tags, attributing total time, latency, rows read, and errors to an application, route, job, or user. Takes tag IDs from `planetscale_list_query_tags`. 26. `planetscale_get_branch_metrics` - Infrastructure metrics for a branch: CPU, memory, disk, replication lag, and connections, for Vitess, Postgres, and Neki. The default summary covers the last hour (latest, min, average, p95, and max) plus current disk and connection counts. Memory utilization includes page cache, so the summary also includes RSS and cache. Use `view` `series` or `instant` to drill into specific metrics. Query stats stay on `planetscale_get_insights`. The MCP server tools are [open source and available on GitHub](https://github.com/planetscale/mcp-server). ## Troubleshooting If your MCP client cannot connect or tools fail to run: 1. **Restart your client or CLI tool** After installation, you may need to restart your client or CLI tool to load the new MCP configuration. 2. **Check the server URL** Make sure the MCP server URL is set to the hosted endpoint (for example, `https://mcp.pscale.dev/mcp/planetscale`). 3. **Re-authorize the MCP server** If scopes or tokens have changed, reauthorize the PlanetScale MCP server in your client so it can request fresh tokens. 4. **Verify organization and database access** Confirm that your PlanetScale user account has access to the orgs and databases you expect to see. 5. **Using a service token** If you authenticated with a service token instead of OAuth, see [MCP service tokens](/docs/mcp/service-token). ## PlanetScale CLI MCP server The local-only `pscale mcp` MCP server has been removed. Use the hosted MCP server documented above. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Authenticate the MCP server with a service token Source: https://planetscale.com/docs/mcp/service-token Use a PlanetScale service token to authenticate the hosted MCP server in CI, headless agents, and clients that send a custom HTTP header Vitess and Postgres OAuth is the default for the [hosted MCP server](/docs/mcp-server). Use a [service token](/docs/api/service-tokens) when you cannot complete a browser login, such as CI or a headless agent. The hosted MCP server accepts the token secret as a Bearer token. 1. Create a service token in your organization settings. 2. Copy the **token** value. It starts with `pscale_tkn_`. Do not use the token ID, and do not send `id:secret`. 3. Grant only the [organization and database permissions](/docs/api/service-tokens#assign-service-token-permissions) you want the MCP tools to have. Leave the rest off. 4. Set `PLANETSCALE_API_TOKEN` to that token value in the environment where the client runs. 5. Send it as `Authorization: Bearer pscale_tkn_...` to `https://mcp.pscale.dev/mcp/planetscale`. The same header works for the insights-only server: `https://mcp.pscale.dev/mcp/planetscale-insights-only`. > [!WARNING] > > Use the token secret only. `Authorization: Bearer :` is rejected. Do not commit the token in a config file. Claude.ai, Claude for desktop, Claude Managed Agents, and Notion authenticate with OAuth only. They do not accept a service token header. ## Permissions The organization **MCP query access** setting applies when someone authorizes an MCP client with OAuth. It does not limit a service token. The token's own permissions do. The MCP server calls the PlanetScale API with the token you send. Each tool fails if the token is missing the permission for that API call. Grant the smallest set that covers the tools you want. Add organization permissions on the token, then add database permissions on each database the token should see, or use [permissions for all databases](/docs/api/service-tokens#add-permissions-for-all-databases). ### Read-only access This covers listing organizations, databases, branches, schema, Insights, schema recommendations, and branch metrics. It does not allow SQL execution. **Organization** * `read_organization` * `read_databases` **Database** * `read_database` * `read_branch` `read_database` returns metadata about a database (name, state, default branch). It does not connect to MySQL or Postgres, and it does not run queries. Connecting and executing SQL requires the `connect_*` permissions below. `read_databases` lets the token list every database in the organization. If you only want one database, skip `read_databases` and grant `read_database` plus `read_branch` on that database only. Add `read_invoices` on the organization if you want the invoice tools. Add `read_payment_method` to view the saved card, and `write_payment_method` to start Checkout or confirm a setup. ### Read queries Add these database permissions on top of the read-only set. The MCP server creates a short-lived reader password or Postgres role, runs the query, then deletes the credential. * `connect_branch` (development branches) * `connect_production_read_only_branch` (production branches) * `delete_branch_password` (development branches) * `delete_production_read_only_branch_password` (production branches) If you only query production, you can leave off `connect_branch` and `delete_branch_password`. If you only query development branches, leave off the production pair. `delete_production_branch_password` also works for deleting reader credentials on production. Do not grant `connect_production_branch`. That permission can create write credentials on production. ### Write queries `planetscale_execute_write_query` creates an admin or write role, so it needs the full connect permissions, not the read-only ones. Add these on top of the read-only set: * `connect_branch` * `connect_production_branch` * `delete_branch_password` * `delete_production_branch_password` The full connect permissions can also run read queries. You do not need the read-only connect pair if you already grant these. > [!WARNING] > > Write query permissions can run `INSERT`, `UPDATE`, `DELETE`, and DDL. Only grant them on databases where that is acceptable. ### Tool permissions | Tools | Organization permissions | Database permissions | | - | - | - | | `planetscale_list_organizations` | None. Any valid token can list the organization it belongs to. | | | `planetscale_get_organization`, `planetscale_list_regions_for_organization`, `planetscale_list_cluster_size_skus` | `read_organization` | | | `planetscale_list_databases`, `planetscale_get_database` | `read_databases` to see every database. Otherwise grant `read_database` on each database. | `read_database` | | `planetscale_list_branches`, `planetscale_get_branch`, `planetscale_get_branch_schema`, `planetscale_get_branch_metrics` | | `read_branch` | | `planetscale_get_insights`, `planetscale_list_schema_recommendations` | `read_databases` also works in place of `read_database` | `read_database` and `read_branch` | | `planetscale_list_invoices`, `planetscale_get_invoice_line_items` | `read_invoices` | | | `planetscale_get_organization_billing_payment_method` | `read_payment_method` | | | `planetscale_update_payment_method`, `planetscale_get_payment_method_setup` | `write_payment_method` | | | `planetscale_execute_read_query` | | Read-only set, plus the read-query connect and delete permissions above | | `planetscale_execute_write_query` | | Read-only set, plus the write-query connect and delete permissions above | | `planetscale_search_documentation` | None | None | ## Client configuration **Cursor** Add the header to `.cursor/mcp.json` or `~/.cursor/mcp.json`. Cursor and the Cursor CLI (`agent`) read `PLANETSCALE_API_TOKEN` from the environment at connect time. ```json .cursor/mcp.json theme={null} { "mcpServers": { "planetscale": { "url": "https://mcp.pscale.dev/mcp/planetscale", "headers": { "Authorization": "Bearer ${env:PLANETSCALE_API_TOKEN}" } } } } ``` **Claude Code** ```bash theme={null} claude mcp add --transport http "planetscale" https://mcp.pscale.dev/mcp/planetscale \ --header "Authorization: Bearer ${PLANETSCALE_API_TOKEN}" ``` The shell expands the variable when you run the command, so the token is written into your Claude MCP config. Don't commit that file. **Codex** Codex reads the env var name and sends it as a Bearer token. The variable should be the `pscale_tkn_...` secret, not `Bearer ...`. ```bash theme={null} codex mcp add planetscale \ --url https://mcp.pscale.dev/mcp/planetscale \ --bearer-token-env-var PLANETSCALE_API_TOKEN ``` Or in `~/.codex/config.toml`: ```toml theme={null} [mcp_servers.planetscale] url = "https://mcp.pscale.dev/mcp/planetscale" bearer_token_env_var = "PLANETSCALE_API_TOKEN" ``` **VS Code** Add the header to `.vscode/mcp.json`. VS Code reads `PLANETSCALE_API_TOKEN` from the environment at connect time. ```json .vscode/mcp.json theme={null} { "servers": { "planetscale": { "type": "http", "url": "https://mcp.pscale.dev/mcp/planetscale", "headers": { "Authorization": "Bearer ${env:PLANETSCALE_API_TOKEN}" } } } } ``` **Gemini CLI** ```bash theme={null} gemini mcp add --transport http \ --header "Authorization: Bearer ${PLANETSCALE_API_TOKEN}" \ planetscale https://mcp.pscale.dev/mcp/planetscale ``` The shell expands the variable when you run the command, so the token is written into your Gemini MCP config. Don't commit that file. **OpenCode** Set `oauth` to `false` so OpenCode uses the header instead of starting a browser login. ```json ~/.config/opencode/opencode.json theme={null} { "$schema": "https://opencode.ai/config.json", "mcp": { "planetscale": { "type": "remote", "url": "https://mcp.pscale.dev/mcp/planetscale", "oauth": false, "headers": { "Authorization": "Bearer {env:PLANETSCALE_API_TOKEN}" } } } } ``` ## Troubleshooting If a tool returns `invalid_token`, the header is the wrong shape. Send `Authorization: Bearer pscale_tkn_...` (the token secret only). The token ID, or `id:secret`, is rejected. If a tool returns permission denied, the token is missing the grant for that API call. Add the permission from the table above. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Metal Source: https://planetscale.com/docs/metal PlanetScale Metal databases are the same PlanetScale databases you know and love, powered by blazing-fast, locally-attached NVMe SSD drives instead of network-attached storage. Vitess, Neki, and Postgres ![Metal SSD](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/metal/metal.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=29fca543db198da06b42f346f7e8f5ea) This translates to significant latency reduction, more consistent IO performance, and unlimited I/O Operations Per Second (IOPS). Metal is an excellent choice for high-IOPS and other performance-critical workloads. With Metal, your database now has the ability to use modern NVMe SSD technology to its full potential. > [!TIP] > > Metal nodes for Postgres now [start at \$50/month](https://planetscale.com/blog/50-dollar-planetscale-metal-is-ga-for-postgres). ## Using a Metal database PlanetScale databases come in two main flavors: **Metal** and **network-attached storage**. When you create a database on PlanetScale, you can choose between these two options: ![Choose between network attached storage and Metal](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/metal/nas-and-metal.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=d5c52a5146457cd2bb41df0d411a3669) > [!WARNING] > > The storage for Metal databases does not autoscale. > It is important to keep a close eye on the storage capacity of Metal databases, and upgrade well before running out of space. When you create a Metal database, you must choose a drive size up front. ![Select storage drive size for a Metal database](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/metal/metal-drive-size.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=4619ff56ac0b7927853673027c817f86) You should select a drive size that best suits your current data size, while also taking into account growth trends. When the time comes that you need more storage, we make it easy to upgrade to larger NVMe drives with just a few clicks of a button. You can learn more about creating and resizing Metal databases in our [creation and upgrade documentation](/docs/metal/create-a-metal-database). ## Monitoring your storage Fixed-sized drives also means that you must closely monitor how much storage your database is using. You can do so by looking at the storage information on the PlanetScale dashboard: ![Storage indicator in PlanetScale](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/metal/storage.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=83ff544d4bf373bbc2e1f31001cd6434) We will send you email notices when your database storage reaches the following thresholds: 60%, 75%, 85%, 90%, 95%. We will also email you when we estimate that your storage will run out in 1 week and 24 hours, based on recent usage trends. Reaching or getting close to a drive's max capacity is dangerous and can lead to failures. It's important to closely monitor your database's disk usage in the dashboard and check your regular storage email notifications. We have an additional safeguard in place to protect your data: When we detect that your Metal disk has 6GB or less of available space, we automatically switch it to read-only mode until resized. Ideally, you should resize to a larger drive long before reaching this point. ## Workload suitability Using Metal is a clear win for many workloads. Many of our current Metal customers have been able to either (A) save money, (B) increase performance, or (C) do both at the same time by switching to Metal. The per-GB cost of metal storage is more affordable than network-attached storage with high IOPS capacity, and the improved IO performance allows you to use smaller compute instances in some cases. There are some scenarios where a network-attached storage database may be a better choice. Here, we provide some general suggestions to help you choose the ideal type of database for your needs. If you'd like a more personalized assessment, please [reach out to support](https://planetscale.com/contact?initial=support) with the specifics of your workload. Generally, these types of database workloads are ideal for Metal: * If your workload has significant I/O demands, Metal is an ideal choice. A network-attached storage database has limitations on how quickly it can read and write data due to the additional network hops. Metal databases allow you to unlock the full potential of modern NVMe technology, providing ultra-high throughput. * If you have experience running up against the limits of AWS EBS IOPS or have a large `gp3` or `io2` `EBS` volume bill. Metal provides unlimited IOPS and will likely yield performance improvement, cost savings, or both. There is no need to pay extra for access to the I/O throughput of the local drive. * If low-latency database performance is critical to your business needs. * If you are concerned about long-tail p99+ performance. However, there are some scenarios where choosing network-attached storage may still be preferable: * Very small databases that have both low compute and low storage requirements may be better suited for network-attached storage. * Databases where the majority of active rows fit in RAM. In this case, the I/O demand on the storage is probably low, and you won't see as much of a performance boost by using Metal. * If you frequently resize your database, Metal may not be the best option. Metal instance resizes generally take longer than a network-attached storage resize as they require copying data between drives. ## Metal Performance We've mentioned several times that Metal can provide you better performance. What does that look like in practice? Let's look at two examples. ### PlanetScale Insights Below is a screenshot showing the p50 and p95 response times for the database that was powering PlanetScale Insights as of Q4 2024. ![The effect of Metal on the Insights database](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/metal/metal-insights-p50-p95.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=72e13b6aa3194554e9df33b8c4f0784a) You can pretty clearly see when the database was switched over from network-attached storage instance types to Metal. The p50 response times were cut in half, and the p95 had approximately a 7x improvement. The workload was and continues to be very I/O bound. The Insights database ingests a large amount of time-series data, and is frequently queried to pull the data that we use to generate graphs in Insights. Metal provides a huge improvement for this type of workload. ### Large, sharded database One of our existing customers runs several large, sharded databases. We migrated these databases to Metal during the internal release of our product. Below is a screenshot of the p99 latencies of a set of shards that we migrated from network-attached storage database to Metal instances. ![The effect of Metal on a large sharded database](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/metal/customer-p99.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=a84d3817e70911f12f7efeb5f16d7fae) Though their p99 response times were already very good, Metal was able to further cut it in half. ### Cost and performance We migrated yet another large customer during our internal release. After switching to Metal, they saw a significant improvement in API call latency for one of their critical APIs. ![API latency improvement](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/metal/customer-api-improvement.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=6c4e45811787200857a4ec36facc8625) We can clearly see that starting on Dec 20, the long tail of latency was reduced significantly. This is due to the lower latency and improved consistency of local NVMe disk performance. This same customer also saw some significant cost savings with their move to Metal. ![Cost savings with Metal](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/metal/customer-pricing-drop.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=42a404dc0d886e8a93647798ed63e609) The performance of the database improved and the AWS costs to run dropped from over \$100 per day to \~\$30 per day. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Create a Metal database Source: https://planetscale.com/docs/metal/create-a-metal-database Create a new Metal database or upgrade an existing one to locally attached NVMe storage. Vitess, Neki, and Postgres [PlanetScale Metal](/docs/metal) databases can be created in a similar way to other PlanetScale databases. However, there are a few important things to keep in mind when creating a new Metal database or upgrading an existing database to Metal, which will be covered here. [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ## Create a new Metal database After logging in to `app.planetscale.com`, click "New database" -> "Create new database." Next, enter the name of your new database and select the "PlanetScale Metal" option. ![New Metal database](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/metal-new-db.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=7510bcae08556f479cc2430f39bddce4) This brings up a set of options to choose from for the size of your Metal database. Start by choosing the vCPU and RAM combination that best suits your needs, then use the dropdown to select the drive size for the instance. ![Select Metal database size](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/metal-new-db-choose-size.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=bdffd572677fed8825cad1116d40f4b5) As opposed to [network-attached storage](/docs/plans/planetscale-skus#network-attached-storage) databases, [Metal](/docs/plans/planetscale-skus#metal) databases do not autoscale their storage size. Therefore, it's important to make a good size choice from the start. If you are starting a new project from scratch on a Metal database and you do not expect massive initial growth, it is likely best to choose the smallest drive possible. If you intend to migrate an existing database into this in the near future, ensure that your drive will fit all of the data while also allowing room for further growth. When ready, click "Create database." After database initialization completes, you can begin using the database. ## Upgrading an existing database to Metal You can also upgrade an existing database cluster to Metal. This is a no-downtime operation. **Step 1** Select your database **Step 2** Click on the "**Clusters**" in the navigation pane on the left side of the dashboard **Step 3** Click on the cluster size drop-down and scroll down to the Metal instance types **Step 4** Select the desired compute and storage combination, and then click "Save changes." ![Upgrade cluster to Metal](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/metal-upgrade.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=7089d1bff5b6692712aadd01cab5bbf9) Keep in mind that this is not an immediate operation. If you have a large database, it may take a while for the upgrade to complete since behind the scenes, your entire database needs to be migrated to the new NVMe drives. Ensure that you upgrade well before reaching max drive capacity. We recommend upgrading at no later than 75% in most cases, and even earlier than that if you are growing quickly. ## Monitoring Metal storage > [!WARNING] > > The storage for Metal databases does not autoscale. > It is important to keep a close eye on the storage capacity of Metal databases, and upgrade well before running out of space. There are several ways to monitor this. You can view storage information on the right side of the main PlanetScale dashboard for Metal databases. After the upgrade is complete, we recommend going to the Insights page and view your query latency diagrams. If you transitioned from a network-attached storage cluster to a Metal one of the same size, you should see a reduction in query latency. ![Metal storage info](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/storage.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=8168dd59a52845461aaf50ca1dff8f27) You should make a habit of regularly logging in and checking the health of your database, keep an eye on this number. If PlanetScale detects that you have only 6GB or less of available storage, it will cause your database to reject writes, preferring to keep the database available rather than cause a total system failure due to running out of storage. This is a safety measure put in place to protect your data. You should upgrade to a larger instance long before reaching this point. You can upgrade to a larger Metal instance / drive using the same set of steps described above. Additionally, operations such as deploy requests may not run if you do not have enough storage. The exception to this is if you are performing an [instant deployment](/docs/vitess/schema-changes/deploy-requests#instant-deployments). Deploying online schema changes with VReplication requires that we [make a copy of the affected tables](/docs/vitess/schema-changes/how-online-schema-change-tools-work#initializing-the-ghost-table-schema). If you are nearing max capacity or making a change on a very large table, you risk not having enough storage to begin the online schema change. We will let you know that there is not enough space to create a deploy request in these cases. It is critical to upgrade your instance storage size well before you are nearing max capacity. We will sent you email notices when your database storage reaches the following thresholds: 60%, 75%, 85%, 90%, 95%. We will also email you when we estimate that your storage will run out in 1 week and 24 hours, based on recent usage trends. The exact point at which you should upgrade depends on your data growth rate, drive size, and other factors. We recommend upgrading no later than at 75% capacity, and even before that in some cases of fast growth. Upgrading to a larger drive takes time, as it requires copying your database to new drives, so it's important to upgrade well before hitting max capacity. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # How to check performance after upgrading to Metal Source: https://planetscale.com/docs/metal/metal-performance PlanetScale [Metal](/metal) databases offer performance benefits for many types of workloads. Vitess, Neki, and Postgres ## Upgrading to Metal When upgrading your existing PlanetScale database to Metal, it's useful to know where to look to see how the upgrade has improved performance. Here, we cover the two main places you can inspect: Insights and the database metrics panel. > [!NOTE] > > The examples shown on this page use a Vitess database, but Metal is available for Vitess, Neki, and Postgres databases. In this page, we will show the results of upgrading from a `PS-640` to an `M-640`. These both have 8 vCPUs and 64GB of RAM, but use a different underlying storage system that allows for the `M-640` to have improved performance. Let's look at the effects of this in PlanetScale. ## Insights After upgrading to Metal, Insights is a great place to look to see how performance has changed. When you visit the Insights page, there are several tabs you can use to view graphs for different statistics. The first and default one is a query latency chart. ### Query latency After zooming in to the time period when we upgraded to Metal, here is what we see: ![Insights query latency](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/insights-p50-p95-p99.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=55387327f20323aed172a29afd9ab3a0) Insights shows indicators on cluster resize and VTGate resize events. We can see in the graph above the purple vertical line indicates when we upgraded to Metal (plus or minus a few minutes). This clearly correlates with an order-of-magnitude or more improvement in p95, and p99 query latencies. You can also click the "p99.9" icon to enable this additional metric, and see yet another significant improvement. ![Insights query latency](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/insights-p50-p95-p99-p99.9.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=893a0e726f768025c815afec1de833cb) ### Anomalies The anomalies graph shows the % of queries that are considered anomalous, or slow-running. ![Insights anomalies](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/insights-anomalies.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=d4b08adc924d5fa0936b04ae32f0ece1) We already had a very small number (less than 0.3%) but even so we see an improvement. ### Queries ![Insights queries](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/insights-qps.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=0e1de73bcda719baa1518378f3ca1d2f) We also see a drastic improvement in average QPS. Prior to switching over we were averaging around 2k queries per second, and after the average is closer to 14k QPS. At the moment PlanetScale switched your traffic from the non-Metal database to the new Metal one, queries are buffered for a short span of time (seconds) to allow Vitess to do the failover to the new Metal primary and replicas. After the failover completed, the buffered queries are quickly pushed to the database to be fulfilled. This is a zero-downtime operation, but you may see a short spike in QPS or latency when the failover occurs. In this example, we are using synthetic TPCC load. In a real production workload you likely would not see such a large QPS jump, as that is dependant on demand from connected application servers. This similarly applies for row reads and writes below. However, what you may find is that an instance size with the same compute specs is able to handle more QPS load before needing to upgrade. The lower IO latency and unlimited IOPS allow you to squeeze more work out of a given node in your database cluster. ### Rows read This is the rows read graph: ![Insights rows read](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/insights-rows-read.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=4f8c887d32f2c96af9063962070e8468) We can definitely see that there was an increase from the pre-Metal rows read. However, we also see a jump for a short burst before the switch over to Metal. This increase appears due to the way upgrading to Metal works. When you upgrade a database to Metal, the (greatly simplified) steps that happen in the background are: * (A) PlanetScale spins up three new NVMe-backed instances for you (one primary and two replicas) * (B) Once these are up and running with all the necessary MySQL and Vitess components, we begin copying the data from your existing database onto the three NVMe drives, starting with restoring from the most recent backup. * (C) Once the backup is restored, we catch the backup up to the current state of your other database. This can be as quick as a few minutes or seconds if there isn't much new data, but can take some time, as we see here, if it has a lot of data to catch up. * (D) Once the state of the old and new databases are identical, the cut over is made and the Metal instances begin serving your database traffic. * (E) The old instances are torn down. Depending on your database and growth rate, you may see a similar spike due to step (C). This could be as short as a minute, or as slow as over an hour, depending on the characteristics of your database workload. ### Rows written We get a clear jump in rows written after the upgrade: ![Insights rows written](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/insights-rows-written.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=ef0e858ed33ed88bbd6018c5470b4319) Unlimited IOPS allows for increased write throughput. We can see that before the switch to Metal, we were only utilizing 20-25% of our CPU capabilities, and much of this was due to being I/O bottlenecked for this particular workload. ![Insights rows written](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/metal/cpu-ram.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=7fc3ab4897a39c993693d83d0870b1e7) Once we switched to Metal and had faster IO operations and unlimited IOPS, we were suddenly able to increase our CPU utilization to 80%+. ## Primary metrics Another good place to look after upgrading to Metal is the metrics for your primary database node. From the database's Dashboard page, click on your primary from the architecture diagram. A panel should display on the right side of your screen with metrics from this instance. ![Database primary metrics](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metal/primary-metrics.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=f2587e892a4c1f3ff7246fcabb840a61) If the change was recently made, you can change the graph to display metrics for only the past hour using the drop down. This gives you good insight into how the upgrade affected your IOPS capabilities, as well as CPU and RAM. Note specifically the large jump in IOPS after the switch to Metal. ## Why is Metal so much faster? PlanetScale databases come in two main flavors: **Metal** and **network-attached storage**. [**Network-attached storage**](/docs/plans/planetscale-skus#network-attached-storage) (Amazon Elastic Block Storage or Google Persistent Disk) databases store all data on storage volumes that are attached to your database's compute resources over the network. For databases in AWS we use Elastic Block Storage (EBS) and in GCP we use Persistent Disk. These network-attached storage solutions are convenient for several reasons. For one, it is easy to resize such storage volumes. PlanetScale leverages this to auto-scale your storage as the size of your database grows or shrinks, allowing you to pay a per-GB storage price. This also means that you can pair a tiny compute instance with a large amount of storage. This works well for large data sets that are not frequently queried. The opposite is also true — you can pair a small amount of storage with a large compute instance for workloads that are heavily CPU-bound. One disadvantage of using **network-attached storage** storage is I/O latency. Reads from and writes to disk need to make network round-trips to be fulfilled. The intra-AZ network speeds in AWS, GCP, and Azure data centers are generally very good, but still slower than accessing a locally-attached solid-state drive. There is also the issue of IOPS. The popular `gp3` EBS volume class provides 3000 IOPS included, but using more than this requires paying for additional IO bandwidth, leading to more expensive databases. These disadvantages do not just apply to PlanetScale. Many of the popular cloud-hosted database solutions, including those offered by Amazon and Google, use network-attached storage to simplify storage scalability. This convenience and scalability comes at a performance cost. **Metal** databases store all data on locally-attached NVMe SSDs. Using direct-attached storage provides a clear solution to the performance issues described above. The removal of network round-trips for I/O operations means low-latency IO, and we sidestep the issue of needing to pay for increased IOPS completely. Your database now has the ability to use modern NVMe SSD technology to it's full potential. ## Data durability One of the most important aspects of a database is storing data durably, even in the face of unexpected outages or hardware failure. By default, all databases on PlanetScale have their data replicated three times: once on the primary database node, and then again on each of the two replicas. More replicas can be added if desired. When using a network-attached storage database, each of the three database instances stores a copy of the data on three distinct network-attached volumes. When using network-attached storage, the compute instances and storage volumes are decoupled. If one of your database compute nodes gets taken down due to an underlying hardware failure, the data is still preserved on the EBS or PD volume and can be quickly re-attached to a new node. PlanetScale handles the detection, re-attachment, and recovery from such failures automatically. A Metal cluster also has high durability, but with different characteristics. Each of the three database instances (1 primary, 2 replicas) stores a copy of the data on the NVMe drives attached to the instance. However, in this case, if one of the three compute nodes goes down, the data also goes with it. This is why replication is so critical. In this scenario, we still have two other copies of the data, and PlanetScale will automatically detect and replace the node that failed, bringing the total back up to three. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki Source: https://planetscale.com/docs/neki Horizontal sharding for Postgres > [!NOTE] > > Neki is currently in Platform Preview. Platform Preview features are “Beta Features” under the PlanetScale Terms of Service or your applicable agreement with PlanetScale. Accordingly, Neki is subject to the limitations and disclaimers applicable to Beta Features and is not covered by any service level agreement. Neki is the best way to scale a highly-available Postgres database. It provides sharding, zero-downtime operations, online DDL, replication workflows, and sophisticated cluster management capabilities, all using real Postgres. Neki abstracts all of this capability behind a single connection string that speaks the Postgres wire protocol. It accomplishes this by placing a sophisticated proxy (router) between clients and the Postgres nodes. This router parses, plans, and coordinates all of the Postgres traffic, routing it to the correct nodes in an extremely efficient way. All of these capabilities come with the other PlanetScale features like [Insights](/docs/neki/monitoring/query-insights), [anomalies](/docs/neki/monitoring/anomalies), and [schema recommendations](/docs/neki/monitoring/schema-recommendations). This makes it a perfect solution for anything from a massive database storing hundreds of terabytes across many shards, all the way down to small, unsharded Postgres databases with a handful of gigabytes. You do not have to shard to use Neki. A new database starts as an unsharded cluster. You still get all the benefits of zero-downtime upgrades, cluster management, connection pooling and more. Whether you later shard depends on your traffic, access patterns, and data — see [When to shard](/docs/neki/when-to-shard). > **[Architecture](/docs/neki/overview)** > > The components of a Neki deployment and how they fit together. > **[When to shard](/docs/neki/when-to-shard)** > > Whether sharding is the right answer to the limits you're hitting. > **[Quickstart](/docs/neki/tutorials/planetscale-quick-start-guide)** > > Create your first Neki database, connect, and insert data. > **[Neki vs PlanetScale Postgres](/docs/neki/coming-from-postgres)** > > The upgrade from a single primary, and the habits that change. ## Availability and access Neki is currently available through the PlanetScale Platform Preview. An organization administrator can select **Join the Platform Preview** from the organization dashboard or open **Organization settings** > **Platform Preview** and accept the preview terms. Neki then appears as a database engine. ## From the maintainers of Vitess Neki is built by PlanetScale, the maintainers of [Vitess](/docs/vitess). Vitess already runs some of the largest services on the internet, including Slack, Square, and Cursor. Neki is not a Vitess port. It is built from scratch to specifically solve the scalability and operational challenges of Postgres. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Back up and restore Source: https://planetscale.com/docs/neki/backups Create, schedule, and restore backups for Neki database branches. Neki supports scheduled and on-demand backups for production and development branches. A backup covers the schema and data stored on every managed shard in the branch. Restoring a backup does not replace the source branch. PlanetScale creates a new Neki branch from the selected backup or point in time. The Neki orchestration layer coordinates the per-shard backup and restore work and reports its status to the PlanetScale control plane. ## How Neki backups work When a backup begins, PlanetScale starts backup work for every managed shard in the branch. Each shard reports its own status, size, and recovery position. The Neki backup is successful only after every shard backup succeeds. The Backups page reports the overall backup as pending, running, successful, or failed. A successful backup records the per-shard recovery information required to restore the branch later. Backups capture each shard primary. Replica instances are not backed up separately. ## Replicas and backups Adding a replica to a shard restores that replica from the shard's last backup. The [admin](/docs/neki/overview#the-admin) then joins the replica to the shard primary so it can start replication and catch up. The same path can repair a replica that can no longer catch up from WAL. That restore does not create a new backup on the Backups page. See [Database replicas](/docs/neki/replicas). ## Automatic backups PlanetScale creates required backup schedules for production and development branches when a database becomes ready. These schedules run every 12 hours and retain each backup for two days. Required schedules cannot be modified or removed. Those backups use the included backup-storage allowance. See [Backup pricing](#backup-pricing). ## View backups **Step 1** From the PlanetScale dashboard, select your Neki database. **Step 2** Select **Backups**. **Step 3** Select **Production branches** or **Development branches**. **Step 4** Optionally filter the list by branch or date range. **Step 5** Review the backup name, source branch, size, completion time, and current status. ![Backups page showing production branch backups, date filters, and the point-in-time recovery card](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/backups-page.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=0958ca1774000daf91c0c5049ca8ecbc) Selecting a backup opens its detail page, where you can review its retention and restore it after it succeeds. ## Create a manual backup You can create one manual backup at a time for a branch. **Step 1** From the database's **Backups** page, select **Create new backup**. **Step 2** Select the branch to back up. **Step 3** Enter a backup name, or leave the field empty to generate one automatically. **Step 4** Set how long to retain the backup. Retention can be measured in hours, days, weeks, months, or years. **Step 5** Select **Create backup**. ### Emergency backups The manual-backup form also provides an **Emergency backup** option. > [!WARNING] > > Emergency backups may affect database performance. Use one only in a critical > situation that requires an immediate backup. ![Create backup dialog with branch, name, Emergency backup, and retention controls](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/create-backup.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=da79791a16a9ca2c0749f5cb85d569a4) ## Create a custom backup schedule A database-level schedule applies to every branch of the selected type. For example, a production schedule runs for every production branch in the database. **Step 1** From the database's **Backups** page, select **Production branches** or **Development branches**. **Step 2** Select **Add new schedule**. **Step 3** Choose an hourly, daily, weekly, or monthly frequency and when the backup should begin. Times are configured in UTC. **Step 4** Set the retention period in hours, days, weeks, months, or years. **Step 5** Enter a schedule name, or leave the field empty to generate one automatically. **Step 6** Select **Save schedule**. You can edit or delete a custom schedule. Required schedules cannot be changed or removed. ## Restore a backup A completed Neki backup can be restored to a new branch. **Step 1** From the database's **Backups** page, select a successful backup. **Step 2** Select **Restore to new branch**. **Step 3** Enter a name for the new branch. **Step 4** For each configuration profile, select a cluster size and replica count. Cluster sizes must use the same CPU architecture as the source profile. Use development SKUs for every profile to create a development branch, or production SKUs for every profile to create a production branch. You cannot mix development and production profile sizes. **Step 5** For each router group, select a router size and the number of replicas per availability zone. **Step 6** Review the estimated monthly cost, then select **Restore backup**. PlanetScale creates the new branch and restores every shard recorded by the backup. The source branch remains available while the new branch is created. The restored branch is billed on its own for compute, storage, backups, and network transfer. See [Neki pricing](/docs/neki/pricing). The default configuration profile's selected SKU determines whether the restored branch is development or production. If you do not override profile sizes, the restore inherits the source sizes and branch type. Development profiles use one Postgres node per shard and a development router. Production profiles use the selected production replica shape and production routers across three AZs. These choices determine the restored branch's availability, branch-limit category, and billing. ## Point-in-time recovery Point-in-time recovery restores a new branch to a selected time rather than to the exact completion time of a backup. Neki starts from an eligible successful backup completed at or before the selected time, then replays the Postgres write-ahead log (WAL) for each restored shard until that time. The dashboard limits the available window to times between the oldest eligible backup and five minutes before the current time. **Step 1** From the database's **Backups** page, find **Point-in-time recovery**. **Step 2** Select the source branch. **Step 3** Select the date and time to restore. **Step 4** Enter a name. **Step 5** Choose a compatible cluster size and replica count for each configuration profile. **Step 6** Choose a size and replicas per availability zone for each router group. **Step 7** Review the estimated monthly cost. **Step 8** Select **Restore backup** to create the branch. If no eligible successful backup completed before the selected time, the restore cannot start. Point-in-time recovery is also available in the PlanetScale CLI: ```bash theme={null} pscale branch create \ --from \ --restore-point 2026-09-09T18:00:00Z ``` The CLI chooses an eligible backup from the source branch. To select the backup and restore sizing explicitly, use its ID and repeat `--config-profile` and `--router` for each resource that needs an override. This example assumes the source has only its default profile and router: ```bash theme={null} pscale branch create \ --restore \ --restore-point 2026-09-09T18:00:00Z \ --config-profile name=default,cluster-size=PS_DEV,replicas=0 \ --router name=default,size=NKR_DEV,replicas-per-cell=1 ``` Omitted configuration profiles and routers inherit their source settings. Profile sizes must retain the source CPU architecture, and all restored profiles must use either development or production SKUs. > [!WARNING] > > A shard-set change can create a gap in the point-in-time recovery window. A > restore point after a shard was added is unavailable until a later successful > backup includes every shard in the new set. Choose a time covered by that > later backup, or restore the later backup without a restore point. ## Retention, protection, and deletion Manual backups and custom schedules use the retention period selected when they are created. After a backup succeeds, its detail page shows when it expires. Enable **Prevent backup deletion** on an individual backup to prevent both automatic and manual deletion. A required backup cannot be manually deleted. A backup that is being used by an active restore also cannot be deleted. ## Backup pricing Each branch includes backup storage equal to twice the allocated disk on that branch. Required 12-hour, two-day backups use this allowance. Compressed backup data and WAL above the included amount are billed at **\$0.023 per GB per month**. Longer retention, manual backups, and custom schedules increase usage. See [Neki pricing](/docs/neki/pricing#backups) for the full usage model. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki best practices Source: https://planetscale.com/docs/neki/best-practices Data placement, transaction scope, and workload isolation ## Design data placement before adding shards For sharded data, choose shard keys from queries and transactions your application relies on. Prefer keys that: * Distribute data and writes evenly. * Appear in the predicates of latency-sensitive queries. * Keep rows that are joined or updated together in the same shard group. * Remain stable for the lifetime of a row. Index shard-key columns and keep their statistics current for query planning. See [Data topology](/docs/neki/data-topology#shard-indexes) for details. Use [reference tables or global secondary indexes (GSIs)](/docs/neki/reference-tables-and-gsis) when queries need data that their shard-key route does not provide: * Reference tables duplicate a small shared dataset across shards so joins to sharded data remain local. * GSIs map another lookup key to the owner row's shard key. Topology declarations do not bootstrap historical data. Populate and verify every reference-table copy before binding it. Create a GSI disabled, backfill and verify its lookup table while following owner changes, and enable it only at cutover. An enabled but incomplete GSI can return incomplete results without an error. `COPY FROM` does not maintain GSI lookup rows for a sharded owner table. Keep the GSI disabled while copying, then backfill, verify, and enable it through the same activation process. Once application data is placed across shards, leave the [authoritative shard group](/docs/neki/data-topology#authoritative-shard-group) as a separate shard. Size it for metadata, catalog work, sequence reservations, and remaining unsharded tables. ## Keep transactions on one shard when possible Neki can route a transaction to more than one shard, but cross-shard transactions do not provide atomic commit across all shards. Include the shard key in transactional queries and verify the query plan before depending on multi-statement behavior. See [Transactions across shards](/docs/neki/transactions). ## Watch for scatter queries A query without enough routing information can run on every shard. Use `EXPLAIN` and [Query Insights](/docs/neki/monitoring/query-insights) to look for an unexpected number of shard calls, then add a routing predicate or revisit the topology. ## Separate workload classes Shards do not need to be evenly sized or identically configured. Assign resources, replicas, parameters, extensions, and more according to the requirements of the data within the shard. Use additional [router groups](/docs/neki/cluster-configuration#configure-router-groups) when a workload needs independent router sizing or autoscaling. Choose the group on the **Connect** page. Profile, router, and Admin changes are asynchronous. Follow the object's **Changes** tab and wait for it to finish before submitting a dependent change. PlanetScale's [orchestration layer](/docs/neki/terminology#orchestration-layer) rolls the requested configuration through the affected resources. ## Monitor capacity and recovery Review [Metrics](/docs/neki/monitoring/metrics) for CPU, memory, connections, storage, router errors, and replication lag. On the **Storage** tab, also watch WAL growth and archive failures. [Anomalies](/docs/neki/monitoring/anomalies) flags queries that run slower than their baseline. Schedule an [MCP](/docs/mcp-server) agent to review [Query Insights](/docs/neki/monitoring/query-insights), query errors, and [schema recommendations](/docs/neki/monitoring/schema-recommendations). [Self-improving database](/docs/self-improving-database) has prompts you can run with Cursor Automations. Required [backup](/docs/neki/backups) schedules run automatically. Choose retention for manual backups, and periodically restore into a disposable branch. A successful backup verifies that data was captured; a restore test verifies the recovery workflow your application depends on. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Branching Source: https://planetscale.com/docs/neki/branching Create development branches, restore from backups, and run production with high availability. A Neki branch is an isolated database environment with its own routers, Postgres instances, data, configuration, backups, and billing. Use production branches for serving workloads and development branches for disposable testing. ## Development branches start empty Creating a Neki development branch provisions a new, isolated environment. It does not copy the parent branch's schema or data. Apply your schema migrations to the new branch and load only the test data you need. To create a branch containing existing data, restore a [backup](/docs/neki/backups) to a new branch instead of creating an ordinary development branch. ## Create a development branch **Step 1** From the PlanetScale dashboard, select your Neki database. **Step 2** Open the branch menu and select **New branch**. **Step 3** Choose **Development**, enter a branch name, and review the selected region and estimated price. **Step 4** Select **Create branch** and wait for the branch to become ready. **Step 5** Apply your application's schema migrations and seed data. The new branch uses development shard and router SKUs. Its infrastructure and usage are billed independently from the parent. See [Neki pricing](/docs/neki/pricing). ## Work with an isolated branch Connections, roles, configuration profiles, parameters, extensions, and data topology are branch-scoped. Verify the selected branch before changing its configuration or running DDL. An empty development branch is useful for testing schema setup from scratch. See [Development environments](/docs/neki/development-environments) for a repeatable setup and guidance for production-like data. ## Move development data to production To keep the data from a development branch, create a backup and restore it to a new branch with production configuration. See [Restore a backup](/docs/neki/backups#restore-a-backup) for the required configuration-profile and router sizes. ## Delete a branch Deleting a branch removes its database infrastructure and data. The default branch cannot be deleted while other branches remain. Confirm that any backup or export you need has completed before deletion. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster configuration Source: https://planetscale.com/docs/neki/cluster-configuration Configure Neki admin, routers, configuration profiles, and shards. The **Clusters** area in the PlanetScale dashboard controls the infrastructure for each Neki branch. It separates the Postgres instances that store data, the routers that receive application connections, and the admin service that monitors and repairs the cluster. | Configuration | What it controls | | - | - | | **Configuration profile** | The cluster size, replica count, Postgres version, storage, Postgres and sidecar parameters, extensions, and shard assignments for one or more shards | | **Admin** | The Admin size and its health-check and recovery parameters | | **Router group** | The router size, router instances per availability zone, autoscaling, and router parameters | Configuration is scoped to the selected branch. Use the branch selector at the top of the page before making a change. By default, an organization can use up to 16 shards in one Neki cluster and 51 total cluster nodes. The node budget includes every Postgres primary and replica plus router instances. When router autoscaling is enabled, its configured maximum counts toward the limit. Organization-specific limits can differ; the dashboard and API return the applicable limit when a requested change would exceed it. > [!NOTE] > > Only a database administrator can change cluster configuration. The branch > must also be ready before the dashboard enables its configuration controls. ![Clusters page showing the configuration-profile list, Instances tab, and branch selector](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/cluster-configuration/clusters-profiles.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=f460ad17911a4c3bc0785c18231ed32a) ## Configuration profiles Every shard is assigned to a configuration profile. Multiple profiles let you give different groups of shards different cluster sizes, replica counts, storage settings, parameters, or extensions. | Tab | What you can manage | | - | - | | **Instances** | Profile name, Postgres minor version, cluster size, and replica count for every shard assigned to the profile | | **Storage** | Minimum disk size, storage autoscaling, and configurable IOPS and bandwidth for network-attached storage profiles; fixed local capacity for Metal profiles | | **Shards** | Shard creation, names, storage usage, deletion, and assignment to profiles | | **Postgres** | Postgres parameters applied to every Postgres instance in the profile | | **Sidecars** | Connection-pool parameters applied to the sidecars beside those Postgres instances | | **Replicator** | Advanced parameters for connection management of the replicator | | **Extensions** | Postgres extensions and extension parameters available to the profile | | **Changes** | Current and previous profile and sidecar changes | The profile list shows the cluster size and number of assigned shards. Change a profile's name from its **Instances** tab. The profile's action menu lets you make it the default or delete it when the current configuration permits deletion. Setting a new default does not move existing shards between profiles. You cannot delete the default profile or a profile that has shards assigned to it. To delete another profile, first assign all of its shards to other profiles. ### Postgres versions The Postgres major version of an existing configuration profile cannot be changed. You can select the version when creating a new configuration profile. ### Configure profile instances The **Instances** tab changes the profile name, Postgres minor version, cluster size, replica count, or [read-only replica](/docs/neki/read-only-replicas) count. Those changes apply to every shard assigned to the profile. See [Cluster sizing](/docs/neki/cluster-configuration/cluster-sizing) to choose and apply a size, and [Database replicas](/docs/neki/replicas) for high availability and read-only routing. Production branches receive scheduled platform updates during their weekly [maintenance window](/docs/neki/cluster-configuration/maintenance-windows). ### Configure profile storage The **Storage** tab controls network-attached disk capacity, storage autoscaling, and configurable IOPS and bandwidth for a network-attached storage profile. A Metal profile shows the fixed local NVMe capacity provided by its cluster size. See [Neki pricing](/docs/neki/pricing#storage) for billing implications. ## Manage shards The **Shards** tab shows the shards assigned to the selected configuration profile, including their readiness and storage usage. It also highlights the authoritative shard. From this tab, you can: * Create one or more shards in the selected profile. * Rename a shard. * Assign compatible shards to another configuration profile. * Delete shards. To create shards, select **Create new shards**, enter the number to add, review the per-shard and total estimated cost, and select **Add shards**. Each new shard uses the selected profile's cluster size and replica count. Creating a shard provisions its Postgres instances, but it does not change which tables or rows Neki stores there. The [data topology](/docs/neki/data-topology) controls data placement. > [!WARNING] > > Deleting a shard is irreversible. Remove it from the data topology and > confirm that it no longer contains data you need before deleting it. Neki > refuses to delete a shard that the current data topology still references. ### Assign shards to another profile Moving a shard to another configuration profile changes the infrastructure settings that apply to that shard. The target profile must use the same architecture, and the shard's current storage usage must fit within the target cluster's storage capacity. If the branch has only one configuration profile, create another compatible profile before **Change profile** becomes available. **Step 1** Open **Clusters** and select the configuration profile that currently contains the shard. **Step 2** Select **Shards**. **Step 3** Select one or more shards, then select **Change profile…**. **Step 4** Select a compatible configuration profile. **Step 5** Select **Change profile**. Wait for the affected shards and branch to return to a ready state before making a dependent change. ## Configure the admin service The Neki admin service monitors the sidecars beside each Postgres instance, analyzes cluster health, and coordinates repairs such as replica recovery and primary failover. Its configuration is separate from configuration profiles and router groups. ![Admin Configuration tab showing the Admin size selector and health-check and recovery parameters](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/cluster-configuration/admin-configuration.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=32339a81b704eb0d47f40aadc8c6a247) Open **Clusters**, then select **Admin**. The **Configuration** tab sets the Admin size and its health-check and recovery parameters. The **Changes** tab records those updates. You can cancel an admin change while it is pending or applying. See [Cluster sizing](/docs/neki/cluster-configuration/cluster-sizing#size-the-admin-service-independently) to choose an Admin size, and [Configuration parameters](/docs/neki/cluster-configuration/parameters#configure-admin-parameters) for the available admin settings. ## Configure router groups Router groups are managed separately from configuration profiles. Open **Clusters**, then select **Routers** from the navigation. Each named router in this area represents one router group. Selecting it opens these tabs: | Tab | What you can manage | | - | - | | **Instances** | Router size and router instances per availability zone | | **Autoscaling** | Horizontal autoscaling, target CPU utilization, and minimum and maximum instances per availability zone | | **Parameters** | Replica-routing lag thresholds | | **Changes** | Current and previous router changes | The default router group cannot be deleted. You can create additional groups, then choose the group when you [connect to Neki](/docs/neki/connecting). ### Create a router **Step 1** From the PlanetScale dashboard, select your Neki database and branch. **Step 2** Open **Clusters**, then select **Routers**. **Step 3** Select **New router**. **Step 4** Enter a name, select a router size, and choose the number of replicas per availability zone. **Step 5** Select **Create router**. On the **Autoscaling** tab, target CPU utilization is 40%, 50%, 60%, or 70%. The maximum replicas per availability zone must be greater than the minimum. The default organization limit is 32 router replicas per availability zone; the dashboard shows a different limit when your organization has one. An autoscaling router's maximum, rather than its current replica count, consumes the cluster node budget. See [Cluster sizing](/docs/neki/cluster-configuration/cluster-sizing#size-routers-independently) to change a group's size or autoscaling. See [Postgres parameters](/docs/neki/cluster-configuration/parameters#configure-postgres-parameters), [admin parameters](/docs/neki/cluster-configuration/parameters#configure-admin-parameters), and [router parameters](/docs/neki/cluster-configuration/parameters#configure-router-parameters). ## Track configuration changes Selecting **Apply now** creates a change that PlanetScale applies asynchronously. View the selected object's **Changes** tab for a summary. Wait for a change to complete before relying on the new configuration. PlanetScale's [orchestration layer](/docs/neki/terminology#orchestration-layer) reconciles these requested settings with what is already running. The change history reflects that asynchronous work rather than an immediate in-place edit. You can cancel a pending configuration change from the **Changes** tab. If you make another configuration-profile change while one is active, Neki saves the new change as a draft. You can apply or discard that draft from the profile or its **Changes** tab. ## Manage Neki infrastructure with the CLI Neki infrastructure commands are grouped beneath `pscale branch`: | Command group | What it manages | | - | - | | `pscale branch config-profile` | Configuration profiles, their Postgres parameters, storage, the default profile, profile changes, extensions (`list`, `enable`, `disable`), and per-profile maintenance | | `pscale branch shard` | Shard creation, assignment, display names, listing, show, and deletion | | `pscale branch admin` | Admin configuration, parameters, sizes, and changes | | `pscale branch router` | Router groups, sizes, configuration, and changes | | `pscale branch sidecar` | Connection-pool parameters and their changes | | `pscale branch data-topology` | The branch data-topology document and its resolved relationships | The [`pscale branch`](/docs/cli/branch) reference lists every nested operation and flag. Run a command group with `--help` to see the same details in the CLI: ```bash theme={null} pscale branch config-profile --help pscale branch shard --help pscale branch router --help ``` The Admin, router, and sidecar `changes list` commands include a **Changes** column that summarizes each size or parameter change from its previous value to its new value. List available Neki cluster sizes with `pscale size cluster list --engine neki`, admin SKUs with `pscale branch admin sizes `, and router SKUs with `pscale branch router sizes `. Extension commands require the database, branch, and profile. Enable and disable also require the extension name: ```bash theme={null} pscale branch config-profile extensions pscale branch config-profile extensions enable pscale branch config-profile extensions disable ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster sizing Source: https://planetscale.com/docs/neki/cluster-configuration/cluster-sizing Choose and change configuration-profile, admin, and router sizes for a Neki database. Neki separates the Postgres instances that store data, the admin service that monitors and repairs clusters, and the routers that receive application connections. These layers are independently sized. Size and configuration are not shared across these layers: | Setting | What it sizes | | - | - | | **Configuration-profile size** | Postgres primary and any configured replicas in each shard assigned to the profile | | **Admin size** | The Neki admin-service instances for the branch | | **Router size** | Every router instance in the selected router group | ## Choose a configuration-profile size Open a configuration profile's **Instances** tab to compare the cluster sizes available for the selected branch. The size menu shows the vCPU, memory, storage, and estimated price of each option. The available choices depend on the branch's infrastructure, region, and the sizes enabled for your organization. Use the dashboard as the source of truth for the sizes and prices currently available to a profile. You can also list cluster sizes with `pscale size cluster list --engine neki`. ### Network-attached storage and Metal The cluster-size menu separates network-attached storage and Metal options. | Option | Storage and availability | | - | - | | **Network-attached storage** | Uses configurable network-attached storage billed separately from compute. The dashboard shows the storage type, usage, and applicable estimate. | | **Metal** | Uses fixed-capacity, locally attached NVMe storage with unlimited IOPS. Metal requires a primary with replicas. | Metal sizes have fixed storage capacity. The dashboard checks current storage usage and disables sizes that do not leave the required storage margin. On AWS, when using the smaller Metal sizes (**M-10** through **M-160**) you can select the storage capacity separately from the vCPU and memory. The capacity is still fixed after you select it and does not autoscale. To get more storage, select a larger storage option or a larger size. See [Metal options on AWS](/docs/plans/planetscale-skus#metal-options-on-aws) for the full list. Use the profile's **Storage** tab to configure network-attached storage disk capacity, autoscaling, IOPS, and bandwidth. See [Configure profile storage](/docs/neki/cluster-configuration#configure-profile-storage). The CPU architecture of an existing configuration profile cannot be changed. The dashboard only lists compatible sizes when you update the profile. ![Configuration profile Instances tab showing cluster size, replica count, and estimated monthly cost](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/cluster-configuration/profile-instances.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=0843c73f07d38e86f1e99964355fcaec) ## Size profiles for their assigned shards Every shard assigned to a profile receives the same cluster size and replica count, but each shard can have a different workload. Compare the Postgres [metrics](/docs/neki/monitoring/metrics) for the assigned shards before changing the profile: * Review CPU, memory, IOPS, storage, connections, and WAL activity. * Size the profile for the shard with the greatest resource requirements. * Check whether uneven utilization comes from the workload or [data topology](/docs/neki/data-topology) before adding capacity. If one shard consistently needs a different size, create a compatible profile for that shard and [assign the shard to it](/docs/neki/cluster-configuration#assign-shards-to-another-profile). This avoids resizing unrelated shards. Replica count is configured separately from cluster size. Production configuration profiles are highly available and include one primary and two replicas. See [Database replicas](/docs/neki/replicas) when adding capacity for replica reads. ## Change a configuration profile > [!WARNING] > > Changing the Instances settings affects every shard assigned to the profile. > Review the profile's **Shards** tab and the estimated monthly cost before > applying the change. **Step 1** From the PlanetScale dashboard, select your Neki database and branch. **Step 2** Open **Clusters** and select the configuration profile. **Step 3** Select **Instances**. **Step 4** Choose the cluster size and replica count. **Step 5** Review the estimated monthly cost, then select **Apply now**. **Step 6** Use the profile's **Changes** tab to follow the change and review any parameter defaults updated by the new size. PlanetScale applies the change asynchronously. Wait for it to complete before making a dependent configuration change. Some Postgres parameter defaults depend on the profile's resources. When the size changes, PlanetScale updates resource-dependent defaults while preserving manually configured values unless the new configuration requires a different valid range. See [Configuration parameters](/docs/neki/cluster-configuration/parameters#defaults-and-profile-changes). ### Profiles without shards Creating a configuration profile does not create Postgres instances. A new profile starts with no assigned shards, so its current profile total is zero. The creation form shows the estimated cost per shard that would apply after a shard is created in or assigned to the profile. ## Size the admin service independently Changing the admin size does not change the Postgres resources assigned to a shard or the size of any router. The available admin sizes are **NKA-0**, **NKA-1**, **NKA-2**, **NKA-5**, **NKA-20**, and **NKA-40**. A new Neki cluster uses **NKA-0** by default; change the size after creation from the branch's Admin configuration. Use the dashboard as the source of truth for the sizes currently available to your branch. You can also list admin SKUs with `pscale branch admin sizes `. The size you select establishes Admin's baseline capacity. PlanetScale automatically increases Admin memory when it needs more capacity. Choose a larger size when the branch needs more capacity available immediately for cluster-health and recovery work. Admin capacity is separate from application query capacity. Increasing the Admin size does not add router throughput, Postgres CPU or memory, replicas, or shards. Size those resources separately. Admin is not billed during Platform Preview; see [what appears on the bill](/docs/neki/pricing#what-appears-on-the-bill). **Step 1** From the selected Neki branch, open **Clusters**, then select **Admin**. **Step 2** Select **Configuration**. **Step 3** Choose an **Admin size**, then select **Apply now**. **Step 4** Use the **Changes** tab to follow the change. PlanetScale applies the change asynchronously. The same change can also update the admin's health-check and recovery parameters. See [Configuration parameters](/docs/neki/cluster-configuration/parameters#configure-admin-parameters). ## Size routers independently Router groups have their own sizes and replica counts. Changing a router does not change the Postgres resources assigned to any shard. ![Router Instances tab showing the selected NKR-1 size with vCPU, memory, replicas per availability zone, and monthly router cost](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/cluster-configuration/router-instances.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=ffc7dc97050a1d75ce1cdcda014e9f1c) Use router [CPU utilization, query rate, latency, and error metrics](/docs/neki/monitoring/metrics#router-metrics) to evaluate the router layer. The router-size menu shows the vCPU, memory, and estimated price of each available option. You can also list router SKUs with `pscale branch router sizes `. **Step 1** From the selected Neki branch, open **Clusters**, then select **Routers**. **Step 2** Select the router group, then select **Instances**. **Step 3** Choose a router size and the number of replicas per availability zone. **Step 4** Review the estimated monthly cost, then select **Apply now**. **Step 5** Use the router group's **Changes** tab to follow the change. When horizontal autoscaling is enabled, configure the target CPU utilization and minimum and maximum replicas per availability zone from the **Autoscaling** tab. Autoscaling manages the replica count; the **Instances** tab continues to control the size of each router instance. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Configure Neki maintenance windows Source: https://planetscale.com/docs/neki/cluster-configuration/maintenance-windows Weekly maintenance scheduling for production Neki branches. Every production Neki branch has a weekly maintenance window. PlanetScale uses this window for maintenance that can restart cluster components, including: * Neki image updates. * Postgres minor-version updates within the configured major version. * Updates to default parameters and platform-managed configuration. On a highly available shard, Neki performs a planned switchover before maintenance on the primary. The switchover waits for a replica to catch up before promoting it, which preserves commits made on the old primary. Maintenance can still close active connections, so applications should reconnect automatically. > [!NOTE] > > The maintenance window controls scheduled weekly maintenance. Configuration > changes that you request in the dashboard, API, CLI, or Terraform apply > independently of this window. ## Change the weekly schedule Maintenance windows use UTC. The dashboard also shows the corresponding time in your configured local time zone. ![Settings Maintenance page showing the weekly schedule card for a production branch, with Change schedule and Run maintenance tasks now](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/cluster-configuration/maintenance-windows.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=265c268fbc5ad3f5310cae26457d79da) **Step 1** From the PlanetScale dashboard, select your Neki database. **Step 2** Open **Settings**, then select **Maintenance**. **Step 3** Find the production branch and select **Change schedule**. **Step 4** Choose the day of the week and hour in UTC. **Step 5** Select **Save**. The schedule card shows the previous maintenance window and the next scheduled window. Only database administrators can change maintenance schedules. ## Run maintenance immediately To apply available weekly maintenance without waiting for the next window, select **Run maintenance tasks now** on the branch's schedule card. Running maintenance immediately applies the same available updates as the weekly schedule and can restart cluster components. Confirm that applications can reconnect before starting it. Neki cannot start customer-requested maintenance while a branch is sleeping. Run branch-wide maintenance from the CLI with: ```bash theme={null} pscale branch maintenance run ``` ## Development branches Development branches do not have configurable maintenance windows. Scheduled maintenance uses PlanetScale's maintenance schedule for these branches. ## Related documentation * [Cluster configuration](/docs/neki/cluster-configuration) * [Postgres versions](/docs/neki/cluster-configuration#postgres-versions) * [Replication and planned switchovers](/docs/neki/overview#the-admin) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Configuration parameters Source: https://planetscale.com/docs/neki/cluster-configuration/parameters Configure Postgres, connection-pool, admin, and router parameters for a Neki database. Neki exposes separate parameters for the Postgres instances and connection pools in a configuration profile, the branch's admin service, and each router group. Parameter changes are scoped to the selected profile, admin, or router. ## Parameter scopes | Scope | What it controls | Where it applies | | - | - | - | | **Configuration profile** | Postgres settings such as memory, autovacuum, write-ahead log, and connection limits | Every Postgres instance in every shard assigned to the profile | | **Sidecars** | The connection pools between routers and Postgres instances | The sidecar beside every Postgres instance in every shard assigned to the configuration profile | | **Replicator** | Connections used by the distributed replicator | The replicator for every shard assigned to the configuration profile | | **Admin** | Cluster discovery, health checks, and recovery behavior | The admin service for the selected branch | | **Router** | Replica-routing lag thresholds | Every router instance in the selected router group | Different configuration profiles can use different Postgres parameter values. For example, a profile assigned to a memory-intensive shard can use different resource settings from the default profile. The effective defaults shown in the dashboard can depend on the configuration profile's cluster size, Postgres version, enabled extensions, and current PlanetScale configuration. Use the dashboard value as the default for the selected profile instead of assuming that one default applies to every profile. ## Configure Postgres parameters You must be a database administrator to change a configuration profile. **Step 1** From the PlanetScale organization dashboard, select your database. **Step 2** Select the branch you want to configure. **Step 3** Open **Clusters** and select a configuration profile. **Step 4** Select the **Postgres** tab. **Step 5** Search for a parameter or browse the available categories. **Step 6** Update one or more values, then select **Apply now**. **Step 7** Use the **Changes** tab to follow the configuration change. The Postgres tab shows commonly adjusted settings first. Select **Show advanced parameters** to view settings that normally do not need to be changed. Searching includes both common and advanced parameters. Each field shows its effective default. A configured value that differs from that default is highlighted, along with who changed it and when. Parameters that Postgres cannot apply without restarting are labeled **Requires restart**. > [!WARNING] > > Parameter changes apply to every shard assigned to the configuration profile. > Review the assigned shards before changing memory, worker, connection, or WAL > settings. ### Defaults and profile changes PlanetScale chooses resource-dependent defaults for each cluster size. When a profile's cluster size or Postgres version changes, its default-owned parameters adopt the applicable defaults. Manually configured values are preserved unless the new configuration imposes a different valid range or another parameter constraint. The API validates individual values and related settings together. For example: * `huge_pages` and `shared_buffers` must use a supported combination. * `max_replication_slots` must provide enough capacity for the profile's replica configuration. * Postgres restart-sensitive capacity settings cannot be increased and decreased in opposite directions in the same change. If a change is rejected, the dashboard displays the validation error next to the affected parameter. ## Postgres parameter reference The following parameters are currently available in the configuration-profile Postgres tab. **Common** parameters appear immediately. **Advanced** parameters appear in the advanced section or in search results. The dashboard remains authoritative for the valid range, allowed values, effective default, and availability for a particular profile. ### Autovacuum | Parameter | Level | Restart | Description | | - | - | - | - | | `autovacuum` | Advanced | No | Starts the autovacuum subprocess. | | `autovacuum_analyze_scale_factor` | Common | No | Sets the fraction of table changes added to the analyze threshold. | | `autovacuum_max_workers` | Advanced | No | Sets the maximum number of autovacuum workers that can run at the same time. | | `autovacuum_naptime` | Advanced | No | Sets the delay between autovacuum runs. | | `autovacuum_vacuum_cost_delay` | Advanced | No | Sets the cost delay used by autovacuum. | | `autovacuum_vacuum_cost_limit` | Advanced | No | Sets the cost limit available to autovacuum before it pauses. | | `autovacuum_vacuum_insert_scale_factor` | Advanced | No | Sets the fraction of inserted rows added to the vacuum insert threshold. | | `autovacuum_vacuum_insert_threshold` | Advanced | No | Sets the minimum number of inserted rows before vacuuming. | | `autovacuum_vacuum_scale_factor` | Common | No | Sets the fraction of updated or deleted rows added to the vacuum threshold. | ### Client connection defaults | Parameter | Level | Restart | Description | | - | - | - | - | | `lock_timeout` | Advanced | No | Sets how long a statement waits to acquire a lock before it is aborted. | | `statement_timeout` | Advanced | No | Sets the maximum duration allowed for a statement. Set it to `0` to disable the timeout. | ### Connections and authentication | Parameter | Level | Restart | Description | | - | - | - | - | | `max_connections` | Common | Yes | Sets the maximum number of concurrent Postgres connections. | ### Lock management | Parameter | Level | Restart | Description | | - | - | - | - | | `deadlock_timeout` | Advanced | No | Sets how long Postgres waits on a lock before checking for a deadlock. | | `max_locks_per_transaction` | Advanced | Yes | Sets the average number of object locks allocated for each transaction. | | `max_pred_locks_per_page` | Advanced | No | Sets how many predicate-locked rows on one page cause Postgres to promote them to a page-level lock. | | `max_pred_locks_per_relation` | Advanced | No | Sets how many predicate-locked pages and rows in one relation cause Postgres to promote them to a relation-level lock. | | `max_pred_locks_per_transaction` | Advanced | Yes | Sets the average number of predicate locks allocated for each transaction. | ### Query tuning | Parameter | Level | Restart | Description | | - | - | - | - | | `default_statistics_target` | Advanced | No | Sets the default statistics target for columns without a column-specific target. | | `effective_cache_size` | Common | No | Sets the query planner's assumption about the total size of available data caches. | | `jit` | Advanced | No | Enables just-in-time compilation. | | `jit_above_cost` | Advanced | No | Sets the estimated query cost above which just-in-time compilation is considered. | | `random_page_cost` | Advanced | No | Sets the planner's estimated cost of fetching a nonsequential disk page. | | `seq_page_cost` | Advanced | No | Sets the planner's estimated cost of fetching a sequential disk page. | ### Reporting and logging | Parameter | Level | Restart | Description | | - | - | - | - | | `log_autovacuum_min_duration` | Advanced | No | Logs autovacuum operations that run for at least the configured duration. | | `log_connections` | Advanced | No | Selects which stages of connection establishment and setup Postgres logs. | | `log_line_prefix` | Advanced | No | Sets the information prefixed to each Postgres log line. | | `log_lock_waits` | Common | No | Logs lock waits that exceed `deadlock_timeout`. | | `log_min_duration_statement` | Common | No | Logs statements that run for at least the configured duration. | | `log_statement` | Advanced | No | Selects which types of SQL statements Postgres logs. | | `log_temp_files` | Advanced | No | Logs temporary files that meet the configured size threshold. | ### Replication | Parameter | Level | Restart | Description | | - | - | - | - | | `max_logical_replication_workers` | Advanced | Yes | Sets the maximum number of logical replication workers. | | `max_replication_slots` | Advanced | Yes | Sets the maximum number of replication slots. | | `max_slot_wal_keep_size` | Advanced | No | Sets the maximum amount of WAL that replication slots can retain. | | `max_sync_workers_per_subscription` | Advanced | No | Sets the maximum number of table and sequence synchronization workers for each subscription. | | `max_wal_senders` | Advanced | Yes | Sets the maximum number of concurrent WAL sender processes. | ### Resource usage | Parameter | Level | Restart | Description | | - | - | - | - | | `bgwriter_delay` | Advanced | No | Sets the delay between background-writer rounds. | | `bgwriter_lru_maxpages` | Advanced | No | Sets the maximum number of least-recently-used buffers the background writer can flush per round. | | `effective_io_concurrency` | Advanced | No | Sets the number of concurrent requests the storage subsystem can handle efficiently. | | `huge_pages` | Advanced | Yes | Controls whether Postgres requests huge pages for its main shared-memory area. | | `io_workers` | Advanced | No | Sets the number of I/O worker processes when `io_method` is `worker`. | | `maintenance_io_concurrency` | Advanced | No | Sets the storage concurrency available to maintenance operations. | | `maintenance_work_mem` | Advanced | No | Sets the maximum memory available to a maintenance operation. | | `max_parallel_maintenance_workers` | Advanced | No | Sets the maximum number of parallel workers for one maintenance operation. | | `max_parallel_workers` | Advanced | No | Sets the maximum number of parallel workers active at one time. | | `max_parallel_workers_per_gather` | Common | No | Sets the maximum number of parallel workers for one executor node. | | `max_worker_processes` | Advanced | Yes | Sets the maximum number of background processes. | | `shared_buffers` | Common | Yes | Sets the memory Postgres uses for shared buffers. | | `track_activity_query_size` | Advanced | Yes | Sets the amount of memory reserved for each query shown in `pg_stat_activity`. | | `work_mem` | Common | No | Sets the memory available to query operations such as sorting and hashing. | ### Statistics | Parameter | Level | Restart | Description | | - | - | - | - | | `track_io_timing` | Common | No | Collects timing statistics for database I/O operations. | ### Write-ahead log | Parameter | Level | Restart | Description | | - | - | - | - | | `archive_timeout` | Advanced | No | Forces a WAL segment switch when the current segment has remained open for the configured duration. | | `checkpoint_timeout` | Advanced | No | Sets the maximum time between automatic WAL checkpoints. | | `max_wal_size` | Common | No | Sets the WAL size that triggers a checkpoint. | | `min_wal_size` | Advanced | No | Sets the minimum size to which WAL can shrink. | | `wal_buffers` | Advanced | Yes | Sets the number of shared-memory disk-page buffers used for WAL. | | `wal_compression` | Common | No | Selects the compression method for full-page images in WAL. | | `wal_level` | Advanced | Yes | Sets how much information Postgres writes to WAL. | > [!NOTE] > > Extension preload settings are managed from the configuration profile's > **Extensions** tab rather than its **Postgres** tab. ## Configure replicator parameters The configuration profile's **Replicator** tab exposes connection management for the distributed replicator. Changes apply to every shard assigned to the profile. | Parameter | Default | Restart | Description | | - | - | - | - | | `max-conns-per-server` | Profile-dependent | Yes | Maximum connections each distributed replicator can hold open to one Postgres server. The valid range is 4 through 16. | ## Configure connection pooling Every Postgres instance runs one sidecar beside it. A configuration profile's **Sidecars** tab controls the connection pools those sidecars use between routers and Postgres. Changes apply to every sidecar in every shard assigned to the profile. **Step 1** From the PlanetScale organization dashboard, select your database and branch. **Step 2** Open **Clusters** and select a configuration profile. **Step 3** Select the **Sidecars** tab. **Step 4** Update one or more connection-pool values, then select **Apply now**. **Step 5** Use the **Changes** tab to follow the connection-pool change. | Parameter | Description | | - | - | | `external-conn-reservation` | Connections reserved for the replicator, backups, and metrics clients instead of application pools. | | `pool-capacity` | Maximum number of Postgres connections in each database pool. | | `pool-min-conns` | Minimum number of live connections the pool maintains. Set it to `0` to disable the minimum. | | `pool-max-lifetime` | Base maximum lifetime of a connection before it is recycled. Actual lifetimes are jittered up to twice this value. Set it to `0` for no maximum. | | `pool-max-wait-time` | Maximum time a query waits for a pooled connection when the pool is full. | | `pool-idle-timeout` | Time after which an idle connection above the `pool-min-conns` minimum is closed. Set it to `0` to disable idle reaping. | | `tx-idle-timeout` | Maximum time a connection can remain idle in an open transaction before the transaction is ended and the connection is recycled. Set it to `0` to disable the timeout. | All seven settings are applied dynamically and do not restart Postgres. The dashboard shows the effective value and valid range for the selected profile. `pool-min-conns` cannot exceed `pool-capacity`. The sum of `pool-capacity` and `external-conn-reservation` cannot exceed the profile's available connection budget after PlanetScale's internal reservations. ## Configure admin parameters Admin parameters apply to the admin service for the selected branch. **Step 1** From the PlanetScale organization dashboard, select your database and branch. **Step 2** Open **Clusters**, then select **Admin**. **Step 3** Select the **Configuration** tab. **Step 4** Update the admin parameters, then select **Apply now**. **Step 5** Use the **Changes** tab to follow the change. | Parameter | Default | Description | | - | - | - | | `discovery-poll-interval` | `5s` | How often the admin discovers sidecars and checks their health. | | `recovery-poll-interval` | `10s` | How often the admin analyzes cluster state and starts repairs. | | `acceptable-replication-lag` | `10s` | The maximum replication lag before the admin considers a replica lagging. | | `planned-switchover-timeout` | `1m` | The default window for a planned switchover when the request does not specify one. | The dashboard remains authoritative for each parameter's current valid range. These admin recovery thresholds are separate from the router parameters that control which replicas can receive read traffic. ## Configure router parameters Router parameters apply only to the selected router group. **Step 1** From the PlanetScale organization dashboard, select your database and branch. **Step 2** Open **Clusters**, then select **Routers**. **Step 3** Select the router group you want to configure. **Step 4** Select the **Parameters** tab. **Step 5** Update the replica-routing thresholds, then select **Apply now**. **Step 6** Use the router group's **Changes** tab to follow the change. | Parameter | Default | Description | | - | - | - | | `insights-raw-queries` | `false` | Sends full query text for slow, large, or failed queries to Insights. Query text may contain sensitive data. | | `replication-lag-tolerable-min` | `30s` | Replica reads prefer replicas with replication lag at or below this value. | | `replication-lag-tolerable-max` | `15m` | Replica reads never route to a replica with replication lag above this value. | | `idle-session-timeout` | `30m` | Disconnects a client that remains idle outside a transaction for longer than this value. Set it to `0` to disable the timeout. | | `idle-in-transaction-session-timeout` | `5m` | Disconnects a client that remains idle inside a transaction for longer than this value. Set it to `0` to disable the timeout. | | `stream-pool-size` | `10` | Number of streams kept ready for each sidecar connection. | | `sidecar-conns` | Derived from router memory | Number of gRPC connections to each sidecar. When unset, Neki derives the value from router memory, up to 16. | The minimum threshold must be less than or equal to the maximum threshold. The dashboard validates both values together when you apply the change. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki vs PlanetScale Postgres Source: https://planetscale.com/docs/neki/coming-from-postgres The upgrade path from regular Postgres to Neki PlanetScale lets you create Neki and regular Postgres databases. Both products lead to great performance, and have a full slew of common features like Insights, metrics, automated backups, a hosted MCP server, and much more. Since both are ways of getting a great Postgres database in the cloud, how do you know which to choose when? This article provides guidance. ## What stays the same Both products include: * Insights, metrics, anomalies, and schema recommendations * Automated backups, backup schedules, and point-in-time restore * CLI, API, and MCP support * Standard Postgres clients, drivers, and ORMs * Roles and credentials * Production branches with high availability ## Connections Applications connect in the same way for Postgres and Neki. Ultimately, both are powered by real Postgres nodes. In PlanetScale Postgres, connections from clients either are made as [direct connections](/docs/postgres/connecting) or [through PgBouncer](/docs/postgres/connecting/pgbouncer). Either way, these are normal postgres connections, and work with all postgres clients, drivers, and ORMs. All client connections to a Neki database are made to [Neki routers](/docs/neki/overview#routers). These are also just regular connections, with the same authentication and communication protocol as Postgres. Neki routers are not just a pooling proxy like PgBouncer. Rather, each router contains a full Postgres query parser, planner, buffering capabilities, and is tied in with Neki's health-monitoring system. These then route query requests along to the Postgres node(s) that are needed to respond to the query. In most cases, connections are more scalable in Neki compared to regular Postgres. Making direct connections to Postgres is notoriously difficult to scale, due to Postgres creating a new full unix process per connection. Using PgBouncer helps this significantly, but there can still be some performance and latency issues using PgBouncer. Neki routers allow for much safer connection scalability. Not only do they use more sophisticated connection pooling than PgBouncer, but they also have the advantage of connecting to [sidecars](/docs/neki/overview#what-runs-alongside-postgres) next to each Postgres instance. Neki also makes it easy to scale the number, size, and grouping of these routers, giving you more precise control over how to handle incoming traffic. ## Scale Whereas Neki has full capabilities for massive data sharding, PlanetScale Postgres does not. On PlanetScale Postgres, each database cluster has a single primary. These primaries can get very large, up to 96 vCPUs and over 700 gigabytes of RAM. However, for large databases this can still become a bottleneck. PlanetScale Postgres also allows you to create many read replicas, both in the same region as your primary and in other regions. However, this only allows for read scalability, and does not help scale data size, only query processing capabilities. On the other hand, Neki allows for near-unlimited scalability for reads, writes, and data. With [sharding](/docs/neki/when-to-shard), Neki is capable of spreading out the data from a single logical database onto many distinct shards, each with their own primary and replicas. For example, we may have a large, 100 terabyte database, which we spread out across 64 shards, each storing approximately 1.5 terabytes. This allows us to scale data, but also distribute queries across many more nodes. Neki has been proven to scale to 100 million queries per second and over a petabyte of data. Regular Postgres can scale to tens of terabytes and hundreds of thousands of queries per second. However, the best scalability is on Neki. It's also good to note that Neki is also a great option for smaller, unsharded databases. You get all the other features like online DDL, zero-downtime operations, connection pooling, and version upgrades, even at smaller scales. ## Availability We maintain [high standards for operational excellence](/docs/postgres/operations-philosophy) for our Postgres databases. Due to this, we have industry-leading uptime and availability for Postgres. However, there are some fundamental limitations of Postgres and PgBouncer that lead to availability imperfections. When a switchover occurs, whether due to a node failure or an upgrade cycle, direct connections to PlanetScale Postgres databases must be severed. PgBouncer improves the situation, however its ability to buffer queries and maintain HA during failovers is limited. The nature of running raw Postgres means we must take small windows of downtime during version upgrades, primary server failure, and other events. Neki provides much better guarantees. Because the query path is client → Router → Sidecar → Postgres, we have greater control over keeping the system fully available during switchovers and failovers. Router failure is also easily handled, since we can deploy any number of them we need to have reserve capacity to unexpected node failures. When running Neki in sharded mode, there is the additional benefit of the shared-nothing architecture. If we have 64 shards, a failure of a database primary only means 1/64th of our data becomes impacted during the switchover to replace this node. Neki allows us to operate Postgres with true high-availability, and most maintenance operations complete with zero downtime. ## Backups We have [written extensively](https://planetscale.com/blog/massively-parallel-postgres-backups) about the benefits sharding provides for backups. When your data is sharded, Neki allows for extremely fast backups, since each shard can perform backup work in parallel. PlanetScale Postgres also has automated backups, custom backup schedules, and point-in-time recovery. However, large databases can lead to long backups, since all data is managed by the primary. If backup time is a pain in your current OLTP database, Neki and sharding is a great solution to improve performance. ## Summary ### What features stay familiar * Applications connect with standard Postgres clients and TLS. * Insights, Anomalies, Schema Recommendations * Full CLI, API, and MCP support * Automated backups, backup schedules, point-in-time restore * Choose between network-backed and local NVMe storage * Roles grant permissions. * Production branches are highly available. ### What is different Familiar conventions from PlanetScale Postgres take a different path in Neki. | PlanetScale Postgres | In Neki | | - | - | | Connect to Postgres, or to PgBouncer on port `6432` | Connect to a [router](/docs/neki/connecting) on port `5432` | | Specify connection type by appending a replica name or PgBouncer suffix to the username | Set [`__neki.target`](/docs/neki/connecting#primary-and-replica-routing) or target a connection at a router group | | One primary owns every row | The [data topology](/docs/neki/data-topology) assigns each table to a shard group | | Adding replicas or vertical size adds capacity for the whole database | Adding a shard adds capacity only after topology and a [data-migration workflow](/docs/neki/data-migration) place data there | | A statement runs on one Postgres server | A statement can reach one shard, several shards, or every shard in a table's group | | Transactions execute on a single node | [Cross-shard work](/docs/neki/transactions) does not share a snapshot or atomic commit; single-shard work does | | `ALTER TABLE` runs on the one database | DDL through a router [fans out](/docs/neki/schema-changes) to every managed shard | | Size one Postgres cluster (and optional PgBouncers) | Size [shards, routers, and Admin](/docs/neki/cluster-configuration) separately | ### Send replica reads with a Neki target PlanetScale Postgres routes replica traffic by appending `|replica` to the username, or by using a dedicated replica PgBouncer. Neki does not use that username form. Select **Replica** under **Query target** on the **Connect** page, or set the startup option `-c __neki.target=REPLICA`. You can also run `SET __neki.target = 'replica'` before a transaction. Replica connections are read-only. Writes fail instead of going to a primary. If no eligible replica is available for a destination shard, the query fails instead of falling back to that shard's primary. See [Primary and replica routing](/docs/neki/connecting#primary-and-replica-routing) and [Replicas](/docs/neki/replicas). ### Placement is in the topology On PlanetScale Postgres, connecting to the database is enough. Every table's rows live on that one primary. On Neki, once you begin to shard data, the [data topology](/docs/neki/data-topology) decides where rows go. A table belongs to one shard group, either by an explicit binding or by inheriting a schema, database, or topology default. An unsharded group routes every row of those tables to one physical shard. A sharded group uses a shard key to split rows across shards. Creating a shard provisions Postgres capacity. It does not move tables or rows onto that shard. [Reference tables](/docs/neki/reference-tables-and-gsis) are the exception that stores the same rows on every shard in the bound groups. ### Query planning and routing On PlanetScale Postgres, every query runs on the one cluster. On Neki, a shard key determines how rows of sharded tables are assigned to specific shards using the [data topology](/docs/neki/data-topology). For example, tables that share a `tenant_id` key can keep that tenant's rows on the same shard. That key determines how a Router routes a query. If the query is looking up rows by a single `tenant_id`, the router can send the statement to one shard. Without it, the router sends the same statement to multiple or all shards and combines the response. Use `EXPLAIN` and [Query Insights](/docs/neki/monitoring/query-insights) to see how many shards a statement reaches. See [Query planning](/docs/neki/query-planning) and [When to shard](/docs/neki/when-to-shard). ### Transactions and snapshots On PlanetScale Postgres, a transaction is one Postgres transaction: one snapshot, one commit. On Neki that is still true for work that stays on a single shard. Once a transaction spans more than one shard, those shards do not share a snapshot or an atomic commit. See [Transactions across shards](/docs/neki/transactions) for routing modes and how to keep a transaction on one shard. ### Schema changes Native DDL issued through a router runs on every managed shard immediately. After it commits, wait with `__neki.wait_for_ddl` before sending dependent SQL through other router instances. Managed [schema-change workflows](/docs/neki/schema-changes) add readiness checks and an explicit completion step. Online DDL builds a shadow table and uses the same copy-and-stream process as other Replicator workflows. ### Branches and clusters A Neki branch is still an isolated environment: its own routers, Postgres instances, data, configuration, backups, and billing. An ordinary development branch starts empty. It does not copy the parent branch's schema or data. Just like on PlanetScale Postgres, restore a [backup](/docs/neki/backups) to create a new branch with existing data. On PlanetScale Postgres, configuration such as parameters affects the single cluster. On Neki, shards, routers and the admin service can all be individually resourced and configured. * A [configuration profile](/docs/neki/cluster-configuration#configuration-profiles) sets Postgres size, replica count, parameters, extensions, and sidecar pools for the shards assigned to it. * [Router groups](/docs/neki/cluster-configuration) accept application connections and have their own size. * The [admin service](/docs/neki/cluster-configuration#configure-the-admin-service) health-checks sidecars and coordinates failover. It is not a Postgres role. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect to Neki Source: https://planetscale.com/docs/neki/connecting Postgres clients, roles, router groups, and replica routing Neki uses the Postgres wire protocol, so you can connect with `psql` or any standard Postgres driver. Your applications connect to a Neki router instead of connecting to each shard's primary or replicas. The router plans each statement and sends it to the required Postgres shards. Create a role for each application or permission boundary. The same role can be reused by that application's connection pool and other connections that need the same permissions. ## Get connection details **Dashboard** **Step 1** Open the database in the PlanetScale dashboard and click **Connect**. **Step 2** Optionally, select the branch you want to connect to. **Step 3** Select an existing role, or create a [user-defined role](/docs/neki/connecting/roles) with the permissions your application needs. **Step 4** Under **Connection target**, choose a router group. The `default` router group is selected by default. **Step 5** To send reads to replicas, select **Replica** or **Read-only replica** under **Query target**. This option is separate from the router-group target. **Step 6** Select your framework, language, or **Postgres CLI**, then copy the generated connection details. The generated `psql` command has this form: ```bash theme={null} psql 'host= port=5432 user= password= dbname=postgres sslnegotiation=direct sslmode=verify-full sslrootcert=system' ``` Replace the placeholders with the values from the dashboard. Generated connections use the `postgres` logical database within a cluster. **CLI** Install and authenticate the [PlanetScale CLI](/docs/cli/planetscale-environment-setup) first. **Step 1** Open a `psql` session. `pscale shell` creates temporary credentials, so you do not need a saved role for this step. ```bash theme={null} pscale shell --org ``` **Step 2** To target a router group other than the default, add `--router`. ```bash theme={null} pscale shell \ --org \ --router ``` For a persistent role that your application can reuse, create one with `pscale role`. See [Roles and credentials](/docs/neki/connecting/roles). ## Connection parameters | Parameter | Value | | :- | :- | | Host | The hostname shown on the **Connect** page | | Port | `5432` | | Username | The complete generated username, including any suffix for a non-default router group | | Password | The password shown when the role is created or reset | | Database | logical database, `postgres` by default | | TLS | Required; use the verification settings generated for your client | All connections to Neki databases use port `5432`. ## Secure connections Neki requires TLS. Your client should verify both the certificate chain and the server hostname instead of only encrypting the connection. The generated `psql` command uses `sslmode=verify-full` and the system CA store. Other drivers use different names for the same settings. Select your framework or language on the **Connect** page to get the correct TLS configuration for that client. ## Private connections Keep traffic between your cloud network and Neki off the public internet with [AWS PrivateLink](/docs/neki/connecting/private-connections/aws-privatelink) or [GCP Private Service Connect](/docs/neki/connecting/private-connections/gcp-private-service-connect). Private connections use the same roles, router groups, and required TLS settings as public connections. ## Primary and replica routing By default, Neki sends work to shard primaries. Use that for writes and for reads that need the latest committed data. Set `__neki.target` to send reads to replicas. Selecting **Replica** under **Query target** on the **Connect** page adds this setting to the generated connection options. It does not change the username. To send reads to [read-only replicas](/docs/neki/read-only-replicas), select **Read-only replica** instead. ```sql theme={null} SET __neki.target = 'replica'; ``` Or set it when the connection starts. For `psql`, use `PGOPTIONS`: ```bash theme={null} PGOPTIONS='-c __neki.target=REPLICA' \ psql 'host= port=5432 user= password= dbname=postgres sslnegotiation=direct sslmode=verify-full sslrootcert=system' ``` For a connection URI, add it as the Postgres `options` parameter: ```text theme={null} postgresql://:@:5432/postgres?sslmode=verify-full&sslrootcert=system&options=-c%20__neki.target%3DREPLICA ``` For `pscale shell`, add `--replica`. That sets `__neki.target=REPLICA` for the session. See [Choosing where reads run](/docs/neki/shard-targeting-fanout#choosing-where-reads-run) for Neki's replica selection settings and lag behavior. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect privately with AWS PrivateLink Source: https://planetscale.com/docs/neki/connecting/private-connections/aws-privatelink Private connectivity between an AWS VPC and Neki through AWS PrivateLink. [AWS PrivateLink](https://aws.amazon.com/privatelink/) keeps traffic between your VPC and Neki on the AWS network without traversing the public internet. Use it when your compliance requirements or network architecture require private database connectivity. Standard Neki connections use TLS over the public internet and are appropriate for most applications. PrivateLink adds network isolation; Neki still requires TLS for connections through the private endpoint. ## Pricing PlanetScale bills PrivateLink ingress and egress at **\$0.01 per GB** after the branch's included allowance. AWS also charges for VPC endpoints and data processing. See [Neki pricing](/docs/neki/pricing#private-traffic) and the [AWS PrivateLink pricing page](https://aws.amazon.com/privatelink/pricing/). ## Prerequisites * A Neki database in an AWS region. * An AWS VPC in the same region as the Neki database. * IAM permissions to create VPC endpoints and manage their security groups. * A host in the VPC from which you can test DNS and Postgres connectivity. ## Get the private connection details **Step 1** From the PlanetScale dashboard, select your Neki database. **Step 2** Open **Settings**, then select **Roles**. **Step 3** Select the branch and open the role your application uses. **Step 4** Select the required router group under **Connection target**. **Step 5** Copy the **Private Host** and **Private Service Name**. The private host and service name identify the Neki service. The role's generated username still identifies the branch and selected router group. Copy the complete username from the dashboard instead of constructing it. Use the service name shown for each role and connection target. Do not assume that every Neki service in an AWS region has the same service name. An existing VPC endpoint can be reused only when the dashboard shows the same **Private Service Name** for the new connection. ## Create the VPC endpoint **Step 1** Create a security group for the endpoint that allows inbound TCP traffic on port `5432` from your application resources. Referencing the applications' security group is more restrictive than allowing the entire VPC CIDR. **Step 2** In the AWS console, open **VPC** > **Endpoints** and select **Create endpoint**. **Step 3** For the endpoint type, select **Endpoint services that use NLBs and GWLBs**. **Step 4** Enter the **Private Service Name** from PlanetScale and select **Verify service**. **Step 5** Select the VPC and the subnets your application can reach. Select at least two subnets for availability. **Step 6** Under **Additional settings**, enable the DNS name. **Step 7** Attach the endpoint security group and create the endpoint. **Step 8** Wait until the endpoint status is **Available**. Your application resources must also allow outbound TCP traffic on port `5432`. If the selected subnets use restrictive network ACLs, allow port `5432` and the ephemeral ports required for response traffic. ## Verify the endpoint From a host in the configured VPC, confirm that the **Private Host** resolves to private addresses: ```bash theme={null} dig +short ``` Then connect using the private host and the username and password for your role: ```bash theme={null} psql 'host= port=5432 user= password= dbname=postgres sslnegotiation=direct sslmode=verify-full sslrootcert=system' ``` Update application connection strings by replacing the public host with the private host. Keep the generated username, port, database, TLS settings, and password unchanged. ## Troubleshooting If the connection fails: 1. Confirm that the VPC endpoint status is **Available**. 2. Confirm that the endpoint is in the same AWS region as the Neki database. 3. Check inbound rules on the endpoint security group and outbound rules on the application security group. 4. Check the network ACLs for the endpoint and application subnets. 5. Verify that the private host resolves to addresses in the VPC. 6. Use [AWS Reachability Analyzer](https://docs.aws.amazon.com/vpc/latest/reachability/what-is-reachability-analyzer.html) to inspect the path between the application and endpoint. ## Related documentation * [Connect to Neki](/docs/neki/connecting) * [Roles and credentials](/docs/neki/connecting/roles) * [Neki pricing](/docs/neki/pricing#private-traffic) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect privately with GCP Private Service Connect Source: https://planetscale.com/docs/neki/connecting/private-connections/gcp-private-service-connect Private connectivity between a Google Cloud VPC and Neki through Private Service Connect. [GCP Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect) keeps traffic between your VPC and Neki on the Google Cloud network without traversing the public internet. Use it when your compliance requirements or network architecture require private database connectivity. Standard Neki connections use TLS over the public internet and are appropriate for most applications. Private Service Connect adds network isolation; Neki still requires TLS for connections through the private endpoint. ## Pricing PlanetScale bills Private Service Connect ingress and egress at **\$0.01 per GB** after the branch's included allowance. Google Cloud also charges for Private Service Connect endpoints and applicable network transfer. See [Neki pricing](/docs/neki/pricing#private-traffic) and the [Google Cloud VPC pricing page](https://cloud.google.com/vpc/pricing#private-service-connect). ## Prerequisites * A Neki database in a GCP region. * A Google Cloud VPC in the same region as the Neki database. * IAM permissions to create Private Service Connect endpoints. * The Cloud DNS and Service Directory APIs enabled in the Google Cloud project. * A host in the VPC from which you can test DNS and Postgres connectivity. ## Get the private connection details **Step 1** From the PlanetScale dashboard, select your Neki database. **Step 2** Open **Settings**, then select **Roles**. **Step 3** Select the branch and open the role your application uses. **Step 4** Select the required router group under **Connection target**. **Step 5** Copy the **Private Host** and **Private Service Name**. The private host and service name identify the Neki service. The role's generated username still identifies the branch and selected router group. Copy the complete username from the dashboard instead of constructing it. Use the service name shown for each role and connection target. Do not assume that every Neki service in a Google Cloud region has the same service name. An existing endpoint can be reused only when the dashboard shows the same **Private Service Name** for the new connection. ## Create the Private Service Connect endpoint **Step 1** In the Google Cloud console, open **Private Service Connect** and select **Connected endpoints**. **Step 2** Select **Connect endpoint**. **Step 3** Select **Published service** as the target. **Step 4** Enter the **Private Service Name** from PlanetScale as the target service. **Step 5** Enter a descriptive endpoint name. The name becomes the first segment of the private connection hostname. **Step 6** Select the VPC and a subnet that your application resources can reach, then reserve an IP address for the endpoint. **Step 7** Enable global access if resources in other Google Cloud regions need to reach this endpoint. **Step 8** Create a Service Directory namespace so Google Cloud can create the private DNS record automatically. **Step 9** Create the endpoint and wait until its status is **Accepted**. ## Build and verify the private hostname The hostname combines the endpoint name from Google Cloud with the **Private Host** from PlanetScale: ```text theme={null} . ``` For example, an endpoint named `planetscale-main` and a private host of `gcp-us-central1-1.private-pg.psdb.cloud` produce: ```text theme={null} planetscale-main.gcp-us-central1-1.private-pg.psdb.cloud ``` From a host in the configured VPC, confirm that the hostname resolves to the endpoint's private address: ```bash theme={null} dig +short . ``` Then connect using the private hostname and the username and password for your role: ```bash theme={null} psql 'host=. port=5432 user= password= dbname=postgres sslnegotiation=direct sslmode=verify-full sslrootcert=system' ``` Update application connection strings by replacing the public host with the combined private hostname. Keep the generated username, port, database, TLS settings, and password unchanged. ## Troubleshooting If the connection fails: 1. Confirm that the endpoint status is **Accepted**. 2. Confirm that the endpoint is in the same GCP region as the Neki database. 3. Verify that the Cloud DNS and Service Directory APIs are enabled. 4. Check that the application can route to the endpoint subnet. 5. Confirm that the combined hostname resolves to the endpoint's private address. 6. Check firewall rules for outbound TCP traffic on port `5432`. ## Related documentation * [Connect to Neki](/docs/neki/connecting) * [Roles and credentials](/docs/neki/connecting/roles) * [Neki pricing](/docs/neki/pricing#private-traffic) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Roles and credentials Source: https://planetscale.com/docs/neki/connecting/roles Create and manage branch-scoped Postgres roles for Neki. A role in Neki has scoped access across all shards within a single branch. The **Roles** dashboard page tracks roles created through the dashboard, API, or PlanetScale CLI. Roles created directly with Postgres `CREATE ROLE` are not managed on that page. You are responsible for their passwords, permissions, and lifecycle. ## Default and user-defined roles The default `postgres` role is best reserved for administrative work. All application traffic should connect via user-defined roles with the least permissions that the connection needs. Resetting the default role's credentials replaces the password, so every application using that role must be updated together. User-defined roles can be rotated independently. The dashboard selects `pg_read_all_data` and `pg_write_all_data` by default, which fits a typical read-write application. Remove either permission when the application does not need it. Common role configurations are: | Use case | Inherited roles | | :- | :- | | Read-only application or reporting tool | `pg_read_all_data` | | Read-write application | `pg_read_all_data`, `pg_write_all_data` | | Monitoring tool | `pg_monitor` | | Read Neki workflow status and use DDL barriers | `pg_read_all_data`, `neki_viewer` | | Run Neki workflow mutations | `postgres`, `neki_operator` | | Administrative work | `postgres` | ## Create a role **Dashboard** **Step 1** Open the database, then go to **Settings** and select **Roles**. **Step 2** Select the branch and click **New role**. **Step 3** Enter a name and select the inherited roles that grant the required permissions. Names can contain up to 63 lowercase letters, numbers, dashes, or underscores. A name cannot start or end with a dash or underscore. **Step 4** Create the role and save its password. Wait until the role is ready before connecting with it. PlanetScale does not show the password again after you leave the credentials view. You can also create a role while connecting. Open the database, click **Connect**, select the branch, and click **Create new role**. **CLI** Install and authenticate the [PlanetScale CLI](/docs/cli/planetscale-environment-setup) first. **Step 1** Create a role on the branch. The command prints the password once. Save it before you close the terminal. ```bash theme={null} pscale role create app \ --inherited-roles pg_read_all_data,pg_write_all_data ``` Change `--inherited-roles` to match the permissions the connection needs. Wait until the role is ready before connecting with it. Add `--ttl 24h` to create temporary credentials. The CLI accepts a duration or a number of seconds. A role created without a TTL does not expire automatically. ```bash theme={null} pscale role create app \ --inherited-roles pg_read_all_data,pg_write_all_data \ --ttl 24h ``` Add `--with-replication` only for a client that needs to start logical replication. It requires `--inherited-roles` to include `postgres`. ```bash theme={null} pscale role create replicator \ --inherited-roles postgres \ --with-replication ``` ## Available inherited roles | Role | Grants | | :- | :- | | `pg_read_all_data` | Read all tables, views, and sequences | | `pg_write_all_data` | `INSERT`, `UPDATE`, and `DELETE` on all tables, views, and sequences. | | `pg_read_all_settings` | Read all configuration variables | | `pg_read_all_stats` | Read all `pg_stat_*` views | | `pg_stat_scan_tables` | Run monitoring functions that may take `ACCESS SHARE` locks | | `pg_monitor` | Use Postgres monitoring views and functions | | `pg_signal_backend` | Cancel a query or terminate another backend session | | `pg_checkpoint` | Run `CHECKPOINT` | | `pg_maintain` | Run maintenance commands such as `VACUUM`, `ANALYZE`, and `REINDEX` | | `pg_use_reserved_connections` | Use reserved connection slots | | `pg_create_subscription` | Create logical replication subscriptions when the role also has the required database permission | | `neki_viewer` | Read Neki workflow status and run functions such as the DDL propagation barrier; requires `pg_read_all_data` | | `neki_operator` | Run Neki workflow mutation functions; requires `postgres` | | `postgres` | Perform broad administrative work | `pg_read_all_data` and `pg_write_all_data` grant data access. They do not grant permission to create or alter tables. Add the `postgres` inherited role only when the connection needs administrative access. Neki enforces inherited-role dependencies. Selecting `neki_viewer` also requires `pg_read_all_data`; selecting `neki_operator` also requires `postgres`. Use separate least-privilege application and migration roles when an application does not need DDL or workflow access. The **WITH REPLICATION** attribute is available only when the role inherits `postgres`. Enable it only for a client that needs to start logical replication. ## Connection credentials Each managed role has: * A display name used in the dashboard. * A generated Postgres login name. * A password shown only after the role is created or reset. * A branch assignment. The complete connection username includes a branch suffix. Use the username from the **Connect** page rather than constructing it yourself. Renaming a role changes its display name, not its generated login name. See [Connect to Neki](/docs/neki/connecting) for the complete connection string and replica-routing options. ## Manage a role Resetting credentials creates a new password. Update every client that opens new connections with that role. Before deleting a user-defined role that owns tables, schemas, or other objects, reassign those objects to another role. If the role is still referenced by another object, Neki leaves it in place and reports that it cannot be deleted. **Dashboard** From **Settings** > **Roles**, open a role to: * View its connection strings. * Rename its display name. * Reset its credentials. * Reassign objects it owns. * Delete it. Use **Reassign objects**, or select **Reassign owned objects** while deleting the role. Both options transfer ownership to `postgres`. **CLI** List roles on a branch, then copy the role ID for later commands: ```bash theme={null} pscale role list ``` Get a role's details: ```bash theme={null} pscale role get ``` Rename a role's display name: ```bash theme={null} pscale role update \ --name ``` Reset a user-defined role's password. The command prints the new password once: ```bash theme={null} pscale role reset ``` Reset the default `postgres` role: ```bash theme={null} pscale role reset-default ``` Reassign objects owned by a role. The CLI can transfer ownership to `postgres` or another role: ```bash theme={null} pscale role reassign \ --successor postgres ``` Delete a role. Pass `--successor` when the role owns objects: ```bash theme={null} pscale role delete \ --successor postgres ``` Extend a temporary role before it expires: ```bash theme={null} pscale role renew ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Data migration Source: https://planetscale.com/docs/neki/data-migration Table movement, resharding, verification, and traffic cutover Data migrations move tables to a different placement or reshard them without taking the application offline. Neki copies the existing rows and keeps the target updated as new writes come in. The source continues serving traffic until traffic switches to the target. If the data starts outside Neki, first [import it into an unsharded Neki database](/docs/neki/imports/postgres). After that placement is stable, **Reshard** redistributes declared tables in one source shard group onto a new shard group in the same database. **MoveTables** is a different workflow. It copies selected tables to another database, or to a shard group whose physical shards are not the source shards. > [!NOTE] > > During Platform Preview, MoveTables and Reshard are not managed in the > PlanetScale dashboard and have no separate CLI workflow. Create, inspect, and > operate them with Neki metafunctions on a SQL connection to a Neki router. > Mutation functions require `neki_operator`; status and report functions > require `neki_viewer`. The default `postgres` role is broader than this > workflow authorization contract. Connect to a router that serves primary > traffic. ## How workflows are managed The router is the management entry point for a Neki data-movement workflow. Create, status, start, stop, differ, traffic switch, cancel, and complete are Neki metafunctions. Use `reshard_create` for Reshard and `move_tables_create` for MoveTables. After create, the shared `workflow_*` metafunctions operate either workflow. There is no dashboard or CLI fallback during Platform Preview. A workflow created through the router continues running after the SQL session that created it disconnects. Later sessions use the workflow name to inspect it or request its next operation. The sections below explain the lifecycle and the checks required before each operation. They describe Neki workflows for data that is already inside Neki. ### Router metafunctions | Operation | Metafunction | | - | - | | Create a Reshard workflow | `__neki.reshard_create(workflow, database, source_shard_group_uid, target_shard_group, target_shard_group_uid, options)` | | Create a MoveTables workflow | `__neki.move_tables_create(workflow, source_db, source_database_topology, target_db, target_database_topology, source_tables, target_tables, options)` | | List workflows | `__neki.list_workflows()` | | Status | `__neki.workflow_status(workflow)` | | Start or resume | `__neki.workflow_start(workflow)` | | Stop | `__neki.workflow_stop(workflow)` | | Create and start a differ | `__neki.differ_create(workflow, differ_name, options)` | | Differ status | `__neki.differ_status(workflow, differ_name)` | | Differ report | `__neki.differ_report(workflow, differ_name)` | | Delete a differ | `__neki.differ_delete(workflow, differ_name)` | | Switch reads | `__neki.workflow_switch_reads(workflow)` | | Switch writes | `__neki.workflow_switch_writes(workflow, options)` | | Switch reads and writes | `__neki.workflow_switch_traffic(workflow, options)` | | Cancel before traffic switch | `__neki.workflow_cancel(workflow, options)` | | Complete after cutover | `__neki.workflow_complete(workflow, options)` | Workflows start when created unless the create options include `"create_stopped": true`. Review a stopped workflow, then call `workflow_start`. ### Workflow options Create options are a JSON object. Unknown values are rejected. The most useful correctness and load controls are: | Option | Behavior and constraints | | - | - | | `create_stopped` | Create the schema, streams, topology merge, and target block without starting data movement | | `stop_after_copy` | Stop after the initial copy; unlike `create_stopped`, copying begins | | `copy_batch_size` | Rows per copy batch; `0` uses the format-specific default | | `copy_phase_duration` | Maximum snapshot-hold duration per copy cycle; `0` uses the default | | `max_concurrent_table_streams` | Requested streams per source/target shard pair; connection limits can queue excess streams | | `read_from_standby` | Read the snapshot and bulk-copy data from a source standby; CDC stays on the primary. Requires Postgres 16 or later on the source | | `on_ddl` | Exact values are `ON_DDL_ACTION_STOP` and `ON_DDL_ACTION_IGNORE`; omitted means `ON_DDL_ACTION_IGNORE` | | `skip_defer_secondary_keys` | Keep target secondary indexes in place during copy instead of rebuilding them afterward | | `skip_grants` | Do not replay source grants; apply and verify target privileges separately | | `skip_source_extensions` | Omit source extension creation from schema deployment; defaults to `true` for an external source and `false` for a Neki-managed source | | `dry_run` | Validate and report a MoveTables plan without creating it | | `publication_name` | Reuse a MoveTables source publication; MoveTables only and incompatible with `ON_DDL_ACTION_STOP` | | `import_cluster_auth` | Import source roles for MoveTables; choose `IMPORT_CLUSTER_AUTH_FULL` or `IMPORT_CLUSTER_AUTH_WITHOUT_PASSWORDS` only after reviewing their target-role effects | | `skip_existing_roles` | Leave same-named target roles unchanged; requires `import_cluster_auth` | `dry_run`, `publication_name`, `import_cluster_auth`, and `skip_existing_roles` are MoveTables-only. `ON_DDL_ACTION_STOP` requires PostgreSQL 14 or later and a superuser source login because Neki must create an event trigger. Many managed external sources do not grant that privilege. Only `copy_batch_size` and `copy_phase_duration` can be changed on a running workflow: ```sql theme={null} SELECT * FROM __neki.workflow_set_options( 'reshard_events', '{"copy_batch_size": 500, "copy_phase_duration": "60s"}' ); ``` List the metafunctions your router currently exposes: ```sql theme={null} SELECT name, arguments, purpose FROM __neki.list_metafuncs() WHERE group_name IN ('workflows', 'cutover', 'verification'); ``` ![A migration starts, initializes, copies existing rows, streams new changes, switches traffic, and completes. The source serves traffic until the switch, then the target serves. Stop and Start preserve progress. Cancel is available before the switch.](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/data-migration/migration-lifecycle.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=fb172876596864c55d14f2e0e7c96309) ## What a migration workflow moves Neki has two workflows for changing where existing data lives: | Workflow | What changes | | - | - | | **MoveTables** | Selected tables move to another database, or to a shard group whose physical shards are not the source shards | | **Reshard** | Declared base tables in one source shard group are redistributed into a target shard group within the same database | Workflows start by default. They can instead be created in a stopped state, reviewed, and started later. Throughout the move, the [data topology](/docs/neki/data-topology) determines which target shard receives each row. Before creating a Reshard workflow, declare every physical base table that resolves to the source group in the data topology. A declared table can inherit its shard group from a database, schema, or cluster default. Neki refuses to start if a physical table uses that default but has no table entry in the data topology. At cutover, database and schema defaults that point to the source group move to the target group. If the database used the cluster default, Neki adds an explicit database default for the target group. A Reshard workflow can split the initial authoritative shard group. When its target contains multiple shards, table placement moves to the target group but cluster authority remains on the original single-shard group. The original group continues to provide database authority and hold system tables and sequences that cannot move to the multi-shard target. The original shard does not become one of the data shards in the split. A two-way split of existing data therefore uses three shards. ## Reshard existing tables The example redistributes `public.events` by `tenant_id` from one unsharded source shard onto two new shards. Replace shard UIDs, the database name, and the table list with values from your branch. You do not need to shard every table. Bind only the tables that must move, and the tables that must stay colocated with them, to the source group. Leave tables that should stay unsharded on the [authoritative shard group](/docs/neki/data-topology#authoritative-shard-group). Declare every physical base table that resolves to the source group, including tables that inherit that group from a database, schema, or cluster default. Put related foreign-key tables in the same group. Reshard does not automatically repoint dependent views. ### Add destination shards Add two new shards for a two-way split. Wait until each destination shard has a ready primary. Do not reuse the source shard as a target. **Dashboard** Open the database, go to **Clusters**, select the configuration profile, open the **Shards** tab, and select **Create new shards**. Enter `2`, review the estimated cost, and add the shards. **CLI** ```bash theme={null} pscale branch shard create main \ --config-profile default \ --count 2 ``` ```sql theme={null} SELECT uid, name FROM __neki.list_shards(); ``` ### Declare the source group Creating shards does not move existing rows. Apply a topology that keeps serving the current shard and binds the tables you will reshard to a named source group on that same shard. Declare the shard index the target group will use. A topology update is a complete replacement. You can edit the document in **Clusters** > **Data topology**, or apply it with SQL; see [Data topology](/docs/neki/data-topology). ```sql theme={null} SELECT * FROM __neki.set_data_topology( $$ { "shard_indexes": { "xxhash_tenant_id": { "type": "xxhash", "columns": ["tenant_id"] } }, "shard_groups": [ { "uid": "", "key_ranges": [ { "shard_uid": "" } ] }, { "uid": "imported", "key_ranges": [ { "shard_uid": "" } ] } ], "databases": { "postgres": { "schemas": { "public": { "tables": { "events": { "shard_group": "imported" } } } } } }, "default_shard_group": "", "authoritative_shard_group": "" } $$::text, true ); ``` ```sql theme={null} SELECT __neki.wait_for_data_topology(); ``` ### Create, copy, switch, and complete Workflows start when created unless you pass `"create_stopped": true`. The target group's `shard_uid` values must be the two new shards only. ```sql theme={null} SELECT __neki.reshard_create( 'reshard_events', 'postgres', 'imported', '{ "default_shard_index": "xxhash_tenant_id", "key_ranges": [ {"shard_uid": "", "end": "80"}, {"shard_uid": "", "start": "80"} ] }', 'events_by_tenant', '{"create_stopped": true}' ); SELECT __neki.workflow_start('reshard_events'); SELECT * FROM __neki.workflow_status('reshard_events'); ``` Wait until every expected stream is `running` in the `streaming` phase. Then create a differ, review the report, and switch traffic. Confirm streams are streaming again after the differ; cutover refuses streams that are not. ```sql theme={null} SELECT __neki.differ_create('reshard_events', 'pre_cutover'); SELECT * FROM __neki.differ_status('reshard_events', 'pre_cutover'); SELECT * FROM __neki.differ_report('reshard_events', 'pre_cutover'); SELECT * FROM __neki.workflow_switch_traffic('reshard_events'); SELECT __neki.workflow_complete('reshard_events'); ``` The later sections cover differ acceptance, staged read/write switches, cancel, and source cleanup. ## From copy to streaming A migration first copies rows that already exist. Neki works in batches while continuing to incorporate changes committed on the source. Once the initial copy is finished, the workflow enters `streaming` and continues applying new source changes to the target. Reaching `streaming` means the target is being kept current. Reads and writes still go to the source until traffic switches to the target. ### Copying with an iteration key Each table needs an **iteration key**: a stable, unique order in which Neki can copy its rows. Neki prefers the primary key. If a table has no primary key, a suitable non-nullable unique key can serve instead. The iteration key controls copy order and resumption. The shard key determines where a row belongs. `public.events` is sharded by `tenant_id` and has a primary key named `event_id`. Neki can copy the table in `event_id` order while using `tenant_id` to choose the target shard. When a batch commits, its rows and copy progress are saved together. Later batches continue from that committed point while source changes affecting earlier rows are incorporated into the target. > [!WARNING] > > A migration stream does not apply source data definition language (DDL) > changes to the target. Set `on_ddl` to the exact value > `ON_DDL_ACTION_STOP` to watch migrated source tables and stop for relevant > table-shape changes, schema renames, and unrecognized changes associated with > a migrated table. The stop occurs before row changes from that source > transaction are applied. This stop is terminal. Continuing requires > cancelling and recreating the workflow, which copies the data again under > the new schema. > > `ON_DDL_ACTION_STOP` requires PostgreSQL 14 or later and a superuser source > login because Neki must create an event trigger. Many managed external > sources do not grant that privilege, so this mode is unavailable for those > imports. It also cannot be combined with `publication_name`. > > If `on_ddl` is omitted, Neki treats it as `ON_DDL_ACTION_IGNORE`. Set that > exact value to select the behavior explicitly. The stream keeps running, so > you must keep the source and target schemas compatible. > > Index and constraint changes do not trigger that stop, even when they alter > replica identity indirectly. Replica identity determines how PostgreSQL finds > rows for updates and deletes. Continued streaming does not prove that the > schemas remain compatible. The migration stream does not apply those schema > changes to the target. Because the migration stream does not apply DDL, source and target schemas must be kept compatible through separate schema changes. See [Schema changes](/docs/neki/schema-changes) for Online DDL and direct schema-change workflows. ### Durable progress and resumption Neki saves copy progress with the rows in each committed batch. It also saves streaming progress with the changes applied on the target. If the workflow is interrupted, it resumes from durable progress rather than recopying the entire table. ## Reading migration progress Inspect one workflow or list every workflow: ```sql theme={null} SELECT * FROM __neki.workflow_status('reshard_events'); SELECT workflow, type, status, traffic_state, phase, streams_streaming, streams_copying, streams_error FROM __neki.list_workflows(); ``` An active migration stream moves through these phases: | Phase | Meaning | | - | - | | `initializing` | Neki is validating and preparing the migration | | `copying` | Existing rows are still being transferred | | `streaming` | Initial copy is complete and source changes continue reaching the target | A stream also reports whether it is running, stopped, or in error. The traffic state shows whether the source or target serves application traffic. Status is reported for each stream and target shard. Before traffic can move, every expected stream must be running and in the `streaming` phase. Copy progress ends at `streaming`. Cutover and completion are separate, explicitly requested stages. ### Stop and Start preserve progress Stop pauses a workflow without abandoning the migration. Its saved phase and progress remain in place. Start resumes the workflow from that state. A stopped workflow stays stopped until it is started again. Use Stop when the migration should continue later. Cancel removes the workflow. By default, Cancel also cleans up data created or copied on the target. You can choose to keep that data. ```sql theme={null} SELECT __neki.workflow_stop('reshard_events'); SELECT __neki.workflow_start('reshard_events'); ``` ## Validate before cutover A **differ** compares the source and target rows for a workflow. Creating a differ starts it. Its report shows mismatched rows and rows found only on the source or target. ```sql theme={null} SELECT __neki.differ_create('reshard_events', 'pre_cutover'); SELECT * FROM __neki.differ_status('reshard_events', 'pre_cutover'); SELECT * FROM __neki.differ_report('reshard_events', 'pre_cutover'); ``` Review the differ report before switching traffic. Cutover readiness checks that every expected stream is running and streaming. It does not check the differ. Approve cutover only when all of the following are true: * The top-level report has `complete` set to `true`. * `mismatch` is `false`. * Shard errors are empty or absent. * Every expected table is present in `table_results`. * Every table has `completed` set to `true`. * The report finds no missing, extra, or mismatched rows. Counts are partial when `complete` is false or a table's `completed` field is false. `rows_compared` of `0` does not prove that an unfinished table is empty. Do not treat a complete differ without mismatches as the only proof that a migration is ready. Confirm that every expected table and stream was included, review the compared and unmatched row counts, and perform an independent application-level check of critical tables before cutover. After the differ finishes, confirm every stream is `running` in `streaming` again before switching traffic. The differ can restart streams, and cutover refuses streams that are not streaming. ## Moving traffic Traffic can move all at once or in stages. A staged switch moves non-primary reads (`replica` and `rdonly`) first while primary reads and writes stay on the source. Writes move in a separate step after the target has served read traffic. ```sql theme={null} SELECT * FROM __neki.workflow_switch_reads('reshard_events'); SELECT * FROM __neki.workflow_switch_writes('reshard_events'); ``` To move reads and writes together: ```sql theme={null} SELECT * FROM __neki.workflow_switch_traffic('reshard_events'); ``` During a write switch, Neki buffers affected queries, prevents new source writes to the tables being moved, and waits for previously accepted changes to reach the target. It then updates the routing topology so reads and writes use the target. Before updating the topology, Neki synchronizes sequences owned by the tables being moved. It finds the highest value in each owning column across the target shards and sets the sequence on its resolved shard to that value. This prevents the next sequence value from colliding with a copied row. The switch can be retried if it is interrupted. Once the cutover decision is durable, retries finish the switch instead of returning serving authority to the source. Returning traffic to the source after writes have moved is not a documented Platform Preview rollback path. Do not plan cutover around reversing a write switch. ## Cancel or Complete Choose the operation from the current traffic state and the intended outcome: | Situation | Operation | Result | | - | - | - | | Traffic has not moved and the migration should be abandoned | **Cancel** | Stop and remove the workflow. Disposable target data is removed by default unless it is explicitly retained | | Traffic has moved and the target is final | **Complete** | Retire migration resources and close the workflow | ```sql theme={null} SELECT __neki.workflow_cancel('reshard_events'); ``` You cannot use Cancel after traffic has moved. Unlike Stop, Cancel cannot be resumed. Pass `{"keep_data": true}` to retain target data. Complete is separate from the traffic switch. It removes the workflow's temporary migration resources after the target has become final. Source data is retained by default for safety. ```sql theme={null} SELECT __neki.workflow_complete('reshard_events'); ``` To truncate old source rows after you no longer need them: ```sql theme={null} SELECT __neki.workflow_complete( 'reshard_events', '{"drop_source_data": true}' ); ``` After you verify the move, you can choose how to handle the old source data. For MoveTables, you can rename an old source table to `__old`, so `events` becomes `_events_old`. If you choose source cleanup, Neki truncates tables that keep the same database, schema, and name. It drops old source tables that moved to a different database, schema, or name. You cannot both rename and remove the old source data. Reshard keeps table identifiers unchanged, so its source tables cannot be renamed. Neki does not drop these tables during Complete. If you choose source cleanup, it truncates their rows on the old source shards. You can remove an old source shard later if it no longer holds other data and it is not the [authoritative shard group](/docs/neki/data-topology). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Data topology Source: https://planetscale.com/docs/neki/data-topology How Neki maps tables and shard keys to shards The data topology is the document that describes a Neki cluster. It declares four kinds of object: * Databases — the logical schema (databases, schemas, tables, sequences). The same shape as a Postgres catalog. * Shards — the units of physical storage. Each one is a Postgres primary plus its replicas, holding bytes on disk. Failover, lag, and disk capacity are properties of shards. * Shard groups — named key-range layouts. Each one is a list of key ranges, each pointing at a shard. Tables attach to a shard group. * Shard indexes — named hashing recipes (type, columns, params). Shard groups and tables reference them by name. The data topology is what we use to manage which tables are sharded, which are not, and how the data is spread across shards. Having a good data topology is critical for a Neki database that has good performance. Below is an example of a small, albeit complete, data topology that shards an `events` table by `tenant_id`: ```json theme={null} { "databases": { "analytics": { "schemas": { "public": { "tables": { "events": { "shard_group": "tenant_data" } } } } } }, "shard_groups": [ { "uid": "tenant_data", "default_shard_index": "xxhash_tenant_id", "key_ranges": [ { "shard_uid": "shard-a", "end": "40" }, { "shard_uid": "shard-b", "start": "40", "end": "80" }, { "shard_uid": "shard-c", "start": "80", "end": "c0" }, { "shard_uid": "shard-d", "start": "c0" } ] }, { "uid": "authoritative", "key_ranges": [ { "shard_uid": "shard-z" } ] } ], "shard_indexes": { "xxhash_tenant_id": { "type": "xxhash", "columns": ["tenant_id"] } }, "authoritative_shard_group": "authoritative" } ``` This is a lot to take in, so let's review section by section. ## Databases The databases section is where we specify which Postgres logical databases and schemas this Neki cluster will contain. In this example, we are only specifying a single `events` table: ```json theme={null} "databases": { "analytics": { "schemas": { "public": { "tables": { "events": { "shard_group": "tenant_data" } } } } } } ``` It may seem unnecessarily nested, but all of this is for good reason. Using this level of detail in the hierarchy allows us to precisely specify the names of the logical databases, schemas, and tables. The one table listed here is `events` within the default `public` schema inside of the `analytics` logical database. The primary thing that needs to be specified for a table is the `shard_group`. This file tells Neki to put this in the `tenant_data` group. ## Shards Every Neki cluster is made up of one or more shards. However, shards are not specified or created here in the data topology. Instead, new shards are created on the [cluster configuration page](/docs/neki/cluster-configuration) of your Neki database. Each shard is assigned to a configuration profile, which is how you specify the size, extensions, and Postgres configuration for that set of shards. Each of these shards has a `shard_uid` which will be needed when setting up shard groups here in the data topology. ## Shard groups After having created one or more shards, shards are assigned to one or more shard groups. Our example from earlier contained two groups: ```json theme={null} "shard_groups": [ { "uid": "tenant_data", "default_shard_index": "xxhash_tenant_id", "key_ranges": [ { "shard_uid": "shard-a", "end": "40" }, { "shard_uid": "shard-b", "start": "40", "end": "80" }, { "shard_uid": "shard-c", "start": "80", "end": "c0" }, { "shard_uid": "shard-d", "start": "c0" } ] }, { "uid": "authoritative", "key_ranges": [ { "shard_uid": "shard-z" } ] } ] ``` The first one, `tenant_data`, is where we will store the `events` table data, since we assigned that table to this shard group. This group has four shards, `shard-a`, `shard-b`, `shard-c`, and `shard-d`, and uses the `xxhash_tenant_id` shard index (more on that soon). Any table assigned to this shard group will be distributed across these shards. The way in which the row distribution happens depends on the shard index. Here, we use `shard-?` labels as the shard id, but on a real Neki cluster in your PlanetScale dashboard, you will see these as longer, unique hashed values (e.g., `sh57wz7p7tblk2`). The second shard group is named `authoritative`. Every Neki cluster must specify an [authoritative shard](/docs/neki/terminology#authoritative-shard-group). The authoritative shard group provides the canonical Postgres object identifiers (OIDs) and schema events for a database. We tell the cluster that this is the authoritative shard with this extra line in the data topology JSON: ```json theme={null} "authoritative_shard_group": "authoritative" ``` A Postgres table can name the shard group or inherit a default. ![A value from events.tenant\_id passes through the xxhash\_tenant\_id shard index, becomes a routing value, matches a key range in tenant\_data, and resolves to that range's shard UID](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/data-topology/topology-map.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=1caf2b88746bd96d7e7e12d131724364) The final piece, shard indexes, will control how all of the rows from `events` get placed onto shards A-D, and in turn determine how queries will be routed from the Neki Router. ## Shard indexes A shard index is a named routing function that converts a table's shard key (the columns that will be used to determine shard placement) into a routing key (how the Router distributes the rows). Neki supports three shard-index types. ```json theme={null} "shard_indexes": { "xxhash_tenant_id": { "type": "xxhash", "columns": ["tenant_id"] } } ``` This shard index is named `xxhash_tenant_id`. Any table using this will shard rows based on an `xxhash` of the single entry in `columns` (in this case, the `tenant_id` column). `columns` accepts one entry; a list of more than one column is rejected when the topology is applied with NK013 code 10. Every time the cluster receives a new row for a table using this hash index, Neki will: 1. Extract the `tenant_id` column 2. Run this value through the deterministic `xxhash` function (specifically `XXH3-64`). 3. Using the `start` and `end` values from the corresponding `shard_group`, determine which shard this row must be routed to. 4. Send the row to the correct shard for storage. ## Choosing a shard index Neki supports three shard-index types. A topology that attempts to name another type is rejected. | Type | Use | | - | - | | `xxhash` | Distribute supported shard-key values across a hexadecimal keyspace | | `modulo` | Map an integer key into a fixed number of buckets | | `range` | Route an integer key by its value | `modulo` requires a positive `modulus` in `index_params`. Both `modulo` and `range` accept Postgres `int2`, `int4`, and `int8` values. `xxhash` accepts `text`, `varchar`, `bpchar`, `bytea`, `int2`, `int4`, `int8`, `float4`, `float8`, `numeric`, `date`, `timestamp`, `timestamptz`, and `uuid` column types. It hashes enum values by label. Arrays, `json`, `jsonb`, and anonymous records are not supported. > [!WARNING] > > Neki rejects a shard-key definition whose Postgres type does not have a > routing rule. > It does not wait for the first row or query to discover an unsupported type. After defaults and table overrides are resolved, the `columns` list contains exactly one entry. That entry is a column name or a deterministic expression over a single unqualified column, such as `lower(email)` or `tenant_id * 1000`. An expression that names more than one column is still one entry, so the topology accepts it: ```json theme={null} { "type": "xxhash", "columns": ["tenant_id * 1000 + user_id"] } ``` > [!WARNING] > > `INSERT ... VALUES` into a table sharded by an expression over more than one > column returns NK013 code 10. Its message is > `multi column shard index inserts are not supported`. `INSERT ... SELECT` > into that table returns NK013 code 117. > A one-column expression routes. `SELECT` and `DELETE` scatter unless `=` > names every base column of that expression, which returns NK013 code 115. > A comparison other than `=` still scatters. `UPDATE` of a column that is > not part of the expression follows the same rule. `UPDATE` of a base column > of the expression returns NK013 code 145, including when there is no > `WHERE`. Shard indexes live in the top-level `shard_indexes` catalog. Multiple shard groups can contain the same shard. The topology's **shard index** defines how Neki routes rows; it is not a PostgreSQL table index. Create a PostgreSQL index on the shard-key columns when they are not already covered by a primary key or another suitable index. These columns commonly appear in joins and predicates. When planning queries, Neki uses PostgreSQL's column statistics to estimate which plan will work best. PostgreSQL's autovacuum refreshes these statistics as data changes. After a bulk load or import, collect them explicitly with `ANALYZE`, as described in the [Postgres import guide](/docs/neki/imports/postgres#restore-into-neki). > [!WARNING] > > A table-level primary index must produce values that fit the shard group's > range layout. Changing the index type without changing the ranges can > concentrate rows on one shard or leave some values without a matching > destination. ## Single-shard and multi-shard queries When the Neki router receives a query, it uses the database schema and topology to determine which shard(s) will fulfill the query. For example, a predicate on `events.tenant_id` can narrow it to be able to execute on a single shard: ```sql theme={null} SELECT * FROM events WHERE tenant_id = 4821; ``` Neki uses `tenant_id = 4821` to route this query. It maps `4821` through `xxhash_tenant_id` and finds the matching key range in `tenant_data`. It then sends the query to that range's shard. Co-location means that related rows from different tables are stored on the same shard. To co-locate, say, the `events` and `event_metadata` tables, bind both tables to `tenant_data` and route both through `xxhash_tenant_id`. Rows with the same `tenant_id` then map to the same shard while the topology is unchanged. If Neki cannot narrow the scope of a query using a shard-key predicate, it sends the query to every shard in the table's group. Each shard calculates its part of an aggregate such as `COUNT(*)`, and Neki combines those results before returning the answer. More about inspecting a statement's routing reach can be found in [Query planning](/docs/neki/shard-targeting-fanout#controlling-fanout). ## Schema and database defaults Neki resolves an application table's shard group in this order: 1. The table's `shard_group`. 2. The schema's `default_shard_group`. 3. The database's `default_shard_group`. 4. The topology's `default_shard_group`. Use a default when tables share one placement and an explicit binding for exceptions. A table does not need an entry in `tables` to inherit a default: Neki applies the effective schema, database, or cluster default to unlisted application tables as their schemas are loaded. If no binding or default resolves, the table is not routable. ## Sequence placement A sequence is a Postgres object that returns numeric values, such as values used by `nextval()`. In Neki, each sequence resolves to a shard group that must contain exactly one shard. A sequence owned only by columns of unsharded tables follows those tables when they all resolve to the same single-shard group. This includes the ownership link created for `SERIAL` and identity columns. A sequence falls back to its database's authoritative shard group when it has no owner, has a sharded owner, or has owners in different groups. A sequence entry in the data topology can set an explicit `shard_group`, which overrides the derived placement. > [!WARNING] > > Changing a sequence's placement in the topology does not copy or synchronize > its current value. Coordinate sequence placement changes with PlanetScale > Support as part of a migration plan. When the router's local range is empty, it reserves another batch from the sequence shard's primary. Neki uses the router's configured reservation size, or the Postgres sequence's `CACHE` size when it is larger, and then returns values from that cached range. A larger `CACHE` reduces how often routers return to the sequence shard. Routers reserve a new range when they start or exhaust a batch, so adding or removing routers does not require moving the sequence. ## Authoritative shard group An authoritative shard group identifies the single shard Neki uses as the source of truth for database metadata. That shard publishes schema changes to routers. It is also the fallback for sequences that do not derive their placement from an owned unsharded table and do not name another shard group. It does not control where application table data is stored. Postgres assigns object identifiers (OIDs) independently on each shard. Custom types and other catalog objects expose those OIDs over the wire protocol, so clients see them. Separate shards do not agree on those numbers. The authoritative shard decides the catalog clients see. Neki rewrites type identifiers in result sets so every client receives the authoritative OIDs. Every topology must name a default authoritative shard group. An authoritative group must contain one key range with no start or end bound. This keeps the metadata source on one shard. A new Neki cluster starts with one shard that is both the default and the authoritative group. After application data is sharded, that original shard remains the authoritative group. Leave it there as a standalone shard. Size that shard for catalog work, sequence reservations, and any unsharded tables that stay on it, not for large application tables. > [!WARNING] > > At the cluster level, `default_shard_group` must be omitted or match > `authoritative_shard_group`. To give application tables a different default, > set `default_shard_group` on the database or schema instead. After authority is established, it is currently not possible to move it to a different physical shard. Renaming the authoritative group is allowed only when the renamed group still resolves to the same shard. ![The meta authoritative shard group sends schema events to routers and receives fallback sequence queries. Application table queries follow tenant\_data routing to the table shards.](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/data-topology/authoritative-group.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=87cf7bd9078eb000cb3c07fea56805bc) ## View and update the topology You can view and replace the topology in the PlanetScale dashboard, with the CLI, or with the `__neki.set_data_topology` metafunction. An update is a complete replacement, not a partial patch. Retrieve and save the current document before preparing a replacement. **Dashboard** Open the database, go to **Clusters**, then **Data topology**. The page shows the current JSON document. Edit it there and select **Save changes**. **CLI** Retrieve the complete document or view its resolved relationships: ```bash theme={null} pscale branch data-topology get --org pscale branch data-topology ls --org ``` To replace the topology, pass the complete JSON document through standard input: ```bash theme={null} pscale branch data-topology update \ --org \ --format json < data-topology.json ``` ### Replace the topology with SQL The SQL signature is: ```sql theme={null} __neki.set_data_topology( data_topology_json text, overwrite boolean, options text DEFAULT NULL ) RETURNS record(success boolean, revision bigint) ``` Use `overwrite => false` only to create the first topology. It fails when a topology already exists. Use `overwrite => true` to replace an existing topology: ```sql theme={null} SELECT * FROM __neki.set_data_topology( $topology$ { "authoritative_shard_group": "meta", "shard_groups": [ { "uid": "meta", "key_ranges": [{ "shard_uid": "shard-meta" }] } ], "databases": { "analytics": { "default_shard_group": "meta" } } } $topology$, true, '{ "comment": "Route analytics tables to meta", "expected_revision": 42 }' ); ``` The optional JSON object accepts: | Option | Effect | | - | - | | `comment` | Records why the topology changed. | | `expected_revision` | Applies the replacement only if the stored revision still matches the revision you read. This prevents one editor from silently overwriting another editor's change. | | `force` | Applies a change despite validation problems introduced by that change and records the suppressed findings. Existing problems do not require this option. | Unknown and misspelled options are rejected. `overwrite` controls create versus replace; it is not a concurrency check. Use `expected_revision` for concurrent edits. Use `force` only with PlanetScale Support because it can persist routing problems. ### Wait for router visibility after a SQL update When you apply a topology with the `__neki.set_data_topology` metafunction, it returns `success` and the stored topology `revision`. The change is visible on the router that accepts it before that router reports the normal success notice, but other routers may still have the previous topology in memory. Use the returned revision to wait until every router can see the replacement: ```sql theme={null} SELECT __neki.wait_for_data_topology(); ``` If the original call warns that the topology was stored but did not become visible on that router in time, the stored update was not rolled back. Use the same wait function before running queries that depend on the new topology. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Development environments Source: https://planetscale.com/docs/neki/development-environments Set up empty branches for schema work and disposable test data. Use a Neki development branch for application development, integration tests, schema rehearsal, and other non-production work. Each branch is isolated from its parent and has its own connection strings and infrastructure. ## Initialize the environment [Development branches start empty](/docs/neki/branching#development-branches-start-empty). Treat environment creation as a repeatable application setup: **Step 1** Create a [development branch](/docs/neki/branching#create-a-development-branch) and wait for it to become ready. **Step 2** Create a role or password for the branch. **Step 3** Run the same schema migration command used by a fresh application deployment. **Step 4** Load deterministic seed or fixture data. **Step 5** Run a connection check and the application's integration tests. This workflow catches migrations that accidentally depend on objects or data already present in a long-lived database. ## Test with representative topology A new development branch begins with one managed shard. One shard is enough for most schema and compatibility tests, but it cannot reveal every distributed query problem. When testing sharding behavior: * Create the additional shards and data topology deliberately. * Seed rows that cover every key range. * Test single-shard and scatter query plans. * Exercise joins, transactions, sequences, reference tables, and GSIs used by the application. * Check [Query Insights](/docs/neki/monitoring/query-insights) and router metrics for unexpected fan-out. ## Use production-like data safely Restore a backup to a new branch when a test requires existing data. Apply your organization's access-control, retention, and sensitive-data policies to the restored branch. Delete the branch and any retained manual backups after the test is complete. Restore sizing determines the new branch type. Select a development cluster SKU for every configuration profile and a development router SKU when creating a disposable environment. If you omit sizing, the restore inherits the source profile and router settings; restoring a production backup without development overrides creates a production branch with production availability, branch limits, and billing. Development and production profile sizes cannot be mixed in one restore. Do not use a development branch as a production failover target. To serve production traffic from a tested environment, [promote the branch](/docs/neki/branching#promote-a-development-branch) first. Promotion is not a substitute for a production availability and recovery plan. ## Control cost Development branches have separate shard, router, storage, backup, and network usage. Delete environments that are no longer needed and keep test backup retention intentional. See [Neki pricing](/docs/neki/pricing#development-branches). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki error codes Source: https://planetscale.com/docs/neki/error-codes The numeric codes Neki reports with a not implemented error, the query shapes that trigger them, and what to do instead. > [!NOTE] > > Neki is currently in Platform Preview. Platform Preview features are “Beta Features” under the PlanetScale Terms of Service or your applicable agreement with PlanetScale. Accordingly, Neki is subject to the limitations and disclaimers applicable to Beta Features and is not covered by any service level agreement. When Neki's router cannot plan or execute a statement because an implementation is still missing, it returns SQLSTATE `NK013` and a message in this form: ``` not implemented: [108] This subquery shape is unavailable in ORDER BY. ``` The bracketed number is a stable catalog code. It identifies one specific implementation gap, and it keeps that meaning for the life of the product: a code is never reused for a different gap, even after the gap is closed. Quote it when you report a rejected query, and use it to match a rejection your application sees against the entry below. `NK013` means the gap is one Neki intends to close. A limit that is a design decision rather than a missing implementation is reported as `not supported` instead, and the broad ones are listed in [Platform preview limitations](/docs/neki/platform-preview-limitations). For how other Neki errors use the DETAIL and HINT fields, see [Error hints and details](/docs/neki/error-hints-and-details). ## How to read an entry Each entry gives the code, the message sentence the router renders after the bracketed code, an example that triggers it, and a note on which part of the query causes the rejection. The examples use the [example schema](/docs/neki/example-schema). A few entries name another small table whose own name describes the property that matters to that code. Where a rejection depends on how the data topology places the tables rather than on the SQL text, the note says so — the same statement can plan successfully with a different topology. Codes that only an internal invariant can raise are not listed, because no query produces them. A code absent from this page is one an application is unlikely to meet. A single statement can be rejected by more than one of these codes. Which one you see is the first gap the planner reaches, so fixing one can reveal another. ## Statements, transactions, and sessions Codes raised while the router decides what a statement is and how the session should behave. ### 1 — SELECT INTO statements are unavailable. ```sql theme={null} SELECT user_id INTO users_copy FROM users WHERE user_id = 1; ``` The `INTO` clause is what Neki rejects. At the top level Postgres treats `SELECT ... INTO` as `CREATE TABLE AS`; write that form instead. ### 20 — This statement type is unavailable. ```sql theme={null} ALTER SYSTEM SET work_mem = '4MB'; ``` The fallback for a statement no plan builder handles and for which the DDL surface has no more specific decision. A statement that the surface does recognize reports its own code instead. ### 21 — Reading an inheritance parent without ONLY is unavailable. ```sql theme={null} CREATE TABLE inheritance_parent (inheritance_id bigint PRIMARY KEY); CREATE TABLE inheritance_child () INHERITS (inheritance_parent); SELECT inheritance_id FROM inheritance_parent; ``` The rejection only occurs when `inheritance_parent` has child tables. Reading the parent without `ONLY` would have to expand to the children, so add `ONLY` to read just the parent. ### 22 — AND CHAIN is unavailable for this transaction command. ```sql theme={null} COMMIT AND CHAIN; ``` The `AND CHAIN` clause is the rejected part. Issue the `COMMIT` (or `ROLLBACK`, `END`, `ABORT`) and start the next transaction explicitly. ### 23 — This transaction command is unavailable. ```sql theme={null} PREPARE TRANSACTION 'gid'; ``` Raised for a transaction-control command the router has no handler for. `BEGIN`, `COMMIT`, `ROLLBACK`, `SAVEPOINT`, `RELEASE SAVEPOINT`, and `ROLLBACK TO SAVEPOINT` succeed. `PREPARE TRANSACTION`, `COMMIT PREPARED`, and `ROLLBACK PREPARED` return this code. ### 26 — Atomic transaction mode is unavailable. ```sql theme={null} SET __neki.tx_mode = 'atomic'; ``` Atomic distributed transactions are not available. `__neki.tx_mode` accepts `multi` (the default) and `single`. ### 28 — SET FROM CURRENT is unavailable. ```sql theme={null} SET work_mem FROM CURRENT; ``` The `FROM CURRENT` form is rejected; give the parameter an explicit value. ### 29 — SET TRANSACTION SNAPSHOT is unavailable. ```sql theme={null} SET TRANSACTION SNAPSHOT '00000003-0000001B-1'; ``` Importing another session's snapshot would have to be coordinated across every shard, so the statement is rejected outright. ### 30 — This statement cannot be analyzed with EXPLAIN neki\_plan. ```sql theme={null} EXPLAIN (NEKI_PLAN, ANALYZE) CREATE TABLE users_copy AS SELECT 1; ``` `EXPLAIN (NEKI_PLAN, ANALYZE)` runs the statement to show its router plan, and some statement types, such as `CREATE TABLE AS`, have none to show. The statement itself may still be supported when run directly. ### 31 — Data-modifying CTEs cannot be analyzed with EXPLAIN neki\_plan. ```sql theme={null} EXPLAIN (NEKI_PLAN, ANALYZE) WITH u AS (UPDATE users SET name = 'x' WHERE user_id = 1 RETURNING user_id) SELECT * FROM u; ``` The data-modifying CTE is the rejected part: `EXPLAIN (NEKI_PLAN, ANALYZE)` does not run a statement whose CTE writes. ### 32 — This EXPLAIN option is unavailable. ```sql theme={null} EXPLAIN (NEKI_PLAN, GENERIC_PLAN) SELECT user_id FROM users; ``` `GENERIC_PLAN` is one of the PostgreSQL `EXPLAIN` options the `NEKI_PLAN` renderer does not honor; the message names the option. Drop it, or run the `EXPLAIN` without `NEKI_PLAN`. ### 518 — This configuration parameter value is unavailable. ```sql theme={null} SET client_min_messages = 'debug1'; ``` The parameter accepts only some of its PostgreSQL values on Neki; the error's DETAIL names them. ### 563 — Changing DateStyle, TimeZone, IntervalStyle, extra\_float\_digits, or client\_encoding while a cursor over a router-computed query holds rows is unavailable. ```sql theme={null} BEGIN; DECLARE c CURSOR FOR SELECT g FROM generate_series(1, 5) g; FETCH 1 FROM c; SET DateStyle = 'SQL, DMY'; FETCH 1 FROM c; ``` The router encodes a cursor's rows under the output settings in force at its first read, so it cannot honor a `FETCH` after one of them changes. Change the setting before the first `FETCH`. ### 660 — A cursor over this query shape is unavailable. ```sql theme={null} DECLARE c CURSOR WITH HOLD FOR SELECT count(*) FROM users; ``` A `WITH HOLD` cursor over a query the router computes, here an aggregate across shards, is refused; the message names the shape. Declare the cursor without `WITH HOLD` inside a transaction. ### 817 — Planning against … is unavailable: earlier DDL in this transaction is not reflected in the router's schema view. ```sql theme={null} BEGIN; CREATE AGGREGATE my_moving_sum(int) (SFUNC = int4pl, STYPE = int, MSFUNC = int4pl, MINVFUNC = int4mi, MSTYPE = int); SELECT my_moving_sum(user_id) OVER (ORDER BY user_id ROWS 1 PRECEDING) FROM users; ``` The router plans against its own view of the schema, which DDL earlier in the same transaction has not reached yet. Commit the DDL first. ## Topology and evaluation context Codes that depend on how the data topology is configured, or on the context the router is evaluating in. ### 10 — Multi-column partitioning indexes are unavailable. ```sql theme={null} INSERT INTO devices (owner_id, device_no) VALUES (1, 2); ``` `devices` is sharded on `owner_id + device_no`, an expression over two columns, and `INSERT ... VALUES` cannot route on it. A `columns` list with more than one entry is rejected when the topology is applied; an expression over one column, such as `lower(email)`, routes. ### 11 — This shard index cannot route advisory locks. ```sql theme={null} SELECT __neki.set_data_topology($${ "authoritative_shard_group": "meta", "shard_indexes": { "modulo_user_id": {"type": "modulo", "columns": ["user_id"], "index_params": {"modulus": "2"}} }, "shard_groups": [ {"uid": "meta", "key_ranges": [{"shard_uid": "shard-meta"}]}, {"uid": "user_data", "default_shard_index": "modulo_user_id", "key_ranges": [{"shard_uid": "shard-a", "end": "80"}, {"shard_uid": "shard-b", "start": "80"}]} ], "databases": {"app": {"advisory_lock_shard_group": "user_data"}} }$$, true); ``` The topology names `user_data` as the advisory-lock shard group, and `user_data` routes on a `modulo` shard index. Neki hashes advisory-lock keys with xxhash, so a multi-shard advisory-lock group must use an `xxhash` default shard index, as the Example schema's does. ### 41 — File COPY is unavailable for sharded tables. Client `COPY TO STDOUT` works for sharded tables, reference tables, and materialized views, including text, CSV, and binary. The codes in this section are for file `COPY`. ```sql theme={null} COPY users FROM '/tmp/users.csv'; ``` File-based `COPY` against a sharded table. Use `COPY ... FROM STDIN` for a sharded table, or an unsharded table for file `COPY`. ### 43 — File COPY is unavailable for reference tables. ```sql theme={null} COPY ref_countries FROM '/tmp/countries.csv'; ``` File-based `COPY` into a reference table. Use `COPY ... FROM STDIN` instead. ### 44 — COPY FROM cannot maintain this table's secondary index. ```sql theme={null} COPY public.accounts (account_id, label) FROM STDIN; ``` Depends on the topology: the rejection occurs because the target table owns an active global secondary index, whose lookup rows `COPY FROM` cannot maintain. Load the table with `INSERT` instead. ### 45 — COPY does not support a multi-column primary shard index. ```sql theme={null} COPY devices (owner_id, device_no) FROM STDIN; ``` `COPY FROM` routes each row on a single-column shard key, and `devices` is sharded on an expression over two columns. ### 46 — COPY FROM cannot evaluate this reference-table default consistently. ```sql theme={null} ALTER TABLE ref_countries ADD COLUMN created_at timestamptz DEFAULT clock_timestamp(); COPY ref_countries (country_id, name) FROM STDIN; ``` The omitted column's default is volatile, and a reference table stores the same logical row in several shard groups, so each copy could receive a different value. Supply the column explicitly. ### 47 — COPY TO PROGRAM is unavailable. ```sql theme={null} COPY users TO PROGRAM 'cat'; ``` The `TO PROGRAM` destination is rejected; it would run a command on a Postgres host. ### 49 — COPY of a SELECT result is unavailable. ```sql theme={null} COPY (SELECT user_id FROM users) TO STDOUT; ``` The parenthesized `SELECT` source is what is rejected. Copy a table, or run the query and handle the rows in your application. ### 50 — COPY FROM with a WHERE clause is unavailable. ```sql theme={null} COPY public.notes FROM STDIN WHERE id > 1; ``` The `WHERE` clause on `COPY FROM` is rejected. Filter the data before feeding it to `COPY`. ### 51 — COPY FROM cannot evaluate this column default on the router. ```sql theme={null} COPY users (user_id, name, email) FROM STDIN; ``` The column list omits `created_at`, whose default `now()` the router would have to evaluate per row. Include the column and supply its values. ### 52 — This COPY option is unavailable for sharded COPY. ```sql theme={null} COPY users (user_id, name, email, created_at) FROM STDIN WITH (ON_ERROR ignore); ``` Depends on the topology: for a sharded target, `COPY` options such as `ON_ERROR`, `REJECT_LIMIT`, and `LOG_VERBOSITY` are rejected. The same option is accepted for an unsharded table. ### 53 — This input encoding is unavailable for sharded COPY. ```sql theme={null} COPY users (user_id, name, email, created_at) FROM STDIN WITH (ENCODING 'SJIS'); ``` The router frames the rows of a sharded `COPY` stream byte by byte, which is unsafe for an encoding such as `SJIS`, whose multi-byte characters can contain a backslash byte. Send UTF-8. ### 54 — COPY is unavailable in a multi-statement query. ```sql theme={null} SELECT 1; COPY notes FROM STDIN; ``` `COPY` must be the only statement in the query. The second statement in the batch is what makes this fail. ### 55 — COPY is unavailable in the extended query protocol. ```sql theme={null} COPY notes FROM STDIN; ``` Reachable only over the extended query protocol — that is, when the client prepares and binds the statement. The same `COPY` works over the simple query protocol. ### 350 — Notify channel routing requires an xxhash shard index. ```sql theme={null} SELECT __neki.set_data_topology($${ "authoritative_shard_group": "meta", "shard_indexes": { "modulo_user_id": {"type": "modulo", "columns": ["user_id"], "index_params": {"modulus": "2"}} }, "shard_groups": [ {"uid": "meta", "key_ranges": [{"shard_uid": "shard-meta"}]}, {"uid": "user_data", "default_shard_index": "modulo_user_id", "key_ranges": [{"shard_uid": "shard-a", "end": "80"}, {"shard_uid": "shard-b", "start": "80"}]} ], "databases": {"app": {"notify_shard_group": "user_data"}} }$$, true); ``` As code 11, for `notify_shard_group`: Neki hashes notification channel names with xxhash, so a multi-shard notify group must use an `xxhash` default shard index. ### 531 — COPY FROM PROGRAM is unavailable. ```sql theme={null} COPY users FROM PROGRAM 'cat users.csv'; ``` Use `COPY ... FROM STDIN` and send the data from the client. ## Advisory locks Codes raised when the router cannot place an advisory lock on a single shard, or take it at all. ### 60 — A session cannot hold session advisory locks on multiple shards. ```sql theme={null} SELECT pg_advisory_lock(1); SELECT pg_advisory_lock(2); ``` Depends on the data topology: `user_data` is the advisory-lock shard group, and keys 1 and 2 hash to different shards. A session holds its session-level locks on one shard, so release the first lock before taking one that lands elsewhere. ### 61 — Mixing session and transaction advisory locks is unavailable. ```sql theme={null} SELECT pg_advisory_lock(1), pg_advisory_xact_lock(1) FROM users WHERE user_id = 1; ``` The statement mixes a session-scoped and a transaction-scoped advisory lock. Use one scope. ### 139 — Advisory locks are unavailable in this plan position. ```sql theme={null} CREATE TABLE users_adv_lock AS SELECT pg_advisory_xact_lock(1); ``` The advisory-lock call lands in a plan position — the body of `CREATE TABLE AS` — where the router cannot take the lock. Take the lock in a separate statement. ## PL/pgSQL Codes raised while translating or executing a PL/pgSQL routine on the router. ### 82 — Nested and array-slice assignment targets are unavailable in PL/pgSQL. ```sql theme={null} DO $$DECLARE a int[]; BEGIN a[1:2] := ARRAY[1,2]; END$$; ``` A nested or array-slice assignment target in PL/pgSQL. Assign the whole variable, or a single subscript. ### 85 — Composite INTO targets are unavailable in PL/pgSQL. ```sql theme={null} DO $$DECLARE c refcursor := 'c'; u users; BEGIN OPEN c FOR SELECT * FROM users; FETCH c INTO u; END$$; ``` The `FETCH` target `u` is a named composite (`users`) variable. Fetch into a `record` variable, or into a list of scalar variables. ### 86 — Record fields with varying runtime types are unavailable in PL/pgSQL. ```sql theme={null} DO $$DECLARE r record; x text; BEGIN IF random() > 2 THEN SELECT 1 AS a INTO r; ELSE SELECT 'x'::text AS a INTO r; END IF; x := r.a; END$$; ``` `r.a` is an `integer` after one branch and `text` after the other, so the read has no single type. Give the field the same type on every path, for example with a cast. ### 87 — Function configuration parameters are unavailable for router-side execution. ```sql theme={null} CREATE FUNCTION pinned_to_primary() RETURNS int LANGUAGE plpgsql SET __neki.target = 'primary' AS $$BEGIN RETURN 1; END$$; SELECT pinned_to_primary(); ``` The function's `SET` clause names a parameter the router cannot scope around a call, here the Neki session option `__neki.target`. Set it on the session instead. ### 88 — Polymorphic routines are unavailable for router-side execution. ```sql theme={null} CREATE FUNCTION identity_of(anyelement) RETURNS anyelement LANGUAGE plpgsql AS $$BEGIN RETURN $1; END$$; SELECT identity_of(1); ``` The routine's polymorphic signature (`anyelement`, `anyarray`, and similar) is what is rejected for router-side execution. Declare concrete argument types. ### 90 — This PL/pgSQL statement is unavailable. ```sql theme={null} DO $$DECLARE x text; BEGIN GET DIAGNOSTICS x = PG_CONTEXT; END$$; ``` A PL/pgSQL statement form the executor has no implementation for, here `GET DIAGNOSTICS ... PG_CONTEXT`. ### 91 — EXIT to a block label is unavailable in PL/pgSQL. ```sql theme={null} DO $$<> BEGIN EXIT outer; END$$; ``` `EXIT` naming a *block* label. `EXIT` from a loop, optionally naming the loop's label. ### 575 — RETURN QUERY EXECUTE of a FETCH or SQL EXECUTE into a function result with a domain column is unavailable unless the cursor was opened by a PL/pgSQL OPEN over a static query. ```sql theme={null} CREATE DOMAIN positive_int AS int CHECK (VALUE > 0); CREATE FUNCTION fetch_positive() RETURNS SETOF positive_int LANGUAGE plpgsql AS $$BEGIN RETURN QUERY EXECUTE 'FETCH ALL FROM c'; END$$; BEGIN; DECLARE c CURSOR FOR SELECT 1::positive_int; SELECT * FROM fetch_positive(); ``` The cursor was opened by SQL `DECLARE`, so the router cannot check its rows against the function's domain result. Open the cursor with a PL/pgSQL `OPEN` over a static query. ### 757 — DISTINCT, and an ORDER BY, GROUP BY or DISTINCT ON key naming the result column, are unavailable in a PL/pgSQL assignment that coerces its value or assigns an array element. ```sql theme={null} DO $$DECLARE n int; BEGIN n := DISTINCT 1.5; END$$; ``` The assignment coerces its value (here `numeric` to `int`) and its query uses `DISTINCT`. Cast the value in the query, or assign without `DISTINCT`. ### 807 — RETURN QUERY in a PL/pgSQL function that returns SETOF record without OUT parameters is unavailable. ```sql theme={null} CREATE FUNCTION all_users() RETURNS SETOF record LANGUAGE plpgsql AS $$BEGIN RETURN QUERY SELECT user_id, name FROM users; END$$; SELECT * FROM all_users() AS t(user_id bigint, name varchar); ``` Declare `OUT` parameters, or return a named composite type. ### 837 — A record or composite INTO target of a non-constant EXECUTE command string is unavailable in PL/pgSQL. ```sql theme={null} DO $$DECLARE r record; t text := 'users'; BEGIN EXECUTE 'SELECT * FROM ' || t || ' LIMIT 1' INTO r; END$$; ``` The `EXECUTE` command string is built at run time, so the router cannot know the shape of the record it fills. Select into scalar variables, or use a constant command string. ### 842 — A record loop target of a non-constant FOR IN EXECUTE command string is unavailable in PL/pgSQL. ```sql theme={null} DO $$DECLARE r record; t text := 'users'; BEGIN FOR r IN EXECUTE 'SELECT * FROM ' || t LOOP END LOOP; END$$; ``` As code 837, for the loop variable of `FOR ... IN EXECUTE`. ### 844 — Reading a PL/pgSQL cursor over a data-modifying query or an EXPLAIN after search\_path, the role, or the shard or sidecar target changed since its OPEN is unavailable. ```sql theme={null} DO $$DECLARE c refcursor; x text; BEGIN OPEN c FOR EXPLAIN SELECT * FROM users; SET search_path = pg_catalog; FETCH c INTO x; END$$; ``` The cursor is over an `EXPLAIN`, and `search_path` changed between its `OPEN` and the `FETCH`. Read the cursor before changing the setting. ### 857 — FETCH into a record variable requires the cursor to be opened by a PL/pgSQL OPEN or cursor FOR loop over a static query whose row shape, collations and nested record shapes included, equals a query the fetching function opens a cursor over; a cursor opened with a dynamic command string, by SQL DECLARE, or over a query of another shape has none. ```sql theme={null} DO $$DECLARE c refcursor := 'c'; r record; BEGIN OPEN c FOR EXECUTE 'SELECT 1 AS a'; FETCH c INTO r; END$$; ``` The cursor was opened with a dynamic command string, so the router cannot know the shape of the record it fills. Open it over a static query, or fetch into scalar variables. ## Query shapes and routing The largest group: codes raised while the planner turns a query into routed work. Many depend on the data topology, and the same statement can plan against an unsharded table and be rejected against a sharded one. ### 67 — Array-slice and multidimensional array assignment are unavailable. ```sql theme={null} DO $$DECLARE a int[] := '{{1,2},{3,4}}'; BEGIN a[1] := 5; END$$; ``` Assigning one element of a multidimensional array is the rejected part, as is assigning an array slice. Replace the whole array value instead. ### 69 — Subqueries in VALUES are unavailable. ```sql theme={null} VALUES ((SELECT 1)); ``` A subquery inside a `VALUES` cell. Compute the value in a `SELECT` and feed it in, or use `INSERT ... SELECT`. ### 100 — Subqueries are unavailable in this DML clause. ```sql theme={null} UPDATE users SET name = (SELECT 'x') FROM orders WHERE users.user_id = orders.user_id; ``` A subquery appears in a DML clause that cannot host one — here the `SET` value of an `UPDATE ... FROM`. Move the subquery into the `FROM` source and reference its column. ### 101 — Per-row router-only expression evaluation is unavailable here. ```sql theme={null} UPDATE users SET name = 'x' WHERE current_setting(name, true) = 'foo'; ``` The expression must be evaluated on the router, but its value depends on each row (`name` is a column), so the router cannot compute it before the statement is routed. ### 102 — This RETURNING expression extraction is unavailable. ```sql theme={null} UPDATE users SET name = 'x' WHERE user_id = 1 RETURNING WITH (OLD AS o) now(); ``` `RETURNING` mixes an `OLD`/`NEW` row alias with an expression the router must evaluate itself. Return the columns and compute the router-side expression in your application. ### 103 — This DML source join cannot be planned safely. ```sql theme={null} DELETE FROM users USING orders WHERE users.created_at < orders.created_at + (extract(epoch from now()) || ' seconds')::interval; ``` The predicate bridging the DML target to its `USING`/`FROM` source is not something the planner can evaluate on one side, so the join cannot be executed safely. ### 104 — This mutation cannot maintain active secondary indexes. ```sql theme={null} DELETE FROM accounts WHERE account_id = 1 RETURNING account_id; ``` Depends on the topology: `accounts` owns global secondary indexes, and this mutation shape cannot keep their lookup rows in step with the rows they point at. ### 106 — This window-function shape is unavailable across shards. ```sql theme={null} SELECT p.user_id, row_number() OVER (PARTITION BY p.user_id ORDER BY p.product_id) FROM users u JOIN products p ON u.user_id % 2 = p.user_id % 2; ``` The window function would have to be computed after a cross-shard join, so no shard sees the whole partition. Rejected only when the query actually spans shards. ### 107 — Expression extraction is unavailable in this clause. ```sql theme={null} SELECT count(*), date_trunc('day', now()) FROM users GROUP BY date_trunc('day', now()); ``` A router-only expression (`now()`) appears in a clause the planner cannot lift it out of — here `GROUP BY`. Compute the value in your application and pass it as a parameter. ### 108 — This subquery shape is unavailable in ORDER BY. ```sql theme={null} SELECT name FROM users WHERE nextval('public.my_sequence') > 0 ORDER BY (SELECT max(total_amount) FROM orders); ``` The `ORDER BY` subquery cannot be evaluated once up front because the volatile `WHERE` expression forces per-row router evaluation. Remove the volatile predicate, or sort in your application. ### 109 — This quantified-subquery shape is unavailable. ```sql theme={null} SELECT * FROM users WHERE (user_id, name) > ALL(SELECT user_id, name FROM orders); ``` The quantified comparison uses a row constructor, so the `ALL` cannot be rewritten into a routable form. Compare a single column. ### 110 — This correlated-subquery shape cannot be decorrelated safely. ```sql theme={null} SELECT u.user_id FROM users u WHERE u.name = (SELECT min(o.status) FROM orders o WHERE o.total_amount = u.user_id GROUP BY o.status); ``` The correlated subquery has its own `GROUP BY`, so decorrelating it would change which groups exist. Rewrite it as an explicit join. ### 111 — This lateral or range-function shape is unavailable. ```sql theme={null} SELECT k FROM jsonb_each((SELECT '{"a": 1}'::jsonb)) AS s(k, v); ``` A subquery as the argument of a function in `FROM` is the rejected part. Compute the argument in a CTE or a join instead. ### 112 — This full outer join has no usable equality key. ```sql theme={null} SELECT 1 FROM users u FULL JOIN products p ON u.user_id > p.user_id; ``` A full outer join needs an equality key it can hash on both sides; `>` does not give one. Join on an equality. ### 116 — This INSERT subquery shape is unavailable. ```sql theme={null} INSERT INTO users (user_id, name) VALUES (1, 'a' || (SELECT 'b')); ``` The subquery is nested inside a larger expression in the `VALUES` cell rather than being the whole cell. Use `INSERT ... SELECT`. ### 117 — This INSERT SELECT shape is unavailable. ```sql theme={null} INSERT INTO users (user_id, email, name) SELECT p.product_id, p.name, p.name FROM products p ON CONFLICT (user_id) DO UPDATE SET name = excluded.name; ``` The combination of an `INSERT ... SELECT` source with `ON CONFLICT DO UPDATE` against a sharded target returns this code, except when the source is one table in the same shard group and the inserted shard key is copied from that table's shard key. Insert explicit `VALUES`, or drop the `DO UPDATE`. `INSERT ... SELECT` into a table sharded by an expression over more than one column also returns this code. `INSERT ... VALUES` into that table returns code 10. ### 118 — These INSERT values cannot be materialized on the router. ```sql theme={null} INSERT INTO users (user_id) VALUES (generate_series(1,3)); ``` A set-returning function in a `VALUES` cell means the router cannot know the rows — and therefore the routing values — before executing. Expand the rows in your application, or use `INSERT ... SELECT`. ### 119 — This ON CONFLICT shape is unavailable. ```sql theme={null} INSERT INTO orders (order_id, user_id, order_date, status) VALUES (91021, 91020, DATE '2026-09-09', 'new'), (91022, 91020, DATE '2026-09-09', 'held') ON CONFLICT (order_id) DO NOTHING RETURNING order_id, user_id; ``` Depends on the topology: the `ON CONFLICT` arbiter index (`order_id`) does not cover the table's shard key (`user_id`), so a conflict cannot be resolved on one shard. Arbitrate on an index that includes the shard key. ### 120 — This column default cannot be materialized on the router. ```sql theme={null} ALTER TABLE ref_countries ADD COLUMN seq bigint GENERATED BY DEFAULT AS IDENTITY; UPDATE ref_countries SET seq = DEFAULT; ``` A reference table's copies must all store the same value, so the router evaluates the default itself, and it cannot draw an identity value. Supply the value explicitly. ### 121 — This MERGE shape is unavailable. ```sql theme={null} MERGE INTO users t USING (SELECT 1 AS user_id) s ON t.user_id = s.user_id WHEN MATCHED THEN UPDATE SET name = 'x'; ``` The `MERGE` source is a derived table, so the planner cannot route the merge to the target's shards. Use a `MERGE` whose source is a table in the same shard group. ### 122 — This view cannot be routed safely. ```sql theme={null} CREATE VIEW view_users AS SELECT user_id, name FROM users; UPDATE view_users SET name = 'x' WHERE user_id = 1; ``` Depends on the topology: Neki does not route a write through a view whose underlying table is sharded, whatever the predicate. Write to the base table. ### 123 — This materialized view cannot be routed safely. ```sql theme={null} CREATE MATERIALIZED VIEW mv_users AS SELECT user_id, name FROM users; SELECT * FROM mv_users WHERE user_id = 1; ``` Depends on the topology. A materialized view over sharded data stores one shard's result in a single heap, which a read cannot reassemble. ### 124 — This set-operation shape is unavailable. ```sql theme={null} SELECT account_id FROM accounts WHERE email = 'a@example.com' INTERSECT SELECT user_id FROM products WHERE user_id = 500; ``` `INTERSECT` and `EXCEPT` are not planned for application tables; only `UNION` is. This limit does not apply to statements Neki forwards unchanged. ### 125 — This grouping-set form is unavailable. ```sql theme={null} SELECT name, count(*) FROM users GROUP BY GROUPING SETS ((name), ()); ``` The `GROUPING SETS` (or `ROLLUP`/`CUBE`) form passes validation but has no cross-shard execution: each grouping set would need its own aggregation pass. Issue the grouping levels as separate queries. ### 126 — TABLESAMPLE is unavailable. ```sql theme={null} SELECT * FROM users TABLESAMPLE bernoulli(50); ``` The `TABLESAMPLE` clause is rejected; a per-shard sample would not be a sample of the whole table. Sample with a `WHERE` predicate such as `random() < 0.5`. ### 128 — WHERE CURRENT OF is unavailable for DML. ```sql theme={null} DELETE FROM users WHERE CURRENT OF c; ``` `WHERE CURRENT OF` identifies a row by cursor position, which the router does not track across shards. Delete or update by key. ### 129 — DISTINCT cannot safely evaluate this volatile record expression. ```sql theme={null} SELECT DISTINCT ROW(random()) FROM users; ``` `DISTINCT` would have to compare a record value built from a volatile function, whose result changes per evaluation, so duplicate elimination is not well defined. ### 130 — This combination of select-list subqueries is unavailable. ```sql theme={null} SELECT nextval('public.my_sequence') + (SELECT max(order_id) FROM orders WHERE orders.user_id = users.user_id) FROM users; ``` One select-list expression combines a router-only value with a correlated subquery; the two need incompatible evaluation strategies in the same slot. Split them into separate expressions or queries. ### 131 — TRUNCATE RESTART IDENTITY with CASCADE is unavailable. ```sql theme={null} TRUNCATE users RESTART IDENTITY CASCADE; ``` `RESTART IDENTITY` together with `CASCADE` is rejected because the cascade set — and so the sequences to restart — is only known per shard. Truncate the tables explicitly. ### 132 — TRUNCATE RESTART IDENTITY cannot include descendant tables. ```sql theme={null} CREATE TABLE logs (log_id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY); CREATE TABLE logs_archive () INHERITS (logs); TRUNCATE logs RESTART IDENTITY; ``` Depends on the schema: rejected when the table has descendants, since `TRUNCATE` would reach them and their sequences. Use `TRUNCATE ONLY`, or truncate each table separately. ### 133 — This whole-row reference is ambiguous to the SQL builder. ```sql theme={null} SELECT json_agg(t.*) FROM (SELECT order_id AS t FROM orders) t; ``` The whole-row reference `t.*` is shadowed by a column also named `t`, so the SQL builder cannot tell which the query means. Rename the column or the alias. ### 135 — Aggregates over columns of an enclosing query are unavailable. ```sql theme={null} SELECT (SELECT max(u.user_id)) FROM users u; ``` The aggregate's argument belongs to the enclosing query, so the aggregation happens at the outer level from inside the subquery. Compute the aggregate in the outer select list. ### 136 — Correlated column references are unavailable in this clause. ```sql theme={null} SELECT user_id FROM users WHERE EXISTS (SELECT 1 FROM orders LIMIT users.user_id); ``` The correlated reference sits in `LIMIT` (or `OFFSET`), a clause that is evaluated once rather than per outer row. Pass the limit as a parameter. ### 138 — FETCH FIRST WITH TIES is unavailable across shards. ```sql theme={null} SELECT name FROM users ORDER BY name FETCH FIRST 2 ROWS WITH TIES; ``` `WITH TIES` needs to know every row tied at the cut-off, which no single shard can determine. Use a plain `LIMIT`, or rank in a subquery and filter. ### 140 — Set-returning functions are unavailable in router-side ORDER BY. ```sql theme={null} SELECT u.user_id FROM users u LEFT JOIN orders o ON u.user_id = o.user_id ORDER BY generate_series(u.user_id::int, (u.user_id + 1)::int) LIMIT 2; ``` A set-returning function in `ORDER BY` must be expanded on the router, which cannot happen while merging sorted streams from several shards. Move the expansion into the `FROM` clause. ### 141 — INSERT OVERRIDING USER VALUE is unavailable. ```sql theme={null} CREATE TABLE audit_log (log_id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, user_id bigint, action text); INSERT INTO audit_log (user_id, action) OVERRIDING USER VALUE VALUES (100, 'login'); ``` The `OVERRIDING USER VALUE` clause is the rejected part. Omit it, or use `OVERRIDING SYSTEM VALUE` where you need to write an `ALWAYS` identity column. ### 142 — Parameterized subscripts are unavailable in INSERT targets. ```sql theme={null} INSERT INTO user_tags (user_id, tag_ids[$1]) VALUES (1, 2); ``` The subscript in the `INSERT` target list is a parameter, so the router does not know which element is being written at plan time. Use a literal subscript, or write the whole array. ### 143 — OLD and NEW references are unavailable in sharded INSERT ON CONFLICT RETURNING. ```sql theme={null} INSERT INTO users (user_id, name, email) VALUES (1, 'a', 'a@example.com'), (900, 'b', 'b@example.com') ON CONFLICT (user_id) DO NOTHING RETURNING old.user_id; ``` Depends on the topology: for a sharded target, `OLD`/`NEW` aliases in the `RETURNING` of an `ON CONFLICT` statement are not available. Return plain columns. ### 144 — Router-only expressions are unavailable in sharded INSERT ON CONFLICT RETURNING. ```sql theme={null} INSERT INTO users (user_id, name, email) VALUES (1, 'a', 'a@example.com'), (900, 'b', 'b@example.com') ON CONFLICT (user_id) DO NOTHING RETURNING now(); ``` As code 143, for an expression the router must evaluate itself rather than an `OLD`/`NEW` reference. Return columns and compute the expression in your application. ### 145 — Updating an index column is unavailable. ```sql theme={null} UPDATE users SET user_id = nextval('public.my_sequence') WHERE user_id = 1; ``` The statement assigns to `user_id`, which is the table's shard index column — changing it could move the row to another shard. Delete the row and insert the new one. ### 146 — This volatile router-only WHERE expression is unavailable. ```sql theme={null} UPDATE users SET name = 'x' WHERE COALESCE(nextval('public.my_sequence') > 0, false); ``` The volatile router-only call is buried inside another expression in `WHERE`, so the router cannot evaluate it once and route on the result. Compute the value first and pass it in. ### 147 — This volatile router-only UPDATE assignment is unavailable. ```sql theme={null} UPDATE users SET name = nextval('public.my_sequence')::text WHERE random() < 0.5; ``` Both the assigned value and the predicate are volatile router-only expressions, so the rows to change and the values to write cannot be determined together. Select the rows first, then update them by key. ### 149 — Window functions inside aggregates are unavailable. ```sql theme={null} SELECT count(*), row_number() OVER () FROM products; ``` A plain aggregate and a window function appear in the same select list, which would need two incompatible aggregation passes across shards. Compute the window function over a subquery that does the aggregation. ### 150 — This ORDER BY expression is unavailable for VALUES. ```sql theme={null} SELECT x FROM (VALUES (1),(2) ORDER BY column1 + 1) v(x); ``` The `ORDER BY` attached to a `VALUES` list is an expression rather than a plain column reference. Sort in the enclosing query instead. ### 151 — INTERSECT and EXCEPT subqueries are unavailable. ```sql theme={null} SELECT cardinality(ARRAY(SELECT unnest(ARRAY['a','b']) INTERSECT SELECT unnest(ARRAY['b','c']))); ``` The subquery's top level is an `INTERSECT` (or `EXCEPT`). Rewrite it as a join or an `IN`/`NOT IN` predicate, or use array functions. ### 152 — This subquery statement type is unavailable. ```sql theme={null} SELECT (VALUES (1)); ``` A `VALUES` list used as a scalar subquery is not a `SELECT`. Write `(SELECT 1)` instead. ### 153 — ARRAY subqueries are unavailable. ```sql theme={null} SELECT user_id FROM users ORDER BY ARRAY(SELECT order_id FROM orders WHERE user_id = 1); ``` An `ARRAY(...)` subquery the router would have to evaluate, here in `ORDER BY`, is the rejected part. Use `array_agg` over a normal subquery: `SELECT array_agg(order_id) FROM ...`. ### 576 — A whole-row reference to a join is unavailable. ```sql theme={null} SELECT j FROM (users u JOIN orders o ON u.user_id = o.user_id) AS j; ``` `j` names the whole row of an aliased join. Select the columns you need instead. ### 597 — Nested grouped expressions in SELECT targets are unavailable. ```sql theme={null} SELECT count(*) + (random() + length(name)) FROM users GROUP BY random() + length(name); ``` The grouped expression appears nested inside a larger target expression. Select the grouped expression on its own, or group by a column. ### 733 — Tables "…" from different schemas cannot both appear unaliased in one query. ```sql theme={null} CREATE SCHEMA archive; CREATE TABLE archive.notes (id bigint PRIMARY KEY, body text); SELECT 1 FROM public.notes, archive.notes; ``` Both tables are named `notes`. Give one of them an alias. ### 739 — JSON\_ARRAY(query) evaluated by the router is unavailable. ```sql theme={null} SELECT JSON_ARRAY(SELECT 1); ``` `JSON_ARRAY` over a subquery, evaluated by the router. Use `json_agg` over the subquery instead. ### 822 — A LATERAL XMLTABLE or JSON\_TABLE is unavailable. ```sql theme={null} SELECT * FROM users u, LATERAL JSON_TABLE(to_jsonb(u.name), '$' COLUMNS (a text PATH '$')) jt; ``` A `LATERAL` `JSON_TABLE` or `XMLTABLE` in `FROM` is the rejected part. ### 824 — Table "…" shares its name with another table in the query, which a reference past a column alias list needs to tell apart. ```sql theme={null} SELECT (SELECT a FROM orders u LIMIT 1) FROM users u(a) WHERE u.a = 1; ``` The outer `users u(a)` renames its first column to `a`, and the subquery's `orders u` reuses the name `u`. Give the two tables distinct aliases. ### 945 — Table "…" inside an aliased join cannot share its name with another table in the query. ```sql theme={null} SELECT 1 FROM (public.users JOIN orders USING (user_id)) j, pg_catalog.pg_class users; ``` `public.users` sits inside the aliased join `j`, and the alias `users` names another table. Give the tables distinct names. ## Evaluation engine Codes raised when the router — rather than a shard — has to compute an expression and has no implementation for it. A query that a shard can evaluate on its own is unaffected. ### 64 — Set-returning function … has no opcode factory. ```sql theme={null} SELECT * FROM pg_lock_status(); ``` The router has no implementation of this set-returning built-in function, so it cannot run it. ### 200 — Built-in function … is unavailable in the evaluation engine. ```sql theme={null} SELECT pg_relation_size('users'); ``` The built-in function has a catalog entry but no router implementation, so it cannot be evaluated when the router — rather than a shard — has to compute it. ### 202 — Built-in type input function … is unavailable in the evaluation engine. ```sql theme={null} SELECT 'x'::text::gtsvector; ``` The literal has to be parsed by `gtsvectorin`, which the evaluation engine does not implement, so the router cannot build the value. `brin_bloom_summary` and `brin_minmax_multi_summary` behave the same way. ## DDL Codes raised by the DDL surface, mostly for objects that are local to one Postgres host and so would not reach a shard that joins the cluster later. ### 300 — Creating an operator class is unavailable. ```sql theme={null} CREATE OPERATOR CLASS cls DEFAULT FOR TYPE int USING btree AS OPERATOR 1 <; ``` An operator class created through the router is not carried to a shard that joins the cluster later. Use one Postgres provides, or install one through an extension. ### 301 — Creating an operator family is unavailable. ```sql theme={null} CREATE OPERATOR FAMILY fam USING btree; ``` As code 300: an operator family created here would not reach a later-joining shard. ### 302 — Creating a text search configuration is unavailable. ```sql theme={null} CREATE TEXT SEARCH CONFIGURATION public.neki_config (COPY = pg_catalog.simple); ``` Router-side text-search evaluation supports only Postgres's built-in configurations, so a user-created one could not be used by a routed text-search expression. ### 303 — Creating a text search dictionary is unavailable. ```sql theme={null} CREATE TEXT SEARCH DICTIONARY public.neki_simple (TEMPLATE = pg_catalog.simple); ``` As code 302, for a text-search dictionary. ### 304 — Event triggers are unavailable. ```sql theme={null} CREATE EVENT TRIGGER e ON ddl_command_start EXECUTE FUNCTION f(); ``` Event triggers fire on DDL, which the router executes on every shard, so the trigger's behavior is not well defined. `ALTER EVENT TRIGGER` is rejected the same way. ### 305 — Rewrite rules are unavailable. ```sql theme={null} CREATE RULE r AS ON DELETE TO public.users DO INSTEAD NOTHING; ``` A rewrite rule would change a statement's meaning below the router, after planning and routing. Use a trigger, or do the rewrite in your application. ### 308 — Creating a tablespace is unavailable. ```sql theme={null} CREATE TABLESPACE ts LOCATION '/tmp/ts'; ``` A tablespace is storage local to one Postgres host, so it is not carried to a later-joining shard. Use the default tablespace. ### 309 — Creating an access method is unavailable. ```sql theme={null} CREATE ACCESS METHOD am TYPE INDEX HANDLER f; ``` An access method needs a C handler local to the host and is not carried to a later-joining shard. Use a built-in method, or install one through an extension. ### 310 — Creating a collation is unavailable. ```sql theme={null} CREATE COLLATION c (LOCALE = 'C'); ``` A collation created through the router is not carried to a later-joining shard, and rows sorted with it could then compare differently per shard. Use a collation Postgres already provides. ### 311 — Creating a conversion is unavailable. ```sql theme={null} CREATE CONVERSION cv FOR 'UTF8' TO 'LATIN1' FROM f; ``` As code 310, for an encoding conversion. ### 312 — Foreign data wrappers are unavailable. ```sql theme={null} CREATE FOREIGN DATA WRAPPER w; ``` Foreign data wrappers reach outside the cluster from a single host. Import the data into a Neki table, or reach the remote system from your application. `ALTER FOREIGN DATA WRAPPER` is rejected the same way. ### 313 — Creating a foreign server is unavailable. ```sql theme={null} CREATE SERVER srv FOREIGN DATA WRAPPER w; ``` A foreign server exists only to serve foreign data wrappers and foreign tables, neither of which Neki supports. ### 314 — Foreign tables are unavailable. ```sql theme={null} CREATE FOREIGN TABLE ft (a int) SERVER srv; ``` A foreign table's data lives outside the cluster, so the router has no topology for it and cannot route to it. ### 315 — User mappings are unavailable. ```sql theme={null} CREATE USER MAPPING FOR testuser SERVER srv; ``` User mappings apply only to foreign data wrappers, which Neki does not support. `ALTER` and `DROP USER MAPPING` are rejected the same way. ### 316 — Creating a text search parser is unavailable. ```sql theme={null} CREATE TEXT SEARCH PARSER p (START = f, GETTOKEN = g, END = h, LEXTYPES = i); ``` A text-search parser needs host-local C functions and is not carried to a later-joining shard. Use the built-in parser, or one an extension installs. ### 317 — Creating a text search template is unavailable. ```sql theme={null} CREATE TEXT SEARCH TEMPLATE tm (LEXIZE = f); ``` As code 316, for a text-search template. ### 318 — Creating a transform is unavailable. ```sql theme={null} CREATE TRANSFORM FOR int LANGUAGE sql (FROM SQL WITH FUNCTION f(int), TO SQL WITH FUNCTION g(int)); ``` A transform created through the router is not carried to a later-joining shard. Install one through an extension. ### 319 — Changing composite-type attributes is unavailable. ```sql theme={null} ALTER TYPE pair ADD ATTRIBUTE third int; ``` Changing a composite type's attributes rewrites every table using that type, which the router does not coordinate across shards. Create a new type and migrate to it. ### 320 — Shared library loading is unavailable. ```sql theme={null} LOAD 'libexample'; ``` `LOAD` loads a shared library into one Postgres backend, and it is not carried to a later-joining shard. Install the library on every shard host before starting Neki. ### 321 — IMPORT FOREIGN SCHEMA is unavailable. ```sql theme={null} IMPORT FOREIGN SCHEMA remote FROM SERVER srv INTO public; ``` `IMPORT FOREIGN SCHEMA` creates foreign tables, which Neki does not support (code 314). ### 323 — Per-column statistics overrides are unavailable. ```sql theme={null} ALTER TABLE t ALTER COLUMN a SET STATISTICS 100; ``` A per-column statistics target is planner tuning local to one Postgres instance, so setting it through the router would not apply uniformly. ### 324 — Per-column foreign-table options are unavailable. ```sql theme={null} ALTER FOREIGN TABLE ft ALTER COLUMN a OPTIONS (ADD b 'c'); ``` Per-column foreign-table options presuppose foreign tables, which Neki does not support (code 314). ### 325 — This function language is unavailable. ```sql theme={null} CREATE FUNCTION f() RETURNS int LANGUAGE c AS 'lib', 'sym'; ``` The function's language is the rejected part: only SQL and PL/pgSQL are supported. A language other than those needs a handler local to the host. ### 326 — This procedure language is unavailable. ```sql theme={null} CREATE PROCEDURE p() LANGUAGE plperl AS 'return 1'; ``` As code 325, for `CREATE PROCEDURE`. ### 327 — CREATE LANGUAGE with a handler is unavailable. ```sql theme={null} CREATE LANGUAGE l HANDLER h; ``` The `HANDLER` clause names a host-local C function. Install the language through a supported extension instead. ### 328 — Renaming a domain constraint is unavailable. ```sql theme={null} ALTER DOMAIN positive_int RENAME CONSTRAINT positive_int_check TO positive_check; ``` Renaming a domain constraint is rejected, while `ALTER DOMAIN ... RENAME TO`, `SET SCHEMA`, and `OWNER TO` are supported. ### 329 — ALTER DOMAIN is unavailable. ```sql theme={null} ALTER DOMAIN positive_int SET DEFAULT 1; ``` Changing a domain's default or constraints affects every column using the domain, across shards, so the statement is rejected. ### 330 — The IS\_TEMPLATE database option is unavailable. ```sql theme={null} CREATE DATABASE db IS_TEMPLATE true; ``` The `IS_TEMPLATE` option is what is rejected; `ALTER DATABASE ... IS_TEMPLATE` reports the same code. ### 331 — Database tablespace selection is unavailable. ```sql theme={null} CREATE DATABASE db TABLESPACE ts; ``` The `TABLESPACE` option on a database selects host-local storage (code 308). `ALTER DATABASE ... SET TABLESPACE` reports the same code. ### 332 — This database locale is unavailable to the router. ```sql theme={null} CREATE DATABASE db LOCALE 'xx_not_a_locale'; ``` The named locale cannot be resolved by the routers, so the database would exist in a state the router could not read back. Use a locale every router provides, or the `icu` or `builtin` locale provider. ### 333 — Renaming an enum label used as a sharding key is unavailable. ```sql theme={null} ALTER TYPE region RENAME VALUE 'eu' TO 'europe'; ``` `regional_users` is sharded on the enum column `region`, so renaming a label would change where existing rows route. The same statement is allowed for an enum not used as a sharding key. ### 334 — Column storage selection is unavailable. ```sql theme={null} ALTER TABLE t ALTER COLUMN a SET STORAGE EXTERNAL; ``` The `SET STORAGE` action is the rejected part: Neki cannot preserve a column's storage setting for a shard that joins the cluster later, so the setting would not be uniform across shards. ### 335 — Column compression selection is unavailable. ```sql theme={null} ALTER TABLE t ALTER COLUMN a SET COMPRESSION pglz; ``` As code 334, for a column's compression method. ### 336 — Changing a routing column's generated expression is unavailable. ```sql theme={null} ALTER TABLE regional_accounts ALTER COLUMN account_id SET EXPRESSION AS (region_id * 2000000 + local_id); ``` `account_id` is both a generated column and the shard key of `regional_accounts`. Changing or dropping its generation expression could send existing rows to another shard without moving them. The same change is allowed for a generated column that does not drive routing. ### 337 — Restarting an identity column through ALTER TABLE is unavailable. ```sql theme={null} ALTER TABLE notes ADD COLUMN seq bigint GENERATED BY DEFAULT AS IDENTITY; ALTER TABLE notes ALTER COLUMN seq RESTART; ``` Neki cannot reset the router's reservations for the identity sequence. Restart the column's owned sequence with `ALTER SEQUENCE` instead. ### 338 — Temporary tables, views and sequences are unavailable. ```sql theme={null} CREATE TEMP TABLE session_cart (sku text PRIMARY KEY, quantity integer NOT NULL); ``` A temporary relation lives in the PostgreSQL backend that created it, and the router holds no shard backend for a session. The relation would be created on each shard's pooled backend, where no later statement could see it, so the statement is rejected instead. `CREATE TEMP TABLE ... AS`, `CREATE TEMP VIEW`, `CREATE TEMP SEQUENCE`, and a name qualified with `pg_temp` report the same code. Create a regular relation and drop it when done. ### 339 — Temporary types and domains are unavailable. ```sql theme={null} CREATE TYPE pg_temp.mood AS ENUM ('happy', 'sad'); ``` A temporary type would be created on each shard's pooled backend, where it outlives the session. Create a regular type and drop it when done. ### 340 — Temporary functions, procedures, aggregates and operators are unavailable. ```sql theme={null} CREATE FUNCTION pg_temp.one() RETURNS int LANGUAGE sql AS 'SELECT 1'; ``` A temporary routine would be created on each shard's pooled backend, where it outlives the session. Create a regular routine and drop it when done. ### 341 — Temporary statistics are unavailable. ```sql theme={null} CREATE STATISTICS pg_temp.users_stats ON user_id, name FROM users; ``` Temporary statistics would be created on each shard's pooled backend, where they outlive the session. Create regular statistics and drop them when done. ### 342 — This database encoding is unavailable to the router. ```sql theme={null} CREATE DATABASE reports ENCODING 'EUC_TW'; ``` The router reads and evaluates text as UTF-8. Use `UTF8`, the default, or another encoding the router supports. ### 343 — Converting an existing table to a typed table is unavailable. ```sql theme={null} CREATE TYPE note_type AS (id bigint, body text); ALTER TABLE notes OF note_type; ``` Neki cannot restore the typed-table relationship when a shard joins. ### 347 — Attaching or detaching partitions with different routing is unavailable. ```sql theme={null} CREATE TABLE events (event_id bigint, event_date date) PARTITION BY RANGE (event_date); CREATE TABLE events_2026 (event_id bigint, event_date date); ALTER TABLE events ATTACH PARTITION events_2026 FOR VALUES FROM ('2026-01-01') TO ('2027-01-01'); ``` Depends on the topology: the partition is not configured in the data topology, so attaching it would change how its rows are routed. ### 348 — Changing all triggers through ALTER TABLE is unavailable. ```sql theme={null} ALTER TABLE users DISABLE TRIGGER ALL; ``` Neki cannot restore an `ALL` change to a shard that joins later. Name the trigger, or use the `USER` form. ### 351 — LISTEN, UNLISTEN and NOTIFY are unavailable in stored expressions shards evaluate. ```sql theme={null} CREATE TABLE notifications AS SELECT pg_notify('c', 'p'); ``` A stored expression runs on the shard that performs the write, not on the notification channel's shard, so the notification would never reach a listener. ### 515 — A recursive CTE with a composite-typed column is unavailable on the router. ```sql theme={null} CREATE TYPE user_location AS (region_id integer, city text); ALTER TABLE users ADD COLUMN location user_location; WITH RECURSIVE r AS (SELECT u.user_id, u.location FROM users u WHERE u.user_id = 1 UNION ALL SELECT u.user_id, u.location FROM users u JOIN r ON u.location = r.location AND u.user_id = r.user_id + 1) SELECT * FROM r; ``` The recursive CTE crosses shards and carries a composite column, whose type the router cannot carry through this recursive plan. ### 554 — A multiply referenced MATERIALIZED CTE returning an anonymous record with a non-static shape is unavailable on the router. ```sql theme={null} WITH c AS MATERIALIZED (SELECT CASE WHEN user_id > 0 THEN ROW(user_id) ELSE ROW(user_id, user_id) END AS r, user_id FROM users) SELECT r FROM c UNION ALL SELECT r FROM c; ``` The two references share a materialized result, but the record has a different number of fields in each `CASE` branch. Use a fixed record shape. ### 568 — Creating user-defined base types is unavailable. ```sql theme={null} CREATE TYPE base_type (INPUT = int4in, OUTPUT = int4out, INTERNALLENGTH = 4); ``` Neki rejects both shell declarations and complete definitions of user-defined base types. Install a custom base type through an extension available on every shard. ### 573 — A data-modifying CTE statement that reads a written table outside a pre-write stage is unavailable on the router. ```sql theme={null} WITH batch AS MATERIALIZED (SELECT user_id FROM users), updated AS (UPDATE users t SET name = 'x' FROM batch b WHERE t.user_id = b.user_id RETURNING 1) SELECT count(*), (SELECT count(*) FROM updated) FROM users WHERE name = 'x'; ``` The outer query reads `users`, which the data-modifying CTE writes, outside the stage that runs before the write. ### 579 — This data-modifying CTE body is unavailable on the router. ```sql theme={null} WITH m AS (MERGE INTO users t USING ref_countries s ON t.user_id = s.country_id WHEN MATCHED THEN UPDATE SET name = 'x' RETURNING t.user_id) SELECT * FROM m; ``` The `MERGE` body cannot be hosted by the router CTE materialization boundary. ### 626 — Refreshing a database collation version is unavailable. ```sql theme={null} ALTER DATABASE postgres REFRESH COLLATION VERSION; ``` Neki manages collation versions across shards. ### 631 — Nested CTE name shadowing is unavailable on the router. ```sql theme={null} WITH c AS (SELECT user_id FROM users WHERE user_id = 2) SELECT c.user_id, d.user_id AS du FROM c, (WITH c AS (SELECT user_id FROM orders WHERE user_id = 2) SELECT user_id FROM c) d; ``` The CTE `c` inside the derived table shadows the outer CTE `c`. Give the inner CTE a distinct name. ### 645 — A data-modifying CTE body whose output differs from its write's RETURNING columns is unavailable on the router. ```sql theme={null} WITH upd AS (UPDATE users SET name='x' WHERE user_id IN (SELECT user_id FROM orders WHERE order_id IN (1,3)) RETURNING user_id, now()) SELECT * FROM upd; ``` The router-computed `now()` output does not match the write producer's `RETURNING` columns. This rejection depends on the query plan and data topology. ### 696 — ALTER EXTENSION membership changes are unavailable. ```sql theme={null} ALTER EXTENSION plpgsql ADD TABLE notes; ``` Develop extension membership outside Neki, then apply an extension update with `ALTER EXTENSION ... UPDATE`. ### 700 — Changing base-type I/O options is unavailable. ```sql theme={null} ALTER TYPE mood SET (RECEIVE = int4recv); ``` The `SET` option changes a base type's input or output functions. Neki cannot refresh the router's type metadata after that change, so it rejects the statement before it reaches a shard. ### 726 — A data-modifying CTE needing a join parameter at its execution fence is unavailable on the router. ```sql theme={null} WITH a AS NOT MATERIALIZED (INSERT INTO users (user_id, name, email) VALUES (1, 'a', 'a@example.com') RETURNING user_id), b AS (UPDATE orders SET status = 'x' WHERE user_id IN (SELECT user_id FROM a) RETURNING order_id, user_id) SELECT count(*) FROM a JOIN b USING (user_id) WHERE a.user_id IN (SELECT user_id FROM a); ``` The `UPDATE` in `b` needs a value from `a` that is not available where the router runs it. ### 801 — A whole-row CTE reference with a target-list subquery is unavailable on the router. ```sql theme={null} WITH c AS (SELECT EXISTS(SELECT 1 FROM users) AS e FROM orders) SELECT c FROM c; ``` The whole-row reference includes a subquery that the planner has extracted from the CTE target list. Select the individual columns instead of the whole row. ### 810 — A NOT MATERIALIZED SELECT CTE read more than once by a statement with a data-modifying CTE is unavailable on the router. ```sql theme={null} WITH batch AS NOT MATERIALIZED (SELECT user_id FROM users WHERE user_id > 1 ORDER BY user_id LIMIT 10), updated AS (UPDATE users SET name = 'x' FROM batch WHERE users.user_id = batch.user_id RETURNING 1) SELECT coalesce(max(batch.user_id), 1), count(*), (SELECT count(*) FROM updated) FROM batch; ``` `batch` is read by the `UPDATE` and by the outer query. Mark it `MATERIALIZED`. ### 816 — A data-modifying CTE body without verifiable write output columns is unavailable on the router. ```sql theme={null} WITH d AS (DELETE FROM orders o USING (SELECT 'x' AS s) v WHERE o.status = v.s) SELECT 1; ``` The router cannot verify the write output columns of this `DELETE ... USING` CTE, which has no `RETURNING`. This rejection depends on the data topology. ### 821 — A CTE exposing an unnamed subquery column is unavailable on the router. ```sql theme={null} WITH c(x) AS MATERIALIZED (SELECT EXISTS(SELECT 1 FROM users) FROM orders) SELECT c.x FROM c; ``` The extracted subquery has no output name in the CTE body. Give the expression an explicit alias inside the CTE. ### 896 — SECURITY LABEL statements are unavailable. ```sql theme={null} SECURITY LABEL ON TYPE mood IS 'classification'; ``` Neki does not yet install a security label provider, so it rejects `SECURITY LABEL` statements for all object types. ### 900 — This CREATE TABLE AS body cannot be populated safely. ```sql theme={null} CREATE TABLE users_copy AS SELECT * FROM users; ``` Depends on the topology: the new table has no routing configured, so the router cannot tell where its rows would be read. Create the table `WITH NO DATA`, add routing for it, then fill it with `INSERT ... SELECT`. A topology update is a complete replacement, not a partial patch. Retrieve the current document, add the new table to the shard group the source rows are read from — beside `users`, under `databases.app.schemas.public.tables` — and re-apply it: ```json theme={null} "users_copy": { "shard_group": "user_data" } ``` The group's default shard index routes on `user_id`, which the copied table keeps. ### 902 — CREATE TABLE AS EXECUTE is unavailable. ```sql theme={null} PREPARE recent_users AS SELECT user_id FROM users; CREATE TABLE users_copy AS EXECUTE recent_users; ``` Write the prepared query into the `CREATE TABLE AS` in place of `EXECUTE`. ### 905 — GROUP BY or DISTINCT on column "…" of CTE "…", which an outer join inside the CTE body can null, is unavailable on the router. ```sql theme={null} WITH c AS (SELECT o.user_id, u2.user_id AS uid FROM users u2 LEFT JOIN orders o ON u2.user_id = o.user_id) SELECT DISTINCT c.user_id FROM c LEFT JOIN users u ON u.user_id = c.uid; ``` `c.user_id` comes from the nullable side of the CTE's `LEFT JOIN`, and the outer query deduplicates on it. Mark the CTE `AS MATERIALIZED`, or write it as a derived table. ### 933 — This nested CTE inside a data-modifying statement is unavailable on the router. ```sql theme={null} WITH a AS (SELECT user_id FROM users WHERE user_id = 1), b AS (SELECT user_id FROM a) UPDATE orders SET status = 'cascaded' FROM b WHERE orders.user_id = b.user_id AND orders.user_id = 1 RETURNING orders.order_id; ``` The CTE chain feeds a data-modifying statement. This rejection depends on the query plan and data topology. ### 961 — This nested recursive CTE is unavailable on the router. ```sql theme={null} WITH RECURSIVE r AS (SELECT user_id FROM users WHERE user_id = 1 UNION ALL SELECT u.user_id FROM users u JOIN r ON u.user_id = r.user_id), cA AS (SELECT user_id FROM r) SELECT user_id FROM r UNION SELECT user_id FROM cA; ``` The recursive CTE is nested through another CTE reference. This rejection depends on the query plan and data topology. ### 967 — A nested CTE referenced more than once is unavailable on the router. ```sql theme={null} WITH c1 AS (SELECT user_id FROM users WHERE name = 'x'), c2 AS MATERIALIZED (SELECT user_id FROM c1) SELECT user_id FROM c2 UNION ALL SELECT user_id FROM c2; ``` A CTE body references another CTE and is itself referenced more than once. This rejection depends on the query plan and data topology. ### 972 — An unreferenced data-modifying CTE that references another CTE is unavailable on the router. ```sql theme={null} WITH a AS (UPDATE users SET name='a' WHERE user_id = 1 RETURNING user_id), b AS (UPDATE orders SET status='x' WHERE user_id IN (SELECT user_id FROM a) RETURNING order_id) SELECT 1; ``` The final query does not reference the second data-modifying CTE, whose body reads the first CTE. The router cannot execute this dependency chain as an unreferenced body. ### 981 — Changing a shard-index routing column is unavailable. ```sql theme={null} ALTER TABLE users ALTER COLUMN user_id TYPE text; ``` `user_id` drives routing, and changing it would alter routing without moving existing rows. ### 992 — A data-modifying CTE body with ON CONFLICT is unavailable on the router. ```sql theme={null} WITH x AS MATERIALIZED (INSERT INTO users (user_id, name, email) VALUES (1, 'a', 'b') ON CONFLICT (user_id) DO NOTHING RETURNING name) SELECT * FROM x; ``` The data-modifying CTE uses `ON CONFLICT`. This rejection depends on the query plan and data topology. ### 1018 — A data-modifying CTE returning an anonymous record with a non-static shape is unavailable on the router. ```sql theme={null} WITH d AS (DELETE FROM users RETURNING CASE WHEN user_id % 2 = 1 THEN ROW(user_id) ELSE ROW(user_id, name) END AS r) SELECT r FROM d UNION ALL SELECT r FROM d; ``` The `RETURNING` record has a different number of fields in each `CASE` branch. The router cannot serialize this record through the CTE materialization boundary. Use a fixed record shape. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Error hints and details Source: https://planetscale.com/docs/neki/error-hints-and-details How Neki uses the DETAIL and HINT fields of Postgres errors and notices, and why clients should not hide them. > [!NOTE] > > Neki is currently in Platform Preview. Platform Preview features are “Beta Features” under the PlanetScale Terms of Service or your applicable agreement with PlanetScale. Accordingly, Neki is subject to the limitations and disclaimers applicable to Beta Features and is not covered by any service level agreement. A Postgres error or notice can include more than its one-line message. The [protocol](https://www.postgresql.org/docs/current/protocol-error-fields.html) defines optional fields beside it, and two of them matter most to an application developer: * **DETAIL** gives more facts about what went wrong. * **HINT** suggests what to do about it. Neki generally follows [Postgres's own convention](https://www.postgresql.org/docs/current/error-style-guide.html) for these fields. The primary message stays short and factual, facts that do not fit on one line go in DETAIL, and advice goes in HINT. Configure the client tools and application logging you use against a Neki database to show DETAIL and HINT, and keep notices enabled so this guidance remains visible. ## What Neki puts in these fields ### Postgres errors keep their fields Neki reproduces Postgres's error messages, including their DETAIL and HINT text. An error that a shard's Postgres raises reaches the client with its fields intact. Anything you rely on these fields for in Postgres works the same way on Neki: ```text theme={null} ERROR: function unknown_fn(integer) does not exist HINT: No function matches the given name and argument types. You might need to add explicit type casts. ``` ### Neki points to the Neki way of doing it Some statements behave differently on a sharded database, and some have a Neki-specific replacement. When Neki rejects one of these, or answers it differently from single-node Postgres, the HINT usually names the Neki mechanism to use instead. The message on its own only says that something failed. A plain `EXPLAIN` of a statement that needs router processing fails, and the hint names the Neki `EXPLAIN` option that shows the plan: ```sql theme={null} EXPLAIN SELECT user_id FROM users ORDER BY user_id; ``` ```text theme={null} ERROR: This statement requires Neki router processing HINT: Use `EXPLAIN (NEKI_PG_PLAN)` (with ANALYZE to execute) to see Neki's plan ``` This error has SQLSTATE `NK017`. It marks an `EXPLAIN` shape that has no faithful single-server answer on a sharded topology, and the HINT is always set. See [Query planning](/docs/neki/query-planning#explain-neki_pg_plan) for `NEKI_PG_PLAN`. Reading `pg_locks` from a [replica](/docs/neki/replicas) session fails, because advisory locks exist only on the primary. The hint names the setting to change: ```text theme={null} ERROR: reading pg_catalog.pg_locks under a non-primary session target HINT: advisory locks are held on the primary and are never replicated: set __neki.target to primary before the read ``` A replication connection must name the shard it reads from. The hint shows the connection option to add: ```text theme={null} FATAL: replication connections must target a specific shard HINT: Set the target shard in the connection string, e.g. options=-c __neki.shard=. ``` ### Notices carry advice too Neki also reports conditions that do not fail a statement as `NOTICE` or `WARNING` messages, and these can carry a hint of their own. When a database's notifications span more than one shard, `pg_notification_queue_usage()` returns the fullest queue among them and adds this notice: ```text theme={null} NOTICE: pg_notification_queue_usage() reports the fullest queue across this database's notify shard group HINT: For the per-shard breakdown, query __neki.notification_queue_usage(). ``` Text-search notices use DETAIL for facts that do not fit in the primary message. When an input word is too long to index, Neki reports the limit separately: ```text theme={null} NOTICE: word is too long to be indexed DETAIL: Words longer than 2047 characters are ignored. ``` ### Where the fields are sparse A query that hits an implementation gap still being worked on fails with SQLSTATE `NK013` and a catalog code, as described in [Error codes](/docs/neki/error-codes). These rejections often carry no HINT. The catalog code and its entry are what tell you how to rewrite the query. The broad limits in [Platform preview limitations](/docs/neki/platform-preview-limitations) also aren't always reported with a hint. For everything else, expect Neki to use DETAIL and HINT the way Postgres does, and to point to a Neki mechanism where one exists. ## Keep hints and details visible Look for these settings in the tools and code that talk to Neki: * **`psql`.** `\set VERBOSITY terse` and `\set VERBOSITY sqlstate` drop DETAIL and HINT from every error and notice. Keep the default, `default`, or use `verbose`. * **`client_min_messages`.** Setting it to `warning` or higher in a session, role, or connection string stops notices from reaching the client, including the DDL propagation notice. Leave it at the default, `notice`. * **libpq-based clients.** `PQsetErrorVerbosity` with `PQERRORS_TERSE` or `PQERRORS_SQLSTATE` leaves DETAIL and HINT out of the formatted message. The fields are still available through `PQresultErrorField`. * **Drivers and ORMs.** Most drivers expose the fields separately from the message, for example `hint` and `detail` on the error object, and many application error handlers log only the message. Log the DETAIL and HINT fields too, and forward notices to your logs if your driver delivers them through a callback. * **Error-reporting and monitoring pipelines.** Make sure the fields survive when an error is serialized into your logs, alerts, or exception tracker. When you report a problem with a rejected query, include the full error: the SQLSTATE, the message, and any DETAIL and HINT. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Example schema Source: https://planetscale.com/docs/neki/example-schema The tables and data topology used by the SQL examples in the Neki documentation The SQL examples in the Neki documentation use one small example schema. This page defines its tables and the data topology that places them on shards. | Table | Placement | | - | - | | `users`, `orders`, `products`, `user_tags` | Sharded on `user_id` in the `user_data` shard group | | `accounts` | Sharded on `account_id` in the `user_data` shard group. Owns the global secondary indexes (GSIs) `by_email` and `by_phone` | | `accounts_by_email`, `accounts_by_phone` | Unsharded lookup tables for the `accounts` GSIs, in the `meta` shard group | | `notes` | Unsharded, in the `meta` shard group | | `ref_countries` | A reference table, with a copy on every shard in the `user_data` shard group | | `regional_users` | Sharded on the enum column `region` in the `user_data` shard group | | `regional_accounts` | Sharded on the generated column `account_id` in the `user_data` shard group | | `devices` | Sharded on the expression `owner_id + device_no` in the `user_data` shard group | ## Tables Every table is in the `public` schema of the `app` database: ```sql theme={null} CREATE TABLE public.users ( user_id bigint PRIMARY KEY, name varchar(255) NOT NULL, email varchar(255) NOT NULL, created_at timestamp NOT NULL DEFAULT now() ); CREATE TABLE public.orders ( order_id bigint PRIMARY KEY, user_id bigint NOT NULL, order_date date NOT NULL, total_amount numeric(10, 2), status varchar(50), created_at timestamp NOT NULL DEFAULT now() ); CREATE INDEX orders_user_id_idx ON public.orders (user_id); CREATE TABLE public.products ( product_id bigint PRIMARY KEY, user_id bigint NOT NULL, name varchar(255) NOT NULL, price numeric(10, 2) ); CREATE INDEX products_user_id_idx ON public.products (user_id); CREATE TABLE public.user_tags ( user_id bigint PRIMARY KEY, tag_ids integer[] NOT NULL ); CREATE TABLE public.accounts ( account_id bigint PRIMARY KEY, label text, email text NOT NULL, phone text NOT NULL, category integer ); CREATE TABLE public.accounts_by_email ( email text PRIMARY KEY, account_id bigint NOT NULL ); CREATE TABLE public.accounts_by_phone ( phone text PRIMARY KEY, account_id bigint NOT NULL ); CREATE TABLE public.notes ( id bigint PRIMARY KEY, body text NOT NULL ); CREATE TABLE public.ref_countries ( country_id integer PRIMARY KEY, name text NOT NULL ); CREATE TYPE public.region AS ENUM ('eu', 'us'); CREATE TABLE public.regional_users ( region public.region NOT NULL, user_id bigint NOT NULL, PRIMARY KEY (region, user_id) ); CREATE TABLE public.regional_accounts ( region_id integer NOT NULL, local_id bigint NOT NULL, account_id bigint GENERATED ALWAYS AS (region_id * 1000000 + local_id) STORED PRIMARY KEY ); CREATE TABLE public.devices ( owner_id bigint NOT NULL, device_no bigint NOT NULL, PRIMARY KEY (owner_id, device_no) ); ``` `users`, `orders`, `products`, and `user_tags` share the shard key `user_id`. Rows with the same `user_id` are on the same shard, so Neki can send a join of these tables on `user_id` to the shards as one query. `orders` and `products` have a Postgres index on `user_id`. The other sharded tables have their shard-key column as their primary key, which already indexes it. `accounts` is in the same shard group but uses a different shard key, `account_id`. Its rows are not co-located with the rows of the tables sharded on `user_id`. `regional_users`, `regional_accounts`, and `devices` show other ways to choose a shard key: an enum column, a generated column, and an expression over two columns. ## Data topology ```json theme={null} { "authoritative_shard_group": "meta", "shard_indexes": { "xxhash_user_id": { "type": "xxhash", "columns": ["user_id"] }, "xxhash_account_id": { "type": "xxhash", "columns": ["account_id"] }, "xxhash_region": { "type": "xxhash", "columns": ["region"] }, "xxhash_device": { "type": "xxhash", "columns": ["owner_id + device_no"] } }, "shard_groups": [ { "uid": "meta", "key_ranges": [{ "shard_uid": "shard-meta" }] }, { "uid": "user_data", "default_shard_index": "xxhash_user_id", "key_ranges": [ { "shard_uid": "shard-a", "end": "80" }, { "shard_uid": "shard-b", "start": "80" } ] } ], "databases": { "app": { "advisory_lock_shard_group": "user_data", "global_secondary_indexes": { "by_email": { "schema": "public", "table": "accounts_by_email", "owner_table": "accounts", "columns": ["email"], "unique": true, "enabled": true }, "by_phone": { "schema": "public", "table": "accounts_by_phone", "owner_table": "accounts", "columns": ["phone"], "unique": true, "enabled": true } }, "schemas": { "public": { "tables": { "users": { "shard_group": "user_data" }, "orders": { "shard_group": "user_data" }, "products": { "shard_group": "user_data" }, "user_tags": { "shard_group": "user_data" }, "accounts": { "shard_group": "user_data", "secondary_indexes": [ { "name": "by_email", "columns": ["email"] }, { "name": "by_phone", "columns": ["phone"] } ], "primary_indexes": [ { "shard_index": "xxhash_account_id" } ] }, "regional_users": { "shard_group": "user_data", "primary_indexes": [{ "shard_index": "xxhash_region" }] }, "regional_accounts": { "shard_group": "user_data", "primary_indexes": [{ "shard_index": "xxhash_account_id" }] }, "devices": { "shard_group": "user_data", "primary_indexes": [{ "shard_index": "xxhash_device" }] }, "accounts_by_email": { "shard_group": "meta" }, "accounts_by_phone": { "shard_group": "meta" }, "notes": { "shard_group": "meta" } }, "reference_tables": { "ref_countries": { "shard_groups": ["user_data"] } } } } } } } ``` The topology has two shard groups: * `user_data` places the sharded tables on `shard-a` and `shard-b`. Its default shard index, `xxhash_user_id`, hashes `user_id`. The `accounts` and `regional_accounts` entries override it with `xxhash_account_id`, which hashes `account_id`; `regional_users` and `devices` override it with `xxhash_region` and `xxhash_device`. It is also the advisory-lock shard group, so Neki hashes each advisory-lock key to one of its shards. * `meta` is the [authoritative shard group](/docs/neki/data-topology#authoritative-shard-group). Its one shard, `shard-meta`, holds the unsharded tables: `notes` and the two GSI lookup tables. `ref_countries` is a reference table. Each shard in `user_data` holds a full copy, so Neki can send a join of a sharded table with `ref_countries` to the shards. [Data topology](/docs/neki/data-topology) explains each part of the topology document. [Reference tables and GSIs](/docs/neki/reference-tables-and-gsis) explains the GSI and reference-table declarations. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # EXPLAIN output reference Source: https://planetscale.com/docs/neki/explain-reference Complete EXPLAIN (NEKI_PLAN) and EXPLAIN (NEKI_PG_PLAN) output for one query in every format The following query adds up the order totals after a date, as a monthly report would. It reads the [example schema](/docs/neki/example-schema): ```sql theme={null} SELECT sum(total_amount) FROM orders WHERE created_at > '2026-09-09'; ``` The query has no predicate on the shard key `user_id`, so the plan scatters to every shard in the group. Each shard computes a partial sum, and the router sums the partial sums. The following sections give the complete output of the query for `EXPLAIN (NEKI_PLAN)` and `EXPLAIN (NEKI_PG_PLAN)`, in each of the four output formats. [Query planning](/docs/neki/query-planning#explain-in-neki) explains the `EXPLAIN` modes and their options. ## `EXPLAIN (NEKI_PLAN)` ```sql theme={null} EXPLAIN (NEKI_PLAN, FORMAT TEXT) SELECT sum(total_amount) FROM orders WHERE created_at > '2026-09-09'; ``` Replace `TEXT` with `JSON`, `XML`, or `YAML` for the other formats. **TEXT** ```text theme={null} Aggregate [Ordered] (cost=0.00..2002.00 rows=1 width=32) -> Collapse (cost=0.00..2002.00 rows=2 width=32) -> Route [Scatter] (cost=0.00..2002.00 rows=2 width=32) Query: SELECT sum(total_amount) FROM public.orders WHERE created_at > $1::timestamp without time zone ShardGroup: user_data ``` **JSON** ```json theme={null} [ { "Plan": { "Node Type": "Aggregate", "Variant": "Ordered", "Startup Cost": 0.00, "Total Cost": 2002.00, "Plan Rows": 1, "Plan Width": 32, "Aggregates": [ "pg_catalog.sum(sum(total_amount) /*0*/) :: out_0" ], "Cost": { "Self": { "RowsProcessed": 2, "Total": 0 }, "Subtree": { "NetworkRows": 2, "RemoteCalls": 2, "RowsProcessed": 2, "Total": 2002 } }, "Plans": [ { "Node Type": "Collapse", "Startup Cost": 0.00, "Total Cost": 2002.00, "Plan Rows": 2, "Plan Width": 32, "Cost": { "Subtree": { "NetworkRows": 2, "RemoteCalls": 2, "Total": 2002 } }, "Plans": [ { "Node Type": "Route", "Variant": "Scatter", "Startup Cost": 0.00, "Total Cost": 2002.00, "Plan Rows": 2, "Plan Width": 32, "ColumnOutputMapping": "[-1]", "OuterParams": "[0]", "ParamTypes": "[text]", "Query": "SELECT sum(total_amount) FROM public.orders WHERE created_at \u003e $1::timestamp without time zone", "ShardGroupUid": "user_data", "Cost": { "Subtree": { "NetworkRows": 2, "RemoteCalls": 2, "Total": 2002 } } } ] } ] } } ] ``` **XML** ```xml theme={null} Aggregate Ordered 0.00 2002.00 1 32 pg_catalog.sum(sum(total_amount) /*0*/) :: out_0 2 0 2 2 2 2002 Collapse 0.00 2002.00 2 32 2 2 2002 Route Scatter 0.00 2002.00 2 32 [-1] [0] [text] SELECT sum(total_amount) FROM public.orders WHERE created_at > $1::timestamp without time zone user_data 2 2 2002 ``` **YAML** ```yaml theme={null} - Plan: Node Type: Aggregate Variant: Ordered Startup Cost: 0.00 Total Cost: 2002.00 Plan Rows: 1 Plan Width: 32 Aggregates: - 'pg_catalog.sum(sum(total_amount) /*0*/) :: out_0' Cost: Self: RowsProcessed: 2 Total: 0 Subtree: NetworkRows: 2 RemoteCalls: 2 RowsProcessed: 2 Total: 2002 Plans: - Node Type: Collapse Startup Cost: 0.00 Total Cost: 2002.00 Plan Rows: 2 Plan Width: 32 Cost: Subtree: NetworkRows: 2 RemoteCalls: 2 Total: 2002 Plans: - Node Type: Route Variant: Scatter Startup Cost: 0.00 Total Cost: 2002.00 Plan Rows: 2 Plan Width: 32 ColumnOutputMapping: '[-1]' OuterParams: '[0]' ParamTypes: '[text]' Query: SELECT sum(total_amount) FROM public.orders WHERE created_at > $1::timestamp without time zone ShardGroupUid: user_data Cost: Subtree: NetworkRows: 2 RemoteCalls: 2 Total: 2002 ``` The route's `Query` shows the date literal as `$1`. Neki replaces literal values with parameters before it looks up a plan, as [Query plan cache and protocols](/docs/neki/query-planning#query-plan-cache-and-protocols) describes. ## `EXPLAIN (NEKI_PG_PLAN)` ```sql theme={null} EXPLAIN (NEKI_PG_PLAN, FORMAT TEXT) SELECT sum(total_amount) FROM orders WHERE created_at > '2026-09-09'; ``` The route also shows the Postgres plan from every shard in its shard group. **TEXT** ```text theme={null} Aggregate [Ordered] (cost=0.00..2002.00 rows=1 width=32) -> Collapse (cost=0.00..2002.00 rows=2 width=32) -> Route [Scatter] (cost=0.00..2002.00 rows=2 width=32) Query: SELECT sum(total_amount) FROM public.orders WHERE created_at > $1::timestamp without time zone ShardGroup: user_data All shard plans -> Shard shard-a (cost=0.00..0.00 rows=0 width=0) -> Aggregate (cost=2.45..2.46 rows=1 width=32) -> Seq Scan on orders (cost=0.00..2.40 rows=18 width=5) Filter: (created_at > ('2026-09-09'::cstring)::timestamp without time zone) -> Shard shard-b (cost=0.00..0.00 rows=0 width=0) -> Aggregate (cost=4.17..4.18 rows=1 width=32) -> Seq Scan on orders (cost=0.00..4.10 rows=26 width=5) Filter: (created_at > ('2026-09-09'::cstring)::timestamp without time zone) ``` **JSON** ```json theme={null} [ { "Plan": { "Node Type": "Aggregate", "Variant": "Ordered", "Startup Cost": 0.00, "Total Cost": 2002.00, "Plan Rows": 1, "Plan Width": 32, "Aggregates": [ "pg_catalog.sum(sum(total_amount) /*0*/) :: out_0" ], "Cost": { "Self": { "RowsProcessed": 2, "Total": 0 }, "Subtree": { "NetworkRows": 2, "RemoteCalls": 2, "RowsProcessed": 2, "Total": 2002 } }, "Plans": [ { "Node Type": "Collapse", "Startup Cost": 0.00, "Total Cost": 2002.00, "Plan Rows": 2, "Plan Width": 32, "Cost": { "Subtree": { "NetworkRows": 2, "RemoteCalls": 2, "Total": 2002 } }, "Plans": [ { "Node Type": "Route", "Variant": "Scatter", "Startup Cost": 0.00, "Total Cost": 2002.00, "Plan Rows": 2, "Plan Width": 32, "ColumnOutputMapping": "[-1]", "OuterParams": "[0]", "ParamTypes": "[text]", "Query": "SELECT sum(total_amount) FROM public.orders WHERE created_at \u003e $1::timestamp without time zone", "ShardGroupUid": "user_data", "ShardPlans": { "All": [ { "ShardPlan": { "Shard": "shard-a", "Plan": { "Node Type": "Aggregate", "Strategy": "Plain", "Partial Mode": "Simple", "Parallel Aware": false, "Async Capable": false, "Startup Cost": 2.45, "Total Cost": 2.46, "Plan Rows": 1, "Plan Width": 32, "Disabled": false, "Plans": [ { "Node Type": "Seq Scan", "Parent Relationship": "Outer", "Parallel Aware": false, "Async Capable": false, "Relation Name": "orders", "Alias": "orders", "Startup Cost": 0.00, "Total Cost": 2.40, "Plan Rows": 18, "Plan Width": 5, "Disabled": false, "Filter": "(created_at \u003e ('2026-09-09'::cstring)::timestamp without time zone)" } ] } } }, { "ShardPlan": { "Shard": "shard-b", "Plan": { "Node Type": "Aggregate", "Strategy": "Plain", "Partial Mode": "Simple", "Parallel Aware": false, "Async Capable": false, "Startup Cost": 4.17, "Total Cost": 4.18, "Plan Rows": 1, "Plan Width": 32, "Disabled": false, "Plans": [ { "Node Type": "Seq Scan", "Parent Relationship": "Outer", "Parallel Aware": false, "Async Capable": false, "Relation Name": "orders", "Alias": "orders", "Startup Cost": 0.00, "Total Cost": 4.10, "Plan Rows": 26, "Plan Width": 5, "Disabled": false, "Filter": "(created_at \u003e ('2026-09-09'::cstring)::timestamp without time zone)" } ] } } } ] }, "Cost": { "Subtree": { "NetworkRows": 2, "RemoteCalls": 2, "Total": 2002 } } } ] } ] } } ] ``` **XML** ```xml theme={null} Aggregate Ordered 0.00 2002.00 1 32 pg_catalog.sum(sum(total_amount) /*0*/) :: out_0 2 0 2 2 2 2002 Collapse 0.00 2002.00 2 32 2 2 2002 Route Scatter 0.00 2002.00 2 32 [-1] [0] [text] SELECT sum(total_amount) FROM public.orders WHERE created_at > $1::timestamp without time zone user_data shard-a Aggregate Plain Simple false false 2.45 2.46 1 32 false Seq Scan Outer false false orders orders 0.00 2.40 18 5 false (created_at > ('2026-09-09'::cstring)::timestamp without time zone) shard-b Aggregate Plain Simple false false 4.17 4.18 1 32 false Seq Scan Outer false false orders orders 0.00 4.10 26 5 false (created_at > ('2026-09-09'::cstring)::timestamp without time zone) 2 2 2002 ``` **YAML** ```yaml theme={null} - Plan: Node Type: Aggregate Variant: Ordered Startup Cost: 0.00 Total Cost: 2002.00 Plan Rows: 1 Plan Width: 32 Aggregates: - 'pg_catalog.sum(sum(total_amount) /*0*/) :: out_0' Cost: Self: RowsProcessed: 2 Total: 0 Subtree: NetworkRows: 2 RemoteCalls: 2 RowsProcessed: 2 Total: 2002 Plans: - Node Type: Collapse Startup Cost: 0.00 Total Cost: 2002.00 Plan Rows: 2 Plan Width: 32 Cost: Subtree: NetworkRows: 2 RemoteCalls: 2 Total: 2002 Plans: - Node Type: Route Variant: Scatter Startup Cost: 0.00 Total Cost: 2002.00 Plan Rows: 2 Plan Width: 32 ColumnOutputMapping: '[-1]' OuterParams: '[0]' ParamTypes: '[text]' Query: SELECT sum(total_amount) FROM public.orders WHERE created_at > $1::timestamp without time zone ShardGroupUid: user_data ShardPlans: All: - ShardPlan: Shard: shard-a Plan: Node Type: "Aggregate" Strategy: "Plain" Partial Mode: "Simple" Parallel Aware: false Async Capable: false Startup Cost: 2.45 Total Cost: 2.46 Plan Rows: 1 Plan Width: 32 Disabled: false Plans: - Node Type: "Seq Scan" Parent Relationship: "Outer" Parallel Aware: false Async Capable: false Relation Name: "orders" Alias: "orders" Startup Cost: 0.00 Total Cost: 2.40 Plan Rows: 18 Plan Width: 5 Disabled: false Filter: "(created_at > ('2026-09-09'::cstring)::timestamp without time zone)" - ShardPlan: Shard: shard-b Plan: Node Type: "Aggregate" Strategy: "Plain" Partial Mode: "Simple" Parallel Aware: false Async Capable: false Startup Cost: 4.17 Total Cost: 4.18 Plan Rows: 1 Plan Width: 32 Disabled: false Plans: - Node Type: "Seq Scan" Parent Relationship: "Outer" Parallel Aware: false Async Capable: false Relation Name: "orders" Alias: "orders" Startup Cost: 0.00 Total Cost: 4.10 Plan Rows: 26 Plan Width: 5 Disabled: false Filter: "(created_at > ('2026-09-09'::cstring)::timestamp without time zone)" Cost: Subtree: NetworkRows: 2 RemoteCalls: 2 Total: 2002 ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres extensions Source: https://planetscale.com/docs/neki/extensions Built-in and community Postgres extensions available on Neki. Postgres extensions add data types, functions, operators, background workers, and other capabilities to Postgres. Extensions that do not require a profile-level toggle install in a logical database with `CREATE EXTENSION`. ## Extension availability Extensions marked **Always enabled** are part of Neki's managed configuration and cannot be toggled. Other extensions can use either of these workflows: * A checkbox enables server configuration that the extension needs, such as a preload setting. Changing it creates a configuration-profile change. * An extension without a checkbox does not require a profile-level toggle. You install it in a logical database with `CREATE EXTENSION` when it is available. Enabling an extension for a configuration profile and installing its database objects are separate operations. A profile-level change applies to the Postgres instances in shards assigned to that profile. `CREATE EXTENSION` installs objects in the logical database to which you are connected. ## Enable an extension for a profile You must have permission to update the database, and the branch must be ready. **Step 1** From the PlanetScale dashboard, select your Neki database and branch. **Step 2** Open **Clusters** and select the configuration profile used by the target shards. **Step 3** Select the **Extensions** tab. **Step 4** Enable the extension. If the extension exposes parameters, review and update them. **Step 5** Select **Apply now** for any parameter changes. **Step 6** Open **Changes** and wait for the configuration change to finish before using the extension. > [!WARNING] > > A configuration-profile change affects every shard assigned to that profile. > Confirm the assigned shards before changing an extension or its parameters. Some profile changes can restart Postgres processes or roll through the instances in the profile. Use the change status to follow the rollout and avoid making dependent application changes before it completes. The available extension list for a profile is also available from `pscale branch config-profile extensions`. Enable or disable a catalog extension that the profile marks as enablable with `pscale branch config-profile extensions enable` or `pscale branch config-profile extensions disable`. ## Install an extension in a logical database After any required profile change has completed, connect to the target logical database with its default role. When the extension's instructions require database installation, run: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS extension_name; ``` Replace `extension_name` with the extension you want to install. Repeat the command for each logical database that needs the extension. To see extensions that are installed in the current logical database: ```sql theme={null} SELECT name, default_version, installed_version, comment FROM pg_available_extensions WHERE installed_version IS NOT NULL ORDER BY name; ``` > [!WARNING] > > During Platform Preview, do not use `ALTER EXTENSION ... ADD` to attach a > user-created table, type, function, sequence, view, or other object to an > extension. A shard created or rebuilt later may not recreate that object. > Contact PlanetScale Support before using extension-membership DDL. ## Supported extensions ### Always enabled These extensions are part of Neki's managed configuration and cannot be disabled. | Extension | Version | Purpose | | - | - | - | | `neki_xxhash` | 0.4 | Hash functions used by Neki distribution | | `pg_pscale_utils` | — | Handles privileged actions without granting superuser access | | `pgextwlist` | — | Controls which extensions can be installed | | `pginsights` | — | Collects per-query execution statistics for [Query Insights](/docs/neki/monitoring/query-insights) | | `plpgsql` | 1.0 | Built-in PL/pgSQL procedural language | ### Built-in PostgreSQL extensions These modules ship with PostgreSQL. They do not need a configuration-profile checkbox. Install each one in the logical databases that need it: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS btree_gist; ``` | Extension | Version | Database install | Purpose | | - | - | - | - | | [`bloom`](https://www.postgresql.org/docs/current/bloom.html) | 1.0 | `CREATE EXTENSION bloom` | Bloom filter index access method | | [`btree_gin`](https://www.postgresql.org/docs/current/btree-gin.html) | 1.3 | `CREATE EXTENSION btree_gin` | B-tree equivalent operator classes for GIN | | [`btree_gist`](https://www.postgresql.org/docs/current/btree-gist.html) | 1.8 | `CREATE EXTENSION btree_gist` | B-tree equivalent operator classes for GiST | | [`citext`](https://www.postgresql.org/docs/current/citext.html) | 1.8 | `CREATE EXTENSION citext` | Case-insensitive text type | | [`cube`](https://www.postgresql.org/docs/current/cube.html) | 1.5 | `CREATE EXTENSION cube` | Multidimensional cube type | | [`fuzzystrmatch`](https://www.postgresql.org/docs/current/fuzzystrmatch.html) | 1.2 | `CREATE EXTENSION fuzzystrmatch` | String similarity and distance functions | | [`hstore`](https://www.postgresql.org/docs/current/hstore.html) | 1.8 | `CREATE EXTENSION hstore` | Key/value pairs stored in a single value | | [`insert_username`](https://www.postgresql.org/docs/current/contrib-spi.html#CONTRIB-SPI-INSERT-USERNAME) | 1.0 | `CREATE EXTENSION insert_username` | Trigger functions that record who changed a table | | [`intarray`](https://www.postgresql.org/docs/current/intarray.html) | 1.5 | `CREATE EXTENSION intarray` | Functions and operators for integer arrays | | [`ltree`](https://www.postgresql.org/docs/current/ltree.html) | 1.3 | `CREATE EXTENSION ltree` | Labels in a hierarchical tree | | [`moddatetime`](https://www.postgresql.org/docs/current/contrib-spi.html#CONTRIB-SPI-MODDATETIME) | 1.0 | `CREATE EXTENSION moddatetime` | Trigger functions that record last modification time | | [`pg_trgm`](https://www.postgresql.org/docs/current/pgtrgm.html) | 1.6 | `CREATE EXTENSION pg_trgm` | Trigram text similarity | | [`pgcrypto`](https://www.postgresql.org/docs/current/pgcrypto.html) | 1.4 | `CREATE EXTENSION pgcrypto` | Cryptographic functions | | [`tcn`](https://www.postgresql.org/docs/current/tcn.html) | 1.0 | `CREATE EXTENSION tcn` | Trigger function that notifies listeners of table changes | | [`tsm_system_rows`](https://www.postgresql.org/docs/current/tsm-system-rows.html) | 1.0 | `CREATE EXTENSION tsm_system_rows` | `TABLESAMPLE` method `SYSTEM_ROWS` | | [`tsm_system_time`](https://www.postgresql.org/docs/current/tsm-system-time.html) | 1.0 | `CREATE EXTENSION tsm_system_time` | `TABLESAMPLE` method `SYSTEM_TIME` | | [`unaccent`](https://www.postgresql.org/docs/current/unaccent.html) | 1.1 | `CREATE EXTENSION unaccent` | Text-search dictionary that strips accents | | [`uuid-ossp`](https://www.postgresql.org/docs/current/uuid-ossp.html) | 1.1 | `CREATE EXTENSION "uuid-ossp"` | UUID generation functions | ### Community extensions | Extension | Version | Profile toggle | Database install | Notes | | - | - | - | - | - | | [`vector`](#pgvector) (pgvector) | 0.8.5 | Enable | `CREATE EXTENSION vector` | Usable on unsharded and sharded Neki with [query-shape limits](#pgvector) | | [`vectorscale`](#vectorscale) | 0.9.0 | — | `CREATE EXTENSION vectorscale` | Requires `vector`. Enable on the profile to expose DiskANN query parameters. Inherits the [pgvector query-shape limits](#pgvector) | ## Extension-specific caveats Neki can use an extension without implementing every function, cast, or cross-shard query shape that Postgres accepts on a single node. Prefer the working forms below. A rejected shape often returns `NK013` for an unimplemented input, receive, or user-defined function. ### pgvector Install [pgvector](https://github.com/pgvector/pgvector) after you enable it on the configuration profile: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS vector; ``` Nearest-neighbor search works for `vector`, `halfvec`, `sparsevec`, and binary vectors, including HNSW and IVFFlat indexes. Tenant-scoped, shard-targeted, and cross-shard searches work. You can project the distance and order by the alias, or repeat the distance expression in `ORDER BY`: ```sql theme={null} SELECT tenant_id, item_id, embedding <-> '[1,0,0]'::vector AS distance FROM public.vector_items ORDER BY distance LIMIT 20; ``` Repeating the operator in `ORDER BY`, and ordering by an alias, both work for cosine (`<=>`), inner product (`<#>`), and L1 (`<+>`) on `vector`, `halfvec`, and `sparsevec`. Hamming (`<~>`) and Jaccard (`<%>`) work the same way on `bit`. These also work on a sharded table: * A prepared parameter declared as `vector`, and a `text` parameter cast with `$1::vector`. * `ARRAY[1,0,0]::vector` and a vector literal such as `'[1,0,0]'::vector`. Cast the literal when the distance is in the select list. An uncast `embedding <-> '[1,0,0]'` there fails with `operator does not exist: vector <-> text`, for every distance operator. The uncast form resolves in `ORDER BY` and `WHERE`. * Global `avg(vector)` and `sum(vector)`. The router scatters and combines the aggregate. Tenant-scoped and direct-shard aggregates work too. * `INSERT ... SELECT` of vector values, including a statement that reads more than one shard. `l2_norm(embedding)` is ambiguous (`42725`) even when the column type is `vector`. IVFFlat recall is poor at the default `ivfflat.probes=1`. Raise `probes` when you need higher recall. HNSW recall depends on `hnsw.ef_search`. ### vectorscale [vectorscale](https://github.com/timescale/pgvectorscale) adds StreamingDiskANN indexes on top of pgvector. Enable and install `vector` first, then: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS vectorscale; ``` Tenant-routed and scatter searches that use a StreamingDiskANN index work when they follow the [pgvector `ORDER BY distance` shape](#pgvector). Enabling `vectorscale` on the **Extensions** tab exposes these DiskANN query parameters: | Parameter | Dashboard default | Description | | - | - | - | | `diskann.query_search_list_size` | `100` | The size of the search list used in queries. | | `diskann.query_rescore` | `50` | The number of elements rescored (`0` disables rescoring). | ## Change extension parameters Parameters associated with an extension appear beneath it after the extension is enabled. Their valid values, defaults, and restart requirements are shown in the dashboard. Updating them creates the same kind of configuration-profile change as other [configuration parameters](/docs/neki/cluster-configuration/parameters). ## Disable or remove an extension Disabling a profile-level extension setting does not run `DROP EXTENSION` and does not remove database objects from logical databases. Before disabling it, remove or migrate any database objects and application features that depend on it according to that extension's instructions. `DROP EXTENSION` is a separate Postgres operation and can remove dependent objects. Review the objects that depend on an extension before running it. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Bun application to Neki Source: https://planetscale.com/docs/neki/guides/bun Bun SQL connections and replica routing Bun includes a native Postgres client that can connect directly to Neki. Before you start, you need: * A Neki database with a ready branch. * An application [role](/docs/neki/connecting/roles) for that branch. Use `pg_read_all_data` for read traffic and add `pg_write_all_data` when the application writes rows. These inherited data roles do not grant `CREATE` or `ALTER`, so application credentials cannot change the schema. * A separate migration role, if you run schema changes against the branch. DDL requires the `postgres` inherited role. Neki restricts execution of the cross-router DDL barrier function `__neki.wait_for_ddl` to its `neki_viewer` role, so add `neki_viewer`, which requires `pg_read_all_data`, to the same role. Keep the migration credentials out of the application's runtime configuration. * The **Primary** connection details from the database's **Connect** page. Use the complete generated username. It contains the branch routing information required by Neki. ## Create a Bun project Install [Bun](https://bun.com/docs/installation), then initialize a project: ```bash theme={null} bun init ``` ## Add the connection string Bun automatically loads local environment variables from `.env`. Copy the connection URI from the **Connect** page: ```bash .env theme={null} DATABASE_URL='postgresql://:@:5432/postgres?sslmode=verify-full' ``` Use the generated URI so reserved characters in the credentials remain URL encoded. Bun supports `sslmode=verify-full` directly, verifying the server certificate against trusted CAs and checking the hostname. Do not add `sslrootcert=system`; Bun does not use it as a trust-store selector. Do not commit the `.env` file. ## Connect and run a query Use Bun's `sql` template literal: ```typescript index.ts theme={null} import { sql } from 'bun' const result = await sql` SELECT current_database() AS database, current_user AS role ` console.log(result[0]) await sql.close() ``` Run the application: ```bash theme={null} bun index.ts ``` ## Apply schema changes > [!NOTE] > > Framework migration commands send DDL through a normal Neki connection. The > router fans that DDL out to the managed shards, but it does not create a > managed schema-change workflow. Use a [schema-change > workflow](/docs/neki/schema-changes) when you want Neki to prepare and coordinate > an Online DDL change. Run migrations with the migration role described in the prerequisites. An application role that inherits only `pg_read_all_data` and `pg_write_all_data` cannot run `CREATE` or `ALTER`, so a migration that uses application credentials fails on its first DDL statement. Apply migration DDL with `psql` and the migration role so the client prints PostgreSQL notices: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f ``` The router that commits the DDL emits a `NOTICE` containing the barrier call for that transaction: ```text theme={null} NOTICE: DDL is committed and the change is visible on this router; other routers may not see it yet. To wait until it is visible on every router, use __neki.wait_for_ddl(42, 7) ``` The schema and cluster versions belong to that transaction, and the notice is the only place Neki reports them. Before you deploy application code that depends on the new schema, run the emitted call verbatim on a primary connection that uses the migration role: ```sql theme={null} SELECT __neki.wait_for_ddl(42, 7); ``` The call returns after every router has applied both versions. A successful migration only confirms that the DDL committed and is visible through the router that ran it; it does not replace this cross-router barrier. Some framework migration commands discard notices. Generate migration SQL with the framework, but apply it with `psql` when the framework cannot preserve the notice. If the notice is lost, its version pair cannot be reconstructed. Bun SQL does not expose PostgreSQL notices. Apply migration SQL with `psql` so you can capture and run the DDL barrier. For read-only traffic, add the Neki replica startup option described in [Primary and replica routing](/docs/neki/connecting#primary-and-replica-routing) to the connection URI. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect Cloudflare Workers to Neki Source: https://planetscale.com/docs/neki/guides/cloudflare-workers Cloudflare Workers connection pooling through Hyperdrive [Cloudflare Hyperdrive](https://developers.cloudflare.com/hyperdrive/) maintains a connection pool between Cloudflare Workers and Neki. The Worker connects to the Hyperdrive binding, and Hyperdrive connects to the Neki router. Before you start, you need: * A Neki database with a ready branch. * An application [role](/docs/neki/connecting/roles) for that branch. Use `pg_read_all_data` for read traffic and add `pg_write_all_data` when the application writes rows. These inherited data roles do not grant `CREATE` or `ALTER`, so application credentials cannot change the schema. * A separate migration role, if you run schema changes against the branch. DDL requires the `postgres` inherited role. Neki restricts execution of the cross-router DDL barrier function `__neki.wait_for_ddl` to its `neki_viewer` role, so add `neki_viewer`, which requires `pg_read_all_data`, to the same role. Keep the migration credentials out of the application's runtime configuration. * The **Primary** connection details from the database's **Connect** page. Use the complete generated username. It contains the branch routing information required by Neki. You also need a Cloudflare account and a Workers project with Wrangler installed and authenticated. ## Create a Workers project Create a TypeScript Worker and install `pg`: ```bash theme={null} npm create cloudflare@latest -- neki-worker cd neki-worker npm install 'pg@>8.16.3' npm install --save-dev @types/pg ``` When prompted, choose a Worker-only application and do not deploy it yet. ## Create a Hyperdrive configuration Cloudflare Hyperdrive requires a `pg` version newer than 8.16.3. > [!WARNING] > > Cloudflare currently documents Hyperdrive compatibility through Postgres 17. > Neki identifies as Postgres 18. Validate your required query paths before > using this integration in production. Download the ISRG Root X1 certificate in PEM format from [Let's Encrypt](https://letsencrypt.org/certificates/), then upload the single certificate to Cloudflare: ```bash theme={null} npx wrangler cert upload certificate-authority \ --ca-cert isrg-root-x1.pem \ --name neki-isrg-root-x1 ``` The command returns a CA certificate ID. Copy the **Primary** connection URI from Neki's **Connect** page and remove the `sslmode` and `sslrootcert` query parameters. Hyperdrive configures TLS to the Neki origin separately from the connection URI. Create the Hyperdrive configuration with the CA certificate ID and `verify-full` SSL mode: ```bash theme={null} npx wrangler hyperdrive create neki-main \ --connection-string='' \ --ca-certificate-id \ --sslmode verify-full ``` `verify-full` verifies the certificate chain and checks that the Neki hostname matches the certificate. See Cloudflare's [Hyperdrive TLS configuration](https://developers.cloudflare.com/hyperdrive/configuration/tls-ssl-certificates-for-hyperdrive/) for details. Wrangler verifies the database connection before creating the configuration. The command returns a Hyperdrive ID. Add that ID to `wrangler.jsonc`. Use a recent `compatibility_date` rather than a hardcoded older date: ```jsonc wrangler.jsonc theme={null} { "compatibility_date": "", "compatibility_flags": ["nodejs_compat"], "hyperdrive": [ { "binding": "HYPERDRIVE", "id": "" } ] } ``` ## Query Neki from the Worker Create a `pg` client from the Hyperdrive connection string: ```typescript src/index.ts theme={null} import { Client } from 'pg' interface Env { HYPERDRIVE: Hyperdrive } export default { async fetch(_request: Request, env: Env): Promise { const client = new Client({ connectionString: env.HYPERDRIVE.connectionString, }) await client.connect() const result = await client.query( 'SELECT current_database() AS database, current_user AS role' ) return Response.json(result.rows[0]) }, } satisfies ExportedHandler ``` Create the client inside the request handler. Hyperdrive pools the underlying origin connections, and Workers automatically clean up the client connection when the request ends. You do not need to call `client.end()`. See Cloudflare's [connection lifecycle](https://developers.cloudflare.com/hyperdrive/concepts/connection-lifecycle/) guidance. Deploy the Worker: ```bash theme={null} npx wrangler deploy ``` > [!NOTE] > > Hyperdrive can cache eligible read queries. Its cache does not automatically > invalidate after an application write. Disable caching for reads that require > read-after-write consistency. ## Apply schema changes > [!NOTE] > > Framework migration commands send DDL through a normal Neki connection. The > router fans that DDL out to the managed shards, but it does not create a > managed schema-change workflow. Use a [schema-change > workflow](/docs/neki/schema-changes) when you want Neki to prepare and coordinate > an Online DDL change. Run migrations with the migration role described in the prerequisites. An application role that inherits only `pg_read_all_data` and `pg_write_all_data` cannot run `CREATE` or `ALTER`, so a migration that uses application credentials fails on its first DDL statement. Apply migration DDL with `psql` and the migration role so the client prints PostgreSQL notices: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f ``` The router that commits the DDL emits a `NOTICE` containing the barrier call for that transaction: ```text theme={null} NOTICE: DDL is committed and the change is visible on this router; other routers may not see it yet. To wait until it is visible on every router, use __neki.wait_for_ddl(42, 7) ``` The schema and cluster versions belong to that transaction, and the notice is the only place Neki reports them. Before you deploy application code that depends on the new schema, run the emitted call verbatim on a primary connection that uses the migration role: ```sql theme={null} SELECT __neki.wait_for_ddl(42, 7); ``` The call returns after every router has applied both versions. A successful migration only confirms that the DDL committed and is visible through the router that ran it; it does not replace this cross-router barrier. Some framework migration commands discard notices. Generate migration SQL with the framework, but apply it with `psql` when the framework cannot preserve the notice. If the notice is lost, its version pair cannot be reconstructed. Run schema migrations outside the Worker through a direct Neki connection. Hyperdrive's transaction pooling does not preserve the per-session state or notices required by the DDL barrier workflow. Hyperdrive does not currently provide a documented way to pass Neki's replica startup option to its origin connections. Keep Hyperdrive traffic on the primary target. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Django application to Neki Source: https://planetscale.com/docs/neki/guides/django Django Postgres backend and replica routing Django connects to Neki through its Postgres database backend. Before you start, you need: * A Neki database with a ready branch. * An application [role](/docs/neki/connecting/roles) for that branch. Use `pg_read_all_data` for read traffic and add `pg_write_all_data` when the application writes rows. These inherited data roles do not grant `CREATE` or `ALTER`, so application credentials cannot change the schema. * A separate migration role, if you run schema changes against the branch. DDL requires the `postgres` inherited role. Neki restricts execution of the cross-router DDL barrier function `__neki.wait_for_ddl` to its `neki_viewer` role, so add `neki_viewer`, which requires `pg_read_all_data`, to the same role. Keep the migration credentials out of the application's runtime configuration. * The **Primary** connection details from the database's **Connect** page. Use the complete generated username. It contains the branch routing information required by Neki. ## Install the Postgres driver Install Psycopg. This example also uses `python-dotenv` for local environment variables: ```bash theme={null} pip install "psycopg[binary]" python-dotenv ``` ## Add the connection details Create a `.env` file with the values from the **Connect** page. Set `DB_SSLROOTCERT` to the path of the system CA bundle: ```bash .env theme={null} DB_HOST= DB_PORT=5432 DB_NAME=postgres DB_USER= DB_PASSWORD= DB_SSLROOTCERT= ``` ## Configure Django Load the environment file and configure the Postgres backend in `settings.py`: ```python settings.py theme={null} import os from dotenv import load_dotenv load_dotenv() DATABASES = { "default": { "ENGINE": "django.db.backends.postgresql", "HOST": os.environ["DB_HOST"], "PORT": os.environ.get("DB_PORT", "5432"), "NAME": os.environ.get("DB_NAME", "postgres"), "USER": os.environ["DB_USER"], "PASSWORD": os.environ["DB_PASSWORD"], "OPTIONS": { "sslmode": "verify-full", "sslrootcert": os.environ["DB_SSLROOTCERT"], }, } } ``` Confirm that Django can connect: ```bash theme={null} python manage.py check --database default ``` > [!NOTE] > > Framework migration commands send DDL through a normal Neki connection. The > router fans that DDL out to the managed shards, but it does not create a > managed schema-change workflow. Use a [schema-change > workflow](/docs/neki/schema-changes) when you want Neki to prepare and coordinate > an Online DDL change. Run migrations with the migration role described in the prerequisites. An application role that inherits only `pg_read_all_data` and `pg_write_all_data` cannot run `CREATE` or `ALTER`, so a migration that uses application credentials fails on its first DDL statement. Apply migration DDL with `psql` and the migration role so the client prints PostgreSQL notices: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f ``` The router that commits the DDL emits a `NOTICE` containing the barrier call for that transaction: ```text theme={null} NOTICE: DDL is committed and the change is visible on this router; other routers may not see it yet. To wait until it is visible on every router, use __neki.wait_for_ddl(42, 7) ``` The schema and cluster versions belong to that transaction, and the notice is the only place Neki reports them. Before you deploy application code that depends on the new schema, run the emitted call verbatim on a primary connection that uses the migration role: ```sql theme={null} SELECT __neki.wait_for_ddl(42, 7); ``` The call returns after every router has applied both versions. A successful migration only confirms that the DDL committed and is visible through the router that ran it; it does not replace this cross-router barrier. Some framework migration commands discard notices. Generate migration SQL with the framework, but apply it with `psql` when the framework cannot preserve the notice. If the notice is lost, its version pair cannot be reconstructed. To send a separate Django connection to replicas, add the Neki startup option described in [Primary and replica routing](/docs/neki/connecting#primary-and-replica-routing) to its `OPTIONS`. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect Drizzle to Neki Source: https://planetscale.com/docs/neki/guides/drizzle Drizzle ORM with the node-postgres driver Drizzle connects to Neki through the `node-postgres` driver. Before you start, you need: * A Neki database with a ready branch. * An application [role](/docs/neki/connecting/roles) for that branch. Use `pg_read_all_data` for read traffic and add `pg_write_all_data` when the application writes rows. These inherited data roles do not grant `CREATE` or `ALTER`, so application credentials cannot change the schema. * A separate migration role, if you run schema changes against the branch. DDL requires the `postgres` inherited role. Neki restricts execution of the cross-router DDL barrier function `__neki.wait_for_ddl` to its `neki_viewer` role, so add `neki_viewer`, which requires `pg_read_all_data`, to the same role. Keep the migration credentials out of the application's runtime configuration. * The **Primary** connection details from the database's **Connect** page. Use the complete generated username. It contains the branch routing information required by Neki. ## Install the dependencies Install Drizzle, `pg`, and `dotenv`: ```bash theme={null} npm install drizzle-orm pg dotenv ``` ## Add the connection string Copy the connection URI from the **Connect** page into `.env`: ```bash .env theme={null} DATABASE_URL='postgresql://:@:5432/postgres?sslmode=verify-full' ``` Use the generated URI so reserved characters in the credentials remain URL encoded. The `node-postgres` driver uses Node.js TLS certificate and hostname verification for `sslmode=verify-full`. Do not add `sslrootcert=system`; `node-postgres` treats its value as a certificate filename. Do not commit the `.env` file. ## Connect and run a query Create a `pg` pool and pass it to Drizzle: ```javascript app.js theme={null} import 'dotenv/config' import { sql } from 'drizzle-orm' import { drizzle } from 'drizzle-orm/node-postgres' import pg from 'pg' const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL, }) const db = drizzle(pool) const result = await db.execute(sql` SELECT current_database() AS database, current_user AS role `) console.log(result.rows[0]) await pool.end() ``` This example uses top-level `await`. Set `"type": "module"` in `package.json`, or wrap the query in an async `main()` function. Run the application: ```bash theme={null} node app.js ``` > [!NOTE] > > Framework migration commands send DDL through a normal Neki connection. The > router fans that DDL out to the managed shards, but it does not create a > managed schema-change workflow. Use a [schema-change > workflow](/docs/neki/schema-changes) when you want Neki to prepare and coordinate > an Online DDL change. Run migrations with the migration role described in the prerequisites. An application role that inherits only `pg_read_all_data` and `pg_write_all_data` cannot run `CREATE` or `ALTER`, so a migration that uses application credentials fails on its first DDL statement. Apply migration DDL with `psql` and the migration role so the client prints PostgreSQL notices: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f ``` The router that commits the DDL emits a `NOTICE` containing the barrier call for that transaction: ```text theme={null} NOTICE: DDL is committed and the change is visible on this router; other routers may not see it yet. To wait until it is visible on every router, use __neki.wait_for_ddl(42, 7) ``` The schema and cluster versions belong to that transaction, and the notice is the only place Neki reports them. Before you deploy application code that depends on the new schema, run the emitted call verbatim on a primary connection that uses the migration role: ```sql theme={null} SELECT __neki.wait_for_ddl(42, 7); ``` The call returns after every router has applied both versions. A successful migration only confirms that the DDL committed and is visible through the router that ran it; it does not replace this cross-router barrier. Some framework migration commands discard notices. Generate migration SQL with the framework, but apply it with `psql` when the framework cannot preserve the notice. If the notice is lost, its version pair cannot be reconstructed. For read-only traffic, add the Neki replica startup option described in [Primary and replica routing](/docs/neki/connecting#primary-and-replica-routing) to the connection URI. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect Kysely to Neki Source: https://planetscale.com/docs/neki/guides/kysely Kysely with the node-postgres driver Kysely connects to Neki through its Postgres dialect and a `pg` connection pool. Before you start, you need: * A Neki database with a ready branch. * An application [role](/docs/neki/connecting/roles) for that branch. Use `pg_read_all_data` for read traffic and add `pg_write_all_data` when the application writes rows. These inherited data roles do not grant `CREATE` or `ALTER`, so application credentials cannot change the schema. * A separate migration role, if you run schema changes against the branch. DDL requires the `postgres` inherited role. Neki restricts execution of the cross-router DDL barrier function `__neki.wait_for_ddl` to its `neki_viewer` role, so add `neki_viewer`, which requires `pg_read_all_data`, to the same role. Keep the migration credentials out of the application's runtime configuration. * The **Primary** connection details from the database's **Connect** page. Use the complete generated username. It contains the branch routing information required by Neki. ## Install the dependencies Install Kysely, `pg`, and `dotenv`. This example uses `tsx` to run TypeScript: ```bash theme={null} npm install kysely pg dotenv npm install --save-dev typescript tsx @types/node @types/pg ``` ## Add the connection string Copy the connection URI from the **Connect** page into `.env`: ```bash .env theme={null} DATABASE_URL='postgresql://:@:5432/postgres?sslmode=verify-full' ``` Use the generated URI so reserved characters in the credentials remain URL encoded. The `node-postgres` driver uses Node.js TLS certificate and hostname verification for `sslmode=verify-full`. Do not add `sslrootcert=system`; `node-postgres` treats its value as a certificate filename. Do not commit the `.env` file. ## Connect and run a query Create a Kysely instance with the Postgres dialect: ```typescript app.ts theme={null} import 'dotenv/config' import { Kysely, PostgresDialect, sql } from 'kysely' import pg from 'pg' interface Database {} const db = new Kysely({ dialect: new PostgresDialect({ pool: new pg.Pool({ connectionString: process.env.DATABASE_URL, }), }), }) const result = await sql<{ database: string; role: string }>` SELECT current_database() AS database, current_user AS role `.execute(db) console.log(result.rows[0]) await db.destroy() ``` Run the application: ```bash theme={null} npx tsx app.ts ``` > [!NOTE] > > Framework migration commands send DDL through a normal Neki connection. The > router fans that DDL out to the managed shards, but it does not create a > managed schema-change workflow. Use a [schema-change > workflow](/docs/neki/schema-changes) when you want Neki to prepare and coordinate > an Online DDL change. Run migrations with the migration role described in the prerequisites. An application role that inherits only `pg_read_all_data` and `pg_write_all_data` cannot run `CREATE` or `ALTER`, so a migration that uses application credentials fails on its first DDL statement. Apply migration DDL with `psql` and the migration role so the client prints PostgreSQL notices: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f ``` The router that commits the DDL emits a `NOTICE` containing the barrier call for that transaction: ```text theme={null} NOTICE: DDL is committed and the change is visible on this router; other routers may not see it yet. To wait until it is visible on every router, use __neki.wait_for_ddl(42, 7) ``` The schema and cluster versions belong to that transaction, and the notice is the only place Neki reports them. Before you deploy application code that depends on the new schema, run the emitted call verbatim on a primary connection that uses the migration role: ```sql theme={null} SELECT __neki.wait_for_ddl(42, 7); ``` The call returns after every router has applied both versions. A successful migration only confirms that the DDL committed and is visible through the router that ran it; it does not replace this cross-router barrier. Some framework migration commands discard notices. Generate migration SQL with the framework, but apply it with `psql` when the framework cannot preserve the notice. If the notice is lost, its version pair cannot be reconstructed. For read-only traffic, add the Neki replica startup option described in [Primary and replica routing](/docs/neki/connecting#primary-and-replica-routing) to the connection URI. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Laravel application to Neki Source: https://planetscale.com/docs/neki/guides/laravel Laravel Postgres connections and replica routing Laravel connects to Neki through its Postgres database driver. Eloquent and the query builder work through the same connection. Before you start, you need: * A Neki database with a ready branch. * An application [role](/docs/neki/connecting/roles) for that branch. Use `pg_read_all_data` for read traffic and add `pg_write_all_data` when the application writes rows. These inherited data roles do not grant `CREATE` or `ALTER`, so application credentials cannot change the schema. * A separate migration role, if you run schema changes against the branch. DDL requires the `postgres` inherited role. Neki restricts execution of the cross-router DDL barrier function `__neki.wait_for_ddl` to its `neki_viewer` role, so add `neki_viewer`, which requires `pg_read_all_data`, to the same role. Keep the migration credentials out of the application's runtime configuration. * The **Primary** connection details from the database's **Connect** page. Use the complete generated username. It contains the branch routing information required by Neki. ## Enable the Postgres PDO driver Laravel's Postgres connector requires PHP's `pdo_pgsql` extension. Install or enable the extension for every environment that runs the application, then confirm that PHP loaded it: ```bash theme={null} php --ri pdo_pgsql ``` ## Add the connection details Add the values from the **Connect** page to `.env`. Set `DB_SSLROOTCERT` to the path of the system CA bundle: ```bash .env theme={null} DB_CONNECTION=pgsql DB_HOST= DB_PORT=5432 DB_DATABASE=postgres DB_USERNAME= DB_PASSWORD= DB_SSLMODE=verify-full DB_SSLROOTCERT= ``` Do not commit this file. Set the same values through your deployment platform's secret manager in production. ## Configure the Postgres connection Make sure the `pgsql` entry in `config/database.php` reads the TLS settings: ```php config/database.php theme={null} 'pgsql' => [ 'driver' => 'pgsql', 'host' => env('DB_HOST'), 'port' => env('DB_PORT', '5432'), 'database' => env('DB_DATABASE', 'postgres'), 'username' => env('DB_USERNAME'), 'password' => env('DB_PASSWORD'), 'charset' => 'utf8', 'prefix' => '', 'prefix_indexes' => true, 'search_path' => 'public', 'sslmode' => env('DB_SSLMODE', 'verify-full'), 'sslrootcert' => env('DB_SSLROOTCERT'), ], ``` Confirm the connection with Tinker: ```bash theme={null} php artisan tinker ``` ```php theme={null} DB::select('SELECT current_database() AS database'); ``` > [!NOTE] > > Framework migration commands send DDL through a normal Neki connection. The > router fans that DDL out to the managed shards, but it does not create a > managed schema-change workflow. Use a [schema-change > workflow](/docs/neki/schema-changes) when you want Neki to prepare and coordinate > an Online DDL change. Run migrations with the migration role described in the prerequisites. An application role that inherits only `pg_read_all_data` and `pg_write_all_data` cannot run `CREATE` or `ALTER`, so a migration that uses application credentials fails on its first DDL statement. Apply migration DDL with `psql` and the migration role so the client prints PostgreSQL notices: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f ``` The router that commits the DDL emits a `NOTICE` containing the barrier call for that transaction: ```text theme={null} NOTICE: DDL is committed and the change is visible on this router; other routers may not see it yet. To wait until it is visible on every router, use __neki.wait_for_ddl(42, 7) ``` The schema and cluster versions belong to that transaction, and the notice is the only place Neki reports them. Before you deploy application code that depends on the new schema, run the emitted call verbatim on a primary connection that uses the migration role: ```sql theme={null} SELECT __neki.wait_for_ddl(42, 7); ``` The call returns after every router has applied both versions. A successful migration only confirms that the DDL committed and is visible through the router that ran it; it does not replace this cross-router barrier. Some framework migration commands discard notices. Generate migration SQL with the framework, but apply it with `psql` when the framework cannot preserve the notice. If the notice is lost, its version pair cannot be reconstructed. ## Route a Laravel process to replicas Laravel 12's `PostgresConnector` does not add arbitrary libpq startup parameters from `config/database.php` to its PDO DSN. Set `PGOPTIONS` in the process environment before PHP starts so `pdo_pgsql` sends the Neki target when it opens each connection: ```bash theme={null} PGOPTIONS='-c __neki.target=REPLICA' php artisan tinker ``` Confirm the target in Tinker: ```php theme={null} DB::scalar('SHOW __neki.target'); // "REPLICA" ``` In production, set the same `PGOPTIONS` value on a dedicated read-only worker or service. The setting applies to every `pdo_pgsql` connection created by that PHP process, so do not set it on a process that also handles writes. Replica-routed connections reject inserts, updates, and deletes, and reads can return stale data. See [Primary and replica routing](/docs/neki/connecting#primary-and-replica-routing). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Node.js application to Neki Source: https://planetscale.com/docs/neki/guides/node node-postgres connections and replica routing Neki uses the Postgres wire protocol, so a Node.js application can connect with [`pg`](https://node-postgres.com/). Before you start, you need: * A Neki database with a ready branch. * An application [role](/docs/neki/connecting/roles) for that branch. Use `pg_read_all_data` for read traffic and add `pg_write_all_data` when the application writes rows. These inherited data roles do not grant `CREATE` or `ALTER`, so application credentials cannot change the schema. * A separate migration role, if you run schema changes against the branch. DDL requires the `postgres` inherited role. Neki restricts execution of the cross-router DDL barrier function `__neki.wait_for_ddl` to its `neki_viewer` role, so add `neki_viewer`, which requires `pg_read_all_data`, to the same role. Keep the migration credentials out of the application's runtime configuration. * The **Primary** connection details from the database's **Connect** page. Use the complete generated username. It contains the branch routing information required by Neki. ## Install the Postgres client Install `pg`. This example also uses `dotenv` to load local environment variables: ```bash theme={null} npm install pg dotenv ``` ## Add the connection string Copy the **Primary** connection URI from the **Connect** page into `.env`: ```bash .env theme={null} DATABASE_URL='postgresql://:@:5432/postgres?sslmode=verify-full' ``` Use the generated URI so reserved characters in the credentials remain URL encoded. Do not commit the `.env` file. Set the same value through your deployment platform's secret manager in production. ## Connect and run a query Create a client from `DATABASE_URL`. The `node-postgres` connection-string parser maps `sslmode=verify-full` to Node.js TLS certificate and hostname verification. Do not add `sslrootcert=system`; `node-postgres` treats its value as a certificate filename. ```javascript index.js theme={null} require('dotenv').config() const { Client } = require('pg') const client = new Client({ connectionString: process.env.DATABASE_URL, }) async function main() { await client.connect() const result = await client.query( 'SELECT current_database() AS database, current_user AS role, NOW() AS time' ) console.log(result.rows[0]) await client.end() } main().catch((error) => { console.error(error) process.exit(1) }) ``` Run the application: ```bash theme={null} node index.js ``` The query travels through a Neki router, which plans it and sends it to the appropriate shard. ## Apply schema changes > [!NOTE] > > Framework migration commands send DDL through a normal Neki connection. The > router fans that DDL out to the managed shards, but it does not create a > managed schema-change workflow. Use a [schema-change > workflow](/docs/neki/schema-changes) when you want Neki to prepare and coordinate > an Online DDL change. Run migrations with the migration role described in the prerequisites. An application role that inherits only `pg_read_all_data` and `pg_write_all_data` cannot run `CREATE` or `ALTER`, so a migration that uses application credentials fails on its first DDL statement. Apply migration DDL with `psql` and the migration role so the client prints PostgreSQL notices: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f ``` The router that commits the DDL emits a `NOTICE` containing the barrier call for that transaction: ```text theme={null} NOTICE: DDL is committed and the change is visible on this router; other routers may not see it yet. To wait until it is visible on every router, use __neki.wait_for_ddl(42, 7) ``` The schema and cluster versions belong to that transaction, and the notice is the only place Neki reports them. Before you deploy application code that depends on the new schema, run the emitted call verbatim on a primary connection that uses the migration role: ```sql theme={null} SELECT __neki.wait_for_ddl(42, 7); ``` The call returns after every router has applied both versions. A successful migration only confirms that the DDL committed and is visible through the router that ran it; it does not replace this cross-router barrier. Some framework migration commands discard notices. Generate migration SQL with the framework, but apply it with `psql` when the framework cannot preserve the notice. If the notice is lost, its version pair cannot be reconstructed. For read-only traffic, see [Primary and replica routing](/docs/neki/connecting#primary-and-replica-routing). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect Prisma to Neki Source: https://planetscale.com/docs/neki/guides/prisma Prisma ORM 7 with the Postgres driver adapter Prisma ORM 7 connects to Neki through the `@prisma/adapter-pg` driver adapter. Before you start, you need: * A Neki database with a ready branch. * An application [role](/docs/neki/connecting/roles) for that branch. Use `pg_read_all_data` for read traffic and add `pg_write_all_data` when the application writes rows. These inherited data roles do not grant `CREATE` or `ALTER`, so application credentials cannot change the schema. * A separate migration role, if you run schema changes against the branch. DDL requires the `postgres` inherited role. Neki restricts execution of the cross-router DDL barrier function `__neki.wait_for_ddl` to its `neki_viewer` role, so add `neki_viewer`, which requires `pg_read_all_data`, to the same role. Keep the migration credentials out of the application's runtime configuration. * The **Primary** connection details from the database's **Connect** page. Use the complete generated username. It contains the branch routing information required by Neki. ## Install and initialize Prisma Install Prisma, the Postgres adapter, and `pg`, then initialize Prisma: ```bash theme={null} npm install prisma @prisma/client @prisma/adapter-pg pg dotenv npx prisma init ``` ## Add the connection strings Set `DATABASE_URL` to the application role and `MIGRATION_DATABASE_URL` to the migration role. Copy both URIs from the **Connect** page: ```bash .env theme={null} DATABASE_URL='postgresql://:@:5432/postgres?sslmode=verify-full' MIGRATION_DATABASE_URL='postgresql://:@:5432/postgres?sslmode=verify-full' ``` If the username or password contains reserved URL characters, use the encoded value from the generated URI instead of inserting the raw value. ## Configure Prisma Use the Postgres provider in `prisma/schema.prisma`: ```prisma prisma/schema.prisma theme={null} generator client { provider = "prisma-client" output = "../generated/prisma" } datasource db { provider = "postgresql" } ``` Configure the Prisma CLI to use the migration credentials: ```typescript prisma.config.ts theme={null} import 'dotenv/config' import { defineConfig, env } from 'prisma/config' export default defineConfig({ schema: 'prisma/schema.prisma', migrations: { path: 'prisma/migrations', }, datasource: { url: env('MIGRATION_DATABASE_URL'), }, }) ``` ## Create the client Prisma ORM 7 requires a driver adapter at runtime: ```typescript src/db.ts theme={null} import 'dotenv/config' import { PrismaPg } from '@prisma/adapter-pg' import { PrismaClient } from '../generated/prisma/client' const adapter = new PrismaPg({ connectionString: process.env.DATABASE_URL, }) export const prisma = new PrismaClient({ adapter }) ``` ## Apply schema changes > [!NOTE] > > Framework migration commands send DDL through a normal Neki connection. The > router fans that DDL out to the managed shards, but it does not create a > managed schema-change workflow. Use a [schema-change > workflow](/docs/neki/schema-changes) when you want Neki to prepare and coordinate > an Online DDL change. Run migrations with the migration role described in the prerequisites. An application role that inherits only `pg_read_all_data` and `pg_write_all_data` cannot run `CREATE` or `ALTER`, so a migration that uses application credentials fails on its first DDL statement. Apply migration DDL with `psql` and the migration role so the client prints PostgreSQL notices: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f ``` The router that commits the DDL emits a `NOTICE` containing the barrier call for that transaction: ```text theme={null} NOTICE: DDL is committed and the change is visible on this router; other routers may not see it yet. To wait until it is visible on every router, use __neki.wait_for_ddl(42, 7) ``` The schema and cluster versions belong to that transaction, and the notice is the only place Neki reports them. Before you deploy application code that depends on the new schema, run the emitted call verbatim on a primary connection that uses the migration role: ```sql theme={null} SELECT __neki.wait_for_ddl(42, 7); ``` The call returns after every router has applied both versions. A successful migration only confirms that the DDL committed and is visible through the router that ran it; it does not replace this cross-router barrier. Some framework migration commands discard notices. Generate migration SQL with the framework, but apply it with `psql` when the framework cannot preserve the notice. If the notice is lost, its version pair cannot be reconstructed. Prisma Migrate does not expose the PostgreSQL notice that contains Neki's DDL barrier. Create migration files against a local Postgres database, not against the Neki branch: ```bash theme={null} MIGRATION_DATABASE_URL="$LOCAL_DATABASE_URL" npx prisma migrate dev --create-only ``` Apply the generated SQL with `psql`, run the barrier call printed in the notice, then record the migration in Prisma's history: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f prisma/migrations//migration.sql npx prisma migrate resolve --applied ``` Do not run `prisma migrate dev` or `prisma migrate deploy` against Neki because those commands do not preserve the DDL barrier notice. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Ruby on Rails application to Neki Source: https://planetscale.com/docs/neki/guides/rails Active Record Postgres connections and replica routing Rails connects to Neki through Active Record's Postgres adapter. Before you start, you need: * A Neki database with a ready branch. * An application [role](/docs/neki/connecting/roles) for that branch. Use `pg_read_all_data` for read traffic and add `pg_write_all_data` when the application writes rows. These inherited data roles do not grant `CREATE` or `ALTER`, so application credentials cannot change the schema. * A separate migration role, if you run schema changes against the branch. DDL requires the `postgres` inherited role. Neki restricts execution of the cross-router DDL barrier function `__neki.wait_for_ddl` to its `neki_viewer` role, so add `neki_viewer`, which requires `pg_read_all_data`, to the same role. Keep the migration credentials out of the application's runtime configuration. * The **Primary** connection details from the database's **Connect** page. Use the complete generated username. It contains the branch routing information required by Neki. ## Install the Postgres adapter Add `pg` to the application's Gemfile: ```ruby Gemfile theme={null} gem "pg" ``` Install the dependency: ```bash theme={null} bundle install ``` ## Store the connection details Open the encrypted credentials for the target environment: ```bash theme={null} bin/rails credentials:edit --environment production ``` Add the values from the **Connect** page. Set `sslrootcert` to the path of the system CA bundle. Common paths include `/etc/ssl/certs/ca-certificates.crt` on Debian and Ubuntu, `/etc/pki/tls/certs/ca-bundle.crt` on RHEL, and `/etc/ssl/cert.pem` on macOS: ```yaml config/credentials/production.yml.enc theme={null} planetscale: host: port: 5432 database: postgres username: password: sslrootcert: ``` > [!NOTE] > > Environment-specific credentials replace the global credentials for that > environment. Keep `secret_key_base` in the production credentials, or set > `SECRET_KEY_BASE` where the application runs, so Rails can start in > production. For local development, use > `bin/rails credentials:edit --environment development` and a matching > `development` entry in `config/database.yml`. ## Configure Active Record Add the production connection to `config/database.yml`: ```yaml config/database.yml theme={null} production: primary: &primary <<: *default adapter: postgresql host: <%= Rails.application.credentials.planetscale.fetch(:host) %> port: <%= Rails.application.credentials.planetscale.fetch(:port) %> database: <%= Rails.application.credentials.planetscale.fetch(:database) %> username: <%= Rails.application.credentials.planetscale.fetch(:username) %> password: <%= Rails.application.credentials.planetscale.fetch(:password) %> sslmode: verify-full sslrootcert: <%= Rails.application.credentials.planetscale.fetch(:sslrootcert) %> pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5) %> primary_replica: <<: *primary options: "-c __neki.target=REPLICA" replica: true ``` Confirm the connection: ```bash theme={null} RAILS_ENV=production bin/rails runner \ 'puts ActiveRecord::Base.connection.select_value("SELECT current_database()")' ``` > [!NOTE] > > Framework migration commands send DDL through a normal Neki connection. The > router fans that DDL out to the managed shards, but it does not create a > managed schema-change workflow. Use a [schema-change > workflow](/docs/neki/schema-changes) when you want Neki to prepare and coordinate > an Online DDL change. Run migrations with the migration role described in the prerequisites. An application role that inherits only `pg_read_all_data` and `pg_write_all_data` cannot run `CREATE` or `ALTER`, so a migration that uses application credentials fails on its first DDL statement. Apply migration DDL with `psql` and the migration role so the client prints PostgreSQL notices: ```bash theme={null} psql "$MIGRATION_DATABASE_URL" -f ``` The router that commits the DDL emits a `NOTICE` containing the barrier call for that transaction: ```text theme={null} NOTICE: DDL is committed and the change is visible on this router; other routers may not see it yet. To wait until it is visible on every router, use __neki.wait_for_ddl(42, 7) ``` The schema and cluster versions belong to that transaction, and the notice is the only place Neki reports them. Before you deploy application code that depends on the new schema, run the emitted call verbatim on a primary connection that uses the migration role: ```sql theme={null} SELECT __neki.wait_for_ddl(42, 7); ``` The call returns after every router has applied both versions. A successful migration only confirms that the DDL committed and is visible through the router that ran it; it does not replace this cross-router barrier. Some framework migration commands discard notices. Generate migration SQL with the framework, but apply it with `psql` when the framework cannot preserve the notice. If the notice is lost, its version pair cannot be reconstructed. ## Route Active Record reads to replicas The `primary_replica` entry connects to the same Neki endpoint and database, but the libpq `options` parameter sets the target when the connection starts. `replica: true` prevents Rails database tasks, including migrations, from running through the reader. Map Active Record's writing and reading roles in `ApplicationRecord`: ```ruby app/models/application_record.rb theme={null} class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end ``` Confirm that the reading role uses replicas: ```bash theme={null} RAILS_ENV=production bin/rails runner \ 'ActiveRecord::Base.connected_to(role: :reading) { puts ActiveRecord::Base.connection.select_value("SHOW __neki.target") }' ``` The command prints `REPLICA`. Replica reads can return stale data, so keep reads that require the latest committed data on the writing role. See [Primary and replica routing](/docs/neki/connecting#primary-and-replica-routing). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Import a Postgres database Source: https://planetscale.com/docs/neki/imports/postgres Import Postgres into Neki with pg_dump and pg_restore Import Postgres into a new, unsharded Neki database: 1. Stop source writes and create a custom-format dump with `pg_dump`. 2. Restore into a fresh Neki database with `pg_restore`. 3. Validate the imported database and move application traffic to Neki. The import is offline. The source application must stop writing before the dump and remain stopped until import validation and cutover are complete. > [!NOTE] > > Automated Neki imports are not available during Platform Preview. This guide > describes the manual dump-and-restore path. To have PlanetScale help plan and > run the migration, email [migrations@planetscale.com](mailto:migrations@planetscale.com). > [!WARNING] > > Restore into an unsharded Neki database. Although Neki supports some > sharded and reference-table `COPY FROM STDIN` operations, `pg_restore` is not > a topology-aware sharded import workflow. See [COPY > limits](/docs/neki/platform-preview-limitations#copy-limits). ## Assess the source Before scheduling the migration, record: * The Postgres version and total database size. * The largest tables and indexes and the expected growth during the migration. * Primary keys, foreign keys, checks, unique constraints, and indexes. * Identity and serial columns, owned sequences, enums, JSONB values, timestamps, views, and quoted identifiers. * Schemas, roles, ownership, grants, and default privileges. * Installed extensions and their versions. * The time for which application and background writes can remain stopped. Inventory extensions on the source before creating the dump: ```sql theme={null} SELECT extname, extversion FROM pg_extension WHERE extname <> 'plpgsql' ORDER BY extname; ``` Compare every result with the target configuration profile's **Extensions** tab. The dashboard catalog is the authoritative list of extensions supported for that profile. Confirm any required profile-level enablement and `CREATE EXTENSION` workflow before the migration. An unsupported extension statement can cause `pg_restore --exit-on-error` to stop before loading data. Time a representative dump, restore, and validation run with a similar data volume so that the maintenance window includes more than data transfer. ## Prepare the target 1. Create the Neki database and wait for its default branch to become ready. 2. Keep the initial one-shard, unsharded data topology for the import. 3. Create the application roles needed on the target and test their connections. 4. Complete any required profile-level extension enablement and decide whether `pg_restore` or a separate administrative step will install each extension's database objects. See [Postgres extensions](/docs/neki/extensions). 5. Confirm that the target is empty and that no application is writing to it. ## Stop writes and create the dump Put the source application into maintenance or read-only mode. Verify that background workers, scheduled jobs, and other clients cannot write, then keep source writes stopped through the remaining steps. The recommended path uses a custom-format dump without source ownership or access-control commands: ```bash theme={null} pg_dump \ --format=custom \ --no-owner \ --no-privileges \ --verbose \ --file=source.dump \ "$SOURCE_DATABASE_URL" ``` Keep the source database and dump available until the rollback window closes. ## Restore into Neki Restore with the target branch's generated connection string: ```bash theme={null} pg_restore \ --dbname="$NEKI_DATABASE_URL" \ --no-owner \ --no-privileges \ --exit-on-error \ --verbose \ --jobs=4 \ source.dump ``` Use a custom-format dump with `pg_restore` for the recommended import path. The example runs four restore jobs in parallel. Omit `--jobs=4` to run the restore serially. If you created a plain-text SQL dump instead, restore it with `psql`. Review the complete verbose output before continuing. After the restore, collect statistics for query planning: ```sql theme={null} ANALYZE; ``` ## Validate the target Keep both databases unavailable for writes while you compare: * Expected schemas, tables, views, indexes, and constraints. * Exact row counts for critical tables and aggregate counts for every table. * Checksums or deterministic fingerprints for critical business data. * Representative application reads, writes, transactions, and query plans. * Inserts that exercise identity and sequence-backed columns. * Role permissions and TLS connections from the production runtime. If any row count or checksum differs, do not cut over. ## Cut over 1. Confirm that source writes are still stopped and validation is complete. 2. Update application secrets to use the Neki branch connection string. 3. Start a small number of application instances and run smoke tests. 4. Increase traffic while watching router errors, query latency, connections, CPU, storage, and Postgres errors. 5. Keep the old source read-only until the rollback boundary has passed. Avoid schema or topology changes during cutover. Establish a stable Neki baseline before planning later changes. ## Roll back For this offline path, rollback means stopping writes to Neki and reconnecting the application to the unchanged source. Writes accepted by Neki after cutover are not copied back automatically. Decide before cutover whether those writes can be discarded or require a separate reconciliation procedure. See [Migration troubleshooting](/docs/neki/imports/troubleshooting) for common dump, restore, and validation failures. > [!NOTE] > > An imported database can stay unsharded. To shard tables that already have > rows, see [Data migration](/docs/neki/data-migration#reshard-existing-tables). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Troubleshoot a Neki migration Source: https://planetscale.com/docs/neki/imports/troubleshooting Resolve discovery, topology-planning, data-movement, validation, and cutover problems. Preserve the exact failing command, Postgres error, object name, and restore log before retrying. A retry against a partially restored target can create a different failure; use a fresh empty target unless the tool and recovery plan explicitly support resumption. ## `NK013` for an unsupported `COPY` form Compare the statement with the implemented [COPY limits](/docs/neki/platform-preview-limitations#copy-limits). In particular, `COPY TO` and file-based `COPY` do not support sharded or reference tables. `COPY FROM STDIN` supports both, subject to its shard-key, option, and default-value requirements. `COPY FROM STDIN` against a sharded table is also rejected while that table has an enabled global secondary index (GSI). COPY does not maintain the GSI lookup rows, so allowing it would make indexed reads incomplete. Load the rows before enabling the GSI, then backfill and verify the lookup table before cutover, or use supported `INSERT` statements that maintain the index. For `pg_restore`, restore into an unsharded Neki database. Moving the imported rows into a sharded layout is a separate [data migration workflow](/docs/neki/data-migration). ## An extension cannot be created Preserve the extension name and the first `pg_restore` error. Compare the source extension inventory from `pg_extension` with the target configuration profile's **Extensions** tab. An extension reported by `pg_available_extensions` but absent from the dashboard catalog is not supported for customer use on Neki. If the extension is supported, complete any required profile-level enablement and wait for the configuration change before retrying on a fresh target. If it is not supported, stop the restore and determine whether the dependent schema objects and data can be migrated without it. Do not continue after `--exit-on-error` stops the restore. ## Owner, role, or privilege errors Use `--no-owner --no-privileges` for the dump-and-restore path. Create application roles through PlanetScale, then apply only the grants those roles need after the restore. Do not create or depend on source superuser roles. ## A retry fails against a partially restored target Create a fresh Neki target and repeat the restore. Do not use a cleanup script that drops and recreates the `public` schema; doing so can remove Neki-managed objects required by the cluster. ## The target shard group includes the source shard MoveTables and Reshard copy onto a disjoint placement. The target group's shards cannot include a source shard. Create the destination shards required by the planned key ranges and capacity, put only those shard UIDs in the target group, and leave the import shard as the source and [authoritative shard group](/docs/neki/data-topology). See [Data migration](/docs/neki/data-migration#what-a-migration-workflow-moves) for the placement requirements. ## Reshard refuses undeclared tables Reshard requires a table entry in the data topology for every physical base table that resolves to the source group, including tables that inherit that group from a database, schema, or cluster default. Declare those tables, or move tables that should stay unsharded onto the authoritative group, before creating the workflow. ## `NK016` reports shared dependencies An object used by a moving table is also used by a table that will remain on the source placement. Shared objects can include sequences, functions used by column defaults or checks, trigger functions, and functions used by expression indexes. The error lists the conflicting object IDs. Move all tables that use the shared objects together, or change the schema so the moving and non-moving tables no longer share them. Inventory dependent views separately because neither MoveTables nor Reshard automatically repoints them. ## The workflow differ exhausts source connections Do not switch traffic. Stop unrelated administrative work and check the source profile's connection usage before retrying the differ. Copy streams, change streaming, and differ workers use managed connections in addition to application and monitoring sessions. Do not retry until the source can provide reliable headroom for the workflow. ## The comparison does not finish or reports shard errors Do not switch traffic. Results from an unfinished comparison are partial. A table that never started may be absent from the report entirely, and a zero-row result does not prove that an unfinished table is empty. Resolve the reported shard errors, then run the comparison again. Approve cutover only after every expected table finishes on every target shard and the report finds no missing, extra, or mismatched rows. ## A dependent view returns stale rows after Reshard or MoveTables The view can still resolve to the retained source copy of a table after the table's traffic moves. Recreate or update the view for the new placement, then compare its result with a direct query against the moved table before completing the workflow or removing source rows. ## Row counts or checksums differ Do not cut over. Confirm that source writes were stopped before the dump and that every restore error was reviewed. Compare by primary-key ranges to locate the first mismatch, then determine whether the dump, restore, unsupported object, or application validation query caused it. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Monitor a Neki database Source: https://planetscale.com/docs/neki/monitoring Use metrics, logs, and Query Insights to monitor Neki routers, shards, Postgres instances, and queries. Monitor Neki routers, the Postgres instances within each shard, and query patterns across the database. PlanetScale provides these complementary monitoring tools: | Tool | Use it to | | :- | :- | | [Metrics](/docs/neki/monitoring/metrics) | Track router traffic and latency, Postgres resource utilization, storage, WAL activity, replication lag, and instance health. | | [Logs](/docs/neki/monitoring/logs) | Search individual log events and filter them by shard, server, severity, and time. | | [Connections](/docs/neki/monitoring/connections) | See live client sessions on your routers, and cancel a query or terminate a session. | | [Query Insights](/docs/neki/monitoring/query-insights) | Find expensive or frequently executed query patterns and inspect their shard-level cost. | | [Anomalies](/docs/neki/monitoring/anomalies) | Investigate periods when queries run slower than their established baseline. | | [Schema recommendations](/docs/neki/monitoring/schema-recommendations) | Review automatic DDL suggestions from production query telemetry and schema. | ## Router monitoring Applications connect to Neki routers rather than directly to individual shards. Routers plan each statement, determine which shards need to participate, and combine results when a query runs on more than one shard. Router metrics include: * Queries per second. * Query latency, including per-database detail. * Query errors per second. * Router CPU and memory utilization. * Container restarts, out-of-memory restarts, and pod status. Use the **Routers** tab to view all router groups or select one named group. ## Shard and Postgres monitoring Each Neki shard contains a primary Postgres instance and can contain one or more replicas. The **Shards** and **Storage** metrics tabs can limit graphs to one [configuration profile](/docs/neki/cluster-configuration#configuration-profiles). The Logs page lets you select shards and then narrow the results to servers within those shards. Postgres monitoring includes: * CPU, memory, IOPS, and storage utilization. * Connections, transaction rate, and locks. * WAL storage and archival activity. * Replication lag between primaries and replicas. * Out-of-memory events, container restarts, and instance health. Use shard and server filters together when investigating whether a problem is isolated to one shard, affects a configuration profile, or appears throughout the database. ## Query monitoring Query Insights groups executions into query patterns and shows their behavior over time. Use it to investigate: * Query latency and execution count. * Rows read and rows written. * Errors and [anomalous performance](/docs/neki/monitoring/anomalies). * Query tags. * Shard calls made by each query. * Parallel worker activity. * [Schema recommendations](/docs/neki/monitoring/schema-recommendations). For Neki queries, **Qualified table** shows the complete resolved table name, including its database and schema. For example, if a query connected to `appdb` refers to `orders` and Postgres resolves it from the `sales` schema, the qualified table is `appdb.sales.orders`. A query can reference more than one qualified table. The **Schema** column is a connection grouping rather than a PostgreSQL schema name; see [Table and schema names in Neki](/docs/neki/monitoring/query-insights#table-and-schema-names-in-neki). A high shard-call count can come from a query that fans out across shards or from repeated dispatches to the same shard. Review the query plan and the database's [data topology](/docs/neki/data-topology) when investigating an unexpected value. ## Investigate a performance problem A typical investigation moves between Query Insights, Metrics, and Logs: 1. Start with Query Insights to identify the affected query pattern and time range. 2. Check Metrics for changes in router latency, shard resource utilization, storage activity, or replication lag during the same period. 3. Search Logs for errors or server events from the affected shards and Postgres instances. 4. Compare the results across shards to determine whether the issue is isolated or branch-wide. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Anomalies Source: https://planetscale.com/docs/neki/monitoring/anomalies Periods when a high share of queries run slower than the established baseline. PlanetScale Insights continuously analyzes query performance to establish a baseline. When a high enough percentage of queries run more slowly than that baseline, Insights records an anomaly. Anomalies are available on every Neki cluster. ## Open Anomalies From the [PlanetScale dashboard](https://app.planetscale.com), select a Neki database and branch, then select **Insights** and **Anomalies**. ## Read the Anomalies graph The graph shows the percentage of queries executing slower than the 97.7th (2-sigma) percentile baseline on the y-axis and time on the x-axis. The **expected** line is the share of slow queries that is statistically normal for a database with uniform performance over time. Small deviations are normal. Only substantial, sustained deviations are treated as an anomaly. Periods of unhealthy performance are highlighted. Select an anomaly to open its detail view, which shows when the period started and ended, how long it lasted, and the slow-query percentage before, during, and after it. The detail view then lists signals from the same period: * **Correlated queries**, with the execution rate of each one. * Queries per second, rows written per second, rows read per second, and errors per second for the branch. * Shard-primary CPU utilization, memory utilization, and IOPS. * Backups that were running during the period. These signals are measured over the anomaly's window; none of them is identified as the cause. A correlated query ran while the anomaly was open, which makes it worth investigating, but it can equally be a victim of the slowdown or unrelated to it. Compare the same period on [Metrics](/docs/neki/monitoring/metrics) before deciding what to change. A shard-primary CPU or IOPS spike, rising replica lag, or a router latency increase without matching Postgres load each point at a different explanation. ## Anomalies vs query latency A query-latency spike does not always indicate an anomaly, because Insights compares each query against its own baseline rather than against an absolute threshold. For example, a weekly report that always runs a few slow queries raises branch query latency at the same time each week. Those queries are running at their expected latency, so Insights does not record an anomaly. ## Investigate an anomaly Use the anomaly details to decide whether you need to act. A common case is a new query that runs often, reads many rows, and consumes enough resources to slow other traffic. Open each correlated query in [Query Insights](/docs/neki/monitoring/query-insights) and check its execution count, rows read, and shard calls over the anomaly window to establish whether it is a likely contributor, a victim, or unrelated. If it contributed, consider adding an index, reducing the rows read, or changing the routing predicate. After you change a query or schema, return to Anomalies and Query Insights to confirm the pattern returned to its baseline. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Inspect live Neki connections Source: https://planetscale.com/docs/neki/monitoring/connections See live client sessions on your Neki routers, and cancel a query or terminate a session from the dashboard or the PlanetScale CLI. ## Overview Connections shows the client sessions currently open on a Neki branch's routers. If a session is stuck, you can cancel its query or terminate the session. > [!NOTE] > > Connections only shows router sessions, not the connections between routers and each shard's Postgres instances. PlanetScale's own administrative sessions and non-client backends aren't shown. ## View connections in the dashboard Select a database and branch, then select **Connections**. The list refreshes automatically and shows one row per router session: | Column | Meaning | | :- | :- | | PID | The session's process ID on its router. PIDs are unique per router, not per branch. | | Router group | The router group the session is connected to. | | Availability zone | The zone of the router serving the session. | | State | The session state, for example `active`, `idle`, or `idle in transaction`. | | Duration | Time since the session's transaction started, or its query if it has no open transaction, or the session itself if neither. | | User | The database role the client connected as. | | Application | The client's `application_name`. | | Query | The session's current query, or its most recent one if it's idle. | Select a row to see the session's details, including the shard backends serving it. ## View connections in the CLI `show` prints the current sessions once, and `top` opens a live terminal view: ```bash theme={null} pscale branch connections show pscale branch connections top ``` `show` prints each session with its IDs and shard backends. Add `--format json` for scripts and AI agents. In `top`, press `s` to switch the sort between transaction start and duration, and `enter` to open a session's details. ### Capture and replay Record a `top` session to a trace file and replay it later: ```bash theme={null} # record while you watch pscale branch connections top --capture trace.jsonl # record without the interactive view pscale branch connections top --capture trace.jsonl --duration 30m # replay the trace pscale branch connections top --replay trace.jsonl ``` In the live view, press `C` to start or stop a capture. The capture includes the last few minutes of history that `top` already has in memory. If you didn't pass `--capture`, the trace is written to `connections-.jsonl` in the current directory. ## Cancel a query or terminate a session | Action | Dashboard / `top` | CLI command | What happens | | :- | :- | :- | :- | | **Cancel query** | Cancel query / `c` | `kill --query` | The router cancels the statement the session is running. The connection stays open, and the client receives a canceled-query error (SQLSTATE `57014`). If the session was in a transaction, the transaction is aborted and the client must roll back before running more statements, as in Postgres. | | **Kill transaction** | Terminate transaction / `k` | `kill-transaction ` | Terminates the session, but only if it still has the open transaction you saw. | | **Force terminate** | Terminate connection / `K` | `kill ` | The router cancels anything the session is running, then closes the client's connection. Any open transaction is rolled back, the same as if the client had disconnected. | Before acting, PlanetScale checks that the session is still the one you saw. If the query finished, the transaction ended, or the PID now belongs to a new session, the action is rejected. In the CLI, the action commands take effect immediately with no confirmation prompt. Scripts and AI agents should get IDs from `show --format json`, act on one session at a time, and run `show` again to confirm the result. An agent should get a person's approval before running an action. ## Permissions Connections is available to signed-in users (for the CLI, authenticate with `pscale auth login`) but not when using service tokens. What you can do depends on your role and whether the branch is a production branch. Acting means cancelling a query or terminating a session. | Role | View dev | Act dev | View production | Act production | | :- | :-: | :-: | :-: | :-: | | Organization Member | ✓ | ✓ | | | | Organization Analyst | ✓ | ✓ | ✓ | | | Organization Administrator | ✓ | ✓ | ✓ | ✓ | | Database Administrator (for that database) | ✓ | ✓ | ✓ | ✓ | Database Administrator access can be granted directly or through a [team](/docs/security/teams). If you don't have the required role, the request fails with a permission error. See [Access control](/docs/security/access-control) for the full role model. ## Related documentation > **[connections CLI reference](/docs/cli/connections)** > **[Monitoring overview](/docs/neki/monitoring)** > **[Query Insights](/docs/neki/monitoring/query-insights)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Logs Source: https://planetscale.com/docs/neki/monitoring/logs Search and filter Neki logs by branch, shard, server, level, and time. The Logs page displays log events from the routers and Postgres instances running a Neki database branch. Use its filters and search syntax to find events that occurred during a selected time range. To open the page, select a database and branch in the PlanetScale dashboard, then select **Logs**. ## Filter logs The Logs page provides the following controls: | Control | Behavior | | :- | :- | | **Branch** | Select the database branch whose logs you want to inspect. | | **Shards** | Show events from one or more Neki shards. | | **Servers** | Select individual servers, grouped by role. By default, logs from every shard primary are shown. | | **Levels** | Filter by `DEBUG`, `INFO`, `WARNING`, or `ERROR`. With no specific levels selected, all levels are shown. | | **Time range** | Select a preset or custom range. The custom date picker is limited to the previous seven days, and the default range is the previous hour. | | **Live** | Refresh the results approximately every 10 seconds. Live mode is enabled by default. | | **Refresh** | Request the latest matching events immediately. | | **Download** | Download the results on the current page as CSV or JSON, or copy them as JSON when clipboard access is available. | Selecting shards narrows the server menu to the servers in those shards. Neki routers are represented separately from shards, so selecting fewer than all shards removes routers from the server menu. To select routers, leave the shard filter set to all shards. Use the shard and server filters together when an issue appears to affect one shard or one primary or replica. The Logs page returns up to 100 events per page. Use **Previous** and **Next** to move through additional results. ## Search logs Use the search field to find text or filter structured log fields. Select the question-mark icon beside the search field to open the syntax reference. ### Search for text Enter a word to find messages containing it: ```text theme={null} error ``` Use quotation marks to search for an exact phrase: ```text theme={null} "Received message" ``` Exclude matching messages with `NOT`: ```text theme={null} NOT "Received message" ``` Use `OR` when either expression can match: ```text theme={null} "Received message" OR planetscale.container:postgres ``` ### Search structured fields Log entries contain structured fields that can be searched directly: | Field | Example | | :- | :- | | Pod | `planetscale.pod:hzi-pod` | | Role | `planetscale.role:replica` | | Container | `planetscale.container:postgres` | | Availability zone | `planetscale.availability_zone:us-east-1a` | You can combine text, field filters, level, shard, server, and time controls to narrow an investigation. PlanetScale log search uses [VictoriaLogs LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/). The search help in the dashboard shows the syntax supported by the Logs page. ## Read a log entry The results table shows each event's time, level, and message. Enable or disable **Single line** to control whether each message is displayed on one line in the table. Select an event to open its details panel. Depending on the event, the panel shows: * Timestamp and configured display timezone. * Log level. * Server role, when present. * Pod. * Neki shard. * Container. * Availability Zone. * Complete raw message. The shard field shows the shard identifier and, when available, its customer-facing display name. Use the controls on a structured field to copy its value or add it to the current search. You can also copy the complete raw message and move to the previous or next event without closing the panel. ## Export log results The download menu operates on the results currently loaded on the page: * **Download as CSV** exports timestamp, level, message, role, container, availability zone, and pod fields. * **Download as JSON** exports the same results as structured JSON. * **Copy JSON to clipboard** copies the current results when clipboard access is available. Narrow the shard, server, time, and search filters before exporting if you need a focused set of events. ## Investigate an incident When investigating a performance or availability problem: 1. Match the Logs time range to the affected period in [Metrics](/docs/neki/monitoring/metrics) or [Query Insights](/docs/neki/monitoring/query-insights). 2. Select the affected shards and servers. 3. Start with `ERROR` and `WARNING`, then include the other levels for more context. 4. Open a relevant event and filter by its shard, pod, role, or container. 5. Compare events across instances to determine whether the problem is isolated or branch-wide. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Metrics Source: https://planetscale.com/docs/neki/monitoring/metrics Monitor Neki router traffic, Postgres resource utilization, storage, WAL activity, replication lag, and instance health. The **Metrics** page shows live and historical performance data for a Neki database branch. Its three tabs separate metrics for **Shards**, **Storage**, and **Routers**. To open the page, select a database and branch in the PlanetScale dashboard, then select **Metrics**. ## Filter metrics All three tabs provide a branch selector, time range, **Live** toggle, **Refresh** button, and controls to expand or collapse every graph. Live mode is enabled by default and refreshes the graphs approximately every 30 seconds. The default time range is the previous 12 hours. The custom date picker is limited to the previous seven days. The remaining filters depend on the selected tab: | Tab | Filters | | :- | :- | | **Shards** | **Shard configuration profile** (`All profiles` or one named profile). | | **Storage** | **Shard configuration profile** (`All profiles` or one named profile). | | **Routers** | **Router** (`All routers` or one named router group). | Opening **Metrics** from a shard or Postgres instance on the database overview highlights that shard or server in the graphs. The Metrics page does not have separate shard or server filter menus. For ranges longer than 15 minutes, click and drag across a graph to narrow the selected time range. Moving the pointer across one graph highlights the same timestamp on the other graphs. The menu on an individual graph can save that graph as an image. ## Shard metrics The **Shards** tab covers Postgres resource utilization, database activity, replication lag, and instance health. CPU, memory, IOPS, connections, locks, and pod status each have separate primary and replica graphs so you can compare their behavior. | Graph | What it shows | | :- | :- | | **Primary CPU utilization** | CPU utilization for shard primaries. | | **Replica CPU utilization** | CPU utilization for shard replicas. | | **Replication lag** | The time by which each replica trails its shard primary. | | **Primary memory utilization** | Percentage of available memory in use by primaries. | | **Replica memory utilization** | Percentage of available memory in use by replicas. | | **Primary IOPS** | Storage input/output operations per second for primaries. | | **Replica IOPS** | Storage input/output operations per second for replicas. | | **Primary connections** | Primary connections grouped by Postgres connection state. | | **Replica connections** | Replica connections grouped by Postgres connection state. | | **Transaction rate** | Committed transactions per second, grouped by shard. | | **Transaction rate by database** | Committed transactions per second, grouped by logical database. | | **Primary locks** | Locks on primaries grouped by lock mode. | | **Replica locks** | Locks on replicas grouped by lock mode. | | **Container out-of-memory restarts** | Containers restarted after exhausting available memory. | | **Container restarts** | Container restarts grouped by reason. | | **Primary pod status** | The lifecycle state reported by each selected primary. | | **Replica pod status** | The lifecycle state reported by each selected replica. | | **Container waiting reasons** | Reasons that a container is waiting to start or resume. | The main graphs summarize the selected instances. Expanding a graph shows its larger chart and, where available, its per-shard or per-instance breakdown. The expanded CPU graphs also show memory utilization for the same instance, and the expanded memory graphs break memory down into mapped, resident, active cache, and inactive cache bytes. Increasing replication lag means replica reads may return older data. If no replica satisfies the router's configured health and lag requirements, a query targeted to replicas fails instead of running on a primary. See [Choosing where reads run](/docs/neki/shard-targeting-fanout#choosing-where-reads-run). When an out-of-memory event occurs in the selected period, the page also shows a warning banner. Correlate the event with memory, connections, and query activity before changing the configuration-profile size. ## Storage metrics The **Storage** tab separates primary and replica disk use and reports write-ahead log (WAL) archiving health. | Graph | What it shows | | :- | :- | | **Primary disk usage** | Percentage of the primary volume in use. | | **Replica disk usage** | Percentage of replica volumes in use. | | **Primary storage usage** | Storage consumed by primaries in bytes. | | **Replica storage usage** | Storage consumed by replicas in bytes. | | **WAL storage** | Storage currently occupied by WAL. | | **WAL archive success rate** | Successful WAL archive operations per second. | | **WAL archive failure rate** | Failed WAL archive operations per second. | | **WAL archive age** | Time since the last successful WAL archive. | | **Unarchived WAL** | WAL waiting to be archived, measured in bytes. | Expanding either archive-rate graph shows a per-shard breakdown of successful and failed operations together. Investigate a rising archive age, failed archive operations, and unarchived WAL together. Compare them with disk usage and IOPS from the same period. ## Router metrics Routers plan incoming statements, determine which shards need to participate, and combine distributed results. The **Routers** tab shows the following graphs for the selected branch or router group: | Graph | What it shows | | :- | :- | | **Queries per second** | Query rate grouped by logical database. | | **Query latency** | Branch-wide p50 and p95 latency, with average, p50, p95, and p99 detail for each logical database. | | **Query errors per second** | Error rate grouped by logical database. | | **Router utilization** | CPU and memory utilization, with detail for individual router instances. | | **Container out-of-memory restarts** | Router containers restarted after exhausting available memory. | | **Container restarts** | Router container restarts grouped by reason. | | **Pod status** | The lifecycle state reported by each router instance. | | **Container waiting reasons** | Reasons that a router container is waiting to start or resume. | A query latency increase without a corresponding increase in Postgres resource utilization can indicate that you should inspect routing behavior, query fanout, or the work required to combine results. ## Metrics elsewhere in the dashboard The database overview and cluster-configuration pages use recent metrics to summarize router memory, primary CPU and memory, replica lag, and branch or per-shard storage. These summaries provide current infrastructure context. Use the **Metrics** tabs for historical time-series graphs. ## Interpret metrics in context There is no single healthy value for every Neki workload. Establish a baseline for each branch and investigate changes from its normal behavior. * If one shard has higher CPU, IOPS, or storage use than the others, review its routed workload and [data topology](/docs/neki/data-topology). * If router latency rises while one shard is saturated, use [Query Insights](/docs/neki/monitoring/query-insights) to find queries that reach that shard or make many shard calls. * If replica lag increases, compare replica CPU, IOPS, and connection activity on **Shards** with WAL activity on **Storage**. * If multiple instances approach their resource limits, review [cluster sizing](/docs/neki/cluster-configuration/cluster-sizing). * If a graph changes suddenly, use its time range when searching [Logs](/docs/neki/monitoring/logs) and Query Insights. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Prometheus metrics for PlanetScale Neki Source: https://planetscale.com/docs/neki/monitoring/prometheus-metrics-neki PlanetScale Neki exposes a Prometheus-compatible endpoint per branch that lets you scrape metrics for routers, sidecars, and Postgres instances. ## Overview PlanetScale serves a Prometheus-compatible metrics endpoint for each Neki branch. Prometheus finds these endpoints through HTTP service discovery. The same discovery list covers every active branch in your organization, including Neki branches. If you use Datadog, the Datadog agent can scrape the same Prometheus endpoint. ### Connect Prometheus You need Prometheus 3.x, or a Prometheus-compatible engine such as VictoriaMetrics, that supports [HTTP SD](https://prometheus.io/docs/prometheus/latest/http_sd/). 1. Create a [service token](/docs/api/reference/service-tokens) in your organization settings. Save the token ID and the token. 2. Grant the token the `read_metrics_endpoints` permission. 3. Add a scrape configuration. Replace `${ORG}`, `${TOKEN_ID}`, and `${TOKEN}` with your values: ```yaml theme={null} scrape_configs: - job_name: "${ORG}-neki" http_sd_configs: - url: https://api.planetscale.com/v1/organizations/${ORG}/metrics authorization: type: "token" credentials: "${TOKEN_ID}:${TOKEN}" refresh_interval: 10m ``` Each discovered target carries a signature that expires after one day. Keep the refresh interval short so Prometheus always holds a valid signature. Each target returns the metrics of one branch. Prometheus attaches the `planetscale_organization_name`, `planetscale_database_name`, and `planetscale_branch_name` target labels, so you can tell branches apart. Every series also carries `planetscale_database_branch_id`. For managed Prometheus services that do not scrape, run Prometheus yourself and forward the metrics with `remote_write`. ## Metrics PlanetScale Neki emits the following metrics to be scraped. Every metric name starts with `planetscale_`. A histogram appears as three series with the suffixes `_bucket`, `_sum`, and `_count`. Counters and gauges appear under the name shown. ## Router Metrics Routers accept application connections, plan each statement, and send work to shards. These metrics describe the statements that routers handle. | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_neki\_router\_queries\_total** The total number of queries routed, by query type | Counter | cluster, database, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_router, query\_type | | **planetscale\_neki\_router\_query\_duration\_seconds** The distribution of query latency, in seconds, through the router | Histogram | cluster, database, le, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_router | | **planetscale\_neki\_router\_query\_errors\_total** The total number of query errors returned by the router, by error class | Counter | cluster, database, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_router, error\_class | ## Sidecar Pool Metrics Each sidecar keeps a connection pool to its Postgres instance for each database. These metrics describe the pool. Use them to see when clients wait for a connection. | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_neki\_sidecar\_pool\_capacity** The configured maximum number of connections in a sidecar pool | Gauge | cluster, database, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_neki\_sidecar\_pool\_connections\_active** The number of open connections in a sidecar pool, checked out or idle | Gauge | cluster, database, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_neki\_sidecar\_pool\_connections\_in\_use** The number of connections in a sidecar pool that clients currently hold | Gauge | cluster, database, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_neki\_sidecar\_pool\_connections\_available** The number of idle connections in a sidecar pool that a client can take | Gauge | cluster, database, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_neki\_sidecar\_pool\_connections\_waiting** The number of clients waiting for a connection from a sidecar pool | Gauge | cluster, database, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_neki\_sidecar\_pool\_waits\_total** The number of checkouts that waited for a pooled Postgres connection | Counter | cluster, database, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_neki\_sidecar\_pool\_wait\_seconds\_total** The total seconds that checkouts spent waiting for a pooled Postgres connection. Recorded only once a checkout has waited | Counter | cluster, database, planetscale\_cell, planetscale\_cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | ## Postgres Metrics These metrics come from the Postgres instances in each shard. Series from a primary carry `planetscale_role="primary"`. Series from a replica carry `planetscale_role="replica"`. | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_postgres\_connection\_state** The count and state of Postgres connections | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_connection\_state, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_wal\_archiver\_succeeded\_count** The count of successfully archived WALs. Primary only | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_wal\_archiver\_failed\_count** The count of unsuccessfully archived WALs. Primary only | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_wal\_archiver\_last\_age\_succeeded** The age of the last successfully archived WAL. Primary only | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_wal\_archiver\_lag\_bytes** WAL bytes written but not yet archived (current LSN minus last archived LSN). Primary only | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_wal\_size\_bytes** The cumulative disk size of WALs waiting to be archived | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_settings\_max\_connections** The current value of the max\_connections setting | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_settings\_max\_wal\_size\_bytes** The current value of the max\_wal\_size\_bytes setting | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_replica\_lag\_seconds** Replica lag in fine-grained seconds from Postgres. Replicas only | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_locks** The count of current lock modes | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_lock\_mode, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_postgres\_database\_xact\_commit\_total** The total committed transactions on Postgres databases | Counter | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_database\_name, planetscale\_pod, planetscale\_role, planetscale\_shard | ## Backup Metrics | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_backup\_restore\_active** Whether a backup restore is active, tagged with the current phase (1 = active) | Gauge | cluster, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_phase, planetscale\_pod, planetscale\_shard | | **planetscale\_backup\_fetch\_percent** The percentage progress of a backup fetch operation (0-100) | Gauge | cluster, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_shard | ## Infrastructure Metrics These metrics describe the pods that run your branch. Router pods carry `planetscale_router`. Sidecar pods carry `planetscale_shard` and `planetscale_role`. | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_pods\_cpu\_util\_percentages** The CPU utilization percentage of router and sidecar pods | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_router, planetscale\_shard | | **planetscale\_pods\_mem\_util\_percentages** The memory utilization percentage of router and sidecar pods | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_router, planetscale\_shard | | **planetscale\_pods\_iops\_total** The total IOPS (Input/Output Operations Per Second) of sidecar pods | Counter | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_volume\_available\_bytes** The available storage space in bytes on Postgres volumes | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_volume\_capacity\_bytes** The total storage capacity in bytes on Postgres volumes | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_pods\_mem\_rss\_bytes** The RSS memory usage in bytes of sidecar pods | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_pods\_mem\_mmap\_bytes** The memory-mapped file usage in bytes of sidecar pods | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_pods\_mem\_active\_cache\_bytes** The active cache memory in bytes of sidecar pods | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_pods\_mem\_inactive\_cache\_bytes** The inactive cache memory in bytes of sidecar pods | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_shard | | **planetscale\_pods\_container\_status\_restarts\_total** The total container restarts, across all restart reasons. Join **planetscale\_pods\_container\_last\_terminated\_reason** to attribute them | Counter | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_router, planetscale\_shard | | **planetscale\_pods\_container\_last\_terminated\_reason** The reason a container was last terminated (1 = the current reason). Join against **planetscale\_pods\_container\_status\_restarts\_total** to attribute restarts to a reason | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_restart\_reason, planetscale\_router, planetscale\_shard | | **planetscale\_pods\_container\_waiting\_reason** The reason a container is waiting, such as CrashLoopBackOff or ImagePullBackOff | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_router, planetscale\_shard, planetscale\_waiting\_reason | | **planetscale\_pods\_status\_phase** The pod status phase (Running, Pending, Failed, Succeeded, Unknown). Carries the pod's role, so join it to break container restarts down by role | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_phase, planetscale\_pod, planetscale\_role, planetscale\_router, planetscale\_shard | ## Example Queries Replace `your-branch-id` with your branch ID. ### Average router query latency ```promql theme={null} sum by (planetscale_router) ( rate(planetscale_neki_router_query_duration_seconds_sum{planetscale_database_branch_id="your-branch-id"}[5m]) ) / sum by (planetscale_router) ( rate(planetscale_neki_router_query_duration_seconds_count{planetscale_database_branch_id="your-branch-id"}[5m]) ) ``` ### Sidecar pool utilization The fraction of the pool that clients currently hold: ```promql theme={null} planetscale_neki_sidecar_pool_connections_in_use{planetscale_database_branch_id="your-branch-id"} / planetscale_neki_sidecar_pool_capacity{planetscale_database_branch_id="your-branch-id"} ``` ### Average time a checkout waits for a pooled connection ```promql theme={null} rate(planetscale_neki_sidecar_pool_wait_seconds_total{planetscale_database_branch_id="your-branch-id"}[5m]) / rate(planetscale_neki_sidecar_pool_waits_total{planetscale_database_branch_id="your-branch-id"}[5m]) ``` ## Attributing Container Restarts `planetscale_pods_container_status_restarts_total` counts every restart and carries no role or reason label. Join the metric that carries the one you want. To break restarts down by role, join `planetscale_pods_status_phase`: ```promql theme={null} sum by (planetscale_role) ( increase(planetscale_pods_container_status_restarts_total[1h]) * on (planetscale_pod) group_left(planetscale_role) group by (planetscale_pod, planetscale_role) (planetscale_pods_status_phase) ) ``` To break restarts down by reason as well, filter against `planetscale_pods_container_last_terminated_reason` one minute at a time and join the role onto the result: ```promql theme={null} sum by (planetscale_role) ( sum_over_time(( increase(planetscale_pods_container_status_restarts_total[1m]) and on (planetscale_pod, planetscale_container) planetscale_pods_container_last_terminated_reason{planetscale_restart_reason="OOMKilled"} )[1h:1m]) * on (planetscale_pod) group_left(planetscale_role) group by (planetscale_pod, planetscale_role) (planetscale_pods_status_phase) ) ``` The gauge only reports a container's latest termination reason, so the query attributes each minute's restarts to the reason current in that minute, then sums over the hour. Change `1h` to your own window and leave the `1m` alone. A container that restarts twice inside one minute under two different reasons has both restarts attributed to the later one. Router pods carry no role, so their restarts appear in a series without a role label. ## Tag Glossary * **cluster**: The PlanetScale cluster identifier * **planetscale\_cluster**: The name of the Neki cluster that serves the branch * **planetscale\_database\_branch\_id**: The unique identifier for the database branch * **planetscale\_pod**: The Kubernetes pod name * **planetscale\_container**: The container name (for example postgres, walg-daemon, router) * **planetscale\_component**: The Neki component that owns the pod: nkrouter (router), nktablet (sidecar and Postgres), or nkadmin (admin) * **planetscale\_router**: The router group name. Present on router pods * **planetscale\_shard**: The shard identifier. Present on sidecar pods and Postgres metrics * **planetscale\_role**: The role of the sidecar and its Postgres instance (for example primary or replica). Absent on router pods * **planetscale\_cell**: The PlanetScale cell identifier * **planetscale\_phase**: The pod's status phase (Running, Pending, Failed, Succeeded, Unknown), or the phase of a backup restore * **database**: The database that a router session connected to, or the database that a sidecar pool serves * **query\_type**: The type of SQL statement, such as SELECT or INSERT * **error\_class**: The two-character class of the Postgres error code (SQLSTATE) of a failed query * **le**: The upper bound of a histogram bucket (less than or equal to) * **planetscale\_connection\_state**: The state of database connections * **planetscale\_lock\_mode**: The Postgres lock mode * **planetscale\_database\_name**: The Postgres database name * **planetscale\_restart\_reason**: Reason for a container's **most recent** termination (OOMKilled, Error, Completed). Carried by **planetscale\_pods\_container\_last\_terminated\_reason** * **planetscale\_waiting\_reason**: Reason a container is waiting (CrashLoopBackOff, ImagePullBackOff, and others). Carried by **planetscale\_pods\_container\_waiting\_reason** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Query Insights Source: https://planetscale.com/docs/neki/monitoring/query-insights Analyze query latency, resource use, and cross-shard execution in Neki. Query Insights groups executions of the same normalized SQL into query patterns. Use it to find queries that run frequently, consume a large share of database time, read unnecessary rows, fan out across shards, or use more parallel workers than expected. Insights is available on every Neki cluster. From the same tab you can also open [Anomalies](/docs/neki/monitoring/anomalies) and [schema recommendations](/docs/neki/monitoring/schema-recommendations). ## Open Query Insights From the [PlanetScale dashboard](https://app.planetscale.com), select a Neki database and branch, then select **Insights**. The page opens to the last 24 hours. Use the branch selector to choose the branch you want to analyze. ## Explore query activity The graph provides the following views: * **Query latency** shows p50 and p95 latency by default. You can also display p99, p99.9, and maximum latency. * **Queries** shows the query rate over the selected period. * **Rows read** and **Rows written** show row activity over the selected period. * The final tab graphs one additional query metric that you select. Select a day from the previous seven days or use the **Last** menu to view the last 15 minutes, 1 hour, 3 hours, 6 hours, 12 hours, or 24 hours. You can also drag across a graph to examine a smaller time range, change the automatic refresh interval, or save the graph as an image. ## Query patterns Insights replaces literal values with numbered placeholders so executions that differ only by their values can be grouped together. For example, these executions belong to the same pattern: ```sql theme={null} SELECT * FROM orders WHERE customer_id = 17; SELECT * FROM orders WHERE customer_id = 42; ``` The normalized pattern uses a placeholder: ```sql theme={null} SELECT * FROM orders WHERE customer_id = $1; ``` Normalized SQL is not the only thing that separates patterns. A router groups executions by the database the session is connected to, what the statement's names resolved to, and whether the execution ran on a primary or a replica. The same SQL text can therefore appear as more than one row: once for each `search_path` resolution that produced a different set of relations, and again for the replica traffic that ran it. ### Pattern cardinality Each router tracks up to 2,000 patterns per aggregation interval. Additional patterns are combined into an overflow entry, so not every distinct pattern is listed. A branch that runs many distinct statement shapes, such as generated SQL with inlined literals that normalization cannot collapse, reaches this limit most often. The table below the graph summarizes each pattern. Use the **Overview**, **Data**, **Resources**, and **Performance** presets to switch between related columns, or select **Custom** to choose columns individually. Numeric columns can display sparklines for the selected time range. The default **Overview** preset includes the query, percentage of runtime, execution count, total time, p50 and p99 latency, rows read, and the ratio of rows read to rows returned. ### Table and schema names in Neki A Postgres query can use an unqualified table name, such as `orders`, while `search_path` determines the schema that Postgres actually uses. Four columns describe the result of that resolution: | Column | Example | Meaning | | - | - | - | | **Table** | `orders` | The relation name without its database or schema. | | **Qualified table** | `appdb.sales.orders` | The complete resolved relation name, as `database.schema.table`. | | **Table schema** | `appdb.sales` | The database and schema of each resolved relation. | | **Schema** | `appdb.a91f4c02` | The connection grouping the pattern was recorded under. | For example, `SELECT * FROM orders` can resolve to `appdb.sales.orders`. Use **Qualified table** when you need to know which database object Neki queried. If a pattern accesses multiple tables, **Qualified table** and **Table schema** list a value for each of them. **Schema** is not a PostgreSQL schema name on Neki. Its value combines the connected database with an identifier for what the statement's names resolved to, so two rows with the same SQL and different **Schema** values resolved to different relations. Read the actual schema from **Table schema** or **Qualified table**. ### Available query statistics The Neki query table can show the following groups of statistics. A column is shown only when its data is collected for the selected branch. | Group | Statistics | | - | - | | Query identity | Query, Schema, Table schema, Qualified table, Table | | Time | % of runtime, % of CPU time, % of IO time, Count, Total time, CPU time, I/O time, Last run | | Latency | p50 latency, p99 latency, Max latency | | Rows | Rows read, Rows read per query, Rows returned, Rows returned per query, Rows affected, Rows affected per query, Rows read / returned | | Routing | Shard calls per query, Max shard calls per query, Parallel workers per query | | Postgres blocks | Block cache hit ratio, Blocks hit, Blocks read, Blocks dirtied, Blocks written | | Network | Bytes returned, Bytes returned per query, Max bytes returned per query, Bytes received, Bytes received per query, Max bytes received per query | **Shard calls per query** is the average number of shard dispatches per execution of the pattern, and **Max shard calls per query** is the highest number recorded for a single execution. A query scattered to four shards records four shard calls. Several statements produced by a rewrite within one dispatch count as one shard call, and repeated dispatches to the same shard count separately, so these metrics are not a count of distinct shards. Buffered or replayed work and failed scatter executions can make the reported count lower than the work performed. A value greater than one indicates either multi-shard execution or repeated work on a shard. Review the query plan and the database's [data topology](/docs/neki/data-topology) when investigating an unexpected value. **Parallel workers per query** is the average number of Postgres backend processes that executed the statement, counted across every shard it reached. The count includes the leader process, so a statement that ran without parallelism on a single shard reports one, and a statement that reached several shards adds up each shard's processes. Parallel maintenance workers, such as those used by an index build, count the same way. This describes process use inside Postgres and is separate from the number of shard calls. ## Filter query patterns The filter field accepts free text and named terms. Select **SYNTAX** beside the field to see the operators the dashboard supports. ```text theme={null} statement_type:select table:orders table_schema:appdb.sales multishard:true indexed:false query_count:>5000 p50:<25 p99:>250 max_latency:>1000 index:orders_customer_id_idx ``` ### Filter on tables and schemas Four named terms filter on the relations a pattern resolved to. All of them match case-insensitively. | Term | Matches | | - | - | | `table:orders` | Any resolved relation named `orders`, in any database and schema. | | `table:sales.orders` | A resolved relation whose qualified name ends in `sales.orders`. A fully qualified value, such as `table:appdb.sales.orders`, narrows the match further. | | `table_schema:appdb.sales` | Patterns that resolved a relation in the `sales` schema of `appdb`. Neki records this value as `database.schema`, so both halves are required. | | `schema:appdb.a91f4c02` | The connection grouping described in [Table and schema names in Neki](#table-and-schema-names-in-neki). This term requires the whole value, so copy it from the **Schema** column rather than typing a schema name. | `qualified_table:orders` finds patterns that name `orders` with an explicit qualifier in their SQL. It accepts a bare relation name only, so use `table:` when you want to filter by a database or schema prefix. ### Free text A term with no name matches the normalized SQL with a case-insensitive substring comparison. Free text does not search the resolved table metadata, so `appdb.sales.orders` finds only patterns whose SQL text contains that string, not every pattern that resolved to that relation. Use `table:` or `table_schema:` for the resolved relation instead. A named term whose value the term type rejects becomes free text as well. Because `qualified_table:` accepts only a bare relation name, `qualified_table:appdb.sales.orders` searches SQL text rather than table metadata. ### Other filter behavior Latency filter values are in milliseconds. Wrap text in double quotes for an exact string match, and prefix a term with `!` to exclude it. Tag filters are also available when query dimensions are enabled for the branch. ## Query-pattern details Select a query pattern to open its detail page. The page shows the normalized SQL and graphs the selected pattern separately from the rest of the branch. It also provides summary statistics for count, total time, rows read relative to rows returned, p50 latency, p99 latency, and errors. Depending on the statement and the data collected for the branch, the detail page can also show: * Index usage over the selected time range. * Notable executions that were slow, read a large number of rows, or returned an error. * Query tags and their values. * An optional AI-generated summary of the normalized query. See [How PlanetScale uses AI](/docs/how-we-use-ai). ## Anomalies and schema recommendations Insights also surfaces two related views for every Neki cluster: * [Anomalies](/docs/neki/monitoring/anomalies) flags periods when a high share of queries run slower than their established baseline. * [Schema recommendations](/docs/neki/monitoring/schema-recommendations) suggest DDL that can improve performance, reduce storage, or prevent ID exhaustion. From **Insights**, open **Anomalies** or **View recommendations**. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Schema recommendations Source: https://planetscale.com/docs/neki/monitoring/schema-recommendations Automatic DDL suggestions from production query telemetry and schema. Schema recommendations use query-level telemetry, Postgres system tables, and your database schema to suggest DDL that can improve performance, reduce memory and storage, or prevent primary-key exhaustion. Recommendations are available on every Neki cluster. They are also available through the [PlanetScale MCP server](/docs/mcp-server). AI coding agents can evaluate and apply them on a recurring schedule; see [Self-improving database](/docs/self-improving-database). ## Open schema recommendations From the [PlanetScale dashboard](https://app.planetscale.com), select a Neki database and branch, then select **Insights** and **View recommendations**. Each open recommendation includes: * An explanation of the change and its intended benefit, such as lower execution time, reduced storage, or prevention of ID exhaustion. * The schema objects or queries it affects. * DDL that applies the recommendation. Recommendations that depend on database traffic run once per day. Recommendations that depend only on schema run when the default branch schema changes. Insights generates recommendations only for the database's default branch. > [!NOTE] > > Schema recommendations may not match your intended outcomes. Evaluate each > suggestion before applying it. PlanetScale is not liable for actions you take > based on these recommendations. ## Apply a recommendation Schema recommendations include either a DDL statement or an Online DDL script. Apply it through a Neki router using native DDL or a [managed schema-change workflow](/docs/neki/schema-changes), as directed by the recommendation. You can also make the equivalent change in your application or ORM migrations. Evaluate lock behavior, rewrite cost, and downtime before running the statement on a production branch. Application DDL is cluster-wide: Neki sends the change to every managed shard. ## Close a recommendation Insights compares each open recommendation with the current default-branch schema and closes it when the schema shows the work was done: the suggested index exists, or the flagged index or table is gone. A recommendation also closes when other schema changes make it unnecessary. Bloat recommendations behave differently. A rebuild leaves the schema unchanged, so Insights has no signal that it happened. A table or index bloat recommendation stays open after `VACUUM FULL` or `REINDEX INDEX CONCURRENTLY` and has to be closed manually. Once a recommendation is closed, Insights does not suggest it again for that object. If you decide to keep the current schema, close the recommendation. ## Supported schema recommendations Insights can suggest the following changes: * [Adding indexes for inefficient queries](#adding-indexes-for-inefficient-queries) * [Removing redundant indexes](#removing-redundant-indexes) * [Preventing primary key ID exhaustion](#preventing-primary-key-id-exhaustion) * [Dropping unused tables](#dropping-unused-tables) * [Dropping unused indexes](#dropping-unused-indexes) * [Rebuilding bloated tables](#rebuilding-bloated-tables) * [Rebuilding bloated indexes](#rebuilding-bloated-indexes) ### Adding indexes for inefficient queries Indexes often reduce rows read and query time. Insights scans query telemetry daily for expensive patterns, proposes indexes, and estimates the improvement before opening a recommendation. A new index recommendation shows the `CREATE INDEX` DDL, the queries that benefit, and an estimated performance change. Indexes speed up reads and increase write cost, memory use, and table size. After you apply an index, use [Query Insights](/docs/neki/monitoring/query-insights) to confirm the intended queries use it. Index suggestions can use AI tools. See [How PlanetScale uses AI](/docs/how-we-use-ai). You can disable new index suggestions and other LLM-based features in organization settings. ### Removing redundant indexes Unnecessary indexes slow writes and consume memory and storage. Insights scans schema changes for: * Exact duplicate indexes that use the same columns in the same order. * Left-prefix duplicate indexes that match the leading columns of a larger index. Exact duplicates are safe to remove. Left-prefix duplicates are usually safe, but removing one can regress queries that relied on the shorter index. ### Preventing primary key ID exhaustion Sequence-backed primary keys can reach the maximum value of their integer type. Further inserts then fail. Insights checks sequence-owned columns daily. When a column has reached 60% or more of its type maximum, it recommends a larger type, typically `BIGINT`. It can also suggest widening foreign-key columns that join to that primary key. Widening a column rewrites every row, and a routed `ALTER TABLE` would hold an exclusive lock on the table cluster-wide until the rewrite finished. The recommendation therefore shows an [Online DDL](/docs/neki/schema-changes) script instead, which builds the new table beside the original and keeps the live table available for reads and writes. The script has four steps: create the migration, poll until every shard is ready, cut over, then clean up the shadow tables and replication slots. Prefer `BIGINT` primary keys unless you know the table will stay small. ### Dropping unused tables Insights flags tables that are more than four weeks old and have not been queried in the last four weeks. Dropping an unused table can reduce storage and shorten backups. Create a [manual backup](/docs/neki/backups) before dropping a table if you are unsure it can be discarded. Confirm that the application no longer uses the table. If the table should be kept, close the recommendation. Once opened, an unused-table recommendation stays open even if the table is queried later. Check Query Insights before dropping it. ### Dropping unused indexes Insights flags indexes that are more than four weeks old and have not been used in the last four weeks. Removing them can lower write cost and save memory and storage. Dropping an unused index can still affect future queries. Confirm that no workload depends on it. Filter Query Insights with `index:index_name` to check recent use. Once opened, the recommendation stays open even if the index is used later. ### Rebuilding bloated tables Postgres MVCC leaves dead tuples after updates and deletes. High table bloat wastes disk, slows queries, and lengthens backups and DDL. Insights estimates bloat daily from system tables. A recommendation opens when estimated wasted space is over 25% and 100 MB for a table. The recommendation shows `VACUUM FULL`. That statement takes an exclusive lock on the table and rewrites it on every shard. Do not run it on a table your application is actively using. It is not an online operation, and a large table can take hours. `VACUUM FULL` writes a new copy of the table and its indexes. Each shard needs free disk space for a second copy of its portion of the table, plus a safety buffer. Plain `VACUUM` is not a substitute. It makes dead space reusable but does not return it to the operating system, so the recommendation stays open. Bloat can also come from long-running transactions or infrequent vacuuming. Address those causes if a rebuild does not hold. Once opened, a bloat recommendation stays open even if bloat later drops below the threshold. Once closed, it is not opened again for that table. ### Rebuilding bloated indexes Index bloat has the same MVCC cause as table bloat. Insights opens a recommendation when estimated wasted space is over 30% and 100 MB for an index. The recommendation typically shows `REINDEX INDEX CONCURRENTLY ...`. `REINDEX INDEX CONCURRENTLY` consumes database resources. Confirm that the branch has enough capacity before rebuilding large indexes. The same open-and-close rules as table bloat apply. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki database dashboard Source: https://planetscale.com/docs/neki/neki-dashboard Cluster topology, summary statistics, and query performance for a Neki database Open a Neki database to see the selected [branch](/docs/neki/branching): its cluster topology, summary statistics, and recent query performance. Each branch in Neki is a distinct instance. Switch branches from the dropdown at the top right. From the dashboard you can review: * The [routers](/docs/neki/terminology#router) and the Postgres instances in each [shard](/docs/neki/terminology#shard) * Live CPU and memory usage on the routers, primary, and replicas * Database summary statistics * Query performance for the branch * Connection details and a new-branch action ## Cluster topology The diagram in the center of the dashboard shows the infrastructure that serves the selected branch. ![Dashboard infrastructure diagram for a Neki production branch: a router in three availability zones above shard sh1, which has one primary and two replicas in AWS us-east-1](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/neki-dashboard/infrastructure-diagram.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=b28585fecdce842dcd1c641a0bc7d229) [Routers](/docs/neki/terminology#router) sit above the Postgres instances. A branch can have more than one router. Each router card in the diagram shows how many instances run across availability zones, its size, live CPU, and memory. Below the routers, the diagram shows each shard: * **Primary**: the writable Postgres instance for that shard, with region, cluster size, and live CPU and memory * **Replicas**: read-only Postgres instances that copy from the primary. Each replica shows region, cluster size, and replication lag A production branch is marked **HA**. Development branches contain a primary database with no replicas. If the branch has more than one shard, the control under the diagram lists them and their readiness. Select a shard to show its primary and replicas. **Manage** opens [cluster configuration](/docs/neki/cluster-configuration) to add, assign, or resize shards. How data arrives into shards is configured by the database's [data topology](/docs/neki/data-topology). ## Database summary The panel on the right shows: * **Postgres version** and CPU architecture * **Tables**, **Branches**, **Region**, and **Next backup** * **Egress bandwidth**, **Provisioned IOPS**, and **Total storage** for the selected branch * The current billing cycle * The next [maintenance window](/docs/neki/cluster-configuration/maintenance-windows), with a **Manage** link ## Performance metrics Below the diagram, a dropdown selects a time-series graph for the branch. Available metrics include: * Query latency (p50, p95, p99, p99.9, and max) * Queries per second * Rows read * Rows written * Query errors **View all query insights** opens [Query Insights](/docs/neki/monitoring/query-insights) for query-pattern detail. Historical router, shard, and storage graphs live on the [Metrics](/docs/neki/monitoring/metrics) page. ## Connecting to your database **Connect** generates credentials for a router on the selected branch. See [Connect to Neki](/docs/neki/connecting). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Overview Source: https://planetscale.com/docs/neki/overview The components of a Neki deployment and how they fit together Neki provides sharding, zero-downtime operations, and sophisticated cluster observability and management for Postgres. To make this work and scale well, we build a number of components that work closely with real Postgres nodes. Let's walk through the architecture of a Neki database cluster. The first component that a client of a Neki cluster will interact with is a Neki router. These routers then connect to either the primary or replicas in a shard, but do so via an intermediary sidecar component, which helps with connection pooling and managing the Postgres nodes. The topology service holds the data topology, the admin watches shard health, and the Replicator runs data-movement workflows. ![Applications send Postgres queries through interchangeable routers and sidecars to a Postgres primary and two replicas in a Neki-managed shard. The topology service, admin, and Replicator form the control plane.](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/overview/system-architecture.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=754fe0d71553c980f50767d7bee744fb) Let's take a closer look at each component. ## Routers A router is the Neki service that accepts Postgres connections and decides which shard or shards should run each statement. It uses the [data topology](/docs/neki/data-topology) and table definitions to build plans for and route incoming queries. When a query touches multiple shards, the router can combine their results before returning one result to the client. Applications connect through the Postgres wire protocol, so they can continue to use standard Postgres clients and ORMs. See [Query planning](/docs/neki/query-planning#how-a-statement-becomes-a-plan) for how Neki builds and reuses query plans. Routers are stateless, in that they are not responsible for durably storing information about the cluster, data topology, or database schema. The data topology and database schema are stored elsewhere, and locally cached and kept in sync on all routers. Because of this, routers can be easily scaled both vertically and horizontally to meet traffic demands. A single Neki cluster can have anywhere from one router to hundreds of routers. Learn more about how to manage routers in our [cluster configuration](/docs/neki/cluster-configuration) documentation. There is no primary router and no leader election. Any router can serve any connection. If a router fails, the connections it was holding at the time are dropped. To a client, this looks like any other lost Postgres connection, and it can reconnect to a different router node. This lends Neki to being highly-available by default. Every Neki cluster on PlanetScale comes with a minimum of 3 routers spread across three availability zones. ## What runs alongside Postgres A shard consists of one Postgres primary and its replicas. Each shard is its own failure domain. It handles switchovers and failovers independently from any other shard in the cluster. Every Postgres instance within a shard has two Neki components running alongside it: a **Sidecar** and a **PostgresManager**. The **Sidecar** is the per-instance endpoint for router queries and admin operations. All communication from a Neki Router to a Postgres node happens via a sidecar. The Neki admin service also queries it during health checks. Each sidecar streams its serving status to the routers, which use the latest status when selecting an instance. The **PostgresManager** manages the startup, teardown, and data directory of it's attached Postgres instance. When a replica promotion happens due to a planned switchover, the sidecar and PostgresManager on each instance keep running throughout. ## The control plane The control plane stores topology, monitors managed shards, repairs failures, and runs workflows that move data. ### The admin **Admin** is a service that manages the health of all other components of the Neki cluster. For Neki-managed shards, it health-checks each Postgres instance through that instance's sidecar and acts on what it finds. More than one admin can run at once. One of them holds recovery leadership, and only the leader performs node repairs. One of Admin's main roles is handling planned switchover and unexpected failovers within the Postgres primary and replicas within a shard. An **unplanned failover** replaces a primary that is no longer available. This could be due to a Postgres-level software crash, or the failure of underlying hardware. When choosing a replica to replace the primary with, Neki checks how much write-ahead log (WAL) each reachable replica has received. It prefers a replica that has already replayed all available WAL and waits briefly for one to catch up if needed. If none is ready, Neki promotes the replica that has received the most WAL. When using synchronous or semi-sync durability, there should always be at least one replica candidate with a fully caught up LSN. After promotion, Neki points the other replicas at the new primary. If it cannot update a replica, a later health check finds and repairs it. A **planned switchover** moves the primary role while the old primary is still healthy. Neki does this before maintenance on the primary, before taking that node out of service, or during a Neki rolling upgrade. Neki makes the old primary read-only and waits for the selected replica to catch up before promoting it. A completed switchover preserves the commits made on the old primary. If the switchover fails before promotion, Neki makes the old primary writable again. A **durability policy** decides whether the primary waits for a replica before confirming a commit. New shards start with `sync` as their configured policy. With `sync`, the primary normally waits for one replica to receive the write. ### The Replicator The Replicator is the service that runs data-movement workflows. It reads changes out of the source Postgres instances and applies them to the targets while the deployment keeps serving traffic. It connects to Postgres directly rather than going through the routers. For example, both [Moving tables amongst shards and resharding](/docs/neki/data-migration#what-a-migration-workflow-moves) use Replicator workflows. [Online schema changes](/docs/neki/schema-changes#one-workflow-across-all-shards) use the same copy-and-stream process. ### The topology service The topology service stores the data topology and service-discovery records for routers, admins, replicators, and sidecars, including each sidecar's Postgres endpoint. Routers and sidecars communicate with the topology service regularly, so when they pick up new versions of the topology without restarting. ## How failure looks to a client Neki routers support query buffering, which is key to maintaing connections and high-availability during switchovers and failovers within shards. When buffering is enabled, Neki can hold queries that target a primary or replica during these times. For a planned switchover, the admin publishes a buffering signal to routers before it takes place, so the breif period of query buffering can be tightly controlled. To a client, this would be observed as a short period of elevated query latency. During an unplanned failure, a qualifying query error can cause buffering to initiate. Neki will buffer queries for a bounded time. If the failover replacement completes within that time, clients will again only observe this event as a short period of elevated query latency. If it takes too long, the router will begin sending errors back to the client. After buffering ends, Neki replans queued statements against the current topology and selects targets using the current health state. If a shard reports that disk space is low, the router treats it as read-only and rejects writes from this until the issue is remidiated with a disk or node resize. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Platform preview limitations Source: https://planetscale.com/docs/neki/platform-preview-limitations Postgres features and query shapes that Neki does not support during Platform Preview > [!NOTE] > > Neki is currently in Platform Preview. Platform Preview features are “Beta Features” under the PlanetScale Terms of Service or your applicable agreement with PlanetScale. Accordingly, Neki is subject to the limitations and disclaimers applicable to Beta Features and is not covered by any service level agreement. During the Platform Preview, Neki's router does not support the following Postgres features or query shapes. These broad limits can change how an application connects, writes SQL, or manages a shard. The planner can reject more specific query combinations with a `not supported` or `not implemented` error. [Neki error codes](/docs/neki/error-codes) lists these rejections, the query shapes that trigger them, and what to do instead. ## High availability Single-node, non-high-availability configurations are not supported during Platform Preview. ## SQL and relation limits The router rejects the following operations: * `SELECT ... INTO`. * `INTERSECT` and `EXCEPT` when Neki must plan a query for an application table. `UNION` is supported. This limit does not apply to statements that Neki forwards unchanged, such as direct-shard statements. * `COMMIT AND CHAIN` and `ROLLBACK AND CHAIN`. * Cross-database object references. * Reading a Postgres inheritance parent without `ONLY` when it has child tables. * A `search_path` that includes `pg_temp`. ## COPY limits `COPY` must use the simple query protocol and must be the only statement in the query. Neki rejects `COPY` over the extended query protocol. The router supports `COPY FROM STDIN` and client `COPY TO STDOUT` (text, CSV, and binary) for unsharded tables, sharded tables, and reference tables. Client `COPY TO STDOUT` also works from a materialized view. For a sharded table, include the single-column shard key in the copied columns. Sharded `COPY FROM` does not support `ON_ERROR`, `REJECT_LIMIT`, or `LOG_VERBOSITY`. File-based `COPY` supports unsharded tables only. Neki rejects `COPY FROM PROGRAM`, `COPY TO PROGRAM`, `COPY (SELECT ...) TO`, and `COPY FROM` with a `WHERE` clause. When `COPY FROM` omits an identity column or a column with a sequence-backed or router-only default, supply that column's values explicitly. ## User-defined text search Neki supports Postgres's built-in text-search configurations and dictionaries. During Platform Preview, the router rejects creating a user-defined text-search configuration or dictionary and rejects changes to their definitions. Router-side text-search evaluation cannot use those custom definitions. ## Cluster-level objects Neki rejects cluster-level objects that depend on storage or code local to one Postgres host. This includes: * Creating a tablespace or moving a database into a tablespace. * Creating Postgres large objects. * Creating a procedural language with a custom handler. Built-in languages and languages installed by a supported extension remain available. * Loading a shared library with `LOAD`. ## Extensions that need shard targeting The router rejects the extension calls in this section when Neki must plan them. Each call acts on one Postgres server, or reports the state of one server. To use these calls, target one shard directly: 1. Get the shard UIDs from `__neki.list_shards()`. 2. Set `__neki.shard` to one shard UID. Neki forwards each call to that shard unchanged. 3. Run the call. The result covers the targeted shard only. 4. Repeat steps 2 and 3 for each shard that the result must cover. 5. Reset `__neki.shard` to restore normal routing. ```sql theme={null} SELECT uid, name FROM __neki.list_shards(); SET __neki.shard = ''; -- Run the extension call here. RESET __neki.shard; ``` See [Targeting one shard directly](/docs/neki/shard-targeting-fanout#targeting-one-shard-directly). ### pg\_squeeze The router rejects a `squeeze.squeeze_table()` call when Neki must plan it. The call squeezes the table on the targeted shard only. Repeat it for each shard that holds rows of the table. ### pgrowlocks The router rejects a `pgrowlocks()` call when Neki must plan it. On the targeted shard, the call shows the locked rows of the table on that shard only. Repeat it for each shard that holds rows of the table. ### pgstattuple The router rejects the `pgstattuple` functions, for example `pgstattuple()`, when Neki must plan them. On the targeted shard, a call returns the statistics of the rows on that shard only. Repeat it for each shard that holds rows of the table. To get the statistics of the full table, add the counts and the sizes from all shards. Do not add the percentages. Calculate each percentage again from the added values. ## External Postgres Connecting an externally managed Postgres source to a Neki migration workflow is not supported during Platform Preview. For the preview, import Postgres data into a Neki-managed, unsharded database instead. See [Import a Postgres database](/docs/neki/imports/postgres) for the supported dump-and-restore path. ## Cross-shard transactions A cross-shard transaction is a transaction that reaches more than one Postgres shard. Cross-shard transactions do not provide atomic commit across all shards. See [Transactions across shards](/docs/neki/transactions). Neki supports a `single` transaction routing mode that rejects an operation when a second shard would join the transaction. Atomic distributed transaction mode is not supported. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki pricing Source: https://planetscale.com/docs/neki/pricing How shard, router, storage, backup, and network resources contribute to Neki pricing. Each Neki branch has its own database infrastructure. Its estimated monthly price depends on the Postgres clusters provisioned for its shards, its router compute, and its storage, backup, and network usage. The PlanetScale dashboard shows the rates available to your organization. SKU prices vary by cloud provider, region, and your organization's commercial terms. Use the dashboard estimate or the [pricing calculator](https://planetscale.com/pricing) for the current rate of a specific size. Do not assume a us-east-1 example applies everywhere. > **[pricing.md](https://planetscale.com/pricing.md)** > > For an agent-friendly catalog of live cluster SKU rates and example SQL, fetch `https://planetscale.com/pricing.md`. Prefer [https://planetscale.com/pricing.md](https://planetscale.com/pricing.md) for live Vitess, Neki, and Postgres cluster SKU rates. ## What appears on the bill | Charge | What you pay for | | - | - | | **Shard compute** | The selected cluster SKU, applied once per shard | | **Additional replicas** | Additional replicas added to each shard | | **Router compute** | The selected router SKU and number of routers | | **Network-attached storage** | Allocated disk, plus IOPS and throughput above the included baselines | | **Backup storage** | Compressed backup and WAL data above the included allowance | | **Network transfer** | Public egress and private traffic above the included allowance | A configuration profile has no compute cost by itself. Adding a shard adds another copy of that profile's cluster SKU to the bill. Routers are not multiplied by shard count. There are no upfront commitments. Clusters are billed based on the selected instance size and prorated to the millisecond. Router compute is billed based on the selected router size and number of routers and prorated to the millisecond. ## Cluster SKU rates Every shard assigned to a configuration profile uses that profile's cluster size. Neki does not offer a single-node configuration. The highly available SKU rate covers **one primary and two replicas**. ```text theme={null} shard compute = highly available SKU rate × shards on the profile ``` The size menu in **Clusters** shows the monthly rate, additional-replica rate, vCPU, and memory for each option. Metal SKUs include local NVMe storage in that rate. Network-attached storage SKUs bill network-attached storage separately. If four shards share one profile, you pay four times that profile's highly available SKU rate. See [Cluster sizing](/docs/neki/cluster-configuration/cluster-sizing). ## Additional replicas You can add replicas above the two included with the SKU. You cannot set the replica count to anything lower than `2`. Extra replicas use the SKU's additional-replica rate, which is about one third of the highly available rate. ```text theme={null} additional replica compute = additional-replica rate × extra replicas × shards on the profile ``` Each extra network-attached storage replica also gets its own network-attached disk. Those volumes do not receive the 10 GB included allowance; you pay for the full allocated size. See [Database replicas](/docs/neki/replicas). [Read-only replicas](/docs/neki/read-only-replicas) also use the additional-replica rate. The two included replicas do not cover them, so every read-only replica is billed: ```text theme={null} read-only replica compute = additional-replica rate × read-only replicas × shards on the profile ``` Each read-only replica on network-attached storage gets its own volume, billed as an additional replica volume. ## Router SKU rates Every Neki branch has routers, which use a separate SKU from shard compute. A production branch starts with one router in each availability zone. Upsizing shards does not change the router size. The monthly production rate shown for a router size covers the three-router baseline: one router in each of three availability zones (AZ). For a fixed production router configuration with the same replica count in every AZ: ```text theme={null} router compute = router-size monthly rate × replicas per AZ ``` When AZs do not have the same number of running routers, production billing uses the observed total: ```text theme={null} router compute = router-size monthly rate × observed running routers ÷ 3 ``` A development branch uses one AZ, so its development router rate applies to each running router without the three-AZ divisor. Autoscaling and size changes are prorated to the millisecond. ## Storage > [!NOTE] > > We use GB as > **[gibibytes, otherwise known as binary gigabytes](https://simple.wikipedia.org/wiki/Gibibyte)**, > to calculate storage and usage limits. For reference, 1 binary gigabyte is > equivalent to 2^30 bytes. ### Storage types explained PlanetScale offers two storage options with different pricing models: | [PlanetScale Metal](/docs/metal) | [Network-attached storage](/docs/neki/cluster-configuration#configure-profile-storage) | | - | - | | **All-inclusive pricing** | **Configuration-based pricing** | | Storage is included in the cluster SKU. High-performance NVMe SSDs are attached directly to each server. Choose a larger cluster size to get more storage. Best for high-performance applications that need predictable costs. | The first 10 GB is included on each primary and included replica volume. Additional storage is billed separately from compute. Add storage without changing the cluster size, with optional automatic disk scaling. Best for applications with variable storage needs or cost optimization. | ### Metal Local NVMe capacity is included in the Metal SKU rate. To get more local storage, choose a larger Metal size. On AWS, the smaller Metal sizes (**M-10** through **M-160**) let you select the storage capacity separately from vCPU and memory. Each storage option has its own SKU rate. To get more local storage on these sizes, choose a larger storage option. See [Metal options on AWS](/docs/plans/planetscale-skus#metal-options-on-aws). ### Network-attached storage Network-attached storage clusters bill allocated disk separately from compute. Neki measures capacity per shard and attributes it to that shard's configuration profile. You pay for current allocated capacity, not the configured minimum. AWS gp3 volumes include 10 GB, 3,000 IOPS, and 125 MiB/s per volume on the primary and the two replicas that come with the SKU. Charges apply only to capacity and provisioned IOPS or throughput above those baselines. GCP uses Hyperdisk Balanced. IOPS and throughput scale with disk size at no separate charge; you pay for allocated disk. | Usage | Included per volume | Additional monthly rate (AWS us-east-1) | | - | -: | -: | | Primary and included replica volumes | First 10 GB | \$0.125 per GB | | Additional replica volumes | None | \$0.125 per GB | | Provisioned IOPS | First 3,000 IOPS | \$0.009 per IOPS | | Provisioned throughput | First 125 MiB/s | \$0.073 per MiB/s | The configured IOPS and throughput apply to each network volume on every shard in the profile. Regional rates are below. Configure minimum disk, autoscaling, IOPS, and bandwidth from the profile **Storage** tab. ### AWS storage rates | Region | Storage (per GB/month) | Additional IOPS (per IOPS/month) | Additional throughput (per MiB/s/month) | | :- | -: | -: | -: | | ap-northeast-1 (Tokyo) | \$0.150 | \$0.011 | \$0.088 | | ap-south-1 (Mumbai) | \$0.143 | \$0.103 | \$0.084 | | ap-southeast-1 (Singapore) | \$0.150 | \$0.011 | \$0.088 | | ap-southeast-2 (Sydney) | \$0.150 | \$0.011 | \$0.088 | | ca-central-1 (Montreal) | \$0.138 | \$0.010 | \$0.081 | | eu-central-1 (Frankfurt) | \$0.149 | \$0.011 | \$0.088 | | eu-west-1 (Dublin) | \$0.138 | \$0.010 | \$0.081 | | eu-west-2 (London) | \$0.145 | \$0.011 | \$0.084 | | sa-east-1 (Sao Paulo) | \$0.238 | \$0.018 | \$0.139 | | us-east-1 (N. Virginia) | \$0.125 | \$0.009 | \$0.073 | | us-east-2 (Ohio) | \$0.125 | \$0.009 | \$0.073 | | us-west-2 (Oregon) | \$0.125 | \$0.009 | \$0.073 | ### GCP storage rates | Region | Storage (per GB/month) | | :- | -: | | asia-northeast3 (Seoul) | \$0.312 | | europe-west1 (Belgium) | \$0.24 | | europe-west4 (Netherlands) | \$0.264 | | northamerica-northeast1 (Montréal) | \$0.264 | | us-central1 (Iowa) | \$0.24 | | us-east1 (South Carolina) | \$0.24 | | us-east4 (Virginia) | \$0.264 | ## Backups Each branch includes backup storage equal to **twice the allocated disk** on that branch. Required schedules (every 12 hours, two-day retention) use this allowance. Compressed backup data and WAL above the included amount are billed at **\$0.023 per GB per month**. A protected backup that outlives the default two-day window continues to contribute storage until you turn protection off or delete it. See [Back up and restore](/docs/neki/backups). ## Network transfer Replication between a shard's primary and its replicas, and backup traffic, are not billed as network transfer. ### Public traffic Outgoing public traffic (egress) is billed. Public ingress is free. | Branch type | Included public egress | | :- | :- | | Production | 100 GB per month | | Development | 10 GB per month | ### Private traffic AWS PrivateLink and GCP Private Service Connect traffic — ingress and egress — is billed at **\$0.01 per GB**, regardless of region. Each branch includes the same allowance as public traffic: 100 GB for production, 10 GB for development. ### Public egress rates | Cloud | Region | Egress (per GB) | | :- | :- | -: | | AWS | ap-northeast-1 (Tokyo) | \$0.101 | | AWS | ap-south-1 (Mumbai) | \$0.096 | | AWS | ap-southeast-1 (Singapore) | \$0.096 | | AWS | ap-southeast-2 (Sydney) | \$0.111 | | AWS | ca-central-1 (Montreal) | \$0.060 | | AWS | eu-central-1 (Frankfurt) | \$0.060 | | AWS | eu-west-1 (Dublin) | \$0.060 | | AWS | eu-west-2 (London) | \$0.060 | | AWS | sa-east-1 (Sao Paulo) | \$0.137 | | AWS | us-east-1 (N. Virginia) | \$0.060 | | AWS | us-east-2 (Ohio) | \$0.060 | | AWS | us-west-2 (Oregon) | \$0.060 | | GCP | asia-northeast3 (Seoul) | \$0.117 | | GCP | europe-west1 (Belgium) | \$0.060 | | GCP | europe-west4 (Netherlands) | \$0.060 | | GCP | northamerica-northeast1 (Montréal) | \$0.060 | | GCP | us-central1 (Iowa) | \$0.060 | | GCP | us-east1 (South Carolina) | \$0.060 | | GCP | us-east4 (Virginia) | \$0.060 | ## Development branches A development branch is billed on its own. It uses development SKU rates and the development network allowance. Each development shard is highly available (primary plus at least two replicas). Compute and router charges are prorated to the millisecond and end when you delete the branch. Restoring a backup or point-in-time recovery always creates a new branch, so the restored environment is an additional billed branch until you delete it. ## Estimate a monthly total | Component | How the SKU or usage rate is applied | | - | - | | Shard compute | Highly available SKU rate × shards, plus additional-replica rate × (extra replicas + read-only replicas) × shards | | Router compute | Production: router SKU rate × observed running routers ÷ 3. Development: router SKU rate × running routers | | Storage | Allocated disk and provisioned IOPS/throughput above the per-volume allowances, per shard | | Backups | Compressed backup and WAL above 2× allocated disk, at \$0.023 per GB-month | | Network | Public egress and private traffic above the branch allowance | | **Estimated monthly total** | **Sum of the components above** | Actual charges differ from a full-month estimate when resources or usage change during the billing period. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Query planning Source: https://planetscale.com/docs/neki/query-planning How Neki plans a statement, and how to inspect its plan with EXPLAIN A query plan defines where each part of a SQL statement runs. It contains routes that send work to shards and, when needed, router operations that combine shard results. ## How a statement becomes a plan Incoming queries to a Neki database cluster are planned after being parsed at the router. The planner builds the plan from four inputs: * **The SQL statement.** The statement defines the tables, predicates, and operations that the plan must cover. * **The database schema.** The schema defines the tables, columns, types, and indexes that the statement refers to. * **The data topology.** The [data topology](/docs/neki/data-topology) defines which shards hold the rows of each table. * **Statistics.** Statistics collected from the shards give the planner row estimates to compare alternative plans. See [Cost model, cardinality estimation, and statistics](#cost-model-cardinality-estimation-and-statistics). ### Query plan cache and protocols The query protocol determines whether two statements can share a plan. With Postgres's simple query protocol, Neki replaces eligible literal values with parameters before looking up a plan. `user_id = 1337` and `user_id = 999`, for example, both become `user_id = $1` and can share a plan. The value for `$1` is bound at execution, when it can direct the route to a different shard. > [!NOTE] > > Preparing a statement does not pin its plan. The router looks it up again > using the current `search_path` and session role, and may rebuild it if the > cached copy was evicted or invalidated by a schema, topology, or statistics > change. ## EXPLAIN in Neki A Neki plan has two levels. At the router level, Neki routes work to shards and combines the results. At the shard level, each shard's Postgres plans the SQL that Neki sends to it. The three `EXPLAIN` modes differ in which levels they show: | Mode | Shows | | - | - | | `EXPLAIN` | Only the shard level: a Postgres plan from one shard | | `EXPLAIN (NEKI_PLAN)` | Only the router level: Neki's routes and router operations | | `EXPLAIN (NEKI_PG_PLAN)` | Both levels: Neki's plan with the Postgres plan from each shard that a route involves | `COSTS` works in all three modes. `COSTS OFF` removes the cost estimates from Neki's operators and from the shard-level Postgres plans. `VERBOSE`, `BUFFERS`, `SETTINGS`, `WAL`, and `MEMORY` have no effect on Neki's own operators. `EXPLAIN` and `EXPLAIN (NEKI_PG_PLAN)` pass them as written to the shard-level Postgres `EXPLAIN`. `EXPLAIN (NEKI_PLAN)` accepts them and ignores them. All four output formats (`TEXT`, `JSON`, `XML`, and `YAML`) are supported in all three modes. The default format is `TEXT`. The examples in this section use the [example schema](/docs/neki/example-schema), where `public.orders` belongs to the `user_data` shard group and uses `user_id` as its primary shard index. ### `EXPLAIN` A plain Postgres `EXPLAIN` works when the whole statement runs on the shards with no router operations. The plan is then a single route, which can also be a route that scatters to every shard in a group, because every shard runs the same SQL. Neki sends the `EXPLAIN` to one of the route's shards and returns that shard's output unchanged. The output is the plan of a real Postgres, with no Neki-specific content: ```sql theme={null} EXPLAIN SELECT order_id FROM public.orders WHERE user_id = 1337; ``` ```text theme={null} Seq Scan on orders (cost=0.00..2.00 rows=1 width=8) Filter: (user_id = 1337) ``` When the plan has router operations, Neki rejects the statement with SQLSTATE `NK017`: ```text theme={null} ERROR: This statement requires Neki router processing HINT: Use `EXPLAIN (NEKI_PG_PLAN)` (with ANALYZE to execute) to see Neki's plan ``` `EXPLAIN ANALYZE` executes the statement. Neki accepts it with a plain `EXPLAIN` only when the plan sends the whole statement to exactly one shard. ### `EXPLAIN (NEKI_PLAN)` `EXPLAIN (NEKI_PLAN)` shows Neki's plan: the routes that send work to shards, and the router operations that combine their results. Without `ANALYZE`, it does not contact the shards. The output shows only router-level plan nodes. The route node is the boundary: it shows the SQL that runs on the shards, but not the shard-level plans. An equality predicate on `orders.user_id` produces one route: ```sql theme={null} EXPLAIN (NEKI_PLAN) SELECT order_id FROM public.orders WHERE user_id = 1337; ``` ```text theme={null} Route [EqualUnique] (cost=0.00..1100.00 rows=100 width=8) Query: SELECT order_id FROM public.orders WHERE user_id = $1 ShardGroup: user_data Values: $1 ``` The `Query` field uses a reusable parameterized form of the SQL, so the literal `1337` appears as `$1`. `Values: $1` is the parameter the route resolves, and `ShardGroup: user_data` names the topology group that receives the route. `Route [EqualUnique]` is the single-shard routing choice. An `IN` predicate produces a route for the selected values: ```sql theme={null} EXPLAIN (NEKI_PLAN) SELECT order_id FROM public.orders WHERE user_id IN (42, 999); ``` ```text theme={null} Collapse (cost=0.00..2500.00 rows=500 width=8) -> Route [IN] (cost=0.00..2500.00 rows=500 width=8) Query: SELECT order_id FROM public.orders WHERE user_id = ANY($1) ShardGroup: user_data Values: $1 ``` `Route [IN]` narrows the query to the shards holding the listed values. It has `multi` [fanout](/docs/neki/shard-targeting-fanout#controlling-fanout). `Collapse` indicates where Neki turns those responses into one result. An aggregate without a usable shard-key predicate scatters across the group: ```sql theme={null} EXPLAIN (NEKI_PLAN) SELECT count(*) FROM public.orders; ``` ```text theme={null} Aggregate [Ordered] (cost=0.00..2002.00 rows=1 width=8) -> Collapse (cost=0.00..2002.00 rows=2 width=8) -> Route [Scatter] (cost=0.00..2002.00 rows=2 width=8) Query: SELECT count(*) FROM public.orders ShardGroup: user_data ``` `Route [Scatter]` indicates this is a scatter-gather query, which requires contacting all shards in the shard group to complete the query. Each shard computes a partial count, and the router-side `Aggregate` adds the partial results before returning the final count. As a shard group grows, the number of shards used by scatter queries increases. This may be acceptable for occasional full-group aggregates and other whole-table operations, but scatter-gather execution of frequent queries should be avoided because it increases utilization of database and network resources. If you expect a query to be single-shard but are seeing `Route [Scatter]` in the query plan, check for a suitable shard-key predicate or consider using a [global secondary index (GSI) lookup](/docs/neki/reference-tables-and-gsis#gsi-mapping). ### `EXPLAIN (NEKI_PG_PLAN)` `EXPLAIN (NEKI_PG_PLAN)` shows Neki's plan together with the Postgres plan from each shard that a route involves. A Postgres plan shows how a shard plans to run the SQL that Neki sends to it. The output shows both router-level and shard-level plan nodes, and a scatter route shows a plan for each shard in the group. Neki designs this mode for maximal compatibility with Postgres, so tools that read Postgres `EXPLAIN` output can also read Neki plans. Without `ANALYZE`, Neki does not execute the statement. For each route, Neki sends a plain Postgres `EXPLAIN` to every shard that the route involves: | Route | Shards that receive the `EXPLAIN` | | - | - | | Scatter | Every shard in the shard group | | Equality or `IN` | The shards that hold the listed values | | Any shard, such as a read from a reference table | Every candidate shard | Each route shows its shard plans under one of two labels. `All shard plans` (`"ShardPlans": {"All": [...]}` in the structured formats) means that every listed shard runs the route's SQL. `Any one shard plan` (`"Any"`) means that the route runs on one of the listed shards, selected arbitrarily. A scatter read from `users` in the [example schema](/docs/neki/example-schema) shows the plan of both shards in the group: ```sql theme={null} EXPLAIN (NEKI_PG_PLAN) SELECT user_id, name FROM users; ``` ```text theme={null} Collapse (cost=0.00..3000.00 rows=1000 width=151) -> Route [Scatter] (cost=0.00..3000.00 rows=1000 width=151) Query: SELECT user_id, name FROM public.users ShardGroup: user_data All shard plans -> Shard shard-a (cost=0.00..0.00 rows=0 width=0) -> Seq Scan on users (cost=0.00..1.20 rows=20 width=15) -> Shard shard-b (cost=0.00..0.00 rows=0 width=0) -> Seq Scan on users (cost=0.00..1.30 rows=30 width=15) ``` When the whole plan is one route on one shard, Neki omits the route node. The output names the shard on the first line, followed by the shard's Postgres plan, similar to the output of a plain `EXPLAIN`: ```sql theme={null} EXPLAIN (NEKI_PG_PLAN) SELECT user_id, name FROM users WHERE user_id = 42; ``` ```text theme={null} Shard shard-a Seq Scan on users (cost=0.00..1.25 rows=1 width=15) Filter: (user_id = 42) ``` Some routes take values that exist only after the statement starts to run, for example the inner side of a correlated subquery. Without `ANALYZE`, Neki rejects `NEKI_PG_PLAN` for such a plan. Use `ANALYZE`, or drop `NEKI_PG_PLAN` to see only Neki's plan. With `ANALYZE`, Neki executes the statement. The output adds Neki's actual row counts and timings, and per-shard statistics for each route. A route whose rows go directly to the client shows each shard's `EXPLAIN ANALYZE` output. A route whose rows feed a router operation, such as one input of a join, shows a plain Postgres `EXPLAIN` from each shard that the route used. In the following example, the route's rows feed the router-side `Aggregate`, so each shard shows a plain `EXPLAIN` with its statistics: ```sql theme={null} EXPLAIN (NEKI_PG_PLAN, ANALYZE) SELECT sum(total_amount) FROM orders WHERE created_at > '2026-09-09'; ``` ```text theme={null} Aggregate [Ordered] (cost=0.00..2002.00 rows=1 width=32) (actual time=0.236..0.238 rows=1 loops=1) RowsTotal: 1 CostAtActualRows: Total=2002 RemoteCalls=2 RowsProcessed=2 NetworkRows=2 -> Collapse (cost=0.00..2002.00 rows=2 width=32) (actual time=0.217..0.220 rows=2 loops=1) RowsTotal: 2 CostAtActualRows: Total=2002 RemoteCalls=2 NetworkRows=2 -> Route [Scatter] (cost=0.00..2002.00 rows=2 width=32) (actual time=0.216..0.217 rows=2 loops=1) RowsTotal: 2 RemoteCalls: 2 NetworkBytesIn: 102 AvgWidth: 51 CostAtActualRows: Total=2002 RemoteCalls=2 NetworkRows=2 Query: SELECT sum(total_amount) FROM public.orders WHERE created_at > $1::timestamp without time zone ShardGroup: user_data -> Shard shard-a (cost=0.00..0.00 rows=0 width=0) Shard Stats: Executions=1 TimeMs=0.21 RowsOut=1 BytesIn=51 -> Aggregate (cost=2.45..2.46 rows=1 width=32) -> Seq Scan on orders (cost=0.00..2.40 rows=18 width=5) Filter: (created_at > ('2026-09-09'::cstring)::timestamp without time zone) -> Shard shard-b (cost=0.00..0.00 rows=0 width=0) Shard Stats: Executions=1 TimeMs=0.215 RowsOut=1 BytesIn=51 -> Aggregate (cost=4.17..4.18 rows=1 width=32) -> Seq Scan on orders (cost=0.00..4.10 rows=26 width=5) Filter: (created_at > ('2026-09-09'::cstring)::timestamp without time zone) Planning Time: 0.192 ms Execution Time: 0.239 ms ``` > [!WARNING] > > `ANALYZE` executes the statement being explained. This is especially risky > for data-changing statements: an `INSERT`, `UPDATE`, `DELETE`, or `MERGE` > under `ANALYZE` changes rows on every shard that the plan reaches, and the > changes stay. [EXPLAIN output reference](/docs/neki/explain-reference) shows the complete `NEKI_PLAN` and `NEKI_PG_PLAN` output of a query with router operations, in every format. ### Cost model, cardinality estimation, and statistics > *A perfect search over bad estimates is just a very efficient way to find the wrong plan.* Neki's query planner uses statistics collected from the shards. The statistics must exist on the shards before Neki can collect them, and keeping them current is your responsibility. Postgres has many kinds of statistics. The [Postgres documentation on planner statistics](https://www.postgresql.org/docs/current/planner-stats.html) describes them. The planner estimates the number of rows that each part of a plan produces. These cardinality estimates come from the collected statistics. The cost model then uses the estimates to compare the alternative plans and choose the cheapest one. Neki collects the following statistics from the catalog of each shard's Postgres. This list grows as Neki collects more kinds of statistics. * The estimated row count of each table. * The number of distinct values (`n_distinct`) and the fraction of NULL values (`null_frac`) of each column that is part of an index, or of a primary key, foreign key, or unique constraint. A collector on each shard reads the catalog periodically and sends changes to the routers. A significant change invalidates the cached plans that depend on the changed statistics. Postgres writes statistics when `ANALYZE`, autoanalyze, `VACUUM`, or `CREATE INDEX` runs. A freshly loaded table has no statistics until one of these runs. Until then, the planner uses default estimates. `ANALYZE` and `VACUUM` sent through a router run on every shard. [Data topology](/docs/neki/data-topology#choosing-a-shard-index) recommends a Postgres index on the shard-key columns, and [Best practices](/docs/neki/best-practices) recommends that you keep their statistics current. Indexes and current statistics give the planner the estimates it needs to find a good plan. An index also adds its columns to the set whose statistics Neki collects. `EXPLAIN (NEKI_PG_PLAN)` shows what the planner does with its inputs. Creating and dropping indexes, and running `ANALYZE`, change what the planner has to work with. Repeat the two steps to keep your queries fast: 1. Inspect the plan with `EXPLAIN (NEKI_PG_PLAN)`. 2. Change the indexes or refresh the statistics, then inspect the plan again. Insights helps in this process. Use [Query Insights](/docs/neki/monitoring/query-insights) to observe the workload, and [schema recommendations](/docs/neki/monitoring/schema-recommendations) for DDL suggestions based on production telemetry. ## Statement types Neki plans read queries (`SELECT`) and data-changing statements (`INSERT`, `UPDATE`, `DELETE`, and `MERGE`) the standard way, which [EXPLAIN in Neki](#explain-in-neki) describes. When a plan reaches several shards, Neki sends the shard requests in parallel and combines the responses into one Postgres result. Some results require more than concatenating rows from each shard. Neki can combine partial aggregates, apply ordering and limits across shards, or complete joins that cannot be pushed into the shards. Neki plans a statement the same way inside and outside an explicit transaction. The transaction limits only how the plan runs, which [Transactions](/docs/neki/transactions) describes. The statement types in the following sections have their own planning. ### DDL Neki keeps the same schema on every shard, so data definition language (DDL) sent through a router runs on every managed shard. Multi-shard DDL is not atomic across the deployment. A failure can leave the change applied on some shards but not others. Managed [schema-change workflows](/docs/neki/schema-changes) add readiness checks and coordinate completion through online or direct execution paths. ### COPY Neki plans `COPY` on its own path because the rows arrive as a data stream after the statement, not in the statement text. For `COPY FROM` into a sharded table, Neki routes each incoming row to a shard. `COPY FROM` can route rows into a sharded table when the input includes the columns needed by its primary shard indexes. It is rejected when that table has an enabled GSI because COPY does not maintain GSI lookup rows. Keep the GSI disabled during the copy, then backfill and verify the lookup table before enabling it. Client `COPY TO STDOUT` works for a sharded table, a reference table, and a materialized view, including binary. File `COPY` remains unsharded-only. ### Cursors Neki plans the query of a `DECLARE` statement the standard way, then builds the cursor from the plan: * When the plan is a single route, each shard that the route resolves to opens its own Postgres cursor. * When the plan has router operations, the router serves the cursor. It runs the plan and returns rows as `FETCH` statements ask for them. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Read-only replicas Source: https://planetscale.com/docs/neki/read-only-replicas Add Neki read-only replicas to serve long-running reads without using the replicas that take over during failover. A read-only replica is a Postgres instance in a shard that copies changes from the primary and serves reads. Neki never promotes a read-only replica to primary. Use read-only replicas for long-running reads, such as analytics, reports, exports, and batch jobs. This work then does not use the [replicas](/docs/neki/replicas) that serve live read traffic and that take over when the primary fails. Read-only replicas give these guarantees: * A read-only replica never becomes the primary, during a failover or at any other time. * A read-only replica does not acknowledge commits. It does not make a write more durable. * A read-only replica rejects writes. * A session reads from read-only replicas only when it sets the `rdonly` target. Neki does not send other traffic to them. ## Read-only replicas and replicas Neki uses the name `rdonly` for read-only replicas in routing targets and node roles. The two types of instance compare as follows: | | Replica | Read-only replica | | :- | :- | :- | | Can become the primary | Yes | No | | Counts for write durability | Yes | No | | Routing target | `replica` | `rdonly` | | Node role in the API | `replica` | `rdonly` | | Count per shard | At least 2 on a production branch | 0 to 8 | | Cluster size and storage | From the configuration profile | From the configuration profile | Both types use Postgres physical replication from the shard primary. After a failover, both types replicate from the new primary. Read-only replicas add read capacity. They do not add availability. A shard can fail over only to a replica. For example, a development branch with one primary and two read-only replicas has no failover target. ## Add read-only replicas The [configuration profile](/docs/neki/cluster-configuration) sets the number of read-only replicas. Every shard assigned to the profile gets that number of read-only replicas. The count does not include the primary or the replicas. To set the count in the dashboard: **Step 1** From the PlanetScale dashboard, select your Neki database and branch. **Step 2** Open **Clusters** and select the configuration profile. **Step 3** Select **Instances**. **Step 4** Choose the number of **Read-only replicas**. **Step 5** Review the estimated monthly cost, then select **Apply now**. **Step 6** Use the profile's **Changes** tab to follow the change. You can also set the count when you create a configuration profile. To set the count with the [PlanetScale CLI](/docs/cli/branch), use the `--read-only-replicas` flag: ```bash theme={null} pscale branch config-profile update --read-only-replicas 2 ``` `pscale branch config-profile create` accepts the same flag. In the [PlanetScale API](/docs/api/reference/update_shard_configuration_profile), the field is `read_only_replicas`. An update creates a change request, as other configuration profile changes do. Use `pscale branch config-profile changes list` to follow the change. These rules apply to the count: * The value is 0 to 8 for each shard. * Read-only replicas count toward the node limit of the cluster. A change that goes above the limit fails. * PlanetScale spreads the read-only replicas across the availability zones of the shard. For example, two read-only replicas on a production branch go into two different availability zones. A change to the read-only replica count does not move the primary. The change does not restart the primary or the replicas. Neki adds or removes only read-only replicas. To remove all read-only replicas, set the count to `0`. ## Send reads to read-only replicas Neki sends reads to shard primaries by default. To use read-only replicas, select **Read-only replica** under **Query target** on the **Connect** page. The dashboard then adds `-c __neki.target=RDONLY` to the generated connection options. The **Read-only replica** option shows only when the branch has read-only replicas. You can also set the Neki target to `rdonly` when the connection starts. For `psql`, use `PGOPTIONS`: ```bash theme={null} PGOPTIONS='-c __neki.target=RDONLY' \ psql 'host= port=5432 user= password= dbname=postgres sslnegotiation=direct sslmode=verify-full sslrootcert=system' ``` You can also change the target before you start a transaction: ```sql theme={null} SET __neki.target = 'rdonly'; ``` The target value is not case-sensitive. These rules apply to a session with the `rdonly` target: * Inserts, updates, deletes, and other writes fail with SQLSTATE `25006` (`read_only_sql_transaction`). * If no eligible read-only replica is available for a destination shard, the query fails. Neki does not send the query to a replica or to the primary. * Neki selects a read-only replica with the same recency, locality, and affinity policy that it uses for replicas. See [How Neki chooses a replica](/docs/neki/replicas#how-neki-chooses-a-replica). The replication lag and read consistency rules for replicas also apply to read-only replicas. See [Replication lag and read consistency](/docs/neki/replicas#replication-lag-and-read-consistency). ## Changes and maintenance A read-only replica uses the cluster size, storage, and Postgres parameters of its configuration profile. Some changes to the profile, such as a new cluster size, restart or replace each read-only replica. Queries with the `rdonly` target can fail while their read-only replica restarts. If your application uses the `rdonly` target, retry failed connections and queries. If a shard has only one read-only replica, queries with the `rdonly` target fail while that read-only replica is not available. If your application cannot accept these failures, use two or more read-only replicas. ## Limitations * A read-only replica has the same cluster size and storage as the other instances of its configuration profile. You cannot select a different size for read-only replicas. * A restore from a backup does not copy the read-only replica count. The new branch has no read-only replicas. Add them after the restore. ## Pricing PlanetScale bills each read-only replica at the additional-replica rate of the cluster size. The two replicas included with the cluster size do not cover read-only replicas. Each read-only replica on network-attached storage has its own volume, billed as an additional replica volume. See [Additional replicas](/docs/neki/pricing#additional-replicas). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Reference tables and GSIs Source: https://planetscale.com/docs/neki/reference-tables-and-gsis Shared data placement and secondary lookup routing A reference table keeps a full copy of shared data on the shards that need it. A global secondary index (GSI) maps another lookup key to the shard key of the original row. Both give Neki a way to route queries that cannot rely on a table's shard key alone. Consider a sharded table that has a single shard index on the `tenant_id` column. Without a reference table or GSI, looking up a row by `email` instead of `tenant_id`, for example, may require searching every shard. Both can avoid unnecessary scatter reads, but add storage and work to writes. The original table and row behind a GSI are called its owner table and owner row. ## Reference tables Reference tables are best for shared data that every shard in a group needs. If the `customers` table is sharded across the `tenant_data` shard group and is often joined with common data from the `countries` table, keep a full copy of `countries` on each shard and declare it a reference table. ![A query joins customers to countries with a tenant\_id predicate, routing through Neki to one shard in tenant\_data. Postgres uses the local countries copy, while the other shards hold their own complete copies](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/reference-tables-and-gsis/reference-read.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=7dee929318495c39be1eac25f7e24a01) This tells Neki that `countries` data exists on every shard in the `tenant_data` shard group, and Neki can send the join to shards based on the predicates on `customers`. If a `tenant_id` predicate narrows the query to one shard, the join executes on one shard. If the query scatters, each shard joins its customer rows to its local `countries` copy. The final result is calculated by the router, but joins stay local to each shard because any row from `customers` is co-located with all rows from `countries`. A query against `countries` alone can use any copy in the group. > [!NOTE] > > The binding changes routing only. Neki assumes the copies already exist and > contain the same rows. It does not verify them, so an incomplete copy can > produce an incomplete result. ## Configure a reference table and GSI The following complete topology declares: * `countries` as a reference table expected on both shards in `tenant_data`. * `users` as a table sharded by `tenant_id`. * `users_by_email` as an unsharded lookup table for the unique `by_email` GSI. ```json theme={null} { "authoritative_shard_group": "meta", "shard_indexes": { "xxhash_tenant_id": { "type": "xxhash", "columns": ["tenant_id"] } }, "shard_groups": [ { "uid": "meta", "key_ranges": [ { "shard_uid": "shard-a" } ] }, { "uid": "tenant_data", "default_shard_index": "xxhash_tenant_id", "key_ranges": [ { "shard_uid": "shard-a", "end": "80" }, { "shard_uid": "shard-b", "start": "80" } ] } ], "databases": { "app": { "global_secondary_indexes": { "by_email": { "schema": "public", "table": "users_by_email", "owner_table": "users", "columns": ["email"], "owner_sk_columns": ["tenant_id"], "unique": true, "ignore_null": false, "enabled": false } }, "schemas": { "public": { "tables": { "users": { "shard_group": "tenant_data", "secondary_indexes": [ { "name": "by_email", "columns": ["email"] } ] }, "users_by_email": { "shard_group": "meta" } }, "reference_tables": { "countries": { "shard_groups": ["tenant_data"] } } } } } } } ``` The Postgres tables must already exist on the shards named by their bindings. Applying the topology does not create `countries`, `users`, or `users_by_email`, and it does not copy existing rows. The GSI has two linked declarations. The database-level `global_secondary_indexes.by_email` entry defines the lookup table and its owner. The `users.secondary_indexes` entry activates that GSI for owner reads and writes. The lookup table also needs an ordinary table binding so Neki can route lookup rows. `owner_sk_columns` names the owner primary-index columns stored in each lookup row. It can be omitted when the owner has exactly one primary index whose `ignore_null` value is `false`; Neki infers that index. Name `owner_sk_columns` when the owner has more than one such primary index. For a GSI, omitted scalar fields use their protobuf defaults: `schema` is resolved from the lookup table when unambiguous, and `unique`, `ignore_null`, and `enabled` default to `false`. A non-unique GSI must also set `owner_pk_columns` so lookup rows for different owner rows remain distinct. A reference table can omit `shard_groups` only when a schema, database, or cluster default shard group resolves for it. Neki then uses that effective default. Listing the groups explicitly makes every copy location visible in the topology. ### Activate a GSI safely An enabled GSI does not backfill historical lookup rows. Enabling an incomplete GSI can return incomplete results without an error. Use this sequence: 1. Create the lookup table with the required keys and constraints. 2. Declare both linked GSI entries and set `global_secondary_indexes..enabled` to `false`. 3. Backfill the lookup table while also following owner-table changes. 4. Verify every expected owner row has the correct lookup row. 5. Set `enabled: true` and apply the topology at cutover. Keep the GSI disabled until both backfill and change capture are complete. Re-run verification immediately before activation. ## Reference-table writes An ordinary `INSERT`, `UPDATE`, or `DELETE` against `countries` runs on every distinct shard that holds a copy. Neki sends the statements concurrently and waits for every shard before reporting completion. For a group with four shards, one inserted row produces four shard executions and four stored copies. Postgres replicas and indexes add their usual storage overhead. > [!NOTE] > > The router coordinates one backend transaction per participating shard. > Those transactions commit independently, so a commit can succeed on some > shards and fail on another. ## GSI mapping In the example, `users` is sharded by `tenant_id`, but the application also needs to look up a user by email: ```sql theme={null} SELECT * FROM users WHERE email = 'adam@example.com'; ``` The query has no `tenant_id`, so the table's primary routing cannot identify one shard. A GSI on `email` adds a lookup table that maps the email address to the owning row's `tenant_id`. For a supported single-table equality query, Neki reads that mapping and uses it to route the original query to the correct shard. The lookup table has its own placement, so this first step may itself reach one shard or several. ![An email predicate routes to the email\_lookup GSI, which returns tenant\_id 42. Neki uses that tenant\_id to route the users query to shard-b in the tenant\_data group](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/reference-tables-and-gsis/gsi-read.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=2bee0cc9300ab83e9b1d933e9ef6909a) A unique GSI can sometimes answer the query without sending an additional query to the `users` table. This covering path applies only when the lookup row contains every owner column the plan needs. Locking reads still fetch the owner row so Postgres locks the underlying record rather than the lookup entry. A non-unique GSI may return several `tenant_id` values. Neki uses them to narrow the owner query to the shard(s) that contain matches. Because reading the GSI itself requires work, it's best to use GSIs for highly selective lookups. Otherwise, if the looked up value appears across most shards, the work needed to run a GSI-assisted query approaches a scatter query, and the GSI also adds overhead on the write path. [Query planning](/docs/neki/query-planning#explain-in-neki) shows how to inspect the resulting plan. ## GSI mappings for writes Neki maintains GSI lookup rows in the same request that writes the owner row. For an insert into `users`, the router first checks mappings owned by other tables that the new row references. It writes the `users` row, then writes the email GSI mapping owned by `users`. Neki supports one narrow kind of GSI-column update. It updates the owner row, removes the old lookup row, and adds the new lookup row. All of these conditions must be true: * The update affects one active, globally unique, single-column GSI. * The `SET` list changes only that indexed column. * The `WHERE` clause contains only equality predicates. They pin every shard-key column and a unique key, so the update reaches one row on one shard. * The statement has no common table expression (CTE) or `RETURNING` clause. * The new value can be computed on the shard. `DEFAULT` and values that the router must compute are not supported. Neki rejects other index-column updates, including updates involving a non-unique or composite GSI or affecting more than one GSI. ## Global uniqueness Postgres enforces uniqueness within each lookup-table shard. To make that constraint global, equal indexed values must always reach the same shard. An unsharded lookup table meets that condition. For a sharded lookup table, every shard-key column must also be one of the GSI's indexed columns. If `email_lookup` is sharded by `email`, duplicate addresses meet on the same shard and conflict. > [!WARNING] > > Neki rejects a lookup table sharded by a column that is not part of the GSI. > It also disables a unique GSI if the lookup table's Postgres primary key no > longer matches the indexed columns. ## GSI limits and fallbacks | Condition | Outcome | | - | - | | A statement directly names a GSI lookup table | Reject the statement | | Unsupported read shape, such as `IN`, a set operation, or a multi-table route | Use primary-index routing or scatter when possible; otherwise reject the query | | `UPDATE` or `DELETE` routed through a GSI | Reject the statement | | Single-row update that meets the supported globally unique GSI shape | Update the owner row and its lookup row | | Supported lookup without all columns needed for covering | Read the lookup, then the owner row | | Covering read with different row-level security (RLS) on the owner and lookup tables | Reject the query during planning | | `ignore_null: true` and an indexed value is NULL | Do not create or verify a lookup row | | `ignore_null: false` and an indexed value is NULL | Reject the write | | Any owner-key value or computed lookup routing value is NULL | Reject the write | | A required GSI column is absent from an insert | Reject the insert during planning | | `DELETE ... USING` targets an active GSI owner | Reject the statement as unsupported | | `COPY FROM` targets a sharded table with an enabled GSI | Reject the copy because `COPY` does not maintain lookup rows | | The GSI is disabled | Stop maintenance and non-owner verification, but continue rejecting updates to its declared columns | | A unique GSI's lookup primary key no longer matches its indexed columns | Disable the GSI | A covering read replaces the owner table with the lookup table in the SQL sent to Postgres. Different RLS could change which rows are visible, so Neki fails planning. `ignore_null` applies only to indexed inputs. Owner keys and lookup routing values remain strict because index-driven reads cannot reach a NULL-keyed mapping. For a non-unique GSI, Neki compares the estimated lookup cost with the fallback route. It uses the GSI only when the lookup is expected to cost less. ## Choose what to duplicate A reference table duplicates the complete shared dataset on every shard it covers. A GSI duplicates only the columns needed to find the owner row. Use a reference table when the whole dataset is small and many shards need it for reads or joins. Use a GSI when a large sharded table needs selective access through another key. Both choices add work to writes. Neither removes the need for a primary routing design that distributes load and keeps common queries narrow. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Database replicas Source: https://planetscale.com/docs/neki/replicas Use Neki replicas for high availability and read-only traffic. Each Neki shard on a production branch has one primary Postgres instance and at least two replicas. The primary accepts writes. Replicas copy changes from the primary, can serve read-only queries, and can take over if the primary becomes unavailable. A new production configuration profile starts with two replicas. You can add replicas for a configuration profile on the **Clusters** page. The change applies to every shard assigned to that profile. Each new replica is restored from the shard's last backup, then joined to the primary to start replication. See [Replicas and backups](/docs/neki/backups#replicas-and-backups). A shard can also have [read-only replicas](/docs/neki/read-only-replicas). A read-only replica serves reads, but Neki never promotes it to primary. ## High availability Neki uses Postgres physical replication to copy write-ahead log (WAL) records from each shard's primary to its replicas. If a primary becomes unavailable, Neki can promote an eligible replica and update the shard topology to use the new primary. Neki handles failures through a division of responsibility between the [admin](/docs/neki/overview#the-admin) and the [orchestration layer](/docs/neki/terminology#orchestration-layer). The orchestration layer owns the lifecycle of the [components that run alongside each Postgres instance](/docs/neki/overview#what-runs-alongside-postgres), the sidecar and PostgresManager: it creates, restarts, and replaces them. The admin monitors each Postgres instance through its sidecar, not by connecting to Postgres directly. It directs Postgres through PostgresManager, and owns replication and failover: it detects a failed primary, promotes a replica, and repairs replicas. Some operations need both. For example, before the orchestration layer replaces the components of a shard's primary, it asks the admin for a planned switchover to a healthy replica, and waits for the admin to confirm the new primary. A failover can briefly interrupt queries or connections while the shard changes primaries. Applications should retry transient connection and query failures. If a replica needs a WAL segment that no source can supply, Neki fences that replica from normal use. Once the shard primary is reachable, Neki restores the replica from the last backup and the [admin](/docs/neki/overview#the-admin) joins it to the primary so replication can resume. This repairs the replica without changing the shard primary. Replication durability and read visibility are separate. WAL can be durable on a replica before that replica has replayed the change. A successful write therefore does not guarantee that an immediate replica read will return that change. Send reads that require the latest committed data to the primary. ## Send reads to replicas Neki sends reads to shard primaries by default. To use replicas, select **Replica** under **Query target** on the **Connect** page or set the Neki target when the connection starts. For `psql`, use `PGOPTIONS`: ```bash theme={null} PGOPTIONS='-c __neki.target=REPLICA' \ psql 'host= port=5432 user= password= dbname=postgres sslnegotiation=direct sslmode=verify-full sslrootcert=system' ``` You can also change the target before starting a transaction: ```sql theme={null} SET __neki.target = 'replica'; ``` Replica connections are read-only. Inserts, updates, and deletes fail instead of being sent to a primary. If no eligible replica is available for a destination shard, the query also fails instead of falling back to that shard's primary. ## How Neki chooses a replica For each shard, the router starts with replicas that are serving and have known replication lag. It then orders the candidates using recency, locality, and session affinity. | Policy | Default behavior | | :- | :- | | Recency | Prefer replicas with no more than 30 seconds of reported lag | | Locality | Within the same lag tier, prefer a replica in the router's availability zone | | Affinity | For reads, choose again for each statement to spread reads across eligible replicas | The default maximum reported lag is 15 minutes. A replica above that limit, or one whose lag is unknown, is not eligible for reads. A replica between the preferred and maximum thresholds is used only when no preferred replica is available. The `__neki.replica_recency`, `__neki.replica_locality`, and `__neki.replica_affinity` settings can make these preferences stricter, disable them, or keep a session on the same preferred replica. See [Choosing where reads run](/docs/neki/shard-targeting-fanout#choosing-where-reads-run) for the supported values. When a query reaches more than one shard, Neki chooses a replica separately for each shard. Cross-shard transactions do not provide atomic commit across all shards, so the combined result can reflect slightly different points in time. ## Replication lag and read consistency Replication lag measures how far a replica is behind its primary. Zero seconds means the replica has replayed everything it has received from the primary. A higher value means recent changes may not be visible there yet. Replica reads work well for dashboards, analytics, exports, and other read-heavy workloads that can tolerate some delay. Use a primary connection for read-after-write paths or any query that must return the newest committed data. Replica reads do not guarantee read-your-writes or monotonic reads. A later read can move to another eligible replica and see older data than an earlier read. Session affinity reduces how often a connection moves between replicas, but it does not provide either guarantee. Neki can choose another replica when the preferred one is unavailable, no longer meets the routing policy, or fails before serving a query. ## Monitor replicas The database dashboard shows replication lag on each replica card. A value of `0s` means that replica is caught up. Each replica also has a chart shortcut that opens the **Shards** tab of the **Metrics** page with that node selected. If a replica has high or unknown lag, Neki can exclude it from read traffic. Check the node's replication lag, health, CPU, memory, and IOPS on **Shards**, then compare its disk, storage, and WAL activity on **Storage**. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Replicator workflows Source: https://planetscale.com/docs/neki/replication Copy and change streaming for migrations and online schema changes The Neki Replicator builds a copy of live table data on a new target while applications continue using the source shards. It copies the existing rows first, then streams new changes until the target catches up. Neki uses this process to move tables, redistribute rows across shards, and rebuild tables for online schema changes. It lasts only for the workflow and is separate from Postgres physical replication, which supports replicas used for reads and failover. ## Workflows Three operations use Replicator workflows: * **MoveTables** moves selected tables to another database, or to a shard group whose physical shards are not the source shards. * **Reshard** redistributes declared tables in one source shard group into a target shard group in the same database. After an [unsharded import](/docs/neki/imports/postgres), Reshard is the path that shards those tables. * **Online schema changes** can [build a shadow table and keep it current](/docs/neki/schema-changes#one-workflow-across-all-shards) until the shadow is ready. Not every schema change requires data movement. Changes that Neki can apply directly do not run a copy-and-stream workflow. Each workflow creates one or more independent Replicator streams. A stream connects one source shard to one target shard. When parallel table streams are enabled, several streams can run between the same pair. A workflow that spans several source or target shards therefore makes progress through several streams rather than through one deployment-wide stream. During a Reshard of a large `events` table, each stream reads from one source shard and writes to one target shard. The [data topology](/docs/neki/data-topology) determines which copied rows and later source changes belong to that target. ## Copy and stream The copying stage transfers rows that already exist. Neki copies tables in batches so that a large table does not need to be moved in one transaction. After each copy cycle, Neki applies WAL changes for the rows already copied, reducing WAL retention. Within one stream, tables are copied one after another. A workflow can still copy concurrently because it can have several streams. Each copy cycle reads from a consistent source snapshot while the source remains available for reads and writes. Changes made during the copy are also sent to the target, so it can catch up without requiring the application to stop writing. Copying and streaming add work to both the source and target. The impact is greater for large tables and write-heavy workloads. After the initial copy finishes, Neki streams new source changes to the target until the workflow completes or switches traffic. ## Progress and resumption Neki records progress as data is copied and streamed. If a stream is interrupted, it resumes from committed progress instead of restarting the entire copy. Reaching the streaming stage does not switch application traffic by itself. Before routing changes or a shadow table takes over, Neki waits for the target to catch up. Until the target takes over, applications continue reading from and writing to the source. Replicator workflows do not merge changes written independently to the target. [Data migration](/docs/neki/data-migration#moving-traffic) explains the traffic switch, cutover checks, and completion stages for online DDL. ## What Replicator does not cover Replicator workflows are only used for temporary data-movement workflows. Other Neki features use different systems: * **Postgres replicas** use physical replication for availability and failover. * **[Reference-table writes](/docs/neki/reference-tables-and-gsis#reference-table-writes)** update every copy as part of the original write. * **[Global secondary indexes](/docs/neki/reference-tables-and-gsis#gsi-mappings-for-writes)** are updated with the original table write. * **Query fanout** sends a query to multiple existing shards without creating another copy of the data. * **Direct schema changes** are [applied without a copy-and-stream workflow](/docs/neki/schema-changes#how-direct-ddl-works). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Schema changes Source: https://planetscale.com/docs/neki/schema-changes Native DDL and managed workflows across Neki shards A statement that changes a large or busy table can hold locks, rewrite existing rows, build an index, generate large volumes of write-ahead log (WAL), and consume substantial database resources. These effects can increase query latency or stop application traffic. Sharding adds another requirement: every shard must end with the same table definition. Applying a change independently on each shard makes progress and failure difficult to track. It can also leave some shards on the old definition and others on the new definition. Neki supports both native DDL and managed DDL. Native DDL means issuing a supported statement such as `ALTER TABLE` through a router. Neki sends the statement to every managed shard, where Postgres executes it immediately. Native DDL has no workflow record, progress tracking, readiness gate, or explicit completion step. After native DDL commits, the change is committed across the managed shards, but another router may not have refreshed its local schema view yet. The router that accepts the DDL emits a notice containing the exact `__neki.wait_for_ddl(schema_version, cluster_version)` call for that change. Run that call before sending dependent SQL through other router instances: ```sql theme={null} SELECT __neki.wait_for_ddl(, ); ``` The wait completes when the schema change is visible on every router. A notice or warning about router visibility does not mean that the DDL transaction failed; read the notice and use the supplied wait call to establish the cluster-wide visibility barrier. > [!WARNING] > > During Platform Preview, a shard added later receives existing publications > with the rest of the schema. Subscriptions are not part of that schema, so a > shard added later does not receive them. Contact PlanetScale Support before > combining Postgres logical replication with shard lifecycle changes. Managed DDL uses a workflow. The workflow records the requested DDL, tracks progress on every managed shard, waits until the change is ready everywhere, and gives you explicit control over completion and cleanup. ## Online and direct schema changes Neki's managed DDL supports two execution paths. **Online DDL** is designed for changes that would be disruptive when applied to the live table. Examples include changes that rewrite many rows and index creation on a large table. Neki builds an updated shadow table in the background while the application continues to use the original table. **Direct DDL** sends the change directly to Postgres in a transaction. This path is appropriate for statements that do not need a table copy, and it is the only managed path for statements that the shadow-table process cannot carry. Examples include creating or dropping a table and changes to types, sequences, and views. Neki examines the requested DDL and selects the appropriate path. You can also choose direct execution for an online-compatible change when applying the change directly is preferable. A request that combines incompatible online and direct work may need to be split into separate workflows. Both paths use the same managed workflow. Direct DDL receives the same progress tracking, readiness gate, explicit completion, retry handling, and multi-shard coordination as Online DDL. ## How Online DDL works Online DDL keeps the live table available while Neki builds its replacement: 1. It creates a shadow table beside the live table. 2. It applies the requested DDL to the shadow table. 3. It copies the existing rows from the live table to the shadow table in batches, using a sequence of transactions instead of one transaction for the entire copy. 4. Meanwhile, it streams ongoing inserts, updates, and deletes to the shadow table. 5. It keeps applying changes until the shadow table is caught up and ready for cutover. The application continues to read and write the original table during these steps. Copying rows and catching up can take a long time, but the batched work limits the size and duration of each transaction. Cutover is the short final step. Neki first asks routers to buffer new queries for the affected table. Neki then locks the table, applies the final streamed changes, and swaps the original and shadow tables in one Postgres transaction. When the transaction commits, the new table definition is live under the original table name. Buffered queries then continue against the new table. If Neki cannot acquire the lock within the cutover timeout, it backs off and retries. Application traffic continues between attempts. ## How direct DDL works Direct DDL does not create a shadow table or copy rows. The shard records that it is ready, then waits for you to complete the workflow. At completion, Postgres applies the DDL in a transaction. Direct execution can require locks on the live table. The safety of this path therefore depends on the requested operation and the table workload. Neki uses it for statements that are best handled directly, and you can select it deliberately for changes that are safe to run in place. ## One workflow across all shards A Neki database has one logical table definition across its managed shards. Some shards store rows for a table. Other shards keep the table definition but do not store rows for that table. For most online-compatible table changes: * Data-bearing shards copy and catch up a shadow table with Online DDL. * Schema-only shards wait to apply the same change directly. For a direct schema change, every managed shard waits on the direct path. The workflow reports each shard's progress and does not become ready to complete until every shard is ready: * An online shard is ready when its shadow table is copied and caught up. * A direct shard is ready when it is waiting to apply its transaction. When every shard is ready, complete the workflow once. Neki tells each shard to cut over or apply its direct DDL. ![A schema-change workflow prepares and catches up shadow tables on data-bearing shards while schema-only shards wait for direct DDL. After every shard reports ready, each shard commits independently. Online shards atomically swap tables and schema-only shards apply direct DDL before the Postgres schema converges across managed shards.](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/schema-changes/online-change.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=98241d0d143e60b6c5a7462922d507df) Each shard commits its own Postgres transaction. Completion is coordinated, but it is not one atomic transaction across the whole Neki database. In normal operation, shards finish within seconds of one another, although failures can extend that interval. If one shard completes and another shard fails, Neki preserves the workflow and its per-shard status. The safe recovery direction is forward: clean up the failed attempt, reissue the workflow, then complete the remaining shards so every shard reaches the same table definition. ## Managing the workflow A schema-change workflow has a small lifecycle. ### Create and track Creating the workflow records the DDL and starts the requested work on every managed shard. Status reports show overall progress and per-shard progress. Create a workflow from any Postgres client connected through a Neki router. Double the single quotes inside the DDL string: ```sql theme={null} SELECT * FROM __neki.online_ddl_create( 'add-refund-state', 'orders_db', 'ALTER TABLE public.orders ADD COLUMN refund_state text NOT NULL DEFAULT ''none''', '' ); ``` The arguments are the workflow name, database name, DDL, and migration ID. An empty migration ID tells Neki to generate one. The function returns the workflow name and migration ID. Check the aggregate and per-shard status: ```sql theme={null} SELECT workflow, ddl, status, jsonb_pretty(status_by_shard) FROM __neki.online_ddl_status('add-refund-state'); ``` Wait until every shard reports `ddl_status: "running"`, `current_readiness: true`, and `controller_state: "running"`. A `postgres_error` means Neki could not read that shard's state. Reissuing Create for an interrupted workflow can restore missing controllers and resume work from persisted state. Depending on where the interruption occurred, a shard may resume or restart its copy and streaming work. A failed attempt must be cleaned up before it can restart. ### Complete Completion is a separate, explicit action. Neki first verifies that every shard is ready and that no shard has failed. Online shards then cut over, and direct shards apply their DDL. ```sql theme={null} SELECT __neki.workflow_complete('add-refund-state'); SELECT workflow, status, jsonb_pretty(status_by_shard) FROM __neki.online_ddl_status('add-refund-state'); ``` Repeated completion requests are safe. A shard that has already completed does not apply the change again. An unfinished shard continues toward the requested end state. ### Cancel Cancel abandons the workflow before completion. In the normal path, Neki marks each shard cancelled, removes its Online DDL artifacts, and deletes the workflow record only after every shard reaches a safe final state. ```sql theme={null} SELECT __neki.workflow_cancel('add-refund-state'); ``` Cancellation is rejected while a shard is actively cutting over because its final table state is not yet known. Wait for cutover to settle, inspect `online_ddl_status`, and retry Cancel. If some shards completed while others cancelled, Neki retains the workflow record instead of hiding the schema skew. Reissue Create with the stored configuration, allow the remaining shards to become ready, and Complete the workflow so every shard converges on the new definition. Do not run blind artifact cleanup across a mixed final state. ### Clean up Cleanup is a separate action. After a successful complete, it drops the retained original table and the remaining Online DDL artifacts, then removes the workflow record when every shard has completed. ```sql theme={null} SELECT __neki.online_ddl_cleanup('add-refund-state'); ``` After a failed shard, cleanup removes that shard's artifacts so you can reissue Create. Neki keeps the workflow in that case. Normal cancellation performs its own cleanup; do not follow a successful Cancel with a separate Cleanup. If cancellation or cleanup is interrupted, reissuing the operation is safe. Neki uses persisted workflow and per-shard state to resume, retry, or reject an action that would conflict with the recorded outcome. ### Retry a failed shard Clean up the failed attempt, then reissue Create with the stored workflow configuration: ```sql theme={null} SELECT __neki.online_ddl_cleanup('add-refund-state'); SELECT * FROM __neki.online_ddl_create( 'add-refund-state', '', '', '' ); ``` Completed shards do not apply the change again. After all shards become ready, run `workflow_complete` and `online_ddl_cleanup` as above. ## Choosing a path Use native DDL when the change is safe to execute immediately and does not need workflow tracking or an explicit completion gate. Use managed Online DDL when direct execution could block traffic, rewrite a large live table, or build an index aggressively on the serving table. Use managed direct DDL when the statement cannot use the shadow-table process, when no copy is needed, or when you want workflow coordination for a change that is safe to run in place. Common single-table `ALTER TABLE` changes can use the online path, including adding or dropping columns, changing a column type, adding or dropping constraints, and adding an index. `CREATE INDEX` and `DROP INDEX` also use the online path. Do not include `CONCURRENTLY` when creating an index through Online DDL. Statements that do not need a shadow-table copy use the managed direct path. These include `CREATE TABLE`, `DROP TABLE`, and operations on types, sequences, and views. A table change can also be forced onto the direct path by passing `'{"applyDirect": true}'` as the fifth argument to `online_ddl_create`. Each workflow must resolve to one affected table and one execution category. Split changes across workflows when they affect multiple tables or combine online and direct work. Changing a table's shard key or primary routing index is a data-movement operation; use a resharding workflow instead. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Shard targeting and fanout Source: https://planetscale.com/docs/neki/shard-targeting-fanout Limit how broadly statements route, choose where reads run, and target one shard Neki routes each statement according to its [query plan](/docs/neki/query-planning) and the [data topology](/docs/neki/data-topology). Session settings control three parts of that routing: how many shards a statement can reach, which type of Postgres instance serves reads, and whether data queries go to one shard directly. ## Controlling fanout The topology determines which shards contain rows. The fanout component of a query plan specifies how broadly a statement will route: | Level | Permitted routing shape | | - | - | | `single` | A plan with single-shard routing and at most one shard-touching operation | | `multi` | `single`, plus plans limited to a bounded set of shards | | `scatter` | Any plan, including one that reaches every shard in a group | An `IN` predicate on the shard key produces a `Route [IN]`, as shown in [Query planning](/docs/neki/query-planning#explain-neki_plan), and has `multi` fanout. Its values may resolve to one shard or several, but the plan is limited to the shards holding those values. ### The `__neki.fanout` setting Set `__neki.fanout` for a session to reject `SELECT` and data-changing statements whose routing shape exceeds the selected level: ```sql theme={null} SET __neki.fanout = 'single'; ``` The default is `scatter`. A statement that exceeds the setting fails instead of running: ```text theme={null} statement's fan-out (scatter) exceeds __neki.fanout (single) ``` Use a stricter setting in development or CI to catch queries that route more broadly than intended. Set `single` for request paths expected to use a single-shard routing shape, or `multi` when a bounded set of destinations is acceptable. Allow `scatter` for deliberate whole-group work. The setting also applies to `EXPLAIN (NEKI_PG_PLAN, ANALYZE)` because `ANALYZE` executes the underlying statement. This setting does not apply to DDL. Fanout required to maintain [reference tables and GSIs](/docs/neki/reference-tables-and-gsis) is also exempt. ## Choosing where reads run Read targeting selects which type of Postgres instance serves a session's read queries. The default target is `primary`. Set `__neki.target` to use another target: ```sql theme={null} SET __neki.target = 'replica'; ``` | Target | Use | | - | - | | `primary` | Send reads to the shard primary. This is the only target that accepts DML | | `replica` | Send reads to replicas used for live traffic | | `rdonly` | Send reads to [read-only replicas](/docs/neki/read-only-replicas) used for long-running work | The following settings control how Neki chooses a `replica` or `rdonly` instance: | Setting | Values | Default | | - | - | - | | `__neki.replica_recency` | `prefer` favors the freshest tier, `require` allows only that tier, and `off` does not order by lag | `prefer` | | `__neki.replica_locality` | `prefer` favors the router's cell, `require` allows only that cell, and `off` does not order by cell | `prefer` | | `__neki.replica_affinity` | `session` favors the same replica for later reads, while `none` chooses again for each statement | `none` | Neki excludes a replica when its lag is unknown or above the maximum allowed lag. A `require` policy can narrow the candidates further. The query fails if no candidate remains. These settings cannot change during a transaction. Set them in separate statements before starting the transaction. ## Targeting one shard directly Direct shard targeting sends data queries to one shard UID without using the data topology to route them. ```sql theme={null} SET __neki.shard = 'shard-a'; ``` Data statements follow these rules while `__neki.shard` is set: | Statement | Behavior | | - | - | | `SELECT`, `INSERT`, `UPDATE`, `DELETE`, and `MERGE` | Forward to the targeted shard | | `SET`, `SHOW`, transaction control, and prepared-statement control | Follow the normal router path | | Schema-changing DDL, such as `CREATE TABLE`, `ALTER TABLE`, `DROP TABLE`, `TRUNCATE`, and `SELECT INTO` | Reject | | `COPY`, `DO`, and unsupported statement types | Reject | Neki rejects a schema change while a direct shard target is set because schema changes run across the cluster rather than on one shard. Use a separate connection without `__neki.shard` to run the change cluster-wide. Publication DDL and maintenance, privilege, and annotation statements still follow the normal router path and can run across every shard. While a shard target is set, a data query cannot also call a router-managed function such as `current_setting`, `set_config`, or `nextval`. Neki rejects the statement because it cannot forward the table access and evaluate the function in the router. You cannot change `__neki.shard` during a transaction. Clear the target to restore normal routing: ```sql theme={null} RESET __neki.shard; ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki terminology Source: https://planetscale.com/docs/neki/terminology Definitions for the Postgres, PlanetScale, and Neki concepts used throughout the documentation. ## Database technologies ### Postgres Postgres is an open-source relational database. Applications connect to a Neki database using the Postgres wire protocol and standard Postgres clients. ### Neki Neki is PlanetScale's distributed Postgres system. It places routers in front of Postgres shards and provides query routing, topology management, failover, sharding, and online data-movement workflows. ### PlanetScale PlanetScale is a managed database platform focused on reliability, scalability, and developer productivity. PlanetScale operates the infrastructure around Neki and provides the application, APIs, and workflows used to create and manage databases. ## PlanetScale concepts ### Organization An organization contains PlanetScale databases and the people who can access them. Organization settings control membership, billing, and access to shared resources. ### Database A database is the top-level resource an application connects to. It belongs to an organization and contains one or more branches. ### Branch A branch is an isolated database environment. A database has a default branch, often named `main`, and can have additional branches for development and testing. Production branches are intended for production workloads and use a highly available configuration. Development branches are intended for development and testing. ### Configuration profile A configuration profile is a set of infrastructure settings shared by one or more shards. It defines the cluster size, replica count, Postgres version, parameters, and enabled extensions. Changing a profile applies the new settings to every shard assigned to it. ### Cluster size A cluster size defines the CPU, memory, and storage resources assigned to a Postgres instance. Every shard assigned to a configuration profile uses the profile's selected cluster size. ## Neki concepts ### Router A router accepts Postgres connections from applications. It plans each query, uses the data topology to find its destination shards, sends work to those shards, and combines results when a query reaches more than one shard. Routers do not store durable application data and are interchangeable. ### Admin The admin monitors the Postgres instances in a Neki cluster through their sidecars. It analyzes cluster health and coordinates repairs, including replica recovery and primary failover. This control-plane service is different from a database administrator or Postgres role. Its selected size establishes a capacity baseline, and PlanetScale automatically increases its memory when needed. ### Orchestration layer The orchestration layer, implemented by the Neki operator, reconciles the configuration requested through PlanetScale into running routers, Admin services, shards, and Postgres instances. It coordinates rolling changes, resizing, backups, restores, and replacement instances and reports their readiness and progress. It is outside the SQL query path and does not determine data placement. ### Postgres instance A Postgres instance is one Postgres server within a shard. A serving instance can be the primary or a replica. ### Primary The primary is the writable Postgres instance for a shard. Each shard has one primary at a time. ### Replica A replica is a Postgres instance that copies changes from the primary through Postgres physical replication. Replicas serve reads and can be promoted during failover. ### Read-only replica A read-only replica is a Postgres instance that copies changes from the primary and serves reads. Neki never promotes a read-only replica to primary, and it does not count for write durability. Routing targets and node roles call it `rdonly`. See [Read-only replicas](/docs/neki/read-only-replicas). ### Shard A shard is a set of Postgres instances that replicate the same data. It has one primary, zero or more replicas, and zero or more read-only replicas. Provisioning a shard creates database capacity. It does not assign tables or rows to that shard; the [data topology](/docs/neki/data-topology) controls data placement. ### Shard group A shard group is a routing layout that maps key ranges to physical shards. Tables in the same group can keep related rows together when they use the same shard index. A shard group in the data topology is different from a PlanetScale configuration profile. A configuration profile controls infrastructure settings; a shard group controls data placement and query routing. ### Data topology The data topology is Neki's routing map. It defines shard groups, shard indexes, key ranges, table bindings, and the shards that own each range. ### Shard index A shard index turns values from one or more table columns into routing values. Neki compares those routing values with a shard group's key ranges to choose a destination shard. ### Key range A key range assigns part of a shard group's routing space to a physical shard. Together, a shard group's key ranges determine where its rows belong. ### Authoritative shard group The authoritative shard group provides the canonical Postgres object identifiers (OIDs) and schema events for a database. Neki rewrites result sets so clients always see those OIDs. Sequence metadata operations also use this group by default. It contains exactly one key range that spans the entire keyspace. ### Reference table A reference table stores a copy of the same logical rows in multiple shard groups. This allows local reads and joins without routing every lookup back to one owner shard. ### Global secondary index A global secondary index, or GSI, uses a lookup table to map another key to the shard key of an owner row. Neki can read this mapping before routing a query to the owner table. Applications cannot query, write to, or change the lookup table directly through a router. ### Replicator Replicator is the Neki component that copies existing rows and then streams changes from a source to a target. It runs the workflows used to move tables, reshard data, and apply online schema changes. ### Workflow A workflow is the multi-shard unit of a Replicator migration. It tracks an online data movement operation while the database continues serving traffic. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Transactions Source: https://planetscale.com/docs/neki/transactions Snapshot and commit behavior for transactions that reach more than one shard Routing one statement to several Postgres shards does not turn those shards into one transactional database. ## Reads do not share a snapshot An ordinary multi-shard read does not establish one shared Postgres snapshot across its destinations. Concurrent writes can therefore cause different shards to reflect different points in time within the same result. ## Multi-shard writes are not atomic Neki commits each shard separately. It does not use two-phase commit to make every shard succeed or fail together. If one shard fails after another commits, the statement can leave different outcomes on the participating shards. ## Enforce a one-shard transaction The default transaction routing mode is `multi`, which permits a transaction to reach more than one shard without making its commit atomic. Set the mode to `single` before starting a transaction when the application wants Neki to prevent that transaction from expanding to another shard: ```sql theme={null} SET __neki.tx_mode = 'single'; BEGIN; ``` > [!NOTE] > > Distributed and atomic cross-shard transactions are not yet supported. Keep a > transaction on one shard when its writes must commit or roll back together. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki quickstart Source: https://planetscale.com/docs/neki/tutorials/planetscale-quick-start-guide Create a single-shard Neki database, connect, and insert data ## Getting started ### Prerequisites 1. You will need [a PlanetScale account](https://auth.planetscale.com/sign-up). 2. An administrator of your PlanetScale organization must [join the Neki Platform Preview](/docs/neki#availability-and-access). ### Create a Neki database **Dashboard** **Step 1** Click "**New database**" > "**Create new database**" on your organization's overview page. **Step 2** Name your database. **Step 3** Select a region. For the lowest latency, select a region near you or your application's hosting location. **Step 4** Select **Neki** as the database engine. ![Create database form with the Neki engine selected](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/tutorials/select-neki-engine.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=41490be17a8a313a2bb81d9075e4ff7e) **Step 5** Select the desired cluster configuration, storage option, and cluster size for your database. **Step 6** If prompted, add a valid credit or debit card. **Step 7** Click "**Create database**" to deploy your database. **CLI** Install and authenticate the [PlanetScale CLI](/docs/cli/planetscale-environment-setup). **Step 1** List organizations, then switch to the one enrolled in the Platform Preview: ```bash theme={null} pscale org list pscale org switch ``` **Step 2** List regions and pick a slug close to your application: ```bash theme={null} pscale region list ``` **Step 3** Create an HA Neki database. `--replicas 2` requests two replicas in addition to the primary, for three Postgres nodes on the initial shard. `PS_10` is a valid starting cluster size. ```bash theme={null} pscale database create \ --engine neki \ --region \ --cluster-size PS_10 \ --replicas 2 \ --wait ``` Provisioning can take a few minutes. `--wait` keeps the command open until the database is ready. > [!NOTE] > > Database names can contain up to 63 lowercase letters, numbers, > dashes, or underscores. A name cannot start or end with a dash or > underscore. ### Add a schema to your database Use the dashboard's web console to create some example tables. To use another SQL client, first see [Connect to your database](#connect-to-your-database). **Step 1** [Enable web console access for production branches](/docs/neki/web-console#enable-for-production-branches). **Step 2** Select **Console** and connect to the `main` branch. The console connects to the **Primary** server and the `postgres` database by default. See [Web console](/docs/neki/web-console) for connection controls and supported commands. ![Web console connected to the main branch](https://mintcdn.com/planetscale-2/gWONWhlM_S1jInv7/neki/tutorials/web-console-main.png?fit=max\&auto=format\&n=gWONWhlM_S1jInv7\&q=85\&s=9c900b11594e861e011a75c953a4c041) **Step 3** Create some example tables by running the following commands in the web console: ```sql theme={null} CREATE TABLE categories ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, name varchar(255) NOT NULL ); ``` ```sql theme={null} CREATE TABLE products ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, name varchar(255) NOT NULL, image_url varchar(255), category_id bigint ); ``` ```sql theme={null} CREATE INDEX category_id_idx ON products (category_id); ``` **Step 4** You can confirm that the tables have been added by running: ```sql theme={null} SELECT table_name FROM information_schema.tables WHERE table_schema = 'public' AND table_name IN ('categories', 'products') ORDER BY table_name; ``` ![Web console showing the categories and products tables](https://mintcdn.com/planetscale-2/gWONWhlM_S1jInv7/neki/tutorials/web-console-tables.png?fit=max\&auto=format\&n=gWONWhlM_S1jInv7\&q=85\&s=46ae8b2fd775222c13f6798b5cbb0990) ### Insert data into your database Run the following SQL to add some rows to the tables: ```sql theme={null} INSERT INTO categories (name) VALUES ('Office supplies'); ``` ```sql theme={null} INSERT INTO products (name, image_url, category_id) VALUES ('Ballpoint pen', 'https://example.com/500x500', 1); ``` You can confirm the data has been added with: ```sql theme={null} SELECT * FROM products; ``` ```sql theme={null} SELECT * FROM categories; ``` ![Web console showing SELECT results for the products and categories tables](https://mintcdn.com/planetscale-2/gWONWhlM_S1jInv7/neki/tutorials/web-console-results.png?fit=max\&auto=format\&n=gWONWhlM_S1jInv7\&q=85\&s=76fdf615a7caa99aef5e67f185381814) You can view the schema of your database by navigating to the "**Branches**" tab and selecting the branch you want to view. You may need to click "**Refresh schema**". For now, select `main`, and it will display the names of the two tables you just created. Click on the name of each table to see further schema details. ![Branches page with the products table expanded to show its schema](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/tutorials/branch-schema.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=03b874caa3898600faef8101cae0835c) ### Connect to your database Connect from outside the PlanetScale dashboard by creating a new role in the **Settings** -> **Roles** page, or open `psql` through the PlanetScale CLI. On the dashboard page, you can also click **Connect** to create a new role. **Dashboard** **Step 1** With the default branch selected, click **Connect** from your database dashboard. **Step 2** Click **Create new role**. A role determines which operations the connection can perform. See [Roles and credentials](/docs/neki/connecting/roles) for more details. **Step 3** Copy the generated `psql` command and enter it into your terminal. ![Connect page showing generated role credentials and the psql command](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/tutorials/connect-credentials.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=e984936fa0ec61f9763422ea93367fde) **CLI** Open a `psql` session through the PlanetScale CLI. `pscale shell` creates temporary credentials, so you do not need a saved role for this step. ```bash theme={null} pscale shell main ``` For a persistent role that your application can reuse, create one from the dashboard **Connect** tab or with `pscale role`. See [Roles and credentials](/docs/neki/connecting/roles). When the `postgres` prompt appears, query the data you added: ```sql theme={null} SELECT c.name AS category, p.name AS product FROM categories AS c JOIN products AS p ON p.category_id = c.id; ``` ### What's next? Open the **Connect** page and select a framework or language to configure your application. See [Connect to Neki](/docs/neki/connecting) for connection parameters, replica routing, and router groups. A new Neki database starts on one shard. Continue with the [Sharding quickstart](/docs/neki/tutorials/sharding-quick-start-guide) to see how sharding works in Neki. When you want to continue developing your database, create a [development branch](/docs/neki/branching) from `main` so you can work independently of the production branch. See [Development environments](/docs/neki/development-environments) to initialize an empty branch for schema work and tests. ### Clean up Delete your database from **Settings** > **Delete database**, or use the CLI: ```bash theme={null} pscale database delete ``` Deleting the database permanently removes all of its branches and data. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki sharding quickstart Source: https://planetscale.com/docs/neki/tutorials/sharding-quick-start-guide Add a shard and route a new table with a data topology In [Database quickstart](/docs/neki/tutorials/planetscale-quick-start-guide) you created a single-shard Neki database with `categories` and `products`. Those tables stay on the original shard while you add a second shard and create an `orders` table sharded by a `tenant_id` column. ## Prerequisites 1. You will need [a PlanetScale account](https://auth.planetscale.com/sign-up) 2. An administrator of your PlanetScale organization must [join the Neki Platform Preview](/docs/neki#availability-and-access). > [!NOTE] > > This quickstart configures the placement of an **empty** table. For a table that > already contains rows, plan a [data migration](/docs/neki/data-migration). ## Add a shard Use the same database and `main` branch from the database quickstart guide. Each new shard will use the selected configuration profile's cluster size and replica count. **Dashboard** **Step 1** Open the dashboard, then go to **Clusters** and select the `default` configuration profile. **Step 2** Open the **Shards** tab and select **Create new shards**. **Step 3** Enter `1` for the number of shards to add, review the estimated cost, and select **Add shards**. **Step 4** Wait until the new shard is ready. **CLI** **Step 1** Create one shard on the `default` configuration profile: ```bash theme={null} pscale branch shard create main \ --config-profile default \ --count 1 ``` **Step 2** List shards and wait until both report ready. ```bash theme={null} pscale branch shard list main ``` ![Clusters Shards tab showing two ready shards, sh1 and sh2](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/tutorials/clusters-shards.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=d9efd57998a20dfba80f63dbbb0f2894) ## Inspect the default topology Go to **Clusters** > **Data topology** to view and edit the JSON document in the dashboard. Alternatively, make the following changes with the PlanetScale CLI. A new Neki database starts with one unsharded shard group. That group is both the default and the [authoritative shard group](/docs/neki/data-topology#authoritative-shard-group). ```bash theme={null} pscale branch data-topology get main --format json ``` The document looks like this, with your first shard's UID in place of ``: ```json theme={null} { "shard_groups": [ { "uid": "", "key_ranges": [ { "shard_uid": "" } ] } ], "default_shard_group": "", "authoritative_shard_group": "" } ``` `categories` and `products` inherited the default shard group, so they stay on ``. The authoritative shard group must resolve to exactly one complete shard range. Neki uses that group for cluster-wide metadata and sequences, including identity columns. Changing it is a topology-wide operational change; verify that the new group is complete and available before applying the replacement document. Save a copy of the current document before you replace it. ## Create the `orders` table Create a schema for the `orders` table. Do not insert rows yet. ```sql theme={null} CREATE TABLE orders ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, tenant_id bigint NOT NULL, product_id bigint NOT NULL, quantity integer NOT NULL ); ``` ## Apply a two-shard topology Replace `` and `` with the shard `name` from `pscale branch shard list`. The `orders_by_tenant` group splits the keyspace at `80`. `orders` uses `xxhash` on `tenant_id`. Everything else keeps the original default. ```json theme={null} { "shard_indexes": { "xxhash_tenant_id": { "type": "xxhash", "columns": ["tenant_id"] } }, "shard_groups": [ { "uid": "", "key_ranges": [ { "shard_uid": "" } ] }, { "uid": "orders_by_tenant", "default_shard_index": "xxhash_tenant_id", "key_ranges": [ { "shard_uid": "", "end": "80" }, { "shard_uid": "", "start": "80" } ] } ], "databases": { "postgres": { "schemas": { "public": { "tables": { "orders": { "shard_group": "orders_by_tenant" } } } } } }, "default_shard_group": "", "authoritative_shard_group": "" } ``` Save that document as `data-topology.json` and apply it via the CLI, or paste the same document into **Clusters** > **Data topology** and select **Save changes**. ```bash theme={null} pscale branch data-topology update main \ --format json < data-topology.json ``` Confirm the resolved bindings: ```bash theme={null} pscale branch data-topology ls main ``` `orders` should list under `orders_by_tenant`. `categories` and `products` should still resolve to ``. See [Data topology](/docs/neki/data-topology) for shard-index types, defaults, the dashboard editor, and the SQL `__neki.set_data_topology` metafunction. ## Insert and check routing Insert two tenants. Neki hashes `tenant_id` and sends each row to the matching shard in `orders_by_tenant`. ```sql theme={null} INSERT INTO orders (tenant_id, product_id, quantity) VALUES (1, 1, 2); INSERT INTO orders (tenant_id, product_id, quantity) VALUES (2, 1, 1); ``` A predicate on `tenant_id` can narrow the query to one shard: ```sql theme={null} EXPLAIN (NEKI_PLAN, COSTS OFF, FORMAT TEXT) SELECT id, quantity FROM orders WHERE tenant_id = 1; ``` `Route [EqualUnique]` means the query can be sent to a single shard. ![Web console EXPLAIN (NEKI\_PLAN) showing Route EqualUnique on orders\_by\_tenant](https://mintcdn.com/planetscale-2/6i1lJ6cNfB-VSsaH/neki/tutorials/explain-equal-unique.png?fit=max\&auto=format\&n=6i1lJ6cNfB-VSsaH\&q=85\&s=b4147c5100672905108b2856f035aaa0) A query without that predicate can scatter across the group: ```sql theme={null} EXPLAIN (NEKI_PLAN, COSTS OFF, FORMAT TEXT) SELECT count(*) FROM orders; ``` `Route [Scatter]` means each shard computes a partial count, and Neki combines the results. `` still holds `categories` and `products`. Rows added to the `orders` table are now split across two shards. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Neki vs Vitess Source: https://planetscale.com/docs/neki/vitess-and-postgres Vitess concepts mapped to Neki's Postgres architecture If you already know Vitess, Neki will look familiar. A router sits in front of shards, configuration defines data placement, and live workflows can copy and stream data for operations such as resharding while the database continues serving traffic. Cutover, failures, and client reconnects can still produce transient errors that applications must handle. However, Neki is not Vitess with MySQL swapped for Postgres. Many parts of Vitess's design are MySQL-specific. Neki serves a similar purpose but implements routing and cluster operations for Postgres semantics. ## The shared model | | Vitess | Neki | | - | - | - | | Client protocol | MySQL | Postgres | | Query router | VTGate | [Router](/docs/neki/overview#routers) | | Database in a shard | MySQL | Postgres | | Routing model | Keyspaces, VSchema, and vindexes | [Data topology](/docs/neki/data-topology), shard groups, and shard indexes | | Data movement | VReplication | [Replicator](/docs/neki/replication) workflows | | Cluster operations | vtctld and related control plane | [Admin service](/docs/neki/overview#the-admin) | Both systems route queries, control fanout, coordinate primary changes, and move data while continuing to serve eligible traffic. The implementation for each system is designed with the underlying relational database in mind. ## Related comparisons * [Neki vs PlanetScale Postgres](/docs/neki/coming-from-postgres) * [PlanetScale Postgres vs Vitess](/docs/postgres-vs-vitess) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Web console Source: https://planetscale.com/docs/neki/web-console Run Postgres queries and DDL from the dashboard. ## Get started Run Postgres queries and DDL from the **Console** page. The console is available on any database branch. It is [disabled for production branches](#enable-for-production-branches) by default to protect production data. **Step 1** From the PlanetScale dashboard, select your Neki database. **Step 2** Select **Console**. **Step 3** Choose the branch and select **Connect**. Once connected, run queries or apply DDL against the branch. Use `EXPLAIN` or `EXPLAIN ANALYZE` in front of `SELECT`, `INSERT`, `UPDATE`, and `DELETE` statements to see how the database executes a query. ## Inspect database objects The console supports a subset of `psql` meta commands. Type `\?` or `help;` to show the complete list in the console. Use `\d` without a name to list tables, partitioned tables, views, materialized views, sequences, and foreign tables. Add a relation name to inspect its columns, data types, nullability, and defaults: ```text theme={null} \d public.orders ``` Use the more specific commands when you only want one type of object. For example, `\dt` lists tables and `\di` lists indexes and their definitions. Commands that accept a pattern support `*` for any number of characters and `?` for one character. You can also include a schema name: ```text theme={null} \dt public.* \di *created_at* ``` > [!NOTE] > > `\d` and `\dt` also show tables in Neki's internal `__neki` schema. Filter > with a pattern such as `public.*` to show only objects in your application > schema. Do not use the `__neki` schema for application objects. The console accepts `\d+`, but it returns the same information as `\d`. Other `psql` meta commands that are not listed below are not supported. ## Logical databases After connecting to a branch, use the **Database** dropdown in the console header to choose which logical database to connect to. The default database name is `postgres`. You can also switch databases from the console prompt with `\connect database_name` or `\c database_name`, just like in `psql`. Run `\l` to list the available databases on the branch. Switching databases reconnects the console and clears the current output. ## Primary and replica connections After connecting to a branch, use the **Server** dropdown in the console header to choose which instance to connect to: * **Primary** — Connect to the primary Postgres instance for read and write queries. * **Replica** — Connect to a replica for read-only queries. If a branch has no replicas, the console connects to the primary and the **Replica** option is unavailable. Switching between primary and replica reconnects the console and clears the current output. ### Connection roles The console creates a short-lived Postgres role for each connection. The inherited permissions depend on the selected server and your organization or database role. Tables created during a web console session are reassigned to the `postgres` role shortly after the session ends. Until reassignment completes, newly created tables may temporarily appear owned by the console's short-lived role. #### Development branches All organization roles (member, analyst, administrator) have full read and write access to development branch consoles on the primary. On a replica, the console always uses a read-only role. #### Production branches Production branch console access also requires the [web console setting](#enable-for-production-branches) to be enabled. Once enabled, access depends on your role: | Role | Console access | Primary permissions | | :- | :- | :- | | Organization or database **Administrator** | Read and write | `pg_read_all_data`, `pg_write_all_data`, `postgres` | | Organization or database **Analyst** | Read-only | `pg_read_all_data` only | | **Member** | No access | — | On a **replica**, the console always uses a read-only role with `pg_read_all_data`, regardless of your primary permissions. ## Supported console commands | Command | Description | | :- | :- | | ?, \\? | Synonym for `help` | | clear, reset | Clear the current input statement | | help, \h | Display list of commands | | ; | Send SQL statement to server | | \connect \[database], \c \[database] | Connect to a different logical database | | \d \[name] | List relations, or show a relation's columns, types, nullability, and defaults | | \dt \[pattern] | List tables and partitioned tables | | \dv \[pattern] | List views and materialized views | | \di \[pattern] | List indexes and their definitions | | \ds \[pattern] | List sequences | | \da \[pattern] | List aggregate functions | | \db \[pattern] | List tablespaces | | \dc \[pattern] | List casts | | \dconfig \[pattern] | List configuration parameters with non-default values | | \dD \[pattern] | List domains | | \det \[pattern] | List foreign tables | | \dn \[pattern] | List schemas | | \do \[pattern] | List operators | | \dO \[pattern] | List collations | | \dp, \z \[pattern] | List access privileges | | \dT \[pattern] | List data types | | \du \[pattern] | List roles | | \l \[pattern] | List databases | | \df \[pattern] | List functions | | \dx \[pattern] | List extensions | Commands with uppercase letters, including `\dD`, `\dO`, and `\dT`, are case sensitive. ## Keyboard shortcuts | Shortcut | Action | | :- | :- | | Up and Down | Move through query history when the cursor is on the first or last line of the current input | | Command+K on macOS or Ctrl+K elsewhere | Clear console output | | Command+Up and Command+Down on macOS, or Ctrl with the arrow keys elsewhere | Move between previous result blocks | | Up and Down while autocomplete is open | Select a suggestion | | Enter or Tab while autocomplete is open | Apply the selected suggestion | | Escape | Close the current autocomplete suggestions | ## Enable for production branches By default, the web console is disabled for production branches to prevent accidental data loss. You can enable the web console for production branches on the **Settings** page for the given database, `app.planetscale.com///settings`. Select the checkbox for **Allow web console access to production branches**, then scroll down and select **Save database settings** to save your changes. This enables the web console to run queries against production branches for the given database. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # When to shard Source: https://planetscale.com/docs/neki/when-to-shard Whether a Postgres workload needs horizontal sharding Horizontal sharding distributes a logical table's rows across Postgres shards, each with its own primary and replicas. Sharding is recommended when one primary remains a bottleneck, even after you tune Postgres configuration, query performance, indexes, and storage. Sharding allows you to spread data across multiple primary Postgres servers, alleviating both storage pressure and read and write throughput limitations of single-primary Postgres setups. ## What sharding changes | | Scale up | Shard horizontally | | - | - | - | | Unit of capacity | One primary | Many distinct primaries working together | | Write path | Every write hits a single primary | Writes are spread across shard primaries | | Row placement | One primary stores every row | The data topology assigns each row to a shard | | Query path | Postgres executes the query locally | Clients send queries to routers, which then route them to the appropriate shard | Each shard can be configured to have a different number of replicas. Replicas can provide failover capacity in a multi-node profile and can offload read traffic from the primary. A shard configured with no replicas is not highly available. ## Partitioning vs sharding [Postgres declarative partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html) splits a logical table into physical tables. In a typical Postgres deployment, it can improve performance and make data pruning cheaper. However, it does not remove a single-primary write bottleneck or let data exceed the capacity of one Postgres instance. Neki distributes a logical table across independent Postgres shards, each with its own primary, storage, and replication state. ## When sharding is the correct choice Sharding becomes a good next step for scaling when one or more of these are true: * The primary is persistently limited by write throughput or disk IOPS. * The working set of your relational data no longer fits within the RAM available on a single instance. * Independent shards would reduce the amount of data affected by one primary failure. * The workload has a stable routing key that keeps related data and common queries on one shard. A tenant key (such as a `customer_id` or `company_id`) often keeps each tenant's rows together in the query workload. ## Evaluate single-shard changes first Before sharding, evaluate whether configuration tuning, schema changes, query optimization, or a larger cluster size can remove the bottleneck. [Query Insights](/docs/neki/monitoring/query-insights) identifies expensive and frequently run queries. It can also surface [schema recommendations](/docs/neki/monitoring/schema-recommendations). These tools can help identify improvements for an existing Postgres database. In addition, [Metal](/docs/metal) uses locally attached NVMe and can provide higher I/O throughput and lower latency than network-attached storage. ## Design considerations Taking a database from unsharded to sharded requires an up-front decision about how to distribute the rows of logical tables. For predictable performance, design the topology so most queries reach one shard instead of requiring several shards to fulfill each query. The [data topology](/docs/neki/data-topology) controls which tables are distributed across which shards and how their rows are divided. Before moving data, validate the topology against the schema and the application's common access patterns. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale plans Source: https://planetscale.com/docs/planetscale-plans Vitess, Neki, and Postgres ## Overview PlanetScale is built to accommodate all companies at all stages. Whether you need a hassle-free managed database for your side project or you’re running millions of queries per second at the scale of YouTube, we have a solution for you. Our plans are split into two general offerings: [Base (self-serve)](#base-plan) and [Enterprise](#enterprise-plan). > **[Base](#base-plan)** > > Self-serve plan with fully-managed Vitess (MySQL-compatible), [Neki](/docs/neki), and Postgres databases. Features include sharding, branching, query insights, and more. Available with network-attached storage or Metal options. > > [Learn more](#base-plan) > **[Enterprise](#enterprise-plan)** > > Enhanced support, customizations, and deployment options. Available for Vitess, Neki, and Postgres with features like dedicated accounts, PCI compliance, and expert assistance. > > [Learn more](#enterprise-plan) ## Base Plan Our Base plan is completely self-serviceable. [Sign up for a PlanetScale account to get started](https://auth.planetscale.com/sign-up). PlanetScale offers **Vitess** (MySQL-compatible), **Neki**, and **Postgres**. All three are available with two storage options: **network-attached storage** and **Metal**. * [**Network-attached storage**](/docs/plans/planetscale-skus#network-attached-storage) (Amazon Elastic Block Storage or Google Persistent Disk) databases come with autoscaling storage and have varying levels of compute power. * [**Metal databases**](/docs/plans/planetscale-skus#metal) are backed by locally-attached NVMe drives for storage, unlocking incredible performance and cost-efficiencies. Because the drives are locally-attached, you need to choose both your compute and storage resources when you create your database. > [!NOTE] > > Cluster size options are capped to `PS-160` and `M-320` SKUs until you have a successfully paid an invoice of at least \$100. > > If you need larger sizes immediately, please [contact us](https://planetscale.com/contact) to unlock all sizes. On top of processing and memory, all **Base** cluster sizes share the following: | | **Vitess** | **Neki** | **Postgres** | | :- | :- | :- | :- | | **Storage/month (network-attached storage)** | 10 GB included; \$0.50 per instance per additional GB\* | 10 GB included on each primary and included replica volume; additional storage [varies by region](/docs/neki/pricing#storage) | 10 GB included; additional storage pricing [varies by region, cloud provider, and storage type](/docs/postgres/pricing#storage-pricing) | | **Storage/month (Metal)** | Depends on selected NVMe drive size | Depends on selected NVMe drive size | Depends on selected NVMe drive size | | **Available cluster sizes** | 22 | 22 | 22 | | **Availability zones** | 3 | 3 | 3 | | **Production branches** | 1 included\*\* | 1 included | 1 included | | **Development branches** | \~1,440 hours included (2× hours of current month) | Billed as a new branch | Billed as a new cluster | | **Concurrent Connections** | *Unmetered* | *Unmetered* | *Unmetered* | | **Single node** | Not available | Not available | Available starting at \$5/mo | | **Query Insights retention** | 7 days | 7 days | 7 days | | **Horizontal sharding** | Included | Included | Only in [Neki](/docs/neki) | | **Vertical sharding** | Included | Included | Included | | [**Deployment options**](/docs/plans/deployment-options) | Multi-tenant | Multi-tenant | Multi-tenant | | **Read-only regions** | Available as an add-on | Coming soon | Coming soon | | **Web console** | Included | Included | Included | | **PlanetScale CLI** | Included | Included | Included | | **Connection pooling** | [VTGates](/docs/vitess/scaling/vtgates) based on cluster size | [Routers](/docs/neki/overview#routers) | [PgBouncer](/docs/postgres/connecting/pgbouncer) | | [**Database Traffic Control®**](/docs/postgres/traffic-control) | Not available | Coming soon | Included | | **SSO** | Available as an add-on\*\*\* | Available as an add-on\*\*\* | Available as an add-on\*\*\* | | **Audit log retention** | 6 months | 6 months | 6 months | | **Private connections** | [AWS](/docs/vitess/connecting/private-connections) and [GCP](/docs/vitess/connecting/private-connections-gcp) | [AWS](/docs/neki/connecting/private-connections/aws-privatelink) and [GCP](/docs/neki/connecting/private-connections/gcp-private-service-connect) | [AWS](/docs/postgres/connecting/private-connections/aws-privatelink) and [GCP](/docs/postgres/connecting/private-connections/gcp-private-service-connect) | | **BAAs** | Included | Included | Included | | **Automatic backups** | Every 12 hours | Every 12 hours | Every 12 hours | | **Support** | Standard, upgrade available\*\*\* | Standard, upgrade available\*\*\* | Standard, upgrade available\*\*\* | | [**Data Branching®**](/docs/vitess/schema-changes/data-branching) | Included | Not available | Not available | \* For HA network-attached storage databases, production branch storage is billed at \$1.50/GB (1 primary + 2 replicas) and development branch storage is billed at \$0.50/GB (1 primary). \*\* Additional production branches are billed at the cost of your selected cluster size per month. \*\*\* [SSO](/docs/security/sso) and [Business support](/docs/support#business) options are available on the Base plan for an additional fee. ### Additional production branches Each HA production branch in the Base plan provisions a separate, production database cluster in our infrastructure. Upon adding an additional production branch, you will be prompted to select the cluster size for the new branch. Each cluster size is priced based on the selected region. See [Vitess](/docs/plans/cluster-sizing), [Neki](/docs/neki/pricing), and [Postgres](/docs/postgres/pricing) pricing. If you have a `main` network-attached storage production branch using the **PS-40** cluster size and two additional production branches using the **PS-20** cluster size, the total cost for the database would be **\$217.00** per month. | **Production branch cluster** | **Cost per unit** | **Quantity** | **Total per month** | | :- | :- | :- | :- | | PS-40 | \$99.00 | 1 | \$99.00 | | PS-20 | \$59.00 | 2 | \$118.00 | | **Grand total** | | | **\$217.00** | Also note that pricing is prorated. If you create a new database in the middle of a billing cycle, you will only be charged for the appropriate fraction of the month. This also applies to changes to an existing database, such as upsizing. For example, if you have a database that started the month as a `PS-10` and a resize to `PS-20` completes halfway through the month, you would be charged \$39/2 + \$59/2 = \$49 (assuming no other charges for storage, etc.). Billing for the new size begins when the cluster resize completes. ### Development branches Billing for development branches differs depending on whether you're using Vitess, Neki, or Postgres. ### Vitess development branches Development branches, `PS-DEV` are billed for the time that they are running, prorated to the millisecond. Databases include `hours_in_current_month * 2` of development branch time per month (1,440 hours for a 30 day month) at no additional cost. Any time used over the included is billed at a rate of \~\$0.014 per hour (\$10 / hours\_in\_current\_month). You may see how many development branch hours have been used at any time by visiting your [organization billing page](https://app.planetscale.com/~/settings/billing/). Data is updated hourly. ### Neki and Postgres development branches Postgres development branches, `PS-DEV`, are billed for the time that they are running, prorated to the millisecond. Each development branch is \$5 per month. Development branches are not intended for HA production traffic, as they do not come with any replicas or have maintenance windows. > [!NOTE] > > You can set spend email alerts from your billing page. See the [Spend management documentation](/docs/billing#spend-management) for more information. If a database is created in the middle of a billing cycle, the included development branch hours are prorated. For example, if you create your database with 15 days remaining in the current month, the database will have `15 days * 2` (720 hours) included for that billing period. ### Fractional vCPU allocation Some tiers of the Base plan indicate a fractional vCPU allocation. These branches run on our multi-tenant platform and this indicates the minimum number of cycles dedicated to your workload. The vast majority of the time, there are spare compute cycles available on the underlying machine instances hosting your branch, and those are available to be used by your workload as needed for no additional charge. If you find the performance of a given query to be substantially inconsistent over the course of a given day, you may want to upgrade to a higher tier for more consistent performance. ## Enterprise plan PlanetScale's Enterprise plan is great for large-scale businesses who require enterprise-level SLAs, want expert assistance through enterprise support, and would prefer to run in their own AWS, GCP, or Azure account. We offer many different deployment options, all of which come with the same set of standard features. * **PlanetScale Single-tenant** runs in PlanetScale's infrastructure with optional dedicated AWS, GCP, or Azure accounts. * [**PlanetScale Managed**](/docs/plans/managed) is a deployment option where your database runs entirely in your own AWS, GCP, or Azure account, giving you complete control over your cloud infrastructure while still benefiting from PlanetScale's fully managed service. The table below covers those shared features, as well as the different options that vary depending on your chosen deployment. | | Single-tenant | **[PlanetScale Managed](/docs/plans/managed)** | | :- | :- | :- | | Customizable feature limits | | | | [Support](/docs/support) | Business — Enterprise upgrade available | Enterprise | | PCI compliant | | | | Dedicated cloud account, project, or subscription | Optional | | | Bring your own cloud account, project, or subscription | | | | Billing | Directly from PlanetScale | Partial payment through PlanetScale and infrastructure costs through your cloud provider. Take advantage of your own discounts. Optionally, purchase through [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-luy3krhkpjne4). | ## How do I know if I need an Enterprise plan? If you’re not sure whether your use case requires an Enterprise plan, we’re more than happy to chat with you to figure it out together. You can [fill out this form](https://planetscale.com/contact), and we’ll be in touch. In general, if you need any of the following, Enterprise may be the best solution for you: * Enhanced support — our expert team becomes an extension of your own. Additional options for technical account management, Slack-based support, and phone escalation. * You need additional support to horizontally shard and migrate your database(s) to PlanetScale * You need your database deployed in a single-tenant environment * You need to keep your data in **your own** AWS, GCP, or Azure account * You need a PCI DSS certified service provider * Mission-critical [response times](/docs/support#initial-response-times) including continuous support coverage * Any other customizations — Our Enterprise plans offer a lot of flexibility, so if you have a requirement that’s not listed here, it’s best to [reach out](https://planetscale.com/contact) and we can see how we can help ## Plan add-ons ### Single Sign-on (SSO) Add [Single Sign-on (SSO)](/docs/security/sso) for your organization to the Base plan for an additional fee of \$199/month. ### Business support Add [Business support](/docs/support#business) to the Base plan for an additional fee. See the Support page for more details. ### User-scheduled backups On the Base plan, we run automated backups every 12 hours. Disk space for default backups is not counted against your plan's storage limit. You can also [schedule additional backups yourself](/docs/vitess/backups#create-manual-backups) as needed. For these additional user-scheduled backups, storage is billed at **\$0.023 per GB** per month. Backups include system tables as well as your data and start at around 140 MB. ## Free plan PlanetScale does not offer a free plan, previously known as the "Developer" or "Hobby" plan. All databases require a paid subscription starting with our Base plan, which provides self-service access to fully-managed database clusters. Single-node Postgres databases are available starting at \$5 per month. ## Scaler Pro plan The Scaler Pro plan has been renamed to the Base plan. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster sizing Source: https://planetscale.com/docs/plans/cluster-sizing You can easily upsize and downsize your database cluster from within the PlanetScale dashboard. This documentation covers some information about selecting a cluster size upon database creation as well as how to upsize and downsize. > [!NOTE] > > If you are on a consumption commitment plan, please be aware that any changes in cluster size will be reflected against your monthly or annual consumption commitment amount. Changes to the originally selected cluster size may cause you to utilize this amount either more quickly or slowly. If you have further questions, please reach out to your account manager or our [Support](https://planetscale.com/contact) team. [Resize your database](https://www.youtube-nocookie.com/embed/y94VExata3A?rel=0) ## Selecting a cluster size Selecting the correct cluster size for your database can have a dramatic impact on how it performs and how much it costs. A good rule of thumb is when you notice CPU usage is consistently at or close to 100% for an extended period of time, you may benefit from [upsizing your cluster](/docs/plans/planetscale-skus#upsizing-and-downsizing-clusters). Conversely, if your CPU usage is consistently below 50%, you may be able to downsize. You can monitor your CPU usage by clicking on your database, clicking "Primary" in your architecture diagram, and referencing the chart under "Metrics and performance". > [!WARNING] > > For Metal instances, you have to consider both the compute and the storage, as storage does not autoscale. For more information about adjusting a Metal instance, see [Upgrading an existing database to Metal](/docs/metal/create-a-metal-database#upgrading-an-existing-database-to-metal). There are also special cases where you may want to temporarily upsize out of caution if you're anticipating a large spike in traffic, such as during a launch or event. In these cases, you can easily [upsize](/docs/plans/planetscale-skus#upsizing-and-downsizing-clusters) ahead of your event, and then downsize after. If you are switching between [network-attached storage](/docs/plans/planetscale-skus#network-attached-storage) (Amazon Elastic Block Storage or Google Persistent Disk) and [Metal](/docs/plans/planetscale-skus#metal), or changing the size of your Metal instance, be aware this switch takes additional time. ### Comparing PlanetScale to other database providers If you are migrating from an existing cloud provider with resource-based pricing, be sure to compare your currently selected instance with our available cluster sizes. Keep in mind, each database comes with a production branch with two replicas. Vitess databases include 1,440 hours worth of development branches. The development branches essentially equate to two extra "always on" databases. In many cases, you can deprecate your dev/staging databases that you pay extra for with other providers in favor of the development branches. In the end, this usually results in significant cost savings. Databases in PlanetScale also come with additional beneficial infrastructure that is not easily configured or available in other hosted database solutions. For more information on what is provisioned with each database, read our [Vitess Architecture](/docs/vitess/architecture) and [Postgres Architecture](/docs/vitess/architecture) docs. If you are unsure which plan or cluster size is right for your application, [contact us](https://planetscale.com/contact) to get further assistance. Our self-serve plans are flexible enough to handle the majority of customers. However, there are several use cases where you may need a more custom plan. This is where our Enterprise offerings shine. ## Sharding with Vitess You can create sharded Vitess keyspaces on any plan by adding a new sharded keyspace using the [cluster configuration page](/docs/vitess/cluster-configuration) and moving tables into it with [MoveTables](/docs/vitess/sharding/sharding-quickstart). Sharded keyspaces can [size shards independently](/docs/vitess/cluster-configuration/per-shard-sizing) after you enable per-shard sizing. If you would like additional support from our expert team, our [Enterprise plan](/docs/planetscale-plans#enterprise-plan) may be a good fit. [Get in touch](https://planetscale.com/contact) for a quick assessment. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Deployment options Source: https://planetscale.com/docs/plans/deployment-options PlanetScale offers two deployment options to accommodate your application and business needs: multi-tenancy and single-tenancy. Vitess and Postgres ## Overview **Multi-tenancy** — In a multi-tenant environment, your database is hosted on infrastructure shared with other users/customers. **Single-tenancy** — In a single-tenant environment, your database is hosted on dedicated infrastructure that is not shared with any other users/customers. Your deployment options depend on the [PlanetScale plan](/docs/planetscale-plans) you choose. This documentation covers some differences between single-tenancy and multi-tenancy so you can evaluate which plan may be suitable for you. For an in-depth look at the different plans we offer, see the [PlanetScale plans documentation](/docs/planetscale-plans). ## Multi-tenancy deployment on PlanetScale Multi-tenancy is the default deployment option. When you sign up for a PlanetScale account with the Base plan, your databases will be created in our multi-tenancy deployment offering. Our Enterprise plan also offers a multi-tenancy deployment option. ### Multi-tenancy highlights * Your database runs on PlanetScale's [secure cloud infrastructure](/docs/security) * Lowest cost plans * No configuration requirements on your end * BAAs available for HIPAA compliance * [Private connection support](/docs/vitess/connecting/private-connections) via AWS PrivateLink * [Private connection support](/docs/vitess/connecting/private-connections-gcp) via GCP Private Service Connect ## Single-tenancy deployment on PlanetScale Single-tenant deployment options are available with [PlanetScale Enterprise](/docs/planetscale-plans#enterprise-plan). Companies that require their PlanetScale databases to be hosted in a single-tenant environment have two options: Enterprise — Single-tenant and Managed. > [!NOTE] > > In both options, your database is deployed in a single-tenant environment. The main difference between Enterprise — Single-tenant and Managed is who owns the underlying account. With [Managed](/docs/plans/managed), your database is deployed in your own AWS, GCP, or Azure account. With Enterprise — Single-tenant, PlanetScale owns and manages the account. > > If you're interested in learning more, please [reach out](https://planetscale.com/contact), and we can figure out the best solution for your use case. ### Single-tenancy highlights * Your resources run in a separate isolated AWS account, GCP project, or Azure subscription, apart from other customers * Your databases can be deployed to supported cloud provider regions that offer three availability zones, including regions that PlanetScale does not offer on our self-serve plans * Azure does not make every PlanetScale-supported VM SKU available to every subscription. Coordinate region selection with your PlanetScale Solutions Engineer. PlanetScale may ask you to verify that your dedicated subscription has access to the required VM SKUs and sufficient quota across three availability zones in each requested region. * You can continue to use the PlanetScale UI and CLI in the same way that you would our general self-serve offerings, but the infrastructure runs in an isolated environment * BAAs available for HIPAA compliance * Support for private database connectivity via: * **AWS** — [AWS PrivateLink](/docs/plans/managed/aws/privatelink) (recommended) or [VPC peering](/docs/plans/managed/aws/vpc-peering) * **GCP** — [Private Service Connect](https://cloud.google.com/vpc/docs/private-access-options) * **Azure** — [Azure Private Link](/docs/plans/managed/azure/private-link) If you have any questions about which deployment option may be best for you, [reach out to us](https://planetscale.com/contact) and we can help you figure it out. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Managed overview Source: https://planetscale.com/docs/plans/managed Run PlanetScale in your own AWS, GCP, or Azure account, operated by our team with your data in your cloud. Vitess, Neki, and Postgres ## What is PlanetScale Managed? PlanetScale Managed is a single-tenant deployment of PlanetScale within your Amazon Web Services (AWS), Google Cloud Platform (GCP), or Microsoft Azure account. In this configuration, you can use the same API, CLI, and web interface that PlanetScale offers, with the benefit of running entirely in your own cloud account. We have packaged the best parts of PlanetScale into a container and can deploy and operate them in your own account, bringing you the best of SaaS with the added benefit of a deployment free of noisy neighbors, enhanced support, and additional security guarantees. With PlanetScale Managed, it is more than just an on-premises deployment of your database; you are getting the PlanetScale expert team operating your database alongside your team for a *truly* fully managed database solution. The PlanetScale team is on-call for your databases. ## How does PlanetScale Managed work? PlanetScale Managed is a packaged [data plane](https://en.wikipedia.org/wiki/Data_plane), deployed to an AWS Organizations member account, GCP project, or Azure subscription that you own and we operate. PlanetScale Managed supports [Vitess](/docs/vitess), [Neki](/docs/neki), and [Postgres](/docs/postgres). Existing Managed customers should [contact us](https://planetscale.com/contact) to enable Neki. For MySQL databases, the data plane is built on [Vitess and Kubernetes](https://planetscale.com/blog/scaling-hundreds-of-thousands-of-database-clusters-on-kubernetes). For Postgres databases, it runs a dedicated PostgreSQL cluster orchestrated by Kubernetes. Your database lives entirely inside a dedicated account, project, or subscription within your cloud organization. PlanetScale will not have access to other accounts, projects, or subscriptions nor your organization-level settings within the cloud service provider. At the same time, you still get to interact with your databases through the web application, pscale CLI, or the PlanetScale API, as you usually would with our hosted product. This includes developer experience features such as non-blocking schema changes, safe migrations, database branching, query insights, and more. If you are an existing PlanetScale user, moving to PlanetScale Managed requires no changes to your existing developer workflows. The database is deployed in a single-tenant environment and isolated in an AWS Organizations member account, GCP project, or Azure subscription from the rest of your organization's infrastructure. By default, all connections are encrypted, but public. PlanetScale Managed supports private database connectivity through [AWS PrivateLink](/docs/plans/managed/aws/privatelink), [GCP Private Service Connect](/docs/plans/managed/gcp/private-service-connect), or [Azure Private Link](/docs/plans/managed/azure/private-link). Private connections are also available for supported databases on the [Base plan](/docs/planetscale-plans#base-plan). Read more about PlanetScale Managed on each cloud provider: > **[PlanetScale Managed on Amazon Web Services](/docs/plans/managed/aws)** > **[PlanetScale Managed on Microsoft Azure](/docs/plans/managed/azure)** > **[PlanetScale Managed on Google Cloud Platform](/docs/plans/managed/gcp)** ## Benefits of PlanetScale Managed Single-tenancy is one of many benefits when it comes to PlanetScale Managed. Still, with this PlanetScale Enterprise service, you also get: * Assistance with [horizontal sharding](/docs/vitess/sharding) for MySQL and [Neki](/docs/neki) for Postgres * Deployment to additional regions * [PCI compliance](/docs/security#pci-compliance) * Additional [support options](/docs/support#enterprise) * Available on [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-luy3krhkpjne4). Your PlanetScale purchase through the AWS Marketplace and the resources you use on PlanetScale will qualify against your EDP commitment. ## How do I get PlanetScale Managed? If you are interested in seeing if PlanetScale Managed fits your needs, [contact us](https://planetscale.com/contact), and we can chat more about your requirements and see if PlanetScale Managed is a good fit for you. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Managed on AWS overview Source: https://planetscale.com/docs/plans/managed/aws PlanetScale Managed on Amazon Web Services (AWS) is a single-tenant deployment of PlanetScale in your AWS organization within an isolated AWS Organizations member account. Vitess, Neki, and Postgres ## Overview In this configuration, you can use the same API, CLI, and web interface that PlanetScale offers, with the benefit of running entirely in an AWS Organizations member account that you own and PlanetScale manages for you. ## Architecture The PlanetScale data plane is deployed inside of a PlanetScale-controlled AWS Organizations member account in your AWS organization. The database cluster will run within this member account, orchestrated via Kubernetes. PlanetScale Managed supports Vitess (MySQL-compatible), Neki, and Postgres clusters. Existing Managed customers should [contact us](https://planetscale.com/contact) to enable Neki. We distribute components of the cluster across three AWS availability zones within your selected region to ensure high availability. You can deploy PlanetScale Managed to any AWS region with at least three availability zones, including those not supported by the PlanetScale self-serve product. Backups, part of the data plane, are stored in S3 inside the same member account. PlanetScale Managed uses isolated Amazon Elastic Compute Cloud (Amazon EC2) instances as part of the deployment. ![Architecture diagram for PlanetScale Managed in AWS](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/aws-arch-diagram.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=36a9f1e69370e041a7ee576e3f0f42ba) Your database lives entirely inside a dedicated AWS Organizations member account within your AWS organization. PlanetScale will not have access to other member accounts nor your organization-level settings within AWS. Outside of your AWS organization, we run the PlanetScale control plane, which includes the PlanetScale API and web application, including the dashboard you see at `app.planetscale.com`. In a MySQL deployment, the Vitess cluster running inside Kubernetes is composed of a number of Vitess components. All incoming queries are received by one of the **VTGates**, which then routes them to the appropriate **VTTablet**. The VTGates, VTTablets, and MySQL instances are distributed across 3 availability zones. ![Diagram of Vitess cluster on AWS](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/aws-vitess.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=8963ce811fda1505db16f995ea7d0d26) Several additional required Vitess components are run in the Kubernetes cluster as well. The topology server keeps track of cluster configuration. **VTOrc** monitors cluster health and handles repairs, including managing automatic failover in case of an issue with a primary. **vtctld** along with the client **vtctl** can be used to make changes to the cluster configuration and run workflows. In a Postgres deployment, the architecture uses a dedicated PostgreSQL cluster with similar high-availability and distribution across availability zones. ## Security and compliance PlanetScale Managed is an excellent option for organizations with specific security and compliance requirements. You own the AWS organization and member account that PlanetScale is deployed within an isolated architecture. This differs from when your PlanetScale database is deployed within our AWS organizations. > [!NOTE] > > PlanetScale manages the entire member account and can NOT support customers running Terraform or other configuration management in the member account. ### PCI compliance Along with System and Organization Controls (SOC) 2 Type 2 and other [security and compliance](/docs/security) practices that PlanetScale has been issued and follows, PlanetScale Managed has been issued an Attestation of Compliance (AoC) and Report on Compliance (RoC), certifying our compliance with the PCI DSS 4.0 as a [Level 1 Service Provider](https://www.pcisecuritystandards.org/glossary/service-provider/). This enables PlanetScale Managed to be used via a shared responsibility model across merchants, acquirers, issuers, and other roles in storing and processing cardholder data. > [!NOTE] > > If you have any questions or concerns related to the security and compliance of PlanetScale Managed, please [contact us](https://planetscale.com/contact), and we will be happy to discuss them further. ### Private database connectivity By default, all database connections are encrypted but use public endpoints. [AWS PrivateLink](/docs/plans/managed/aws/privatelink) and [AWS VPC peering](/docs/plans/managed/aws/vpc-peering) keep database traffic on private AWS networks, but they use different network models. | | AWS PrivateLink | AWS VPC peering | | - | - | - | | Connectivity | Exposes a specific service through interface endpoints | Routes private traffic directly between two VPCs | | CIDR requirements | Overlapping VPC CIDR blocks are supported | VPC CIDR blocks must not overlap | | Routing | No route to the service VPC is required | Both VPC owners maintain routes | | Access scope | Connectivity is limited to the endpoint service | Routes and security controls determine which resources can communicate | | Topology | Each VPC uses its own interface endpoint | One-to-one and non-transitive; each additional VPC needs its own peering connection | | DNS | Uses PrivateLink private DNS | Requires a cross-account private hosted zone association | | AWS charges | Hourly charges per interface endpoint and Availability Zone, plus data processing charges | No hourly peering connection charge; data transfer charges can apply | AWS PrivateLink is the recommended default because it exposes only the database endpoint and does not require non-overlapping CIDR blocks. VPC peering can be useful when your architecture requires direct, routed connectivity between the application VPC and the Managed VPC. Contact your PlanetScale Solutions Engineer to choose an option. See the AWS documentation for current [AWS PrivateLink pricing](https://aws.amazon.com/privatelink/pricing/) and [VPC peering pricing](https://aws.amazon.com/vpc/pricing/). ### Fully private network isolation You can also turn off public database access with a dual AWS PrivateLink setup. PlanetScale's control plane will talk to your member account over AWS PrivateLink, and your VPCs will also communicate with your database over AWS PrivateLink. Please get in touch with your PlanetScale Account Manager for more information on how to set up fully private network isolation. ### Third-account customer-controlled public key infrastructure PlanetScale Managed on AWS supports public key infrastructure (PKI) services. PlanetScale Managed customers can provide PlanetScale the use of a set of customer-managed keys in a third AWS account inside your organization. This third account is controlled by you, the customer. PlanetScale has no administrative access. Your organization is the custodian for this key material. PlanetScale uses the customer-managed keys to encrypt EBS volumes, S3 buckets, and for envelope encryption of backups. ## Billing With any of the PlanetScale Enterprise offerings, including PlanetScale Managed, you have the option to purchase PlanetScale through the [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-luy3krhkpjne4). In addition to this, the resources you use on PlanetScale will qualify against your EDP commitment. > [!NOTE] > > If you have any billing-related questions for PlanetScale Managed, please [contact us](https://planetscale.com/contact), and we will be happy to discuss them further. ## Getting started with PlanetScale Managed in AWS If you want to see what is involved in getting set up with PlanetScale Managed in AWS, you can see the [AWS set up documentation](/docs/plans/managed/aws/getting-started). If you are interested in exploring PlanetScale Managed further, please [contact us](https://planetscale.com/contact), and we can chat more about your requirements and see if PlanetScale Managed is a good fit for you. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Back up and restore in AWS Source: https://planetscale.com/docs/plans/managed/aws/back-up-and-restore PlanetScale Managed backup and restore functions like the hosted PlanetScale product. For more info, see [how to create, schedule, and restore backups for your PlanetScale databases](/docs/vitess/backups). To learn more about the backup and restore access levels, see the [database level permissions documentation](/docs/security/access-control#database-level-permissions). By default, databases are automatically backed up once per day to an S3 bucket in the customer's AWS Organizations member account. This default can be adjusted when working with PlanetScale Support. Configuring and validating additional backup frequencies is the customer's responsibility. During the initial provisioning process, PlanetScale applies an S3 configuration to ensure that backups are encrypted at rest on Amazon S3. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Set up PlanetScale Managed in AWS Source: https://planetscale.com/docs/plans/managed/aws/getting-started The following guide will walk you through setting up a PlanetScale Managed cluster in your Amazon Web Services (AWS) organization. ## Overview If you have any questions while working through this documentation, contact your PlanetScale Solutions Engineer for assistance. > [!NOTE] > > This guide is only intended for PlanetScale Managed customers currently working with the PlanetScale team. You cannot set PlanetScale Managed up on your own without PlanetScale enabling it for your organization. If you are interested in [PlanetScale Managed](/docs/plans/managed), please [contact us](https://planetscale.com/contact). ## Step 1: Account requirements A new AWS Organizations member account must be set up following this documentation to successfully bootstrap a new PlanetScale Managed cluster. An existing AWS organization is required to proceed with this guide. ### Dedicated AWS Organizations member account PlanetScale Managed requires the use of a standalone AWS Organizations member account in Amazon Web Services. This account should not have any existing resources running within it. The [creating a member account in your organization document](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts_create.html#orgs_manage_accounts_create-new) covers how to create a new member account in an existing AWS organization. The document also includes the required permissions to create a member account in your AWS organization. ### Modification of accounts Once the member account is handed over to PlanetScale via granting IAM permissions, it should not be modified. Issues caused by modifications of the member account or its resources void the PlanetScale Managed SLA. [Contact support](https://planetscale.com/contact?initial=support) to discuss configuration changes or customization. ### Recommendations During the initial provisioning process, PlanetScale applies the following recommendations to the AWS Organizations member account and we recommend that a customer not change these once the member account has been created: * **Encryption by default:** PlanetScale enables EBS encryption by default using the AWS-managed keys in the relevant regions in the member account. If you want to change this behavior, please consult PlanetScale before the initial deployment process. * **AWS CloudTrail + AWS Config:** Enable AWS CloudTrail for management events and resource tracking using AWS Config. ### PCI Compliance Customers of PlanetScale Managed should ensure the following additional configurations are applied and maintained to ensure that the customer environment remains PCI-compliant for the storage and protection of cardholder data: #### Local Authentication Parameters PlanetScale does not set specific authentication parameters for accessing the customer database during initial provisioning. To maintain compliance with PCI requirements 8.2 and 8.3, it is the customer's responsibility to set the following authentication parameters in line with the respective requirements set by the PCI Data Security Standards: * Account lockout * Account lockout duration * Password minimum length * Password complexity * Password expiration * Password history #### Log Level Configuration The PlanetScale-controlled AWS Organizations member account will be pre-configured by PlanetScale with [AWS CloudTrail](https://aws.amazon.com/cloudtrail/) enabled and configured to emit logging events from the customer application. As PlanetScale does not retain access to these logs after the account is configured, to maintain compliance with PCI requirement 10.2.1.1 (audit logs capture all individual user access to cardholder data), it is the customer's responsibility to ensure this logging remains enabled and to regularly review and verify the following events: * All administrative action * Accessing cardholder data * Accessing audit trails * Invalid access attempts * Successful access attempts * Elevation of privileges * Creation/deletion/changing an account with admin privileges * Start/stop/pausing of audit logs As a best practice, it is recommended that these logs be captured and continuously analyzed by a Security Information & Event Management (SIEM) platform. ## Step 2: Bootstrap with CloudFormation We've created a CloudFormation template to complete the setup of required permissions in your AWS Organizations member account. Save the following as `planetscale-bootstrap.json`: ```json expandable theme={null} { "Resources": { "GrantPlanetscaleBootstrapAccess": { "Type": "AWS::IAM::Role", "Properties": { "RoleName": "PlanetscaleBootstrap", "ManagedPolicyArns": ["arn:aws:iam::aws:policy/AdministratorAccess"], "AssumeRolePolicyDocument": { "Statement": [ { "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::313573332105:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_Ops_feec88bc3aad314d" ] }, "Action": ["sts:AssumeRole"] } ] } } } } } ``` Next, apply the CloudFormation template as a new stack: ```bash theme={null} aws cloudformation create-stack --stack-name planetscale-bootstrap \ --template-body file://planetscale-bootstrap.json \ --capabilities CAPABILITY_NAMED_IAM ``` Let your Solutions Engineer know once the new stack reaches the `CREATE_COMPLETE` state in AWS. This is a temporary stack used only for initial provisioning and you will clean it up later. ## Step 3: Requesting an initial quota increase By default, AWS may provision new member accounts with EC2 On-Demand quotas that may be too small for: * PlanetScale's initial provisioning process * The databases you may want to provision on your PlanetScale Managed cluster Although the PlanetScale Support and Operations teams will have the ability to request quota increases on your behalf after you give us access to the AWS Organizations member account, we recommend that you review the following quotas and request increases as necessary, as requesting quota increases later will delay the process: * [Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances](https://console.aws.amazon.com/servicequotas/home/services/ec2/quotas/L-1216C47A) — Since PlanetScale Managed typically runs small instances by default, it is generally best to set this high enough to avoid any later issues. At least **300** is sufficient for most customers. * [Storage for General Purpose SSD (gp3) volumes, in TB](https://console.aws.amazon.com/servicequotas/home/services/ebs/quotas/L-7A658B76) — Note that we typically will keep 3 copies of all data (primary plus 2 replicas), so you have to consider that here. We will also create volumes at backup time, which could be a temporary 4th copy for quota purposes. **50** TB should be sufficient for most customers. * [Storage modifications for General Purpose SSD (gp3) volumes, in TB](https://console.aws.amazon.com/servicequotas/home/services/ebs/quotas/L-59C8FC87) — Ensure this is large enough, if possible, to cover your largest database so that storage volume performance modifications can be made (if necessary), without replacing volumes. Again, **50** TB or more should be sufficient in most cases. You can read more about how to request a quota increase in the [AWS requesting a quota increase documentation](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html). > [!NOTE] > > If you have AWS Enterprise Support, you can contact your account manager to expedite quota requests; otherwise, quota requests above your current limit can take at least one business day. There is also a limit on how often you can make quota requests. A quota request can only be made once every 6 hours. ## Step 4: Initiating the provisioning process Once the CloudFormation stack has returned as `CREATE_COMPLETE`, notify your Solutions Engineer, providing them the following information: * The name of the organization that you have created on `app.planetscale.com`. * The AWS Account ID of the member account, which can be found by using one of the choices in the [AWS account ID and alias documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html). * A confirmation of the region(s) that you have chosen for the deployment to reside in. The canonical list of regions can be found in the [AWS Regions and Zones documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions). Once your Solutions Engineer receives this information, they will forward it to the team responsible for provisioning your deployment. Provisioning the deployment takes PlanetScale, on average, one business day. Once the deployment has been provisioned, your Solutions Engineer will contact you to confirm that your team can start creating databases. Finally, delete the temporary CloudFormation stack now that it's no longer needed. During provisioning, PlanetScale creates other, permanent, IAM resources to manage your deployment. ```bash theme={null} aws cloudformation delete-stack --stack-name planetscale-bootstrap ``` > [!NOTE] > > Private database connectivity is available through [AWS PrivateLink](/docs/plans/managed/aws/privatelink) or [AWS VPC peering](/docs/plans/managed/aws/vpc-peering). Contact your Solutions Engineer to choose an option before provisioning. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Set up AWS PrivateLink with PlanetScale Managed Source: https://planetscale.com/docs/plans/managed/aws/privatelink PlanetScale Managed can connect you to your databases via [AWS PrivateLink](https://aws.amazon.com/privatelink/). The following guide describes how PlanetScale Managed with AWS PrivateLink works and how to set it up. Vitess and Postgres ## Overview If you are on the Base plan and would like to set up AWS PrivateLink, see our [Private connections documentation](/docs/vitess/connecting/private-connections). ## How PlanetScale Managed and AWS PrivateLink work AWS PrivateLink requires two components: * A [VPC endpoint service](https://docs.aws.amazon.com/vpc/latest/privatelink/endpoint-service-overview.html) deployed in the AWS Organizations member account that PlanetScale controls. * A [VPC endpoint interface](https://docs.aws.amazon.com/vpc/latest/privatelink/vpce-interface.html), sometimes referred to as a "VPC endpoint" in AWS, deployed in the account that your applications operate in. Once both components are operating correctly, the EC2 instances in the VPC that the VPC endpoint has been assigned to will leverage [Private DNS](https://docs.aws.amazon.com/vpc/latest/privatelink/vpce-interface.html#vpce-private-dns) to connect to your VPC endpoint instead of the publicly accessible endpoint. The connection strings that PlanetScale provides will operate successfully inside and outside your VPC, creating PrivateLink connections inside of your VPC and regular connections outside of your VPC. ## Step 1: Initiating the setup process There is no fully automated way to establish a PrivateLink connection. If you would like to initiate the process, please get in touch with your Solutions Engineer and let them know the [AWS Account ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html) that you intend to create the [VPC endpoint](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints.html) in. Once they receive your AWS Account ID and forward it to the team responsible for provisioning your deployment, the team will provide the Solutions Engineer (and ultimately you) with the Service Name of the [VPC endpoint service](https://docs.aws.amazon.com/vpc/latest/privatelink/endpoint-service-overview.html) that will be responsible for accepting your connection. > [!NOTE] > > It is important to keep the service name in your records. It is the only piece of information you need to input when creating your VPC endpoint. ## Step 2: Establishing a VPC endpoint connection > [!WARNING] > > Only proceed to the next steps once a PlanetScale Solutions Engineer has provided the service name and confirmed cross-account authentication has been configured. The following steps are an example of establishing a VPC endpoint connection in the AWS Console. In this example, the customer has requested that their deployment be in the `eu-west-1` region. When you go through the steps, make sure that you have selected the region that matches the region that your PlanetScale Managed cluster deployment has been provisioned into. **Step 1** Navigate to the Endpoints section on the Virtual Private Cloud page and select "**Create Endpoint**." ![nav\_to\_splash](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/nav_to_splash.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=f47c7cb8652a121d20c31645fa65d932) **Step 2** Select the "**Find service by name**" selector, input the provided Service Name, and select the "**Verify**" button. ![verified](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/verified.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=469e7a63f0d2cf0d2a858c3535c6a07a) **Step 3** Select the VPC in the drop-down where you wish to provision this VPC endpoint and the relevant subnets inside your VPC. ![cyo\_vpc](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/cyo_vpc.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=66a5cd7586905f3f8a102116147a12f7) **Step 4** Select the "**Enable DNS Name**" checkbox. Take note of the value of your "**Private DNS Name**" field. That is how we will verify that the connection is operating successfully. ![enable\_dns\_name](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/enable_dns_name.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=cf722a66139f93210f5ccbfa4e6a6a98) **Step 5** Select the relevant Security Groups you want your VPC endpoint to adhere to. ![select\_sgs](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/select_sgs.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=f90b6530c68f3c41f6133cc3caa14ead) **Step 6** Add as many tags as your heart desires (up to 50) and select "**Create endpoint**." ![click\_it](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/click_it.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=e0cbec31162fc455f92fba292c7f3fa1) **Step 7** The "Creating" spinner will spin momentarily and then deliver you the news of the endpoint creation. You should see a VPC endpoint in the `pending` state if it was successful. If the creation failed, record the reason and consult your Solutions Engineer. ![pending](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/pending.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=a8754d136979d8347c35682445a313aa) **Step 8** After 2-10 minutes (make sure to refresh), your VPC endpoint will report an `available` state. ![available](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/available.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=ff418c62e571302611620ed4638aead6) ## Step 3: Verifying a VPC endpoint connection PlanetScale publishes a [wildcard DNS record](https://en.wikipedia.org/wiki/Wildcard_DNS_record) for your private region. AWS PrivateLink will override the DNS record in your VPC to point to your VPC endpoint instead of the publicly published record. To verify that the DNS override is working correctly, issue the following `dig` command using the value of your "Private DNS Name" instead of the value in the example: ```shell theme={null} dig +short wildcard.frzzbztuqm3h-euwest1-1.psdb.cloud 172.31.16.197 172.31.13.7 ``` If your `dig` command returns a set of static IP addresses, your VPC Endpoint connection is operating successfully. If it returns a `CNAME` to an ELB record (for example, something like `something.elb.region.amazoneaws.com`), your connection is not operating successfully, and you should consult your Solutions Engineer. Once you've verified that your connection is operating successfully, you will need to verify that you can reach a database you've provisioned: **Step 1** Create a connection string for your PlanetScale database using the "**Connect**" button in the dashboard or the [pscale CLI](/docs/cli). **Step 2** From an EC2 instance running in your VPC, connect using the appropriate client for your database engine: **MySQL** Ensure the `mysql-client` package is installed, then paste your MySQL CLI command: ```shell theme={null} mysql -h -u -p --ssl-mode=VERIFY_IDENTITY --ssl-ca=/etc/pki/tls/certs/ca-bundle.crt Enter password: ... mysql> ``` If you receive the `mysql>` prompt, your connection is operating successfully. > [!NOTE] > > The correct path for the CA root configuration for the `--ssl-ca` flag depends on your operating system. See the [CA root configuration documentation](/docs/vitess/connecting/secure-connections#ca-root-configuration) for the correct path. **Postgres** Ensure `psql` (version 17+ recommended for `sslnegotiation=direct` support) is installed, then connect: ```shell theme={null} psql 'host= \ port=5432 \ user= \ password= \ dbname= \ sslnegotiation=direct \ sslmode=verify-full \ sslrootcert=system' ``` If you reach the `postgres=>` prompt, your connection is operating successfully. > [!NOTE] > > If `sslrootcert=system` doesn't work in your environment, replace it with the path to your system's root CA certificate. See the [Postgres connecting documentation](/docs/postgres/connecting/quickstart#secure-connections) for more details. If your client connects successfully, you have confirmed that your connections to PlanetScale will be established through AWS PrivateLink. If you cannot connect, please consult your Solutions Engineer. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Set up AWS Reverse PrivateLink with PlanetScale Managed Source: https://planetscale.com/docs/plans/managed/aws/reverse-privatelink [PlanetScale Managed](/docs/plans/managed/aws) can connect to your existing databases via AWS PrivateLink for the purposes of data imports. Vitess and Postgres Often, one of the first tasks for a new Managed deployment is to import data from an existing database housed in a separate AWS Organizations member account. This guide explains how to set up AWS PrivateLink components that enable cross-account communication to facilitate this process. In PrivateLink parlance, the source database is known as the "producer", and PlanetScale is the "consumer". While there are a number of different ways to set up and configure PrivateLink components, in this guide we'll be using the AWS CLI tool. ## How PlanetScale Managed and AWS PrivateLink work Broadly speaking, there are three major components to this PrivateLink setup: * A [VPC Endpoint Service](https://docs.aws.amazon.com/vpc/latest/privatelink/endpoint-service-overview.html) conceptually lives in the producer's VPC and AWS account. Once configured, it allows authorized principals (e.g. AWS accounts, IAM users) to establish a connection to an existing VPC Endpoint Service ID. * A [VPC Endpoint Interface](https://docs.aws.amazon.com/vpc/latest/privatelink/vpce-interface.html) exists on the consumer side, and once configured with an endpoint service address exposes an internal IP address and port which consumers may use for cross-account communication. * A [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) (NLB) is created on the producer side, and by managing the IP target group on this NLB you can choose which internal services to expose via the PrivateLink Endpoint Service. ## Starting state Below is a simplified diagram of our initial state. We've got two separate accounts, each with their own VPCs and availability zones. We're using AWS [AZ IDs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#az-ids) as well as of AZ names to ensure that we're always referring to the correct AZ, as AZ names are not consistent across AWS accounts. On the producer side, we have a network subnet inside each AZ. ![starting state](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/starting-state.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=2628332316a16768b3b3cff8f1e1a224) ## Create and configure the NLB First, create the Network Load Balancer in the producer AWS account. You want the NLB to be available in each of the three AZs that the two accounts share. This is done by adding the ids of the three subnets associated with those AZs using the `--subnets` option. ```bash theme={null} aws elbv2 create-load-balancer \ --name pl-nlb \ --scheme internal \ --type network \ --subnets subnet-cafed00d subnet-c0ffee subnet-cafef00d --security-groups sg-f00 ``` If you want to attach a security group to your NLB, you must do so at creation time. This allows you to restrict the inbound protocol and port range to the appropriate port (e.g. `TCP:3306` for MySQL or `TCP:5432` for Postgres). This command will return an ARN that uniquely identifies the newly created NLB. You'll need this later. ### Create an IP target group Run the following to create an empty IP target group on the appropriate port (e.g. `TCP:3306` for MySQL, `TCP:5432` for Postgres) in the producer VPC: ```bash theme={null} aws elbv2 create-target-group \ --name pl-nlb-target-group \ --protocol TCP \ --port 3306 \ --vpc-id vpc-f00 \ --target-type ip ``` This will return an ARN that uniquely identifies the target group. ### Register a target This step adds the IP address of the existing Primary RDS instance to the target group. You want to make sure that only the RDS writer is registered as a target. ```bash theme={null} aws elbv2 register-targets \ --target-group-arn --targets Id=10.0.0.1 ``` ### Create a listener Finally, associate the target group NLB by creating a listener. ```bash theme={null} aws elbv2 create-listener \ --load-balancer-arn \ --protocol TCP \ --port 3306 \ --default-actions Type=forward,TargetGroupArn= ``` Now that you've created the NLB and configured it, you can add the NLB to the diagram. Although on the diagram it looks like there are three separate NLBs, it's meant to represent a single NLB object that has network interfaces in three different subnets. ![create and configure nlb](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/create-and-configure-nlb.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=250dacc873bf297af9a692cb35bab8d4) ## Create and configure the PrivateLink VPC Endpoint Service Now, let's configure the endpoint service. You'll need the ARN of your load balancer for this. ```bash theme={null} aws ec2 create-vpc-endpoint-service-configuration \ --network-load-balancer-arn \ --no-acceptance-required ``` This will return a VPC service id. ## Allow inbound traffic from VPC's subnet After setting up the NLB and the VPC Endpoint Service, you need to ensure that the security group attached to the NLB permits inbound traffic from the subnet(s) of the consumer VPC where the interface VPC endpoint resides. **Step 1** Locate the security group associated with your NLB **Step 2** Edit Inbound Rules: * Protocol: TCP * Port Range: Specify the port your database is listening on (default is 3306 for MySQL, 5432 for Postgres) * Source: Enter the CIDR block of the consumer VPC's subnet (e.g., 10.0.1.0/24) **Step 3** Save changes to apply the updated rules ### Configure service permissions You only want to allow incoming connections on this endpoint service from the PlanetScale Managed AWS Organizations member account. This step requires the account number. ```bash theme={null} aws ec2 modify-vpc-endpoint-service-permissions \ --service-id vpce-svc-f00 \ --add-allowed-principals '["arn:aws:iam::123456789012:root"]' ``` ![create and configure vpce svc](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/create-and-configure-vpce-svc.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=f02a0e0352ac6384f6e23d826ed214eb) ## Wrapping Up Once you've communicated your newly created VPC Service Endpoint ID to your Solutions Engineer, the PlanetScale Engineering team can then complete the rest of the process. This involves creating, configuring, and testing the PrivateLink VPC Interface Endpoint. The diagram below illustrates the completed system. ![wrapping up](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/aws/privatelink/wrapping-up.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=3504e380f0ae82a6c7755597676266a7) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Set up AWS VPC peering with PlanetScale Managed Source: https://planetscale.com/docs/plans/managed/aws/vpc-peering Private database connectivity between your VPC and a PlanetScale Managed deployment over AWS VPC peering. ## Overview [AWS VPC peering](https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html) establishes private IP connectivity between your application VPC and the VPC containing your PlanetScale Managed deployment. Traffic uses the AWS network instead of traversing the public internet. PlanetScale will send a peering request to your AWS account, configure routes and database access in the Managed VPC, and authorize your VPC to use a Route 53 private hosted zone. In your AWS console you will need to accept the request and configure routes and network access in your VPC, and associate your VPC with the private hosted zone. > [!NOTE] > > VPC peering is available for PlanetScale Managed deployments on AWS. Contact your PlanetScale Solutions Engineer to confirm that it is appropriate for your network architecture. Peering does not disable the public database endpoint; clients must use the dedicated private hostname to connect over the peering connection. ## Prerequisites Before requesting a peering connection: * Install and configure the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). The AWS CLI is required to associate your VPC with the private hosted zone. * Confirm that you own the VPC. Participants in a shared VPC cannot accept peering requests. * Your VPC's IPv4 CIDR blocks must not overlap the PlanetScale Managed VPC's IPv4 CIDR blocks. PlanetScale provides the Managed VPC CIDR blocks. * Enable both `enableDnsSupport` and `enableDnsHostnames` on your VPC. In the [Amazon VPC console](https://console.aws.amazon.com/vpc/), select **Your VPCs**, select your VPC, then select **Actions** and **Edit VPC settings**. Enable both **DNS resolution** and **DNS hostnames**, then save your changes. For AWS CLI instructions, see [View and update DNS attributes for your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-dns-updating.html). * Identify every route table used by clients that connect to PlanetScale. * Review security groups and network ACLs for database traffic: * Vitess uses TCP ports `443` and `3306`. * Postgres uses TCP ports `443`, `5432`, and `6432`. VPC peering is not transitive. Clients must reside in the directly peered VPC; they cannot reach the Managed VPC through a transit gateway, VPN, Direct Connect connection, NAT device, or another peered VPC. ## Step 1: Initiate the setup Contact your PlanetScale Solutions Engineer and provide: * Your AWS account ID * The AWS Region of your VPC * Your VPC ID * Every IPv4 CIDR block associated with your VPC PlanetScale verifies that the CIDR blocks do not overlap, creates the peering request, and configures the PlanetScale side of the connection. Your Solutions Engineer then provides: * The VPC peering connection ID, which starts with `pcx-` * The AWS account ID and VPC ID of the PlanetScale side of the connection * The PlanetScale Managed VPC CIDR blocks * The Route 53 private hosted zone ID * The private hosted zone name * Your dedicated private database hostname > [!WARNING] > > Accept only the peering connection ID supplied by PlanetScale. An unexpected peering request can grant an unknown AWS account network access to your VPC. ## Step 2: Accept the VPC peering request The peering request remains in `pending-acceptance` for seven days. Accept it in the Region containing your VPC. **AWS Console** **Step 1** Open the [Amazon VPC console](https://console.aws.amazon.com/vpc/) and select the Region containing your VPC. **Step 2** In the navigation pane, select **Peering connections**. **Step 3** Select the peering connection ID supplied by PlanetScale. Confirm that its status is **Pending acceptance**, the requester account and VPC match the information from PlanetScale, and the accepter account and VPC are yours. **Step 4** Select **Actions**, then **Accept request**. Confirm the request. **Step 5** Wait for the connection status to become **Active**. **AWS CLI** Inspect the request before accepting it: ```shell theme={null} aws ec2 describe-vpc-peering-connections \ --region \ --vpc-peering-connection-ids \ --query 'VpcPeeringConnections[0].{Status:Status.Code,RequesterAccount:RequesterVpcInfo.OwnerId,RequesterVpc:RequesterVpcInfo.VpcId,AccepterAccount:AccepterVpcInfo.OwnerId,AccepterVpc:AccepterVpcInfo.VpcId}' \ --output table ``` Confirm that the status is `pending-acceptance`, the requester account and VPC match the information from PlanetScale, and the accepter account and VPC are yours. Accept the request: ```shell theme={null} aws ec2 accept-vpc-peering-connection \ --region \ --vpc-peering-connection-id ``` Check the status again: ```shell theme={null} aws ec2 describe-vpc-peering-connections \ --region \ --vpc-peering-connection-ids \ --query 'VpcPeeringConnections[0].Status.Code' \ --output text ``` Continue when the command returns: ```text theme={null} active ``` ## Step 3: Configure routes and network access Add a route to each route table associated with clients that connect to PlanetScale. The destination is a PlanetScale Managed VPC CIDR block and the target is the VPC peering connection. **AWS Console** **Step 1** In the Amazon VPC console, select **Route tables**. **Step 2** Select a route table associated with a client subnet, then select **Actions** and **Edit routes**. **Step 3** Add a route with a PlanetScale Managed VPC CIDR block as the **Destination** and the peering connection as the **Target**. **Step 4** Save the route and repeat these steps for every Managed VPC CIDR block and client route table supplied by PlanetScale. **AWS CLI** Run this command for every combination of client route table and Managed VPC CIDR block: ```shell theme={null} aws ec2 create-route \ --region \ --route-table-id \ --destination-cidr-block \ --vpc-peering-connection-id ``` Verify the routes: ```shell theme={null} aws ec2 describe-route-tables \ --region \ --route-table-ids \ --query 'RouteTables[0].Routes[?VpcPeeringConnectionId!=`null`].[DestinationCidrBlock,VpcPeeringConnectionId,State]' \ --output table ``` Each route should reference the supplied peering connection ID and have an `active` state. If your client security groups restrict outbound traffic, allow TCP traffic to the Managed VPC CIDR blocks on the required ports. If your network ACLs restrict traffic, allow the database connection and return traffic. PlanetScale configures the return routes and database access controls in the Managed VPC. Confirm with your Solutions Engineer that the PlanetScale route tables contain return routes for every client CIDR block you supplied. ## Step 4: Associate the private hosted zone VPC peering does not automatically change how the dedicated database hostname resolves. PlanetScale creates a Route 53 private hosted zone containing the private database records and authorizes your VPC to use it. You must complete the cross-account association. > [!WARNING] > > AWS does not support cross-account private hosted zone associations in the Route 53 console. Use the AWS CLI, an AWS SDK, or the Route 53 API. After your Solutions Engineer confirms that PlanetScale created the association authorization, run: ```shell theme={null} aws route53 associate-vpc-with-hosted-zone \ --hosted-zone-id \ --vpc VPCRegion=,VPCId= ``` The command returns a change ID. Check the change status: ```shell theme={null} aws route53 get-change \ --id \ --query 'ChangeInfo.Status' \ --output text ``` Continue when the command returns `INSYNC`. Verify the association: ```shell theme={null} aws route53 list-hosted-zones-by-vpc \ --vpc-id \ --vpc-region \ --query 'HostedZoneSummaries[].{Id:HostedZoneId,Name:Name,Owner:Owner}' ``` Confirm that the output contains the private hosted zone ID and narrowly scoped zone name supplied by PlanetScale. PlanetScale can remove the unused association authorization after the association succeeds without affecting DNS. ## Step 5: Verify DNS and database connectivity Run DNS and connection tests from a resource in a subnet that uses the configured route table. Use the dedicated private database hostname supplied by your Solutions Engineer: * Vitess hostnames end in `.private-connect.psdb.cloud`. * Postgres hostnames end in `.private-pg.psdb.cloud`. First, resolve the hostname: ```shell theme={null} dig +short ``` The hostname should resolve to private IP addresses within the PlanetScale Managed VPC CIDR blocks. If it resolves to a public IP address, traffic will not use the peering connection. Verify the private hosted zone association and the DNS settings on your VPC. > [!NOTE] > > AmazonProvidedDNS resolves the private hosted zone automatically. If your clients use a custom DNS resolver, ask your infrastructure team to configure conditional forwarding for the private hosted zone through a [Route 53 Resolver inbound endpoint](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resolver-forwarding-inbound-queries.html) in your VPC. Next, create a connection string using the **Connect** button in the PlanetScale dashboard or the [pscale CLI](/docs/cli), then connect with the appropriate database client: **MySQL** ```shell theme={null} mysql -h -u -p \ --ssl-mode=VERIFY_IDENTITY \ --ssl-ca=/etc/pki/tls/certs/ca-bundle.crt ``` A successful connection displays the `mysql>` prompt. > [!NOTE] > > The CA root path depends on your operating system. See the [CA root configuration documentation](/docs/vitess/connecting/secure-connections#ca-root-configuration) for the correct path. **Postgres** ```shell theme={null} psql -W 'host= port=5432 user= dbname= sslnegotiation=direct sslmode=verify-full sslrootcert=system' ``` A successful connection displays the `postgres=>` prompt. > [!NOTE] > > PostgreSQL 17 or later is required for `sslnegotiation=direct`. If `sslrootcert=system` does not work in your environment, use the path to your system root CA certificate. See the [Postgres connection documentation](/docs/postgres/connecting/quickstart#secure-connections). If DNS resolves to a private address but the client cannot connect: * Confirm that the peering connection and routes are `active`. * Confirm that the route tables associated with the client subnets contain every Managed VPC CIDR block. * Confirm that security groups and network ACLs allow the database connection and return traffic. * Ask your Solutions Engineer to verify the return routes and access controls in the Managed VPC. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Managed on Azure overview Source: https://planetscale.com/docs/plans/managed/azure A single-tenant deployment of PlanetScale in a dedicated Azure subscription that you own and PlanetScale operates. ## Overview PlanetScale Managed on Microsoft Azure runs the PlanetScale data plane in a dedicated Azure subscription that you own and PlanetScale operates. You continue to use the PlanetScale dashboard, API, and CLI while your database infrastructure and data remain in your Azure environment. ## Architecture PlanetScale deploys the data plane to a dedicated subscription in your Microsoft Entra tenant. The database runs on Azure Kubernetes Service (AKS), distributed across three availability zones in regions that support them. The subscription also contains the networking, managed identities, Azure Key Vaults, Azure Container Registry, storage accounts, and other resources required to operate the deployment. Backups are encrypted and stored in Azure Blob Storage in the same subscription. PlanetScale's control plane runs outside your Azure subscription. It provides the PlanetScale dashboard, API, automation, and operational workflows used to manage your databases. PlanetScale receives access only to the dedicated subscription. It does not receive administrative access to your Microsoft Entra tenant or other Azure subscriptions. ## Subscription ownership and management The subscription must be dedicated to PlanetScale Managed and must not contain existing workloads. You retain ownership of the subscription, Microsoft Entra tenant, and Azure billing relationship. PlanetScale manages all resources inside the subscription after the initial bootstrap. Do not modify, remove, or manage those resources with your own infrastructure-as-code tooling. [Contact support](https://planetscale.com/contact?initial=support) to discuss configuration changes or customization. The one-time [Azure bootstrap](/docs/plans/managed/azure/getting-started) creates customer-owned managed identities for PlanetScale automation and support. The bootstrap uses federated credentials rather than client secrets. ## Security and compliance The dedicated subscription is the security and operational boundary for the deployment. PlanetScale automation receives the subscription-scoped permissions required to provision and operate the data plane. PlanetScale does not require tenant-wide Microsoft Entra administrator roles. PlanetScale Managed is designed for organizations with additional security, compliance, and isolation requirements. PlanetScale maintains [security and compliance](/docs/security) programs including SOC 2 Type 2 and PCI DSS 4.0 Level 1 Service Provider compliance. > [!NOTE] > > If you have questions about the security or compliance of PlanetScale Managed on Azure, contact your PlanetScale Solutions Engineer. ## Private database connectivity Database connections use public, TLS-encrypted endpoints by default. PlanetScale Managed on Azure supports private connectivity through [Azure Private Link](/docs/plans/managed/azure/private-link). Azure Private Link exposes only the database service through a customer-owned Private Endpoint. It does not provide routed access between the PlanetScale Managed virtual network and your application virtual networks. VNet peering is not currently supported. Contact your Solutions Engineer if you need VNet peering. Non-overlapping IP ranges between the PlanetScale Managed subscription and any peered application subscriptions are required. The IP ranges to use in the PlanetScale subscription must be coordinated with your Solutions Engineer before PlanetScale provisions resources in the managed Azure subscription. ## Billing Azure bills the resources in the dedicated Managed subscription directly to your Azure billing account. Azure also bills Private Endpoint resources to the subscription where you create them. PlanetScale pricing is defined in your Enterprise agreement. Contact your PlanetScale account team with billing questions. ## Get started Follow the [Azure setup guide](/docs/plans/managed/azure/getting-started) to prepare a dedicated subscription and grant PlanetScale access. If you are interested in PlanetScale Managed on Azure, [contact us](https://planetscale.com/contact) to discuss your requirements. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Set up PlanetScale Managed in Azure Source: https://planetscale.com/docs/plans/managed/azure/getting-started Prepare a dedicated Azure subscription and deploy the one-time bootstrap required for PlanetScale Managed. ## Overview This guide explains how to prepare an Azure subscription for PlanetScale Managed and deploy the one-time authorization bootstrap. > [!NOTE] > > This guide is intended for PlanetScale Managed customers currently working with the PlanetScale team. You cannot complete provisioning without PlanetScale enabling Managed for your organization. If you are interested in [PlanetScale Managed](/docs/plans/managed), [contact us](https://planetscale.com/contact). ## Step 1: Prepare a dedicated Azure subscription Create a new Azure subscription in your Microsoft Entra tenant. The subscription must: * Be attached to an active Azure billing account * Be dedicated to one PlanetScale organization * Contain no existing workloads or customer-managed infrastructure * Allow resource deployment in the Azure regions selected with your Solutions Engineer Azure VM SKU availability varies by subscription and region. Coordinate region selection with your Solutions Engineer. PlanetScale may ask you to verify that the dedicated subscription has access to the required VM SKUs and sufficient quota across three availability zones in each requested region. The person deploying the bootstrap must have the Azure **Owner** role on the subscription. Owner is required because the template creates resources, managed identities, a custom role definition, and role assignments. Once PlanetScale begins managing the subscription, do not modify its resources or use your own infrastructure-as-code tooling inside it. [Contact support](https://planetscale.com/contact?initial=support) to discuss changes or customization. ## Step 2: Deploy the PlanetScale bootstrap The PlanetScale bootstrap is a subscription-scoped Azure Resource Manager (ARM) template generated from a reviewed Bicep source. It has no input parameters and creates no database infrastructure. [Deploy the PlanetScale bootstrap to Azure](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fplanetscale.com%2Fdocs%2Fassets%2Fmanaged%2Fazure%2Fazure-mn-bootstrap.json) You can also [download the ARM template](/docs/assets/managed/azure/azure-mn-bootstrap.json) for review before deployment. ### Deploy through the Azure portal **Step 1** Open the **Deploy the PlanetScale bootstrap to Azure** link and sign in with the account that has Owner on the dedicated subscription. **Step 2** Select the dedicated subscription. Select the deployment region agreed upon with your PlanetScale Solutions Engineer. **Step 3** Review the template and select **Create**. The template does not require any parameters. **Step 4** Wait for the subscription deployment to reach **Succeeded**. ### Deploy with the Azure CLI Download the template and deploy it at subscription scope: ```bash theme={null} az deployment sub create \ --name planetscale-managed-bootstrap \ --location \ --template-file azure-mn-bootstrap.json ``` Replace `` with the primary deployment region agreed upon with your Solutions Engineer. ### Resources and permissions created The template creates the `planetscale-bootstrap` resource group and three user-assigned managed identities: | Identity | Initial authorization | Purpose | | - | - | - | | Planner | Reader | Read Azure resources during Terraform planning | | Runner | Contributor, Role Based Access Control Administrator, and a custom role for managing role definitions | Provision resources and manage authorization within the dedicated subscription | | Support | None | Reserved for scoped operational access configured by PlanetScale after onboarding | The planner and runner identities use federated credentials. The template creates no passwords, client secrets, or database resources. > [!WARNING] > > The runner's combined permissions are intentionally equivalent to Owner within the dedicated subscription. The subscription is the PlanetScale Managed security boundary and must not contain unrelated resources. ## Step 3: Collect the deployment output The deployment outputs the subscription and Microsoft Entra tenant identifiers plus the resource, client, and principal IDs of all three managed identities. These values are identifiers, not credentials or secrets. In the Azure portal: 1. Open the dedicated **Subscription**. 2. Select **Deployments**. 3. Open the successful bootstrap deployment. 4. Select **Outputs** and copy the `managed` output. If you deployed with the Azure CLI, retrieve the same output with: ```bash theme={null} az deployment sub show \ --name planetscale-managed-bootstrap \ --query properties.outputs.managed.value \ --output json ``` ## Step 4: Send the onboarding information to PlanetScale Send your Solutions Engineer: * The name and public ID of your PlanetScale organization * The complete `managed` deployment output * The Azure region or regions selected for the deployment Do not send Azure credentials or create client secrets for PlanetScale. ## Step 5: PlanetScale validates access and provisions the deployment PlanetScale validates planner and runner federation, confirms the bootstrap identities and role assignments, and then provisions the Managed data plane. PlanetScale may work with you to request regional Azure quota increases based on the selected database products and expected capacity. Do not restrict quotas or resource providers after provisioning begins. Your Solutions Engineer will notify you when provisioning is complete and your team can create databases through the PlanetScale dashboard, API, CLI, or [MCP](/docs/mcp-server). ## Step 6: Verify the deployment After PlanetScale confirms provisioning is complete: 1. Sign in to the [PlanetScale dashboard](https://app.planetscale.com). 2. Confirm that the Managed region is available to your organization. 3. Create or select a database in that region. 4. Verify a TLS connection through the public endpoint. 5. If private connectivity is required, contact your Solutions Engineer. > [!NOTE] > > Private database connectivity is available through [Azure Private Link](/docs/plans/managed/azure/private-link) or Azure VNet peering (coming soon). Contact your Solutions Engineer to choose an option before provisioning. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Set up Azure Private Link with PlanetScale Managed Source: https://planetscale.com/docs/plans/managed/azure/private-link Connect an application virtual network privately to a PlanetScale Managed deployment in Azure. [Azure Private Link](https://learn.microsoft.com/azure/private-link/private-link-overview) connects an application virtual network to your PlanetScale Managed deployment without sending database traffic over the public internet. ## How the connection works PlanetScale operates an Azure Private Link Service in your Managed subscription. You create a Private Endpoint in an application VNet and manage the endpoint's private IP address and DNS. PlanetScale provides: * The Private Link Service alias * Confirmation that each application subscription is allowed to discover and connect to the service You manage: * The Private Endpoint in the application subscription * Network access from your applications to the endpoint * Optional private DNS zones and DNS records > [!WARNING] > > The Private Link Service alias identifies the Azure service. It is not a database hostname and cannot be used in a connection string. ## Prerequisites Before you begin, you need: * A [provisioned PlanetScale Managed deployment on Azure](/docs/plans/managed/azure/getting-started) * The Azure subscription and VNet where the applications that will connect to PlanetScale run * A subnet in that VNet where Azure can create the Private Endpoint * Permission to create Private Endpoints and network interfaces * The `Microsoft.Network` resource provider registered in the application subscription * Network access from your applications to the database ports supplied by PlanetScale * If you plan to use private DNS, permission to create Private DNS zones, VNet links, and DNS records ## Step 1: Ask PlanetScale to enable the application subscription Give your PlanetScale Solutions Engineer the subscription ID for each Azure subscription where you will create a Private Endpoint. PlanetScale configures the Private Link Service and gives you: * A Private Link Service alias, such as `SERVICE.GUID.REGION.azure.privatelinkservice` * Confirmation that your application subscription is configured for automatic approval If your deployment spans multiple Azure regions, PlanetScale provides a separate service alias for each region. Create a Private Endpoint for each region your applications use. > [!WARNING] > > Wait for PlanetScale to confirm that the application subscription is configured before creating the endpoint. Azure does not retroactively auto-approve an endpoint created before its subscription was added. ## Step 2: Create a Private Endpoint You can create the endpoint in the Azure portal or with the Azure CLI. ### Azure portal **Step 1** In the Azure portal, open **Private Link Center**, select **Private endpoints**, and then select **Create**. **Step 2** Select the application subscription and resource group where you want to create the endpoint. Enter an endpoint name and select its region. **Step 3** On the resource page, select the option to connect by resource ID or alias. Enter the Private Link Service alias supplied by PlanetScale. **Step 4** Select the VNet and subnet used by your applications. Complete the endpoint creation without configuring automatic DNS integration. **Step 5** Open the new endpoint and confirm that its connection status is **Approved**. Record the private IP address assigned to its network interface. ### Azure CLI Set values for your environment and retrieve the endpoint subnet ID: ```bash theme={null} ENDPOINT_RESOURCE_GROUP="" LOCATION="" VNET_RESOURCE_GROUP="" VNET_NAME="" SUBNET_NAME="" ENDPOINT_NAME="" PLS_ALIAS="" SUBNET_ID=$(az network vnet subnet show \ --resource-group "$VNET_RESOURCE_GROUP" \ --vnet-name "$VNET_NAME" \ --name "$SUBNET_NAME" \ --query id \ --output tsv) ``` Create the endpoint by using the service alias and the manual-request flow. Azure automatically approves the request when PlanetScale has configured the application subscription: ```bash theme={null} az network private-endpoint create \ --resource-group "$ENDPOINT_RESOURCE_GROUP" \ --name "$ENDPOINT_NAME" \ --location "$LOCATION" \ --subnet "$SUBNET_ID" \ --private-connection-resource-id "$PLS_ALIAS" \ --connection-name "$ENDPOINT_NAME" \ --manual-request true \ --request-message "PlanetScale private connection" ``` Confirm that the connection is approved: ```bash theme={null} az network private-endpoint show \ --resource-group "$ENDPOINT_RESOURCE_GROUP" \ --name "$ENDPOINT_NAME" \ --query 'manualPrivateLinkServiceConnections[0].privateLinkServiceConnectionState.status' \ --output tsv ``` The expected result is `Approved`. Get the endpoint's private IP address: ```bash theme={null} NIC_ID=$(az network private-endpoint show \ --resource-group "$ENDPOINT_RESOURCE_GROUP" \ --name "$ENDPOINT_NAME" \ --query 'networkInterfaces[0].id' \ --output tsv) PRIVATE_ENDPOINT_IP=$(az network nic show \ --ids "$NIC_ID" \ --query 'ipConfigurations[0].privateIPAddress' \ --output tsv) ``` ## Step 3: Choose how applications address the endpoint Applications can connect through the Private Endpoint IP directly or through a private DNS hostname. ### Use `hostaddr` for Postgres without private DNS Private DNS is optional for Postgres. Start with the connection string provided by the PlanetScale dashboard and add `hostaddr=`. Keep the existing `host` value so PostgreSQL can use it for TLS hostname verification while connecting to the Private Endpoint IP. For example: ```bash theme={null} psql "host= hostaddr=$PRIVATE_ENDPOINT_IP port=5432 user= password= dbname= sslnegotiation=direct sslmode=verify-full sslrootcert=system" ``` Use port `6432` instead when connecting through PgBouncer. PostgreSQL 17 or later is recommended when using `sslnegotiation=direct`. ### Configure private DNS The public PlanetScale hostname continues to resolve to the public endpoint. Private Link uses a separate private DNS name supplied by PlanetScale, such as `.private-pg.psdb.cloud`; it does not override the public `.pg.psdb.cloud` zone. Public connectivity is not automatically disabled when Private Link is enabled. We recommend confirming private connectivity first. Contact your Solutions Engineer to disable public connectivity. Choose a single DNS record label for the Private Endpoint. The resulting private hostname is: ```text theme={null} . ``` Create the private zone, link it to your application VNet, and add an A record for the Private Endpoint IP: ```bash theme={null} DNS_RESOURCE_GROUP="" PRIVATE_DNS_ZONE="" DNS_RECORD_NAME="" VNET_ID=$(az network vnet show \ --resource-group "$VNET_RESOURCE_GROUP" \ --name "$VNET_NAME" \ --query id \ --output tsv) az network private-dns zone create \ --resource-group "$DNS_RESOURCE_GROUP" \ --name "$PRIVATE_DNS_ZONE" az network private-dns link vnet create \ --resource-group "$DNS_RESOURCE_GROUP" \ --zone-name "$PRIVATE_DNS_ZONE" \ --name "${ENDPOINT_NAME}-vnet-link" \ --virtual-network "$VNET_ID" \ --registration-enabled false az network private-dns record-set a add-record \ --resource-group "$DNS_RESOURCE_GROUP" \ --zone-name "$PRIVATE_DNS_ZONE" \ --record-set-name "$DNS_RECORD_NAME" \ --ipv4-address "$PRIVATE_ENDPOINT_IP" ``` If your applications use custom or on-premises DNS servers, configure those servers to resolve or forward the private zone to Azure DNS. ## Step 4: Verify connectivity Run these checks from a host that uses the application VNet. Confirm that the Private Endpoint accepts connections on the required database port. For direct PostgreSQL and PgBouncer connections: ```bash theme={null} nc -vz "$PRIVATE_ENDPOINT_IP" 5432 nc -vz "$PRIVATE_ENDPOINT_IP" 6432 ``` If you configured private DNS, confirm that the private hostname resolves to the Private Endpoint IP: ```bash theme={null} dig +short . ``` Then use either the dashboard hostname with `hostaddr=$PRIVATE_ENDPOINT_IP` or the private hostname in your connection string. ## Public and private access > [!WARNING] > > Creating a Private Endpoint does not disable the deployment's public database endpoint. Private connectivity and public ingress are configured separately. After you verify private connectivity and update your applications, contact your PlanetScale Solutions Engineer if your deployment should accept private connections only. ## Troubleshooting ### The endpoint creation fails with `LinkedAuthorizationFailed` For a cross-tenant connection, use the Private Link Service alias rather than its Azure resource ID. Set `--manual-request true` when using the Azure CLI. ### The endpoint remains pending Confirm that PlanetScale configured the application subscription before you created the endpoint. If the subscription was added afterward, contact your Solutions Engineer. You may need to recreate the endpoint after PlanetScale confirms the configuration. ### The private hostname does not resolve to the Private Endpoint IP The public `.pg.psdb.cloud` hostname continues to resolve to the public endpoint while public ingress is enabled. If you ask PlanetScale to disable public ingress, PlanetScale removes the public DNS record. For private DNS, confirm that: * You created the separate private DNS zone supplied by PlanetScale * The A record points to the Private Endpoint's network-interface IP * The private DNS zone is linked to the VNet used by your application * Custom DNS servers forward or resolve the private zone correctly This DNS configuration is not required when Postgres clients use both the dashboard `host` value and `hostaddr=`. ### TLS hostname verification fails When using `hostaddr`, keep the `host` value from the PlanetScale dashboard in the connection string. When using private DNS, use a single record label directly under the private DNS zone supplied by PlanetScale. Do not use the Private Endpoint IP or Private Link Service alias as the `host` value. ### Connections time out Confirm that network security groups, user-defined routes, firewalls, and application egress policies allow the required database ports between the application and the Private Endpoint. ## Next steps * [Connect to PlanetScale Postgres](/docs/postgres/connecting) * [Configure connection pooling with PgBouncer](/docs/postgres/connecting/pgbouncer) * [Learn about PlanetScale Managed on Azure](/docs/plans/managed/azure) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Managed on GCP overview Source: https://planetscale.com/docs/plans/managed/gcp PlanetScale Managed on Google Cloud Platform (GCP) is a single-tenant deployment of PlanetScale in your GCP organization within an isolated project. Vitess, Neki, and Postgres ## Overview In this configuration, you can use the same API, CLI, and web interface that PlanetScale offers, with the benefit of running entirely in a GCP project that you own and PlanetScale manages for you. ## Architecture As you can see in the architecture diagram below, the PlanetScale data plane is deployed inside of a PlanetScale-controlled project in your GCP organization. The database cluster will run within this project, orchestrated by Kubernetes. PlanetScale Managed supports Vitess (MySQL-compatible), Neki, and Postgres clusters. Existing Managed customers should [contact us](https://planetscale.com/contact) to enable Neki. We distribute components of the cluster across three GCP zones within a region to ensure high availability. You can deploy PlanetScale Managed to any GCP region with at least three zones, including zones not supported by the PlanetScale self-serve product, so long as the region supports the required GCP services (including but not limited to Google Compute Engine (GCE), Google Kubernetes Engine (GKE), Cloud Storage, Persistent Disk, Cloud Key Management Service (Cloud KMS), Cloud Logging). Backups, part of the data plane, are stored in Cloud Storage inside the same project. PlanetScale Managed uses isolated GCE instances as part of the deployment. ![Architecture diagram for PlanetScale Managed in GCP](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/gcp/gcp-arch-diagram.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=6a6894b126235de690bce817ccda0dcf) Your database lives entirely inside a dedicated project within GCP. PlanetScale will not have access to other projects nor your organization-level settings within GCP. Outside of your GCP organization, we run the PlanetScale control plane, which includes the PlanetScale API and web application, including the dashboard you see at `app.planetscale.com`. In a MySQL deployment, the Vitess cluster running inside Kubernetes is composed of a number of Vitess components. All incoming queries are received by one of the **VTGates**, which then routes them to the appropriate **VTTablet**. The VTGates, VTTablets, and MySQL instances are distributed across 3 availability zones. ![Diagram of Vitess cluster on GCP](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/gcp/gcp-vitess.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=8834fe7fca2553cb21c2b3d5b66b4fb3) Several additional required Vitess components are run in the Kubernetes cluster as well. The topology server keeps track of cluster configuration. **VTOrc** monitors cluster health and handles repairs, including managing automatic failover in case of an issue with a primary. **vtctld** along with the client **vtctl** can be used to make changes to the cluster configuration and run workflows. In a Postgres deployment, the architecture uses a dedicated PostgreSQL cluster with similar high-availability and distribution across availability zones. ## Security and compliance PlanetScale Managed is an excellent option for organizations with specific security and compliance requirements. You own the GCP organization and project that PlanetScale is deployed within an isolated architecture. This differs from when your PlanetScale database is deployed within our GCP organizations. Along with System and Organization Controls (SOC) 2 Type 2 and PlanetScale [security and compliance](/docs/security) practices that PlanetScale has been issued and follows, we can also sign BAAs for [HIPAA compliance](https://planetscale.com/blog/planetscale-and-hipaa) on PlanetScale Managed. > [!NOTE] > > PlanetScale manages the entire project and can NOT support customers running Terraform or other configuration management in the project. ### GCP Private Service Connect By default, all connections are encrypted, but public. Optionally, you also have the option to use private database connectivity through [GCP Private Service Connect](/docs/plans/managed/gcp/private-service-connect), which is only available on single-tenancy deployment options, including PlanetScale Managed. > [!NOTE] > > If you have any questions or concerns related to the security and compliance of PlanetScale Managed, please [contact us](https://planetscale.com/contact), and we will be happy to discuss them further. ## Getting started with PlanetScale Managed in GCP If you want to see what is involved in getting set up with PlanetScale Managed in GCP, you can see the [GCP set up documentation](/docs/plans/managed/gcp/getting-started). If you are interested in exploring PlanetScale Managed further, please [contact us](https://planetscale.com/contact), and we can chat more about your requirements and see if PlanetScale Managed is a good fit for you. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Back up and restore in GCP Source: https://planetscale.com/docs/plans/managed/gcp/back-up-and-restore PlanetScale Managed backup and restore functions like the hosted PlanetScale product. For more info, see [how to create, schedule, and restore backups for your PlanetScale databases](/docs/vitess/backups). To learn more about the backup and restore access levels, see the [database level permissions documentation](/docs/security/access-control#database-level-permissions). By default, databases are automatically backed up once per day to a Cloud Storage bucket in the customer's GCP project. This default can be adjusted when working with PlanetScale Support. However, configuring and validating additional backup frequencies is the customer's responsibility. During the initial provisioning process, PlanetScale applies a Cloud Storage configuration to ensure backups are encrypted at rest on GCP Cloud Storage. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Set up PlanetScale Managed in GCP Source: https://planetscale.com/docs/plans/managed/gcp/getting-started The following guide will walk you through setting up a PlanetScale Managed cluster in your Google Cloud Platform (GCP) organization. ## Overview If you have any questions while working through this documentation, contact your PlanetScale Solutions Engineer for assistance. > [!NOTE] > > This guide is only intended for PlanetScale Managed customers currently working with the PlanetScale team. You cannot set PlanetScale Managed up on your own without PlanetScale enabling it for your organization. If you are interested in [PlanetScale Managed](/docs/plans/managed), please [contact us](https://planetscale.com/contact). ## Step 1: Account requirements A new GCP project must be set up following this documentation to successfully bootstrap a new PlanetScale Managed cluster. To proceed with this guide, an existing GCP organization and an active Cloud Billing account are required. Further information on creating GCP organizations can be found in the [creating and managing organization resources documentation](https://cloud.google.com/resource-manager/docs/creating-managing-organization). ## Dedicated GCP project PlanetScale Managed requires the use of a standalone project in GCP. This project should not have any existing resources running within it, as PlanetScale will request a set of permissions as defined in step 2. ## Modification of accounts Once the GCP project is handed over to PlanetScale via granting IAM permissions, it should not be modified. Issues caused by modifications of the GCP project or its resources void the PlanetScale Managed SLA. [Contact support](https://planetscale.com/contact?initial=support) to discuss configuration changes or customization. ## Step 2: Bootstrap GCP project Before setting up the IAM roles, you must create a new GCP project, assign it to a GCP Billing Account, and enable the Compute Engine API. ### Create a new GCP project A new GCP project can be created via the command line if the [gcloud](https://cloud.google.com/sdk/install) SDK is installed and configured: ```bash theme={null} gcloud projects create ``` Projects can also be created through the [GCP console](https://console.cloud.google.com/projectcreate). Further information on creating GCP projects is available in the [Google Cloud Resource Manager documentation](https://cloud.google.com/resource-manager/docs/creating-managing-projects). ### Assign the new project to a GCP Billing Account Next, assign the new project to a GCP Billing Account inside your organization. The account to use will depend on your organization and its policies. > [!NOTE] > > If the user who created the project has the Billing Administrator role, the project may already have billing enabled. Please review the settings to ensure it is attached to the intended Billing Account. Further information on assigning projects to Billing Accounts is available [here](https://cloud.google.com/billing/docs/how-to/modify-project). ### Enable Compute Engine API The Compute Engine API must be enabled on the new project. This can be done via the command line: ```bash theme={null} gcloud services enable compute.googleapis.com --project "" ``` Further information on enabling an API is available [here](https://cloud.google.com/apis/docs/getting-started#enabling_apis). ### Assign IAM Roles For PlanetScale to provision resources in the project, the following IAM roles must be granted to the following service accounts: * `terraform-planner@planetscale-operations.iam.gserviceaccount.com` service account: * `roles/viewer` - Viewer * `terraform-runner@planetscale-operations.iam.gserviceaccount.com` service account: * `roles/cloudkms.admin` - Cloud KMS Admin * `roles/compute.admin` - Compute Admin * `roles/container.admin` - Kubernetes Engine Admin * `roles/container.clusterAdmin` - Kubernetes Engine Cluster Admin * `roles/iam.roleAdmin` - IAM Role Admin * `roles/iam.securityAdmin` - Security Admin * `roles/iam.serviceAccountAdmin` - Service Account Admin * `roles/iam.serviceAccountKeyAdmin` - Service Account Key Admin * `roles/logging.admin` - Logging Admin * `roles/serviceusage.serviceUsageAdmin` - Service Usage Admin * `roles/storage.admin` - Storage Admin * `roles/viewer` - Viewer These can be assigned using the `gcloud` command line tool: ```bash theme={null} gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-planner@planetscale-operations.iam.gserviceaccount.com --role roles/viewer gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/cloudkms.admin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/compute.admin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/container.admin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/container.clusterAdmin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/iam.roleAdmin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/iam.securityAdmin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/iam.serviceAccountAdmin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/iam.serviceAccountKeyAdmin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/logging.admin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/serviceusage.serviceUsageAdmin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/storage.admin gcloud projects add-iam-policy-binding "" --member serviceAccount:terraform-runner@planetscale-operations.iam.gserviceaccount.com --role roles/viewer ``` Alternatively, they can be assigned through the GCP console under the project's "**IAM & Admin > IAM**" section of the [GCP console](https://console.cloud.google.com/iam-admin/iam). Further information on assigning IAM roles to projects is available in the [GCP IAM documentation](https://cloud.google.com/iam/docs/granting-changing-revoking-access). ## Step 3: Requesting an initial quota increase By default, GCP provides most new projects with quotas that are too small for PlanetScale's initial provisioning process. Submit increase requests for the following quotas. This must be done for all regions in which PlanetScale will provision resources. Depending on your organization, the default quotas may already be at or above these levels: 1. `compute.googleapis.com/ssd_total_storage`: 10000 GB 2. `compute.googleapis.com/disks_total_storage`: 10000 GB 3. `compute.googleapis.com/n2_cpus`: 256 4. `compute.googleapis.com/n2d_cpus`: 256 5. `compute.googleapis.com/cpus_all_regions`: 256 6. `compute.googleapis.com/instances`: 100 You can submit GCP quota increase requests via the project's "**IAM & Admin > Quotas**" section of the [GCP console](https://console.cloud.google.com/iam-admin/quotas). Copy and paste the quota metrics from above into the table to search for them in the quota interface. While PlanetScale does not immediately consume all requested resources, we recommend these values to ensure enough resources are available for autoscaling, growth, and upgrades. PlanetScale will request the quota increase if the customer does not but recommends that the customer initiate the request due to an unknown turnaround time for quota requests. Further information on requesting and managing GCP quotas can be found in the [Google Cloud Allocation quotas documentation](https://cloud.google.com/compute/docs/resource-quotas). > [!NOTE] > > After initial provisioning, PlanetScale will manage quotas on behalf of the customer. Customers do not need to request quota increases for future upgrades or scaling and should not restrict quotas. Limiting quotas may result in service interruptions. ## Step 4: Initiating the provisioning process Once the GCP project has been created, the IAM roles have been applied, and the quota increases have been granted, notify your Solutions Engineer, providing them the following information: * The name of the organization that you have created on `app.planetscale.com`. * The GCP project name * A confirmation of the region(s) that you have chosen for the deployment to reside in. The canonical list of regions can be found in the [Google Cloud Regions and Zones documentation](https://cloud.google.com/compute/docs/regions-zones). Once your Solutions Engineer receives this information, they will forward it to the team responsible for provisioning your deployment. Provisioning the deployment takes PlanetScale, on average, one business day. Once the deployment has been provisioned, your Solutions Engineer will contact you to confirm that your team can start creating databases. > [!NOTE] > > Optionally, PlanetScale can connect you to your databases via [GCP Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect) with PlanetScale Managed. See the [GCP Private Service Connect documentation](/docs/plans/managed/gcp/private-service-connect) for more information on establishing a Private Service Connect connection. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Set up GCP Private Service Connect with PlanetScale Managed Source: https://planetscale.com/docs/plans/managed/gcp/private-service-connect PlanetScale Managed can connect you to your databases via [GCP Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect). Vitess and Postgres ## Overview The following guide describes how PlanetScale Managed with GCP Private Service Connect works and how to set it up. > [!NOTE] > > If you are on the Base plan and would like to set up GCP Private Service Connect endpoint, see our [Private connections documentation](/docs/vitess/connecting/private-connections-gcp). ## How PlanetScale Managed and GCP Private Service Connect work Private Service Connect (PSC) allows a service producer (PlanetScale) offer services to a service consumer without the consumer being a member of the service producer's organization. The service producer is the Google Cloud project controlled by PlanetScale, and the service consumer is the project(s) where your applications operate. Your applications connect to a private IP you allocate in your project, which is routed to your PlanetScale databases in the project that PlanetScale controls. GCP PSC requires multiple components: * A Private Service Connect [Service Attachment](https://cloud.google.com/vpc/docs/private-service-connect#service-attachments) (also known as a Published Service) deployed in the project that PlanetScale controls on your behalf. * A Private Service Connect [Endpoint](https://cloud.google.com/vpc/docs/private-service-connect#endpoints) deployed in the project(s) that your applications operate in. Once all components are operating correctly, the applications in the project with the endpoint configured will connect to the service attachment using private IP addresses instead of the publicly accessible endpoint. ## Step 1: Initiating the setup process If you would like to initiate the process, please contact your Solutions Engineer and let them know the Google Cloud project ID(s) in which you intend to create Private Service Connect endpoints. If you need to add additional projects to the allowlist, please get in touch with your Solutions Engineer. > [!WARNING] > > Google Cloud project IDs cannot be changed after initial setup. Please be sure to choose an ID that you will continue to use. Once they receive your project IDs and forward them to the team responsible for provisioning your deployment, the team will provide them (and ultimately you) with the Private Service Connect Service Attachment URI, which will be in the form `projects/PROJECT/regions/REGION/serviceAttachments/SERVICE_NAME`. > [!WARNING] > > If you use VPC Service Controls in your VPC, you must ensure that the policy allows access to the PlanetScale-controlled project. Your Solutions Engineer will provide you the following information when the setup is complete: * `Target Service` (example: `projects/PROJECT/regions/REGION/serviceAttachments/SERVICE_NAME`) You will use these values when configuring the Private Service Connect in your application projects. If you have databases in multiple regions, each region will have a unique `Target Service`, and you will need to configure consumer endpoints for each region. ## Step 2: Establishing Private Service Connect > [!WARNING] > > Only proceed to the next steps once a PlanetScale Solutions Engineer has provided the `Target Service`. Refer to Google Cloud's [Access published services through endpoints](https://cloud.google.com/vpc/docs/configure-private-service-connect-services) document for more information on connecting to services via Private Service Connect. This document covers additional details not covered here, including the IAM roles required to perform the configuration process. ### Using the GCP console The following steps are an example of establishing a Private Service Connect endpoint in the [GCP Console](https://console.cloud.google.com/). **Step 1** Obtain the Private Service Connect Attachment URI (`Target Service`) from your Solutions Engineer. It will be in the format: `projects/PROJECT/regions/REGION/serviceAttachments/SERVICE_NAME`. **Step 2** Create a Private Service Connect Endpoint. In the GCP console, go to ["Network Service > Private Service Connect"](https://console.cloud.google.com/net-services/psc) page, select the "**Connected endpoints**" tab, and select the "**+ Connect endpoint**" button. **Step 3** Add a Private Service Connect Endpoint with the following details: * **Target**: Published Service. * **Target Service**: Paste the `Target Service` attachment URI provided by your Solutions Engineer in step 1. * **Name**: Pick any `Endpoint Name`. The examples in this document use `"edge"`. * **Network and subnet**: Select the network (VPC) to create the endpoint in. The endpoint will reserve a static IP address in the subnet. The VPC and subnet must be reachable by the applications you intend to connect to your PlanetScale databases from. * **Create an IP Address**: Create a reserved IP address. This is the address your applications will use to access your PlanetScale databases. * **Enable Global Access**: PlanetScale recommends enabling this option. When enabled, this allows applications in other regions of your VPC to reach the PSC endpoint. Finally, click **Add Endpoint** to start the process. Setup will take approximately 1-2 minutes. ![connect\_endpoint\_details](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/gcp/private-service-connect/connect_endpoint_details.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=2215c7182cb05e332dad7b8dbe278d4c) **Step 4** The endpoint creation process will take a minute or two. When finished, select the endpoint and verify the status is **Accepted**: ![Showing endpoint status as "Accepted"](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/gcp/private-service-connect/endpoint_status.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=99266fa77047512a6c58e2bf2d29abf0) > [!NOTE] > > You must provide the list of projects you wish to connect from to your Solutions Engineer. Your endpoint will only function once they have been added to the allowlist in the published service. Repeat steps 2-4 to create an endpoint in **each project** you wish to connect to the Private Service Connect Attachment. ## Step 3: Verifying Connectivity ## DNS > [!NOTE] > > Private Service Connect services created after **May 8, 2024** automatically create private Cloud DNS records in the project where the PSC consumer endpoints are created. > > PSC services published before **May 8, 2024** may need to create a private Cloud DNS zone and configure records pointing to the PSC endpoint IP's manually if you wish to use DNS names to connect to your PlanetScale databases. > > Google maintains additional documentation covering DNS and Private Service Connect here: > > * [Automatic DNS configuration for Service Consumers](https://cloud.google.com/vpc/docs/dns-vpc-hosted-services#auto-dns-consumer) > * [Other ways to configure DNS for Service Consumers](https://cloud.google.com/vpc/docs/configure-private-service-connect-services#other-dns) Private Service Connect endpoints automatically create a private DNS records in the project where the PSC consumer endpoints are created that resolve to the endpoint's reserved IP. The domain name used varies by region. You can view the domain name by clicking on `Network Services > Cloud DNS`. If Google was able to set up automatic DNS, you will see a new private DNS zone labeled by `DNS Name`: ![cloud dns zone list](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/managed/gcp/private-service-connect/cloud_dns.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=4159a9fad7d72a68109ceb69e14a61a7) Your consumer endpoints will be available via DNS records visible only within your VPC using the format: * `.` If your endpoint was created with automatic DNS or your created your own DNS records manually, you can verify resolution with `dig`. In this example, the endpoint was created with the name `edge` and the service's domain name was `izkpm55j334u-uscentral1.private-connect.psdb.cloud`: ```shell theme={null} $ dig +short edge.izkpm55j334u-uscentral1.private-connect.psdb.cloud 10.128.0.14 ``` ## Test connectivity Run `curl https://.` to verify your connectivity. A successful response will yield `Welcome to PlanetScale`. ```shell theme={null} curl https://edge.izkpm55j334u-uscentral1.private-connect.psdb.cloud Welcome to PlanetScale. ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Maintenance schedules Source: https://planetscale.com/docs/plans/managed/maintenance-schedules Enterprise customers have the option to enable scheduled maintenance windows on databases. Vitess and Postgres If you have a maintenance schedule enabled, any changes to your cluster configuration will only roll out during the set maintenance period. This includes database engine version upgrades (MySQL/Vitess or Postgres), increasing or decreasing cluster size, changes to the size of cluster components, and anything else that modifies your cluster configuration. The only exceptions to this are the following: * For MySQL databases: changes to the *number* of VTGates Modifications to your schema, such as [deploy requests](/docs/vitess/schema-changes/deploy-requests) and [workflows](/docs/vitess/scaling/workflows), are not included in the maintenance schedule and will complete as soon as you apply the changes. ## Enabling maintenance schedules To enable the use of maintenance schedules, you must first reach out to our [Support team](https://planetscale.com/contact). Once the feature is enabled on a database, you can configure the maintenance windows for each of your databases by clicking the database > Settings > Maintenance. Maintenance schedules can be set up on a daily, weekly, or monthly basis at a specific time in UTC. When occurring weekly, you can set the day of the week. When occurring monthly, you can set the day of the week and which week of the month over which the maintenance will occur. To modify a maintenance schedule, click on the available schedule, and then adjust via the dropdowns. If you need to make a change during an emergency, you can disable the maintenance schedule. That will result in the branch immediately running any queued tasks. If you are making a change in an emergency, we recommend that you perform the sizing operation first to queue the change, and then disable the maintenance schedule. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # The proof of concept phase of an enterprise contract Source: https://planetscale.com/docs/plans/managed/proof-of-concept We get many customers that come to us with existing large, production databases. Vitess and Postgres ## PlanetScale proof of concept [How we do PoCs with customers (Proof of Concept)](https://www.youtube-nocookie.com/embed/yj7nylLeyog?rel=0) These databases can be many terabytes of data, serving millions or billions of queries per day. Such customers would like an opportunity to build confidence that PlanetScale will meet and exceed their expectations for database performance, reliability, and compatibility with their existing systems. To help build this confidence, we often include a **Proof of Concept** (PoC) phase as a part of our contract with enterprise customers. These are also sometimes referred to as a **Proof of Value** phases (PoV). ## What is a proof of concept? We often sign multi-year contracts with our enterprise customers, and typically also include a Proof of Concept phase at the beginning. The PoC phase is a short window of time at the beginning of a longer contract during which our customers work with a Solutions Architect from our team to test our product and ensure it will meet their needs going forward. These can range from a few weeks to a few months depending on the details of the customers database, size, compute requirements, and testing needs. The last day of the PoC phase is known as the opt-out date. If a customer decides that they no longer want to move forward using PlanetScale before this opt-out date, they must notify our team of this decision in a timely manner. Payment will still be required for all resource usage and services provided during the PoC, and anything else specified in the contract, but the customer will not be on the hook for the full contract amount. This gives you the opportunity to test drive PlanetScale before committing to the full contract period. ## What happens during a Proof of Concept phase? Before signing a contract, customers typically have discussions with one of our Solutions Architects to gather the information needed to create your contract: * Which database(s) you want to bring in to PlanetScale * Your database storage and resource requirements * Whether or not vertical or horizontal sharding will be needed * The load that the database will need to be able to handle * Other details about your database infrastructure After signing a contract with PlanetScale, you will start working with your Solutions Architects on implementation and testing. Though the details change from customer to customer, the flow of work during the first few weeks or months of an enterprise contract is as follows: **Step 1** First, it's good to determine which database you want to start with. Long term, you may intend to move multiple databases and workloads into PlanetScale, but during the proof-of-concept phase it is often useful to focus on one specific database and workload to begin testing with. **Step 2** Based on this choice, your Solutions Architect will help you get the infrastructure set up for your database, and configured appropriately. **Step 3** For MySQL databases that require sharding, you will also work with us to determine your [sharding strategy](/docs/vitess/sharding), and we will help you decide the best [VSchema](https://vitess.io/reference/features/vschema/) for your tables. **Step 4** The next step is to get your data imported. We have a [self-serve tool for importing databases](/docs/vitess/imports/database-imports) into PlanetScale, but we often provide direct assistance with this, and for larger, more complex workloads our team can handle running the migration for you. **Step 5** After this, customers typically want to start testing the capabilities of their imported database. Testing can be done in a number of ways. Some customers have testing scripts they have already developed, which can be run against the PlanetScale database. You may also want to do more specific testing with queries or workloads that you know to be problematic to see how PlanetScale handles them. You can use our powerful [Insights panel](/docs/vitess/monitoring/query-insights) as well as your own custom tooling to assess how well these workloads are handled. **Step 6** If any issues arise during the testing phase, you can work with our team to come up with solutions before going into production. Ideally, customers have had sufficient time to test their database and build confidence in our platform before the opt-out date. We often see customers completely cut over to PlanetScale during this phase as well — making PlanetScale their production database before the opt-out period is even over. The process has been optimized to quickly prove that PlanetScale can handle your workloads while still providing flexibility to ensure you're able to test everything you want to before the opt-out date. ## After the opt-out date After a phase of testing has completed, our customers continue to work with our Customer Engineering team to continue onboarding. This process includes: * Migrating other database(s) that you intend to use PlanetScale for * Continued testing * Performance tuning where needed * Setting a cutover date to get your database into production ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cloud accounts and contents Source: https://planetscale.com/docs/plans/managed/security-and-access/cloud-accounts-and-contents ## Cloud accounts PlanetScale is not responsible for the general configuration of services shared across the cloud organization in which the AWS Organizations member account, GCP project, or Azure subscription is provisioned. The customer is solely responsible for managing access outside that granted to PlanetScale. ## Content restrictions The data stored in PlanetScale Managed databases is contained entirely in the customer's AWS Organizations member account, GCP project, or Azure subscription. Customers are responsible for all content that is stored in the databases they have created. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Data requests Source: https://planetscale.com/docs/plans/managed/security-and-access/data-requests ## End-user data requests Some laws require you and your company to allow your end-users to control their personal data. PlanetScale allows you to access, export, or delete their personal data. [Reach out to support](https://planetscale.com/contact?initial=support) for additional help with accessing, exporting, or deleting personal data. ## Data portability PlanetScale does not offer the migration of specific metadata between regions. PlanetScale does support migrating a database between regions for PlanetScale Managed. ## Data deletion Users can request to have their personal data removed by [contacting support](https://planetscale.com/contact?initial=support). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Schema snapshots Source: https://planetscale.com/docs/plans/managed/security-and-access/schema-snapshots Vitess and Postgres Snapshots of database schemas within PlanetScale Managed accounts are transferred to PlanetScale's control plane to facilitate features requiring visibility into schemas. Schema snapshots include the structure of databases managed by PlanetScale, but do not include any row-level data. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # User management Source: https://planetscale.com/docs/plans/managed/security-and-access/user-management ## Initial onboarding ### Administrative onboarding The customer's initial administrative user creates an organization on PlanetScale. Administrative accounts have the `Organization Administrator` role assigned. ### User onboarding Users can be onboarded either manually or using single sign-on. Manual onboarding is handled by the Administrator once they are initially granted access. Users managed via SSO are onboarded once the SSO provider is connected and configured. ### Single sign-on PlanetScale Managed requires single sign-on (SSO) for the API and web interface, enabling organizations to manage access through their existing directory services. You can read more about single sign-on and how to set it up in the [PlanetScale single sign-on documentation](/docs/security/sso). ## Access levels PlanetScale currently supports three different roles inside of organizations: * `Organization Administrator` * `Organization Member` * `Database Administrator` See the [PlanetScale access control documentation](/docs/security/access-control) for a further breakdown of each role's permissions. ## Separation of accounts PlanetScale Managed provides integration with numerous single sign-on providers. Users can have entirely separate personal and corporate accounts with PlanetScale when their organization uses SSO. It is up to the customer to ensure that they maintain their SSO setup and do not invite or allow employees to use any other authentication method to access PlanetScale. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster sizes Source: https://planetscale.com/docs/plans/planetscale-skus Here you'll find a complete list of all the database sizes available to you on PlanetScale. Vitess and Postgres If your needs exceed what is available here, we can easily spin up additional sizes for you. Just [reach out](https://planetscale.com/contact) and let us know. We also support [sharding](https://planetscale.com/sharding) on Vitess clusters. ## Network-attached storage **Network-attached storage** databases come with autoscaling storage and have varying levels of compute power. | | **Processor** | **Memory** | | :- | :- | :- | | **PS-10** | 1/8 vCPU | 1 GB RAM | | **PS-20** | 1/4 vCPU | 2 GB RAM | | **PS-40** | 1/2 vCPU | 4 GB RAM | | **PS-80** | 1 vCPU | 8 GB RAM | | **PS-160** | 2 vCPUs | 16 GB RAM | | **PS-320** | 4 vCPUs | 32 GB RAM | | **PS-400** | 8 vCPUs | 32 GB RAM | | **PS-640** | 8 vCPUs | 64 GB RAM | | **PS-700** | 16 vCPUs | 32 GB RAM | | **PS-900** | 16 vCPUs | 64 GB RAM | | **PS-1280** | 16 vCPUs | 128 GB RAM | | **PS-1400** | 32 vCPUs | 64 GB RAM | | **PS-1800** | 32 vCPUs | 128 GB RAM | | **PS-2100** | 48 vCPUs | 96 GB RAM | | **PS-2560** | 32 vCPUs | 256 GB RAM | | **PS-2700** | 48 vCPUs | 128 GB RAM | | **PS-2800** | 64 vCPUs | 128 GB RAM | ## Metal [Metal databases](/docs/metal) are backed by locally-attached NVMe drives for storage, unlocking incredible performance and cost-efficiencies. Because the drives are locally-attached, you need to choose both your compute and storage resources when you create your database. The storage options vary by cloud provided, so we break out the options into AWS and GCP sections. ### Metal options on AWS | | **Processor** | **Memory** | **NVMe Storage options** | **Notes** | | :- | :- | :- | :- | - | | **M-10** | 1/8 vCPU | 1 GB RAM | configurable | Postgres and Neki | | **M-20** | 1/4 vCPU | 2 GB RAM | configurable | Postgres and Neki | | **M-40** | 1/2 vCPU | 4 GB RAM | configurable | Postgres and Neki | | **M-80** | 1 vCPU | 8 GB RAM | configurable | Postgres and Neki | | **M-160** | 2 vCPUs | 16 GB RAM | configurable | | | **M-320** | 4 vCPUs | 32 GB RAM | 229 GB | | | **M-320** | 4 vCPUs | 32 GB RAM | 929 GB | | | **M-320** | 4 vCPUs | 32 GB RAM | 2,490 GB | | | **M-640** | 8 vCPUs | 64 GB RAM | 466 GB | | | **M-640** | 8 vCPUs | 64 GB RAM | 1,866 GB | | | **M-640** | 8 vCPUs | 64 GB RAM | 4,992 GB | | | **M-1280** | 16 vCPUs | 128 GB RAM | 942 GB | | | **M-1280** | 16 vCPUs | 128 GB RAM | 3,739 GB | | | **M-2560** | 32 vCPUs | 256 GB RAM | 1,891 GB | | | **M-2560** | 32 vCPUs | 256 GB RAM | 7,492 GB | | ### Metal options on GCP | | **Processor** | **Memory** | **NVMe Storage options** | | :- | :- | :- | :- | | **M-160** | 2 vCPUs | 16 GB RAM | 367 GB | | **M-160** | 2 vCPUs | 16 GB RAM | 742 GB | | **M-160** | 2 vCPUs | 16 GB RAM | 1,492 GB | | **M-160** | 2 vCPUs | 16 GB RAM | 2,992 GB | | **M-160** | 2 vCPUs | 16 GB RAM | 5,992 GB | | **M-160** | 2 vCPUs | 16 GB RAM | 8,992 GB | | **M-320** | 4 vCPUs | 32 GB RAM | 367 GB | | **M-320** | 4 vCPUs | 32 GB RAM | 742 GB | | **M-320** | 4 vCPUs | 32 GB RAM | 1,492 GB | | **M-320** | 4 vCPUs | 32 GB RAM | 2,992 GB | | **M-320** | 4 vCPUs | 32 GB RAM | 5,992 GB | | **M-320** | 4 vCPUs | 32 GB RAM | 8,992 GB | | **M-640** | 8 vCPUs | 64 GB RAM | 367 GB | | **M-640** | 8 vCPUs | 64 GB RAM | 742 GB | | **M-640** | 8 vCPUs | 64 GB RAM | 1,492 GB | | **M-640** | 8 vCPUs | 64 GB RAM | 2,992 GB | | **M-640** | 8 vCPUs | 64 GB RAM | 5,992 GB | | **M-640** | 8 vCPUs | 64 GB RAM | 8,992 GB | | **M-1280** | 16 vCPUs | 128 GB RAM | 742 GB | | **M-1280** | 16 vCPUs | 128 GB RAM | 1,492 GB | | **M-1280** | 16 vCPUs | 128 GB RAM | 2,992 GB | | **M-1280** | 16 vCPUs | 128 GB RAM | 5,992 GB | | **M-1280** | 16 vCPUs | 128 GB RAM | 8,992 GB | | **M-2560** | 32 vCPUs | 256 GB RAM | 1,492 GB | | **M-2560** | 32 vCPUs | 256 GB RAM | 2,992 GB | | **M-2560** | 32 vCPUs | 256 GB RAM | 5,992 GB | | **M-2560** | 32 vCPUs | 256 GB RAM | 8,992 GB | ## Selecting a cluster size Selecting the correct cluster size for your database can have a dramatic impact on how it performs and how much it costs. A good rule of thumb is when you notice CPU usage is consistently at or close to 100% for an extended period of time, you may benefit from [upsizing your cluster](#upsizing-and-downsizing-clusters). Conversely, if your CPU usage is consistently below 50%, you may be able to downsize. You can monitor your CPU usage by clicking on your database, clicking "Primary" in your architecture diagram, and referencing the chart under "Metrics and performance". There are also special cases where you may want to temporarily upsize out of caution if you're anticipating a large spike in traffic, such as during a launch or event. In these cases, you can easily [upsize](#upsizing-and-downsizing-clusters) ahead of your event, and then downsize after. Changing cluster sizes is a seamless operation that requires no downtime. If you are migrating from an existing cloud provider with resource-based pricing, be sure to compare your currently selected instance with our available cluster sizes. Keep in mind, each database comes with a production branch with two replicas, as well as 1,440 hours worth of development branches (for Vitess databases). The development branches essentially equate to two extra "always on" databases. In many cases, you can deprecate your dev/staging databases that you pay extra for with other providers in favor of the development branches. In the end, this usually results in significant cost savings. Databases in PlanetScale also come with additional beneficial infrastructure that is not easily configured or available in other hosted database solutions. For more information on what is provisioned with each database, read our [Vitess architecture](/docs/vitess/architecture) and [Postgres architecture](/docs/postgres/postgres-architecture) docs. If you are unsure which plan or cluster size is right for your application, [contact us](https://planetscale.com/contact) to get further assistance. Our self-serve plans are flexible enough to handle the majority of customers. However, there are several use cases where you may need a more custom plan. This is where our Enterprise offerings shine. ## Upsizing and downsizing clusters As your application scales, upgrading or downgrading your cluster is a seamless operation that does not involve any downtime. To change cluster sizes, go to your PlanetScale dashboard, click on your database, click the gear icon that specifies your current cluster size, select the new cluster size, and click "Update". The time it takes to change sizes depends on the size and region of your database. Larger databases may take 20 minutes to upsize/downsize. However, this is all done online, so you will not experience any downtime. Keep in mind, once you update your cluster size, you cannot change sizes again until the first size change completes. When you choose to change cluster size, we upgrade each of your replicas one by one: delete the tablet container, create a new tablet container of the new size, attach the persistent volume, start it up, and connect it to the primary. Once that's complete, we fail the primary over to one of those new replicas, and do the same thing to the old primary. ## Transaction pools (Vitess) Each Vitess keyspace (logical database) has a predefined limit on the number of simultaneous transactions it supports. This is also known as the *transaction pool*. These limits are put in place as a protection mechanism for each keyspace. The limits for each size are shown in the table below: | | **Transaction Pool\*** | | :- | :- | | **PS-10** and **M-10** | 70 | | **PS-20** and **M-20** | 75 | | **PS-40** and **M-40** | 75 | | **PS-80** and **M-80** | 110 | | **PS-160** and **M-160** | 158 | | **PS-320** and **M-320** | 211 | | **PS-400** | 211 | | **PS-640** and **M-640** | 281 | | **PS-700** | 211 | | **PS-900** | 281 | | **PS-1280** and **M-1280** | 375 | | **PS-1400** | 281 | | **PS-1800** | 375 | | **PS-2100** | 328 | | **PS-2560** and **M-2560** | 500 | | **PS-2700** | 438 | | **PS-2800** | 375 | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Selecting a region Source: https://planetscale.com/docs/plans/regions Vitess, Neki, and Postgres ## Overview PlanetScale currently offers database deployment for Vitess, Neki, and Postgres in multiple regions. Select the region closest to your application servers to reduce latency between your database and application. Deploy development branches in the region closest to your own location to reduce latency when working with the branch. For Vitess databases, you may also add read-only regions to your production database. See our [Read-only regions documentation](/docs/vitess/scaling/read-only-regions) for more information. A number of resources exist to help find which region has the lowest latency from your location – such as [CloudPing](https://github.com/estahn/cloudping). ## Available regions > [!NOTE] > > If you don't see your preferred region(s) in the following list, [get in touch](https://planetscale.com/contact) to let us know what region(s) you would like to see added. Also, Enterprise plans can be deployed in any region(s) with three availability zones. See the [Deployment options documentation](/docs/plans/deployment-options#single-tenancy-highlights) for more information. Currently, the following regions are supported, with their respective PlanetScale slugs: ### AWS regions * AWS ap-northeast-1 (Tokyo) — `ap-northeast` * AWS ap-south-1 (Mumbai) — `ap-south` * AWS ap-southeast-1 (Singapore) — `ap-southeast` * AWS ap-southeast-2 (Sydney) — `aws-ap-southeast-2` * AWS ca-central-1 (Montreal) — `aws-ca-central-1` * AWS eu-central-1 (Frankfurt) — `eu-central` * AWS eu-west-1 (Dublin) — `eu-west` * AWS eu-west-2 (London) — `aws-eu-west-2` * AWS sa-east-1 (Sao Paulo) — `aws-sa-east-1` * AWS us-east-1 (Northern Virginia) — `us-east` * AWS us-east-2 (Ohio) — `aws-us-east-2` * AWS us-west-2 (Oregon) — `us-west` ### GCP regions * GCP us-central1 (Council Bluffs, Iowa) — `gcp-us-central1` * GCP us-east4 (Ashburn, Virginia) — `gcp-us-east4` * GCP northamerica-northeast1 (Montréal, Québec, Canada) — `gcp-northamerica-northeast1` * GCP asia-northeast3 (Seoul, South Korea) — `gcp-asia-northeast3` * GCP us-east1 (Moncks Corner, South Carolina) — `gcp-us-east1` * GCP europe-west1 (St Ghislain, Belgium) — `gcp-europe-west1` * GCP europe-west4 (Eemshaven, Netherlands) — `gcp-europe-west4` ### Azure regions > [!NOTE] > > Azure regions support Postgres and Neki databases. Vitess databases are not currently available on Azure. * Azure centralus (Iowa, USA) — `azure-centralus` * Azure eastus2 (Virginia, USA) — `azure-eastus2` * Azure westeurope (Netherlands) — `azure-westeurope` ## Selecting the database region PlanetScale allows you to select the region for the default branch of your database during database creation. By default, all database branches created within this database will also be created in this region. Once you select a region for your default branch, it cannot be changed directly, but you can [migrate to a new region](#changing-branch-and-database-regions) if needed. You can also select the region while creating a database via the CLI by using the `--region` flag with the region's slug. > [!NOTE] > > The default region for all new databases is AWS us-east-2. Here's an example command for creating a database with a different region: ```shell theme={null} pscale database create --region us-west ``` ## Selecting the branch region PlanetScale allows you to select a region for development branches during creation as well. By default, it is set to the same region as its database. ![Select your branch region](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/regions/branch.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=398b71d7bbaa9316998c40839defd715) > [!NOTE] > > Once you select a branch region, it cannot be changed. You can also select the region while creating a branch via the CLI by using the `--region` flag with the region's slug. Here's an example command for creating a branch with a different region: ```shell theme={null} pscale branch create my-production-database add-tables --region eu-west ``` ## Restricting the branch regions [Organization Administrators](/docs/security/access-control#organization-administrator) can restrict branches to only being created in the same region as the one selected during database creation. To enable this setting, check the *Restrict region* setting in the settings page for the database: `app.planetscale.com///settings`. ![Restrict your branches to one region](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/regions/restrict-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=6d9c2ed20e45871805ea0619d36aebfa) ## Changing branch and database regions Once you select a region for a production or development branch, it cannot be changed. If you do need to move to a different region, we recommend taking the following steps: **Step 1** Create a new branch in the new region. **Step 2** [Backup and dump](/docs/cli/database) the original branch with: ```bash theme={null} pscale database dump ``` **Step 3** Restore the dump to the new branch with: ```bash theme={null} pscale database restore-dump ``` **Step 4** If this is for a production branch, [promote the new branch](/docs/cli/branch) to production: ```bash theme={null} pscale branch promote ``` **Step 5** Swap out the credentials in your app with the new branch. It's important to note that this will require downtime if done on a production branch, as the dump and restore process will take time to complete. To avoid data loss, you can temporarily block writes in your application before doing the dump, and re-enable them after the final credential swap. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres Source: https://planetscale.com/docs/postgres PlanetScale Postgres is a fully-managed PostgreSQL-compatible database that brings you scale, performance, and reliability without sacrificing developer experience. With PlanetScale Postgres, you get the power of branching, query insights, automated backups, replicas, and many more powerful database features without the operational complexity. > **[Postgres quickstart guide](/docs/postgres/tutorials/planetscale-postgres-quickstart)** > > Deploy a Postgres database and learn the basics of using PlanetScale. > **[Architecture](/docs/postgres/postgres-architecture)** > > PlanetScale Postgres architecture. > **[Database operations](/docs/postgres/operations-philosophy)** > > The operational philosophy behind PlanetScale Postgres databases. > **[Import a database](/docs/postgres/imports/postgres-imports)** > > Import your existing Postgres database to PlanetScale. > **[Horizontal sharding with Neki](/docs/neki)** > > Scale Postgres beyond a single cluster with Neki. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vitess, Neki, and Postgres Source: https://planetscale.com/docs/postgres-vs-vitess Compare PlanetScale's Vitess, Neki, and Postgres products PlanetScale offers [**Vitess**](/docs/vitess) (our MySQL-compatible engine), [**Neki**](/docs/neki) (sharded Postgres), and [**Postgres**](/docs/postgres). All three provide PlanetScale's signature branching workflow and enterprise-grade scaling capabilities. ## Feature comparison | Feature | Vitess | Neki | Postgres | | - | - | - | - | | **Branching** | ✅ Schema and data branches | ✅ Isolated branches | ✅ Schema and data branches | | **Deploy requests** | ✅ Online schema changes | ❌ Not available | ❌ Not available | | **Horizontal sharding** | ✅ | ✅ | ❌ | | **Read replicas** | ✅ | ✅ | ✅ | | **Read-only regions** | ✅ | ❌ | ❌ | | **Serverless driver** | ✅ | ✅ | ✅ | | **Connection pooling** | ✅ Built-in | ✅ Built-in | ✅ Built-in (PgBouncer) | | **Query Insights** | ✅ | ✅ | ✅ | | **Automatic and custom backups** | ✅ | ✅ | ✅ | | **PITR** | ❌ | ✅ | ✅ | | **Multi-region** | ✅ | ✅ | ✅ | | **SQL compatibility** | Some [MySQL compatibility](/docs/vitess/mysql-compatibility) limitations | PostgreSQL compatible; some [platform preview limitations](/docs/neki/platform-preview-limitations) | Fully PostgreSQL compatible | | **Multiple logical databases per cluster** | ❌ (use [keyspaces](/docs/vitess/terminology#keyspace) instead) | ✅ Logical databases | ✅ `CREATE DATABASE` | | **Max cluster size** | Unlimited shards | Unlimited shards | Single cluster | ## Which product should you choose? For most teams, **choose based on your existing database experience**: * **Choose Vitess** if you're currently using MySQL or have a large-scale cluster that requires [horizontal sharding](/docs/vitess/sharding) * **Choose Neki** when your Postgres workload could benefit from [online schema changes](/docs/neki/schema-changes) or [horizontal sharding](/docs/neki) * **Choose Postgres** if you're currently using PostgreSQL or prefer its feature set ## Scale considerations [Metal](https://planetscale.com/metal) clusters with over 100 TB of storage handle throughput and latency much better than traditional EBS-backed instances. Check out our [benchmarks](https://planetscale.com/blog/benchmarking-postgres) to see how Metal stacks up against other providers. When a single cluster is no longer enough, [Vitess](/docs/vitess/sharding) provides explicit sharding for MySQL-compatible workloads, and [Neki](/docs/neki) adds horizontal sharding for Postgres. See [Neki vs Vitess](/docs/neki/vitess-and-postgres) to compare their differences. If your workload can run on a single shard, [Postgres](/docs/postgres) is a strong choice. There are also benefits to using [Neki](/docs/neki) on a single shard, which puts you in a better position if you later need to shard. See [Neki vs PlanetScale Postgres](/docs/neki/coming-from-postgres). # Back up and restore Source: https://planetscale.com/docs/postgres/backups PlanetScale Postgres databases include comprehensive backup and restore capabilities to protect your data. The backup system provides both automated scheduled backups and manual on-demand backups, as well as [point-in-time recovery (PITR)](/docs/postgres/backups/point-in-time-recovery) support through Write-Ahead Log (WAL) archiving. ## Automatic scheduled backups ### Default backup schedule All PlanetScale Postgres databases include automatic scheduled backups that run every 12 hours for production and development branches. Default backups are created automatically and don't require any setup. There are no additional charges for these backups. WAL logs are retained for the same period as your oldest backup. Both backups and WAL files are stored in highly durable, available, and secure storage off of your cluster, in the same cloud provider and region as the cluster. ## Additional backups > [!NOTE] > > Custom backup schedules and manually created on-demand backups can incur additional charges. See [Backup Pricing](#backup-pricing) for more information. ### Custom backup schedules You can create additional custom backup schedules for more frequent backups or specific timing requirements. You must be an organization administrator or database administrator to create or modify a custom backup schedule. **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Backups** page from the menu on the left **Step 3** Click **Add new schedule** **Step 4** Configure your schedule: * **Run every**: Configure the frequency of the backup schedule based on hours, days, weeks, months * **Time**: Specify when the backup should start. Depending on frequency pick either the offset of the hour, the time of the day, the day of the week and time of that day, or which day of the week of the month, and the hour of that day * **Retention**: How long to keep backups (hours, days, weeks, months, or years) * **Name**: The name for your custom backup schedule. If left blank, an auto-generated name will be created. **Step 5** Click **Save schedule** ### Manual backups You can create a one-off on-demand backup of your database branch: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Backups** page from the menu on the left **Step 3** Click **Create new backup** **Step 4** Select a branch **Step 5** Enter a name **Step 6** Select if this backup is an [Emergency backup](#emergency-backups) **Step 7** Specify how long you want to keep the backup (hours, days, weeks, months, or years) **Step 8** Click **Create backup** ### Emergency backups Emergency backups are available when you need an immediate backup regardless of database state or ongoing operations. Unlike regular backups, emergency backups do not rely on previous backups or previous WALs. They also may use additional primary database resources during creation. They are only recommended when regular backups are failing due to WAL corruption or other issues. > [!NOTE] > > Emergency backups may impact database performance and should only be used in critical situations where immediate backup is required. ## Viewing backups ### Backup List All backups can be viewed on your Backups page. **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Backups** page from the menu on the left **Step 3** View all backups with details: * Backup name * Branch name * Size * Backed up (a timestamp will be displayed if backup has completed) You can click into any backup to see the branch, expiration date, restore to a new branch, and to toggle preventing backup deletion. ## Restoring from Backups ### Creating a Restored Branch Restore any backup to a new database branch: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Backups** page from the menu on the left **Step 3** Select the backup you wish to restore from **Step 4** Click **Restore to new branch** **Step 5** Configure the restored branch: * **Branch name**: Name for the new branch * **Cluster size**: Choose the desired size of the new cluster **Step 6** Click **Restore backup** A new branch will be created based on this backup and will become visible under the **Branches** page. ### Point-in-time recovery PlanetScale Postgres supports precise [point-in-time recovery (PITR)](/docs/postgres/backups/point-in-time-recovery) using Write-Ahead Log (WAL) files. This allows you to restore to any specific moment within your backup retention window, not just to backup snapshots. **Key benefits:** * Restore to any timestamp between your oldest backup and 5 minutes prior to the current time * More precise than traditional backup-only restoration * Essential for recovering from data corruption or accidental changes **Restore time considerations:** The time required depends on your cluster size, data volume, and how far back you're restoring. Restoring to recent points (within default 12-hour backup intervals) is typically quick, while restoring to older points may require replaying multiple days of WAL files. For detailed examples and step-by-step instructions, see the [Point-in-time recovery documentation](/docs/postgres/backups/point-in-time-recovery). ### Restored Branch Behavior * Restored branches are independent database branches * They include all data as of the restore point * Restored branches are always created in the **same cloud region** as the source branch * Database extensions are **not** restored - you'll need to reinstall any extensions * Restored branches can be promoted to production or used for testing * Backup expiration is automatically extended to ensure the restored branch can create new backups ### Roles on a restored branch [User-defined roles](/docs/postgres/connecting/roles) created in PlanetScale are restored with the branch. They appear on the new branch's Roles page, and they keep ownership of the objects they owned in the backup. Passwords are reset. Credentials from the source branch do not work on the restored branch. [Reset the role's password](/docs/postgres/connecting/roles#resetting-role-credentials) on the restored branch to get a new one. Short-lived roles PlanetScale creates for schema recommendations and the web console are not kept. Objects they owned are reassigned to the default `postgres` role. Roles created with `CREATE ROLE` inside Postgres are part of the restored data, but they still do not appear on the Roles page. ## Backup retention The default, included backup schedule retains backups for 2 days. You can set a custom retention period for custom backup schedules and manual one-off backups. Additionally, you can choose to protect any individual backup from deletion. Click on the backup you'd like to protect, and click the "**Prevent backup deletion**" toggle. A protected backup retains both the base backup and the Write-Ahead Log (WAL) files needed for [point-in-time recovery](/docs/postgres/backups/point-in-time-recovery). Keep in mind that custom backups and included backups that are retained beyond the default retention policy will incur additional charges. ## Backup pricing PlanetScale Postgres includes backup storage with every database branch. Each branch receives backup storage equal to **twice the allocated disk size**. For example, a database with 50 GB of storage includes 100 GB of backup storage. To learn more about how backup storage is measured and pricing for additional storage, see [pricing](/docs/postgres/pricing). ### Monitor your usage Track your backup storage usage and costs on the **Backups** page in your database dashboard. The page shows a breakdown of storage usage and any overage charges per branch, helping you optimize retention policies if needed. If you need additional assistance, [contact](https://planetscale.com/contact?initial=support) the PlanetScale support team. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Point-in-time recovery Source: https://planetscale.com/docs/postgres/backups/point-in-time-recovery Point-in-time recovery (PITR) allows you to restore your PlanetScale Postgres database to any specific moment within your backup retention window. Unlike traditional backups that restore to specific backup snapshots, PITR gives you precise control over when to restore your data. ## How PITR Works PlanetScale Postgres uses [PostgreSQL's Write-Ahead Logging (WAL)](https://www.postgresql.org/docs/current/wal-intro.html) system combined with periodic backups to enable PITR. PostgreSQL's WAL system records all changes to the database before they're applied to the actual data files. Each transaction is first written to the WAL log, ensuring durability even if the system crashes. These WAL files are continuously archived, creating a complete record of every change made to your database. During PITR, PostgreSQL starts with a base backup and then "replays" the relevant WAL files to reconstruct the exact database state at your specified point in time. In short, there are 3 key things to know about: * **Backups**: Snapshots of your database taken at regular intervals * **WAL Archiving**: Continuous archiving of all database changes as they occur * **Recovery Process**: Restores from a base backup and replays WAL files to reach the exact point in time requested ## Performing a point-in-time recovery PITR is available for any point in time within your retention window — by default from 2 days ago up to 5 minutes before the current time (the 5-minute buffer ensures WAL files are fully processed and archived). If you have created manual on-demand backups, have a custom backup schedule, or have enabled the **Prevent backup deletion** toggle on any backup, then PITR is available from the oldest backup available up to 5 minutes before the current time. > [!NOTE] > > Additional backups beyond the default incur additional charges. See [Backups](/docs/postgres/pricing#backups) for more information. ### Create the PITR backup branch **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Backups** page from the menu on the left **Step 3** On the right side of the page, you'll see a box labeled "**Point-in-time recovery**" **Step 4** Click the Branch dropdown to select the branch you'd like to restore from **Step 5** Select your target date and time within the available window **Step 6** Click **Restore backup** **Step 1** You'll see a summary of the selected branch, time you are restoring to, and size **Step 2** Name your new branch **Step 3** Select a cluster size **Step 4** Click "Restore backup" This will create a new branch with the schema and data from the selected point in time. User-defined roles are restored the same way as a [backup restore](/docs/postgres/backups#roles-on-a-restored-branch): they keep object ownership, and their passwords are reset. See [Branching](/docs/postgres/branching) for next steps on how to connect and potentially promote your new branch. ## Restore time considerations Restore time depends on three key factors: * **Cluster size**: Data can only be restored as fast as your cluster can handle. Consider up-scaling the disk configuration (for network-attached storage) or upgrading your cluster size (for PlanetScale Metal) * **Data volume**: The amount of data being restored affects initial backup restoration time * **WAL replay duration**: The time between your chosen restore point and the nearest available backup affects WAL replay time PostgreSQL must start with the closest available backup taken *before* your target restore point and replay all WAL files from that backup until the target time. ### Example 1: Restore to a point 8 hours ago Your database uses the default backup schedule: backups every 12 hours, retained for 2 days. You want to restore to a point 8 hours ago. ```text theme={null} Available Backups: [-------- 2 days retention --------] |-------|-------|-------|-------| 48hrs 36hrs 24hrs 12hrs NOW (exp) ✓ ✓ ✓ Restore Process: |===============|~~~~~~~| Base Backup WAL Replay (12hrs ago) (4 hours) ↑ Target: 8hrs ago Result: Quick restore (typically minutes to under an hour) ``` ### Example 2: Restore to a point 16 days ago (with a custom backup schedule) In addition to the default backup schedule, your database has a custom schedule: weekly backups every Sunday, retained for 31 days. You want to restore to a point 16 days ago (mid-week). ```text theme={null} Available Backups: [-------------- 31 days retention --------------] |----------|----------|----------|----------| 28d (Sun) 21d (Sun) 14d (Sun) 7d (Sun) NOW ✓ ✓ ✓ ✓ Restore Process: |===============|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Base Backup WAL Replay (21 days ago) (5 days) ↑ Target: 16 days ago Result: Longer restore time (hours, depending on database activity volume) ``` The total restore time increases with the gap between your target restore point and the nearest prior backup. Databases with high transaction volume will have more WAL files to replay, extending the process. ## When to Use PITR Point-in-time recovery is ideal for: * **Data corruption recovery**: Restore to just before corruption occurred * **Accidental deletions**: Recover data that was mistakenly removed * **Failed migrations**: Roll back to before a problematic schema change * **Compliance requirements**: Access historical data states for auditing ## Troubleshooting ### "Recovery point not available" * The requested time may be outside your retention window * Check that the timestamp is at least 5 minutes ago ### "WAL files missing" * WAL archiving may have been interrupted * Network connectivity issues during archival * Storage system failures * Consider using the nearest available backup point instead If you need additional assistance, [contact](https://planetscale.com/contact?initial=support) the PlanetScale support team. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Branching Source: https://planetscale.com/docs/postgres/branching Branches on PlanetScale Postgres are isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups. When your PostgreSQL database is first initialized, a single production branch is created called `main` which acts as the default branch. You can then create development branches that you can use for development before shipping schema changes to production. Branches are completely isolated databases. Changes made in one branch, whether to the schema or the data, do not affect any other branches for a given database. There is no data replication between branches, so writing data into one will not replicate to another. > [!NOTE] > > New branches incur additional charges. The cost depends on the selected resources for the branch. The monthly price for the new branch will be shown during branch creation. See [pricing](/docs/postgres/pricing) for more information. > [!WARNING] > > **A branch's region is set at creation.** To serve traffic from a different region, you can migrate to a new branch. See [Changing regions](#changing-regions) for the steps. ## Development and production branches PlanetScale Postgres provides two types of database branches: * **Development branches** — Development branches run on `PS-DEV` instances that have limited performance capabilities, different egress rates, and no replicas. This is great for experimentation, testing new schemas, and limited exploration of your data. * **Production branches** — Production branches are intended for production traffic and include optional replicas for high availability. Production branches can be created on non-HA ([single node](/docs/postgres/cluster-configuration/single-node)) or HA (primary + 2 replicas) nodes. New branches created from the **Branches** page always start as development branches on `PS-DEV` instances. To convert a branch into a production branch, [upsize the cluster](/docs/postgres/cluster-configuration#adjusting-cluster-size) and [add replicas](/docs/postgres/cluster-configuration#managing-replicas) from the **Clusters** page. Only production branches can be set as the [default branch](#set-as-default-branch). Each database allows `100` development branches by default. Change the limit on the database [Settings](/docs/postgres/settings#development-branch-limit) page. The limit can be any number from 1 to 5,000. When a database is at its limit, delete development branches you no longer need, or raise the limit, before creating another one. For a limit above 5,000, [contact support](https://planetscale.com/contact). ## Create a branch There are two ways to create a new Postgres branch: from the Branches page (no schema or data included) or by restoring from a backup (schema and data included). Each branch is its own isolated database and uses its own storage separate from production. You will be charged for the storage consumed by all production and development branches. We do not recommend using production data for development environments. ### From the Branches page This method does not include schema or data. Branches created via this method will always initialize as a single node `PS-DEV` database, which does not have any replicas and begins at \$5/month, depending on the region. After initialization, you have the option to upsize the branch or add replicas from the "Clusters" page in the dashboard. **Step 1** Click **Branches** **Step 2** Click **New branch** **Step 3** Give the branch a name **Step 4** Choose the base branch. This currently does not copy the schema. **Step 5** Select a region. See [Changing regions](#changing-regions) to migrate to a different region later. **Step 6** Click **Create branch** > [!NOTE] > > If [Restrict branch regions](/docs/postgres/settings#restrict-branch-regions) is enabled in database settings, new branches can only be created in the same region as the default branch. ### From a backup This method includes both the schema and the data for the selected backup. The cluster size, and therefore cost, is inherited from your main parent branch. **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Backups** page from the menu on the left **Step 3** Select the backup you wish to restore from **Step 4** Click **Restore to new branch** **Step 5** Configure the restored branch: * **Branch name**: Name for the new branch * **Cluster size**: Choose the desired size of the new cluster **Step 6** Click **Restore backup** A new branch will be created based on this backup and will become visible under the **Branches** page. ## Changing regions A branch stays in the region you select at creation, so switching regions means migrating to a new branch. The process is straightforward — create a branch in the target region, bring your schema and data over, and promote it when ready. **Step 1** Create a new empty branch from the **Branches** page and select the desired region. **Step 2** Apply your schema to the new branch using standard PostgreSQL DDL. **Step 3** Copy data from the existing branch using a tool such as `pg_dump`/`pg_restore` or [pgcopydb](/docs/postgres/imports/postgres-migrate-pgcopydb). **Step 4** [Upsize the cluster](/docs/postgres/cluster-configuration#adjusting-cluster-size) and [add replicas](/docs/postgres/cluster-configuration#managing-replicas) to make it a production branch. **Step 5** [Set it as the default branch](#set-as-default-branch) if needed, and update your application connection strings to point at the new branch. **Keep in mind:** * There is no data replication between branches, so schema and data need to be copied manually as described above. * Branches created [from a backup](#from-a-backup) inherit the region of the source branch, so backups alone won't get you into a new region — use the migration steps instead. * To run production in a new region, you'll need to promote the new branch to default. See [Set as default branch](#set-as-default-branch). ## View all branches To view all branches for your PostgreSQL database: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Click on the **Branches** drop down **Step 3** You'll see a list of all branches with their type (development/production) **Step 4** You can select a branch by its name to see its **Dashboard** Similarly you can navigate directly to the Branches page to see all branches: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Branches** page from the menu on the left ## Connect to a branch Each branch has its own connection details. To connect to a specific branch: **Step 1** Navigate to the desired branch following either of the paths shown above **Step 2** Click the **Connect** button **Step 3** If you haven't already, follow the instructions to **Create the default role** **Step 4** Choose your preferred connection method ([direct or PgBouncer](/docs/postgres/connecting)) **Step 5** Copy the connection string or credentials **Step 6** You should create new roles for specific use-cases and tailor their permissions appropriately for them **Step 7** In your application, use the specific role(s) and the preferred connection method. For detailed connection instructions, see the [PostgreSQL connection documentation](/docs/postgres/connecting). ## Rename a branch To rename a branch: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Branches** page from the menu on the left **Step 3** Click on the three dots ("...") next to the branch you want to rename **Step 4** Select **Rename branch** **Step 5** Enter the new branch name **Step 6** Click **Save changes** Renaming a branch does not affect that branch's credentials. You do not need to regenerate credentials if you rename a branch. ## Set as default branch The default branch serves as the source branch when creating new development branches. Only production branches can be the default branch. New branches created from the **Branches** page start as development branches, so before a branch can be promoted to the default, you must first [upsize the cluster](/docs/postgres/cluster-configuration#adjusting-cluster-size) and [add replicas](/docs/postgres/cluster-configuration#managing-replicas) from the **Clusters** page to turn it into a production branch. To change the default branch: ### From the Branches page **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Go to the **Branches** tab in your database dashboard **Step 3** Click on the three dots ("...") next to the branch you want to set as default **Step 4** Click **Set as default branch** ### From the Settings page **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Settings** page from the menu on the left **Step 3** Select the branch from the **Default branch** dropdown you want to be the default branch **Step 4** Scroll down and click **Save database settings** If you change the default branch and intend for it to power your production application, you may need to update your application credentials to reference the new default branch. ## Delete a branch You can delete a branch that you no longer need: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Go to the **Branches** tab in your database dashboard **Step 3** Click on the three dots ("...") next to the branch you want to delete **Step 4** Select **Delete** **Step 5** Confirm the deletion **Important notes:** * You cannot delete the default branch. You must first set another branch as the default branch. * You cannot set development branches as the default branch. * Only [Organization Administrators and Database Administrators](/docs/security/access-control) have permission to delete production branches. Database Members can delete non-production branches. * Development branches are billed only for the time they are used, so it's beneficial to delete them when no longer needed. ## Schema changes Since PlanetScale Postgres branches don't use [deploy requests like in Vitess](/docs/vitess/schema-changes/deploy-requests), you make schema changes directly on each branch: **Step 1** Connect to your development branch **Step 2** Make your schema changes using standard PostgreSQL DDL commands **Step 3** Test your changes in the development environment **Step 4** When ready, manually apply the same changes to your production branch There's currently no automated way to merge schema changes between PlanetScale Postgres branches. You must manually copy your changes from development to production branches. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster configuration Source: https://planetscale.com/docs/postgres/cluster-configuration The Clusters page in your PlanetScale dashboard allows you to monitor your cluster utilization and configure cluster settings for each branch in your database. You can: * Adjust the cluster size and instance type ([Metal](/docs/metal) vs network-attached storage) * Configure the number of replicas * Monitor cluster utilization with real-time graphs * Configure storage settings including: * Disk size configuration * Autoscaling settings and thresholds * Storage limits * IOPS configuration (default: 3,000 IOPS included) * Bandwidth settings (default: 125 MiB/s included) * Modify PostgreSQL [parameters](/docs/postgres/cluster-configuration/parameters) * Manage PostgreSQL [extensions](/docs/postgres/extensions) * View and track configuration changes These settings may only be changed by a [database administrator or organization administrator](/docs/security/access-control). ## Adjusting cluster size To adjust your cluster size: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** In the Cluster size section, click on the cluster size dropdown **Step 4** Select the new cluster size, (Metal or network-attached storage). If you select Metal, make sure you select the correct storage size. **Step 5** Click Queue instance changes > Apply changes to apply the new cluster size > [!NOTE] > > You cannot change the [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) (`aarch64` or `x86-64`) of an existing cluster. New branches and backup restores retain the source architecture. To use a different architecture, create a new PlanetScale Postgres database and [migrate your data](/docs/postgres/imports/postgres-imports). Cluster resizing may take several minutes to complete and you cannot make additional configuration changes until the resize is finished. Consider the case where you need to upgrade from an `M-160` database cluster to an `M-320`, doubling the compute resources of each node. After applying the upgrade, three new `M-320` nodes are created. These are caught up with the primary through a combination of a backup restore and data replication. ![Cluster Resize 1](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/cluster-resize-1.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=75487a1f038e75172bac84e1af3a424c) Once these new `M-320` replicas are sufficiently caught up, the operator transitions primaryship to the one of the new `M-320` nodes. After this, the old `M-160` replicas are decommissioned, using the new ones for all replica traffic. During each node replacement, the connections to the decommissioned node will be terminated. Your clients will need to establish new connections with the new nodes. ![Cluster Resize 2](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/cluster-resize-2.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=a9f1f7f07b884fa33ab45a8b2a587e22) During the primary cutover all database connections will be terminated. Normally a primary promotion proceeds in a fast and orderly manner in less than 5 seconds. In cases where the operator is not able to quickly and cleanly shutdown the primary due to unresponsive user queries or transactions, the the operator will failover to a replica after a timeout of 30 seconds. For all the steps leading up to the node replacement, your existing M-160 database cluster remains fully functional. Due to the way this works, it's important for your application to have connection retry logic. ## Managing replicas [Replicas](/docs/postgres/scaling/replicas) provide read scalability and high availability for your PostgreSQL database. Each production branch (excluding single node) comes with 2 replicas by default. > [!NOTE] > > If you add additional replicas beyond the default 2, you will incur additional charges. Billing for additional replicas begins once the replica change has completed. See [pricing](/docs/postgres/pricing) to understand the additional costs. To adjust the number of replicas: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** In the **Cluster size** section, use the dropdown **Replicas** to select your desired number of replicas **Step 4** Click **Queue instance changes** **Step 5** Click **Apply changes** to immediately begin the process to update the number of replicas ## Monitoring cluster utilization The cluster utilization graph provides real-time insights into your database's CPU and memory utilization. You can change the graph view to the past hour, 6 hours, 12 hours, day, or week. The graph also displays indicators when cluster settings were changed. > [!NOTE] > > If you are consistently close to 100% CPU, we recommend upsizing your cluster. Likewise, if you are consistently below 50% CPU, you can likely safely downgrade. ## Parameters and settings You can customize your PostgreSQL cluster's behavior in the **Parameters** tab on the Clusters page. This allows you to configure the following settings: * PgBouncer * Resource usage * Write-ahead logs (WAL) * Query tuning * Connections and authentication * Failover For more information about each setting, refer to the [Parameters documentation](/docs/postgres/cluster-configuration/parameters). ## Configuring storage You can configure your cluster's storage settings in the "**Storage**" tab on the Clusters page. Storage configuration includes: * Setting the minimum disk size for your cluster * Enabling autoscaling to automatically increase storage when approaching limits * Configuring storage limits to control maximum storage allocation * Monitoring current storage usage * Configuring IOPS and bandwidth settings (AWS gp3 storage includes 3,000 IOPS and 125 MiB/s throughput by default at no additional cost) For detailed information about storage configuration options, refer to the [Storage configuration documentation](/docs/postgres/cluster-configuration/cluster-storage). ## Managing extensions PostgreSQL [extensions](/docs/postgres/extensions) provide additional functionality for your database. The "**Extensions**" tab on the Clusters page allows you to view and enable/disable available extensions. ### Extension management * View currently installed extensions * See available extensions for installation * Enable or disable extensions * Configure extension parameters ## Monitoring configuration changes The Clusters page provides visibility into all changes made to your cluster settings. * Navigate to the Changes tab for the selected branch on the Clusters page * View a chronological list of all configuration changes * See details including: * What was changed (cluster size, replicas, parameters) * Previous and new values * Status of the change * Who initiated the change * When the change was started and completed > [!NOTE] > > When updating a cluster's size, some [parameters](/docs/postgres/cluster-configuration/parameters) will automatically be adjusted. Each cluster size is associated with default parameter settings, changing the cluster size will also update those defaults. The exception to this is if you manually override a default parameter setting. In that case, a cluster size adjustment will not automatically change that setting. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster storage configuration Source: https://planetscale.com/docs/postgres/cluster-configuration/cluster-storage You can configure storage settings for network-attached storage PlanetScale Postgres clusters in the "**Storage**" tab on the Clusters page for your database. > [!NOTE] > > For PlanetScale Postgres clusters launched on PlanetScale Metal instances, storage is scaled by directly scaling the cluster instance size. Storage autoscaling is not available for Metal clusters. To learn more see the documentation for [PlanetScale Metal](/docs/metal) ## Configuring storage settings You must be a database or organization administrator to modify these settings. Adjusting these settings may incur additional charges. To learn more about pricing for storage, see [pricing](/docs/postgres/pricing). **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose storage settings you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Storage** tab **Step 5** Configure your disk size and autoscaling settings **Step 6** Set your storage limit as needed **Step 7** Click "**Queue storage changes**" **Step 8** Once you're ready to apply the changes, click "**Apply changes**" ## Minimum disk size configuration Configure the minimum disk size for your database cluster. This setting determines the initial storage capacity allocated to your database. The disk size is specified in GB and serves as the baseline storage allocation for your cluster. You're billed for the allocated size, which can be larger after the disk grows via autoscaling. To shrink, lower the minimum, then queue and apply the change. See [Manually shrinking](/docs/postgres/cluster-configuration/disk-autoscaling#manually-shrinking). > [!NOTE] > > The maximum disk size for network-attached storage is 65536 GB (64 TB). ## Enable autoscaling > [!NOTE] > > PlanetScale storage autoscaling is only for network-attached storage database clusters. For [PlanetScale Metal](/docs/metal) based clusters you will need to increase the cluster instance size. Disk autoscaling is enabled by default upon database creation. Disk autoscaling automatically increases storage when your database approaches a disk size utilization threshold, preventing storage-related outages without manual intervention. You can also manually shrink your disk. Autoscaling can be configured by going to "Clusters" > "Storage" > and clicking the "Enable autoscaling" checkbox. For more information, see the [Disk autoscaling documentation](/docs/postgres/cluster-configuration/disk-autoscaling). ## Storage limit The storage limit sets the maximum amount of storage that can be allocated to your database cluster through autoscaling. This acts as a ceiling to prevent unlimited storage growth and helps control costs. When autoscaling is enabled, your storage can grow from the minimum disk size up to the storage limit you specify. The storage limit should be set higher than your initial disk size to allow for growth while providing a reasonable upper bound for your storage costs. > [!NOTE] > > The maximum disk size for network-attached storage is 65536 GB (64 TB). ## IOPS Configure the maximum input/output operations per second for your database. This will be limited by your database cluster size and disk size. ### Storage volume type and IOPS | Storage type | Default IOPS | Maximum IOPS | | - | - | - | | AWS gp3 | 3,000 | 80,000 (at 160 GB or larger disk size) | > [!NOTE] > > The default of **3,000 IOPS** is included with all AWS gp3 network-attached storage clusters at no additional cost. You are only charged for IOPS above this baseline if you manually increase the IOPS setting. > [!NOTE] > > On GCP Persistent Disk clusters, IOPS are not configured manually. Instead, IOPS scale automatically with your disk size at no additional cost. ## Bandwidth The maximum amount of data that can be read or written to your database in a single second. This will be limited by your database cluster size and configured IOPS. ### Storage volume type and bandwidth | Storage type | Default bandwidth | Maximum bandwidth | | - | - | - | | AWS gp3 | 125 MiB/s | 2,000 MiB/s (at 8,000 IOPS or higher) | > [!NOTE] > > The default of **125 MiB/s** throughput is included with all AWS gp3 network-attached storage clusters at no additional cost. You are only charged for throughput above this baseline if you manually increase the bandwidth setting. > [!NOTE] > > On GCP Persistent Disk clusters, bandwidth is not configured manually. Instead, throughput scales automatically with your disk size at no additional cost. ### AWS storage throughput limits For databases created on AWS-based clusters the **maximum configurable throughput** your cluster can support is based on CPU architecture and cluster size. | CPU Architecture | Cluster Size | Maximum Throughput (in MiB/s) | | - | - | - | | AWS x86-64 | PS-DEV, PS-10, PS-20, PS-40, PS-80, PS-160, PS-320, PS-640, PS-1280, PS-2560 | 2000 | | AWS aarch64/ARM64 | PS-DEV, PS-10, PS-20, PS-40, PS-80, PS-160, PS-320, PS-640, PS-1280 | 593 | | AWS aarch64/ARM64 | PS-2560 | 2000 | ## Monthly storage cost Displays the estimated monthly cost for your current storage configuration. If you adjust your storage configuration the number shown represents the new monthly estimate for the configured values. Billing for storage changes begins once the storage change has completed. ## Tracking changes to storage settings You can click on the "Changes" tab on the Clusters page to view a log of any changes made to your storage settings. The log will include the settings affected, the original and updated values, status, user that made the changes, start time, and end time. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # CPU Architectures Source: https://planetscale.com/docs/postgres/cluster-configuration/cpu-architectures When deploying PostgreSQL databases, choosing the right CPU architecture is crucial for optimizing performance, cost, and efficiency. PlanetScale Postgres supports both x86-64 and ARM64 (aarch64) architectures, with ARM64 instances on AWS powered by Graviton processors. ## Architecture Overview ### x86-64 (Intel/AMD) The x86-64 architecture has been the dominant server architecture for decades. These processors offer: * Mature ecosystem with extensive software optimization * Wide compatibility with existing applications and tools * High single-threaded performance * Established performance benchmarks and tuning practices ### ARM64 (AWS Graviton) ARM64 represents the next generation of server processors, with [AWS Graviton chips](https://aws.amazon.com/ec2/graviton/) specifically designed for cloud workloads: * Custom silicon optimized for cloud applications * Potential price-performance benefits, depending on the workload and cluster size * Designs focused on energy efficiency * Built on modern 64-bit ARM architecture ## Performance Comparison ### CPU Performance On AWS Graviton and Google Axion instances, each cloud vCPU corresponds to a physical core. On x86 instances with simultaneous multithreading, two vCPUs share a physical core. Some PlanetScale sizes provide fractional vCPU allocations, so choosing ARM64 does not mean every cluster size receives a whole core. * **Single-threaded**: Compare individual query latency on the available processors; architecture alone does not determine single-threaded performance * **Concurrent workloads**: More physical cores at the same vCPU count can help with concurrent queries and parallel query workers * **Memory bandwidth**: Depends on the processor and instance configuration, and can affect queries that scan large amounts of data ### PostgreSQL-Specific Performance * **OLTP workloads**: ARM64 is a good default for ordinary reads and writes; test your workload to compare performance per dollar * **Analytics workloads**: Compare your analytical queries on both architectures, including the parallelism and extension optimizations they use * **Concurrent queries**: Test at production-like concurrency to compare throughput and latency * **Background processes**: Additional cores can help background maintenance run alongside application queries ## Cost Considerations ### Infrastructure Costs * **Instance prices**: Compare current prices for the sizes and region you need * **Performance per dollar**: Compare the cost of configurations that meet your throughput and latency requirements * **Storage**: Include storage charges and check [throughput limits](/docs/postgres/cluster-configuration/cluster-storage), which can vary by architecture and cluster size ### Total Cost of Ownership * **Development overhead**: Client applications do not need to share the database server's architecture * **Migration effort**: Include the work required to migrate an existing database when evaluating potential savings * **Scaling costs**: Compare the available sizes and prices as your workload grows ## Compatibility and Ecosystem ### Software Compatibility * **PostgreSQL**: Full native support on both architectures ([PostgreSQL supported platforms](https://www.postgresql.org/docs/current/supported-platforms.html)) * **Extensions**: Check the [extensions available on PlanetScale](/docs/postgres/extensions); implementations can use different optimizations on each architecture * **Tools**: Client-side administration tools connect through the Postgres protocol and do not need to match the server architecture * **Drivers**: An application on x86 can connect to an ARM64 database using the same Postgres driver and SQL ### Migration Considerations * **Existing applications**: Changing the database server architecture alone does not require rebuilding applications that use standard PostgreSQL drivers * **Binary extensions**: Code running inside Postgres must support the server architecture; use PlanetScale's supported extensions * **Performance tooling**: Some x86-specific optimization tools may not be available on ARM64 ## Decision Matrix | Consideration | x86-64 | ARM64 (Graviton) | | :- | :- | :- | | **Price-performance ratio** | Measure for your workload | Measure for your workload | | **Single-threaded performance** | Depends on the processor | Depends on the processor | | **Concurrent workloads** | vCPUs can share a physical core | One physical core per Graviton vCPU | | **PostgreSQL extensions** | Check supported extensions | Check supported extensions | **Choose x86-64 when:** * Your workload benefits from x86-specific optimizations in supported extensions * Tests show better performance for your CPU-heavy queries **Choose ARM64 (Graviton) when:** * It meets your performance requirements at a lower cost * Running concurrent, multi-threaded workloads * Starting new projects without legacy constraints ## Getting Started Both architectures are available when creating new PostgreSQL databases in PlanetScale Postgres. You can specify your preferred architecture during the database creation process. For most new deployments doing ordinary reads and writes, ARM64 is a good default. Consider running performance tests with your specific workload on both architectures to make the most informed decision for your use case. > [!NOTE] > > You cannot change the CPU architecture of an existing cluster or mix architectures within its primary and replicas. New branches and backup restores retain the source architecture. To use a different architecture, create a new PlanetScale Postgres database and [migrate your data](/docs/postgres/imports/postgres-imports). For help planning the migration, please [reach out to support](https://planetscale.com/contact?initial=support). ## CPU Architecture availability Depending on your target region for your deployment, there may or may not be certain cluster size configurations that are available from the underlying provider. PlanetScale aims to have the most complete availability of resources for you to use and as is such may enable or disable certain configurations over time based on availability. > [!NOTE] > > The most accurate source of this information is on the [Create a new database](https://app.planetscale.com/new) page and then selecting the desired region. > > For customers of managed deployments, please [reach out to support](https://planetscale.com/contact?initial=support) for assistance in confirming availability for your deployment. ## Additional Resources > **[AWS Graviton processors](https://aws.amazon.com/ec2/graviton)** > **[PostgreSQL Performance Tuning](https://wiki.postgresql.org/wiki/Performance_Optimization)** > **[PostgreSQL supported platforms](https://www.postgresql.org/docs/current/supported-platforms.html)** > **[PlanetScale Postgres migration guides](/docs/postgres/imports/postgres-imports)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Disk autoscaling Source: https://planetscale.com/docs/postgres/cluster-configuration/disk-autoscaling Disk autoscaling automatically increases storage when your database approaches a disk size utilization threshold, preventing storage-related outages without manual intervention. You can also manually shrink your disk allocation. > [!NOTE] > > PlanetScale storage autoscaling is only for network-attached storage database clusters. For [PlanetScale > Metal](/docs/metal) clusters, you need to increase the cluster instance size to increase storage space. Cloud providers like AWS and GCP limit how frequently network-attached disks can be resized. In both cases, there is a multi-hour cooldown period between resizing operations. These volumes also typically do not support shrinking. PlanetScale disk autoscaling grows disks past those limits. To shrink, we copy your data onto a new, smaller disk and replace the old one. **Disk autoscaling is enabled by default upon database creation.** We provide two growth modes: * **In-place growth mode** — This is the default scaling mode that expands storage capacity by resizing existing volumes directly, without requiring failovers or connection disruption. This method leverages the native volume-resize capability of AWS EBS and GCP Persistent Disk. * **Surge growth mode** — Surge growth creates new volumes with larger capacity and orchestrates failover to the new storage, circumventing AWS EBS and GCP Persistent Disk [resize limitations](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifyVolume.html). * **Shrink mode** — In shrink mode, the autoscaler reduces storage capacity for underutilized volumes. This can help to optimize costs after a surge event. Autoscaling will grow the cluster's storage without requiring you to make any configuration changes. The new additional space will become available as soon as the scaling action has completed. Disks are never shrunk automatically. After usage drops, use [Manually shrinking](#manually-shrinking) to shrink the disk. > [!NOTE] > > You're billed for the allocated disk, not how much data you have. ## Autoscaling thresholds and behaviors When enabled, disk growth kicks in when your disk utilization reaches the following thresholds. Shrink is never automatic. A shrink is only suggested in the dashboard when utilization falls below the thresholds below. ### Surge mode thresholds Automatic disk growth activates when disk utilization reaches these thresholds: * **70%** for disks smaller than 4 TB * **25%** for disks larger than or equal to 4 TB ### Suggested shrink thresholds A disk shrink will be suggested in the "Clusters" configuration storage tab when disk utilization falls below these thresholds: * **12.5%** for disks smaller than 1 TB * **15%** for disks between 1 TB and 2 TB * **25%** for disks larger than 2 TB For example, if you have a 200 GB disk allocation and are only using 20 GB, we will suggest the disk can be shrunk because it's below the existing 12.5% threshold. ### Key disk autoscaling behavior: * Cluster storage can only scale once in a multi-hour period * Cluster storage scales proportionally based on current size * Smaller disks receive larger percentage increases, while larger disks receive smaller percentage increases * All disks grow by a minimum of 50% when autoscaling occurs * A disk is never shrunk automatically and disk shrinking must be initiated manually * If you need to scale cluster storage by more than 200% within 24 hours, manually scale disk size ahead of time * Autoscaling will not scale past your configured [**Storage limits**](#storage-limits) ## Surge growth mode Our surge growth creates new volumes with larger capacity and orchestrates failover to the new storage, circumventing any resize rate limits. When our disk autoscaler is able to spread out disk scale-up sufficiently, no downtime is required to scale the disks. When data growth is rapid, the autoscaler may need to complete a **surge resize** to support the writes. In this case, PlanetScale creates brand new, larger network-attached storage volumes to replace the old ones. Surge growth causes a brief failover event that severs existing database connections. Applications must handle connection recovery. If the surge autoscaler is able to complete the resize before your disk fills, downtime will be minimal for growing the disk (seconds). If your disk fills before the new disks are ready, you will experience a longer period of downtime. We make every effort to keep your network-attached storage disk from filling, but it's important for the database administrators to pay close attention to storage and take manual intervention when necessary. ## Manually shrinking **Step 1** Open the database, go to **Clusters**, pick the branch, and open the **Storage** tab **Step 2** Lower **Minimum disk size** to the size you want **Step 3** Click **Queue storage changes** **Step 4** Click **Apply changes** When initiated, the disk scaler reduces storage capacity for underutilized volumes through surge operations, as neither AWS EBS nor GCP Persistent Disk supports in-place volume shrinking. Shrink operations cause a brief failover event that severs existing database connections. Applications must handle connection recovery. ## Enable or disable disk autoscaling Disk autoscaling is enabled by default upon database creation. Turn it on or off from "Clusters" > "Storage" with the "Enable autoscaling" checkbox. ## Storage limits The storage limit sets the maximum amount of storage that can be allocated to your database cluster through autoscaling. This acts as a ceiling to prevent unlimited storage growth and helps control costs. When autoscaling is enabled, your storage can grow from the minimum disk size up to the storage limit you specify. The storage limit should be set higher than your initial disk size to allow for growth while providing a reasonable upper bound for your storage costs. > [!NOTE] > > The maximum disk size for network-attached storage is 65536 GB (64 TB). # Maintenance windows Source: https://planetscale.com/docs/postgres/cluster-configuration/maintenance-windows Maintenance windows have been temporarily disabled for PlanetScale Postgres databases. PlanetScale Postgres databases previously supported weekly maintenance windows, during which Postgres image upgrades would occur. These required connections to be terminated and re-established on upgraded nodes. To ensure better connection stability for customers, we've moved to optional updates and have temporarily disabled maintenance windows for Postgres databases. You can now [manually update your cluster](/docs/postgres/cluster-configuration/updates) to access new extensions and other software updates. # Cluster configuration parameters Source: https://planetscale.com/docs/postgres/cluster-configuration/parameters You can configure your PlanetScale Postgres cluster settings in the "**Parameters**" tab on the Clusters page for your database. You can configure your PlanetScale Postgres cluster settings in the "**Parameters**" tab on the Clusters page for your database. The defaults for each parameter depend on the configuration of your cluster. The defaults have been chosen to optimize performance, resource usage, and connection handling for each cluster size. However, you are able to fine-tune each of these settings as needed. ## Configuring parameter values You must be a database or organization administrator to modify these settings. 1. From the PlanetScale organization dashboard, select the desired database 2. Navigate to the **Clusters** page from the menu on the left 3. Choose the branch whose parameters you'd like to configure in the "**Branch**" dropdown 4. Select the **Parameters** tab 5. Search for a specific parameter or scroll through the page to see all configurable parameters 6. Update the value for the parameter(s) you wish to adjust 7. Click "**Queue parameter changes**" 8. Once you're ready to apply the changes, click "**Apply changes**" ### Configuring parameters with the CLI You can also inspect and update parameters with the [PlanetScale CLI](/docs/cli/branch): ```bash theme={null} # List the parameters you can change, their current values, and allowed ranges pscale branch parameters list --internal=false # Queue and apply a parameter change (repeat --parameters to set several at once) pscale branch resize --parameters pgconf.max_connections=200 --wait # Check on or cancel a change request pscale branch resize status pscale branch resize cancel ``` The `--internal=false` flag excludes internal parameters, which cannot be changed. Parameters are addressed as `namespace.name`, for example `pgconf.max_connections` for PostgreSQL settings or `pgbouncer.default_pool_size` for PgBouncer settings. See the [`branch` command reference](/docs/cli/branch) for all flags. ## Tracking changes to parameters You can click on the "Changes" tab on the Clusters page to view a log of any changes made to your parameter settings. The log will include the settings affected, the original and updated values, status, user that made the changes, start time, and end time. > [!NOTE] > > When updating a cluster's size, some [parameters](/docs/postgres/cluster-configuration/parameters) will automatically > be adjusted. Each cluster size is associated with default parameter settings, changing the cluster size will also > update those defaults. The exception to this is if you manually override a default parameter setting. In that case, a > cluster size adjustment will not automatically change that setting. ## Parameter change types PostgreSQL parameter changes fall into two categories based on how they are applied: * **Reloadable changes**: The parameter can be updated without restarting PostgreSQL, resulting in zero downtime * **Restart-required changes**: PostgreSQL requires a cluster restart for the parameter to take effect Parameters that require a restart are marked with a ✅ in the "Restart Required" column in the [parameter reference table](#default-parameter-values) below. ### Restart behavior for production clusters When you apply parameters that require a restart, PlanetScale performs a rolling restart process to minimize downtime: 1. Configuration changes are first applied to replica instances and they are restarted 2. Once replicas are ready, a switchover promotes one replica to become the new primary 3. The configuration is applied to the former primary (now a replica) and it is restarted ![Config change with restart](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/cluster-configuration/config-change-restart.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=f3d127b5d60863c67275ba78ddbb75d2) This rolling restart process minimizes downtime, but there remains a brief several-second window of unavailability during the primary switchover. All direct database connections will be terminated during this process, so your application should implement connection retry logic. With the exception of the `Number of processes` parameter for PgBouncer, PgBouncer connections persist through all parameter changes and do not require reconnection. Some parameters are [required by PostgreSQL](https://www.postgresql.org/docs/current/hot-standby.html#HOT-STANDBY-ADMIN) to be applied to the primary before replicas, which may result in a slightly longer unavailability period. ## Default parameter values The following table shows the default values for parameters that are displayed by default to customers. You can find additional parameters in the search field. | Parameter | Restart Required | Description | | - | - | - | | **PgBouncer** | | | | Number of processes | ✅ (restarts PGBouncer only) | Sets the number of PgBouncer processes that will run on each node in this branch's cluster | | default\_pool\_size | | Sets the number of server connections to allow per user/database pair | | max\_client\_conn | | Sets the maximum number of client connections allowed | | max\_db\_client\_connections | | Sets the maximum number of client connections allowed per database (regardless of user). 0 is unlimited | | max\_db\_connections | | Sets the maximum number of server connections allowed per database (regardless of user). 0 is unlimited | | max\_prepared\_statements | | Sets the maximum number of client-prepared statements available across server connections | | max\_user\_connections | | Sets the maximum number of server connections allowed per user (regardless of database). 0 is unlimited | | server\_lifetime | | Sets how long an unused server connection stays open | | server\_idle\_timeout | | Sets how long an idle server connection stays open | | **Resource usage** | | | | effective\_io\_concurrency | | Sets the number of simultaneous requests that can be handled efficiently by the disk subsystem | | effective\_cache\_size | | Sets the planner's assumption about the total size of the data caches | | huge\_pages | ✅ | Controls whether huge pages are requested for the main shared memory area | | maintenance\_io\_concurrency | | Sets the number of simultaneous requests that can be handled efficiently by the disk subsystem for maintenance operations | | maintenance\_work\_mem | | Sets the maximum memory to be used for maintenance operations | | max\_parallel\_maintenance\_workers | | Sets the maximum number of parallel processes per maintenance operation | | max\_parallel\_workers | | Sets the maximum number of parallel workers that can be active at one time | | max\_parallel\_workers\_per\_gather | | Sets the maximum number of parallel processes per executor node | | max\_worker\_processes | ✅ | Sets the maximum number of background processes that the cluster can support | | shared\_buffers | ✅ | Sets the amount of memory the database server uses for shared memory buffers | | work\_mem | | Sets the amount of memory the database will use for internal operations like sorting and hashing | | **Write-ahead log** | | | | max\_slot\_wal\_keep\_size | | Sets the maximum WAL size that can be reserved by replication slots | | max\_wal\_size | | Sets the WAL size that triggers a checkpoint | | min\_wal\_size | | Sets the minimum size to shrink the WAL to | | wal\_buffers | ✅ | Sets the number of disk-page buffers in shared memory for WAL | | wal\_level | ✅ | Sets the level of information written to the WAL | | **Query tuning** | | | | deadlock\_timeout | | Sets the maximum time to wait on a lock before checking for deadlocks | | default\_statistics\_target | | Sets the default statistics target for table columns without a column-specific target set | | random\_page\_cost | | Sets the planner's estimate of the cost of a nonsequentially fetched disk page | | seq\_page\_cost | | Sets the planner's estimate of the cost of a sequentially fetched disk page | | **Connections and authentication** | | | | max\_connections | ✅ | Sets the maximum number of concurrent connections | | **Replication** | | | | hot\_standby\_feedback | | Sends feedback to the primary about queries being executed on the standby. Required for logical replication failover | | max\_logical\_replication\_workers | ✅ | Sets the maximum number of logical replication workers | | max\_replication\_slots | ✅ | Sets the maximum number of replication slots that the server can support | | max\_sync\_workers\_per\_subscription | | Sets the maximum number of synchronization workers per subscription | | max\_wal\_senders | ✅ | Sets the maximum number of WAL senders | | sync\_replication\_slots | | Enables standbys to synchronize logical replication streams from the primary. Required for logical replication failover | | **Failover** | | | | failover\_delay | | Sets the time to wait before triggering a failover to drain inflight transactions | | **Statistics** | | | | track\_io\_timing | | Enables timing of database I/O calls. This may cause significant overhead | | **Logging** | | | | log\_lock\_waits | | Logs the duration of lock waits that exceed the deadlock\_timeout | | log\_min\_duration\_statement | | Sets the minimum execution time above which all statements will be logged | | log\_line\_prefix | | Sets the prefix format for each log line. Uses PostgreSQL format sequences such as %m (timestamp), %p (process ID), %d (database name), and %u (user name) | | **Client connection defaults** | | | | shared\_preload\_libraries | ✅ | Specifies shared libraries to preload into the server at server start | | **Autovacuum** | | | | autovacuum\_vacuum\_scale\_factor | | Specifies a fraction of the table size to add to autovacuum\_vacuum\_threshold when deciding whether to trigger a VACUUM | | autovacuum\_analyze\_scale\_factor | | Specifies a fraction of the table size to add to autovacuum\_analyze\_threshold when deciding whether to trigger an ANALYZE | ## Parameters not in the dashboard The Parameters tab covers the most common cluster-level settings, but PostgreSQL exposes many more configuration parameters than what appears in the dashboard. If a parameter isn't listed in the dashboard, that doesn't mean it can't be changed — it means it's typically managed at a different scope. You can set many other PostgreSQL runtime parameters directly via SQL at the **session**, **role**, or **database** level using standard PostgreSQL commands: * **Session** — Applies only to the current connection. Resets when the connection closes. ```sql theme={null} SET idle_in_transaction_session_timeout = '30s'; ``` * **Role** — Applies to all future sessions for a specific database role. Useful for enforcing defaults per application user. ```sql theme={null} ALTER ROLE app_user SET idle_in_transaction_session_timeout = '30s'; ``` * **Database** — Applies to all future sessions connecting to a specific database. ```sql theme={null} ALTER DATABASE mydb SET idle_in_transaction_session_timeout = '30s'; ``` These SQL-level settings take effect immediately for new sessions and do not require a cluster restart. ### Common examples Some frequently useful parameters that can be set via SQL include: * **`idle_in_transaction_session_timeout`** — Terminates sessions that sit idle inside an open transaction for too long. This is an important safety net for preventing long-lived transactions from holding locks and blocking writes. See also the PgBouncer [`idle_transaction_timeout`](/docs/postgres/connecting/pgbouncer#advanced-pgbouncer-settings) setting for a complementary connection-pooler-level control. * **`statement_timeout`** and **`transaction_timeout`** — Bound how long individual queries and transactions can run. See the [connection resilience guide](/docs/postgres/connection-resilience#postgres-timeouts) for recommended configuration. * **`lock_timeout`** — Prevents statements from waiting indefinitely to acquire locks. Particularly useful for DDL operations to avoid blocking other queries. * **`application_name`** — Identifies your application in monitoring tools and `pg_stat_activity`. > [!NOTE] > > Dashboard parameters are applied at the cluster level and are managed through PlanetScale's rolling restart process. SQL-level settings (`SET`, `ALTER ROLE`, `ALTER DATABASE`) operate at a different scope and do not interact with or override cluster-level parameters set in the dashboard. SQL-level settings take precedence for the sessions they apply to. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Single node Source: https://planetscale.com/docs/postgres/cluster-configuration/single-node Single node Postgres databases come with a single primary and no replicas, and are a great cost-effective option for development or production workloads that do not require high availability. > [!NOTE] > > Single node is only available for network-attached storage databases. You cannot create a single node > [Metal](/docs/metal) database. Single node databases are available on all network-attached storage cluster sizes and begin at \$5/month. For full single node pricing, see the [pricing](https://planetscale.com/pricing) page. To create a single node database, just select "**Single node**" and your cluster size during the database creation process. ## Resizing your single node database You can upsize or downsize your single node Postgres database by going to "**Clusters**", selecting the cluster size from the dropdown, and clicking "**Queue instance changes**", "**Apply changes**". During the resize, we will surge a new replica on the selected size, sync your data, promote the replica to primary, and decommission the old node. ## Switch from single node to HA To upgrade your single node database to an HA cluster, go to the Clusters page, select the "HA Primary + Multi-replica" option, choose your cluster size from the dropdown (network-attached storage or [Metal](/docs/metal)), click "Queue instance changes", and click "Apply changes". During this process, 2 additional replicas across 3 Availability Zones are added to your database cluster. If you selected a Metal cluster, your data is copied over to the new instances with locally-attached storage. It follows the process described in the [Upgrading to a Metal database documentation](/docs/metal/create-a-metal-database#upgrading-an-existing-database-to-metal). ## Switch from HA to single node To switch an HA database with 1 primary and 2+ replicas to single node, go to your Clusters page, select "Single node", choose a cluster size, click "Queue instance changes", and click "Apply changes". This will remove any existing replicas you have and leave you with a single non-HA primary. # Cluster updates Source: https://planetscale.com/docs/postgres/cluster-configuration/updates PlanetScale occasionally releases software updates for your Postgres cluster. ## How to update your cluster To check if your cluster has any available updates, go to the **Clusters** page in your PlanetScale dashboard and look for the "Cluster update available" indicator. > [!NOTE] > > If you make any other cluster changes that require a cluster restart, such as enabling an extension, we will roll out the latest updates during that cluster restart. To update your cluster: **Step 1** Click the **"Queue update"** button on your Clusters page. This will queue the updates without immediately starting the update process. **Step 2** You should see "This cluster has queued changes". You can batch additional cluster changes before applying them if needed. **Step 3** Once you're ready, click **"Apply changes"**. **Step 4** In the confirmation modal, review the warning that applying changes will close any open connections. Make sure your application is equipped to handle closed connections, then click **"Apply changes"** to confirm. **Step 5** You can view the status of your changes under the **Changes** tab. # Supported versions Source: https://planetscale.com/docs/postgres/cluster-configuration/versions When creating a PlanetScale Postgres cluster, you can choose from any of the Postgres versions we support. PlanetScale Postgres supports versions 17 and 18 of Postgres. Specifically, the following versions are currently supported: * 17.11 * 18.6 New databases will be created using the latest version by default and we recommend sticking to that default, but you can choose an older version if you need to. ## Major version upgrades PlanetScale doesn't currently offer in-place upgrades between major versions of Postgres. To upgrade from Postgres 17 to 18, create a new Postgres 18 database and perform an online migration from your existing PlanetScale Postgres 17 database using our [import guides](/docs/postgres/imports/postgres-imports). The same import process also applies if you're migrating from another provider such as RDS, Aurora, or a self-hosted Postgres instance. You can import into any supported major version regardless of where the source database is hosted. ## Benchmarking Postgres 17 vs 18 Our [Benchmarking Postgres 17 vs 18](https://planetscale.com/blog/benchmarking-postgres-17-vs-18) blog post covers sysbench `oltp_read_only` results across multiple EC2 instance types and I/O configurations. * **Postgres 18's new `io_method=worker` default delivers the best overall read performance**, outperforming both `io_uring` and the legacy synchronous I/O in most scenarios. * **Local NVMe disks dramatically outperform network-attached storage** (gp3, io2) regardless of Postgres version or I/O setting. ## What's new in Postgres 18 Postgres 18 builds on Postgres 17 with new features, changed defaults, and some deprecations. For full details, see the official [Postgres 18 release notes](https://www.postgresql.org/docs/18/release-18.html). | Area | What Postgres 18 adds | | :- | :- | | **I/O architecture** | New asynchronous I/O (AIO) subsystem with `io_method` setting (`worker`, `io_uring`, `sync`); up to 3× faster reads in benchmarks | | **Vacuum** | Proactive page-freezing during regular vacuums, reducing the need for aggressive vacuum passes | | **WAL / write throughput** | Per-connection WAL and I/O statistics for finer-grained monitoring | | **Indexes** | Skip-scan lookups on multicolumn B-tree indexes; parallel GIN index builds; broader `OR`-clause index usage | | **`pg_upgrade`** | Retains optimizer statistics through upgrades, reducing post-upgrade performance dips; parallel checks via `--jobs`; `--swap` flag | | **SQL / developer features** | Virtual generated columns (default); `uuidv7()` / `uuidv4()` functions; `OLD`/`NEW` in `RETURNING` for DML statements; temporal `PRIMARY KEY`/`UNIQUE`/`FOREIGN KEY` constraints (`WITHOUT OVERLAPS`, `PERIOD`) | | **Authentication** | OAuth 2.0 authentication; FIPS mode validation; TLS v1.3 cipher suite allowlisting; SCRAM passthrough for `postgres_fdw`/`dblink`; `md5` auth deprecated | | **Logical replication** | Write-conflict logging; parallel streaming for subscriptions by default; `pg_createsubscriber --all` for all databases; auto-invalidation of idle replication slots | | **Observability** | `EXPLAIN ANALYZE` auto-shows buffer access and index-lookup counts; `ANALYZE (VERBOSE)` now reports WAL, CPU, and average read stats | | **Text processing** | `PG_UNICODE_FAST` collation; `casefold()` function; `LIKE` over nondeterministic collations; full-text search uses cluster default collation provider | | **Wire protocol** | Protocol version 3.2 (first update since 2003); `libpq` still defaults to 3.0 for backward compatibility | | **Data checksums** | On by default at `initdb` (off by default in 17) | # Connections overview Source: https://planetscale.com/docs/postgres/connecting There are several ways to connect to Postgres databases, each with their advantages and tradeoffs. ## Connecting to your PlanetScale Postgres database Connecting to your PlanetScale Postgres database involves understanding several key components. This page provides an overview of connection options — for detailed instructions, see the linked documentation below. > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). ### AI assistant integrations You can expose read-only schema and query capabilities to tools like Claude, Cursor, or Zed through the shared [Model Context Protocol (MCP)](/docs/mcp-server) guide. Point each tool at the hosted PlanetScale MCP server, which streams data over a secure, read-only connection. ### Roles and credentials PlanetScale provides two types of roles for database access: * **Default postgres role** — A near-superuser role with extensive permissions, ideal for administrative tasks and initial database setup. This role should not be used for application connections. * **User-defined roles** — Custom roles with specific permission sets that follow the principle of least privilege. These are recommended for all application connections and allow credential rotation without downtime. Connection credentials include a hostname, username (formatted as `{role}.{branch_id}`), password (prefixed with `pscale_pw_`), and database name. Learn more about [managing roles and creating credentials](/docs/postgres/connecting/roles). ### Connection strings PlanetScale databases require SSL/TLS encryption for all connections. Connection strings include parameters for the host, port, username, password, database name, and SSL configuration. The port determines the connection method: * **Port 5432** — Direct connections to Postgres, bypassing PgBouncer * **Port 6432** — Connections through PgBouncer for connection pooling The [connections quickstart](/docs/postgres/connecting/quickstart) provides detailed connection string examples and explains when to use each connection method. ### Private connectivity For enhanced security and reduced latency, PlanetScale supports private connectivity that keeps traffic within cloud provider networks: * **AWS PrivateLink** — Establishes private connections from your AWS VPC to PlanetScale databases without exposing traffic to the public internet. See the [AWS PrivateLink documentation](/docs/postgres/connecting/private-connections/aws-privatelink). * **GCP Private Service Connect** — Provides private connectivity from your Google Cloud VPC to PlanetScale databases. See the [GCP Private Service Connect documentation](/docs/postgres/connecting/private-connections/gcp-private-service-connect). ### Choosing a connection method for your platform Most JavaScript and TypeScript applications should connect with a standard TCP driver such as [`pg`](/docs/postgres/tutorials/planetscale-postgres-node) through [PgBouncer](/docs/postgres/connecting/pgbouncer) on port **6432**. This applies to Vercel (with [`@vercel/functions`](https://vercel.com/guides/connection-pooling-with-functions) connection pooling), AWS Lambda, Railway, Render, and long-running servers. For platforms without reliable TCP pooling, PlanetScale supports the [Neon serverless driver](/docs/postgres/connecting/neon-serverless-driver) over HTTP or WebSockets. Use it for [Netlify Functions](https://docs.netlify.com/functions/overview/), [Deno Deploy](https://deno.com/deploy/docs), and edge runtimes where TCP is not available. For [Cloudflare Workers](https://developers.cloudflare.com/workers/), prefer [Hyperdrive with `pg`](/docs/postgres/tutorials/planetscale-postgres-cloudflare-workers) over the serverless driver when possible. ## Understanding Postgres connections Postgres uses a connection-per-process architecture. Each connection made to a Postgres server [spawns a new process](https://planetscale.com/blog/processes-and-threads), which consumes system resources including memory and CPU. For this reason, it's important to manage the number of direct connections to keep the system performant. Connection pooling is the primary solution to this challenge. In the Postgres ecosystem, [PgBouncer](https://www.pgbouncer.org/) is the most widely-used connection pooler. PgBouncer instances sit between clients and the Postgres server, maintaining a small pool of connections to Postgres while accepting a much larger number of client connections. PgBouncer routes client requests through these pooled connections efficiently. ## Connection options PlanetScale provides several ways to connect to your Postgres database: 1. **Direct primary connections** - Connect directly to your Postgres primary server on port `5432`. This provides the lowest latency and full Postgres session capabilities. Use this for administrative tasks, long-running operations, and data imports. 2. **Direct replica connections** - Connect directly to read-only replicas on port `5432` by appending `|replica` to your username. Use this for read-only queries that can tolerate replication lag. 3. **Local PgBouncer (primary only)** - All Postgres databases include a local PgBouncer running on the same host as the primary. Connect via port `6432`. This is recommended for all application connections to the primary. 4. **Dedicated replica PgBouncer** - Create dedicated PgBouncer instances that pool connections to your replicas. These run on separate nodes and are useful for read-heavy workloads. Connect via port `6432` with the PgBouncer name appended to your username. 5. **Dedicated primary PgBouncer** - Create dedicated PgBouncer instances that pool connections to your primary database. These run on separate nodes and provide improved high availability, with connections persisting through cluster resizes, upgrades, and most failover scenarios. Connect via port `6432` with the PgBouncer name appended to your username. The following sections describe each option in detail to help you choose the right connection method for your use case. ## Direct primary connections Direct connections provide the lowest-latency access to your Postgres primary instance. ![Direct connections](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/postgres/connecting/diagram-direct-connect.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=d14a418b2ca34b52d64211181da8bd48) However, these connections are considered *heavy-weight* since each one consumes significant resources. Direct connections are recommended only for specific scenarios: 1. Administrative tasks, like creating new databases/schemas, manual DDL commands, and installing extensions. 2. Long-running operations like `VACUUM`s and large analytical queries that are executed infrequently. 3. Importing data during a migration or other bulk-loading operations. 4. When you need features like `SET`, pub/sub, and other features not provided by PgBouncer pooled connections. Because having too many direct connections degrades performance, PlanetScale sets `max_connections` to a conservative default value that varies depending on cluster size. To find this value, navigate to the "Clusters" page and select the "Parameters" tab. ![Navigate to the Cluster Parameters page](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/postgres/connecting/cluster-configuration-parameters-darkmode.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=11be7d48f5090ae0fb4442d04bae7d73) Search for `max_connections` to view the current configured value. This can be increased if necessary, though doing so requires careful consideration as increasing direct connections can negatively impact performance. When the `max_connections` limit is reached, error messages like the following will appear: ``` FATAL: sorry, too many clients already ``` Or variations such as: ``` FATAL: remaining connection slots are reserved for non-replication superuser connections ``` For application connections outside of the specific use cases listed above, PgBouncer should be used instead. ## Direct replica connections The main purpose for the default [Replicas](/docs/postgres/scaling/replicas) in a cluster is to maintain [high-availability](/docs/postgres/operations-philosophy), but they can also be used to handle read traffic. Since replicas are read-only, they are only capable of serving `SELECT` queries. All write traffic (`INSERT`, `UPDATE`, etc) must be sent to the primary. ![Direct replica connections](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/postgres/connecting/diagram-replica-direct-connect.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=daa3dc082440c3c1928d615e4c53df6d) Replicas always experience some level of replication lag — the delay between data arriving at the primary and being replicated to a replica. Frequently, replication lag is measured in milliseconds, but it can grow to multiple seconds, especially when the server is experiencing high write traffic or network issues. Because of these factors, queries should only be sent to replicas if they meet the following criteria: (A) they are read-only and (B) they can tolerate being slightly out-of-sync with the data on the primary. For reads that cannot tolerate this lag, send them to the primary. To connect to a replica, append `|replica` to your credential username and use port `5432`. For example: ```bash theme={null} psql 'host=xxxxxxxxxx-useast1-1.horizon.psdb.cloud \ port=5432 \ user=postgres.xxxxxxxxxx|replica \ password=pscale_pw_xxxxxxxxxxxxxxxxxx \ dbname=my_database \ sslnegotiation=direct \ sslmode=verify-full \ sslrootcert=system' ``` Learn more about replicas and when to use them in the [database replicas documentation](/docs/postgres/scaling/replicas). ## PgBouncer connections PgBouncer provides connection pooling for your Postgres database, allowing applications to scale beyond the constraints of direct connections. Connections from application servers should be made via PgBouncer whenever possible. PlanetScale provides three types of PgBouncer instances: All managed PgBouncers run in transaction pooling mode. Session pooling is not offered because it creates a 1:1 mapping between clients and Postgres connections, so it hits the same connection limits as direct connections while introducing additional latency through another proxy layer. If you require session-specific behavior, use a direct connection on port `5432` instead of requesting session pooling. ### Local PgBouncer ![Local PgBouncer connections](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/postgres/connecting/diagram-local-pgbouncer.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=23da28858fc2cb08d57a6d44d571dce6) All PlanetScale Postgres databases include a local PgBouncer instance running on the same host node as the Postgres primary. This is recommended for all application connections to the primary. To connect via the local PgBouncer, use the same credentials as a direct connection but change the port from `5432` to `6432`. > [!NOTE] > > The local PgBouncer only routes connections to the primary. To pool connections to replicas, use a [dedicated replica PgBouncer](#dedicated-replica-pgbouncer). ### Dedicated replica PgBouncer ![Dedicated replica PgBouncer connections](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/postgres/connecting/diagram-dedicated-replica-pgbouncer.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=b455ccb9b6d2063f47bad676115bccd1) [Dedicated replica PgBouncers](/docs/postgres/connecting/pgbouncer#dedicated-replica-pgbouncers) run on nodes separate from the Postgres instances and pool connections to your replicas. These are useful for read-heavy workloads that send significant read traffic to replicas. ### Dedicated primary PgBouncers ![Dedicated primary PgBouncer connections](https://mintcdn.com/planetscale-2/kiyN17feiApl1in5/images/assets/docs/postgres/connecting/diagram-dedicated-primary-pgbouncer.png?fit=max\&auto=format\&n=kiyN17feiApl1in5\&q=85\&s=55b200856bc1bbf5cb7214c27676b24f) [Dedicated primary PgBouncers](/docs/postgres/connecting/pgbouncer#dedicated-primary-pgbouncers) provide connection pooling for your primary database on nodes separate from the Postgres servers. Traffic routes from the dedicated primary PgBouncer to the local PgBouncer on the primary node, and then to Postgres. This lets client connections persist through cluster resizes, upgrades, and most failover scenarios, providing improved high availability. ## Connecting to dedicated PgBouncers Connect to replica or primary PgBouncers via port `6432` and append the name of the PgBouncer to your username. For example, if your PgBouncer is named `read-bouncer`, the connection username should be `postgres.xxxxxxxxxx|read-bouncer`. ```bash theme={null} psql 'host=xxxxxxxxxx-useast1-1.horizon.psdb.cloud \ port=6432 \ user=postgres.xxxxxxxxxx|read-bouncer \ password=pscale_pw_xxxxxxxxxxxxxxxxxx \ dbname=my_database \ sslnegotiation=direct \ sslmode=verify-full \ sslrootcert=system' ``` Learn more about [creating, configuring, and connecting to PgBouncers](/docs/postgres/connecting/pgbouncer). # PostgreSQL Proxy - Client Error Reference Source: https://planetscale.com/docs/postgres/connecting/client-error-reference This document provides a comprehensive reference of all error messages that the Exosphere PostgreSQL proxy may send to clients. These errors follow the PostgreSQL wire protocol format and include standard SQLSTATE error codes. ## Error format All errors sent by Exosphere follow the PostgreSQL ErrorResponse message format: | Field | Description | | :- | :- | | Severity | ERROR, FATAL, PANIC, WARNING, NOTICE, etc. | | Code | 5-character SQLSTATE code (e.g., "28P01") | | Message | Human-readable error description | | Hint | Optional suggestion for resolution | ## Error severity levels | Severity | Description | Client Action | | - | - | - | | **FATAL** | Connection-terminating error | Must reconnect | | **ERROR** | Request failed but connection remains | Can retry or continue | | **WARNING** | Potential issue but operation continues | Take note, prepare for action | | **NOTICE** | Informational message | For awareness only | ## Error categories ### Authentication and authorization errors #### SSL/TLS required * **Severity:** FATAL * **Code:** 28000 (invalid\_authorization\_specification) * **Message:** "SSL connection is required" * **Hint:** "Use sslmode=require or connect with SSL enabled" * **When:** Client attempts unencrypted connection when TLS is mandatory * **Resolution:** Configure client to use SSL/TLS (e.g., `sslmode=verify-full` in connection string) #### Invalid user format * **Severity:** FATAL * **Code:** 28000 (invalid\_authorization\_specification) * **Message:** "invalid user format: username must include branch (e.g., user.branch)" * **When:** Username doesn't follow required format for branch routing * **Resolution:** Include branch identifier in username (format: `username.branchname`) #### Authentication failure * **Severity:** FATAL * **Code:** 28P01 (invalid\_password) * **Message:** Various authentication-specific messages * **When:** Password validation fails * **Resolution:** Verify credentials are correct #### Insufficient schema privileges * **Severity:** ERROR * **Code:** 42501 (insufficient\_privilege) * **Message:** "permission denied for schema public" * **When:** A role runs DDL (such as `CREATE TABLE`) in a schema where it lacks the `CREATE` privilege. Data permissions like `pg_read_all_data`/`pg_write_all_data` do not grant DDL rights. * **Resolution:** Grant the role `CREATE` on the schema (`GRANT CREATE ON SCHEMA public TO "";`) as the default `postgres` role, or run DDL as the `postgres` role. See [Managing roles](/docs/postgres/connecting/roles#user-defined-roles). ### Connection and network errors #### Startup message errors * **Severity:** FATAL * **Code:** 08006 (connection\_failure) * **Messages:** * "failed to read startup message" * "startup message too short: %d bytes" * "incomplete startup message: expected %d bytes, got %d" * "failed to parse startup header" * **When:** Initial connection handshake fails * **Resolution:** Likely a client library bug - check for driver updates #### Backend connection failures * **Severity:** FATAL * **Code:** 08006 (connection\_failure) * **Messages:** * "failed to connect to upstream" * "failed to send startup message" * "failed to setup backend" * "connection retry timeout - branch %s unavailable after %s" * **When:** Proxy cannot establish connection to backend database * **Resolution:** Retry with exponential backoff - this is often transient ### Routing and branch resolution errors #### Branch not found * **Severity:** FATAL * **Code:** 28000 (invalid\_authorization\_specification) * **Message:** "branch %s does not exist" * **When:** Specified branch identifier is not recognized * **Resolution:** Verify branch name is correct #### Member not found * **Severity:** FATAL * **Code:** 28000 (invalid\_authorization\_specification) * **Message:** "member %s not found in branch %s" * **When:** Specific database member requested doesn't exist * **Resolution:** Check member name and branch configuration #### No primary available * **Severity:** FATAL * **Code:** 08006 (connection\_failure) * **Message:** "no primary available for branch %s" * **When:** Primary database instance is unavailable * **Resolution:** Database outage - retry with exponential backoff #### No replica available * **Severity:** FATAL * **Code:** 08006 (connection\_failure) * **Message:** "no replica available for branch %s" * **When:** No read replicas are available for the branch * **Resolution:** All replicas are down - retry with exponential backoff #### No running members * **Severity:** FATAL * **Code:** 08006 (connection\_failure) * **Message:** "no running members available for branch %s" * **When:** All database instances in branch are down * **Resolution:** Total database outage - retry, but investigate the cause as this indicates all instances are down #### Pooler restriction * **Severity:** FATAL * **Code:** 28000 (invalid\_authorization\_specification) * **Message:** "pooler only supports primary destinations for branch %s" * **When:** Attempting to use pooler with non-primary target * **Resolution:** Connect to replicas via port 5432 instead of 6432 (pgbouncer port doesn't support replicas) ## SQLSTATE error codes Exosphere uses standard PostgreSQL SQLSTATE codes for compatibility: | Code | Class | Description | Common Scenarios | | :- | :- | :- | :- | | **08006** | Connection Exception | connection\_failure | Network issues, backend unavailable | | **22001** | Data Exception | string\_data\_right\_truncation | Value exceeds field length | | **23505** | Integrity Constraint | unique\_violation | Duplicate key violation | | **28000** | Invalid Authorization | invalid\_authorization\_specification | Auth configuration issues | | **28P01** | Invalid Authorization | invalid\_password | Authentication failure | ## Client library considerations ### Connection retry logic * Errors with code `08006` are typically transient and safe to retry * Errors with code `28000` or `28P01` indicate configuration issues - don't retry without changes ### Error handling best practices 1. Always check the SQLSTATE code, not just the message text 2. Implement exponential backoff for connection failures (08006) 3. Handle shutdown notices gracefully by proactively reconnecting 4. Log full error details including severity and code for debugging ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # IP restrictions Source: https://planetscale.com/docs/postgres/connecting/ip-restrictions Manage which IP addresses can establish connections to your database. You can manage IP address restrictions for database connections in the "**IP restrictions**" tab under Settings for your database. IP restrictions control which networks can connect to your database, providing an additional layer of security beyond authentication. IP restrictions restrict database connections to the specified IP ranges. Rules apply to all roles and schemas unless specified otherwise. ## Creating an IP restriction rule You must be a database or organization administrator to create or modify an IP restriction rule. **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to **Settings** from the menu on the left **Step 3** Select the **IP restrictions** tab **Step 4** Click "**New rule**" **Step 5** Configure the rule settings: * **Role** (optional): Leave empty to apply to all PostgreSQL roles, or specify a particular role. If specifying a role name, please note that the descriptive name cannot be used. Instead, you will use the role username as provided in the UI with the following caveat: rather than use the full value, e.g., `pscale_api_.`, you should only use the first part, `pscale_api_`. * **Schema** (optional): Leave empty to apply to all PostgreSQL schemas, or specify a particular schema * **IP ranges** (required): Enter a comma-separated list of IP addresses or CIDR ranges (e.g., `1.2.3.4/32, 10.0.0.0/8`) **Step 6** Click "**Create rule**" ## How IP restriction rules work IP restrictions restrict database connections to the specified IP ranges. The behavior depends on how you configure each rule: * **Apply to all roles and schemas**: Leave both the **Role** and **Schema** fields empty * **Apply to specific role**: Specify a role name in the **Role** field to restrict connections for that role across all schemas * **Apply to specific schema**: Specify a schema name in the **Schema** field to restrict connections to that schema from all roles * **Apply to specific role and schema**: Specify both to create a rule that applies only when that role connects to that schema Multiple rules can be created to build complex access policies for your database cluster. ## IP range format The **IP ranges** field accepts: * Individual IP addresses in CIDR notation (e.g., `1.2.3.4/32`) * IP ranges in CIDR notation (e.g., `10.0.0.0/8`, `192.168.1.0/24`) * Multiple entries separated by commas (e.g., `1.2.3.4/32, 10.0.0.0/8`) ## Editing an IP restriction rule **Step 1** Navigate to **Settings** → **IP restrictions** **Step 2** Click the menu icon (**...**) on the right side of the rule you want to edit **Step 3** Select **Edit** from the menu **Step 4** Modify the rule settings as needed: * **Role**: Change the role or leave empty to apply to all PostgreSQL roles * **Schema**: Change the schema or leave empty to apply to all PostgreSQL schemas * **IP ranges**: Update the comma-separated list of IP addresses or CIDR ranges **Step 5** Click "**Update rule**" to save your changes ## Deleting an IP restriction rule **Step 1** Navigate to **Settings** → **IP restrictions** **Step 2** Click the menu icon (**...**) on the right side of the rule you want to delete **Step 3** Select **Delete** from the menu **Step 4** Review the confirmation dialog showing the rule details (role, schema, and IP ranges) **Step 5** Click "**Delete rule**" to confirm deletion, or "**Cancel**" to keep the rule > [!WARNING] > > Deleting an IP restrictions rule is irreversible. After deletion, connections from those IP ranges will no longer be restricted, potentially allowing broader access to your database. > [!NOTE] > > Rule creation and modification only applies to connections established after the change. > It does not impact or disconnect existing connections, even if they break the newly-established rules. ## Using IP restrictions with private connections If you're connecting to your database through [AWS PrivateLink](/docs/postgres/connecting/private-connections/aws-privatelink) or [GCP Private Service Connect](/docs/postgres/connecting/private-connections/gcp-private-service-connect), the source IP address that PlanetScale sees is not your application's IP address or your VPC's CIDR range. Instead, PlanetScale sees the private IP address from our internal network infrastructure that handles the private connection. ### Restricting access to private connections only If you want to ensure your database only accepts connections through PrivateLink or Private Service Connect (blocking all public internet access), you should configure your IP restrictions to allow the following RFC1918 private IP ranges: | CIDR Range | Description | | - | - | | `10.0.0.0/8` | Class A private range | | `172.16.0.0/12` | Class B private range | | `192.168.0.0/16` | Class C private range | By allowing only these private ranges, you effectively block all connections from the public internet while still allowing connections through your private endpoint. > [!NOTE] > > Adding IP restrictions to your database is independent from setting up PrivateLink or Private Service Connect. Simply enabling a private connection does not automatically block public internet access. You must explicitly configure IP restrictions if you want to enforce private-only connectivity. **Example configuration:** To restrict a database to only accept connections through private endpoints, create an IP restriction rule with the following IP ranges: ``` 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 ``` This configuration allows connections from any of the RFC1918 private IP ranges while blocking all public internet traffic. ## Best practices When configuring IP restrictions rules: * Start with the most restrictive rules that meet your requirements * Use CIDR notation to define ranges efficiently (e.g., `/24` for a subnet rather than listing individual IPs) * Document the purpose of each rule by using descriptive role names or organizing rules by application * Regularly audit your IP restrictions rules to remove access that is no longer needed * Consider creating separate roles for different applications or environments to enable fine-grained access control * Complement network restrictions with [pg\_strict](/docs/postgres/extensions/pg_strict) for query-level protection against accidental mass mutations ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Using the Neon serverless driver Source: https://planetscale.com/docs/postgres/connecting/neon-serverless-driver The [Neon serverless driver](https://neon.com/docs/serverless/serverless-driver) (`@neondatabase/serverless`) runs Postgres queries from JavaScript and TypeScript over HTTP or WebSockets instead of TCP. PlanetScale supports this driver on platforms where standard TCP drivers and connection pooling are not viable. For full API details, see [Neon's serverless driver documentation](https://neon.com/docs/serverless/serverless-driver) and the [GitHub repository](https://github.com/neondatabase/serverless). > [!NOTE] > > Driver v1.0.0 and later require Node.js 19 or higher. ## When to use this driver Use the Neon serverless driver when your deployment platform cannot maintain TCP connections or a connection pool between requests. | Platform | Recommended approach | | - | - | | [Netlify Functions](https://docs.netlify.com/functions/overview/) | Neon serverless driver (this page) | | [Deno Deploy](https://deno.com/deploy/docs) | Neon serverless driver (this page) | | [Cloudflare Workers](https://developers.cloudflare.com/workers/) (without Hyperdrive) | Neon serverless driver (this page) | | [Vercel](https://vercel.com/docs/functions) | [`pg`](https://node-postgres.com/) with [`@vercel/functions`](https://vercel.com/guides/connection-pooling-with-functions) and [PgBouncer on port 6432](/docs/postgres/connecting/pgbouncer) | | [Cloudflare Workers](https://developers.cloudflare.com/workers/) + Hyperdrive | [`pg` with Hyperdrive](/docs/postgres/tutorials/planetscale-postgres-cloudflare-workers) | | AWS Lambda, Railway, Render, VPS, Docker | [`pg`](/docs/postgres/tutorials/planetscale-postgres-node) with [PgBouncer on port 6432](/docs/postgres/connecting/pgbouncer) | ### Vercel [Vercel Fluid compute](https://vercel.com/docs/functions/fluid-compute) reuses warm function instances, making TCP connection pooling safe. For Vercel deployments, connect with [`pg`](https://node-postgres.com/) through [PgBouncer](/docs/postgres/connecting/pgbouncer) on port **6432** and pool connections with [`attachDatabasePool`](https://vercel.com/guides/connection-pooling-with-functions) from `@vercel/functions`. The Neon serverless driver remains an option on Vercel when you cannot use connection pooling (for example, classic serverless without Fluid). See [Neon's Vercel connection guide](https://neon.com/docs/guides/vercel-connection-methods) for a comparison of TCP, HTTP, and WebSocket latency on that platform. ### Cloudflare Workers For Cloudflare Workers, [Hyperdrive with `pg`](/docs/postgres/tutorials/planetscale-postgres-cloudflare-workers) is the recommended approach. It provides connection pooling and lower latency for warm connections. Use the Neon serverless driver only when Hyperdrive is not available for your setup. ### Deno Deploy Install the driver from JSR for Deno projects: ```bash theme={null} deno add jsr:@neon/serverless ``` See [Neon's Deno Deploy guide](https://neon.com/docs/guides/deno) for deployment details. ## HTTP vs WebSocket modes The Neon serverless driver supports two connection modes: * **HTTP mode** — Uses the `neon` function to execute queries over HTTP. Faster for single queries and non-interactive transactions. No connection state is maintained between requests. * **WebSocket mode** — Uses the `Pool` object to establish a WebSocket connection. Required for session support, interactive transactions, or `node-postgres` compatibility. PlanetScale supports both modes. Choose based on your use case: | Use case | Recommended mode | | - | - | | Single queries | HTTP | | Non-interactive transactions (batch of queries) | HTTP | | Interactive transactions | WebSocket | | Session-based features | WebSocket | | `node-postgres` compatibility | WebSocket | ## Setting up credentials Both modes use the same credentials setup. **Step 1** Install the driver via npm: ```bash theme={null} npm install @neondatabase/serverless ``` **Step 2** You'll need to create a [Postgres role](/docs/postgres/connecting/roles) to use with the driver. Once you have these credentials, place them in environment variables: ```bash theme={null} DATABASE_HOST=XXXX.pg.psdb.cloud DATABASE_PORT=5432 DATABASE_NAME=XXXX DATABASE_USERNAME=XXXX DATABASE_PASSWORD=pscale_pw_XXXX ``` These can all be added to a unified Postgres connection URL for use by the driver: ```bash theme={null} DATABASE_URL="postgresql://$DATABASE_USERNAME:$DATABASE_PASSWORD@$DATABASE_HOST:$DATABASE_PORT/$DATABASE_NAME" ``` ## Using HTTP mode HTTP mode is the simplest way to execute queries. You must configure the `fetchEndpoint` to use PlanetScale's SQL endpoint. ```ts theme={null} import { neon, neonConfig } from "@neondatabase/serverless"; // This MUST be set for PlanetScale Postgres connections neonConfig.fetchEndpoint = (host) => `https://${host}/sql`; const sql = neon(process.env.DATABASE_URL!); const posts = await sql`SELECT * FROM posts WHERE id = ${postId}`; ``` The `neon` function returns a tagged template literal that automatically handles parameterized queries, protecting against SQL injection. For manually parameterized queries, use `sql.query()`: ```ts theme={null} const posts = await sql.query("SELECT * FROM posts WHERE id = $1", [postId]); ``` See [Neon's HTTP mode documentation](https://neon.com/docs/serverless/serverless-driver#use-the-driver-over-http) for `arrayMode`, `fullResults`, `fetchOptions`, and other configuration options. ### Non-interactive transactions HTTP mode supports non-interactive transactions where you send a batch of queries to be executed together. Use the `transaction` function: ```ts theme={null} import { neon, neonConfig } from "@neondatabase/serverless"; // This MUST be set for PlanetScale Postgres connections neonConfig.fetchEndpoint = (host) => `https://${host}/sql`; const sql = neon(process.env.DATABASE_URL!); const [posts, tags] = await sql.transaction( [ sql`SELECT * FROM posts ORDER BY posted_at DESC LIMIT ${limit}`, sql`SELECT * FROM tags`, ], { isolationLevel: "ReadCommitted", readOnly: true } ); ``` ## Using WebSocket mode WebSocket mode provides a full `Pool` interface compatible with the `pg` library. See [Neon's WebSocket documentation](https://neon.com/docs/serverless/serverless-driver#use-the-driver-over-websockets) for the full API reference. This mode requires additional configuration for PlanetScale connections. > [!WARNING] > > In serverless and edge environments, WebSocket connections cannot outlive a single request. Create, use, and close `Pool` or `Client` objects within the same request handler. Do not create pools outside a handler or reuse them across handlers. **Step 1** When connecting, you must set the following configuration options: ```ts theme={null} neonConfig.pipelineConnect = false; neonConfig.wsProxy = (host, port) => `${host}/v2?address=${host}:${port}`; ``` **Step 2** Here's a complete example: ```ts theme={null} import ws from "ws"; import { Pool, neonConfig } from "@neondatabase/serverless"; neonConfig.webSocketConstructor = ws; // These MUST be set for PlanetScale Postgres connections neonConfig.pipelineConnect = false; neonConfig.wsProxy = (host, port) => `${host}/v2?address=${host}:${port}`; const pool = new Pool({ connectionString: process.env.DATABASE_URL }); const posts = await pool.query("SELECT * FROM posts WHERE id = $1", [postId]); pool.end(); ``` > [!NOTE] > > In browser or edge environments that have a native `WebSocket` global, you don't need to import `ws` or set `neonConfig.webSocketConstructor`. ## Security PlanetScale requires `SCRAM-SHA-256` for all authentication to Postgres servers. We maintain this strict requirement for security purposes. For WebSocket connections, you must set `neonConfig.pipelineConnect = false;`. This adds a bit of additional latency, but is necessary to connect using `SCRAM-SHA-256`. When this is `"password"` (the default value) it requires using cleartext password authentication, reducing connection security. HTTP mode connections handle authentication automatically and don't require this configuration. # PgBouncer Source: https://planetscale.com/docs/postgres/connecting/pgbouncer PgBouncer provides connection pooling for your Postgres database. ## When to use PgBouncer PgBouncer is generally recommended for OLTP workloads. All application connections should be routed through PgBouncer whenever possible. Learn more about the pros and cons of the different connection methods on the [connections overview page](/docs/postgres/connecting). PlanetScale provides several options for using PgBouncer, including local PgBouncers, dedicated primary PgBouncers, and dedicated replica PgBouncers. PgBouncer connections operate in transaction mode, which means pooled server connections are assigned to client connections on a per-transaction level. This provides excellent performance for OLTP workloads but limits certain PostgreSQL features that require persistent connections. Learn more at the [PgBouncer documentation](https://www.pgbouncer.org/features.html). ## When to NOT use PgBouncer For use cases that require long-running operations, direct connections on port `5432` are recommended. For example: * Schema changes and DDL * OLAP, analytics, reporting, or batch processing * Session-specific features: Custom session variables, temporary tables * ETL processes and data streaming * Long-running transactions or queries that span multiple transactions * Creating a local backup with `pg_dump` ## Local PgBouncer Every PlanetScale Postgres database includes an instance of PgBouncer running on the same node as the primary Postgres database (local PgBouncer). To connect to PgBouncer, use the same credentials as a direct connection, but use port `6432` instead of the Postgres default of `5432`. For example: ```bash theme={null} psql 'host=xxxxxxxxxx-useast1-1.horizon.psdb.cloud port=6432 user=postgres.xxxxxxxxxx password=pscale_pw_xxxxxxxxxxxxxxxxxx dbname=my_database sslnegotiation=direct sslmode=verify-full sslrootcert=system' ``` > [!NOTE] > > The local PgBouncer does not support routing queries to replicas. All connections through the local PgBouncer are automatically routed to the primary database, regardless of the username specification. Use a [dedicated replica PgBouncer](#dedicated-replica-pgbouncers) for replica access. ## Dedicated replica PgBouncers Dedicated replica PgBouncers can be created to run on nodes separate from the Postgres servers. This is useful for applications that send significant read traffic to replicas and need connection pooling. This offers similar high-availability benefits as the local PgBouncer but is used for read-only replica traffic. ### Creating a dedicated replica PgBouncer You must be a database or organization administrator to create PgBouncers. 1. From the PlanetScale organization dashboard, select the desired database 2. Navigate to the **Clusters** page from the menu on the left 3. Choose the branch where you want to add a PgBouncer in the "**Branch**" dropdown 4. Select the **PgBouncers** tab 5. Scroll down to the "**Dedicated replica PgBouncers**" section 6. Click the "**Add a replica PgBouncer**" button ![Dedicated replica PgBouncer](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/dedicated-replica-pgbouncer-darkmode.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=c4fe58ffe61cf43cd00bf19e30e7b3c8) 7. In the pop-up dialog, give the new PgBouncer a descriptive name. Note that names can not be modified after creation. 8. Select a size based on your connection pooling needs (see [PgBouncer pricing](/docs/postgres/pricing#pgbouncer-pricing) for available sizes) ![Create a PgBouncer](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/create-pgbouncer-darkmode.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=fb8b7b3c7ab9b4b4860d1b30c8adea74) 9. Click "**Create PgBouncer**" 10. Wait a few minutes for the creation to complete A new entry for the PgBouncer will appear in the Dedicated replica PgBouncers section once provisioning is complete. Multiple replica PgBouncers can be created if needed. This is useful for adding additional PgBouncer capacity or for having distinct bouncers for different client applications to manage connection pooling with more precision. #### Availability zone affinity Dedicated replica PgBouncers can be configured to prefer routing to the Postgres replica servers inside their own availability zone. Applications deployed across several zones can benefit from lower replica query latency in this configuration. However, if your application is deployed to a single zone, this mode may direct most queries to one replica server while replicas in other zones receive little traffic. Allowing the bouncer to load balance across availability zones, without preferring its own zone, will spread the query volume across the replica servers for single-zone applications. Select the **Prefer routing to replicas in the same availability zone** checkbox to enable affinity. ### Connecting to dedicated replica PgBouncers Connect to dedicated replica PgBouncers by appending `|pgbouncer-name` to the username of any [role you have created](/docs/postgres/connecting/roles). For example, if your username is `user1.abcdefghi` and the dedicated replica PgBouncer is named `read-bouncer`, the connection username should be `user1.abcdefghi|read-bouncer`. The hostname and password remain the same. Use port `6432` for dedicated PgBouncer connections: ```bash theme={null} psql 'host=xxxxxxxxxx-useast1-1.horizon.psdb.cloud \ port=6432 \ user=postgres.xxxxxxxxxx|read-bouncer \ password=pscale_pw_xxxxxxxxxxxxxxxxxx \ dbname=my_database \ sslnegotiation=direct \ sslmode=verify-full \ sslrootcert=system' ``` ## Dedicated primary PgBouncers A dedicated primary PgBouncer provides connection pooling for your primary database while running on nodes separate from the Postgres servers. Traffic flows from the dedicated primary PgBouncer to the local PgBouncer on the primary node, and then to Postgres. This extra layer lets client connections persist through cluster resizes, upgrades, and most failover scenarios, providing improved high availability. Primary bouncers are configured in the same way as replica bouncers. For production OLTP workloads requiring maximum availability, see the [connection resilience guide](/docs/postgres/connection-resilience). > [!NOTE] > > A dedicated pool runs multiple PgBouncers spread across availability zones for resiliency (configured with **PgBouncers per availability zone**). Connection-count settings such as `max_client_conn` and `default_pool_size` apply to each PgBouncer in the pool, so the pool's effective totals are the sum across all of them: the client-connection ceiling shown in the dashboard is the total across every PgBouncer (for example, three PgBouncers each with `max_client_conn = 100` give a ceiling of 300), and total server connections to Postgres are likewise the sum of each PgBouncer's `default_pool_size`. See [Configuring PgBouncers](#configuring-pgbouncers). ### Connecting to dedicated primary PgBouncers Connect to dedicated primary PgBouncers by appending `|pgbouncer-name` to the username of any [role you have created](/docs/postgres/connecting/roles). For example, if your username is `user1.abcdefghi` and the dedicated primary PgBouncer is named `write-pool`, the connection username should be `user1.abcdefghi|write-pool`. The hostname and password remain the same. Use port `6432` for dedicated PgBouncer connections. For dedicated primary PgBouncers, requests route through the dedicated primary PgBouncer first, then the local PgBouncer, and finally Postgres: ```bash theme={null} psql 'host=xxxxxxxxxx-useast1-1.horizon.psdb.cloud \ port=6432 \ user=postgres.xxxxxxxxxx|write-pool \ password=pscale_pw_xxxxxxxxxxxxxxxxxx \ dbname=my_database \ sslnegotiation=direct \ sslmode=verify-full \ sslrootcert=system' ``` ## Configuring PgBouncers Each PgBouncer on the "PgBouncers" tab can be individually configured with a section like this under each PgBouncer: ![Configure a PgBouncer](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/pgbouncer-settings-darkmode.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=72389b7ec1f889a41cedc22bff0ad3f2) The basic settings are at the top, with advanced settings available as an option. Adjusting advanced settings is not recommended unless there is a good understanding of how PgBouncer works. ### Applying configuration changes When you save a PgBouncer configuration change, PlanetScale applies it with a live reload. PgBouncer re-reads the updated configuration without a process restart. In normal operation, existing client connections are preserved during this reload, so applying configuration changes does not introduce downtime or a connection reset event. ### Configurable parameters The following parameters can be configured for both the local and dedicated replica PgBouncers. #### Basic settings | Parameter | Description | | - | - | | default\_pool\_size | How many server connections to allow per user/database pair. Default: `20` | | min\_pool\_size | Add more server connections to pool if below this number. Improves behavior when load returns after inactivity. Default: `0` | | max\_client\_conn | Maximum number of client connections allowed, per PgBouncer. Default: `100` | | server\_lifetime | The pooler will close unused server connections that have been connected longer than this. 0 means use once then close. Default: `3600` seconds | | server\_idle\_timeout | Close server connections idle longer than this many seconds. 0 disables this timeout. Default: `600` seconds | #### Advanced settings Advanced parameters should only be adjusted with a thorough understanding of PgBouncer internals. | Parameter | Description | | - | - | | **Connection Limits** | | | max\_prepared\_statements | When non-zero, PgBouncer tracks protocol-level named prepared statements in transaction and statement pooling mode. Default: `200` | | max\_db\_connections | Do not allow more than this many server connections per database (regardless of user). 0 is unlimited. Default: `0` | | max\_db\_client\_connections | Do not allow more than this many client connections per database (regardless of user). 0 is unlimited. Default: `0` | | max\_user\_connections | Do not allow more than this many server connections per user (regardless of database). 0 is unlimited. Default: `0` | | max\_user\_client\_connections | Do not allow more than this many client connections per user (regardless of database). 0 is unlimited. Default: `0` | | reserve\_pool\_size | How many additional connections to allow to a pool. 0 disables. Default: `0` | | reserve\_pool\_timeout | If a client has not been serviced in this time, use additional connections from the reserve pool. 0 disables. Default: `5` seconds | | **Timeouts** | | | query\_timeout | Cancel queries running longer than this. Use with smaller server-side statement\_timeout for network problems. Default: `0` seconds | | query\_wait\_timeout | Maximum time queries wait for execution. Client disconnected if query not assigned to server in time. 0 disables. Default: `120` seconds | | client\_idle\_timeout | Close client connections idle longer than this. Should be larger than client-side lifetime settings. Default: `0` seconds | | client\_login\_timeout | Disconnect clients that don't log in within this time. Prevents dead connections stalling SUSPEND and restart. Default: `60` seconds | | idle\_transaction\_timeout | If a client has been in "idle in transaction" state longer, it will be disconnected. Default: `0` seconds | | cancel\_wait\_timeout | Maximum time cancel requests wait for execution. Client disconnected if not assigned to server in time. 0 disables. Default: `10` seconds | | autodb\_idle\_timeout | How long database pools stay cached after last use. After timeout, unused pools are freed and stats reset. Default: `3600` seconds | | suspend\_timeout | How long to wait for buffer flush during SUSPEND or reboot (-R). Connection dropped if flush fails. Default: `10` seconds | | **Server Health** | | | server\_check\_query | Simple query to check if server connection is alive. Empty string disables sanity checking. Default: `select 1` | | server\_check\_delay | How long to keep released connections available for immediate re-use without running server\_check\_query. 0 always runs check. Default: `30` seconds | | **Logging** | | | log\_connections | Log successful logins. Default: `1` (enabled) | | log\_disconnections | Log disconnections with reasons. Default: `1` (enabled) | | log\_pooler\_errors | Log error messages the pooler sends to clients. Default: `1` (enabled) | | **Parameter Handling** | | | ignore\_startup\_parameters | Allow additional startup parameters that PgBouncer normally rejects. Specify here so PgBouncer knows admin handles them. Default: `extra_float_digits` | | track\_extra\_parameters | Additional parameters to track per client beyond the defaults. Maintained in client cache and restored when client active. Default: `IntervalStyle` | | **Low-Level Performance** | | | pkt\_buf | Internal buffer size for packets. Affects TCP packet size and memory usage. No need to set large for libpq packets. Default: `4096` bytes | | sbuf\_loopcnt | How many times to process data on one connection before proceeding. Prevents big result sets stalling PgBouncer. 0 = no limit. Default: `5` | | disable\_pqexec | Disable Simple Query protocol (PQexec). Improves security by preventing some SQL injection attacks. 0 = enabled, 1 = disabled. Default: `0` | | **Infrastructure (Local PgBouncer only)** | | | Number of processes | Sets the number of PgBouncer processes that will run on each node in this branch's cluster. Default: `1` | Learn more about PgBouncer configuration [on their official website](https://www.pgbouncer.org/features.html). ## How PgBouncer works Connection reuse is the key mechanism that makes PgBouncer effective. When a client completes a transaction, PgBouncer returns the server connection to the pool rather than closing it. The next client transaction can immediately reuse that existing connection without incurring the overhead of spawning a new Postgres process. This allows a single pooled connection to serve hundreds or thousands of client connections over its lifetime, enabling applications to scale far beyond the constraints of direct connections. ### Pooling modes PgBouncer supports three pooling modes, but PlanetScale-managed PgBouncers operate in transaction pooling mode only. * **Transaction pooling**: Assigns client connections to pooled server connections on a per-transaction level and allows multi-statement transactions. This mode provides the multiplexing benefits PlanetScale optimizes for and is the default on every PgBouncer we run. * **Statement pooling**: Assigns client connections to pooled server connections on a per-query basis. This mode does not allow multi-statement transactions, which is unsuitable for most use cases. * **Session pooling**: Each client connection is given a dedicated connection from the PgBouncer pool for its entire duration. Because the mapping stays 1:1, Postgres connection counts do not decrease, connection limits are reached at the same rate, and PgBouncer adds another proxy layer that introduces latency. If your workload needs persistent sessions, connect directly to Postgres on port `5432` instead of adding a PgBouncer hop. If you are worried about exhausting connection limits by connecting directly, switching to session pooling reaches the same limit while still adding another proxy layer. ### Limitations of transaction pooling PgBouncer's transaction pooling mode provides excellent performance for OLTP workloads but limits certain PostgreSQL features that require persistent connections: * Prepared statements that persist across transactions (protocol-level prepared statements work with `max_prepared_statements` configured) * Temporary tables * `LISTEN`/`NOTIFY` * Session-level advisory locks * `SET` commands that persist beyond a transaction For operations requiring these features, use a direct connection instead (see the [connections overview](/docs/postgres/connecting#direct-primary-connections)). ### Benefits during maintenance operations Using PgBouncer provides improved availability during configuration changes. When modifying Postgres Parameters, some changes require the server to be restarted. When these restarts happen, any direct connections to Postgres will be terminated. However, when using PgBouncer, client connections are maintained and PgBouncer handles reconnecting to Postgres after it restarts. The [operations philosophy documentation](/docs/postgres/operations-philosophy) covers more details on how connections are managed during various database lifecycle operations. ### Scaling PgBouncer PgBouncer itself is a lightweight process, but high connection volumes or high query throughput can eventually exhaust its capacity. PlanetScale offers multiple PgBouncer sizes to handle different workload demands. Each size provides increased CPU and memory resources, allowing PgBouncer to handle more concurrent client connections and higher query throughput without becoming a bottleneck. See [PgBouncer pricing](/docs/postgres/pricing#pgbouncer-pricing) for available sizes. ### PgBouncer error messages PgBouncer has custom error messages that may be encountered in addition to standard Postgres errors. The [PgBouncer config documentation](https://www.pgbouncer.org/config.html) describes these errors and can be a helpful resource for troubleshooting connection issues. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect privately with AWS PrivateLink Source: https://planetscale.com/docs/postgres/connecting/private-connections/aws-privatelink When you use AWS PrivateLink, your network traffic between your VPC and PlanetScale stays within the AWS network, without traversing the public internet. [AWS PrivateLink](https://aws.amazon.com/privatelink/) is a highly available, scalable technology that enables you to privately connect your VPC to supported AWS services, VPC endpoint services, and AWS Marketplace partner services. ### When to use AWS PrivateLink By default, PlanetScale Postgres databases use secure connections over the public internet with industry-standard TLS encryption. This approach is secure and meets the needs of most customers. However, you may want to consider AWS PrivateLink if: * **Compliance requirements**: Your organization has stronger regulatory or compliance mandates that require database connections to avoid the public internet entirely * **Enhanced security posture**: You want an additional layer of network isolation for sensitive data workloads * **Network architecture**: Your existing AWS infrastructure is designed around private connectivity patterns * **Reduced network latency**: AWS PrivateLink can help reduce latency by avoiding the extra network hop through a [NAT gateway](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html) that's typically required for outbound internet connections from private subnets. While this latency difference is often minimal (typically single-digit milliseconds), it may be noticeable if you're migrating from a database that was previously hosted directly within your VPC AWS PrivateLink provides these security and compliance benefits by ensuring your database traffic never leaves the AWS backbone network. > [!NOTE] > > Normal PlanetScale Postgres connectivity (as described in our [standard connection documentation](/docs/postgres/connecting)) uses secure TLS encryption over the public internet and is appropriate for most use cases. AWS PrivateLink is primarily beneficial for compliance and enhanced security requirements. ### PrivateLink pricing PlanetScale charges a flat rate of **\$0.01/GB** for all network traffic (both egress and ingress) over private connections. This replaces the standard [egress pricing](/docs/postgres/pricing#network-data-transfer) that applies to public connections. Each branch has a default amount of included private network traffic before this rate takes effect. AWS also charges standard PrivateLink pricing for VPC endpoints, which includes: | Charge Type | Rate | Description | | - | - | - | | **VPC endpoint hourly charges** | \~\$0.01 per hour | Per VPC endpoint (varies by region) | | **Data processing charges** | \~\$0.01 per GB | Data processed through the VPC endpoint (varies by region) | For current pricing in your region, see the [AWS PrivateLink pricing page](https://aws.amazon.com/privatelink/pricing/). ## Prerequisites * A PlanetScale Postgres database in an AWS region * An AWS VPC in the same region where you want to establish the private connection * Appropriate AWS IAM permissions to create VPC endpoints (see [AWS VPC endpoint permissions documentation](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-iam.html)) * Appropriate AWS IAM permissions to create and modify Security Groups (see [AWS IAM permissions for security groups documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-policies-for-amazon-ec2.html#iam-policies-security-groups)) ## Establishing a VPC endpoint 1. **Retrieve the Private Service Name**: 1. From the PlanetScale organization dashboard, select the desired database 2. Navigate to **Settings** from the menu on the left 3. Select **Roles** 4. Click on a role with permissions to the relevant `Branch` 5. Copy the `Private Host` and `Private Service Name` from the role details ![Private connection strings](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/aws-private-host-names.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=1b5f557ed732b7f5929e949b76848114) Save these two attributes for your records and the rest of the configuration. > [!NOTE] > > Both the `Private Host` and `Private Service Name` values are the same for all roles for a given PlanetScale database `Branch`. Once enabled, any role can use the PrivateLink endpoint. You do not need to configure this per PlanetScale `Role`. 1. **Create a Security Group for the Endpoint**: You will need an AWS Security Group configured to allow inbound traffic for the required ports. You can configure access using either the security group ID of your application hosts, your VPC's CIDR configuration, or specific subnet CIDR configurations. Ensure your [security groups](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-security-groups.html) allow: * **Inbound PostgreSQL (port 5432)**: For direct database connections * **Inbound PgBouncer (port 6432)**: For pooled connections via PgBouncer An example using the AWS CLI: ```bash theme={null} # Create the security group (capture its ID) SG_ID=$(aws ec2 create-security-group \ --group-name PScalePrivateLinkEndpointSG \ --description "Security group for PlanetScale PrivateLink endpoint" \ --vpc-id \ --query 'GroupId' --output text) # Option A (preferred): allow only from a client SG (replace sg-CLIENT) aws ec2 authorize-security-group-ingress \ --group-id "$SG_ID" \ --ip-permissions '[ {"IpProtocol":"tcp","FromPort":5432,"ToPort":5432,"UserIdGroupPairs":[{"GroupId":"sg-CLIENT"}]}, {"IpProtocol":"tcp","FromPort":6432,"ToPort":6432,"UserIdGroupPairs":[{"GroupId":"sg-CLIENT"}]} ]' # Option B: allow from entire VPC CIDR (replace with your actual CIDR) #aws ec2 authorize-security-group-ingress \ #--group-id "$SG_ID" \ #--ip-permissions '[ # {"IpProtocol":"tcp","FromPort":5432,"ToPort":5432,"IpRanges":[{"CidrIp":"10.0.0.0/16"}]}, # {"IpProtocol":"tcp","FromPort":6432,"ToPort":6432,"IpRanges":[{"CidrIp":"10.0.0.0/16"}]} #]' ``` Replace `` with your actual VPC ID. You can find your VPC ID and its CIDR block using: ```bash theme={null} aws ec2 describe-vpcs --query 'Vpcs[*].[VpcId,CidrBlock]' --output table ``` 2. **Navigate to VPC Endpoints**: In your AWS Console: 1. Confirm you are in the proper `` from the dropdown on the top right 2. In the search field at the top left enter "Endpoints". 3. Click the link listed as a **VPC Feature**. ![Endpoint search](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/endpoint-search.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=056b9b38fd36f778e9551a17c89cb37b) 3. **Create a new endpoint**: Click "**Create Endpoint**". ![Create a new endpoint](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/create-new-endpoint.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=7a530c11ec6ab94a467203f45e0369ea) 4. **Select endpoint type**: Choose "Endpoint services that use NLBs and GWLBs". ![Menu to select endpoint type](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/type-of-endpoint.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=3d06cd7cc0a99c988fbe6c6a7f698e9a) 5. **Enter service name**: Enter in the "Service name" text box the `Private Service Name` retrieved from the PlanetScale dashboard. Click "**Verify service**" to confirm the service exists. ![Endpoint service name and verification](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/verified-endpoint.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=e689939f8891b0ebb649774c200c4910) 6. **Configure VPCs**: Choose the VPC that should have access to the PlanetScale service endpoint. 7. **Enable DNS names**: Click the "Additional settings" dropdown arrow to reveal DNS configuration options, and select the "**Enable DNS name**" checkbox. 8. **Configure Subnets**: Choose the subnets that should have endpoint interfaces for the PlanetScale service endpoint. It is recommended that you select at least 2. You should select subnets that your application servers have access to. ![Subnets](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/network-subnets-config.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=7bc08b7d82f5f6b3697de4bad729d932) 9. **Configure security groups**: Choose the appropriate security group to control which resources can send traffic to the PlanetScale service endpoint. Use the one created earlier if you created one for this purpose. ![Security Groups](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/security-groups.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=d12a4e463598ec26ae735a0c8b03800a) 10. **Create the endpoint**: Click "**Create endpoint**" and wait for the VPC endpoint status to show "Available" (this may take several minutes). ![Available Endpoint](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/available-endpoint.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=e7f0a0f31f851f90f8b245460b1f9e16) ## Verifying your VPC endpoint connectivity 1. **Confirm endpoint status**: In the AWS Console, verify that your endpoint's status shows "Available". 2. **Test DNS resolution**: From an EC2 instance in your configured VPC, run a DNS lookup to confirm resolution to your VPC's IP range. Use the `Private Host` you recorded earlier from the PlanetScale dashboard: ```bash theme={null} dig +short .private-pg.psdb.cloud 10.0.2.120 10.0.1.118 ``` 3. **Test your new connection**: Once you have confirmed DNS resolution, test the private endpoint: ```bash theme={null} psql 'host=.private-pg.psdb.cloud port=5432 user=postgres.XYZ234 password=pscale_pw_REDACTED dbname=postgres sslnegotiation=direct sslmode=verify-full sslrootcert=system' ``` ## Update your connection strings Once your VPC endpoint is established and verified, you're ready to update your application's connection strings to use the private endpoint address instead of the standard public endpoint. ## Security group considerations Ensure your [security groups](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-security-groups.html) allow: * **Outbound PostgreSQL (port 5432)**: For direct database connections * **Outbound PgBouncer (port 6432)**: For pooled connections via PgBouncer * **Inbound - any application-specific ports**: Based on your connection requirements For more details about connection types and when to use each port, see our [connection documentation](/docs/postgres/connecting) and [PgBouncer guide](/docs/postgres/connecting/pgbouncer). ## Network ACL considerations VPC [Network ACLs (NACLs)](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html) operate at the subnet level and provide an additional layer of security beyond security groups. For AWS PrivateLink connections to PlanetScale, ensure your NACLs allow: * **Outbound PostgreSQL (ports 5432, 6432)**: For database connections * **Ephemeral ports (1024-65535)**: For return traffic from AWS PrivateLink endpoints Most default NACL configurations allow all outbound traffic and are compatible with PrivateLink. If using custom restrictive NACLs, add explicit allow rules for the above ports. ## Restricting access to PrivateLink only By default, setting up AWS PrivateLink does not block public internet access to your database. If you want to ensure your database only accepts connections through PrivateLink, you can use [IP restrictions](/docs/postgres/connecting/ip-restrictions) to block public internet traffic. When connections come through PrivateLink, PlanetScale sees the source IP as coming from our internal network infrastructure (private IP ranges), not from your VPC's CIDR range. To restrict access to PrivateLink-only connections, create an IP restriction rule allowing the RFC1918 private ranges: ``` 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 ``` This configuration blocks all public internet connections while allowing traffic through your PrivateLink endpoint. For more details on configuring IP restrictions, see our [IP restrictions documentation](/docs/postgres/connecting/ip-restrictions). ## Troubleshooting If you're experiencing connectivity issues: 1. **Verify endpoint status**: Ensure your VPC endpoint shows "Available" status 2. **Check security groups**: Confirm your security groups allow the required ports 3. **Check NACLs**: Confirm that your VPC's NACLs are configured to allow the correct network traffic 4. **Test DNS resolution**: Verify DNS is resolving to private IP addresses in your VPC CIDR range 5. **Use AWS Reachability Analyzer**: The [Reachability Analyzer](https://docs.aws.amazon.com/vpc/latest/reachability/what-is-reachability-analyzer.html) allows you to inspect the path between two resources (such as a client and your PlanetScale Postgres endpoint) and provides guidance on why connectivity might be failing 6. **Contact support**: If issues persist, contact PlanetScale support with your endpoint configuration details ## Next steps * [Learn about PostgreSQL roles and permissions](/docs/postgres/connecting/roles) * [Configure connection pooling with PgBouncer](/docs/postgres/connecting/pgbouncer) * [Monitor your connections and performance](/docs/postgres/monitoring/query-insights) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect privately with GCP Private Service Connect Source: https://planetscale.com/docs/postgres/connecting/private-connections/gcp-private-service-connect When you use GCP Private Service Connect, your network traffic between your VPC and PlanetScale stays within the Google Cloud network, without traversing the public internet. [GCP Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect) is a highly available, scalable technology that enables you to privately connect your VPC to supported GCP services, endpoint services, and partner services. ### When to use GCP Private Service Connect By default, PlanetScale Postgres databases use secure connections over the public internet with industry-standard TLS encryption. This approach is secure and meets the needs of most customers. However, you may want to consider GCP Private Service Connect if: * **Compliance requirements**: Your organization has regulatory or compliance mandates that require database connections to avoid the public internet entirely * **Enhanced security posture**: You want an additional layer of network isolation for sensitive data workloads * **Network architecture**: Your existing GCP infrastructure is designed around private connectivity patterns * **Reduced network latency**: GCP Private Service Connect can help reduce latency by keeping traffic within Google's network backbone GCP Private Service Connect provides these security and compliance benefits by ensuring your database traffic never leaves the Google Cloud network. > [!NOTE] > > Normal PlanetScale Postgres connectivity (as described in our [standard connection documentation](/docs/postgres/connecting)) uses secure TLS encryption over the public internet and is appropriate for most use cases. GCP Private Service Connect is primarily beneficial for compliance and enhanced security requirements. ### Private Service Connect pricing PlanetScale charges a flat rate of **\$0.01/GB** for all network traffic (both egress and ingress) over private connections. This replaces the standard [egress pricing](/docs/postgres/pricing#network-data-transfer) that applies to public connections. Each branch has a default amount of included private network traffic before this rate takes effect. Google Cloud also charges standard Private Service Connect pricing for endpoints, which includes: * **Private Service Connect endpoint charges**: Based on your endpoint configuration and usage * **Network egress charges**: Standard GCP egress pricing may apply for data transfer For current GCP pricing in your region, see the [GCP Private Service Connect pricing page](https://cloud.google.com/vpc/pricing#private-service-connect). ## Prerequisites * A PlanetScale Postgres database in a GCP region * A GCP VPC in the same region where you want to establish the private connection * Appropriate GCP IAM permissions to create Private Service Connect endpoints * **Required APIs enabled**: Cloud DNS API and Service Directory API must be enabled in your project for automatic DNS zone creation (see [GCP documentation](https://cloud.google.com/vpc/docs/dns-vpc-hosted-services#auto-dns-consumer)) ## Establishing a Private Service Connect endpoint 1. **Retrieve the Private Service Name**: 1. From the PlanetScale organization dashboard, select the desired database 2. Navigate to **Settings** from the menu on the left 3. Select **Roles** 4. Click on a role with permissions to the relevant `Branch` 5. Copy the `Private Host` and `Private Service Name` from the role details ![Private connection strings](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/psc-private-host-names.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=039af70e430f615759d661889f2975c3) Save these two attributes for your records and the rest of the configuration. > [!NOTE] > > Both the `Private Host` and `Private Service Name` values are the same for all roles for a given PlanetScale database `Branch`. Once enabled, any role can use the Private Service Connect endpoint. You do not need to configure this per PlanetScale `Role`. 2. **Enable required APIs**: Ensure Cloud DNS and Service Directory APIs are enabled for automatic DNS zone creation: Via GCP Console: 1. Confirm you are in the proper `` from the project selector 2. From the top search bar, search for `CloudDNS` and select it from the results 3. Click to enable the API 4. From the top search bar, search for `Service Directory` and select it from the results 5. Click to enable the API 3. **Navigate to Private Service Connect**: In the GCP Console: 1. Confirm you are in the proper `` from the project selector 2. From the top search bar, search for `Private Service Connect` and select it from the results ![Private Service Connect console](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/psc-search.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=1512642665fe31e4be2881493cfe2f01) 4. **Create a new endpoint**: Click "**+ Connect endpoint**". 5. **Configure endpoint details**: * **Target**: Select "Published Service" * **Target Service**: Enter the `Private Service Name` recorded from the PlanetScale Dashboard * **Endpoint name**: Choose a descriptive name (e.g., "planetscale-main" uses the `Branch` name in it. This name will be part of the connection host string you use going forward) * **Network**: Select your VPC network * **Subnet**: Choose a subnet that your application servers can access * **Create an IP Address**: Reserve a static IP address for the endpoint * **Enable Global Access**: Recommended - allows applications in other regions to reach the endpoint * **Create a namespace**: Recommended - Set a namespace in `Service Directory` to enable creation of an entry for this endpoint ![Endpoint configuration details](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/psc-endpoint-configuration.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=9550639d2cf4a5f50845ae8c17846d62) 6. **Create the endpoint**: Click "**Add Endpoint**" and wait for the endpoint status to show "Accepted" (this may take several minutes). ## Verifying your Private Service Connect endpoint connectivity 1. **Confirm endpoint status**: In the GCP Console, verify that your endpoint's status shows "Accepted". ![Active Endpoint](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/psc-active-endpoint.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=56ab9c43aae6344f38c047687ff401a8) 2. **Test DNS resolution**: If Cloud DNS is enabled, GCP automatically creates a private Cloud DNS zone for your endpoint. The DNS zone will match the `Private Host` (recorded earlier from the PlanetScale Dashboard). You can verify by navigating to the `Cloud DNS` page from the left Nav: ![Cloud DNS resource](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/connecting/private-connections/psc-clouddns-record.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=121bfa531bf8fa5e8006365685bbb318) To address the actual host endpoint you will use, you need to combine the `Endpoint name` you defined with the DNS zone name here (same as the `Private Host` recorded earlier). From this example: * `Endpoint name` = planetscale-main * `Private Host` = gcp-us-central1-1.private-pg.psdb.cloud Which results in: ``` planetscale-main.gcp-us-central1-1.private-pg.psdb.cloud ``` To confirm DNS is working properly in your VPC, run a DNS lookup from a VM instance in your configured VPC: ```bash theme={null} dig +short planetscale-main.gcp-us-central1-1.private-pg.psdb.cloud 10.128.0.17 ``` 3. **Test your PostgreSQL connection**: Once you have confirmed DNS resolution, test the private endpoint: ```bash theme={null} psql 'host=planetscale-main.gcp-us-central1-1.private-pg.psdb.cloud port=5432 user=postgres.XYZ234 password=pscale_pw_REDACTED dbname=postgres sslmode=require' ``` ## Update your connection strings Once your Private Service Connect endpoint is established and verified, update your application's connection strings to use the private endpoint address. Note that you need both the `Endpoint name` you configured and the `Private Host` from the PlanetScale dashboard to form the full hostname for your application to use. * **Original**: `gcp-us-central1-1.pg.psdb.cloud` * **Private**: `planetscale-main.gcp-us-central1-1.private-pg.psdb.cloud` Replace the hostname in your connection strings while keeping all other parameters (user, password, database name, etc.) the same. ## VPC considerations Your VPC configuration should allow: * **Private Google Access**: Enable this if your compute instances don't have external IP addresses * **Subnet routing**: Ensure proper routing between your application subnets and the PSC endpoint subnet * **Network tags**: Use network tags to organize and control access to your PSC endpoint ## Restricting access to Private Service Connect only By default, setting up GCP Private Service Connect does not block public internet access to your database. If you want to ensure your database only accepts connections through Private Service Connect, you can use [IP restrictions](/docs/postgres/connecting/ip-restrictions) to block public internet traffic. When connections come through Private Service Connect, PlanetScale sees the source IP as coming from our internal network infrastructure (private IP ranges), not from your VPC's CIDR range. To restrict access to Private Service Connect-only connections, create an IP restriction rule allowing the RFC1918 private ranges: ``` 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 ``` This configuration blocks all public internet connections while allowing traffic through your Private Service Connect endpoint. For more details on configuring IP restrictions, see our [IP restrictions documentation](/docs/postgres/connecting/ip-restrictions). ## Troubleshooting If you're experiencing connectivity issues: 1. **Verify endpoint status**: Ensure your Private Service Connect endpoint shows "Accepted" status 2. **Test DNS resolution**: Verify DNS is resolving to the private IP address in your VPC 3. **Check VPC routing**: Ensure there are no route conflicts or missing routes 4. **Verify network connectivity**: Use tools like `telnet` or `nc` to test port connectivity 5. **Contact support**: If issues persist, contact PlanetScale support with your endpoint configuration details ## Next steps * [Learn about PostgreSQL roles and permissions](/docs/postgres/connecting/roles) * [Configure connection pooling with PgBouncer](/docs/postgres/connecting/pgbouncer) * [Monitor your connections and performance](/docs/postgres/monitoring/query-insights) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connections quickstart Source: https://planetscale.com/docs/postgres/connecting/quickstart PlanetScale Postgres works with any standard PostgreSQL driver or library that supports SSL connections. We recommend reading the [Connections overview documentation](/docs/postgres/connecting) before going through this quickstart. It contains important information such as when and how to use PgBouncer over direct connections. > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). ## Creating a role After creating a Postgres database, you will need to create a new role to connect. **Step 1** From your PlanetScale organization, select the desired database **Step 2** From the database dashboard, click the "**Connect**" button in the top-right corner **Step 3** Select the desired database branch from the "**Branch**" dropdown **Step 4** On this page, you can create either a default role (with near-superuser privileges) or a user-defined role. The default role has the most privileged access and should be reserved for administrative tasks. Do not connect to your database from application servers using the default role. A user-defined role lets you pick from the standard Postgres permission set to grant only what is needed, which should be used in most cases. **Step 5** If the database has [**pg\_strict**](/docs/postgres/extensions/pg_strict) enabled, the Connect modal also surfaces pg\_strict options for user-defined roles. pg\_strict blocks `UPDATE` and `DELETE` statements without `WHERE` clauses, preventing accidental mass mutations in production. **Step 6** Click "**Create role**" **Step 7** This generates the unique username and password pair for your user that you can use to access the designated branch of your database. > [!TIP] > > Make sure you copy the credentials, as we will not display them once you leave the page. If you need to access your default credentials after leaving the page, you have to reset the default password. You'll be provided with the following: ```bash theme={null} DATABASE_HOST=**********-.horizon.psdb.cloud DATABASE_NAME= DATABASE_USERNAME=postgres. DATABASE_PASSWORD=pscale_pw_************************** ``` Here is an example of the connection string that connects directly (without [PgBouncer](/docs/postgres/connecting/pgbouncer)) you can use to connect to the Postgres CLI: ```bash theme={null} psql 'host=xxxxxxxxxx-useast1-1.horizon.psdb.cloud \ port=5432 \ user=postgres.xxxxxxxxxx \ password=pscale_pw_xxxxxxxxxxxxxxxxxx \ dbname=my_database \ sslnegotiation=direct \ sslmode=verify-full \ sslrootcert=system' ``` ### Connection parameters | Parameter | Description | | :- | :- | | **host** | Your database hostname in the format `{id+region}.horizon.psdb.cloud` | | **port** | Connection port (`5432` for direct, `6432` for PgBouncer) | | **user** | Your role username in the format `{role}.{branch_id}` | | **password** | Your role password (begins with `pscale_pw_`) | | **dbname** | Your database name | | **sslmode** | Set to `verify-full` for secure connections (required) | | **sslrootcert** | Set to `system`. See the [Secure connections documentation](/docs/postgres/connecting/quickstart#secure-connections) if that produces an error. | | **sslnegotiation** | Set to `direct` for improved performance (optional) | ## Secure connections All PlanetScale Postgres connections require SSL/TLS encryption. The following parameters are used to enforce this: | Parameter | Required | Description | | :- | :- | :- | | `sslmode=verify-full` | Required | Verifies both encryption and server identity | | `sslrootcert=system` | Required | Uses [system certificate](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-SSLROOTCERT) store by default. If this method doesn't work, you can enter the path to your root certificate instead. The exact [locations of root certificates](/docs/postgres/connecting) differ by SSL implementation and platform. | | `sslnegotiation=direct` | Optional | Enables direct SSL negotiation for better performance. | ## Additional roles You should not connect to your database from application servers with the default role, as you will need to take some downtime if you ever have to rotate your password. Instead, you can [create additional roles](/docs/postgres/connecting/roles) with fine-grained permission settings for this purpose. Check out our [roles](/docs/postgres/connecting/roles) documentation for more information. ## Resetting the default role credentials If you need to reset the password for the default `postgres` role: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Roles** page in **Settings** **Step 3** Find the role labeled **default** for the branch in question **Step 4** Open the three dot menu for the role and select **Reset credentials** **Step 5** Update the credentials any place that they are in use > [!WARNING] > > Resetting the default password will disconnect any existing connections using the previous credentials. ## Default `postgres` role The default `postgres` role is similar to the Postgres `superuser`, but with fewer permissions. It is defined by the following statement: ```sql theme={null} CREATE ROLE $POSTGRES_USERNAME NOSUPERUSER CREATEDB CREATEROLE INHERIT LOGIN REPLICATION BYPASSRLS PASSWORD '$PASSWORD'; ``` It also inherits the following permissions: ```sql theme={null} GRANT pg_read_all_data, pg_write_all_data, pg_read_all_settings, pg_read_all_stats, pg_stat_scan_tables, pg_monitor, pg_signal_backend, pg_checkpoint, pg_maintain, pg_use_reserved_connections, pg_create_subscription TO $ALMOST_SUPERUSER_ROLENAME WITH ADMIN OPTION; ``` You should not connect to your database from your applications using the default role, as you will need to take some downtime if you ever have to rotate your password. Instead, you should [create additional roles](./roles) as needed. ## Connection types: Direct vs PgBouncer PlanetScale offers two connection methods for PostgreSQL databases: direct (port `5432`) and via PgBouncer (port `6432`). > [!NOTE] > > **PlanetScale's use of PgBouncer (port `6432`) does not support replica routing.** All connections through PgBouncer are automatically routed to the primary database, regardless of the username specification. Use direct connections (port `5432`) for replica access. ### Direct connection (Port 5432) Using port 5432 bypasses PgBouncer and connects directly to Postgres. This is the recommended way to connect for any operations that require long-running queries or persistent connections. PgBouncer operates in transaction pooling mode, where connections are returned to the pool after each transaction completes. This means that session-level features and long-running operations are interrupted between transactions. Direct connections are recommended for: * Schema changes and DDL * OLAP, analytics, reporting, or batch processing * Session-specific features: Custom session variables, temporary tables * ETL processes and data streaming * Long-running transactions or queries that span multiple transactions Additionally, if you're connecting to a replica, you must connect directly. PgBouncer is not supported for replica connections. ### PgBouncer connection (Port 6432) [PgBouncer](/docs/postgres/connecting/pgbouncer) enables high availability for a Postgres database by efficiently pooling connections and buffering queries during failovers. PgBouncer is generally recommended for OLTP workloads. For example, we'd recommend routing your application connections through PgBouncer. You can connect through PgBouncer by updating your connection string to use port `6432`. PgBouncer connections operate in transaction mode, which means each connection is only held for the duration of a single transaction. This provides excellent performance for OLTP workloads but limits certain PostgreSQL features that require persistent connections. For use cases that require long-running operations, we recommend a direct connection on port `5432`. ## Routing to replicas PlanetScale Postgres supports routing connections to replicas for improved read performance and load distribution. To connect to a replica, append `|replica` to your credential username. For example: ```bash theme={null} # Connect to replica user=postgres.xxxxxxxxx|replica # where postgres.xxxxxxxxx is your username ``` You can append `|replica` to any role you create on your Postgres database. > [!NOTE] > > **PlanetScale's use of PgBouncer (port `6432`) does not support replica routing.** All connections through PgBouncer are automatically routed to the primary database, regardless of the username specification. Use direct connections (port `5432`) for replica access. ## Authentication PlanetScale Postgres uses SCRAM-SHA-256 authentication, which provides enhanced security over traditional password authentication methods. ### Username format All usernames follow the format `{role}.{branch_id}`: | Component | Description | Examples | | :- | :- | :- | | **role** | The role name | `postgres`, `app_user` | | **branch\_id** | The unique identifier for your database branch | `cnlmx96ec5kw` | | **Full username** | Complete format: `{role}.{branch_id}` | `postgres.cnlmx96ec5kw`, `app_user.cnlmx96ec5kw` | The branch ID in the username tells PlanetScale's routing layer (Exosphere) which specific database branch to connect to. ### Password format All PlanetScale Postgres passwords begin with `pscale_pw_` followed by a unique string: ```bash theme={null} pscale_pw_XXXXXXXXXXXXXXXXXXXX ``` ## Strong security model PlanetScale roles are created for use with a single database branch. This strong security model allows you to generate roles that are tied to a branch, and cannot access data/schema from another branch. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Managing Roles for your Postgres database Source: https://planetscale.com/docs/postgres/connecting/roles When you connect to your database, you have the option to connect with the default role or a user-generated role. This document covers the differences between each option and how to create new roles. You should not connect to the database from your application servers using the default role. If you ever need to rotate your default role credentials and you use the default role to connect to your application, you will have to take some downtime while rotating the credentials. Instead, we recommend creating [user-defined roles](#user-defined-roles) for this purpose. We'll first cover the default role below, and then explain how to generate user-defined roles. ## Default role The default `postgres` role is similar to the Postgres `superuser`, but with fewer permissions. It is defined by the following statement: ```sql theme={null} CREATE ROLE $POSTGRES_USERNAME NOSUPERUSER CREATEDB CREATEROLE INHERIT LOGIN REPLICATION BYPASSRLS PASSWORD '$PASSWORD'; ``` It also inherits the following permissions: ```sql theme={null} GRANT pg_read_all_data, pg_write_all_data, pg_read_all_settings, pg_read_all_stats, pg_stat_scan_tables, pg_monitor, pg_signal_backend, pg_checkpoint, pg_maintain, pg_use_reserved_connections, pg_create_subscription TO pscale_superuser WITH ADMIN OPTION; ``` ## User-defined roles When creating custom roles for your application, you can select from a variety of permissions to grant specific capabilities. User-defined roles allow you to implement the principle of least privilege by granting only the permissions necessary for each use case. For examples of common user-defined roles, see [User-defined role examples and use cases](#user-defined-role-examples-and-use-cases). ### Available permissions Below is a list of available permissions you can set on user-defined roles. **Data access permissions** * **pg\_read\_all\_data** — Read data from all tables, views, and sequences. This permission allows `SELECT` queries across all database objects. * **pg\_write\_all\_data** — Write data to all tables, views, and sequences. This permission allows `INSERT`, `UPDATE`, and `DELETE` operations. Note that write operations typically require `pg_read_all_data` as well to read the data being modified. > [!NOTE] > > `TRUNCATE` is a separate table privilege. Grant it explicitly on the tables that need it, or run `TRUNCATE` as a table owner or a role that inherits `postgres`: > > ```sql theme={null} > GRANT TRUNCATE ON TABLE public.my_table TO ""; > ``` **Configuration and monitoring permissions** * **pg\_read\_all\_settings** — Read all configuration variables. This allows viewing database configuration parameters. * **pg\_read\_all\_stats** — Read all `pg_stat_*` views. This provides access to database statistics and performance metrics. * **pg\_stat\_scan\_tables** — Execute monitoring functions that may take `ACCESS SHARE` locks on tables. This is useful for running database monitoring and analysis operations. * **pg\_monitor** — Read and execute monitoring views and functions. This is a convenience role that combines several monitoring-related permissions. **Administrative permissions** * **pg\_signal\_backend** — Signal another backend to cancel a query or terminate its session. This is useful for managing long-running queries and terminating problematic connections. * **pg\_checkpoint** — Execute the `CHECKPOINT` command. Checkpoints ensure that all data is written to disk and are important for database recovery. * **pg\_maintain** — Execute maintenance operations including `VACUUM`, `ANALYZE`, `CLUSTER`, `REFRESH MATERIALIZED VIEW`, `REINDEX`, and `LOCK TABLE`. These operations are essential for database performance and maintenance. * **pg\_use\_reserved\_connections** — Use connection slots reserved via `reserved_connections`. This allows connecting to the database even when all regular connection slots are in use. * **pg\_create\_subscription** — Allow users with `CREATE` permission on the database to issue `CREATE SUBSCRIPTION`. This is used for logical replication scenarios. **Superuser-equivalent permission** * **postgres** — The default near-superuser role with extensive permissions. This role can create, modify, and drop databases, users, roles, tables, schemas, and all other objects. Use this permission carefully, as it grants broad administrative capabilities. ## Creating new user-defined roles There are several ways to create a new role: * Using the "Connect" button in your dashboard * Using "Roles" section in your database settings * Using the [`CREATE ROLE`](https://www.postgresql.org/docs/current/sql-createrole.html) command as the default role (which has elevated privileges). * Using the Postgres [Roles API](/docs/api/reference/list_roles) * Using the PlanetScale CLI [pscale role commands](/docs/cli/role) ### Creating roles in the dashboard To create a new role in the dashboard, you can either click the "Connect" button on the database overview page, or navigate to "Settings" > "Roles" and click "New role". ![Configure the new role](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/roles/image3.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=1bcbf03cd4421f06490316447d97ed67) ### Creating roles via the CLI You can also manage roles directly from the command line using the PlanetScale `pscale role` CLI. This provides a convenient way to create, list, and manage roles as part of your development workflow or automation scripts. Make sure you have the [PlanetScale CLI](/docs/cli) installed #### Available commands **Create a new role:** ``` pscale role create [flags] ``` Example: ``` pscale role create my-database main api-user --inherited-roles pg_read_all_data --ttl 24h ``` **List all roles for a branch:** ``` pscale role list ``` **Get details for a specific role:** ``` pscale role get ``` **Delete a role:** ``` pscale role delete [--successor ] ``` **Update a role's name:** ``` pscale role update --name ``` **Renew a role's expiration:** ``` pscale role renew ``` **Reset the default postgres role credentials:** ``` pscale role reset-default ``` **Reset a role's password:** ``` pscale role reset --org ``` This command resets the password for a role after prompting for confirmation. It returns the role object with the new password. **Reassign objects owned by a role:** ``` pscale role reassign --org --successor ``` This command assigns all objects owned by the donor role to the recipient role. Roles created via the CLI will appear in your database settings and can be managed through the dashboard as well. ### Creating roles via `CREATE ROLE` Use this path when you need fine-grained permissions that the managed role builder doesn't expose (for example, table- or column-level `GRANT`s). For most use cases, prefer the dashboard, [`pscale role`](/docs/cli/role), or the [Roles API](/docs/api/reference/create_role), which produce managed roles that appear in the dashboard and handle credential and lifecycle management for you. When you create a role via the Postgres [`CREATE ROLE`](https://www.postgresql.org/docs/current/sql-createrole.html) command, these will not display on your database settings. It is up to you to manage these via the `psql` CLI. PlanetScale's routing layer uses the `user` to identify which database or branch we are sending queries to. For example, the user `matt.nk35mx55qq` routes to the PlanetScale database with branch id `nk35mx55qq`. When you create a new role, you do not need to specify the branch id on the user. You can simply set the user to `matt`. However, when you connect, you must append the branch id to the user so we know which branch to route to. **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Select the desired branch from the dropdown **Step 3** Click "**Connect**" **Step 4** Copy the branch id **Step 5** Append it to your user with `.branch_id` ## Roles on a restored branch Restoring a backup or point-in-time recovery copies user-defined roles onto the new branch. They keep ownership of their objects, and their passwords are reset. See [Roles on a restored branch](/docs/postgres/backups#roles-on-a-restored-branch). ## Viewing, deleting, and renaming roles On the roles page, you will see all roles created via the dashboard, API, and the `pscale role` CLI. However, we will not display roles created manually via `CREATE ROLE` commands. You can rename a role by clicking the "..." button for the role on the Roles page at "Settings" > "Roles". ### Deleting a role To delete a role, click the "..." for the role on the database Roles page at "Settings" > "Roles". ![Rename or delete a role](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/roles/image5.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=4732d6c610379b15c35aed87b8e8d974) Deleting a role requires an extra step if the role has created any objects like tables or schemas, if the role has been granted any additional permissions, or if the role has created any other roles. If you try to delete a role that is still referenced, you may see this error: `Role is still referenced and cannot be dropped.`. Such roles must designate a successor role, to which allowed objects are reassigned. Additional granted permissions are dropped as part of the transfer process. The usual successor role is `postgres`, which you can indicate in the "Delete role" modal. You can reassign owned objects when deleting a role directly in the dashboard. When you click "Delete role", check the "**Reassign owned objects**" box on the modal. ![Specify a successor for a role](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/roles/image6.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=76636a882fba3970dbea04d295ed0189) You can choose successors other than `postgres`, but only by using the API or the `pscale role` CLI. Deleting a role that owns objects, has additional permissions, or has created other roles will fail if no successor is specified. You can [delete a role using the `pscale role` CLI](/docs/cli/role#the-delete-sub-command) with: ```bash theme={null} pscale role delete --org main --successor postgres ``` ### Reassigning objects owned by a role If you need to transfer ownership of database objects from one role to another without deleting the role, you can reassign the objects from the dashboard or the CLI. **In the dashboard**, click the "..." button for the role on the Roles page at "Settings" > "Roles", then select "**Reassign objects**". This will transfer all objects owned by the role to the `postgres` role. Reassigning objects to a recipient other than `postgres` is only possible through the API and the CLI. **Using the CLI**, you can reassign objects with: ```bash theme={null} pscale role reassign --org --successor ``` This command will prompt for confirmation before transferring ownership of all objects from the donor role to the recipient role. ### Resetting role credentials If you need to reset a role's password, you can do so from the dashboard or the CLI. **In the dashboard**, click the "..." button for the role on the Roles page at "Settings" > "Roles", then select "**Reset credentials**". This will generate a new password for the role. **Using the CLI**, you can reset a role's password with: ```bash theme={null} pscale role reset --org ``` This command will prompt for confirmation before resetting the password and return the role object with the new password. ## User-defined role examples and use cases Understanding when to use user-defined roles versus the default role is essential for maintaining secure and maintainable database access patterns. This section provides practical examples of role configurations for common scenarios. ### When to use user-defined roles vs. the default role **Use user-defined roles when:** * **Connecting from application servers**: Application connections should never use the default role. This allows you to rotate the default role credentials without application downtime. * **Principle of least privilege**: Different parts of your application or different services may need different levels of access. Create specific roles for each use case. * **Managing team access**: Different team members may need different permissions (e.g., developers vs. data analysts vs. DBAs). * **Integrating third-party tools**: External tools and services should have their own roles with limited permissions appropriate to their function. **Use the default role when:** * **Performing administrative tasks**: Creating schemas, managing database structure, or performing major database migrations. * **Initial database setup**: Setting up the initial database structure and creating the first set of user-defined roles. ### Example role configurations The following are some example permission configurations that you may use for user-defined roles. Your use cases may vary, but these are generic examples. **Application read-write role** For a typical web application that needs to read and write data, you may consider these permissions: * `pg_read_all_data` * `pg_write_all_data` > [!WARNING] > > `pg_read_all_data` and `pg_write_all_data` grant **data** access only (`SELECT`, > `INSERT`, `UPDATE`, `DELETE`). They do **not** grant `TRUNCATE`, and they do > **not** grant the ability to run DDL such as `CREATE TABLE`, `CREATE INDEX`, or > `CREATE SCHEMA`. Creating an object in a schema additionally requires the > `CREATE` privilege **on that schema**. > > A role with only these two permissions that tries to create a table in the > `public` schema will fail with: > > ``` > ERROR: permission denied for schema public > ``` > > To let a role create objects in `public`, grant it explicitly (run this as the > default `postgres` role, which has the necessary privileges): > > ```sql theme={null} > GRANT CREATE ON SCHEMA public TO ""; > ``` > > Alternatively, run schema migrations and other DDL as the default `postgres` > role and reserve the least-privilege application role for reads and writes. > [!TIP] > > Enable [pg\_strict](/docs/postgres/extensions/pg_strict) for production roles to block UPDATE and DELETE statements without WHERE clauses. **Read-only analytics role** For analytics tools or reporting dashboards that only need to query data: * `pg_read_all_data` * `pg_read_all_settings` * `pg_read_all_stats` **Monitoring and observability role** For monitoring tools like Datadog, New Relic, or custom monitoring solutions: * `pg_monitor` * `pg_read_all_stats` * `pg_read_all_settings` * `pg_stat_scan_tables` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connection resilience guide Source: https://planetscale.com/docs/postgres/connection-resilience Configure PlanetScale Postgres for maximum availability in demanding OLTP workloads. ## Summary * Use dedicated PgBouncers (not local) for production OLTP * Connect via port 6432 with `username|bouncer-name` format * Do not hold any connection longer than 24 hours, regardless of target * Keep transactions under 3 seconds for clean failovers * Add retry logic for read queries to handle transient failures ## Overview Production teams running OLTP workloads need connection strategies that maximize availability during failovers and planned maintenance. This guide covers how to configure PlanetScale Postgres for resilience and what to expect during failovers. For background on PlanetScale Postgres cluster architecture, see [operations philosophy](/docs/postgres/operations-philosophy). ## Connection interruption Postgres design dictates connection interruptions for several reasons: * **Parameter changes** — Changing certain [parameters](/docs/postgres/cluster-configuration/parameters) requires connection interruption. * **Cluster resizing** — If you're upgrading which cluster size you're running on, or the amount of CPU/memory, connections will be interrupted as we move the cluster to new hardware. * **Enabling extensions** — The design of the extension ecosystem means most new extensions need connection interruption. * **PlanetScale routine maintenance** — Very occasionally, PlanetScale has to perform routine maintenance on your database (e.g. for operating system updates and other infrastructure changes). We notify your organization's admin contacts at least 24 hours before scheduled maintenance (typically weeks beforehand). * **PlanetScale emergency maintenance** — Critical security vulnerabilities may require immediate patching to protect your data. These patches are deployed as soon as possible and may not include advance notice. Many applications don't tolerate a Postgres connection being closed. Typically this results in an HTTP 500 error and an error in your exception tracker/logs. ## Connection lifetime Do not hold any connection open longer than 24 hours, regardless of the target. This applies to every connection method: direct connections to the primary or replicas, the local PgBouncer, and dedicated primary or replica PgBouncers. Connections held longer than 24 hours may be terminated unexpectedly. Configure your connection pool or client to close and reconnect at least once per day. Set a maximum connection lifetime of 24 hours or less (for example, `maxLifetime` or the equivalent setting in your driver or pool). ## Use Dedicated PgBouncer PlanetScale Postgres offers one connection path that is significantly more resilient in the face of maintenance and other failovers, which is a dedicated PgBouncer. We strongly recommend production customers use a dedicated PgBouncer. A dedicated PgBouncer allows us to turn all planned failovers into a brief period of elevated latency with no connection drops (using `PAUSE`). It's worth noting that PgBouncer has several important compatibility problems that can cause unexpected application issues. See [transaction pooling limitations](/docs/postgres/connecting/pgbouncer#limitations-of-transaction-pooling). See the [PgBouncer documentation](/docs/postgres/connecting/pgbouncer) for configuration options and the [connections overview](/docs/postgres/connecting) for a comparison of all connection methods. If you have replica traffic that is production critical, we also recommend using dedicated PgBouncers for your replicas. Without a dedicated PgBouncer, every connection interruption means every client connection gets terminated. Your application must detect the closed connection, reconnect, and retry the failed query. During a failover, this can cause a burst of errors and retries across your entire application fleet. If you cannot use a dedicated PgBouncer due to compatibility issues, you will see errors during failovers. We recommend tuning timeouts (see below), but also to expect brief error windows during these events. ### Dedicated bouncer maintenance Configuration changes to dedicated bouncers are very safe. When you modify a dedicated bouncer's settings, we create a new bouncer with the updated configuration and drain connections from the old one over 24 hours. New connections go to the new bouncer immediately; existing connections continue on the old bouncer until they disconnect naturally or the drain period ends. This means: * Bouncer configuration changes cause no connection drops for well-behaved clients * Clients that hold connections longer than 24 hours will eventually be disconnected when the drain period ends * Long-lived connections will continue using old settings until they reconnect See [Connection lifetime](#connection-lifetime). Recycle every connection at least once per day so clients disconnect during the drain rather than being terminated at the end of it. ## Failover behavior ### What happens during a failover PlanetScale failovers typically complete in seconds. See [operations philosophy](/docs/postgres/operations-philosophy#minimizing-the-impact-of-configuration-changes) for details on our failover approach. During a planned failover (e.g. maintenance, parameter changes, etc.), we can use `PAUSE` with a dedicated PgBouncer to replace connection interruptions with a latency increase on starting new transactions. ### How PAUSE keeps connections alive Dedicated PgBouncers use PgBouncer's `PAUSE` command during failovers to maintain client connections: 1. PAUSE stops assigning new transactions to backend Postgres connections 2. New queries queue in PgBouncer rather than erroring 3. Once the new primary/replica is ready, RESUME allows queued queries to execute Clients see a brief latency spike while queries are paused, but their connections stay alive. They don't need to reconnect or retry. ## Long-running queries and failovers ### Why long queries delay failovers Fast failovers depend on clean primary shutdowns. When you change a configuration or PlanetScale needs to failover, PlanetScale asks Postgres to shut down gracefully. If queries are still running, Postgres waits for them to complete. Long-running queries can extend unavailability from a few seconds to 30 seconds (at which point we hard cancel queries). ### Keeping transactions short For OLTP workloads, keep transactions as short as possible (at least under 3 seconds, but ideally a few milliseconds max per transaction for performance-critical workloads). Move network calls, external API requests, and heavy computation outside the transaction boundary. Do that work before `BEGIN` or after `COMMIT`. Use `statement_timeout` to bound how long individual queries can run. Route analytical queries and batch jobs to replicas where they won't block failovers on the primary. ## Monitoring query latency Use [Query Insights](/docs/postgres/monitoring/query-insights) to identify slow queries that could block failovers. ## Timeouts ### Postgres timeouts Use Postgres timeouts to prevent runaway queries from blocking failovers: * **`statement_timeout`** — Aborts any statement that takes longer than the specified duration. Set this to bound individual query execution time. * **`transaction_timeout`** — Aborts any transaction that exceeds the specified duration. This catches cases where an application opens a transaction but takes too long between statements (e.g. due to network calls or slow application logic within the transaction). * **`idle_in_transaction_session_timeout`** — Terminates any session that has been idle inside an open transaction for longer than the specified duration. This is an important safety net for preventing abandoned or forgotten transactions from holding locks and blocking other writes. If your application occasionally leaves transactions open due to unhandled exceptions or network issues, this timeout will clean them up automatically. Treat these as session-level controls. Set them at the role level with `ALTER ROLE ... SET`: ```sql theme={null} ALTER ROLE app_user SET statement_timeout = '2s'; ALTER ROLE app_user SET transaction_timeout = '3s'; ALTER ROLE app_user SET idle_in_transaction_session_timeout = '30s'; ``` ### Application timeouts Configure timeouts in your connection pool and database driver: * **Maximum connection lifetime** — How long a pooled connection may remain open before being closed and replaced. Set this to 24 hours or less so connections are recycled before they can be [terminated unexpectedly](#connection-lifetime). This applies to every target, including direct Postgres and all PgBouncer paths. * **Connection timeout** — How long to wait when establishing a new connection. Set this low enough to fail fast during outages rather than queueing requests indefinitely. * **Query timeout** — Application query timeout should be set slightly higher than your Postgres `statement_timeout` to allow Postgres to cancel the query cleanly. * **Pool checkout timeout** — How long to wait for a connection from the pool. During failovers, this determines how long requests queue before failing. Most connection pools also support health checks (e.g. `test on borrow`) to detect stale connections before using them. ## Retries Adding retry logic to your application improves availability by handling transient failures from network issues, brief unavailability windows, and connection drops. Read queries are safe to retry since they don't modify data. Write queries require more care—only retry if you can verify the original request didn't complete, or if the operation is idempotent. For read-heavy workloads, implementing retries in your database client or application layer can significantly reduce user-visible errors during failovers and intermittent network issues. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres database dashboard Source: https://planetscale.com/docs/postgres/dashboard When you navigate to a database in your PlanetScale organization, you'll see a comprehensive view of your cluster health, performance metrics, and management options. You can filter this view by [branch](/docs/postgres/branching) by selecting from the branch dropdown at the top. From the dashboard you can review: * Your cluster's topology diagram * Real-time performance metrics * Summary and statistics * Connection management * Branch-specific views and controls ## Cluster topology The cluster topology diagram provides a visual representation of your PostgreSQL database infrastructure, including: * **Primary node**: The main database instance that handles all write operations * **Replica nodes**: Read-only copies of your primary database for improved read performance and high availability Each node includes information about the region, instance type, real-time resource utilization (CPU and memory percentage), and cluster size. If you had additional replicas beyond the 2 default, you'll see them in this diagram. ## Database summary The database summary section on the right-hand side displays key statistics about your PostgreSQL environment, including: * **PostgreSQL version**: Shows the current PostgreSQL version (e.g., "17.4") * **Tables**: Total number of tables across all schemas * **Branches**: Count of database branches in your environment * **CPU utilization**: Percentage of CPU currently used * **Next backup**: Shows when the next scheduled backup will occur (e.g., "in 8 hours") * **Total storage**: Amount of storage currently used Production branches are clearly marked with visual indicators and badges to distinguish them from development branches. The summary also shows the current state and health of each branch, making it easy to assess your database environment at a glance. ## Performance metrics The performance metrics section includes a dropdown to select different metrics and a time-series graph showing data over the selected time period. Available metrics include: * Query latency (shown as p95, p99, p50, and p99.9 percentiles) * Queries per second * Rows read * Rows written * Query errors You can select each metric from the dropdown to update the graph. There's also a "View all query insights" link to access more detailed query performance data. ### Slowest queries The "Slowest queries during the last 24 hours" section at the bottom of the dashboard shows a detailed table with: * **Query**: The actual SQL query text * **Count**: Number of times the query was executed * **p50 latency (ms)**: The median query execution time in milliseconds This helps you identify performance bottlenecks and queries that may need optimization. ## Connecting to your database The "**Connect**" button allows you to generate or reset your default credentials for your Postgres database. For more information, see the [Connecting documentation](/docs/postgres/connecting). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Day one with PlanetScale Postgres Source: https://planetscale.com/docs/postgres/day-one-with-planetscale-postgres Top ten things you should be aware of to kill your p99, lower your egress, and get your application flying. Just created a new PlanetScale Postgres database? Don't miss the platform's best features to get started. ## Agent setup Run the [Agent setup prompt](/docs/agent-setup/prompt) to have your AI agent assess your database configuration, install the PlanetScale skills and suggest improvements. ## Top ten features ### 1. [Migrate](/docs/postgres/imports/postgres-imports) an existing database We have guides and tools available to help migrate from multiple sources. Start with our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. ### 2. Create [roles and connections](/docs/postgres/connecting/roles) Create user-defined roles for application connections to enable least-privilege permissions and credential rotation without downtime. Your cluster's primary database has a local [PgBouncer](/docs/postgres/connecting/pgbouncer), but dedicated PgBouncers for primaries and replicas are available as an optional add-on for improved connection resilience and scalability. ### 3. Browse [Query Insights](/docs/postgres/monitoring/query-insights) Built-in monitoring for past and current queries. Includes latency percentiles, QPS, rows read/written, and per-query pattern analytics. Use it to find bottlenecks before they become outages. As you begin receiving traffic, Insights will alert you or your agent to errors, anomalies, and provide schema recommendations to improve performance. > [!TIP] > > Watch "[Agents and Infrastructure](https://www.youtube.com/watch?v=zxvyO5vnknI)" with PlanetScale CEO Sam Lambert for a live demo of how infrastructure can be safely maintained and improved by a team of agents. Query Insights data is available in the dashboard, API, MCP, and CLI. [Fix Database Faults with PlanetScale Insights](https://www.youtube-nocookie.com/embed/OAPHvq51hWU?rel=0) ### 4. Install the [PlanetScale MCP server](/docs/mcp-server) Give your agent access to Query Insights' production data alongside your application code. With this combined context, it's the best way to do deep, automated analysis on database health and performance. [The PlanetScale MCP server: AI for your database](https://www.youtube-nocookie.com/embed/d3bc3PryOfY?rel=0) ### 5. Explore the [pscale CLI](/docs/postgres/postgres-cli) and [PlanetScale API](/docs/api/reference/getting-started-with-planetscale-api) Perform administrative actions against your databases from the command line, or the same workflows programmatically via the API. Create branches, manage roles and service tokens, and wire PlanetScale into CI/CD. The pscale CLI is the best way [automate processes with GitHub Actions](/docs/postgres/integrations/github-actions). ### 6. Configure [Database Traffic Control®](/docs/postgres/traffic-control) Define resource budgets to prioritize traffic slices by throttling others. Fine-grained limits on CPU, I/O, and concurrency per query pattern or tag. Exclusive to PlanetScale Postgres. [Introducing Database Traffic Control](https://www.youtube-nocookie.com/embed/YgrEGqABD8U?rel=0) ### 7. [Tag](/docs/postgres/monitoring/query-tags) your queries Embed SQLCommenter key-value tags in SQL comments to closely monitor related queries and define Traffic Control rules by application, route, or feature. ### 8. Create your first [branch](/docs/postgres/branching) Perform development and testing in isolated database environments per branch. Your new database started with a default branch called `main`. Development branches run on low-cost `PS-DEV` instances and can be created empty or from a backup. ### 9. Protect yourself with [pg\_strict](/docs/postgres/extensions/pg_strict) Per-role safety guardrails that block dangerous query shapes, like `UPDATE`/`DELETE` without a `WHERE` clause, before they run. [Protect your database. Use the pg\_strict Postgres extension.](https://www.youtube-nocookie.com/embed/noPnqjXsZxs?rel=0) ### 10. Enable and install [extensions](/docs/postgres/extensions) Install standard and community extensions from the dashboard or via `CREATE EXTENSION`. Some require a cluster restart and must be enabled in the dashboard. Vote for new extensions at [ps-extensions.io](https://ps-extensions.io). ### 11. Join the Discord community Bonus! Ask questions and share what you're building. [Join the PlanetScale Discord community](https://pscale.link/community) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Development Environments Source: https://planetscale.com/docs/postgres/development-environments When using development environments, it's important to have a way to work with your database and test schema changes in isolation from production. We have several recommendations for how to accomplish this when using PlanetScale Postgres. > [!NOTE] > > Schema and code changes must be deployed in the correct sequence. When adding a new column to a table along with a new application feature, you likely want to execute the schema change before deploying the corresponding code changes. In other cases, such as removing a feature and with it a table in your database, you would make your code changes first and then drop the table second. > > We recommend reviewing [this blog post](https://planetscale.com/blog/safely-making-database-schema-changes) for more information about deploying database schema changes alongside application changes. ## Use local Postgres Since PlanetScale uses standard PostgreSQL, one option is to use locally-running Postgres servers for all of your development work. In this case, you would create a new local database for each branch a developer is working on. After creating the database, run your database migration scripts to populate it with your project's schema. Optionally, you could have a script to populate your tables with dummy data to improve local testing. When you are ready to commit your code changes, execute the necessary schema changes on your production branch, being [careful about ordering](https://planetscale.com/blog/safely-making-database-schema-changes). While this is a convenient option in some cases, local databases can make it difficult for multiple developers to collaborate on shared branches, as schema changes must be manually synchronized across each developer's local environment. ## Use PlanetScale branches PlanetScale Postgres supports [branching](/docs/postgres/branching). This allows you to spin up lightweight Postgres databases separate from production that can be used for development and testing. Creating a new branch produces a completely empty Postgres database. No schema or data is copied to the branch, unless restoring from a backup (more on this later). Branches can be used similarly to local Postgres instances. The advantage is, since they are hosted by PlanetScale, they can be easily shared amongst development teams. If using branches for your development environments, you would [create new database branch](/docs/postgres/branching#create-a-branch) for each code branch a developer/team is working on. You would then run your schema migrations and any data loading scripts you need to initialize the branch. When you are ready to commit your code changes, execute the necessary schema changes on your production branch, and then [delete the branch](/docs/postgres/branching#delete-a-branch). ## Use PlanetScale branches + backups PlanetScale gives you the ability to create new branches that are [initialized from a recent backup](/docs/postgres/branching#from-a-backup). This can be used instead of empty branches to create development environments that are pre-populated with schema and data. There are some downsides to this approach, however. This approach copies production data to development branches, which may pose security risks if your database contains sensitive customer information. Another consideration is cost. Each branch is its own, isolated database and uses its own storage separate from production. You will be charged for the storage consumed by both your production branches and your development branches. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Extensions Source: https://planetscale.com/docs/postgres/extensions PostgreSQL extensions are add-on modules that extend the functionality of your database beyond the core features. They provide additional data types, functions, operators, indexes, and other capabilities without requiring modifications to the PostgreSQL source code. > [!NOTE] > > You can submit and vote for the extensions you want us to support next at [ps-extensions.io](https://ps-extensions.io/). Extensions work by packaging related database objects (functions, data types, operators, etc.) into a single installable unit. When you install an extension, PostgreSQL loads the necessary code and creates the required database objects in your database schema. This modular approach allows you to add only the functionality you need while keeping your database lightweight and maintainable. ## Types of Extensions PostgreSQL extensions generally fall into two main categories: ### Native Extensions These are extensions that ship with PostgreSQL itself and are maintained by the PostgreSQL core team. Examples include: * **Data type extensions**: `citext`, `hstore`, `uuid-ossp` for specialized data types * **Index extensions**: `btree_gin`, `btree_gist` for advanced indexing capabilities * **Text search extensions**: `unaccent`, `pg_trgm` for full-text search enhancements * **Utility extensions**: `pgcrypto` for cryptographic functions, `tablefunc` for table manipulation ### Community Extensions These are developed and maintained by the community or third-party organizations. Popular examples include: * **Vector search**: `pgvector` for AI/ML vector operations * **Query optimization**: `pg_hint_plan` for query plan hints * **Partitioning**: `pg_partman` for automated table partitioning > [!WARNING] > > Extensions that modify shared memory settings, background worker processes, or core database functionality typically require a restart. You can only enable these extensions through the [Dashboard](#configuring-extensions-in-the-dashboard). > > Most simple extensions like data types or functions can be installed without a restart. ## Managing Extensions > [!NOTE] > > Some extensions may require that you use a role with superuser privileges to enable them on your PlanetScale Postgres database. See [supported extensions](#supported-extensions) for more information. ### Installing an Extension To install an extension in your database, use the `CREATE EXTENSION` command: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS pgcrypto; ``` ### Removing an Extension To remove an extension from your database, use the `DROP EXTENSION` command: ```sql theme={null} DROP EXTENSION IF EXISTS pgcrypto; ``` ### Updating an Extension To update an extension to a newer version, use the `ALTER EXTENSION` command: ```sql theme={null} ALTER EXTENSION pgcrypto UPDATE; ``` Some extensions, including [TIN](/docs/postgres/search), also need a [cluster update](/docs/postgres/cluster-configuration/updates) so the new library is on the server before `ALTER EXTENSION ... UPDATE` can expose new options. ### Viewing Installed Extensions To see all extensions currently installed in your database, use this query: ```sql theme={null} SELECT name, default_version, installed_version, comment FROM pg_available_extensions WHERE installed_version IS NOT NULL ORDER BY name; ``` To view all available extensions (both installed and available for installation): ```sql theme={null} SELECT name, default_version, comment FROM pg_available_extensions ORDER BY name; ``` ### Installing Extensions in Specific Schemas By default, extensions are installed in the `public` schema. To install an extension in a specific schema: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS pgcrypto SCHEMA my_schema; ``` To move an existing extension to a different schema: ```sql theme={null} ALTER EXTENSION pgcrypto SET SCHEMA new_schema; ``` > [!NOTE] > > Not all extensions support being installed in non-public schemas. Most utility and data type extensions work fine, but some system-level extensions must remain in the public schema. ## Configuring Extensions in the Dashboard ![Configuring Extensions](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/extensions/configure-extensions.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=6f4a305debf0d9e406e3655873ec3c97) Some extensions require explicit configuration through the PlanetScale dashboard and may consume additional resources on your database instances. Configuring these extensions may require a database restart. The extensions that require a restart are marked with a ✅ in the "Restart required" column. For `Production` clusters, these reboots are applied in a rolling cadence through the cluster's instances. For detailed configuration instructions and parameter descriptions for each extension, please refer to their individual documentation pages via the 📝 links in the "Additional Notes" column. ## Troubleshooting ### Common Extension Issues #### Extension not found ```sql theme={null} ERROR: extension "extension_name" is not available ``` This means the extension is not installed on the server or not supported. Check the [supported extensions list](#supported-extensions) below. #### Permission denied ```sql theme={null} ERROR: permission denied to create extension "extension_name" ``` Extensions typically require database owner or superuser privileges. Contact support if you encounter permission issues. #### Extension already exists ```sql theme={null} ERROR: extension "extension_name" already exists ``` Use `CREATE EXTENSION IF NOT EXISTS extension_name;` to avoid this error, or check installed extensions first. #### Dependency conflicts ```sql theme={null} ERROR: required extension "dependency_name" is not installed ``` Some extensions depend on others. Install the required dependency first, then retry installing your desired extension. ## Supported Extensions ### Table Column Legend > [!NOTE] > > Newer Postgres versions ship with newer versions of extensions. When upgrading to a newer Postgres version, make sure to test your application for compatibility with those extension versions.\* * **Extension**: Extension name with link to official documentation * **Description**: Brief description of extension functionality * **Version**: Version number available in PlanetScale (or *TBD* if placeholder) * **Superuser required**: ⭐ = Extension requires superuser privileges to use its functionality * **Restart required**: ✅ = Must be enabled through PlanetScale dashboard first because the extension requires shared libraries and a restart (blank = install directly with `CREATE EXTENSION`) * **Additional Notes**: 📝 = Link to detailed configuration documentation ### Supported Native PostgreSQL Extensions #### Postgres 18.6 | Extension | Description | Version | Superuser required | Restart required | Additional Notes | | - | - | - | - | - | - | | [autoinc](https://www.postgresql.org/docs/current/contrib-spi.html#CONTRIB-SPI-AUTOINC) | Functions for autoincrementing fields | 1.0 | ⭐ | | | | [auto\_explain](https://www.postgresql.org/docs/current/auto-explain.html) | Logs execution plans for statements, including slow queries | 1.0 | | | [📝](/docs/postgres/extensions/auto_explain) | | [bloom](https://www.postgresql.org/docs/current/bloom.html) | Bloom filter index access method | 1.0 | ⭐ | | | | [btree\_gin](https://www.postgresql.org/docs/current/btree-gin.html) | Provides GIN operator classes that implement B-tree equivalent behavior for various data types | 1.3 | | | | | [btree\_gist](https://www.postgresql.org/docs/current/btree-gist.html) | Provides GiST index operator classes that implement B-tree equivalent behavior | 1.8 | | | | | [citext](https://www.postgresql.org/docs/current/citext.html) | Provides case-insensitive character string type | 1.8 | | | | | [cube](https://www.postgresql.org/docs/current/cube.html) | Data type for representing multidimensional cubes | 1.5 | | | | | [dict\_int](https://www.postgresql.org/docs/current/dict-int.html) | Text search dictionary template for integers | 1.0 | | | | | [dict\_xsyn](https://www.postgresql.org/docs/current/dict-xsyn.html) | Text search dictionary for extended synonym processing | 1.0 | ⭐ | | | | [earthdistance](https://www.postgresql.org/docs/current/earthdistance.html) | Calculate great circle distances on the surface of the Earth | 1.2 | ⭐ | | | | [fuzzystrmatch](https://www.postgresql.org/docs/current/fuzzystrmatch.html) | Functions for determining similarities and distance between strings | 1.2 | | | | | [hstore](https://www.postgresql.org/docs/current/hstore.html) | Data type for storing sets of key/value pairs within a single PostgreSQL value | 1.8 | | | | | [insert\_username](https://www.postgresql.org/docs/current/contrib-spi.html#CONTRIB-SPI-INSERT-USERNAME) | Functions for tracking who changed a table | 1.0 | ⭐ | | | | [intagg](https://www.postgresql.org/docs/current/intagg.html) | Integer aggregator and enumerator (deprecated) | 1.1 | ⭐ | | | | [intarray](https://www.postgresql.org/docs/current/intarray.html) | Functions and operators for manipulating null-free arrays of integers | 1.5 | | | | | [isn](https://www.postgresql.org/docs/current/isn.html) | Data types for international product numbering standards | 1.3 | | | | | [lo](https://www.postgresql.org/docs/current/lo.html) | Support for managing Large Objects (also called BLOBs) | 1.2 | | | | | [ltree](https://www.postgresql.org/docs/current/ltree.html) | Data type for representing labels of data stored in hierarchical tree-like structures | 1.3 | | | | | [moddatetime](https://www.postgresql.org/docs/current/contrib-spi.html#CONTRIB-SPI-MODDATETIME) | Functions for tracking last modification time | 1.0 | ⭐ | | | | [pg\_stat\_statements](https://www.postgresql.org/docs/current/pgstatstatements.html) | Provides a means for tracking planning and execution statistics of all SQL statements | 1.12 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_stat_statements) | | [pg\_trgm](https://www.postgresql.org/docs/current/pgtrgm.html) | Functions and operators for determining the similarity of alphanumeric text based on trigram matching | 1.6 | | | | | [pgcrypto](https://www.postgresql.org/docs/current/pgcrypto.html) | Cryptographic functions for PostgreSQL | 1.4 | | | | | [pg\_prewarm](https://www.postgresql.org/docs/current/pgprewarm.html) | Provides a means for preloading relation data into the operating system buffer cache | 1.2 | ⭐ | | | | [pgrowlocks](https://www.postgresql.org/docs/current/pgrowlocks.html) | Show row-level locking information | 1.2 | ⭐ | | | | [pgstattuple](https://www.postgresql.org/docs/current/pgstattuple.html) | Obtain tuple-level statistics | 1.5 | ⭐ | | | | [plpgsql](https://www.postgresql.org/docs/current/plpgsql.html) | Loadable procedural language for the PostgreSQL database system | 1.0 | | | | | [postgres\_fdw](https://www.postgresql.org/docs/current/postgres-fdw.html) | Foreign data wrapper for remote PostgreSQL servers | 1.2 | ⭐ | | | | [seg](https://www.postgresql.org/docs/current/seg.html) | Data type for representing line segments or floating-point intervals | 1.4 | | | | | [tablefunc](https://www.postgresql.org/docs/current/tablefunc.html) | Functions that return tables (multiple rows) | 1.0 | | | | | [tcn](https://www.postgresql.org/docs/current/tcn.html) | Provides a trigger function that notifies listeners of changes to any table | 1.0 | | | | | [tsm\_system\_rows](https://www.postgresql.org/docs/current/tsm-system-rows.html) | Table sampling method SYSTEM\_ROWS which can be used in TABLESAMPLE clause | 1.0 | | | | | [tsm\_system\_time](https://www.postgresql.org/docs/current/tsm-system-time.html) | Table sampling method SYSTEM\_TIME which can be used in TABLESAMPLE clause | 1.0 | | | | | [unaccent](https://www.postgresql.org/docs/current/unaccent.html) | Text search dictionary for removing accents (diacritic signs) from lexemes | 1.1 | | | | | [uuid-ossp](https://www.postgresql.org/docs/current/uuid-ossp.html) | Functions to generate universally unique identifiers (UUIDs) | 1.1 | | | | #### Postgres 17.11 | Extension | Description | Version | Superuser required | Restart required | Additional Notes | | - | - | - | - | - | - | | [autoinc](https://www.postgresql.org/docs/current/contrib-spi.html#CONTRIB-SPI-AUTOINC) | Functions for autoincrementing fields | 1.0 | ⭐ | | | | [auto\_explain](https://www.postgresql.org/docs/current/auto-explain.html) | Logs execution plans for statements, including slow queries | 1.0 | | | [📝](/docs/postgres/extensions/auto_explain) | | [bloom](https://www.postgresql.org/docs/current/bloom.html) | Bloom filter index access method | 1.0 | ⭐ | | | | [btree\_gin](https://www.postgresql.org/docs/current/btree-gin.html) | Provides GIN operator classes that implement B-tree equivalent behavior for various data types | 1.3 | | | | | [btree\_gist](https://www.postgresql.org/docs/current/btree-gist.html) | Provides GiST index operator classes that implement B-tree equivalent behavior | 1.7 | | | | | [citext](https://www.postgresql.org/docs/current/citext.html) | Provides case-insensitive character string type | 1.6 | | | | | [cube](https://www.postgresql.org/docs/current/cube.html) | Data type for representing multidimensional cubes | 1.5 | | | | | [dict\_int](https://www.postgresql.org/docs/current/dict-int.html) | Text search dictionary template for integers | 1.0 | | | | | [dict\_xsyn](https://www.postgresql.org/docs/current/dict-xsyn.html) | Text search dictionary for extended synonym processing | 1.0 | ⭐ | | | | [earthdistance](https://www.postgresql.org/docs/current/earthdistance.html) | Calculate great circle distances on the surface of the Earth | 1.2 | ⭐ | | | | [fuzzystrmatch](https://www.postgresql.org/docs/current/fuzzystrmatch.html) | Functions for determining similarities and distance between strings | 1.2 | | | | | [hstore](https://www.postgresql.org/docs/current/hstore.html) | Data type for storing sets of key/value pairs within a single PostgreSQL value | 1.8 | | | | | [insert\_username](https://www.postgresql.org/docs/current/contrib-spi.html#CONTRIB-SPI-INSERT-USERNAME) | Functions for tracking who changed a table | 1.0 | ⭐ | | | | [intagg](https://www.postgresql.org/docs/current/intagg.html) | Integer aggregator and enumerator (deprecated) | 1.1 | ⭐ | | | | [intarray](https://www.postgresql.org/docs/current/intarray.html) | Functions and operators for manipulating null-free arrays of integers | 1.5 | | | | | [isn](https://www.postgresql.org/docs/current/isn.html) | Data types for international product numbering standards | 1.2 | | | | | [lo](https://www.postgresql.org/docs/current/lo.html) | Support for managing Large Objects (also called BLOBs) | 1.1 | | | | | [ltree](https://www.postgresql.org/docs/current/ltree.html) | Data type for representing labels of data stored in hierarchical tree-like structures | 1.3 | | | | | [moddatetime](https://www.postgresql.org/docs/current/contrib-spi.html#CONTRIB-SPI-MODDATETIME) | Functions for tracking last modification time | 1.0 | ⭐ | | | | [pg\_stat\_statements](https://www.postgresql.org/docs/current/pgstatstatements.html) | Provides a means for tracking planning and execution statistics of all SQL statements | 1.11 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_stat_statements) | | [pg\_trgm](https://www.postgresql.org/docs/current/pgtrgm.html) | Functions and operators for determining the similarity of alphanumeric text based on trigram matching | 1.6 | | | | | [pgcrypto](https://www.postgresql.org/docs/current/pgcrypto.html) | Cryptographic functions for PostgreSQL | 1.3 | | | | | [pg\_prewarm](https://www.postgresql.org/docs/current/pgprewarm.html) | Provides a means for preloading relation data into the operating system buffer cache | 1.2 | ⭐ | | | | [pgrowlocks](https://www.postgresql.org/docs/current/pgrowlocks.html) | Show row-level locking information | 1.2 | ⭐ | | | | [pgstattuple](https://www.postgresql.org/docs/current/pgstattuple.html) | Obtain tuple-level statistics | 1.5 | ⭐ | | | | [plpgsql](https://www.postgresql.org/docs/current/plpgsql.html) | Loadable procedural language for the PostgreSQL database system | 1.0 | | | | | [postgres\_fdw](https://www.postgresql.org/docs/current/postgres-fdw.html) | Foreign data wrapper for remote PostgreSQL servers | 1.1 | ⭐ | | | | [seg](https://www.postgresql.org/docs/current/seg.html) | Data type for representing line segments or floating-point intervals | 1.4 | | | | | [tablefunc](https://www.postgresql.org/docs/current/tablefunc.html) | Functions that return tables (multiple rows) | 1.0 | | | | | [tcn](https://www.postgresql.org/docs/current/tcn.html) | Provides a trigger function that notifies listeners of changes to any table | 1.0 | | | | | [tsm\_system\_rows](https://www.postgresql.org/docs/current/tsm-system-rows.html) | Table sampling method SYSTEM\_ROWS which can be used in TABLESAMPLE clause | 1.0 | | | | | [tsm\_system\_time](https://www.postgresql.org/docs/current/tsm-system-time.html) | Table sampling method SYSTEM\_TIME which can be used in TABLESAMPLE clause | 1.0 | | | | | [unaccent](https://www.postgresql.org/docs/current/unaccent.html) | Text search dictionary for removing accents (diacritic signs) from lexemes | 1.1 | | | | | [uuid-ossp](https://www.postgresql.org/docs/current/uuid-ossp.html) | Functions to generate universally unique identifiers (UUIDs) | 1.1 | | | | ### Supported Community Extensions #### Postgres 18.6 | Extension | Description | Version | Superuser required | Restart required | Additional Notes | | - | - | - | - | - | - | | [address\_standardizer](https://postgis.net/docs/manual-3.5/Extras.html#Address_Standardizer) | Used to parse an address into constituent elements. Generally used to support geocoding address normalization step. (Part of PostGIS) | 3.6.4 | ⭐ | | | | [address\_standardizer\_data\_us](https://postgis.net/docs/manual-3.5/Extras.html#Address_Standardizer) | Address Standardizer US dataset example. (Part of PostGIS) | 3.6.4 | ⭐ | | | | [hypopg](https://github.com/HypoPG/hypopg) (installed by default) | Hypothetical indexes - allows testing the impact of indexes without actually creating them | 1.4.3 | | | | | [pg\_cron](https://github.com/citusdata/pg_cron) | Simple cron-based job scheduler for PostgreSQL that allows you to run SQL commands on a schedule | 1.6.7 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_cron) | | [pg\_duckdb](https://github.com/duckdb/pg_duckdb) | Embeds DuckDB's analytical database engine directly into PostgreSQL for high-performance analytical queries | 1.1.1 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_duckdb) | | [pg\_hint\_plan](https://github.com/ossc-db/pg_hint_plan) | Provides query plan hints to control the PostgreSQL planner | 1.8.0 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_hint_plan) | | [pg\_partman](https://github.com/pgpartman/pg_partman) | Extension to create and manage both time-based and serial-based table partition sets | 5.5.0 | | | | | [pg\_partman\_bgw](https://github.com/pgpartman/pg_partman) | Background worker to automatically run partition maintenance for pg\_partman | 5.5.0 | | ✅ | [📝](/docs/postgres/extensions/pg_partman_bgw) | | [pg\_squeeze](https://github.com/cybertec-postgresql/pg_squeeze/) | Automatically cleans up unused space in tables (table bloat) | 1.9.1 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_squeeze) | | [pgvector](https://github.com/pgvector/pgvector) | Open-source vector similarity search for PostgreSQL, designed for AI/ML applications | 0.8.5 | ⭐ | | [📝](/docs/postgres/extensions/pgvector) | | [pgvectorscale](https://github.com/timescale/pgvectorscale) | Open-source complement to pgvector for PostgreSQL, adding increased compression, additional filtering, and faster indexes for vector search | 0.9.0 | ⭐ | | | | [postgis](https://postgis.net/) | PostGIS extends the capabilities of the PostgreSQL relational database by adding support for storing, indexing, and querying geospatial data | 3.6.4 | ⭐ | | | | [postgis\_sfcgal](https://postgis.net/docs/manual-3.5/reference_sfcgal.html) | SFCGAL is a C++ wrapper library around CGAL that provides advanced 2D and 3D spatial functions | 3.6.4 | ⭐ | | | | [postgis\_tiger\_geocoder](https://postgis.net/docs/manual-3.5/Extras.html#Tiger_Geocoder) | A plpgsql based geocoder written to work with the TIGER (Topologically Integrated Geographic Encoding and Referencing system) / Line and Master Address database export released by the US Census Bureau | 3.6.4 | | | | | [postgis\_topology](https://postgis.net/docs/manual-3.5/Topology.html) | The PostGIS Topology types and functions are used to manage topological objects such as faces, edges and nodes | 3.6.4 | ⭐ | | | | [roaringbitmap](https://github.com/ChenHuajun/pg_roaringbitmap) | Compressed bitmap data type for efficient set operations such as union, intersection, difference, and cardinality | 1.2.0 | ⭐ | | | | [TimescaleDB](https://github.com/timescale/timescaledb) | A time-series database for high-performance real-time analytics packaged as a Postgres extension ([Apache 2 Edition](https://docs.tigerdata.com/about/latest/timescaledb-editions/)) | 2.29.2 | | ✅ | [📝](/docs/postgres/extensions/timescaledb) | | [tin](/docs/postgres/search) | High-performance full-text search index with BM25 ranking and TINQL query language | 1.0.4 | ⭐ | | [📝](/docs/postgres/search) | | [wal2json](https://github.com/eulerto/wal2json) | Logical decoding output plugin that produces JSON format change data capture (CDC) output for streaming database changes | 2.6 | ⭐ | | | #### Postgres 17.11 | Extension | Description | Version | Superuser required | Restart required | Additional Notes | | - | - | - | - | - | - | | [address\_standardizer](https://postgis.net/docs/manual-3.5/Extras.html#Address_Standardizer) | Used to parse an address into constituent elements. Generally used to support geocoding address normalization step. (Part of PostGIS) | 3.6.4 | ⭐ | | | | [address\_standardizer\_data\_us](https://postgis.net/docs/manual-3.5/Extras.html#Address_Standardizer) | Address Standardizer US dataset example. (Part of PostGIS) | 3.6.4 | ⭐ | | | | [hypopg](https://github.com/HypoPG/hypopg) (installed by default) | Hypothetical indexes - allows testing the impact of indexes without actually creating them | 1.4.3 | | | | | [pg\_cron](https://github.com/citusdata/pg_cron) | Simple cron-based job scheduler for PostgreSQL that allows you to run SQL commands on a schedule | 1.6.7 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_cron) | | [pg\_duckdb](https://github.com/duckdb/pg_duckdb) | Embeds DuckDB's analytical database engine directly into PostgreSQL for high-performance analytical queries | 1.0.0 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_duckdb) | | [pg\_hint\_plan](https://github.com/ossc-db/pg_hint_plan) | Provides query plan hints to control the PostgreSQL planner | 1.7.1 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_hint_plan) | | [pg\_partman](https://github.com/pgpartman/pg_partman) | Extension to create and manage both time-based and serial-based table partition sets | 5.5.0 | | | | | [pg\_partman\_bgw](https://github.com/pgpartman/pg_partman) | Background worker to automatically run partition maintenance for pg\_partman | 5.5.0 | | ✅ | [📝](/docs/postgres/extensions/pg_partman_bgw) | | [pg\_squeeze](https://github.com/cybertec-postgresql/pg_squeeze/) | Automatically cleans up unused space in tables (table bloat) | 1.9.1 | ⭐ | ✅ | [📝](/docs/postgres/extensions/pg_squeeze) | | [pgvector](https://github.com/pgvector/pgvector) | Open-source vector similarity search for PostgreSQL, designed for AI/ML applications | 0.8.5 | ⭐ | | [📝](/docs/postgres/extensions/pgvector) | | [pgvectorscale](https://github.com/timescale/pgvectorscale) | Open-source complement to pgvector for PostgreSQL, adding increased compression, additional filtering, and faster indexes for vector search | 0.9.0 | ⭐ | | | | [postgis](https://postgis.net/) | PostGIS extends the capabilities of the PostgreSQL relational database by adding support for storing, indexing, and querying geospatial data | 3.6.4 | ⭐ | | | | [postgis\_sfcgal](https://postgis.net/docs/manual-3.5/reference_sfcgal.html) | SFCGAL is a C++ wrapper library around CGAL that provides advanced 2D and 3D spatial functions | 3.6.4 | ⭐ | | | | [postgis\_tiger\_geocoder](https://postgis.net/docs/manual-3.5/Extras.html#Tiger_Geocoder) | A plpgsql based geocoder written to work with the TIGER (Topologically Integrated Geographic Encoding and Referencing system) / Line and Master Address database export released by the US Census Bureau | 3.6.4 | | | | | [postgis\_topology](https://postgis.net/docs/manual-3.5/Topology.html) | The PostGIS Topology types and functions are used to manage topological objects such as faces, edges and nodes | 3.6.4 | ⭐ | | | | [roaringbitmap](https://github.com/ChenHuajun/pg_roaringbitmap) | Compressed bitmap data type for efficient set operations such as union, intersection, difference, and cardinality | 1.2.0 | ⭐ | | | | [TimescaleDB](https://github.com/timescale/timescaledb) | A time-series database for high-performance real-time analytics packaged as a Postgres extension ([Apache 2 Edition](https://docs.tigerdata.com/about/latest/timescaledb-editions/)) | 2.29.2 | | ✅ | [📝](/docs/postgres/extensions/timescaledb) | | [tin](/docs/postgres/search) | High-performance full-text search index with BM25 ranking and TINQL query language | 1.0.4 | ⭐ | | [📝](/docs/postgres/search) | | [wal2json](https://github.com/eulerto/wal2json) | Logical decoding output plugin that produces JSON format change data capture (CDC) output for streaming database changes | 2.6 | ⭐ | | | ### PlanetScale Extensions These extensions are installed by PlanetScale and cannot be disabled. | Extension | Description | Version | Additional Notes | | - | - | - | - | | pgextwlist | PostgreSQL Extension Allowlist | N/A | | | pginsights | PlanetScale Insights plugin | N/A | [📝](/docs/postgres/extensions/pginsights) | | pg\_pscale\_utils | Utility extension to handle superuser actions in PlanetScale Postgres | N/A | | | pg\_strict | Block UPDATE and DELETE statements without WHERE clauses to prevent accidental mass mutations | N/A | [📝](/docs/postgres/extensions/pg_strict) | ## Need Additional Extensions? If you require an extension that is not currently supported, please [reach out to support](https://planetscale.com/contact?initial=support). We regularly evaluate and add new extensions based on customer needs and security considerations. # Extensions: auto_explain Source: https://planetscale.com/docs/postgres/extensions/auto_explain auto_explain logs execution plans for slow statements automatically to help troubleshoot query performance. ## Dashboard Configuration This extension requires activation via the PlanetScale dashboard before it can be used. It is loaded through `session_preload_libraries` and does not require a database restart. To enable auto\_explain: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose extensions you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Extensions** tab **Step 5** Enable auto\_explain and configure its parameters **Step 6** Click **Queue extension changes** to apply the configuration **Step 7** Once you're ready to apply the changes, click "**Apply changes**" ## Parameters ### auto\_explain.log\_min\_duration * **Type**: Integer (milliseconds) * **Default**: `-1` * **Description**: Minimum statement duration to log a plan for. Set to `-1` to disable logging by duration. ### auto\_explain.log\_analyze * **Type**: Boolean * **Default**: `false` * **Description**: Uses `EXPLAIN ANALYZE` for logged plans to include actual execution statistics. ### auto\_explain.log\_buffers * **Type**: Boolean * **Default**: `false` * **Description**: Includes buffer usage statistics in logged plans. ### auto\_explain.log\_wal * **Type**: Boolean * **Default**: `false` * **Description**: Includes WAL usage statistics in logged plans. ### auto\_explain.log\_timing * **Type**: Boolean * **Default**: `true` * **Description**: Includes per-node timing information in logged plans. ### auto\_explain.log\_triggers * **Type**: Boolean * **Default**: `false` * **Description**: Includes trigger execution statistics in logged plans. ### auto\_explain.log\_verbose * **Type**: Boolean * **Default**: `false` * **Description**: Uses `EXPLAIN VERBOSE` output for logged plans. ### auto\_explain.log\_format * **Type**: Select * **Options**: text, xml, json, yaml * **Default**: `text` * **Description**: Output format used when logging plans. ### auto\_explain.log\_level * **Type**: Select * **Default**: `log` * **Description**: Log severity used when writing query plans. ### auto\_explain.log\_nested\_statements * **Type**: Boolean * **Default**: `false` * **Description**: Logs plans for nested statements that run inside functions. ### auto\_explain.sample\_rate * **Type**: Decimal * **Default**: `1` * **Minimum**: `0` * **Maximum**: `1` * **Description**: Fraction of statements that are sampled for plan logging. ## Usage `auto_explain` is not installed with `CREATE EXTENSION`. After enabling it in the dashboard, sessions load it through `session_preload_libraries`. You can verify and tune settings in SQL: ```sql theme={null} SHOW session_preload_libraries; SET auto_explain.log_min_duration = '250ms'; SET auto_explain.sample_rate = 0.25; ``` Logged plans appear in your cluster logs when statements match the configured thresholds. ## External Documentation For more detail about `auto_explain` behavior and parameter semantics, see the [official PostgreSQL documentation](https://www.postgresql.org/docs/current/auto-explain.html). # Extensions: pg_cron Source: https://planetscale.com/docs/postgres/extensions/pg_cron pg_cron is a simple cron-based job scheduler for PostgreSQL. It allows you to run SQL commands on a schedule, similar to how cron jobs work in Unix-like systems. ## Dashboard Configuration This extension requires activation via the PlanetScale Dashboard before it can be used. It must be enabled through shared libraries and requires a database restart. To enable pg\_cron: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose extensions you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Extensions** tab **Step 5** Enable pg\_cron and configure its parameters **Step 6** Click **Queue extension changes** to apply the configuration **Step 7** Once you're ready to apply the changes, click "**Apply changes**" ## Parameters ### cron.database\_name * **Type**: String * **Default**: `postgres` * **Description**: Name of the database where pg\_cron is installed (via CREATE EXTENSION). ### cron.launch\_active\_jobs * **Type**: Boolean * **Default**: `true` * **Description**: Switch to enable/disable running cron jobs - applies to all jobs. ### cron.log\_min\_messages * **Type**: Select * **Options**: error, warning, notice, info, log, debug * **Default**: `warning` * **Description**: Lowest severity messages to log from the launcher background worker. ### cron.log\_run * **Type**: Boolean * **Default**: `true` * **Description**: Log all cron runs in the cron.job\_run\_details table. ### cron.log\_statement * **Type**: Boolean * **Default**: `true` * **Description**: Log all cron statements before running them. ### cron.max\_running\_jobs * **Type**: Integer * **Default**: `1` * **Minimum**: `1` * **Description**: Maximum number of jobs that can run at once. Must be less than or equal to cluster-level max\_worker\_processes. ## Usage After enabling the extension through the dashboard, you can install it in your database: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS pg_cron; ``` Then you can schedule jobs using the `cron.schedule` function: ```sql theme={null} -- Schedule a job to run every minute SELECT cron.schedule('job-name', '* * * * *', 'SELECT 1;'); ``` ## External Documentation For more detailed information about pg\_cron usage and functionality, see the [official pg\_cron documentation](https://github.com/citusdata/pg_cron). # Extensions: pg_duckdb Source: https://planetscale.com/docs/postgres/extensions/pg_duckdb pg_duckdb is a Postgres extension that embeds DuckDB, a high-performance analytical database, directly into Postgres. > [!WARNING] > > We don't recommend running `pg_duckdb` directly on your PlanetScale Postgres database as it can consume significant resources during analytical queries. If you want to use DuckDB for analytics, we recommend using [MotherDuck](https://motherduck.com/) to host your analytical workloads separately. ## Dashboard Configuration This extension requires activation via the PlanetScale dashboard before it can be used. It must be enabled through shared libraries and requires a database restart. To enable pg\_duckdb: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose extensions you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Extensions** tab **Step 5** Enable `pg_duckdb` and configure its parameters **Step 6** Click **Queue extension changes** to apply the configuration **Step 7** Once you're ready to apply the changes, click "**Apply changes**" ## Parameters ### duckdb.postgres\_role * **Type**: String * **Default**: `pscale_superuser` * **Description**: Specifies the Postgres role that is allowed to use DuckDB execution and manage secrets. ### duckdb.memory\_limit * **Type**: Integer * **Default**: 0 * **Description**: Maximum memory DuckDB can use per connection in megabytes. Setting to 0 activates DuckDB's default (80% of available RAM). ## Usage After enabling the extension through the dashboard, you can install it in your database: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS pg_duckdb; ``` Once installed, you can use DuckDB's analytical capabilities directly from PostgreSQL. For example: ```sql theme={null} -- Query PostgreSQL tables using DuckDB's analytical engine SELECT * FROM duckdb.read_parquet('s3://bucket/data.parquet'); ``` ## External Documentation For more detailed information about `pg_duckdb` usage and functionality, see the [official `pg_duckdb` documentation](https://github.com/duckdb/pg_duckdb). # Extensions: pg_hint_plan Source: https://planetscale.com/docs/postgres/extensions/pg_hint_plan The pg_hint_plan extension allows you to influence the query planner's decisions by providing hints. It can be used to optimize query performance by guiding the planner towards more efficient execution plans. ## Dashboard Configuration The pg\_hint\_plan extension requires activation via the PlanetScale Dashboard before it can be used. It must be enabled through shared libraries and requires a database restart. To enable pg\_hint\_plan: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose extensions you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Extensions** tab **Step 5** Enable pg\_hint\_plan and configure its parameters **Step 6** Click **Queue extension changes** to apply the configuration **Step 7** Once you're ready to apply the changes, click "**Apply changes**" ## Parameters ### pg\_hint\_plan.enable\_hint * **Type**: Boolean * **Default**: `true` * **Description**: Enable/disable hint functionality. ### pg\_hint\_plan.enable\_hint\_table * **Type**: Boolean * **Default**: `false` * **Description**: Enable hint table functionality. ### pg\_hint\_plan.parse\_messages * **Type**: Select * **Options**: error, warning, notice, info, log, debug * **Default**: `info` * **Description**: Control parsing message output. ### pg\_hint\_plan.debug\_print * **Type**: Select * **Options**: off, on, detailed, verbose * **Default**: `off` * **Description**: Enable debug output. ### pg\_hint\_plan.message\_level * **Type**: Select * **Options**: error, warning, notice, info, log, debug * **Default**: `info` * **Description**: Set message verbosity level. ## Usage > [!NOTE] > > Unlike most PostgreSQL extensions, `pg_hint_plan` does not require `CREATE EXTENSION` to function. Once enabled > through the dashboard, it's automatically loaded and available for use. You only need to run `CREATE EXTENSION` if you > plan to use the hint table functionality (when `pg_hint_plan.enable_hint_table` is enabled). After enabling the extension through the dashboard, you can optionally install it in your database for hint table functionality: ```sql theme={null} -- Only required if using hint tables CREATE EXTENSION IF NOT EXISTS pg_hint_plan; ``` Then you can use hints in your queries: ```sql theme={null} /*+ SeqScan(t1) IndexScan(t2 t2_pkey) */ SELECT * FROM table1 t1 JOIN table2 t2 ON t1.id = t2.id; ``` ## External Documentation For more detailed information about pg\_hint\_plan usage and hint syntax, see the [official pg\_hint\_plan documentation](https://github.com/ossc-db/pg_hint_plan). # Extensions: pg_partman_bgw Source: https://planetscale.com/docs/postgres/extensions/pg_partman_bgw The pg_partman_bgw extension is a background worker for managing partitioned tables in PostgreSQL. It automates the creation and maintenance of partitioned tables, making it easier to manage large datasets. ## Dashboard Configuration This extension requires activation via the PlanetScale Dashboard before it can be used. It must be enabled through shared libraries and requires a database restart. To enable pg\_partman\_bgw: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose extensions you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Extensions** tab **Step 5** Enable pg\_partman\_bgw and configure its parameters **Step 6** Click **Queue extension changes** to apply the configuration **Step 7** Once you're ready to apply the changes, click "**Apply changes**" ## Parameters ### pg\_partman\_bgw\.interval * **Type**: Integer * **Default**: `3600` * **Description**: Interval (in seconds) between partition maintenance runs. ### pg\_partman\_bgw\.dbname * **Type**: String * **Default**: `postgres` * **Description**: Database name where the background worker should connect. ### pg\_partman\_bgw\.role * **Type**: String * **Default**: `postgres` * **Description**: Role that the background worker should use when connecting. ### pg\_partman\_bgw\.analyze * **Type**: Boolean * **Default**: `false` * **Description**: Whether to run ANALYZE on newly created partitions. ### pg\_partman\_bgw\.jobmon * **Type**: Boolean * **Default**: `true` * **Description**: Enable job monitoring for partition maintenance activities. ## Usage After enabling the extension through the dashboard, the background worker will automatically run partition maintenance based on your pg\_partman configuration. You'll also need to install the pg\_partman extension: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS pg_partman; ``` The background worker works in conjunction with pg\_partman to automatically maintain your partitioned tables. ## External Documentation For more detailed information about pg\_partman and pg\_partman\_bgw usage, see the [official pg\_partman documentation](https://github.com/pgpartman/pg_partman). # Extensions: pg_squeeze Source: https://planetscale.com/docs/postgres/extensions/pg_squeeze pg_squeeze automatically cleans up unused space in tables (table bloat). pg_squeeze helps maintain optimal table performance by removing dead space that accumulates over time from UPDATE and DELETE operations. ## Dashboard Configuration This extension requires activation via the PlanetScale Dashboard before it can be used. It must be enabled through shared libraries and requires a database restart. To enable pg\_squeeze: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose extensions you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Extensions** tab **Step 5** Enable pg\_squeeze and configure its parameters **Step 6** Click **Queue extension changes** to apply the configuration **Step 7** Once you're ready to apply the changes, click "**Apply changes**" ## Parameters ### squeeze.max\_xlock\_time * **Type**: Integer (milliseconds) * **Default**: `0` * **Minimum**: `0` * **Description**: The maximum time the processed table may be locked exclusively. ### squeeze.worker\_autostart * **Type**: String * **Default**: `postgres` * **Description**: Space-separated list of databases to start background workers for automatically. ### squeeze.workers\_per\_database * **Type**: Integer * **Default**: `1` * **Minimum**: `1` * **Description**: Maximum number of worker processes launched per database. Must be less than or equal to cluster-level max\_worker\_processes. ## Usage After enabling the extension through the dashboard, you can install it in your database: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS pg_squeeze; ``` Then you can schedule tables for squeezing: ```sql theme={null} -- Schedule a table to be squeezed SELECT squeeze.squeeze_table('public', 'your_table_name'); ``` > [!NOTE] > > Only the [default role](/docs/postgres/connecting/roles#default-role) has the permissions necessary to start a squeeze process. ## External Documentation For more detailed information about pg\_squeeze usage and configuration, see the [official pg\_squeeze documentation](https://github.com/cybertec-postgresql/pg_squeeze/). # Extensions: pg_stat_statements Source: https://planetscale.com/docs/postgres/extensions/pg_stat_statements pg_stat_statements is a PostgreSQL extension that tracks execution statistics of all SQL statements executed by a server. It provides insights into query performance, allowing you to identify slow queries and optimize them ## Dashboard Configuration This extension requires activation via the PlanetScale Dashboard before it can be used. It must be enabled through shared libraries and requires a database restart. To enable pg\_stat\_statements: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose extensions you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Extensions** tab **Step 5** Enable pg\_stat\_statements and configure its parameters **Step 6** Click **Queue extension changes** to apply the configuration **Step 7** Once you're ready to apply the changes, click "**Apply changes**" ## Parameters ### pg\_stat\_statements.max * **Type**: Integer * **Default**: `5000` * **Description**: Maximum number of statements tracked by the extension. ### pg\_stat\_statements.track * **Type**: Select * **Options**: top, all * **Default**: `top` * **Description**: Which statements to track. 'top' tracks top-level statements only, 'all' tracks all statements including nested ones. ### pg\_stat\_statements.track\_utility * **Type**: Boolean * **Default**: `true` * **Description**: Whether to track utility statements (such as PREPARE, EXPLAIN, etc.). ### pg\_stat\_statements.track\_planning * **Type**: Boolean * **Default**: `false` * **Description**: Whether to track planning statistics for statements. ### pg\_stat\_statements.save * **Type**: Boolean * **Default**: `true` * **Description**: Whether to save statement statistics across server restarts. ## Usage After enabling the extension through the dashboard, you can install it in your database: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS pg_stat_statements; ``` Then you can query the statistics view: ```sql theme={null} -- View the most time-consuming queries SELECT query, calls, total_exec_time, mean_exec_time, rows FROM pg_stat_statements ORDER BY total_exec_time DESC LIMIT 10; ``` ## External Documentation For more detailed information about pg\_stat\_statements usage and the available statistics, see the [official PostgreSQL documentation](https://www.postgresql.org/docs/current/pgstatstatements.html). # Extensions: pg_strict Source: https://planetscale.com/docs/postgres/extensions/pg_strict pg_strict blocks dangerous queries before they run. It prevents common mistakes like UPDATE or DELETE statements without WHERE clauses that would affect every row in a table. ## Overview pg\_strict intercepts queries at parse time and blocks operations that are likely to be mistakes. When enabled, it prevents accidental mass updates or deletes that could affect your entire table. [Protect your database. Use the pg\_strict Postgres extension.](https://www.youtube-nocookie.com/embed/noPnqjXsZxs?rel=0) > [!NOTE] > > pg\_strict configuration only applies to connections established after the change. It does not impact existing connections, even if they would violate the newly-configured rules. ## Enabling pg\_strict pg\_strict is configured per-role when creating new roles in the dashboard. To add pg\_strict to existing roles, see [per-role configuration](#per-role-configuration). **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to **Settings** > **Roles** **Step 3** Click **New role** **Step 4** Configure the role permissions **Step 5** Under **pg\_strict**, select a mode for each setting **Step 6** Click **Create role** ## Modes Each pg\_strict setting supports three modes: | Mode | Behavior | | - | - | | `off` | Disabled, standard PostgreSQL behavior | | `warn` | Log a warning but allow the query to run | | `on` | Block the query with an error | ## What it blocks > [!NOTE] > > We're actively expanding pg\_strict with additional safety checks. More configurations will be added soon to help prevent other common mistakes. ### `UPDATE` without `WHERE` When `pg_strict.require_where_on_update` is enabled, UPDATE statements must include a WHERE clause: ```sql theme={null} -- Blocked (affects all rows) UPDATE users SET status = 'inactive'; -- Allowed (targets specific rows) UPDATE users SET status = 'inactive' WHERE last_login < '2024-01-01'; ``` ### `DELETE` without `WHERE` When `pg_strict.require_where_on_delete` is enabled, DELETE statements must include a WHERE clause: ```sql theme={null} -- Blocked (deletes all rows) DELETE FROM sessions; -- Allowed (targets specific rows) DELETE FROM sessions WHERE expired_at < NOW(); ``` ## Overriding for one-off operations For intentional bulk operations, use `SET LOCAL` within a transaction to temporarily disable a specific check: ```sql theme={null} BEGIN; SET LOCAL pg_strict.require_where_on_delete = off; DELETE FROM temp_import_data; -- Allowed within this transaction COMMIT; -- Setting restored after commit ``` ## Database and role configuration Settings can be applied at the database level (affects all roles) or per-role. ### Database-wide default Enable pg\_strict for all connections to a database: ```sql theme={null} ALTER DATABASE postgres SET pg_strict.require_where_on_update = 'on'; ALTER DATABASE postgres SET pg_strict.require_where_on_delete = 'on'; ``` ### Per-role configuration Configure pg\_strict for existing or new roles: ```sql theme={null} -- App role: block dangerous queries ALTER ROLE app_service SET pg_strict.require_where_on_update = 'on'; ALTER ROLE app_service SET pg_strict.require_where_on_delete = 'on'; -- Migration role: warn only ALTER ROLE migration_user SET pg_strict.require_where_on_update = 'warn'; ALTER ROLE migration_user SET pg_strict.require_where_on_delete = 'warn'; -- Admin role: full access ALTER ROLE dba_admin SET pg_strict.require_where_on_update = 'off'; ALTER ROLE dba_admin SET pg_strict.require_where_on_delete = 'off'; ``` This allows you to set a strict default at the database level while relaxing restrictions for specific roles that need to perform bulk operations. # Extensions: pginsights Source: https://planetscale.com/docs/postgres/extensions/pginsights The pginsights extension is a PlanetScale Insights plugin that provides query performance monitoring and analysis capabilities. This extension is always enabled for PlanetScale databases and integrates with the Query Insights feature. ## Dashboard Configuration This extension is always enabled for PlanetScale databases and requires activation via the PlanetScale Dashboard. While the extension itself is always active, some of its parameters can be configured through the dashboard. To configure pginsights parameters: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose extensions you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Extensions** tab **Step 5** Enable pginsights and configure its parameters **Step 6** Click **Queue extension changes** to apply the configuration **Step 7** Once you're ready to apply the changes, click "**Apply changes**" ## Parameters ### Raw query collection * **Setting:** `pginsights.raw_queries` * **Type:** Boolean * **Default:** `false` When enabled, Insights collects the full query text for notable queries, including all literal values. When disabled, only the normalized SQL, with literals removed, will be collected. Enabling complete query collection can be helpful when performance varies significantly within the same query pattern, and you need to see the full SQL statement without placeholders to troubleshoot the underlying issue. > [!NOTE] > > Enabling this setting may result in sensitive data that appears in queries being sent to PlanetScale, where it will be processed and stored in accordance with our privacy policy. ### Schema name normalization * **Setting:** `pginsights.normalize_schema_names` * **Type:** Boolean * **Default:** `false` When enabled, [schema](https://www.postgresql.org/docs/current/ddl-schemas.html) names appearing in queries are normalized as if they were literal values. Consider the following example query: `select * from myschema.users where id = 1`. * With `pginsights.normalize_schema_names` set to false, the query will be reported in insights as `select * from myschema.users where id = $1` * With `pginsights.normalize_schema_names` set to true, the query will be reported in insights as `select * from $1.users where id = $2` This setting is useful for databases using a schema-per-tenant design, where each user or tenant's data is stored in an isolated Postgres schema. With this feature enabled, query patterns that are identical except for the namespace will be grouped together, resulting in fewer distinct query patterns and more navigable insights. ### Database Traffic Control shared memory size * **Setting:** `traffic_control.shm_size` * **Type:** Integer (bytes) * **Default:** `64kB` Controls how much shared memory is allocated to store [Database Traffic Control](/docs/postgres/traffic-control) resource budgets. This setting matters most for Traffic Control rules that match each unique value for a tag key. The default `64kB` size tracks about 600 active budget entries at once. The maximum `1MB` size tracks about 10,000 active budget entries. Active budget entries are stored in an LRU cache. The limit applies to recently active values, not the total number of possible tag values. ## Usage The pginsights extension is automatically installed and enabled for all PlanetScale databases. You don't need to manually create the extension - it's always active and collecting query insights. The extension integrates with PlanetScale's Query Insights dashboard to provide: * Query performance metrics * Slow query identification * Query pattern analysis * Resource usage tracking ## External Documentation For more information about Query Insights and how to use the collected data, see the [Query Insights documentation](/docs/postgres/monitoring/query-insights). # Extensions: pgvector Source: https://planetscale.com/docs/postgres/extensions/pgvector Open-source vector similarity search for Postgres, Store embeddings and run nearest-neighbor queries ## Overview [pgvector](https://github.com/pgvector/pgvector) adds a `vector` column type, distance operators, and approximate nearest-neighbor indexes (HNSW and IVFFlat) to PostgreSQL. Use it to store embeddings alongside relational data and query by similarity. PlanetScale Postgres also supports [pgvectorscale](https://github.com/timescale/pgvectorscale), a companion extension for higher-performance approximate search. See the [supported extensions](/docs/postgres/extensions#supported-community-extensions) table for available versions. ## Enabling pgvector pgvector does not require a cluster restart. Install it with a role that has superuser privileges: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS vector; ``` Confirm the extension is installed: ```sql theme={null} SELECT extname, extversion FROM pg_extension WHERE extname = 'vector'; ``` ## Usage Create a table with a vector column, insert embeddings, and query by distance: ```sql theme={null} CREATE TABLE items ( id bigserial PRIMARY KEY, content text, embedding vector(3) ); INSERT INTO items (content, embedding) VALUES ('red', '[1,0,0]'), ('green', '[0,1,0]'), ('blue', '[0,0,1]'); -- Nearest neighbors by L2 distance SELECT id, content FROM items ORDER BY embedding <-> '[1,0.2,0]' LIMIT 5; ``` Common distance operators: | Operator | Meaning | | - | - | | `<->` | L2 distance | | `<#>` | Negative inner product | | `<=>` | Cosine distance | Add an HNSW index when you need approximate search at scale: ```sql theme={null} CREATE INDEX ON items USING hnsw (embedding vector_l2_ops); ``` ## External documentation For indexing options, half-precision and sparse vectors, and language clients, see the [official pgvector documentation](https://github.com/pgvector/pgvector). # Extensions: TimescaleDB Source: https://planetscale.com/docs/postgres/extensions/timescaledb TimescaleDB is a time-series database for high-performance real-time analytics packaged as a Postgres extension. > [!NOTE] > > Only features from the [Apache 2 Edition](https://docs.tigerdata.com/about/latest/timescaledb-editions/) of TimescaleDB are supported. ## Dashboard Configuration This extension requires activation via the PlanetScale Dashboard before it can be used. It must be enabled through shared libraries and requires a database restart. To enable TimescaleDB: **Step 1** From the PlanetScale organization dashboard, select the desired database **Step 2** Navigate to the **Clusters** page from the menu on the left **Step 3** Choose the branch whose extensions you'd like to configure in the "**Branch**" dropdown **Step 4** Select the **Extensions** tab **Step 5** Enable timescaledb and configure its parameters **Step 6** Click **Queue extension changes** to apply the configuration **Step 7** Once you're ready to apply the changes, click "**Apply changes**" ## Usage After enabling the extension through the dashboard, you'll be able to install it in your Postgres cluster: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS timescaledb; ``` ## External Documentation For more detailed information about TimescaleDB usage, see the [official documentation](https://github.com/timescale/timescaledb). ## Parameters ### timescaledb.enable\_chunk\_append * **Type**: Boolean * **Default**: `true` * **Description**: Enable chunk append node ### timescaledb.enable\_chunk\_skipping * **Type**: Boolean * **Default**: `false` * **Description**: Enable chunk skipping functionality ### timescaledb.enable\_constraint\_aware\_append * **Type**: Boolean * **Default**: `true` * **Description**: Enable constraint-aware append scans ### timescaledb.enable\_constraint\_exclusion * **Type**: Boolean * **Default**: `true` * **Description**: Enable constraint exclusion ### timescaledb.enable\_custom\_hashagg * **Type**: Boolean * **Default**: `false` * **Description**: Enable custom hash aggregation ### timescaledb.enable\_deprecation\_warnings * **Type**: Boolean * **Default**: `true` * **Description**: Enable warnings when using deprecated functionality ### timescaledb.enable\_event\_triggers * **Type**: Boolean * **Default**: `false` * **Description**: Enable event triggers for chunks creation ### timescaledb.enable\_foreign\_key\_propagation * **Type**: Boolean * **Default**: `true` * **Description**: Enable foreign key propagation ### timescaledb.enable\_job\_execution\_logging * **Type**: Boolean * **Default**: `false` * **Description**: Enable job execution logging ### timescaledb.enable\_now\_constify * **Type**: Boolean * **Default**: `true` * **Description**: Enable now() constify ### timescaledb.enable\_optimizations * **Type**: Boolean * **Default**: `true` * **Description**: Enable TimescaleDB query optimizations ### timescaledb.enable\_ordered\_append * **Type**: Boolean * **Default**: `true` * **Description**: Enable ordered append scans ### timescaledb.enable\_parallel\_chunk\_append * **Type**: Boolean * **Default**: `true` * **Description**: Enable parallel chunk append node ### timescaledb.enable\_qual\_propagation * **Type**: Boolean * **Default**: `true` * **Description**: Enable qualifier propagation ### timescaledb.enable\_runtime\_exclusion * **Type**: Boolean * **Default**: `true` * **Description**: Enable runtime chunk exclusion ### timescaledb.enable\_tiered\_reads * **Type**: Boolean * **Default**: `true` * **Description**: Enable tiered data reads ### timescaledb.enable\_tss\_callbacks * **Type**: Boolean * **Default**: `true` * **Description**: Enable ts\_stat\_statements callbacks ### timescaledb.max\_cached\_chunks\_per\_hypertable * **Type**: Integer * **Default**: `1024` * **Min**: `0` * **Max**: `65536` * **Description**: Maximum cached chunks ### timescaledb.max\_open\_chunks\_per\_insert * **Type**: Integer * **Default**: `1024` * **Min**: `0` * **Max**: `32767` * **Description**: Maximum open chunks per insert ### timescaledb.restoring * **Type**: Boolean * **Default**: `false` * **Description**: Enable restoring mode for TimescaleDB # Migrate from Aurora to PlanetScale Source: https://planetscale.com/docs/postgres/imports/aurora Use this guide to migrate an existing Aurora (Postgres) database to PlanetScale Postgres. > [!NOTE] > > Before beginning your migration, we recommend running our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. This guide will cover a no-downtime approach to migrating using Postgres logical replication. If you are willing to tolerate downtime during a maintenance window, you may also use [`pg_dump` and restore](/docs/postgres/imports/postgres-migrate-dumprestore). The `pg_dump`/restore approach is simpler, but is only for applications where downtime is acceptable. This guide assumes that public internet access is enabled on your Aurora database, as it will be needed to connect and replicate to the new PlanetScale host. If you cannot enable this due to security policies, consider using [AWS DMS](/docs/postgres/imports/aurora-dms) for your migration, or [contact support](https://planetscale.com/contact?initial=support) for more specific guidance. These instructions work for all versions of Postgres that support logical replication (version 10+). If you have an older version you want to bring to PlanetScale, [contact us](https://planetscale.com/contact?initial=support) for guidance. Before beginning a migration, you should check our [extensions documentation](/docs/postgres/extensions) to ensure that all of the extensions you rely on will work on PlanetScale. As an alternative to this guide, you can also try our [Postgres migration scripts](https://github.com/planetscale/migration-scripts/tree/main/postgres-direct). These allow you to automate some of the manual steps that we describe in this guide. > [!NOTE] > > Want expert guidance for your migration? PlanetScale's [migration services](https://planetscale.com/migrate) are available to help you plan and execute a smooth, successful move. ## 1. Prepare your PlanetScale database Create a new database in the [PlanetScale dashboard](https://app.planetscale.com) or using the [PlanetScale CLI](/docs/cli). A few things to check when configuring your database: * Ensure you select the correct cloud region. You typically want to use the same region that you deploy your other application infrastructure to. * This guide assumes you are migrating from a Postgres Aurora database, so also choose the Postgres option in PlanetScale. * Choose the best storage option for your needs. For applications needing high-performance and low-latency I/O, use [PlanetScale Metal](/docs/metal). For applications that need more flexible storage options or smaller compute instances, choose "Elastic Block Storage" or "Persistent Disk." ![Create a new PlanetScale Postgres database](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=ed2edfd9919a5be57329cb6fba2794cb) Once the database is created and ready, navigate to your dashboard and click the "Connect" button. ![Connect to a PlanetScale Postgres database](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image2.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=d4e761af35a5af36b2e09519bdea8857) From here, follow the instructions to create a new default role. This role will act as your admin role, with the highest level of privileges. Though you may use this one for your migration, we recommend you use a separate role with lesser privileges for your migration and general database connections. To create a new role, navigate to the [Role management page](/docs/postgres/connecting/roles) in your database settings. Click "New role" and give the role a memorable name. By default, `pg_read_all_data` and `pg_write_all_data` are enabled. In addition to these, enable `pg_create_subscription` and `postgres`, and then create the role. ![New Postgres role privileges](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image3.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=0bd2fc3186412fbe6b82837dae75a6ee) Copy the password and all other connection credentials into environment variables for later use: ```bash theme={null} PLANETSCALE_USERNAME=pscale_api_XXXXXXXXXX.XXXXXXXXXX PLANETSCALE_PASSWORD=pscale_pw_XXXXXXXXXXXXXXXXXXXXXXX PLANETSCALE_HOST=XXXX.pg.psdb.cloud PLANETSCALE_DBNAME=postgres ``` We also recommend that you increase `max_worker_processes` for the duration of the migration, in order to speed up data copying. Go to the "Parameters" tab of the "Clusters" page: ![Configure parameters](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image4.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=06e3429bbf27c259b8ed232c02a1ca30) On this page, increase this value from the default of `4` to `10` or more: ![Configure max worker processes](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image5.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=b1834c66b40c0a4ff56d7e1540d4474a) You can decrease these values after the migration is complete. ## 2. Configure disk size on PlanetScale If you are importing into a database backed by network-attached storage, you must configure your disk in advance to ensure your database will fit. Though we support disk autoscaling for these, AWS and GCP limit how frequently disks can be resized. If you don't ensure your disk is large enough for the import in advance, it will not be able to resize fast enough for a large data import. To configure this, navigate to "Clusters" and then the "Storage" tab: ![Storage configuration min size](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/imports/storage-configuration-min-size.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=41d36935a4bf7fc12799794d0cde1a51) On this page, adjust the "Minimum disk size." You should set this value to at least 150% of the size of the database you are migrating. For example, if the database you are importing is 330 GB, you should set your minimum disk size to at least 500 GB. The 50% overhead is to account for: 1. Data growth during the import process and 2. Table and index bloat that can occur during the import process. This can be later mitigated with careful [VACUUMing](https://www.postgresql.org/docs/current/sql-vacuum.html) or using an extension like [pg\_squeeze](/docs/postgres/extensions/pg_squeeze), but is difficult to avoid during the migration itself. When ready, queue and apply the changes. You can check the "Changes" tab to see the status of the resize: ![Confirm disk size change](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/imports/confirm-disk-size-change.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=3642cee5a9eddb534263ecd8a70d81eb) Wait for it to indicate completion. If you are importing to a Metal database, you must choose a disk size when first creating your database. You should launch your cluster with a disk size at least 50% larger than the storage used by your current source database (150% of the existing total). As an example, if you need to import a 330 GB database onto a PlanetScale `M-160` there are three storage sizes available: ![Metal disk size](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/imports/metal-disk-size.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=18c7b2f2b9dbdfe9d20b22fbd3e634c7) You should use the largest, 1.25TB option during the import. After importing and cleaning up table bloat, you may be able to downsize to the 468 GB option. Resizing is a no-downtime operation that can be performed on the [Clusters](/docs/postgres/cluster-configuration) page. ## 3. Prepare the Aurora database For PlanetScale to import your database, it needs to be publicly accessible. You can check this in your AWS dashboard. In the writer instance of your database cluster, go to the “Connectivity & security” tab, and under “Security” you will see if your database is publicly accessible. If it says “No,” you will need to change it to be publicly accessible through the “Modify” button. If this is an issue, you cannot do this, or you have questions, please [contact support](https://planetscale.com/contact?initial=support) to explore your migration options. You will also need to change some parameters and ensure that logical replication is enabled. If you don't already have a parameter group for your Aurora cluster, create one from the "Parameter groups" page in the AWS console: ![AWS parameter groups](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image6.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=62984e0293e198442598bc9ce5e26c77) From here, click the button to create a new group. Choose whichever name and description you want. Set the `Engine type` to `Aurora Postgres` and the `Parameter family group` to the version that matches your Aurora Postgres database. Set the `Type` to `DB Cluster Parameter Group`. ![Create an AWS parameter group](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image7.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=9763f41e56d02fe071fa728cc34bd808) If you already have a custom parameter group for your cluster, you can use the existing one instead. The two key parameters you need to update are adding `pglogical` to `shared_preload_libraries` and setting `rds.logical_replication` to `1`: ![Preload libraries parameter](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image8.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=cc6925ec16a6ae35e304dbfc00883ce6) ![Logical replication parameter](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image9.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=66aac0e4a8d9c3da045a2b655b679ef2) Once these are set, you need to make sure your Aurora database is configured to use them. Navigate to your Aurora database in the AWS console, click the "Modify" button, and then ensure your database is using the parameter group: ![Set parameter group for Aurora](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image10.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=5aeb15ae094f092cf1d51a683b840a8e) When you go to save the changes, select the option to either apply immediately or during your next maintenance window. The changes may take time to propagate. You can confirm that the `wal_level` is set to `logical` by running `SHOW wal_level;` on your Aurora database: ```sql theme={null} postgres=> SHOW wal_level; wal_level ----------- logical ``` If you see a result other than `logical`, then it is not configured correctly. If you are having trouble getting the settings to propagate, you can try restarting the Aurora instance, though that will cause a period of downtime. ## 4. Copy schema from Aurora to PlanetScale Before we begin migrating data, we first must copy the schema from Aurora to PlanetScale. We do this as a distinct set of steps using `pg_dump`. > [!WARNING] > > You should not make any schema changes during the migration process. You may continue to select, insert, update, and delete data, keeping your application fully online during this process. For these instructions, you'll need to connect to Aurora with a role that has permissions to create replication publications and read all data. Your default role that was generated by Aurora when you first created your database should suffice here, but you may also use other roles. We will assume that the credentials for this user and other connection info are stored in the following environment variables. ```bash theme={null} AURORA_USERNAME=XXXX AURORA_PASSWORD=XXXX AURORA_HOST=XXX AURORA_DBNAME=XXX ``` Run the below command to take a snapshot of the full schema of the `$AURORA_DBNAME` that you want to migrate: ```bash theme={null} PGPASSWORD=$AURORA_PASSWORD \ pg_dump -h $AURORA_HOST \ -p 5432 \ -U $AURORA_USERNAME \ -d $AURORA_DBNAME \ --schema-only \ --no-owner \ --no-privileges \ -f schema.sql ``` This saves the schema into a file named `schema.sql`. > [!NOTE] > > The above command will dump the tables for all schemas in the current database. If you want to migrate only one specific schema, you can add the `--schema=SCHEMA_NAME` option. The schema then needs to be loaded into your new PlanetScale database: ```bash theme={null} PGPASSWORD=$PLANETSCALE_PASSWORD \ psql -h $PLANETSCALE_HOST \ -p 5432 \ -U $PLANETSCALE_USERNAME \ -d $PLANETSCALE_DBNAME \ -f schema.sql ``` In the output of this command, you might see some error messages of the form: ``` psql:schema.sql:LINE: ERROR: DESCRIPTION ``` You should inspect these to see if they are of any concern. You can [reach out to our support](https://planetscale.com/contact) if you need assistance at this step. ## 5. Set up logical replication We now must create a `PUBLICATION` on Aurora that the PlanetScale database can subscribe to for data copying and replication. To create a publication for all tables in all schemas of the current database, run the following command on your Aurora database: ```sql theme={null} CREATE PUBLICATION replicate_to_planetscale FOR ALL TABLES; ``` You should see this if it created correctly: ```sql theme={null} CREATE PUBLICATION ``` > [!NOTE] > > To publish changes for only one specific schema, run the following query: > > ```sql theme={null} > SELECT 'CREATE PUBLICATION replicate_to_planetscale FOR TABLE ' || > string_agg(format('%I.%I', schemaname, tablename), ', ') || ';' > FROM pg_tables > WHERE schemaname = 'YOUR_SCHEMA_NAME'; > ``` > > This will generate a query that looks like this: > > ```sql theme={null} > CREATE PUBLICATION replicate_to_planetscale FOR TABLE > public.table_1, > public.table_2, > ... > public.table_n; > ``` > > You can then copy/paste this and execute on Aurora. This will create a publication that only publishes changes for the tables in `YOUR_SCHEMA_NAME` After creating the publication on Aurora, we then need to tell PlanetScale to `SUBSCRIBE` to this publication. ```sql theme={null} PGPASSWORD=$PLANETSCALE_PASSWORD psql \ -h $PLANETSCALE_HOST \ -U $PLANETSCALE_USERNAME \ -p 5432 $PLANETSCALE_DBNAME \ -c " CREATE SUBSCRIPTION replicate_from_aurora CONNECTION 'host=$AURORA_HOST dbname=$AURORA_DBNAME user=$AURORA_USERNAME password=$AURORA_PASSWORD' PUBLICATION replicate_to_planetscale WITH (copy_data = true);" ``` Data copying and replication will begin at this point. To check in on the row counts for the tables, you can run a query like this on your source and target databases: ```sql theme={null} SELECT table_name, row_count FROM ( SELECT 'table_name_1' as table_name, COUNT(*) as row_count FROM table_name_1 UNION ALL SELECT 'table_name_2', COUNT(*) FROM table_name_2 UNION ALL ... SELECT 'table_name_N', COUNT(*) FROM table_name_N ) t ORDER BY table_name; ``` When the row counts match (or nearly match) you can begin testing and prepare for your application to cutover to use PlanetScale. ## 6. Handling sequences Logical replication is great at migrating all of your data over to PlanetScale. However, logical replication does *not* synchronize the `nextval` values for [sequences](https://www.postgresql.org/docs/current/sql-createsequence.html) in your database. Sequences are often used for things like auto incrementing IDs, so it's important to ensure we update this before you switch your traffic to PlanetScale. You can see all of the sequences and their corresponding `nextval`s on your source Aurora database using this command: ```sql theme={null} SELECT schemaname, sequencename, last_value + increment_by AS next_value FROM pg_sequences; ``` An example output from this command: ```sql theme={null} schemaname | sequencename | next_value ------------+------------------+------------ public | users_id_seq | 105 public | posts_id_seq | 1417 public | followers_id_seq | 3014 ``` What this means is that we have three sequences in our database. In this case, they are all being used for auto-incrementing primary keys. The `nextval` for the `users_id_seq` is 105, the `nextval` for the `posts_id_seq` is 1417, and the `nextval` for the `followers_id_seq` is 3014. If you run the same query on your new PlanetScale database, you'll see something like: ```sql theme={null} schemaname | sequencename | next_value ------------+------------------+------------ public | users_id_seq | 0 public | posts_id_seq | 0 public | followers_id_seq | 0 ``` If you switch traffic over to PlanetScale in this state, you'll likely encounter errors when inserting new rows: ```sql theme={null} ERROR: duplicate key value violates unique constraint "XXXX" DETAIL: Key (id)=(ZZZZ) already exists. ``` Before switching over, you need to progress all of these sequences forward so that the `nextval`s produced will be greater than any of the values previously produced on the source Aurora database, avoiding constraint violations. There are several approaches you can take for this. A simple way to solve the problem is to first run this query on your source Aurora database: ```sql theme={null} SELECT 'SELECT setval(''' || schemaname || '.' || sequencename || ''', ' || (last_value + 10000) || ');' AS query FROM pg_sequences; ``` This will generate a sequence of queries that will advance the `nextval` by 10,000 for each sequence: ```sql theme={null} query -------------------------------------------------- SELECT setval('public.users_id_seq', 10104); SELECT setval('public.posts_id_seq', 11416); SELECT setval('public.followers_id_seq', 13013); ``` You would then execute these on your target PlanetScale database. You need to ensure you advance each sequence far enough forward so that the sequences in the Aurora database will not reach these `nextval`s before you switch your primary to PlanetScale. For tables that have a high insertion rate, you might need to increase this by a larger value (say, 100,000 or 1,000,000). ## 7. Cutting over to PlanetScale Before you cutover, it's good to have confidence that the replication is fully caught up between Aurora and PlanetScale. You can do this using Log Sequence Numbers (LSNs). The goal is to see these match up between the source Aurora database and the target PlanetScale database exactly. If they don't, it indicates that the PlanetScale database is not fully caught-up with the changes happening on Aurora. You can run this on Aurora to see the current LSN: ```sql theme={null} postgres=> SELECT pg_current_wal_lsn(); pg_current_wal_lsn -------------------- 0/703FE460 ``` Then on PlanetScale, you would run the following query to check for a match: ```sql theme={null} postgres=> SELECT received_lsn, latest_end_lsn FROM pg_stat_subscription WHERE subname = 'replicate_from_aurora'; received_lsn | latest_end_lsn --------------+---------------- 0/703FE460 | 0/703FE460 ``` Once you are comfortable that all your data has successfully copied over and replication is sufficiently caught up, it's time to switch to PlanetScale. In your application code, prepare the cutover by changing the database connection credentials to go to PlanetScale rather than Aurora. Then, you can deploy this new version of your application, which will begin using PlanetScale as your primary database. After doing this, new rows written to PlanetScale will not be reverse-replicated to Aurora. Thus, it's important to ensure you are fully ready for the cutover at this point. Once this is complete, PlanetScale is now your primary database! We recommend you keep your old database around for at least a few days, just in case you discover any data or schemas you forgot to copy over to PlanetScale. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Migrate from Aurora using Amazon DMS with CloudFormation Source: https://planetscale.com/docs/postgres/imports/aurora-dms This method uses Infrastructure as Code with Step Functions workflow automation for a completely managed migration experience. > [!NOTE] > > Before beginning your migration, we recommend running our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. [Amazon Database Migration Service (DMS)](https://aws.amazon.com/dms/) provides a fully automated approach to migrate your PostgreSQL database to PlanetScale Postgres. > [!NOTE] > > Want expert guidance for your migration? PlanetScale's [migration services](https://planetscale.com/migrate) are available to help you plan and execute a smooth, successful move. ## Overview This automated migration method: **Step 1** **Pre-migration schema setup** (essential for production) **Step 2** Deploys DMS infrastructure via CloudFormation template **Step 3** Configures source and target database endpoints automatically **Step 4** Creates Step Functions workflow for automated migration orchestration **Step 5** Provides built-in monitoring, notifications, and automated cleanup **Step 6** Requires minimal manual intervention - mostly console clicks > [!WARNING] > > **Critical: AWS DMS Schema Object Limitations** > > AWS DMS **only migrates table data and primary keys**. All other PostgreSQL schema objects must be handled separately: > > * Secondary indexes > * Sequences and their current values > * Views, functions, and stored procedures > * Constraints (foreign keys, unique, check) > * Triggers and custom data types > > Deploy your complete Aurora schema to PlanetScale BEFORE starting DMS migration to preserve performance and avoid application errors. > [!NOTE] > > This method requires an AWS account and will incur AWS DMS charges. The CloudFormation template includes cost optimization features. Review [AWS DMS pricing](https://aws.amazon.com/dms/pricing/) before proceeding. ## General Prerequisites Before starting the migration: * Have an active AWS user with CloudFormation, EC2, DMS, SNS, and Step Functions permissions * Source Aurora database accessible from AWS VPC * Connection details for your PlanetScale Postgres database from the console * VPC with at least 2 subnets in different Availability Zones ## Source database prerequisites The Task that AWS DMS runs will automatically perform these 7 validation checks before starting the migration. Confirm before starting this process that they will succeed. | Check Name | What It Validates | Required Action (if needed) | | :- | :- | :- | | Database Version Compatibility | Verifies your PostgreSQL version is supported by AWS DMS | Ensure you're running a supported PostgreSQL version (9.4+) | | Target Database Privileges | Confirms PlanetScale user has sufficient permissions | No action should be needed - PlanetScale credentials include required permissions | | Replication Slots Available | Checks that replication slots are available for CDC | Verify `max_replication_slots >= 1` in Aurora parameter group | | Source Database Read Privileges | Validates source user can read all tables for migration | Ensure source user has SELECT privileges on all tables to migrate | | WAL Level Configuration | Confirms WAL level is set to 'logical' for CDC replication | Set `wal_level = logical` in Aurora parameter group (requires restart) | | WAL Sender Timeout | Ensures WAL sender timeout is at least 10 seconds | Set `wal_sender_timeout >= 10000` (10 seconds) in parameter group | | Maximum WAL Senders | Verifies enough WAL sender processes for CDC | Set `max_wal_senders >= 2` in Aurora parameter group | ## Step 1: Pre-Migration Schema Setup Deploy your complete Aurora schema to PlanetScale BEFORE starting the CloudFormation stack. This ensures optimal performance and prevents application errors. ### Extract and Apply Schema **Step 1** Extract your complete schema from Aurora: ```bash theme={null} pg_dump -h aurora-cluster-endpoint.amazonaws.com -p 5432 \ -U username -d database --schema-only \ --no-owner --no-privileges -f aurora_schema.sql ``` **Step 2** Apply the schema to PlanetScale: ```bash theme={null} psql -h your-planetscale-host -p 5432 -U username -d database -f aurora_schema.sql ``` > [!NOTE] > > **Foreign Key Constraints** > > If the schema application fails due to foreign key constraint issues, you can temporarily remove them from the schema file and apply them after DMS completes the data migration. ### Verify Schema Application Quickly verify your schema was applied successfully: ```sql theme={null} -- Check that tables and sequences exist SELECT (SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'public') as tables, (SELECT COUNT(*) FROM information_schema.sequences WHERE sequence_schema = 'public') as sequences, (SELECT COUNT(*) FROM pg_indexes WHERE schemaname = 'public') as indexes; ``` ## Step 2: Check DMS IAM Roles Before deploying, check if DMS service roles already exist in your AWS account: **Step 1** Go to [IAM Console](https://console.aws.amazon.com/iam/) **Step 2** Click "Roles" in the left sidebar **Step 3** Search for these role names: * `dms-vpc-role` * `dms-cloudwatch-logs-role` - **If neither role exists**: Set `CreateDMSRoles` parameter to `true` in Step 4 - **If both roles exist**: Set `CreateDMSRoles` parameter to `false` in Step 4 - **If one role exists but not the other**: Consider manually creating the roles per guidance in the [AWS DMS documentation](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_VPC_Endpoints.html#CHAP_VPC_Endpoints.prereq) and set `CreateDMSRoles` parameter to `false` in Step 4 ## Step 3: Download CloudFormation Template Get the CloudFormation template: **Step 1** Visit: [https://github.com/planetscale/migration-scripts/tree/main/postgres-planetscale](https://github.com/planetscale/migration-scripts/tree/main/postgres-planetscale) **Step 2** Right-click on `aurora-to-ps-dms.yaml` → "Save link as" **Step 3** Save the file to your computer ## Step 4: Deploy CloudFormation Stack **Step 1** Navigate to [AWS CloudFormation Console](https://console.aws.amazon.com/cloudformation/) **Step 2** Click **"Create stack"** → **"With new resources (standard)"** **Step 3** Select **"Upload a template file"** **Step 4** Click **"Choose file"** and select the downloaded template **Step 5** Click **"Next"** ### Configure Stack Parameters **Stack name**: `postgres2planetscale` or any name you want but note that overly-long names can cause resource creation issues #### VPC Information * **VPC ID**: Select your VPC from dropdown * **Subnet IDs**: Select 2+ subnets in different AZs which are "public" subnets in that they have route tables through an Internet Gateway (IGW) and NACLs that allow outbound routing. See [Subnet types](https://docs.aws.amazon.com/vpc/latest/userguide/configure-subnets.html#subnet-types) in the AWS documentation for more information #### Source Database (Your Aurora Postgres database) * **Source Endpoint Host**: Your Aurora hostname (primary write endpoint is best, not a proxy) * **Source Port**: `5432` (or your custom port, valid range: 1024-65535) * **Source Database Name**: Your database name (1-63 characters, must start with a letter, alphanumeric and underscore only) * **Source Username**: Your database username (1-63 characters, must start with a letter, alphanumeric and underscore only) * **Source Password**: Your database password (4-128 characters, will be hidden in console) #### Target Database (PlanetScale Postgres) From your PlanetScale console connection details: * **Target Endpoint Host**: PlanetScale host (from connection details) * **Target Port**: `5432` (standard PostgreSQL port) * **Target Database Name**: PlanetScale database name (1-63 characters, must start with a letter, alphanumeric and underscore only) * **Target Username**: PlanetScale username (1-63 characters, must start with a letter, alphanumeric and underscore only) * **Target Password**: PlanetScale password (4-128 characters, will be hidden in console) #### Additional Features * **DMS Instance Class**: `dms.c6i.xlarge` (template default, can select from dropdown) * Options include: dms.t3.micro, dms.t3.small, dms.t3.medium, dms.t3.large, dms.c6i.large, dms.c6i.xlarge, dms.c6i.2xlarge, dms.c6i.4xlarge * Recommended: dms.c6i.large or larger for production workloads * **Migration Type**: `full-load-and-cdc` (recommended) * Options: `full-load`, `cdc`, `full-load-and-cdc` * **Migration Bucket Name**: Base name for S3 bucket that stores DMS assessment reports * Must be 3-35 characters, start/end with lowercase letter or number * Can contain lowercase letters, numbers, hyphens, and periods * Region and account ID will be automatically appended to create unique bucket name * Example: `my-migration-bucket` becomes `my-migration-bucket-us-east-1-123456789012` * **Enable Automation**: `true` ⭐ **Important: This creates the Step Functions workflow** * **Create DMS Roles**: `true` or `false` (based on Step 2 findings) * **Notification Email**: Your email address for migration status updates and alerts > [!NOTE] > > **Schema-First Approach Built-In** > > The CloudFormation template is pre-configured for schema-first migrations with: > > * `TargetTablePrepMode: DO_NOTHING` (automatically set, uses your existing schema) > * Enhanced performance tuning settings > * Built-in row-level validation > * Batch processing optimizations > * Memory tuning for large datasets > > Complete Step 1 (pre-migration schema setup) before deploying this stack for optimal results. **Step 1** Click **"Next"** → **"Next"** → Check **"I acknowledge that AWS CloudFormation might create IAM resources"** **Step 2** Click **"Submit"** ## Step 5: Wait for Stack Completion **Step 1** Stay on the CloudFormation console **Step 2** Click on your stack name to view details **Step 3** Watch the **"Events"** tab for progress **Step 4** Wait for stack status to show **`CREATE_COMPLETE`** (typically 10-15 minutes) ## Step 6: Confirm your email notification subscription **Step 1** Check the inbox for the email used above **Step 2** You will get an email from `no-reply@sns.amazonaws.com` "DMS Migration Workflow Notifications" **Step 3** Click the link for **Confirm Subscription** Note that after the migration task completes and the stack is deleted, you would receive no further communications from this AWS SNS Topic, but other SNS topics may use the same address. ## Step 7: Get Workflow Input Configuration **Step 1** In your completed CloudFormation stack, click the **"Outputs"** tab **Step 2** Find the output key **`StepFunctionsPayloadTemplate`** **Step 3** **Copy the entire JSON value** (this contains the configuration for your migration) The JSON should look like the following example: ```json theme={null} { "replicationInstanceArn": "arn:aws:dms:us-east-2:1234567890:rep:YMZ2AH4YAJCRNJKOWBR7EEIRGE", "sourceEndpointArn": "arn:aws:dms:us-east-2:1234567890:endpoint:SIVNPTNFJZDCVK4ODTN6ZLONN4", "targetEndpointArn": "arn:aws:dms:us-east-2:1234567890:endpoint:MLSCVENBKVBWJKRVJ27EWB32IU", "replicationTaskArn": "arn:aws:dms:us-east-2:1234567890:task:QZCBNW565VH2JG2KE5UXX42LS4", "sourceEndpointHost": "prod-aurora-cluster.cluster-abc1234567.us-east-2.rds.amazonaws.com", "sourcePort": "5432" } ``` ## Step 8: Locate Step Functions Workflow **Step 1** While still in the **"Outputs"** tab find the key **`StepFunctionsConsoleURL`** **Step 2** Click on the URL link to open the AWS Step Functions console for the workflow created here The workflow includes these automated steps: * **Network Connectivity Check**: Tests connections to both source and target databases * **Security Group Auto-Fix**: Automatically corrects Aurora security group settings if DMS connectivity fails * **Pre-Migration Validation**: Validates database schemas, table structures, and data types with row-level validation * **Migration Task Start**: Launches optimized DMS full-load and CDC replication with performance tuning * **Progress Monitoring**: Continuously monitors migration progress with enhanced error handling and batch processing * **Built-in Optimization**: Uses tuned task settings for improved throughput and memory management ## Step 9: Start Migration Workflow **Step 1** In the Step Functions state machine, click **"Start execution"** **Step 2** **Execution name**: Leave as auto-generated **Step 3** **Input**: **Paste the JSON** you copied from CloudFormation outputs **Step 4** Click **"Start execution"** The workflow will automatically: * Test database connections * Start the DMS migration task * Monitor progress * Send notifications * Handle errors and retries ## Step 10: Monitor Migration Progress There are several tools you can use to monitor the progress and if necessary troubleshoot potential failures. ### DMS Console **Step 1** Go to [DMS Console](https://console.aws.amazon.com/dms/) **Step 2** Click **"Tasks"** **Step 3** Select the task **Identifier** for this migration **Step 4** View your task for detailed table-level progress ### Step Functions Console **Step 1** Watch the visual workflow progress in the Step Functions console **Step 2** Each step will show green (success), red (failure), or blue (in progress) **Step 3** Click on individual steps to see detailed logs ### CloudWatch Dashboard **Step 1** Navigate to [CloudWatch Console](https://console.aws.amazon.com/cloudwatch/) **Step 2** Click **"Dashboards"** → **"Automatic Dashboards"** → **"DMS-Migration-Dashboard"** **Step 3** Monitor key metrics: * Full load progress percentage * CDC latency * Error counts * Throughput ### Wait for automated emails You will receive an email once the migration has reached a 100% full load and CDC replication is ongoing. If the workflow does fail at any point, you will instead receive an email with where the failure occurred and then you can review the previously mentioned tools for more information. ## Step 11: Post-Migration Sequence Synchronization After DMS completes, sequences need their values synchronized: > [!WARNING] > > **Critical: Sequence Synchronization** > > Sequence values must be set ahead of Aurora values to prevent duplicate key errors when applications start using PlanetScale. ### Get Current Sequence Values from Aurora ```sql theme={null} -- Run on Aurora database to get all current sequence values SELECT sequence_name, last_value, 'SELECT setval(''' || sequence_name || ''', ' || (last_value + 1000) || ');' as update_command FROM information_schema.sequences WHERE sequence_schema = 'public' ORDER BY sequence_name; ``` ### Update Sequences in PlanetScale ```sql theme={null} -- For each sequence, run the update command from above -- Example commands (values set ahead of Aurora): SELECT setval('users_id_seq', 16234); -- Aurora value + 1000 SELECT setval('orders_id_seq', 99765); -- Aurora value + 1000 SELECT setval('products_id_seq', 6432); -- Aurora value + 1000 -- Verify sequence values are ahead of Aurora SELECT sequence_name, last_value FROM information_schema.sequences WHERE sequence_schema = 'public' ORDER BY sequence_name; ``` ### Apply Remaining Constraints Now apply foreign key constraints that were deferred: ```sql theme={null} -- Apply foreign key constraints \i constraints.sql -- Verify constraints were applied successfully SELECT conname, contype, conrelid::regclass AS table_name FROM pg_constraint WHERE connamespace = 'public'::regnamespace AND contype = 'f' -- foreign key constraints ORDER BY conrelid::regclass::text; ``` ## Step 12: Application Cutover When the Step Functions workflow or DMS task itself indicates migration is ready (Status is "Load completed, replication ongoing"), then you can begin your cutover process. ### Comprehensive Pre-Cutover Validation > [!WARNING] > > **Complete Validation Required** > > Validate ALL schema objects and data integrity before cutover. Missing objects will cause application failures. ```sql theme={null} -- Validate table row counts match Aurora SELECT schemaname, tablename, n_tup_ins as estimated_rows FROM pg_stat_user_tables WHERE schemaname = 'public' ORDER BY tablename; ``` ### Pre-Cutover Checklist (Automated) AWS DMS ensures: * Full load is 100% complete * CDC latency is under 5 seconds * Data validation passes * Both databases are synchronized ### Cutover Process **Step 1** **Complete sequence synchronization and constraint application** using steps above **Step 2** **Run comprehensive validation** to ensure all objects are functional **Step 3** **Put application in maintenance mode**, pause all writes from your application to Aurora **Step 4** **Wait for DMS to confirm final sync** **Step 5** **Update your application's database connection strings** to use PlanetScale details **Step 6** **Restart or redeploy your application** **Step 7** **Test critical functionality**, especially features using sequences and indexes ## Automated Cleanup (mostly) > [!WARNING] > > **Schema Objects and Cleanup** > > The CloudFormation stack cleanup **does not** affect your migrated schema objects in PlanetScale. Your indexes, sequences, and other objects remain intact. **Step 1** Go to your CloudFormation stack **Step 2** Click **"Delete"** **Step 3** Click the **"Delete"** confirmation popup > [!NOTE] > > The first time you attempt to delete the stack, the full process will fail to delete some of the resources. Minimally the S3 bucket created to store the DMS pre-migration test reports will need to be manually emptied before it can be deleted. If the Step Functions workflow had to modify Aurora security group, then the rule added needs to be deleted as well. Both of these resources could be safely left behind, however the S3 bucket's data will incur ongoing charges. **Step 4** In the CloudFormation stack **"Resources"** tab, find the resources where deletion failed **Step 5** Find the Resource named **PreMigrationAssessmentBucket** **Step 6** Click to open the link under the **Physical ID** heading for this resource This will take you to the Amazon S3 console for this bucket. The assessment-folder is a versioned object in S3, which means that directly deleting it here does not actually remove it, but instead places a deleted marker on the version. **Step 7** To fully empty the bucket you will need to navigate to the main console page. Click **"Buckets"** from the top nav, or **"General purpose buckets"** from the left nav **Step 8** Select the bucket used, it will start with the name of the CloudFormation stack **Step 9** Click **"Empty"** **Step 10** You can now re-attempt the stack deletion from the 1st step here ## Troubleshooting ### Stack Creation Issues **Permission Errors:** * Ensure your AWS user has CloudFormation, DMS, Step Functions, and IAM permissions * Check that you acknowledged IAM resource creation during stack creation **Network Issues:** * Verify your VPC allows internet access for DMS to reach PlanetScale * Check security groups allow port 5432 access * Ensure subnets are in different Availability Zones ### Step Functions Workflow Issues **Workflow Creation Fails:** * Verify you copied the complete JSON from CloudFormation outputs * Check that the Step Functions execution role exists **Migration Task Fails:** * Check Step Functions execution details for specific error messages * Verify database connection details are correct * Ensure source database has logical replication enabled ### Connection Problems **Source Database:** * Verify hostname, port, username, and password * Check that source database allows connections from DMS subnet * Ensure database has logical replication enabled (`rds.logical_replication = 1` for RDS) **Target Database (PlanetScale):** * Double-check connection details from PlanetScale console * Verify PlanetScale database is active and accessible * Test connectivity from AWS region to PlanetScale ### Schema-Related Issues **"sequence does not exist" errors after cutover:** ```sql theme={null} -- Check if sequences exist SELECT * FROM information_schema.sequences WHERE sequence_name = 'your_sequence'; -- Recreate missing sequence CREATE SEQUENCE your_sequence START WITH 1; SELECT setval('your_sequence', (SELECT MAX(id) FROM your_table)); ``` **Application slowness after migration:** * Missing indexes are the most common cause * Run `EXPLAIN ANALYZE` on slow queries to identify missing indexes * Apply indexes from your schema extraction **Foreign key constraint violations:** ```sql theme={null} -- Find constraint violations before applying constraints SELECT COUNT(*) FROM child_table c WHERE NOT EXISTS (SELECT 1 FROM parent_table p WHERE p.id = c.parent_id); ``` **Function/view dependency errors:** * Apply objects in correct order: sequences → indexes → views → functions → constraints * Check for Aurora-specific functions that may need modification for PlanetScale **Permission errors during schema application:** * Ensure PlanetScale user has CREATE privileges * Check if objects already exist and need to be dropped first ## Step Functions Workflow Benefits Using the automated Step Functions workflow provides: * **Visual Progress Tracking**: See each migration phase in real-time * **Automatic Error Handling**: Built-in retry logic and error notifications * **Audit Trail**: Complete log of migration steps and timings ## Advanced Options ### CloudFormation Template Optimizations The updated CloudFormation template includes these performance enhancements: **Task Configuration Improvements:** * `BatchApplyEnabled: true` - Improves target database write performance * `ValidationMode: ROW_LEVEL` - Built-in data validation with 10K failure tolerance * Memory tuning: 2GB total memory limit with optimized batch sizing * Enhanced CDC processing with 5-second commit timeout * Statement caching for improved query performance **Monitoring Enhancements:** * Comprehensive logging for all DMS components * CloudWatch integration for real-time metrics * Automated failure handling and notifications **Schema-First Integration:** * `TargetTablePrepMode: DO_NOTHING` preserves your pre-deployed schema * `FullLoadIgnoreConflicts: true` handles edge cases gracefully * Optimized for existing table structures and indexes ### Custom Migration Settings Modify template parameters for: * Different DMS instance sizes * Custom migration types (full-load only, CDC only) * Extended monitoring periods * Custom notification settings ### Multiple Database Migration Deploy multiple stacks with different names to migrate multiple databases in parallel. ## Migration Considerations Before migration, review: > **[PostgreSQL version compatibility](/docs/postgres/imports/postgres-imports#postgresql-version-compatibility)** > **[Extension support limitations](/docs/postgres/imports/postgres-imports#extension-support)** > **[Third-party enhancement restrictions](/docs/postgres/imports/postgres-imports#third-party-enhancements-and-tools)** **Important:** Allow additional time for post-migration schema object setup. Aurora databases with many indexes or complex constraints may require several hours for complete schema migration. ## Support and Resources > **[PlanetScale migration scripts](https://github.com/planetscale/migration-scripts)** > **[AWS Documentation](https://docs.aws.amazon.com/cloudformation/)** > **[PlanetScale Support](https://planetscale.com/contact?initial=support)** For simpler migrations, consider [pg\_dump/restore](/docs/postgres/imports/postgres-migrate-dumprestore), or [logical replication](/docs/postgres/imports/postgres-migrate-walstream) methods. **Post-Migration Success Checklist:** * All schema objects migrated and validated * Sequence values synchronized with Aurora * Application performance matches pre-migration levels * All critical application features tested * Constraints and foreign keys working correctly * No application errors in logs for 24+ hours * Query performance baseline established **Migration Timeline Expectations with Optimized Template:** * Schema setup: 30 minutes - 2 hours (depending on complexity) * DMS full load: Improved by \~25-40% due to batch processing optimizations * Small databases (under 10GB): 30 minutes - 2 hours * Medium databases (10-100GB): 2-6 hours * Large databases (100GB+): 4-12 hours * Sequence synchronization: 5-15 minutes * Validation and cutover: 30-60 minutes * Total downtime for cutover: 5-30 minutes **Performance Improvements:** * Batch apply processing reduces target database load * Enhanced memory management improves large table handling * Row-level validation catches issues early without stopping migration ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Discovery Tool Source: https://planetscale.com/docs/postgres/imports/discovery-tool Assess your PostgreSQL database and cloud infrastructure before migrating to PlanetScale Postgres. The PlanetScale Discovery Tool analyzes your existing PostgreSQL database and cloud infrastructure to help plan your migration to PlanetScale Postgres. It collects metadata about your database configuration, schema structure, performance characteristics, security settings, and cloud resources. It never reads or stores actual table data. The Discovery CLI also supports MySQL and MySQL-compatible database discovery. See the [MySQL setup guide](https://github.com/planetscale/ps-discovery/blob/main/docs/mysql.md) for MySQL, Vitess, and PlanetScale-specific details. ![Where ps-discovery looks: cloud and vendor discovery, database discovery, and workload discovery](https://mintcdn.com/planetscale-2/YxnMMno34vnTE3Ub/images/imports/discovery-tool-topology.png?fit=max\&auto=format\&n=YxnMMno34vnTE3Ub\&q=85\&s=f2efb9d5a29c396484b0c677b60e66bf) The tool produces a structured JSON report that PlanetScale uses to provide migration guidance tailored to your environment. You can upload this report to our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on your migration readiness. > [!NOTE] > > The discovery tool is open source and available on [GitHub](https://github.com/planetscale/ps-discovery). Essential setup steps are below. Advanced usage, troubleshooting, and detailed reference are available in the [full documentation](https://github.com/planetscale/ps-discovery/tree/main/docs). ## What it discovers **Database analysis:** * PostgreSQL version, configuration, and installed extensions * Schema structure, including schemas, tables, columns, indexes, constraints, and sizes * Performance statistics such as cache hit ratios, table/index usage, and active locks * Security configuration: roles, permissions, and SSL settings * Feature usage: foreign data wrappers, partitioning, PostGIS, and more **Cloud infrastructure analysis:** * Database instances, clusters, and their configurations * Azure Database for PostgreSQL Flexible Servers * Snowflake Postgres instances * Supabase, Heroku Postgres, and Neon project metadata * VPC networking, subnets, security groups, and firewall rules * Performance metrics from cloud monitoring services * High availability and replica configurations ## Installation The discovery tool requires Python 3.10 or later. **1. Download and extract** Download the latest release from [GitHub](https://github.com/planetscale/ps-discovery/releases) and extract it: ```bash theme={null} unzip ps-discovery-2.2.0.zip cd ps-discovery-2.2.0 ``` **2. Run setup** The setup script verifies your Python version, creates a virtual environment, installs dependencies, and writes a starter `config.yaml`: ```bash theme={null} chmod +x setup.sh ./setup.sh ``` **3. Configure credentials** Edit the generated configuration file to include your database and cloud provider credentials: ```bash theme={null} nano config.yaml ``` At a minimum, configure your database connection. See [Configuration](#configuration) below for the full format. For advanced installation options, see the [Discovery Tool README](https://github.com/planetscale/ps-discovery/blob/main/README.md). ## Database user setup Create a dedicated read-only user for the discovery tool. Connect to your PostgreSQL database as a superuser or privileged role and run the following: ```sql theme={null} -- Create a dedicated user for database discovery CREATE USER planetscale_discovery WITH PASSWORD 'secure_password_here'; -- Grant basic connection and usage permissions GRANT CONNECT ON DATABASE your_database TO planetscale_discovery; GRANT USAGE ON SCHEMA public TO planetscale_discovery; GRANT USAGE ON SCHEMA information_schema TO planetscale_discovery; -- Grant read access to all tables and views GRANT SELECT ON ALL TABLES IN SCHEMA public TO planetscale_discovery; GRANT SELECT ON ALL TABLES IN SCHEMA information_schema TO planetscale_discovery; GRANT SELECT ON ALL TABLES IN SCHEMA pg_catalog TO planetscale_discovery; -- Grant permissions for system catalogs and statistics GRANT SELECT ON pg_stat_database TO planetscale_discovery; GRANT SELECT ON pg_stat_user_tables TO planetscale_discovery; GRANT SELECT ON pg_stat_user_indexes TO planetscale_discovery; GRANT SELECT ON pg_stat_activity TO planetscale_discovery; GRANT SELECT ON pg_stat_replication TO planetscale_discovery; GRANT SELECT ON pg_settings TO planetscale_discovery; GRANT SELECT ON pg_database TO planetscale_discovery; GRANT SELECT ON pg_user TO planetscale_discovery; GRANT SELECT ON pg_roles TO planetscale_discovery; GRANT SELECT ON pg_user_mappings TO planetscale_discovery; -- For foreign data wrapper analysis GRANT SELECT ON pg_foreign_server TO planetscale_discovery; GRANT SELECT ON pg_foreign_data_wrapper TO planetscale_discovery; -- For advanced performance analysis (if pg_stat_statements is enabled) GRANT SELECT ON pg_stat_statements TO planetscale_discovery; -- For replication analysis GRANT SELECT ON pg_stat_wal_receiver TO planetscale_discovery; GRANT SELECT ON pg_stat_subscription TO planetscale_discovery; -- For PostgreSQL 10+ enhanced privileges (recommended) GRANT pg_read_all_stats TO planetscale_discovery; GRANT pg_read_all_settings TO planetscale_discovery; ``` > [!NOTE] > > If your database has additional schemas beyond `public`, repeat the `GRANT USAGE ON SCHEMA` and `GRANT SELECT ON ALL TABLES IN SCHEMA` statements for each schema you want analyzed. ### PostgreSQL cleanup > [!WARNING] > > After PostgreSQL discovery is complete, remove the `planetscale_discovery` user from your database. This user has read access to your schema and system catalogs and should not be left in place. ```sql theme={null} DROP USER IF EXISTS planetscale_discovery; ``` If the user owns any objects, reassign ownership first: ```sql theme={null} REASSIGN OWNED BY planetscale_discovery TO postgres; DROP OWNED BY planetscale_discovery; DROP USER planetscale_discovery; ``` ## Configuration The discovery tool uses a YAML configuration file. Running `./ps-discovery` loads `./config.yaml` by default. Configure the `database` block for database discovery and enable any cloud providers you want included: ```yaml theme={null} engine: postgres database: host: your-db-host.example.com port: 5432 database: your_database username: planetscale_discovery password: secure_password_here ssl_mode: require providers: aws: enabled: true regions: - us-east-1 gcp: enabled: false azure: enabled: false supabase: enabled: false heroku: enabled: false neon: enabled: false snowflake: enabled: false output: output_dir: ./discovery_output ``` ## Running discovery Run discovery with the generated `config.yaml`: ```bash theme={null} ./ps-discovery ``` Or point to a specific config file: ```bash theme={null} ./ps-discovery --config config.yaml ``` The tool produces a timestamped JSON file in your configured output directory, such as `planetscale_discovery_results_20260708T072229.json`. Upload this report to the [migration assessment tool](https://planetscale.com/liftoff) for instant feedback, or share it with PlanetScale for migration planning assistance. > [!NOTE] > > Once PostgreSQL discovery is complete, remember to [clean up](#postgresql-cleanup) the `planetscale_discovery` user you created on your source database. ## Cloud provider setup Each cloud provider requires specific credentials and permissions. Below is a summary of what you need for each. For detailed instructions including IAM policies and API enablement steps, see the [provider documentation](https://github.com/planetscale/ps-discovery/tree/main/docs/providers). For third-party hosted Postgres providers, the discovery tool supports Snowflake, Supabase, Heroku, and Neon. ### AWS (RDS / Aurora) The tool discovers RDS instances, Aurora clusters, VPC networking, security groups, and CloudWatch metrics. **Authentication** (choose one): * IAM instance profile (recommended when running on EC2) * Access key and secret key * IAM role assumption (for cross-account access) **Required permissions:** * RDS: `DescribeDBInstances`, `DescribeDBClusters`, `DescribeDBSubnetGroups` * EC2: `DescribeVpcs`, `DescribeSubnets`, `DescribeSecurityGroups`, `DescribeRouteTables` * CloudWatch: `GetMetricData`, `ListMetrics` **Configuration:** ```yaml theme={null} providers: aws: enabled: true regions: - us-east-1 - us-west-2 # Authentication - choose one approach: # Option 1: Use instance profile or environment variables (recommended) # Option 2: Explicit credentials access_key_id: AKIA... secret_access_key: ... ``` ### Google Cloud (Cloud SQL / AlloyDB) The tool discovers Cloud SQL instances, AlloyDB clusters, VPC networks, firewall rules, and Cloud Monitoring metrics. **Authentication** (choose one): * Application Default Credentials (recommended) * Service account key file **Required APIs** (must be enabled in your project): * Cloud SQL Admin API * Compute Engine API * Cloud Monitoring API * AlloyDB API (if using AlloyDB) **Configuration:** ```yaml theme={null} providers: gcp: enabled: true project_id: your-project-id # Optional: path to service account key credentials_file: /path/to/service-account-key.json ``` ### Azure (Database for PostgreSQL) The tool discovers Azure Database for PostgreSQL Flexible Servers, virtual networks, network security groups, and firewall rules. **Authentication** (choose one): * Service principal with the Reader role (recommended) * Azure CLI (`az login`) or a managed identity * `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_TENANT_ID`, and `AZURE_SUBSCRIPTION_ID` environment variables **Required permissions:** * Built-in Reader role at subscription or resource-group scope * Resource providers registered on the subscription: `Microsoft.DBforPostgreSQL` and `Microsoft.Network` **Configuration:** ```yaml theme={null} providers: azure: enabled: true subscription_id: 22222222-2222-2222-2222-222222222222 credentials: tenant_id: 11111111-1111-1111-1111-111111111111 client_id: 00000000-0000-0000-0000-000000000000 client_secret: your-client-secret discover_all: true ``` You can also sign in with `az login` and omit the `credentials` block, or focus discovery on specific servers: ```yaml theme={null} providers: azure: enabled: true subscription_id: 22222222-2222-2222-2222-222222222222 discover_all: false resources: postgresql_flexible_servers: - pg-prod-01 regions: - eastus ``` When you're ready to migrate from Azure Database for PostgreSQL, see the [pgcopydb migration guide](/docs/postgres/imports/postgres-migrate-pgcopydb) — it includes an Azure Terraform template for the migration instance and Azure-specific logical replication setup. ### Supabase The tool discovers project metadata, database configuration, PgBouncer settings, and connection details. **Authentication:** * [Personal Access Token](https://supabase.com/dashboard/account/tokens) (recommended, read-only) **Configuration:** ```yaml theme={null} providers: supabase: enabled: true access_token: sbp_... ``` ### Neon The tool discovers Neon project metadata, branch topology, compute endpoints, autoscaling configuration, connection pooling, and database names. **Authentication:** * API key from Neon. See [Manage API keys](https://neon.com/docs/manage/api-keys) in the Neon docs. * Or the `NEON_API_KEY` environment variable **Configuration:** ```yaml theme={null} providers: neon: enabled: true api_key: your-neon-api-key discover_all: true ``` ### Heroku The tool discovers Heroku Postgres add-ons across all your apps, including plan details, database sizes, replica configurations, and connection pooling. **Authentication:** * API key from the [Heroku dashboard](https://dashboard.heroku.com/account) * Or a Heroku CLI authorization token **Configuration:** ```yaml theme={null} providers: heroku: enabled: true api_key: your-heroku-api-key ``` ### Snowflake The tool inventories Snowflake Postgres instances with `SHOW POSTGRES INSTANCES` and `DESCRIBE POSTGRES INSTANCE`. Results land under `cloud_results.providers.snowflake`. Full setup is in the [Snowflake provider guide](https://github.com/planetscale/ps-discovery/blob/main/docs/providers/snowflake.md). **Authentication** (choose one): These are Snowflake account credentials, not the Postgres instance password. * Key pair on a `TYPE = SERVICE` user (recommended) * SSO (`authentication: sso`) for an interactive one-off run * Password via the `SNOWFLAKE_PASSWORD` environment variable if the account still allows it. Do not put the password in the config file. Snowflake is retiring password-only sign-in. **Required permissions:** * `OPERATE` on each Snowflake Postgres instance, including read replicas. Replicas do not inherit grants from their primary. > [!NOTE] > > `OPERATE` can also suspend, resume, and change Postgres settings. Grant it to a dedicated `PS_DISCOVERY` role, run discovery, then drop that role and user. **Configuration:** ```yaml theme={null} providers: snowflake: enabled: true account: your-account-locator user: PS_DISCOVERY_SVC role: PS_DISCOVERY authentication: key_pair private_key_path: /path/to/rsa_key.p8 discover_all: true ``` Do not enable `providers.aws` or `providers.gcp` for Snowflake-hosted hostnames. Those tokens in the hostname are location labels. The inventory provider is Snowflake. You can also focus inventory on specific instances: ```yaml theme={null} providers: snowflake: enabled: true account: your-account-locator user: PS_DISCOVERY_SVC role: PS_DISCOVERY authentication: key_pair private_key_path: /path/to/rsa_key.p8 discover_all: false resources: postgres_instances: - your_primary_instance - your_replica_instance ``` ## Performance and safety The default database analyzers are safe to run against production databases. They use read-only queries against system catalogs and statistics views, with very low performance impact. > [!WARNING] > > The optional **data size analyzer** performs sampling queries against actual tables and can have a significant performance impact on large databases. If you need this analysis, consider running it against a read replica and starting with a low sampling percentage. This analyzer is disabled by default and must be explicitly opted into via configuration. ## Privacy and security The discovery tool runs entirely on your infrastructure. No data is sent to external services during analysis. **Collected:** Schema metadata, database configuration, usage statistics, infrastructure topology, and role names. **Not collected:** Table contents, row data, application queries, passwords, or secrets. Passwords are used only to establish the database connection and are never included in the output. ## Get an instant migration assessment Once you have your discovery report, upload it to our [migration assessment tool](https://planetscale.com/liftoff) for immediate feedback. The tool analyzes your report and provides: * An overview of your existing database and optionally hosting provider * Extension compatibility issues, schema defects, and any migration blockers * A recommended migration path with main steps involved and the estimated duration All analysis happens client-side in your browser — your discovery data is never sent to a server. > [!NOTE] > > Liftoff needs both the database and cloud sections. A cloud-only file cannot be reviewed. If you will upload the output to Liftoff, put a `database` block and an enabled cloud provider in the same config, or run the [`both`](https://github.com/planetscale/ps-discovery/blob/main/docs/advanced-usage.md#selecting-what-runs) subcommand. ## Next steps If you want tailored guidance from our migration team, [share your discovery report with us](https://planetscale.com/contact). You can also follow one of our migration guides on your own: > **[Migrate using pgdump/restore](/docs/postgres/imports/postgres-migrate-dumprestore)** > **[Migrate using WAL streaming](/docs/postgres/imports/postgres-migrate-walstream)** > **[Migrate using Amazon DMS](/docs/postgres/imports/postgres-migrate-dms)** > **[Migrate from Heroku](/docs/postgres/imports/heroku)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Migrate from Heroku to PlanetScale Source: https://planetscale.com/docs/postgres/imports/heroku Migrate from Heroku Postgres to PlanetScale with minimal downtime using the Heroku Migrator tool. The [PlanetScale Heroku Migrator](https://github.com/planetscale/heroku-migrator) helps you migrate your Heroku Postgres database to PlanetScale with minimal downtime. It runs as a temporary Heroku app that copies your data and keeps both databases in sync until you're ready to cut over. The entire process is managed through a web dashboard. Under the hood, the migrator uses [Bucardo](https://bucardo.org/Bucardo/), an open-source trigger-based replication tool. Heroku does not support logical replication, so trigger-based replication is the most reliable migration strategy with the least downtime. Before beginning your migration, we recommend running our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. We also offer complimentary hands-on migration assistance for Heroku migrations on a case-by-case basis. [Reach out](https://planetscale.com/contact) to learn more. [Automate your Heroku database migration, fast!](https://www.youtube-nocookie.com/embed/92VnUY9_Hpk?rel=0) > [!WARNING] > > **Databases over \~1 TB:** Heroku force-restarts dynos every 24 hours, which resets the initial data copy. If your database is large enough that the copy could take longer than 24 hours, do not deploy this tool on Heroku. On AWS, use the [CloudFormation template](https://github.com/planetscale/heroku-migrator/tree/main/templates/aws-cloudformation). See [Heroku's 24-hour restart limit](#7-heroku-24-hour-restart-limit) for details. ## Before you start > [!WARNING] > > **Do not make schema changes during the migration.** Bucardo discovers your tables, columns, and constraints when replication is configured and installs triggers based on that snapshot. Running any DDL on Heroku after the migration starts will break replication. If you need to ship a schema change, finish or abort the migration first. ### 1. Get your Heroku database credentials You'll need your Heroku Postgres connection URL. Run this command to get it: ```sh theme={null} heroku config:get DATABASE_URL -a your-app-name ``` It will look something like: ``` postgres://username:password@host:5432/dbname ``` Copy this value. You'll paste it as the `HEROKU_URL` when deploying the migrator. ### 2. Create your PlanetScale database and get credentials Follow the [PlanetScale Postgres quickstart](/docs/postgres/tutorials/planetscale-postgres-quickstart) to create a database and generate a password. When creating the password, make sure you select the **Postgres** permission. Copy the Postgres connection string. This is your `PLANETSCALE_URL`. ### 3. Check your Heroku Postgres extensions The migrator replicates your data, but it doesn't install Postgres extensions. You need to make sure any extensions you use on Heroku are also enabled on PlanetScale **before** starting the migration. Run this command to see which extensions your Heroku database uses: ```sh theme={null} heroku pg:psql -a your-app-name -c "SELECT extname, extversion FROM pg_extension WHERE extname != 'plpgsql' ORDER BY extname;" ``` For each extension listed, enable it on your PlanetScale database before starting the migration. See the [PlanetScale Postgres extensions documentation](/docs/postgres/extensions) for supported extensions and how to enable them. If `pg_partman` is in that list, read [pg\_partman support](#pg_partman-support) before you start. It needs one manual step. ### 4. Check for blocking vacuum processes The migrator creates triggers on your Heroku tables to track changes. In rare cases, a long-running autovacuum process can block trigger creation, which can also block your application's queries. Before starting the migration, check for wraparound vacuum processes: ```sh theme={null} heroku pg:locks -a your-app-name ``` If you see any `VACUUM` queries with `(to prevent wraparound)` in the output, wait for them to finish before starting the migration. ### 5. Size your PlanetScale database **Cluster size:** Choose a PlanetScale cluster with similar CPU and RAM to your Heroku Postgres plan. You don't need to get this exactly right. [Resizing in PlanetScale is an online operation](/docs/postgres/cluster-configuration) with no downtime, and you are only billed for the time you use. **Storage:** Make sure your PlanetScale database has at least **twice the storage** that Heroku reports using. Bucardo is not very space-efficient during migration, and Postgres disk usage can vary significantly between providers. It's not uncommon for a database to use 50% more or less space on PlanetScale than on Heroku. Automatic vacuuming will reclaim the extra space over time after the migration completes. To check your current Heroku storage usage: ```sh theme={null} heroku pg:info -a your-app-name ``` Look for the "Data Size" field. If your Heroku database uses 10 GB, provision at least 20 GB on PlanetScale. You can adjust storage at any time via the [Storage tab in Cluster Configuration](/docs/postgres/cluster-configuration/cluster-storage). ### 6. Consider your database size and performance The migrator uses trigger-based replication. When the migration starts, it installs triggers on your Heroku database tables to track every insert, update, and delete so changes can be replicated to PlanetScale. * **Small to medium databases** (under 10 million rows): You likely won't notice any performance impact. * **Large databases** (tens of millions to billions of rows): The initial data copy puts additional read load on your Heroku database. The triggers add a small amount of write overhead. If your database is already under heavy load, consider: * **Upgrade your Heroku Postgres plan temporarily.** A larger plan gives your database more headroom during the migration. You can downgrade after. * **Run the migration during off-peak hours.** Start the initial copy when your app has less traffic. * **Use the Pause button, but be aware of the phase.** Once the dashboard shows "Your databases are in sync", **Pause Sync** stops data transfer to PlanetScale while changes continue to queue. Resuming catches up cleanly. Do not pause while the dashboard still says "Copying data to PlanetScale". Bucardo cannot resume a partial `COPY`, so resuming restarts the initial copy from scratch. ### 7. Heroku 24-hour restart limit Heroku restarts every dyno at least once every 24 hours. If a restart happens during the initial data copy, the copy starts over from the beginning. Copy speed varies by database, but most users can expect around **200 GB per hour**. Actual throughput depends on the number and size of indexes, average row width, network conditions between Heroku and PlanetScale, and the configuration of the target database. > [!WARNING] > > If your Heroku database is large enough that the initial copy could take close to or longer than 24 hours, **do not deploy this tool on Heroku**. Run it on infrastructure that will not force-restart the container. **On AWS, use the CloudFormation template.** That is the default way to run the migrator outside Heroku. The [aws-cloudformation templates](https://github.com/planetscale/heroku-migrator/tree/main/templates/aws-cloudformation) provision an EC2 instance, build the container, and run it under systemd with no 24-hour restart. The dashboard binds to localhost only, so you reach it through an SSH or SSM tunnel. On other hosts, build and run the container yourself with the same environment variables you would set on Heroku, including index-rebuild tuning: ```sh theme={null} docker build -t heroku-migrator . docker run -d \ -e HEROKU_URL="postgres://..." \ -e PLANETSCALE_URL="postgresql://..." \ -e PASSWORD="your-password" \ -e INDEX_REBUILD_WORKERS=4 \ -e MAINTENANCE_WORK_MEM=1GB \ -e PARALLEL_MAINTENANCE_WORKERS=2 \ -p 8080:8080 \ heroku-migrator ``` For databases under \~1 TB, Heroku is typically fine. For anything larger, use a host without forced restarts to avoid re-copying data. ## Deploy the migrator Deploy the migrator as a temporary Heroku app. You can use the Deploy to Heroku button in the [heroku-migrator repository](https://github.com/planetscale/heroku-migrator), or deploy manually: **1. Clone the repository** ```sh theme={null} git clone https://github.com/planetscale/heroku-migrator.git cd heroku-migrator ``` **2. Create a Heroku app** ```sh theme={null} heroku create my-migration --stack container ``` **3. Set the required config vars** ```sh theme={null} heroku config:set \ HEROKU_URL="postgres://..." \ PLANETSCALE_URL="postgresql://..." \ PASSWORD="choose-a-password" ``` Optionally, tune the post-copy index rebuild (defaults shown; see [Faster initial copy](#faster-initial-copy-deferred-index-rebuild)). You can also change the parallel-jobs count from the dashboard before the copy starts: ```sh theme={null} heroku config:set \ INDEX_REBUILD_WORKERS=4 \ MAINTENANCE_WORK_MEM=1GB \ PARALLEL_MAINTENANCE_WORKERS=2 ``` **4. Deploy** ```sh theme={null} git push heroku main ``` **5. Resize the dyno (recommended)** ```sh theme={null} heroku ps:resize web=standard-2x -a my-migration ``` **6. Open the dashboard** ```sh theme={null} heroku open ``` You'll be prompted for a password. Enter the `PASSWORD` you set above. The username is `admin`. ### Which dyno size should I use? The migrator runs PostgreSQL and Bucardo inside the dyno, so it needs more memory than a typical web app. Memory usage scales with both **data size** and **write volume**. * **Standard-1x (512 MB)**: Small databases with low write volume (under 1 million rows, fewer than 20 tables). * **Standard-2x (1 GB)**: Databases under 50 GB with moderate write volume. * **Performance-M (2.5 GB)**: Databases between 50-100 GB, high write throughput, or many tables. Also use this if you see R14 memory errors on Standard-2x. * **Performance-L (14 GB)**: Databases over 100 GB, or any database with very high write volume. The initial copy and ongoing replication for large databases can consume several GB of RAM. Smaller dynos will OOM and crash. ## How the migration works Once you open the dashboard and click **Start Migration**, the process follows these steps: ### Step 1: Setup The migrator copies your database structure (tables, indexes, constraints) from Heroku to PlanetScale and configures replication. This is fully automatic and typically takes a minute or two. It then stops and waits for you: no data is copied until you start the next step. Bookkeeping schemas (`bucardo`, `_ps_migrator`, `pscale_extensions`), extension home schemas, extension-owned tables, and tables the migrator cannot put a trigger on are excluded from the dump and the sync. ### Step 2: Data sync Click **Start Data Copy** to begin. All existing rows are copied from Heroku to PlanetScale (the "initial copy"). By default, secondary and unique indexes are dropped on PlanetScale before that copy and rebuilt afterward. See [Faster initial copy](#faster-initial-copy-deferred-index-rebuild). Once the copy and index rebuild finish, the migrator enters real-time replication mode, where every new write to your Heroku database is automatically replicated to PlanetScale. Your Heroku app continues running normally throughout this entire process. For large databases, the initial copy can take hours. The dashboard shows progress and you can safely close the browser and come back later. The **Pause Sync** button behaves differently depending on the phase: * **While the initial copy is running** (dashboard title "Copying data to PlanetScale"): Bucardo cannot resume a partial `COPY`. Pausing and then resuming will restart the initial copy from scratch. * **Once you reach delta replication** (dashboard title "Your databases are in sync"): pausing is safe. Changes are queued while paused and apply on Resume. In either case, Bucardo's triggers stay on your Heroku database while paused, so writes still carry trigger overhead. Use **Abort Migration** if you need to remove the triggers entirely. ### Step 3: Switch traffic When the dashboard shows your databases are in sync, you're ready to cut over. Click **Switch Traffic** to block writes on your Heroku database. This runs a SQL `REVOKE` command that removes `INSERT`, `UPDATE`, and `DELETE` privileges from your Heroku database user, once for each schema being migrated: ```sql theme={null} REVOKE INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA your_schema FROM your_heroku_user; ``` > [!NOTE] > > For production apps, consider enabling Heroku maintenance mode before clicking **Switch Traffic**. The migrator blocks writes on the source database, but it does not update your application's config vars, restart dynos, or verify the app against PlanetScale. Maintenance mode keeps user traffic out of the handoff window while you update and test the app. After this, your app can still **read** from Heroku, but any **write** query will fail with a permission error. This ensures no new data is written to Heroku while you switch over. **Verify the migration.** Now that writes are blocked, the source is frozen and the two databases can fully match. Click **Verify Migration** in the dashboard to run an automated source-vs-target comparison. It usually finishes in under a minute and checks: * **Tables, columns, indexes, constraints, and sequences** all present and matching on PlanetScale. * **Extensions** present and the same version. * **Row counts** on a random sample of up to 10 tables (skipping any over 10 GB, with a 60s-per-table cap). * **Pending replication** — every queued change must have been applied to PlanetScale. If you run this right after Switch Traffic, it often still reports a backlog, which fails the check. Wait a minute and re-run. The check does not modify either database, so you can re-run it as many times as you like. Then update your app to use PlanetScale: ```sh theme={null} heroku config:set DATABASE_URL="your-planetscale-connection-string" -a your-app-name ``` > [!WARNING] > > If your app has a Heroku Postgres add-on attached, `DATABASE_URL` may be owned by the add-on attachment rather than being a normal config var. In that case, `heroku config:set DATABASE_URL=...` can fail with `Cannot overwrite attachment values`. Rehearse this before you click **Switch Traffic**. Your app restarts and begins using PlanetScale. Test it to make sure everything works. If something goes wrong, click **Revert Switch** in the dashboard. This runs the inverse `GRANT` command to restore write access to your Heroku database, so your app can write to Heroku again immediately. > [!NOTE] > > During **Switch Traffic** or **Revert Switch**, you may see PostgreSQL warnings about `pg_stat_statements` privileges in the dashboard output. These warnings are expected on some Heroku Postgres setups and do not indicate a failure. ### Step 4: Complete Once you've verified everything is working on PlanetScale, click **Complete Migration** in the dashboard. This removes the replication triggers from your Heroku database. ## Faster initial copy (deferred index rebuild) You have two options for how indexes are handled during the initial copy. ### Option A: Deferred index rebuild (default) This is the default behavior (`DISABLE_INDEX_DEFERRAL` is `false`). To speed up the initial copy, the migrator drops the target's **secondary and unique indexes before copying** and rebuilds them **after** the copy completes. How it works: 1. After the schema is copied, the migrator records the exact definition of every secondary and unique index (and any foreign keys that depend on a dropped unique) in a `_ps_migrator` schema on the target, then drops them. **Primary keys, exclusion constraints, and `REPLICA IDENTITY USING INDEX` indexes are kept.** 2. The initial copy runs against the index-light tables. 3. When the copy finishes the indexes are rebuilt in parallel (`INDEX_REBUILD_WORKERS` at a time). Progress and any failures are shown on the dashboard. 4. Once each index is rebuilt, replication resumes automatically. If any index fails, the run **pauses for review** so you can fix the issue and **Retry Failed Indexes**, or **Proceed Anyway** (replication resumes without the failed indexes). The parallelism, per-job memory, and parallel-maintenance-workers of this rebuild are tunable with `INDEX_REBUILD_WORKERS`, `MAINTENANCE_WORK_MEM`, and `PARALLEL_MAINTENANCE_WORKERS`. Peak memory on the target is roughly `INDEX_REBUILD_WORKERS × MAINTENANCE_WORK_MEM`. You can also change `INDEX_REBUILD_WORKERS` from the dashboard before starting the copy. ### Option B: Keep all indexes in place If you'd rather not drop and rebuild anything, set `DISABLE_INDEX_DEFERRAL=true`. All indexes stay in place for the entire initial copy. The trade-off is a **slower initial copy**, because every index is maintained row-by-row as data is copied. When `DISABLE_INDEX_DEFERRAL=true`, there is no post-copy rebuild, so `INDEX_REBUILD_WORKERS`, `MAINTENANCE_WORK_MEM`, and `PARALLEL_MAINTENANCE_WORKERS` have no effect and the dashboard hides the index-rebuild tuning control. ## pg\_partman support pg\_partman is supported, with one manual step: pause pg\_partman maintenance before you start, and resume it on PlanetScale after cutover. Everything else is automated. The migrator replicates your **leaf partitions** and recreates the partitioned parent through the schema copy. The `partman` schema is not replicated, because its config and template tables can't be. > [!WARNING] > > Pause the job that calls `partman.run_maintenance_proc()` (usually a pg\_cron entry) and leave it paused until after cutover. Bucardo's relation list is fixed when the sync is created. A partition that maintenance creates after that point is **never replicated, and nothing reports an error**. With pg\_partman's default `automatic_maintenance = 'on'`, this can happen unattended. Right after the data copy, the migrator reads your partition configuration from the source and recreates it on PlanetScale, so maintenance keeps working after cutover. Two cases still need your attention, both reported in the Setup Log: * **Sub-partitioned sets.** pg\_partman's own dump function covers single-level sets only, so if you use sub-partitioning (`partman.part_config_sub`) re-register those by hand after cutover. * **A partition set that failed to register on PlanetScale.** This shows as a warning and does not stop the migration; all your data is still copied. Only pg\_partman's settings for that table are missing on PlanetScale, so no new partitions will be created for it until you register it again with `partman.create_parent()` after cutover, using the same settings as on Heroku. Then resume maintenance on PlanetScale. ## Cleanup After completing the migration: 1. Delete the migration app: `heroku apps:destroy my-migration` 2. Remove the Heroku Postgres add-on from your main app when you're confident everything is working. To verify that all replication objects were removed from Heroku, you can run: ```sql theme={null} SELECT count(*) FROM pg_trigger WHERE tgname LIKE 'bucardo_%'; SELECT count(*) FROM pg_namespace WHERE nspname = 'bucardo'; ``` Both queries should return `0`. ## Running another migration If you want to run another migration test after finishing one: 1. Create a fresh PlanetScale target (database/branch and credentials) and use that as the new `PLANETSCALE_URL`. 2. Keep or reset `HEROKU_URL` depending on your source test dataset. 3. Update the migrator app config vars: ```sh theme={null} heroku config:set \ HEROKU_URL="postgres://..." \ PLANETSCALE_URL="postgresql://..." \ PASSWORD="your-password" \ -a my-migration ``` 4. Open the dashboard and start a new run. Using a fresh PlanetScale target for each rerun keeps validation clean and avoids mixing data from previous migration attempts. ## Environment variables | Variable | Required | Description | | :- | :- | :- | | `HEROKU_URL` | Yes | Heroku Postgres connection URL | | `PLANETSCALE_URL` | Yes | PlanetScale Postgres connection URL | | `PASSWORD` | Yes | Password to access the migration dashboard | | `DISABLE_NOTIFICATIONS` | No | Set to `true` to disable migration progress notifications to PlanetScale (enabled by default) | | `INDEX_REBUILD_WORKERS` | No | How many indexes to rebuild in parallel after the initial copy (default `4`). See [Faster initial copy](#faster-initial-copy-deferred-index-rebuild). | | `MAINTENANCE_WORK_MEM` | No | `maintenance_work_mem` applied to each index build, e.g. `1GB` (default `1GB`). Allocated per concurrent build. | | `PARALLEL_MAINTENANCE_WORKERS` | No | `max_parallel_maintenance_workers` per index build (default `2`). | | `DISABLE_INDEX_DEFERRAL` | No | Set to `true` to keep all indexes in place during the initial copy (disables the speedup; default `false`). When `true`, the three rebuild variables have no effect. | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Import public IP addresses Source: https://planetscale.com/docs/postgres/imports/import-ips When importing a database, you may have IP restrictions in place on your source database and may need to grant a set of IP addresses access to your Postgres database so that PlanetScale can make the connection. If you have IP restrictions on your source database, you'll need to allowlist the PlanetScale IP addresses so we can connect during the import process. The IPs you need depend on the region of your PlanetScale database. ## Retrieve IP addresses via API You can get the list of IP addresses for your database's region by visiting the following URL in your browser while logged into PlanetScale: ``` https://api.planetscale.com/v1/organizations//databases/ ``` Replace `` with your organization name and `` with your database name. The response includes a `region` object with a `public_ip_addresses` array containing all the IPs you need to allowlist: ```json theme={null} { "id": "rr0qdj0nin8q", "type": "Database", "region": { "id": "ri0pbcmdkjsh", "type": "Region", "provider": "AWS", "enabled": true, "public_ip_addresses": [ "18.117.23.127", "3.131.243.164", "3.132.168.252", "3.131.252.213", "3.132.182.173", "3.15.49.114", "3.209.149.66", "3.215.97.46", "34.193.111.15" ], "display_name": "AWS us-east-2", "location": "Ohio", "slug": "aws-us-east-2" } } ``` Add each IP address from the `public_ip_addresses` array to your source database's allowlist before starting the import. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Migrate from Neon to PlanetScale Source: https://planetscale.com/docs/postgres/imports/neon Use this guide to migrate an existing Neon database to PlanetScale Postgres. > [!NOTE] > > Before beginning your migration, we recommend running our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. This guide will cover a no-downtime approach to migrating using Postgres logical replication. If you are willing to tolerate downtime during a maintenance window, you may also use [`pg_dump` and restore](/docs/postgres/imports/postgres-migrate-dumprestore). The `pg_dump`/restore approach is simpler, but is only for applications where downtime is acceptable. These instructions work for all versions of Postgres that support logical replication (version 10+). If you have an older version you want to bring to PlanetScale, [contact us](https://planetscale.com/contact?initial=support) for guidance. Before beginning a migration, you should check our [extensions documentation](/docs/postgres/extensions) to ensure that all of the extensions you rely on will work on PlanetScale. As an alternative to this guide, you can also try our [Postgres migration scripts](https://github.com/planetscale/migration-scripts/tree/main/postgres-direct). These allow you to automate some of the manual steps that we describe in this guide. > [!NOTE] > > Want expert guidance for your migration? PlanetScale's [migration services](https://planetscale.com/migrate) are available to help you plan and execute a smooth, successful move. ## 1. Prepare your PlanetScale database Create a new database in the [PlanetScale dashboard](https://app.planetscale.com) or using the [PlanetScale CLI](/docs/cli). A few things to check when configuring your database: * Ensure you select the correct cloud region. You typically want to use the same region that you deploy your other application infrastructure to. * This guide assumes you are migrating from a Neon database, so also choose the Postgres option in PlanetScale. * Choose the best storage option for your needs. For applications needing high-performance and low-latency I/O, use [PlanetScale Metal](/docs/metal). For applications that need more flexible storage options or smaller compute instances, choose "Elastic Block Storage" or "Persistent Disk." ![Create a new PlanetScale Postgres database](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=ed2edfd9919a5be57329cb6fba2794cb) Once the database is created and ready, navigate to your dashboard and click the "Connect" button. ![Connect to a PlanetScale Postgres database](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image2.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=d4e761af35a5af36b2e09519bdea8857) From here, follow the instructions to create a new default role. This role will act as your admin role, with the highest level of privileges. Though you may use this one for your migration, we recommend you use a separate role with lesser privileges for your migration and general database connections. To create a new role, navigate to the [Role management page](/docs/postgres/connecting/roles) in your database settings. Click "New role" and give the role a memorable name. By default, `pg_read_all_data` and `pg_write_all_data` are enabled. In addition to these, enable `pg_create_subscription` and `postgres`, and then create the role. ![New Postgres role privileges](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image3.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=0bd2fc3186412fbe6b82837dae75a6ee) Copy the password and all other connection credentials into environment variables for later use: ```bash theme={null} PLANETSCALE_USERNAME=pscale_api_XXXXXXXXXX.XXXXXXXXXX PLANETSCALE_PASSWORD=pscale_pw_XXXXXXXXXXXXXXXXXXXXXXX PLANETSCALE_HOST=XXXX.pg.psdb.cloud PLANETSCALE_DBNAME=postgres ``` We also recommend that you increase `max_worker_processes` for the duration of the migration, in order to speed up data copying. Go to the "Parameters" tab of the "Clusters" page: ![Configure parameters](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image4.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=06e3429bbf27c259b8ed232c02a1ca30) On this page, increase this value from the default of `4` to `10` or more: ![Configure max worker processes](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/neon/image5.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=b1834c66b40c0a4ff56d7e1540d4474a) You can decrease these values after the migration is complete. ## 2. Configure disk size on PlanetScale If you are importing into a database backed by network-attached storage, you must configure your disk in advance to ensure your database will fit. Though we support disk autoscaling for these, AWS and GCP limit how frequently disks can be resized. If you don't ensure your disk is large enough for the import in advance, it will not be able to resize fast enough for a large data import. To configure this, navigate to "Clusters" and then the "Storage" tab: ![Storage configuration min size](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/imports/storage-configuration-min-size.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=41d36935a4bf7fc12799794d0cde1a51) On this page, adjust the "Minimum disk size." You should set this value to at least 150% of the size of the database you are migrating. For example, if the database you are importing is 330 GB, you should set your minimum disk size to at least 500 GB. The 50% overhead is to account for: 1. Data growth during the import process and 2. Table and index bloat that can occur during the import process. This can be later mitigated with careful [VACUUMing](https://www.postgresql.org/docs/current/sql-vacuum.html) or using an extension like [pg\_squeeze](/docs/postgres/extensions/pg_squeeze), but is difficult to avoid during the migration itself. When ready, queue and apply the changes. You can check the "Changes" tab to see the status of the resize: ![Confirm disk size change](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/imports/confirm-disk-size-change.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=3642cee5a9eddb534263ecd8a70d81eb) Wait for it to indicate completion. If you are importing to a Metal database, you must choose a disk size when first creating your database. You should launch your cluster with a disk size at least 50% larger than the storage used by your current source database (150% of the existing total). As an example, if you need to import a 330 GB database onto a PlanetScale `M-160` there are three storage sizes available: ![Metal disk size](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/imports/metal-disk-size.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=18c7b2f2b9dbdfe9d20b22fbd3e634c7) You should use the largest, 1.25TB option during the import. After importing and cleaning up table bloat, you may be able to downsize to the 468 GB option. Resizing is a no-downtime operation that can be performed on the [clusters](/docs/postgres/cluster-configuration) page. ## 3. Prepare the Neon database You will need to enable logical replication to use this import guide. To do so, go to the settings page in your Neon project: ![Neon dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/imports/neon-dashboard-darkmode.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=09bc40b0c2a54559d4a1fd66bb98c52e) Navigate to the "Logical Replication" section and ensure it is enabled. > [!WARNING] > > Note the warnings in the Neon dashboard. Changing this setting will sever all existing connections and restart all computes. ![Neon logical replication](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/imports/neon-logical-replication-darkmode.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=00a04d8ff0031bc4d957a7c265577064) You can confirm that the `wal_level` is set to `logical` by running `SHOW wal_level;` on your Neon database: ```sql theme={null} neondb=> SHOW wal_level; wal_level ----------- logical ``` If you see a result other than `logical`, then it is not configured correctly. Once this is enabled, return to the dashboard. For these instructions, you'll need to connect to Neon with a role that has permissions to create replication publications and read all data. Your default role that was generated by Neon when you first created your database should suffice here. To get these connection credentials to use for the migration, click on the "Connect" button from your project dashboard: ![Neon dashboard connect](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/imports/neon-dashboard-connect-darkmode.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=f007391c166d5124ca9cf75bee100cdb) In the connection modal that appears, ensure you have "connection pooling" disabled. ![Neon connect](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/imports/neon-connect-darkmode.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=4c4f5ede484b1cbad3820aa8944ecd94) The credentials displayed here are the ones you can use for the migration. For the rest of this guide, we'll assume these are saved into the following environment variables: ```bash theme={null} NEON_USERNAME=XXXX NEON_PASSWORD=XXXX NEON_HOST=XXX NEON_DBNAME=XXX ``` ## 4. Copy schema from Neon to PlanetScale Before we begin migrating data, we first must copy the schema from Neon to PlanetScale. We do this as a distinct set of steps using `pg_dump`. > [!WARNING] > > You should not make any schema changes during the migration process. You may continue to select, insert, update, and delete data, keeping your application fully online during this process. Run the below command to take a snapshot of the schema for the `$NEON_DBNAME` database that you want to migrate: ```bash theme={null} PGPASSWORD=$NEON_PASSWORD \ pg_dump -h $NEON_HOST \ -p 5432 \ -U $NEON_USERNAME \ -d $NEON_DBNAME \ --schema-only \ --no-owner \ --no-privileges \ -f schema.sql ``` This saves the schema into a file named `schema.sql`. > [!NOTE] > > The above command will dump the tables for all schemas in the current database. If you want to migrate only one specific schema, you can add the `--schema=SCHEMA_NAME` option. The schema then needs to be loaded into your new PlanetScale database: ```bash theme={null} PGPASSWORD=$PLANETSCALE_PASSWORD \ psql -h $PLANETSCALE_HOST \ -p 5432 \ -U $PLANETSCALE_USERNAME \ -d $PLANETSCALE_DBNAME \ -f schema.sql ``` In the output of this command, you might see some error messages of the form: ``` psql:schema.sql:LINE: ERROR: DESCRIPTION ``` You should inspect these to see if they are of any concern. You can [reach out to our support](https://planetscale.com/contact) if you need assistance at this step. ## 5. Set up logical replication We now must create a `PUBLICATION` on Neon that the PlanetScale database can subscribe to for data copying and replication. To create a publication for all tables in all schemas of the current database, connect to the Neon database: ``` PGPASSWORD=$NEON_PASSWORD \ PGSSLMODE=require \ PGCHANNELBINDING=require \ psql \ -h $NEON_HOST \ -U $NEON_USERNAME \ -p 5432 \ $NEON_DBNAME ``` Then run the following command: ```sql theme={null} CREATE PUBLICATION replicate_to_planetscale FOR ALL TABLES; ``` You should see this output if it created correctly: ```sql theme={null} CREATE PUBLICATION ``` > [!NOTE] > > To publish changes for only one specific schema, run the following query: > > ```sql theme={null} > SELECT 'CREATE PUBLICATION replicate_to_planetscale FOR TABLE ' || > string_agg(format('%I.%I', schemaname, tablename), ', ') || ';' > FROM pg_tables > WHERE schemaname = 'YOUR_SCHEMA_NAME'; > ``` > > This will generate a query that looks like this: > > ```sql theme={null} > CREATE PUBLICATION replicate_to_planetscale FOR TABLE > public.table_1, > public.table_2, > ... > public.table_n; > ``` > > You can then copy/paste this and execute on Neon. This will create a publication that only publishes changes for the tables in `YOUR_SCHEMA_NAME` After creating the publication on Neon, we need to tell PlanetScale to `SUBSCRIBE` to this publication. ```sql theme={null} PGPASSWORD=$PLANETSCALE_PASSWORD psql \ -h $PLANETSCALE_HOST \ -U $PLANETSCALE_USERNAME \ -p 5432 $PLANETSCALE_DBNAME \ -c " CREATE SUBSCRIPTION replicate_from_neon CONNECTION 'host=$NEON_HOST dbname=$NEON_DBNAME user=$NEON_USERNAME password=$NEON_PASSWORD sslmode=require channel_binding=require' PUBLICATION replicate_to_planetscale WITH (copy_data = true);" ``` Data copying and replication will begin at this point. To check in on the row counts for the tables, you can run a query like this on your source and target databases: ```sql theme={null} SELECT table_name, row_count FROM ( SELECT 'table_name_1' as table_name, COUNT(*) as row_count FROM table_name_1 UNION ALL SELECT 'table_name_2', COUNT(*) FROM table_name_2 UNION ALL ... SELECT 'table_name_N', COUNT(*) FROM table_name_N ) t ORDER BY table_name; ``` When the row counts match (or nearly match) you can begin testing and prepare for your application to cutover to use PlanetScale. ## 6. Handling sequences Logical replication is great at migrating all of your data over to PlanetScale. However, logical replication does *not* synchronize the `nextval` values for [sequences](https://www.postgresql.org/docs/current/sql-createsequence.html) in your database. Sequences are often used for things like auto incrementing IDs, so it's important to ensure we update this before you switch your traffic to PlanetScale. You can see all of the sequences and their corresponding `nextval`s on your source Neon database using this command: ```sql theme={null} SELECT schemaname, sequencename, last_value + increment_by AS next_value FROM pg_sequences; ``` An example output from this command: ```sql theme={null} schemaname | sequencename | next_value ------------+------------------+------------ public | users_id_seq | 105 public | posts_id_seq | 1417 public | followers_id_seq | 3014 ``` What this means is that we have three sequences in our database. In this case, they are all being used for auto-incrementing primary keys. The `nextval` for the `users_id_seq` is 105, the `nextval` for the `posts_id_seq` is 1417, and the `nextval` for the `followers_id_seq` is 3014. If you run the same query on your new PlanetScale database, you'll see something like: ```sql theme={null} schemaname | sequencename | next_value ------------+------------------+------------ public | users_id_seq | 0 public | posts_id_seq | 0 public | followers_id_seq | 0 ``` If you switch traffic over to PlanetScale in this state, you'll likely encounter errors when inserting new rows: ```sql theme={null} ERROR: duplicate key value violates unique constraint "XXXX" DETAIL: Key (id)=(ZZZZ) already exists. ``` Before switching over, you need to progress all of these sequences forward so that the `nextval`s produced will be greater than any of the values previously produced on the source Neon database, avoiding constraint violations. There are several approaches you can take for this. A simple way to solve the problem is to first run this query on your source Neon database: ```sql theme={null} SELECT 'SELECT setval(''' || schemaname || '.' || sequencename || ''', ' || (last_value + 10000) || ');' AS query FROM pg_sequences; ``` This will generate a sequence of queries that will advance the `nextval` by 10,000 for each sequence: ```sql theme={null} query -------------------------------------------------- SELECT setval('public.users_id_seq', 10104); SELECT setval('public.posts_id_seq', 11416); SELECT setval('public.followers_id_seq', 13013); ``` You would then execute these on your target PlanetScale database. You need to ensure you advance each sequence far enough forward so that the sequences in the Neon database will not reach these `nextval`s before you switch your primary to PlanetScale. For tables that have a high insertion rate, you might need to increase this by a larger value (say, 100,000 or 1,000,000). ## 7. Cutting over to PlanetScale Before you cutover, it's good to have confidence that the replication between Neon and PlanetScale is fully caught up. You can do this using Log Sequence Numbers (LSNs). The goal is to see these match up between the source Neon database and the target PlanetScale database exactly. If they don't, it indicates that the PlanetScale database is not fully caught-up with the changes happening on Neon. You can run this on Neon to see the current LSN: ```sql theme={null} postgres=> SELECT pg_current_wal_lsn(); pg_current_wal_lsn -------------------- 0/703FE460 ``` Then on PlanetScale, you would run the following query to check for a match: ```sql theme={null} postgres=> SELECT received_lsn, latest_end_lsn FROM pg_stat_subscription WHERE subname = 'replicate_from_neon'; received_lsn | latest_end_lsn --------------+---------------- 0/703FE460 | 0/703FE460 ``` Once you are comfortable that all your data has successfully copied over and replication is sufficiently caught up, it's time to switch to PlanetScale. In your application code, prepare the cutover by changing the database connection credentials to go to PlanetScale rather than Neon. Then, you can deploy this new version of your application, which will begin using PlanetScale as your primary database. After doing this, new rows written to PlanetScale will not be reverse-replicated to Neon. Thus, it's important to ensure you are fully ready for the cutover at this point. Once this is complete, PlanetScale is now your primary database! We recommend you keep your old database around for at least a few days, just in case you discover any data or schemas you forgot to copy over to PlanetScale. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres Imports Source: https://planetscale.com/docs/postgres/imports/postgres-imports For customers looking to migrate their Postgres databases to PlanetScale Postgres, you have several options for how to make this the smoothest event for your business. Use this guide if you are importing from platforms like Aurora Postgres, RDS Postgres, Neon, Supabase, Azure Database for PostgreSQL, Cloudflare D1, and other Postgres or SQLite instances. Before beginning your migration, we recommend running our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. > [!NOTE] > > If you have IP restrictions in place on your source database and need to grant a set of IP addresses access, see our [Import public IP addresses documentation](/docs/postgres/imports/import-ips). ## Migration Options Overview PlanetScale Postgres provides several migration approaches to suit different business requirements, database sizes, and downtime tolerances: > **[Migrate using pg\_dump & pg\_restore](/docs/postgres/imports/postgres-migrate-dumprestore)** > **[Migrate using WAL streaming](/docs/postgres/imports/postgres-migrate-walstream)** > **[Migrate using pgcopydb](/docs/postgres/imports/postgres-migrate-pgcopydb)** > **[Migrate using pscale from Cloudflare D1](/docs/postgres/imports/postgres-migrate-d1)** You can also utilize our [migration scripts](https://github.com/planetscale/migration-scripts/tree/main/postgres-direct) directly if you prefer. These scripts can be used to migrate straight from any Postgres source that supports logical replication into PlanetScale Postgres. ### 1. pg\_dump & pg\_restore The [pg\_dump](https://www.postgresql.org/docs/current/app-pgdump.html) method is the simplest approach for migrating smaller PostgreSQL databases. This method involves creating a full backup of your source database using PostgreSQL's built-in `pg_dump` utility and then restoring it to your PlanetScale Postgres database. **How it works:** * Export your entire database schema and data using [pg\_dump](https://www.postgresql.org/docs/current/app-pgdump.html) * Transfer the dump file to PlanetScale Postgres * Restore the database using [pg\_restore](https://www.postgresql.org/docs/current/app-pgrestore.html) or [psql](https://www.postgresql.org/docs/current/app-psql.html) This approach is straightforward and doesn't require additional infrastructure, making it ideal for databases that can tolerate some downtime during the migration process. ### 2. WAL Log Replication [Write-Ahead Logging (WAL)](https://www.postgresql.org/docs/current/wal-intro.html) replication provides a near-zero downtime migration by continuously streaming transaction logs from your source PostgreSQL database to PlanetScale Postgres. **How it works:** * Set up [logical replication](https://www.postgresql.org/docs/current/logical-replication.html) between your source database and PlanetScale Postgres * Stream [WAL logs](https://www.postgresql.org/docs/current/wal-intro.html) in real-time to keep the target database synchronized * Perform a quick cutover when ready to switch to the new database This method is ideal for production databases that require minimal downtime and need to maintain data consistency during the migration process. ### 3. pgcopydb [pgcopydb](/docs/postgres/imports/postgres-migrate-pgcopydb) automates the `pg_dump | pg_restore` pipeline by copying table data in parallel without intermediate files and building indexes concurrently. PlanetScale maintains a fork with PostgreSQL 17 and 18 support, improved filtering, resilient retry, and improved CDC support. PlanetScale also provides [helper scripts](https://github.com/planetscale/migration-scripts/tree/main/pgcopydb-helpers) for setup, monitoring, cutover, and cleanup. **How it works:** * Copy schema from source using `pg_dump` * Copy table data in parallel using `COPY` with configurable concurrency * Build indexes and constraints concurrently on the target * Optionally stream changes via [logical decoding](https://www.postgresql.org/docs/current/logicaldecoding.html) for near-zero downtime CDC * Perform cutover when replication is caught up pgcopydb is the recommended approach for large databases requiring high-throughput parallel copying, near-zero downtime, and fine-grained control over the migration process. PlanetScale has used pgcopydb and these helper scripts to migrate multi-terabyte databases from live customer environments, with speeds as fast as 2 TB per hour using [Metal](/docs/metal). ### 4. Cloudflare D1 If you are moving from **Cloudflare D1** (SQLite) rather than Postgres, use the dedicated [D1 import guide](/docs/postgres/imports/postgres-migrate-d1). Export with Wrangler, then run `pscale import d1` to convert schema and load data into PlanetScale Postgres. ## Migration Method Comparison | Feature | pg\_dump & pg\_restore | WAL Log Replication | pgcopydb | | :- | :- | :- | :- | | **Best For** | Small databases | Production databases requiring minimal downtime | Large databases requiring fast parallel migration | | **Data Size** | \<20GB | \<150GB | >=150GB | | **Downtime** | High (up to an hour) | Minimal (minutes) | Minimal (minutes) | | **Setup Complexity** | Low | Medium | Medium | | **Infrastructure Requirements** | None (built-in tools) | Source DB configuration changes | Migration instance with network access | | **Data Consistency** | [Point-in-time snapshot](https://www.postgresql.org/docs/current/backup-dump.html) | Real-time sync | Real-time sync via CDC | | **Cost** | Free (built-in tools) | Low (minimal resources) | Low (migration instance only) | | **Schema Changes During Migration** | Not supported | Limited support | Not supported | | **Data Transformation** | None | Limited | None | | **Error Handling** | Manual intervention required | Basic retry mechanisms | Resumable with retry and exponential backoff | | **Rollback Options** | Manual restore from backup | Stop replication, switch back | Stop CDC, switch back | ## Migration Considerations Before migrating your PostgreSQL database to PlanetScale Postgres, there are several important factors to consider to ensure a smooth migration process. ### PostgreSQL Version Compatibility PlanetScale Postgres supports [PostgreSQL 18](https://www.postgresql.org/docs/18/index.html) and [PostgreSQL 17](https://www.postgresql.org/docs/17/index.html). If your source database is running an older version of PostgreSQL, you should verify compatibility and consider upgrading your source database before migration, or plan for potential compatibility issues during the migration process. For major version upgrades, use [pgcopydb](/docs/postgres/imports/postgres-migrate-pgcopydb). Only pgcopydb supports migrations across PostgreSQL major versions; pg\_dump & pg\_restore and WAL log replication are intended for same-major-version imports. PlanetScale has successfully migrated companies from PostgreSQL 13 and newer to PostgreSQL 18 with pgcopydb. **Version considerations:** * **PostgreSQL 18**: Fully supported * **PostgreSQL 17**: Fully supported * **Earlier versions**: May require additional testing and validation * **Version-specific features**: Newer features may not be available in older versions For detailed information about PostgreSQL version differences, refer to the [PostgreSQL 18 release notes](https://www.postgresql.org/docs/18/release.html) and [PostgreSQL 17 release notes](https://www.postgresql.org/docs/17/release.html). ### Upgrading from PostgreSQL 17 to 18 on PlanetScale We don't currently offer an automated in-place major version upgrade from PostgreSQL 17 to 18. You can perform an online upgrade by migrating from your existing PlanetScale Postgres 17 database to a new PostgreSQL 18 database using pgcopydb: * For major version upgrades: follow the [pgcopydb guide](/docs/postgres/imports/postgres-migrate-pgcopydb) At a high level, the process is: 1. Create a new PostgreSQL 18 database (same region and similar configuration). 2. Use pgcopydb to sync data from your PostgreSQL 17 database. 3. Validate data and application behavior, then update your application connection string to the new database. 4. Decommission the old PostgreSQL 17 database when you're ready. ### Extension Support Many PostgreSQL databases rely on extensions to provide additional functionality, but not all extensions are supported by PlanetScale Postgres. See the list of [supported extensions](/docs/postgres/extensions). **Important notes about extensions:** * Review your current database's installed extensions using `\dx` in psql or by querying `pg_extension` * Identify which extensions are critical to your application's functionality * Plan for alternative approaches if critical extensions are not supported * Test your application thoroughly in a staging environment before migrating production data Before migrating, compare your source extension versions with [PlanetScale's supported versions](/docs/postgres/extensions) and review the extension's changelog for any breaking changes. Common extensions to verify: * **[PostGIS](https://postgis.net/)** — Geospatial functions and data types may change between versions * **[Full-text search extensions](https://www.postgresql.org/docs/current/textsearch.html)** (`pg_trgm`, `unaccent`, `dict_int`) — Dictionary and tokenization behavior can vary between versions * **[UUID extensions](https://www.postgresql.org/docs/current/uuid-ossp.html)** — During replication, UUID generation can cause value misalignment if source and target use different generation methods * **[pg\_stat\_statements](https://www.postgresql.org/docs/current/pgstatstatements.html)** ### Third-Party Enhancements and Tools PlanetScale Postgres does **not support third-party enhancements** to PostgreSQL's core capabilities. This includes: **Currently unsupported:** * Custom background workers * Third-party connection poolers (like [PgBouncer](https://www.pgbouncer.org/)) * External procedural languages beyond the standard ones * Third-party monitoring tools that require database-level access * Custom shared libraries or plugins > [!NOTE] > > PlanetScale Postgres includes connection pooling by default. **Alternatives to consider:** * Migrate custom functions to standard PostgreSQL syntax where possible. * Utilize [Metrics](/docs/postgres/monitoring/metrics), [Insights](/docs/postgres/monitoring/query-insights), and 3rd party integrations for monitoring. ### Pre-Migration Checklist Before starting your migration: **Step 1** **Database Assessment** * Document your current PostgreSQL version * List all installed extensions and their usage * Identify any third-party tools or enhancements in use * Review custom functions and stored procedures **Step 2** **Compatibility Testing** * Test your application against your target PostgreSQL version (18 or 17) * Validate that critical extensions are supported or have alternatives * Identify any custom code that may need modification **Step 3** **Migration Planning** * Choose the appropriate migration method based on your requirements * Plan for testing in a staging environment * Prepare rollback procedures if needed * Schedule migration during low-traffic periods if possible For the most up-to-date information on supported features and extensions, refer to the [PostgreSQL documentation](https://www.postgresql.org/docs/current/) and PlanetScale Postgres release notes. ## Get Started Follow the migration guide that's right for you: > **[Import from Cloudflare D1](/docs/postgres/imports/postgres-migrate-d1)** > **[Migrate using pg\_dump & pg\_restore](/docs/postgres/imports/postgres-migrate-dumprestore)** > **[Migrate using WAL streaming](/docs/postgres/imports/postgres-migrate-walstream)** > **[Migrate using pgcopydb](/docs/postgres/imports/postgres-migrate-pgcopydb)** > **[Migrate from Heroku](/docs/postgres/imports/heroku)** > [!NOTE] > > Heroku does not support logical replication, so the WAL streaming approach above won't work for Heroku Postgres. Use the dedicated [Heroku migration guide](/docs/postgres/imports/heroku) instead, which provides a migration tool with a web dashboard. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres Imports - Cloudflare D1 Source: https://planetscale.com/docs/postgres/imports/postgres-migrate-d1 Migrate a Cloudflare D1 (SQLite) database to PlanetScale Postgres using wrangler and pscale import d1. Use this guide to move an existing **Cloudflare D1** database to **PlanetScale Postgres**. The PlanetScale CLI converts SQLite schema to Postgres, loads your data, and verifies the result. This is an **offline** migration: export D1, import into PlanetScale, then point your app at the new database. > [!NOTE] > > Want help planning the move? PlanetScale's [migration services](https://planetscale.com/migrate) can walk through export, import, and cutover with you. ## Overview The workflow has three parts: 1. Export your remote D1 database with **Wrangler** 2. Lint and import the SQL dump with **`pscale import d1`** 3. Verify row counts, then update your application connection string ## Prerequisites Before you start, install and configure: * **[PlanetScale CLI](/docs/cli)** — authenticated to your org (`pscale auth login`, then `pscale org set `) * **[Cloudflare Wrangler](https://developers.cloudflare.com/workers/wrangler/)** — to export the remote D1 database * **[pgloader](https://pgloader.io/)** — required for loading data (used for all import sizes) * **`psql`** — PostgreSQL client (version 10 or newer) * **`sqlite3`** *(optional)* — helps with deeper verification checks You also need: * A **PlanetScale Postgres** database with enough storage for the imported data * A **target branch** that is empty or disposable (the import replaces schema on that branch) * **Cloudflare credentials** for remote export (`CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID`), unless you already have a D1 SQL export file Check that your machine is ready: ```bash theme={null} pscale import d1 doctor ``` Fix anything marked as failed before continuing. ## Step 1: Export D1 with Wrangler Export your production D1 database to a SQL file: ```bash theme={null} wrangler d1 export --remote --output ./d1-export.sql ``` Keep this file — every later `pscale import d1` command uses `--input ./d1-export.sql`. ## Step 2: Lint the export Review schema and data compatibility issues before touching Postgres: ```bash theme={null} pscale import d1 lint --input ./d1-export.sql ``` Fix **errors** before importing. **Warnings** (for example integer booleans or text timestamps) are usually handled automatically during import. ## Step 3: Preview the import Run a dry run on the default branch to lint again, build the import plan, and get a migration ID: ```bash theme={null} pscale import d1 start my-database --input ./d1-export.sql --dbname myapp --dry-run ``` `my-database` is the PlanetScale database (the cluster). `--dbname` is the Postgres database *inside* it. If you omit `--dbname`, data lands in `postgres`. Use the same `--dbname` on `start` and `verify`. Use `--format json` if you want machine-readable output. Save the `migration_id` from the response. The command above targets the default branch (`main`). To target a different branch, include the branch name after the database name: ```bash theme={null} pscale import d1 start my-database --input ./d1-export.sql --dbname myapp --dry-run ``` ## Step 4: Import into PlanetScale Postgres Run the import against your PlanetScale database: ```bash theme={null} pscale import d1 start my-database --input ./d1-export.sql --dbname myapp --migration-id ``` The CLI converts SQLite DDL to Postgres, loads data with pgloader, and creates indexes after the load. Use `--force` to skip the confirmation prompt in scripts. ## Step 5: Verify the import Compare source and destination row counts and run spot checks: ```bash theme={null} pscale import d1 verify my-database --migration-id --input ./d1-export.sql --dbname myapp ``` Check migration progress at any time: ```bash theme={null} pscale import d1 status my-database --migration-id ``` ## Step 6: Complete and cut over When verification passes, mark the migration complete: ```bash theme={null} pscale import d1 complete my-database --migration-id ``` Update your application to use the PlanetScale Postgres connection string from the console. ## Important notes ### ORM migration tables are skipped `pscale import d1` **does not import** ORM or framework migration bookkeeping tables. These SQLite-only history tables are not valid on Postgres and are excluded from schema conversion and data load. Examples the linter detects and the importer skips: * Drizzle — `__drizzle_migrations` and related tables * Prisma — `_prisma_migrations` * Knex — `knex_migrations`, `knex_migrations_lock` * Rails — `schema_migrations`, `ar_internal_metadata` * Django — `django_migrations` * Sequelize, Flyway, Liquibase, Alembic, TypeORM, Goose — their version/history tables **After import, re-baseline your ORM on Postgres** (for example `prisma db pull` + a fresh migration, or `drizzle-kit push`). Do not expect SQLite migration history to carry over. Run `pscale import d1 lint --input ./d1-export.sql` to see which tables in your export match these patterns. ### Plan for downtime This path is a **point-in-time snapshot**. Applications should stop writing to D1 before export, or you accept that rows written after export will not be on PlanetScale. ### Storage and branch choice Size your PlanetScale database for the full imported dataset. Run the import on a branch you can replace if something goes wrong. ### Application table name collisions If you have an application table named `schema_migrations` that is **not** Rails migration metadata, lint may warn about a name collision. Rename that table in D1 before export, or confirm lint output carefully so real data is not skipped by mistake. ## Next steps > **[Postgres import overview](/docs/postgres/imports/postgres-imports)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres Imports - Amazon DMS Source: https://planetscale.com/docs/postgres/imports/postgres-migrate-dms > [!NOTE] > > Before beginning your migration, we recommend running our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. [Amazon Database Migration Service (DMS)](https://aws.amazon.com/dms/) provides a managed migration service that can handle complex database migrations with built-in monitoring, error handling, and data validation. This method is ideal for large, complex databases that require robust migration capabilities. > [!NOTE] > > Want expert guidance for your migration? PlanetScale's [migration services](https://planetscale.com/migrate) are available to help you plan and execute a smooth, successful move. ## Overview This migration method involves: **Step 1** **Pre-migration schema setup** (strongly recommended for production) **Step 2** Setting up AWS DMS replication instance and endpoints **Step 3** Configuring source and target database connections **Step 4** Creating and running migration tasks with full load and CDC **Step 5** Monitoring migration progress and performing cutover > [!WARNING] > > **Critical: AWS DMS Schema Object Limitations** > > AWS DMS **only migrates table data and primary keys**. All other PostgreSQL schema objects must be handled separately: > > * Secondary indexes > * Sequences and their current values > * Views, functions, and stored procedures > * Constraints (foreign keys, unique, check) > * Triggers and custom data types > > Deploy your complete schema to PlanetScale BEFORE starting DMS migration to preserve performance and avoid application errors. > [!NOTE] > > This method requires an AWS account and will incur AWS DMS charges. Review [AWS DMS pricing](https://aws.amazon.com/dms/pricing/) before proceeding. > [!NOTE] > > **For Aurora users**: Consider the [Aurora to PlanetScale CloudFormation & DMS tutorial](/docs/postgres/imports/aurora-dms) for a fully automated approach using CloudFormation templates and Step Functions workflows instead of manual DMS setup. ## Prerequisites Before starting the migration: * Active AWS account with appropriate DMS permissions * Source PostgreSQL database accessible from AWS (consider VPC configuration) * Connection details for your PlanetScale Postgres database from the console * Ensure the disk on your PlanetScale database has at least 150% of the capacity of your source database. If you are migrating to a PlanetScale database backed by network-attached storage, you can [resize](/docs/postgres/cluster-configuration/cluster-storage) your disk manually by setting the "Minimum disk size." If you are using Metal, you will need to select a size when first creating your database. For example, if your source database is 330GB, you should have at least 500GB of storage available on PlanetScale. * Understanding of your data transformation requirements (if any) * Network connectivity between AWS and both source and target databases ## Step 1: Pre-Migration Schema Setup Deploy your complete schema to PlanetScale BEFORE starting DMS migration. This ensures optimal performance and prevents application errors. ### Extract and Apply Schema **Step 1** Extract your complete schema from the source PostgreSQL database: ```bash theme={null} pg_dump -h your-postgres-host -p 5432 -U username -d database \ --schema-only --no-owner --no-privileges \ --exclude-table-data='*' -f schema_objects.sql ``` **Step 2** Apply the schema to PlanetScale: ```bash theme={null} psql -h your-planetscale-host -p 5432 -U username -d database -f schema_objects.sql ``` > [!NOTE] > > **Foreign Key Constraints** > > If the schema application fails due to foreign key constraint issues, you can temporarily remove them from the schema file and apply them after DMS completes the data migration. ### Verify Schema Application Quickly verify your schema was applied successfully: ```sql theme={null} -- Check that tables and sequences exist SELECT (SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'public') as tables, (SELECT COUNT(*) FROM information_schema.sequences WHERE sequence_schema = 'public') as sequences, (SELECT COUNT(*) FROM pg_indexes WHERE schemaname = 'public') as indexes; ``` ## Step 2: Set Up AWS DMS ### Create DMS Replication Instance **Step 1** Navigate to the [AWS DMS Console](https://console.aws.amazon.com/dms/) **Step 2** Click "Create replication instance" **Step 3** Configure the instance: ```yaml theme={null} Name: planetscale-postgres-migration Description: Migration to PlanetScale Postgres Instance class: dms.t3.medium (adjust based on your needs) Engine version: Latest available VPC: Select appropriate VPC Multi-AZ: No (for cost savings, Yes for production) Publicly accessible: Yes (if needed for connectivity) ``` ### Configure Security Groups Ensure your replication instance can connect to: * Source PostgreSQL database (port 5432) * PlanetScale Postgres (port 5432) * Internet for PlanetScale connectivity ## Step 3: Create Source Endpoint ### Configure PostgreSQL source endpoint: **Step 1** In DMS Console, go to "Endpoints" > "Create endpoint" **Step 2** Configure source endpoint: ```yaml theme={null} Endpoint type: Source endpoint Endpoint identifier: postgres-source Source engine: postgres Server name: your-postgres-host Port: 5432 Database name: your-database-name Username: your-username Password: your-password ``` ### Advanced settings for PostgreSQL: ```yaml theme={null} Extra connection attributes: pluginName=test_decoding; slotName=dms_slot_planetscale; captureDDLs=false; maxFileSize=32768; ``` ## Step 4: Create Target Endpoint ### Configure PlanetScale Postgres target endpoint: **Step 1** Create target endpoint with PlanetScale connection details: ```yaml theme={null} Endpoint type: Target endpoint Endpoint identifier: planetscale-target Target engine: postgres Server name: [from PlanetScale console] Port: [from PlanetScale console] Database name: [from PlanetScale console] Username: [from PlanetScale console] Password: [from PlanetScale console] ``` ### SSL Configuration: ```yaml theme={null} SSL mode: require ``` ## Step 5: Test Endpoints **Step 1** Select your source endpoint and click "Test connection" **Step 2** Select your target endpoint and click "Test connection" **Step 3** Ensure both tests pass before proceeding ## Step 6: Create Migration Task ### Configure the migration task: **Step 1** Go to "Database migration tasks" > "Create task" **Step 2** Configure task settings: ```yaml theme={null} Task identifier: postgres-to-planetscale Replication instance: planetscale-postgres-migration Source database endpoint: postgres-source Target database endpoint: planetscale-target Migration type: Migrate existing data and replicate ongoing changes ``` ### Task Settings **Option 1: Schema-first approach** (recommended for production): ```json expandable theme={null} { "TargetMetadata": { "TargetSchema": "", "SupportLobs": true, "FullLobMode": true, "LobChunkSize": 32, "LimitedSizeLobMode": false, "LobMaxSize": 0, "InlineLobMaxSize": 32, "BatchApplyEnabled": true, "TaskRecoveryTableEnabled": false }, "FullLoadSettings": { "TargetTablePrepMode": "DO_NOTHING", "CreatePkAfterFullLoad": false, "StopTaskCachedChangesApplied": false, "MaxFullLoadSubTasks": 8, "TransactionConsistencyTimeout": 600, "CommitRate": 10000, "FullLoadIgnoreConflicts": true }, "ValidationSettings": { "EnableValidation": true, "ValidationMode": "ROW_LEVEL", "ThreadCount": 5, "FailureMaxCount": 10000, "TableFailureMaxCount": 1000 }, "ChangeProcessingTuning": { "StatementCacheSize": 50, "CommitTimeout": 5, "BatchApplyPreserveTransaction": true, "BatchApplyTimeoutMin": 1, "BatchApplyTimeoutMax": 30, "MinTransactionSize": 5000, "MemoryKeepTime": 60, "BatchApplyMemoryLimit": 1000, "MemoryLimitTotal": 2048 }, "Logging": { "EnableLogging": true, "LogComponents": [ { "Id": "TRANSFORMATION", "Severity": "LOGGER_SEVERITY_DEFAULT" }, { "Id": "SOURCE_UNLOAD", "Severity": "LOGGER_SEVERITY_DEFAULT" }, { "Id": "TARGET_LOAD", "Severity": "LOGGER_SEVERITY_DEFAULT" }, { "Id": "SOURCE_CAPTURE", "Severity": "LOGGER_SEVERITY_DEFAULT" }, { "Id": "TARGET_APPLY", "Severity": "LOGGER_SEVERITY_DEFAULT" } ] }, "ControlTablesSettings": { "historyTimeslotInMinutes": 5, "ControlSchema": "", "HistoryTableEnabled": false, "SuspendedTablesTableEnabled": false, "StatusTableEnabled": false, "FullLoadExceptionTableEnabled": false } } ``` **Option 2: Standard approach** (matches CloudFormation template): ```json expandable theme={null} { "TargetMetadata": { "SupportLobs": true, "FullLobMode": true, "LobChunkSize": 32, "BatchApplyEnabled": true, "TaskRecoveryTableEnabled": false }, "FullLoadSettings": { "TargetTablePrepMode": "DROP_AND_CREATE", "CreatePkAfterFullLoad": false, "MaxFullLoadSubTasks": 8, "TransactionConsistencyTimeout": 600, "CommitRate": 10000, "FullLoadIgnoreConflicts": true }, "ValidationSettings": { "EnableValidation": true, "ValidationMode": "ROW_LEVEL", "ThreadCount": 5, "FailureMaxCount": 10000, "TableFailureMaxCount": 1000 }, "Logging": { "EnableLogging": true, "LogComponents": [ { "Id": "SOURCE_UNLOAD", "Severity": "LOGGER_SEVERITY_DEFAULT" }, { "Id": "SOURCE_CAPTURE", "Severity": "LOGGER_SEVERITY_DEFAULT" }, { "Id": "TARGET_LOAD", "Severity": "LOGGER_SEVERITY_DEFAULT" }, { "Id": "TARGET_APPLY", "Severity": "LOGGER_SEVERITY_DEFAULT" }, { "Id": "TASK_MANAGER", "Severity": "LOGGER_SEVERITY_DEFAULT" } ] } } ``` **Configuration Comparison:** | Setting | Schema-First | Standard | Notes | | - | - | - | - | | TargetTablePrepMode | DO\_NOTHING | DROP\_AND\_CREATE | Schema-first uses existing schema | | ChangeProcessingTuning | Included | Not needed | Extra optimization for manual setup | | Logging Components | 5 components | 5 components | Both include all DMS components | | ValidationSettings | Same | Same | Both use row-level validation | **When to use each approach:** * **Schema-First**: Production systems, complex schemas, performance-critical applications * **Standard**: Simple migrations, dev/test environments, when schema objects aren't critical during migration ## Step 7: Configure Table Mappings ### Basic table mapping (migrate all tables): ```json theme={null} { "rules": [ { "rule-type": "selection", "rule-id": "1", "rule-name": "1", "object-locator": { "schema-name": "public", "table-name": "%" }, "rule-action": "include", "filters": [] } ] } ``` ### Advanced table mapping with transformations: ```json expandable theme={null} { "rules": [ { "rule-type": "selection", "rule-id": "1", "rule-name": "1", "object-locator": { "schema-name": "public", "table-name": "%" }, "rule-action": "include" }, { "rule-type": "transformation", "rule-id": "2", "rule-name": "2", "rule-target": "schema", "object-locator": { "schema-name": "public" }, "rule-action": "rename", "value": "public" } ] } ``` ## Step 8: Start Migration Task **Step 1** Review all task configurations **Step 2** Click "Create task" to start the migration **Step 3** Monitor the task status in the DMS console ## Step 9: Monitor Migration Progress ### Key metrics to monitor: * **Full load progress**: Percentage of tables loaded * **CDC lag**: Latency between source and target * **Error count**: Any migration errors * **Throughput**: Records per second ### Using CloudWatch: Set up [CloudWatch alarms](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Monitoring.html) for: * High CDC latency * Migration errors * Task failures ```bash theme={null} # CLI command to check task status aws dms describe-replication-tasks \ --filters Name=replication-task-id,Values=your-task-id ``` ## Step 10: Verify Data Migration ### Check table counts and data integrity: ```sql theme={null} -- Run on both source and target databases SELECT schemaname, tablename, n_tup_ins as estimated_rows, n_tup_upd as updated_rows, n_tup_del as deleted_rows FROM pg_stat_user_tables ORDER BY schemaname, tablename; ``` ### Validate specific data: ```sql theme={null} -- Compare checksums for critical tables SELECT count(*), md5(string_agg(column_name::text, '')) FROM your_important_table ORDER BY primary_key; ``` ## Step 11: Prepare for Cutover ### Monitor CDC lag: Ensure CDC latency is minimal (under 5 seconds) before cutover: ```sql theme={null} -- Check DMS validation status SELECT * FROM awsdms_validation_failures_v1; ``` ### Test application connectivity: 1. Create a read-only connection to PlanetScale Postgres 2. Test critical application queries with EXPLAIN ANALYZE 3. Verify performance matches expectations (indexes should be working) 4. Test sequence-dependent operations (INSERT operations) ## Step 12: Post-Migration Sequence Synchronization After DMS completes, sequences need their values synchronized: > [!WARNING] > > **Critical: Sequence Synchronization** > > Sequence values must be set ahead of source database values to prevent duplicate key errors when applications start using PlanetScale. ### Get Current Sequence Values from Source ```sql theme={null} -- Run on source database to get all current sequence values SELECT sequence_name, last_value, 'SELECT setval(''' || sequence_name || ''', ' || (last_value + 1000) || ');' as update_command FROM information_schema.sequences WHERE sequence_schema = 'public' ORDER BY sequence_name; ``` ### Update Sequences in PlanetScale ```sql theme={null} -- For each sequence, run the update command from above -- Example commands (values set ahead of source): SELECT setval('users_id_seq', 16234); -- Source value + 1000 SELECT setval('orders_id_seq', 99765); -- Source value + 1000 SELECT setval('products_id_seq', 6432); -- Source value + 1000 -- Verify sequence values are ahead of source SELECT sequence_name, last_value FROM information_schema.sequences WHERE sequence_schema = 'public' ORDER BY sequence_name; ``` ### Apply Remaining Constraints Now apply foreign key constraints that were deferred: ```sql theme={null} -- Apply foreign key constraints \i constraints.sql -- Verify constraints were applied successfully SELECT conname, contype, conrelid::regclass AS table_name FROM pg_constraint WHERE connamespace = 'public'::regnamespace AND contype = 'f' -- foreign key constraints ORDER BY conrelid::regclass::text; ``` ## Step 13: Comprehensive Pre-Cutover Validation > [!WARNING] > > **Complete Validation Required** > > Validate ALL schema objects and data integrity before cutover. Missing objects will cause application failures. ```sql theme={null} -- Validate table row counts match source SELECT schemaname, tablename, n_tup_ins as estimated_rows FROM pg_stat_user_tables WHERE schemaname = 'public' ORDER BY tablename; ``` ## Step 14: Perform Cutover When ready to switch to PlanetScale Postgres: **Step 1** **Stop application writes** to source database **Step 2** **Wait for CDC to catch up** (monitor lag in DMS console) **Step 3** **Verify final data consistency** **Step 4** **Update application connection strings** to point to PlanetScale **Step 5** **Start application** with new connections **Step 6** **Stop DMS task** once satisfied with cutover ### Stop the migration task: ```bash theme={null} aws dms stop-replication-task \ --replication-task-arn arn:aws:dms:region:account:task:task-id ``` ## Step 15: Cleanup The task configuration above is already optimized for schema-first migrations. Key settings: * **DO\_NOTHING** prep mode preserves your existing schema * **Row-level validation** ensures data integrity * **Batch processing** optimizations improve performance * **Memory tuning** handles large datasets efficiently > [!NOTE] > > **Automated vs Manual Configuration** > > For Aurora migrations, consider using the [automated CloudFormation approach](/docs/postgres/imports/aurora-dms) which includes these optimized settings and additional automation features. After successful cutover and schema migration: **Step 1** **Delete DMS task** **Step 2** **Delete replication instance** (to stop charges) **Step 3** **Remove source and target endpoints** **Step 4** **Clean up security groups** if created specifically for migration ```bash theme={null} # Cleanup commands aws dms delete-replication-task --replication-task-arn your-task-arn aws dms delete-replication-instance --replication-instance-arn your-instance-arn aws dms delete-endpoint --endpoint-arn your-source-endpoint-arn aws dms delete-endpoint --endpoint-arn your-target-endpoint-arn ``` ## Troubleshooting ### Common Issues: **Connectivity problems:** * Check security groups and network ACLs * Verify endpoint configurations * Test network connectivity from replication instance **Performance issues:** * Increase replication instance size * Adjust parallel load settings * Monitor source database performance **Data type mapping issues:** * Review [DMS data type mapping](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Reference.DataTypes.html) * Use transformation rules for custom mappings **Large object (LOB) handling:** ```json theme={null} { "TargetMetadata": { "SupportLobs": true, "FullLobMode": true, "LobChunkSize": 32768, "LimitedSizeLobMode": false } } ``` ### Schema-Related Troubleshooting: **"sequence does not exist" errors:** ```sql theme={null} -- Check if sequence exists SELECT * FROM information_schema.sequences WHERE sequence_name = 'your_sequence'; -- Recreate missing sequence CREATE SEQUENCE your_sequence START WITH 1; SELECT setval('your_sequence', (SELECT MAX(id) FROM your_table)); ``` **Missing indexes causing performance issues:** ```sql theme={null} -- Find missing indexes by comparing to source -- Run on source database to get index list SELECT indexname, indexdef FROM pg_indexes WHERE schemaname = 'public'; -- Check query performance EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM your_table WHERE indexed_column = 'value'; ``` **Foreign key constraint violations:** ```sql theme={null} -- Check for constraint violations before applying SELECT COUNT(*) FROM child_table c WHERE NOT EXISTS (SELECT 1 FROM parent_table p WHERE p.id = c.parent_id); -- Apply constraints one by one to isolate issues ALTER TABLE child_table ADD CONSTRAINT fk_name FOREIGN KEY (parent_id) REFERENCES parent_table(id); ``` **Functions/views with dependency errors:** ```sql theme={null} -- Check dependencies SELECT * FROM pg_depend WHERE objid = 'your_function'::regproc; -- Apply in dependency order: functions before views that use them ``` **Permission errors during schema application:** * Ensure PlanetScale database user has CREATE permissions * Check if objects already exist and need DROP statements * Verify user has permissions on referenced objects **Sequence values too low causing duplicate key errors:** ```sql theme={null} -- Check current sequence value vs max table value SELECT last_value FROM your_sequence; SELECT MAX(id) FROM your_table; -- Update sequence to safe value SELECT setval('your_sequence', (SELECT MAX(id) FROM your_table)); ``` ### Performance Optimization: 1. **Parallel loading**: Increase `MaxFullLoadSubTasks` 2. **Batch apply**: Enable for better target performance 3. **Memory allocation**: Increase replication instance size 4. **Network optimization**: Use placement groups for better network performance ## Cost Optimization * **Instance sizing**: Start with smaller instances and scale up if needed * **Multi-AZ**: Disable for dev/test migrations * **Task lifecycle**: Delete resources immediately after successful migration * **Data transfer**: Consider AWS region placement to minimize transfer costs ## Schema Considerations Before migration, review: > **[PostgreSQL version compatibility](/docs/postgres/imports/postgres-imports#postgresql-version-compatibility)** > **[Extension support limitations](/docs/postgres/imports/postgres-imports#extension-support)** > **[Third-party enhancement restrictions](/docs/postgres/imports/postgres-imports#third-party-enhancements-and-tools)** **Important:** Plan additional time for post-migration schema object setup. Complex databases may require several hours for index recreation and sequence synchronization. **Performance Impact Note:** Large indexes can take hours to rebuild on populated tables. Consider the schema-first approach to avoid this performance penalty. ## Next Steps After successful migration and schema setup: **Step 1** **Run comprehensive post-cutover validation** using all verification queries above **Step 2** **Monitor application logs** for any sequence or constraint errors **Step 3** **Performance baseline comparison** - compare query performance to source database **Step 4** **Test critical business workflows** end-to-end **Step 5** Set up monitoring and alerting for PlanetScale Postgres **Step 6** Plan for ongoing maintenance and backup strategies **Step 7** Consider implementing additional PlanetScale features **Success Criteria:** * ✅ All schema objects validated and functional * ✅ Sequence values synchronized and tested * ✅ Query performance matches or exceeds source database * ✅ No application errors in logs for 24+ hours * ✅ All foreign key constraints working correctly For simpler migrations, consider [pg\_dump/restore](./postgres-migrate-dumprestore) or [WAL streaming](./postgres-migrate-walstream) methods. If you encounter issues during migration, please [reach out to support](https://planetscale.com/contact?initial=support) for assistance. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres Imports - PGDump and Restore Source: https://planetscale.com/docs/postgres/imports/postgres-migrate-dumprestore > [!NOTE] > > Before beginning your migration, we recommend running our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. The [pg\_dump](https://www.postgresql.org/docs/current/app-pgdump.html) and restore method is the simplest approach for migrating PostgreSQL databases to PlanetScale Postgres. This method is ideal for smaller databases that can tolerate downtime during the migration process. > [!NOTE] > > Want expert guidance for your migration? PlanetScale's [migration services](https://planetscale.com/migrate) are available to help you plan and execute a smooth, successful move. ## Overview This migration method involves: 1. Creating a full backup of your source database using `pg_dump` 2. Transferring the dump file to your local environment or staging area 3. Restoring the database to PlanetScale Postgres using `pg_restore` or `psql` ## Prerequisites Before starting the migration: * Ensure you have PostgreSQL client tools installed (`pg_dump`, `pg_restore`, `psql`) * Have read access to your source PostgreSQL database * Have connection details for your PlanetScale Postgres database from the console * Ensure the disk on your PlanetScale database has at least 150% of the capacity of your source database. If you are migrating to a PlanetScale database backed by network-attached storage, you can [resize](/docs/postgres/cluster-configuration/cluster-storage) your disk manually by setting the "Minimum disk size." If you are using Metal, you will need to select a size when first creating your database. For example, if your source database is 330GB, you should have at least 500GB of storage available on PlanetScale. * Verify sufficient storage space for the dump file * Plan for application downtime during the migration ## Step 1: Create Database Dump ### For a complete database dump: ```bash theme={null} pg_dump -h source-host \ -p source-port \ -U source-username \ -d source-database \ --verbose \ --no-owner \ --no-privileges \ -f database_dump.sql ``` ### For a custom format dump (recommended for large databases): ```bash theme={null} pg_dump -h source-host \ -p source-port \ -U source-username \ -d source-database \ --verbose \ --no-owner \ --no-privileges \ -Fc \ -f database_dump.dump ``` ### Command options explained: * `--verbose`: Provides detailed output during the dump process * `--no-owner`: Excludes ownership information from the dump * `--no-privileges`: Excludes privilege information from the dump * `-Fc`: Creates a custom format dump (binary, compressed) * `-f`: Specifies the output file name ## Step 2: Get PlanetScale Connection Details From your PlanetScale console: **Step 1** Navigate to your PlanetScale Postgres database **Step 2** Go to the "Connect" section **Step 3** Copy the connection details including: * Host * Port * Database name * Username * Password ## Step 3: Restore to PlanetScale Postgres ### For SQL format dumps: ```bash theme={null} psql -h planetscale-host \ -p planetscale-port \ -U planetscale-username \ -d planetscale-database \ -f database_dump.sql ``` ### For custom format dumps: ```bash theme={null} pg_restore -h planetscale-host \ -p planetscale-port \ -U planetscale-username \ -d planetscale-database \ --verbose \ --no-owner \ --no-privileges \ database_dump.dump ``` ### For parallel restoration (faster for large databases): ```bash theme={null} pg_restore -h planetscale-host \ -p planetscale-port \ -U planetscale-username \ -d planetscale-database \ --verbose \ --no-owner \ --no-privileges \ --jobs=4 \ database_dump.dump ``` ## Step 4: Verify Migration After the restore completes, verify your migration: ### Check table counts: ```sql theme={null} SELECT schemaname, relname, n_tup_ins as row_count FROM pg_stat_user_tables ORDER BY schemaname, relname; ``` ### Verify data integrity: ```sql theme={null} -- Check a few sample records from key tables SELECT count(*) FROM your_main_table; SELECT * FROM your_main_table LIMIT 5; ``` ### Check for errors in logs: Review the output from the pg\_restore command for any errors or warnings. ## Troubleshooting ### Common Issues: **Permission errors:** * Ensure your PlanetScale user has appropriate permissions * Check that connection details are correct **"must be able to SET ROLE" error:** If you encounter an error like `must be able to SET ROLE "some_role"` when running `CREATE DATABASE ... OWNER ...` or during a restore, this occurs because your current user doesn't have membership in the role you're trying to assign as owner. To resolve this, grant the role to your current user first: ```sql theme={null} -- Grant the role to your current user GRANT some_role TO current_user; -- Now you can create databases owned by that role CREATE DATABASE mydb OWNER some_role; ``` Alternatively, if you're restoring a dump and don't need to preserve ownership, use the `--no-owner` flag with `pg_restore` or `pg_dump` to skip ownership assignments entirely: ```bash theme={null} pg_restore --no-owner --no-privileges -h planetscale-host -U planetscale-username -d planetscale-database database_dump.dump ``` **Extension errors:** * Some PostgreSQL extensions may not be available in PlanetScale Postgres * Review the [extension compatibility guide](/docs/postgres/imports/postgres-imports#extension-support) **Large object errors:** * If using large objects (BLOBs), add `--blobs` flag to pg\_dump ```bash theme={null} pg_dump --blobs -h source-host -U source-username -d source-database -f database_dump.sql ``` **Timeout errors:** * For large databases, consider breaking the dump into smaller chunks * Use custom format with parallel restoration ### Performance Tips: 1. **Use custom format**: Binary format with compression is more efficient 2. **Parallel jobs**: Use `--jobs` parameter for faster restoration 3. **Network considerations**: Ensure stable network connection for large transfers 4. **Disk space**: Monitor available disk space during dump creation ## Schema Considerations Before migration, review: > **[PostgreSQL version compatibility](/docs/postgres/imports/postgres-imports#postgresql-version-compatibility)** > **[Extension support limitations](/docs/postgres/imports/postgres-imports#extension-support)** > **[Third-party enhancement restrictions](/docs/postgres/imports/postgres-imports#third-party-enhancements-and-tools)** ## Next Steps After successful migration: **Step 1** Update your application connection strings to point to PlanetScale Postgres **Step 2** Test your application thoroughly in a staging environment **Step 3** Plan your production cutover **Step 4** Monitor performance and optimize as needed For more advanced migration scenarios or larger databases, consider [WAL streaming](./postgres-migrate-walstream) or [Amazon DMS](./postgres-migrate-dms) methods. If you encounter issues during migration, please [reach out to support](https://planetscale.com/contact?initial=support) for assistance. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres Imports - pgcopydb Source: https://planetscale.com/docs/postgres/imports/postgres-migrate-pgcopydb Migrate your PostgreSQL database to PlanetScale Postgres using pgcopydb with near-zero downtime through parallel data copying and change data capture. [pgcopydb](https://github.com/planetscale/pgcopydb) is a tool that automates migrating data between two running PostgreSQL servers. PlanetScale maintains a fork that supports PostgreSQL 17 and 18, improved filtering, resilient retry, and improved CDC support. This guide uses PlanetScale's fork throughout. PlanetScale also provides [helper scripts](https://github.com/planetscale/migration-scripts/tree/main/pgcopydb-helpers) that automate the setup, monitoring, cutover, and cleanup of the migration process. PlanetScale has used pgcopydb and these scripts to migrate multi-terabyte databases from live customer environments, with speeds as fast as 2 TB per hour using [Metal](/docs/metal). > [!NOTE] > > pgcopydb is a highly capable tool with extensive tunables and flags for handling a wide range of migration scenarios. The helper scripts in this guide use sensible defaults, but you can customize them for your specific environment. See the [pgcopydb documentation](https://github.com/planetscale/pgcopydb) for in-depth coverage of all available options. We recommend testing the migration on a staging environment or from a backup/snapshot before running it on production, and always ensure you have a backup of your source database before starting. ## How pgcopydb works Traditional `pg_dump | pg_restore` pipelines write all data to intermediate files on disk before restoring to the target. This doubles disk I/O, creates storage bottlenecks, and builds indexes one at a time. For large databases, this can take days. ```mermaid theme={null} flowchart LR Source[("Source PostgreSQL")] --> pgcopydb{{"pgcopydb Migration Instance"}} --> Target[("PlanetScale Postgres")] style Source fill:#4a90d9,stroke:#2c5f8a,color:#fff style pgcopydb fill:#6c757d,stroke:#495057,color:#fff style Target fill:#f15817,stroke:#c44512,color:#fff ``` pgcopydb takes a different approach: 1. **Direct streaming with no intermediate files.** Table data flows directly from source to target using PostgreSQL's native COPY protocol. No dump files are written to disk. Data moves over the network in a single pass. 2. **Parallel operations at every stage.** pgcopydb runs multiple operations concurrently: * Multiple tables are copied in parallel * Large tables are automatically split into partitions and copied by multiple workers simultaneously * Indexes are built concurrently on the target, not sequentially like `pg_restore` * VACUUM, sequence resets, and constraint creation all run in parallel 3. **Live migration with Change Data Capture (CDC).** With the `--follow` flag, pgcopydb captures changes from the source via logical decoding while the initial copy is still running. Changes are prefetched to disk during the copy phase and replayed once the copy finishes. The target stays in sync with the source, and cutover requires only a brief pause while the final changes apply. Internally, pgcopydb proceeds through these phases: | Phase | Description | | :- | :- | | **Catalog source** | Query source metadata, apply filters, estimate table sizes | | **Dump schema** | Export schema using `pg_dump` with a consistent snapshot | | **Restore schema** | Create tables and types on the target with `pg_restore` | | **Copy table data** | Stream data in parallel using COPY protocol | | **Restore post-data** | Create views, materialized views, and functions | | **Create indexes** | Build all indexes concurrently | | **Create constraints** | Apply foreign keys and check constraints | | **Vacuum and analyze** | Update table statistics and reclaim space | | **Reset sequences** | Synchronize sequence values from source | | **Finalize** | Complete post-data restoration | When `--follow` is enabled, CDC runs alongside these phases. Changes are prefetched during the copy and switch to live replay once the copy completes. The migration is ready for cutover when CDC lag drops to near zero. ## Overview The migration follows this high-level flow: **Step 1** **Prepare your environment** Set up a migration instance, install pgcopydb and the helper scripts, and configure database credentials. **Step 2** **Prepare your source database** Create a migration user, enable logical replication, and configure replication settings. **Step 3** **Prepare your PlanetScale database** Create a database, configure disk size and parameters, and compare source parameters. **Step 4** **Configure filtering** Define which schemas, tables, and extensions to exclude from the migration. **Step 5** **Run the migration** Execute pgcopydb to copy schema and table data in parallel, then build indexes and constraints. **Step 6** **Monitor progress** Track copy progress, index creation, and CDC replication lag using the helper scripts. **Step 7** **Prepare for cutover** Create application roles, confirm connectivity, configure observability, and prepare connection strings. **Step 8** **Cutover** Stop writes to the source, wait for CDC to catch up, and switch your application to PlanetScale. **Step 9** **Clean up** Remove replication slots and the migration user from the source database. > [!NOTE] > > If you do not need near-zero downtime and can tolerate a maintenance window, you can use pgcopydb without CDC (omit the `--follow` flag). This is simpler but requires stopping writes to the source for the duration of the copy. ## 1. Prepare your environment The migration runs from a dedicated **migration instance**, a compute instance (EC2, GCE, Azure VM, or similar) with network access to both your source database and PlanetScale. This instance runs pgcopydb and the helper scripts. It does not store any persistent data beyond the migration working directory. PlanetScale provides infrastructure-as-code templates that provision a fully configured migration instance with pgcopydb v0.20.0, PostgreSQL 18 client tools, and the helper scripts pre-installed. Choose the template that matches your cloud provider and provisioning tool: **AWS CloudFormation** Upload the [CloudFormation template](https://github.com/planetscale/migration-scripts/tree/main/pgcopydb-templates/aws-cloudformation) to CloudFormation and fill in VPC ID, Subnet ID, instance type, and volume size. Or deploy via the CLI: ```bash theme={null} aws cloudformation create-stack \ --stack-name planetscale-migration \ --template-body file://pgcopydb-migration-instance.yaml \ --parameters \ ParameterKey=VpcId,ParameterValue=vpc-xxxxxxxxx \ ParameterKey=SubnetId,ParameterValue=subnet-xxxxxxxxx \ ParameterKey=InstanceType,ParameterValue=c7i.xlarge \ ParameterKey=VolumeSize,ParameterValue=1000 \ --capabilities CAPABILITY_NAMED_IAM ``` **Key parameters:** * `VpcId` / `SubnetId` — The VPC and public subnet where the instance will launch. The subnet must have an internet gateway route for PlanetScale connectivity. * `InstanceType` — `c7i.xlarge` for databases under 100 GB, `c7i.2xlarge` for 100–500 GB, `c7i.4xlarge` for 500 GB+. * `VolumeSize` — EBS io2 Block Express volume in GB. Choose based on your source database size (500, 1000, 3000, 6000, or 12000). * `KeyPairName` — Optional. Leave empty to use EC2 Instance Connect for SSH access. **Connect to the instance** via SSM Session Manager (no SSH key needed): ```bash theme={null} aws ssm start-session --target INSTANCE_ID ``` You can also use EC2 Instance Connect. Open the instance in the EC2 console and click **Connect** for browser-based SSH. **AWS Terraform** Save the [Terraform template](https://github.com/planetscale/migration-scripts/tree/main/pgcopydb-templates/aws-terraform) files (`pgcopydb-migration-instance.tf` and `user-data.sh`) in the same directory and deploy: ```bash theme={null} terraform init terraform apply \ -var="vpc_id=vpc-xxxxxxxxx" \ -var="subnet_id=subnet-xxxxxxxxx" \ -var="instance_type=c7i.xlarge" \ -var="volume_size=1000" ``` **Key variables:** * `vpc_id` / `subnet_id` — The VPC and public subnet where the instance will launch. The subnet must have an internet gateway route for PlanetScale connectivity. * `instance_type` — `c7i.xlarge` for databases under 100 GB, `c7i.2xlarge` for 100–500 GB, `c7i.4xlarge` for 500 GB+. * `volume_size` — EBS io2 Block Express volume in GB (500, 1000, 3000, 6000, or 12000). * `key_pair_name` — Optional. Leave empty to use EC2 Instance Connect for SSH access. * `region` — AWS region (default: `us-east-1`). **Connect to the instance** via SSM Session Manager: ```bash theme={null} aws ssm start-session --target $(terraform output -raw instance_id) ``` You can also use EC2 Instance Connect. Open the instance in the EC2 console and click **Connect** for browser-based SSH. **GCP Terraform** Save the [Terraform template](https://github.com/planetscale/migration-scripts/tree/main/pgcopydb-templates/gcp-terraform) files (`pgcopydb-migration-instance.tf` and `startup-script.sh`) in the same directory and deploy: ```bash theme={null} terraform init terraform apply \ -var="project_id=my-project" \ -var="vpc_name=my-vpc" \ -var="subnet_name=my-subnet" \ -var="service_account_email=sa@my-project.iam.gserviceaccount.com" \ -var="machine_type=n2-standard-8" \ -var="disk_size_gb=500" ``` **Key variables:** * `project_id` — Your GCP project ID. * `vpc_name` / `subnet_name` — The VPC and subnet where the instance will launch. * `service_account_email` — Service account to attach to the instance. * `machine_type` — `n2-standard-8` or `c3d-standard` family. Size based on your source database. * `disk_size_gb` — Boot disk size in GB (100–10000). Choose based on your source database size. * `region` / `zone` — GCP region and zone (default: `us-central1-a`). **Connect to the instance** via IAP tunnel: ```bash theme={null} gcloud compute ssh $(terraform output -raw instance_name) \ --zone=$(terraform output -raw zone) \ --project=YOUR_PROJECT \ --tunnel-through-iap ``` You can also use the GCP Console. Open the instance in Compute Engine and click **SSH** for browser-based access. **Azure Terraform** Save the [Terraform template](https://github.com/planetscale/migration-scripts/tree/main/pgcopydb-templates/azure-terraform) files (`pgcopydb-migration-instance.tf` and `custom-data.sh`) in the same directory and deploy: ```bash theme={null} terraform init terraform apply \ -var="subscription_id=$(az account show --query id -o tsv)" \ -var="vnet_resource_group_name=YOUR_NETWORK_RG" \ -var="vnet_name=YOUR_VNET" \ -var="subnet_name=YOUR_SUBNET" \ -var="your_public_ip=$(curl -4 icanhazip.com)" ``` **Key variables:** * `subscription_id` — Your Azure subscription ID. * `vnet_resource_group_name` / `vnet_name` / `subnet_name` — An existing virtual network and subnet where the VM will launch, and the resource group that holds them. Everything the template creates lands in its own `planetscale-migration-rg` resource group, so tearing it down never touches your resources. * `your_public_ip` — Restricts SSH access to your IP address. * `vm_size` — `Standard_E4bds_v5` for databases under 100 GB, `Standard_E8bds_v5` (default) for 100–500 GB, `Standard_E16bds_v5` for 500 GB+. The data disk's provisioned IOPS are matched to the VM size. * `data_disk_size_gb` — Premium SSD v2 data disk size in GB (500, 1000, or 3000). Choose based on your source database size. * `region` / `zone` — Azure region and availability zone (default: `eastus`). The region must support zonal Premium SSD v2 disks. If `terraform apply` fails with an allocation error, retry with a different `zone`. **Connect to the instance** as the `ubuntu` user with the generated SSH key: ```bash theme={null} terraform output -raw generated_ssh_private_key > ./migration_key chmod 600 ./migration_key ssh -i ./migration_key ubuntu@$(terraform output -raw public_ip) ``` > [!WARNING] > > The generated private key is stored in `terraform.tfstate` as well as `./migration_key`. Do not commit or share either file, and delete `./migration_key` once the migration is complete. To skip the generated key entirely, pass your own key with `-var="ssh_public_key=..."`. You can also connect with Entra ID (`az ssh vm ...`, or **Connect** in the portal), which logs you in as your own user rather than `ubuntu` — run `sudo su - ubuntu` after connecting. First boot takes 10–15 minutes while pgcopydb is built from source. Watch progress with `sudo tail -f /var/log/pgcopydb-setup.log`. > [!NOTE] > > If you prefer to set up your own instance manually, see the [pgcopydb installation documentation](https://github.com/planetscale/pgcopydb#install) and the [helper scripts README](https://github.com/planetscale/migration-scripts/tree/main/pgcopydb-helpers) for manual installation steps. Once your migration instance is running, **connect to it** using one of the methods shown above. All commands and scripts in the remaining steps are run from the migration instance. The examples in this guide assume you are logged in as the `ubuntu` user on an instance created by the templates, with helper scripts installed in `/home/ubuntu/`. ### Configure connection credentials The templates create a `~/.env` file with placeholder connection strings. Update this file with your actual source and target credentials: ```bash theme={null} # Source Database PGCOPYDB_SOURCE_PGURI="postgresql://user:password@source-host:5432/dbname?sslmode=require" # Target Database (PlanetScale) PGCOPYDB_TARGET_PGURI="postgresql://user:password@target-host.connect.psdb.cloud:5432/dbname?sslmode=require" ``` All helper scripts read from this file automatically. > [!WARNING] > > This file contains database credentials. Restrict permissions with `chmod 600 ~/.env`. ## 2. Prepare your source database ### Create a migration user The migration user needs read access to all schemas and tables being migrated. For CDC migrations using `--follow`, it also needs the `REPLICATION` attribute. If you are using a dedicated migration user rather than the database owner, create one and grant the required permissions: ```sql theme={null} -- Create the migration user CREATE ROLE migration_user WITH LOGIN PASSWORD 'your-strong-password' REPLICATION; -- Connect and schema access GRANT CONNECT ON DATABASE mydb TO migration_user; GRANT USAGE ON SCHEMA public TO migration_user; -- Read access to all tables and sequences GRANT SELECT ON ALL TABLES IN SCHEMA public TO migration_user; GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO migration_user; -- For future tables created before migration starts ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO migration_user; -- Prevent idle replication connection from being dropped during long COPY phases ALTER ROLE migration_user SET wal_sender_timeout = 0; ``` Repeat the `GRANT USAGE`, `GRANT SELECT`, and `ALTER DEFAULT PRIVILEGES` statements for each schema being migrated. Setting `wal_sender_timeout` to `0` on the source prevents the replication slot from being dropped during long COPY phases. The initial data copy can take hours on large databases, and the default timeout (60s) may cause PostgreSQL to drop the idle replication connection before CDC streaming begins. Once the user is created, update the `PGCOPYDB_SOURCE_PGURI` in your `~/.env` file on the migration instance with the new credentials. ### Enable logical replication (CDC only) For CDC migrations using `--follow`, logical replication must be enabled on the source. How to enable it depends on your platform: * **PlanetScale Postgres:** Configure `wal_level = logical` and the required replication parameters in **Clusters > Parameters**. Connect with a role that has replication permissions and access to the tables being migrated. See [Logical replication and CDC](/docs/postgres/integrations/logical-cdc) for configuration and role setup. * **Amazon RDS / Aurora:** Set `rds.logical_replication = 1` in the parameter group and reboot the instance. * **Google Cloud SQL:** Set the `cloudsql.logical_decoding` database flag to `on` and restart the instance. * **Google AlloyDB:** Set `alloydb.logical_decoding = on` and restart the instance. * **Azure Database for PostgreSQL (Flexible Server):** On the **Server parameters** page, set `wal_level` to `logical` and `max_worker_processes` to at least 16, then restart the server. Grant replication permissions to your migration user with `ALTER ROLE migration_user WITH REPLICATION;`. CDC from Azure sources requires pgcopydb v0.20.0 or newer — older versions send `TIMELINE_HISTORY` on the replication connection, which Azure rejects. The templates above install v0.20.0. * **Supabase:** Logical replication is enabled by default. Make sure you use a **direct connection** string (not pooled) from your Supabase dashboard under Project Settings > Database. * **Self-hosted PostgreSQL:** Set `wal_level = logical` in `postgresql.conf` and restart PostgreSQL. Verify logical replication is enabled: ```sql theme={null} SHOW wal_level; -- should return 'logical' ``` ### Fix replica identity (CDC migrations only) For CDC migrations, tables without a primary key need `REPLICA IDENTITY FULL` set so that UPDATE and DELETE operations can be replicated. The [fix-replica-identity script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/fix-replica-identity.sh) identifies affected tables and applies the change: ```bash theme={null} ~/fix-replica-identity.sh ``` The script queries the source for tables with default replica identity that lack a primary key or unique index, previews the `ALTER TABLE` statements, and prompts for confirmation before applying. This script requires table ownership on the source, not just `SELECT`. If you are using a dedicated migration user, grant it membership in the role(s) that own the affected tables: ```sql theme={null} GRANT TO migration_user; ``` After running the script, revoke the membership: ```sql theme={null} REVOKE FROM migration_user; ``` > [!WARNING] > > Setting `REPLICA IDENTITY FULL` increases WAL volume for UPDATE and DELETE operations on affected tables, as PostgreSQL must log full row contents instead of just key columns. Consider the impact on your source database's disk usage and replication throughput. ## 3. Prepare your PlanetScale database Create a new database in the [PlanetScale dashboard](https://app.planetscale.com/new) or using the [PlanetScale CLI](/docs/cli). When configuring the database: * Select the correct **cloud region**. You typically want to use the same region as your source database and application infrastructure. * Choose **Postgres** as the database type. * Choose the best **storage option** for your needs. For high-performance, low-latency I/O, use [PlanetScale Metal](/docs/metal). For more flexible storage options, choose "Elastic Block Storage" (AWS), "Persistent Disk" (GCP), or "Azure Managed Disks" (Azure). ### Configure disk size Set your disk size to at least **150%** of your source database size. For example, if your source is 330 GB, provision at least 500 GB. The overhead accounts for data growth during import and table/index bloat that occurs during the copy process. * **Network-attached storage (EBS, Persistent Disk, Azure Managed Disks):** Configure the disk in advance by navigating to **Clusters > Storage** and setting the **Minimum disk size**. Cloud providers limit how frequently disks can be resized, so the disk may not be able to resize fast enough during a large import. * **Metal:** Choose the disk size when creating the cluster. You can resize after import and cleanup. ### Enable extensions Review the extensions installed on your source database and compare them against the [PlanetScale supported extensions](/docs/postgres/extensions) list. Any extensions your application depends on should be enabled on PlanetScale before running the migration. Some extensions require enabling through the PlanetScale dashboard and may need a cluster restart. See the [extensions documentation](/docs/postgres/extensions#configuring-extensions-in-the-dashboard) for details. ### Use the Default role Use the [**Default role**](/docs/postgres/connecting/roles#default-role) on your PlanetScale database for the migration. This role has the necessary permissions to create schemas, tables, indexes, and manage replication. You can find the Default role credentials by navigating to your database in the [PlanetScale dashboard](https://app.planetscale.com) and clicking **Connect**. Copy the connection credentials into the `PGCOPYDB_TARGET_PGURI` in your `~/.env` file. ### Compare and configure parameters Use the [compare-pg-params script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/compare-pg-params.sh) to compare PostgreSQL parameters between your source and PlanetScale target to identify any differences that could affect application behavior: ```bash theme={null} ~/compare-pg-params.sh ``` This script checks 50+ performance-relevant parameters across resource usage, query tuning, WAL configuration, connections, replication, autovacuum, statistics, and shared libraries. The output shows a side-by-side comparison with indicators for which parameters require a restart versus a reload to change. Review the output and adjust parameters on your PlanetScale database as needed through the **Clusters > Parameters** page. Note that many parameters on a PlanetScale cluster are already tuned appropriately, so copying values directly from another provider may not be beneficial. You should also increase `max_worker_processes` above its default value. This allows pgcopydb to run more parallel workers during the copy phase. You can decrease this value after the migration is complete. ### Run the preflight check The [preflight check script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/preflight-check.sh) validates all migration prerequisites before you start: ```bash theme={null} ~/preflight-check.sh ``` This checks: **Source database:** * Connectivity * `wal_level = logical` * Replication permissions (`REPLICATION`, `SUPERUSER`, or `rds_replication`) * Available replication slots * Available WAL senders * No leftover pgcopydb replication slots from previous attempts * `wal_sender_timeout` configuration * No pending prepared transactions **Target database:** * Connectivity * Replication permissions * No leftover pgcopydb schema from previous attempts * Extension compatibility (all source extensions are available on the target) **Migration instance:** * `~/filters.ini` exists * `pgcopydb` binary is on `PATH` The output is color-coded: **PASS**, **WARN**, or **FAIL**. Resolve all FAILs before proceeding. ## 4. Configure filtering The `~/filters.ini` file controls which schemas, tables, extensions, and event triggers are excluded from the migration. Edit this file to match your source environment: > [!WARNING] > > Do not place comments inside sections. pgcopydb treats `#` characters as literal object names. Place all comments before the first section header. Any extensions not supported by PlanetScale should be added to the `[exclude-extension]` section. If unsupported extensions are not excluded, the migration may fail when trying to create those objects on the target. Check the [PlanetScale extensions documentation](/docs/postgres/extensions) to verify which extensions are supported. The following are starting-point filters for common source providers. Copy the one that matches your source and adjust as needed: **AWS RDS/Aurora** ```ini theme={null} [exclude-extension] rds_tools aws_commons aws_s3 pg_repack ``` **Supabase** ```ini theme={null} [exclude-schema] auth supabase_functions extensions storage cron realtime supabase_migrations net _supabase graphql graphql_public [exclude-extension] pg_net pg_graphql pgsodium supabase_vault pg_repack http pg_stat_monitor [exclude-event-trigger] supabase_functions.hooks ``` **Google AlloyDB** ```ini theme={null} [exclude-schema] ai google_ml helpobj perfsnap pgsnap [exclude-extension] g_stats google_columnar_engine google_db_advisor google_ml_integration ``` ## 5. Run the migration Start the migration using the [start-migration-screen script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/start-migration-screen.sh) which runs in a detached [screen](https://www.gnu.org/software/screen/) session so it survives SSH disconnects: ```bash theme={null} ~/start-migration-screen.sh ``` To attach and watch the migration progress: ```bash theme={null} screen -r migration ``` To detach without stopping the migration, press `Ctrl-A D`. ### What happens during migration The [run-migration script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/run-migration.sh) executes pgcopydb with the following settings: * **16 parallel table COPY workers** (`TABLE_JOBS=16`) * **12 parallel index creation workers** (`INDEX_JOBS=12`) * Tables larger than **50 GB** are automatically split into parts for parallel copying * The [`wal2json`](https://github.com/eulerto/wal2json) logical decoding plugin is used for CDC The migration proceeds through the phases described in [How pgcopydb works](#how-pgcopydb-works). All output is logged to `~/migration_YYYYMMDD-HHMMSS/migration.log`. ### Tuning parallelism You can tune the parallel job counts by editing the variables at the top of `~/run-migration.sh` before starting: ```bash theme={null} TABLE_JOBS=16 # Parallel COPY workers (keep below source and target vCPU count) INDEX_JOBS=12 # Parallel index workers (keep below target vCPU count) ``` Higher values speed up the migration but put more load on both databases. As a guideline, keep `TABLE_JOBS` below the lesser of your source and target vCPU counts, and keep `INDEX_JOBS` below the target vCPU count. ## 6. Monitor progress ### During the copy phase Use the [check-migration-status script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/check-migration-status.sh) to see a real-time dashboard of migration progress: ```bash theme={null} ~/check-migration-status.sh ``` This displays: * Phase status (1-10: pending, running, or done) * Copy progress, including tables in progress, completed, and percentage * Index and constraint creation progress * Data transferred in GB * Error counts with context * Elapsed runtime * Active database operations on the target (COPY, CREATE INDEX, VACUUM, ALTER TABLE with durations) You can run this as frequently as you like to check status. ### During CDC replication Once the initial copy completes and pgcopydb enters the CDC phase, use the [check-cdc-status script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/check-cdc-status.sh) to monitor replication lag: ```bash theme={null} ~/check-cdc-status.sh ``` This displays: * Apply LSN and Streaming LSN * Apply backlog gap in MB/GB * Apply rate (GB/hr) * Estimated time to catch up * Source replication slot health * Total end-to-end lag When the apply backlog gap drops below 100 MB, the script shows **"CDC IS CAUGHT UP"** and you are ready to proceed with cutover. ## 7. Prepare for cutover While CDC is running and caught up, prepare your PlanetScale database for production traffic before performing the cutover. **Create application roles.** pgcopydb does not migrate roles from the source database. The Default role used for migration has elevated privileges and should not be used by your application. Create dedicated roles with appropriate permissions for your application's needs using PlanetScale's built-in role management. See [Managing roles](/docs/postgres/connecting/roles) for details. **Confirm application connectivity.** Verify that your application can connect to the PlanetScale database using the new role credentials. Test read queries to confirm connectivity, but do not write to the target while CDC is still replicating. **Configure observability.** Set up monitoring for your PlanetScale database so you have visibility from the moment you cut over. See [Prometheus integration](/docs/postgres/monitoring/prometheus-postgres) and [Metrics](/docs/postgres/monitoring/metrics) for available options. **Prepare connection strings.** Have your new PlanetScale connection strings ready to deploy in your application configuration. If your application uses environment variables or a secrets manager, stage the new values so the switch can happen quickly. **Configure connection pooling if needed.** If your application requires dedicated connection pooling, set up [PgBouncer](/docs/postgres/connecting/pgbouncer) ahead of the cutover. ## 8. Cutover The cutover process switches your application from the source database to PlanetScale. This is the only point where downtime occurs. **Step 1** **Stop writes to the source database** Put your application into maintenance mode, switch to read-only, or drain connections. The goal is to ensure no new writes reach the source. **Step 2** **Set the CDC endpoint** Run the [stop-cdc script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/stop-cdc.sh) — it fetches the current WAL LSN from the source, displays it, and prompts for confirmation before applying: ```bash theme={null} ~/stop-cdc.sh ``` The script updates pgcopydb's internal state so it stops streaming once it has applied all changes up to the confirmed LSN. **Step 3** **Wait for CDC to reach the endpoint** Monitor with `~/check-cdc-status.sh` until the apply LSN matches the endpoint. pgcopydb will exit cleanly once it has applied all changes. **Step 4** **Verify data on the target** Run the [verify-migration script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/verify-migration.sh) to compare source and target databases: ```bash theme={null} ~/verify-migration.sh ``` This script compares schemas, estimated row counts, table sizes, indexes, constraints, and sequences between source and target. It also performs spot-checks on your largest tables without full table scans. It typically completes in under 2 minutes, even on multi-terabyte databases. The output is color-coded: **PASS**, **WARN**, or **FAIL**. **Step 5** **Switch your application to PlanetScale** Update your application's database connection string to point to PlanetScale and deploy. > [!WARNING] > > After switching traffic to PlanetScale, new writes will not be replicated back to the source. Ensure you are fully ready before performing the cutover. We recommend keeping the source database available for a few days in case you need to verify any data. ## 9. Clean up After a successful cutover, clean up replication artifacts on the source database: ```bash theme={null} ~/drop-replication-slots.sh ``` The [drop-replication-slots script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/drop-replication-slots.sh) removes: * The logical replication slot on the source (terminates the active consumer if needed) * The replication origin on the target * The pgcopydb sentinel schema on the target > [!WARNING] > > Always run this cleanup after migration. Unconsumed replication slots cause WAL to accumulate on the source database until the disk fills, which can take down your source database. You should also: * **Remove the migration user on the source database.** If you created a dedicated user for pgcopydb to connect to your source, drop it now that the migration is complete: ```sql theme={null} DROP USER IF EXISTS migration_user; ``` * Decrease `max_worker_processes` on the PlanetScale database back to its default value. * Run `VACUUM ANALYZE` on the target to reclaim any bloat from the import. You can also use [pg\_squeeze](/docs/postgres/extensions/pg_squeeze) for online table compaction. ## Resuming after interruption If the migration is interrupted by a crash, reboot, or connection failure, use the [resume-migration script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/resume-migration.sh) to continue from where it left off: ```bash theme={null} # Resume the most recent migration ~/resume-migration.sh # Or specify an explicit migration directory ~/resume-migration.sh ~/migration_YYYYMMDD-HHMMSS ``` The script backs up pgcopydb's internal tracking database before resuming and uses `--not-consistent` to allow resuming from a mid-transaction state. It passes `--split-tables-larger-than` to match `run-migration.sh`, since pgcopydb requires catalog consistency between the original run and the resume. > [!WARNING] > > Do not use `pgcopydb --restart`. It wipes the CDC directory and internal tracking databases without cleaning the target database or correcting previous failures. To start over, use `~/target-clean.sh` + `~/drop-replication-slots.sh` + `~/start-migration-screen.sh` instead. ### Starting over If you need to restart the migration from scratch, first clean the target database and replication state: ```bash theme={null} # Wipe all migrated objects from the target ~/target-clean.sh # Remove replication slots from the source ~/drop-replication-slots.sh # Start a fresh migration ~/start-migration-screen.sh ``` The [target-clean script](https://github.com/planetscale/migration-scripts/blob/main/pgcopydb-helpers/target-clean.sh) drops all non-default schemas, materialized views, publications, subscriptions, event triggers, standalone custom types, and partitioned tables. It preserves `pg_catalog`, `information_schema`, and `pscale_extensions`. The script previews what will be dropped and prompts for confirmation before proceeding. ## Troubleshooting ### Checking logs All migration output is captured in `~/migration_YYYYMMDD-HHMMSS/migration.log`. Common diagnostic commands: ```bash theme={null} # Find errors grep ERROR ~/migration_*/migration.log # Check pg_restore errors grep "errors ignored on restore" ~/migration_*/migration.log # Verify extension filtering grep s_depend ~/migration_*/migration.log # Check the exit code (last line of log, success = "Exit code: 0") tail -1 ~/migration_*/migration.log ``` ### Common issues **Preflight check shows `wal_level` is not `logical`:** Changing `wal_level` requires a database restart. On managed services like RDS, update the parameter group and reboot the instance. On self-managed PostgreSQL, update `postgresql.conf` and restart. **Extension filtering not working:** If extension-owned objects are still being copied despite `[exclude-extension]` entries in `filters.ini`, verify that the extension names match exactly and that the extensions are actually installed on the source database. **Migration fails with restore errors:** pgcopydb tolerates up to 10 `pg_restore` errors by default. Check the migration log for specific errors. Common causes include unsupported extensions or objects that depend on excluded schemas. Adjust your `filters.ini` and restart. **Replication slot already exists:** If a previous migration attempt left a replication slot, run `~/drop-replication-slots.sh` to clean it up before starting a new migration. **Tables missing data after resume:** If the resume did not recover correctly, run `~/target-clean.sh` and `~/drop-replication-slots.sh`, then start the migration over with `~/start-migration-screen.sh`. ## Next steps After your migration is complete: * Review [Query Insights](/docs/postgres/monitoring/query-insights) to monitor query performance on PlanetScale. * Set up [Metrics](/docs/postgres/monitoring/metrics) and alerting for your new database. * Review [Schema recommendations](/docs/postgres/monitoring/schema-recommendations) for optimization suggestions. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres Imports - WAL Streaming Source: https://planetscale.com/docs/postgres/imports/postgres-migrate-walstream > [!NOTE] > > Before beginning your migration, we recommend running our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. [Write-Ahead Logging (WAL)](https://www.postgresql.org/docs/current/wal-intro.html) streaming provides a near-zero downtime migration method by continuously replicating changes from your source PostgreSQL database to PlanetScale Postgres using [logical replication](https://www.postgresql.org/docs/current/logical-replication.html). > [!NOTE] > > Want expert guidance for your migration? PlanetScale's [migration services](https://planetscale.com/migrate) are available to help you plan and execute a smooth, successful move. ## Overview This migration method involves: **Step 1** Setting up logical replication from your source database **Step 2** Creating an initial data snapshot **Step 3** Continuously streaming WAL changes to keep databases synchronized **Step 4** Performing a quick cutover when ready > [!WARNING] > > This method requires administrative access to your source PostgreSQL database to configure replication settings. ## Prerequisites Before starting the migration: * PostgreSQL 10+ on the source database (logical replication support) * Administrative access to source database configuration * Network connectivity between source and PlanetScale Postgres * Connection details for your PlanetScale Postgres database from the console * Ensure the disk on your PlanetScale database has at least 150% of the capacity of your source database. If you are migrating to a PlanetScale database backed by network-attached storage, you can [resize](/docs/postgres/cluster-configuration/cluster-storage) your disk manually by setting the "Minimum disk size." If you are using Metal, you will need to select a size when first creating your database. For example, if your source database is 330GB, you should have at least 500GB of storage available on PlanetScale. * Understanding of your application's write patterns for cutover planning ## Step 1: Configure Source Database ### Enable logical replication on source database: Edit your PostgreSQL configuration (`postgresql.conf`): ```ini theme={null} # Enable logical replication wal_level = logical # Set maximum replication slots max_replication_slots = 10 # Set maximum WAL senders max_wal_senders = 10 # Enable logical replication workers max_logical_replication_workers = 10 ``` ### Configure authentication (`pg_hba.conf`): Add an entry to allow replication connections: ```ini theme={null} # Allow replication connections host replication replication_user source_ip/32 md5 ``` ### Restart PostgreSQL service: ```bash theme={null} # On systemd systems sudo systemctl restart postgresql # On older systems sudo service postgresql restart ``` ## Step 2: Create Replication User Connect to your source database and create a replication user: ```sql theme={null} -- Create replication user CREATE USER replication_user WITH REPLICATION LOGIN; -- Grant necessary permissions GRANT CONNECT ON DATABASE your_database TO replication_user; GRANT USAGE ON SCHEMA public TO replication_user; GRANT SELECT ON ALL TABLES IN SCHEMA public TO replication_user; -- Grant permissions for future tables ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO replication_user; ``` ## Step 3: Create Publication on Source Create a publication for the tables you want to replicate: ```sql theme={null} -- Create publication for all tables CREATE PUBLICATION planetscale_migration FOR ALL TABLES; -- Or create publication for specific tables CREATE PUBLICATION planetscale_migration FOR TABLE table1, table2, table3; -- Verify publication SELECT * FROM pg_publication; ``` ## Step 4: Get PlanetScale Connection Details From your PlanetScale console: **Step 1** Navigate to your PlanetScale Postgres database **Step 2** Go to the "Connect" section **Step 3** Copy the connection details including: * Host * Port * Database name * Username * Password ## Step 5: Create Initial Schema on PlanetScale Export and import the schema structure: ```bash theme={null} # Export schema from source pg_dump -h source-host \ -p source-port \ -U source-username \ -d source-database \ --schema-only \ --no-owner \ --no-privileges \ -f schema.sql # Import schema to PlanetScale psql -h planetscale-host \ -p planetscale-port \ -U planetscale-username \ -d planetscale-database \ -f schema.sql ``` ## Step 6: Create replication slot and capture snapshot Before exporting data, create the replication slot on the source database. This captures a consistent snapshot you will use for export so the dump and replication start point stay aligned with no gaps or duplicates. ```sql theme={null} -- Run on source database in a dedicated session -- This creates the slot and returns a snapshot name SELECT slot_name, snapshot_name, confirmed_flush_lsn FROM pg_create_logical_replication_slot('planetscale_slot', 'pgoutput'); ``` Record the `snapshot_name` value from the output and use it in the next step. > [!WARNING] > > Keep the source database session open after running this command. The snapshot is valid only while the creating session remains open. If the session closes, drop the slot and repeat this step to get a new snapshot. ## Step 7: Export and import initial data Use the snapshot captured in Step 6 when running `pg_dump` so the data export matches the replication slot start position: ```bash theme={null} # Export data using the consistent snapshot from Step 6 pg_dump -h source-host \ -p source-port \ -U source-username \ -d source-database \ --data-only \ --no-owner \ --no-privileges \ --snapshot= \ -f data.sql # Import data to PlanetScale psql -h planetscale-host \ -p planetscale-port \ -U planetscale-username \ -d planetscale-database \ -f data.sql ``` After import completes, close the source session that was holding the snapshot. ## Step 8: Set up logical replication Create the subscription on PlanetScale referencing the existing slot from Step 6: > [!WARNING] > > Always include `copy_data = false` when creating a subscription after a manual data import. Without it, PostgreSQL attempts to copy table data again and can trigger duplicate key errors. ```sql theme={null} -- Create subscription referencing the pre-created slot CREATE SUBSCRIPTION planetscale_subscription CONNECTION 'host=source-host port=source-port dbname=source-database user=replication_user password=replication-password' PUBLICATION planetscale_migration WITH ( copy_data = false, create_slot = false, slot_name = 'planetscale_slot' ); -- Verify subscription is active SELECT subname, subenabled, subslotname FROM pg_subscription; -- Confirm replication is streaming SELECT subname, received_lsn, latest_end_lsn, latest_end_time FROM pg_stat_subscription; ``` ## Step 9: Monitor replication ### Check replication lag: ```sql theme={null} -- On source database SELECT * FROM pg_replication_slots; -- On PlanetScale database SELECT subname, received_lsn, latest_end_lsn, latest_end_time FROM pg_stat_subscription; ``` ### Monitor for conflicts: ```sql theme={null} -- Check for subscription errors SELECT * FROM pg_stat_subscription WHERE last_msg_failure_time IS NOT NULL; ``` ## Step 10: Prepare for cutover ### Verify data consistency: ```sql theme={null} -- Compare row counts between source and target -- Run on both databases SELECT schemaname, tablename, n_tup_ins as estimated_rows FROM pg_stat_user_tables ORDER BY schemaname, tablename; ``` ### Check replication lag: Ensure replication lag is minimal (ideally under 1 second) before cutover. ## Step 11: Handle sequences Logical replication synchronizes table data, but it does not synchronize the `nextval` values for [sequences](https://www.postgresql.org/docs/current/sql-createsequence.html) in your database. Sequences are often used for auto-incrementing IDs, so update them on PlanetScale before switching application traffic. You can see all sequences and their corresponding `nextval` values on your source database using this command: ```sql theme={null} SELECT schemaname, sequencename, last_value + increment_by AS next_value FROM pg_sequences; ``` Example output: ``` schemaname | sequencename | next_value ------------+------------------+------------ public | users_id_seq | 105 public | posts_id_seq | 1417 public | followers_id_seq | 3014 ``` In this example, the database has three sequences used for auto-incrementing primary keys. The `nextval` for `users_id_seq` is 105, the `nextval` for `posts_id_seq` is 1417, and the `nextval` for `followers_id_seq` is 3014. If you run the same query on your PlanetScale database, the sequence values may be behind the source database. If you switch traffic to PlanetScale while sequences are behind, inserts can fail with duplicate key errors: ```bash theme={null} ERROR: duplicate key value violates unique constraint "XXXX" DETAIL: Key (id)=(ZZZZ) already exists. ``` Before switching over, advance these sequences so the `nextval` values produced on PlanetScale are greater than any values previously produced on the source database. A simple approach is to run this query on your source database: ```sql theme={null} SELECT 'SELECT setval(''' || schemaname || '.' || sequencename || ''', ' || (last_value + 10000) || ');' AS query FROM pg_sequences; ``` This generates queries that advance each sequence by 10,000: ```sql theme={null} query -------------------------------------------------- SELECT setval('public.users_id_seq', 10104); SELECT setval('public.posts_id_seq', 11416); SELECT setval('public.followers_id_seq', 13013); ``` Execute the generated queries on your PlanetScale database. Advance each sequence far enough that the source database will not reach those `nextval` values before you switch your primary database to PlanetScale. For tables with a high insertion rate, increase the offset to a larger value, such as 100,000 or 1,000,000. ## Step 12: Perform cutover When ready to switch to PlanetScale Postgres: **Step 1** **Stop application writes** to the source database **Step 2** **Wait for replication to catch up** (monitor lag) **Step 3** **Update application connection strings** to point to PlanetScale **Step 4** **Start application** with new connection **Step 5** **Monitor** for any issues ### Verify cutover success: ```sql theme={null} -- Check that latest data is present SELECT count(*), max(updated_at) FROM your_main_table; ``` ## Step 13: Cleanup (after successful cutover) ### Drop subscription on PlanetScale: ```sql theme={null} DROP SUBSCRIPTION planetscale_subscription; ``` ### Drop publication on source: ```sql theme={null} DROP PUBLICATION planetscale_migration; ``` ## Troubleshooting ### Common Issues: **Replication slot conflicts:** ```sql theme={null} -- Check existing slots SELECT * FROM pg_replication_slots; -- Drop unused slots SELECT pg_drop_replication_slot('slot_name'); ``` **Permission errors:** * Verify replication user has correct permissions * Check pg\_hba.conf configuration * Ensure network connectivity **Large transaction delays:** * Monitor for long-running transactions on source * Consider breaking large operations into smaller batches **Subscription conflicts:** ```sql theme={null} -- Check subscription worker status SELECT * FROM pg_stat_subscription; -- Restart subscription if needed ALTER SUBSCRIPTION planetscale_subscription DISABLE; ALTER SUBSCRIPTION planetscale_subscription ENABLE; ``` **Duplicate key errors immediately after subscription creation:** The subscription was likely created without `copy_data = false` after manual data import. Drop the subscription immediately to stop retry loops: ```sql theme={null} DROP SUBSCRIPTION planetscale_subscription; ``` Clean up leaked replication origins: ```sql theme={null} DO $$ DECLARE r RECORD; BEGIN FOR r IN SELECT roname FROM pg_replication_origin WHERE roname NOT IN ( SELECT 'pg_' || oid::text FROM pg_subscription ) LOOP PERFORM pg_replication_origin_drop(r.roname); END LOOP; END; $$; ``` Then drop the replication slot on the source, repeat Steps 6-8 with a fresh snapshot, and recreate the subscription with `copy_data = false`. **`could not find free replication state slot` errors:** This is commonly caused by the duplicate key crash loop above. Each unclean worker exit can leak a replication origin until `max_active_replication_origins` is exhausted. Resolve the duplicate key issue first and run the origin cleanup. If the error persists, contact PlanetScale support. ## Performance Considerations 1. **Network bandwidth**: Ensure sufficient bandwidth for initial sync and ongoing replication 2. **Disk I/O**: Monitor disk usage on both source and target during replication 3. **Replication lag**: Keep lag minimal by optimizing source database performance 4. **Conflict resolution**: Understand how PostgreSQL handles replication conflicts ## Schema Considerations Before migration, review: > **[PostgreSQL version compatibility](/docs/postgres/imports/postgres-imports#postgresql-version-compatibility)** > **[Extension support limitations](/docs/postgres/imports/postgres-imports#extension-support)** > **[Third-party enhancement restrictions](/docs/postgres/imports/postgres-imports#third-party-enhancements-and-tools)** ## Next Steps After successful migration: **Step 1** Monitor replication performance and lag **Step 2** Test application functionality thoroughly **Step 3** Set up monitoring and alerting for the new database **Step 4** Plan for ongoing maintenance and optimization For simpler migrations or if you don't have administrative access to your source database, consider the [pg\_dump/restore method](./postgres-migrate-dumprestore). For more complex scenarios, explore [Amazon DMS](./postgres-migrate-dms). If you encounter issues during migration, please [reach out to support](https://planetscale.com/contact?initial=support) for assistance. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Migrate from Supabase to PlanetScale Source: https://planetscale.com/docs/postgres/imports/supabase Use this guide to migrate an existing Supabase database to PlanetScale Postgres. > [!NOTE] > > Before beginning your migration, we recommend running our [migration assessment tool](https://planetscale.com/liftoff) for instant feedback on migration complexity, potential blockers, and the recommended migration path. Use this guide to migrate an existing Supabase database to PlanetScale Postgres. This guide will cover a no-downtime approach to migrating using Postgres logical replication. If you are willing to tolerate downtime during a maintenance window, you may also use [`pg_dump` and restore](/docs/postgres/imports/postgres-migrate-dumprestore). The `pg_dump`/restore approach is simpler, but is only for applications where downtime is acceptable. These instructions work for all versions of Postgres that support logical replication (version 10+). If you have an older version you want to bring to PlanetScale, [contact us](https://planetscale.com/contact?initial=support) for guidance. Before beginning a migration, you should check our [extensions documentation](/docs/postgres/extensions) to ensure that all of the extensions you rely on will work on PlanetScale. As an alternative to this guide, you can also try our [Postgres migration scripts](https://github.com/planetscale/migration-scripts/tree/main/postgres-direct). These allow you to automate some of the manual steps that we describe in this guide. > [!NOTE] > > Want expert guidance for your migration? PlanetScale's [migration services](https://planetscale.com/migrate) are available to help you plan and execute a smooth, successful move. ## 1. Prepare your PlanetScale database Create a new database in the [PlanetScale dashboard](https://app.planetscale.com) or using the [PlanetScale CLI](/docs/cli). A few things to check when configuring your database: * Ensure you select the correct cloud region. You typically want to use the same region that you deploy your other application infrastructure to. * Since Supabase uses Postgres, you'll also want to create a Postgres database in PlanetScale. * Choose the best storage option for your needs. For applications needing high-performance and low-latency I/O, use [PlanetScale Metal](/docs/metal). For applications that need more flexible storage options or smaller compute instances, choose "Elastic Block Storage" or "Persistent Disk." * Choose between aarch64 and x86-64 architecture. If you don't know which to choose, `aarch64` is a good default choice. ![Create a new PlanetScale Postgres database](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/imports/image.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=ed12dab5fee65e01a22bc3bf8b7e3e04) Once the database is created and ready, navigate to your dashboard and click the "Connect" button. ![Connect to a PlanetScale Postgres database](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/imports/migration-dashboard-connect.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=9bc8cdd874472873d94478ac18cb08b6) From here, follow the instructions to create a new default role. This role will act as your admin role, with the highest level of privileges. Though you may use this one for your migration, we recommend you use a separate role with lesser privileges for your migration and general database connections. To create a new role, navigate to the [Role management page](/docs/postgres/connecting/roles). Click "New role" and give the role a memorable name. By default, `pg_read_all_data` and `pg_write_all_data` are enabled. In addition to these, enable `pg_create_subscription` and `postgres`, and then create the role. ![New Postgres role privileges](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/imports/image3.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=b6592510d75100ab2e87208d2aacec2e) Copy the password and all other connection credentials into environment variables for later use: ``` PLANETSCALE_USERNAME=pscale_api_XXXXXXXXXX.XXXXXXXXXX PLANETSCALE_PASSWORD=pscale_pw_XXXXXXXXXXXXXXXXXXXXXXX PLANETSCALE_HOST=XXXX.pg.psdb.cloud PLANETSCALE_DBNAME=postgres ``` We also recommend that you increase `max_worker_processes` for the duration of the migration in order to speed up data copying. Go to the "Parameters" tab of the "Clusters" page: ![Configure parameters](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/imports/image4.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=314a03ba73e5cdf908e0ccedb0123e95) On this page, increase this value from the default of `4` to `10` or more: ![Configure max worker processes](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/imports/image5.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=d29190a32bd982451a5b52ecd2211097) You can decrease these values after the migration is complete. ## 2. Configure disk size on PlanetScale If you are importing into a database backed by network-attached storage, you must configure your disk in advance to ensure your database will fit. Though we support disk autoscaling for these, AWS and GCP limit how frequently disks can be resized. If you don't ensure your disk is large enough for the import in advance, it will not be able to resize fast enough for a large data import. To configure this, navigate to "Clusters" and then the "Storage" tab: ![Storage configuration min size](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/imports/storage-configuration-min-size.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=41d36935a4bf7fc12799794d0cde1a51) On this page, adjust the "Minimum disk size." You should set this value to at least 150% of the size of the database you are migrating. For example, if the database you are importing is 330 GB, you should set your minimum disk size to at least 500 GB. The 50% overhead is a baseline to account for: 1. Data growth during the import process 2. Table and index bloat that can occur during the import process Schemas with many secondary indexes often need more headroom. Each index is built during the copy phase, which increases copy time and temporary disk usage. If your source database has a large number of indexes, provision more than 150% of your source size. Bloat can be mitigated after import with careful [VACUUMing](https://www.postgresql.org/docs/current/sql-vacuum.html) or an extension like [pg\_squeeze](/docs/postgres/extensions/pg_squeeze), but is difficult to avoid during the migration itself. When ready, queue and apply the changes. You can check the "Changes" tab to see the status of the resize: ![Confirm disk size change](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/imports/confirm-disk-size-change.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=3642cee5a9eddb534263ecd8a70d81eb) Wait for it to indicate completion. If you are importing to a Metal database, you must choose a disk size when first creating your database. You should launch your cluster with a disk size at least 50% larger than the storage used by your current source database (150% of the existing total). As an example, if you need to import a 330 GB database onto a PlanetScale `M-160`, there are three storage sizes available: ![Metal disk size](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/imports/metal-disk-size.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=18c7b2f2b9dbdfe9d20b22fbd3e634c7) You should use the largest, 1.25TB option during the import. After importing and cleaning up table bloat, you may be able to downsize to the 468 GB option. ## 3. Enable IPv4 direct connections in Supabase In Supabase, logical replication to external sources requires direct connections. Direct IPv4 connections are not enabled by default. If you have not enabled them yet, go to your project dashboard in Supabase and click the "Connect" button: ![Supabase dashboard](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/imports/image6.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=3457b2e1f89ee76070960a7eb6927c4e) In the connection modal, click "IPv4 add-on." ![Supabase direct](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/imports/image7.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=0479c0d05e9c16e1b352d345891e77b2) In the menu that appears, enable the IPv4 add-on: ![Supabase IPV4](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/postgres/imports/image8.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=da38d6f1b5729ac4371dd2aced55c9e8) Supabase notes that enabling this might incur downtime. Take that into account when planning your migration. Logical replication requires a direct connection to the primary Postgres instance. Use direct connection credentials (not pooled connection credentials) for the rest of this guide. > [!WARNING] > > Use direct connection host and port `5432` for `pg_dump`, `CREATE PUBLICATION`, and `CREATE SUBSCRIPTION`. Supabase pooled endpoints are not suitable for logical replication. ## 4. Copy schema from Supabase to PlanetScale Before we begin migrating data, we first must copy the schema from Supabase to PlanetScale. We do this as a distinct set of steps using `pg_dump`. > [!WARNING] > > You should not make any schema changes during the migration process. You may continue to select, insert, update, and delete data, keeping your application fully online during this process. For these instructions, you'll need to connect to your Supabase role that has permissions to create replication publications and read all data. You also must use a direct IPv4 connection. Your default role that was generated by Supabase when you first created your database should suffice here. We will assume that the credentials for this user and other connection info are stored in the following environment variables. ``` SUPABASE_USERNAME=XXXX SUPABASE_PASSWORD=XXXX SUPABASE_HOST=XXX SUPABASE_PORT=5432 SUPABASE_DBNAME=XXX ``` Run the below command to take a snapshot of the full schema of the `$SUPABASE_DBNAME` that you want to migrate: ``` PGPASSWORD=$SUPABASE_PASSWORD \ pg_dump -h $SUPABASE_HOST \ -p $SUPABASE_PORT \ -U $SUPABASE_USERNAME \ -d $SUPABASE_DBNAME \ --schema-only \ --no-owner \ --no-privileges \ --schema=public \ -f schema.sql ``` This saves the schema into a file named `schema.sql`. > [!NOTE] > > The above command will dump the tables only for the `public` schema. If you want to include other schemas in the migration, you can repeat these steps for each, or customize the commands to dump multiple schemas at once. The schema then needs to be loaded into your new PlanetScale database: ```bash theme={null} PGPASSWORD=$PLANETSCALE_PASSWORD \ psql -h $PLANETSCALE_HOST \ -p 5432 \ -U $PLANETSCALE_USERNAME \ -d $PLANETSCALE_DBNAME \ -f schema.sql ``` In the output of this command, you might see some error messages of the form: ```bash theme={null} psql:schema.sql:LINE: ERROR: DESCRIPTION ``` You should inspect these to see if they are of any concern. You can [reach out to our support](https://planetscale.com/contact?initial=support) if you need assistance at this step. ## 5. Set up logical replication We now must create a `PUBLICATION` on Supabase that the PlanetScale database can subscribe to for data copying and replication. This example shows how to create a publication that only publishes changes to tables in the `public` schema of your Postgres database. You can adjust the commands if you want to do so for a different schema, or have multiple schemas to migrate. ### Configure the Supabase replication role Before creating the publication, check the `statement_timeout` on the Supabase role used for replication. Supabase roles may have a low default (for example, 10 seconds). During the initial table copy, replication reads full table contents from Supabase. Large rows, such as tables with `jsonb` columns, can exceed this limit. When statements time out on the source, table copy can fail repeatedly without cleaning up. This can cause WAL and disk usage to grow on Supabase. On Supabase, disable or raise the timeout for your replication role: ```sql theme={null} ALTER ROLE your_supabase_role SET statement_timeout = 0; ``` You can restore the previous value after migration completes. First, run this command on your Supabase database to get all of the tables in the `public` schema: ```sql theme={null} SELECT 'CREATE PUBLICATION replicate_to_planetscale FOR TABLE ' || string_agg(format('%I.%I', schemaname, tablename), ', ') || ';' FROM pg_tables WHERE schemaname = 'public'; ``` This will generate a query that looks like this: ```sql theme={null} CREATE PUBLICATION replicate_to_planetscale FOR TABLE public.table_1, public.table_2, ... public.table_n; ``` Take this command and execute it on your Supabase database. You should see this if it created correctly: ```sql theme={null} CREATE PUBLICATION ``` We then need to tell PlanetScale to `SUBSCRIBE` to this publication. ```sql theme={null} PGPASSWORD=$PLANETSCALE_PASSWORD psql \ -h $PLANETSCALE_HOST \ -U $PLANETSCALE_USERNAME \ -p 5432 $PLANETSCALE_DBNAME \ -c " CREATE SUBSCRIPTION replicate_from_supabase CONNECTION 'host=$SUPABASE_HOST port=$SUPABASE_PORT dbname=$SUPABASE_DBNAME user=$SUPABASE_USERNAME password=$SUPABASE_PASSWORD' PUBLICATION replicate_to_planetscale WITH (copy_data = true);" ``` Data copying and replication will begin at this point. ### Monitoring your migration Once the subscription is created, PlanetScale begins copying data from Supabase. This section explains what is happening internally and how to track progress. #### What happens during the copy When the subscription is created with `copy_data = true`, PostgreSQL proceeds in two phases: **Initial table sync (copy phase)** PostgreSQL spawns tablesync workers on the PlanetScale side. Each worker opens a replication connection to Supabase and copies one table at a time using a consistent snapshot taken at subscription creation. Up to `max_sync_workers_per_subscription` tables are copied in parallel (the default is 2; we recommend increasing `max_worker_processes` as described above to allow more parallelism). Because your schema was loaded before the subscription was created, all indexes are live during this phase. Expect elevated CPU on PlanetScale for the duration — this is normal. The larger and more heavily indexed your tables, the longer this phase takes. > [!NOTE] > > For schemas with many secondary indexes, one approach is to load only primary key and foreign key indexes in the initial `schema.sql`, run the copy, then create remaining indexes after all tables reach the `ready` state. This reduces CPU usage, copy time, and disk overhead during the sync phase. **Steady-state replication (streaming phase)** Once all tables are copied, the tablesync workers exit and a single apply worker takes over, streaming WAL changes from Supabase in real time. CPU usage will drop significantly at this point. This is the state you want to reach and maintain until cutover. #### Tracking table sync progress Run this on your PlanetScale database to see the sync state of each table: ```sql theme={null} SELECT srrelid::regclass AS table_name, CASE srsubstate WHEN 'i' THEN 'queued' WHEN 'd' THEN 'copying' WHEN 's' THEN 'catching up' WHEN 'r' THEN 'ready' END AS state FROM pg_subscription_rel ORDER BY srsubstate, table_name; ``` A summary view: ```sql theme={null} SELECT CASE srsubstate WHEN 'i' THEN 'queued' WHEN 'd' THEN 'copying' WHEN 's' THEN 'catching up' WHEN 'r' THEN 'ready' END AS state, count(*) AS tables FROM pg_subscription_rel GROUP BY srsubstate ORDER BY srsubstate; ``` Once all tables show `ready`, the initial copy is complete and you are in steady-state replication. #### Checking replication lag You can compare Log Sequence Numbers (LSNs) between Supabase and PlanetScale to measure how far behind the subscriber is. This is useful both for monitoring progress during the copy phase and for confirming that replication is fully caught up before [cutting over](#7-cutting-over-to-planetscale). Run this on PlanetScale to see the last LSN received by the subscription: ```sql theme={null} SELECT subname, received_lsn, latest_end_lsn, last_msg_send_time, last_msg_receipt_time FROM pg_stat_subscription WHERE subname = 'replicate_from_supabase'; ``` And on Supabase, to see the current WAL position: ```sql theme={null} SELECT pg_current_wal_lsn(); ``` Compare `received_lsn` from PlanetScale against `pg_current_wal_lsn()` from Supabase. During the initial copy phase, these values will diverge — this is expected. Once all tables are in the `ready` state, they should converge quickly. When both values match, the subscriber is fully caught up with the source. #### Troubleshooting **CPU is elevated on PlanetScale** This is expected during the copy phase. Each tablesync worker is writing rows and maintaining indexes simultaneously. CPU will return to normal once all tables reach the `ready` state. **Rows are not appearing on PlanetScale** Check that tablesync workers are active: ```sql theme={null} SELECT pid, backend_type, state FROM pg_stat_activity WHERE backend_type LIKE '%worker%'; ``` If no workers are running, verify that `max_worker_processes` is high enough to accommodate the subscription workers plus any other background processes (autovacuum, etc.). **A table is stuck in the `copying` or `catching up` state** Check for locks on the PlanetScale side that may be blocking the tablesync worker: ```sql theme={null} SELECT pid, wait_event_type, wait_event, query FROM pg_stat_activity WHERE wait_event IS NOT NULL AND backend_type LIKE '%worker%'; ``` **Replication lag is growing after the copy phase** If `received_lsn` is falling further behind `pg_current_wal_lsn()` during steady-state replication, your source may be generating changes faster than the single apply worker can apply them. This is uncommon for typical workloads but can occur with very high write volume. Contact [PlanetScale support](https://planetscale.com/contact?initial=support) if you observe this. **Supabase disk usage is growing during migration** A low `statement_timeout` on the Supabase replication role can cause table copy to fail repeatedly on large rows. Failed copy operations can leave replication slots active and allow WAL to accumulate on Supabase. See [Configure the Supabase replication role](#configure-the-supabase-replication-role) and verify that table copy is completing on PlanetScale using the [table sync progress](#tracking-table-sync-progress) queries. **Copy phase is slow or PlanetScale disk usage is high** Schemas with many secondary indexes take longer to copy and use more disk during import. See [Configure disk size on PlanetScale](#2-configure-disk-size-on-planetscale) for sizing guidance. For heavily indexed schemas, consider deferring non-primary-key and non-foreign-key indexes until after all tables reach the `ready` state. ## 6. Handling sequences Logical replication is great at migrating all of your data over to PlanetScale. However, logical replication does *not* synchronize the `nextval` values for [sequences](https://www.postgresql.org/docs/current/sql-createsequence.html) in your database. Sequences are often used for things like auto incrementing IDs, so it's important to ensure we update this before you switch your traffic to PlanetScale. You can see all of the sequences and their corresponding `nextval`s on your source Supabase database using this command: ```sql theme={null} SELECT schemaname, sequencename, last_value + increment_by AS next_value FROM pg_sequences; ``` An example output from this command: ``` schemaname | sequencename | next_value ------------+------------------+------------ public | users_id_seq | 105 public | posts_id_seq | 1417 public | followers_id_seq | 3014 ``` What this means is that we have three sequences in our database. In this case, they are all being used for auto-incrementing primary keys. The `nextval` for the `users_id_seq` is 105, the `nextval` for the `posts_id_seq` is 1417, and the `nextval` for the `followers_id_seq` is 3014. If you run the same query on your new PlanetScale database, you'll see something like: ``` schemaname | sequencename | next_value ------------+------------------+------------ public | users_id_seq | 0 public | posts_id_seq | 0 public | followers_id_seq | 0 ``` If you switch traffic over to PlanetScale in this state, you'll likely encounter errors when inserting new rows: ```bash theme={null} ERROR: duplicate key value violates unique constraint "XXXX" DETAIL: Key (id)=(ZZZZ) already exists. ``` Before switching over, you need to progress all of these sequences forward so that the `nextval`s produced will be greater than any of the values previously produced on the source Supabase database, avoiding constraint violations. There are several approaches you can take for this. A simple way to solve the problem is to first run this query on your source Supabase database: ```sql theme={null} SELECT 'SELECT setval(''' || schemaname || '.' || sequencename || ''', ' || (last_value + 10000) || ');' AS query FROM pg_sequences; ``` This will generate a sequence of queries that will advance the `nextval` by 10,000 for each sequence: ```sql theme={null} query -------------------------------------------------- SELECT setval('public.users_id_seq', 10104); SELECT setval('public.posts_id_seq', 11416); SELECT setval('public.followers_id_seq', 13013); ``` You would then execute these on your target PlanetScale database. You need to ensure you advance each sequence far enough forward so that the sequences in the Supabase database will not reach these `nextval`s before you switch your primary to PlanetScale. For tables that have a high insertion rate, you might need to increase this by a larger value (say, 100,000 or 1,000,000). ## 7. Cutting over to PlanetScale Before cutting over, confirm that replication is fully caught up by [checking replication lag](#checking-replication-lag). The `received_lsn` on PlanetScale should match `pg_current_wal_lsn()` on Supabase. If they do not match, the PlanetScale database has not yet applied all changes from Supabase — wait for the values to converge before proceeding. Once replication is caught up, update your application's database connection credentials to point to PlanetScale and deploy. After doing this, new rows written to PlanetScale will not be reverse-replicated to Supabase. Thus, it's important to ensure you are fully ready for the cutover at this point. Once this is complete, PlanetScale is now your primary database! We recommend you keep the old Supabase database around for a few days, in case you discover any data or schemas you forgot to copy over to PlanetScale. If necessary, you can switch traffic back to the old database. However, keep in mind that any database writes that happened with PlanetScale as the primary will not appear on Supabase. This is why it's good to test the database thoroughly before performing the cutover. ## 8. Post-cutover cleanup (optional) After confirming your application is fully running on PlanetScale, you can clean up logical replication resources: 1. On PlanetScale, drop the subscription: ```sql theme={null} DROP SUBSCRIPTION IF EXISTS replicate_from_supabase; ``` 2. On Supabase, drop the publication: ```sql theme={null} DROP PUBLICATION IF EXISTS replicate_to_planetscale; ``` If you no longer need direct external connections from Supabase, you can also disable the IPv4 add-on from the Supabase dashboard. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale GitHub Actions Source: https://planetscale.com/docs/postgres/integrations/github-actions With GitHub Actions, you can automate branch creation and role credentials within your CI workflow. > **[Getting Started](#getting-started)** > **[Authentication](#authentication)** > **[Convert GitHub branch name to PlanetScale branch name](#convert-github-branch-name-to-planetscale-branch-name)** > **[Create a PlanetScale branch](#create-a-planetscale-branch)** > **[Create a role for a branch](#create-a-role-for-a-branch)** > **[Delete a role](#delete-a-role)** > **[Delete a PlanetScale branch](#delete-a-planetscale-branch)** ## Getting started The best way to use PlanetScale within GitHub Actions is via the `pscale` CLI. Use [`planetscale/setup-pscale-action`](https://github.com/planetscale/setup-pscale-action) to make pscale available within your GitHub Actions. ```yaml theme={null} - name: Setup pscale uses: planetscale/setup-pscale-action@v1 ``` The action works with Linux, Windows, and Mac runners. Once installed it will be added to your tool cache for subsequent runs. ## Authentication Authentication for pscale is via service token environment variables. You will need to [create a service token](/docs/cli/service-token). Make sure to give your service token the proper permissions to the database you'll be using in your workflow. Add your `PLANETSCALE_SERVICE_TOKEN_ID` and `PLANETSCALE_SERVICE_TOKEN` to your [Actions secrets](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository). In your Actions workflow, you will need to make the secrets available as environment variables. ```yaml theme={null} - name: Run pscale command env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: pscale database list ``` ## Examples The following examples show how to accomplish common Actions workflows with PlanetScale. In each example, notice that we use secrets for the service token, database and organization names. You will need to set them in your GitHub repository to target your own database. ``` PLANETSCALE_ORG_NAME PLANETSCALE_DATABASE_NAME PLANETSCALE_SERVICE_TOKEN_ID PLANETSCALE_SERVICE_TOKEN ``` ### Convert GitHub branch name to PlanetScale branch name PlanetScale branch names must be lowercase, alphanumeric characters and hyphens are allowed. Since git branch names allow more possibilities, you can use the following code to transform a git branch name into an acceptable PlanetScale branch name. ```yaml theme={null} - name: Rename branch name run: | branch_name=$(printf '%s' "$HEAD_REF" | tr -cd '[:alnum:]-' | tr '[:upper:]' '[:lower:]') echo "PSCALE_BRANCH_NAME=$branch_name" >> "$GITHUB_ENV" env: HEAD_REF: ${{ github.head_ref }} ``` This makes `${{ env.PSCALE_BRANCH_NAME }}` available for use in the rest of the workflow. This is useful to run in any scenario where you are creating a PlanetScale branch to correspond with a git branch. ### Create a PlanetScale branch You can use `pscale branch create` to create a branch that matches your GitHub branch name. ```yaml theme={null} - name: Create branch env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | set +e pscale branch show ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} exit_code=$? set -e if [ $exit_code -eq 0 ]; then echo "Branch exists. Skipping branch creation." else echo "Branch does not exist. Creating." pscale branch create ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --wait --org ${{ secrets.PLANETSCALE_ORG_NAME }} fi ``` Notice that we first check if the branch exists. If it does, we do nothing. Otherwise we create it and pass the `--wait` flag. This is useful when running in CI, as the workflow may run multiple times and you'll want the branch ready before creating roles or running migrations. > [!NOTE] > > New branches created this way start empty (no schema or data). To create a branch that includes schema and data, restore from a [backup](/docs/postgres/backups) with `--restore`. See [Branching](/docs/postgres/branching). ### Create a role for a branch You can use `pscale role create` to generate [credentials](/docs/postgres/connecting/roles) for your database branch. ```yaml expandable theme={null} - name: Generate role for branch env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | response=$(pscale role create ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} "github-actions" \ --inherited-roles postgres \ --ttl 1h \ -f json \ --org ${{ secrets.PLANETSCALE_ORG_NAME }}) id=$(echo "$response" | jq -r '.id') host=$(echo "$response" | jq -r '.access_host_url') username=$(echo "$response" | jq -r '.username') password=$(echo "$response" | jq -r '.password') database_name=$(echo "$response" | jq -r '.database_name') # Set the role ID, allows us to later delete it if wanted. echo "ROLE_ID=$id" >> $GITHUB_ENV # Create the DATABASE_URL database_url="postgresql://$username:$password@$host:5432/$database_name?sslmode=verify-full" echo "DATABASE_URL=$database_url" >> $GITHUB_ENV echo "::add-mask::$database_url" - name: Use the DATABASE_URL in a subsequent step run: | echo "Using DATABASE_URL: $DATABASE_URL" ``` This example shows creating the role and getting back a response in JSON. The JSON is then parsed to create a `DATABASE_URL` which can be used in later steps. > [!NOTE] > > `pscale role create` accepts a `--ttl` flag which lets you limit how long the role is valid. In CI, a short TTL (such as `1h`) is recommended so credentials expire automatically if cleanup does not run. ### Delete a role When your workflow finishes, you can delete the temporary role with `pscale role delete`. ```yaml theme={null} - name: Delete role if: always() env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | pscale role delete ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} ${{ env.ROLE_ID }} \ --force \ --successor postgres \ --org ${{ secrets.PLANETSCALE_ORG_NAME }} ``` `--force` skips the interactive confirmation prompt. `--successor postgres` reassigns any objects owned by the role before deletion. ### Delete a PlanetScale branch You can clean up ephemeral CI branches with `pscale branch delete`. ```yaml theme={null} - name: Delete branch if: always() env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | pscale branch delete ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} \ --force \ --org ${{ secrets.PLANETSCALE_ORG_NAME }} ``` `--force` skips the interactive confirmation prompt so the command can run non-interactively in CI. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Logical replication and Change Data Capture (CDC) Source: https://planetscale.com/docs/postgres/integrations/logical-cdc Change Data Capture (CDC) allows you to track and stream data changes from your PostgreSQL database to external systems in real-time. PlanetScale Postgres supports logical replication, enabling CDC through various tools and integrations. ## What is logical replication? Logical replication is a PostgreSQL feature that replicates data objects and their changes at the logical level, rather than the physical level. It works by: 1. **WAL Level**: The database must be configured with `wal_level = logical` to capture logical changes 2. **Publication**: Creates a logical replication stream on the source database 3. **Replication Slot**: Maintains position in the WAL stream and ensures data consistency 4. **Subscription/Consumer**: External tools consume the logical replication stream This setup enables: * Replication of individual transactions and row changes * Selective replication of specific tables or databases * Cross-version replication between different PostgreSQL versions * CDC integration with external tools and data pipelines ## Configuration requirements ### PlanetScale cluster parameters To enable logical replication on your PlanetScale Postgres cluster, configure these parameters in the **Clusters > Parameters** tab: | Parameter | Required Value | Description | | - | - | - | | `wal_level` | `logical` | Setting the `wal_level` to `logical` enables logical replication, which captures row-level changes in a format that can be flexibly replayed on target systems. | | `max_replication_slots` | 2 x replicas | Set `max_replication_slots` to twice the number of replicas or subscribers. Each replica uses one slot, with the extra slots reserved for operations like failover. | | `max_wal_senders` | 2 x replicas | Likewise, set `max_wal_senders` to twice the number of replicas or targets, and not less than `max_replication_slots`. | | `max_slot_wal_keep_size` | > 4GB | The value of `max_slot_wal_keep_size` should be tuned to ensure you are keeping WAL files long enough for subscribers to consume them, while being sure that the source disk is not overrun by files. A reasonable starting point is > 4GB, and monitor your replication lag, your database's change rate (inserts, updates, deletes), and available disk space. | In addition, for production environments, configure the following to ensure that your CDC stream is maintained during any switchover or failover. Without these settings, manual intervention will be required to restore data pipelines after these events. | Parameter | Required Value | Description | | - | - | - | | `sync_replication_slots` | `on` | Set to `on` to enable synchronization of replication slots to the subcribers. | | `hot_standby_feedback` | `on` | Set to `on` to prevent query conflicts during replication. | If your CDC client sleeps for long periods, a failover can proceed before slots are synced, which risks losing the logical slot. Use `logical_slot_sync_timeout` to bound how long failover waits for slot sync in that scenario. For more details, see [Postgres High Availability with CDC](https://planetscale.com/blog/postgres-ha-with-cdc). | Parameter | Default | Description | | - | - | - | | `logical_slot_sync_timeout` | `300s` | Time to wait for logical slot sync on standbys before failover proceeds. | ### Verify configuration After setting these configuration parameters in the dashboard, you can verify them in the CLI. For example, to verify the WAL level: ```sql theme={null} SHOW wal_level; ``` The result should show `logical`: ```sql theme={null} wal_level ----------- logical ``` ### CDC tool configuration Ensure your CDC tool is configured properly: * **Airbyte**: Ensure replication slots are created with failover support ([Setup Guide](https://docs.airbyte.com/integrations/sources/postgres)) * **AWS DMS**: Manually create failover-enabled replication slots before configuring DMS ([Setup Guide](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html)) * **ClickHouse**: See ClickPipes documentation for PlanetScale configuration ([Setup Guide](https://clickhouse.com/docs/en/integrations/clickpipes/postgres/source/planetscale)) * **Debezium**: Configure connector to use failover-enabled replication slots ([Setup Guide](https://debezium.io/documentation/reference/stable/connectors/postgresql.html)) * **Fivetran**: Create your own replication slot with `failover = true` ([Setup Guide](https://fivetran.com/docs/connectors/databases/postgresql)) > [!WARNING] > > Some CDC tools create replication slots automatically. **You must verify** that any auto-created slots have `failover = true` enabled, or manually create the slots yourself with the proper configuration. ## Create and manage users For production CDC deployments, create a dedicated replication role. You can do this through PlanetScale (API, dashboard, CLI, or Terraform) or with SQL as the default user. ### Create a role with REPLICATION via PlanetScale You can create a PlanetScale [user-defined role](/docs/postgres/connecting/roles) with the `REPLICATION` attribute enabled. When creating the role: * Include the `postgres` inherited role. * Enable the `REPLICATION` attribute on the role. In the dashboard, select **WITH REPLICATION** when creating a role (Settings > Roles, or the Connect dialog). Through the API, pass `with_replication: true` when creating a role with `inherited_roles` that includes `postgres`. Roles created this way appear in your database settings and can be managed through the dashboard, API, and CLI. Use this role's credentials to connect from your subscriber/consumer side. ### Create a user with SQL Alternatively, log in as the default user and create a dedicated replication user with SQL: ```sql theme={null} -- Create dedicated CDC user CREATE USER cdc_user WITH REPLICATION PASSWORD 'strong_password'; ``` The `WITH REPLICATION` clause allows the user to connect to the server using the replication protocol, to create and use replication slots, to stream WAL files, and to perform the logical decoding operations. You will configure this user to connect from your subscriber/consumer side. Because of the edge connection settings, to log in as this user, add the branch ID after the username, like this: ```sql theme={null} cdc_user.12345678 ``` Some target systems, like Fivetran, will need to match on the exact login username for some operations after it has logged in. As a workaround for those cases, also create a user with the name + branch ID, like this: ```sql theme={null} -- Create dedicated CDC user CREATE USER "cdc_user.12345678" WITH REPLICATION PASSWORD 'strong_password'; ``` To find the branch ID, look at the Settings > Roles page and observe the branch ID in roles that were created in the UI. Roles created via SQL do not appear on the Settings > Roles page and must be managed directly in Postgres. ## Create and manage replication streams ### Create a replication slot Using the dedicated replication role, create logical replication slots with the `failover` option enabled to preserve the slots during any switchover or failover events: ```sql theme={null} SELECT pg_create_logical_replication_slot( 'my_cdc_slot', -- slot_name 'pgoutput', -- plugin false, -- temporary false, -- two_phase true -- failover = true (REQUIRED) ); ``` > [!NOTE] > > If PlanetScale detects a logical replication slot with `failover = false`, you will be notified and given 3 days to update the configuration. If the slot is not reconfigured with `failover = true` within that window, PlanetScale will resume performing planned failovers as normal. ### Enabling failover on an existing replication slot If you created a logical replication slot before enabling failover support on your cluster, the slot will not survive a switchover or failover on its own. Use the steps below to enable failover on an already-created slot without having to drop and recreate it. > [!WARNING] > > Before following these steps, confirm the following in your cluster's **Parameters** tab: > > * `hot_standby_feedback` is set to `on` > * `sync_replication_slots` is set to `on` > > Altering the slot's failover setting without these prerequisites in place will not protect it during a switchover or failover. **Step 1** Using the [default role](/docs/postgres/connecting/roles#default-role) for the branch, connect through a replication connection. Append `&replication=database` to the connection URI, or pass `replication=database` in a psql connection string: ```bash theme={null} psql 'host=xxxxxxxxxx-useast1-1.horizon.psdb.cloud \ port=5432 \ user=postgres.xxxxxxxxxx \ password=pscale_pw_xxxxxxxxxxxxxxxxxx \ dbname=my_database \ replication=database \ sslmode=verify-full' ``` **Step 2** Query the slot name and its current consumer's pid: ```sql theme={null} SELECT slot_name, active_pid, failover FROM pg_replication_slots WHERE slot_type = 'logical' AND failover is false; ``` **Step 3** Alter the replication slot to enable failover: ```sql theme={null} ALTER_REPLICATION_SLOT (FAILOVER true); ``` > [!NOTE] > > This command will hang until the slot's current consumer disconnects. Move on to the next step to disconnect it. **Step 4** In a new session to the database using the default role, terminate the slot's current consumer by looking up its `active_pid` from the `slot_name` in step 2: ```sql theme={null} SELECT pg_terminate_backend(active_pid) FROM pg_replication_slots WHERE slot_name = '' AND active_pid IS NOT NULL; ``` **Step 5** The `ALTER_REPLICATION_SLOT` command from step 3 should now complete. Confirm the slot's failover flag is enabled: ```sql theme={null} SELECT slot_name, active_pid, failover FROM pg_replication_slots WHERE slot_name = ''; ``` **Step 6** Your CDC consumer should reconnect automatically on its next resync window. ### Create initial publication Some CDC tools require you to create publications to specify which tables to replicate. You will need to do this as the owner of the tables or the default role. This example uses the [default role](/docs/postgres/connecting/roles#default-role). ```sql theme={null} CREATE PUBLICATION my_cdc_publication FOR TABLE table1, table2; ``` ### Add tables to a publication Currently, tables must be added to the publication individually or as a comma-delimited list. Remember to update your publication when adding new tables that should be replicated. ```sql theme={null} ALTER PUBLICATION my_cdc_publication ADD TABLE table3; ``` ### Replica identity configuration For complete change tracking of both row values before and after changes (as well as to support any tables without a primary key), set the replica identity to FULL: ```sql theme={null} -- Enable full replica identity for complete change tracking ALTER TABLE table1 REPLICA IDENTITY FULL; ALTER TABLE table2 REPLICA IDENTITY FULL; ALTER TABLE table3 REPLICA IDENTITY FULL; ``` ### Verify publications Issue the following to see active publications with tables. Do this as the default user. ```sql theme={null} SELECT p.pubname, c.relname AS tablename FROM pg_publication p JOIN pg_publication_rel pr ON p.oid = pr.prpubid JOIN pg_class c ON pr.prrelid = c.oid; ``` ## Monitoring and troubleshooting ### PlanetScale metrics for CDC monitoring PlanetScale provides built-in metrics that are essential for monitoring your CDC setup. Access these through your **Metrics dashboard** to track replication health and performance: | Metric Category | Key Indicators for CDC | What to Monitor | | - | - | - | | **WAL archival rate** | Success/Failed counts | Monitor failed WAL archival attempts that could impact CDC streams | | **WAL archive age** | Seconds behind | Age of oldest unarchived WAL - should be under 60 seconds for healthy CDC | | **WAL storage** | Storage usage in MB | Track WAL disk usage; high usage may indicate CDC consumers falling behind | | **Replication lag** | Lag in seconds | Monitor delay between primary and replicas; high lag may indicate CDC consumer performance issues | | **Transaction rate** | Transactions per second | Track database workload intensity affecting CDC processing | | **Memory** | RSS and Memory mapped | Monitor memory pressure that could impact logical decoding performance | | **Primary Storage Usage** | MB disk utilization | Monitor disk utilization to be sure WAL files are being consumed quickly enough | > [!NOTE] > > For detailed information about interpreting these metrics, see the [Cluster > Metrics](/docs/postgres/monitoring/metrics) documentation. ### Monitoring replication lag Check replication slot lag. The replication\_lag column shows how much WAL data the publisher is keeping because the subscriber has not confirmed or processed it yet. This value should be kept well below `max_wal_size`. ```sql theme={null} SELECT slot_name, database, active, restart_lsn, confirmed_flush_lsn, pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS replication_lag FROM pg_replication_slots WHERE slot_type = 'logical'; ``` ### WAL retention and disk usage Monitor WAL retention to prevent disk space issues. This is another way to see similar information, and will include any PlanetScale HA replicas. ```sql theme={null} SELECT slot_name, pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained_wal FROM pg_replication_slots; ``` ### Common issues **Issue**: WAL disk space growing rapidly\ **Cause**: Inactive or slow CDC consumers\ **Solution**: Remove unused slots or troubleshoot slow consumers **Issue**: Failover breaks CDC stream\ **Cause**: Replication slot not properly synchronized\ **Solution**: Verify failover configuration and slot synchronization status ## Best practices 1. **Always enable failover**: **Never** deploy CDC to production without `failover = true` on replication slots and proper PlanetScale cluster configuration 2. **Verify configuration**: Double-check that both your CDC tool and PlanetScale settings are properly configured before going live 3. **Test failover scenarios**: Test actual failover events in staging environments to ensure your configuration works 4. **Regular monitoring**: Monitor replication lag, WAL retention, and slot synchronization status 5. **Slot cleanup**: Remove unused logical replication slots to prevent WAL accumulation 6. **CDC client resilience**: Ensure CDC clients can handle connection interruptions gracefully ## Security considerations * Logical replication exposes table data - ensure proper access controls * Use dedicated database users with minimal required privileges for CDC * Consider network security when streaming to external systems * Monitor for unauthorized replication slots For more information about cluster configuration parameters, see the [Cluster configuration parameters](/docs/postgres/cluster-configuration/parameters) documentation. # Anomalies Source: https://planetscale.com/docs/postgres/monitoring/anomalies Anomalies are defined as periods with a substantially elevated percentage of slow-running queries. ## Overview PlanetScale Insights continuously analyzes your query performance to establish a baseline for expected performance. When a high enough percentage of queries are running more slowly than the baseline expectation, we call this an anomaly. ## Using the Anomalies graph The graph shown on the Anomalies page shows the percentage of queries executing slower than the 97.7th (2-sigma) percentile baseline on the y-axis and the period of time on the x-axis. The "expected" line shows the percent of queries that are statistically expected in a database with uniform query performance over time. Slight deviations from the expected value are normal. Only substantial and sustained deviations from the expected value are considered an anomaly. ![Database health graph showing two anomalies](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/anomalies/database-health-graph.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=5b6c409b85f1638ae8a318e4d3c64453) Any periods where your database was unhealthy will be highlighted with a red icon representing a performance anomaly. Each anomaly on the graph is clickable. Clicking on it will pull up more details about it in the table below the graph, such as: duration, percentage of increase, and when the anomaly occurred. We also overlay any deploy requests that happened during that period over the anomaly graph. On top of this, we also surface any impact to the following: * The query that triggered the anomaly * CPU utilization * Memory * IOPS * Queries per second * Rows written per second * Rows read per second * Errors per second ## Anomalies vs query latency You may notice a correlation between some areas in the query latency graph and the anomalies graph. Conversely, in some cases, you may see a spike in query latency, but no corresponding anomaly. Increased query latency *can* be indicative of an anomaly, but not always. Query latency may increase and decrease in ways that don't always indicate an actual problem with your database. For example, you may run a weekly report that consists of a few slow-running queries. These queries are always slow. Every week, you'll see a spike on your query latency graph during the time that your weekly report is generated, but not on your anomaly violations graph. The queries are running at their *expected* latency, so this is not considered an anomaly. ## What should I do if my database has an anomaly? The purpose of the Anomalies page is to show you relevant information so you can determine what caused an anomaly and correct the issue. Let's look at an example scenario. You deploy a feature in your application that contains a new query. This query is slow, running frequently, and is hogging database resources. This new slow query is running so often that it's slowing down the rest of your database. Because your other queries are now running slower than expected, an anomaly is triggered. In this case, we will surface the new slow-running query so that you can find ways to optimize it to free up some of the resources it's using. Adding an index will often solve the problem. You can test this by adding the index, creating a deploy request, and deploying it. If it's successful, you'll quickly see the anomaly end. On the other hand, an anomaly does not necessarily mean you need to take any action. One common example where you may see an anomaly is in the case of large active-running backups. In this case, we will tell you that a backup was running during the time of the anomaly. > [!NOTE] > > Even if it causes an anomaly, we do not recommend you turn off backups to prevent possible data loss. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Authentication attempts Source: https://planetscale.com/docs/postgres/monitoring/authentication-attempts Export a report of every attempt to authenticate against your Postgres databases, including the real client IP, the credential used, and whether access was allowed or denied. Authentication attempt reports provide a record of attempts to authenticate against your Postgres databases. You can use them to review connection activity, troubleshoot authentication failures, investigate unexpected access, or export authentication data for further analysis. Each record includes information such as the client IP address, credential used, targeted database and branch, and whether the attempt was allowed or denied. The `source_ip` in the report is the address of the client that connected to PlanetScale. This is especially useful for connections routed through a pooler, where Postgres server logs may show a PlanetScale infrastructure address instead of the original client address. > [!NOTE] > > Generating a report is limited to [Organization Administrators](/docs/security/access-control#organization-administrator). Service tokens cannot generate reports. ## Generate a report Use the [`pscale` CLI](/docs/cli/audit-log) to generate and download a report. Specify a time window and, optionally, filters: ```bash theme={null} pscale audit-log auth-attempts download --since 24h --output auth-attempts.zip ``` ```bash theme={null} Generating authentication-attempt export for acme... Successfully downloaded auth attempts (csv) to auth-attempts.zip ``` The download link expires 24 hours after the report is generated. If you need to download the report again after that, generate a new report. ## What's in the archive Every archive contains exactly two files: | File | Contents | | :- | :- | | `auth-attempts.csv` (or `.jsonl`, `.parquet`) | One row per authentication attempt | | `manifest.json` | The time window, filters, format, and other metadata for the report | ### The data | Column | Description | | :- | :- | | `occurred_at` | When the attempt happened, UTC, to the millisecond | | `source_ip` | The IP address the client connected from | | `source_port` | The client's source port | | `organization_id`, `organization_name` | The organization | | `database_id`, `database_name` | The database the attempt targeted | | `branch_id`, `branch_name` | The branch the attempt targeted | | `username` | The credential used | | `startup_database` | The database named in the connection's startup message | | `outcome` | `allow` or `deny` | | `failure_reason` | Why it was denied. Empty when the attempt succeeded | | `sqlstate` | The Postgres SQLSTATE returned on failure | | `backend_route` | `postgres` or `pgbouncer`, depending on the connection path | | `auth_protocol` | The wire protocol used | Denials carry one of these reasons: | `failure_reason` | Meaning | | :- | :- | | `bad_password` | The credential exists but the password did not match | | `unknown_user` | No such credential | | `authorization_failed` | Authenticated, but not permitted to proceed | | `ip_not_allowed` | Blocked by an IP allow list | | `other` | Anything else | ### The manifest `manifest.json` records how the report was generated: ```json theme={null} { "schema_version": 1, "format": "csv", "generated_at": "2026-08-10T14:46:20Z", "organization": { "id": "...", "name": "acme" }, "window": { "start_at": "2026-08-09T00:00:00Z", "end_at": "2026-08-10T00:00:00Z", "semantics": "[start_at,end_at)" }, "filters": { "source_ips": ["203.0.113.10/32"] }, "resolved_branch_public_ids": ["..."] } ``` Keep the manifest with the exported data when storing or sharing a report. It records the time window and filters used to produce the export. The report window uses `[start_at,end_at)` semantics: `start_at` is inclusive and `end_at` is exclusive. ## Example queries ### Filter by source IP Pass a single address or a whole CIDR range: ```bash theme={null} pscale audit-log auth-attempts download --since 7d \ --source-ip 203.0.113.10 --output suspect-ip.zip ``` ```bash theme={null} pscale audit-log auth-attempts download --since 7d \ --source-ip 203.0.113.0/24 --source-ip 198.51.100.0/24 --output suspect-range.zip ``` ### Filter by credential Use `--username` to find attempts made with a particular credential. For example, to return only denied attempts: ```bash theme={null} pscale audit-log auth-attempts download --since 7d \ --username app_readonly --outcome deny --output credential.zip ``` ### Export a time window for analysis Omit filters to export all authentication attempts in a time window. For larger datasets, Parquet can be useful for analysis with tools such as DuckDB: ```bash theme={null} pscale audit-log auth-attempts download \ --start-at 2026-07-29T00:00:00Z --end-at 2026-07-30T00:00:00Z \ --export-format parquet --output window.zip unzip -q window.zip -d window duckdb -c " SELECT username, source_ip, count(*) AS attempts FROM 'window/auth-attempts.parquet' WHERE outcome = 'deny' GROUP BY username, source_ip ORDER BY attempts DESC LIMIT 20;" ``` Because the archive includes its manifest, you can also share or retain the complete archive without separately recording the parameters used to generate it. ## Combining filters Filters combine with AND across different flags, and OR within a repeated flag. This request means "denied attempts, from either of these two ranges, using either of these two credentials": ```bash theme={null} pscale audit-log auth-attempts download --since 24h \ --outcome deny \ --source-ip 203.0.113.0/24 --source-ip 198.51.100.0/24 \ --username app_readonly --username reporting ``` ## Choosing a format | Format | Good for | | :- | :- | | `csv` | Spreadsheets, quick inspection, most SIEM ingest paths | | `jsonl` | Log pipelines and tools that process one object per line | | `parquet` | Analysis over larger datasets and querying with tools such as DuckDB or a data warehouse | CSV is the default. JSONL is selected by `--format json`, and Parquet must be requested with `--export-format parquet`. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Inspect live Postgres connections Source: https://planetscale.com/docs/postgres/monitoring/connections Watch live Postgres connection activity, find what is blocking what, and cancel or terminate connections from the PlanetScale CLI. ## Overview The `pscale branch connections` command shows what is happening inside a Postgres database branch right now, including which sessions are active, which queries are running, what they are waiting on, and which connections are blocking others. If something is stuck, you can cancel the query or terminate the connection directly. [See what's happening in your database RIGHT NOW](https://www.youtube-nocookie.com/embed/j67zO-MixK0?rel=0) It is the live equivalent of reading `pg_stat_activity` on a self-managed Postgres, with the ability to act on what you find. This guide covers Postgres branches; the same command works for Vitess branches, with the differences described in [Inspect live Vitess connections](/docs/vitess/monitoring/connections). There are two ways to use it: * **Interactively**, with `pscale branch connections top` — a terminal UI for watching activity live, drilling into blocker chains, and taking action from the keyboard. * **From scripts or AI agents**, with `pscale branch connections show` plus the `kill` and `kill-transaction` commands — one-shot, structured output designed to be safe to automate. The full command and flag reference lives in the [`connections` CLI reference](/docs/cli/connections). ## Watch live activity with `top` Run `top` in an interactive terminal to launch the live view: ```bash theme={null} pscale branch connections top ``` The view refreshes about once per second; press `space` to pause. You can also scrub back through the recent in-memory history without leaving the live view: `[` and `]` step one sample at a time, `{` and `}` jump to the oldest or newest, and `}` returns to live-follow. ![The pscale branch connections top view, with a stuck checkout transaction blocking a queue of updates](https://mintcdn.com/planetscale-2/Kp51U6wavfDUDSGO/postgres/monitoring/live-connections-top.png?fit=max\&auto=format\&n=Kp51U6wavfDUDSGO\&q=85\&s=5a8782e1e87c5a992c94fe662b331903) ### Reading the table Each row is one Postgres backend connection. The columns are: | Column | Meaning | | :- | :- | | (marker) | `▶` marks the selected row. `R` marks a connection on a replica; primary connections are unmarked. | | `PID` | The backend process ID — the connection's identity. | | `STATE` | The session state: `active`, `idle`, or `idle/xact` (idle in a transaction). | | `BLOCK` | Contention signal. A number is how many other sessions this connection is blocking. `W` means it is itself waiting on a lock. `3 W` means both. `-` means quiet. | | `WAIT` | What the session is waiting on, as `type/event` (for example `Lock/transactionid`). | | `DURATION` | Time since the last transaction or query started. | | `APP` | The client's `application_name`. | | `START` | When the transaction or query started (shown on wider terminals). | | `QUERY` | A preview of the current query. | State is color-coded so trouble stands out: active queries read clearly, idle sessions are muted, and idle-in-transaction sessions are highlighted because they tend to hold locks. Rows that are blocking other sessions are shaded more heavily as the blocking chain deepens. Press `s` to cycle the sort order between transaction start, duration, and blocking impact. ### Header and freshness The header shows the target (`database / branch`), a live connection count, the current sort, and how fresh the data is — for example `captured 17:10:46 (2s ago)`. If the data ages past a few refresh intervals it turns yellow, and if it becomes badly stale it turns red, so you always know whether you are looking at the current state. When you pause, the header still shows the data's real age and adds a `paused` indicator. If some instances in the branch are unreachable, a banner reports how many, and the rest of the rows still load. ### Drilling into blockers Press `enter` on a row to open the detail view. It has two tabs: * **Query** — the connection's full metadata and complete query text, scrollable and never truncated. * **Blockers** — the blocking graph: what this connection is **blocked by** upstream, and what it is **blocking** downstream. Inside the detail view, `left` and `right` switch between the two tabs, and `b` jumps straight to Blockers. From the table you can also open a row directly onto a tab: `v` opens it on the Query tab and `b` opens it on the Blockers tab. On the Blockers tab, press `enter` on a session in the chain to re-center the detail view on it, so you can walk a lock wait all the way to its root cause. Press `esc` to go back, and `?` at any time for the in-app key reference. ![The Blockers tab: one idle checkout-api transaction holding up the refund, payment, and cancel updates queued behind it](https://mintcdn.com/planetscale-2/Kp51U6wavfDUDSGO/postgres/monitoring/live-connections-blockers.png?fit=max\&auto=format\&n=Kp51U6wavfDUDSGO\&q=85\&s=c04a0dd0f479d78cdd5cfee9dae3f6da) ### Capture and replay You can record a session to a trace file and review it later: ```bash theme={null} # record while you watch pscale branch connections top --capture trace.jsonl # or record headlessly, e.g. during an incident pscale branch connections top --capture trace.jsonl --duration 30m # replay the trace in the same UI pscale branch connections top --replay trace.jsonl ``` In the live view, press `C` to start or stop capturing on the fly. A capture started this way includes the recent history already buffered in memory — up to the last 300 samples, about five minutes at the default refresh interval — not just from the moment you pressed `C`, so you keep the lead-up to whatever you just noticed. The same `[ ] { }` history keys work in replay to move through the recording. > [!NOTE] > > When you start a capture with `C` and did not pass `--capture`, pscale writes the trace to > `connections-.jsonl` in the directory you launched from. These files hold the full buffered history, so > they can reach a few megabytes. Clean them up or pass `--capture ` to control where they land. ## Take action on a connection `connections` offers three actions, ordered by how much they disrupt the client — from cancelling one query to dropping the whole connection: | Action | TUI key | What it does | | :- | :- | :- | | **Cancel query** | `c` | Cancels the running query (`pg_cancel_backend`). The connection stays open, so the client just sees a cancelled query and can keep going. | | **Kill transaction** | `k` | Ends the whole connection (`pg_terminate_backend`) — the client is disconnected — but only if it is still running the transaction you saw. | | **Force terminate** | `K` | Ends the connection (`pg_terminate_backend`) regardless of what it is running. Still verified by backend start, so it can't hit a recycled PID. The client is disconnected. | In the TUI, the action applies to the selected connection (or, on the Blockers tab, to the highlighted session in the chain). > [!NOTE] > > **Every action verifies it is hitting the connection you saw.** Each one carries a guard from the snapshot: cancel > query checks the query, kill transaction checks the transaction's start, and force terminate checks the connection's > backend start. So kill transaction only fires if that exact transaction is still open — if it has committed or rolled > back, the action is rejected and you re-list and try again. Force terminate has no such *transaction* check. It drops > the connection no matter what it is now running while confirming the backend is the same one you saw. Reach for kill transaction > first; use force terminate when a connection is fully idle or you specifically want to drop it. Because the data can be up to about a second old (see [How fresh the data is](#how-fresh-the-data-is)), a connection you act on may have moved on by the time the action lands. That is expected and safe: kill transaction's guard rejects the action rather than terminating different work, and you should run `show` again to confirm the outcome either way. ## Inspect and act from scripts or AI agents For automation — including AI agents that cannot drive an interactive UI — use `show` with `--format json`, then act with the server-issued IDs it returns. ```bash theme={null} pscale branch connections show --format json ``` The output is an envelope with `captured_at`, the `instances` in the branch, and the `connections`. Each connection carries opaque `query_id`, `transaction_id`, and `connection_id` fields. Pass those to the action commands: ```bash theme={null} pscale branch connections kill --query pscale branch connections kill-transaction pscale branch connections kill ``` The recommended loop is: **1. Show** Run `show --format json` to get the current snapshot. **2. Inspect** Read the row you care about. `connection_id` is always present; `transaction_id` is `null` when the connection has no open transaction. So an idle connection can be ended with `kill` but not `kill-transaction`. **3. Explain and confirm** For a destructive action, describe what you are about to do and get a human's approval before running it. **4. Act once** Run exactly one action command, targeting the connection by its server-issued ID. **5. Verify** Run `show` again to confirm the result. > [!WARNING] > > The action sub-commands take effect immediately and have **no confirmation prompt** — the confirmation only exists in > the interactive TUI. Always target a connection by the `query_id`, `transaction_id`, or `connection_id` from a recent > `show`, never by PID or position. If the data was stale, a destructive action may be rejected with a verification mismatch — the transaction or query changed since you listed it. That is the safety guard working: run `show` again and retry. ## Permissions Access to Connections is tied to your user account and your role on the database, and what you can do depends on whether the branch is a [production branch](/docs/postgres/branching). It is available to signed-in users (for the CLI, authenticate with `pscale auth login`), not to service tokens. There are two levels of access: * **View** — see the connection list and the `top` view. * **Act** — cancel a query or terminate a connection. Production branches are gated more tightly than development branches: | Role | View dev | Act dev | View production | Act production | | :- | :-: | :-: | :-: | :-: | | Organization Member | ✓ | ✓ | | | | Organization Analyst | ✓ | ✓ | ✓ | | | Organization Administrator | ✓ | ✓ | ✓ | ✓ | | Database Administrator (for that database) | ✓ | ✓ | ✓ | ✓ | In short: any organization member can view and act on development branches; viewing a production branch requires Analyst or higher; acting on a production branch requires Administrator. `Database Administrator` access can be granted directly or through a [team](/docs/security/teams) — everyone on a team that includes the database has `Database Administrator` access to it. If you do not have the required role, the command returns a permission error (HTTP 403) for the action you attempted. See [Access control](/docs/security/access-control) for the full role and team model. ## How fresh the data is `connections` polls about once per second, and snapshots are cached for roughly a second on the server, so the data you see can be up to about a second old. The `captured_at` timestamp (and the freshness indicator in the TUI header) always tells you exactly how old the current snapshot is. After you take an action, you may briefly see the connection in the next `show` because of that caching window; run `show` again a moment later to confirm. If the server is briefly busy refreshing its view, `show` retries automatically for a couple of seconds. If it still cannot get fresh data, you see `server is warming up, please retry in a moment`. ## What the data maps to If you know Postgres, these map to what you would expect: * A **connection** (or session) is a Postgres backend process, identified by its `PID`. * A **query** is the statement that connection is currently running. Cancel it with `kill --query`. * A **transaction** is the open transaction on that connection. End it with `kill-transaction`. * A **blocker** is a connection holding a lock that another connection is waiting on. The `BLOCK` column and the Blockers tab show these relationships. * An **instance** is a database server in your branch — your primary and any replicas. ## Related documentation > **[connections CLI reference](/docs/cli/connections)** > **[Query Insights](/docs/postgres/monitoring/query-insights)** > **[PlanetScale CLI environment setup](/docs/cli/planetscale-environment-setup)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Errors Source: https://planetscale.com/docs/postgres/monitoring/errors The Errors page surfaces any errors that have been captured on your database in a 24 hour period. ## Overview The Errors page surfaces any errors that have been captured on your database in a 24 hour period. Underneath the graph, you'll find a list of database error messages that have been captured over the selected period. You can click on any of the error messages on the Errors page to open a more detailed view. This view shows you the individual queries that produced the error, when they ran, how long they ran, and any query tags attached to them. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster Logs Source: https://planetscale.com/docs/postgres/monitoring/logs The Logs dashboard provides comprehensive logging and debugging capabilities for your PlanetScale Postgres database cluster. This centralized view helps you monitor database activity, troubleshoot issues, and analyze system behavior in real-time. ## Dashboard overview The Logs dashboard displays real-time and historical log entries from your database cluster across all servers. You can filter and analyze logs by: * **Server Filter**: Monitor all servers or focus on specific replica instances * **Log Level**: Filter by severity (DEBUG, INFO, WARNING, ERROR) * **Time Range**: View logs from the past hour up to custom time ranges * **Branch**: Select which database branch to monitor * **Search**: Full-text search across log messages * **Live update**: Toggle on/off the auto-refresh of data every \~10 seconds ## Log filtering and navigation ### Server selection The server filter dropdown allows you to choose which database instances to view logs from. You can select the Primary, Replicas, or all servers. By default just the Primary is shown: | Filter Option | Purpose | | :- | :- | | **Primary** | Monitor main database server logs | | **Replicas** | Monitor individual replica servers | ### Log level filtering By default all log levels are displayed. You can filter them based on the following log levels: | Level | Color Code | Purpose | | :- | :- | :- | | **DEBUG** | Gray | Detailed system information | | **INFO** | Blue | General information messages | | **WARNING** | Yellow | Potential issues requiring attention | | **ERROR** | Red | Critical issues requiring immediate action | ### Time range selection The time range selector offers several preset options: * **Past 15 minutes** * **Past hour** * **Past 3 hours** * **Past 6 hours** * **Past 12 hours** * **Past day** **Custom range**: Set specific date and time ranges for targeted analysis > [!NOTE] > > All log timestamps are displayed in the timezone configured in the top right of the console under the drop down 'Timezone display' ## Log retention By default, logs are retained for **7 days**. This retention period ensures you have access to recent log history for troubleshooting and analysis while managing storage efficiently. ## Search and filtering ### Full-text search Use the search bar to find specific: * Error messages and stack traces * Connection identifiers * Query patterns * Configuration changes ### LogQL search syntax PlanetScale's logging capabilities utilize Victoria Logs, which supports powerful LogQL query syntax. Click the `?` icon in the search box to see syntax help. Common examples include: **Text search:** * `error` - Find logs containing "error" * `"received message"` - Exact phrase search **Field filtering:** * `planetscale.pod:h2l-pod` - Filter by specific pod * `planetscale.role:replica` - Filter by database role * `planetscale.container:psbouncer` - Filter by container type * `planetscale.availability_zone:us-east-1a` - Filter by availability zone **Exclusion filters:** * `NOT error` - Exclude logs containing "error" * `NOT "received message"` - Exclude specific phrases **Logical operators:** * `"received message" OR planetscale.container:psbouncer` - Match either condition For complete LogQL syntax documentation, see: [Victoria Logs LogQL Documentation](https://docs.victoriametrics.com/victorialogs/logsql/) ### Advanced filtering Combine multiple filters for precise log analysis and advanced log discovery: * **Server type + Log level**: Filter Primary/Replica servers by ERROR/WARNING levels for targeted troubleshooting * **Time window + Search terms**: Narrow down to specific time ranges with LogQL queries for incident investigation * **Log level + Search bar**: Combine severity filtering with text search for comprehensive debugging * **Server selection + Time range + LogQL**: Use all filtering options together for precise log discovery ## Log message structure When you click on a log entry, a detailed view appears on the right side showing: | Field | Description | Example | | :- | :- | :- | | **Time** | Exact timestamp of the log event with timezone | `2025-06-23 17:31:27 UTC` | | **Level** | Log severity level with color coding | `INFO`, `DEBUG`, `ERROR`, `WARNING` | | **Role** | Database server role | `primary` | | **Pod** | Kubernetes pod identifier | `h2l-8abod1fll1tn-aws-useast1b-1-1335211508-467a6a97` | | **Container** | Container identifier | `postgres` | | **Availability Zone** | Geographic server location | `us-east-1a` | | **Message** | Full log message content | Complete error messages, SQL queries, system events | The Logs dashboard serves as your primary debugging and monitoring tool, providing the detailed visibility needed to maintain optimal database performance and quickly resolve issues. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster Metrics Source: https://planetscale.com/docs/postgres/monitoring/metrics The Metrics dashboard provides comprehensive monitoring and observability for your PostgreSQL database cluster. This centralized view helps you track performance, identify bottlenecks, and ensure optimal database health. ![Metrics Dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/monitoring/metrics.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=11d691b3843a6ba2772c68bc29cabefa) ## Dashboard overview The Metrics dashboard displays real-time and historical data about your database cluster's performance across multiple dimensions. You can filter metrics by: * **Server Filter**: Monitor all servers or focus on specific instances * **Branch**: Select which database branch to monitor * **Time Range**: View data from the past 15 minutes up to custom time ranges * **Live update**: Toggle on/off the auto-refresh of data every \~30 seconds [Data drop: Metrics for Postgres](https://www.youtube-nocookie.com/embed/0Rg1dS2vPjs?rel=0) ## Key metrics categories ### Primary cluster utilization The primary cluster utilization panel shows your primary database server's resource consumption: | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **CPU** | Percent | Real-time CPU utilization | Monitor for consistent performance and identify when optimization may be needed | | **Memory** | Percent | Current memory consumption | Track memory usage patterns and plan for scaling when approaching limits | ### Replica monitoring Each replica displays individual performance metrics in dedicated panels: | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **CPU** | Percent | Individual CPU tracking per replica | Compare replica performance against primary and identify load distribution | | **Memory** | Percent | Individual memory tracking per replica | Monitor replica resource consumption and ensure balanced utilization | ### Primary IOPS | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **IOPS** | Operations/second | Tracks database read/write operations per second | Monitor I/O patterns and identify peak usage periods for performance optimization | ### Primary storage usage Storage metrics vary depending on your cluster's storage type: **For Network-attached Storage clusters:** | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Storage Usage** | MB/GB | Current storage consumption | Track storage growth trends for capacity planning and ensure adequate free space | **For PlanetScale Metal clusters:** | Metric | Unit | Purpose | Key Insights | | - | - | - | - | | **Primary disk usage** | Percentage | Current disk utilization | Monitor disk space consumption on Metal instances where storage is directly tied to instance size | > [!NOTE] > > **Storage metric differences**: Network-attached storage clusters show absolute storage usage (MB/GB), while PlanetScale Metal clusters display disk usage as a percentage since storage scaling requires changing the entire instance size. ### Memory | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Memory mapped** | MB | Memory-mapped files used by PostgreSQL | Monitor shared memory usage for buffer pools and shared data | | **RSS** | MB | Resident Set Size - physical memory currently used | Track actual memory consumption by PostgreSQL processes | | **Active cache** | MB | Recently accessed cached data in memory | Monitor hot data retention and cache effectiveness | | **Inactive cache** | MB | Less recently accessed cached data | Track memory available for eviction under pressure | ### Transaction rate | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Transactions per second** | TPS | Database transaction throughput | Monitor database workload intensity and performance trends | ### Postgres connections | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Active** | Count | Currently executing connections | Monitor active database workload and concurrent operations | | **Disabled** | Count | Disabled connections | Track connection state management and administrative actions | | **Fastpath function call** | Count | Connections using fastpath protocol | Monitor specialized connection types for performance optimization | | **Idle** | Count | Idle but connected sessions | Track connection pool efficiency and unused capacity | | **Idle in transaction** | Count | Idle connections holding transactions | Identify potential long-running transactions that may cause blocking | | **Idle in transaction (aborted)** | Count | Idle connections with aborted transactions | Monitor failed transaction cleanup and connection state recovery | ### PgBouncer connections | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Total Connections** | Count | Active database connections | Monitor connection patterns and trends for capacity planning cluster size | ### PgBouncer peer utilization | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **CPU** | Percent | PgBouncer process CPU usage | Monitor connection pooler performance and resource consumption | | **Memory** | Percent | PgBouncer process memory usage | Track memory usage of the connection pooling layer | ### PgBouncer server pools | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Active** | Count | Active server connections | Monitor backend database connections from the pool | | **Active Cancel** | Count | Connections being cancelled | Track connection cleanup and cancellation events | | **Being Cancelled** | Count | Connections in cancellation process | Monitor connection state transitions | | **Idle** | Count | Idle server connections | Track connection pool efficiency and unused connections | | **Login** | Count | Connections in login state | Monitor authentication and connection establishment | | **Testing** | Count | Connections being tested | Track connection health check activities | | **Tested** | Count | Recently tested connections | Monitor connection validation processes | | **Used** | Count | Total used connections | Overall connection utilization from the pool | ### PgBouncer client pools | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Active** | Count | Active client connections | Monitor incoming client connection load | | **Active Cancel** | Count | Client connections being cancelled | Track client-side connection cleanup | | **Waiting** | Count | Client connections waiting for server | Identify connection queue buildup and potential bottlenecks | > [!NOTE] > > The client connection ceiling shown for a dedicated pool is the total across all of its PgBouncers, not the `max_client_conn` value alone, because a dedicated pool runs multiple PgBouncers spread across availability zones (configured with "PgBouncers per availability zone"). See [Configuring PgBouncers](/docs/postgres/connecting/pgbouncer#configuring-pgbouncers). ### WAL archival rate | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Success** | Count | Successfully archived WAL files | Monitor backup and replication health | | **Failed** | Count | Failed WAL archival attempts | Track archival failures that could impact recovery capabilities | ### WAL archive age | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Seconds** | Time | Age of oldest unarchived WAL | Monitor WAL archival latency and ensure timely backup operations | ### WAL storage | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Storage Usage** | MB | Write-ahead log storage consumption | Track WAL disk usage for capacity planning and cleanup monitoring | ### Replication lag | Metric | Unit | Purpose | Key Insights | | :- | :- | :- | :- | | **Lag** | Seconds | Time delay between primary and replica | Monitor replication health and ensure acceptable lag for read replica consistency | ## Interpreting metrics ### Normal operating ranges * **CPU**: 0-30% for typical workloads * **Memory**: 20-80% depending on dataset size * **IOPS**: Varies by workload type (OLTP vs. analytics) * **Storage Usage**: Keep below 80% for optimal performance (applies to both absolute storage usage and disk usage percentage) ### Performance indicators * **Consistent Low CPU/Memory**: Indicates healthy, optimized queries * **Spiky IOPS**: May indicate batch processing or analytical workloads * **Low Connection Pool Utilization**: Suggests efficient connection management * **High Active Cache vs Inactive Cache**: Indicates good data locality and efficient query patterns * **Low Transaction Rate**: May indicate application bottlenecks or connection pooling issues * **High Idle in Transaction**: Suggests application issues with transaction management ### Troubleshooting with metrics * **High CPU**: Check for inefficient queries or missing indexes * **High Memory**: Monitor for high memory usage from large queries or buffer cache pressure * **High IOPS**: Analyze query patterns and consider query optimization * **High Storage/Disk Usage**: Plan for storage scaling or data archiving (for Metal clusters, this requires instance resizing) * **High RSS vs Memory Mapped**: May indicate memory pressure or suboptimal shared\_buffers configuration * **Low Transaction Rate**: Investigate connection pooling, application logic, or database locks * **High Idle in Transaction**: Review application transaction handling and connection management * **Imbalanced Cache (Active/Inactive)**: Consider adjusting memory settings or query optimization ### WAL monitoring best practices * **Archive age**: Should typically be under 60 seconds for healthy systems * **Archival success rate**: Aim for 100% success rate with zero failures * **WAL storage**: Monitor for steady-state usage with periodic cleanup cycles * **Replication lag**: High lag may indicate WAL transmission issues ## Best practices 1. **Baseline Establishment**: Understand your normal operating ranges 2. **Alert Thresholds**: Set up monitoring alerts for critical thresholds 3. **Trend Analysis**: Use historical data to predict scaling needs 4. **Performance Correlation**: Cross-reference metrics with application performance The Metrics dashboard serves as your primary tool for maintaining optimal database performance and ensuring reliable service delivery. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # pganalyze collector Source: https://planetscale.com/docs/postgres/monitoring/pganalyze Set up pganalyze monitoring on your PlanetScale Postgres databases for query analysis, performance insights, log collection, and alerting through the pganalyze platform. ## Overview pganalyze is an advanced monitoring and analytics platform built specifically for PostgreSQL databases, enabling visibility into query performance, system activity, and logs. To use this with PlanetScale, you must run an instance of the [pganalyze collector](https://github.com/pganalyze/collector). This is a monitoring tool designed to collect PostgreSQL logs, query statistics, and system metrics. Data is sent to the pganalyze platform through this collector for visualization, alerting, and analysis, providing deep insights into your database's performance. > [!NOTE] > > PlanetScale provides [Query Insights](/docs/postgres/monitoring/query-insights) and [detailed metrics](/docs/postgres/monitoring/metrics) as a part of every Postgres database. > These combined with our in-app logs view typically provide more than enough detail to monitor cluster health and query performance. > We recommend pganalyze for those who have specific additional analytics needs not covered by our built-in tools. ## Prerequisites Before setting up the pganalyze collector, you'll need: * A [pganalyze account](https://pganalyze.com/) * A PlanetScale Postgres database * If you want to use log analytics, you'll also need a [PlanetScale Service Token](/docs/api/reference/service-tokens) with appropriate permissions. This guide assumes you will set up log analytics. ## Getting started ### Step 1: Create a PlanetScale Service Token First, create a Service Token in your PlanetScale organization that the collector will use to access your database branch information and logs: **Step 1** Navigate to your Organization Settings in PlanetScale **Step 2** Go to the **Service tokens** section and click **Create service token** **Step 3** Give your token a descriptive name (e.g., "pganalyze-collector") **Step 4** Grant the token `read_branch` capabilities, which is required for pganalyze to fetch branch information **Step 5** Click **Create service token** and save both the **Token ID** and **Token** securely. You won't be able to see the token again. ### Step 2: Create a pganalyze API key **Step 1** Log in to your [pganalyze account](https://app.pganalyze.com/) **Step 2** Navigate to **Settings** > **API Keys** **Step 3** Select **Collector API key** and then create the key ### Step 3: Get your PlanetScale database connection details You'll need the following information from your PlanetScale database: your **organization name** (the organization slug), your **database name** (the name of your Postgres database), and the **branch name** that you want to monitor (for example, `main`). You will also need to create a role that the pganalyze collector will connect to your database with. Follow our [connections quickstart guide](/docs/postgres/connecting/quickstart) for instructions on how to create a new role. When doing so, ensure to give it a descriptive name (e.g., `pganalyze-collector`) and enable only the `pg_monitor` role permission. After creating this role, you must run a few additional SQL commands to ensure it has the necessary schema and access for monitoring. Run the following to create the pganalyze schema: ```sql theme={null} CREATE SCHEMA pganalyze; GRANT USAGE ON SCHEMA pganalyze TO your_username; GRANT USAGE ON SCHEMA public TO your_username; ``` Note that `your_username` should not include the `.branch_id` at the end of it. So if the username you were given from the PlanetScale UI is `pscale_api_abcd.efgh` you should use `pscale_api_abcd` for these statements. Finally, add these two functions to this newly-created schema: ```sql theme={null} DROP FUNCTION IF EXISTS pganalyze.get_column_stats; CREATE FUNCTION pganalyze.get_column_stats() RETURNS TABLE( schemaname name, tablename name, attname name, inherited bool, null_frac real, avg_width int, n_distinct real, correlation real ) AS $$ /* pganalyze-collector */ SELECT schemaname, tablename, attname, inherited, null_frac, avg_width, n_distinct, correlation FROM pg_catalog.pg_stats WHERE schemaname NOT IN ('pg_catalog', 'information_schema') AND tablename <> 'pg_subscription'; $$ LANGUAGE sql VOLATILE SECURITY DEFINER; DROP FUNCTION IF EXISTS pganalyze.get_relation_stats_ext; CREATE FUNCTION pganalyze.get_relation_stats_ext() RETURNS TABLE( statistics_schemaname text, statistics_name text, inherited boolean, n_distinct pg_ndistinct, dependencies pg_dependencies, most_common_val_nulls boolean[], most_common_freqs float8[], most_common_base_freqs float8[] ) AS $$ /* pganalyze-collector */ SELECT statistics_schemaname::text, statistics_name::text, (row_to_json(se.*)::jsonb ->> 'inherited')::boolean AS inherited, n_distinct, dependencies, most_common_val_nulls, most_common_freqs, most_common_base_freqs FROM pg_catalog.pg_stats_ext se WHERE schemaname NOT IN ('pg_catalog', 'information_schema') AND tablename <> 'pg_subscription'; $$ LANGUAGE sql VOLATILE SECURITY DEFINER; ``` You can also read about this in [the pganalyze docs](https://pganalyze.com/docs/install/self_managed/01_create_monitoring_user). ### Step 4: Enable the pg\_stat\_statements extension On PlanetScale, you must first enable the `pg_stat_statements` extension in the UI: 1. From your database dashboard, navigate to the **Clusters** page then to the **Extensions** tab. 2. Find `pg_stat_statements` and click the checkbox to enable it, tuning parameters from the defaults if needed. 3. Enqueue and apply the change. Applying the change to enable this extension will cause direct connections to be terminated. Ensure you are able to handle connection interruptions before applying this change. After enabling in the UI, you must also enable the extension in your database by running: ```sql theme={null} CREATE EXTENSION IF NOT EXISTS pg_stat_statements; ``` Verify the extension is enabled by running: ```sql theme={null} SELECT * FROM pg_extension WHERE extname = 'pg_stat_statements'; ``` ## Configuration The pganalyze collector is configured using a configuration file. Create a file named `pganalyze_collector.conf`: ```ini theme={null} [pganalyze] # pganalyze API key api_key = your_pganalyze_api_key [planetscale_production] # PlanetScale Postgres connection (for query stats and monitoring) db_host = your_db_host db_port = 5432 db_name = your_database_name db_username = your_username.your_branch_id db_password = your_password db_sslmode = require # Specify type and scope for pganalyze api_system_type = planetscale api_system_scope = your_database_name/your_branch_name # PlanetScale integration (for log collection) planetscale_org = your_org_name planetscale_database = your_database_name planetscale_branch = your_branch_name planetscale_token_id = your_service_token_id planetscale_token_secret = your_service_token_secret ``` > [!NOTE] > > The `db_username` should include your branch ID in the format `username.branch_id`. You can find this in the connection string provided by PlanetScale. ## Installation and deployment The collector can be downloaded using [pre-built packages](https://pganalyze.com/docs/collector/packages) or optionally built from source. The pganalyze website has [instructions for installing it](https://pganalyze.com/docs/install/self_managed/03_install_the_collector) on various operating systems and Docker. Choose whichever is most appropriate for your development environment. When deploying the collector, use the configuration options covered in the previous section. ## Verifying the setup After deploying the collector, you should verify that it is working correctly. **Step 1** Log in to your [pganalyze dashboard](https://app.pganalyze.com/) **Step 2** Select your PlanetScale database and branch combination from the servers dropdown list **Step 3** Navigate through the different sections to verify data collection. For example: * **Query Performance**: View slow query analysis, execution times, and optimization recommendations * **Log Insights**: Review parsed log entries with error detection and categorization * **Connections**: Check connection counts and database activity metrics If data is not appearing, check the collector logs to see if there are authentication or connectivity issues between pganalyze and PlanetScale. ## Limitations Current settings / limitations of the pganalyze integration with PlanetScale Postgres: * **System metrics**: Detailed system metrics (CPU, memory, disk I/O) from the database host are not available through the collector. Use [PlanetScale's Prometheus metrics](/docs/postgres/monitoring/prometheus-postgres) for system-level monitoring. * **Log polling interval**: Logs are collected every 30 seconds, matching the collector's default polling interval. ## Next steps * **Explore pganalyze**: Learn about [query performance insights](https://pganalyze.com/docs/query-performance) and [log insights](https://pganalyze.com/log-insights) * **Monitor additional metrics**: Complement pganalyze monitoring with [Prometheus metrics](/docs/postgres/monitoring/prometheus-postgres) * **Leverage PlanetScale Insights and schema recommendations**: Use [PlanetScale Insights](/docs/postgres/monitoring/query-insights) for actionable query analytics, and explore [schema recommendations](/docs/postgres/monitoring/schema-recommendations) to improve your database structure alongside pganalyze monitoring. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sending Prometheus Metrics to Datadog for PlanetScale Postgres Source: https://planetscale.com/docs/postgres/monitoring/prometheus-metrics-datadog-postgres If you're looking for more metrics than PlanetScale's native Datadog integration provides, this tutorial will show how to configure your [Datadog agent](https://docs.datadoghq.com/agent/) to scrape PlanetScale's [Prometheus infrastructure](/docs/vitess/integrations/prometheus-metrics) automatically, allowing you to collect detailed metrics for all of your PlanetScale Postgres branches. ## Overview In this tutorial, this guide assumes that you have a Datadog Agent Version 7 running. For more information on what the Datadog Agent is and how to install it, start with the [Datadog Agent documentation](https://docs.datadoghq.com/getting_started/agent/). For the purposes of this guide, the examples use a Datadog agent running with the recommended installation steps on a Linux system. ## Prerequisites You'll need a working Datadog agent and access to add a [Custom Agent Check](https://docs.datadoghq.com/developers/custom_checks/) to that instance. This may require `root` or `sudo` access on the machine running the Datadog agent. You'll also need a [Service token](/docs/api/reference/service-tokens) in your Organization, with the `read_metrics_endpoints` permission granted. ## Adding the Plugin to the Datadog Agent Go to [https://github.com/planetscale/planetscale-datadog](https://github.com/planetscale/planetscale-datadog), which is the repository that has our custom OpenMetrics Check. Place the unedited `planetscale.py` in the `checks.d` directory of your Datadog Agent. * On Linux, that is `/etc/datadog-agent/checks.d/` * On macOS, that is `/opt/datadog-agent/etc/checks.d/` Make sure that it belongs to the appropriate user. If you're using the recommended Linux installation steps, it will have created a `dd-agent` user: ```bash theme={null} $ pwd /etc/datadog-agent/checks.d $ ls -al planetscale.py -rw-r--r-- 1 dd-agent dd-agent 9261 Apr 2 22:54 planetscale.py ``` This file is owned by the `dd-agent` user and group in the `/etc/datadog-agent/checks.d` directory. If you're on macOS, it will depend on whether you installed the agent as a 'Single User Agent' or a 'Systemwide Agent'. If you picked Single User, there should be no additional permission changes needed. If you installed it as a Systemwide agent, make sure the user and group you installed the agent with have ownership of the file. ## Configuring the Datadog Agent Now that you have the plugin installed, you need to configure it. In the `conf.d` directory of the Datadog agent take the `conf.d/planetscale.yaml.example` file and edit it with your organization name and Service Token information. It should look like this: ```yaml expandable theme={null} instances: - planetscale_organization: 'your-org' # Required: Your PlanetScale organization ID ps_service_token_id: '${TOKEN_ID}' # Required: Your PlanetScale Service Token ID ps_service_token_secret: '${TOKEN}' # Required: Your PlanetScale Service Token Secret. Consider using Datadog secrets management: https://docs.datadoghq.com/agent/guide/secrets-management/ namespace: 'planetscale_postgres' # Required: Namespace for the metrics metrics: # Required: List of metrics to collect. Use mapping for renaming/type overrides. - planetscale_postgres_connection_state: postgres_connection_state - planetscale_edge_postgres_active_connections: edge_active\_connections min_collection_interval: 60 send_distribution_buckets: true collect_counters_with_distributions: true ``` This configures the integration to look for all of the branches in your PlanetScale organization, collect specific Postgres metrics, and put them inside the `planetscale_postgres` namespace. ### Collecting all metrics If you want to collect all available PlanetScale Postgres metrics instead of specifying individual metrics, you can use a wildcard pattern: ```yaml theme={null} instances: - planetscale_organization: 'your-org' ps_service_token_id: '${TOKEN_ID}' ps_service_token_secret: '${TOKEN}' namespace: 'planetscale_postgres' metrics: [".*"] min_collection_interval: 60 send_distribution_buckets: true collect_counters_with_distributions: true ``` Using `'.*'` in the metrics list tells the Datadog agent to scrape all metrics exposed by PlanetScale. This is useful when you want comprehensive monitoring without manually maintaining a list of specific metrics. Save the file as `planetscale.yaml`, making sure to double check permissions: ```bash theme={null} $ pwd /etc/datadog-agent/conf.d $ ls -al planetscale.yaml -rw-r--r-- 1 root root 1518 Apr 2 22:57 planetscale.yaml ``` ## Restart the Datadog Agent Now that this is configured and installed, restart the Agent: ```bash theme={null} $ sudo systemctl restart datadog-agent ``` ## Validating the PlanetScale Plugin Now that the Datadog Agent is running the PlanetScale plugin, metrics should start flowing into Datadog within a couple of minutes. To validate, you can check the Datadog Agent: ```bash theme={null} sudo -u dd-agent -- datadog-agent check planetscale ``` If the plugin is installed successfully, this should output the scrape targets for your branches, as well as metadata about when it was last run and how many metrics were emitted. ## Adding Metrics In the earlier configuration, only two metrics were added. For a complete list of what PlanetScale Postgres exposes, please take a look at the [Postgres Metrics Reference Documentation](/docs/postgres/monitoring/prometheus-metrics-postgres). Note that the Datadog agent [normalizes metrics with certain suffixes starting in v7.32.0](https://github.com/DataDog/integrations-core/blob/master/openmetrics/README.md): > Starting in Datadog Agent v7.32.0, in adherence to the OpenMetrics specification standard, counter names ending in \_total must be specified without the \_total suffix. For example, to collect promhttp\_metric\_handler\_requests\_total, specify the metric name promhttp\_metric\_handler\_requests. This submits to Datadog the metric name appended with .count, promhttp\_metric\_handler\_requests.count. This means that to scrape a metric such as `planetscale_postgres_database_xact_commit_total`, you would configure the Datadog agent for `planetscale_postgres_database_xact_commit`. If you want to collect additional metrics, you can add them to the list: ```yaml theme={null} metrics: # Required: List of metrics to collect. Use mapping for renaming/type overrides. - planetscale_postgres_connection_state: postgres_connection_state - planetscale_edge_postgres_active_connections: edge_active_connections - planetscale_postgres_wal_size_bytes: postgres_wal_size - planetscale_pgbouncer_current_connections: pgbouncer_connections ``` Then, restart the Datadog Agent: ```bash theme={null} $ sudo systemctl restart datadog-agent ``` ## What's Next? Now that you're sending Postgres metrics from PlanetScale to Datadog, take a look at our [full list](/docs/postgres/monitoring/prometheus-metrics-postgres) and start building dashboards to monitor your PostgreSQL database performance, connection health, and resource utilization! ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Prometheus metrics for PlanetScale Postgres Source: https://planetscale.com/docs/postgres/monitoring/prometheus-metrics-postgres PlanetScale Postgres exposes a Prometheus-compatible endpoint per-branch that allows you to scrape metrics for your database. ## Overview See our [Prometheus integration](/docs/vitess/integrations/prometheus) documentation for how to set Prometheus up to automatically discover and scrape metrics for your database branches. If you're using Datadog, see our [Datadog tutorial](/docs/postgres/monitoring/prometheus-metrics-datadog-postgres) for how to setup your Datadog agent to scrape metrics for your branch. ## Metrics PlanetScale Postgres emits the following metrics to be scraped. For metrics emitted by a dedicated read replica, `planetscale_database_branch_id` identifies the replica and `planetscale_upstream_database_branch_id` identifies its parent database branch. ## Database Metrics | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_postgres\_connection\_state** The count and state of Postgres connections | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_connection\_state, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_wal\_archiver\_succeeded\_count** The count of successfully archived WALs | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_wal\_archiver\_failed\_count** The count of unsuccessfully archived WALs | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_wal\_archiver\_last\_age\_succeeded** The age of the last successfully archived WAL | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_wal\_archiver\_lag\_bytes** WAL bytes written but not yet archived (current LSN minus last archived LSN) | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_wal\_size\_bytes** The cumulative disk size of WALs waiting to be archived | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_settings\_max\_connections** The current value of the max\_connections setting | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_settings\_max\_wal\_size\_bytes** The current value of the max\_wal\_size\_bytes setting | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_settings\_max\_slot\_wal\_keep\_size\_bytes** The current value of the max\_slot\_wal\_keep\_size setting (-1 means unlimited) | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_replication\_slot\_max\_wal\_retained\_bytes** The largest amount of WAL retained behind any logical replication slot on the primary; grows toward max\_slot\_wal\_keep\_size as a slot's consumer falls behind | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_replication\_slot\_min\_safe\_wal\_size\_bytes** The smallest remaining headroom before a logical replication slot risks invalidation; absent when max\_slot\_wal\_keep\_size is unlimited | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_replication\_slots\_lost** The number of logical replication slots on the primary that have been invalidated (lost) and must be resynced | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_replica\_lag\_seconds** Replica lag in fine-grained seconds from Postgres | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_locks** Count of current lock modes | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_lock\_mode, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_wait\_events** Count of active Postgres backends on the primary by wait event. An active backend with no wait event is running on CPU and is reported with wait event and wait event type **CPU**. Primary only; replicas emit no series | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_wait\_event\_type, planetscale\_wait\_event, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_postgres\_database\_xact\_commit\_total** Total committed transactions on Postgres databases | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_database\_name, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_backup\_restore\_active** Determines if a backup restore is active, tagged with the current phase (1 = active) | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_phase, planetscale\_component | | **planetscale\_backup\_fetch\_percent** Percentage progress of a backup fetch operation (0-100) | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_component | ## Edge Metrics | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_edge\_postgres\_active\_connections** The number of active Postgres and PgBouncer connections to the branch | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_port, planetscale\_region | | **planetscale\_edge\_postgres\_connection\_drops\_total** The total number of Postgres and PgBouncer connections that have been dropped | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_port, planetscale\_region | | **planetscale\_edge\_postgres\_connection\_errors\_total** The total number of Postgres and PgBouncer connections that have resulted in an error | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_port, planetscale\_region | | **planetscale\_edge\_postgres\_bytes\_sent\_total** The total number of Postgres and PgBouncer bytes sent | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_port, planetscale\_connector\_id, planetscale\_region | | **planetscale\_edge\_postgres\_bytes\_received\_total** The total number of Postgres and PgBouncer bytes received | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_port, planetscale\_connector\_id, planetscale\_region | ## PgBouncer Metrics | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_pgbouncer\_total\_peers** The total count of peered processes PgBouncer is running | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pgbouncer\_cpu\_util\_per\_peer\_percentages** CPU utilization percentage of PgBouncer peered processes | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pgbouncer\_current\_connections** The current count PgBouncer connections to Postgres | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pgbouncer\_current\_client\_connections** The current count client connections to PgBouncer | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pgbouncer\_pools\_client** The count and state of PgBouncer client connections | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_pgbouncer\_pool, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pgbouncer\_pools\_client\_maxwait\_seconds** How long the first client connection has waited to connect to PgBouncer | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pgbouncer\_pools\_server** The count and state of PgBouncer server connections | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_pgbouncer\_pool, planetscale\_role, planetscale\_cell, planetscale\_component | ## Infrastructure Metrics | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_pods\_cpu\_util\_percentages** CPU utilization percentage of database pods | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pods\_mem\_util\_percentages** Memory utilization percentage of database pods | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pods\_iops\_total** Total IOPS (Input/Output Operations Per Second) of database pods | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_volume\_available\_bytes** Available storage space in bytes on Postgres volumes | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_volume\_capacity\_bytes** Total storage capacity in bytes on Postgres volumes | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pods\_mem\_rss\_bytes** RSS memory usage in bytes of database pods | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pods\_mem\_mmap\_bytes** Memory-mapped file usage in bytes of database pods | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pods\_mem\_active\_cache\_bytes** Active cache memory in bytes of database pods | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pods\_mem\_inactive\_cache\_bytes** Inactive cache memory in bytes of database pods | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component | | **planetscale\_pods\_container\_status\_restarts\_total** Total container restarts, across all restart reasons. Join **planetscale\_pods\_container\_last\_terminated\_reason** to attribute them | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_cell, planetscale\_component | | **planetscale\_pods\_container\_last\_terminated\_reason** Reason a container was last terminated (1 = the current reason). Join against **planetscale\_pods\_container\_status\_restarts\_total** to attribute restarts to a reason | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_cell, planetscale\_component, planetscale\_restart\_reason | | **planetscale\_pods\_status\_phase** Pod status phase (Running, Pending, Failed, Succeeded, Unknown). Carries the pod's role, so join it to break container restarts down by role | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_phase, planetscale\_role, planetscale\_cell, planetscale\_component | ## Deprecated Metrics These metrics are deprecated and will be removed eventually. | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_pods\_container\_restarts\_total** Total container restart events detected. The reason label is attached to an all-reasons total and the counter cannot be observed at zero, so rates and increases over it are wrong in both directions. Replaced by planetscale\_pods\_container\_status\_restarts\_total. | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_container, planetscale\_role, planetscale\_cell, planetscale\_component, planetscale\_restart\_reason | ## Attributing Container Restarts `planetscale_pods_container_status_restarts_total` counts every restart and carries no role or reason label. Join the metric that carries the one you want. To break restarts down by role, join `planetscale_pods_status_phase`: ```promql theme={null} sum by (planetscale_role) ( increase(planetscale_pods_container_status_restarts_total[1h]) * on (planetscale_pod) group_left(planetscale_role) group by (planetscale_pod, planetscale_role) (planetscale_pods_status_phase) ) ``` To break restarts down by reason as well, filter against `planetscale_pods_container_last_terminated_reason` one minute at a time and join the role onto the result: ```promql theme={null} sum by (planetscale_role) ( sum_over_time(( increase(planetscale_pods_container_status_restarts_total[1m]) and on (planetscale_pod, planetscale_container) planetscale_pods_container_last_terminated_reason{planetscale_restart_reason="OOMKilled"} )[1h:1m]) * on (planetscale_pod) group_left(planetscale_role) group by (planetscale_pod, planetscale_role) (planetscale_pods_status_phase) ) ``` The gauge only reports a container's latest termination reason, so the query attributes each minute's restarts to the reason current in that minute, then sums over the hour. Change `1h` to your own window and leave the `1m` alone. A container that restarts twice inside one minute under two different reasons has both restarts attributed to the later one. ## Tag Glossary * **cluster**: The PlanetScale cluster identifier * **planetscale\_access**: The access path on bytes sent/received metrics (**public** for traffic over the public internet, or **private** for traffic over AWS PrivateLink or GCP Private Service Connect) * **planetscale\_database\_branch\_id**: The unique identifier for the database branch * **planetscale\_upstream\_database\_branch\_id**: The unique identifier for the parent database branch of a dedicated read replica. Present only on dedicated read replica metrics * **planetscale\_pod**: The Kubernetes pod name * **planetscale\_container**: The container name (postgres, pgbouncer, walg-daemon) * **planetscale\_role**: The database role (primary, replica). Absent on pods that have no role, such as pgbouncer * **planetscale\_cell**: The PlanetScale cell identifier * **planetscale\_phase**: The pod's status phase (Running, Pending, Failed, Succeeded, Unknown) * **planetscale\_component**: The PlanetScale component identifier * **planetscale\_connection\_state**: The state of database connections * **planetscale\_lock\_mode**: The PostgreSQL lock mode * **planetscale\_wait\_event\_type**: The PostgreSQL wait event type of an active backend (for example Lock, LWLock, IO, Client), or **CPU** when the backend is not waiting * **planetscale\_wait\_event**: The PostgreSQL wait event of an active backend (for example transactionid, DataFileRead, ClientRead), or **CPU** when the backend is not waiting * **planetscale\_database\_name**: The PostgreSQL database name * **planetscale\_port**: The connection port number * **planetscale\_region**: The geographic region * **planetscale\_pgbouncer\_pool**: The PgBouncer connection pool identifier * **planetscale\_connector\_id**: The edge connector identifier * **planetscale\_restart\_reason**: Reason for a container's **most recent** termination (OOMKilled, Error, Completed). Carried by **planetscale\_pods\_container\_last\_terminated\_reason** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Prometheus integration for PlanetScale Postgres Source: https://planetscale.com/docs/postgres/monitoring/prometheus-postgres PlanetScale Postgres exposes Prometheus-compatible metrics endpoints for scraping metrics about your database branches. This, along with our API-driven service discovery, allows you to automatically get in-depth information about all of the Postgres databases in your organization. In order to collect and store these, you will need to use Prometheus or a Prometheus-compatible metrics engine (such as VictoriaMetrics) that is capable of using the [HTTP SD](https://prometheus.io/docs/prometheus/latest/http_sd/) protocol. ## Prerequisites This document assumes you'll be configuring a Prometheus 3.x instance via a configuration file running on your local machine. If you are using managed Prometheus via AWS, GCP or another provider, you will have to deploy Prometheus to scrape and forward metrics via `remote_write`, as these services do not support scraping metrics. ## Getting Started First, provision a new PlanetScale [Service token](/docs/api/reference/service-tokens) in your Organization settings. Make sure to save the ID and token, as they will not be visible after they've been generated. When that's created, grant the token `read_metrics_endpoints` permissions and click "Save permissions". Your token should have the necessary permissions to access Postgres metrics endpoints. ## Configuring Prometheus Now that you have a Service Token, you can add a scrape configuration for your PlanetScale organization. A minimal Prometheus configuration should look like the following: ```yaml theme={null} scrape_configs: - job_name: "${ORG}-postgres" http_sd_configs: - url: https://api.planetscale.com/v1/organizations/${ORG}/metrics authorization: type: "token" credentials: "${TOKEN_ID}:${TOKEN}" refresh_interval: 10m ``` Fill in your organization name in the `job_name` and `url`, and place the Service Token and ID that you created in the previous step for the credentials. Save this file to `prometheus.yml` in your working directory. ## Start Prometheus Run Prometheus pointed at this configuration file: ```bash theme={null} $ prometheus --config.file=prometheus.yml ``` By default, Prometheus will listen at `0.0.0.0:9090`, which means you can access it in your browser at [http://127.0.0.1:9090](http://127.0.0.1:9090/). ### Validating Service Discovery First, make sure that Prometheus is properly querying the PlanetScale API for the right branches. If you go to `http://127.0.0.1:9090/service-discovery` you should see the job that you created earlier, with all of your Postgres branches listed under `Discovered labels`. You can confirm that this matches what's in your organization by using the PlanetScale CLI: ```bash theme={null} $ pscale branch list your-postgres-database --org your-org ``` Now, if you go to your list of targets you should see each Postgres branch as an Endpoint with properly configured scraping targets. ## Querying Prometheus Now that you're collecting metrics for your Postgres branches, the [reference guide](/docs/postgres/monitoring/prometheus-metrics-postgres) has a list of everything that PlanetScale exports. Here are some example queries you can run: ### Database Connection State To see the current connection states across your Postgres instances: ```sql theme={null} planetscale_postgres_connection_state{planetscale_database_branch_id="your-branch-id"} ``` ### Active Edge Connections To monitor active connections at the edge: ```sql theme={null} planetscale_edge_postgres_active_connections{planetscale_database_branch_id="your-branch-id"} ``` ### WAL Size Monitoring To track WAL size in bytes: ```sql theme={null} planetscale_postgres_wal_size_bytes{planetscale_database_branch_id="your-branch-id"} ``` ### PgBouncer Connection Pools To monitor PgBouncer connection pool status: ```sql theme={null} planetscale_pgbouncer_current_connections{planetscale_database_branch_id="your-branch-id"} ``` ### Resource Utilization To check CPU utilization across your database pods: ```sql theme={null} planetscale_pods_cpu_util_percentages{planetscale_database_branch_id="your-branch-id"} ``` Make sure the graph is set to stacked for multi-series metrics to get the best visualization. ## Next Steps If you keep this Prometheus instance running, it will collect metrics every 30 seconds, and refresh the list of branches every 10 minutes. For more information, see: * [Postgres Metrics reference](/docs/postgres/monitoring/prometheus-metrics-postgres) for a complete list of metrics PlanetScale exposes * [Sending metrics to Datadog](/docs/postgres/monitoring/prometheus-metrics-datadog-postgres) tutorial for using the Datadog agent to collect these metrics * [Grafana dashboards](/docs/vitess/tutorials/prometheus-metrics-grafana) for visualizing these metrics (instructions applicable for Postgres metrics along with our [Postgres dashboard template](https://github.com/planetscale/grafana-dashboard/blob/main/postgres.json)) ## Troubleshooting ### Service Discovery Issues If you're not seeing your Postgres branches in the service discovery: 1. Verify your service token has the `read_metrics_endpoints` permission 2. Check that your organization name in the URL matches exactly 3. Ensure your service token credentials are correctly formatted as `${TOKEN_ID}:${TOKEN}` ### Missing Metrics If specific metrics aren't appearing: 1. Confirm the branch is active and healthy 2. Check the scrape interval - some metrics may take time to appear 3. Verify the metric names against our [metrics reference](/docs/postgres/monitoring/prometheus-metrics-postgres) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Query Insights Source: https://planetscale.com/docs/postgres/monitoring/query-insights PlanetScale Insights gives you a detailed look into **all active queries** running against your database. ## Overview This in-dashboard tool allows you to identify queries that are running too often, too long, returning too much data, producing errors, and more. You can scroll through the performance graph to detect the time of impacted performance, quickly identifying any recent issues. You can also see a [list of all queries](#queries-overview) performed on your database in the last 24 hours. For further analysis, you can sort these by metrics like amount of rows read, time per query, and more. With this built-in tool, you can easily diagnose issues with your queries, allowing you to optimize individual queries without much digging. We will also alert you of any active issues your database may be having in the [Anomalies](/docs/postgres/monitoring/anomalies) page. This feature flags queries that are running significantly slower than expected. [Fix Database Faults with PlanetScale Insights](https://www.youtube-nocookie.com/embed/OAPHvq51hWU?rel=0) ## Insights page overview To view Insights for your database, head to the [PlanetScale dashboard](https://app.planetscale.com), select your database, and click the "**Insights**" tab. The dropdown on the top left lets you select which branch you want to analyze. You can also choose which servers you want to view insights for: primary or replicas. ![PlanetScale Insights overview page](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/monitoring/query-insights-overview.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=95790fa60341d086b21ba14466419680) You can click the dates listed above the graph to scroll through the past seven days. To further narrow down query analysis, you can select a time range by clicking on the graph and dragging the cursor across. This will zoom in on the selected timeframe. You also have the option to save a screenshot of the graph by clicking "Save". ### Queries overview table The table underneath the graph shows all queries performed on your database in the selected timeframe (last 24 hours by default). For more information about how to read and interpret this data, see the [Queries overview](#queries-overview) section. ### Insights graphs Once you have selected the branch and server you want to analyze, you can begin exploring the insights for them in the following sections: > **[Query latency](#query-latency)** > **[Queries](#queries)** > **[Rows read](#rows-read)** > **[Rows written](#rows-written)** > **[Any query metric](#any-query-metric)** The remaining sections of this doc walk through how to interpret and act on the data in each graph. If you'd like to see a practical example of how to use Insights to debug a performance issue, check out our [Problem solving with PlanetScale Insights blog post](https://planetscale.com/blog/problem-solving-with-insights) or [this YouTube video](https://www.youtube.com/watch?v=OAPHvq51hWU) walking you through an example. ### Database Traffic Control [Database Traffic Control](/docs/postgres/traffic-control) also lives within the Insights tab. It lets you define resource budgets that limit how much CPU, I/O, or connection time specific queries or workloads can consume. You can create budgets directly from query patterns surfaced in Insights, making it easy to put guardrails around problematic queries. See the [Database Traffic Control documentation](/docs/postgres/traffic-control) for full details. ### MCP server access Query Insights data is also available through the [PlanetScale MCP server](/docs/mcp-server). This lets MCP-compatible tools like Cursor, Claude, and VS Code query your Insights data directly — useful for analyzing slow queries, spotting pattern changes, or getting index recommendations without leaving your editor. See the [MCP server documentation](/docs/mcp-server) for setup instructions. ## Query latency The default graph depicts your database's query latency in milliseconds over the last 24 hours. By default, the graph contains two line charts showing `p50` and `p95` latency. This means 50% and 95% of requests, respectively, completed faster than the time listed. You can also click on the `p99` and `p99.9` pills to toggle those on, or click `p50` or `p95` to toggle those off. ## Queries The Queries graph displays insights about all active running queries in your database. The graph displays total queries per second against the specified time period. ## Rows read The Rows read graph displays the total number of rows read per second across the selected time period. ## Rows written The Rows written graph displays the total number of rows written per second across the selected time period. ## Any query metric The last graph tab plots any single query metric you choose across the selected time period. This tab is named after the metric you have selected, so it reads "Total time" or "Bytes returned" rather than a fixed title. To graph a metric: 1. Click the chevron on the last graph tab to open the metric list. 2. Select a metric. The graph redraws with that metric plotted over the time period you already have selected. The list offers the same metrics you can add as columns in the queries table, described in [Available query statistics](#available-query-statistics). It includes only the metrics your branch collects, so the options vary from branch to branch. Sparklines in the queries table lead to the same graphs. Clicking a sparkline opens that query pattern's [deep dive](#query-deep-dive) with the matching metric already selected, which takes you from every query on the database to one query pattern in a single click. ## Queries overview The table underneath the graph shows queries performed on your database in the selected timeframe (last 24 hours by default). > [!NOTE] > > The queries table does not show following statements types: `BEGIN`, `COMMIT`, `RELEASE`, `ROLLBACK`, `SAVEPOINT`, `SAVEPOINT_ROLLBACK`, `SET`. Queries are listed with literals replaced by ordinal placeholder values (e.g. \$1). Normalizing queries in this way allows them to be grouped together into patterns, irrespective of the specific parameters used in the underlying query. You may also see one or more orange icons next to some queries. * An exclamation point icon indicates that the query is not currently using an index and requires a full table scan. Hovering over the icon will show a tooltip with information about the meaning of the icon. This query overviews table shows the same data for all graphs on this page. For information about [Anomalies](/docs/postgres/monitoring/anomalies) and [Errors](/docs/postgres/monitoring/errors), refer to those pages. ### Available query statistics You can customize the metrics that show up on the Queries list by selecting columns in the "View options" dropdown. | Statistic | Description | | - | - | | **Query** | The query that was run. | | **Schema** | The default database and schema associated with the connection that issued the query, in the format `database.schema`. | | **Table schema** | The schema(s) associated with the tables referenced in the query, in the format `database.schema`. (This may differ from the connection schema.) | | **Qualified table** | The table(s) referenced in the query, in the format `database.schema.table_name`. | | **Table** | The table(s) being queried or modified. | | **% of runtime** | The percent of the total runtime the query pattern is responsible for (query pattern time divided by the cumulative time of all query patterns on your database). | | **% of CPU time** | The percent of the total CPU time the query pattern is responsible for (query pattern CPU time divided by the cumulative CPU time of all query patterns on your database). | | **% of IO time** | The percent of the total I/O time the query pattern is responsible for (query pattern I/O time divided by the cumulative I/O time of all query patterns on your database). This column is only present if `track_io_timing` parameter is set in your database's cluster configuration. | | **Count** | The number of times this query has run. | | **Total time** | The total time the query has run, in seconds. | | **CPU time** | The cumulative CPU time the query has consumed, in seconds. | | **I/O time** | The cumulative I/O time the query has consumed, in seconds. This column is only present if `track_io_timing` parameter is set in your database's cluster configuration. | | **p50 latency** | The p50 latency for the query in milliseconds. This means that 50% of requests completed faster than the time listed. | | **p99 latency** | The p99 latency for the query in milliseconds. This means that 99% of requests completed faster than the time listed. | | **Max latency** | The maximum observed latency for the query in milliseconds. | | **Rows read** | The total number of rows read. This includes all times the query has run in the displayed time frame. | | **Rows read per query** | The average number of rows read per execution of the query. | | **Rows returned** | The total number of rows fetched by a `SELECT` statement. This includes all times the query has run in the displayed time frame. | | **Rows returned per query** | The average number of rows returned per execution of the query. | | **Rows affected** | The total number of rows modified by an `INSERT`, `UPDATE`, or `DELETE` statement. This includes all times the query has run in the displayed time frame. | | **Rows affected per query** | The average number of rows affected per execution of the query. | | **Rows read / returned** | The result of dividing total rows read by rows returned in a query. A high number can indicate that your database is reading unnecessary rows, and the query may be improved by adding an index. | | **Block cache hit ratio** | The percentage of blocks read from the shared buffers cache for this query during its execution, avoiding more costly disk reads. | | **Blocks hit** | The total number of blocks read from the shared buffers cache when executing this query. | | **Blocks read** | The total number of blocks read from disk when executing this query. | | **Blocks dirtied** | The total number of blocks modified (but not necessarily flushed to disk) during query execution. | | **Blocks written** | The total number of blocks written to disk during query execution. | | **Bytes returned** | The total number of bytes returned to clients in query responses. | | **Bytes returned per query** | The total number of bytes returned to clients in query responses divided by the number of queries. | | **Last run** | The last time a query was run. | You can also sort the columns for quick analysis by clicking on the title at the top of each column. If `Show sparklines` is selected, numeric columns in the queries table show a time series graph of the value within the selected time period. Click a sparkline to open that query pattern's [deep dive](#query-deep-dive) with the same metric graphed full size. #### Enabling I/O columns The **% of I/O** and **I/O time** columns require the `track_io_timing` PostgreSQL config setting to be set to 'on'. This setting can be changed in the "Parameters" tab of the datatbase's cluster configuration. Note that we only begin collection I/O query performance after `track_io_timing` is enabled. Enabling `track_io_timing` may impact query performance. ### Query filtering The search bar above the table allows you to filter queries as needed. You can filter for query SQL, schema (connection schema, and/or schema of tables referenced by the query), table name, query count, query latency, index name, and if the query was indexed. Click on the `?` next to the search bar for the full list of search syntax. ### Query deep dive Clicking on a query in the Queries list will open a new page with more information about that query. You'll first see the full query pattern, which displays the query with data normalized away. This query may run several times with different values, which Insights combines into a single query pattern. You can display an LLM-generated summary of the query by clicking "Summarize query." [Learn more about how PlanetScale uses AI here](/docs/how-we-use-ai). #### Additional query information Beneath the query pattern is a graph with more information about the query. The set of available metrics/tabs include: Query latency, Queries, Rows read, Rows written, Errors and Indexes, along with a final tab that graphs [any query metric](#any-query-metric) you select. The Indexes graph (which is not shown on the database-level page) shows the percentage of queries that used each of the listed indexes in each time bucket. Beneath the time series graphs you will see summary statistics for the query pattern. These data are scoped to the same time period shown in the main query pattern graphs. The available metrics have the same definitions as the query statistics listed in the main insights page. Queries that use an index include a horizontal bar graph that shows the cumulative usage of each index over the complete time period shown in the main query pattern graphs. To change the time period reflected in the graphs and summary statistics, click and drag to restrict the time window, or click on one of the day icons above the graph to select a different day. #### Notable queries Underneath the graph, you'll see a table with more information about notable instances of the query, which are defined as queries that took longer than 1s, read more than 10,000 rows, or produced an error. If any of the selected queries have [query tags](/docs/postgres/monitoring/query-tags) attached, you'll see the key-value pairs in the table under `Tags`. See [Query tags](/docs/postgres/monitoring/query-tags) for how to add tags to your queries. The table also surfaces when the query started, rows returned, rows read, rows affected, the time it took the query to run (in ms), and the user associated with the query. ## Extension configuration The `pginsights` extension is responsible for sending query telemetry to the PlanetScale Insights pipeline. Its parameters, including raw query collection and schema name normalization, can be changed in the Extensions tab on your database's Clusters page. For full parameter documentation, see the [pginsights extension reference](/docs/postgres/extensions/pginsights#parameters). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Tags Source: https://planetscale.com/docs/postgres/monitoring/query-tags Attach key-value metadata to SQL statements to filter performance data in Insights and Database Traffic Control. ![Insights tags](https://mintcdn.com/planetscale-2/Pc1vKrfqALwLm39x/postgres/traffic-control/assets/insights-tags-light.png?fit=max\&auto=format\&n=Pc1vKrfqALwLm39x\&q=85\&s=3412fe11b21e0060139b8f661e70f0e9) ## Overview Query tags are key-value pairs embedded in SQL comments written into each statement to the database. PlanetScale parses these tags automatically and uses them to: * Show filtered tag-based performance data alongside individual query executions in [Insights](/docs/postgres/monitoring/query-insights) * Enable [tag-based rules](/docs/postgres/traffic-control/concepts#tag-based-rules) in [Database Traffic Control](/docs/postgres/traffic-control), so you can set resource budgets for specific applications, features, or routes Tags follow the [SQLCommenter](https://google.github.io/sqlcommenter/) specification, an open standard created by Google for attaching metadata to SQL statements as comments. ## Tag format A tag comment is appended to the SQL statement inside `/* */` delimiters. Each tag is a `key='value'` pair, and multiple tags are separated by commas: ```sql theme={null} SELECT * FROM orders WHERE status = 'pending' /*app='web',route='api-export'*/; ``` This example would create the following tags in the PlanetScale dashboard: * **app:** `web` * **route:** `api-export` Values are URL-encoded and wrapped in single quotes, following the [SQLCommenter specification](https://google.github.io/sqlcommenter/spec/). > [!WARNING] > > Tags must appear **before** the semicolon that ends the SQL statement. Tags placed after the semicolon are not recognized by PlanetScale. > > ```sql theme={null} > -- Correct: tag before semicolon > SELECT 1 /*category='analytics'*/; > > -- Incorrect: tag after semicolon > SELECT 1; /*category='analytics'*/ > ``` > > When a tag appears after the semicolon, the database treats it as a separate statement (a no-op comment), so Insights and Database Traffic Control see only the original query without any tags. ## Key and value limits After URL-decoding, tag keys and values must match `[\w\-./:,]+`: alphanumeric characters, underscores, dashes, periods, slashes, colons, and commas. Keys or values containing other characters are not accepted as query tags. Keys may be up to 65 bytes. Values may be up to 1024 bytes. > [!NOTE] > > The [SQLCommenter specification](https://google.github.io/sqlcommenter/spec/) URL-encodes keys and values without limiting the source character set. PlanetScale applies these character and length limits when ingesting tags. ## Where tags appear in PlanetScale ### Insights Queries in the Insights page can be filtered by tags by using the search bar. For example, to filter for queries with the tag `action='analytics'`, use the following syntax: ``` tag:action:analytics ``` ### Insights → Tags "Tags" is a top-level item under [Insights](/docs/postgres/monitoring/query-insights). Select one or many key/value pairs to filter the performance data. Or see the tags of all queries within the currently selected time range. You can also see the tags associated with a query on the query details page (click on the query text from the list in Insights). ### Tag cardinality limits You can attach tags with any cardinality. When a tag produces too many distinct values, PlanetScale may collapse the value to keep tag data usable in Insights. Tag values may be collapsed when either of these limits is exceeded: * Any query pattern has more than 20 values for the tag * Any query pattern has more than 50 distinct combinations of tag values within a 15-second period When a tag value is collapsed, PlanetScale removes the original value and reports it as `Collapsed` in the query details page. If you search for a tag with collapsed values, or open the Tags view for a tag with collapsed values, PlanetScale shows a warning with the percentage of results affected by collapsing. ### Database Traffic Control Tags can be used as matching criteria when creating resource budgets. A tag-based rule targets all queries that share a given tag, letting you set limits on traffic from a specific application, feature, or route without creating separate database users. See [tag-based rules](/docs/postgres/traffic-control/concepts#tag-based-rules) for details. ## Adding tags to your queries There are two approaches to tagging queries: using an SQLCommenter integration that adds tags automatically, or appending tag comments manually. ### SQLCommenter integrations The [SQLCommenter project](https://google.github.io/sqlcommenter/) provides middleware and plugins for several frameworks and ORMs that automatically attach tags to every query: | Language | Supported frameworks / ORMs | | - | - | | Python | Django, SQLAlchemy, psycopg2, Flask | | Java | Hibernate, Spring | | Node.js | Knex.js, Sequelize.js, Express.js | | Go | `database/sql`, gorilla/mux, `net/http` | | PHP | Laravel | | Ruby | Ruby on Rails | [Drizzle](https://orm.drizzle.team/docs/sql-comments) and Prisma support SQLCommenter natively. These integrations typically inject framework-level context like the route, controller, and framework version. Some support adding custom key-value pairs. > [!NOTE] > > Several popular ORMs and query builders used with PlanetScale do not have official SQLCommenter support, including **Bun** and **Kysely**. If you use one of these, see [manual tagging](#manual-tagging) below. ### Manual tagging If your ORM or query builder does not have an SQLCommenter plugin, you can append the tag comment directly to your SQL statements. The format is straightforward: add `/*key='value'*/` before the semicolon. #### Raw SQL ```sql theme={null} SELECT * FROM orders WHERE user_id = $1 /*app='checkout',feature='order-history'*/; ``` #### With a query builder ```typescript theme={null} // Example using Kysely const result = await db .selectFrom('orders') .where('user_id', '=', userId) .selectAll() .execute({ // Append the tag comment to the generated SQL queryId: "/*app='checkout',feature='order-history'*/" }); ``` > [!NOTE] > > The exact mechanism for appending a comment varies by library. Some query builders support a comment or annotation option, while others require you to modify the SQL string directly. Check your library's documentation for the best approach. #### Encoding rules When adding tags manually, follow these rules from the SQLCommenter specification: * **URL-encode** values that contain special characters. For example, `/api/export` becomes `%2Fapi%2Fexport`. * **Escape** single quotes within values with a backslash: `name='O\'Brien'`. * **Sort** key-value pairs lexicographically by key if you want to match the specification exactly, though PlanetScale does not require a specific order. * After URL-decoding, keys and values must conform to PlanetScale's [allowed character set and byte limits](#key-and-value-limits). ## Troubleshooting ### Tags are not appearing in Insights The most common cause is **tag placement**. Tags must appear before the semicolon. See the [warning above](#tag-format) for the correct format. If your tags are correctly placed and still not appearing: * Verify that your queries are being executed and appear in the [Insights queries table](/docs/postgres/monitoring/query-insights) * Check that you are searching with the correct syntax: `tag:key:value` (for example, `tag:category:analytics`) ### ORM is stripping comments Some ORMs and connection poolers may strip SQL comments before sending queries to the database. If you have added tags but they are not appearing: * Check whether your ORM has a setting to preserve SQL comments (for example, Django's `DISABLE_SERVER_SIDE_CURSORS` or specific comment-stripping options) * If you use a connection pooler like PgBouncer, verify that it is not configured to strip comments ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Schema recommendations Source: https://planetscale.com/docs/postgres/monitoring/schema-recommendations With schema recommendations inside of [PlanetScale Insights](/docs/postgres/monitoring/query-insights), you will automatically receive recommendations to improve database performance, reduce memory and storage, and improve your schema. Schema recommendations use query-level telemetry data, PostgreSQL system tables, and your database's schema to generate tailored recommendations in the form of DDL statements. > [!NOTE] > > Schema recommendations are also available through the [PlanetScale MCP server](/docs/mcp-server), and can be automatically evaluated and implemented by AI coding agents on a recurring schedule. See [Self-improving database](/docs/self-improving-database). ![Schema recommendations](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/monitoring/schema-recommendations-postgres.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=d4d4061daf0784da45c5d675962f093f) ## How to use schema recommendations To find the schema recommendations for your database, go to the “**Insights**” tab in your PlanetScale database and click “**View recommendations**.” You will see the current open recommendations that may help improve database performance. Select a recommendation to learn more. Each recommendation will have the following: * An explanation of the recommended changes, including some of the benefits of the recommended change (E.g., reduced memory and storage, decreased execution time, prevent ID exhaustion) * The schema or query that it will affect * DDL that can be applied to resolve the recommendation > [!NOTE] > > Schema recommendations that depend on your database traffic run **once per day**. Recommendations that depend only on > database schema are run whenever the the schema of your default branch is modified. Schema recommendations are > generated only for the database's default branch. ![Example of a recommendation to reduce bloat](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/monitoring/recommendation.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=02758260e1e8aab35ef77b72993f4b9a) ### Applying a recommendation Schema recommendations include a DDL statement that can be applied directly to your database, or integrated into your application or ORM framework. We recommend carefully evaluating the performance impacts or downtime assoicated with running the DDL before applying it to a production environment. ### Closing a recommendation Recommendations are automatically closed when: * The changes have been deployed to the default branch * Other schema changes to the default branch make the recommendation unnecessary Once a recommendation is closed, PlanetScale will never re-suggest it. ## Supported schema recommendations The following are the currently supported schema recommendations: * [Adding indexes for inefficient queries](#adding-indexes-for-inefficient-queries) * [Removing redundant indexes](#removing-redundant-indexes) * [Preventing primary key ID exhaustion](#preventing-primary-key-id-exhaustion) * [Dropping unused tables](#dropping-unused-tables) * [Dropping unused indexes](#dropping-unused-indexes) * [Rebuilding bloated tables](#rebuilding-bloated-tables) * [Rebuilding bloated indexes](#rebuilding-bloated-indexes) The impact of schema recommendations can vary by recommendation. In the following sections, we will inform you of each recommendation’s potential impacts and explain the recommendation further. > [!NOTE] > > Schema recommendations may not be in line with your desired outcomes. PlanetScale shall not be held liable for any > actions you take based on these recommendations. ### Adding indexes for inefficient queries Indexes are crucial for relational database performance. With no indexes or suboptimal indexes, PostgreSQL may have to read a large number of rows to satisfy queries that only match a few records. This results in slow queries and poor database performance. PlanetScale Insights automatically scans your database and suggests indexes tailored to your schema and workload. #### How PlanetScale recommends adding indexes To find indexes that will improve database performance, Insights scans query telemetry data every 24 hours to find query patterns that use significant resources. Normalized versions of these queries, along with relevant schema information, are provided as input to an LLM prompted to return index suggestions. After validating the returned suggestions for syntactic correctness, we evaluate the estimated performance of each query pattern with and without the suggested index. If any suggested index results in substantial query performance improvement, a new index recommendation is created. New index recommendations show the DDL required to create the index, a list of the queries that will benefit from the index, and an estimate of the performance improvement of each query pattern after creating the index. #### Caveats * Indexes can dramatically improve read query performance, but they also increase write costs, memory usage, and the size of tables on disk. Evaluating these additional costs against your application’s improvement in read performance is always a good idea. * Indexing is a complicated topic and depends on many factors, such as the distribution of values in your database and the particular queries your database receives. If you decide to deploy a suggested index to production, it is a good idea to use Insights to verify that your index has the desired effect on relevant queries. * Query performance with a suggested index is estimated with the [HypoPG extension](https://github.com/HypoPG/hypopg). This extension is installed by default on all PlanetScale Postgres databases. Removing this extension will disable new index recommendations. * Index suggestions are generated using AI tools. [Learn more about how PlanetScale uses AI here](/docs/how-we-use-ai). * New index suggestions, and other LLM-based features, can be disabled in the organization settings page. * By default, cost estimates use placeholder parameter values for each normalized query pattern. These values may not match your actual workload, which makes index suggestions less accurate. To improve accuracy, enable the [`pginsights.raw_queries` setting](/docs/postgres/extensions/pginsights#raw-query-collection). When enabled, candidate indexes are evaluated using actual parameter values from slow queries, resulting in better suggestions. Note: Full query text is only used for generating `EXPLAIN` plans. No parameter values are sent to the LLM. ### Removing redundant indexes While indexes can drastically improve query performance, having unnecessary indexes slows down writes and consumes additional storage and memory. #### How PlanetScale recommends removing indexes Insights scans your schema every time it is changed to find redundant indexes. We suggest removing two types of indexes: * Exact duplicate indexes - an index that has the same columns in the same order * Left prefix duplicate indexes - an index that has the same columns in the same order as the prefix of another index There are differences between the two, so note your exact recommendation and the following caveats. #### Caveats Removing redundant indexes is more nuanced than adding an index. * Exact duplicate indexes are *always* safe to remove. * Left prefix duplicate indexes are *almost always* safe to remove, but in some cases can lead to a performance regression. Usually, the larger index can be used instead of the left prefix duplicate indexes. ### Preventing primary key ID exhaustion As new rows are inserted, it’s possible for sequence-driven primary keys to exceed the maximum allowable value for the underlying column type. When the column reaches the maximum value, subsequent inserts into the table will fail, which can cause a severe outage to your application. #### How PlanetScale detects primary key ID exhaustion Insights scans all of the sequences in your database, finds the owning columns and checks the most recent value daily to identify where you might be approaching primary key ID exhaustion. If Insights detects that one of the columns is above 60% of the maximum allowable type, it will recommend changing the underlying column to a larger type. Additionally, Insights scans queries to parse joins and correlated subqueries to find foreign keys and suggests increasing the column size for those columns. #### Changing primary key types To make space for additional table growth, the primary key column and any foreign key columns that reference the primary key need to be updated to a larger integer type: `BIGINT`. The most straightforward approach is to issue an `ALTER TABLE ... ALTER TYPE ...` command, and this is the DDL that is shown in the schema recommendation. However, there is a *significant* downside to this approach: the table will be completely locked while the entire table is rewritten which will result in downtime if your application interacts with this table. `ALTER TABLE ... ALTER TYPE ...` is not an online operation and rewriting a large table can take many hours. > [!NOTE] > > Do not run `ALTER TABLE ... ALTER TYPE ...` if the table is actively used by your application. Assuming table unavailability is not an option for your application, the best approach is usually to create a new column and migrate to it in a controlled manner. The process will depend on the constraints of your application and your tolerance for downtime, and needs to be carefully planned and executed. For this reason we suggest always creating primary keys with the `BIGINT` data type unless you are certain that the number of rows in the table will be small. #### Caveats * Running `ALTER TABLE ... ALTER TYPE ...` will lead to significant downtime for large tables. * Changing data types without downtime is complicated and will need to be carefully planned and tested. ### Dropping unused tables Dropping unused tables can help clean up data that is no longer needed and reduce storage. If the table is large, it can also decrease backup and restore time. If you are unsure if a table should be retained but decide to drop the table, make sure to [create a manual backup](/docs/postgres/backups#manual-backups) of your database before you deploy the change. If you determine that the table should be retained, close the recommendation, preventing the suggestion from being remade. #### How PlanetScale recommends dropping unused tables Insights scans your query performance data daily to identify if any tables are more than four weeks old and haven’t been queried in the last four weeks. #### Caveats * Only you can know if the table’s data is no longer needed. Ensure that the table is never used (even infrequently) and does not contain important data before removing it. * Once a drop unused table recommendation is opened, it will remain open even if it is subsequently queried. Check your Insights data to verify that the table is still unused before permanently dropping it. ### Dropping unused indexes Dropping unused indexes can help reduce the cost of inserts and updates, and save memory and storage. #### How PlanetScale recommends dropping unused indexes Insights scans your query performance data daily to identify if any indexes are more than four weeks old and haven’t been used by any queries in the last four weeks. #### Caveats * Dropping an index, even if hasn't been recently used, can affect the performance of future queries. Ensure that your application doesn't make any queries that depend on the index to run efficiently. * Once a drop unused index recommendation is opened, it will remain open even if the index is subsequently used. Check your Insights data to verify that the index is still unused before permanently dropping it. Insights queries can be filtered by index usage by specifying `index:index_name` in the Insights search box. ### Rebuilding bloated tables Table bloat is excess physical storage space created by PostgreSQL’s MVCC system when rows are updated or deleted. Excessive table bloat wastes disk space, negatively impacts query performance and slows down backup and DDL operations. Some amount of table bloat is normal, but high or ever-increasing bloat is a situation that needs to be remediated. Table bloat can occur when frequent `update`s or `delete`s lead to fragmentation in the physical storage layer over time. When this occurs, the table needs to be rebuilt to optimize the physical layout of tuples on disk. To rebuild a bloated table with minimal disruption to your application, we recommend using the `pg_squeeze` extension, which can be enabled on your database's Clusters page. Once that is enabled a table can be registered for regular processing or a one-time cleanup can be performed. The schema recommendation shows the command necessary to perform a one-time cleanup. See the [pg\_squeeze documentation](https://github.com/cybertec-postgresql/pg_squeeze/) for further information. #### How PlanetScale detects table bloat Insights scans your database once a day to estimate table bloat based on system tables. Table bloat recommendations are triggered when the estimated percent of physical space lost due to bloat is over 25% and 100MB for a given table. #### Caveats * Tables can become bloated for reasons other than physical storage fragmentation, such as very long running transactions that prevent dead tuple reclamation, or infrequent vacuuming possibly caused by inadequate auto vacuum settings. If either of these are the case, you will need to address those underlying causes to resolve table bloat. * Once a table bloat recommendation is created, it will remain open even if bloat drops below the triggering thresholds. * Once a table bloat recommendation is closed, it will never be opened for that table again. * Running `pg_squeeze` will consume database resources, so be sure that your database has sufficient capacity before rebuilding tables. ### Rebuilding bloated indexes Like table bloat, index bloat is excess physical storage space created by PostgreSQL’s MVCC system when rows are updated or deleted. Excessive index bloat wastes disk space and negatively impacts query performance. Some amount of index bloat is normal, but high or ever-increasing bloat is a situation that needs to be remediated. Index bloat can occur when frequent `update`s or `delete`s lead to fragmentation in the physical storage layer over time. When this occurs, the index needs to be rebuilt to optimize the physical layout of tuples on disk. The index bloat recommendation shows the requisite `REINDEX INDEX CONCURRENTLY ...` command to rebuild the index and minimize accumulated bloat. #### How PlanetScale detects index bloat Insights scans your database once a day to estimate index bloat based on system tables. Index bloat recommendations are triggered when the estimated percent of physical space lost due to bloat is over 30% and 100MB for a given index. #### Caveats * Indexes can become bloated for reasons other than physical storage fragmentation, such as very long running transactions that prevent dead tuple reclamation, or infrequent vacuuming possibly caused by inadequate auto vacuum settings. If either of these are the case, you will need to address those underlying causes to resolve index bloat. * Once an index bloat recommendation is created, it will remain open even if bloat drops below the triggering thresholds. * Once an index bloat recommendation is closed, it will never be opened for that index again. * Running `REINDEX INDEX CONCURRENTLY ...` will consume database resources, so be sure that your database has sufficient capacity before rebuilding indexes. # PlanetScale Postgres operations philosophy Source: https://planetscale.com/docs/postgres/operations-philosophy PlanetScale has a high standard for uptime and availability of Postgres databases. The foundation of achieving this goal in the cloud is our custom Kubernetes operator, the software that manages new database creation, version upgrades, resizes, failovers, and other operations in an inherently failure-prone environment. For more detail on how the operator orchestrates clusters, see the [Postgres architecture overview](/docs/postgres/postgres-architecture#custom-operator). This document presents our architecture and philosophy on operating PlanetScale Postgres databases. All customers should read it, as it also covers what can be expected during failovers, querying replicas vs the primary, and the tradeoffs of using direct vs pooled connections. ## Database cluster architecture The cloud is an inherently failure-prone environment. Servers can experience degraded performance or become unavailable at any time and for unknown reasons. Because of this, we must treat all cloud server nodes as ephemeral. We can embrace failure and build database clusters that are resilient in the cloud environment by understanding our constraints and leveraging the cloud's elasticity within known and practical limits. Every production database cluster in PlanetScale has a primary and two replicas. This allows us to embrace server failures through failovers. Some consider failovers as a problem, but we see that as a fundamental building block for operating reliable databases. Tolerating the brief disruptions produced by failovers has huge upside: PlanetScale can rapidly deliver changes which continuously increase the quality, reliability, and performance of customer databases. ![Primary and Replicas](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/postgres-arch-diagram.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=c20f53698428814045e9bc242956ffd9) This is the most proven architecture for highly-available database clusters for several reasons: 1. Replicating data to multiple servers is crucial for data durability, especially in a cloud environment. All data is replicated 3 ways at minimum. 2. Each instance is in a distinct availability zone. This means we can be resilient to zone-wide incidents. 3. Replicas can receive read queries that can tolerate replication lag. This allows offloading work from the primary, reserving it for writes and critical reads. 4. In the event of an incident in the primary's availability zone, we can failover to a healthy replica in a different availability zone. Likewise, when performing a planned upgrade on the primary instance, we first promote a healthy replica to become primary before upgrading the old primary (now demoted to a replica) Failovers take on the order of seconds, not minutes or hours, getting you back online quickly. Having a three-node configuration as the starting point for all clusters allows PlanetScale to operate these databases in ways many other providers cannot. Failovers are used to replace primaries when they go down unexpectedly. The same, proven cluster failover techniques are used for other changes like upgrading instances, version patches, and reconfiguration. This means customers benefit from tried-and-true database management techniques to increase availability, and minimize (though not eliminate) downtime during these operations. ## Continual improvement Most Postgres database platforms shy away from version updates and other incremental improvements due to their disruptive nature. Many services require minutes or even hours of downtime to handle this. The PlanetScale philosophy is different. To give our customers the best possible Postgres experience, we believe that regular version bumps, bugfixes, and quality-of-life improvements are necessary. Image upgrades are performed in cases of emergencies (such as patching security issues) or when initiated by the user via the PlanetScale application or API. When these upgrades occur, they require node failovers, which lead to a short period of database unavailability (seconds). ## The ideal Postgres experience PlanetScale provides a well-rounded, out-of-the-box Postgres experience while also giving users what they need to tune it to their liking. This shows up in several aspects of the product: * Each database is given custom default tunings depending on CPU, RAM, and disk characteristics. Users can adjust and carefully monitor the effects of their own changes via the Clusters and Insights pages. * In addition, we tune `effective_io_concurrency` based on the IOPS values specific to your database. For example, each Metal node type has unique and very large IOPS capacity, so we customize it on a per-instance-type basis. But we also let you experiment with your own tunings. * Users have lots of freedom to customize their Postgres roles via our UI, giving them fine-grained control over permissions. * Both direct-to-Postgres and PgBouncer connections are available by default. Each has tradeoffs, and we let you choose what makes the most sense for your applications. The aim is to set you up with a strong foundation, and give users the tools they need to tune for their use-cases. ## Minimizing the impact of configuration changes PlanetScale takes the database unavailability produced by upgrades extremely seriously. We monitor this downtime across the fleet, and make continual efforts to reduce its impact. We apply context-appropriate, rather than a one-size fits all, upgrade strategies to ensure that a given upgrade takes no more time than absolutely necessary. PlanetScale allows manual configuration of a number of parameters via the [clusters](/docs/postgres/cluster-configuration) page. Each configuration change falls into one of two categories: * Reloadable changes: The change can be applied with no need to restart Postgres. When applying these, there is zero downtime. * Restart-required: Postgres requires the server be restarted for the change to take effect. When applying these, you can expect a brief period of server unavailability due to the restart. In the latter case, what we do is more sophisticated than a simple server restart. Because we have a primary and two replicas, we typically start these configuration changes by applying them to and restarting the replicas. Once these are ready, we do a switchover from the old primary to one of the replicas with the upgraded configuration. The config is then applied and the final instance restarted, now demoted to a replica. ![Config change with restart](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/config-change-restart.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=f4424ed370e5b727bc300b5bade335fe) This process allows us to minimize end-user downtime. There remains an unavoidable several-second window of unavailability. In addition, all direct connections will be terminated, so your application should have retry logic to accommodate for this. There are a few configuration options that are [required by Postgres](https://www.postgresql.org/docs/current/hot-standby.html#HOT-STANDBY-ADMIN) to be applied to the primary before replicas. For these, you will notice a slightly longer unavailability period. Normally a primary shutdown proceeds in a fast and orderly manner within a second or two, allowing the operator to promote a replica to primary. In cases where the operator is not able to quickly and cleanly shutdown the primary due to unresponsive user queries or transactions, the operator will failover to a replica after a timeout of 30 seconds. PgBouncer connections will persist through all config changes. ## Database cluster sizing PlanetScale does not support autoscaling of CPU and RAM. Instead, users are put in the driver's seat to choose the hardware the database runs on, and are given tools to closely monitor performance and adjust resources when needed. There are two main ways to increase the compute capacity of a cluster: Adding replicas and resizing all nodes. ### Adding Replicas Each production PlanetScale database (excluding single node) has 1 primary and 2 [replicas](/docs/postgres/scaling/replicas). In this standard configuration, read traffic can be [routed to a replica](/docs/postgres/scaling/replicas#how-to-query-postgres-replicas) rather than relying on the primary for everything. It's important not to over-stress your replicas, however. If all three nodes in the cluster experience too much strain, failovers will be more disruptive, since the cluster will temporarily go from 3 nodes down to 2. Reads from replicas must also be able to tolerate replication lag. If you only need to increase the read capacity of the cluster, a good option can be to add 1 or several additional replicas. This is done through the [Clusters](/docs/postgres/cluster-configuration) menu, and has no negative impact on the other nodes when being added. ### Resizing a cluster Resizing a cluster leads to database connections being terminated. Therefore, it's important for your application to have connection retry logic. Consider the case where you need to upgrade from an `M-160` database cluster to an `M-320`, doubling the compute resources of each node. After applying the upgrade, three new `M-320` nodes are created. These are caught up with the primary through a combination of a backup restore and data replication. ![Cluster Resize 1](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/cluster-resize-1.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=75487a1f038e75172bac84e1af3a424c) Once these new `M-320` replicas are sufficiently caught up, the operator transitions primaryship to one of the new `M-320` nodes. After this, the old `M-160` replicas are decommissioned, using the new ones for all replica traffic. During each node replacement, connections to the decommissioned node will be destroyed. New connections will need to be made to re-establish with one of the new nodes. ![Cluster Resize 2](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/cluster-resize-2.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=a9f1f7f07b884fa33ab45a8b2a587e22) During the upgrade process, all database connections will be terminated. Normally a primary promotion proceeds in a fast and orderly manner in less than 5 seconds. In cases where the operator is not able to quickly and cleanly shutdown the primary due to unresponsive user queries or transactions, the operator will failover to a replica after a timeout of 30 seconds. For all the steps leading up to the node replacement, we keep your existing `M-160` database cluster fully functional. Only brief periods of unavailability are required to ensure we can smoothly replace each node. Many ORMs have built-in connection retry logic to help with these scenarios. It's likely your application already has such capabilities if you are using libraries like Rails ActiveRecord, Laravel Eloquent, and Drizzle. ## Unplanned failures In a cloud-native environment, server failure is to be expected. This is unavoidable, so we embrace the behavior and have proven failover systems in place to address issues as quickly as possible. PlanetScale has executed millions of failovers across our Postgres and Vitess database clusters. They are a well-exercised path and are proven to be an invaluable mechanism for database operations and handling node failures. ### Replicas Replica health is important to keep your PlanetScale database cluster operating smoothly. Replica failures are automatically detected and automatically replaced. There will be a period of time where your cluster has only the primary and a single replica while the replacement replica is created and data restored to it. This time varies from minutes to hours depending on database size and your instance type. ![Failed replica](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/failed-replica.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=0ab7de4bfa00c15e925e8aff8fa48540) Any connections routed to the failed replica will be terminated and need to be re-established. When re-established, we will route the connections to the other available replica. ### Primaries The PlanetScale operator will detect a primary failure within seconds and immediately begin the process of replacing the node. The new primary is chosen based on which one is the most caught-up from the state of the primary. All Postgres databases are run with semi-sync replication, meaning that at least one replica should have received all the writes that the primary had seen. Once chosen, this replica is promoted to primary and a new node is brought up to replace it. ![Failed primary](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/failed-primary.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=38b162ee4ab9e2d09c23199d99e3daed) The primary failure, detection, and election process typically takes 10 seconds or less when all other components of the cluster are healthy. During this time your primary database will be inaccessible and no connections can be made. The time it takes for the new replica to catch-up to the primary varies from minutes to hours depending on database size and your instance type. For detailed guidance on configuring your application for maximum availability during failovers, see the [connection resilience guide](/docs/postgres/connection-resilience). ## Image upgrades We have fast iteration cycles at PlanetScale, always aiming to improve the features and capabilities of our customer's Postgres databases. Postgres minor version updates, PgBouncer updates, and adding support for new extensions all require us to update the container images for your database. When this happens, the Kubernetes pods that power your primary and replicas are updated. ![Image upgrade](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/image-upgrade.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=f2f03e225e63fc1d0606456053b21490) As with other operations described earlier, this leads to terminated connections and a brief period of database inaccessibility. Image upgrades are performed in cases of emergencies (such as patching security issues) or when initiated by the user via the PlanetScale application or API. ## Disk availability, scaling, and cost For databases using network-attached storage, autoscaling is enabled by default. This is to protect availability of your database. Reaching 100% full on a storage volume leads to database downtime. ![Disk autoscaling](https://mintcdn.com/planetscale-2/o_cHHlFu3sW-NBEp/postgres/disk-autoscaling.png?fit=max\&auto=format\&n=o_cHHlFu3sW-NBEp\&q=85\&s=9c69d1e0e590defb94d73d0633ea0aab) Cloud providers like AWS and GCP limit how frequently network-attached disks can be resized. In both cases, there is a multi-hour cooldown period between resizing operations. Also, these volumes typically do not support shrinking. When our disk auto-scaler is able to spread out disk scale-up sufficiently, no downtime is needed to scale the disks. This is true in the vast majority of cases. When data growth is rapid, the auto-scaler may need to complete a **surge resize** to support the writes. In this case, PlanetScale creates brand new database nodes with larger network-attached storage volumes. Once ready, the nodes in your cluster get replaced with these new ones to increase capacity. If the surge autoscaler is able to complete the resize before your disk fills, downtime will be minimal for growing the disk (seconds). If your disk fills before the new disks are ready, you will experience a longer period of downtime. We make every effort to keep your network-attached storage disk from filling, but it's important for the database administrators to pay close attention to storage and take manual intervention when necessary. # PlanetScale Postgres architecture Source: https://planetscale.com/docs/postgres/postgres-architecture PlanetScale Postgres is a managed PostgreSQL service built on modern cloud infrastructure. ## Overview Our architecture provides high availability through availability zone distribution, automated failover, and redundancy across multiple components. PlanetScale Postgres is built around a `shared nothing` architecture in alignment with PlanetScale's [principles of extreme fault tolerance](https://planetscale.com/blog/the-principles-of-extreme-fault-tolerance). This document explains the core components of our PostgreSQL infrastructure, from regional deployment patterns to cluster configuration options and operational capabilities. ## Regional and availability zone architecture ### Geographic distribution PlanetScale Postgres deploys database clusters across multiple availability zones within a single region. This design provides: * **Zone-level fault tolerance**: Database instances are automatically distributed across separate availability zones * **Network isolation**: Each availability zone operates independently with its own network infrastructure ### Cluster design ![PlanetScale Postgres cluster topology](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/postgres-arch-diagram.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=c20f53698428814045e9bc242956ffd9) PlanetScale Postgres uses a primary-replica architecture distributed across availability zones to provide high availability without compromising performance. This design ensures that your database can survive infrastructure failures while maintaining fast read and write operations. ### Primary-replica architecture Each production PostgreSQL cluster consists of: **Primary instance (1)**: * Handles all write operations * Located in one availability zone * Serves read operations when replicas are unavailable * Source of truth for data replication **Replica instances (2)**: * Handle read-only operations * Located in separate availability zones from primary * Continuously synchronized with primary through streaming replication * Available for promotion to primary during failover events ## Orchestration layer The orchestration layer manages the lifecycle, health, and operations of PostgreSQL clusters across multiple availability zones. This infrastructure layer handles everything from initial deployment to ongoing maintenance and failover operations. ### Kubernetes-based management Our PostgreSQL clusters run on Kubernetes infrastructure, providing: * **Automated deployment**: Consistent cluster provisioning across availability zones * **Health monitoring**: Continuous monitoring of database instance health * **Resource management**: Dynamic allocation of compute and storage resources * **Configuration management**: Centralized management of PostgreSQL parameters and settings ### Custom operator Our custom [Kubernetes operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) manages all cluster nodes and handles PostgreSQL operations that would otherwise require deep database knowledge. Based on our extensive experience running databases at massive scale, our operator was built for rock-solid handling of PostgreSQL replication, backup requirements, and the specific steps needed for safe failover operations: * **Failover coordination**: Automated promotion of replica instances when primary fails * **Backup scheduling**: Coordinated backup operations across all instances * **Configuration synchronization**: Ensures consistent settings across primary and replicas * **Scaling operations**: Manages instance resizing and replica addition/removal ## Data replication **Streaming replication**: PostgreSQL's built-in streaming replication keeps replicas synchronized with the primary in near real-time. Changes are continuously streamed to replicas, ensuring data consistency across all instances: * Continuous data streaming from primary to replicas * Changes confirmed by at least one replica * Automatic lag monitoring and alerting **Connection routing**: * Port 5432: Direct PostgreSQL connections * Port 6432: [PgBouncer](/docs/postgres/connecting/pgbouncer) connection pooling for optimized connection management * SSL/TLS encryption required for all connections ## Instance configurability PlanetScale Postgres offers flexible instance configuration to match your workload requirements and cost constraints. You can choose different CPU architectures, storage types, and performance characteristics to optimize for your specific use case. ![Creating a database](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/postgres-create-config.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=94073bce12d09880d08626279e653d03) ### CPU architecture options **ARM64 (AWS Graviton)**: * Cost-optimized instances with good performance characteristics * Compatible with all PostgreSQL features and extensions * Lower power consumption and cost per compute unit **x86-64 (Intel/AMD)**: * Traditional architecture with broad compatibility * Optimized for single-threaded performance requirements * Full compatibility with existing tooling and applications Learn more about [CPU architecture selection](/docs/postgres/cluster-configuration/cpu-architectures). ### Storage types Storage choice significantly impacts database performance and cost. PlanetScale offers two distinct storage options optimized for different workload patterns: **PlanetScale Metal**: * Direct-attached NVMe storage for maximum performance * Lower I/O latency compared to network-attached storage * Fixed storage capacity based on instance size * Optimal for I/O-intensive workloads **Network-attached storage (EBS)**: * Flexible storage scaling up to 64 TB * Configurable IOPS and throughput settings * Automatic storage scaling based on usage patterns * Cost-effective for variable storage requirements Learn more about [storage configuration](/docs/postgres/cluster-configuration/cluster-storage). ### Performance and scaling relationships ![Database dashboard summary](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/postgres-dashboard-summary-metal.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=4e48e13d3af6eb8460d9617af8131113) Understanding how different configuration choices affect performance helps you optimize your database for both cost and performance. These relationships guide capacity planning and scaling decisions: **Compute scaling**: * Instance size directly affects CPU, memory, and network capacity * Larger instances support more concurrent connections * Memory allocation affects query performance and caching efficiency **Storage performance**: * Network-attached storage can autoscale to meet storage needs * Network-attached storage allows for configurable IOPS and disk bandwidth * PlanetScale Metal provides increased IOPS and ultra-low latency storage * Throughput limits vary by instance size and storage configuration ## Operational capabilities PlanetScale Postgres provides comprehensive operational tools that give you visibility into database performance, health, and behavior. These capabilities help you monitor, troubleshoot, and optimize your database without requiring additional setup or external tools. ### Insights and query analysis ![PlanetScale Insights](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/postgres-insights.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=9ecdae873efd3bca8329b8f3494c54d8) **Query performance insights**: * Automatic detection of slow-running queries and anomalies * Query execution plan analysis and recommendations * Historical query performance tracking * Identification of resource-intensive operations ### Metrics and monitoring Comprehensive metrics collection provides real-time visibility into database health and performance trends. All metrics are collected automatically and made available through both the dashboard and programmatic interfaces: **Built-in metrics collection**: * CPU, memory, and storage utilization across all instances * Connection count and pooling efficiency * Replication lag and synchronization status Learn more about [monitoring and metrics](/docs/postgres/monitoring/metrics). ### Logs and diagnostics Centralized logging aggregates all database-related logs in a searchable format with 7-day retention. This unified logging system helps with troubleshooting, security monitoring, and performance analysis: **Centralized logging**: * PostgreSQL server logs from all instances * Query execution logs with configurable detail levels * Error logs with automatic categorization and alerting * Connection and authentication logs for security monitoring **Log analysis tools**: * Search and filtering capabilities across all log sources * Export capabilities for external analysis tools ### Cluster configuration options ![Tracking changes](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/postgres-config-changes.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=ad1554779e5fba5ddd2144d44bad095d) Flexible configuration options allow you to customize PostgreSQL behavior, enable additional functionality, and optimize performance for your specific workload requirements: **Extension management**: * Curated set of PostgreSQL extensions tested for compatibility * Both Native and Community extensions available * Managed through dashboard or CLI Learn more about [extension configuration](/docs/postgres/extensions). **Parameter tuning**: * Pre-configured parameter sets optimized for different workload types * Customizable PostgreSQL configuration parameters * Automatic parameter adjustment based on instance size changes * Configuration change tracking Learn more about [parameter configuration](/docs/postgres/cluster-configuration/parameters). **Custom Backups and Point-in-time recovery**: ![Postgres PITR](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/postgres-pitr.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=ab0d860a3f430875b0544458371dc534) * Automated backup scheduling with configurable retention periods * Custom backup timing to minimize impact on production workloads * Point-in-time recovery capabilities within retention windows Learn more about [backup and restore operations](/docs/postgres/backups). ## Development workflow integration PlanetScale Postgres integrates with development workflows through database branching and comprehensive migration support. These features enable safe schema changes and smooth transitions from other PostgreSQL platforms. ### Database branching **Environment isolation**: * Create isolated database environments from production backups * Independent schema and data changes without affecting production * Cost-optimized single-instance architecture for development branches * Automatic promotion to full high-availability architecture when needed ### Third-party integrations PlanetScale Postgres integrates with popular monitoring and development tools through standard protocols and APIs. These integrations allow you to incorporate database metrics into existing workflows and toolchains: **Monitoring system integrations**: * [Prometheus endpoints](/docs/postgres/monitoring/prometheus-postgres) for custom metrics collection * [Datadog integration](/docs/postgres/monitoring/prometheus-metrics-datadog-postgres) for unified monitoring dashboards **Development tool integrations**: * Standard PostgreSQL connection protocols for universal tool compatibility * Support for popular database administration tools * API access for programmatic management and monitoring ## Related documentation > **[Cluster Configuration](/docs/postgres/cluster-configuration)** > **[Scaling with Replicas](/docs/postgres/scaling/replicas)** > **[Backup and Recovery](/docs/postgres/backups)** > **[Monitoring and Metrics](/docs/postgres/monitoring/metrics)** > **[Database Branching](/docs/postgres/branching)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres compatibility Source: https://planetscale.com/docs/postgres/postgres-compatibility PlanetScale Postgres is built on standard PostgreSQL, providing full compatibility with PostgreSQL features. This page documents the specific behaviors, limitations, and differences you should be aware of when using PlanetScale Postgres. ## Overview PlanetScale Postgres runs on PostgreSQL versions 17 and 18. See our [supported versions](/docs/postgres/cluster-configuration/versions) documentation for the specific versions available. PlanetScale Postgres is designed to be fully compatible with standard PostgreSQL. Unlike PlanetScale Vitess, which has [certain MySQL compatibility limitations](/docs/vitess/mysql-compatibility), PlanetScale Postgres supports the full PostgreSQL feature set, including stored procedures, triggers, and foreign keys. ## Full PostgreSQL feature support PlanetScale Postgres supports all core PostgreSQL features: | Feature | Support | Notes | | - | - | - | | Stored procedures and functions | ✅ | Full support for `CREATE FUNCTION` and `CREATE PROCEDURE` | | Foreign keys and constraints | ✅ | All constraint types supported | | Triggers | ✅ | All trigger types supported | | Materialized views | ✅ | Full support including `REFRESH MATERIALIZED VIEW` | | Transactions and ACID | ✅ | Full transactional support | | JSON/JSONB | ✅ | All JSON functions and operators | | Full-text search | ✅ | Including `tsvector`, `tsquery`, and related functions | | Table partitioning | ✅ | Range, list, and hash partitioning | | Logical replication | ✅ | As source or target — see [logical replication documentation](/docs/postgres/integrations/logical-cdc) | | Window functions | ✅ | Full support | | Common Table Expressions (CTEs) | ✅ | Including recursive CTEs | | `LISTEN`/`NOTIFY` | ✅ | Full support for pub/sub messaging | | `CREATE DATABASE` / multiple logical databases | ✅ | Create multiple PostgreSQL databases within a single cluster | ## Tool compatibility PlanetScale Postgres works with the PostgreSQL ecosystem: * **Drivers**: Standard PostgreSQL drivers and connection strings (libpq, psycopg, pg, node-postgres, etc.) * **ORMs**: Prisma, ActiveRecord, Sequelize, Drizzle, Django ORM, SQLAlchemy, and others * **Monitoring**: pganalyze, Datadog DBM — see [monitoring documentation](/docs/postgres/monitoring/pganalyze) * **Migration tools**: pgcopydb, AWS DMS — see [import documentation](/docs/postgres/imports/postgres-imports) * **CDC tools**: Debezium, Fivetran, Airbyte, ClickPipes — see [CDC documentation](/docs/postgres/integrations/logical-cdc) ## PlanetScale-specific behaviors While PlanetScale Postgres is fully PostgreSQL-compatible, there are some platform-specific behaviors to be aware of. ### Roles and permissions > [!NOTE] > > The following reference describes PlanetScale-specific permission behaviors. | Behavior | Description | | - | - | | **No SUPERUSER access** | PlanetScale does not grant `SUPERUSER` privileges as part of our security model. The default role has near-superuser capabilities. See [role management](/docs/postgres/connecting/roles) for details. | | **Default role permissions** | The default `postgres` role has `NOSUPERUSER CREATEDB CREATEROLE INHERIT LOGIN REPLICATION BYPASSRLS` and inherits permissions including `pg_read_all_data`, `pg_write_all_data`, `pg_monitor`, `pg_read_all_settings`, `pg_read_all_stats`, `pg_stat_scan_tables`, `pg_signal_backend`, `pg_checkpoint`, `pg_maintain`, `pg_use_reserved_connections`, and `pg_create_subscription`. See [role management](/docs/postgres/connecting/roles) for the complete list. | | **User-defined roles** | Create application-specific roles with minimal required permissions. Roles created via the dashboard, CLI, or API are managed by PlanetScale. | Most operations that would require `SUPERUSER` in vanilla PostgreSQL work with PlanetScale's default role. If you encounter a specific use case requiring elevated permissions, [contact support](https://planetscale.com/contact?initial=support). ### Extensions PlanetScale supports a curated list of vetted PostgreSQL extensions. See the [extensions documentation](/docs/postgres/extensions) for the complete list. | Behavior | Description | | - | - | | **Curated extension list** | Extensions must be on our supported list. You can [request new extensions](https://ps-extensions.io/) or contact support. | | **Version pinning** | Extension versions are pinned to each PostgreSQL major version and do not auto-upgrade. | | **Restart requirements** | Some extensions require a cluster restart to enable. These are marked in the [extensions documentation](/docs/postgres/extensions#supported-extensions). | | **Dashboard configuration** | Extensions requiring shared memory or background workers must be enabled through the PlanetScale dashboard. | > [!WARNING] > > When migrating to PlanetScale Postgres, verify that your required extensions are supported and check version compatibility. Extensions like PostGIS may have version differences that affect functionality. ### Branch behavior PlanetScale Postgres uses [branches](/docs/postgres/branching) for isolated database environments. | Behavior | Description | | - | - | | **Configuration not inherited** | New branches start with default configuration. Extensions and custom parameters must be re-enabled on each branch. | | **Isolated environments** | Branches are completely isolated. Schema and data changes in one branch do not affect others. | | **No automatic schema sync** | Unlike PlanetScale Vitess deploy requests, schema changes must be manually applied to each branch. | ### Configuration and logging | Behavior | Description | | - | - | | **Log line prefix** | The `log_line_prefix` does not include database (`%d`) and user (`%u`) identifiers by default, but can be [customized](/docs/postgres/cluster-configuration/parameters) to include them. | | **Default search path** | Default `search_path` is `"$user", public`. Custom search paths from source databases need manual configuration via `ALTER ROLE ... SET search_path = 'your_schema'`. | | **Parameter changes** | Some parameters require a cluster restart. See [parameters documentation](/docs/postgres/cluster-configuration/parameters) for details. | ### Operational characteristics | Behavior | Description | | - | - | | **Failover behavior** | Production clusters have 1 primary and 2 replicas. Failovers typically complete in seconds. See [operations philosophy](/docs/postgres/operations-philosophy). | | **Disk autoscaling** | Storage autoscaling is enabled by default for network-attached storage clusters, automatically growing disk when utilization thresholds are reached. See [storage configuration](/docs/postgres/cluster-configuration/cluster-storage). | | **No CPU/RAM autoscaling** | Cluster sizing is manual. Use the Clusters page to monitor utilization and resize as needed. | ## Migrating to PlanetScale Postgres For detailed migration guidance, including step-by-step instructions for importing your existing PostgreSQL database, see our [import documentation](/docs/postgres/imports/postgres-imports). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres pricing Source: https://planetscale.com/docs/postgres/pricing PlanetScale Postgres databases are billed based on their configuration and usage. ## Overview Understanding the relationship between **databases**, **branches**, and **clusters** is key to understanding your billing: * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups. [Learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch **Each branch runs on its own dedicated cluster** and is billed separately based on its configuration and usage. * You are billed for the compute and storage resources (cluster) that power each branch (prorated to the millisecond each month) * Every database branch you launch has default included amounts of: * [Cluster disk storage](#cluster-disk-storage) * [Backup storage](#backups) * [Network egress data transfer](#network-data-transfer) * [Replicas](#additional-replicas) * Usage beyond the included defaults is charged based on the costs for that resource, this includes: * Cluster disk storage used beyond the included amount (for network-attached storage) * Backup storage used beyond the included amount * Network egress beyond the included amount * Additional replicas * [Dedicated PgBouncer](#pgbouncer-pricing) instances There are no upfront commitments, and as you adjust your configuration or usage changes, pricing will automatically adjust from that point. ## Query live prices with SQL Cluster SKU prices are available in a public, read-only Postgres table: `public.planetscale_prices`. You and your agent can query live prices with `SELECT`. The role can only `SELECT` on that table. SSL is required. Connect through PgBouncer: ``` postgresql://pscale_api_hhdcchpiodja.dzken160d27q:pscale_pw_wa09rWwuQZQEWGS2eL0v7BlSGriZ6xgv@aws-us-east-1-3.pg.psdb.cloud:6432/postgres?sslmode=verify-full&sslrootcert=system ``` Each row is one priced cluster. Filter on `product` (`Postgres` or `Vitess`), `provider`, `region`, `machine_sku`, `replication_factor`, and `disk_sku` (empty string for `PS-*` network-attached SKUs). `price` is the monthly cluster cost in whole USD. It does not include extra storage, backups, egress, dedicated PgBouncers, or extra replicas. ```sql theme={null} -- Postgres, 3-node HA PS-160 ARM in us-east-1 SELECT machine_sku, replication_factor, price FROM planetscale_prices WHERE product = 'Postgres' AND machine_sku = 'PS-160-ARM' AND provider = 'AWS' AND region = 'us-east-1' AND disk_sku = '' AND replication_factor = 3; ``` ```sql theme={null} -- Vitess, 3-node HA PS-160 in us-east-1 SELECT machine_sku, replication_factor, price FROM planetscale_prices WHERE product = 'Vitess' AND machine_sku = 'PS-160' AND provider = 'AWS' AND region = 'us-east-1' AND disk_sku = '' AND replication_factor = 3; ``` > [!NOTE] > > Not every SKU exists in every region. If a query returns no rows, that configuration is not sold there. Do not invent a price. Vitess SKUs in this catalog are mostly x86; there is no `PS-160-ARM` Vitess row. Column meanings, more example queries, and an agent-friendly catalog are in [pricing.md](https://planetscale.com/pricing.md). The connection URI is also on the [pricing page](https://planetscale.com/pricing#pricing-database). > **[pricing.md](https://planetscale.com/pricing.md)** > > For an agent-friendly catalog of live cluster SKU rates and example SQL, fetch `https://planetscale.com/pricing.md`. Prefer [https://planetscale.com/pricing.md](https://planetscale.com/pricing.md) for live Vitess, Neki, and Postgres cluster SKU rates. > [!NOTE] > > For information on Vitess cluster pricing on the Base plan, see [Base plan cluster pricing](/docs/plans/cluster-sizing). ## Cluster instance size Each database branch runs on its own dedicated cluster infrastructure. Clusters are billed based on the selected instance size and prorated to the millisecond. Each instance size includes defined amounts of vCPU cores and memory, as well as storage based on the storage type selected ([network-attached storage](/docs/postgres/cluster-configuration/cluster-storage) or [PlanetScale Metal](/docs/metal)). The dashboard and [pricing calculator](https://planetscale.com/pricing) show the current monthly rate for each size. How that rate is applied is described under [cluster pricing](#cluster-pricing). You can [increase or decrease your cluster size](/docs/postgres/cluster-configuration) at any time. Pricing is prorated to the millisecond, so if you temporarily increase, you will only be charged for the larger cluster size for the time that it was running. Billing for a modified cluster size begins once the resize is completed. You can also [spin up additional production branches](/docs/postgres/branching) at any time for additional cost. The pricing for these is also prorated. ## Cluster disk storage ### Storage types explained PlanetScale offers two storage options with different pricing models: | [PlanetScale Metal](/docs/metal) | [Network-Attached Storage](/docs/postgres/cluster-configuration/cluster-storage) | | - | - | | **All-inclusive pricing** | **Configuration-based pricing** | | - Storage cost is included in your cluster instance price - High-performance NVMe SSDs directly attached to your servers - To get more storage, you upgrade to a larger cluster size - Best for: High-performance applications that need predictable costs | - First 10 GB included with each cluster - Additional storage is billed separately from your cluster instance configuration - Flexible scaling - add storage without changing cluster size - Automatic disk scaling with [autoscaling](/docs/postgres/cluster-configuration/cluster-storage#enable-autoscaling) - Best for: Applications with variable storage needs or cost optimization | ### Network-attached storage billing For network-attached storage clusters, pricing varies by region and cloud provider: * **Base storage**: you manually configure this in cluster settings, unless [autoscaling](/docs/postgres/cluster-configuration/cluster-storage#enable-autoscaling) is enabled * **Additional IOPS**: you manually configure this in cluster settings (AWS only) * **Additional Throughput**: you manually configure this in cluster settings (AWS only) > [!NOTE] > > AWS gp3 and GCP Hyperdisk Balanced bill provisioned IOPS and throughput above their included baselines. GCP PD-SSD has volume-only pricing, with performance determined by disk size. Billing for modified storage attributes begins once the change for the cluster has completed. **What are IOPS and Throughput?** * **IOPS** (Input/Output Operations Per Second): How many read/write operations your database can handle per second. Higher IOPS = faster response for many small database queries. * **Throughput**: How much data can be transferred per second (MiB/s). Higher throughput = faster for large data operations like bulk imports. **Default values for AWS gp3 storage:** All network-attached storage clusters on AWS come with the following baseline performance included at no additional cost: | Setting | Default value | | - | - | | IOPS | 3,000 IOPS | | Throughput (Bandwidth) | 125 MiB/s | These defaults apply to all PS-series cluster sizes (PS-5 through PS-2560). You only pay additional IOPS or throughput charges if you manually configure values above these baselines. For more information on changing your cluster's storage, see the [Cluster storage configuration](/docs/postgres/cluster-configuration/cluster-storage) documentation. ## Backups ### Included backup storage PlanetScale automatically includes backup storage with every database branch at no extra cost. Each branch gets backup storage equal to **2x your disk size**. Examples: | Disk Size | Backup Storage Included | | :- | :- | | 25 GB | 50 GB | | 50 GB | 100 GB | | 1000 GB | 2000 GB | > [!NOTE] > > For clusters with [network-attached storage](/docs/postgres/cluster-configuration/cluster-storage) with [autoscaling](/docs/postgres/cluster-configuration/cluster-storage#enable-autoscaling) enabled, **backup** storage changes as your **disk storage** autoscales. When your backup and WAL storage exceeds the included amount, additional storage is billed at **\$0.023 per GB per month**. **Storage overage examples**: | Storage Overage | Duration | Cost | | :- | :- | :- | | 100 GB | 30 days | \$2.30 | | 100 GB | 15 days | \$1.15 | ### How backup billing works **Storage calculation**: Both database data backups and WAL files are automatically compressed before being sent to backup storage. You're billed only on the compressed size, which is typically much smaller than your active database size. Backup storage usage billing data is updated hourly. For more details on backup functionality, see [Back up and restore](/docs/postgres/backups). **What affects storage costs**: | Factor | Impact | | :- | :- | | **Database activity** | More active databases generate more WAL files | | **Backup retention** | Longer retention periods increase storage usage | | **Oldest backup age** | WAL files are kept as long as your oldest backup exists | ## Network data transfer ### Public traffic Outgoing network traffic (egress) is billed based on the amount of data transferred out of PlanetScale Postgres instances to any other host or destination. Data transfer for [backups](/docs/postgres/backups) and [replication within a primary cluster](/docs/postgres/scaling/replicas) is free. Cross-region Postgres dedicated read replica traffic is billed separately. Each branch has a default amount of included public network egress that is aggregated for all cluster instances in the entire branch. | Branch Type | Included Public Network Egress | | - | - | | Production | 100 GB per month | | PS-5 non-HA | 10 GB per month | | Development (PS-DEV) | 10 GB per month | Fetch the current public egress rate for a region from the [`/www/egress-rates`](https://api.planetscale.com/www/egress-rates?region=us-east) endpoint. Pass a supported region identifier in the `region` query parameter. Incoming public network traffic (ingress) is free. ### Private traffic For databases using [private connections](/docs/postgres/connecting/private-connections) (AWS PrivateLink or GCP Private Service Connect), all network traffic over that connection — both egress and ingress — is charged at a flat rate of **\$0.01/GB**, regardless of region. This replaces the standard regional egress pricing for traffic routed over private connections. Each branch has a default amount of included private network traffic that is aggregated for all cluster instances in the entire branch. | Branch Type | Included Private Network Traffic | | - | - | | Production | 100 GB per month | | PS-5 non-HA | 10 GB per month | | Development (PS-DEV) | 10 GB per month | ### Cross-region dedicated read replica traffic Cross-region traffic between a Postgres primary cluster and a dedicated read replica is billed per GB with no included allowance. * **Streaming**: WAL streamed from the primary cluster to the dedicated read replica. * **Archive**: WAL fetched from the primary cluster's backups after streaming is interrupted. * **Bootstrap**: The base backup and WAL downloaded when a dedicated read replica is created. Each replica downloads a separate copy. > [!NOTE] > > Traffic is billed at the higher rate of the primary cluster’s region or the dedicated read replica’s region. | Cloud provider | Regions | Streaming per GB | Archive + bootstrap per GB | | :- | :- | :- | :- | | AWS | `us-east-1` Northern Virginia `us-east-2` Ohio `us-west-2` Oregon `ca-central-1` Montreal `eu-central-1` Frankfurt `eu-west-1` Dublin `eu-west-2` London | \$0.04 | \$0.03 | | AWS | `ap-south-1` Mumbai `ap-northeast-1` Tokyo `ap-southeast-1` Singapore | \$0.11 | \$0.10 | | AWS | `ap-southeast-2` Sydney | \$0.12 | \$0.11 | | AWS | `sa-east-1` Sao Paulo | \$0.16 | \$0.15 | | GCP | `us-central1` Iowa `us-east1` South Carolina `us-east4` Virginia `northamerica-northeast1` Montreal | \$0.02 | \$0.02 | | GCP | `europe-west1` Belgium `europe-west4` Netherlands | \$0.05 | \$0.05 | | GCP | `asia-northeast3` Seoul | \$0.08 | \$0.08 | ## Additional replicas Each production cluster includes 2 [replicas](/docs/postgres/scaling/replicas) (excluding [Single Node](/docs/postgres/cluster-configuration/single-node)) to provide high availability and additional read capacity alongside the primary. Certain read-heavy workloads may demand additional read replicas. If you need additional replicas beyond what is included, you can add them for an additional cost based on the branch's instance size and storage requirements. The cost of an additional replica instance is about one third of the highly available SKU rate. New replica billing begins once the change has completed. Replicas with network-attached storage do not receive the 10 GB included disk allowance. You pay for the full configured disk size on each extra replica, plus any IOPS or throughput you provision above the baselines. Development branches and Single Node databases do not support replicas. See [Replicas](/docs/postgres/scaling/replicas). ## Cluster pricing The dashboard and [pricing calculator](https://planetscale.com/pricing) show the monthly SKU rate for each cluster size in your region. That rate is applied as follows: * **Single Node** — you pay the single-node SKU rate for the time that size is running. * **Highly Available** — you pay the highly available SKU rate, which covers one primary and two replicas. * **Resizes** — billing for the new size starts when the resize completes and is prorated to the millisecond. * **Metal** — local NVMe storage is included in the Metal SKU rate. Network-attached storage is billed separately, as described under [Cluster disk storage](#cluster-disk-storage). Query live SKU prices with SQL from the public `planetscale_prices` table. See [Query live prices with SQL](#query-live-prices-with-sql). > [!TIP] > > Visit the [pricing calculator](https://planetscale.com/pricing) for interactive, region-specific SKU rates. ## Storage pricing Network-attached storage is billed per database instance. The primary and two replicas in an HA cluster each use the configured volume, IOPS, and throughput. Additional replicas use another full copy of the configured volume and do not receive the 10 GB included allowance. For storage types with an included allowance, monthly volume cost before proration is: ```text theme={null} included-instance volume cost = max(average storage GB - 10 GB, 0) × volume rate × included instances additional-replica volume cost = average storage GB × volume rate × additional replicas ``` IOPS and throughput costs before proration are: ```text theme={null} IOPS cost = max(provisioned IOPS - included IOPS, 0) × IOPS rate × total instances throughput cost = max(provisioned MiB/s - included MiB/s, 0) × throughput rate × total instances ``` | Storage type | Included performance | Additional charges | | :- | :- | :- | | AWS gp3 | 3,000 IOPS and 125 MiB/s | Provisioned IOPS and throughput above the included baselines | | GCP Hyperdisk Balanced | 3,000 IOPS and 140 MiB/s | Provisioned IOPS and throughput above the included baselines | | GCP PD-SSD | Performance scales with disk size | No separate IOPS or throughput charge | Volume rates vary by region. Fetch the current rate from the [`/www/storage-rates`](https://api.planetscale.com/www/storage-rates?region=us-east) endpoint: * AWS regions return the gp3 volume rate. * GCP regions return the Hyperdisk Balanced volume rate by default. * Add `pd-ssd=true` for the GCP PD-SSD volume rate, for example [`gcp-us-central1&pd-ssd=true`](https://api.planetscale.com/www/storage-rates?region=gcp-us-central1\&pd-ssd=true). The endpoint returns the volume rate per GB per month. The dashboard shows the IOPS and throughput rates available for the selected storage type and region. ## PgBouncer pricing The local PgBouncer on the primary is included in the cluster SKU rate. A dedicated PgBouncer is a separate SKU. Its cost is the selected size's monthly rate multiplied by the configured **PgBouncers per availability zone**, prorated for the time each configuration runs. ```text theme={null} dedicated PgBouncer cost = PgBouncer SKU rate × PgBouncers per availability zone ``` Available dedicated sizes: | Size | CPU | Memory | | - | - | - | | PGB-5 | 1/16 vCPU | 128 MB | | PGB-10 | 1/8 vCPU | 256 MB | | PGB-20 | 1/4 vCPU | 512 MB | | PGB-40 | 1/2 vCPU | 1 GB | | PGB-80 | 1 vCPU | 2 GB | | PGB-160 | 2 vCPU | 4 GB | The dashboard and [pricing calculator](https://planetscale.com/pricing) show the current monthly rate for each size in your region. See [PgBouncer](/docs/postgres/connecting/pgbouncer). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Dedicated read replicas Source: https://planetscale.com/docs/postgres/scaling/dedicated-read-replicas Independently sized dedicated read replicas for scaling Postgres reads across locations. ## Overview Dedicated read replicas maintain continuously updated copies of a production branch in locations you select. Use them to serve read traffic closer to your applications and users or to isolate read-heavy workloads from the primary cluster. A dedicated read replica differs from the [replicas in your primary cluster](/docs/postgres/scaling/replicas): * Primary-cluster replicas provide high availability and read capacity within the primary location. * Dedicated read replicas are distincy Postgres nodes from the primary cluster and provide independently sized read capacity in a location you select. Each dedicated read replica has a unique name and contains one or more read-only instances. Configure its location, cluster size, replica count, storage, and supported Postgres parameters independently of the primary cluster. You can create multiple dedicated read replicas in the same location. Dedicated read replicas can be created in the same cloud region as the primary or different ones from the same cloud privider. You may not, for example, have your primary in an AWS region and a dedicated read replica in a GCP region. Database administrators and organization administrators can create and manage dedicated read replicas. > [!WARNING] > > PlanetScale asynchronously replicates changes to dedicated read replicas. Applications that query them must tolerate [replication lag and temporarily stale results](/docs/postgres/scaling/replicas#data-consistency-and-replication-lag). ## Create a dedicated read replica **Step 1** In the [PlanetScale dashboard](https://app.planetscale.com), select your database and production branch. **Step 2** Select **Clusters** in the left navigation, then select the **Dedicated read replicas** tab. **Step 3** Select **Add a dedicated read replica**. **Step 4** Enter a unique name, then choose the location, number of replicas, cluster size, and storage settings. The dashboard shows the estimated monthly compute and storage cost. **Step 5** Select **Add dedicated read replica**. The dedicated read replica appears with a **Provisioning** status until it is ready. ## Change a dedicated read replica Set the cluster size, replica count, storage, and supported Postgres parameters independently for each dedicated read replica. **Step 1** Open the production branch's **Clusters** page and select **Dedicated read replicas**. **Step 2** Change the **Replica size** or **Number of replicas** for the cluster. **Step 3** For network-attached storage, configure the minimum disk size, storage limit, autoscaling, IOPS, or throughput. **Step 4** To change a supported Postgres parameter, expand **Customize cluster parameters** and enter the new value. **Step 5** Select **Save replica changes**. Changes apply immediately. The **Changes** tab shows the status and history of dedicated read replica changes. Storage settings apply independently to each dedicated read replica. For storage controls and limits, see [Cluster storage configuration](/docs/postgres/cluster-configuration/cluster-storage). Set parameter values on a dedicated read replica at or above the corresponding primary-cluster values. Changing a parameter restarts the dedicated read replica. For PlanetScale Metal, a dedicated read replica size must have enough storage for the primary cluster's current disk usage and required headroom. ## Connect to a dedicated read replica Managed roles and passwords work across the primary cluster and its dedicated read replicas. The host and username are different for each dedicated read replica. The dashboard displays the connection details at creation time. **Step 1** Open the production branch and select **Connect**. **Step 2** Create or select a role. **Step 3** Under **Connection target**, choose the named dedicated read replica that you want to connect to. **Step 4** Copy the connection string or individual connection values that the dashboard provides. You can also select a read-only connection target from a role's details page under **Settings** > **Roles**. > [!NOTE] > > The generated username ends in `|replica`, which distributes connections across the read-only instances in the selected cluster. Dedicated read replicas reject `INSERT`, `UPDATE`, `DELETE`, and other write operations. To retrieve connection details from the CLI, pass the dedicated read replica name to `pscale role get`: ```bash theme={null} pscale role get --dedicated-read-replica ``` ## Monitor a dedicated read replica Prometheus metrics for a dedicated read replica use `planetscale_database_branch_id` to identify the replica and `planetscale_upstream_database_branch_id` to identify its parent database branch. See the [Prometheus metrics reference](/docs/postgres/monitoring/prometheus-metrics-postgres). ## Manage dedicated read replicas with the API The [PlanetScale API](/docs/api/reference/getting-started-with-planetscale-api) supports dedicated read replica management: * [List dedicated read replicas](/docs/api/reference/list_dedicated_read_replicas) * [Create a dedicated read replica](/docs/api/reference/create_dedicated_read_replica) * [Get a dedicated read replica](/docs/api/reference/get_dedicated_read_replica) * [Update a dedicated read replica](/docs/api/reference/update_dedicated_read_replica) * [Delete a dedicated read replica](/docs/api/reference/delete_dedicated_read_replica) * [List dedicated read replica change requests](/docs/api/reference/list_dedicated_read_replica_change_requests) Creating a dedicated read replica requires a unique name and region slug. The API defaults to one instance and the primary cluster's size when you omit `replicas` and `cluster_size`. Create and update requests accept a `storage` object containing `minimum_storage_bytes`, `maximum_storage_bytes`, `storage_autoscaling`, `storage_iops`, and `storage_throughput_mibs`. Get, update, and delete requests identify a dedicated read replica by name. To retrieve role connection details for a specific cluster, pass its name in the [`dedicated_read_replica` query parameter](/docs/api/reference/get_role). ## Delete a dedicated read replica **Step 1** Open the production branch's **Clusters** page and select **Dedicated read replicas**. **Step 2** Select **Delete replica cluster** for the cluster you want to remove. **Step 3** Confirm by selecting **Delete replica cluster**. > [!WARNING] > > Deleting a dedicated read replica is irreversible and stops its connection endpoint. ## Pricing The dashboard displays the estimated monthly compute and storage cost when you create or change a dedicated read replica. PlanetScale prorates charges when you create, delete, or resize clusters. Compute charges consist of: * The location's dedicated read replica rate for the first instance. * The standard additional-replica rate for every instance after the first. For network-attached storage, PlanetScale bills every instance in a dedicated read replica for its configured storage, IOPS, and throughput. PlanetScale Metal cluster prices include their local storage. Cross-region replication traffic for Postgres dedicated read replicas is billed per GB with no included allowance. See [cross-region dedicated read replica traffic pricing](/docs/postgres/pricing#cross-region-dedicated-read-replica-traffic). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Database replicas Source: https://planetscale.com/docs/postgres/scaling/replicas A replica is a continuously updated copy of your Postgres database. ## Overview Replicas serve two main purposes: * They provide a way to reduce load on your primary instance by allowing you to read from a replica. * They increase database availability by enabling fast failovers for maintenance or unexpected failure. > [!WARNING] > > Before utilizing replicas for reducing load on the primary, it's important to understand the trade-offs. For more information, see the [Data consistency and replication lag](#data-consistency-and-replication-lag) section. > [!NOTE] > > The replicas described here run within the primary cluster and its location. To create independently sized read capacity in the same or other locations, see [Postgres dedicated read replicas](/docs/postgres/scaling/dedicated-read-replicas). ## How to query Postgres replicas Postgres replicas can be used to read data and reduce load on the primary. PlanetScale does not automatically route queries to replicas unless you explicitly use a replica credential or tell your application to do so. > [!NOTE] > > **PlanetScale's use of PgBouncer (port `6432`) does not support replica routing.** All connections through PgBouncer are automatically routed to the primary database, regardless of the username specification. Use direct connections (port `5432`) for replica access. To query a replica: 1. Append `|replica` to the end of your username for the branch you want to target: ```bash theme={null} user=postgres.cnlmx96ec5kw|replica ``` > [!NOTE] > > If you're using a URI-formatted connection string (for example, `DATABASE_URL`), URL-encode the pipe character in the username suffix. Use `%7Creplica` instead of `|replica`. > > ```text theme={null} > postgresql://postgres.cnlmx96ec5kw%7Creplica:pscale_pw_xxx@host:5432/my_database?sslmode=verify-full&sslrootcert=/etc/ssl/certs/ca-certificates.crt > ``` 1. Make sure the port in your connection string is set to `5432`. To pool connections to replicas, use a [dedicated replica PgBouncer](/docs/postgres/connecting/pgbouncer#dedicated-replica-pgbouncers) instead. Your connection string should look something like this: ```bash theme={null} psql 'host=xxxxxxxxxx-useast1-1.horizon.psdb.cloud \ port=5432 \ user=postgres.xxxxxxxxxx|replica \ password=pscale_pw_xxxxxxxxxxxxxxxxxx \ dbname=my_database \ sslnegotiation=direct \ sslmode=verify-full \ sslrootcert=system' ``` ## High availability Replicas within PlanetScale are used to enable high availability of your database. This is a part of the reason all production branches (excluding [single node](/docs/postgres/cluster-configuration/single-node)) in PlanetScale are provided 2 replicas. In situations where the underlying hardware or service hosting the primary Postgres node fails, our system will automatically elect a new primary node from the available replicas and reroute traffic to that new primary. This process is known as **reparenting** and typically is all handled within milliseconds or seconds. If you're using [PgBouncer](/docs/postgres/connecting/pgbouncer), querying the primary during a reparent typically goes unnoticed, other than a bit of additional query latency. This is because PlanetScale's PgBouncer buffers queries under the hood during failovers and directs them to the new primary once the failover is complete. ## Multiple availability zones In cloud architecture, regions are further broken down into logical groupings of data centers known as availability zones (or AZs for short). For example, the `us-east-1` region on AWS contains multiple AZs available to customers starting with `us-east-1a` through `us-east-1f`. The infrastructure for your PlanetScale Postgres database cluster is distributed across 3 availability zones. In the instance of an AZ failure, your database will automatically failover to an available AZ. ## Data consistency and replication lag PlanetScale Postgres utilizes a consistency model in which data committed (`INSERT`, `UPDATE`, `DELETE`, etc) on the primary must be durably stored on and confirmed by at least one replica before the primary reports the commit succeeded to client. > [!NOTE] > > This consistency model does mean that it's possible for a commit to be visible to other clients/transactions on the primary or the replicas, before the primary reports commit success back to the client. For development branches there is no data replication. The delay between when a change is applied to a primary and the same change is applied to a replica is known as `replication lag`. Your database's replication lag is viewable on your database main "Dashboard" page. It is important to be aware of replication lag whenever querying data from your replicas. For example, if you make an update and then immediately try to query for that updated data via a replica, it may not be available yet due to replication lag. ## When should you use replicas in your application? Replicas are useful for offloading read-heavy workloads from the primary node. By using replicas, you can distribute the read load across multiple nodes, which can help improve the performance of your application. Some examples of where you might want to query a replica are: scheduled jobs, analytics jobs, search features, or aggregate queries. Replicas can also be used to provide a read-only view of your data to users or applications that do not need to write data, such as when a user is logged out or writing one-off queries for debugging purposes. ## Configuring replicas for your database cluster By default, production databases (excluding [single node](/docs/postgres/cluster-configuration/single-node)) are created with 2 replicas. You may add additional replicas if you need to scale your read traffic. Adding additional replicas beyond the default does not guarantee an increase in availability of your database. > [!NOTE] > > You are charged for additional replicas you add beyond the default. Billing for additional replicas begins once the replica change has completed. The additional cost is \~ 1/3 the original price of the base cluster running (initial primary + 2 replicas). To see how to increase and manage the number of additional replicas you have, see [Cluster configuration](/docs/postgres/cluster-configuration). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # TIN: PlanetScale Postgres Search Source: https://planetscale.com/docs/postgres/search A fully featured and fast search engine inside Postgres TIN (**T**ext **IN**dex) brings blazing-fast full-text search to PlanetScale Postgres. Enabling the `tin` extension adds an inverted index type built for search, BM25 ranking, and the **[TINQL](/docs/postgres/search/tinql)** query language. For local development and CI, use [Lead](https://github.com/planetscale/lead), a TIN-compatible Postgres extension for testing your application's search queries on small datasets. See [Local development and CI](/docs/postgres/search/get-started#local-development-and-ci) for setup instructions. Lead is intentionally unsuitable for production workloads. ## A search engine inside Postgres A TIN index is a normal Postgres index, so searches see a consistent snapshot while writes continue. You also get ranking, highlighting, exact counts, and a query language that can express phrases, proximity, and spans. TIN offers richer retrieval features and drastically better performance than typical Postgres search extensions and external services. ### Compared with other Postgres search [TINQL](/docs/postgres/search/tinql) is a broader retrieval language than `tsquery` or `pg_textsearch`, and TIN is substantially faster than `pg_textsearch` and ParadeDB `pg_search` on the count and top-k workloads those engines can run. | Capability | TIN | Postgres FTS | ParadeDB `pg_search` | `pg_textsearch` | | - | - | - | - | - | | Available on PlanetScale | ✅ | ✅ | ❌ | ❌ | | BM25 ranking | ✅ | ❌ | ✅ | ✅ | | Top-k retrieval | ✅ | Limited | ✅ | ✅ | | `AND` / `OR` / `NOT` | ✅ | ✅ | ✅ | ❌ | | Phrases with word gaps | ✅ | Limited | ✅ | ❌ | | Proximity and span queries | ✅ | ❌ | Limited | ❌ | | Fuzzy terms, wildcards, boosts | ✅ | Limited | ✅ | ❌ | | Query language | TINQL | `tsquery` | Query API | Term string | | Highlighting | ✅ | ✅ | ✅ | ❌ | | Indexes `text` directly | ✅ | ❌ | ✅ | ✅ | | Stemming | ✅ | ✅ | ✅ | ✅ | An external engine such as Elasticsearch has a similar feature set, but requires a second cluster and a sync path with Postgres. ### Key features * **[The `==>` operator](/docs/postgres/search/reference/operator).** `WHERE body ==> '…'` searches a TIN-indexed column with a TINQL string. `==>` is boolean: the row matches if the text satisfies the query. * **[TIN index](/docs/postgres/search/reference/indexes).** Create a TIN index with `CREATE INDEX ... USING tin` on a `text` column. * **[BM25 ranking with top-k retrieval](/docs/postgres/search/scoring).** `tin.score(ctid)` returns the BM25 relevance of each matching row, and `ORDER BY tin.score(ctid) DESC LIMIT 10` returns the 10 best documents in score order. The same row gets the same score however the query is executed. `tin.max_score(ctid)` gives a denominator for normalized relevance, and `k1` and `b` can be tuned per index or per query. * **[The TINQL query language](/docs/postgres/search/tinql).** One query string expresses phrases with word gaps, ordered and unordered proximity (`THEN/5`, `NEAR/5`), span relations (`ENCLOSES`, `OVERLAPPING`, `BEFORE`, `AFTER`), positional filters (`IN FIRST 100 WORDS`), wildcards, fuzzy terms, regular expressions, term ranges, `AT LEAST 2 OF`, and boosts. Every expression produces spans, so the operators compose freely. * **Exact counts.** `count(*)` with `WHERE body ==> '…'` is answered from the index and stays exact under concurrent writes, `VACUUM`, and on read replicas. * **[Highlighting](/docs/postgres/search/highlighting).** `tin.highlight` adds configurable markers around the text that produced the match. When highlighted, a match on `pineapple` returns `'pineapple'`. `tin.highlight_ansi` does the same for terminals. * **[Cross-column scoring](/docs/postgres/search/reference/sql-shapes#several-tin-indexed-columns).** Search several TIN-indexed columns in one query, and `tin.score(ctid)` combines relevance across them. * **[Configurable tokenization](/docs/postgres/search/reference/indexes#index-options-with).** The default tokenizer runs on indexed columns and queries, splitting text on Unicode word boundaries, folding case and accents, and indexes emoji as terms. Optional [Snowball stemming](/docs/postgres/search/reference/indexes#stemming) matches inflected words (`runs`, `running`, `run`) as the same term. * **[Parallel build and parallel query](/docs/postgres/search/operations#parallel-query).** Index builds use parallel workers, and based on cost, queries and `count(*)` can run in parallel too. ### TIN example Create a table, insert some rows, index the text column, and query it: > [!WARNING] > > If `CREATE EXTENSION` fails with `permission denied to create extension "tin"` and `Must be superuser to create this extension.`, your cluster needs an update before it can install TIN. Go to the **Clusters** page for your branch, find the "Cluster update available" indicator, and [update your cluster](/docs/postgres/cluster-configuration/updates). After the update completes, run `CREATE EXTENSION` again. ```sql theme={null} CREATE EXTENSION IF NOT EXISTS tin; CREATE TABLE posts ( id bigint PRIMARY KEY, body text NOT NULL ); INSERT INTO posts (id, body) VALUES (1, 'Fuji apple slices with citrus and melon make a bright fruit salad'), (2, 'Fuji apple, citrus, and melon for lunch'), (3, 'Peel the fuji apple and toss it with citrus and melon'), (4, 'Citrus and melon salad with fresh mint'), (5, 'Grape tasting notes from the orchard'); CREATE INDEX posts_body_tin ON posts USING tin (body); SELECT id, tin.score(ctid) AS score, tin.highlight(body) FROM posts WHERE body ==> '(citrus NEAR/5 melon) AND "fuji apple" AND NOT peel' ORDER BY score DESC LIMIT 10; ``` The query matches rows 1 and 2. Each contains the phrase `fuji apple`, has `citrus` within five words of `melon`, and excludes `peel`. Row 3 has all three terms but is excluded by `AND NOT peel`, and rows 4 and 5 lack the phrase. `tin.score(ctid)` ranks the matches by BM25 relevance, and `tin.highlight(body)` adds ``/`` tags around the text that produced the match. ## Hybrid search with pgvector Hybrid search requires a keyword (lexical) search and a meaning (semantic) search over the same documents, then combining the two ranked lists. TIN does the lexical half. Semantic search needs an embedding model outside Postgres and a vector index on the same table. The model embeds each document when you write it, and embeds the user's query at search time. [pgvector](/docs/postgres/extensions/pgvector) stores those vectors and returns the nearest rows. ```sql theme={null} CREATE TABLE posts ( id bigint PRIMARY KEY, body text NOT NULL, embedding vector(1536) NOT NULL ); CREATE INDEX posts_body_tin ON posts USING tin (body); CREATE INDEX posts_embedding ON posts USING hnsw (embedding vector_cosine_ops); -- Lexical: words and phrases in the query SELECT id FROM posts WHERE body ==> '"fuji apple"' ORDER BY tin.score(ctid) DESC LIMIT 10; -- Semantic: embedding of that same query, from the same model that embedded `body` SELECT id FROM posts ORDER BY embedding <=> $query_embedding LIMIT 10; ``` You would then combine these results in your application, keeping every `id` from either query, and treating an `id` that appears in both as a stronger match. ## Next steps * [Get started](/docs/postgres/search/get-started) — install the extension, index a column, run your first query * [Scoring](/docs/postgres/search/scoring) — BM25 relevance and top-k retrieval * [Highlighting](/docs/postgres/search/highlighting) — mark the text that matched * [TINQL](/docs/postgres/search/tinql) — the query language * [Operational guidance](/docs/postgres/search/operations) — vacuum settings for tables with churn * [Reference](/docs/postgres/search/reference/indexes) — index options and the SQL API ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Get started with TIN Source: https://planetscale.com/docs/postgres/search/get-started Install the `tin` extension, create a TIN index, and run TINQL queries with BM25 ranking. ## Install the extension On PlanetScale, enable TIN with the SQL below. For a local or CI database, first [set up Lead](#local-development-and-ci). The database encoding must be `UTF8` or `SQL_ASCII`. `CREATE EXTENSION tin` refuses other encodings (for example, `LATIN1`). ```sql theme={null} CREATE EXTENSION IF NOT EXISTS tin; ``` > [!WARNING] > > If `CREATE EXTENSION` fails with `permission denied to create extension "tin"` and `Must be superuser to create this extension.`, your cluster needs an update before it can install TIN. Go to the **Clusters** page for your branch, find the "Cluster update available" indicator, and [update your cluster](/docs/postgres/cluster-configuration/updates). After the update completes, run `CREATE EXTENSION` again. ### Upgrade the extension When a new TIN version is available, [update your cluster](/docs/postgres/cluster-configuration/updates) so it restarts onto the new library. Then, in every database that already has the extension, run: ```sql theme={null} ALTER EXTENSION tin UPDATE; ``` The cluster update installs the new binary. `ALTER EXTENSION` updates the SQL catalog so new options and function signatures are visible. Changing analysis options such as [stemming](/docs/postgres/search/reference/indexes#stemming) on a populated index also needs `REINDEX`. ## Create a table and index The example below creates a table, inserts rows, and creates a TIN index on the `text` column: ```sql theme={null} CREATE TABLE posts ( id bigint PRIMARY KEY, category text NOT NULL, body text NOT NULL ); INSERT INTO posts (id, category, body) VALUES (1, 'fruit', 'I love fuji apples and juicy mangoes'), (2, 'tasting', 'Grape tasting notes from the orchard'), (3, 'fruit', 'The best juicy fuji apple in town'); CREATE INDEX posts_body_tin ON posts USING tin (body); ``` Each TIN index covers one text column (or a text-producing expression). The default tokenizer folds case and accents and indexes emoji as terms, so `Jalapeño` and `jalapeno` match, and "😀" is searchable. The same tokenizer applies to indexed columns and queries. Stemming is off by default. Add `WITH (stemmer = 'en')` so `runs`, `running`, and `run` match as the same term. See [Stemming](/docs/postgres/search/reference/indexes#stemming). You can preview how a string is tokenized with `tin.tokenize`: ```sql theme={null} SELECT * FROM tin.tokenize('Jalapeño 😀'); SELECT * FROM tin.tokenize('Jalapeño 😀', accent_folding => 'preserve'); SELECT * FROM tin.tokenize('runs running runner', stemmer => 'en'); ``` You can also create [partial indexes](/docs/postgres/search/reference/indexes#partial-and-expression-indexes), [expression indexes](/docs/postgres/search/reference/indexes#partial-and-expression-indexes), and set per-index BM25 and tokenization options with [`WITH (k1, b, tokenizer, stemmer, …)`](/docs/postgres/search/reference/indexes#index-options-with). After changing analysis options on a populated index, `REINDEX` so existing rows are re-tokenized. ## Your first queries TINQL keywords are UPPERCASE. Lowercase tokens are terms. Quote a multi-word phrase ("fuji apple"). ### Filter ```sql theme={null} SELECT id, body FROM posts WHERE body ==> 'apple AND "fuji apple"'; ``` ### Ranked results (BM25) Order responses in a ranked list with `tin.score`. Add more sort keys after the score when equal scores must stay in a stable order: ```sql theme={null} SELECT id, tin.score(ctid) AS score, body FROM posts WHERE body ==> 'apple OR grape' ORDER BY score DESC, id LIMIT 10; ``` To normalize scores against the query's best match, divide by `tin.max_score(ctid)`, which is constant for the scan and identical on every row: ```sql theme={null} SELECT id, tin.score(ctid) AS score, tin.score(ctid) / tin.max_score(ctid) AS relative FROM posts WHERE body ==> 'apple OR grape' ORDER BY score DESC LIMIT 10; ``` `tin.score` and `tin.max_score` require a TIN index scan in the same query. Outside that context, they raise an error rather than returning NULL. ### Count `count(*)` over a TIN predicate is answered from the index, not a heap scan. ```sql theme={null} SELECT count(*) FROM posts WHERE body ==> 'juicy'; ``` ### Highlight `tin.highlight` adds markers around the text that produced the match. A match on `apple` returns `'apple'`. ```sql theme={null} SELECT id, tin.highlight(body) FROM posts WHERE body ==> 'apple' LIMIT 20; ``` `tin.highlight` is configurable, pass in additional arguments to customize the markers and perform the search. ```sql theme={null} SELECT tin.highlight(body, '', '', 'apple') FROM posts WHERE id = 1; ``` ### Search across columns Each TIN index covers one text column. Index every column you want to search, then combine `==>` in SQL. `tin.score(ctid)` combines BM25 relevance across those fields for the row. To weight one column higher than another, use TINQL boost (`^N`) on that field's query. For example, `name ==> 'fuji^1.5'` makes a `name` match count 1.5 times an unboosted `notes` match. ```sql theme={null} CREATE TABLE fruits ( id bigint PRIMARY KEY, name text NOT NULL, notes text NOT NULL ); CREATE INDEX fruits_name_tin ON fruits USING tin (name); CREATE INDEX fruits_notes_tin ON fruits USING tin (notes); SELECT id, tin.score(ctid) AS score, name, notes FROM fruits WHERE name ==> 'fuji^1.5' AND notes ==> 'citrus' ORDER BY score DESC LIMIT 10; ``` ## Local development and CI [Lead](https://github.com/planetscale/lead) is a Postgres extension for exercising TIN-compatible application SQL in local development and CI. It uses the `tin` extension name so you can test your application's search queries without connecting to PlanetScale. Lead supports Postgres 17 and 18 and is built from source. Follow the [Lead README](https://github.com/planetscale/lead#build) for build prerequisites, local setup, and packaging instructions for your test database. > [!WARNING] > > Lead is intended for small development and test datasets. It scans table rows instead of maintaining TIN's search index, so it is not suitable for production workloads or benchmarking TIN's performance. Use TIN on PlanetScale for production search. ## Common pitfalls | Symptom | Fix | | - | - | | Parse error on `==>` | Fix TINQL syntax. Keywords must be UPPERCASE (`AND`, not `and` as an operator). Explicit empty syntax (`""`, `[]`) is a parse error. | | Zero rows | Broaden the query (`OR`, fewer required terms, wildcards). Confirm you indexed the column you are searching. Check tokenization with `tin.tokenize`. Inputs that analyze to no tokens (`''`, whitespace, bare punctuation) match nothing. | | Score / max\_score error | Call `tin.score(ctid)` / `tin.max_score(ctid)` on a query that also has `col ==> …`. They require a TIN index scan and error in other contexts (including DML `RETURNING` without a scan). On a **partitioned parent**, every planned leaf needs a usable TIN index — see [limitations](/docs/postgres/search/reference/limitations). | | Exclusion not working | Use `AND NOT`, not `-term`. `-` is not negation. | | `MATCHES` finds nothing | `MATCHES` is not folded. Match the dictionary form (usually lowercase, accents folded): `MATCHES apple.*`, not `MATCHES Apple.*`. | | Accent / case mismatch | Defaults fold both. Search `jalapeno` to match `Jalapeño`. Override with `WITH (case_folding = preserve, accent_folding = preserve)` if you need exact surface forms. | | Inflected forms miss | Stemming is off by default. Create the index `WITH (stemmer = 'en')` (or another [language code](/docs/postgres/search/reference/indexes#stemming)), then `REINDEX` if the index already has rows. | | Fuzzy error on hyphenated term | Fuzzy needs one token after tokenization. Prefer `wi-fi` as a phrase (`"wi fi"`) or drop `~N`. | | Out-of-range `k1` / `b` / boost | `k1` and query boosts must be in `[0.0..10000.0]`; `b` must be in `[0.0..1.0]`. | | Index keeps growing | The index relation grows to a high-water mark and does not shrink on its own. `REINDEX` shrinks it. See [limitations](/docs/postgres/search/reference/limitations). | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Highlighting Source: https://planetscale.com/docs/postgres/search/highlighting Mark the text that produced a match in your application or terminal `tin.highlight` adds configurable markers around the text that produced the match. A match on `pineapple` returns `'pineapple'`. `tin.highlight_ansi` does the same with ANSI colors for a terminal. ## `tin.highlight(text [, begin_tag [, end_tag [, query]]])` `tin.highlight(text, begin_tag, end_tag, query) → text` | Argument | Type | Default | Notes | | - | - | - | - | | `text` | `text` | — | Document text to highlight (usually the indexed column). | | `begin_tag` | `text` | `''` | Opening tag wrapped around each match. May contain the placeholders described under [Tag placeholders](#tag-placeholders). | | `end_tag` | `text` | `''` | Closing tag wrapped around each match. | | `query` | `text` | `NULL` | TINQL used to find match spans. When omitted, the `==>` predicate on the same column in the statement's `WHERE` clause is used. | Returns document text with matched spans wrapped in tags. Highlighting marks exactly the text that produced the match. For `a BEFORE b`, only the `b` occurrences that satisfy the relation are wrapped. Stemmed matches wrap the complete original word, so a query for `runs` highlights `running` as `running`. Calls on a searched column use that index's analysis options, including `stemmer`. Standalone calls accept the same analysis arguments as [`tin.tokenize`](/docs/postgres/search/reference/functions). Supplied options must match the selected index when highlighting a searched column. When `query` is omitted, the predicate is found anywhere in the same statement, including subqueries, CTEs, and DML (`RETURNING`, `MERGE` actions, `ON CONFLICT`). ```sql theme={null} -- Query taken from the ==> predicate on body SELECT id, tin.highlight(body) FROM posts WHERE body ==> 'apple' LIMIT 20; -- Witnessing spans only SELECT tin.highlight('b a b', query => 'a BEFORE b'); -- b a b (the leading b precedes no a — unmarked) -- Automatic query in DML RETURNING UPDATE posts SET reviewed = true WHERE body ==> 'urgent' RETURNING id, tin.highlight(body); -- Explicit tags + query SELECT tin.highlight(body, '', '', 'apple') FROM posts WHERE id = 1; -- Stemming highlights the original word SELECT tin.highlight('run runs running runner', query => 'runs', stemmer => 'en'); -- run runs running runner ``` ## Tag placeholders `begin_tag` can carry two placeholders that TIN fills in for each highlighted span. | Placeholder | Replaced with | | - | - | | `$QUERY_PART` | The part of the query that matched the span, HTML-escaped. For a wildcard or regular expression this is the pattern that matched, such as `MATCHES email.*`. | | `$QUERY_LABEL` | The same information as a token that is safe in a CSS class: lowercased, with punctuation collapsed to `-`. | ```sql theme={null} SELECT tin.highlight('send email now', '', '', 'email*'); -- send email now -- One CSS class per query part SELECT id, tin.highlight(body, '', '') FROM posts WHERE body ==> 'apple OR "fuji apple"'; ``` When two matches overlap or touch, they merge into one highlight whose placeholders list every query part involved, so tags never nest or interleave. ## `tin.highlight_ansi(text [, wrap_to [, query]])` `tin.highlight_ansi(text, wrap_to, query) → text` | Argument | Type | Default | Notes | | - | - | - | - | | `text` | `text` | — | Document text to highlight. | | `wrap_to` | `integer` | `NULL` | When set to a positive width, rewraps the text before highlighting. | | `query` | `text` | `NULL` | TINQL used to find match spans. When omitted, the `==>` predicate on the same column in the statement's `WHERE` clause is used. | Returns document text with ANSI color highlighting for matched spans. ```sql theme={null} SELECT tin.highlight_ansi(body) FROM posts WHERE body ==> 'apple' LIMIT 5; ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Operational guidance Source: https://planetscale.com/docs/postgres/search/operations Best practices running TIN in production. ## Memory for index builds A TIN index build runs in parallel, and each worker prefers about 1 GB of memory. The budget comes from `maintenance_work_mem`, split evenly across the workers. TIN starts as many workers as `max_parallel_maintenance_workers` allows and the budget can support, so a small `maintenance_work_mem` means fewer workers and a slower build. A worker can run with less than 1 GB, but the index it writes is more fragmented, and that fragmentation slows every query on the index until you `REINDEX` it with more memory. Before building an index on a large table, raise the budget for that session: ```sql theme={null} SET maintenance_work_mem = '8GB'; SET max_parallel_maintenance_workers = 8; CREATE INDEX posts_body_tin ON posts USING tin (body); ``` Keep the total inside the memory of your cluster size. As a rule of thumb, plan on 1 GB for each core you want the build to use, on top of `shared_buffers` and the memory your normal traffic needs. On [network-attached storage clusters](/docs/plans/planetscale-skus#network-attached-storage), a PS-160 has 16 GB of RAM and a PS-1280 has 128 GB, so an 8 GB build budget fits comfortably on the larger size and not at all on the smaller one. Both settings are also available under [Parameters](/docs/postgres/cluster-configuration/parameters) when you want a cluster-wide default rather than a session setting. ## Storage PlanetScale clusters run on network-attached storage (AWS gp3, Amazon Elastic Block Store) or on [PlanetScale Metal](/docs/metal), which uses locally attached NVMe drives. Network-attached storage supports [storage autoscaling](/docs/postgres/cluster-configuration/cluster-storage) and comes with an IOPS and throughput allowance that you can raise. Every read that misses the cache pays network latency, and a search over a cold index is many such reads. TIN reads index pages as sequentially as it can and returns matching rows in heap order within each segment, which keeps its I/O pattern friendly to network-attached storage and to Postgres' caching. Metal has no IOPS limit and much lower latency per read, and its drive size is fixed when the cluster is created. For a search index that does not fit in memory, Metal is the faster choice, and it is the one to prefer when search latency matters. `effective_io_concurrency` matters on network-attached storage. When it is positive, TIN issues readahead hints for some index reads. Keep it positive when the index is read from disk, and set it to `0` when the index fits in `shared_buffers`, where the hints are pure overhead. ## Parallel query TIN splits an index into segments at build time, one per core by default, and a query can use one parallel worker per segment. Postgres caps the workers with `max_parallel_workers_per_gather` and `max_parallel_workers`, both available under [Parameters](/docs/postgres/cluster-configuration/parameters). As an index grows, the work per query grows with it, and TIN spreads that work across the cores it is given, so a multi-terabyte index performs best on a cluster size with more vCPUs. If you build on one cluster size and serve on another, set `initial_segment_count` for the serving size, because the segment count is fixed at build. See [Index options](/docs/postgres/search/reference/indexes#index-options-with). ## Read replicas Set `hot_standby_feedback = on` on every replica that serves TIN queries. TIN's background maintenance frees index pages once no transaction on the primary needs them, and the feedback is what tells the primary that a query on a replica still does. Without it, replica queries that overlap with maintenance are cancelled with SQLSTATE `40001` more often. Results on a replica are always exact. Under heavy replication traffic a query can still be cancelled with `40001`, and the application should retry it. `hot_standby_feedback` is available under [Parameters](/docs/postgres/cluster-configuration/parameters). See also [Replicas](/docs/postgres/scaling/replicas). ## Vacuum When a row is updated or deleted, Postgres keeps the old version in the table until `VACUUM` removes it, and the row's entry stays in the TIN index. TIN learns that the entry is dead only when `VACUUM` runs on the table. Until then, a query that reaches the entry has to check the row in the heap to find out whether it is still visible. `count(*)` is affected most. A count over a TIN predicate is normally answered from counts stored in the index. TIN trusts those stored counts only where it knows of no dead entries and Postgres's visibility map marks every row visible. Rows deleted or updated since the last `VACUUM` break both conditions, so the count falls back to checking rows in the heap. On a table with steady update or delete traffic, count performance degrades between vacuums and recovers after each one. `VACUUM` also drives background maintenance. It marks deleted entries dead in the index, and once a segment's dead fraction crosses `dead_percent_threshold` (default `0.5`), background maintenance rewrites the segment without them. BM25 statistics keep counting deleted rows until that rewrite. See [Visibility](/docs/postgres/search/scoring#visibility). A long-running transaction delays `VACUUM`, so dead heap rows and dead TIN index entries remain until that transaction ends. ### Autovacuum settings By default Postgres does not vacuum a table until 20% of its rows have changed (`autovacuum_vacuum_scale_factor = 0.2`, plus `autovacuum_vacuum_threshold = 50` rows). On a large table that is millions of dead rows, and a TIN index over that table carries them the whole time. For tables you search with TIN, and especially when `count(*)` is part of the workload, lower the trigger for that table: ```sql theme={null} ALTER TABLE posts SET ( autovacuum_vacuum_scale_factor = 0.01, autovacuum_vacuum_threshold = 1000 ); ``` With these values autovacuum runs after about 1% of the table plus 1,000 rows have changed. Set them per table so that tables you do not search keep the defaults. The cluster-wide `autovacuum_vacuum_scale_factor` can also be changed under [Parameters](/docs/postgres/cluster-configuration/parameters). ## Postgres settings TIN uses TIN has few settings of its own. Most of its behavior follows ordinary Postgres settings, which you can change under [Parameters](/docs/postgres/cluster-configuration/parameters) or, for the session-level ones, with `SET`. | Setting | How TIN uses it | | - | - | | `maintenance_work_mem` | The memory budget for an index build, split evenly across the build workers, and the budget for folding new writes into the index. Together with `tin.rss_baseline` it decides how many build workers can run. | | `max_parallel_maintenance_workers` | The upper bound on build workers. | | `max_worker_processes` | The pool that background maintenance, parallel builds, and the end-of-build WAL pass draw from. When the pool is exhausted, maintenance runs inline in the writing session and builds proceed with fewer workers. | | `max_parallel_workers_per_gather`, `max_parallel_workers` | The cap on parallel workers for a query. TIN uses up to one worker per segment within that cap. | | `work_mem` | Memory for deduplicating and sorting row identifiers when a query combines several predicates, for example an `OR` between a text predicate and a btree lookup. Larger sets spill to disk. | | `shared_buffers` | Where index pages are cached. Size it to hold the part of the index your queries touch. | | `effective_io_concurrency` | When positive, TIN issues readahead hints for some index reads. Keep it positive when the index is read from disk and `0` when the index fits in `shared_buffers`. | | `seq_page_cost`, `random_page_cost`, `cpu_tuple_cost`, `cpu_operator_cost`, `parallel_setup_cost`, `parallel_tuple_cost`, `effective_cache_size` | TIN's cost model measures the work of each candidate plan in physical units and prices them with these settings, so they decide between serial and parallel plans and between index paths. The defaults describe spinning disk. On a cluster whose index fits in memory, `seq_page_cost = 0.1`, `random_page_cost = 0.1`, and an accurate `effective_cache_size` produce better plans. | | `wal_level` | Changes to a TIN index are written to WAL under the same rule as built-in indexes. Only `minimal`, with an index created in the current transaction, skips logging. | | `wal_compression` | The last phase of an index build writes the whole index to WAL. `lz4` or `zstd` reduces that WAL volume at the cost of CPU that is usually idle by then. | | `hot_standby_feedback` | Required on replicas that serve TIN queries. See [Read replicas](#read-replicas). | | `autovacuum_vacuum_scale_factor`, `autovacuum_vacuum_threshold` | Decide how soon dead rows are reported to TIN. See [Vacuum](#vacuum). | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Functions Source: https://planetscale.com/docs/postgres/search/reference/functions SQL functions in the `tin` schema. ## `tin.score(ctid [, dense_ratio [, k1 [, b [, term_add [, term_replace]]]]])` `tin.score(tid, real, real, real, text[], text[]) → real` Returns the BM25 relevance of the current row, leaving out dense terms. See [Scoring](/docs/postgres/search/scoring). ## `tin.full_score(ctid [, k1, b])` `tin.full_score(tid) → real`\ `tin.full_score(tid, real, real) → real` Returns the BM25 relevance of the current row with every query term kept in the sum. See [Scoring](/docs/postgres/search/scoring). ## `tin.max_score(ctid)` `tin.max_score(tid) → real` Returns the highest score among the query's matches. The value is the same on every row. See [Scoring](/docs/postgres/search/scoring). ## `tin.score_inspect(index, query [, dense_ratio [, term_add [, term_replace]]])` `tin.score_inspect(regclass, text, real, text[], text[]) → setof (term text, weight real)` Returns the terms a `tin.score` call with the same arguments would score, with their boost weights. See [Scoring](/docs/postgres/search/scoring). ## `tin.highlight(text [, begin_tag [, end_tag [, query]]])` `tin.highlight(text, text, text, text) → text` Returns the document text with matched spans wrapped in tags. See [Highlighting](/docs/postgres/search/highlighting). ## `tin.highlight_ansi(text [, wrap_to [, query]])` `tin.highlight_ansi(text, integer, text) → text` Returns the document text with matched spans in ANSI colors. See [Highlighting](/docs/postgres/search/highlighting). ## `tin.tokenize(text [, …])` `tin.tokenize(text) → setof text` Returns the tokens the analysis pipeline produces for a string. Optional named arguments match the [index tokenization options](/docs/postgres/search/reference/indexes#index-options-with) and use the same defaults (`tokenizer`, `case_folding`, `accent_folding`, `long_tokens`, `max_token_bytes`, `graphemes`, `position_gaps`, `stemmer`). `stemmer` defaults to unset (no stemming). Useful when debugging why a query does or does not match. ```sql theme={null} SELECT * FROM tin.tokenize('Jalapeño 😀'); SELECT * FROM tin.tokenize('Jalapeño 😀', accent_folding => 'preserve'); SELECT * FROM tin.tokenize('runs running runner', stemmer => 'en'); ``` ## `tin.maybe_quote(text)` `tin.maybe_quote(text) → text` Quotes a term for safe use inside a TINQL string when it would otherwise be parsed as a keyword or special form. ```sql theme={null} SELECT tin.maybe_quote('AND'); -- "AND" ``` ## `tin.fsck(index [, heapcheck])` `tin.fsck(regclass, boolean) → setof record` Runs a read-only structural check of a TIN index. An empty result means the index passed. It never writes or repairs. Rebuild with `REINDEX`. Requires ownership of the index. | Argument | Type | Default | Notes | | - | - | - | - | | `index` | `regclass` | — | TIN index to validate. | | `heapcheck` | `boolean` | `false` | When `true`, also cross-checks index TIDs against the heap. | ```sql theme={null} SELECT * FROM tin.fsck('posts_body_tin'); SELECT * FROM tin.fsck('posts_body_tin', heapcheck => true); ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Indexes Source: https://planetscale.com/docs/postgres/search/reference/indexes Creating TIN indexes and setting their options. ```sql theme={null} CREATE INDEX ON USING tin (); ``` Indexes a `text` or `citext` column (or a text-producing expression) for TINQL search, including phrases, proximity, and spans. Documents and queries are analyzed with the same tokenization pipeline. Each TIN index covers **one** text source. Multi-column `USING tin (…)` indexes are not supported. To search several columns, create one TIN index per column and combine `==>` predicates. `tin.score(ctid)` combines relevance across those columns. ```sql theme={null} -- Create index CREATE INDEX posts_body_tin ON posts USING tin (body); -- Build or rebuild without blocking writes CREATE INDEX CONCURRENTLY posts_body_tin ON posts USING tin (body); REINDEX INDEX CONCURRENTLY posts_body_tin; ``` ## Partial and expression indexes Partial and expression TIN indexes work like other Postgres index types: ```sql theme={null} -- Partial: only index active rows CREATE INDEX posts_active_body_tin ON posts USING tin (body) WHERE active; -- Expression: index a JSON field or transformed text CREATE INDEX docs_title_tin ON docs USING tin ((data->>'title')); CREATE INDEX posts_lower_body_tin ON posts USING tin ((lower(body))); ``` The query must match the indexed expression exactly. `(data->>'title') ==> …` uses the expression index, and a plain `title ==> …` does not. A partial index is used only when Postgres can prove the query implies its `WHERE` predicate (for example `WHERE active AND body ==> …`). ## Index options (`WITH`) ```sql theme={null} CREATE INDEX posts_body_tin ON posts USING tin (body) WITH (k1 = 1.2, b = 0.75); ALTER INDEX posts_body_tin SET (k1 = 1.5); ALTER INDEX posts_body_tin RESET (k1); -- Tokenization policy (same pipeline at index time and query time) CREATE INDEX logs_msg_tin ON logs USING tin (message) WITH ( tokenizer = whitespace, case_folding = preserve, accent_folding = preserve, long_tokens = truncate, max_token_bytes = 64, graphemes = discard, position_gaps = collapse ); -- Optional Snowball stemming (requires case_folding = fold) CREATE INDEX posts_body_en_tin ON posts USING tin (body) WITH (stemmer = 'en'); ``` ### Scoring | Option | Type | Default | Notes | | - | - | - | - | | `k1` | real | `1.2` | BM25 term-frequency saturation. Domain `[0.0..10000.0]`, enforced at DDL. | | `b` | real | `0.75` | BM25 length normalization. Domain `[0.0..1.0]`. | | `score_stop_words` | text | unset | Comma-separated terms that `tin.score` leaves out of the BM25 sum. Each entry must match the stored term exactly and is never tokenized, so write entries in analyzed form (lowercase on a case-folding index, and stemmed when a stemmer is set). Matching, counting, and phrases are unaffected. | Scoring options are read at query time, so `ALTER INDEX ... SET` takes effect on the next scored query without a rebuild. `k1` and `b` can also be overridden for a single query with `tin.score(ctid, k1 => …, b => …)`. [`tin.score_inspect`](/docs/postgres/search/scoring) shows which terms a query would score after stop words and dense-term elision are applied. ### Tokenization | Option | Type | Default | Notes | | - | - | - | - | | `tokenizer` | text | `unicode` | Base token boundary policy: `unicode` or `whitespace`. | | `case_folding` | text | `fold` | Unicode case folding: `fold` or `preserve`. | | `stemmer` | text | unset | Snowball language code. Unset means no stemming. Requires `case_folding = fold`. See [Stemming](#stemming). | | `accent_folding` | text | `fold` | Accent/diacritic folding: `fold` or `preserve`. | | `long_tokens` | text | `split` | Tokens longer than `max_token_bytes`: `split`, `truncate`, or `discard`. | | `max_token_bytes` | int | `256` | Maximum token length in UTF-8 bytes (`[4..2692]`). | | `graphemes` | text | `emoji` | Standalone grapheme clusters (emoji, symbols): `emoji`, `retain`, or `discard`. | | `position_gaps` | text | `preserve` | Position numbering when analysis removes tokens: `preserve` or `collapse`. | Defaults fold **case and accents**, split over-long tokens on grapheme boundaries, and emit emoji as searchable terms. Stemming is off until you set `stemmer`. Indexing and query analysis use the same pipeline, so `==>` searches agree with what the index stored. ```sql theme={null} CREATE INDEX posts_body_tin ON posts USING tin (body); SELECT id FROM posts WHERE body ==> 'jalapeno'; -- matches 'Jalapeño' SELECT id FROM posts WHERE body ==> '😀'; -- emoji are terms ``` Use `tin.tokenize` to inspect the pipeline. Named arguments match the index options and use the same defaults: ```sql theme={null} SELECT * FROM tin.tokenize('Jalapeño 😀'); -- default policy: case + accent folding, emoji as a term SELECT * FROM tin.tokenize('Jalapeño 😀', accent_folding => 'preserve'); ``` Hyphenated surface forms may still split into multiple tokens (for example `wi-fi` becomes the phrase `"wi fi"`). Fuzzy (`term~N`) requires a single token after tokenization. Changing tokenization options on a populated index does not re-tokenize stored rows. Run `REINDEX` to apply the new policy to them. ### Stemming `stemmer` applies Snowball stemming after lowercasing and before accent folding. It is unset by default. Set an ISO language code to match inflected words as the same term: ```sql theme={null} CREATE INDEX posts_body_en_tin ON posts USING tin (body) WITH (stemmer = 'en'); SELECT * FROM tin.tokenize('runs running runner', stemmer => 'en'); -- run -- run -- runner ``` English `run`, `runs`, and `running` share the stored term `run`. `runner` stays a separate term. Stemming requires `case_folding = fold`; unknown language codes and `case_folding = preserve` with a stemmer fail at `CREATE` / `ALTER`. Ordinary query terms and phrases use the same stemmer as stored text. Wildcard, fuzzy, and range literals skip stemming and search the stored stems. Regular expressions are unchanged. `score_stop_words` still match stored terms exactly, so write them in stemmed form on a stemmed index. | Code | Language | Code | Language | | - | - | - | - | | `ar` | Arabic | `da` | Danish | | `nl` | Dutch | `en` | English | | `fi` | Finnish | `fr` | French | | `de` | German | `el` | Greek | | `hu` | Hungarian | `it` | Italian | | `no` | Norwegian | `pt` | Portuguese | | `ro` | Romanian | `ru` | Russian | | `es` | Spanish | `sv` | Swedish | | `ta` | Tamil | `tr` | Turkish | Enable, change, or disable stemming and rebuild together so existing rows and later queries agree: ```sql theme={null} BEGIN; ALTER INDEX posts_body_tin SET (stemmer = 'en'); REINDEX INDEX posts_body_tin; COMMIT; ``` `ALTER INDEX ... RESET (stemmer)` turns stemming off. That also needs `REINDEX` on a populated index. ### Segments A TIN index is stored as segments. The segment count sets how many parallel workers a build and a query can use, and background maintenance folds new writes into segments over time. | Option | Type | Default | Notes | | - | - | - | - | | `initial_segment_count` | int | available parallelism | Segments created at build (`[1..4096]`). Read at `CREATE INDEX` and `REINDEX`, and used as the default for `target_segment_count`. | | `target_segment_count` | int | = `initial_segment_count` | Segment count that background maintenance converges to as writes add segments (`[1..4096]`). | | `max_mutable_segment_size` | int | `4194304` (4 MiB) | Bytes of new writes that accumulate before background maintenance folds them into the index (`≥ 131072`). Folding also happens every 16,384 documents. | | `max_merged_segment_size` | int | `2000` | Size in megabytes above which a segment is no longer merged with others (`≥ 100`). | | `dead_percent_threshold` | real | `0.5` | Fraction of dead rows at which a segment is rewritten to drop them. Domain `[0.0..1.0]`. | Changes to these options take effect on the next maintenance pass, except `initial_segment_count`, which is read only at build. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Limitations Source: https://planetscale.com/docs/postgres/search/reference/limitations Constraints on TIN indexes. * **One column per index** — Create a separate TIN index for each searchable column (or expression). Combine columns in a query with multiple `==>` predicates. `tin.score(ctid)` combines scores across those fields. * **Partitioned tables** — TIN indexes on partitioned tables are supported, and querying the parent works when every partition has a TIN index. BM25 scores are computed **per partition** (each partition has its own corpus statistics), so ordering by score across partitions mixes scores from different subsets. Some query shapes that pass `tin.score` through aggregates or window functions against the parent are refused with an error that names the workaround: compute the score in a `MATERIALIZED` CTE (or query a single partition) and consume it above. * **Index relation size** — The index relation grows to a high-water mark and does not shrink on its own. Space freed by deleted and updated rows is reused internally, and `REINDEX` is the only way to shrink it. Measure it with `pg_relation_size('')`. `CREATE INDEX CONCURRENTLY`, `REINDEX`, and `REINDEX CONCURRENTLY` are supported. * **Tokenizer** — Tokenization is configured per index via `WITH (…)` (see [Index options](/docs/postgres/search/reference/indexes#index-options-with)), including optional [Snowball stemming](/docs/postgres/search/reference/indexes#stemming). After changing analysis options on a populated index, `REINDEX` so existing rows are re-tokenized. `tin.tokenize` accepts the same arguments. * **Read replicas** — On physical standbys, set `hot_standby_feedback = on`. Standby results are exact. Under heavy replay a query can be cancelled with SQLSTATE `40001` (serialization failure) and should be retried. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # The ==> operator Source: https://planetscale.com/docs/postgres/search/reference/operator Search a TIN-indexed column with a TINQL query string. TIN introduces the `==>` operator which is followed by a TINQL query. ```sql theme={null} SELECT id, body FROM posts WHERE body ==> 'apple AND "fuji apple"'; ``` | Side | Type | Notes | | - | - | - | | Left | `text` | Indexed text column or matching expression. | | Right | `text` | TINQL query string. May be a prepared-statement parameter (`$1`). | Returns `true` when the document matches the TINQL expression. An input that analyzes to no tokens (`''`, whitespace, bare punctuation) matches nothing. Explicit empty syntax (`""`, `[]`) is a parse error. Combine multiple `==>` predicates and ordinary SQL filters with `AND` / `OR`. When several indexed columns appear in the same query, `tin.score(ctid)` combines BM25 relevance across those fields. The right-hand side may also be an array via `==> ANY (…)`, which matches if any element matches: ```sql theme={null} SELECT * FROM posts WHERE body ==> 'juicy AND apple'; -- Zero-token input matches nothing; explicit empty syntax is a parse error SELECT count(*) FROM posts WHERE body ==> ''; SELECT count(*) FROM posts WHERE body ==> '""'; -- ERROR -- Match any of several TINQL strings SELECT id FROM posts WHERE body ==> ANY (ARRAY['apple', 'grape']); -- Cross-column: one TIN index per column, combine predicates SELECT id, tin.score(ctid) AS score, name, notes FROM fruits WHERE name ==> 'fuji' AND notes ==> 'citrus' ORDER BY score DESC LIMIT 10; ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Settings Source: https://planetscale.com/docs/postgres/search/reference/settings Postgres settings that control TIN. These are ordinary Postgres GUCs (`SET`, `ALTER DATABASE … SET`, or `postgresql.conf`). Defaults are tuned for good performance, and you usually do not need to change them. | Setting | Default | Notes | | - | - | - | | `tin.enable_custom_scan` | `on` | When `off`, falls back to generic index scanning (troubleshooting / comparison). | | `tin.index_maintenance_mode` | `background` | How index maintenance runs: `background`, `foreground` (in the writing session), or `manual` (no automatic maintenance). | | `tin.build_io_concurrency` | `0` (auto) | Caps concurrent disk I/O during `CREATE INDEX` / `REINDEX`. `1` serializes build I/O. | | `tin.rss_baseline` | server-derived (MB) | Per-worker RSS estimate used when planning build parallelism. | | `tin.maintenance_jobs_per_db` | `0` | How many of its own database's maintenance jobs a cluster-wide maintenance worker finishes before yielding to another database with queued work. `0` drains the current database first. Set it in `postgresql.conf` or with `ALTER SYSTEM`. A session `SET` is rejected. | | `tin.track_page_reuse_stats` | `off` | When `on`, `EXPLAIN ANALYZE` reports distinct vs repeated index page reads. | ## Debug settings The `tin.debug_*` settings are for debugging and testing TIN, not for production use. Each one forces a planning choice the cost model would otherwise make, so that an alternative plan can be measured. They never change query results, and a query shape that would be incorrect is still refused. All of them are session-level (`SET`) and default to following the cost model. | Setting | Values | Default | Notes | | - | - | - | - | | `tin.debug_force_boolean_family` | `auto`, `fused`, `factored` | `auto` | For a query over several TIN indexes, force one combined scan (`fused`) or separate scans joined with boolean operators (`factored`). | | `tin.debug_force_topk` | `auto`, `topk`, `exhaustive` | `auto` | For `ORDER BY tin.score(ctid) DESC LIMIT k`, force the bounded top-k scan or an exhaustive scored scan. | | `tin.debug_force_multi_index_drive` | `auto`, `sparse`, `stripe` | `auto` | For a query over several TIN indexes, drive from the rarest required term (`sparse`) or evaluate by block range (`stripe`). | | `tin.debug_force_conjunction_mode` | `auto`, `generic`, `pushdown`, `noadaptive` | `auto` | Override how an `AND` between a text predicate and another indexed predicate is executed. `noadaptive` only disables the adaptive verifier. | | `tin.debug_force_visibility` | `auto`, `streaming`, `sorted` | `auto` | Where a count-only `AND` can check row visibility in streaming or sorted order, force one. | | `tin.debug_disable_count_pushdown` | `on`, `off` | `off` | Disable the index-answered `count(*)` path so a plain aggregate over a scan runs instead. | | `tin.debug_disable_index_probe` | `on`, `off` | `off` | Disable probing a btree index alongside the text search; mixed predicates fall back to a TIN scan with Postgres filters. | | `tin.debug_force_parallel` | `on`, `off` | `off` | Drop serial plans that have a parallel sibling so the parallel plan is chosen, including on tables too small to qualify otherwise. | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Recommended SQL shapes Source: https://planetscale.com/docs/postgres/search/reference/sql-shapes Query shapes that use the index well. The examples on this page use two tables. `posts` has TIN indexes on `title` and `body` and btree indexes on `author_id` and `created_at`. `authors` has a TIN index on `bio`. ```sql theme={null} CREATE TABLE authors ( id bigint PRIMARY KEY, name text NOT NULL, bio text NOT NULL, topics text NOT NULL ); CREATE INDEX authors_bio_tin ON authors USING tin (bio); CREATE TABLE posts ( id bigint PRIMARY KEY, author_id bigint NOT NULL REFERENCES authors (id), title text NOT NULL, body text NOT NULL, ups integer NOT NULL DEFAULT 0, created_at timestamptz NOT NULL DEFAULT now() ); CREATE INDEX posts_title_tin ON posts USING tin (title); CREATE INDEX posts_body_tin ON posts USING tin (body); CREATE INDEX posts_author_id ON posts (author_id); CREATE INDEX posts_created_at ON posts (created_at); ``` ## Ranked results ```sql theme={null} -- $1 is the TINQL query, $2 the page size SELECT id, title, tin.score(ctid) AS score FROM posts WHERE body ==> $1 ORDER BY score DESC LIMIT $2; ``` `ORDER BY tin.score(ctid) DESC LIMIT k` is the shape TIN is built around. It returns the k best rows in score order and stops as soon as it has them. Passing the query as a parameter keeps user input out of the SQL text. Add more sort keys after the score when equal scores must stay in a stable order. TIN still uses the top-k scan and applies those keys only where scores tie. See [Ties and secondary sort keys](/docs/postgres/search/scoring#ties-and-secondary-sort-keys). ```sql theme={null} SELECT id, title, tin.score(ctid) AS score FROM posts WHERE body ==> $1 ORDER BY score DESC, created_at DESC, id LIMIT $2; ``` ```sql theme={null} SELECT id, title, tin.score(ctid) AS score, tin.score(ctid) / tin.max_score(ctid) AS relative FROM posts WHERE body ==> $1 ORDER BY score DESC LIMIT $2; ``` `tin.max_score(ctid)` is the best score among the matches and has the same value on every row, so dividing by it gives a relative score between 0 and 1. That is the number to show in a relevance bar, or to compare against a threshold when you want to hide weak matches. ## Filters and counts ```sql theme={null} SELECT id, title FROM posts WHERE body ==> '"fuji apple"'; SELECT count(*) FROM posts WHERE body ==> 'error OR fail'; ``` Without `tin.score`, `==>` is an ordinary boolean filter. A `count(*)` over a `==>` predicate is answered from the index. ## Several TIN-indexed columns ```sql theme={null} SELECT id, title, tin.score(ctid) AS score FROM posts WHERE title ==> 'espresso^2' AND body ==> 'grinder OR "burr grinder"' ORDER BY score DESC LIMIT 10; ``` Each TIN index covers one column, so a query over several columns uses one `==>` per column. `tin.score(ctid)` adds up the relevance from every column that matched. The boost on the title query makes a title hit count twice as much as a body hit, which is the usual way to say that one field matters more than another. ```sql theme={null} SELECT id, title, tin.score(ctid) AS score FROM posts WHERE title ==> 'espresso' OR body ==> 'espresso' ORDER BY score DESC LIMIT 10; ``` With `OR`, a row qualifies when either column matches, and its score sums whatever matched. This is the shape for a single search box that should look in several fields. ## Filtering on an unindexed column ```sql theme={null} SELECT id, title, tin.score(ctid) AS score FROM posts WHERE body ==> 'espresso' AND ups > 100 ORDER BY score DESC LIMIT 10; ``` `ups` has no index. TIN finds the rows that match the text and the `ups` condition is checked on each of them, so it costs one comparison per candidate and needs no index of its own. When the filter rejects most of the text matches, the search has to look further down the ranking to fill the limit, so this shape suits filters that keep a reasonable share of the matches. ## Combining with btree indexes ```sql theme={null} -- Point lookup SELECT id, title FROM posts WHERE body ==> 'espresso' AND author_id = 42; -- Range SELECT id, title, tin.score(ctid) AS score FROM posts WHERE body ==> 'espresso' AND created_at >= now() - interval '7 days' ORDER BY score DESC LIMIT 10; ``` When the other condition has a btree index, TIN can use both indexes. A selective point or range condition narrows the text search rather than filtering its results afterwards, which matters when the text query matches many rows and the other condition matches few. A recency window over a large corpus is the common case. ```sql theme={null} SELECT id, title FROM posts WHERE body ==> '"espresso machine"' OR id = 21462; ``` With `OR`, each index contributes its own rows and the result is the union without duplicates. An `OR` costs about what its more expensive side costs, so a cheap phrase plus a primary-key lookup stays cheap. ## Joins with a score on each side ```sql theme={null} SELECT p.id, p.title, tin.score(p.ctid) AS post_score, tin.score(a.ctid) AS author_score FROM posts p JOIN authors a ON a.id = p.author_id WHERE p.body ==> 'espresso' AND a.bio ==> 'barista OR roaster' ORDER BY post_score + author_score DESC LIMIT 10; ``` `tin.score` takes the `ctid` of the relation it scores, so a join can carry one score per side. Each side needs its own `==>` predicate. A side with no text predicate has nothing to score, and the query is refused. The two scores are independent BM25 values from two different indexes, so combine them deliberately: add them, weight one, or order by one and display the other. ```sql theme={null} SELECT a.name, p.id, p.title, p.score FROM authors a CROSS JOIN LATERAL ( SELECT id, title, tin.score(ctid) AS score FROM posts WHERE body ==> a.topics ORDER BY score DESC LIMIT 3 ) p ORDER BY a.name, p.score DESC; ``` The right-hand side of `==>` can be a column from the outer query, so here the search is bound once per author using that author's `topics` as the TINQL query. A `LATERAL` subquery with `ORDER BY tin.score(ctid) DESC LIMIT k` returns the top k for each outer row. This is the shape behind "recommended for you" lists and any page that shows the best matches per user, per category, or per saved search. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Scoring Source: https://planetscale.com/docs/postgres/search/scoring BM25 relevance with `tin.score`, `tin.full_score`, and `tin.max_score`. `tin.score(ctid)` returns a BM25 score over the rows a TIN scan matched. [The `==>` operator](/docs/postgres/search/reference/operator) performs a TINQL query against a TIN index to find matches, and `tin.score` assigns each matching row a number. Scoring does not sort on its own, so write `ORDER BY tin.score(ctid) DESC` to sort from closest to farthest matches. ## `tin.score(ctid [, dense_ratio [, k1 [, b [, term_add [, term_replace]]]]])` `tin.score(tid, real, real, real, text[], text[]) → real` | Argument | Type | Default | Notes | | - | - | - | - | | `ctid` | `tid` | required | Heap tuple identifier for the current row (`ctid`). | | `dense_ratio` | `real` | `0.10` | Fraction of the document count at which a term is dense and left out of the score. A value above `1` disables elision. | | `k1` | `real` | `NULL` | Runtime BM25 `k1` override. `NULL` inherits the index option. Domain `[0.0..10000.0]`. | | `b` | `real` | `NULL` | Runtime BM25 `b` override. `NULL` inherits the index option. Domain `[0.0..1.0]`. | | `term_add` | `text[]` | `NULL` | Terms added to the scored set. Each element is analyzed with the index tokenizer and pinned so elision cannot drop it. | | `term_replace` | `text[]` | `NULL` | Terms that become the entire scored set, analyzed with the index tokenizer. Cannot be combined with `term_add`. | Returns BM25 relevance for the current row. Requires a TIN index scan in the same query (`col ==> …`). Outside that context, including DML `RETURNING` without a scan, the call raises *requires a tin index scan and cannot be used in this query context*. When the query has multiple `==>` predicates on TIN-indexed columns, the score combines relevance across those fields. Under `FOR UPDATE` / `FOR SHARE`, scores may be `NULL` on concurrently updated rows. **Dense-term elision.** A term whose document frequency reaches `dense_ratio × N`, where `N` is the document count, is dense, and dense terms are left out of the BM25 sum unless the query boosts them explicitly (`^1.0` counts). A common term carries almost no ranking signal and is expensive to score. At the default ratio, a term found in 10% or more of the documents contributes nothing. A matching row that contains no scored term returns exactly `0.0`. Matching itself is unchanged. On a table of only a few rows, every term is dense, and every score is `0.0`. `tin.full_score` scores every term. ```sql theme={null} SELECT id, tin.score(ctid) AS score, body FROM posts WHERE body ==> 'apple OR grape' ORDER BY score DESC LIMIT 10; -- Override BM25 parameters for this query only (does not change the index) SELECT id, tin.score(ctid, k1 => 3.2, b => 0.2) AS score, body FROM posts WHERE body ==> 'apple OR grape' ORDER BY score DESC LIMIT 10; -- Score terms found in up to a quarter of the documents SELECT id, tin.score(ctid, dense_ratio => 0.25) AS score, body FROM posts WHERE body ==> 'apple OR grape' ORDER BY score DESC LIMIT 10; ``` Index defaults come from `WITH (k1 = …, b = …)` on the TIN index (or `1.2` / `0.75` when unset). Runtime overrides must be statement-constant and share the same domains as the index options. `k1` and `b` may differ between `tin.score` calls on the same relation. `dense_ratio`, `term_add`, and `term_replace` must be written identically in every call. ## `tin.full_score(ctid [, k1, b])` `tin.full_score(tid) → real`\ `tin.full_score(tid, real, real) → real` | Argument | Type | Notes | | - | - | - | | `ctid` | `tid` | Heap tuple identifier for the current row (`ctid`). | | `k1` | `real` | Runtime BM25 `k1` override. `NULL` inherits the index option. | | `b` | `real` | Runtime BM25 `b` override. `NULL` inherits the index option. | Returns BM25 relevance with every query term kept in the sum. Dense-term elision and `score_stop_words` do not apply, so common terms contribute to the ranking and cost more to score on a large corpus. It has the same scan requirement as `tin.score`, and you can't combine the two in a single scanned relation. ```sql theme={null} SELECT id, tin.full_score(ctid) AS score, body FROM posts WHERE body ==> 'the AND midnight' ORDER BY score DESC LIMIT 10; ``` ## `tin.max_score(ctid)` `tin.max_score(tid) → real` | Argument | Type | Notes | | - | - | - | | `ctid` | `tid` | Heap tuple identifier for the current row (`ctid`). | Returns the highest BM25 score over the query’s visible matches. The value is constant for the scan, which makes it the natural denominator for normalized relevance. Like `tin.score`, it must be used alongside a TIN index scan, or the query errors. It follows whichever of `tin.score` and `tin.full_score` the query uses. ```sql theme={null} SELECT id, tin.score(ctid) AS score, tin.score(ctid) / tin.max_score(ctid) AS relative FROM posts WHERE body ==> 'quick AND fox' ORDER BY score DESC LIMIT 10; ``` ## `tin.score_inspect(index, query [, dense_ratio [, term_add [, term_replace]]])` `tin.score_inspect(regclass, text, real, text[], text[]) → setof (term text, weight real)` | Argument | Type | Default | Notes | | - | - | - | - | | `index` | `regclass` | required | The TIN index whose statistics and options to use. | | `query` | `text` | required | The TINQL query to inspect. | | `dense_ratio` | `real` | `0.10` | Same meaning as in `tin.score`. | | `term_add` | `text[]` | `NULL` | Same meaning as in `tin.score`. | | `term_replace` | `text[]` | `NULL` | Same meaning as in `tin.score`. | Returns one row per term that a `tin.score` scan with the same arguments would score, with the term's accumulated boost weight. It runs the same steps as `tin.score` without scanning anything: query terms, minus `score_stop_words`, minus dense terms, plus terms pinned by a boost or by `term_add`. Use it to see what `dense_ratio` drops on your corpus and to check that a `score_stop_words` entry matches a stored term. It requires `SELECT` on the index's table. ```sql theme={null} SELECT * FROM tin.score_inspect('posts_body_tin', 'common^1.0 OR mid OR rare', 0.25) ORDER BY term; -- term | weight -- common | 1 -- rare | 1 ``` `mid` is dense at that ratio, so it is left out. `common` is denser still, but the explicit `^1.0` keeps it in. ## Visibility `tin.score` scores only rows your query can see. A row that is not visible to your snapshot, whether it was deleted, replaced by an update, or inserted by a transaction that has not committed, is never returned and never receives a score. It also never influences which rows make a `LIMIT k` cut. The corpus statistics behind BM25 work differently. The document count, the number of documents containing each term, and the average document length come from the index, and they include every document the index still stores. These statistics are totaled across every segment your transaction can see, including the newest segment that is still being written to; a segment your transaction can't see yet (because it was written after your snapshot was taken) contributes nothing to those totals. For a segment your transaction can see, a deleted or updated row keeps its entry until `VACUUM` marks it dead and background maintenance rewrites the segment that holds it, or until you `REINDEX`. Until then, the deleted rows still count in the statistics, and the rows you do see are scored as if the deleted rows were still present. On a table with heavy update or delete churn, keep autovacuum aggressive so that maintenance can drop dead entries promptly. See [Operational guidance](/docs/postgres/search/operations). Scores are comparable within one query. They shift over time as the corpus grows and as maintenance reshapes it. ## Ties and secondary sort keys Documents with the same term statistics get the same BM25 score. Ranked searches can keep a bounded top-k scan when you add more sort keys after the score: ```sql theme={null} SELECT id, tin.score(ctid) AS score, body FROM posts WHERE body ==> 'apple OR grape' ORDER BY score DESC, created_at DESC, id LIMIT 10; ``` TIN ranks by score first. When several rows share a score, the extra keys decide which of those rows stay in the top k and in what order. A unique key such as `id` makes the ranking reproducible across runs. Without a secondary key, the order of equal-score rows can change between queries. Use a tie-break column when you paginate with `LIMIT` / `OFFSET`, or when the order must stay stable. A score threshold alone (`WHERE tin.score(ctid) < last_score`) cannot resume inside a tied group: `<` skips the rest of the group, and `<=` repeats earlier rows. A volatile key (`random()`), `FETCH ... WITH TIES`, or a `LIMIT` that is not constant at plan time keeps PostgreSQL's `Sort` instead of the top-k scan. The results stay correct; only the plan changes. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # TINQL Source: https://planetscale.com/docs/postgres/search/tinql A fully-featured query language unique to TIN. TIN adds [the `==>` operator](/docs/postgres/search/reference/operator). The query on the right-hand side of it is **TINQL**. ```sql theme={null} SELECT id, body FROM posts WHERE body ==> 'apple AND "fuji apple"'; ``` **Rules** * Keywords must be **UPPER CASE** (`AND`, `OR`, `THEN`). Lowercase is a search term (`and` matches the word “and”). * Matching **folds case and accents** under the default tokenizer (`Jalapeño` and `jalapeno` are the same term). Override with `case_folding` / `accent_folding` on the index. * When the index has a [`stemmer`](/docs/postgres/search/reference/indexes#stemming), ordinary terms and phrases are stemmed to match inflected forms (`runs` matches `running`). Wildcard, fuzzy, and range literals stay unstemmed and search stored stems. * Inputs that analyze to **no tokens** (`''`, whitespace, bare punctuation) match nothing. Explicit empty syntax (`""`, `[]`) is a parse error. * Query boost factors (`^N`) must be in `[0.0..10000.0]`. The system rejects out-of-range values at parse time. * Token positions are **0-based**. Prefer `IN FIRST N WORDS` / `IN LAST N WORDS` over raw `IN WORDS` ranges when you mean “the first/last N words.” * Juxtaposition defaults to AND: `apple grape` acts as `apple AND grape`. * Exclude terms with `AND NOT`: `apple AND NOT peel`. * Quote reserved words used as terms: `"AND"`, `"THEN"`. * A hyphen usually splits a term: `wi-fi` is searched as `"wi fi"`. * Fuzzy match (`apple~2`) works on one word. `wi-fi~2` errors because `wi-fi` is two words. ## Terms | Form | Meaning | Example | | - | - | - | | bare term | Single dictionary term (case + accent folded by default) | `apple` (same as `Apple`), `jalapeno` (same as `Jalapeño`) | | `*` | Match-all documents (standalone only) | `*` | | `*` / `?` wildcards | `*` = 0+ chars, `?` = 1 char | `appl*`, `p?ach`, `*house` | | `term~N` | Fuzzy: edit distance N, stable prefix 1 | `apple~2` | | `term~P:N` | Fuzzy: stable prefix P, distance N | `apple~0:2` | | `A TO B` | Inclusive term-dictionary range | `aardvark TO cat` | | `* TO B` / `A TO *` | Open range bound | `* TO cat` | | `MATCHES regex` | Full-term regex (to unescaped whitespace); **not** folded | `MATCHES peach.*` | | `CONTAINS term` | Same as bare term (optional) | `CONTAINS apple` | | `expr^N` | Boost (`N` ∈ `[0.0..10000.0]`) | `apple^2`, `"x"^1.5` | Allowed in terms: anything except whitespace and `( ) [ ] " ~ ^`. Escape literal `*` / `?` with `\`. Space in a regex: `MATCHES foo\ bar`. Because `MATCHES` is not folded, write patterns against the normalized dictionary form (usually lowercase, accents folded): `MATCHES apple.*`, not `MATCHES Apple.*`. ## Phrases | Form | Meaning | Example | | - | - | - | | `"a b c"` | Adjacent ordered words | `"fuji apple"` | | `"a _ c"` | `_` = one any-word gap | `"big _ wolf"` | | `"a [b c] d"` | Per-position alternatives | `"big [bad large] wolf"` | | `"a b"~N` | Up to N extra word gaps (phrase tolerance) | `"fuji apple"~2` | Phrase escapes: `\"` `\\` `\_` `\[` `\]`. Empty `""` is invalid (parse error). Inputs that analyze to no tokens match nothing. ## Boolean operators | Operator | Meaning | Example | | - | - | - | | `A AND B` | Both match | `juicy AND apple` | | `A OR B` | Either matches | `apple OR grape` | | `A AND NOT B` | A matches and B does not | `apple AND NOT peel` | | `A B` | Implicit AND | `apple grape` | `NOT` never stands alone. It only appears in `AND NOT`, `NOT ENCLOSES`, `NOT ENCLOSED BY`, and `NOT OVERLAPPING`. ## Alternatives | Form | Meaning | Example | | - | - | - | | `[a b c]` | OR of whitespace-separated alts (not commas) | `[mango plum pear]` | | `AT LEAST N OF […]` | At least N alts match | `AT LEAST 2 OF [a b c d]` | | `AT LEAST N% OF […]` | At least N% of alts match | `AT LEAST 50% OF [a b c d]` | | `ALL OF […]` | Every alt matches | `ALL OF [a b c]` | Alternatives may hold expressions: `["big bad" house NEAR/2 brick]`. ## Proximity `/N` is required on `THEN` and `NEAR`. N is the maximum number of extra words allowed between the operands, and `0` means adjacent. | Operator | Meaning | Example | | - | - | - | | `A THEN/N B` | Ordered proximity | `fuji THEN/0 apple` | | `A NEAR/N B` | Either order | `peach NEAR/5 blossom` | | `(A … B) WITHIN N` | Match span width ≤ N | `(citrus NEAR/5 melon) WITHIN 6` | ## Span relations Every TINQL expression produces **spans** (position ranges). Relation operators compare two span sets. | Operator | Keeps | Example | | - | - | - | | `A ENCLOSES B` | Spans of A that contain a span of B | `(security NEAR/10 threat) ENCLOSES critical` | | `A NOT ENCLOSES B` | Spans of A that contain no span of B | `* NOT ENCLOSES spam` | | `A ENCLOSED BY B` | Spans of A that fall inside a span of B | `critical ENCLOSED BY (security NEAR/10 threat)` | | `A NOT ENCLOSED BY B` | Spans of A not inside any span of B | `price NOT ENCLOSED BY (disclaimer NEAR/20 terms)` | | `A OVERLAPPING B` | Spans of A that share a position with B | `(apple NEAR/5 fuji) OVERLAPPING (kiwi NEAR/5 citrus)` | | `A NOT OVERLAPPING B` | Spans of A that share no positions with B | `title NOT OVERLAPPING disclaimer` | | `A BEFORE B` | Spans of A that start before B | `abstract BEFORE conclusion` | | `A AFTER B` | Spans of A that start after B | `price AFTER discount` | `ENCLOSES` and `ENCLOSED BY` test the same relationship but return different spans: `ENCLOSES` returns the outer span and `ENCLOSED BY` returns the inner one. ## Positional filters Token positions are **0-based** (the first token is position `0`). Use `IN FIRST` / `IN LAST` when you care about word counts, and `IN WORDS` when you need an explicit position window. | Form | Meaning | Example | | - | - | - | | `A IN FIRST N WORDS` | Match in the first N tokens (positions `0` … `N-1`) | `apple IN FIRST 100 WORDS` | | `A IN FIRST N%` | Match in first N% of the document | `apple IN FIRST 25%` | | `A IN LAST N WORDS` | Match in the last N tokens | `apple IN LAST 50 WORDS` | | `A IN LAST N%` | Match in last N% of the document | `apple IN LAST 25%` | | `A IN MIDDLE N%` | Match in middle N% of the document | `apple IN MIDDLE 50%` | | `A IN WORDS X TO Y` | Match in inclusive 0-based position range `[X, Y]` | `apple IN WORDS 500 TO 1000` | ## Precedence (loose → tight) | Level | Operators | | - | - | | 1 | `OR` | | 2 | `AND` (explicit and implicit) | | 3 | `AND NOT` | | 4 | Positional filters (`IN …`) | | 5 | Relation operators (`ENCLOSES`, `BEFORE`, …) | | 6 | Proximity (`THEN/N`, `NEAR/N`) | | 7 | `WITHIN` | | 8 | Boost (`^N`) | | 9 | Primary (`apple`, `"phrase"`, `[a b]`, `MATCHES …`) | Operators at the same level are left-associative. Use parentheses to override precedence. `A OR B THEN/5 C AND D` → `A OR ((B THEN/5 C) AND D)`. Postfix modifiers compose on the primary: `apple~2^3`, `"big bad"~2^1.5`. ## Keyword index All keywords are case-sensitive **UPPER CASE**. | Category | Keywords | | - | - | | Boolean | `AND`, `OR`, `NOT` (only in compounds) | | Proximity | `THEN`, `NEAR`, `WITHIN` | | Relations | `ENCLOSES`, `ENCLOSED`, `BY`, `OVERLAPPING`, `BEFORE`, `AFTER` | | Range / position | `TO`, `IN`, `FIRST`, `LAST`, `MIDDLE`, `WORDS` | | Terms | `CONTAINS`, `MATCHES` | | Alternatives | `AT`, `LEAST`, `OF`, `ALL` | `IN` is special only before `FIRST` / `LAST` / `MIDDLE` / `WORDS`. `AT` only before `LEAST`. `ALL` only before `OF`. `ENCLOSED` / `BY` only together. ## Special characters | Character | Meaning | | - | - | | `"…"` | Phrase | | `(…)` | Grouping | | `[…]` | Alternatives | | `~N` | Fuzzy (on term) or phrase word-gap tolerance (after `"`) | | `~P:N` | Fuzzy with explicit prefix length | | `^N` | Boost | | `*` | Wildcard, open range bound, or standalone match-all | | `?` | Single-character wildcard | | `_` | One-word gap inside a phrase | | `\` | Escape | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Settings Source: https://planetscale.com/docs/postgres/settings The Settings page allows you to configure general settings and options for your PlanetScale Postgres database. To access the settings: **Step 1** From the PlanetScale organization dashboard, select your database **Step 2** Navigate to **Settings** on the left sidebar menu ## General Settings ### Database Name The database name field displays your current database name. **This name cannot be changed** because it's used to connect to the database. Changing the name would break existing connections and applications. ### Default Branch The default branch dropdown allows you to select which branch serves as the default for your database. This branch will be used as the source when creating new development branches. Only production branches can be set as the default. New branches created from the **Branches** page start as development branches, so you must first [upsize the cluster and add replicas](/docs/postgres/cluster-configuration) on the **Clusters** page to promote them to a production branch before they are eligible. > [!NOTE] > > A branch's region cannot be changed after creation. To run production in a different region, create a new branch in the desired region, migrate your schema and data into it, and then set it as the default branch. See [Changing regions](/docs/postgres/branching#changing-regions) for the full procedure. To change the default branch: **Step 1** Click the **Default branch** dropdown **Step 2** Select the desired branch from the list **Step 3** Click **Save database settings** at the bottom of the page ### Restrict Branch Regions The "Restrict branch regions" option allows you to limit where new branches can be created geographically. When enabled, new branches can only be created in the same region as the default branch. This setting controls where **new** branches can be created. It does not move existing branches, and it cannot change the region of the default branch or any other existing branch — branch regions are fixed at creation. See [Changing regions](/docs/postgres/branching#changing-regions) for more information. To enable region restrictions: **Step 1** Check the **Restrict branch regions** checkbox **Step 2** Click **Save database settings** to apply the change This setting is useful for: * Compliance requirements that mandate data residency * Cost optimization by keeping resources in a specific region * Reducing latency by keeping all branches in the same geographic area ### Development branch limit The development branch limit controls how many development branches the database can have at the same time. The default is 100. Set any number from 1 to 5,000. To change the limit: **Step 1** Enter a number in **Development branch limit** **Step 2** Click **Save database settings** When the database is at its limit, delete development branches you no longer need, or raise the limit, before creating another one. For a limit above 5,000, [contact support](https://planetscale.com/contact). ### Save Settings After making any changes to the general settings, click the **Save database settings** button to apply your changes. ## Delete Database The settings page also provides the option to permanently delete your database. This action is irreversible and will: * Delete the entire database and all of its branches * Remove all data and backups permanently * Disconnect any applications currently connected to the database * Include usage charges in your final invoice through the deletion date > [!WARNING] > > Database deletion cannot be undone. Make sure you have proper backups and that no critical applications depend on this database before proceeding. To delete a database: **Step 1** Scroll to the bottom of the Settings page **Step 2** Click the red **Delete database** button **Step 3** Enter the database name **Step 4** Click **Delete database** to confirm Only Organization Administrators and Database Administrators have permission to delete databases. See the [Access Control documentation](/docs/security/access-control) for more information about user permissions. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Horizontal sharding for Postgres Source: https://planetscale.com/docs/postgres/sharding Scale past a single Postgres primary with Neki > [!NOTE] > > Neki is currently in Platform Preview. Platform Preview features are “Beta Features” under the PlanetScale Terms of Service or your applicable agreement with PlanetScale. Accordingly, Neki is subject to the limitations and disclaimers applicable to Beta Features and is not covered by any service level agreement. [Neki](/docs/neki) scales past a single primary while keeping one connection string. A router sits in front of real Postgres so applications keep talking to one database while data lives on one cluster or many. PlanetScale Postgres already helps with schema recommendations, Query Insights, and faster instances on [Metal](/docs/metal). Neki is the upgrade when you need more than one writer, schema changes that do not stall a large table, and backups that run per cluster instead of against one heap. You do not have to shard on day one. A new Neki database starts as one unsharded cluster and still runs the router, admin service, and sidecar: one connection string, failover, and pooling without a client-facing PgBouncer. [When to shard](/docs/neki/when-to-shard) depends on your traffic, access patterns, and data. > **[Neki documentation](/docs/neki)** > > Architecture, data topology, query planning, and cluster configuration. > **[Quickstart](/docs/neki/tutorials/planetscale-quick-start-guide)** > > Create a Neki database, connect, and insert data. > **[When to shard](/docs/neki/when-to-shard)** > > Whether a single primary is still the cheaper answer. > **[Neki vs PlanetScale Postgres](/docs/neki/coming-from-postgres)** > > The upgrade from a single primary, and the habits that change. ## Why sharding matters **Horizontal sharding** splits a logical table's rows across multiple database instances, each with its own primary. A router sends each query to the shard or shards that hold the rows. That differs from **vertical scaling**, where you add CPU, memory, or faster disks to one server. A larger instance eventually hits a ceiling on write throughput and disk, and one failure takes the whole database with it. If read capacity is your bottleneck, replicas help, but you cannot add more write capacity to a single primary. Sharding is the next step when you need: * **More than one writer.** Writes spread across shard primaries instead of one WAL. * **More storage than one instance can hold.** Each shard has its own disk. * **A smaller blast radius.** A lagged replica, a vacuum, or a restore is a shard problem, not a whole-database problem. * **Tenant isolation.** A noisy customer can live on its own shard instead of starving the rest of the cluster. ### When horizontal sharding makes sense **Multi-tenant SaaS** often fits when a few tenants dominate storage or query volume. A tenant key can keep each tenant's rows together and keep heavy tenants from starving others. **Very large datasets or I/O-heavy workloads** can exhaust one primary even after you tune queries and storage. Spreading rows across shards adds capacity that a larger instance does not. **Write-heavy applications** can scale reads with replicas, but every write still hits one primary. Sharding splits those writes. See [When to shard](/docs/neki/when-to-shard) for the full decision, including what partitioning does not solve. ## Tune queries and Metal first Neki is still in Platform Preview, and PlanetScale Postgres is still the best single-primary Postgres. Use [Query Insights](/docs/postgres/monitoring/query-insights) and [schema recommendations](/docs/postgres/monitoring/schema-recommendations) before you split the database. Many limits are expensive queries or missing indexes, not a need for more primaries. [Metal](/docs/metal) uses locally attached NVMe. Workloads that look like they need sharding are sometimes hitting I/O limits that Metal removes. Tune the workload and the storage class, then move to Neki when that one primary is still the limit. Neki is the next upgrade. You can import into an unsharded Neki database then add shards and split data later. See [Neki vs PlanetScale Postgres](/docs/neki/coming-from-postgres) and [Migrate PostgreSQL to Neki](/docs/neki/imports/postgres). ## Frequently asked questions > **What is horizontal sharding?** > > Rows are split across multiple database servers, and a router sends each query using a shard key. See [Why sharding matters](#why-sharding-matters) for how that differs from vertical scaling. > **When should I shard my Postgres database?** > > After query tuning and [Metal](/docs/metal) still leave one primary as the limit. See [Tune queries and Metal first](#tune-queries-and-metal-first) and [When to shard](/docs/neki/when-to-shard). > **Is Neki a fork of Vitess?** > > No. It is built from scratch for Postgres. See [Neki vs Vitess](/docs/neki/vitess-and-postgres). > **How does Neki relate to PlanetScale Postgres?** > > Neki is the upgrade from a single Postgres primary. Each shard is real Postgres. [Neki vs PlanetScale Postgres](/docs/neki/coming-from-postgres) covers what you gain and the habits that change. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Database Traffic Control Source: https://planetscale.com/docs/postgres/traffic-control Define and enforce limits on how many resources specific operations can use. Database Traffic Control for PlanetScale Postgres introduces **resource budgets**, giving you control over the resources allocated to specific traffic slices. Your database faces two categories of threat: external actors who abuse or overload your system, and your own application code under unexpected conditions. A single runaway query, a traffic spike from a viral feature, or a misbehaving integration can overwhelm your database just as effectively as a deliberate attack. Database Traffic Control protects against both. [Introducing Database Traffic Control](https://www.youtube-nocookie.com/embed/YgrEGqABD8U?rel=0) Instead of all queries competing equally for CPU and connections, you can: * Ensure critical application paths stay alive even when the database is under pressure * Isolate noisy analytics or reporting workloads from user-facing traffic * Put guardrails around third‑party integrations and batch jobs * Gradually introduce new features without risking the stability of production Database Traffic Control operates at the [database branch](/docs/postgres/branching) level and lives inside [Insights](/docs/postgres/monitoring/query-insights), next to the metrics you already use to understand performance. ## Why this matters When a database becomes overwhelmed, the typical outcome is a complete outage. Every query fails. Every user is affected. The entire application goes down. Database Traffic Control changes the failure mode. Instead of total collapse, you define which workloads are allowed to be shed under pressure and which should have higher limits. You choose which non-critical queries fail while higher priority workloads like checkout and authentication continue to function. This is the difference between a **degraded experience** and a **site-wide incident**. Database Traffic Control gives you the ability to keep your application running—even when something has gone wrong. ## When should I use it? Instances where Database Traffic Control is useful include: * **Emergency situations to stop out-of-control queries** Immediately apply resource budgets to a single runaway query to prevent it from running again. * **Running OLTP and OLAP workloads on the same database** For example, user‑facing application traffic shares a cluster with heavy analytics queries. * **Extract, transform, load (ETL) jobs that you do not fully control** An external tool or script might periodically generate high QPS or expensive queries. * **Launching new features with unknown query patterns** You want to ship to production, but be confident that mistakes are contained. * **Supporting multiple tenants on the same cluster** You want to ensure that no single tenant can consume all resources at the expense of others. If you never see contention between workloads and your queries are well understood, you may not need Database Traffic Control today. But as usage grows, more teams access the same database, and the cost of availability issues increases, any overload condition could result in a total-failure outcome. ## Key capabilities ![Graph of blocked and warned queries](https://mintcdn.com/planetscale-2/Pc1vKrfqALwLm39x/postgres/traffic-control/assets/insights-traffic-control-light.png?fit=max\&auto=format\&n=Pc1vKrfqALwLm39x\&q=85\&s=b89119803161c3de1ee2a32646e390b3) The decision to Warn or Block a query is determined by the resource budget limits and mode. See [Concepts](/docs/postgres/traffic-control/concepts) for detailed definitions of **resource budgets**, **rules**, and **modes**. * **Configure limits for query traffic** Resource budgets define how much specific traffic can use within the configured limits. * **Write rules to target queries, system attributes, and tags** Resource budgets can apply to one or many queries matching against one or more rules, including each unique value for a selected tag. * **Define resource budgets per branch** Resource budgets are unique to each Postgres branch, so you can experiment in staging before enforcing in production. * **Enforce, warn, or disable** Dry‑run resource budgets before they block traffic by running in warning mode and inspecting logs. * **Warning thresholds on enforced budgets** Set separate warning thresholds as a percentage of burst, per-query, or concurrency limits. Get early notice when workloads approach their limits, before queries are blocked. ## How Database Traffic Control fits with Insights Insights tracks performance at a per-query-pattern level and provides you with data to take action and improve overall performance. Database Traffic Control allows you to control which queries are allowed to execute by attaching limits to the same query activity that Insights observes. ### Database Traffic Control and pg\_strict Database Traffic Control and [pg\_strict](/docs/postgres/extensions/pg_strict) are separate features that address different concerns: * **pg\_strict** controls **which query shapes** a role is allowed to run (for example, preventing queries without appropriate predicates). * **Database Traffic Control** limits **how many resources** a workload can consume over time. These features operate independently. You can use either one on its own, or both if your needs require it. ## Manage via CLI You can manage Database Traffic Control budgets and rules using the PlanetScale CLI: ```bash theme={null} pscale traffic-control budget create [flags] pscale traffic-control rule create [flags] ``` See the [`traffic-control` CLI reference](/docs/cli/traffic-control) for the full list of sub-commands and flags. ## What Database Traffic Control is not To set expectations clearly, Database Traffic Control is **not**: * A full **web application firewall (WAF)**. It does not inspect HTTP‑level details or replace application‑side validation. * A substitute for **indexing, schema design, or query tuning**. * A global, cross‑cluster rate limiter. * A replacement for application‑level rate limiting. It protects the database, but your application may still need to shape traffic earlier in the stack. * A guarantee that queries will be given the resources they need. Traffic control rules are limits, not capacities. Think of Database Traffic Control as the last line of defense for your database. Application logic, query safety, and good schema design all help prevent problems—but when something slips through, Database Traffic Control helps your database stay alive. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Concepts Source: https://planetscale.com/docs/postgres/traffic-control/concepts Terminology and core concepts: resource budgets, rules, limits, and modes. ## Resource budgets A **resource budget** is the primary Database Traffic Control® configuration mechanism. It defines how much database resource a specific slice of traffic is allowed to consume. A resource budget: * Applies to traffic that matches one or more **rules**, which can be a specific query, a system attribute, or a combination of SQLCommenter tags. * Defines [**limits**](#resource-budget-limits) on resource consumption (capacity, rate, burst, concurrency) * Handles responses in a [**mode**](#modes-enforce-warn-off) (Enforce, Warn, Off) * Is scoped to a specific [database branch](/docs/postgres/branching) ## Resource budget limits When a query to the database arrives, Database Traffic Control first checks whether it matches any resource budget **rules**. If a match is found, the cost of executing the query is estimated, and the **limits** in that resource budget determine whether the query is allowed, warned, or blocked. ![Create a new resource budget](https://mintcdn.com/planetscale-2/Pc1vKrfqALwLm39x/postgres/traffic-control/assets/create-new-resource-budget-light.png?fit=max\&auto=format\&n=Pc1vKrfqALwLm39x\&q=85\&s=df269f5829c69351a84889834280fac4) Each resource budget contains optional settings to control the amount of resources a specific traffic slice can use. Each resource budget has four optional, configurable limits: * **Server share** - The max percentage of overall system resources available to queries managed by this budget. * **Burst limit** - Burst limit is the maximum short-term CPU burst your server can use above its steady share, measured in whole-server seconds and refilled gradually over time. * **Per-query limit** - The maximum resources any individual query can use, as a percentage of server resources in the first second. * **Maximum concurrent queries** - Percentage of available worker processes this policy can use ### How resource budgets are estimated Database Traffic Control estimates the cost of the query based on the recent behavior of similar queries. If the estimated cost exceeds the resource budget limits, the decision to block is enforced or a warning is issued, depending on the **mode** setting. * There is insufficient capacity to handle the query: `fill_level + estimated_cost > capacity` * The query is too expensive on its own: `estimated_cost > burst` These estimates are not precise, real-time measurements. Database Traffic Control uses recent query behavior to make fast decisions about whether to allow, warn, or block a query before it runs. Each unique query pattern learns its own cost model over time, so estimates improve as the system observes more executions of the same query shape. Under normal usage, queries flow through without noticeable impact. When a workload exceeds its configured limits, and the mode is set to **Enforce**, queries are blocked to keep other workloads healthy. ## Modes: Enforce, Warn, Off Each resource budget has a **mode** that controls how strictly it is applied. ![Database Traffic Control mode enforcement](https://mintcdn.com/planetscale-2/Pc1vKrfqALwLm39x/postgres/traffic-control/assets/insights-traffic-enforce-banner-light.png?fit=max\&auto=format\&n=Pc1vKrfqALwLm39x\&q=85\&s=86edb33c8a4e79d498b1e6ae2bc9243f) You can set the mode of a resource budget to Enforce, Warn, or Off. * **Enforce** * Limits are active * Queries that exceed limits are blocked * Violation counts are recorded and visible in the UI * Best choice for stabilizing workloads * Blocked queries return with the code **`53000`** and a message from `[PGINSIGHTS] Traffic Control:` and, the ID of the resource budget that blocked the query and the limit that was exceeded * **Warn** * Limits are evaluated, but not enforced * Queries always continue * Violation counts are recorded and visible in the UI * Best choice for initial rollout or tuning * Query response contains a message indicating the query would have been blocked by Database Traffic Control, and which limit was exceeded * **Off** * The resource budget does not participate in matching * No violations are recorded It is best practice to roll out a new resource budget in stages: 1. Create a resource budget in **Warn** mode 2. Observe violation patterns and adjust limits 3. Switch to **Enforce** once you are confident in the configuration ### Warning thresholds on enforced budgets Enforced resource budgets can also emit warnings before queries are blocked. You can set a separate **warning threshold** as a percentage of the burst limit, per-query limit, or concurrency limit. When a query's estimated cost crosses the warning threshold but stays below the enforced limit, the query runs normally and the client receives a SQL notice with the `[PGINSIGHTS] Traffic Control:` prefix — the same format used in Warn mode. This gives you early visibility into workloads that are approaching their limits, so you can investigate and adjust before queries start getting blocked. Warning thresholds are optional. If not set, only the enforced limits apply. ## Response messages Resource budgets in **Warn** or **Enforce** mode attach a message to the SQL response when a limit would be or was exceeded. Messages use the prefix **`[PGINSIGHTS] Traffic Control:`**, followed by the ID of the resource budget that blocked the query and the limit that was exceeded. * **Enforce** — The query is blocked. The client receives a **SQL error** with **SQLSTATE `53000`** and a message that includes `[PGINSIGHTS] Traffic Control:` and the reason. Treat this as a failed statement and use retry or fallback logic as needed. * **Warn** — The query still runs and returns a normal result. The client also receives a **SQL notice** (warning) whose text includes `[PGINSIGHTS] Traffic Control:` and the reason. Log or surface these notices for observability while tuning limits before enforcement. Applications can detect Traffic Control specifically by **SQLSTATE `53000`** on blocked queries, by matching this prefix in error or notice text, or both. ## Rules Resource budgets apply to specific traffic based on matching against a **rule**. Rules can be individual **queries**, **system attributes**, or **tags**. System attributes and tags are matched on key-value pairs. System attributes are created automatically when using Postgres, while tags must be added to your queries. You can combine rules with `AND` logic so that multiple rules must match, or `OR` logic so that any one of the rules must match, for the resource budget to apply. ![Creating resource budget rules](https://mintcdn.com/planetscale-2/Pc1vKrfqALwLm39x/postgres/traffic-control/assets/insights-traffic-edit-rules-light.png?fit=max\&auto=format\&n=Pc1vKrfqALwLm39x\&q=85\&s=f043a46a9fad0836a517d597678eeecb) This rule will match any query that contains the `action='analytics'` tag ### System attribute rules System attribute rules match key-value pairs that Postgres automatically provides for each connection. You can write rules that match against any of the following: | Attribute | Description | | - | - | | `username` | The database role or user executing the query | | `remote_address` | The client IP address of the connection | | `application_name` | The application name set by the connecting client | To create a system attribute rule, traffic matching that attribute must have recently executed queries. Usage examples include: * Limit analytics queries by having them connect as a dedicated database user, like `analytics_user` and matching `username = analytics_user` * Isolate traffic from a known internal service that connects from a fixed IP range by matching `remote_address` * Target a specific service without creating separate database users by matching `application_name` ### Tag-based rules Tag rules match against key-value pairs that your application attaches to queries using the [SQLCommenter](/docs/postgres/monitoring/query-tags) format. A tag rule targets one or many queries that share the same tag. To create a tag-based rule, recently executed queries must have included the tag. Tag rules can also match each unique value for a selected tag key. This creates separate active budget tracking for each tag value, such as each application, worker pool, user, or tenant, without creating a separate rule for every value. > [!NOTE] > > Rules that match each tag value use shared memory for every active value. The active values are stored in an LRU cache controlled by [`traffic_control.shm_size`](/docs/postgres/extensions/pginsights#database-traffic-control-shared-memory-size). The default `64kB` size tracks about 600 active budget entries at once, while the maximum `1MB` size tracks about 10,000. > > You can use this with more total tag values than fit in the cache, as long as the number of recently active values stays within the configured size for the time window Traffic Control needs to remember. Usage examples include: * Feature flags or code paths, such as `feature='beta_checkout'` * Application tags, such as `app='web'` * Route-level targeting, such as `route='api-export'` For details on the tag format, how to add tags to your queries, and framework support, see [Query tags](/docs/postgres/monitoring/query-tags). ### Query-based rules Target a specific query by using the query itself as the rule. You can create query-based rules by finding the query in the [Insights](/docs/postgres/monitoring/query-insights) page and clicking **Create resource budget**. ![Create resource budget from query](https://mintcdn.com/planetscale-2/Pc1vKrfqALwLm39x/postgres/traffic-control/assets/create-resource-budget-from-query-light.png?fit=max\&auto=format\&n=Pc1vKrfqALwLm39x\&q=85\&s=777bdfbe53b04fab93600933c65d31c3) > [!WARNING] > > This is recommended only for temporary cases where you need to test or limit the resources allocated to a specific query. Such as an emergency situation. Prefer system attribute or tag-based rules. ## Matching multiple rules Resource budget rules may contain several attributes combined with `AND` or `OR` logic. A query may also match more than one resource budget. For example, a query might match one resource budget based on its system attribute username and another based on an application tag. When a query matches multiple resource budgets, it must satisfy **all** of them. If any matching resource budget would block the query, the query is blocked. Multiple resource budgets are combined with `AND` logic, not `OR`. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Examples and recipes Source: https://planetscale.com/docs/postgres/traffic-control/examples-and-recipes Scenarios to help you design effective resource budgets. This video walkthrough demonstrates a situation where high-priority traffic is degraded by slower, low-priority traffic and how to solve it with resource budgets. [DEMO: Protect your database with Traffic Control](https://www.youtube-nocookie.com/embed/OLfeiG63UfA?rel=0) The following are more recipes and examples for common uses of Database Traffic Control. ## Example 1: Protect production from OLAP and ad‑hoc queries **Problem** Your analytics team and business intelligence (BI) tools run heavy queries on the same Postgres database as your user‑facing application. Occasionally, a dashboard or ad‑hoc query slows down the entire system. **Approach** 1. [Create a dedicated database user](/docs/postgres/connecting/roles), for example `olap_user`. 2. Require analytics tools and analysts to connect using `olap_user`. 3. In **Insights → Database Traffic Control**, click **New resource budget**: * **Name**: `OLAP user limits` * **Mode**: Warn (initially) * **Matching criteria**: System attribute `username = olap_user` * **Limits**: * Moderate server share and rate * Burst limit set low enough to prevent any single analytics query from consuming a large share of the budget * Concurrency set lower than the application users 4. Observe violations in **Warn** mode and adjust limits. 5. Switch the resource budget to **Enforce** once the pattern is stable. **Outcome** Analytics queries cannot consume unlimited concurrency or capacity. If a dashboard or exploratory query behaves badly, it hits the resource budget limits before impacting application traffic. *** ## Example 2: Tame a misbehaving integration or extract, transform, load (ETL) job **Problem** A third‑party integration or ETL job runs periodic backfills or syncs. When it runs, it spikes query volume and sometimes slows down production. **Approach** 1. [Assign the integration its own database user](/docs/postgres/connecting/roles), for example `integration_user`. 2. In **Insights → Database Traffic Control**, click **New resource budget**: * **Name**: `Integration limits` * **Mode**: Warn * **Matching criteria**: System attribute `username = integration_user` * **Limits**: * Conservative concurrency (for example, a small percentage of worker processes) * Modest capacity and rate suitable for the job's schedule * Burst low enough to prevent any single integration query from being too expensive 3. Let the resource budget run in Warn mode through several integration runs. 4. Tune limits so that normal runs do not trigger violations, but unexpected spikes do. 5. Change Mode to **Enforce**. **Outcome** The integration runs reliably, but if it is misconfigured or begins to behave differently, Database Traffic Control limits its impact and makes the problem visible via violations. *** ## Example 3: Soft‑launch a new feature with tight initial limits **Problem** You want to launch a new feature that requires several new queries. You are not yet confident about how those queries will behave in production. **Approach** 1. Ensure your application tags queries from the new feature, for example `feature='new_dashboard'`. 2. In **Insights → Database Traffic Control**, click **New resource budget**: * **Name**: `New dashboard feature` * **Mode**: Warn * **Matching criteria**: Tag rule `feature` = `new_dashboard` * **Limits**: * Tight initial limits on server share, burst limit, and maximum concurrent queries 3. Launch the feature to a subset of users. 4. Watch violations and [Insights](/docs/postgres/monitoring/query-insights) metrics to see how the feature behaves. 5. If everything looks healthy, gradually relax limits. 6. Once the feature is stable and widely rolled out, consider whether you still need a dedicated resource budget or whether the traffic can be absorbed into your general application limits. **Outcome** You can safely roll out the feature knowing that any unexpected behavior is constrained to a well‑defined resource budget, without needing to manage separate database users for each feature. *** ## Example 4: Target a specific application via tags **Problem** You want to limit or closely observe traffic from a particular application without relying on username matching. For example, a new frontend app is sending traffic to the same database as an existing monolith. **Approach** 1. Ensure your application emits a tag identifying itself, for example `app='event-website'`. 2. In **Insights → Database Traffic Control**, click **New resource budget**: * **Name**: `Event website limits` * **Mode**: Warn (initially) * **Matching criteria**: Tag rule `app` = `event-website` 3. Configure limits appropriate for this app: * Server share and rate that reflect normal behavior for this traffic * Burst limit set low enough that a single query from this app cannot consume a disproportionate share of the budget * Maximum concurrent queries lower than core app traffic 4. Let the resource budget run in **Warn** mode while the app rolls out. 5. Review violations and performance: * If the app is noisy or misbehaving, you will see frequent violations * If it behaves as expected, violations should be rare or absent 6. When you are confident in the configuration, switch the resource budget to **Enforce** to actively constrain this traffic. **Outcome** You can control and observe traffic from a specific application based on tags, without changing your database users. This is especially useful when multiple apps share the same username for historical reasons. *** ## Example 5: Limit traffic from a specific API route **Problem** A specific API endpoint in your application generates heavy database queries. For example, an `/api/export` or `/api/search` route that allows users to request large result sets, and you want to prevent these queries from impacting the rest of your application. **Approach** 1. Ensure your application uses [query tags](/docs/postgres/monitoring/query-tags) to tag queries with the originating route, for example `route='api-export'`. 2. In **Insights → Database Traffic Control**, click **New resource budget**: * **Name**: `Export endpoint limits` * **Mode**: Warn (initially) * **Matching criteria**: Tag rule `route` = `/api/export` * **Limits**: * Lower maximum concurrent queries than general application traffic * Moderate server share and rate to allow exports to complete, but with limits * Burst limit is sized so that a single large export query cannot exhaust the budget 3. Monitor violations to understand typical export behavior. 4. Adjust limits so that normal exports succeed, but unusually large or concurrent requests are constrained. 5. Switch to **Enforce** once you are confident in the configuration. **Outcome** Heavy queries from specific endpoints are isolated from the rest of your application traffic. Users can still run exports, but a spike in export requests or an unusually expensive query will not degrade performance for other users. *** These examples are starting points. As your organization gains experience, you can develop your own internal playbooks and standard resource budget templates for common workloads. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # FAQ and troubleshooting Source: https://planetscale.com/docs/postgres/traffic-control/faq-and-troubleshooting Common questions and troubleshooting help. ## Behavior and expectations > **Does Database Traffic Control change my schema, indexes, or query plans?** > > No. Database Traffic Control does not modify your schema or indexes, and it does not directly change how queries are planned. It focuses on whether queries are allowed to consume resources based on your configured resource budgets. > **Is Database Traffic Control applied per cluster or per branch?** > > Resource budgets are configured **per [database branch](/docs/postgres/branching)**. Each branch can have its own set of resource budgets. > **What happens when a query exceeds a resource budget in Enforce mode?** > > Database Traffic Control blocks the query and records a **violation**. > > The blocked query returns an error to the client with **SQLSTATE `53000`**. The error message includes **`[PGINSIGHTS] Traffic Control:`** and details about why the query was blocked (for example, which limit was exceeded). Violations are visible in the Database Traffic Control page. > **Does Database Traffic Control affect queries in Warn mode?** > > In **Warn** mode, matching queries are **not** blocked or throttled; they complete normally. > > The client still receives a **SQL notice** (warning) whose text includes **`[PGINSIGHTS] Traffic Control:`** and details about which limit would have been exceeded. Resource budgets also record **violations** for the UI. Violations are visible in the Database Traffic Control page. ## Resource budget matching > **Can a query match multiple resource budgets?** > > Yes. A query can match more than one resource budget if its attributes satisfy multiple matching criteria. For example, a query might match one resource budget based on its username and another based on an application tag. > > When a query matches multiple resource budgets, it must satisfy **all** of them. If any matching resource budget would block the query, the query is blocked. Resource budgets are combined with AND logic, not OR. > **How do I know which resource budgets a query matched?** > > The Database Traffic Control UI shows violations per resource budget. If a query is blocked, you can see which resource budget caused the violation. If a query matches multiple resource budgets but only one blocks it, the violation is attributed to the blocking resource budget. > **My SQLCommenter tags are not showing up in Insights. What's wrong?** > > The most common cause is tag placement. Tags must appear **before** the semicolon that ends the SQL statement. See the [Query tags troubleshooting](/docs/postgres/monitoring/query-tags#troubleshooting) section for the correct format and other common issues. ## Understanding and responding to high violation rates > **I see a sudden spike in violations. What should I do?** > > 1. Narrow the **time range** in the Database Traffic Control UI to the window where the spike occurs. > 2. Identify which resource budget has the highest violation count. > 3. Determine which workload or system attribute that resource budget targets. > 4. Use [Insights](/docs/postgres/monitoring/query-insights) to see what changed (new deployment, new dashboard, larger batch job, etc.). > 5. Decide whether the workload is misbehaving or the limits are too strict: > * If the workload is expected and healthy, consider relaxing limits slightly. > * If the workload is unexpected or clearly problematic, work with the owning team to correct it. > **Violations are constantly high for a critical application workload. Is that bad?** > > Consistently high violations for a key application workload suggest a mismatch between your resource budget limits and real usage. Either: > > * The application's normal behavior exceeds the resource budget limits, in which case you should adjust the resource budget; or > * The application is issuing more or heavier queries than intended, in which case you should treat this as a performance issue. > * Your queries might be poorly optimized, in which case you should inspect results in [Insights](/docs/postgres/monitoring/query-insights) for more details, or debug queries with the [PlanetScale MCP Server](/docs/mcp-server). > * Your database may be under-provisioned, in which case you should consider [upgrading your cluster](/docs/postgres/cluster-configuration) to a larger size. > > In any case, do not ignore sustained high violation counts for important workloads. ## Quickly rolling back a misconfiguration > **I enabled Enforce and now legitimate traffic is failing. How do I revert quickly?** > > You have several options: > > * Change the resource budget **Mode** from **Enforce** back to **Warn** — For new queries, this stops enforcing the resource budget while still recording violations. > * Temporarily set the resource budget **Mode** to **Off** — Use this for emergency rollback when you need to fully disable the resource budget. > * Loosen the resource budget's limits — Increase capacity, burst, or concurrency to better match real traffic. > > After stabilizing the system, review what went wrong before re‑enabling enforcement. ## Configuration tips > **How do I block expensive single queries immediately?** > > Create a resource budget that matches only that query, then set its allowed resource usage to `0` across the board. This will block all executions of that query. > **How does concurrency interact with parallel queries?** > > Concurrency is expressed as a percentage of Postgres worker processes. A query with a parallel execution plan uses multiple worker processes, so a single parallel query may consume more than one unit of concurrency. Keep this in mind when setting concurrency limits for workloads that run parallel queries. > **Can I control how much shared memory is used for resource budgets?** > > Yes. The [`traffic_control.shm_size`](/docs/postgres/extensions/pginsights#database-traffic-control-shared-memory-size) parameter controls how much shared memory is allocated to store resource budgets. The default is `64kB`. > > This is especially important for rules that match each unique value for a tag key. The default size tracks about 600 active budget entries at once. The maximum size is `1MB`, which tracks about 10,000 active budget entries. > > The entries are stored in an LRU cache, so the limit applies to recently active values, not the total number of possible values. For example, a database can have more than 10,000 tenants as long as fewer than 10,000 are active during the time window Traffic Control needs to remember. ## Interaction with connection pooling and clients > **How does Database Traffic Control interact with connection pooling?** > > Database Traffic Control is concerned with the queries that reach the database, regardless of whether they come from a pool or direct connections. > > See [Connection pooling](/docs/postgres/connecting/pgbouncer) for more information. > **Can I rely on Database Traffic Control instead of client‑side rate limiting?** > > Database Traffic Control protects your database, but it does not replace all forms of client‑side rate limiting. You may still need application‑level controls to: > > * Protect upstream services > * Enforce per‑user or per‑tenant limits that are not visible at the database layer > * Set appropriate query timeouts ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Getting started Source: https://planetscale.com/docs/postgres/traffic-control/getting-started Adopt Database Traffic Control gradually. Start with ample limits, observe real behavior, and then tighten. ## Ensure your application handles blocked queries gracefully Database Traffic Control allows you to categorize and place resource budgets on your query traffic. This is useful to opt for the failure of individual **queries** as opposed to the failure of your entire **application**. Before you start, ensure that your application can handle blocked queries gracefully without crashing or cascading failures. For example, a blocked analytics query should not take down your checkout flow. This might mean adding try/catch blocks, implementing circuit breakers, or providing fallback behavior when a query is rejected. Your application may also need retry logic for any blocked queries. For example, you can retry the query after a short delay while waiting for resources to become available. With these in place, you can configure your first resource budget. ### Restart may be required If your PlanetScale Postgres cluster existed before the launch of Database Traffic Control, you may need to update and restart your cluster to enable this feature. ## 1. Open the Database Traffic Control page **Step 1** From the PlanetScale dashboard, select your **Postgres database**. **Step 2** Go to the **Insights** page for that branch. **Step 3** Choose the **Database Traffic Control** page. You should now see: * A branch selector * A time range selector (for example, past 15 minutes, past hour, past day) * A graph for query violations * A table of existing resource budgets, if you have any ## 2. Create your first resource budget in Warn mode Start with a simple, low‑risk resource budget for a non‑critical workload, such as analytics. > [!NOTE] > > To create a resource budget that matches a specific query, you will need to find that query in the Insights page and click **Create resource budget**. It is preferred to create resource budgets based on system attributes or tags. **Step 1** On the Database Traffic Control page, click **New resource budget**. **Step 2** Give the resource budget a name (for example, `OLAP user limits`). **Step 3** Select the **Branch** you want to configure (for example, `main`). **Step 4** Select the key/value pair that will be used to match the resource budget rule. You can select any existing tags that have been included in recent queries, or alternatively create a new key/value pair in advance of those tags. For example to match any query including `username='olap_user'` you would select `username` as the key and `olap_user` as the value. **Step 5** Configure the limits applied to queries matching the resource budget rule. For now, keep them on the higher side of the slider, as you can always tighten them later. * **Server share** * **Burst limit** * **Per-query limit** * **Maximum concurrent queries** New resource budgets are started in **Warn** mode, which means they will not block any queries yet. It will only record where your limits would have been exceeded. ## 3. Observe and tune in "Warn" mode Let the resource budget run in Warn mode for at least a few traffic cycles (hours or days, depending on your workload). On the Database Traffic Control page: * Use the **time range selector** to examine recent activity. * Look at the **violations graph** to see when and how often your resource budget would have triggered. * Check the **resource budgets table** for the violation count on your new resource budget. Based on what you see: * If violations are **very frequent** during normal operations, your limits may be too strict. Consider increasing capacity or concurrency. * If violations are **rare or non‑existent**, your limits may be too generous to provide real protection. Consider tightening them slightly. Aim for a configuration where: * Normal, expected workloads do not hit the limits * Misconfigured jobs or runaway queries quickly produce visible violations Queries that would exceed the resource budget in **Warn** mode still return results, but the driver typically receives a **notice** whose text includes `[PGINSIGHTS] Traffic Control:` and the reason—use that for logging and tuning before you enable **Enforce**. ## 4. Move to Enforce Once you are comfortable with how the resource budget behaves in Warn mode: **Step 1** Communicate with the team that owns the affected workload. **Step 2** On the Database Traffic Control page, edit the resource budget. **Step 3** Change **Mode** from **Warn** to **Enforce**. **Step 4** Save the resource budget. From this point on: * Queries that exceed the configured limits for that workload will be blocked * Violations will continue to appear in the graph and the resource budgets table ## Approaches to adoption Database Traffic Control is flexible enough to support different organizational philosophies. Two common approaches are: ### Restrictive by default Start by creating a broad resource budget that applies to most or all traffic with conservative limits. Then create additional resource budgets for specific workloads that need higher limits. This approach is useful when you want tight control over resource usage and prefer to set higher limits for workloads that need them. ### Permissive by default Leave most traffic unconstrained and only create resource budgets for specific workloads you want to limit, such as analytics, integrations, or known-heavy features. This approach is useful when you want minimal friction for most workloads and only need to rein in specific problem areas. Both approaches are valid. The best approach for your organization may depend on your operational culture, the diversity of your workloads, and the level of visibility you already have into query behavior. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Out of memory (OOM) Source: https://planetscale.com/docs/postgres/troubleshooting/out-of-memory Understanding and resolving out of memory events on PlanetScale Postgres clusters. An out of memory (OOM) event occurs when Postgres processes consume more memory than is available on the cluster. When this happens, the impacted instance is automatically restarted to free up memory. When an OOM happens, we email organization admins. At most one email is sent per database branch per day, even if multiple OOMs happen. ## Impact and risks When an OOM event occurs: * **Database restart required** — The impacted instance must be restarted to recover from the OOM event. * **Downtime** — The instance that OOMed will be unavailable while it restarts. If the primary instance OOMs continuously, an automatic failover will be made. * **Connections terminated** — Direct connections and connections through the local PgBouncer are dropped during the restart, since the local PgBouncer runs on the same node as Postgres. See the [connection resilience guide](/docs/postgres/connection-resilience) for details. * **Transactions rolled back** — Any in-progress transactions that were not committed will be rolled back. ## Primary resolution: Upgrade cluster size The most direct solution to OOM events is to increase your cluster size to provide more memory. Each cluster size includes a defined amount of memory. See [cluster pricing](/docs/postgres/pricing#cluster-pricing) for available sizes and their specifications. Resizing is performed with minimal downtime. See [cluster configuration](/docs/postgres/cluster-configuration) for more details. ## Understanding memory usage High memory usage does not necessarily indicate a problem. Whether memory usage is concerning depends on the *type* of memory being used. The [Metrics](/docs/postgres/monitoring/metrics) page includes a memory chart that breaks down the different types of memory your cluster is using. Understanding these categories helps you determine if action is needed: * **Active cache** — Memory used to cache frequently accessed table and index data. This is *good* memory usage — it improves query performance by reducing disk reads. The operating system manages this cache automatically and will release it when needed. High active cache usage does not increase OOM risk. * **Inactive cache** — Memory used to cache data that hasn't been accessed recently. Like active cache, this memory is managed by the operating system and released when needed. High inactive cache usage does not increase OOM risk. * **RSS (Resident Set Size)** — Memory actively used by Postgres processes, including connection overhead and query working memory. This memory cannot be reclaimed while processes are running. High RSS usage, especially when combined with many connections, can lead to OOM events. * **Memory mapped** — Memory mapped files used by Postgres. This is typically stable and managed by the system. When evaluating memory usage, focus on RSS rather than cache memory. If your cluster shows high total memory usage but most of it is active or inactive cache, the cluster is healthy and performing optimally. The cache will be released automatically if Postgres needs the memory for other operations. ## Common causes of high memory usage ### Too many connections Postgres uses a connection-per-process architecture. Each connection made to a Postgres server [spawns a new process](https://planetscale.com/blog/processes-and-threads), which consumes system resources including memory and CPU. While the base overhead per connection is modest (a few megabytes), the real memory risk comes from `work_mem` allocations — each query operation (sorts, joins, aggregations) can allocate up to `work_mem` of memory, and a single complex query may perform multiple such operations simultaneously. If your application opens many connections — either intentionally or due to connection leaks — memory can be exhausted quickly, especially on smaller cluster sizes. **Recommendations:** * **Use connection pooling** — [PgBouncer](/docs/postgres/connecting/pgbouncer) maintains a small pool of connections to Postgres while accepting many client connections, significantly reducing memory usage. All PlanetScale Postgres databases include a local PgBouncer on port `6432`. * **Monitor active connections** — Use the [Metrics](/docs/postgres/monitoring/metrics) dashboard to track connection counts. * **Review application connection handling** — Ensure your application properly closes connections and uses connection pooling libraries. * **Configure connection limits** — Set appropriate values for `max_connections` in your [Postgres parameters](/docs/postgres/cluster-configuration/parameters). ### Memory-intensive queries Certain query patterns can consume large amounts of memory: | Query Pattern | Why It Uses Memory | | - | - | | Large `ORDER BY` operations | Sorts require memory to hold and reorder rows | | `GROUP BY` on large datasets | Aggregations build hash tables in memory | | `DISTINCT` on large result sets | Requires storing unique values for deduplication | | Hash joins on large tables | Hash tables for join keys are built in memory | | CTEs (Common Table Expressions) | May materialize entire result sets in memory | | Large `IN` clauses | Arrays and lists are stored in memory | | `array_agg()` or similar functions | Accumulates data in memory | | JSON operations on large documents | Parsing and manipulating JSON requires loading entire documents into memory | **Recommendations:** * **Add appropriate indexes** — Indexes can eliminate the need for in-memory sorts by returning data in the correct order. * **Use `LIMIT`** — Restrict result set sizes when you don't need all rows. * **Break large operations into batches** — Process data in smaller chunks rather than all at once. * **Optimize query plans** — Use `EXPLAIN (ANALYZE, VERBOSE, BUFFERS, MEMORY)` to understand how queries use memory. * **Monitor query performance** — Use [Query Insights](/docs/postgres/monitoring/query-insights) to identify memory-heavy queries. ### Postgres memory configuration Several Postgres parameters control memory allocation: | Parameter | Description | | - | - | | `work_mem` | Memory allocated per operation for sorts, hash tables, and similar operations. A single query with multiple operations can allocate this amount multiple times, potentially using several times `work_mem` in total. | | `maintenance_work_mem` | Memory for maintenance operations like `VACUUM`, `CREATE INDEX`, and `ALTER TABLE`. | | `shared_buffers` | Memory for caching table and index data. | These parameters are configured automatically based on your cluster size, but custom configurations via [Postgres parameters](/docs/postgres/cluster-configuration/parameters) may need adjustment if you experience OOM events. > [!WARNING] > > Increasing `work_mem` too high can cause OOM events if many queries run concurrently, as each query operation can allocate up to this amount. ## Monitoring and prevention To prevent OOM events: 1. **Monitor memory usage** — Use the [Metrics](/docs/postgres/monitoring/metrics) dashboard to track memory utilization over time. 2. **Set up alerts** — Use external monitoring tools to alert when memory usage exceeds thresholds. See [Prometheus integration](/docs/postgres/monitoring/prometheus-postgres) for scraping metrics into your monitoring system. 3. **Review connection counts** — High connection counts often correlate with memory pressure. 4. **Analyze slow queries** — Memory-intensive queries often appear in slow query logs. Use [Query Insights](/docs/postgres/monitoring/query-insights) to identify them. 5. **Right-size your cluster** — Ensure your cluster has adequate memory for your workload, with headroom for traffic spikes. > [!NOTE] > > Some queries can spike memory usage so quickly that the increase is not captured in the metrics graph. If you see OOM annotations on the graph but no corresponding memory spike, the OOM did occur — the memory increase simply happened too fast to be recorded. In these cases, review [Query Insights](/docs/postgres/monitoring/query-insights) to identify queries that ran around the time of the OOM event. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Read-only mode Source: https://planetscale.com/docs/postgres/troubleshooting/read-only-mode PlanetScale may automatically enable read-only mode on a cluster to maintain stability and availability when certain conditions are detected. Read-only mode preserves cluster functionality while preventing operations that could lead to instability, allowing you to remediate issues before they become critical. When read-only mode is enabled, attempts to write to the cluster will see the following error: ```sql theme={null} invalid statement because cluster is read-only ``` We will also *pause* all logical replication subscriptions writing data into the cluster. Once your cluster is out of read-only mode you can unpause the subscribers by running ``` ALTER SUBSCRIPTION sub_name ENABLE; ``` on each subscription. There's no way for a PlanetScale customer to enter or exit read-only mode, except by remediating the conditions that caused the cluster to enter that mode. Here are the reasons we'll put the cluster into read-only mode: ## Insufficient space To protect your cluster from crashing, if a cluster has very little space left (typically 5% or less), we will automatically move it into read-only mode. The specific scenarios are: | Storage Type | Scenario | Remediation | | - | - | - | | **Network-attached storage** | Clusters with [Autoscaling](/docs/postgres/cluster-configuration/cluster-storage#enable-autoscaling) disabled or where your current [disk size](/docs/postgres/cluster-configuration/cluster-storage#minimum-disk-size-configuration) is approaching your configured [Autoscaling maximum](/docs/postgres/cluster-configuration/cluster-storage#storage-limit) | Increase the [cluster disk size](/docs/postgres/cluster-configuration/cluster-storage#minimum-disk-size-configuration) manually or increase the [storage limit on disk autoscaling](/docs/postgres/cluster-configuration/cluster-storage#storage-limit) | | **PlanetScale Metal** | When your data is approaching the storage size of your cluster | Increase the [cluster size](/docs/metal) | Once you have remediated the issue, your cluster will automatically exit read-only mode. > [!NOTE] > > PlanetScale will send notifications via email and webhook (if enabled) when your storage exceeds 60, 75, 85, 90, and 95 percent, respectively. You can also track disk utilization via [Metrics](/docs/postgres/monitoring/metrics). Emails are sent to all Organization and Database administrators. It is critical that these email addresses are monitored regularly. If the issue was temporary, say due to errant data being written by mistake, you could remove the data, perform a vacuum, and then reverse the remediation action. See [Cluster storage configuration](/docs/postgres/cluster-configuration/cluster-storage) for more on reducing storage usage. ## Single Node Archiver lag Single node Postgres clusters offer decreased durability compared to HA offerings. In HA, we use Postgres synchronous replication to guarantee every committed transaction reaches the primary and at least one replica. As part of our usual (we do this on HA too) durability posture, we archive postgres Write-Ahead Logs (WAL) to durable object storage (S3, GCP Cloud Storage). For single-node, we will enter read-only mode if this archiving process falls too far behind writes on your cluster. If you see this happening frequently, we recommend sizing your cluster up, or moving to an HA cluster. ## Read-only transactions vs cluster read-only mode Cluster read-only mode and read-only transactions are different failure modes with different error messages. | | Cluster read-only mode | Read-only transaction | | - | - | - | | Error | `invalid statement because cluster is read-only` | `cannot execute in a read-only transaction` (SQLSTATE `25006`) | | Cause | PlanetScale protects the cluster when storage is low or archiver lag is high | The current transaction is read only | | Branch state | Branch enters read-only mode | Branch stays writable | A transaction becomes read only through `BEGIN READ ONLY`, `SET TRANSACTION READ ONLY`, session characteristics, role defaults, `default_transaction_read_only`, and similar settings in application code, an ORM, or a database driver. If only some writes fail while others succeed on the same branch, the branch is likely not in read-only mode. [PgBouncer](/docs/postgres/connecting/pgbouncer) runs in transaction pooling mode and reuses server connections across clients. When session parameters such as `default_transaction_read_only` are not tracked, read-only state from a prior client can persist on a server connection and affect the next client. A plain `SET` can leak across transactions (including `BEGIN; SET ...; COMMIT`); only `SET LOCAL` is limited to the current transaction. See [limitations of transaction pooling](/docs/postgres/connecting/pgbouncer#limitations-of-transaction-pooling) and [PgBouncer error messages](/docs/postgres/connecting/pgbouncer#pgbouncer-error-messages) for more on pooled connection behavior. To resolve read-only transaction errors: 1. Search application code, ORM configuration, and driver options for read-only transaction modes, session characteristics, or replica routing that sets read-only sessions on a write connection. 2. Remove the setting if writes should use the same connection, use `SET LOCAL` when a per-transaction setting is required, or route read-only traffic through a separate [role](/docs/postgres/connecting/roles) and connection (for example, a [replica PgBouncer](/docs/postgres/connecting/pgbouncer#dedicated-replica-pgbouncers)) instead of toggling read-only on a shared pooled connection. # Switchovers Source: https://planetscale.com/docs/postgres/troubleshooting/switchovers Move the primary of a Postgres branch to a replica on demand. ## Overview A switchover moves the primary of a Postgres branch to one of its replicas on demand. The current primary steps down and a replica is promoted in its place, using the same battle-tested mechanism PlanetScale uses for [automatic failovers](/docs/postgres/operations-philosophy). Writes are briefly interrupted (on the order of seconds) while the switch completes. Switchovers are useful when you want to: * Verify that your application tolerates a primary change before you depend on it during an unplanned failover * Move the primary ahead of planned work, instead of waiting for routine maintenance to switch over the primary for you > [!NOTE] > > A branch without replicas has nothing to promote. Requesting a switchover on a single-node branch **restarts the instance in place** instead, and the branch is unreachable while it comes back. Check the `method` field on the switchover (`switchover` or `restart`) to see which one a branch got. ## Start a switchover A branch accepts one switchover at a time. You can start one from the dashboard, the CLI, or the API. ### Dashboard On the branch page, open the cluster actions menu (the kebab icon next to the cluster summary) and select **Switch over to replica**. On a single-node branch the action reads **Restart cluster** instead. ### CLI ```bash theme={null} pscale branch switchover $DATABASE $BRANCH ``` ### API ```bash theme={null} curl -X POST \ "https://api.planetscale.com/v1/organizations/$ORG/databases/$DATABASE/branches/$BRANCH/switchovers" \ -H "Authorization: $API_TOKEN" ``` See the [create switchover API reference](/docs/api/reference/create_switchover) for details. ## Choose the replica to promote By default, an eligible replica is selected automatically. To pick one yourself, first find replica names with `pscale branch infra $DATABASE $BRANCH` or on the branch page in the dashboard, then pass the exact replica `name` as the candidate: ```bash theme={null} pscale branch switchover $DATABASE $BRANCH --candidate $REPLICA_NAME ``` ```bash theme={null} curl -X POST \ "https://api.planetscale.com/v1/organizations/$ORG/databases/$DATABASE/branches/$BRANCH/switchovers" \ -H "Authorization: $API_TOKEN" \ -d '{"candidate": "$REPLICA_NAME"}' ``` A candidate can only be specified for a branch with replicas. ## Track the outcome A switchover moves through `pending`, `running`, and a terminal state of `succeeded` or `failed`. List or fetch switchovers via the API: ```bash theme={null} curl "https://api.planetscale.com/v1/organizations/$ORG/databases/$DATABASE/branches/$BRANCH/switchovers" \ -H "Authorization: $API_TOKEN" ``` > [!WARNING] > > A switchover that ends in the `failed` state has an **unconfirmed outcome**: the primary may still have moved, and nothing is rolled back. Check which instance is currently primary (with `pscale branch infra` or the dashboard) before retrying. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres with Bun Source: https://planetscale.com/docs/postgres/tutorials/planetscale-postgres-bun Create a new Postgres database and integrate it with Bun. Bun provides native support for Postgres databases. Already created a PlanetScale Postgres database? [Jump straight to integration instructions](#integrate-with-bun). We'll cover: * Creating a new Postgres database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Postgres clusters use real Postgres in a [high-availability architecture with one primary and two replicas](/docs/postgres/postgres-architecture/#cluster-design). ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Postgres** to create a PostgreSQL database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > This "name" is referenced in the PlanetScale Dashboard and APIs and not created as a logical database inside of Postgres. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred cluster size and [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Postgres database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You’ll be taken to a screen in the browser where you’ll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Postgres database by running: ```bash theme={null} pscale database create --region --engine postgres ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Postgres database cluster, PlanetScale automatically: * Provisions a PostgreSQL cluster in your selected region * Creates the initial `main` branch * Prepopulates Postgres with required default databases * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Create credentials and connect > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). In this section you'll create the "Default role" in your PlanetScale dashboard to create connection credentials for your database branch. > [!NOTE] > > The "Default role" is meant purely for administrative purposes. You can only create one and it has significant privileges for your database cluster and you should treat these credentials carefully. After completing this quickstart, it is *strongly recommended* that you [create another role](/docs/postgres/connecting/roles) for your application use-cases. **Dashboard** ![Database dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/new-database.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=0dc877c9af2d9be252c165c0c16274b8) **Step 1** Navigate to your database in the [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Click on the **"Connect"** button in the top right **Step 3** Select **"Default role"** **Step 4** Click **"Create default role"**. A new default role is created for your database branch. **Step 5** Record the "Host", "Username", and "Password" for the "Default role" someplace secure. ![Create a new role](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/create-role.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=dd760b8bac6d56d7ea96ef3bb981c666) **Step 6** You can generate connection strings under **"How are you connecting?"** for major languages, frameworks, and tools. ![Connection strings](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/langs-and-frames.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=709ecbea3be475aeadf8c19fae75dc87) Your connection details will include: * **Host**: the DNS name of your database endpoint * **Username**: automatically formatted for routing to the correct `branch` * **Password**: A securely generated password * **Database**: `postgres` (default database) * **Port**: `5432` (standard PostgreSQL port) or `6432` (for using [PgBouncer](/docs/postgres/connecting/pgbouncer)) **CLI** Create a new "Default role" in your PlanetScale CLI to create connection credentials for your database branch. **Step 1** Run the following command to create the "Default role" for your database branch. ```bash theme={null} pscale role reset-default ``` **Step 2** Record the "Host", "Username", and "Password" for the "Default role" somewhere secure. > [!NOTE] > > Passwords are shown only once. If you lose your record of the password, you must [reset the password](/docs/postgres/connecting/roles). ## Integrate with Bun ### Step 1: Initialization You will need to have Bun installed. See the Bun docs for [installation instructions](https://bun.sh/docs/installation). If you do not already have a Bun project, initialize one with the following command: ```bash Terminal theme={null} bun init ``` ### Step 2: Add credentials to .env For local development, you can place your credentials in a `.env` file. For production, we recommend setting your credentials as environment variables wherever your application is deployed. Bun will automatically load your credentials from the `.env` file. Replace the placeholders below with the role credentials created in the previous section. ```bash .env theme={null} DATABASE_URL='postgresql://{username}:{password}@{host}:{port}/{database}?sslmode=verify-full' ``` Choose the appropriate **port** for your use case. Learn more about [Direct vs PgBouncer connections](/docs/postgres/connecting/quickstart#connection-types%3A-direct-vs-pgbouncer). > **PgBouncer** > > Port `6432` enables a lightweight connection pooler for PostgreSQL. This facilitates better performance when there are many simultaneous connections. > **Direct** > > Port `5432` connects directly to PostgreSQL. Total connections are limited by your cluster's `max_connections` setting. > [!NOTE] > > Both connection types will disconnect when your database restarts or handles a failover scenario. ### Step 3: Create a database connection To run a query, you can use the `sql` template literal. As Postgres is the default choice in Bun, using it requires no additional configuration. ```typescript index.ts theme={null} import { sql } from 'bun' const now = await sql` SELECT NOW() ` console.log(now) ``` From the command line, run the following command to execute the script: ```bash Terminal theme={null} bun index.ts ``` See the [Bun SQL documentation](https://bun.sh/docs/runtime/sql#postgresql) for more information about queries and mutations. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres with Cloudflare Workers Source: https://planetscale.com/docs/postgres/tutorials/planetscale-postgres-cloudflare-workers Create a new Postgres database and integrate it with Cloudflare Workers and Hyperdrive. [Cloudflare Workers](https://developers.cloudflare.com/workers/) is a serverless platform that allows you to run your code at the edge, close to your users. [Hyperdrive](https://developers.cloudflare.com/hyperdrive/) accelerates queries you make to existing databases. Want to create and pay for your database from Cloudflare without adding billing to PlanetScale? See [Create PlanetScale databases from Cloudflare](/docs/connect/cloudflare). Already created a PlanetScale Postgres database? [Jump straight to integration instructions](#integrate-with-cloudflare-workers). We'll cover: * Creating a new Postgres database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Postgres clusters use real Postgres in a [high-availability architecture with one primary and two replicas](/docs/postgres/postgres-architecture/#cluster-design). ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Postgres** to create a PostgreSQL database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > This "name" is referenced in the PlanetScale Dashboard and APIs and not created as a logical database inside of Postgres. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred cluster size and [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Postgres database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You’ll be taken to a screen in the browser where you’ll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Postgres database by running: ```bash theme={null} pscale database create --region --engine postgres ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Postgres database cluster, PlanetScale automatically: * Provisions a PostgreSQL cluster in your selected region * Creates the initial `main` branch * Prepopulates Postgres with required default databases * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Integrate with Cloudflare Workers Don't have a Workers project yet? [Create a Workers project from the Postgres Hyperdrive template](https://developers.cloudflare.com/workers/get-started/quickstarts/#postgres-hyperdrive-template). ```bash Terminal theme={null} npm create cloudflare@latest -- --template=cloudflare/templates/postgres-hyperdrive-template ``` ### Step 1: Create a Hyperdrive connection You can automatically create a connection from the PlanetScale dashboard when creating a new role, or use one of the methods below. **Dashboard** **Step 1** Log into the Cloudflare dashboard and navigate to **"Compute & AI"** > **"Workers & Pages"**. You should see your Worker in the list. ![PlanetScale Cloudflare integration wizard - step 1](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_1.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=21239d9ba0e43e1616b020c77fe617b7) **Step 2** Select the **"Storage & databases"** tab, then **"Hyperdrive"**, and finally **"Create configuration"** in the top right corner. ![PlanetScale Cloudflare integration wizard - step 2](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_2.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=a6ca84d0b9f0f7a54dc37911e5d9f0b5) **Step 3** Click **"Connect to PlanetScale database"** then click **Next**. ![PlanetScale Cloudflare integration wizard - step 3](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_3.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=c42406780d8a671ed5c8d01b9be867f0) **Step 4** Click **Login to PlanetScale** to log in to your PlanetScale account. ![PlanetScale Cloudflare integration wizard - step 4](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_4.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=ee9e19bd44f298fe8a85958eee20a965) **Step 5** The next screen will allow you to grant access to your organization, database, and branch. Start by selecting your organization from the list. ![PlanetScale Cloudflare integration wizard - step 5](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_5.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=f22610297689c9d3b6ae4d9a651c1f0b) **Step 6** Select the database you created, the "main" branch and then click **Next**. You must include a **database name**. For MySQL/Vitess databases, the default database name is the same as your project name (for example, `my-project`). For Postgres databases, the default database name is `postgres`. ![PlanetScale Cloudflare integration wizard - step 6](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_6_unedited.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=5135047d924040fb2ce714b86799c146) **Step 7** You should see a success screen with the connection binding details to add to your `wrangler.json` file. ![PlanetScale Cloudflare integration wizard - step 7](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_7.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=83cca64caf0b9e48d33761e8f1551359) **Step 8** You should now have metrics within your Cloudflare dashboard in the Hyperdrive page. ![PlanetScale Cloudflare integration wizard - step 8](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_8.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=6e2001b97e976e539176075edc476dfb) **CLI** Create a Hyperdrive connection by running the `wrangler` command below: * Replace the `--connection-string` placeholders with your database credentials * Replace the `` placeholder with a name for your Hyperdrive connection **For MySQL/Vitess databases:** ```bash Terminal theme={null} npx wrangler hyperdrive create --connection-string="mysql://:@/" ``` **For Postgres databases:** ```bash Terminal theme={null} npx wrangler hyperdrive create --connection-string="postgresql://:@:/?sslmode=verify-full" ``` If successful, the command will output your new Hyperdrive configuration, for example: ```json Terminal theme={null} { "hyperdrive": [ { "binding": "HYPERDRIVE", "id": "" } ] } ``` ### Configure Worker placement By default, Workers run at the edge close to your users. For database-heavy workloads with multiple round trips, you can use [placement hints](https://developers.cloudflare.com/workers/configuration/smart-placement/) to run your Worker closer to your PlanetScale database and reduce latency. Add a `placement` configuration to your `wrangler.jsonc` file with the region that matches your PlanetScale database region: ```json wrangler.json theme={null} { "placement": { "region": "aws:us-east-1" } } ``` > **PlanetScale region to placement hint mapping** > > | PlanetScale Region | Placement Hint | > | - | - | > | AWS ap-northeast-1 (Tokyo) | `aws:ap-northeast-1` | > | AWS ap-south-1 (Mumbai) | `aws:ap-south-1` | > | AWS ap-southeast-1 (Singapore) | `aws:ap-southeast-1` | > | AWS ap-southeast-2 (Sydney) | `aws:ap-southeast-2` | > | AWS ca-central-1 (Montreal) | `aws:ca-central-1` | > | AWS eu-central-1 (Frankfurt) | `aws:eu-central-1` | > | AWS eu-west-1 (Dublin) | `aws:eu-west-1` | > | AWS eu-west-2 (London) | `aws:eu-west-2` | > | AWS sa-east-1 (Sao Paulo) | `aws:sa-east-1` | > | AWS us-east-1 (N. Virginia) | `aws:us-east-1` | > | AWS us-east-2 (Ohio) | `aws:us-east-2` | > | AWS us-west-2 (Oregon) | `aws:us-west-2` | > | GCP asia-northeast3 (Seoul) | `gcp:asia-northeast3` | > | GCP europe-west1 (Belgium) | `gcp:europe-west1` | > | GCP europe-west4 (Netherlands) | `gcp:europe-west4` | > | GCP northamerica-northeast1 (Montreal) | `gcp:northamerica-northeast1` | > | GCP us-central1 (Iowa) | `gcp:us-central1` | > | GCP us-east1 (South Carolina) | `gcp:us-east1` | > | GCP us-east4 (Virginia) | `gcp:us-east4` | Your Worker will run in the Cloudflare data center with the lowest latency to the specified region. If your Worker connects to multiple back-end services and you're unsure which region to specify, use `"mode": "smart"` for automatic placement based on measured latency: ```json wrangler.json theme={null} { "placement": { "mode": "smart" } } ``` ### Step 2: Deploy your Worker Run the following to deploy your Worker. ```bash Terminal theme={null} npx wrangler deploy ``` For more information on using Cloudflare Workers and Hyperdrive, [refer to the Cloudflare documentation](https://developers.cloudflare.com/hyperdrive/). ## What's next? Once you're done with development, consider these next steps: * **Create additional roles**: Set up [application-specific roles](/docs/postgres/connecting/roles) with appropriate permissions for your production workloads. The "Default role" is meant for administrative purposes and has significant privileges. * **Use branching for development**: Create [database branches](/docs/postgres/branching) to safely test schema changes before deploying to production. * **Set up monitoring**: Configure [query insights](/docs/postgres/monitoring/query-insights) to monitor database performance and identify slow queries. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres with Drizzle Source: https://planetscale.com/docs/postgres/tutorials/planetscale-postgres-drizzle Create a new Postgres database and integrate it with Drizzle. Drizzle is a modern SQL ORM for TypeScript that allows you to write type-safe SQL queries and migrations. Already created a PlanetScale Postgres database? [Jump straight to integration instructions](#integrate-with-drizzle). We'll cover: * Creating a new Postgres database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Postgres clusters use real Postgres in a [high-availability architecture with one primary and two replicas](/docs/postgres/postgres-architecture/#cluster-design). ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Postgres** to create a PostgreSQL database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > This "name" is referenced in the PlanetScale Dashboard and APIs and not created as a logical database inside of Postgres. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred cluster size and [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Postgres database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You’ll be taken to a screen in the browser where you’ll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Postgres database by running: ```bash theme={null} pscale database create --region --engine postgres ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Postgres database cluster, PlanetScale automatically: * Provisions a PostgreSQL cluster in your selected region * Creates the initial `main` branch * Prepopulates Postgres with required default databases * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Create credentials and connect > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). In this section you'll create the "Default role" in your PlanetScale dashboard to create connection credentials for your database branch. > [!NOTE] > > The "Default role" is meant purely for administrative purposes. You can only create one and it has significant privileges for your database cluster and you should treat these credentials carefully. After completing this quickstart, it is *strongly recommended* that you [create another role](/docs/postgres/connecting/roles) for your application use-cases. **Dashboard** ![Database dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/new-database.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=0dc877c9af2d9be252c165c0c16274b8) **Step 1** Navigate to your database in the [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Click on the **"Connect"** button in the top right **Step 3** Select **"Default role"** **Step 4** Click **"Create default role"**. A new default role is created for your database branch. **Step 5** Record the "Host", "Username", and "Password" for the "Default role" someplace secure. ![Create a new role](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/create-role.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=dd760b8bac6d56d7ea96ef3bb981c666) **Step 6** You can generate connection strings under **"How are you connecting?"** for major languages, frameworks, and tools. ![Connection strings](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/langs-and-frames.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=709ecbea3be475aeadf8c19fae75dc87) Your connection details will include: * **Host**: the DNS name of your database endpoint * **Username**: automatically formatted for routing to the correct `branch` * **Password**: A securely generated password * **Database**: `postgres` (default database) * **Port**: `5432` (standard PostgreSQL port) or `6432` (for using [PgBouncer](/docs/postgres/connecting/pgbouncer)) **CLI** Create a new "Default role" in your PlanetScale CLI to create connection credentials for your database branch. **Step 1** Run the following command to create the "Default role" for your database branch. ```bash theme={null} pscale role reset-default ``` **Step 2** Record the "Host", "Username", and "Password" for the "Default role" somewhere secure. > [!NOTE] > > Passwords are shown only once. If you lose your record of the password, you must [reset the password](/docs/postgres/connecting/roles). ## Integrate with Drizzle Installation instructions vary depending on your choice of JavaScript runtime or framework. The following are general instructions for installing Drizzle and connecting to your database. ### Step 1: Install packages Run the following to install `drizzle-orm` and `pg`: ```bash Terminal theme={null} npm install drizzle-orm pg ``` Optionally, you can also install `dotenv` to load your credentials from an `.env` file for local development. ```bash Terminal theme={null} npm install dotenv ``` ### Step 2: Add credentials to .env For local development, you can place your credentials in a `.env` file. For production, we recommend setting your credentials as environment variables wherever your application is deployed. Replace the placeholders below with the role credentials created in the previous section. ```bash .env theme={null} DATABASE_URL='postgresql://{username}:{password}@{host}:{port}/{database}?sslmode=verify-full' ``` Choose the appropriate **port** for your use case. Learn more about [Direct vs PgBouncer connections](/docs/postgres/connecting/quickstart#connection-types%3A-direct-vs-pgbouncer). > **PgBouncer** > > Port `6432` enables a lightweight connection pooler for PostgreSQL. This facilitates better performance when there are many simultaneous connections. > **Direct** > > Port `5432` connects directly to PostgreSQL. Total connections are limited by your cluster's `max_connections` setting. > [!NOTE] > > Both connection types will disconnect when your database restarts or handles a failover scenario. ### Step 3: Create a database connection ```typescript src/db/index.ts theme={null} import 'dotenv/config' import { drizzle } from 'drizzle-orm/node-postgres' export const db = drizzle(process.env.DATABASE_URL) ``` Where you import this `db` variable will depend on your choice of JavaScript runtime or framework. For more information on using Drizzle including creating schemas and performing queries, [refer to their documentation](https://orm.drizzle.team/docs). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres with Kysely Source: https://planetscale.com/docs/postgres/tutorials/planetscale-postgres-kysely Create a new Postgres database and integrate it with Kysely. Kysely is a type-safe SQL query builder for TypeScript. Already created a PlanetScale Postgres database? [Jump straight to integration instructions](#integrate-with-kysely). We'll cover: * Creating a new Postgres database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Postgres clusters use real Postgres in a [high-availability architecture with one primary and two replicas](/docs/postgres/postgres-architecture/#cluster-design). ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Postgres** to create a PostgreSQL database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > This "name" is referenced in the PlanetScale Dashboard and APIs and not created as a logical database inside of Postgres. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred cluster size and [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Postgres database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You’ll be taken to a screen in the browser where you’ll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Postgres database by running: ```bash theme={null} pscale database create --region --engine postgres ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Postgres database cluster, PlanetScale automatically: * Provisions a PostgreSQL cluster in your selected region * Creates the initial `main` branch * Prepopulates Postgres with required default databases * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Create credentials and connect > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). In this section you'll create the "Default role" in your PlanetScale dashboard to create connection credentials for your database branch. > [!NOTE] > > The "Default role" is meant purely for administrative purposes. You can only create one and it has significant privileges for your database cluster and you should treat these credentials carefully. After completing this quickstart, it is *strongly recommended* that you [create another role](/docs/postgres/connecting/roles) for your application use-cases. **Dashboard** ![Database dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/new-database.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=0dc877c9af2d9be252c165c0c16274b8) **Step 1** Navigate to your database in the [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Click on the **"Connect"** button in the top right **Step 3** Select **"Default role"** **Step 4** Click **"Create default role"**. A new default role is created for your database branch. **Step 5** Record the "Host", "Username", and "Password" for the "Default role" someplace secure. ![Create a new role](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/create-role.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=dd760b8bac6d56d7ea96ef3bb981c666) **Step 6** You can generate connection strings under **"How are you connecting?"** for major languages, frameworks, and tools. ![Connection strings](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/langs-and-frames.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=709ecbea3be475aeadf8c19fae75dc87) Your connection details will include: * **Host**: the DNS name of your database endpoint * **Username**: automatically formatted for routing to the correct `branch` * **Password**: A securely generated password * **Database**: `postgres` (default database) * **Port**: `5432` (standard PostgreSQL port) or `6432` (for using [PgBouncer](/docs/postgres/connecting/pgbouncer)) **CLI** Create a new "Default role" in your PlanetScale CLI to create connection credentials for your database branch. **Step 1** Run the following command to create the "Default role" for your database branch. ```bash theme={null} pscale role reset-default ``` **Step 2** Record the "Host", "Username", and "Password" for the "Default role" somewhere secure. > [!NOTE] > > Passwords are shown only once. If you lose your record of the password, you must [reset the password](/docs/postgres/connecting/roles). ## Integrate with Kysely Installation instructions vary depending on your choice of JavaScript runtime or framework. The following are general instructions for installing Drizzle and connecting to your database. ### Step 1: Install packages Run the following to install `kysely` and `pg`: ```bash Terminal theme={null} npm install kysely pg ``` Optionally, you can also install `dotenv` to load your credentials from an `.env` file for local development. ```bash Terminal theme={null} npm install dotenv ``` ### Step 2: Add credentials to .env For local development, you can place your credentials in a `.env` file. For production, we recommend setting your credentials as environment variables wherever your application is deployed. Replace the placeholders below with the role credentials created in the previous section. ```bash .env theme={null} DATABASE_URL='postgresql://{username}:{password}@{host}:{port}/{database}?sslmode=verify-full' ``` Choose the appropriate **port** for your use case. Learn more about [Direct vs PgBouncer connections](/docs/postgres/connecting/quickstart#connection-types%3A-direct-vs-pgbouncer). > **PgBouncer** > > Port `6432` enables a lightweight connection pooler for PostgreSQL. This facilitates better performance when there are many simultaneous connections. > **Direct** > > Port `5432` connects directly to PostgreSQL. Total connections are limited by your cluster's `max_connections` setting. > [!NOTE] > > Both connection types will disconnect when your database restarts or handles a failover scenario. ### Step 3: Create a database connection ```typescript app.ts theme={null} import 'dotenv/config' import { Kysely, PostgresDialect } from 'kysely' import { Pool } from 'pg' import { Database } from './types.ts' // you set this up locally const db = new Kysely({ dialect: new PostgresDialect({ pool: new Pool({ connectionString: process.env.DATABASE_URL, database: 'postgres' }) }) }) ``` Where you import this `db` variable will depend on your choice of JavaScript runtime or framework. For more information on using Kysely including creating schemas and performing queries, [refer to their documentation](https://kysely.dev/docs/getting-started). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres with Laravel Source: https://planetscale.com/docs/postgres/tutorials/planetscale-postgres-laravel Create a new Postgres database and integrate it with Laravel. Laravel is a popular, fully-featured PHP framework for building web applications. Already created a PlanetScale Postgres database? [Jump straight to integration instructions](#integrate-with-laravel). We'll cover: * Creating a new Postgres database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Postgres clusters use real Postgres in a [high-availability architecture with one primary and two replicas](/docs/postgres/postgres-architecture/#cluster-design). ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Postgres** to create a PostgreSQL database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > This "name" is referenced in the PlanetScale Dashboard and APIs and not created as a logical database inside of Postgres. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred cluster size and [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Postgres database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You’ll be taken to a screen in the browser where you’ll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Postgres database by running: ```bash theme={null} pscale database create --region --engine postgres ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Postgres database cluster, PlanetScale automatically: * Provisions a PostgreSQL cluster in your selected region * Creates the initial `main` branch * Prepopulates Postgres with required default databases * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Create credentials and connect > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). In this section you'll create the "Default role" in your PlanetScale dashboard to create connection credentials for your database branch. > [!NOTE] > > The "Default role" is meant purely for administrative purposes. You can only create one and it has significant privileges for your database cluster and you should treat these credentials carefully. After completing this quickstart, it is *strongly recommended* that you [create another role](/docs/postgres/connecting/roles) for your application use-cases. **Dashboard** ![Database dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/new-database.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=0dc877c9af2d9be252c165c0c16274b8) **Step 1** Navigate to your database in the [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Click on the **"Connect"** button in the top right **Step 3** Select **"Default role"** **Step 4** Click **"Create default role"**. A new default role is created for your database branch. **Step 5** Record the "Host", "Username", and "Password" for the "Default role" someplace secure. ![Create a new role](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/create-role.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=dd760b8bac6d56d7ea96ef3bb981c666) **Step 6** You can generate connection strings under **"How are you connecting?"** for major languages, frameworks, and tools. ![Connection strings](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/langs-and-frames.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=709ecbea3be475aeadf8c19fae75dc87) Your connection details will include: * **Host**: the DNS name of your database endpoint * **Username**: automatically formatted for routing to the correct `branch` * **Password**: A securely generated password * **Database**: `postgres` (default database) * **Port**: `5432` (standard PostgreSQL port) or `6432` (for using [PgBouncer](/docs/postgres/connecting/pgbouncer)) **CLI** Create a new "Default role" in your PlanetScale CLI to create connection credentials for your database branch. **Step 1** Run the following command to create the "Default role" for your database branch. ```bash theme={null} pscale role reset-default ``` **Step 2** Record the "Host", "Username", and "Password" for the "Default role" somewhere secure. > [!NOTE] > > Passwords are shown only once. If you lose your record of the password, you must [reset the password](/docs/postgres/connecting/roles). ## Integrate with Laravel Laravel contains a built-in ORM, [Eloquent](https://laravel.com/docs/master/eloquent), that allows you to interact with your database using PHP objects. ### Step 1: Add credentials to .env For local development, you can place your credentials in a `.env` file. For production, we recommend setting your credentials as environment variables wherever your application is deployed. Replace the placeholders below with the role credentials created in the previous section. ```bash .env theme={null} DB_CONNECTION=pgsql DB_HOST= DB_PORT= DB_DATABASE= DB_USERNAME= DB_PASSWORD= ``` Choose the appropriate **port** for your use case. Learn more about [Direct vs PgBouncer connections](/docs/postgres/connecting/quickstart#connection-types%3A-direct-vs-pgbouncer). > **PgBouncer** > > Port `6432` enables a lightweight connection pooler for PostgreSQL. This facilitates better performance when there are many simultaneous connections. > **Direct** > > Port `5432` connects directly to PostgreSQL. Total connections are limited by your cluster's `max_connections` setting. > [!NOTE] > > Both connection types will disconnect when your database restarts or handles a failover scenario. > [!NOTE] > > When using [PgBouncer](/docs/postgres/connecting/pgbouncer) (port `6432`), set `DB_POOLED=true` and ensure your `pgsql` connection in `config/database.php` includes `'pooled' => env('DB_POOLED', false)`. This option is available in Laravel 13.17.0 and later. See [Laravel's pooled PostgreSQL connections](https://laravel.com/docs/13.x/database#pooled-postgresql-connections). ### Step 2: Create a database connection The following example loads in your credentials from your `.env` file, establishes a connection, and, if successful, prints what database you are connected to. Add the following code to your `routes/web.php` file: ```php routes/web.php theme={null} use Illuminate\Support\Facades\DB; use Illuminate\Support\Facades\Route; Route::get('/test-database', function () { try { DB::connection()->getPdo(); print_r("Connected successfully to: " . DB::connection()->getDatabaseName()); } catch (\Exception $e) { die("Could not connect to the database. Please check your configuration. Error: " . $e); } }); ``` Run the following to start the Laravel server, then navigate to `http://localhost:8000/test-database` to see the results. ```bash Terminal theme={null} php artisan serve ``` See the [Laravel documentation on databases](https://laravel.com/docs/master/database) for more information. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres with Node.js Source: https://planetscale.com/docs/postgres/tutorials/planetscale-postgres-node Create a new Postgres database and integrate it with Node.js. Node.js is a popular JavaScript runtime for building server-side applications. Already created a PlanetScale Postgres database? [Jump straight to integration instructions](#integrate-with-nodejs). We'll cover: * Creating a new Postgres database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Postgres clusters use real Postgres in a [high-availability architecture with one primary and two replicas](/docs/postgres/postgres-architecture/#cluster-design). ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Postgres** to create a PostgreSQL database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > This "name" is referenced in the PlanetScale Dashboard and APIs and not created as a logical database inside of Postgres. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred cluster size and [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Postgres database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You’ll be taken to a screen in the browser where you’ll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Postgres database by running: ```bash theme={null} pscale database create --region --engine postgres ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Postgres database cluster, PlanetScale automatically: * Provisions a PostgreSQL cluster in your selected region * Creates the initial `main` branch * Prepopulates Postgres with required default databases * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Create credentials and connect > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). In this section you'll create the "Default role" in your PlanetScale dashboard to create connection credentials for your database branch. > [!NOTE] > > The "Default role" is meant purely for administrative purposes. You can only create one and it has significant privileges for your database cluster and you should treat these credentials carefully. After completing this quickstart, it is *strongly recommended* that you [create another role](/docs/postgres/connecting/roles) for your application use-cases. **Dashboard** ![Database dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/new-database.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=0dc877c9af2d9be252c165c0c16274b8) **Step 1** Navigate to your database in the [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Click on the **"Connect"** button in the top right **Step 3** Select **"Default role"** **Step 4** Click **"Create default role"**. A new default role is created for your database branch. **Step 5** Record the "Host", "Username", and "Password" for the "Default role" someplace secure. ![Create a new role](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/create-role.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=dd760b8bac6d56d7ea96ef3bb981c666) **Step 6** You can generate connection strings under **"How are you connecting?"** for major languages, frameworks, and tools. ![Connection strings](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/langs-and-frames.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=709ecbea3be475aeadf8c19fae75dc87) Your connection details will include: * **Host**: the DNS name of your database endpoint * **Username**: automatically formatted for routing to the correct `branch` * **Password**: A securely generated password * **Database**: `postgres` (default database) * **Port**: `5432` (standard PostgreSQL port) or `6432` (for using [PgBouncer](/docs/postgres/connecting/pgbouncer)) **CLI** Create a new "Default role" in your PlanetScale CLI to create connection credentials for your database branch. **Step 1** Run the following command to create the "Default role" for your database branch. ```bash theme={null} pscale role reset-default ``` **Step 2** Record the "Host", "Username", and "Password" for the "Default role" somewhere secure. > [!NOTE] > > Passwords are shown only once. If you lose your record of the password, you must [reset the password](/docs/postgres/connecting/roles). ## Integrate with Node.js ### Step 1: Install packages First, you will need to install the `pg` package: ```bash theme={null} npm install pg ``` Optionally, you can also install `dotenv` to load your credentials from an `.env` file for local development: ```bash theme={null} npm install dotenv ``` ### Step 2: Add credentials to .env For local development, you can place your credentials in a `.env` file. For production, we recommend setting your credentials as environment variables wherever your application is deployed. Replace the placeholders below with the role credentials created in the previous section. ```bash .env theme={null} DATABASE_URL='postgresql://{username}:{password}@{host}:{port}/{database}?sslmode=verify-full' ``` Choose the appropriate **port** for your use case. Learn more about [Direct vs PgBouncer connections](/docs/postgres/connecting/quickstart#connection-types%3A-direct-vs-pgbouncer). > **PgBouncer** > > Port `6432` enables a lightweight connection pooler for PostgreSQL. This facilitates better performance when there are many simultaneous connections. > **Direct** > > Port `5432` connects directly to PostgreSQL. Total connections are limited by your cluster's `max_connections` setting. > [!NOTE] > > Both connection types will disconnect when your database restarts or handles a failover scenario. ### Step 3: Create a database connection To run a query, establish a connection using the `DATABASE_URL` from your `.env` file, then use the `query` function. Use the direct port (`5432`) for schema changes such as `CREATE TABLE`: ```javascript index.js theme={null} require('dotenv').config() const { Client } = require('pg') // The connection string carries the host, credentials, and database. The `ssl` // option verifies PlanetScale's certificate against your system's CA store, // equivalent to sslmode=verify-full. const client = new Client({ connectionString: process.env.DATABASE_URL, ssl: { rejectUnauthorized: true }, }) async function main() { await client.connect() console.log('Connected to PostgreSQL!') // Read smoke test — works with any role that has pg_read_all_data. const now = await client.query('SELECT NOW()') console.log(now.rows[0]) // Write/DDL smoke test — creates a table in the `public` schema of the // `postgres` database. This requires CREATE on the schema (see the note below). await client.query('CREATE TABLE IF NOT EXISTS smoke_test (id serial PRIMARY KEY, created_at timestamptz DEFAULT now())') await client.query('INSERT INTO smoke_test DEFAULT VALUES') const rows = await client.query('SELECT count(*) FROM smoke_test') console.log(`smoke_test rows: ${rows.rows[0].count}`) await client.end() } main().catch((err) => { console.error('Database error', err) process.exit(1) }) ``` From the command line, run the following command to execute the script: ```bash Terminal theme={null} node index.js ``` > [!WARNING] > > The `CREATE TABLE` smoke test runs DDL in the `public` schema. A least-privilege > role (`pg_read_all_data`/`pg_write_all_data` only) will fail with > `permission denied for schema public` — see [Managing > roles](/docs/postgres/connecting/roles#user-defined-roles) for the fix. See the [node-postgres documentation](https://node-postgres.com/) for more information. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres with Prisma Source: https://planetscale.com/docs/postgres/tutorials/planetscale-postgres-prisma Create a new Postgres database and integrate it with Prisma. Prisma ORM allows you to write type-safe SQL queries and automatically generate migrations. Already created a PlanetScale Postgres database? [Jump straight to integration instructions](#integrate-with-prisma). We'll cover: * Creating a new Postgres database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Postgres clusters use real Postgres in a [high-availability architecture with one primary and two replicas](/docs/postgres/postgres-architecture/#cluster-design). ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Postgres** to create a PostgreSQL database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > This "name" is referenced in the PlanetScale Dashboard and APIs and not created as a logical database inside of Postgres. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred cluster size and [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Postgres database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You’ll be taken to a screen in the browser where you’ll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Postgres database by running: ```bash theme={null} pscale database create --region --engine postgres ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Postgres database cluster, PlanetScale automatically: * Provisions a PostgreSQL cluster in your selected region * Creates the initial `main` branch * Prepopulates Postgres with required default databases * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Create credentials and connect > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). In this section you'll create the "Default role" in your PlanetScale dashboard to create connection credentials for your database branch. > [!NOTE] > > The "Default role" is meant purely for administrative purposes. You can only create one and it has significant privileges for your database cluster and you should treat these credentials carefully. After completing this quickstart, it is *strongly recommended* that you [create another role](/docs/postgres/connecting/roles) for your application use-cases. **Dashboard** ![Database dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/new-database.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=0dc877c9af2d9be252c165c0c16274b8) **Step 1** Navigate to your database in the [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Click on the **"Connect"** button in the top right **Step 3** Select **"Default role"** **Step 4** Click **"Create default role"**. A new default role is created for your database branch. **Step 5** Record the "Host", "Username", and "Password" for the "Default role" someplace secure. ![Create a new role](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/create-role.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=dd760b8bac6d56d7ea96ef3bb981c666) **Step 6** You can generate connection strings under **"How are you connecting?"** for major languages, frameworks, and tools. ![Connection strings](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/langs-and-frames.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=709ecbea3be475aeadf8c19fae75dc87) Your connection details will include: * **Host**: the DNS name of your database endpoint * **Username**: automatically formatted for routing to the correct `branch` * **Password**: A securely generated password * **Database**: `postgres` (default database) * **Port**: `5432` (standard PostgreSQL port) or `6432` (for using [PgBouncer](/docs/postgres/connecting/pgbouncer)) **CLI** Create a new "Default role" in your PlanetScale CLI to create connection credentials for your database branch. **Step 1** Run the following command to create the "Default role" for your database branch. ```bash theme={null} pscale role reset-default ``` **Step 2** Record the "Host", "Username", and "Password" for the "Default role" somewhere secure. > [!NOTE] > > Passwords are shown only once. If you lose your record of the password, you must [reset the password](/docs/postgres/connecting/roles). ## Integrate with Prisma Installation instructions vary depending on your choice of JavaScript runtime or framework. The following are general instructions for installing Prisma and connecting to your database. ### Step 1: Install packages and initialize Run the following to install `prisma`: ```bash Terminal theme={null} npm install prisma ``` Next, set up Prisma in your application with the following command: ```bash Terminal theme={null} npx prisma init ``` ### Step 2: Add credentials to .env For local development, you can place your credentials in a `.env` file. For production, we recommend setting your credentials as environment variables wherever your application is deployed. Replace the placeholders below with the role credentials created in the previous section. ```bash .env theme={null} DATABASE_URL='postgresql://{username}:{password}@{host}:{port}/{database}?sslmode=verify-full' ``` Choose the appropriate **port** for your use case. Learn more about [Direct vs PgBouncer connections](/docs/postgres/connecting/quickstart#connection-types%3A-direct-vs-pgbouncer). > **PgBouncer** > > Port `6432` enables a lightweight connection pooler for PostgreSQL. This facilitates better performance when there are many simultaneous connections. > **Direct** > > Port `5432` connects directly to PostgreSQL. Total connections are limited by your cluster's `max_connections` setting. > [!NOTE] > > Both connection types will disconnect when your database restarts or handles a failover scenario. ### Step 3: Create a database connection By default, Prisma should create a `prisma/schema.prisma` file with the `postgresql` provider. ```prisma prisma/schema.prisma theme={null} generator client { provider = "prisma-client" output = "../generated/prisma" } datasource db { provider = "postgresql" } ``` As well as a `prisma.config.ts` file which reads `DATABASE_URL` from your `.env` file. ```ts prisma.config.ts theme={null} import 'dotenv/config' import { defineConfig, env } from 'prisma/config' export default defineConfig({ schema: 'prisma/schema.prisma', migrations: { path: 'prisma/migrations', }, datasource: { url: env('DATABASE_URL'), }, }) ``` [Learn how to query your replicas directly](/docs/postgres/scaling/replicas) For more information on using Prisma including creating schemas and performing queries, [refer to their documentation](https://www.prisma.io/orm). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres quickstart Source: https://planetscale.com/docs/postgres/tutorials/planetscale-postgres-quickstart This guide will walk you through how to create a new PlanetScale Postgres database. We'll cover: * Creating a new Postgres database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Postgres clusters use real Postgres in a [high-availability architecture with one primary and two replicas](/docs/postgres/postgres-architecture/#cluster-design). ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Postgres** to create a PostgreSQL database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > This "name" is referenced in the PlanetScale Dashboard and APIs and not created as a logical database inside of Postgres. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred cluster size and [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Postgres database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You’ll be taken to a screen in the browser where you’ll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Postgres database by running: ```bash theme={null} pscale database create --region --engine postgres ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Postgres database cluster, PlanetScale automatically: * Provisions a PostgreSQL cluster in your selected region * Creates the initial `main` branch * Prepopulates Postgres with required default databases * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Create credentials and connect > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). In this section you'll create the "Default role" in your PlanetScale dashboard to create connection credentials for your database branch. > [!NOTE] > > The "Default role" is meant purely for administrative purposes. You can only create one and it has significant privileges for your database cluster and you should treat these credentials carefully. After completing this quickstart, it is *strongly recommended* that you [create another role](/docs/postgres/connecting/roles) for your application use-cases. **Dashboard** ![Database dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/new-database.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=0dc877c9af2d9be252c165c0c16274b8) **Step 1** Navigate to your database in the [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Click on the **"Connect"** button in the top right **Step 3** Select **"Default role"** **Step 4** Click **"Create default role"**. A new default role is created for your database branch. **Step 5** Record the "Host", "Username", and "Password" for the "Default role" someplace secure. ![Create a new role](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/create-role.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=dd760b8bac6d56d7ea96ef3bb981c666) **Step 6** You can generate connection strings under **"How are you connecting?"** for major languages, frameworks, and tools. ![Connection strings](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/langs-and-frames.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=709ecbea3be475aeadf8c19fae75dc87) Your connection details will include: * **Host**: the DNS name of your database endpoint * **Username**: automatically formatted for routing to the correct `branch` * **Password**: A securely generated password * **Database**: `postgres` (default database) * **Port**: `5432` (standard PostgreSQL port) or `6432` (for using [PgBouncer](/docs/postgres/connecting/pgbouncer)) **CLI** Create a new "Default role" in your PlanetScale CLI to create connection credentials for your database branch. **Step 1** Run the following command to create the "Default role" for your database branch. ```bash theme={null} pscale role reset-default ``` **Step 2** Record the "Host", "Username", and "Password" for the "Default role" somewhere secure. > [!NOTE] > > Passwords are shown only once. If you lose your record of the password, you must [reset the password](/docs/postgres/connecting/roles). ### Connection strings PlanetScale provides connection strings in various formats for different frameworks and languages. Here are some common examples: > **General PostgreSQL URL** > > ```bash theme={null} > postgresql://postgres.{branch-id}:{password}@{host}.horizon.psdb.cloud:5432/postgres?sslmode=verify-full > ``` > **psql command line** > > ```bash terminal theme={null} > psql 'host={host} port=5432 user={user} password={password} dbname=postgres sslnegotiation=direct sslmode=verify-full sslrootcert=system' > ``` > **Node.js (node-postgres)** > > ```typescript db.ts theme={null} > const { Client } = require('pg'); > const client = new Client({ > host: '{host}', > port: 5432, > user: '{user}', > password: '{your-password}', > database: 'postgres', > ssl: { rejectUnauthorized: true } > }); > ``` > **Rails** > > ```ruby theme={null} > planetscale: > username: {user} > host: {host} > port: 5432 > database: postgres > password: {your-password} > ``` ## Default databases When your database branch is first created, there are a number of default databases that are created at the same time. ```sql theme={null} postgres=> \l List of databases Name | Owner | Removed columns... ------------------+------------------+------------------- postgres | postgres | ... pscale_admin | pscale_admin | ... pscale_exporter | pscale_admin | ... pscale_pgbouncer | pscale_admin | ... template0 | pscale_admin | ... template1 | pscale_superuser | ... (6 rows) ``` These databases include those that are used by various PlanetScale platform features such as metrics and logs collection, backups, Insights, or are used by PGBouncer (as examples). They cannot be removed. | Database | Purpose | | - | - | | postgres | Default user database | | pscale\_admin | PlanetScale platform | | pscale\_exporter | PlanetScale platform | | pscale\_pgbouncer | PlanetScale platform | | `template0` and `template1` | [Postgres database defaults](https://www.postgresql.org/docs/current/manage-ag-templatedbs.html) | ## Security requirements All connections to Postgres databases require: * **SSL/TLS encryption** - Use `sslmode=verify-full` (recommended, with `sslrootcert=system`); `sslmode=require` is the weaker minimum * **Certificate verification** - Connections verify PlanetScale's SSL certificates * **Secure passwords** - Generated passwords use cryptographically secure random generation ### Password management * **Password reset**: You can [reset your default user password](/docs/postgres/connecting/roles) anytime from the dashboard * **No password rotation required**: Passwords don't expire unless you set them to * **Single credential per branch**: Each branch has one default user credential ## Next steps Your database is ready. See [Next steps](/docs/postgres/day-one-with-planetscale-postgres) for the top platform features to enable before going to production. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Postgres with Ruby on Rails Source: https://planetscale.com/docs/postgres/tutorials/planetscale-postgres-rails Create a new Postgres database and integrate it with Ruby on Rails. Ruby on Rails is a popular web application framework that includes ActiveRecord, a powerful ORM for working with databases. Already created a PlanetScale Postgres database? [Jump straight to integration instructions](#integrate-with-rails). We'll cover: * Creating a new Postgres database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/postgres/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Postgres clusters use real Postgres in a [high-availability architecture with one primary and two replicas](/docs/postgres/postgres-architecture/#cluster-design). ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Postgres** to create a PostgreSQL database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > This "name" is referenced in the PlanetScale Dashboard and APIs and not created as a logical database inside of Postgres. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred cluster size and [CPU architecture](/docs/postgres/cluster-configuration/cpu-architectures) ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Postgres database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You’ll be taken to a screen in the browser where you’ll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Postgres database by running: ```bash theme={null} pscale database create --region --engine postgres ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Postgres database cluster, PlanetScale automatically: * Provisions a PostgreSQL cluster in your selected region * Creates the initial `main` branch * Prepopulates Postgres with required default databases * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Create credentials and connect > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). In this section you'll create the "Default role" in your PlanetScale dashboard to create connection credentials for your database branch. > [!NOTE] > > The "Default role" is meant purely for administrative purposes. You can only create one and it has significant privileges for your database cluster and you should treat these credentials carefully. After completing this quickstart, it is *strongly recommended* that you [create another role](/docs/postgres/connecting/roles) for your application use-cases. **Dashboard** ![Database dashboard](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/new-database.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=0dc877c9af2d9be252c165c0c16274b8) **Step 1** Navigate to your database in the [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Click on the **"Connect"** button in the top right **Step 3** Select **"Default role"** **Step 4** Click **"Create default role"**. A new default role is created for your database branch. **Step 5** Record the "Host", "Username", and "Password" for the "Default role" someplace secure. ![Create a new role](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/create-role.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=dd760b8bac6d56d7ea96ef3bb981c666) **Step 6** You can generate connection strings under **"How are you connecting?"** for major languages, frameworks, and tools. ![Connection strings](https://mintcdn.com/planetscale-2/NAfHErQ6-kE8SaMw/postgres/tutorials/langs-and-frames.png?fit=max\&auto=format\&n=NAfHErQ6-kE8SaMw\&q=85\&s=709ecbea3be475aeadf8c19fae75dc87) Your connection details will include: * **Host**: the DNS name of your database endpoint * **Username**: automatically formatted for routing to the correct `branch` * **Password**: A securely generated password * **Database**: `postgres` (default database) * **Port**: `5432` (standard PostgreSQL port) or `6432` (for using [PgBouncer](/docs/postgres/connecting/pgbouncer)) **CLI** Create a new "Default role" in your PlanetScale CLI to create connection credentials for your database branch. **Step 1** Run the following command to create the "Default role" for your database branch. ```bash theme={null} pscale role reset-default ``` **Step 2** Record the "Host", "Username", and "Password" for the "Default role" somewhere secure. > [!NOTE] > > Passwords are shown only once. If you lose your record of the password, you must [reset the password](/docs/postgres/connecting/roles). ## Integrate with Rails ### Step 1: Install the pg gem Add the `pg` gem to your Gemfile: ```ruby Gemfile theme={null} gem "pg" ``` Then run: ```bash Terminal theme={null} bundle install ``` ### Step 2: Add credentials We recommend using [Rails encrypted credentials](https://edgeguides.rubyonrails.org/security.html#custom-credentials) to store your database connection details securely. ```bash Terminal theme={null} rails credentials:edit --environment production ``` Add the following, replacing the placeholders with the role credentials created in the previous section: ```yaml config/credentials/production.yml.enc theme={null} planetscale: username: host: port: database: password: ``` Choose the appropriate **port** for your use case. Learn more about [Direct vs PgBouncer connections](/docs/postgres/connecting/quickstart#connection-types%3A-direct-vs-pgbouncer). > **PgBouncer** > > Port `6432` enables a lightweight connection pooler for PostgreSQL. This facilitates better performance when there are many simultaneous connections. > **Direct** > > Port `5432` connects directly to PostgreSQL. Total connections are limited by your cluster's `max_connections` setting. > [!NOTE] > > Both connection types will disconnect when your database restarts or handles a failover scenario. ### Step 3: Configure database.yml Update your `config/database.yml` to use your PlanetScale credentials in production: ```yaml config/database.yml theme={null} production: <<: *default adapter: postgresql username: <%= Rails.application.credentials.planetscale&.fetch(:username) %> password: <%= Rails.application.credentials.planetscale&.fetch(:password) %> database: <%= Rails.application.credentials.planetscale&.fetch(:database) %> host: <%= Rails.application.credentials.planetscale&.fetch(:host) %> port: <%= Rails.application.credentials.planetscale&.fetch(:port) %> sslmode: verify-full ``` > [!NOTE] > > If deploying to Heroku, add `sslrootcert: /etc/ssl/certs/ca-certificates.crt` to your configuration. ### Step 4: Run migrations Run your migrations against PlanetScale to set up the schema: ```bash Terminal theme={null} RAILS_ENV=production rails db:migrate ``` Verify the connection by opening a Rails console: ```bash Terminal theme={null} RAILS_ENV=production rails console > User.first ``` ## Schema migrations Running schema migrations safely is critical in production environments. Here are our recommendations for Rails applications. ### Run migrations during deployment Schema migrations should run as part of your deployment process, before the new code is released. Most deployment platforms support running a release command: ```bash theme={null} rails db:migrate ``` This ensures your database schema is updated before your application code starts serving requests. ### Use strong\_migrations We recommend the [strong\_migrations](https://github.com/ankane/strong_migrations) gem to catch potentially dangerous migrations in development before they reach production. ```ruby Gemfile theme={null} gem "strong_migrations" ``` After installing, run the generator: ```bash Terminal theme={null} bundle install rails generate strong_migrations:install ``` The gem will warn you about migrations that could cause downtime or lock tables for extended periods, such as adding an index without `CONCURRENTLY` or changing column types on large tables. ### Separate schema and code changes For safer deployments, we recommend separating schema changes from code changes into different deployments: 1. **Schema-only deployment**: Contains only the migration file with no application code changes. Deploy this first. 2. **Code deployment**: Contains the application code that uses the new schema. Deploy after the schema change is live. This approach makes it easier to roll back if something goes wrong and reduces the risk of deploying code that depends on schema changes that haven't been applied yet. ### Dropping columns and tables When removing columns or tables, follow this process: 1. **Stop using the column**: Deploy code that no longer reads from or writes to the column. 2. **Verify with Insights**: Use [PlanetScale Insights](/docs/postgres/monitoring/query-insights) to confirm no queries are accessing the column. 3. **Drop the column**: In a separate deployment, add the migration to remove the column. This prevents errors from code trying to access columns that no longer exist. ## Connection management Understanding how many database connections your Rails application uses is essential to avoid exhausting your connection pool. ### Calculating connections with Puma If you're using Puma as your web server, the total number of database connections your application needs is: ``` Total connections = Puma workers × Puma threads ``` For example, with 4 workers and 5 threads per worker: ``` 4 workers × 5 threads = 20 connections ``` If you also have Sidekiq or other background job processors, add their connection counts: ``` Total = (Puma workers × threads) + (Sidekiq processes × Sidekiq concurrency) ``` ### Staying within limits Your total connection count must stay below your Postgres cluster's `max_connections` setting. Check your current limit in the PlanetScale dashboard under cluster configuration. Configure your Rails connection pool in `database.yml`: ```yaml config/database.yml theme={null} production: <<: *default pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %> # ... other settings ``` ### Use PgBouncer for scaling If your application needs more connections than `max_connections` allows, we recommend using [PgBouncer](/docs/postgres/connecting/pgbouncer). PlanetScale provides both a local PgBouncer (included with every database) and dedicated PgBouncers for higher connection counts. To use PgBouncer, change your port from `5432` to `6432`: ```yaml config/credentials/production.yml.enc theme={null} planetscale: port: 6432 # ... other settings ``` This allows many application connections to share a smaller pool of database connections. > [!WARNING] > > **Schema migrations must use direct connections.** Rails uses advisory locks during migrations, which require a persistent session. PgBouncer's transaction pooling mode doesn't support session-level advisory locks. Update your `database.yml` to use direct connections (port `5432`) for migrations while your application uses PgBouncer: ```yaml config/database.yml theme={null} production: <<: *default adapter: postgresql username: <%= Rails.application.credentials.planetscale&.fetch(:username) %> password: <%= Rails.application.credentials.planetscale&.fetch(:password) %> database: <%= Rails.application.credentials.planetscale&.fetch(:database) %> host: <%= Rails.application.credentials.planetscale&.fetch(:host) %> port: <%= ENV.fetch("DATABASE_PORT") { Rails.application.credentials.planetscale&.fetch(:port) } %> sslmode: verify-full ``` Then run migrations with a direct connection: ```bash Terminal theme={null} DATABASE_PORT=5432 RAILS_ENV=production rails db:migrate ``` ## Querying replicas PlanetScale Postgres includes replicas that you can query directly to reduce load on your primary. Rails 6+ has built-in support for multiple databases, making it straightforward to route read queries to replicas. ### Connection options for replicas There are two ways to connect to replicas: | Method | Port | Username suffix | Connection pooling | Replica routing | | :- | :- | :- | :- | :- | | Direct connection | `5432` | `\|replica` | No | Single replica | | Dedicated replica PgBouncer | `6432` | `\|pgbouncer-name` | Yes | Multiple replicas | **Direct connections** work well for all traffic types and are simpler to set up. The tradeoff is that each connection goes to a single replica rather than being distributed, and you're limited by `max_connections` since there's no connection pooling. **Dedicated replica PgBouncers** provide connection pooling and route queries across your replica nodes, allowing your application to scale beyond `max_connections`. This is the better choice when you have high read traffic or many application instances connecting to replicas. > [!NOTE] > > The local PgBouncer (port `6432`) does not support replica routing. To get connection pooling for replica traffic, create a [dedicated replica PgBouncer](/docs/postgres/connecting/pgbouncer#dedicated-replica-pgbouncers) in the PlanetScale dashboard. ### Step 1: Configure database.yml Add a replica configuration. The example below uses a direct connection with the `|replica` suffix: > [!NOTE] > > If you configure Rails with `DATABASE_URL` (URI format), URL-encode the pipe character in username suffixes. For example, use `%7Creplica` instead of `|replica`: > > ```text theme={null} > DATABASE_URL=postgresql://postgres.%7Creplica:pscale_pw_xxx@:5432/?sslmode=verify-full&sslrootcert=/etc/ssl/certs/ca-certificates.crt > ``` ```yaml config/database.yml theme={null} production: primary: <<: *default adapter: postgresql username: <%= Rails.application.credentials.planetscale&.fetch(:username) %> password: <%= Rails.application.credentials.planetscale&.fetch(:password) %> database: <%= Rails.application.credentials.planetscale&.fetch(:database) %> host: <%= Rails.application.credentials.planetscale&.fetch(:host) %> port: 5432 sslmode: verify-full primary_replica: <<: *default adapter: postgresql username: <%= Rails.application.credentials.planetscale&.fetch(:username) %>|replica password: <%= Rails.application.credentials.planetscale&.fetch(:password) %> database: <%= Rails.application.credentials.planetscale&.fetch(:database) %> host: <%= Rails.application.credentials.planetscale&.fetch(:host) %> port: 5432 sslmode: verify-full replica: true ``` > [!TIP] > > **Using a dedicated replica PgBouncer?** Replace `|replica` with your PgBouncer name (e.g., `|read-pool`) and change the port to `6432`. This gives you connection pooling for replica traffic, which is ideal for high-traffic read workloads. ### Step 2: Configure ApplicationRecord Update your `ApplicationRecord` to use multiple databases: ```ruby app/models/application_record.rb theme={null} class ApplicationRecord < ActiveRecord::Base primary_abstract_class connects_to database: { writing: :primary, reading: :primary_replica } end ``` ### Step 3: Query the replica Use `connected_to` to explicitly route queries to the replica: ```ruby theme={null} class UsersController < ApplicationController def index # This query runs against the replica @users = ActiveRecord::Base.connected_to(role: :reading) do User.where(active: true).order(:created_at) end end end ``` You can also enable [automatic connection switching](https://guides.rubyonrails.org/active_record_multiple_databases.html#activating-automatic-role-switching) in Rails to route GET and HEAD requests to the replica automatically. ### When to use replicas Replicas are useful for offloading read-heavy workloads from the primary. Some examples: * **Scheduled and background jobs**: Sidekiq jobs, cron tasks, or batch processing that read data. * **Analytics and reporting**: Aggregations, reports, or dashboards that run expensive queries. * **Search features**: Full-text search or filtering that scans large datasets. * **Read-only views**: Serving data to logged-out users or public pages where [replication lag](/docs/postgres/scaling/replicas#data-consistency-and-replication-lag) is acceptable. * **Debugging and one-off queries**: Running queries against production data without impacting the primary. ### Avoiding read-your-own-writes issues Due to replication lag, data written to the primary may not be immediately available on replicas. This can cause issues where a user creates a record and then can't see it. Rails automatic connection switching handles this by sticking to the primary for a configurable delay after writes. If you're manually routing queries, ensure you read from the primary immediately after writes: ```ruby theme={null} class PostsController < ApplicationController def create @post = Post.create!(post_params) # Read from primary to avoid stale data redirect_to @post end end ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Web console Source: https://planetscale.com/docs/postgres/web-console Interactive interface for running PostgreSQL queries and DDL against PlanetScale Postgres database branches. > [!NOTE] > > The web console is currently in beta for **Postgres** databases. ## Get started The PlanetScale web console can be used to query any database branch; however, it is [disabled for production branches](/docs/postgres/web-console#enable-for-production-branches) by default to protect production data. To access the web console, navigate to a database, and click on the "Console" tab in the page navigation. From here, you can select which branch you'd like to connect to by selecting it in the dropdown and clicking "Connect". Once you have accessed the web console, you can run queries against your database branch or apply DDL. A branch can contain multiple [logical databases](https://planetscale.com/blog/approaches-to-tenancy-in-postgres); use the **Database** dropdown in the console header to choose which one to connect to. The following are examples of PostgreSQL statements and meta commands you may find useful within the web console: Use `\l` to see a list of logical databases on the branch. Use `\dt` to see a list of tables in the current database. Use `\d table_name` to obtain information about a given table's structure. Use `EXPLAIN` or `EXPLAIN ANALYZE` in front of `SELECT`, `INSERT`, `UPDATE`, and `DELETE` statements to learn how the database is executing a query. This can be useful for optimizing slow queries. ## Logical databases After connecting to a branch, use the **Database** dropdown in the console header to choose which logical database to connect to. The default database name is `postgres`. You can also switch databases from the console prompt with `\connect database_name` or `\c database_name`, just like in `psql`. Run `\l` to list the available databases on the branch. Switching databases reconnects the console and clears the current output. ## Primary and replica connections After connecting to a branch, use the **Server** dropdown in the console header to choose which node to connect to: * **Primary** — Connect to the primary node for read and write queries. * **Replica** — Connect to a read replica for read-only queries. If a branch has no replicas, the console connects to the primary and the **Replica** option is unavailable. Switching between primary and replica reconnects the console and clears the current output. ### Connection roles The console creates a short-lived Postgres role for each connection. The inherited permissions depend on the selected server and your [organization or database role](/docs/security/access-control). Tables created during a web console session are reassigned to the `postgres` role shortly after the session ends. Until reassignment completes, newly created tables may temporarily appear owned by the console's short-lived role. #### Development branches All organization roles (member, analyst, administrator) have full read and write access to development branch consoles on the primary. On a replica, the console always uses a read-only role. #### Production branches Production branch console access also requires the [web console setting](#enable-for-production-branches) to be enabled. Once enabled, access depends on your role: | Role | Console access | Primary permissions | | :- | :- | :- | | Organization or database **Administrator** | Read and write | `pg_read_all_data`, `pg_write_all_data`, `postgres` | | Organization or database **Analyst** | Read-only | `pg_read_all_data` only | | **Member** | No access | — | On a **replica**, the console always uses a read-only role with `pg_read_all_data`, regardless of your primary permissions. For more information about these permissions, see [Role management](/docs/postgres/connecting/roles). ## Supported console commands | Command | Description | | :- | :- | | ?, \\? | Synonym for `help` | | clear | Clear the current input statement | | help, \h | Display list of commands | | ; | Send SQL statement to server | | \connect \[database], \c \[database] | Connect to a different logical database | | \d \[name] | List relations or describe a table | | \dt \[pattern] | List tables | | \dv \[pattern] | List views | | \di \[pattern] | List indexes | | \ds \[pattern] | List sequences | | \da \[pattern] | List aggregate functions | | \db \[pattern] | List tablespaces | | \dc \[pattern] | List casts | | \dconfig \[pattern] | List configuration parameters | | \dD \[pattern] | List domains | | \det \[pattern] | List foreign tables | | \dn \[pattern] | List schemas | | \do \[pattern] | List operators | | \dO \[pattern] | List collations | | \dp, \z \[pattern] | List access privileges | | \dT \[pattern] | List data types | | \du \[pattern] | List roles | | \l \[pattern] | List databases | | \df \[pattern] | List functions | | \dx \[pattern] | List extensions | ## Enable for production branches By default, the web console is disabled for production branches to protect production data. Database admins can enable the web console for production branches on the "Settings" page for the given database, `app.planetscale.com///settings`. Select the checkbox for "Allow web console access to production branches", then scroll down and click the "Save database settings" button to save your changes. This will enable the ability to use the web console to run queries against production branches for the given database. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Security and compliance Source: https://planetscale.com/docs/security PlanetScale is committed to delivering a powerful and easy-to-use database platform while keeping your data secure. Vitess and Postgres The security of our systems is of the utmost importance. We consistently aim to improve our security posture by building security into every layer of our products. Below is a breakdown of common security and compliance requirements by PlanetScale plan: | | Base plan | Enterprise multi-tenant | Enterprise single-tenant | PlanetScale Managed | | :- | :- | :- | :- | :- | | [Encryption of data (at rest and in transit)](#encryption-of-data) | | | | | | [SOC 1 Type 2 available](#soc-1-type-2-&-soc-2-type-2+-hipaa) | | | | | | [SOC 2 Type 2+ HIPAA available](#soc-1-type-2-&-soc-2-type-2+-hipaa) | | | | | | [IP restrictions](/docs/vitess/connecting/connection-strings#ip-restrictions) (Vitess only) | | | | | | [Audit logs](/docs/security/audit-log) | | | | | | [Security logs](/docs/security/security-log) | | | | | | [Data Processing Addendum available](#data-processing-addendum) | | | | | | [Private database connectivity](#private-database-connectivity) | | | | | | [Single sign-on (SSO)](/docs/security/sso) | Available as add-on | | | | | [Business Associate Agreements available](#hipaa-and-business-associate-agreements) | | | | | | Dedicated AWS/GCP/Azure account | | | | | | [PCI compliant](#pci-compliance) | | | | | | [Your own AWS/GCP/Azure account](#deploy-in-your-own-aws-gcp-or-azure-account) | | | | | ## Available on all PlanetScale plans ### Private database connectivity By default, all PlanetScale connections are encrypted and routed through the public Internet. Optionally, you can connect privately to databases through [AWS PrivateLink](/docs/vitess/connecting/private-connections) or [GCP Private Service Connect](/docs/vitess/connecting/private-connections-gcp). ### SOC 1 Type 2 & SOC 2 Type 2+ HIPAA PlanetScale continuously monitors and reports primarily using System and Organization Controls (SOC) 1 & 2 Type 2 paired with the HIPAA Security Rule. To request access to our latest reports, please visit PlanetScale's [Trust Center](https://trust.planetscale.com/). ### HIPAA and Business Associate Agreements PlanetScale can enter into Business Associate Agreements (BAAs) with customers who meet qualifying criteria. To request a BAA: 1. Sign into the PlanetScale dashboard 2. Click "Settings" from your Organization page > "Legal" > "Create request" 3. Fill out the form 4. We will review your request and let you know if it's approved 5. Once approved, you'll receive a Docusign sent to the email you specified as the signer The customer must determine whether they are a Covered Entity — or a Business Associate of a Covered Entity — as defined under HIPAA. If so, the customer may require a BAA with PlanetScale for the purposes of our relationship. Responsibility around HIPAA compliance between PlanetScale and the customer is implemented using a shared responsibility model. While PlanetScale plans provide a secure and compliant infrastructure for the storage and processing of Protected Health Information (PHI), the customer is ultimately responsible for ensuring that the environment and applications that they build on top of PlanetScale are properly configured and secured according to HIPAA requirements. The Department of Health and Human Services does not recognize any formal certification for HIPAA. PlanetScale systems, software, networks, and procedures are consistent with the controls outlined in the relevant rules. ### Data security #### Encryption of data PlanetScale databases and their client communications are AES encrypted throughout the PlanetScale platform, both in transit and at rest. ##### At rest Data is encrypted at rest on the underlying storage media that serves database branches and also the underlying storage media that hosts your PlanetScale database backups. This helps mitigate the risk of unintentional or malicious access to user data on storage systems. ##### In transit Data in transit to PlanetScale databases is encrypted and goes through three major paths: * The [PlanetScale CLI](/docs/cli), leverages [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) when initiating a connection to PlanetScale's API and Edge. * PlanetScale [connection strings](/docs/vitess/connecting/connection-strings) require the successful establishment of a TLS session before any SQL commands can be issued. * [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) is used to secure all data transmitted between PlanetScale and [clients using PlanetScale Connect](/docs/vitess/etl). All public PlanetScale endpoints use post-quantum key exchange with clients that support it. #### Additional data protection controls Communications to the PlanetScale API and Dashboard are encrypted using TLS 1.3 with support for post-quantum key exchange. Certificates are issued by established third-party certificate authorities. ### General Data Protection Regulation (GDPR) PlanetScale offers database services in Amazon Web Services, Google Cloud Platform, and Microsoft Azure regions around the world. PlanetScale complies with the EU General Data Protection Regulation (GDPR) and other global privacy regulations, where applicable. Customers are responsible for their applications' compliance with regulatory requirements, including as they relate to data subjects of their systems. #### Data Processing Addendum All PlanetScale plans are covered by our [Data Processing Addendum (DPA)](https://planetscale.com/legal/data-processing-addendum). Markups are accepted for addendums on all PlanetScale Enterprise plans. [Contact us](https://planetscale.com/contact) to talk more about PlanetScale Enterprise plans and changes to our DPA. #### Data locality The infrastructure supporting user databases, backups, etc., is in the provider (AWS, GCP, or Azure) and region where the database is created. Any read-only replicas in other geographies will copy the data set to the selected regions. The following are two examples of data locality in PlanetScale: * If you create a database in a US-based region, all data, including customer data, is stored and processed in the US, except in cases where sub-processors are identified as having other locations. * If you create a database in a Europe-based region, your data does not leave the region the database was created in, unless you create a read-only region in another region. ## Available on all Enterprise plans ### Additional audit logging features In addition to the [audit log](/docs/security/audit-log) feature available to all PlanetScale plans, Enterprise plans can use our EventBridge configuration to send logs to your AWS account. Ask your PlanetScale account manager for more information on how to set it up. > [!NOTE] > > If you have any questions or concerns related to the security and compliance of any PlanetScale Enterprise plans, please [contact us](https://planetscale.com/contact), and we will be happy to discuss them further. ## Available on Enterprise single-tenant plans PlanetScale offers two single-tenant deployment options: Single-tenant and PlanetScale Managed for organizations that require a single-tenant environment. See the [section below](/docs/#available-on-planetscale-managed) for more information on PlanetScale Managed-only security and compliance features. > [!NOTE] > > [Contact us](https://planetscale.com/contact) if you are interested in exploring PlanetScale single-tenant deployment options for your organization. ### Dedicated AWS Organizations member account, GCP organization, or Azure subscription PlanetScale single-tenant deployment options are deployed into a dedicated AWS Organizations member account, GCP organization, or Azure subscription owned by PlanetScale. If you want PlanetScale to deploy into your own AWS Organizations member account, GCP organization, or Azure subscription, owned by your organization, see PlanetScale Managed below. ## Available on PlanetScale Managed PlanetScale Managed is a single-tenant deployment of PlanetScale within your Amazon Web Services (AWS), Google Cloud Platform (GCP), or Microsoft Azure account. In this configuration, you can use the same API, CLI, and web interface that PlanetScale offers, with the benefit of running entirely in your own AWS Organizations member account, GCP organization, or Azure subscription. You can learn more on the [PlanetScale Managed overview page](/docs/plans/managed). ### Deploy in your own AWS, GCP, or Azure account PlanetScale Managed is a packaged [data plane](https://en.wikipedia.org/wiki/Data_plane) that's deployed to an AWS Organizations member account, GCP project, or Azure subscription that you own and we operate. Your database lives entirely inside a dedicated account, project, or subscription within your cloud organization. PlanetScale will not have access to other accounts, projects, or subscriptions nor your organization-level settings within the cloud service provider. Read more about PlanetScale Managed on each cloud provider: * [PlanetScale Managed on Amazon Web Services](/docs/plans/managed/aws) * [PlanetScale Managed on Google Cloud Platform](/docs/plans/managed/gcp) * [PlanetScale Managed on Microsoft Azure](/docs/plans/managed/azure) ### PCI compliance PlanetScale Managed, when deployed with the appropriate controls enabled via our Shared Responsibility Matrix, has been issued an Attestation of Compliance (AoC) and Report on Compliance (RoC), certifying our compliance with the PCI DSS 4.0 as a Level 1 Service Provider. This enables PlanetScale Managed to be used via a shared responsibility model across merchants, acquirers, issuers, and other roles in storing and processing cardholder data. ### Human access flow PlanetScale Managed supports customer counter-approval for access to Managed environment(s) by PlanetScale employees via an integration in the web application. Talk to your Customer Engineer about enabling this feature. ### Other PlanetScale Managed security features PlanetScale Managed on AWS also supports: * [Fully private network isolation](/docs/plans/managed/aws#fully-private-network-isolation) * [Third-account customer-controlled public key infrastructure (PKI)](/docs/plans/managed/aws#third-account-customer-controlled-public-key-infrastructure) ## Corporate security ### Background checks Background checks are performed on new team members during onboarding (within 30 days of their start date) as permitted by local law. ### Security training All team members complete security awareness training covering company security policies and procedures during onboarding (within 30 days of their hire date). Trainings are required annually after that for all employees. The training material is designed to assist the employee in identifying and responding to social engineering and other cybersecurity risks they may encounter as part of their role at PlanetScale. ## Security operations ### Endpoints All company-provided devices are managed with [Kandji](https://kandji.io/). Device configuration is based on the Center for Internet Security (CIS) level 1 benchmark and is continuously enforced. Mobile device management deploys and enables relevant services to ensure corporate endpoints are appropriately monitored. ### Extended Detection and Response (XDR) PlanetScale administrative endpoints, such as devices used by employees, are monitored via endpoint detection and response systems. ### Detection and response PlanetScale employs measures for both corporate endpoints and cloud instances to collect, analyze, and store events, connections, and other potentially relevant metadata in real-time. Automated systems match events against internal and known bad patterns and intelligence streams. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Access control Source: https://planetscale.com/docs/security/access-control Vitess and Postgres ## Organization access control When you set up your PlanetScale account, you're asked to create an **Organization**. An organization is essentially a container for your databases, settings, and members. You can create multiple organizations in the same account for different applications or use cases. Within each organization, you can add members and assign them different roles. This document covers the different roles, the ways you can assign roles, permissions associated with those roles. ## Roles and permissions We currently support four different roles in your organization: * `Organization Administrator` * `Organization Analyst` * `Organization Member` * `Database Administrator` ### Organization Administrator An `Organization Administrator` can perform all actions in an organization, as well as all actions on *every* database within that organization. ### Organization Analyst An `Organization Analyst` can perform read-only operations on production database branches. This role is designed for team members who need to view production data for analysis and reporting without full administrative access. Analysts can create and manage read-only credentials for production branches. ### Organization Member An `Organization Member` can only perform limited actions within an organization and on all databases in that organization. By default, all users added to an organization have this role. ### Database Administrator A `Database Administrator` can perform all actions on the database for which they were assigned the `Databases Administrator` role. This role is assigned at the **database level** and gives elevated permissions for the particular database that an organization member is the `Database Administrator` of. If you want to [grant a member *full* access to manage one or several databases](#assign-roles-at-a-database-level) but not full `Organization Administrator` access, then this is the role you want. Please note, a user that is granted this role must be a member of the organization of which the database exists in, so they will have the permissions associated with `Organization Member` as well. ## Organization-level permissions Each role has a set of permissions assigned to it, which determines what actions that role is allowed to take within an organization or database. The following table describes permissions assigned at the organization level for `Organization Administrators`, `Organization Analysts`, and `Organization Members`. Because `Database Administrators` don't have any organization-level permissions, they are not included in this table. | Action | Description | Member | Analyst | Administrator | | - | - | - | - | - | | View branches | View a database branch | | | | | Create branches | Create a database branch | | | | | Delete non-production branches | Delete a non-production database branch | | | | | View databases | View one or all databases | | | | | Create databases | Create a new database | | | | | Create deploy requests | Create a deploy request for a branch | | | | | Manage service tokens | Create, view, or delete service tokens | | | | | Manage service token grants | Create, view, update, or delete service token grants | | | | | View organization members | View one or all organization members | | | | | View database members | View one or all database members | | | | | View organization | View an organization | | | | | View query statistics | View query statistics for an organization's databases | | | | | Connect to development branches | Create passwords or use pscale shell for development branches | | | | | Connect to production branches (read-only) | Create read-only passwords or use pscale shell for production branches | | | | | Connect to production branches | Create passwords or use pscale shell for production branches | | | | | Delete production branches | Delete a production database branch | | | | | Promote branches | Promote a branch to production | | | | | Modify VSchema (Vitess only) | Edit the VSchema of a keyspace | | | | | Manage databases | Delete, update settings, or import a database | | | | | Manage beta features | Opt-in or opt-out of a beta feature | | | | | Create production service token grants | Create a service token grant to connect or delete a production database branch | | | | | Update an integration | Update a third-party integration | | | | | Manage invitations | View, create, or cancel organization invitations | | | | | Manage invoices | View or download organization invoices | | | | | Manage billing | View or update billing plans and payment methods | | | | | View audit logs | View all audit logs | | | | | Manage organization members | Update member roles or delete organization members | | | | | Manage database members | Update member roles, add, or remove database members | | | | | Manage organization | Update organization settings, SSO, or delete organization | | | | ## Database-level permissions The following table describes the permissions assigned at the **database level** for `Organization Administrators`, `Organization Analysts`, `Organization Members`, and `Database Administrators`. For `Organization Administrators`, `Organization Analysts`, and `Organization Members`, these permissions apply to every database in the organization. Because the `Database Administrator` role is assigned at the database level, the permissions are for the specific database(s) for which they have the `Database Administrator` role. | Action | Description | Member | Analyst | Administrator | | - | - | - | - | - | | Create and view branches | Create or view a database branch | | | | | Delete non-production branches | Delete a non-production branch of a specific database | | | | | View database | View a database in an organization | | | | | Create deploy requests | Create a deploy request for a branch on a specific database | | | | | View database members | View one or all database members | | | | | View query statistics | View query statistics for an organization's databases | | | | | Restore non-production backups | Restore the backup of a development branch | | | | | Connect to development branches | Create passwords or use pscale shell for development branches | | | | | Connect to production branches (read-only) | Create read-only passwords or use pscale shell for production branches | | | | | Connect to production branches | Create passwords or use pscale shell for production branches | | | | | Manage billing | Update the billing plan of a specific database | | | | | Delete production branches | Delete a production database branch of a specific database | | | | | Promote branches | Promote a branch of a specific database to production | | | | | Manage database | Delete, update settings, or import a database | | | | | Manage beta features | Opt-in or opt-out of a beta feature for a database | | | | | Manage database members | Update database member roles, add, or remove database members | | | | | Restore production backups | Restore the backup of a production branch | | | | An organization may have several databases, and an `Organization Member` may have different access to each database depending on whether or not they also have the `Database Administrator` role. ## Assign organization roles to members You can follow the steps below to assign roles to your members. You must be an Organization Administrator to modify member roles. * In the [PlanetScale dashboard](https://app.planetscale.com), click on the Settings tab in the top navigation. * Click on "Members" in the sidebar on the left. * From here, you can click on the dropdown on the right under the "Role" column to select the role you want to apply to each member. You can also invite new members to your organization and choose their role before sending the invitation. If you don't pick a role, they'll join as an [`Organization Member`](#organization-member) by default. > [!NOTE] > > Member role management is issued at the organization level. Each organization in your account may have different > members with different access levels. ## Assign roles at a database level There are two ways to assign database-level roles to Organization members: 1. Individually using the `Database Administrator` role. 2. Creating a Team, adding member(s), and adding database(s) to that team. ### Individually assign the `Database Administrator` role To assign a member the role of `Database Administrator`, follow the steps outlined below. You must be an Organization Administrator or an existing Database Administrator to manage the `Database Administrator` role. > [!NOTE] > > Members that create a database are automatically assigned the role of `Database Administrator` for that database. **Step 1** In the [PlanetScale dashboard](https://app.planetscale.com), click on the name of the database you want to add a Database Administrator to. **Step 2** Click on the "**Settings**" tab in the top navigation. **Step 3** Click on "**Administrators**" in the sidebar on the left. **Step 4** To add an administrator, click on the "**Add administrator**" button and select the member you wish to add as a Database Administrator. **Step 5** From here, you can also remove a Database Administrator by clicking the "**Remove**" button next to their name. ### Add Database Administrator role via Teams If you wish to give several members the Database Administrator role, you may want to [create a Team](/docs/security/teams#create-and-manage-teams). This will allow you to manage the access to that database all in one place. For instructions, see our [Teams documentation](/docs/security/teams). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Account password security Source: https://planetscale.com/docs/security/account-password-security In addition to best practices like [multi-factor authentication](/docs/security/multi-factor-authentication), PlanetScale securely stores your account passwords and validates passwords against known security breaches. ## Password storage PlanetScale uses [Argon2](https://en.wikipedia.org/wiki/Argon2) as the password hashing function. We use the `Argon2id` variant, which provides protection against side channel attacks and GPU-based cracking attacks. A password hashing function is a one-way function which means that your password cannot be reversed or decrypted from the stored value in the database. ## Leaked passwords PlanetScale checks passwords when a user sets them during signup or when changing the password. The first check is that the password needs to have enough entropy. Entropy is a measure for the amount of randomness a password contains. Read more about how we use entropy for [user-friendly strong passwords in the PlanetScale blog](https://planetscale.com/blog/using-entropy-for-user-friendly-strong-passwords). PlanetScale also checks the password against [Have I been pwned](https://haveibeenpwned.com). *Have I been pwned* is a large database containing passwords seen in security breaches. PlanetScale does **not** send the password you enter to *Have I been pwned*. The *Have I been pwned* API provides anonymity through [the Cloudflare k-anonymity implementation](https://blog.cloudflare.com/validating-leaked-passwords-with-k-anonymity/). This ensures that no other provider can identify the password that you have entered. The password is also not associated in any way with the email address you use to sign up. This information is not shared with *Have I been pwned*, nor is this information needed for the leaked passwords API they provide. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Audit log Source: https://planetscale.com/docs/security/audit-log The organization audit log grants [Organization Administrators](/docs/security/access-control#organization-administrator) access to review **actions** performed by individual members of the organization. Vitess and Postgres In addition, each audit log includes **events** detailing who performed the **action** and when it happened. Audit log retainment period is [based on your plan](/docs/planetscale-plans): * **Base plan** — 15 days * **Enterprise** — 15 days > [!NOTE] > > Organization audit log access is limited to [Organization Administrators](/docs/security/access-control#organization-administrator). ## Review your organization audit log You can review your organization audit log under [your PlanetScale **organization** settings](https://app.planetscale.com/~/settings/audit-log). Once there, you can filter the audit log by **Actor** and/or **Action**. Clicking and expanding individual log **event** names empowers you to investigate additional metadata that can provide broader context around what changed. ## Audited organization events You can track the following organization **events** in your PlanetScale account: | PlanetScale organization events | Actions | Database Engine | | :- | :- | :- | | access\_token | created deleted token\_leaked | Vitess, Postgres | | backup\_policy | created deleted | Vitess | | branch\_maintenance\_schedule | created deleted started\_maintenance\_window finished\_maintenance\_window | Vitess | | branch\_maintenance\_window | created deleted | Vitess | | database | created deleted requested\_deletion updated\_default\_branch updated\_migration\_table\_name removed\_member added\_member | Vitess, Postgres | | database\_branch | created deleted enabled\_safe\_migrations disabled\_safe\_migrations enabled\_foreign\_keys disabled\_foreign\_keys enabled\_vectors updated\_cluster\_size updated\_cluster\_config updated\_vtgates updated\_cluster\_update\_strategy | Vitess, Postgres | | database\_branch\_keyspace | created deleted updated requested\_deletion | Vitess | | database\_branch\_password | created deleted password\_leaked updated\_ip\_restrictions | Vitess, Postgres | | database\_branch\_read\_only\_region | created deleted | Vitess | | database\_deploy\_request | created deleted closed | Vitess | | database\_webhook | created deleted | Vitess, Postgres | | deploy\_request\_review | approved | Vitess | | deployment | unqueued | Vitess | | external\_datasource | created deleted | Vitess, Postgres | | integration | created deleted | Vitess, Postgres | | organization | created deleted joined removed\_member left added\_member enabled\_sso disabled\_sso enabled\_sso\_directory disabled\_sso\_directory | Vitess, Postgres | | organization\_invitation | created deleted | Vitess, Postgres | | organization\_membership | created updated\_role | Vitess, Postgres | | organization\_team | created deleted added\_member left removed\_member | Vitess, Postgres | | postgres\_role | created deleted | PostgreSQL | | regional\_failover\_event | created deleted | Vitess, Postgres | | service\_token | created deleted updated\_bulk\_database\_access token\_leaked | Vitess, Postgres | | user | created deleted signed\_in | Vitess, Postgres | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Authentication methods Source: https://planetscale.com/docs/security/authentication-methods There are three ways to authenticate with PlanetScale: _email address and password_, _single sign-on_, and _OAuth via GitHub_. ## Overview Let's break down how each of these work. ## Email address and password This is the only authentication mechanism where PlanetScale maintains user credentials. Additionally, users can opt to configure [two-factor authentication (2FA)](/docs/security/multi-factor-authentication). This option requires **something you know** *(i.e. your password)* and **something you have** *(i.e. recovery codes)*. ## Single sign-on Users can authenticate with their chosen corporate identity provider *(i.e. Okta)* instead of maintaining passwords with PlanetScale. Once [SSO](/docs/security/sso) is enabled for an `organization`, all members are redirected through that identity provider's authentication flow. Moving forward, they must pass through SSO to access their PlanetScale account. ## OAuth via GitHub Users can authenticate with PlanetScale using their GitHub account. > [!WARNING] > > PlanetScale doesn't maintain the passwords for these accounts. Losing access to your GitHub account prevents accessing > your PlanetScale account. > [!NOTE] > > Enabling SSO removes OAuth access for all members of your *organization*, meaning they will no longer be able > to sign in with their GitHub credentials. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Multi-factor authentication Source: https://planetscale.com/docs/security/multi-factor-authentication Multi-factor authentication (MFA) provides better safety for your databases and prevents unauthorized access to your user account. ## Overview MFA strengthens security by requiring two or more methods *(i.e. authentication factors)* to verify your identity. PlanetScale allows users logging in with an email address and password to set MFA as a requirement for logging into the user account. > [!NOTE] > > If you're authenticating via GitHub OAuth or [SSO](/docs/security/sso), MFA settings will not be displayed. ### Authentication providers PlanetScale supports login with a unique *time-based one-time password (TOTP)* that is generated for your user account by using TOTP apps such as [1Password](https://support.1password.com/one-time-passwords/), [Authy](https://help.twilio.com/articles/19753413823643), or [LastPass Authenticator](https://lastpass.com/auth/). ## Enable MFA You can enable MFA for your user account under your PlanetScale account settings. **Step 1** Go to your [PlanetScale account settings](https://app.planetscale.com/account) page. **Step 2** Find the **Security** row and click the **"Setup multi-factor authentication"** button. ![Click the "Setup MFA" button priority](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/mfa/setup.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=759dc07f699bb12c7115b2b24ee035d6) This will bring up a pop-up modal with a *QR code* and some `recovery codes` that you will need to copy. ![Pop-up modal with QR code and recovery codes priority](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/mfa/recovery-codes.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=3f7c5e1ab150e3b48f164a04bfc6c237) > [!WARNING] > > Recovery codes are the only account recovery method accepted when MFA is enabled. If you lose both your TOTP app and the recovery codes, there is no way to regain access to your account. **Step 3** Scan the QR Code with your preferred TOTP app and enter the generated code. **Step 4** Press **"Validate OTP"** to ensure that your application setup is correct. **Step 5** Once the generated code is validated, click the **Copy** button in the `recovery codes` section. ![Copy the recovery codes](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/mfa/copy.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=2a28f8791b68f85570d38818c99256b0) > [!TIP] > > Recovery codes are only visible during the MFA setup process. Make sure the recovery codes are copied and saved some place secure before continuing. **Step 6** Click **Done** to close the pop-up modal. ## Login with two-factor authentication Once you've enabled MFA in your PlanetScale user account, the next time you login, you'll be prompted to enter your two-factor authentication (2FA) code. * Use the **OTP code** generated by your preferred TOTP app to login to your PlanetScale account. ## Recovery code login The `recovery codes` shown during MFA setup are **the only way regain access to your account** in the event that you lose the device that generates your authentication codes. PlanetScale will not accept any other method of authentication or identification. You can use one of the `recovery codes` in the place of a TOTP token on the second screen during login. ## Disable MFA > [!WARNING] > > We strongly recommend that you do not disable MFA to avoid unauthorized access to your user account. > [!NOTE] > > * Any devices setup with the QR code for your account will no longer be able to produce valid OTP tokens. > * Any recovery codes that were generated when MFA was enabled will no longer be valid. You can disable MFA for your user account under your PlanetScale account settings. **Step 1** Go to your [PlanetScale account settings](https://app.planetscale.com/account) page. **Step 2** Click the **Disable** button next to *"Multi-factor authentication enabled"* in the **Security** row. ![Click the "Disable" button](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/mfa/disable.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=7e3ac87c9a3b1a1fd18d7419f757f0c5) **Step 3** Enter an **OTP code** or one of the `recovery codes` generated by your preferred TOTP app to confirm. ![Disable MFA pop-up modal](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/mfa/modal.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=c8315d37e641c8aa2b4200c14169cdb1) **Step 4** Click the **Disable** button to close the pop-up modal. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Security log Source: https://planetscale.com/docs/security/security-log The Security log gives you insight into recent authentication and other security-related `actions` for your account. Your security log also details when you performed each `action`. ## Overview Security log events are retained for 6 months in your PlanetScale account. ## Review your security log You can review your security log in your PlanetScale account settings. 1. Go to your [PlanetScale Security log](https://app.planetscale.com/settings/security-log) page. Once there, you can filter your Security log by `Action`. ![Filter your security log by Action.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/security-log/filter.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=1dcedcf81d1f05b5f6b90756a40bb985) ## Security log events You can track the following `events` in your PlanetScale account: | PlanetScale security log events | Actions | | :- | :- | | user | signed\_out signed\_in accepted\_tos updated\_name updated\_email enabled\_mfa disabled\_mfa | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Single sign-on Source: https://planetscale.com/docs/security/sso Single sign-on (SSO) provides additional account security by allowing company administrators to require the use of an identity provider when logging into PlanetScale. ## Overview Users only need to sign in once with a single set of credentials *(i.e. password and email)* to access all of their tools and applications upon joining the company. Furthermore, SSO allows an administrator to revoke someone’s access to all tools and applications from a single place when they leave a team or the company. PlanetScale uses SAML SSO. You can also enable and manage organization SSO from the [PlanetScale CLI](/docs/cli/org). > [!NOTE] > > SSO is available as an *add-on* for the [Base plan](/docs/planetscale-plans#base-plan) at \$199/month per organization and included in our [Enterprise plans](/docs/planetscale-plans#enterprise-plan). Security is important to us, so we do not profit off of SSO. We only charge enough to cover the WorkOS cost for enrolling a new account. ## Implications It's important to understand how enabling SSO will affect your Organization. Once enabled, the following will happen: * All non-admin members will be removed from the organization. * [Organization Administrators](/docs/security/access-control#organization-administrator) will remain in the Organization so they can configure SSO without losing access. * All administrators will retain access with their old credentials, until they logout and login through their Identity Provider. Once they've authenticated through their Identity Provider the account will only be usable with SSO authentication. * Each [Organization Member](https://app.planetscale.com/planetscale/settings/members) must re-authenticate using SSO. Once they've authenticated, they will be automatically added back to the organization. * Organization Member invites will be disabled when SSO is enabled; all Organization Membership will be managed through SSO. * Organization Members that were [Database Administrators](/docs/security/access-control#database-administrator) before will no longer have that role upon rejoining. You must assign them the role after they re-authenticate with SSO. * Organization Members that were on [Teams](/docs/security/teams) will need to be re-added. * Any database credentials and tokens that were generated by non-admin members will remain active, so you **do not need to regenerate connection credentials**. * Organization Members removed from your SSO will still appear in the Organization until they are manually removed. * While they are visible in the Organization they will not be able to authenticate to PlanetScale. * If you enabled [Directory Sync](#directory-sync), the Member will be removed from the Organization without manual intervention. * Organization Members can only authenticate through SSO when their email address uses a [verified domain](#domain-verification). > [!NOTE] > > If you enable SSO and Directory Sync, the Directory will remain the source of truth, and Teams will map accordingly. Please see the [Directory Sync](#directory-sync) section for more information. ## Domain verification Verify ownership of each email domain your organization uses for SSO. Users can sign in through SSO only when their email address uses a verified domain. You can start verification from the dashboard or with `pscale org sso domain verify`. > [!NOTE] > > The **Configure single sign-on** button stays disabled until at least one domain is verified. ### Verify a domain **Step 1** Go to your [PlanetScale organization's authentication page](https://app.planetscale.com/~/settings/authentication) under **Settings** → **Authentication**. **Step 2** Under the **Single sign-on** section, click the **Verify domain** button. **Step 3** You will be redirected to the WorkOS domain verification portal. Follow the steps there to verify your domain by adding a DNS TXT record to your domain's DNS configuration. **Step 4** Once WorkOS confirms the DNS record, your domain will appear as **Verified** on the authentication page. DNS propagation can take a few minutes to a few hours depending on your DNS provider and TTL settings. When at least one domain shows **Verified**, continue with [Enable SSO for your organization](#enable-sso-for-your-organization). ### Manage verified domains Verified and pending domains are listed on the authentication page. You can delete a domain at any time using the **Delete** button. Verify every domain your members use to sign in (for example, `company.com` and `subsidiary.com`). If SSO is enabled and a member's email uses a domain you have not verified, they cannot authenticate through SSO. > [!WARNING] > > If you delete a verified domain, users with email addresses on that domain can no longer authenticate through SSO. If SSO is already enabled, keep at least one verified domain for every email domain your members use. ## Enable SSO for your organization To enable SSO for your organization, you must be an [Organization Administrator](/docs/security/access-control). Organization administrators can *enable*, *configure*, and *disable* SSO for all members of your organization in the dashboard or with [`pscale org sso`](/docs/cli/org). **Step 1** [Verify your organization's domain(s)](#domain-verification). **Step 2** Head to your [PlanetScale organization's authentication page](https://app.planetscale.com/~/settings/authentication) under **Settings** → **Authentication** and click **Enable single sign-on**. **Step 3** Once SSO is enabled, click **Configure single sign-on**. This will take you to WorkOS where you can choose your identity provider. You can find documentation for your specific identity provider in the [WorkOS integrations documentation](https://workos.com/integrations). You also have the option to manage PlanetScale roles through your identity provider's SSO profile. Just check the box next to that message on the settings page to enable. > [!NOTE] > > If you are using Okta as your identity provider and want to manage organization roles through SSO or Directory Sync, see [Using Okta to manage organization admins](#using-okta-to-manage-organization-admins) for additional configuration steps. ## Multiple organizations with SSO You can enable SSO for multiple organizations using the same identity provider. Users will be able to swap between organizations by using the selector in the top right of the PlanetScale dashboard. > [!TIP] > > When setting up SSO for multiple organizations, make sure to check that the role management settings are configured correctly for each organization. The settings for each organization are independent of each other. ## Disable SSO When SSO is disabled for an organization, users can log in with the password they initially set for their PlanetScale account. If they don't know their password, users can go through the password reset flow to regain access to their account. Should a user lose access to the email address associated with that organization, they'll also lose access to their account after SSO is disabled. ## Directory Sync We also support the use of Directory Sync with SSO. You can use Directory Sync to make the directory the source of truth for organization membership. You can open the directory setup portal with `pscale org sso directory enable`. ### Enable Directory Sync To enable Directory Sync, you first must have SSO enabled. Once enabled, go to your Organization settings page, click "**Authentication**", and click the "**Enable directory sync**" button. You can now configure Directory Sync using your identity provider. ### Directory Sync access control Directory Sync automatically adds and removes members from your PlanetScale organization to match your SSO directory. If you have groups defined within your SSO provider, it can also automatically create [Teams](/docs/security/teams) within your PlanetScale organization mapped to those groups. If you wish to have your identity provider determine user roles in PlanetScale, please make sure to select the option for `Manage PlanetScale roles through identity provider` in Settings > Authentication. ![Manage roles](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/sso/sync.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=55861fac34c727f7a2805771c82cda7d) > [!NOTE] > > Once you enable Directory Sync, existing Teams will be cleared, as all Teams must map to a Directory group. You can find the directory-managed members under "**Settings**" > "**Members**", and directory-managed Teams under "**Settings**" > "**Teams**". ![Dashboard UI - Directory-managed Teams page](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/sso/managed.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=f36262759a8dc656e592aa960eaaa77e) ## Admin failsafe When PlanetScale roles are managed through SSO or Directory Sync, PlanetScale keeps at least one `admin` in the organization. This failsafe helps prevent identity provider or directory role mapping misconfiguration from removing every admin, so an admin can update settings and fix the configuration. ## Using Okta to manage organization admins There are two different ways to configure Okta to manage the user's role in a PlanetScale organization. The first approach uses SAML assertions during Single Sign On to update the role just-in-time. The second approach uses an attribute defined on the directory user to update the role in real time as it changes. Choose the approach that works best for your organization. ### Managing roles during single sign-on (SAML) In order to update roles using SAML, you will first need to configure the SAML application and user profile in Okta before enabling the setting within PlanetScale. #### Configuring the SAML Okta application **Step 1** Click the **Applications** > **Applications** in the Okta dashboard's sidebar. **Step 2** Find your PlanetScale SAML application that you configured earlier for WorkOS and find the **SAML Settings** section. Click on the **Edit** link in this section. **Step 3** In the **Configure SAML** step, find the **Attribute Statements (optional)** section, which defines the attributes of email, first and last name, and id. **Step 4** Add a new Attribute Statement with the following values. * Name — `planetscale_role` * Name Format — `Unspecified` * Value — `user.planetscale_role` #### Creating user profile attributes in Okta In order to define the custom attribute in Okta, you must update the default User profile to have the `planetscale_role` attribute. **Step 1** Click **Directory** > **Profile Editor** in the Okta dashboard's sidebar. **Step 2** Click the **User (default)** profile within Okta **Step 3** Click **Add attribute** and fill in the following values * Data type: `string` * Display name: `PlanetScale Role` * Variable name: `planetscale_role` * Enum: Enable the **Define enumerated list of values** checkbox and add these attributes: * Display name: `Member`, Value: `member` * Display name: `Analyst`, Value: `analyst` * Display name: `Administrator`, Value: `admin` **Step 4** Click **Save attribute** #### Enable in PlanetScale Lastly, we will have to enable role management via SSO within the PlanetScale application. **Step 1** Go to your SSO organization's settings page **Step 2** Click on **Authentication** in the side navigation **Step 3** Check the **Manage PlanetScale roles through identity provider's SSO profile** checkbox After following all of these steps, after you change the PlanetScale role for a user in Okta, their role will be updated accordingly the next time they log in via Single Sign On. If the role is changed during an active session, they will need to log out and log back in for it to be updated once again. ### Managing roles using Directory Sync (SCIM) In order to manage roles using Directory Sync, you must properly configure the SCIM user's profile within Okta and update the Directory Sync custom attributes before enabling the setting within the PlanetScale dashboard. #### Configuring the SCIM user profile in Okta **Step 1** Click **Directory** > **Profile Editor** in the Okta dashboard's sidebar. **Step 2** Click the user profile that maps to your SCIM application, it should be in the format of **\< SCIM application name > User** **Step 3** Click **Add attribute** and fill in the following values * Data type: `string` * Display name: `PlanetScale Role` * Variable name: `planetscale_role` * External name: `planetscale_role` * The External namespace should be defined as `urn:ietf:params:scim:schemas:core:2.0:User`. * Check the box to define an enumerated list of values, add the following: * Display name: `member`, Value: `member` * Display name: `analyst`, Value: `analyst` * Display name: `administrator`, Value: `admin` **Step 4** Click **Save attribute** ![PlanetScale Role](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/sso/planetscale_role.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=c22f15bf288b9222d6ee1f6e22202115) This attribute then needs to be mapped to the PlanetScale application in Okta. **Step 1** Open the PlanetScale SCIM application in Okta Admin Console, then select Provisioning **Step 2** Scroll down to the bottom of the `Okta to App` provisioning page, and click `Show unmapped attributes`. **Step 3** Find `planetscale_role` and click the 🖋️ to map the attribute. **Step 4** Select `Map from Okta Profile` as the type and `planetscale_role` as the string. **Step 5** Save with Create & Update permission. #### Configuring custom attributes within WorkOS Next, you will need to update your WorkOS configuration to detect the custom attribute mappings. **Step 1** Go to the **Authentication** within your organization settings page. **Step 2** In the **Directory sync** section, click the `identity provider` link to open up the directory configuration page **Step 3** Click the **Edit Attribute Map** button and in the form, enter the following: * Directory Provider Value: `planetscale_role` * PlanetScale Attribute: `planetscale_role` **Step 4** Click **Save mappings** #### Enable in PlanetScale Lastly, we will have to enable role management via SSO within the PlanetScale application. **Step 1** Go to your SSO organization's settings page **Step 2** Click on **Authentication** in the side navigation **Step 3** Check the **Manage PlanetScale roles through identity provider's directory** checkbox After following all of these steps, after you change the PlanetScale role for a user in Okta, their role will be updated in PlanetScale without needing to re-authenticate. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Teams Source: https://planetscale.com/docs/security/teams PlanetScale allows you to create teams within organizations. ## Overview This allows you to easily manage administrator access to one or multiple databases all in one spot. ## Create and manage Teams You can manage teams straight from your PlanetScale dashboard by going to "**Settings**" > "**Teams**". > [!NOTE] > > Only [Organization Administrators](/docs/security/access-control#organization-administrator) can create and > manage Teams. Once you add databases to a team, any members on that team will have [Database Administrator access](/docs/security/access-control#database-level-permissions) to those databases. Review our [Access control documentation](/docs/security/access-control) to understand the full scope of Database Administrator access. ### Create a team **Step 1** On your PlanetScale overview page, click "**Settings**". **Step 2** Click "**Teams**" in the left nav. ![Dashboard UI - Create a PlanetScale team](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/teams/create.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=807359c494a2da5af3c5df900cb25361) **Step 3** Give your team a name and description (*optional*). **Step 4** Click "**Create team**". ### Add members **Step 1** Click "**Add a member**". **Step 2** You'll see a list of your Organization members. Select the member(s) one at a time that you wish to add to the team. ### Add databases **Step 1** Click "**Add databases**". **Step 2** Select the databases you want this team to have [database administrator access](/docs/security/access-control#database-level-permissions) to. Now, when you go to the Settings page for any databases you've added to a team, you'll also be able to view and revoke access straight from the database Administrators page. ![Dashboard UI - Database Administrators settings page](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/teams/settings.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=b8c11f131c83aec215dd8da1d5e84567) ### Remove members and databases To remove a member from a team, find their name in the member list and click "**Remove**". At this time, you'll also be able to delete any passwords this member has created to ensure you've completely revoked their access to the database. ![Dashboard UI - Delete a member from a team](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/teams/member.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=a782f3091086aeed095c33ffc3b511e0) To remove a database from a team, click the "**x**" next to the database name under "Administrator permissions". This will remove database administrator access for all members of the team. ![Dashboard UI - Delete a database from a team](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/teams/database.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=8976a32f9c1f8c2fe97d81e095813b39) ## Directory Sync with Teams If you have [SSO with Directory Sync](/docs/security/sso#directory-sync) enabled, all Teams will be managed by your Directory Sync directory. You can add and remove database access to teams, but member management must be done through your directory. ![Dashboard UI - Directory-managed Teams page](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/sso/managed.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=f36262759a8dc656e592aa960eaaa77e) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vulnerability disclosure Source: https://planetscale.com/docs/security/vulnerability-disclosure ## In scope PlanetScale is actively seeking vulnerability reports for the following components that make up the product and its Production Environment: * **Dashboard and API**: The website hosted at app.planetscale.com, along with the API hosted at api.planetscale.com * **Database Operations**: The actions taken within the product to create, branch, backup, and restore databases * **Database Connectivity and Behavior**: The process of provisioning a password and issuing SQL statements against a PlanetScale database * **Command-line Interface**: The open source command-line interface hosted at [planetscale/cli](https://github.com/planetscale/cli) ## Out of scope PlanetScale is not actively seeking the following types of reports: * **Testing software output**: Output generated from automated testing software like [Burp Suite](https://portswigger.net/burp). These include, but aren't limited to: * CSRF on forms that are available to anonymous users or are related to logging out * Disclosure of known public files or directories (i.e. `robots.txt`) * DNSSEC or other DNS configuration suggestions * TLS and security header configuration suggestions * Sender Policy Framework (SPF) configuration suggestions * Flags on cookies that are not sensitive * **Software version reports**: Reports notifying PlanetScale that newer versions of software have been released ## Reporting a vulnerability If you believe you have discovered a security vulnerability in a PlanetScale product or its Production Environment, please let us know immediately. You can submit your vulnerability findings to [security@planetscale.com](mailto:security@planetscale.com). If applicable, please include the following pieces of information in your report: * Steps to reproduce the vulnerability * The word "mochi" to acknowledge that you have read these guidelines * Any relevant software (including versions) used to identify the vulnerability ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Self-improving database Source: https://planetscale.com/docs/self-improving-database Use the PlanetScale MCP server with AI coding agents to continuously optimize database performance based on production Insights data and Schema Recommendations Vitess and Postgres The [PlanetScale MCP server](/docs/mcp-server) gives AI agents access to production query metrics ([Insights](/docs/postgres/monitoring/query-insights)) and [Schema Recommendations](/docs/postgres/monitoring/schema-recommendations) alongside your application code. This combination enables a self-improving optimization loop: an agent identifies the highest-impact performance issues, finds the relevant code, makes improvements, and opens a pull request. This works for both Postgres and Vitess databases. [Self-improving database](https://www.youtube-nocookie.com/embed/T7aof_ilvkQ?rel=0) ## How it works ```mermaid theme={null} graph LR MCP["PlanetScale MCP"] -->|"Insights + Recommendations"| Agent["AI Agent"] Codebase["Your Codebase"] -->|"Application code"| Agent Agent -->|"Opens pull request"| GitHub["GitHub"] ``` 1. The agent connects to PlanetScale through the MCP server and pulls Insights data or open Schema Recommendations. 2. Identifies the highest-impact queries or schema issues. 3. Searches your codebase to find where those queries originate. 4. Makes the change in the codebase. 5. Benchmarks the change in a development environment to validate the improvement. 6. Opens a pull request with the fix and an explanation of the expected impact. This can run as a one-off task or on a recurring schedule. [Cursor Automations](https://cursor.com/docs/cloud-agent/automations) are a great way to run this daily — they run cloud agents in the background on a schedule and support MCP server connections, making them ideal for this workflow. ## Prerequisites * [PlanetScale MCP server](/docs/mcp-server) installed and authenticated in your AI coding tool * [Cursor Automations](https://cursor.com/docs/cloud-agent/automations) (recommended) or another MCP-compatible coding agent such as Claude Code * A PlanetScale database with [Insights](/docs/postgres/monitoring/query-insights) data * An `AGENTS.md` file in your repository with your PlanetScale org, database, and branch (see [Help your agent target your project's database](/docs/mcp-server#help-your-agent-target-your-projects-database)) > [!TIP] > > These workflows only need Insights and Schema Recommendations access. You can use the insights-only MCP server (`https://mcp.pscale.dev/mcp/planetscale-insights-only`) to avoid exposing query execution tools to the agent. ## Getting started For best results, take an iterative approach: 1. **Run the prompt manually first.** Use one of the example prompts below in your AI coding tool and review the pull request it produces. 2. **Refine with guardrails.** Based on the results, add rules specific to your codebase — which directories to touch, which ORM patterns to follow, linter and test requirements, migration commands, etc. 3. **Repeat a few times.** Run it again with your refined prompt. Nudge the agent where needed. Each iteration helps you build a prompt that consistently produces high-quality PRs. 4. **Graduate to a schedule.** Once the agent reliably produces good results, set it up as a [Cursor Automation](https://cursor.com/docs/cloud-agent/automations) on a daily or weekly schedule. > [!NOTE] > > Always review pull requests before merging. Automated agents can surface valuable optimizations, but a human should verify that the changes are safe for your workload. ## Example prompt: Insights optimization This prompt instructs the agent to review your Insights data, find the highest-impact queries, and optimize them. Customize the rules section for your framework, ORM, and linter. ```markdown theme={null} You are a database performance automation expert focused on evaluating data from PlanetScale Insights and improving application and database query performance. Org: [your PlanetScale org] Database: [your database name] Branch: main ## Goal Evaluate PlanetScale Insights data and improve the performance of the application and database. ## How to Use the PlanetScale MCP server to look at available Insights data. Identify which queries are causing the most impact on the production database (slow, high rows read, frequent, high egress). Find where in the codebase the query is being run from and look for ways to optimize it. ## Rules * Only fix one query at a time. * Check existing open pull requests in GitHub. If one already exists solving this query pattern, pick another one. * Before making changes, benchmark the query in a development environment using EXPLAIN to establish a baseline. * After making changes, benchmark again to validate the improvement. * Run the linter and fix any errors before opening the pull request. * When opening the pull request, start the description with "Implementing PlanetScale Insights improvement" and explain why you chose this query to optimize. Share the potential impact of fixing it as well as any potential risks from making the change. Include before/after benchmark results. ``` > [!NOTE] > > Adapt the rules to your stack. For example, a Rails project might add: "Run migrations with `rails db:migrate` to generate the `schema.rb` file" and "Run rubocop and fix any lint errors." ## Example prompt: Schema Recommendations This prompt instructs the agent to evaluate open Schema Recommendations and implement them. Customize the rules section for your framework and workflow. ```markdown theme={null} You are a database performance automation expert focused on evaluating and implementing PlanetScale Schema Recommendations. Org: [your PlanetScale org] Database: [your database name] Branch: main ## Goal Evaluate and implement PlanetScale Schema Recommendations to improve the performance of the application and database. ## How to Use the PlanetScale MCP server to look at all open schema recommendations. If any are available, pick the oldest one and implement it. If none are available, do nothing. ## Rules * Only fix one recommendation at a time. * Check existing open pull requests in GitHub. If one already exists solving this schema recommendation, pick another one. * Run the linter and fix any errors before opening the pull request. * When opening the pull request, start the description with "Implementing PlanetScale schema recommendation" and link to the recommendation. ## For dropping a table or removing a column If you need to remove a table or column, this is a two-step process. 1. Remove all code that references the table or column. When opening the pull request, add an important note cautioning that this action is dangerous. The code changes must be deployed to production before the schema changes. 2. Then make the schema change that drops the table or column once no code in production is making use of it. ``` ## Tips for effective automations * **One fix at a time.** Each run should address a single query or recommendation. This keeps pull requests small, reviewable, and easy to revert. * **Benchmark in development.** Instruct the agent to run `EXPLAIN` or time queries on a development branch before and after the change. This validates the optimization and gives reviewers concrete data. * **Use `AGENTS.md` to provide context.** Include your PlanetScale org, database, and branch so the agent doesn't have to scan through all your organizations. * **Adapt prompts to your stack.** Add rules for your specific framework, ORM, migration tool, and linter. The more specific the guardrails, the better the output. * **Review every PR.** Automated agents surface real improvements, but schema and query changes require human judgment before hitting production. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Support Source: https://planetscale.com/docs/support PlanetScale’s team has a combined 600 years of experience running databases at scale. Our support goes well beyond what you might have experienced with other providers. We partner closely with our customers whether they are launching a AAA game or planning brand new architecture. Even with the Standard tier, you can expect timely, and thorough responses from engineers. We also have an active [Discord server](https://pscale.link/community) where you can chat with staff as well as a [GitHub Discussion board](https://github.com/planetscale/discussion/discussions). If you’re looking into Vitess or already running it on your own, the security of knowing you have a direct line to the core maintainers and co-creators of Vitess is invaluable. PlanetScale believes all our users, from those running small databases to those running large, mission-critical workloads, deserve to get the help they need. We are committed to providing the following support options. ## Standard * Included for Base plan customers * Access to support engineers via [email](mailto:support@planetscale.com) and [web ticketing](https://planetscale.com/contact?initial=support) * Access to the [PlanetScale GitHub Discussion](https://github.com/planetscale/discussion/discussions) board * Ability to [submit support tickets](https://planetscale.com/contact?initial=support) * Target response time of 2 business days ## Business * Available as an upgrade for Base plans and included on Enterprise plans * Access to the [PlanetScale GitHub Discussion](https://github.com/planetscale/discussion/discussions) board * Access to support engineers via email and [web ticketing](https://planetscale.com/contact?initial=support) * Enhanced response times including Urgent and High Priority coverage * [Business Response Times](#initial-response-times) ## Enterprise * Available upgrade for Enterprise Plan customers * Enhanced email and [web ticketing](https://planetscale.com/contact?initial=support) support * Additional options for technical account management, Slack-based support, and phone escalation * Mission-critical response times including continuous support coverage * [Enterprise Response Times](#initial-response-times) > [!NOTE] > > [Contact our sales team](https://planetscale.com/contact) for Business and Enterprise support pricing. ## Initial response times > [!NOTE] > > Tickets must be submitted through web ticketing to be eligible for response times. > > 12x5 means from 6am to 6pm Pacific Standard Time (UTC -8), Monday through Friday, excluding public holidays in the United States. > > Standard support includes a target first response time of 2 business days. | Support plan | P1 (Urgent) | P2 (High) | P3 (Normal) | P4 (Low) | | :- | :- | :- | :- | :- | | Standard (Base plan) | 12x5 | 12x5 | 12x5 | 12x5 | | Business (Optional Upgrade) | 1 hour (24x7) | 4 hours (24x7) | 1 business day (12x5) | 2 business days (12x5) | | Enterprise (Optional Upgrade) | 15 minutes (24x7) | 1 hour (24x7) | 4 hours (12x5) | 12 hours (12x5) | ## Ticket priorities P1 (Urgent) and P2 (High) priority tickets are reserved exclusively for production database issues, excepting confirmed security incidents. Development, staging, or non-production environments should use P3 (Normal) or P4 (Low) priority levels. > [!NOTE] > > PlanetScale Support has the ability to update a ticket's priority if it does not generally match these guidelines. | Priority | Description | | :- | :- | | **P4 (Low)** | One or more of: - General platform and workflow questions - Feature requests - General product feedback | | **P3 (Normal)** | One or more of: - Partial, non-critical loss of functionality of the PlanetScale platform. Users are able to continue using core platform functionality. - A non-production application running against the PlanetScale platform is impaired or down. - There is a partial, non-critical loss of use of the platform with a medium-to-low business impact. Short-term workaround is available. | | **P2 (High)** | One or more of: - A production application is experiencing performance issues due to the PlanetScale platform. - Intermittent issues and reduced quality of service for significant features. - Operations can continue in a restricted fashion, although long-term productivity might be adversely affected. - A temporary workaround is available for an issue that was previously a P1 (Urgent). | | **P1 (Urgent)** | One or more of: - Production applications are down due to the PlanetScale platform and no workaround is immediately available. - A complete loss of ability to serve queries, commit transactions, or a significant feature impacting production workflows is completely unavailable. - All or a substantial portion of your data on PlanetScale production branches is at significant risk of loss or corruption. - A confirmed security incident has been identified. | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Terraform provider Source: https://planetscale.com/docs/terraform A resource model for Vitess, Neki, and Postgres databases, designed for Terraform and OpenTofu. Vitess, Neki, and Postgres Terraform is an open-source infrastructure-as-code tool that lets you define and manage cloud resources through declarative configuration files. With the [PlanetScale Terraform provider](https://github.com/planetscale/terraform-provider-planetscale), you can manage PlanetScale Vitess, Neki, and Postgres databases, branches, credentials, backups, backup policies, Neki routers and shards, dedicated PgBouncers, and cluster parameters alongside the rest of your infrastructure. ## Conceptual model The PlanetScale Terraform provider is organized around a clear distinction between **Vitess**, **Neki**, and **Postgres** resources and is focused on long-lived infrastructure objects. * There are separate resources for each **database kind**—Vitess, Neki, or Postgres. * The provider maps cleanly onto PlanetScale's public API while exposing Terraform-friendly fields and lifecycle behavior. PlanetScale databases do not have a dedicated Terraform resource. Their lifecycle is managed implicitly through branch resources: * Applying a `planetscale_vitess_branch`, `planetscale_neki_branch`, or `planetscale_postgres_branch` creates the parent database if it does not already exist. * Destroying the last branch in a database also destroys the database. See [Deletion protection](#deletion-protection) for safeguards. Creating a Neki branch also provisions a default [configuration profile](/docs/neki/cluster-configuration#configuration-profiles), [shard](/docs/neki/cluster-configuration#manage-shards), [router group](/docs/neki/cluster-configuration#configure-router-groups), [admin](/docs/neki/cluster-configuration#configure-the-admin-service), and [sidecar](/docs/neki/overview#what-runs-alongside-postgres). Import those objects if Terraform should manage them. Do not recreate the defaults with `planetscale_neki_configuration_profile` or `planetscale_neki_shard`. Creating a Vitess branch likewise provisions the branch's default [keyspace](/docs/vitess/sharding/keyspaces); import it to manage its size or replicas. ### Credential models **Vitess**, **Neki**, and **Postgres** use different terminology for database credentials: | Database Type | Resource | How permissions work | | - | - | - | | Vitess | `planetscale_vitess_branch_password` | Set the `role` attribute to `reader`, `writer`, `readwriter`, or `admin` | | Neki | `planetscale_neki_role` | Use `inherited_roles` to inherit from [built-in Postgres roles](/docs/neki/connecting/roles) like `pg_read_all_data` or `pg_write_all_data`. | | Postgres | `planetscale_postgres_branch_role` | Use `inherited_roles` to inherit from [built-in Postgres roles](/docs/postgres/connecting/roles) like `pg_read_all_data` or `pg_write_all_data`. | | Postgres | `planetscale_postgres_redacted_branch_role` | Same as `planetscale_postgres_branch_role`, but the password is not stored in Terraform state. Use this when you manage passwords in a secret manager. | ## Quick start This complete example creates a Postgres branch with application credentials. Vitess and Neki follow the same pattern with their own branch and credential resources. ```hcl theme={null} terraform { required_providers { planetscale = { source = "planetscale/planetscale" version = "~> 1.3" } } } # Manage a branch (the database is created automatically if it does not exist) resource "planetscale_postgres_branch" "main" { organization = "my-org" database = "my-db" name = "main" } # Create application credentials resource "planetscale_postgres_branch_role" "app" { organization = "my-org" database = "my-db" branch = planetscale_postgres_branch.main.id name = "app" inherited_roles = ["pg_read_all_data", "pg_write_all_data"] } # Output connection details output "connection_string" { sensitive = true value = "postgres://${planetscale_postgres_branch_role.app.username}:${planetscale_postgres_branch_role.app.password}@${planetscale_postgres_branch_role.app.access_host_url}/${planetscale_postgres_branch_role.app.database_name}" } ``` Run: ```bash theme={null} export PLANETSCALE_SERVICE_TOKEN_ID="your-token-id" export PLANETSCALE_SERVICE_TOKEN="your-token" terraform init terraform apply terraform output -raw connection_string ``` ## Available resources The provider offers resources for the most common automation scenarios: **Vitess** * `planetscale_vitess_backup_policy` * `planetscale_vitess_branch` * `planetscale_vitess_branch_backup` * `planetscale_vitess_branch_password` * `planetscale_vitess_keyspace` **Neki** * `planetscale_neki_admin` * `planetscale_neki_backup_policy` * `planetscale_neki_branch` * `planetscale_neki_configuration_profile` * `planetscale_neki_role` * `planetscale_neki_router` * `planetscale_neki_shard` * `planetscale_neki_sidecar` **Postgres** * `planetscale_postgres_backup_policy` * `planetscale_postgres_bouncer` * `planetscale_postgres_branch` * `planetscale_postgres_branch_backup` * `planetscale_postgres_branch_role` * `planetscale_postgres_read_only_replica` * `planetscale_postgres_redacted_branch_role` ## Data sources The provider also offers data sources for reading existing resources: **Organization and databases** * `planetscale_databases` * `planetscale_database_postgres` * `planetscale_database_vitess` * `planetscale_organization` * `planetscale_organizations` **Vitess** * `planetscale_vitess_backup_policies` * `planetscale_vitess_backup_policy` * `planetscale_vitess_branch` * `planetscale_vitess_branch_backup` * `planetscale_vitess_branch_backups` * `planetscale_vitess_branch_password` * `planetscale_vitess_branch_passwords` * `planetscale_vitess_keyspace` * `planetscale_vitess_keyspaces` **Neki** * `planetscale_neki_admin` * `planetscale_neki_backup_policies` * `planetscale_neki_backup_policy` * `planetscale_neki_branch` * `planetscale_neki_configuration_profile` * `planetscale_neki_configuration_profiles` * `planetscale_neki_role` * `planetscale_neki_roles` * `planetscale_neki_router` * `planetscale_neki_routers` * `planetscale_neki_shard` * `planetscale_neki_shards` * `planetscale_neki_sidecar` **Postgres** * `planetscale_postgres_backup_policies` * `planetscale_postgres_backup_policy` * `planetscale_postgres_bouncer` * `planetscale_postgres_bouncers` * `planetscale_postgres_branch` * `planetscale_postgres_branch_backup` * `planetscale_postgres_branch_backups` * `planetscale_postgres_branch_role` * `planetscale_postgres_branch_roles` * `planetscale_postgres_read_only_replica` * `planetscale_postgres_read_only_replicas` * `planetscale_postgres_redacted_branch_role` ### Data source behavior | Data Source | Required Inputs | Notes | | - | - | - | | `planetscale_databases` | `organization` | Lists Vitess, Neki, and Postgres databases. Optional `q` parameter for filtering by name. | | `planetscale_database_postgres` | `organization` | Returns Postgres database info. Does not accept a database name filter. | | `planetscale_database_vitess` | `organization` | Returns Vitess database info. Does not accept a database name filter. | | `planetscale_organization` | `organization` | Returns organization details. | | `planetscale_organizations` | (none) | Lists all accessible organizations. | | `planetscale_vitess_backup_policies` | `organization`, `database` | Lists Vitess backup policies for a database. | | `planetscale_vitess_backup_policy` | `organization`, `database`, `id` | Fetches a specific Vitess backup policy by ID. | | `planetscale_vitess_branch` | `organization`, `database` | Returns the default branch. Does not accept a branch name parameter. | | `planetscale_vitess_branch_backups` | `organization`, `database`, `branch` | Lists Vitess backups for a branch. Supports filters like `state`, `from`, `to`, and `policy`. | | `planetscale_vitess_branch_backup` | `organization`, `database`, `branch`, `id` | Fetches a specific Vitess backup by ID. | | `planetscale_vitess_branch_passwords` | `organization`, `database`, `branch` | Lists all passwords on a branch. | | `planetscale_vitess_branch_password` | `organization`, `database`, `branch`, `id` | Fetches a specific password by ID. | | `planetscale_vitess_keyspace` | `organization`, `database`, `branch`, `name` | Fetches a specific keyspace by name. | | `planetscale_vitess_keyspaces` | `organization`, `database`, `branch` | Lists keyspaces for a branch. | | `planetscale_neki_admin` | `organization`, `database`, `branch` | Fetches the admin for a branch. | | `planetscale_neki_backup_policies` | `organization`, `database` | Lists Neki backup policies for a database. | | `planetscale_neki_backup_policy` | `organization`, `database`, `id` | Fetches a specific Neki backup policy by ID. | | `planetscale_neki_branch` | `organization`, `database`, `id` | Fetches a specific branch by ID. Unlike the Vitess and Postgres branch data sources, this requires a branch ID. | | `planetscale_neki_configuration_profile` | `organization`, `database`, `branch`, `name` | Fetches a specific configuration profile by name. | | `planetscale_neki_configuration_profiles` | `organization`, `database`, `branch` | Lists configuration profiles for a branch. | | `planetscale_neki_role` | `organization`, `database`, `branch`, `id` | Fetches a specific role by ID. Does not return the password. | | `planetscale_neki_roles` | `organization`, `database`, `branch` | Lists roles on a branch. Optional `q` and `status` filters. | | `planetscale_neki_router` | `organization`, `database`, `branch`, `name` | Fetches a specific router group by name. | | `planetscale_neki_routers` | `organization`, `database`, `branch` | Lists router groups for a branch. | | `planetscale_neki_shard` | `organization`, `database`, `branch`, `id` | Fetches a specific shard by ID. | | `planetscale_neki_shards` | `organization`, `database`, `branch` | Lists shards for a branch. | | `planetscale_neki_sidecar` | `organization`, `database`, `branch`, `configuration_profile` | Fetches the sidecar for a configuration profile. | | `planetscale_postgres_backup_policies` | `organization`, `database` | Lists Postgres backup policies for a database. | | `planetscale_postgres_backup_policy` | `organization`, `database`, `id` | Fetches a specific Postgres backup policy by ID. | | `planetscale_postgres_bouncer` | `organization`, `database`, `branch`, `name` | Fetches a specific dedicated PgBouncer by name. | | `planetscale_postgres_bouncers` | `organization`, `database`, `branch` | Lists dedicated PgBouncers for a branch. | | `planetscale_postgres_branch` | `organization`, `database` | Returns the default branch. Does not accept a branch name parameter. | | `planetscale_postgres_branch_backups` | `organization`, `database`, `branch` | Lists Postgres backups for a branch. Supports filters like `state`, `from`, `to`, and `policy`. | | `planetscale_postgres_branch_backup` | `organization`, `database`, `branch`, `id` | Fetches a specific Postgres backup by ID. | | `planetscale_postgres_branch_roles` | `organization`, `database`, `branch` | Lists all roles on a branch. | | `planetscale_postgres_branch_role` | `organization`, `database`, `branch`, `id` | Fetches a specific role by ID. Useful for importing existing roles. | | `planetscale_postgres_read_only_replica` | `organization`, `database`, `branch`, `name` | Fetches a specific dedicated read replica by name. | | `planetscale_postgres_read_only_replicas` | `organization`, `database`, `branch` | Lists dedicated read replicas for a branch. | | `planetscale_postgres_redacted_branch_role` | `organization`, `database`, `branch`, `id` | Fetches a specific role by ID without reading the password into state. | Refer to the [Terraform Registry and provider documentation](https://registry.terraform.io/providers/planetscale/planetscale/latest) for the full, up-to-date list of available resources and data sources. ## Provider configuration The provider supports authentication using [service tokens](/docs/api/reference/service-tokens). Example configuration: ```hcl theme={null} terraform { required_providers { planetscale = { source = "planetscale/planetscale" version = "~> 1.3" } } } provider "planetscale" { # Credentials are read from environment variables by default: # PLANETSCALE_SERVICE_TOKEN_ID # PLANETSCALE_SERVICE_TOKEN # # You can also configure them explicitly (not recommended for production): # service_token_id = "..." # service_token = "..." } ``` | Provider Attribute | Environment Variable | Description | | - | - | - | | `service_token_id` | `PLANETSCALE_SERVICE_TOKEN_ID` | PlanetScale Service Token ID | | `service_token` | `PLANETSCALE_SERVICE_TOKEN` | PlanetScale Service Token | | `server_url` | `PLANETSCALE_SERVER_URL` | Optional server URL override | ## Example usage ### Vitess branch and password ```hcl theme={null} resource "planetscale_vitess_branch" "app" { organization = "my-org" database = "my-vitess-db" name = "app-main" # Optional configuration: # cluster_size = "PS_10" # region = "us-east-1" # Region for the branch # parent_branch = "main" # Fork from an existing branch # backup_id = "..." # Restore from a specific backup # safe_migrations = true # vtgate_size = "VTG_320" # vtgate_count = 2 # vtgate_autoscaling = true # vtgate_max_count = 8 # vtgate_target_cpu_utilization = 50 } resource "planetscale_vitess_branch_password" "app_rw" { organization = "my-org" database = "my-vitess-db" branch = planetscale_vitess_branch.app.name name = "app-rw" role = "admin" # Options: reader, writer, readwriter, admin # Optional configuration: # replica = false # Connect to read replica # direct_vtgate = false # Direct VTGate connection # ttl = 86400 # Credentials expire after N seconds # cidrs = ["10.0.0.0/8"] # IP allowlist } ``` ### Vitess keyspace Creating a `planetscale_vitess_branch` already provisions the branch's default keyspace. Use `planetscale_vitess_keyspace` for additional keyspaces, or import the default keyspace to manage its size or replicas. `cluster_size` and `extra_replicas` update in place. Changing `planetscale_vitess_branch.cluster_size` still replaces the branch. ```hcl theme={null} resource "planetscale_vitess_keyspace" "metrics" { organization = planetscale_vitess_branch.app.organization database = planetscale_vitess_branch.app.database branch = planetscale_vitess_branch.app.name name = "metrics" cluster_size = "PS_10" extra_replicas = 0 } ``` ### Neki branch and role ```hcl theme={null} resource "planetscale_neki_branch" "main" { organization = "my-org" database = "my-neki-db" name = "main" # Optional configuration: # region = "us-east-1" # Region for the branch # parent_branch = "main" # Parent branch; development branches start empty # backup_id = "..." # Restore from a specific backup # restore_point = "2024-01-01T00:00:00Z" # Point-in-time recovery # deletion_protected = true } resource "planetscale_neki_role" "app" { organization = "my-org" database = "my-neki-db" branch = planetscale_neki_branch.main.name name = "app" # Grant permissions by inheriting from built-in Postgres roles inherited_roles = [ "pg_read_all_data", "pg_write_all_data", ] # Optional configuration: # ttl = 86400 # Credentials expire after N seconds # successor = "other-role" # Role to reassign ownership to before dropping # with_replication = false } ``` > [!WARNING] > > **Immutable attributes:** Changing `region`, `parent_branch`, `backup_id`, or `restore_point` on a Neki branch will **destroy and recreate** the branch. Set `deletion_protected = true` or use `lifecycle { prevent_destroy = true }` to protect production branches from accidental replacement. ### Neki routers and cluster configuration Creating a Neki branch already provisions a default configuration profile, shard, router group, admin, and sidecar. Import those objects if Terraform should manage their size or parameters. Use `planetscale_neki_configuration_profile`, `planetscale_neki_shard`, and `planetscale_neki_router` to add more. Clients connect through an additional router group by appending the group name to the username, e.g. `user|analytics`. See [Connect to Neki](/docs/neki/connecting). ```hcl theme={null} resource "planetscale_neki_router" "analytics" { organization = planetscale_neki_branch.main.organization database = planetscale_neki_branch.main.database branch = planetscale_neki_branch.main.name name = "analytics" # Optional configuration: # router_size = "NKR_1" # replicas_per_cell = 1 # autoscaling = true # max_replicas_per_cell = 4 # target_cpu_utilization = 50 } ``` See [Neki cluster configuration](/docs/neki/cluster-configuration) for configuration profiles, shards, admin, routers, and sidecars. ### Postgres branch and role ```hcl theme={null} resource "planetscale_postgres_branch" "main" { organization = "my-org" database = "my-postgres-db" name = "main" # Optional configuration: # cluster_size = "PS_10_AWS_ARM" # region = "us-east-1" # Region for the branch # parent_branch = "main" # Fork from an existing branch # major_version = "18" # PostgreSQL major version # backup_id = "..." # Restore from a specific backup # restore_point = "2024-01-01T00:00:00Z" # Point-in-time recovery # parameters = { # pgconf = { # max_connections = 30 # shared_preload_libraries = "pg_cron" # "cron.max_running_jobs" = "2" # } # } } resource "planetscale_postgres_branch_role" "app_role" { organization = "my-org" database = "my-postgres-db" branch = planetscale_postgres_branch.main.id name = "app_role" # Grant permissions by inheriting from built-in Postgres roles inherited_roles = [ "pg_read_all_data", "pg_write_all_data", ] # Optional configuration: # ttl = 86400 # Credentials expire after N seconds # successor = "other-role" # Role to reassign ownership to before dropping } ``` > [!WARNING] > > **Immutable attributes:** Changing `region`, `parent_branch`, `major_version`, `backup_id`, or `restore_point` will **destroy and recreate** the branch. Use `lifecycle { prevent_destroy = true }` to protect production branches from accidental replacement. ### Backups and backup policies Use backup policy resources to define automatic backup schedules. Use branch backup resources to create managed backups. Vitess, Neki, and Postgres support backup policies. Vitess and Postgres also support branch backup resources. Postgres branch backups also support the `emergency` option. ```hcl theme={null} resource "planetscale_postgres_backup_policy" "daily" { organization = "my-org" database = "my-postgres-db" name = "daily-production" target = "production" frequency_unit = "day" frequency_value = 1 retention_unit = "day" retention_value = 7 schedule_time = "03:00" } resource "planetscale_postgres_branch_backup" "before_release" { organization = "my-org" database = "my-postgres-db" branch = planetscale_postgres_branch.main.name name = "before-release" retention_unit = "day" retention_value = 14 } ``` For Vitess databases, use `planetscale_vitess_backup_policy` and `planetscale_vitess_branch_backup` with the same scheduling and retention fields. For Neki databases, use `planetscale_neki_backup_policy` with the same scheduling and retention fields. ### Dedicated PgBouncers (Postgres) Use `planetscale_postgres_bouncer` to manage a dedicated PgBouncer for a Postgres branch. Clients connect through it by appending the bouncer name to the username, e.g. `postgres.abc123|my-bouncer`. ```hcl theme={null} resource "planetscale_postgres_bouncer" "main" { organization = planetscale_postgres_branch.main.organization database = planetscale_postgres_branch.main.database branch = planetscale_postgres_branch.main.name name = "my-bouncer" target = "primary" # primary, replica, or replica_az_affinity bouncer_size = "PGB_5" # Defaults to PGB_5 replicas_per_cell = 1 # Defaults to 1 parameters = { pgbouncer = { default_pool_size = "30" } } } ``` > [!WARNING] > > Changing `name` or `target` destroys and recreates the bouncer. Omitted `parameters` values are reset to their defaults. ### Postgres dedicated read replicas Use `planetscale_postgres_read_only_replica` to add a dedicated read replica in the same region as the primary or in a different region. The replica can use a different cluster size than the primary. ```hcl theme={null} resource "planetscale_postgres_read_only_replica" "analytics" { organization = planetscale_postgres_branch.main.organization database = planetscale_postgres_branch.main.database branch = planetscale_postgres_branch.main.name name = "analytics" region = "us-west" # Optional configuration: # cluster_size = "PS_10_AWS_ARM" # Defaults to the primary's cluster size # replicas = 1 # Instances serving reads } ``` ### Postgres parameters and extensions Postgres branches support a `parameters` map for cluster settings, PgBouncer settings, Patroni settings, and supported extension settings. Parameters are nested by namespace: `pgconf`, `pgbouncer`, and `patroni`. ```hcl theme={null} resource "planetscale_postgres_branch" "configured" { organization = "my-org" database = "my-postgres-db" name = "configured" parameters = { pgconf = { max_connections = 30 shared_preload_libraries = "pg_cron" "cron.max_running_jobs" = "2" } } } ``` > [!WARNING] > > Omitted values are reset to their defaults. > [!WARNING] > > Some Postgres parameter and extension changes require a restart. See [Postgres parameters](/docs/postgres/cluster-configuration/parameters) and [Postgres extensions](/docs/postgres/extensions) for details. ## Retrieving connection details After creating a role or password, use Terraform outputs to retrieve connection information: ### Vitess ```hcl theme={null} output "vitess_connection" { description = "Vitess connection details" sensitive = true value = { host = planetscale_vitess_branch_password.app_rw.access_host_url username = planetscale_vitess_branch_password.app_rw.username password = planetscale_vitess_branch_password.app_rw.plain_text } } ``` ### Neki ```hcl theme={null} output "neki_connection" { description = "Neki connection details" sensitive = true value = { host = planetscale_neki_role.app.access_host_url username = planetscale_neki_role.app.username password = planetscale_neki_role.app.password database = "postgres" } } ``` Generated Neki connections use the `postgres` logical database by default. See [Connect to Neki](/docs/neki/connecting). ### Postgres ```hcl theme={null} output "postgres_connection" { description = "Postgres connection details" sensitive = true value = { host = planetscale_postgres_branch_role.app_role.access_host_url username = planetscale_postgres_branch_role.app_role.username password = planetscale_postgres_branch_role.app_role.password database = planetscale_postgres_branch_role.app_role.database_name } } ``` > [!NOTE] > > The `password` (Neki and Postgres) and `plain_text` (Vitess) fields are only available after the initial `terraform apply`. They are marked as sensitive and stored in Terraform state. To manage a Postgres role without storing its password in state, use `planetscale_postgres_redacted_branch_role` instead. ## Practical examples ### Development branch workflow Create a development branch forked from your main branch. Optionally specify a larger cluster size if you need more resources than the default. Neki development branches start empty and do not copy the parent branch's schema or data; restore a [backup](/docs/neki/backups) when you need existing data. ```hcl theme={null} resource "planetscale_postgres_branch" "feature_auth" { organization = "my-org" database = "my-db" name = "feature-auth" parent_branch = "main" # cluster_size = "PS_10_AWS_ARM" # Optional: specify a larger cluster size } ``` ### Read-only role for reporting Create a role with read-only access for analytics and reporting. Neki uses the same `inherited_roles` pattern with `planetscale_neki_role`: ```hcl theme={null} resource "planetscale_postgres_branch_role" "reporting" { organization = "my-org" database = "my-db" branch = "main" name = "reporting" inherited_roles = ["pg_read_all_data"] # Read-only access } ``` ### Short-lived CI/CD credentials Create credentials that automatically expire for CI/CD pipelines: ```hcl theme={null} resource "planetscale_postgres_branch_role" "ci_runner" { organization = "my-org" database = "my-db" branch = "main" name = "ci-runner" inherited_roles = ["pg_read_all_data", "pg_write_all_data"] ttl = 3600 # Expires in 1 hour } ``` ### Postgres role without a password in state Use `planetscale_postgres_redacted_branch_role` when Terraform should manage the role but the password should live in your secret manager instead of Terraform state. After creating the role, reset its password through the [reset role API](/docs/api/reference/reset_role) and store the new credential in your secret manager. ```hcl theme={null} resource "planetscale_postgres_redacted_branch_role" "app" { organization = "my-org" database = "my-db" branch = planetscale_postgres_branch.main.id name = "app" inherited_roles = ["pg_read_all_data", "pg_write_all_data"] } ``` ## Configuration reference | Attribute | Valid Values | | - | - | | `cluster_size` | Use the [Clusters API](/docs/api/reference/list_cluster_size_skus#list-available-cluster-sizes) to get the list of available cluster sizes | | `region` | See [available regions](/docs/plans/regions) | | `major_version` (Postgres) | `17`, `18` | | `parameters` (Postgres) | Map of string values nested under `pgconf`, `pgbouncer`, or `patroni` | | `parameters` (Neki) | Map of string values nested by namespace on configuration profiles, admin, and sidecars. Enable extensions in `pgconf.session_preload_libraries`. See [Neki parameters](/docs/neki/cluster-configuration/parameters). | | `bouncer_size` (dedicated PgBouncer) | `PGB_5`, `PGB_10`, `PGB_20`, `PGB_40`, `PGB_80`, `PGB_160` | | `target` (dedicated PgBouncer) | `primary`, `replica`, `replica_az_affinity` | | `router_size` (Neki) | Use `pscale branch router sizes` or [Neki cluster sizing](/docs/neki/cluster-configuration/cluster-sizing) | | `admin_size` (Neki) | `NKA_0`, `NKA_1`, `NKA_2`, `NKA_5`, `NKA_20`, `NKA_40` | | `retention_unit` | `hour`, `day`, `week`, `month`, `year` | | `frequency_unit` | `hour`, `day`, `week`, `month` | | `target` (backup policies) | `production`, `development` | | `role` (Vitess) | `reader`, `writer`, `readwriter`, `admin` | | `inherited_roles` (Neki and Postgres) | `pg_read_all_data`, `pg_write_all_data`, or any custom role | | `deletion_protected` (Neki) | `true`, `false` | ## Deletion protection Branch deletion is one of the most sensitive operations in infrastructure automation. To reduce risk, use Terraform's `lifecycle` block to prevent accidental destruction of critical resources. Neki branches also support a native `deletion_protected` attribute: ```hcl theme={null} resource "planetscale_vitess_branch" "production" { organization = "my-org" database = "my-vitess-db" name = "main" lifecycle { prevent_destroy = true } } resource "planetscale_neki_branch" "production" { organization = "my-org" database = "my-neki-db" name = "main" deletion_protected = true lifecycle { prevent_destroy = true } } ``` You should enforce your own review and approval processes around Terraform `apply`, particularly for changes that delete or recreate branches. ## Upgrading from v0.x to v1.x The original [PlanetScale Terraform provider](https://registry.terraform.io/providers/planetscale/planetscale/0.6.1) was not officially supported for production use and is no longer maintained. Because the new v1 provider is a **breaking rewrite**, migration from v0.x to v1.x is a one-time, manual transition. Projects using the v0 provider will continue to work as normal, but this version will not receive further updates. 1. **Pin v0.x in existing workloads** * Ensure all existing Terraform projects using the PlanetScale provider are pinned to `~> 0.6.1` (or a specific v0.x version) to avoid unintentional upgrades. 2. **Create a new Terraform project for v1.x** * Create a separate directory with a new configuration using the v1 provider. * Use the v1 resource types for Vitess, Neki, and Postgres branches, roles, passwords, and other supported resources. * Run `terraform init` to download the v1 provider and initialize your working directory. This creates a new, independent state file. 3. **Import existing resources into v1.x state** * For each resource that you want Terraform to manage going forward, run `terraform import` for the corresponding v1 resource. * Expect import to use IDs that align with the v1 API design (for example, IDs rather than purely name-based identifiers). 4. **Cut over automation** * After resources are imported and `terraform plan` shows no unexpected changes, switch your automation (CI pipelines, etc.) to apply the v1 project. 5. **Sunset v0.x usage** * Once you have validated v1.x in production, retire the v0.x configurations and keep them only for historical reference, if needed. ## Importing existing resources Resources are imported using JSON-encoded identifiers: ```bash theme={null} # Import a Neki branch terraform import planetscale_neki_branch.main \ '{"organization": "my-org", "database": "my-db", "id": "branch-id"}' # Import a Neki role terraform import planetscale_neki_role.app \ '{"organization": "my-org", "database": "my-db", "branch": "main", "id": "role-id"}' # Import a Neki backup policy terraform import planetscale_neki_backup_policy.daily \ '{"organization": "my-org", "database": "my-db", "id": "backup-policy-id"}' # Import a Neki configuration profile terraform import planetscale_neki_configuration_profile.default \ '{"organization": "my-org", "database": "my-db", "branch": "main", "name": "default"}' # Import a Neki router group terraform import planetscale_neki_router.analytics \ '{"organization": "my-org", "database": "my-db", "branch": "main", "name": "analytics"}' # Import a Neki shard terraform import planetscale_neki_shard.extra \ '{"organization": "my-org", "database": "my-db", "branch": "main", "id": "shard-id"}' # Import a Neki admin terraform import planetscale_neki_admin.main \ '{"organization": "my-org", "database": "my-db", "branch": "main"}' # Import a Neki sidecar terraform import planetscale_neki_sidecar.default \ '{"organization": "my-org", "database": "my-db", "branch": "main", "configuration_profile": "default"}' # Import a Postgres branch terraform import planetscale_postgres_branch.main \ '{"organization": "my-org", "database": "my-db", "id": "branch-id"}' # Import a Postgres backup policy terraform import planetscale_postgres_backup_policy.daily \ '{"organization": "my-org", "database": "my-db", "id": "backup-policy-id"}' # Import a Postgres branch backup terraform import planetscale_postgres_branch_backup.before_release \ '{"organization": "my-org", "database": "my-db", "branch": "main", "id": "backup-id"}' # Import a Postgres branch role terraform import planetscale_postgres_branch_role.app \ '{"organization": "my-org", "database": "my-db", "branch": "main", "id": "role-id"}' # Import a dedicated PgBouncer terraform import planetscale_postgres_bouncer.app \ '{"organization": "my-org", "database": "my-db", "branch": "main", "name": "bouncer-name"}' # Import a Postgres dedicated read replica terraform import planetscale_postgres_read_only_replica.analytics \ '{"organization": "my-org", "database": "my-db", "branch": "main", "name": "analytics"}' # Import a Vitess branch terraform import planetscale_vitess_branch.main \ '{"organization": "my-org", "database": "my-db", "id": "branch-id"}' # Import a Vitess backup policy terraform import planetscale_vitess_backup_policy.daily \ '{"organization": "my-org", "database": "my-db", "id": "backup-policy-id"}' # Import a Vitess branch backup terraform import planetscale_vitess_branch_backup.before_release \ '{"organization": "my-org", "database": "my-db", "branch": "main", "id": "backup-id"}' # Import a Vitess branch password terraform import planetscale_vitess_branch_password.app \ '{"organization": "my-org", "database": "my-db", "branch": "main", "id": "password-id"}' # Import a Vitess keyspace terraform import planetscale_vitess_keyspace.metrics \ '{"organization": "my-org", "database": "my-db", "branch": "main", "name": "metrics"}' ``` You can find resource IDs in the PlanetScale dashboard URL or via the API. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale for Vitess Source: https://planetscale.com/docs/vitess [Vitess](https://vitess.io) is a battle-hardened open source technology invented at YouTube for deploying, scaling, and managing large clusters of database instances. It now powers some of the largest sites on the Internet. PlanetScale for Vitess brings you fully-managed Vitess clusters with unlimited scalability through horizontal sharding. > **[Vitess quickstart](/docs/vitess/tutorials/planetscale-quick-start-guide)** > > Deploy a Vitess database and learn the basics of using PlanetScale. > **[Architecture](/docs/vitess/architecture)** > > Learn about Vitess architecture. > **[Branching](/docs/vitess/schema-changes/branching)** > > Schema changes with branching for safe database development. > **[Import a database](/docs/vitess/imports/database-imports)** > > Import your existing MySQL database to PlanetScale. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Vitess architecture Source: https://planetscale.com/docs/vitess/architecture PlanetScale's Vitess product is designed for reliability, scalability, and developer productivity. ## Overview We achieve these goals through a combination of [MySQL](/docs/vitess/terminology#mysql), [Vitess](/docs/vitess/terminology#vitess), and our own application and ecosystem we have built atop these open-source technologies. There is a great deal of infrastructure that enables our databases to be highly-available, secure, and resilient. In this article, you'll learn about what powers PlanetScale databases and how you can view your database's configuration on our app. ## The infrastructure diagram After creating a PlanetScale account and joining at least one organization, you can create a database. Each new database has a single default [keyspace](/docs/vitess/terminology#keyspace) — a logical database — with the same name as the database. On the dashboard of every PlanetScale database is a diagram outlining the infrastructure that powers the database. ![Architecture diagram for a PlanetScale database](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/diagram.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=9ec6943a8d9e7fa4d258fdc00fc5f8e1) By default, the architecture diagram will show the architecture for the keyspace corresponding to your default branch. Here's how you can tell what keyspace and branch you are viewing the diagram of: ![Architecture diagram for a PlanetScale database](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/diagram-labeled.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=f8dfbba16aeda9c7e58a199f7649e9ae) ### Production branches Production branches are designed for production workloads, and as such are given enough resources to ensure high availability. By default, every production branch has a single primary MySQL instance and two replicas. Each primary also comes with 3 [VTGates](/docs/vitess/terminology#vtgate) across 3 availability zones, which act as proxies for your MySQL instances. These are all pictured in the diagram for a production branch: ![Production branch architecture](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/diagram-production-branch.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=d1688f64961aa7260622d92bfc9f0122) Generally, the application connecting to this database need not be aware of these various components. One exception to this is if you are specifically trying to [send queries to a replica](/docs/vitess/scaling/replicas#how-to-query-replicas). ### Development branches Development branches are specced to enable the development and testing of new features and are not designed for production workloads. When a new development branch is created, a single MySQL node is created along with a VTGate that handles connections to that node. This is reflected in the diagram of a development branch. ![Development branch architecture](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/diagram-dev-branch.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=eb7a0c3d1251f39dc99676a50c42e61b) When you promote a development branch to production status, PlanetScale automatically adds additional replicas and VTGates deployed across multiple availability zones in a given region. ### Read-only regions The primary of your database is the only node that can accept writes, and it resides in a single region. You can add [read-only regions](/docs/vitess/scaling/read-only-regions) to each keyspace on a production branch, which adds replicas in another region that can be used to serve read traffic. This can help reduce read latency for application servers that are distributed around the world. Each read-only region's cluster size and number of replicas are configured per keyspace on the [Clusters page](/docs/vitess/cluster-configuration). Below, you can see our database has the primary and two replicas in `us-east-2` with read-only replicas added in both `us-west-2` and `eu-central-1`. ![Production branch with read-only regions architecture](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/read-only-regions.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=e04cbb60812049bae0cd8e02d6ecfea0) The read-only replicas can be identified by the blue globe icon. ## Infrastructure metrics Each element within the infrastructure diagram for PlanetScale database branches can be selected to display additional metrics related to that element. These metrics are displayed in expandable cards that present themselves when an element is selected. By default, the cards display metrics from the last 6 hours but can be adjusted if additional data is needed. ### VTGates The VTGate node displays the total number of VTGates that exist for a given branch, as well as the number of availability zones in which they live. Selecting the VTGates node will show the following metrics: * Number of connections. * Latency. * Queries received. * CPU. * Memory consumption. ![VTGate metrics](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/vtgate-metrics.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=7f63563d72d7aaed530b17d5146045a4) ### MySQL nodes Each MySQL node in the diagram will display whether it is the primary node or a replica, along with the region where that node is deployed to. Clicking any of the MySQL nodes will display the following metrics: * Database reads and writes for that node. * Queries served. * IOPS. * CPU and Memory utilization. * Storage utilization over the past week. ![Primary MySQL node metrics](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/vttablet-metrics.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=67b38884dca40c084cd994284c7e2ae3) Selecting a replica will display the replication lag in addition to the other metrics. ![Replication lag diagram](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/replication-lag-diagram.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=1336d1234b183ef4dfbcf226c291eb3b) ### Replication lag at a glance Within the infrastructure diagram, you'll also notice that there is a number near the connection points for each replica. These numbers are a way to read the replication lag between the Primary node and that given node at a glance. ![Replication lag](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/replication-lag.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=1470e0d88205aaaca54acd2d59f2b5ce) ### Database shards If your database is [sharded](/docs/vitess/sharding), the infrastructure diagram will represent that as a green stack of shards. ![Stacked shards](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/shard-stack.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=18c7dfaa2566c3329b47bdb6dca445aa) Selecting the stack from the diagram will open a card displaying all of the shards belonging to that keyspace. ![Shard list](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/shard-list.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=9b6ce295f63cf8e865594329aa6a33fe) After selecting a shard, you'll be able to choose to look at metrics for either that shard's primary or one of its replicas. ![Shard list](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/shard-primary-replicas-list.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=1f7ba70b238190e895f0cb0411e6327c) Selecting one will show you the metrics for that specific node in your database architecture. ![Shard](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/shard-primary-metrics.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=95d49bd3c5929730dfaebb759604f1dc) ### Resizing You can use the [Clusters page](/docs/vitess/cluster-configuration) menu to resize your keyspaces. On a sharded keyspace, you can also enable [per-shard sizing](/docs/vitess/cluster-configuration/per-shard-sizing) and resize individual shards. When a resize is in progress, this will be indicated at the top of the diagram. ![Architecture diagram with resize indicator](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/diagram-resize.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=6ec8d02f12cd08efd6d29668a491c57d) Click on "**View**" to see the status for each shard being resized: ![Per-shard resize status](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/architecture/resize-progress.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=06fdecd2db2bf1fdace4f830b3d41f10) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Audit log Source: https://planetscale.com/docs/vitess/audit-log The organization audit log grants [Organization Administrators](/docs/security/access-control#organization-administrator) access to review **actions** performed by individual members of the organization. In addition, each audit log includes **events** detailing who performed the **action** and when it happened. Audit log retainment period is [based on your plan](/docs/planetscale-plans): * **Base plan** — 15 days * **Enterprise** — 15 days > [!NOTE] > > Organization audit log access is limited to [Organization Administrators](/docs/security/access-control#organization-administrator). ## Review your organization audit log You can review your organization audit log under [your PlanetScale **organization** settings](https://app.planetscale.com/~/settings/audit-log). Once there, you can filter the audit log by **Actor** and/or **Action**. Clicking and expanding individual log **event** names empowers you to investigate additional metadata that can provide broader context around what changed. ## Audited organization events You can track the following organization **events** in your PlanetScale account: | PlanetScale organization events | Actions | Database Engine | | :- | :- | :- | | access\_token | created deleted token\_leaked | Vitess, Postgres | | backup\_policy | created deleted | Vitess | | branch\_maintenance\_schedule | created deleted started\_maintenance\_window finished\_maintenance\_window | Vitess | | branch\_maintenance\_window | created deleted | Vitess | | database | created deleted requested\_deletion updated\_default\_branch updated\_migration\_table\_name removed\_member added\_member | Vitess, Postgres | | database\_branch | created deleted enabled\_safe\_migrations disabled\_safe\_migrations enabled\_foreign\_keys disabled\_foreign\_keys enabled\_vectors updated\_cluster\_size updated\_cluster\_config updated\_vtgates updated\_cluster\_update\_strategy | Vitess, Postgres | | database\_branch\_keyspace | created deleted updated requested\_deletion | Vitess | | database\_branch\_password | created deleted password\_leaked updated\_ip\_restrictions | Vitess, Postgres | | database\_branch\_read\_only\_region | created deleted | Vitess | | database\_deploy\_request | created deleted closed | Vitess | | database\_webhook | created deleted | Vitess, Postgres | | deploy\_request\_review | approved | Vitess | | deployment | unqueued | Vitess | | external\_datasource | created deleted | Vitess, Postgres | | integration | created deleted | Vitess, Postgres | | organization | created deleted joined removed\_member left added\_member enabled\_sso disabled\_sso enabled\_sso\_directory disabled\_sso\_directory | Vitess, Postgres | | organization\_invitation | created deleted | Vitess, Postgres | | organization\_membership | created updated\_role | Vitess, Postgres | | organization\_team | created deleted added\_member left removed\_member | Vitess, Postgres | | postgres\_role | created deleted | PostgreSQL | | regional\_failover\_event | created deleted | Vitess, Postgres | | service\_token | created deleted updated\_bulk\_database\_access token\_leaked | Vitess, Postgres | | user | created deleted signed\_in | Vitess, Postgres | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Back up and restore Source: https://planetscale.com/docs/vitess/backups ## Overview PlanetScale provides the ability to create, schedule, and restore backups for production and development database branches. > [!NOTE] > > Our [Base plan](/docs/planetscale-plans) includes automated backups every 12 hours. ## View backups To view backups for all of your branches, go to your database backups page: `app.planetscale.com///backups`. Once there, you'll find additional details about your backup history. ![View backups for your database](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/back-up-and-restore/view-backups.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=d69339fc9344ae5325dfbdfa9ba887ca) ## Create manual backups In addition to the daily default backups that PlanetScale schedules for your database branches, you can create additional **manual** backups. To create a manual backup, follow the steps outlined below: **Step 1** Go to your database backups page: `app.planetscale.com///backups`. **Step 2** Click the **Create new backup** button. This will bring up a pop-up modal that prompts you to pick a **branch** to backup, *name* your backup, and select how long you wish to keep the backup. ![Manual backup pop-up modal](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/back-up-and-restore/create-new-backup.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=f077d4384de73a1057de5dc6736cd28b) **Step 3** Click the **Create backup** button to finish the backup and close the pop-up modal. **Step 4** Manual backups are not free and are denoted by the `$` icon next to their name in the list view on the backups page. ![Manual backups are shown with a \$ sign](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/back-up-and-restore/manual-backup-row.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=4d8b0a32c70a633c877977d5b6014ede) **Step 5** To see the cost associated with storage of a manual backup, click on the backup name. You will see the cost per month on the details page. ![Monthly storage cost for a Manual backup](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/back-up-and-restore/manual-backup-cost.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=4cffe60bd30ece07b0f67f72140bb8f9) ## Schedule backups You can add additional **scheduled backups** for your branches, billed at \$0.023 per GB per month. **Step 1** Go to your database backups page: `app.planetscale.com///backups`. **Step 2** Select the type of branch (`Production branches` | `Development branches` ) you'd like to be backed up by the new backup schedule. ![Schedule backup pop-up modal](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/back-up-and-restore/new-backup-schedule.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=a969c26c114388ca3d745a8b0e4b8e20) **Step 3** Click the **Add new schedule** button. This will bring up a pop-up modal that prompts you to configure backup frequency, select how long you wish to keep the backup, and to *name* your **schedule**. ![Schedule backup pop-up modal](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/back-up-and-restore/new-backup-schedule.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=a969c26c114388ca3d745a8b0e4b8e20) **Step 4** Click the **Save schedule** button to save your new scheduling configurations and to close the pop-up modal. > [!NOTE] > > For additional scheduled backups beyond the included default (every 12 hours for the > [Base plan](/docs/planetscale-plans), you will be billed > **\$0.023 per GB per month**. ## Restore from a backup To restore a backup to a new branch, click on the individual backup to see the option to restore them. **Step 1** Go to your database backups page: `app.planetscale.com///backups`. **Step 2** Select the backup you wish to restore. **Step 3** Click the **Restore backup** button. This will bring up a pop-up modal that prompts you to name your branch and select a cluster size. ![Restore backup pop-up modal](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/backups/restore.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=4c122bc21296c2c62c30788494389819) **Step 4** Click the **Restore backup** button to finish restoring your backup and to close the pop-up modal. **Step 5** To see all branches that are `restored` from a backup, head to the backup details and you'll see a list of those branches. ![Branches restored from a backup](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/back-up-and-restore/restored-branches-list.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=c61762f206947680ff0f03d43d85cd06) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale workflow for Vitess Source: https://planetscale.com/docs/vitess/best-practices ![Diagram showing PlanetScale workflow](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/planetscale-workflow.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=cd739ac76edee56fd3c0130b6edfe32c) PlanetScale databases are designed for developers and developer workflows. Deploy a fully managed database cluster with the reliability of MySQL (our databases run on MySQL 8) and the scale of open source Vitess in just minutes. Deploy, branch, and query your database directly from the UI, download our [CLI](https://github.com/planetscale/cli#installation) and run commands there, or automate your deployments using our [GitHub Actions](/docs/vitess/integrations/github-actions) and [API](/docs/api/reference/getting-started-with-planetscale-api). Built-in connection pooling means you’ll never run into connection limits for your database. ## PlanetScale branching The PlanetScale branching feature allows you to treat your databases like code by creating a branch of your production database schema to serve as an isolated development environment. PlanetScale provides two types of database branches: **development** and **production**. Development branches provide isolated copies of your database schema where you can make changes, experiment, or run CI against. Instantly branch your production database to create a staging environment for testing out your schema changes. Production branches are highly available databases intended for production traffic. They include an additional replica for high availability and are automatically backed up daily. Branches can also have [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled for zero-downtime schema migrations, protection against accidental schema changes, and enhanced team collaboration through [deploy requests](/docs/vitess/schema-changes/deploy-requests). We also offer a [Data Branching®](/docs/vitess/schema-changes/data-branching) feature, which allows you to create an isolated replica of your database for development that includes both the schema **and** data. Learn more about [database branching](/docs/vitess/schema-changes/branching). ## Non-blocking schema changes PlanetScale makes it safe to deploy schema changes to production and easy to automate schema management as a part of your CI/CD process. Schema changes to production branches with safe migrations enabled are applied online and protect against changes that block databases, lock individual tables, or slow down production during the migration. Use a development branch to apply schema changes and view the schema diff in the UI or the CLI. Once you’re satisfied with your schema changes, you can open a deploy request. Learn more about [non-blocking schema changes](/docs/vitess/schema-changes). ## Deploy requests > [!NOTE] > > Your database must have a branch with [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled before you can create a deploy request. [Deploy requests](/docs/vitess/schema-changes/deploy-requests) allow you to propose schema changes and get feedback from your team. The deploy requests display DDL statements (`CREATE`, `ALTER`, and `DROP`) for each table changed, with a line-by-line schema diff, making it easy to review the changes. For example, you can pair deploy requests with GitHub pull requests so that your teammates can review the code and the schema changes in parallel. PlanetScale also analyzes your schema changes for conflicts when you open a deploy request. It checks against the production schema at the time the branch was created and against the current production schema which may have changed in the interim. This ensures your changes can be deployed safely without impacting production. ## Deploy a schema change Once a deploy request has been approved, it gets added to the deploy queue. Schema changes are applied to your production database branch in the order in which they are added to the deploy queue. As a part of the process of adding the schema changes to the deploy queue, PlanetScale analyzes the schema changes in the deploy requests ahead of your deploy request for any potential conflicts. If a conflict exists, your deploy request is rejected from the queue, and you are immediately notified of the conflict. This means you’ll never wait for your schema change to deploy only to learn that there’s an unanticipated conflict. This is especially useful if long running schema changes are being applied. ## Revert a schema change If you ever deploy a schema change, only to realize you need to undo it, PlanetScale can handle that. With our [schema revert feature](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change), you have 30 minutes to "undo" a schema change deployment. Simply click the "Revert changes" button on the affected deploy request page and your production database will instantly revert back to the previous schema. Additionally, any data that was added to your production database in the time between deployment and reverting will be retained. Learn more about this process in our [Revert a schema change section](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change) of our deploy requests documentation. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster configuration Source: https://planetscale.com/docs/vitess/cluster-configuration The Clusters page in your PlanetScale dashboard allows you to configure your PlanetScale cluster. From here, you can: * Adjust the instance sizes for [keyspaces](/docs/vitess/sharding/keyspaces) * Enable [per-shard sizing](/docs/vitess/cluster-configuration/per-shard-sizing) on sharded keyspaces * Create sharded or unsharded keyspaces * Create [external keyspaces](#create-an-external-keyspace) that connect a production branch to an existing MySQL database * Switch between [Metal](/docs/plans/planetscale-skus#metal) and [network-attached storage](/docs/plans/planetscale-skus#network-attached-storage) * Adjust the number of replicas for each keyspace * Add and configure [read-only regions](/docs/vitess/scaling/read-only-regions) for each keyspace * Adjust the VSchema * Adjust the number and size of [VTGates](/docs/vitess/scaling/vtgates) * Turn on [VTGate autoscaling](/docs/vitess/scaling/vtgates#autoscaling-vtgates) * View the CPU and memory utilization of your VTGates * View any changes to your VTGates * Adjust the [memory allocated to vector indexes](/docs/vitess/vectors#resource-requirements) and the InnoDB buffer pool, for any branch with [vectors](/docs/vitess/vectors) enabled * Configure [VReplication settings](#vreplication-settings) * Configure [replication durability constraints](#replication-durability-constraints) * Configure [shard rollout concurrency](#shard-rollout-concurrency) * Enable or disable the [tablet throttler](#throttler) and set its replication lag threshold * Tune [VTGate, VTTablet, and MySQL parameters](/docs/vitess/cluster-configuration/parameters) This documentation will cover how to use everything in this Clusters page. For a full walkthrough with an example of setting up a sharded keyspace, refer to the [Sharding quickstart](/docs/vitess/sharding/sharding-quickstart). If you would like additional support configuring your sharded keyspaces, our [Enterprise plan](/docs/planetscale-plans#enterprise-plan) may be a good fit. [Get in touch](https://planetscale.com/contact) for a quick assessment. ## Adjust your cluster size To adjust your cluster size, first select the [keyspace](/docs/vitess/sharding/keyspaces) you'd like to adjust. If you only have one keyspace, it will be selected by default. Click on the "Cluster size" dropdown. Select the new cluster size, ([Metal](/docs/metal) or network-attached storage). If you select Metal, make sure you select the correct storage size. Finally. click "Save". A network-attached storage cluster (Amazon Elastic Block Storage or Google Persistent Disk) may take a few minutes to change sizes. And a Metal cluster can take hours, depending on the size. You can check the status of the resize from the [database homepage](/docs/vitess/architecture#resizing) in the dashboard, [CLI](/docs/cli/keyspace), or [API](/docs/api/reference/get_keyspace_rollout_status). There will be no downtime or locking during this process. For more information about selecting a cluster size, see the [Cluster sizing documentation](/docs/plans/cluster-sizing). ## Per-shard sizing On a sharded keyspace, enable [per-shard sizing](/docs/vitess/cluster-configuration/per-shard-sizing) from the Shards tab, then set a cluster size on individual shards. Shards you don't set keep the keyspace default. ## Configure read-only regions Each keyspace has a "Read-only regions" section where you can add [read-only regions](/docs/vitess/scaling/read-only-regions) and choose the cluster size and number of replicas for each region. The monthly cost of each region is shown as you configure it. See the [Read-only regions documentation](/docs/vitess/scaling/read-only-regions) for more information. ## Create a keyspace > [!WARNING] > > Misconfiguration to keyspaces can cause availability issues. We recommend thoroughly reading through the documentation in the [Sharding section](/docs/vitess/sharding) of the docs prior to adding additional keyspaces. If you have any questions, please [reach out to our support team](https://planetscale.com/contact?initial=support). > [!WARNING] > > If you are using [Vitess global routing](https://vitess.io/docs/reference/features/global-routing/), you must take extra care when adding a sharded keyspace to your database (for example, if you are using `@primary`). > Before creating one, you must ensure that all tables from your first *unsharded keyspace* are added to the `VSchema` of that *unsharded keyspace*. Eg: > > ``` > { > "tables": { > "users": { } > ... > } > } > ``` > > Otherwise, queries will fail. [Learn more about VSchema.](/docs/vitess/sharding/vschema) > [!NOTE] > > Sharded keyspaces are not currently supported on databases that have foreign key constraints enabled. To create a new [keyspace](/docs/vitess/sharding/keyspaces): **Step 1** Select the database you want to configure. **Step 2** Click "Clusters" in the left nav. **Step 3** You should see the existing unsharded keyspace for your database here. **Step 4** Click "New keyspace". **Step 5** Enter the keyspace name. For example, if your existing unsharded keyspace is named `metal`, you may create a sharded keyspace named `metal-sharded`. **Step 6** Select whether you want to keep it unsharded, or, if not, select the number of shards you want to exist in this keyspace. In most cases, you will be adding a new sharded keyspace. Adding a new unsharded keyspace is not a common use case, but is an option if you're looking to do [vertical sharding](https://planetscale.com/learn/courses/database-scaling/sharding/vertical-sharding). > [!NOTE] > > The cost of adding this additional keyspace largely depends on the number of shards you choose, the cluster size, and if you'd like to add additional replicas. **Step 1** Choose the cluster sizes you would like to use for this keyspace. Keep in mind, if you are creating a sharded keyspace, this will spin up multiple clusters of the selected size. For example, if you are creating 4 shards and choose the `PS-80` cluster size, we will create 4 `PS-80`s, each with 1 primary and 2 replicas. **Step 2** Select the number of *additional* replicas, if any, that you'd like to add to each cluster. Each cluster comes with 2 replicas by default, so any number you choose will be in addition to those 2. **Step 3** Review the new monthly cost for this keyspace below. This is in addition to your existing unsharded keyspace, as well as any other keyspaces you add. **Step 4** Once satisfied, click "Create keyspace". ## Create an external keyspace An external keyspace connects a **production** branch to an existing MySQL database. PlanetScale runs an external tablet against that source so Vitess can read from it. External keyspaces are the source of a [database import](/docs/vitess/imports/database-imports): once the keyspace is ready, a [MoveTables workflow](/docs/cli/move-tables#import-from-an-external-keyspace) copies its tables into a PlanetScale keyspace. **Step 1** Select the database you want to configure, then open a production branch. **Step 2** Click "Clusters" in the left nav. **Step 3** Click "New keyspace" and choose **Create external keyspace**. **Step 4** Enter a keyspace name. This is the Vitess keyspace name, not the remote schema name. **Step 5** Enter the source hostname, port, database name, username, password, and SSL mode. These fields match the [database import](/docs/vitess/imports/database-imports#connection-settings) connection settings. **Step 6** Allow the IP addresses listed on the form in your database's firewall or security group, and make sure the user has the [import user permissions](/docs/vitess/imports/import-tool-user-requirements). **Step 7** PlanetScale checks connectivity, server settings, user grants, and schema compatibility, then creates the keyspace. You can create the same keyspace from the CLI with [`pscale keyspace create-external`](/docs/cli/keyspace#create-an-external-keyspace). Resize an existing external keyspace from this page, or with [`pscale keyspace resize`](/docs/cli/keyspace). To import the data, follow [Database imports](/docs/vitess/imports/database-imports#step-4-start-the-import). ## Modify the VSchema of a keyspace via the Clusters page > [!NOTE] > > You can modify the VSchema on your development branch either in the Clusters page, using the [`ALTER VSCHEMA` command](/docs/vitess/sharding/vschema#modifying-vschema), or with the pscale CLI using [`pscale keyspace vschema update`](/docs/cli/keyspace). Once you have created your keyspace, you will see a new tab: **VSchema**. The VSchema contains information about how the keyspace is sharded, sequence tables, and other Vitess schema information. The VSchema tab allows you to configure the Vschema for your new keyspace or modify it for existing keyspaces. We do not recommend modifying the VSchema directly on your production branch. In fact, it is not possible to do if you have [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled (as recommended). Instead, to modify the VSchema, you should first [create a new development branch](/docs/vitess/schema-changes/branching). Once you have your branch ready, follow these steps: **Step 1** To update the VSchema on the Clusters page, select your new development branch from the dropdown at the top, and then select the keyspace below that has the VSchema you'd like to modify. **Step 2** Next, click the tab labeled "VSchema". **Step 3** Modify the VSchema configuration JSON file as needed. Refer to the [VSchema documentation](/docs/vitess/sharding/vschema) for more information about the available options. **Step 4** When finished, click "Save changes". We will validate your VSchema, and if it is valid, the changes will be saved. If there are errors, we will warn you here to change them before saving. **Step 5** Go back to your "Branches" tab and click on the development branch that you modified. You should see a note on the right that says "Updated VSchema configuration" which lets you know the VSchema(s) for this branch has been modified. **Step 6** From here, go through the normal [deploy request process](/docs/vitess/schema-changes/deploy-requests) to deploy this change to production. Once your change is deployed to production, you can come back to the Clusters page, switch to your production branch, and view the updates to your VSchema. You can also click the "Changes" tab to see information, such as the resize event, status, and start/end time for any previous changes to the VSchema. ## Modify routing rules This configuration setting is currently only available for some Enterprise customers. To modify your routing rules, click "Manage routing rules" on the bottom left of the keyspace configuration panel. Again, you will need to create a new branch to modify routing rules, as described in the "Modify the VSchema of a keyspace" section above. ## Modify keyspace settings There are a number of keyspace-specific settings you can use to customize keyspace behavior. ### Replication durability constraints By default, replication to replica and read-only VTTablets is configured to maximize safety and data integrity. This behavior can be relaxed for performance improvements and reduced replication lag. * **Maximum** — Default setting; use maximum durability constraints. * **Dynamic** — Use maximum durability constraints when replication lag is under 5s, and automatically relax durability constraints when replication lag exceeds 5s. Durability constraints are automatically set back to maximum when replication lag reduces to under 5s. * **Minimum** — Reduce durability constraints. Optimizes for replica and read-only performance, but has highest risk of data loss on crashed instances. ### VReplication settings These settings improve performance during [VReplication](https://vitess.io/docs/reference/vreplication/vreplication/) processes like deploy requests and workflows. * **Optimize inserts** — Enabled by default. When enabled, during binlog replication catch-up, skip sending insert events for rows that have yet to be copied. For more technical details, see the corresponding [Vitess implementation](https://github.com/vitessio/vitess/pull/7708). * **Allow NOBLOB binlog row image** — Enabled by default. When enabled, then we support enabling MySQL’s NOBLOB binlog mode, to omit unchanged BLOB and TEXT columns from replication events, reducing binlog size. For more technical details, see the corresponding [Vitess Implementation](https://github.com/vitessio/vitess/pull/14502). * **Batch binlog statements** — Disabled by default. If enabled, batches binlog statements and transactions to limit the number of round-trips to MySQL. For more technical details, see the corresponding [Vitess implementation](https://github.com/vitessio/vitess/pull/14502). ### Shard rollout concurrency By default, PlanetScale rolls out keyspace configuration changes to one shard at a time. For a sharded keyspace, you can set the maximum number of shards updated concurrently to a value from 1 to 32. Increasing the value can shorten rollouts for keyspaces with many shards, but changes more shards at the same time. View and change the setting with the `pscale` CLI: ```bash theme={null} pscale keyspace settings pscale keyspace update-settings --max-rollout ``` You can also set `max_rollout` with the [update keyspace API endpoint](/docs/api/reference/update_keyspace). Omitting `max_rollout` preserves the current setting. Passing `null` through the API clears the configured value and uses the default of 1. ### Throttler The [Vitess tablet throttler](https://vitess.io/docs/reference/features/tablet-throttler/) pauses schema migrations and VReplication workflows when replication lag rises above a threshold. You can enable or disable it per keyspace, and set the lag threshold, from the same keyspace settings form. * **Enable** — On by default. When enabled, migrations and workflows pause while replication lag is above the threshold. This setting applies across all shards in the keyspace. * **Replication lag threshold** — Seconds of lag that trip the throttler. The default is 5 seconds. You can also change these settings from the CLI with `pscale keyspace update-settings`. See the [keyspace CLI docs](/docs/cli/keyspace). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Disk autoscaling Source: https://planetscale.com/docs/vitess/cluster-configuration/disk-autoscaling Disk autoscaling grows the disks in a Vitess keyspace as they fill up, up to a storage limit. You can also turn growth off, or shrink disks you no longer need. > [!NOTE] > > Disk autoscaling is only for keyspaces on network-attached storage. On [PlanetScale Metal](/docs/metal), storage is tied to > the cluster size, so you [resize the keyspace](/docs/vitess/cluster-configuration) to get more storage. Each keyspace has a disk scaling setting that controls how the disks of its tablets are sized: * **Grow automatically**: disks grow as they fill up, until they reach the storage limit. This is the default for new keyspaces. * **Shrink**: disks are recreated at the size you choose, and automatic growth is then turned off. * **Off**: disks stay at their current size and will not grow as they fill up. > [!NOTE] > > You're billed for the disk that is allocated, not the storage limit. ## Grow automatically When a disk fills up, PlanetScale grows it in place, without replacing the tablet or dropping connections. A disk grows when its usage reaches: * **70%** for disks smaller than 4 TB * **90%** for disks of 4 TB and larger Smaller disks grow by a larger percentage than larger disks. A disk can grow at most once an hour, and cloud providers can limit how often a disk can be resized. If you expect data to grow quickly, for example during an import, raise the storage limit ahead of time. ### Storage limit The storage limit is the maximum size a disk can grow to. Once a disk reaches the limit, it stops growing. The limit must be at least the size of the largest disk in the keyspace, and no more than your organization's storage limit (4 TB by default). To raise the limit above that, [contact support](https://planetscale.com/contact?initial=support). After support raises the limit, it can only be changed by support. ## Shrink Disks grow but don't shrink on their own. After your data gets smaller, for example after you drop a large table, use **Shrink** to stop paying for the disk you no longer need. Cloud providers can't shrink a disk in place, so PlanetScale recreates the disks at the size you choose. Tablets are replaced one at a time, replicas first and the primary last. Each new tablet restores from the latest backup and catches up through replication before the next one is replaced. Before the primary's disk is replaced, the primary role moves to another tablet, which closes existing connections to the primary. Make sure your application can reconnect. The size you choose: * Must be a whole number of GB, and no more than your organization's storage limit * Must be large enough for your data. A shrink is rejected if any disk in the keyspace, including in [read-only regions](/docs/vitess/scaling/read-only-regions), is using more than 80% of the requested size. After the shrink, automatic growth stays off. To let disks grow again, switch back to **Grow automatically**. ## Off With disk scaling off, disks keep their current size. They don't grow, even when they are full. If a disk fills up, MySQL can no longer write to it, so keep an eye on disk usage, or switch back to **Grow automatically**. ## Change the disk scaling setting **Dashboard** **Step 1** Select your database, and click [**Clusters**](/docs/vitess/cluster-configuration) in the left nav **Step 2** Select the keyspace, and open the **Storage** tab **Step 3** Choose **Grow automatically**, **Shrink**, or **Off**, and set the storage limit or the size to shrink to **Step 4** Click **Save changes** **CLI** Use `pscale keyspace update-settings` with `--disk-scaling-strategy`, `--max-storage`, and `--storage`. For example: ```bash theme={null} pscale keyspace update-settings \ --disk-scaling-strategy grow \ --max-storage 1TiB ``` See [Configure disk scaling](/docs/cli/keyspace#configure-disk-scaling) in the CLI reference for more examples. **API** Use the `storage` object on the [update keyspace](/docs/api/reference/update_keyspace) endpoint, with `disk_scaling_strategy` (`grow`, `disable`, or `shrink`), `max_storage_bytes`, and `storage_bytes`. > [!NOTE] > > [Read-only regions](/docs/vitess/scaling/read-only-regions) with network-attached storage have their own disk scaling > setting, which is **Grow automatically** by default. Changing the keyspace's setting doesn't change theirs. Like Metal > keyspaces, read-only regions on Metal don't have disk scaling settings. Shards with a custom size from [per-shard > sizing](/docs/vitess/cluster-configuration/per-shard-sizing) follow the keyspace's setting. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster configuration parameters Source: https://planetscale.com/docs/vitess/cluster-configuration/parameters Tune VTGate, VTTablet, and MySQL settings for your PlanetScale for Vitess cluster. PlanetScale for Vitess lets you adjust a curated set of VTGate, VTTablet, and MySQL parameters directly from the [Clusters page](/docs/vitess/cluster-configuration) in the dashboard. The defaults for each parameter depend on the size of your cluster. They are chosen to optimize performance, resource usage, and connection handling for each cluster size, but you can fine-tune them when your workload calls for it. ## Where parameters are configured | Scope | Component | Where it applies | | - | - | - | | Per branch | **VTGate** | Affects all VTGates that route queries for the branch. | | Per keyspace, per branch | **VTTablet** | Affects every VTTablet (primary and replicas) in the keyspace. | | Per keyspace, per branch | **MySQL (`mysqld`)** | Applied to the MySQL instances backing the keyspace's tablets. | VTGate parameters are set once per branch. VTTablet and MySQL parameters are set per keyspace. ## How to configure parameters You must be a database or organization administrator to modify these settings. **Step 1** From your PlanetScale organization dashboard, select the database you want to configure. **Step 2** Click **Clusters** in the left nav. **Step 3** Select the branch you want to configure from the **Branch** dropdown. **Step 4** Choose what you want to tune: * **VTGate settings**: click **VTGates** at the top of the page, then open the **Configuration** tab. * **VTTablet settings**: select the keyspace, then click the **VTTablets** tab. * **MySQL settings**: select the keyspace, then click the **MySQL** tab. **Step 5** Search for a parameter or scroll through the list. Use the **Category** filters to narrow down. **Step 6** Update the value(s) you want to change. Each parameter shows its default, allowed range, and current override (if any). **Step 7** Click **Queue changes** to add the modifications to a draft batch. **Step 8** When you're ready, click **Apply changes** to submit the batch. PlanetScale rolls the changes out to the relevant components. You can stack changes across VTGate, VTTablets, and MySQL into a single batch before applying. To revert a parameter to its default, clear the override and apply the change. ### Change parameters with the CLI You can also change VTTablet and MySQL parameters with [`pscale keyspace parameters`](/docs/cli/keyspace#change-keyspace-parameters). Prefix each parameter with its namespace, `vttablet` or `mysqld`: ```bash theme={null} pscale keyspace parameters list pscale keyspace parameters set \ --parameters vttablet.vreplication-parallel-insert-workers=4 \ --parameters mysqld.wait_timeout=600 ``` Pass `--reset vttablet.vreplication-parallel-insert-workers` to set a parameter back to its default, and run `pscale keyspace parameters changes list ` to follow a change as it rolls out. To change VTGate parameters, use the dashboard. ## Tracking changes The **Changes** tab on the Clusters page shows the history of parameter modifications: which parameter was changed, the original and updated values, who made the change, the status, and the start and end time of the rollout. > [!NOTE] > > Resizing a cluster will adjust size-dependent defaults automatically, but only for parameters you have **not** overridden. If you have set a parameter explicitly, that override is preserved across cluster size changes. ## Configurable parameters PlanetScale for Vitess exposes the following parameters. Defaults vary by cluster size; values shown in the dashboard reflect the current default for your cluster. ### VTGate (per branch) | Category | Parameter | Description | | - | - | - | | **Query execution** | `query-timeout` | Timeout for queries sent to VTGate, in milliseconds. `0` disables the timeout. | | | `max_memory_rows` | Maximum number of rows held in memory for intermediate query results. | | | `max_payload_size` | Maximum query payload size in bytes. `0` for unlimited. | | | `warn_memory_rows` | Warning threshold for in-memory result rows. | | | `warn_payload_size` | Warning threshold for query payload size, in bytes. | | **Failover buffering** | `buffer_size` | Maximum number of buffered requests during a failover. | | | `buffer_window` | Duration requests are buffered during a failover. | | | `buffer_max_failover_duration` | Maximum total duration to buffer a failover before draining. | | **Replication** | `discovery_low_replication_lag` | Lag threshold below which replicas are considered fully caught up. | | | `discovery_high_replication_lag_minimum_serving` | Minimum number of replicas to keep serving when lag is high. | | | `legacy_replication_lag_algorithm` | Use the legacy algorithm for replication lag detection. | | **Health** | `healthcheck_timeout` | Timeout for VTGate health checks against VTTablets. | | | `min_number_serving_vttablets` | Minimum number of VTTablets that must be serving for the tablet type to be considered healthy. | | **Network** | `grpc_max_message_size` | Maximum gRPC message size, in bytes. | ### VTTablet (per keyspace, per branch) | Category | Parameter | Description | | - | - | - | | **Connection pools** | `queryserver-config-pool-size` | Size of the connection pool for serving queries. | | | `queryserver-config-stream-pool-size` | Size of the pool used for streaming queries. | | | `queryserver-config-idle-timeout` | How long idle connections in the query pool stay open. | | | `queryserver-config-query-pool-timeout` | Maximum time to wait for a query pool connection. | | | `queryserver-config-txpool-timeout` | Maximum time to wait for a transaction pool connection. | | **Transactions** | `queryserver-config-transaction-cap` | Maximum number of concurrent transactions. | | | `queryserver-config-transaction-timeout` | Maximum duration of a single transaction. | | **Query execution** | `queryserver-config-query-timeout` | Maximum query duration before VTTablet kills the query. | | | `queryserver-config-max-result-size` | Maximum number of rows returned by a single query. | | | `queryserver-config-warn-result-size` | Warning threshold for query result size. | | | `queryserver-config-query-cache-memory` | Memory used by the query plan cache. | | **Protection** | `enable_hot_row_protection` | Detect and serialize transactions contending on the same row. | | | `hot_row_protection_concurrent_transactions` | Concurrent transactions allowed per hot row. | | **VReplication** | `vreplication_copy_phase_max_innodb_history_list_length` | Pause copy if the InnoDB history list grows beyond this size. | | | `vreplication_copy_phase_duration` | Maximum duration of a single VReplication copy phase. | | | `vreplication_experimental_flags` | Bitmask of experimental VReplication features to enable. | | | `vreplication_max_time_to_retry_on_error` | Maximum time to retry a failing VReplication stream before giving up. | | | `vreplication-parallel-insert-workers` | Number of parallel insert workers used during VReplication copy. | | **Schema** | `track_schema_versions` | Enable historical tracking of schema versions. | | | `schema-version-max-age-seconds` | Maximum age of tracked schema versions, in seconds. | | **Replication** | `throttle_tablet_types` | Comma-separated tablet types that participate in throttling decisions. | | **Network** | `grpc_max_message_size` | Maximum gRPC message size, in bytes. | ### MySQL (per keyspace, per branch) | Category | Parameter | Description | | - | - | - | | **Connections** | `wait_timeout` | Seconds an idle non-interactive connection stays open. | | | `connect_timeout` | Seconds the server waits for a connect packet before responding with `Bad handshake`. | | **Network** | `max_allowed_packet` | Maximum size of one packet or any generated/intermediate string. | | **Memory** | `join_buffer_size` | Buffer used by joins that do not use indexes. | | | `sort_buffer_size` | Buffer used by sessions that perform sorts. | | | `tmp_table_size` | Maximum size of internal in-memory temporary tables. | | | `max_heap_table_size` | Maximum size of user-created `MEMORY` tables. | | **InnoDB** | `innodb_lock_wait_timeout` | Seconds an InnoDB transaction waits for a row lock. | | | `innodb_buffer_pool_dump_pct` | Percentage of the buffer pool's most recent pages dumped at shutdown. | | | `innodb_io_capacity` | Baseline I/O operations per second available to InnoDB background tasks. | | | `innodb_io_capacity_max` | Maximum I/O operations per second available to InnoDB background tasks. | | | `innodb_adaptive_hash_index` | Enable the InnoDB adaptive hash index (AHI). Can block queries during deploy request cutovers on tables with many foreign key child tables; see [Adaptive hash index](/docs/vitess/foreign-key-constraints#adaptive-hash-index). | | | `innodb_log_buffer_size` | Size of the buffer that holds redo log data before flushing. | | | `innodb_max_purge_lag` | Delay DML when the history list exceeds this length. | | | `innodb_max_purge_lag_delay` | Maximum delay, in microseconds, applied when `innodb_max_purge_lag` is exceeded. | | | `innodb_purge_batch_size` | Number of undo log pages purged in one batch. | | | `innodb_purge_rseg_truncate_frequency` | How often the purge system truncates rollback segments. | | | `innodb_purge_threads` | Number of background purge threads. | | | `innodb_write_io_threads` | Number of I/O threads used for write operations. | | | `innodb_redo_log_capacity` | Total size of the InnoDB redo log. | | **Replication** | `binlog_row_value_options` | Enables partial JSON logging in row-based binlog events. | | | `binlog_row_image` | Controls which columns are included in row-based binlog events (`full`, `minimal`, `noblob`). | | | `replica_parallel_workers` | Number of parallel applier threads on replicas. | | **Transactions** | `transaction_isolation` | Default isolation level for new transactions. | Some MySQL parameters have version-specific defaults (for example, 8.0 vs. 8.4). The dashboard always reflects the value that applies to your branch's MySQL version. ## Configuring parameters via the API and CLI The same parameters are available through the [PlanetScale API](/docs/api). The API uses a draft/submit workflow that mirrors the dashboard: list parameters for a branch, queue one or more changes, then submit the batch to apply them. See the API reference for the available operations under **Vitess parameters** and **Config changes**. The `pscale` CLI does not currently expose commands for tuning these parameters. Use the dashboard or the API. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Per-shard sizing Source: https://planetscale.com/docs/vitess/cluster-configuration/per-shard-sizing Adjust VTTablet sizes for individual shards in a Vitess keyspace Per-shard sizing lets you adjust the VTTablet size of individual shards in a sharded Vitess keyspace instead of resizing the whole keyspace. A shard that takes more traffic can run more CPU and memory than the rest of the keyspace, and on [Metal](/docs/plans/planetscale-skus#metal) it can run more local storage. Enable per-shard sizing from the keyspace Shards tab, then set a size on any shard. Shards you don't set keep the keyspace default. ## Eligibility Per-shard sizing is available on a Vitess keyspace that: * Has two or more shards, all covering equal [key ranges](/docs/vitess/sharding/vindexes). A four-shard keyspace named `-40`, `40-80`, `80-c0`, `c0-` is eligible. One named `-80`, `80-c0`, `c0-`, where the first shard covers half the keyspace and the others cover a quarter each, is not. * Is not imported or external If **Enable per-shard sizing** is not shown on the Shards tab, the keyspace is not eligible. You cannot enable per-shard sizing while a [keyspace resize](/docs/vitess/cluster-configuration#adjust-your-cluster-size) or [deploy request](/docs/vitess/schema-changes/deploy-requests) is in progress. ## Enable per-shard sizing Enabling per-shard sizing is metadata only. Existing key ranges, [VSchema](/docs/vitess/sharding/vschema), routing, and the keyspace default cluster size stay the same until you resize a shard. **Step 1** In the [PlanetScale dashboard](https://app.planetscale.com), open the Vitess database and select the branch. **Step 2** Open the keyspace and go to the **Shards** tab. **Step 3** Click **Enable per-shard sizing**. The dialog confirms the shard count and default size, for example: "This keyspace has N shards with a default size of PS-X. Once enabled you will be able to adjust the sizes of individual shards." **Step 4** Confirm. You can then set sizes on individual shards, or leave shards on the keyspace default. ## Resize a shard After per-shard sizing is enabled, resize a shard the same way you [resize a keyspace](/docs/vitess/cluster-configuration#adjust-your-cluster-size): pick a cluster size and save. This changes the CPU, memory, and, on Metal, the local storage for that shard's primary and replicas. Clearing a shard's size returns it to the keyspace default. All shards in a keyspace must stay on the same storage class: all [Metal](/docs/plans/planetscale-skus#metal) or all [network-attached storage](/docs/plans/planetscale-skus#network-attached-storage). The size picker only lists sizes that match the keyspace storage class. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connection strings Source: https://planetscale.com/docs/vitess/connecting/connection-strings > **Postgres** > > Create **role credentials** with [`pscale role`](/docs/cli/role), then connect using a [connection string](/docs/postgres/connecting/quickstart). > > `pscale connect` is not supported. > **Vitess / MySQL** > > Create a **branch password** with [`pscale password`](/docs/cli/password), then connect with a [connection string](/docs/vitess/connecting/connection-strings) or [`pscale connect`](/docs/cli/connect). ## Creating a password **Step 1** To create a password, head to your database dashboard page at `https://app.planetscale.com//` and click on the "**Connect**" button. **Step 2** On the **Connect page**, select the branch you wanted to create a password for, pick a [password role](/docs/vitess/security/password-roles), and provide a recognizable name for the new credentials. Clicking `Create password` will then generate a **unique username and password pair** that can only be used to access the designated branch of your database. Take note of this password, as you won't be able to see it again. **Step 3** Once created, you can browse the connection string in different framework formats by selecting framework in the "Select your language or framework" section. This will also show you all of the files you need to modify to get connected with PlanetScale in your framework or language of choice. > [!NOTE] > > Passwords support three connection types: `Primary`, `Replica`, and `Read-only region`. `Primary` connects to the primary region. `Replica` routes queries to your branch's replicas and read-only regions (nearest replica). `Read-only region` scopes the password to a specific read-only region and can be created with `pscale password create ... --read-only-region`. [Read more about replicas](/docs/vitess/scaling/replicas) and [read-only regions](/docs/vitess/scaling/read-only-regions). > [!TIP] > > Make sure you copy the credentials for your application and the "Other" format. We do not save the password in plaintext, so there will be no way to retrieve the password after you leave this page. ## Managing passwords Once you've created the password, you can head over to the "**Passwords**" settings page available at `Organization > Database > Settings > Passwords` to manage them. > [!TIP] > > You can also create passwords for branches other than `main` on this page. ![Manage passwords page](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/connection-strings/manage-2.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=880492f6d69d4772c79eac4f05291cfe) Clicking on the `...` icon on the row for your password allows you rename or delete the password. ## Renaming a password Since the **username & password** pair is unique, the only metadata you can edit is the `display name` of the password. ## Deleting a password Deleting a password will invalidate the username & password pair and **disconnect any active clients using this password**. > [!NOTE] > > Any open database connections authenticated with a deleted password will be disconnected within five minutes. ## Native MySQL authentication support Use the tools you're familiar with to connect to PlanetScale databases. PlanetScale supports both [MySQL native authentication](https://dev.mysql.com/doc/refman/8.0/en/native-pluggable-authentication.html), which is widely used to provide a secure connection to MySQL servers, and [MySQL Caching SHA-2 authentication](https://dev.mysql.com/doc/refman/8.0/en/caching-sha2-pluggable-authentication.html), which is the most secure authentication mechanism to connect to MySQL. Based on your application needs and platform support, you can switch between the authentication modes, with the same password. For a list of tested MySQL GUI clients, review our article on [how to connect MySQL GUI applications](/docs/vitess/tutorials/connect-mysql-gui). ## Strong security model PlanetScale Passwords are created for use with a single database branch. This strong security model allows you to generate passwords that are tied to a branch, and cannot access data/schema from another branch. ## IP restrictions You can restrict database connections to specific IP ranges for a single password by updating its IP restrictions. For example, if you have a database for a web application and you create a password for use in the deployed application, you can restrict usage of that specific password to the IP ranges of the deployed application. If somebody attempts to connect to the database from outside of the deployed application, the connection will be refused. IP restrictions work on a per-password basis, so if you want to use the same restriction across passwords, they must be applied to each password separately. Some passwords are incompatible with IP restrictions, and you will need to create a new password to use IP restrictions. Examples of when you may want to use IP restrictions: * You want to segment database access so that the production database can only be connected to from production environments or development branches. * You use a bastion in production and want to ensure that all database connections originate or pass through the bastion. * You want to allow a single client to be able to access your database (e.g., for debugging) and want to provide the least amount of access for them to do so. * You have compliance requirements that require implementing a more stringent access control list in your database. ### Updating the IP restrictions for a password 1. Go to your database's "**Settings**" tab. 2. Click "**Passwords**." 3. You can update the IP restrictions for a password in two different ways: The first way is by opening the dropdown menu to the right of any password on the Passwords page and clicking "**Manage IP restrictions**." The second way is by clicking on the password and scrolling to the bottom of its page to update IP restrictions. 4. Add the IP ranges that you want to allow to connect using the selected password. > [!NOTE] > > If your password has no IP restrictions, it is set to **allow all traffic**. Similarly, when you add a new IP range to the restrictions, all IP addresses out of this range cannot connect to your database using that password. ## Disconnect clients by deleting passwords PlanetScale automatically disconnects clients that are using a deleted password. Head on over to the `Organization > Database > Settings > Passwords` page on your database branch to delete passwords for that branch. It may take up to five minutes for all active clients to be disconnected. ## No plain text password storage PlanetScale only stores hashes and metadata about your database passwords. To add an extra layer of security to your database, we do not store any passwords in plaintext. > [!NOTE] > > In the event that you lose a password, we cannot recover it for you. We recommend creating a new password with the > same access level. ## GitHub Secret Scanning integration All passwords and service tokens generated for use with PlanetScale databases are part of [GitHub's Secret Scanning](https://docs.github.com/en/code-security/secret-security/about-secret-scanning) program. If any database passwords or service tokens are committed in plaintext to any public GitHub repository, we will be notified and take corrective action to delete the access tokens and cut off their access. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Network latency Source: https://planetscale.com/docs/vitess/connecting/network-latency When connecting to PlanetScale, it's important to understand how network latency can impact query speed. ## What is network latency, and why is it important? Network latency is the time it takes for data to travel across a network between your application and your database. Minimizing network latency for databases is critical because it adds additional time to your application's queries. One of the primary causes of network latency is the distance between two endpoints. For example, if an application is hosted in Virginia and is communicating to a database in Paris, each query will spend around 80ms traveling between the two servers. For an application that does many queries, this network time adds up quickly and can greatly impact the application's performance. ### Choosing a region for your database The most optimal location for your database is in the same region and cloud provider as your application. PlanetScale has [regions available in both AWS and GCP](/docs/vitess/regions). If your application is hosted outside of AWS or GCP, then we recommend choosing the PlanetScale region that is geographically closest to your application servers. #### Using `pscale ping` You can use the [`pscale ping`](/docs/cli/ping) command to determine the best PlanetScale region for your application. It will measure the latency to each PlanetScale region and display the results in an ordered list from fastest to slowest. It's best to run this command directly from your application's server to get a realistic measure. ```shell theme={null} NAME LATENCY ENDPOINT TYPE ----------------------------- --------- ------------------------------------------------ ----------- AWS us-west-2 34.6ms aws.connect.psdb.cloud optimized AWS us-west-2 34.8ms us-west.connect.psdb.cloud direct GCP us-central1 57.5ms gcp.connect.psdb.cloud optimized GCP us-central1 57.9ms gcp-us-central1.connect.psdb.cloud direct AWS us-east-2 60.5ms aws-us-east-2.connect.psdb.cloud direct GCP us-east4 69.2ms gcp-us-east4.connect.psdb.cloud direct AWS us-east-1 70.2ms us-east.connect.psdb.cloud direct GCP northamerica-northeast1 80.9ms gcp-northamerica-northeast1.connect.psdb.cloud direct ``` ### Network latency and serverless applications In traditional applications, a single region hosts both the application and database. With these both collocated in the same region, network latency is minimized. For serverless or "edge" deployment models, this can become more complex. In these scenarios, the application is often deployed to several different regions while the primary database remains in a single region. This can result in high network latency between the application and database. When looking at network latency, there are two important dimensions to consider. First, the **distance between the application and PlanetScale's edge network**. And second, the **distance between the edge and the database**. While there is no solution to completely eliminating latency, the path the connection takes over the internet can be optimized for these serverless applications. PlanetScale does this by connecting to the nearest edge location and having the traffic backhauled via PlanetScale's network rather than traversing the public internet. For example, consider an application running in `us-east` that is connecting to a database in `eu-west`. This example is not optimized, and traffic is directed over public internet until it connects to PlanetScale's edge in `eu-west` before being directed to the database. ``` <-------> <-> ``` The following example shows a more optimal traffic path. The application connects to PlanetScale's edge, and then the traffic is sent over PlanetScale's network to the database. This minimizes the time spent traversing the public internet. ``` <-> <------> ``` ## How to tell if network latency is impacting my application When experiencing slow query times, it's important to rule out any potential networking issues. First, you need to be able to measure the time your application is spending waiting on the query. The best sources of this data are application performance monitoring (APM) services or query-level logging from your application. Since applications can be complex and many different factors can influence a performance problem, it's important to isolate and measure only the time spent on the query from the application's perspective. Once you have a measurement of the query in your application, you can then compare the query to the data in [Query Insights](/docs/vitess/monitoring/query-insights). The difference between the numbers can give you an idea of how much time was spent transferring the data across the network. If you see a large difference, it is likely due to network latency between the application and database. If you can access your application's host machine, you can also use `netcat` to understand the latency between the host and PlanetScale. ```shell theme={null} time nc -z aws.connect.psdb.cloud 3306 Connection to aws.connect.psdb.cloud port 3306 [tcp/mysql] succeeded! nc -z aws.connect.psdb.cloud 3306 0.01s user 0.00s system 24% cpu 0.044 total ``` In this example, you can see establishing a connection took 44ms. ## PlanetScale hostnames When connecting to PlanetScale, you have the option of two different hostnames: optimized or direct. For almost every case, we recommend using the optimized hostname. But there are some circumstances where using the direct hostname may work better for your application. > [!NOTE] > > This does not apply to [PlanetScale Managed](/docs/plans/managed) and single-tenant customers who have a single hostname for their account. ## Optimized hostname PlanetScale's optimized hostname uses Route 53's [latency-based](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy-latency.html) routing to connect to the closest edge in PlanetScale's network. Using this hostname optimizes latency and improves reliability as it can route around regional network outages. Examples of optimized hostnames: * `aws.connect.psdb.cloud` * `gcp.connect.psdb.cloud` We recommend everyone use this hostname by default as it provides the best connection in most cases. ## Direct hostname In some rare cases, we have found that PlanetScale's optimized hostname may not direct traffic along the most optimal path. If you are experiencing this, one solution is to use the direct hostname. We only recommend using this if your application and database are in the same region and you have confirmed the optimized hostname is directing traffic incorrectly. To test if the direct hostname is better for your application, you can compare it to the optimized hostname by running `netcat` from your application's host machine. ```shell theme={null} time nc -z us-east.connect.psdb.cloud 3306 Connection to us-east.connect.psdb.cloud port 3306 [tcp/mysql] succeeded! nc -z us-east.connect.psdb.cloud 3306 0.01s user 0.00s system 26% cpu 0.037 total ``` > [!NOTE] > > If you have latency issues with the optimized hostname, we'd appreciate [hearing about your experience](https://planetscale.com/contact?initial=support) so that we can improve. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connecting to PlanetScale privately on AWS Source: https://planetscale.com/docs/vitess/connecting/private-connections ## Connecting to PlanetScale privately via AWS PrivateLink When your compliance mandates that your connections do not route through the public Internet, PlanetScale provides private connection endpoints to AWS regions via [AWS PrivateLink](https://aws.amazon.com/privatelink/). AWS PrivateLink is a form of *VPC peering* that does not send your traffic over the public internet. Private connections are included on the Base plan. There is no additional charge on PlanetScale's end, but this may impact your AWS bill. Below is a list of instructions to set up your Virtual Private Cloud (VPC) to utilize a VPC endpoint when communicating with PlanetScale databases. ## Establishing a VPC endpoint **Step 1** Identify the AWS region that your VPC lives in, which we will refer to as `` for the rest of this document. **Step 2** Navigate to the "Endpoints" section on the VPC page and select "**Create Endpoint**." ![Create a new endpoint](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/private-connections/create-new-endpoint.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=cf911b5cbce910b31409f9095ef45cff) **Step 3** Select "Endpoint services that use NLBs and GWLBs" ![Menu to select endpoint type](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/private-connections/type-of-endpoint.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=d13fb4258236fdea6b103ef1aed9fb59) | AWS Region Name | AWS Region | VPC Endpoint Service Name | | :- | :- | :- | | US East (Ohio) | us-east-2 | `com.amazonaws.vpce.us-east-2.vpce-svc-069f88c102c1a7fba` | | US East (N. Virginia) | us-east-1 | `com.amazonaws.vpce.us-east-1.vpce-svc-02fef31be60d3fd35` | | US West (Oregon) | us-west-2 | `com.amazonaws.vpce.us-west-2.vpce-svc-0f63a383cb2d41919` | | Asia Pacific (Mumbai) | ap-south-1 | `com.amazonaws.vpce.ap-south-1.vpce-svc-06556ed2371c5fdd2` | | Asia Pacific (Singapore) | ap-southeast-1 | `com.amazonaws.vpce.ap-southeast-1.vpce-svc-046d8feae38660302` | | Asia Pacific (Sydney) | ap-southeast-2 | `com.amazonaws.vpce.ap-southeast-2.vpce-svc-03e5578eeaf446c90` | | Asia Pacific (Tokyo) | ap-northeast-1 | `com.amazonaws.vpce.ap-northeast-1.vpce-svc-099c246fa320e54d1` | | Europe (Frankfurt) | eu-central-1 | `com.amazonaws.vpce.eu-central-1.vpce-svc-091260498e58d4dc3` | | Europe (Ireland) | eu-west-1 | `com.amazonaws.vpce.eu-west-1.vpce-svc-049577caa775e8648` | | Europe (London) | eu-west-2 | `com.amazonaws.vpce.eu-west-2.vpce-svc-0f69e183c9a555f03` | | South America (São Paulo) | sa-east-1 | `com.amazonaws.vpce.sa-east-1.vpce-svc-09b11604d399b5c58` | | Canada (Montreal) | ca-central-1 | `com.amazonaws.vpce.ca-central-1.vpce-svc-0617a00ea4e327520` | **Step 4** Fill the "Service name" text box according to which region you want to establish AWS PrivateLink for. Once you have filled in the Service Name text box, click "Verify service". ![Endpoint service name and verification](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/private-connections/service-name-and-verification.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=078bc7b8eb16e49acfec3babb8a0014d) **Step 5** Choose the VPC and subnets that should be peered with the PlanetScale service endpoint. **Step 6** Click the "Additional settings" dropdown arrow to reveal the "DNS name" configuration, and select the "Enable DNS name" checkbox. **Step 7** Choose the security group of your choice to control what can send traffic to the PlanetScale service endpoint. **Step 8** Click "Create endpoint" and verify that the VPC endpoint's status reports "Available" after a few minutes. ## Verifying the connectivity of your VPC endpoint **Step 1** In the AWS UI, confirm that the endpoint has successfully been created by verifying that the Status section of the endpoint reads "Available". > [!WARNING] > > Some PlanetScale regions are named differently than AWS Provider regions. We will refer to the PlanetScale region as `` for the rest of this document. **Step 2** Confirm that the Private DNS Names reads: `.private-connect.psdb.cloud`. **Step 3** Log into any EC2 instance in the configured VPC and run `dig +short .private-connect.psdb.cloud` to confirm that DNS resolution is producing IP Addresses in the range of your VPC's CIDR. ```bash theme={null} dig +short us-east.private-connect.psdb.cloud 172.31.16.197 172.31.13.7 ``` **Step 4** Run `curl https://.private-connect.psdb.cloud` to verify your connectivity. A successful response will yield `Welcome to PlanetScale`. ``` curl https://us-east.private-connect.psdb.cloud Welcome to PlanetScale. ``` ## Modifying your Connection Strings to utilize your VPC endpoint. By default, PlanetScale provides users with a connection string that reads `.connect.psdb.cloud`. To utilize your newly configured VPC endpoint, prepend `private-` to the `connect` subdomain as shown above, yielding a connection string that reads `.private-connect.psdb.cloud`. With this configured, you can leverage VPC peering to communicate between your AWS account and PlanetScale. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connecting to PlanetScale privately on GCP Source: https://planetscale.com/docs/vitess/connecting/private-connections-gcp ## Connecting to PlanetScale privately via GCP Private Service Connect When your compliance mandates that your connections do not route through the public Internet, PlanetScale provides private connection endpoints to GCP regions via [GCP Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect). GCP Private Service Connect is a form of *VPC peering* that keeps your traffic within Google Cloud. Private connections are included on the Base plan. There is no additional charge on PlanetScale's end, but this may impact your GCP bill. Below is a list of instructions to set up your VPC network to utilize a Private Service Connect endpoint when communicating with PlanetScale databases. ## Establishing a Private Service Connect Endpoint **Step 1** Identify the GCP region that your VPC lives in, which we will refer to as `` for the rest of this document. **Step 2** In the GCP console, go to ["Network Service > Private Service Connect"](https://console.cloud.google.com/net-services/psc) page, select the "**Connected endpoints**" tab, and select the "**+ Connect endpoint**" button. **Step 3** Add a Private Service Connect Endpoint with the following details: * **Target**: Published Service. * **Target Service**: Select the target service from the table below for the region you want to establish an endpoint in. * **Name**: Pick any endpoint name. The examples in this document use `"edge"`. * **Network and subnet**: Select the network (VPC) to create the endpoint in. The endpoint will reserve a static IP address in the subnet. The VPC and subnet must be reachable by the applications you intend to connect to your PlanetScale databases from. * **Create an IP Address**: Create a reserved IP address for the endpoint. This is the address your applications will use to access your PlanetScale databases. * **Enable Global Access**: PlanetScale recommends enabling this option. When enabled, this allows applications in other regions of your VPC to reach the PSC endpoint. * Finally, click **Add Endpoint** to start the process. Setup will take approximately 1-2 minutes. ![Setup endpoint details](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/multi/gcp/private-service-connect/connect_endpoint_details.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=f4761a812fda86e6df22a7594c06b456) | GCP Region | Target Service | Domain Name | | :- | :- | :- | | asia-northeast3 | `projects/planetscale-production/regions/asia-northeast3/serviceAttachments/edge-gateway-gcp-asia-northeast3` | gcp-asia-northeast3.private-connect.psdb.cloud | | europe-west1 | `projects/planetscale-production/regions/europe-west1/serviceAttachments/edge-gateway-gcp-europe-west1` | gcp-europe-west1.private-connect.psdb.cloud | | europe-west4 | `projects/planetscale-production/regions/europe-west4/serviceAttachments/edge-gateway-gcp-europe-west4` | gcp-europe-west4.private-connect.psdb.cloud | | northamerica-northeast1 | `projects/planetscale-production/regions/northamerica-northeast1/serviceAttachments/edge-gateway-gcp-northamerica-northeast1` | gcp-northamerica-northeast1.private-connect.psdb.cloud | | us-central1 | `projects/planetscale-production/regions/us-central1/serviceAttachments/edge-gateway-gcp-us-central1` | gcp-us-central1.private-connect.psdb.cloud | | us-east1 | `projects/planetscale-production/regions/us-east1/serviceAttachments/edge-gateway-gcp-us-east1` | gcp-us-east1.private-connect.psdb.cloud | | us-east4 | `projects/planetscale-production/regions/us-east4/serviceAttachments/edge-gateway-gcp-us-east4` | gcp-us-east4.private-connect.psdb.cloud | **Step 4** The endpoint creation process will take a minute or two. When finished, select the endpoint and verify the status is **Accepted**: ![Showing endpoint status as 'Accepted'](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/multi/gcp/private-service-connect/endpoint_status.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=a9eb988c765bee899dc7741b3e4d0943) ## Verifying the connectivity of your Private Service Connect endpoint GCP will automatically create a private Cloud DNS zone in the project where the PSC consumer endpoints are created. The domain name depends on the region the consumer endpoint was created in. Refer to the table above. The format of the domain name will be: * `.` For example, if you chose `edge` as the endpoint name in the `us-central1` region, the domain name for the endpoint would be: * `edge.gcp-us-central1.private-connect.psdb.cloud` **Step 1** Log into any VM instance in the configured VPC and run `dig +short .` to confirm that DNS resolution resolves to the static IP address reserved during endpoint creation. ``` $ dig +short edge.gcp-us-central1.private-connect.psdb.cloud 10.128.0.17 ``` **Step 2** Run `curl https://.` to verify your connectivity. A successful response will yield `Welcome to PlanetScale`. ``` curl https://edge.gcp-us-central1.private-connect.psdb.cloud Welcome to PlanetScale. ``` ## Modifying your Connection Strings to utilize your Private Service Connect endpoint By default, PlanetScale provides connection strings based on the `connect.psdb.cloud` domain name. To access your databases over the private endpoint change your connection string to match the `.` pattern. For example, a connection string such as `gcp-us-central1.connect.psdb.cloud` would be changed to `edge.gcp-us-central1.private-connect.psdb.cloud` assuming `edge` was the Endpoint Name chosen during creation of the endpoint. With this configured, you can leverage VPC peering to communicate between your GCP account and PlanetScale. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connecting to PlanetScale securely Source: https://planetscale.com/docs/vitess/connecting/secure-connections When you are using passwords to connect to PlanetScale, it is essential to correctly validate the server side certificate that PlanetScale provides. If you don't configure it properly, your connection will be vulnerable to man-in-the-middle attacks. The server certificates that PlanetScale uses are signed by a commonly available system root. How to configure this properly depends on the client you are using to connect to MySQL. > [!NOTE] > > We don't guarantee we will continue to sign our certificates using the same CA. We reserve the right to change our CA but guarantee that we will use one that is provided on all common systems by default. ## MySQL command line client With the MySQL command line client, you need to use the `VERIFY_IDENTITY` mode and provide the path to the system roots on your system. The examples below use the most common Linux configuration, but the path might be different in your case. Please check the [CA root configuration](#ca-root-configuration) below to find the appropriate path for your operating system and distribution. ```bash theme={null} mysql --ssl-mode=VERIFY_IDENTITY --ssl-ca=/etc/ssl/certs/ca-certificates.crt ``` ## libmysqlclient based clients Many clients are based on `libmysqlclient`. These clients require configuration options similar to the command line MySQL client, as they are using the same driver to connect to PlanetScale. Please reference the documentation of the driver that you are using for how to configure the SSL CA path. The value provided can be found in the [CA root configuration](#ca-root-configuration). Again, if you don't configure this, things will work but your connection to PlanetScale will be vulnerable to man-in-the-middle attacks. For a list of tested MySQL GUI clients, review our article on [how to connect MySQL GUI applications](/docs/vitess/tutorials/connect-mysql-gui). > [!NOTE] > > Some MySQL clients, like Sequel Pro, do not have full SSL support and will not work with PlanetScale. ## Certificate Authorities ### What is a Certificate Authority (CA)? A [Certificate Authority (CA)](https://en.wikipedia.org/wiki/Certificate_authority) is a trusted party that signs digital certificates used to identify websites and other services. Their most well known use is signing the certificates used by websites serving content over [HTTPS](https://en.wikipedia.org/wiki/HTTPS). Using a signed certificate means that you can trust that you really are communicating with the website that you think you are, without someone else listening in. ### Why is a Certificate Authority (CA) needed for TLS / SSL? Encryption alone won't provide trust, only confidentiality. Without a way to verify the entity at the other end of a secure connection, you risk establishing an encrypted connection to a malicious party. This means that TLS / SSL only provides real security when certificates are validated against specific trusted Certificate Authorities. ### What is a CA root store? In day-to-day browsing you don't need to specify which CAs to trust, your operating system and / or browser come with a list of trusted authorities. These trusted authorities verify the identities of domain owners creating HTTPS certificates, and sign those certificates so that you can trust them as well. The root store of certificates from Certificate Authorities is a collection of certificates that are trusted to sign other certificates. ### Why do I need to care about a CA when connecting to PlanetScale? PlanetScale uses certificates from a [Certificate Authority (CA)](https://en.wikipedia.org/wiki/Certificate_authority) that is part of the system root available on almost all platforms. We do this so that we can provide the easiest possible way for you to connect securely to PlanetScale. No one should be able to listen in on traffic, and no one should be able to impersonate PlanetScale to trick you into connecting to them. Operating systems all come with CA root stores, but not every MySQL driver uses these. Drivers like those for Go, Java or .NET will use the system roots. This means they can provide an experience similar to your browser when connecting to PlanetScale. For these drivers you only have to indicate that you want to use TLS / SSL and that you want to run it in a mode that verifies the identity of PlanetScale. That way you can trust that you are safely connected to PlanetScale when the handshake completes. Many other drivers don't automatically load the system roots. Most notably, the `libmysqlclient` C driver and all other drivers that use it, such as the drivers often used in Ruby (`mysql2`) and Python (`mysqlclient`). This means you will need to specify the path to the CA certificates you want to trust. The best option is to point the driver at your system roots. Below we have listed common paths on which the CA root store can be found in various operating systems and distributions. Only configure this path if your driver absolutely needs it and it won't work without it. If you are using a driver where the system roots are loaded by default, such as Go, Java or .NET, we strongly recommend not configuring this option and depending on the default behavior of your driver. ### CA root configuration #### Linux On Linux, the path to the system CA roots depends on the distribution that you are using. #### Debian / Ubuntu / Gentoo / Arch / Slackware This path also applies to Debian or Ubuntu derivatives. You need to make sure the `ca-certificates` package is installed. ``` /etc/ssl/certs/ca-certificates.crt ``` #### RedHat / Fedora / CentOS / Mageia / Vercel / Netlify This path also applies to RedHat or Fedora derivatives like Amazon Linux and Oracle Linux. This is the path to use for applications deployed on Vercel and Netlify. ``` /etc/pki/tls/certs/ca-bundle.crt ``` #### Alpine This is a commonly used distribution for Docker containers. ``` /etc/ssl/cert.pem ``` #### OpenSUSE This also applies to OpenSUSE derivatives. ``` /etc/ssl/ca-bundle.pem ``` #### MacOS / FreeBSD / OpenBSD MacOS provides an extracted version of the system roots on disk that can be used for the CA roots. On FreeBSD you need to install the `ca_root_nss` package for this path to be available. ``` /etc/ssl/cert.pem ``` #### Windows Windows does not provide a file with the CA roots that can be used by your driver. Many languages often used on Windows like C#, Java or Go do not need the CA root path and will use the Windows internal system roots by default. In those environments, you don't need to specify a root CA list. If you are using a language that requires specifying the CA root path, like C or PHP, the [`curl`](https://curl.se) project provides an extracted bundle of root certificates from the [Mozilla CA Certificate program](https://wiki.mozilla.org/CA). You can download the bundle at [https://curl.se/docs/caextract.html](https://curl.se/docs/caextract.html). Once you download the file, you can point at it with the correct configuration options for the driver that you are using. > [!NOTE] > > We strongly discourage downloading only the current CA that we are using, as PlanetScale reserves the right to change > our CA. We will however always use a CA that is commonly available. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Deleting a database Source: https://planetscale.com/docs/vitess/delete-a-database How to remove a database from PlanetScale. To access the settings for deleting a database: **Step 1** From the PlanetScale organization dashboard, select your database **Step 2** Navigate to **Settings** on the left sidebar menu ## Delete Database The settings page provides the option to permanently delete your database. This action is irreversible and will: * Delete the entire database and all of its branches * Remove all data and backups permanently * Disconnect any applications currently connected to the database * Include usage charges in your invoice through the deletion date > [!WARNING] > > Database deletion cannot be undone. Make sure you have proper backups and that no critical applications depend on this database before proceeding. To delete a database: **Step 1** Scroll to the bottom of the Settings page **Step 2** Click the red **Delete database** button **Step 3** Enter the database name **Step 4** Click **Delete database** to confirm Only organization administrators and database administrators have permission to delete databases. See the [Access Control documentation](/docs/security/access-control) for more information about user permissions. ## IP Restrictions If you would like to 'soft delete' your database first, you can instead enable IP restrictions on the passwords for that database. To enable IP restrictions: **Step 1** From the **Settings** page, select **Passwords** from the list **Step 2** Click the three dots button next to the password you wish to restrict **Step 3** Click the **Manage IP restrictions** button **Step 4** Add in an IP range that is NOT used by any clients and click the **Add** button This will restrict the IP range to what is specified and can be used to test if connection are going to the database via the password in question. This is a useful test for determining if the password in question is still in use, before deleting either the database or the password. You should also confirm in [Insights](/docs/vitess/monitoring/query-insights) that no queries are executing on the database. This helps avoid remaking passwords that are still in use or deleting a database that is still in use. Once you've confirmed that no traffic is using any of your database passwords, you can follow the steps above to delete the database if desired and safe. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Connect Source: https://planetscale.com/docs/vitess/etl PlanetScale Connect is one of the underlying mechanisms we use to transfer data between parties. With PlanetScale Connect, you can integrate with supported ETL platforms to extract data from your PlanetScale database and safely load it into other destinations for analysis, transformation, and more. ## ETL integrations We currently support the following connectors: > **[Airbyte](/docs/vitess/integrations/airbyte)** > **[Debezium](/docs/vitess/integrations/debezium)** > **[Fivetran](/docs/vitess/integrations/fivetran)** > **[Hightouch](/docs/vitess/integrations/hightouch)** > **[Stitch](/docs/vitess/integrations/stitch)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Foreign key constraints Source: https://planetscale.com/docs/vitess/foreign-key-constraints ## What is a foreign key constraint? A **foreign key** is a logical association of rows between two tables in a parent-child relationship. A row in a “parent” table may be referenced by one or more rows in a “child” table. A foreign key typically suggests how you should `JOIN` tables in most queries. A **`FOREIGN KEY` *constraint*** is a database construct, an implementation that *forces* the foreign key relationship's integrity (referential integrity). Namely, it ensures that a child table can only reference a parent table when the appropriate row *exists* in the parent table. A constraint also prevents the existence of “orphaned rows” in different methods. ### Advantages and disadvantages of foreign key constraints Foreign key constraints have advantages and disadvantages. While foreign key constraints can help ensure referential integrity, they will cause degraded performance in high concurrency workloads and introduce more complexity in the database. Often, foreign key constraints become problematic when operating on a large scale. You can read more [why we do not recommend foreign key constraints](/docs/vitess/operating-without-foreign-key-constraints#why-does-planetscale-not-recommend-constraints) for some applications. We recommend weighing the advantages and disadvantages for your specific application when using foreign key constraints. If you decide to enforce referential integrity at the application level instead of at the database level, see our documentation on [how to design systems that maintain referential integrity without foreign key constraints](/docs/vitess/strategies-for-maintaining-referential-integrity). ## Foreign key constraints on PlanetScale ### Prerequisites > [!NOTE] > > You must enable foreign key constraint support on your database in the PlanetScale database settings. Before you enable foreign key constraint support, there are a few important things to know: * **No open deploy requests:** You cannot have any open deploy requests before enable foreign key constraint support. * **Possible orphaned rows on reverts:** Some deploy requests may result in orphaned rows if you revert them. You will be warned before you deploy changes. * **Database upgrades:** When you enable foreign key constraint support, we upgrade the MySQL version in some cases. You should experience no downtime during this upgrade, but it can take a few minutes to complete. Older databases may take longer. ### How to enable foreign key constraints Foreign key constraints can be enabled on a **per database** level, not the organization level. **Step 1** Navigate to the database you want to enable foreign key constraint support for, and select the **“Settings”** tab. **Step 2** Under **"General"**, click the check box next to "Allow foreign key constraints", and click **"Save database settings"**. **Step 3** After enable foreign key constraint support, PlanetScale may upgrade your database in the background, which could take several seconds. As always, database upgrades do not involve any downtime or locking and require no action on your part. ## Database imports with foreign key constraints You can import a database with foreign key constraints to PlanetScale. We will automatically detect them after successfully connecting to your external database. If we find any foreign key constraints, we automatically enable foreign key constraint support and continue the import process. We recommend using a replica as your source when doing database imports with foreign key constraints. Read more in the [database import documentation](/docs/vitess/imports/database-imports). ## Limitations For most cases, foreign key constraints should work as expected in PlanetScale. There are a few cases to be aware of as they are currently unsupported or result in less ideal behavior. ### Sharded versus unsharded environments Currently, the foreign key constraints are only supported in unsharded environments. If you use PlanetScale in a sharded environment, contact your PlanetScale account manager for more information. ### Deploy request limitations #### Referential integrity of existing columns Deploy requests do not validate the referential integrity of *existing* columns. `ALTER TABLE… ADD FOREIGN KEY…` does not validate existing row relations within the context of a deploy request. Unlike standard MySQL, it is possible to add the foreign key constraint to a table with orphaned rows, and they will remain orphaned. In standard MySQL, adding a foreign key is a blocking operation, and it fails if any orphaned rows are found. #### Revert limitations In some cases, a revert of a deploy request can result in orphaned rows. When you revert: * Dropping a foreign key constraint: Once a foreign key constraint is dropped, new data written to the table is less constrained. Reverting this change may result in data that is inconsistent with the dropped foreign key constraint. * Dropping a table with foreign key constraints: When a table with foreign key constraints is dropped, the parent table(s) will continue to be written to. If this change is reverted, data in the table that was dropped may no longer be consistent with its foreign key constraints. #### Adaptive hash index When a deploy request modifies a table that is referenced by foreign key child tables, MySQL obtains a metadata lock and, inside that lock, clears adaptive hash index (AHI) entries for every child table with a foreign key reference to the table being modified. Clearing AHI for each child entails a full scan of the InnoDB buffer pool. On the primary, queries block for the duration of the scan while that metadata lock is held. The same work replays on replicas via the binlog, which can raise replication lag high enough that those replicas stop serving queries. Risk is highest when all of the following are true: * AHI is enabled (`innodb_adaptive_hash_index`) * The modified table has many child tables with foreign key constraints * The InnoDB buffer pool is large To mitigate this risk, consider disabling AHI either permanently, or temporarily before deploying a schema change. Set `innodb_adaptive_hash_index` to `false` in [cluster configuration parameters](/docs/vitess/cluster-configuration/parameters). Many workloads see little or no noticeable query impact with AHI off. ### Unsupported queries The following queries are currently unsupported when the query leads to `child` table `CASCADE` or `SET NULL`: #### `INSERT... ON DUPLICATE KEY UPDATE` statement For example: ```sql theme={null} INSERT INTO tbl (id, col) values (1, 2) ON DUPLICATE KEY UPDATE SET col = 3; ``` #### `REPLACE INTO... SELECT...` statement For example: ```sql theme={null} REPLACE INTO tbl SELECT * FROM tbl2; ``` #### `UPDATE` for a foreign key column statement and referencing the column being updated in the same `UPDATE` statement For example: ```sql theme={null} UPDATE tbl SET col = 5, fk_col = col + 1; ``` The workaround is to separate the updates for conflicting columns into separate schema changes. #### Cyclic foreign keys It is possible to create self-referencing tables as well as a groups of tables which compose a cyclic foreign key (where tables reference each other in a loop). There is no restriction to the schema design, but in some scenarios queries will be rejected: * A cycle where the participating foreign keys all have `ON DELETE CASCADE` rule. Example: ```sql theme={null} CREATE TABLE `employee` ( `id` bigint unsigned NOT NULL AUTO_INCREMENT, `manager_id` bigint unsigned NOT NULL, PRIMARY KEY (`id`), KEY `idx_manager_id` (`manager_id`), CONSTRAINT `self_referencing_key_with_cascade` FOREIGN KEY (`manager_id`) REFERENCES `employee` (`id`) ON DELETE CASCADE ); ``` * A cycle where all rules are either `SET NULL` or `CASCADE`, and the loop ends up having columns reference themselves. Example: ```sql theme={null} CREATE TABLE `t1` ( `id` int NOT NULL, `i` int, PRIMARY KEY (`id`), KEY `i_fk` (`i`), CONSTRAINT `i_fk` FOREIGN KEY (`i`) REFERENCES `t2` (`j`) ON UPDATE CASCADE ON DELETE SET NULL ); CREATE TABLE `t2` ( `id` int NOT NULL, `j` int, PRIMARY KEY (`id`), KEY `j_fk` (`j`), CONSTRAINT `j_fk` FOREIGN KEY (`j`) REFERENCES `t1` (`i`) ON UPDATE SET NULL ON DELETE CASCADE ); ``` #### Foreign key constraint names change on every deployment This is mainly due to the MySQL limitation (compatible with ANSI SQL specification), where constraint names must be unique to the schema. For example, in your database branch, you may run the following: ```sql theme={null} create table parent (id int primary key); create table child (id int primary key, pid int, constraint pchild_fk foreign key (pid) references parent (id)); ``` But after you deploy your schema changes: ```sql theme={null} show create table child \G CREATE TABLE `child` ( `id` int NOT NULL, `pid` int DEFAULT NULL, PRIMARY KEY (`id`), KEY `pchild_fk_5vtaqz7kepok6wa91vryrkrje` (`pid`), CONSTRAINT `pchild_fk_5vtaqz7kepok6wa91vryrkrje` FOREIGN KEY (`pid`) REFERENCES `parent` (`id`) ) ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sending Prometheus Metrics to Datadog Source: https://planetscale.com/docs/vitess/guides/prometheus-metrics-datadog If you're looking for more metrics than PlanetScale's native Datadog integration provides, this tutorial will show how to configure your [Datadog agent](https://docs.datadoghq.com/agent/) to scrape PlanetScale's [Prometheus infrastructure](/docs/vitess/integrations/prometheus) automatically, allowing you to collect detailed metrics for all of your PlanetScale branches. ## Overview In this tutorial, we'll assume that you have a Datadog Agent Version 7 running. For more information on what the Datadog Agent is and how to install it, start with the [Datadog Agent documentation](https://docs.datadoghq.com/getting_started/agent/). For the purposes of this guide, we'll be using a Datadog agent running with the recommended installation steps on a Linux system. ## Prerequisites You'll need a working Datadog agent and access to add a [Custom Agent Check](https://docs.datadoghq.com/developers/custom_checks/) to that instance. This may require `root` or `sudo` access on the machine running the Datadog agent. You'll also need a [Service token](/docs/api/reference/service-tokens) in your Organization, with the `read_metrics_endpoints` permission granted. ## Adding the Plugin to the Datadog Agent Go to [https://github.com/planetscale/planetscale-datadog](https://github.com/planetscale/planetscale-datadog), which is the repository that has our custom OpenMetrics Check. Place the unedited `planetscale.py` in the `checks.d` directory of your Datadog Agent. * On Linux, that is `/etc/datadog-agent/checks.d/` * On macOS, that is `/opt/datadog-agent/etc/checks.d/` Make sure that it belongs to the appropriate user. If you're using the recommended Linux installation steps, it will have created a `dd-agent` user: ``` $ pwd /etc/datadog-agent/checks.d $ ls -al planetscale.py -rw-r--r-- 1 dd-agent dd-agent 9261 Apr 2 22:54 planetscale.py ``` This file is owned by the `dd-agent` user and group in the `/etc/datadog-agent/checks.d` directory. If you're on macOS, it will depend on whether you installed the agent as a 'Single User Agent' or a 'Systemwide Agent'. If you picked Single User, there should be no additional permission changes needed. If you installed it as a Systemwide agent, make sure the user and group you installed the agent with as ownership of the file. ## Configuring the Datadog Agent Now that we have the plugin installed, we need to configure it. In the `conf.d` directory of the Datadog agent take the `conf.d/planetscale.yaml.example` file and edit it with your organization name and Service Token information. It should look like this: ```bash expandable theme={null} instances: - planetscale_organization: 'nick' # Required: Your PlanetScale organization ID ps_service_token_id: '${TOKEN_ID}' # Required: Your PlanetScale Service Token ID ps_service_token_secret: '${TOKEN}' # Required: Your PlanetScale Service Token Secret. Consider using Datadog secrets management: https://docs.datadoghq.com/agent/guide/secrets-management/ namespace: 'planetscale' # Required: Namespace for the metrics metrics: # Required: List of metrics to collect. Use mapping for renaming/type overrides. - planetscale_vtgate_queries_duration: vtgate_query_duration min_collection_interval: 60 send_distribution_buckets: true collect_counters_with_distributions: true ``` This configures the integration to look for all of the branches in the `"nick"` PlanetScale organization, only collect the `planetscale_vtgate_queries_duration` metric, which it will rename `vtgate_query_duration` and put it inside of the `planetscale` namespace. ### Collecting all metrics If you want to collect all available PlanetScale metrics instead of specifying individual metrics, you can use a wildcard pattern: ```yaml theme={null} instances: - planetscale_organization: 'your-org' ps_service_token_id: '${TOKEN_ID}' ps_service_token_secret: '${TOKEN}' namespace: 'planetscale' metrics: [".*"] min_collection_interval: 60 send_distribution_buckets: true collect_counters_with_distributions: true ``` Using `'.*'` in the metrics list tells the Datadog agent to scrape all metrics exposed by PlanetScale. This is useful when you want comprehensive monitoring without manually maintaining a list of specific metrics. Save the file at `planetscale.yaml`, making sure to double check permissions: ``` $ pwd /etc/datadog-agent/conf.d $ ls -al planetscale.yaml -rw-r--r-- 1 root root 1518 Apr 2 22:57 planetscale.yaml ``` ## Restart the Datadog Agent Now that this is configured and installed, restart the Agent: ``` $ sudo systemctl restart datadog-agent ``` ## Validating the PlanetScale Plugin Now that the Datadog Agent is running the PlanetScale plugin, metrics should start flowing into Datadog within a couple of minutes. To validate, we can ask the Datadog Agent: ``` sudo -u dd-agent -- datadog-agent check planetscale ``` If the plugin is installed successfuly, this should output the scrape targets for your branches, as well as metadata about when it was last run and how many metrics were emitted: ```bash expandable theme={null} $ sudo -u dd-agent -- datadog-agent check planetscale === Service Checks === [ { "check": "planetscale.api.can_connect", "host_name": "ubuntu", "timestamp": 1743638192, "status": 0, "message": "", "tags": [ "planetscale_org:nick" ] }, { "check": "planetscale.prometheus.health", "host_name": "ubuntu", "timestamp": 1743638192, "status": 0, "message": "", "tags": [ "endpoint:https://metrics.psdb.cloud/metrics/branch/7wxuxewx4l0p?..." ] }, { "check": "planetscale.prometheus.health", "host_name": "ubuntu", "timestamp": 1743638192, "status": 0, "message": "", "tags": [ "endpoint:https://metrics.psdb.cloud/metrics/branch/6o0rr27785fl?..." ] } ] Running Checks ============== planetscale (unversioned) ------------------------- Instance ID: planetscale:planetscale:8d4d64f696d967be [OK] Configuration Source: file:/etc/datadog-agent/conf.d/planetscale.yaml Total Runs: 1 Metric Samples: Last Run: 0, Total: 0 Events: Last Run: 0, Total: 0 Service Checks: Last Run: 3, Total: 3 Histogram Buckets: Last Run: 77, Total: 77 Average Execution Time : 809ms Last Execution Date : 2025-04-02 23:56:32 UTC (1743638192000) Last Successful Execution Date : 2025-04-02 23:56:32 UTC (1743638192000) Metadata ======== config.hash: planetscale:planetscale:8d4d64f696d967be config.provider: file ``` The Service Checks show that it has successfully connected to the PlanetScale API to request information about how to scrape for the branches in my organization, and it has successfully scraped both of what it discovered. We can also see that it successfully executed at `2025-04-02 23:56:32 UTC` and produced 77 Histogram Buckets. ## Adding Metrics In our earlier configuration, we only added one metric. For a complete list of what PlanetScale exposes, please take a look at our [Metrics Reference Documentation](/docs/vitess/integrations/prometheus-metrics). Note that the Datadog agent [normalizes metrics with certain suffixes starting in v7.32.0](https://github.com/DataDog/integrations-core/blob/master/openmetrics/README.md): > Starting in Datadog Agent v7.32.0, in adherence to the OpenMetrics specification standard, counter names ending in \_total must be specified without the \_total suffix. For example, to collect promhttp\_metric\_handler\_requests\_total, specify the metric name promhttp\_metric\_handler\_requests. This submits to Datadog the metric name appended with .count, promhttp\_metric\_handler\_requests.count. This means that to scrape a metric such as `planetscale_mysql_bytes_received_total`, you would configure the Datadog agent for `planetscale_mysql_bytes_received`. If I want to collect additional metrics, I can add them to the list: ``` metrics: # Required: List of metrics to collect. Use mapping for renaming/type overrides. - planetscale_vtgate_queries_duration: vtgate_query_duration - planetscale_edge_active_connections: active_connections ``` Then, restart the Datadog Agent: ``` $ sudo systemctl restart datadog-agent ``` If I check the status of the PlanetScale Plugin, I can see our last run added a Metric Sample: ```bash theme={null} Running Checks ============== planetscale (unversioned) ------------------------- Instance ID: planetscale:planetscale:fde586b60a54a38f [OK] Configuration Source: file:/etc/datadog-agent/conf.d/planetscale.yaml Total Runs: 1 Metric Samples: Last Run: 1, Total: 1 Events: Last Run: 0, Total: 0 Service Checks: Last Run: 3, Total: 3 Histogram Buckets: Last Run: 77, Total: 77 Average Execution Time : 826ms Last Execution Date : 2025-04-03 00:01:45 UTC (1743638505000) Last Successful Execution Date : 2025-04-03 00:01:45 UTC (1743638505000) ``` In the Datadog UI, I can see data for the `planetscale.active_connections` metric: ![Datadog Connections Metric](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/prometheus-datadog-graph.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=a27906622e857038e5b7e7f6c34c5598) ## What's Next? Now that you're sending a couple of metrics from PlanetScale to Datadog, take a look at our [full list](/docs/vitess/integrations/prometheus-metrics) and start building dashboards! ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Grafana Dashboard for PlanetScale Branches Source: https://planetscale.com/docs/vitess/guides/prometheus-metrics-grafana In this tutorial, you'll learn how to set up Grafana and connect it to a Prometheus instance to see metrics about your PlanetScale database. ## Introduction This guide requires that you've set up a Prometheus instance from our documentation. If you're already running Grafana in production and you're just looking for our standard dashboard template, you can find it [on GitHub](https://github.com/planetscale/grafana-dashboard). ## Install Grafana Grafana's [installation documentation](https://grafana.com/docs/grafana/latest/setup-grafana/installation/) contains information for their supported platforms. For this guide, we'll be setting this up locally on a macOS machine. If you're using a hosted Grafana option such as [Grafana Cloud](https://grafana.com/products/cloud/) or [AWS Managed Grafana](https://aws.amazon.com/grafana/) you can skip this step. On macOS, Grafana is availabile via [homebrew](https://brew.sh/), and I can install it with: ```bash theme={null} $ brew install grafana ``` This will download and install Grafana, and I can start it with: ```bash theme={null} $ brew services start grafana ``` When that succeeds, I can go to `http://localhost:3000/` and I should see the Grafana welcome page: ![Grafana Welcome Page](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-grafana-welcome.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=7eb0e85d512da6800feaaeed72519fe8) The default username and password for a new install is `admin` and `admin`. Grafana will ask you to change the password the first time you log in, please pick something more secure than `admin`. ### Adding a Prometheus Endpoint You can skip this step as well if you're already running a managed Prometheus or have added your datasource to Grafana already. If you're running Prometheus locally, you'll need to add that as a datasource. To do this: **Step 1** Open the menu in the top left and click "Connections" **Step 2** Search for "Prometheus" and pick the plain "Prometheus" option **Step 3** Click "Add new data source" in the top right of the page Now, you should look see a page that looks like this: ![Grafana Add Datasource](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-add-prometheus-connection.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=ae460e439d809b5724a517a615ae9271) You can call this whatever you want, we'll use the following: * Name: "PlanetScale" * Prometheus server URL: `http://localhost:9090/` ![Grafana Prometheus Configuration](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-prometheus-configuration.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=bd74325add64e87dd54c2398069f6c98) Because this is running on your local machine, we do not need to use any Authentication or TLS Scroll down to the "Interval behaviour" section and set the "Scrape interval" to `1m`. Finally, scroll to the bottom and click "Save & test". ## Import the PlanetScale Dashboard Now that we have our datasource added, let's import the PlanetScale Dashboard. This is a starter dashboard that PlanetScale has produced which shows an overview of your branch with the metrics that we expose. From the Grafana homepage, go to the top left menu and pick "Dashboards". In the top right, click "New" and then Import": PlanetScale maintains the latest version of the dashboard located here: [https://github.com/planetscale/grafana-dashboard/blob/main/overview.json](https://github.com/planetscale/grafana-dashboard/blob/main/overview.json) Download this file to your computer, and then click "Upload dashboard JSON file". Find the JSON file you downloaded in the previous step, and configure it with the Prometheus datasource that we added in an earlier: ![Grafana Prometheus Configuration](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-prometheus-configuration.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=bd74325add64e87dd54c2398069f6c98) Click 'Import' and you should be directed to the dashboard, configured to query your local Prometheus with the data it's been scraping! ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sending Prometheus Metrics to New Relic Source: https://planetscale.com/docs/vitess/guides/prometheus-metrics-newrelic If you're looking for your PlanetScale database metrics in your New Relic account, this tutorial will show how to configure a [Prometheus](https://prometheus.io/) instance to scrape PlanetScale's [Prometheus infrastructure](/docs/vitess/integrations/prometheus) automatically, allowing you to collect detailed metrics for all of your PlanetScale branches. While this tutorial is written for New Relic, using Prometheus' remote write is a common pattern for sending metrics to [AWS Managed Prometheus](https://aws.amazon.com/prometheus/), [Google Cloud Managed Service for Prometheus](https://cloud.google.com/stackdriver/docs/managed-prometheus), [Grafana hosted Prometheus](https://grafana.com/products/cloud/metrics/) and many other tools. For more information on Prometheus Remote Write and New Relic, see the [New Relic documentation on sending Prometheus metric data](https://docs.newrelic.com/docs/infrastructure/prometheus-integrations/get-started/send-prometheus-metric-data-new-relic/). ## Overview In this tutorial, we will be using an instance of [Prometheus](https://prometheus.io/) running on a Linux VM to scrape metrics from PlanetScale and then forward them to New Relic using [Remote Write](https://prometheus.io/docs/specs/prw/remote_write_spec/). We will make sure that Prometheus stays running by creating a [Systemd Unit File](https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files). The default configuration we will create will send all PlanetScale metrics to New Relic, and we will cover how to filter to drop certain metrics that may not be desired. In order to proceed, you'll need: * A [New Relic API Key](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/), make sure it is the `Ingest - License` type. * PlanetScale [Service token](/docs/api/reference/service-tokens) with `read_metrics_endpoints` permissions. ## Prometheus Installation First, let's download the latest release of Prometheus and create our user that is going to run it. We'll be using the latest 3.x release from the [GitHub Releases Page](https://github.com/prometheus/prometheus/releases). Create a `prometheus` user: ```bash theme={null} $ sudo useradd -M -U prometheus ``` ```bash theme={null} $ wget https://github.com/prometheus/prometheus/releases/download/v3.2.1/prometheus-3.2.1.linux-amd64.tar.gz $ tar xf prometheus-3.2.1.linux-amd64.tar.gz $ sudo mv prometheus-3.2.1.linux-amd64/ /opt/prometheus $ sudo chown prometheus:prometheus -R /opt/prometheus ``` This has put the Prometheus binary in `/opt/prometheus` along with the example configuration file that we can use. ## Create our Systemd Unit File Now that we have the binary in place, let's setup Systemd to run Prometheus by creating a Unit File in `/etc/systemd/system/prometheus.service` with the following contents: ```ini expandable theme={null} [Unit] Description=Prometheus Agent Documentation=https://prometheus.io/ After=network-online.target [Service] User=prometheus Group=prometheus Restart=on-failure ExecStart=/opt/prometheus/prometheus \ --config.file=/opt/prometheus/prometheus.yml \ --storage.agent.path=/opt/prometheus/data \ --web.listen-address=0.0.0.0:9091 \ --agent [Install] WantedBy=multi-user.target ``` ## Configure Prometheus Now that we've got Prometheus installed and a unit file present, let's configure Prometheus. We will be borrowing some of our configuration from the [Prometheus Guide](/docs/vitess/integrations/prometheus), and adding some New Relic specific configuration. Edit `/opt/prometheus/prometheus.yml` in your editor of choice so that it contains this, making sure to replace your org name, service token information, and New Relic API key: ```yaml theme={null} global: scrape_interval: 1m scrape_configs: - job_name: "${ORG}" http_sd_configs: - url: https://api.planetscale.com/v1/organizations/${ORG}/metrics authorization: type: "token" credentials: "${TOKEN_ID}:${TOKEN}" refresh_interval: 10m remote_write: - url: https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscale authorization: credentials: ${NEW_RELIC_API_TOKEN} ``` This configuration file does the following: * Configures Prometheus to discover scraping endpoints from the PlanetScale API using a service token * Points Prometheus to write the metrics it scrapes from PlanetScale to the New Relic API ## Starting Prometheus Now that we have a Systemd unit file and a configured Prometheus, let's run it! ```bash theme={null} $ sudo systemctl daemon-reload $ sudo systemctl start prometheus.service ``` We can also tell Systemd to run Prometheus when my VM boots: ```bash theme={null} $ sudo systemctl enable prometheus.service ``` Now, let's check to make sure everything is running properly: ```bash expandable theme={null} $ sudo systemctl status prometheus.service ● prometheus.service - Prometheus Agent Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; preset: enabled) Active: active (running) since Fri 2025-04-04 17:36:57 UTC; 38s ago Docs: https://prometheus.io/ Main PID: 745542 (prometheus) Tasks: 9 (limit: 9486) Memory: 21.2M (peak: 21.9M) CPU: 264ms CGroup: /system.slice/prometheus.service └─745542 /opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml --storage.agent.path=/opt/prometheus/data --web.listen-address=0.0.0.0:9091 --agent Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.804Z level=INFO source=main.go:1305 msg=EXT4_SUPER_MAGIC Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.804Z level=INFO source=main.go:1308 msg="Agent WAL storage started" Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.804Z level=INFO source=main.go:1437 msg="Loading configuration file" filename=/opt/prometheus/prometheus.yml Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.805Z level=INFO source=watcher.go:225 msg="Starting WAL watcher" component=remote remote_name=fbe64a url="https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscal> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.805Z level=INFO source=metadata_watcher.go:90 msg="Starting scraped metadata watcher" component=remote remote_name=fbe64a url="https://metric-api.newrelic.com/prometheus/v1/write?prometh> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.805Z level=INFO source=watcher.go:277 msg="Replaying WAL" component=remote remote_name=fbe64a url="https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscale" queu> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.806Z level=INFO source=main.go:1476 msg="updated GOGC" old=100 new=75 Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.806Z level=INFO source=main.go:1486 msg="Completed loading of configuration file" db_storage=791ns remote_storage=610.171µs web_handler=897ns query_engine=301ns scrape=517.175µs scrape_s> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.806Z level=INFO source=main.go:1213 msg="Server is ready to receive web requests." ``` This reports that prometheus is `active (running)`, and I can see the logs showing that it started successfully. Great! ## Querying on New Relic After a couple of minutes, head over to your New Relic dashboard and we can query for your database metrics. First, let's get a list of the database branches in the `nick` organization that I'm using to test: ```bash theme={null} $ pscale branch list test --org nick ID NAME PARENT BRANCH REGION PRODUCTION SAFE MIGRATIONS READY CREATED AT UPDATED AT -------------- ------------ --------------- --------- ------------ ----------------- ------- -------------- ---------------- 7wxuxewx4l0p main n/a us-east Yes No Yes 2 years ago 50 minutes ago 6o0rr27785fl partitions main us-east No No Yes 2 months ago 7 minutes ago ``` For this, we'll use the `7wxuxewx4l0p` branch. Using New Relic's NRQL, we can visualize the memory usage of my VTTablet instances with the following query: ```sql theme={null} FROM Metric SELECT average(planetscale_pods_cpu_util_percentages) WHERE planetscale_database_branch_id = '7wxuxewx4l0p' AND planetscale_component='vttablet' SINCE 30 minutes AGO TIMESERIES FACET planetscale_pod ``` Because my `main` branch is production, we will see the memory usage for my primary and both my replicas over the last 30 minutes: ![New Relic Memory Query](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-new-relic-dashboard.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=5ae095318cdab9644ca82f12f11ad58f) ## Filtering Metrics If you don't want to ingest every metric into New Relic, you can tell Prometheus to drop certain metrics. For more information, see the [New Relic Documentation](https://docs.newrelic.com/docs/infrastructure/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration/#allow-deny). If we adjust our Prometheus configuration that we have in `/opt/prometheus/prometheus.yml` we can instruct Prometheus to drop all metrics unless they match a certain naming convention: ```yaml theme={null} remote_write: - url: https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscale authorization: credentials: ${NEW_RELIC_API_TOKEN} - source_labels: [__name__] regex: "planetscale_pods_(.*)" action: keep ``` If you replace your `remote_write` block with what's above, Prometheus will only forward the timeseries that match the `planetscale_pods_*` name. For a full list of metrics, see our [Metric List](/docs/vitess/integrations/prometheus-metrics). ## What's Next? Now that you have your branch metrics in New Relic, you can create dashboards and alerts for conditions such as high CPU or replication delay. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Handling table and column renames Source: https://planetscale.com/docs/vitess/handling-table-and-column-renames ## Overview In SQL, it is possible to `RENAME` either a column or a table using an [`ALTER` statement](https://dev.mysql.com/doc/refman/8.0/en/alter-table.html). This keeps the data and type of the column or entire table the same and changes only the name. PlanetScale does not support this method of table and column renames. If you do need to rename a column or table, the following section covers how you should do it with PlanetScale using [Deploy Requests](/docs/vitess/schema-changes/deploy-requests). ## How to rename a column on PlanetScale **Step 1** Create a new column with the new name. **Step 2** Update the application to write to both columns with new data. **Step 3** Backfill all the data in the new column for rows that are still missing that information. **Step 4** Optionally, add constraints like `NOT NULL` to the new column once all the data is backfilled. **Step 5** Update the application to only use the new column, and remove any references to the old column name. **Step 6** Drop the old column. This means at least two deploy requests are needed (potentially more if you want to enforce `NOT NULL` without a `DEFAULT`), where you first add the newly named column and then drop the old one. ## Why not support renames? There are two reasons why renames are not supported. The first reason is that PlanetScale uses a declarative model for determining the changes between schemas. This means that there is no way to know for sure if something is a rename or if it was an add and drop column. The second reason is that safely performing a rename requires downtime for your application. ### Declarative model for schema migrations PlanetScale uses a declarative model to determine schema changes. This means that we look at the end state of two branches and compare them to find the difference. When you rename a column, there is no way to know whether there was a rename or if a column was added and dropped in a development branch. It would be possible to apply heuristics to determine if something looks like a rename, but this is not 100% guaranteed to be correct. For example, if a column is renamed and a second column is dropped adjacent to the one that was renamed, it is impossible to determine which column to rename with certainty. Because of this, we only detect simple cases that look like a rename and alert the user on that in a deploy request. While it would be possible to generate a `RENAME` in such a case, we don't know if that is correct, and do not want to execute any schema changes that are not requested. ### Downtime with renames When using a `RENAME`, downtime for your application can only be avoided with significant and complex logic on the application side. While the application is running, suddenly a column disappears and another one appears that contains the same data. This means that the application both needs to be able to handle the state when this happens and know that the data is the same. Usually the only way to handle a column rename is to simultaneously deploy the application, but that is very difficult as it races with the database changes and is not perfectly atomic and synchronized with the schema change. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # How to make different types of schema changes Source: https://planetscale.com/docs/vitess/how-to-make-different-types-of-schema-changes Your database has to grow and change with your application and product. ## Overview So schema changes are inevitable. Luckily, PlanetScale has features like [branching](/docs/vitess/schema-changes/branching), [deploy requests](/docs/vitess/schema-changes/deploy-requests), and [revert](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change) to help give you more confidence when making schema changes. But you still have to consider though how to make the schema change. Some schema changes are more complex than others. This document will help you understand how to make different types of schema changes and associated risks. > [!NOTE] > > Most of the following applies to any relational database and not only to PlanetScale. ## Adding a table Adding a table is a low risk since it does not affect the existing schema. Once you add the table to your schema and deploy the database to production, if you want to backfill the data, now would be the time. Otherwise, you add it to your application code after completing your database deployment. Deploying the database first and then changing your application code to avoid downtime is best. ## Adding a column Adding a column without a `NOT NULL` constraint is a low risk and can be handled similarly to [adding a table](#adding-a-table). The one exception is if you want to add a column with a `NOT NULL` constraint and/or no `DEFAULT` value. The risk increases because the previously inserted rows will be expected to be `NOT NULL` or have no default values; this will cause possible errors in your application. To work around this, you can do the following: **Step 1** Add the new column without defining a DEFAULT value, allowing NULL values. **Step 2** Write a script to backfill the new column for all missing data rows. **Step 3** Add the `NOT NULL` constraint once the data is backfilled. ## Changing a column or table Some examples of changing a column or table include: * Renaming an existing column or table * Changing the data type of an existing column * Splitting and other modifications to the data of an existing column or table This includes modifying your application to use a different column or table in existing code. For example, if you are using `username` in the code but instead want to start using a new `user_id` column in the existing code. Changing a column or table your current application uses is a high risk. It can cause disruptions to users and possible downtime. If you want to change a column or table, we recommend following the expand, migrate, and contract pattern described in the [backward compatible schema changes](https://planetscale.com/blog/backward-compatible-databases-changes) blog post. The blog post walks you through each step in both your database and application code and gives a detailed example. ## Removing a column or table ### In use Removing a column or table your current application uses is a high risk. Dropping a column or table can cause disruptions to users and possible downtime. If you want to remove a column or table in use, we recommend following the expand, migrate, and contract pattern described in the [backward compatible schema changes](https://planetscale.com/blog/backward-compatible-databases-changes) blog post. The blog post walks you through each step in both your database and application code and gives a detailed example. ### Not in use Removing a column or table that is no longer in use by your current application is a low risk. The only danger is potentially needing the removed data in the future. ## Other schema changes This is not an exhaustive list of schema changes. Remember, making smaller incremental schema changes is always safer than larger, more complex ones. If you want to read more on making safer schema changes with PlanetScale, read this blog post on [safely making database schema changes](https://planetscale.com/blog/safely-making-database-schema-changes). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Amazon Aurora migration guide Source: https://planetscale.com/docs/vitess/imports/amazon-aurora-migration-guide ## Overview This document will demonstrate how to migrate a database from Amazon Aurora (MySQL compatible) to PlanetScale. > [!NOTE] > > This guide assumes you are using Amazon Aurora (MySQL compatible) on RDS. If you are using MySQL on Amazon RDS, follow the [Amazon RDS for MySQL migration guide](/docs/vitess/imports/aws-rds-migration-guide). Other database systems (non-MySQL or MariaDB databases) available through RDS will not work with this guide. We recommend reading through the [Database import documentation](/docs/vitess/imports/database-imports) to learn how imports work before proceeding. ## Prerequisites Gather the following information from the AWS Console: * **Database cluster endpoint address** - Located in "**Connectivity & security**" tab (use the regional cluster endpoint, not reader or writer instances) * **Port number** - Typically 3306 * **Master username and password** - Your Aurora root credentials ![The Connectivity & security tab of the database in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/amazon-aurora-migration-guide/the-connectivity-and-security-tab-of-the-database-in-aurora.jpg?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=7643e830fa16b79b18be6f2b327a3094) ## Step 1: Configure server settings Your Aurora database needs specific server settings configured before you can import. Follow these steps to configure GTID mode, binlog format, and sql\_mode. ### Check your current parameter group Your Amazon Aurora database is either using the default DB cluster parameter group (e.g., default.aurora-mysql8.0) or a custom one. You can view it in the "**Configuration**" tab of your regional database cluster (not reader or writer instances). ![The Configuration tab of the database view in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/amazon-aurora-migration-guide/the-configuration-tab-of-the-database-view-in-aurora.jpg?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=54d99c8cca6901211645b5b12fb52a42) ### Configure the parameter group **Step 1** If you are using the default DB cluster parameter group, you'll need to create a new parameter group to reconfigure settings. To create a parameter group, select "**Parameter groups**" from the left nav and then "**Create parameter group**". ![The Parameter groups view in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-parameter-groups-view-in-rds.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=b5e55af884e382a3fbc39bb24f8f0168) Specify the **Parameter group family**, **Type**, **Group name**, and **Description**. All fields are required. * Parameter group family: aurora-mysql8.0 * Type: DB Cluster Parameter Group (Note: Not "DB Parameter Group" type) * Group name: psmigrationgroup (or your choice) * Description: Parameter group for PlanetScale migration You'll be brought back to the list of available parameter groups when you save. **Step 2** Edit the settings in your custom DB cluster parameter group. Select your parameter group from the list. Click "**Edit parameters**" to unlock editing. ![The header of the view when editing a parameter group.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-header-of-the-view-when-editing-a-parameter-group.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=350c20efabe62fc1371216cb61c2d420) Search for "**gtid**" and update: * gtid-mode: ON * enforce\_gtid\_consistency: ON Search for "**sql\_mode**" and update: * sql\_mode: NO\_ZERO\_IN\_DATE,NO\_ZERO\_DATE,ONLY\_FULL\_GROUP\_BY Search for "**binlog\_format**" and update: * binlog\_format: ROW Click "**Save changes**". **Step 3** Associate the DB cluster parameter group to your database. Select "**Databases**" from the left nav, select your regional cluster (not writer or reader instance), and click "**Modify**". Scroll to **Additional configuration** section. Update the **DB cluster parameter group** to your new parameter group. Click "**Continue**". ![The Additional configuration section of the database configuration view.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/amazon-aurora-migration-guide/the-additional-configuration-section-of-the-database-configuration-view.jpg?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=3303c0580b4e6c42a3b0e5629dab3a81) Choose when to apply: * **Apply during the next scheduled maintenance window** - Applied during maintenance window * **Apply immediately** - Applied now, but requires manual reboot Click "**Modify DB instance**". **Step 4** Reboot your database's writer instance to apply the settings. Click "**Actions**" > "**Reboot**". (Make sure you're selecting the writer instance, not the regional cluster.) > [!WARNING] > > This will briefly disconnect active users! The parameter group changes won't take effect without a reboot. Confirm the reboot. You can check the status in the databases list (click refresh to update). ## Step 2: Enable binary logging Binary logging must be enabled for the import to work. On Aurora/RDS, binary logging is tied to automated backups. To enable binary logging, [enable automated backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html#USER_WorkingWithAutomatedBackups.Enabling) by setting the backup retention period to any value greater than zero days. Verify binary logging is enabled: ```sql theme={null} mysql> show variables like 'log_bin'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | log_bin | ON | +---------------+-------+ ``` ## Step 3: Configure binlog retention Set the binary log retention period to ensure logs aren't purged during the import. For most cases, 48 hours is sufficient, but larger imports may need more time. > [!WARNING] > > Longer retention periods use more disk space. Evaluate your binlog size to avoid running out of disk space. Contact [PlanetScale Support](https://planetscale.com/contact?initial=support) if you need assistance. Set the retention period using the `mysql.rds_set_configuration()` procedure: ```sql theme={null} CALL mysql.rds_set_configuration('binlog retention hours', 48); ``` Verify the setting: ```sql theme={null} CALL mysql.rds_show_configuration; ``` Expected output: ``` +------------------------+-------+-----------------------------------------------------------------------------------------------------------+ | name | value | description | +------------------------+-------+-----------------------------------------------------------------------------------------------------------+ | binlog retention hours | 48 | binlog retention hours specifies the duration in hours before binary logs are automatically deleted. | +------------------------+-------+-----------------------------------------------------------------------------------------------------------+ ``` ## Step 4: Ensure database is publicly accessible PlanetScale needs to connect to your Aurora database over the internet. Check that your database is publicly accessible. In the writer instance, go to "**Connectivity & security**" tab. Under "**Security**", check if **Publicly accessible** is set to "Yes". If it says "No", you'll need to modify the database settings to enable public access. If you cannot make the database publicly accessible, [contact us](https://planetscale.com/contact) to discuss alternative import options. ## Step 5: Create a migration user Create a dedicated user with limited privileges for the import process. Connect to your Aurora database using the MySQL command line with your master credentials: ```bash theme={null} mysql -u admin -p -h [your-aurora-endpoint] ``` Run the following script, replacing the placeholders: * `` - Password for the migration\_user account * `` - Name of the database you're importing ```sql theme={null} CREATE USER 'migration_user'@'%' IDENTIFIED BY ''; GRANT PROCESS, REPLICATION SLAVE, REPLICATION CLIENT, RELOAD ON *.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, SHOW VIEW, LOCK TABLES ON ``.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER ON `ps\_import\_%`.* TO 'migration_user'@'%'; GRANT EXECUTE ON PROCEDURE mysql.rds_show_configuration TO 'migration_user'@'%'; GRANT SELECT ON mysql.db TO 'migration_user'@'%'; GRANT SELECT ON mysql.func TO 'migration_user'@'%'; GRANT SELECT ON mysql.innodb_table_stats TO 'migration_user'@'%'; GRANT SELECT ON mysql.tables_priv TO 'migration_user'@'%'; GRANT SELECT ON mysql.user TO 'migration_user'@'%'; GRANT SELECT ON performance_schema.* TO 'migration_user'@'%'; FLUSH PRIVILEGES; ``` Save the username and password securely - you'll need them for the import. ## Step 6: Configure RDS security group Allow PlanetScale to connect by adding PlanetScale's IP addresses to your security group. The specific IP addresses depend on your PlanetScale database region. See [Import public IP addresses](/docs/vitess/imports/import-tool-migration-addresses) for how to find them. ### Add IP addresses to security group 1. Navigate to "**Connectivity & security**" tab of your writer instance 2. Click the VPC security group link ![The Connectivity & security tab of the database view in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/amazon-aurora-migration-guide/the-connectivity-and-security-tab-of-the-database-view-in-aurora.jpg?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=54cc849d916f8168e22379b70aa00d88) 3. Select "**Inbound rules**" tab, then "**Edit inbound rules**" ![The view of security groups associated with the RDS instance.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-view-of-security-groups-associated-with-the-rds-instance.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=a00d564d0c17b2e5c4e1f2c44301927b) 4. Click "**Add rule**" 5. **Type**: Select `MYSQL/Aurora` 6. **Source**: Enter the first PlanetScale IP address (AWS will format it as `x.x.x.x/32`) 7. Repeat for each IP address in your region 8. Click "**Save rules**" ![The Edit inbound rules view where source traffic can be allowed.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-edit-inbound-rules-view-where-source-traffic-can-be-allowed.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=ff9336f2b92a2bb5d54f149f87ae6ff3) ## Importing your database Now that your Aurora database is configured, follow the [Database imports guide](/docs/vitess/imports/database-imports) to complete your import. When you [create the external keyspace](/docs/vitess/imports/database-imports#step-3-create-an-external-keyspace), use the following connection settings: * **Host name** (`--host`) - Your Aurora cluster endpoint address (from Prerequisites) * **Port** (`--port`) - 3306 (or your custom port) * **Database name** (`--source-database`) - The exact database name to import * **Username** (`--username`) - `migration_user` * **Password** (`--password`) - The password you set in Step 5 * **SSL verification mode** (`--ssl-mode`) - Select based on your Aurora SSL configuration The Database imports guide will walk you through: * Creating your PlanetScale database * Connecting to your Aurora database with an external keyspace * Checking your server settings, user grants, and schema * Starting the import with MoveTables * Monitoring the import progress * Switching traffic and completing the import ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # AWS RDS migration guide Source: https://planetscale.com/docs/vitess/imports/aws-rds-migration-guide ## Overview This document will demonstrate how to migrate a database from AWS Relational Database Services (RDS) to PlanetScale. > [!NOTE] > > This guide assumes you are using MySQL on Amazon RDS. If you are using Amazon Aurora (MySQL compatible) on RDS, follow the [Amazon Aurora migration guide](/docs/vitess/imports/amazon-aurora-migration-guide). Other database systems (non-MySQL or MariaDB databases) available through RDS will not work with this guide. We recommend reading through the [Database import documentation](/docs/vitess/imports/database-imports) to learn how imports work before proceeding. ## Prerequisites Gather the following information from the AWS Console: * **Database endpoint address** - Located in "**Connectivity & security**" tab of your database instance * **Port number** - Typically 3306 * **Master username and password** - Your RDS root credentials ![The Connectivity & security tab of the database in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-connectivity-and-security-tab-of-the-database-in-rds.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=20474788ee59661d90250f15bb6f0be3) ## Step 1: Configure server settings Your RDS database needs specific server settings configured before you can import. Follow these steps to configure GTID mode, binlog format, and sql\_mode. ### Check your current parameter group Your Amazon RDS database is either using the default DB parameter group (e.g., default.mysql8.0) or a custom one. You can view it in the "**Configuration**" tab of your database instance. ![The Configuration tab of the database view in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-configuration-tab-of-the-database-view-in-rds.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=a30ebea8e2762b69a3b7e33e1fba3b8f) ### Configure the parameter group **Step 1** If you are using the default DB parameter group, you'll need to create a new parameter group to reconfigure settings. To create a parameter group, select "**Parameter groups**" from the left nav and then "**Create parameter group**". ![The Parameter groups view in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-parameter-groups-view-in-rds.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=b5e55af884e382a3fbc39bb24f8f0168) Specify the **Parameter group family**, **Type**, **Group name**, and **Description**. All fields are required. * Parameter group family: mysql8.0 (or your MySQL version) * Type: DB Parameter Group (Note: Not "DB Cluster Parameter Group" type) * Group name: psmigrationgroup (or your choice) * Description: Parameter group for PlanetScale migration ![The form used to create a parameter group.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-form-used-to-create-a-parameter-group.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=3476446fe5a00f8ef0bfd38e9525f57c) You'll be brought back to the list of available parameter groups when you save. **Step 2** Edit the settings in your custom DB parameter group. Select your parameter group from the list. ![The list of parameter groups in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-list-of-parameter-groups-in-rds.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=15cfa3fa50d42549371c6d1d341a8e03) Click "**Edit parameters**" to unlock editing. ![The header of the view when editing a parameter group.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-header-of-the-view-when-editing-a-parameter-group.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=350c20efabe62fc1371216cb61c2d420) Search for "**binlog\_format**" and update: * binlog\_format: ROW ![The binlog\_format configuration required.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-binlog_format-configuration-required.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=d9508a7c11a68b30486c33fb1d1c57dc) Search for "**gtid**" and update: * gtid-mode: ON * enforce\_gtid\_consistency: ON Search for "**sql\_mode**" and update: * sql\_mode: NO\_ZERO\_IN\_DATE,NO\_ZERO\_DATE,ONLY\_FULL\_GROUP\_BY ![The GTID configurations that are required.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-gtid-configurations-that-are-required.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=12cbd3d9ddd38a10b5fbdf4e43b341c8) Click "**Save changes**". **Step 3** Associate the parameter group to your database. Select "**Databases**" from the left nav, check the **select box** next to your database instance, and click "**Modify**". ![The list of databases in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-list-of-databases-in-rds.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=6dfc1a2b2b77a6babd2621aec8647c10) Scroll to **Additional configuration** section. Update the **DB parameter group** to your new parameter group. Click "**Continue**". ![The Additional configuration section of the database configuration view.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-additional-configuration-section-of-the-database-configuration-view.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=ed9606c46e59229a04fc7e5d2ec5b455) Choose when to apply: * **Apply during the next scheduled maintenance window** - Applied during maintenance window * **Apply immediately** - Applied now, but requires manual reboot ![The confirmation view that is displayed when modifying RDS database settings.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-confirmation-view-that-is-displayed-when-modifying-rds-database-settings.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=80c6516675d0491dc1469a18ff8434a2) Click "**Modify DB instance**". **Step 4** Reboot your database instance to apply the settings. Click "**Actions**" > "**Reboot**". > [!WARNING] > > This will briefly disconnect active users! The parameter group changes won't take effect without a reboot. You'll be presented with a page to confirm the database you want to reboot. Click "**Confirm**" and the database will begin rebooting. If you opted to apply changes immediately, monitor the status on the "**Configuration**" tab. The page does not automatically update, so refresh to check status. ![The configuration tab with the new parameter group applying.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-configuration-tab-with-the-new-parameter-group-applying.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=eb449d643c5de82d594b27ab26c0f64a) ## Step 2: Enable binary logging Binary logging must be enabled for the import to work. On RDS, binary logging is tied to automated backups. To enable binary logging, [enable automated backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html#USER_WorkingWithAutomatedBackups.Enabling) by setting the backup retention period to any value greater than zero days. Verify binary logging is enabled: ```sql theme={null} mysql> show variables like 'log_bin'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | log_bin | ON | +---------------+-------+ ``` ## Step 3: Configure binlog retention Set the binary log retention period to ensure logs aren't purged during the import. For most cases, 48 hours is sufficient, but larger imports may need more time. > [!WARNING] > > Longer retention periods use more disk space. Evaluate your binlog size to avoid running out of disk space. Contact [PlanetScale Support](https://planetscale.com/contact?initial=support) if you need assistance. Set the retention period using the `mysql.rds_set_configuration()` procedure: ```sql theme={null} CALL mysql.rds_set_configuration('binlog retention hours', 48); ``` Verify the setting: ```sql theme={null} CALL mysql.rds_show_configuration; ``` Expected output: ``` +------------------------+-------+-----------------------------------------------------------------------------------------------------------+ | name | value | description | +------------------------+-------+-----------------------------------------------------------------------------------------------------------+ | binlog retention hours | 48 | binlog retention hours specifies the duration in hours before binary logs are automatically deleted. | +------------------------+-------+-----------------------------------------------------------------------------------------------------------+ ``` ## Step 4: Ensure database is publicly accessible PlanetScale needs to connect to your RDS database over the internet. Check that your database is publicly accessible. In your database instance, go to "**Connectivity & security**" tab. Under "**Security**", check if **Publicly accessible** is set to "Yes". If it says "No", you'll need to modify the database settings to enable public access. If you cannot make the database publicly accessible, [contact us](https://planetscale.com/contact) to discuss alternative import options. ## Step 5: Create a migration user Create a dedicated user with limited privileges for the import process. Connect to your RDS database using the MySQL command line with your master credentials: ```bash theme={null} mysql -u admin -p -h [your-rds-endpoint] ``` Run the following script, replacing the placeholders: * `` - Password for the migration\_user account * `` - Name of the database you're importing ```sql theme={null} CREATE USER 'migration_user'@'%' IDENTIFIED BY ''; GRANT PROCESS, REPLICATION SLAVE, REPLICATION CLIENT, RELOAD ON *.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, SHOW VIEW, LOCK TABLES ON ``.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER ON `ps\_import\_%`.* TO 'migration_user'@'%'; GRANT EXECUTE ON PROCEDURE mysql.rds_show_configuration TO 'migration_user'@'%'; GRANT SELECT ON mysql.db TO 'migration_user'@'%'; GRANT SELECT ON mysql.func TO 'migration_user'@'%'; GRANT SELECT ON mysql.innodb_table_stats TO 'migration_user'@'%'; GRANT SELECT ON mysql.tables_priv TO 'migration_user'@'%'; GRANT SELECT ON mysql.user TO 'migration_user'@'%'; GRANT SELECT ON performance_schema.* TO 'migration_user'@'%'; FLUSH PRIVILEGES; ``` Save the username and password securely - you'll need them for the import. ## Step 6: Configure RDS security group Allow PlanetScale to connect by adding PlanetScale's IP addresses to your security group. The specific IP addresses depend on your PlanetScale database region. See [Import public IP addresses](/docs/vitess/imports/import-tool-migration-addresses) for how to find them. ### Add IP addresses to security group Navigate to "**Connectivity & security**" tab of your database instance and click the VPC security group link. ![The Connectivity & security tab of the database view in RDS.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-connectivity-and-security-tab-of-the-database-view-in-rds.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=68e1c754e4f5d7f5632443582b845362) Select "**Inbound rules**" tab, then "**Edit inbound rules**". ![The view of security groups associated with the RDS instance.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-view-of-security-groups-associated-with-the-rds-instance.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=a00d564d0c17b2e5c4e1f2c44301927b) Click "**Add rule**", then: * **Type**: Select `MYSQL/Aurora` * **Source**: Enter the first PlanetScale IP address (AWS will format it as `x.x.x.x/32`) Repeat for each IP address in your region, then click "**Save rules**". ![The Edit inbound rules view where source traffic can be allowed.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/aws-rds-migration-guide/the-edit-inbound-rules-view-where-source-traffic-can-be-allowed.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=ff9336f2b92a2bb5d54f149f87ae6ff3) ## Importing your database Now that your RDS database is configured, follow the [Database imports guide](/docs/vitess/imports/database-imports) to complete your import. When you [create the external keyspace](/docs/vitess/imports/database-imports#step-3-create-an-external-keyspace), use the following connection settings: * **Host name** (`--host`) - Your RDS endpoint address (from Prerequisites) * **Port** (`--port`) - 3306 (or your custom port) * **Database name** (`--source-database`) - The exact database name to import * **Username** (`--username`) - `migration_user` * **Password** (`--password`) - The password you set in Step 5 * **SSL verification mode** (`--ssl-mode`) - Select based on your RDS SSL configuration The Database imports guide will walk you through: * Creating your PlanetScale database * Connecting to your RDS database with an external keyspace * Checking your server settings, user grants, and schema * Starting the import with MoveTables * Monitoring the import progress * Switching traffic and completing the import ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Azure Database for MySQL migration guide Source: https://planetscale.com/docs/vitess/imports/azure-database-for-mysql-migration-guide ## Overview This document will demonstrate how to migrate a database from Azure Database for MySQL to PlanetScale. We recommend reading through the [Database import documentation](/docs/vitess/imports/database-imports) to learn how imports work before proceeding. ## Prerequisites Before you can perform the migration, you’ll need to gather the following information from you MySQL instance in Azure: * Server name * Server admin login name * Server admin password * Database name The server name and admin login name can be located on the **Overview** tab of the MySQL instance in Azure. ![The server name and server admin login name located in the Azure dashboard.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/azure-database-for-mysql-migration-guide/the-server-name-and-server-admin-login-name-located-in-the-azure-dashboard.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=a2a57e5a657f72caa1c47b35b50c0a88) The server admin password is the same password you set when initially creating the database instance. To view your available databases, select the **Databases** tab from the sidebar. ![The databases tab of the Azure dashboard.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/azure-database-for-mysql-migration-guide/the-databases-tab-of-the-azure-dashboard.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=17320c7c4a9ce343dbac48c30f9e86ad) ## Configure firewall rules In order for PlanetScale to connect to your Azure database, you must allow traffic into the database through the associated security group. The specific IP addresses you will need to allow depend on the region you plan to host your PlanetScale database. Check the [Import public IP addresses page](/docs/vitess/imports/import-tool-migration-addresses) to determine the IP addresses to allow before continuing. This guide will use the **AWS us-east-1 (North Virginia)** region so we’ll allow the following addresses: ``` 3.209.149.66 3.215.97.46 34.193.111.15 23.23.187.137 52.6.141.108 52.70.2.89 50.17.188.76 52.2.251.189 52.72.234.74 35.174.68.24 52.5.253.172 54.156.81.4 34.200.24.255 35.174.79.154 44.199.177.24 ``` To allow traffic into your Azure database, navigate to the “**Networking**” section from the sidebar and locate the **Firewall rules** section. There are already a series of inputs allowing you to add entries into the Firewall rules, each of which will permit network traffic from that IP address. Add a new entry for each address required, then click “Save” from the toolbar. ![The networking tab of the Azure dashboard.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/azure-database-for-mysql-migration-guide/the-networking-tab-of-the-azure-dashboard.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=5ee562b7199fc5a58c188ea1bad757c8) ## Configure MySQL server settings There are three settings that need to be configured before you can import your database: * gtid\_mode * enforce\_gtid\_consistency * binlog\_row\_image To access these settings in Azure, select “**Server parameters**” from the sidebar and enter “**gtid**” in the search bar. Set both “**enforce\_gtid\_consistency**” and “**gtid\_mode**” to “**ON**”. Next, search for “**binlog\_row\_image**” and set to “**full**”. Click “**Save**”. > [!NOTE] > > For “**gtid\_mode**”, you’ll need to update the value in sequence displayed in the dropdown until it is set to “**ON**”. For example, if the current setting is “**OFF\_PERMISSIVE**”, you’ll need to first change it to “**ON\_PERMISSIVE**”, save the changes, then set it to “**ON**” in that order. ![How to access gtid settings in the Azure dashboard.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/azure-database-for-mysql-migration-guide/how-to-access-gtid-settings-in-the-azure-dashboard.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=8bb053b7f89564cf3b7b120811e4cf8f) ## Import your database Now that your Azure Database for MySQL is configured and ready, follow the [Database imports guide](/docs/vitess/imports/database-imports) to complete your import. When you [create the external keyspace](/docs/vitess/imports/database-imports#step-3-create-an-external-keyspace), use the following connection settings: * **Host name** (`--host`) - Your Azure server name (from Prerequisites) * **Port** (`--port`) - 3306 (default for Azure MySQL) * **Database name** (`--source-database`) - The exact database name to import * **Username** (`--username`) - Your server admin login name * **Password** (`--password`) - Your server admin password * **SSL verification mode** (`--ssl-mode`) - `verify_identity` (verify the certificate and hostname) The Database imports guide will walk you through: * Creating your PlanetScale database * Connecting to your Azure MySQL database with an external keyspace * Checking your server settings, user grants, and schema * Starting the import with MoveTables * Monitoring the import progress * Switching traffic and completing the import ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Database imports Source: https://planetscale.com/docs/vitess/imports/database-imports Import an existing MySQL database into PlanetScale with no downtime by connecting it as an external keyspace and moving its tables with a Vitess MoveTables workflow. ## Overview You can import an existing internet-accessible MySQL database into a PlanetScale Vitess database with **no downtime**. An import has two parts: 1. An [external keyspace](/docs/vitess/cluster-configuration#create-an-external-keyspace) connects your production branch to your existing MySQL database. 2. A [Vitess MoveTables](/docs/cli/move-tables) workflow copies the tables from the external keyspace into a PlanetScale keyspace, keeps them in sync while your application keeps running, and switches traffic to PlanetScale when you're ready. You run the import with the [`pscale` CLI](/docs/cli). The **Workflows** page in the dashboard shows the progress of the import, but it is read-only: every step on this page is a CLI command. > [!NOTE] > > You must be an [Organization Administrator](/docs/security/access-control#organization-administrator) or a [Database Administrator](/docs/security/access-control#database-administrator) of the database to run an import. To run an import with a [service token](/docs/api/service-tokens), give it the `create_branch`, `read_workflow`, `write_workflow`, and `delete_workflow` permissions on the database, plus `delete_production_branch` to delete the external keyspace at the end. Before you begin, it may be helpful to check out our [general MySQL compatibility guide](/docs/vitess/troubleshooting/mysql-compatibility). ## Import process overview 1. **Prepare your source database** - Check its server settings, create a user for PlanetScale, and allow PlanetScale's IP addresses 2. **Create your PlanetScale database** - Create the Vitess database you are importing into, and tune its keyspace for the import 3. **Create an external keyspace** - Connect the production branch to your source database. PlanetScale checks connectivity, server settings, user grants, and schema compatibility first 4. **Start the import** - Create a MoveTables workflow from the external keyspace to your PlanetScale keyspace 5. **Monitor the import** - Watch the copy and replication progress, then verify the data 6. **Connect your application to PlanetScale** - Test your application against PlanetScale while your source database is still serving traffic 7. **Switch traffic** - Move reads, then writes, to PlanetScale 8. **Complete the import** - Stop replication and disconnect your source database > [!NOTE] > > It's recommended to avoid all schema changes / DDL (Data Definition Language) statements during an import on both your source database and the PlanetScale database. This includes `CREATE`, `DROP`, `ALTER`, `TRUNCATE`, etc. ## Step 1: Prepare your source database ### Server configuration These server settings need to be set correctly for the import to work: | Variable | Required value | Documentation | | :- | :- | :- | | `gtid_mode` | `ON` | [Documentation](https://dev.mysql.com/doc/refman/8.0/en/replication-options-gtids.html#sysvar_gtid_mode) | | `log_bin` | `ON` | [Documentation](https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_log_bin) | | `binlog_format` | `ROW` | [Documentation](https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_binlog_format) | | `binlog_row_image` | `FULL` or `NOBLOB` | [Documentation](https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_binlog_row_image) | | `expire_logs_days`\* | `>= 2` | [Documentation](https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_expire_logs_days) | | `binlog_expire_logs_seconds`\* | `>= 172800` | [Documentation](https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_binlog_expire_logs_seconds) | | `sql_mode` | Includes `NO_ZERO_IN_DATE` and `NO_ZERO_DATE`, and does not include `ANSI_QUOTES` | [Documentation](https://dev.mysql.com/doc/refman/8.0/en/sql-mode.html) | | `max_connections` | `>= 10` | [Documentation](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_max_connections) | **\*** Either `expire_logs_days` or `binlog_expire_logs_seconds` needs to be set. If both are set, `binlog_expire_logs_seconds` takes precedence. On Amazon RDS and Aurora, set `binlog retention hours` to at least 48 instead. See the [provider-specific migration guides](#provider-specific-migration-guides) for how to change these settings. MySQL 5.6, 5.7, and 8.0 are supported. The source database must not already contain a `_vt` database. ### Create a user for PlanetScale PlanetScale connects to your source database with a single user. That user needs replication privileges, read and write access to the database you're importing, and access to the `ps_import_*` database that PlanetScale creates on your source to track replication. See [Import user permissions](/docs/vitess/imports/import-tool-user-requirements) for the full list of grants and a script that creates the user. ### Allow PlanetScale's IP addresses Allow connections from PlanetScale's IP addresses for your database's region in your database firewall or security group. See [Import public IP addresses](/docs/vitess/imports/import-tool-migration-addresses) for where to find them. ## Step 2: Create your PlanetScale database Create the Vitess database you're importing into: ```bash theme={null} pscale database create --org --region --cluster-size --wait ``` Pick a [region](/docs/plans/regions) close to your source database and a [cluster size](/docs/vitess/scaling/cluster-sizing) with enough storage for your data. You can also create the database from the dashboard with "**New database**" > "**Create database**". We recommend using the same name as the database you're importing from to avoid updating any database name references throughout your application code. Your PlanetScale database starts with one keyspace, named after the database. That keyspace is the **target** of the import. If you'd prefer to use a different database name, make sure to update your app where applicable once you fully switch over to PlanetScale. ### Tune the target keyspace for the import We recommend setting these [VTTablet parameters](/docs/vitess/cluster-configuration/parameters) on your PlanetScale keyspace before you start the import. They speed up the copy and keep VReplication retrying through errors on your source database, which matters most for large imports. | Parameter | Value for imports | What it does | | :- | :- | :- | | `vreplication_copy_phase_duration` | `24h` | How long each copy phase runs before the next catch-up phase. Only change it if your database uses [foreign key constraints](#foreign-key-constraints). | | `vreplication_max_time_to_retry_on_error` | `720h` | How long VReplication keeps retrying after an error before it gives up. | | `vreplication-parallel-insert-workers` | `4` | Number of parallel insert workers on the target tablet during the copy. Don't set it higher than the number of vCPUs on the keyspace's tablets. | Set them with [`pscale keyspace parameters set`](/docs/cli/keyspace#change-keyspace-parameters). If your database uses foreign key constraints, also pass `--parameters vttablet.vreplication_copy_phase_duration=24h`: ```bash theme={null} pscale keyspace parameters set main \ --org \ --parameters vttablet.vreplication-parallel-insert-workers=4 \ --parameters vttablet.vreplication_max_time_to_retry_on_error=720h ``` The change rolls out to the keyspace's tablets, one shard at a time by default. Wait until `pscale keyspace parameters changes list main --org ` shows it as `completed` before you start the import. You can also set these parameters in the dashboard: click **Clusters** in the left nav, select the keyspace, click the **VTTablets** tab, and search for each parameter. Set these parameters back to their defaults after you [switch primary traffic](#step-7-switch-traffic) to PlanetScale. ## Step 3: Create an external keyspace An external keyspace connects your production branch to your source database. It is the **source** of the import. Give it a name that is different from your PlanetScale keyspace, such as `mydb_source`. ### Connection settings | Setting | `pscale keyspace create-external` flag | Description | | :- | :- | :- | | Host name | `--host` | The address where your database is hosted. | | Port | `--port` | The port where your database is hosted. The default MySQL port is `3306`. | | Database name | `--source-database` | The exact name of the database you want to import on your MySQL server. | | Username | `--username` | The [user you created for PlanetScale](/docs/vitess/imports/import-tool-user-requirements). | | Password | `--password` | The password for that user. | | SSL verification mode | `--ssl-mode` | `disabled`, `preferred`, `required`, `verify_ca`, or `verify_identity`. | | SSL client certificate and key | `--ssl-client-certificate`, `--ssl-client-key` | Authenticate with mTLS instead of a password. Pass the PEM contents or a path to a PEM file. | | SSL CA certificate chain | `--ssl-certificate-authority` | If your database server has a certificate with a non-trusted root CA, provide the full CA certificate chain here. | | SSL server name override | `--ssl-server-name` | Override the server name for SSL certificate verification. | | Minimum TLS version | `--min-tls-version` | The oldest TLS version PlanetScale may use. The default is TLS 1.2. | If your database server has a valid SSL certificate, set the SSL verification mode to `required` or higher. For more information about certificates from a Certificate Authority, check out our [Secure connections documentation](/docs/vitess/connecting/secure-connections#certificate-authorities). ### Check your source database Run `create-external` with `--dry-run` to check your source database without creating anything: ```bash theme={null} pscale keyspace create-external main \ --org \ --host \ --source-database \ --username \ --password \ --ssl-mode required \ --dry-run ``` PlanetScale checks: * **Connectivity** - It can connect with the credentials and SSL/TLS settings you provided. * **Server configuration** - The [server settings](#server-configuration) above are correct. * **User grants** - The user has the [required grants](/docs/vitess/imports/import-tool-user-requirements). * **Schema compatibility** - Every table can be imported. See [Schema compatibility](#schema-compatibility) below. If PlanetScale cannot connect, or the server settings or grants are wrong, it won't create the external keyspace. Fix the issue on your source database and run the check again. The [Import troubleshooting guide](/docs/vitess/imports/import-troubleshooting) covers each error. ### Schema compatibility The check also reports tables that can't be imported: * **Missing unique key** - All tables must have a unique, not-null key. See our [Changing unique keys documentation](/docs/vitess/schema-changes/onlineddl-change-unique-keys) for more info. * **Invalid charset** - We support `utf8`, `utf8mb4`, `utf8mb3`, `latin1`, and `ascii`. Tables with other charsets will be flagged. * **Unsupported storage engines** - Only `InnoDB` is supported. * **Unsupported partitioning** - Only `RANGE` partitioning is supported, without subpartitions. Schema errors don't stop you from creating the external keyspace. Fix those tables on your source database, or leave them out of the import with `--exclude-tables` in the next step. If your database uses foreign key constraints, PlanetScale turns on [foreign key support](/docs/vitess/foreign-key-constraints) for your PlanetScale database when you create the external keyspace. See [Foreign key constraints](#foreign-key-constraints) before you start the import. ### Create the external keyspace Once the check passes, run the same command with `--wait` instead of `--dry-run`: ```bash theme={null} pscale keyspace create-external main \ --org \ --host \ --source-database \ --username \ --password \ --ssl-mode required \ --wait ``` PlanetScale sizes the external keyspace from the size of your source data. You can also create the external keyspace from the dashboard: open your production branch, click "**Clusters**", then "**New keyspace**" > "**Create external keyspace**". The form checks your source database the same way before it creates the keyspace, and lists the IP addresses to allow. ## Step 4: Start the import Create a MoveTables workflow that copies the tables from the external keyspace into your PlanetScale keyspace: ```bash theme={null} pscale branch vtctl move-tables create main \ --org \ --workflow \ --source-keyspace \ --target-keyspace \ --all-tables \ --defer-secondary-keys \ --format json ``` The workflow starts right away. You will pass the workflow name and the target keyspace to every later command. ### Import options * **Tables** - `--all-tables` imports every table. To import only some tables, pass `--tables` with a comma-separated list instead. To import every table except a few, combine `--all-tables` with `--exclude-tables`. * **Defer secondary index creation** - `--defer-secondary-keys` creates secondary (non-primary) indexes after the data is copied instead of during the copy. Maintaining many indexes while inserting data is slow, so this can make your import significantly faster (often 2-3x faster for tables with multiple indexes). Secondary indexes are created during the copy unless you pass this flag. Don't use it if your database has foreign key constraints. * **DDL handling** - `--on-ddl` controls what happens if schema changes (like `ALTER TABLE`, `ADD INDEX`, etc.) run on your source database while the import is running: * `STOP` (default, recommended) - The workflow stops when a schema change is detected. After you review the change, restart the workflow with `pscale branch vtctl move-tables start`. This is the safest option because it lets you verify the schema changes won't cause issues before continuing. * `IGNORE` - Schema changes are skipped and won't be applied to your PlanetScale database. Your import continues without interruption, but your schemas will diverge. Only use this if you're confident you don't need these changes or plan to apply them manually to your PlanetScale database later. * `EXEC` - Schema changes are applied to your PlanetScale database while the import continues running. If applying a schema change fails (for example, if it's not compatible with Vitess), the workflow stops and you'll need to restart it. * `EXEC_IGNORE` - Attempts to apply schema changes but keeps running even if they fail. > [!WARNING] > > `EXEC_IGNORE` can lead to schema mismatches between your external database and PlanetScale database, potentially causing data inconsistencies or unexpected behavior. Only use this if you understand the risks and have a plan to handle failures. See the [`move-tables` reference](/docs/cli/move-tables) for every option. ### Foreign key constraints If your database uses foreign key constraints: * **Import all tables** - Pass `--all-tables` without `--exclude-tables` so referential integrity stays intact. * **Use an atomic copy** - Pass `--atomic-copy`, and don't pass `--defer-secondary-keys`. Foreign key constraints need their indexes to exist during the copy. * **Import retries** - An atomic copy holds a long-running transaction on your source database, which can increase load. If it fails, it starts over from the beginning instead of resuming where it left off. ```bash theme={null} pscale branch vtctl move-tables create main \ --org \ --workflow \ --source-keyspace \ --target-keyspace \ --all-tables \ --atomic-copy \ --format json ``` For more information about foreign key support and limitations, see our [foreign key constraints documentation](/docs/vitess/foreign-key-constraints). ## Step 5: Monitor your import Check the progress of the import with `status`: ```bash theme={null} pscale branch vtctl move-tables status main \ --org \ --workflow \ --target-keyspace \ --format json ``` The output shows: * **`table_copy_state`** - Rows and bytes copied so far for each table that is still copying * **`shard_streams`** - Each replication stream's state and any error message * **`traffic_state`** - Whether reads and writes have switched to PlanetScale Streams are `Copying` while the initial data is copied, then `Running` once the import is replicating new changes from your source database. A `Lagging` stream is running but hasn't caught up with recent changes. A `Stopped` or `Error` stream includes a message that explains why. The output also includes a `next_steps` field with the command to run next. You can also follow the import in the dashboard. Click "**Workflows**" in the left nav, select your production branch, and open the workflow to see its streams, per-table copy progress, replication lag, and traffic routing. To pause the import, run `pscale branch vtctl move-tables stop`. Run `pscale branch vtctl move-tables start` to resume it. ### Verify data (optional) Once the copy completes and the streams are `Running`, you can verify that the data in PlanetScale matches your source database with a [VDiff](https://vitess.io/docs/reference/vreplication/vdiff/): ```bash theme={null} pscale branch vtctl vdiff create main \ --org \ --workflow \ --target-keyspace \ --format json ``` The output includes the VDiff's `uuid`. Pass it to `vdiff show` and repeat until the VDiff completes: ```bash theme={null} pscale branch vtctl vdiff show main \ --org \ --workflow \ --target-keyspace \ --uuid \ --format json ``` The dashboard also shows the VDiff's progress on the workflow page. If the VDiff reports mismatches, don't switch traffic until you've resolved them. ## Step 6: Connect your application to PlanetScale Once the streams are `Running`, you can connect your application to PlanetScale while your source database stays the authoritative source. Until you switch traffic, PlanetScale routes queries for the imported tables to your source database through the external keyspace, so reads and writes still happen there. [Create a password](/docs/vitess/connecting/connection-strings) for your production branch and point a test deployment of your application at PlanetScale. This is the ideal time to test your application end-to-end before switching traffic. ## Step 7: Switch traffic When you're ready, switch traffic to PlanetScale. You can switch replica traffic first to test reads, then switch primary traffic. > [!WARNING] > > **Before switching primary traffic:** Update connection strings for all production writers to PlanetScale, and verify none still writes to the source database. Otherwise, the databases can diverge and data can be lost. 1. **Switch replica traffic** - Serve read queries sent to replicas from PlanetScale while writes still go to your source database. This is an optional intermediate step that lets you test read traffic separately. ```bash theme={null} pscale branch vtctl move-tables switch-traffic main \ --org \ --workflow \ --target-keyspace \ --tablet-types REPLICA,RDONLY \ --format json ``` 2. **Switch primary traffic** - Serve both reads and writes from PlanetScale. ```bash theme={null} pscale branch vtctl move-tables switch-traffic main \ --org \ --workflow \ --target-keyspace \ --tablet-types PRIMARY \ --format json ``` Pass `--dry-run` to see what a switch would do without applying it. After primary traffic switches, PlanetScale replicates writes back to your source database, so both stay in sync until you complete the import. If something goes wrong, switch traffic back to your source database: ```bash theme={null} pscale branch vtctl move-tables reverse-traffic main \ --org \ --workflow \ --target-keyspace \ --format json ``` If you [tuned the target keyspace](#tune-the-target-keyspace-for-the-import) for the import, set its VTTablet parameters back to their defaults once primary traffic has switched. Like the original change, this rolls out to the keyspace's tablets one shard at a time. ```bash theme={null} pscale keyspace parameters set main \ --org \ --reset vttablet.vreplication-parallel-insert-workers \ --reset vttablet.vreplication_max_time_to_retry_on_error ``` Add `--reset vttablet.vreplication_copy_phase_duration` if you set it for foreign key constraints. ## Step 8: Complete the import Once you've switched all traffic to PlanetScale and verified everything is working: **Step 1** Monitor your application for any issues. Check: * Application logs for errors * Replication lag (should be near zero) * Error tracking tools * Application performance metrics **Step 2** Preview what completing the workflow will do: ```bash theme={null} pscale branch vtctl move-tables complete main \ --org \ --workflow \ --target-keyspace \ --keep-data=true \ --keep-routing-rules=false \ --dry-run \ --format json ``` **Step 3** When you're confident everything is working correctly, run the same command without `--dry-run`. **Step 4** Delete the external keyspace to disconnect PlanetScale from your source database: ```bash theme={null} pscale keyspace delete main --org ``` `--keep-data` must be `true` when you complete an import. Your source database's tables are never dropped or renamed. `--keep-routing-rules=false` removes the routing rules that sent queries to the external keyspace. Write both flags with an `=`: a space-separated value such as `--keep-data false` is read as `--keep-data=true`. **What happens when you complete:** * Replication from PlanetScale back to your source database stops * The routing rules for the imported tables are removed * The workflow no longer appears on the Workflows page **What happens when you delete the external keyspace:** * PlanetScale disconnects from your source database * The source database's credentials are removed from PlanetScale > [!WARNING] > > **Important** > > Completing the workflow is not reversible. Make sure your application is running smoothly on PlanetScale before completing the import. Don't delete the external keyspace until the workflow is complete. Once the import is complete, you can drop the `ps_import_*` database from your source database and remove the user you created for PlanetScale. ### Cancel an import To stop an import before you complete it, cancel the workflow: ```bash theme={null} pscale branch vtctl move-tables cancel main \ --org \ --workflow \ --target-keyspace \ --keep-data=false \ --keep-routing-rules=false \ --format json ``` `--keep-data=false` deletes the data already copied into your PlanetScale keyspace. The external keyspace stays connected, so you can fix the problem and start a new workflow. Delete the external keyspace if you don't plan to try again. ## Next steps You just migrated your database to PlanetScale. Here are some things you can do next: * [Create a development branch](/docs/vitess/schema-changes/branching) - Use branching in your development workflow. * [Create a deploy request](/docs/vitess/schema-changes/deploy-requests#create-a-deploy-request) - Test schema changes in dev branches before pushing to production. ## Provider-specific migration guides For detailed instructions on preparing your external database for import, see our provider-specific guides: * [Amazon Aurora](/docs/vitess/imports/amazon-aurora-migration-guide) * [AWS RDS for MySQL](/docs/vitess/imports/aws-rds-migration-guide) * [Azure Database for MySQL](/docs/vitess/imports/azure-database-for-mysql-migration-guide) * [DigitalOcean MySQL](/docs/vitess/imports/digitalocean-database-migration-guide) * [Google Cloud SQL](/docs/vitess/imports/gcp-cloudsql-migration-guide) * [MariaDB](/docs/vitess/imports/mariadb-migration-guide) ## Troubleshooting For detailed troubleshooting guidance, see our [Import troubleshooting guide](/docs/vitess/imports/import-troubleshooting). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # DigitalOcean database migration guide Source: https://planetscale.com/docs/vitess/imports/digitalocean-database-migration-guide ## Introduction In this article, we’ll walk through migrating a MySQL database from DigitalOcean to PlanetScale with a [database import](/docs/vitess/imports/database-imports). > [!NOTE] > > This guide assumes you are using MySQL on DigitalOcean. Other database systems available through DigitalOcean will not work with this guide. We recommend reading through the [Database import documentation](/docs/vitess/imports/database-imports) to learn how imports work before proceeding. ## Prerequisites Before you can start migrating your database, you’ll also need to collect the following information from your DigitalOcean cluster: * The admin username and password * The database host * The port * Database name Most of this information is located on the landing page of your cluster. ![The Overview of the database cluster.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/digitalocean-database-migration-guide/the-overview-of-the-database-cluster.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=1f29311908698e9fe2b2ee1876c1556b) You can view the list of databases in the "**Users & Databases**" tab. This article will use the default database created when the cluster was initialized, named `defaultdb`. ![The Users & Databases view of the cluster with the Databases section highlighted.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/digitalocean-database-migration-guide/the-users-and-databases-view-of-the-cluster-with-the-databases-section-highlighted.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=d8e3ced595e14b3f8089cdb9240a27e6) > [!NOTE] > > If you don’t know the admin password, you can create a new set of credentials using the information on the [Import > tool user permissions page](/docs/vitess/imports/import-tool-user-requirements) to create an account that can be used to > import your database. ## Update trusted sources In order for PlanetScale to connect to your DigitalOcean database, you must allow network traffic into the database by adding the necessary IP addresses to the trusted sources list in DigitalOcean. The specific IP addresses you will need to allow depend on the region you plan to host your PlanetScale database. Check the [Import public IP addresses page](/docs/vitess/imports/import-tool-migration-addresses) to determine the IP addresses to allow before continuing. This guide will use the **AWS us-east-1 (North Virginia)** region so we’ll allow the following addresses: ``` 3.209.149.66 3.215.97.46 34.193.111.15 23.23.187.137 52.6.141.108 52.70.2.89 50.17.188.76 52.2.251.189 52.72.234.74 35.174.68.24 52.5.253.172 54.156.81.4 34.200.24.255 35.174.79.154 44.199.177.24 ``` In the DigitalOcean dashboard, navigate to the “**Settings”** tab of your database and locate **Trusted sources** in the list of configuration items. Click “**Edit”** and the row should change to allow edits to the setting. ![The Settings tab of the database cluster in Digital Ocean.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/digitalocean-database-migration-guide/the-settings-tab-of-the-database-cluster-in-digital-ocean.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=ef3106dc50ca4745df5f37c83b53d28c) When you enter an IP address from the list, a message will appear below the input box asking if you want to add that IP as an address. Click that message to add it to the list. Repeat this step for each IP address that needs to be added, then click “**Save**” once you are done. ![The Trusted sources section of the Settings tab.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/digitalocean-database-migration-guide/the-trusted-sources-section-of-the-settings-tab.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=a553916f587d5a1264a7e62b17a66b3f) ### Disable ANSI\_QUOTES setting The default settings for MySQL databases on DigitalOcean is to have `ANSI_QUOTES` enabled in the global MySQL settings, which is not supported by PlanetScale. To remove this setting, navigate to the "**Settings**" tab of your cluster and locate the section titled **Global SQL mode**. Click “**Edit**” in that section to change the configuration settings. ![The Settings tab of the database cluster in DigitalOcean with the Global SQL mode section highlighted.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/digitalocean-database-migration-guide/the-settings-tab-of-the-database-cluster-in-digitalocean-with-the-global-sql-mode-section-highlighted.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=33f2966ff29f5f1e09dc640ee5d6eef5) To remove the `ANSI_QUOTES` setting, click the “**x**” next to the tag and click “**Save**.” The change should apply immediately. ![An example of removing the ANSI\_QUOTES setting from the Global SQL mode settings.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/digitalocean-database-migration-guide/an-example-of-removing-the-ansi_quotes-setting-from-the-global-sql-mode-settings.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=14ca6da3f6473279e13ab4fe1be4c26e) ### Set the Binlog Retention Period The binary log related [MySQL server variables](/docs/vitess/imports/database-imports#server-configuration) required for PlanetScale imports are already set to acceptable values by default on Digital Ocean managed MySQL servers but there's one more variable to check on the "**Settings**" tab of your existing cluster. Scroll down to "**Advanced configurations**" and ensure the "**Binlog Retention Period**" is set to at least `172800` seconds (48 hours). ![Setting the binlog retention period under Advanced Configurations](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/imports/digitalocean-binlog-retention.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=cc591d82d25ad5a9a5e81a35f6d47f60) ## Importing your database Now that your DigitalOcean database is configured and ready, follow the [Database imports guide](/docs/vitess/imports/database-imports) to complete your import. When you [create the external keyspace](/docs/vitess/imports/database-imports#step-3-create-an-external-keyspace), use the following connection settings: * **Host name** (`--host`) - Your DigitalOcean database host (from Prerequisites) * **Port** (`--port`) - Your port (typically 25060 for DigitalOcean) * **Database name** (`--source-database`) - The exact database name to import (e.g., `defaultdb`) * **Username** (`--username`) - Your admin username * **Password** (`--password`) - Your admin password * **SSL verification mode** (`--ssl-mode`) - Select based on your DigitalOcean SSL configuration The Database imports guide will walk you through: * Creating your PlanetScale database * Connecting to your DigitalOcean database with an external keyspace * Checking your server settings, user grants, and schema * Starting the import with MoveTables * Monitoring the import progress * Switching traffic and completing the import ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Discovery Tool Source: https://planetscale.com/docs/vitess/imports/discovery-tool Assess your MySQL, MariaDB, or Vitess database and cloud infrastructure before migrating to PlanetScale Vitess. The PlanetScale Discovery Tool analyzes your existing MySQL-compatible database and cloud infrastructure to help plan your migration to PlanetScale Vitess. It collects metadata about your database configuration, schema structure, performance characteristics, replication topology, security settings, feature usage, and cloud resources. It never reads or stores actual table data. The Discovery CLI also supports PostgreSQL discovery. See the [Postgres Discovery Tool guide](/docs/postgres/imports/discovery-tool) for PlanetScale Postgres-specific details. ![Where ps-discovery looks: cloud and vendor discovery, database discovery, and workload discovery](https://mintcdn.com/planetscale-2/YxnMMno34vnTE3Ub/images/imports/discovery-tool-topology.png?fit=max\&auto=format\&n=YxnMMno34vnTE3Ub\&q=85\&s=f2efb9d5a29c396484b0c677b60e66bf) The tool produces a structured JSON report that PlanetScale uses to provide migration guidance tailored to your environment. > [!NOTE] > > The discovery tool is open source and available on [GitHub](https://github.com/planetscale/ps-discovery). Essential setup steps are below. Advanced usage, troubleshooting, and detailed reference are available in the [full documentation](https://github.com/planetscale/ps-discovery/tree/main/docs). ## What it discovers **Database analysis:** * MySQL version, distribution, configuration, server variables, and cloud platform * Schema structure, including databases, tables, columns, indexes, constraints, views, stored routines, triggers, and partitioning * Performance statistics such as global status counter rates, process list summaries, InnoDB lock counters, and deadlock detection * Replication configuration, including replica status, binary log inventory, binary log retention settings, and binary log format * Security configuration, including SSL/TLS status, authentication plugin distribution, password policy settings, and aggregate privilege summaries * Feature usage: full-text indexes, geospatial data types, foreign key constraints, table partitioning, InnoDB compression, XA transactions, prepared statements, Galera Cluster, and more **Cloud infrastructure analysis:** * Database instances, clusters, and their configurations * RDS instances, Aurora clusters, Cloud SQL instances, and Azure Database for MySQL Flexible Servers * VPC networking, subnets, security groups, firewall rules, and private connectivity * Performance metrics from cloud monitoring services * High availability and replica configurations ## Installation The discovery tool requires Python 3.10 or later. **1. Download and extract** Download the latest release from [GitHub](https://github.com/planetscale/ps-discovery/releases) and extract it: ```bash theme={null} unzip ps-discovery-2.2.0.zip cd ps-discovery-2.2.0 ``` **2. Run setup** The setup script verifies your Python version, creates a virtual environment, installs dependencies, and writes a starter `config.yaml`: ```bash theme={null} chmod +x setup.sh ./setup.sh ``` Select MySQL as the database engine during setup. **3. Configure credentials** Edit the generated configuration file to include your database and cloud provider credentials: ```bash theme={null} nano config.yaml ``` At a minimum, configure your MySQL connection. See [Configuration](#configuration) below for the full format. For advanced installation options, see the [Discovery Tool README](https://github.com/planetscale/ps-discovery/blob/main/README.md). ## Database user setup Create a dedicated read-only user for the discovery tool. Connect to your MySQL-compatible database as a privileged user and run the following: ```sql theme={null} -- Create a dedicated user for database discovery CREATE USER 'planetscale_discovery'@'%' IDENTIFIED BY 'secure_password_here'; -- Grant read access for schema analysis GRANT SELECT ON *.* TO 'planetscale_discovery'@'%'; -- Grant process privilege for performance analysis and SHOW PROCESSLIST GRANT PROCESS ON *.* TO 'planetscale_discovery'@'%'; -- Grant replication client for binary log and replica status analysis GRANT REPLICATION CLIENT ON *.* TO 'planetscale_discovery'@'%'; -- Apply changes FLUSH PRIVILEGES; ``` > [!NOTE] > > On Amazon RDS, Aurora MySQL, Google Cloud SQL for MySQL, Azure Database for MySQL, MariaDB, and Percona Server, create the user through your administrative database user. Some managed services restrict access to certain system tables. The discovery tool reports those gaps and continues with the data it can collect. ### PlanetScale and Vitess credentials For PlanetScale databases, your existing branch credentials are sufficient. The discovery tool automatically detects PlanetScale and Vitess environments and adapts its queries. Add your branch credentials to `config.yaml`: ```yaml theme={null} engine: mysql mysql: host: aws.connect.psdb.cloud port: 3306 username: your_branch_username password: your_branch_password ssl_mode: required ``` Then run discovery with the values from the config file: ```bash theme={null} ./ps-discovery --config config.yaml ``` **PlanetScale and Vitess-specific behavior:** * The tool detects scoped `information_schema` in Vitess and automatically falls back to per-database iteration * System databases such as `_vt`, `mysql`, and `performance_schema` are excluded * Features not supported by Vitess are detected and reported ### MySQL cleanup > [!WARNING] > > After MySQL discovery is complete, remove the `planetscale_discovery` user from your database. This user has read access to your schema and system metadata and should not be left in place. ```sql theme={null} DROP USER IF EXISTS 'planetscale_discovery'@'%'; ``` ## Configuration The discovery tool uses a YAML configuration file. Running `./ps-discovery` loads `./config.yaml` by default. Configure the `mysql` block for database discovery and enable any cloud providers you want included: ```yaml theme={null} engine: mysql mysql: host: your-db-host.example.com port: 3306 database: "" # Leave empty to discover all databases username: planetscale_discovery password: secure_password_here ssl_mode: required providers: aws: enabled: true regions: - us-east-1 gcp: enabled: false azure: enabled: false output: output_dir: ./mysql_discovery_output ``` Set `database` to a specific database name to focus analysis on one database: ```yaml theme={null} engine: mysql mysql: host: your-db-host.example.com port: 3306 database: my_application_db username: planetscale_discovery password: secure_password_here ssl_mode: required ``` ## Running discovery Run discovery with the generated `config.yaml`: ```bash theme={null} ./ps-discovery ``` Or point to a specific config file: ```bash theme={null} ./ps-discovery --config config.yaml ``` The tool produces a timestamped JSON file in your configured output directory, such as `planetscale_discovery_results_20260708T072229.json`. Share this report with PlanetScale for migration planning assistance. > [!NOTE] > > Once MySQL discovery is complete, remember to [clean up](#mysql-cleanup) the `planetscale_discovery` user you created on your source database. ## Cloud provider setup Each cloud provider requires specific credentials and permissions. Below is a summary of what you need for each. For detailed instructions including IAM policies and API enablement steps, see the [provider documentation](https://github.com/planetscale/ps-discovery/tree/main/docs/providers). ### AWS (RDS / Aurora) The tool discovers RDS instances, Aurora clusters, VPC networking, security groups, and CloudWatch metrics. **Authentication** (choose one): * AWS profile * IAM instance profile when running on EC2 * Access key and secret key * IAM role assumption for cross-account access **Required permissions:** * RDS: `DescribeDBInstances`, `DescribeDBClusters`, `DescribeDBSubnetGroups`, `DescribeDBClusterParameterGroups`, `DescribeDBParameterGroups`, `DescribeOptionGroups` * EC2: `DescribeVpcs`, `DescribeSubnets`, `DescribeSecurityGroups`, `DescribeRouteTables`, `DescribeInternetGateways`, `DescribeNatGateways`, `DescribeVpcEndpoints` * CloudWatch: `GetMetricData`, `ListMetrics` * STS: `GetCallerIdentity` **Configuration:** ```yaml theme={null} providers: aws: enabled: true regions: - us-east-1 - us-west-2 discover_all: true # Authentication - choose one approach: # Option 1: Use an AWS profile credentials: profile: migration-discovery # Option 2: Assume a role # credentials: # role_arn: "arn:aws:iam::123456789012:role/PlanetScaleDiscoveryRole" # external_id: "unique-external-id" ``` You can also focus discovery on specific AWS resources: ```yaml theme={null} providers: aws: enabled: true discover_all: false resources: rds_instances: - production-db-1 - staging-db-1 aurora_clusters: - prod-cluster regions: - us-east-1 ``` ### Google Cloud (Cloud SQL) The tool discovers Cloud SQL instances, VPC networks, firewall rules, and Cloud Monitoring metrics. **Authentication** (choose one): * Service account key file * Application Default Credentials * Environment variables **Required APIs** (must be enabled in your project): * Cloud SQL Admin API * Compute Engine API * Cloud Monitoring API **Configuration:** ```yaml theme={null} providers: gcp: enabled: true project_id: your-project-id regions: - us-central1 - us-east1 discover_all: true credentials: service_account_key: /path/to/ps-discovery-key.json ``` You can also focus discovery on specific Google Cloud resources: ```yaml theme={null} providers: gcp: enabled: true project_id: your-project-id discover_all: false resources: cloud_sql_instances: - prod-db-instance - staging-db-instance regions: - us-central1 ``` ### Azure (Database for MySQL) The tool discovers Azure Database for MySQL Flexible Servers, virtual networks, network security groups, and firewall rules. **Authentication** (choose one): * Service principal with the Reader role (recommended) * Azure CLI (`az login`) or a managed identity * `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_TENANT_ID`, and `AZURE_SUBSCRIPTION_ID` environment variables **Required permissions:** * Built-in Reader role at subscription or resource-group scope * Resource providers registered on the subscription: `Microsoft.DBforMySQL` and `Microsoft.Network` **Configuration:** ```yaml theme={null} providers: azure: enabled: true subscription_id: 22222222-2222-2222-2222-222222222222 credentials: tenant_id: 11111111-1111-1111-1111-111111111111 client_id: 00000000-0000-0000-0000-000000000000 client_secret: your-client-secret discover_all: true ``` You can also sign in with `az login` and omit the `credentials` block, or focus discovery on specific servers: ```yaml theme={null} providers: azure: enabled: true subscription_id: 22222222-2222-2222-2222-222222222222 discover_all: false resources: mysql_flexible_servers: - mysql-reporting regions: - eastus ``` ## Performance and safety The default database analyzers are safe to run against production databases. They use read-only queries against system catalogs and statistics views, with very low performance impact. > [!WARNING] > > The discovery tool can query metadata and statistics across every accessible database when the `database` field is empty. For environments with many databases or very large schemas, consider targeting one database at a time or running the tool against a replica. ## Privacy and security The discovery tool runs entirely on your infrastructure. No data is sent to external services during analysis. **Collected:** Schema metadata, database configuration, usage statistics, replication metadata, infrastructure topology, aggregate security information, and feature usage. **Not collected:** Table contents, row data, query text, slow query log entries, passwords, secrets, connection strings, application code, or individual grant details. Passwords are used only to establish the database connection and are never included in the output. ## Next steps Once you have your discovery report, [share it with us](https://planetscale.com/contact) if you want tailored migration guidance. You can also follow one of our migration guides on your own: > **[Database imports](/docs/vitess/imports/database-imports)** > **[Migrate from AWS RDS](/docs/vitess/imports/aws-rds-migration-guide)** > **[Migrate from Amazon Aurora](/docs/vitess/imports/amazon-aurora-migration-guide)** > **[Migrate from Google Cloud SQL](/docs/vitess/imports/gcp-cloudsql-migration-guide)** > **[Migrate from Azure](/docs/vitess/imports/azure-database-for-mysql-migration-guide)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # GCP Cloud SQL Migration Guide Source: https://planetscale.com/docs/vitess/imports/gcp-cloudsql-migration-guide ## Overview This document will demonstrate how to migrate a database from Google Cloud Platform (GCP) Cloud SQL MySQL Cluster to PlanetScale with a [database import](/docs/vitess/imports/database-imports). > [!NOTE] > > This guide assumes you are using MySQL on GCP. Other database systems available through GCP will not work with this guide. We recommend reading through the [Database import documentation](/docs/vitess/imports/database-imports) to learn how imports work before proceeding. ## Prerequisites Before you can perform a migration, gather the following information from the GCP Console: * **Public IP address** - Found in the **Overview** tab of your Cloud SQL cluster under the **Connect to this instance** section * **Database name** (`--source-database`) - The name of the database you want to import * **Root username and password** - You'll need these to create the migration user ![The GCP Cloud SQL console with the IP address highlighted.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/gcp-cloudsql-migration-guide/cloudsql-ip-address.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=9cd3188f3de4861f6b3be0b094ccb35f) A list of your databases can be found in the **Databases** tab. In this guide, we'll be using the `prod` database. ![The Databases list in the GCP console.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/gcp-cloudsql-migration-guide/cloudsql-databases.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=dc74d3b6e3bf087647a6f0010a0053e2) ## Create a migration user Create a migration user account with limited privileges for the import process. **You must run this as the root user or another user with admin privileges.** Connect to your Cloud SQL instance as root using the MySQL command line: ```bash theme={null} mysql -u root -p -h [your-cloud-sql-ip] ``` Then run the following script, making sure to update the placeholders: * `` - The password for the `migration_user` account * `` - The name of the database you will import into PlanetScale ```sql theme={null} CREATE USER 'migration_user'@'%' IDENTIFIED BY ''; GRANT PROCESS, REPLICATION SLAVE, REPLICATION CLIENT, RELOAD ON *.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, SHOW VIEW, LOCK TABLES ON ``.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER ON *.* TO 'migration_user'@'%'; GRANT SELECT ON mysql.db TO 'migration_user'@'%'; GRANT SELECT ON mysql.func TO 'migration_user'@'%'; GRANT SELECT ON mysql.innodb_table_stats TO 'migration_user'@'%'; GRANT SELECT ON mysql.tables_priv TO 'migration_user'@'%'; GRANT SELECT ON mysql.user TO 'migration_user'@'%'; GRANT SELECT ON performance_schema.* TO 'migration_user'@'%'; FLUSH PRIVILEGES; ``` Verify the grants were applied correctly: ```sql theme={null} SHOW GRANTS FOR 'migration_user'@'%'; ``` You should see all the GRANT statements listed. If you only see one or two lines, the grants didn't apply correctly. > [!NOTE] > > **Important** > > You must create the migration user on the MySQL command line and not in the GCP console. Creating users through the GCP console automatically grants the `cloudsqlsuperuser` role, which will cause the import to fail. ## Allow PlanetScale to connect to your Cloud SQL instance For PlanetScale to connect to your database, you'll need to update the Authorized networks for your cluster. The specific IP addresses to permit depend on your PlanetScale database region. See [Import public IP addresses](/docs/vitess/imports/import-tool-migration-addresses) for how to find them. To permit traffic from these IP addresses to your database in GCP, select **Connections** from the navigation on the left. Under **Authorized networks**, click “**Add network**”. This will display an inline form for you to add a network. The name of the field is arbitrary, but the **Network** field should contain the IP address that needs access to your database. Click “**Done**” to add the new entry. Perform this step for each IP address for the selected region, then click “**Save**” to apply the settings. ![The form to add a new authorized network in the GCP console.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/gcp-cloudsql-migration-guide/cloudsql-networking.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=0db785aa2f986ecab3ee5fefbc8b595b) ## Configure MySQL server settings Certain MySQL server settings may need to be changed before you can begin the import. The initial connection test will fail if these settings are not configured correctly. * binlog\_expire\_logs\_seconds To set a flag in your GCP console, go to your database's “**Overview**” page, select the “**Edit**” button, and then scroll down to the “**Flags**” section. You want to select the “**binlog\_expire\_logs\_seconds**” flag and set it to `172800` seconds. Make sure to select the “**Done**” button. ![The form to set MySQL flags.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/gcp-cloudsql-migration-guide/cloudsql-set-flags.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=5cbff9170b7b587951aed8ead440b118) * log\_bin If `log_bin` is set to OFF you may need to [enable Point in Time Recovery (PITR)](https://cloud.google.com/sql/docs/mysql/backup-recovery/pitr#enablingpitr) from the GCP console to start binary logging. ![The form to set enable point in time recovery.](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/imports/gcp-cloudsql-migration-guide/cloudsql-enable-pitr.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=71f8ae6138e6f5c112415957de2a0216) ## Importing your database Now that your GCP Cloud SQL database is configured and ready, follow the [Database imports guide](/docs/vitess/imports/database-imports) to complete your import. When you [create the external keyspace](/docs/vitess/imports/database-imports#step-3-create-an-external-keyspace), use the following connection settings: * **Host name** (`--host`) - Your GCP Cloud SQL public IP address (from Prerequisites) * **Port** (`--port`) - 3306 (default for Cloud SQL) * **Database name** (`--source-database`) - The exact database name to import * **Username** (`--username`) - `migration_user` * **Password** (`--password`) - The password you set for the migration user * **SSL verification mode** (`--ssl-mode`) - Select based on your Cloud SQL SSL configuration The Database imports guide will walk you through: * Creating your PlanetScale database * Connecting to your Cloud SQL database with an external keyspace * Checking your server settings, user grants, and schema * Starting the import with MoveTables * Monitoring the import progress * Switching traffic and completing the import ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Import public IP addresses Source: https://planetscale.com/docs/vitess/imports/import-tool-migration-addresses When you [import a database](/docs/vitess/imports/database-imports), you need to grant a set of IP addresses access to your external MySQL database so that PlanetScale can connect to it through an external keyspace. ## Overview To import your external database into PlanetScale, you need to allowlist PlanetScale's IP addresses in your database firewall or security group. This lets PlanetScale connect to your database when you create the external keyspace and for the rest of the import. ## Where to find your IP addresses The IP addresses depend on the region of your PlanetScale database. **From the CLI:** list your organization's regions in JSON. Each region's `public_ip_addresses` field lists the IP addresses to allowlist: ```bash theme={null} pscale region list --org --format json ``` Use the addresses for the region with the same `slug` as your database's region. **From the dashboard:** when you [create an external keyspace](/docs/vitess/cluster-configuration#create-an-external-keyspace) from the **Clusters** page, the form lists the IP addresses that need access to your external database. ![IP addresses listed on the external database connection form](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/imports/import-workflows-ip-addresses/import-ips.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=fa15ffe65fbc1bd53493531dc5ec19ec) **From the API:** the [list regions for an organization](/docs/api/reference/list_regions_for_organization) endpoint returns the same `public_ip_addresses` field. ## Provider-specific firewall guides If you're connecting to a known database provider, see the provider's migration guide for how to allow these IP addresses: * [Amazon Aurora](/docs/vitess/imports/amazon-aurora-migration-guide#step-6-configure-rds-security-group) * [AWS RDS for MySQL](/docs/vitess/imports/aws-rds-migration-guide#step-6-configure-rds-security-group) * [Azure Database for MySQL](/docs/vitess/imports/azure-database-for-mysql-migration-guide#configure-firewall-rules) * [DigitalOcean](/docs/vitess/imports/digitalocean-database-migration-guide#update-trusted-sources) * [Google Cloud SQL](/docs/vitess/imports/gcp-cloudsql-migration-guide#allow-planetscale-to-connect-to-your-cloud-sql-instance) ## Important notes * **Region-specific IPs** - The IP addresses differ by region. Make sure you're using the IPs for your database's region. * **IPs may change** - IP addresses can change occasionally. Check the current list before you start an import. * **All IPs required** - You need to allowlist all the IP addresses listed, not just one. > [!NOTE] > > This guide is meant to be used alongside the [Database imports guide](/docs/vitess/imports/database-imports) or one of the provider-specific migration guides. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Import user permissions Source: https://planetscale.com/docs/vitess/imports/import-tool-user-requirements When you [import a database](/docs/vitess/imports/database-imports), PlanetScale connects to your existing database through an external keyspace with a user that has the permissions it needs to copy and replicate your data. PlanetScale uses one user for everything it does on your source database: reading your schema, copying rows, replicating changes, and, after you switch primary traffic, replicating writes back to your source database. When you [create the external keyspace](/docs/vitess/imports/database-imports#step-3-create-an-external-keyspace), PlanetScale checks that this user has the grants below and won't create the keyspace if any are missing. Below is the minimum set of permissions needed and what each allows the user to do: | Scope | Databases | Grant | Description | | :- | :- | :- | :- | | Global | n/a | `PROCESS` | Enable the user to see all processes with SHOW PROCESSLIST. | | Global | n/a | `REPLICATION SLAVE` | Enable replicas to read binary log events from the source. | | Global | n/a | `REPLICATION CLIENT` | Enable the user to ask where source or replica servers are. | | Global | n/a | `RELOAD` | Enable use of FLUSH operations. On MySQL 8.0, `FLUSH_TABLES` can be used instead. | | Database | `` | `SELECT` | Enable use of SELECT. | | Database | `` | `INSERT` | Enable use of INSERT. | | Database | `` | `UPDATE` | Enable use of UPDATE. | | Database | `` | `DELETE` | Enable use of DELETE. | | Database | `` | `LOCK TABLES` | Enable use of LOCK TABLES on tables for which you have the SELECT privilege. | | Database | `` | `SHOW VIEW` | Enable use of SHOW VIEW. Needed if your database has views. | | Database | `ps_import_*` | `SELECT`, `INSERT`, `UPDATE`, `DELETE` | Read and write the replication state PlanetScale keeps on your source database. | | Database | `ps_import_*` | `CREATE`, `DROP`, `ALTER` | Create and maintain the `ps_import_*` database and its tables. | PlanetScale creates a database named `ps_import_` on your source database to track replication. The last portion of the name varies per external keyspace, which is why the grants use a wildcard. > [!NOTE] > > * Create the user with the host `'%'`. Grants for other hosts are not recognized. > * Grant the privileges in the table above directly to the user, at the global or database level. The check does not recognize table-level grants or grants through a role. > * `ALL PRIVILEGES` on a scope covers every grant for that scope. > [!NOTE] > > The descriptions in the table above were taken from the MySQL docs. For a full list of all possible grants and their > impact, please refer to the [GRANT Statement page](https://dev.mysql.com/doc/refman/8.0/en/grant.html) of the MySQL > docs, and locate the section titled **Privileges Supported by MySQL**. ## Script to create user This MySQL script can be used to create a user with the necessary permissions inside of your database. You will need appropriate database permissions to run the script. The username will be `migration_user`. Make sure to update the following variables: * `` — The password for the `migration_user` account. * `` — The name of the database you will import into PlanetScale. ```sql theme={null} CREATE USER 'migration_user'@'%' IDENTIFIED BY ''; GRANT PROCESS, REPLICATION SLAVE, REPLICATION CLIENT, RELOAD ON *.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, SHOW VIEW, LOCK TABLES ON ``.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER ON `ps\_import\_%`.* TO 'migration_user'@'%'; GRANT SELECT ON mysql.db TO 'migration_user'@'%'; GRANT SELECT ON mysql.func TO 'migration_user'@'%'; GRANT SELECT ON mysql.innodb_table_stats TO 'migration_user'@'%'; GRANT SELECT ON mysql.tables_priv TO 'migration_user'@'%'; GRANT SELECT ON mysql.user TO 'migration_user'@'%'; ``` If your database is on Amazon RDS or Aurora, also let the user read your binlog retention setting: ```sql theme={null} GRANT EXECUTE ON PROCEDURE mysql.rds_show_configuration TO 'migration_user'@'%'; ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Import troubleshooting Source: https://planetscale.com/docs/vitess/imports/import-troubleshooting ## Overview This guide covers common issues you might run into when importing a database to PlanetScale and how to fix them. Most problems show up when you create the external keyspace. Run [`pscale keyspace create-external`](/docs/cli/keyspace#create-an-external-keyspace) with `--dry-run` to check your source database without creating anything. Connection, server configuration, and user grant errors stop the external keyspace from being created. Schema errors are listed for each table but don't block creation. ## Connection issues ### Can't connect to external database If PlanetScale can't connect to your database, here's what to check: **Verify your credentials locally** Try connecting with the same credentials using the MySQL CLI: ```bash theme={null} mysql -u -p -h -P -D ``` If this works locally but not in PlanetScale, the issue is likely network-related. **Check IP allowlist** Make sure you've added all PlanetScale IP addresses to your database's firewall or security group. The specific IPs depend on which region you selected for your PlanetScale database. See our [Import public IP addresses](/docs/vitess/imports/import-tool-migration-addresses) page. **Verify database is publicly accessible** PlanetScale needs to reach your database over the internet. Check that: * Your database has a public IP address * Public access is enabled in your database settings * No VPN or private network is required **SSL/TLS issues** If you're getting SSL-related errors: * Try `--ssl-mode disabled` to test if SSL is the issue * If your database uses self-signed certificates, provide the full CA certificate chain * For managed databases (RDS, Azure, etc.), use `--ssl-mode required` or `--ssl-mode verify_ca` ### Connection times out If the connection attempt times out: **Firewall rules** Most timeouts are caused by firewall rules blocking PlanetScale IPs. Double-check: * All IPs for your region are allowlisted * The port (usually 3306) is open * Any cloud provider security groups are configured correctly **Database not running** Verify your database server is running and accepting connections. **Hostname/port incorrect** Make sure you're using the correct hostname and port. For cloud providers: * Use the cluster endpoint, not individual instance endpoints * Default MySQL port is 3306, but some providers use different ports (DigitalOcean uses 25060) ## Server configuration errors ### GTID mode is OFF **Error:** `external database settings are not compatible with PlanetScale: "gtid_mode" must be "ON", but found: "OFF"` **Solution:** You need to enable GTID mode in your database configuration. For AWS RDS/Aurora: 1. Create a custom DB parameter group 2. Set `gtid-mode` to `ON` 3. Set `enforce_gtid_consistency` to `ON` 4. Apply the parameter group to your database 5. Reboot the database For Azure: 1. Go to Server parameters 2. Set `gtid_mode` to `ON` (you may need to go through intermediate states: `OFF_PERMISSIVE` → `ON_PERMISSIVE` → `ON`) 3. Set `enforce_gtid_consistency` to `ON` 4. Save changes For self-hosted MySQL/MariaDB: Add to your `my.cnf` or `my.ini`: ``` gtid_mode = ON enforce_gtid_consistency = ON ``` Then restart MySQL. ### Binary logging not enabled **Error:** `external database settings are not compatible with PlanetScale: "log_bin" must be "ON", but found: "OFF"` **Solution:** For AWS RDS/Aurora: Binary logging is tied to automated backups. Enable automated backups with a retention period >= 2 days. For GCP Cloud SQL: Enable Point in Time Recovery (PITR) from the console. For self-hosted: Add to your configuration: ``` log_bin = /var/log/mysql/mysql-bin.log ``` Restart MySQL. ### Wrong binlog format **Error:** `"binlog_format" must be "ROW", but found: "MIXED"` or `"STATEMENT"` **Solution:** Set `binlog_format` to `ROW` in your database configuration, then restart. If you see `"binlog_row_image" must be "FULL" or "NOBLOB"`, set `binlog_row_image` to `FULL`. For managed databases, update this in your parameter group or server parameters. ### Binlog retention too short **Error:** `"binlog_expire_logs_seconds" must be > 172800` (or similar for `expire_logs_days`). On AWS RDS and Aurora: `"binlog retention hours" must be at least 48 hours` or `binlog retention is not set on this AWS RDS database`. **Solution:** You need at least 48 hours of binlog retention for the import to work. For AWS RDS/Aurora: ```sql theme={null} CALL mysql.rds_set_configuration('binlog retention hours', 48); ``` Verify with: ```sql theme={null} CALL mysql.rds_show_configuration; ``` For other platforms: Set in your database configuration: ``` binlog_expire_logs_seconds = 172800 ``` Or: ``` expire_logs_days = 3 ``` ### sql\_mode is not compatible **Error:** `"sql_mode" cannot have "ANSI_QUOTES" enabled` or `PlanetScale requires "sql_mode" to have the following options set: "NO_ZERO_IN_DATE, NO_ZERO_DATE"` **Solution:** Remove `ANSI_QUOTES` from `sql_mode`, and make sure it includes `NO_ZERO_IN_DATE` and `NO_ZERO_DATE`. For managed databases, update this in your parameter group or server parameters. ### max\_connections is too low **Error:** `PlanetScale requires external databases to support at least a max connection limit of 10` **Solution:** Set `max_connections` to at least 10. PlanetScale uses up to half of your `max_connections` for the import. ### Unsupported MySQL version **Error:** `unsupported MySQL version detected` **Solution:** MySQL 5.6, 5.7, and 8.0 are supported. [Contact support](https://planetscale.com/contact?initial=support) if you need to import from a different version. ### Existing `_vt` database **Error:** `external database may have existing Vitess state: found "_vt" Vitess state database` **Solution:** Your source database already has a `_vt` database, usually left behind by an earlier Vitess-based import or replication tool. Make sure nothing is using it, then drop it and try again: ```sql theme={null} DROP DATABASE `_vt`; ``` ## Schema compatibility issues ### No unique key on table **Error:** Table has no unique key **Solution:** All tables must have a unique, not-null key. This is required for replication to work correctly. Add a primary key or unique index to the table: ```sql theme={null} ALTER TABLE your_table ADD PRIMARY KEY (id); ``` Or add a unique index: ```sql theme={null} ALTER TABLE your_table ADD UNIQUE KEY unique_index (column1, column2); ``` See our [Changing unique keys documentation](/docs/vitess/schema-changes/onlineddl-change-unique-keys) for more details. ### Invalid charset **Error:** Table uses unsupported charset **Solution:** PlanetScale supports: `utf8`, `utf8mb4`, `utf8mb3`, `latin1`, and `ascii`. Convert your table to a supported charset: ```sql theme={null} ALTER TABLE your_table CONVERT TO CHARACTER SET utf8mb4; ``` We recommend `utf8mb4` as it has the widest character support. ### Table names with special characters **Error:** Table name contains unsupported characters **Solution:** Rename tables that have characters outside the standard ASCII set: ```sql theme={null} RENAME TABLE `special-table-name` TO `special_table_name`; ``` Ensure that any queries using the table name get updated as well. ### Views detected Views aren't imported automatically. After your import completes, you'll need to manually recreate any views in your PlanetScale database. ### Unsupported storage engine **Error:** Table uses non-InnoDB storage engine **Solution:** Convert your tables to InnoDB: ```sql theme={null} ALTER TABLE your_table ENGINE=InnoDB; ``` Changing storage engines has significant performance impact. [Contact us](https://planetscale.com/contact?initial=support) if you are using a different storage engine on your source database and cannot change it prior to migrating. ## Foreign key import issues ### Import slower than expected Foreign key imports use an atomic copy (`--atomic-copy`), which holds a long-running transaction on your source database. This can be slow on large databases. **Solution:** Run the import during off-peak hours, and make sure your source database has enough CPU and I/O headroom for the copy. ### Import failed and won't resume Unlike regular imports, foreign key imports must start from the beginning if they fail. **Solution:** Before retrying: 1. Fix any errors that caused the failure 2. Make sure your binlog retention is long enough for the full import 3. Consider importing during off-peak hours To retry, [cancel the workflow](/docs/vitess/imports/database-imports#cancel-an-import) with `--keep-data=false` and create it again. ### Can't select specific tables When your database has foreign keys, import every table with `--all-tables` to maintain referential integrity. If you really only need specific tables, you'll need to: 1. Remove foreign key constraints from your source database 2. Import only the tables you need 3. Recreate foreign key constraints in PlanetScale after import Note: We recommend importing all tables to avoid referential integrity issues. ## Schema errors when creating the external keyspace ### External keyspace created with schema errors Schema errors, such as a table without a unique key, are listed for each table but don't stop the external keyspace from being created. Tables with schema errors will fail to import. **Solution:** * Fix the tables on your source database, then run `create-external --dry-run` again to confirm, or * Leave those tables out of the import with `--exclude-tables` when you create the MoveTables workflow Server configuration and user grant errors always need to be fixed first, because PlanetScale won't create the external keyspace until they pass. ## Import monitoring issues ### Replication lag is high During the initial copy phase, high replication lag is normal. The lag should drop once the copy finishes. **If lag stays high after copy completes:** 1. **Check source database load** - High write activity on source can cause lag 2. **Slow queries** - Look for slow queries or locks on the source database 3. **Network issues** - Check for network latency between source and PlanetScale 4. **Large transactions** - Very large transactions take time to replicate **Solutions:** * Reduce write load on source during import * Wait for off-peak hours * Check binlog retention isn't expiring before lag catches up ### Workflow stopped after a schema change With the default `--on-ddl STOP`, the workflow stops when a schema change runs on your source database. The stream's message starts with `Stopped at DDL`. **Solution:** Review the schema change, apply it to your PlanetScale database if needed, then resume the workflow with `pscale branch vtctl move-tables start`. ### Streams show errors Check the `message` on each stream in the [`status`](/docs/cli/move-tables) output, or the Streams panel on the workflow page in the dashboard. Common ones: **"Access denied"** - Permission issues. See [user requirements](/docs/vitess/imports/import-tool-user-requirements). **"Table doesn't exist"** - Schema may have changed during import. Don't modify schema during import. **"Deadlock found"** - Usually temporary. The import will retry. **Connection lost** - Network issue or source database restarted. The import will retry. ### Import stuck in "Copying" phase The copy phase can take a while for large databases. Check: * Check `table_copy_state` in the `status` output, or the Tables panel on the workflow page, to see if it's actually stuck or just slow * Check the stream messages for any errors * Verify source database is responding If truly stuck: 1. Check source database for locks or slow queries 2. Verify network connectivity 3. Look for errors in the stream messages ## Permission errors ### MySQL error 1045: Access denied **Error:** `Access denied for user 'migration_user'@'%'` **Solution:** Check that your migration user has all required permissions. See our [import user permissions](/docs/vitess/imports/import-tool-user-requirements). For foreign key imports, the user needs either: * `FLUSH_TABLES` or `RELOAD` privileges (preferred) * `LOCK TABLES` privilege (minimum) Verify grants: ```sql theme={null} SHOW GRANTS FOR 'migration_user'@'%'; ``` ### Required grants are missing **Error:** `external database does not have the required user grants: required privileges [...] are not present` **Solution:** Grant the privileges listed in the error. They must be granted directly to the user at the global or database level. See [Import user permissions](/docs/vitess/imports/import-tool-user-requirements). If the error says the permissions `do not match user`, the user was created for a specific host. Create it for the host `'%'` instead. ### Can't create the ps\_import database **Solution:** Grant the migration user permissions on the database PlanetScale creates to track replication: ```sql theme={null} GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER ON `ps\_import\_%`.* TO 'migration_user'@'%'; ``` ## Traffic switching issues ### Can't switch replica traffic Make sure: * Replication lag is low (under a few seconds). `switch-traffic` fails if lag is above `--max-replication-lag-allowed` * Every stream is `Running` * No stream shows an error ### Can't switch primary traffic Make sure: * Your application is connected to PlanetScale * Replication lag is minimal * Every stream is `Running` ### Data inconsistency after switching If you notice missing or stale data after switching traffic: 1. Check replication lag - it may still be catching up 2. Verify your application is actually connecting to PlanetScale 3. Run a [VDiff](/docs/vitess/imports/database-imports#verify-data-optional) to compare your source database with PlanetScale Don't complete the import until you've verified data consistency. ## Completing the import ### Complete fails with `keep_data must be true` **Error:** `keep_data must be true when the source keyspace is external` **Solution:** Completing an import never removes tables from your source database. Pass `--keep-data=true`, with an `=`, and leave out `--rename-tables`. ## Common provider-specific issues ### AWS RDS **Problem:** Can't modify GTID settings on default parameter group **Solution:** Create a custom DB parameter group with your MySQL version, modify settings there, then apply to your database. **Problem:** Binary logs not enabled **Solution:** Enable automated backups with retention >= 2 days. ### Azure **Problem:** Can't set gtid\_mode directly to ON **Solution:** Change through intermediate states: `OFF_PERMISSIVE` → `ON_PERMISSIVE` → `ON` ### DigitalOcean **Problem:** ANSI\_QUOTES mode enabled **Solution:** Remove ANSI\_QUOTES from Global SQL mode in Settings. **Problem:** Binlog retention too short **Solution:** Set Binlog Retention Period to at least 172800 seconds (48 hours). ### GCP Cloud SQL **Problem:** Binary logging disabled **Solution:** Enable Point in Time Recovery (PITR) from the GCP console. ## Still stuck? If you've tried the solutions above and are still having issues: 1. Check your database's error logs 2. Review our [general MySQL compatibility guide](/docs/vitess/troubleshooting/mysql-compatibility) 3. Look at the specific provider guide for your database 4. Check the `status` output or the workflow page in the dashboard for detailed error messages 5. [Contact PlanetScale support](https://planetscale.com/contact?initial=support) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # MariaDB migration guide Source: https://planetscale.com/docs/vitess/imports/mariadb-migration-guide ## Overview In this article, you’ll learn how to migrate a database from MariaDB, a fork of MySQL, into PlanetScale. > [!WARNING] > > The steps outlined in this guide used MariaDB version 10.6.12 on an Ubuntu host. Depending on the version of MariaDB you are using, your results may vary. Don't hesitate to [reach out to us](https://planetscale.com/contact) for further assistance. We recommend reading through the [Database import documentation](/docs/vitess/imports/database-imports) to learn how imports work before proceeding. ### Prerequisites * A PlanetScale account * A MariaDB server with traffic permitted from our [import IP addresses](/docs/vitess/imports/import-tool-migration-addresses) ## Configure MariaDB Before you can start migrating data, there are a number of configuration options that need to be in place for the import to work properly: * `binlog_format` * `log_bin` * `sql_mode` > [!NOTE] > > The steps below edit the configuration file on a self-managed server. If your MariaDB database is hosted on Amazon RDS, you cannot edit these values directly. Instead, set them through a DB parameter group, enable binary logging by turning on automated backups, and set your binlog retention period, as described in the [AWS RDS migration guide](/docs/vitess/imports/aws-rds-migration-guide). Skip the GTID parameters in that guide, since MariaDB uses its own GTID implementation and does not have a `gtid_mode` setting. Then return here to configure the migration account. You may run the following query to check these values: ```sql theme={null} SHOW variables WHERE Variable_name IN ('binlog_format','log_bin','sql_mode'); +---------------+-------------------------------------------------------------------------------------------+ | Variable_name | Value | +---------------+-------------------------------------------------------------------------------------------+ | binlog_format | MIXED | | log_bin | OFF | | sql_mode | STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION | +---------------+-------------------------------------------------------------------------------------------+ ``` In the results listed above, none of the options are configured properly, so let’s set them up now. The exact path to your configuration file varies by operating system. This demo uses Ubuntu, so the MariaDB configuration file is located at `/etc/mysql/mariadb.conf.d/50-server.cnf`. Edit the configuration file and add the following values at the end of the file: ``` binlog_format = ROW log_bin = /var/log/mysql/mysql-bin.log sql_mode = 'NO_ZERO_IN_DATE,NO_ZERO_DATE,ONLY_FULL_GROUP_BY' ``` With the configuration updated, restart the MariaDB service. The exact command varies by the service manager you are using on your host, with this demo using `systemctl`: ```bash theme={null} sudo systemctl restart mariadb ``` ## Configure a migration account PlanetScale requires a user account with a specific set of permissions on the database you wish to migrate, as well as the server itself to set up the databases that track replication changes. Connect to your MariaDB server with an account that has admin privileges, then run the following script, replacing the placeholders: * `` - The password for the `migration_user` account * `` - The name of the database you will import into PlanetScale ```sql theme={null} CREATE USER 'migration_user'@'%' IDENTIFIED BY ''; GRANT PROCESS, REPLICATION SLAVE, REPLICATION CLIENT, RELOAD ON *.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, SHOW VIEW, LOCK TABLES ON ``.* TO 'migration_user'@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER ON `ps\_import\_%`.* TO 'migration_user'@'%'; GRANT SELECT ON mysql.db TO 'migration_user'@'%'; GRANT SELECT ON mysql.func TO 'migration_user'@'%'; GRANT SELECT ON mysql.innodb_table_stats TO 'migration_user'@'%'; GRANT SELECT ON mysql.tables_priv TO 'migration_user'@'%'; GRANT SELECT ON mysql.user TO 'migration_user'@'%'; GRANT SELECT ON performance_schema.* TO 'migration_user'@'%'; FLUSH PRIVILEGES; ``` PlanetScale creates the `ps_import_` database on your server to track replication between MariaDB and PlanetScale. The last portion of the `ps_import_` name varies per external keyspace, which is why the grant uses a wildcard. Your server must not already contain a `_vt` database. > [!NOTE] > > If your MariaDB database is hosted on Amazon RDS, also grant access to the RDS configuration procedure so PlanetScale can read your binlog retention setting: > > ```sql theme={null} > GRANT EXECUTE ON PROCEDURE mysql.rds_show_configuration TO 'migration_user'@'%'; > ``` > > This procedure only exists on Amazon RDS. Running the grant against a self-managed MariaDB server fails with `ERROR 1305: PROCEDURE mysql.rds_show_configuration does not exist`. Verify the grants were applied correctly: ```sql theme={null} SHOW GRANTS FOR 'migration_user'@'%'; ``` You should see all of the `GRANT` statements listed. If you only see one or two lines, the grants didn't apply correctly. For a full explanation on what each of these grants do, [our article on configuring a migration account for MySQL databases](/docs/vitess/imports/import-tool-user-requirements) details each requirement. ## Importing your database Now that your MariaDB database is configured and ready, follow the [Database imports guide](/docs/vitess/imports/database-imports) to complete your import. When you [create the external keyspace](/docs/vitess/imports/database-imports#step-3-create-an-external-keyspace), use the following connection settings: * **Host name** (`--host`) - Your MariaDB server hostname or IP address * **Port** (`--port`) - 3306 (default for MariaDB) * **Database name** (`--source-database`) - The exact database name to import * **Username** (`--username`) - `migration_user` (created in previous section) * **Password** (`--password`) - The password you set for the migration user * **SSL verification mode** (`--ssl-mode`) - Select based on your MariaDB SSL configuration The Database imports guide will walk you through: * Creating your PlanetScale database * Connecting to your MariaDB database with an external keyspace * Checking your server settings, user grants, and schema * Starting the import with MoveTables * Monitoring the import progress * Switching traffic and completing the import ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres to Vitess imports Source: https://planetscale.com/docs/vitess/imports/postgres > [!WARNING] > > **PlanetScale now supports Postgres**. This guide is for migrating from Postgres to PlanetScale's Vitess product. You can still use these scripts if you would like to utilize [Vitess](/docs/vitess). If you prefer to stay on Postgres, refer to the [Postgres import guides](/docs/postgres/imports/postgres-imports). Many customers choose to migrate from Postgres to PlanetScale for Vitess in order to leverage the performance and scalability offered by PlanetScale and Vitess. Use this guide if you are importing from platforms like Aurora Postgres, RDS Postgres, Neon, Supabase, and other Postgres instances. [Data drop: Migrate from Postgres → PlanetScale](https://www.youtube-nocookie.com/embed/UPMinBnujmo?rel=0) ## Postgres to MySQL Vitess is built on MySQL. MySQL and Postgres are similar in that they are both relational databases, broadly function similarly, and are used for similar purposes by many organizations. When you look more closely, there are a number of small differences in types, indexing, and SQL language features. When added up, it makes for enough difference that migrating between the two is typically not straightforward. We provide scripts to make moving your data from Postgres to PlanetScale for Vitess easier. However, moving the data is only a part of the challenge. It is up to you to ensure that your application servers, which connect to your database, are also set up to cut over to MySQL. If you are using an ORM, this could be as simple as changing the target database engine. In some cases, you will have to update your queries manually to follow MySQL syntax. ## Migration Guides We have two recommendations for how to migrate your Postgres database to PlanetScale for Vitess. Both of these solutions leverage the AWS Database Migration Service to handle conversions between Postgres and MySQL types. 1. Use our [Postgres to PlanetScale for Vitess](/docs/vitess/imports/postgres-planetscale-migration-guide) guide along with our [postgres-planetscale scripts](https://github.com/planetscale/migration-scripts/tree/main/postgres-planetscale) to import directly from a Postgres source to a PlanetScale target. This technique is simpler than option 2, as it does not require an intermediate MySQL database, but operates more slowly at a rate of only a few gigabytes per hour. 2. Use our [Postgres to MySQL + PlanetScale for Vitess](/docs/vitess/imports/postgres-mysql-planetscale-migration-guide) guide along with our [postgres-mysql-planetscale scripts](https://github.com/planetscale/migration-scripts/tree/main/postgres-mysql-planetscale) to import from Postgres to an RDS MySQL instance, and then use a PlanetScale [database import](/docs/vitess/imports/database-imports) to bring the data in from MySQL. This technique is more complex since it requires an intermediary MySQL database. However, this technique can be 10x or more faster. This is recommended for larger databases. In some cases, using these scripts and following the necessary steps from our docs are all it takes to get your data imported to PlanetScale. However, there are many possible edge cases and specific Postgres configurations that may cause the scripts to fail. We encourage customers to make modifications to the scripts as needed to support their specific database environment. If you encounter issues while importing from a Postgres database, please [reach out to support](https://planetscale.com/contact?initial=support) for assistance. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres to MySQL to PlanetScale migration guide Source: https://planetscale.com/docs/vitess/imports/postgres-mysql-planetscale-migration-guide > [!WARNING] > > **PlanetScale now supports Postgres**. This guide is for migrating from Postgres to PlanetScale's Vitess product. You can still use these scripts if you would like to utilize [Vitess](/docs/vitess). If you prefer to stay on Postgres, refer to the [Postgres import guides](/docs/postgres/imports/postgres-imports). This guide covers how to import a Postgres database to PlanetScale for Vitess using an intermediate Aurora MySQL database. For this, we will be using the [postgres-mysql-planetscale scripts](https://github.com/planetscale/migration-scripts/tree/main/postgres-mysql-planetscale). This technique is good for larger databases that need a fast import, but requires an intermediate MySQL instance. If you have a smaller database or don't mind a slower import, consider [an alternate approach](/docs/vitess/imports/postgres-planetscale-migration-guide). You are encouraged to modify these scripts to suit your needs if need be. ## Methodology The scripts in this guide leverage [AWS Database Migration Service](https://aws.amazon.com/dms/) to handle conversions between Postgres and MySQL types. It also creates a new Aurora MySQL database to use as a go-between for Postgres and PlanetScale. Even if you are migrating from a non-AWS Postgres provider, such as Neon or Supabase, you will still need an AWS account to perform the migration. When using this script, your data will take the following path: * Data flows from your Postgres source into DMS * DMS does necessary type conversions and copies the data into the Aurora MySQL database * Using a PlanetScale [database import](/docs/vitess/imports/database-imports), your data will flow from Aurora MySQL into your destination PlanetScale database * After the initial copy, changes will continue to flow form Postgres, to Aurora MySQL, to PlanetScale so that your data stays in sync, even if the migration takes several hours or days. ![Import data flow](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/postgres-mysql-planetscale-darkmode.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=165c4b036dfb0c8b39ac411a34ba36cd) It is up to you to determine how to handle the cut over between the two in your application. ### Prerequisites * An AWS account * An empty PlanetScale database as the target * The [AWS CLI](https://aws.amazon.com/cli/) > [!WARNING] > > These import scripts create and modify resources in your AWS account. > Before executing, you should read through the scripts to ensure you are comfortable with the actions they will take. > You will also be billed in AWS for the resources it creates, which include: > > * 1 DMS replication task > * 1 DMS replication instance > * 1 DMS replication subnet group > * 2 DMS endpoints > * 1 Aurora MySQL database ## Importing a database ### 1. Prepare Postgres for migration Before beginning a migration to PlanetScale, you should ensure the following flags are set on your Postgres database. | flag name | flag value | | - | - | | logical\_replication | 1 | | shared\_preload\_libraries | pglogical | Given the variety of Postgres options and versions, there may be additional flags that you will need to adjust to make the import work. If you encounter errors when using the script below, it may be caused by other Postgres options not shown here. You can either update your Postgres configuration based on the error you see, or [contact support](https://planetscale.com/contact?initial=support) for additional assistance. > [!WARNING] > > You should not make any schema changes to the source database during an import. ### 2. Create an EC2 instance These scripts are designed to be run from an EC2 instance on the same account that you will authenticate into. Create one, and then log in to the instance. Ensure that both Postgres and MySQL are installed: ``` sudo apt update sudo apt install postgresql sudo apt install mysql-server ``` ### 3. Install the AWS CLI The migration scripts we provide rely on [AWS Database Migration Service](https://aws.amazon.com/dms/). To use the scripts, you will need to download and install the [AWS CLI](https://aws.amazon.com/cli/). You will also need to authenticate into the AWS account that you would like to run the migration from. This step is necessary, even if you are importing from a non-AWS Postgres provider. Go ahead and download and install the AWS CLI on the EC2 instance you created in the last step. For example, on **Ubuntu**, you would run: ``` curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip sudo ./aws/install ``` Full instructions can be found on the [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). ### 4. Authenticate into AWS After installing the CLI, you must authenticate into the AWS account that you intend to run the import scripts in. There are several ways to authenticate. You can find instructions in the [AWS documentation](https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html). We recommend you authenticate with [short-term credentials](https://docs.aws.amazon.com/cli/v1/userguide/cli-authentication-short-term.html). The authenticated account will need permissions to both create and modify DMS resources, RDS / Aurora databases, security groups, and parameter groups ### 5. Prepare the import Check out the [migration-scripts](https://github.com/planetscale/migration-scripts) repository, and navigate to the `postgres-mysql-planetscale` directory. ``` git clone https://github.com/planetscale/migration-scripts cd migration-scripts/postgres-mysql-planetscale ``` Since these scripts will be running in your AWS account, it is important that you understand what the scripts are doing before executing them. Before running, look through the scripts and confirm what they will be doing in your AWS account is acceptable to you and your organization. At a high level, this script does the following: 1. Creates a DMS source using the Postgres credentials you provide 2. Creates a new Aurora MySQL database and sets it as the target for the DMS import 3. Creates a DMS import instance (a server to handle the migration) 4. Sets up rules for how to handle the migration If there are any concerns, you should modify the scripts to suit your needs. > [!WARNING] > > This script prepares for a migration between your postgres source and a new MySQL database. The MySQL database will be accessible from all IPs. > > If you want a tighter security configuration, modify the script to make the database only accessible from the [required PlanetScale IPs](/docs/vitess/imports/import-tool-migration-addresses). If you are comfortable proceeding, the next step is to execute the `prepare.sh` script. You will need to provide this with a unique identifier for this import, as well as the connection credentials for the source Postgres database. > [!NOTE] > > If you are importing from **Supabase**, the scripts will not work with a transaction pooler or session pooler connection. > You must use a direct connection over ipv4. > In order to use this, you must be on the pro plan or greater, and pay for the ipv4 connection upgrade. > After doing so, use the direct connection credentials and host when using `import.sh`. > > If you are importing from **Neon**, You must use `--tls` mode when importing from Neon. > This sets `SSL_MODE="require"` on the connection, a necessity for Neon. Here's an example of executing this: ``` sh prepare.sh --identifier "PGtoPSImport01" \ --source "${PG_USER}:${PG_PASSWORD}@${PG_HOST}/${PG_DB}/${PG_SCHEMA}" \ --ips "us-east-1" ``` You can choose whatever identifier you want in place of `PGtoPSImport01`. The variables prefixed with `PG_` are for the Postgres source. Running the script like this will give you occasional log messages indicating which phase of the import process it is at. If you want full debug mode, including each command the script executes, add the `--debug` flag: ``` sh prepare.sh --identifier "PGtoPSImport01" \ --source "${PG_USER}:${PG_PASSWORD}@${PG_HOST}/${PG_DB}/${PG_SCHEMA}" \ --ips "us-east-1" \ --debug ``` This script can take upwards of 15 minutes, particularly because of the step to set up the DMS import server. When this script completes, it will provide the connection information for the MySQL instance and instructions for next steps. For example: ``` ====================================================================================== SETUP COMPLETED SUCCESSFULLY ====================================================================================== MySQL RDS instance information: Hostname: TARGET_HOSTNAME Database: TARGET_DATABASE Admin user: Username: TARGET_USERNAME Password: TARGET_PASSWORD Migration user (for PlanetScale): Username: migration_user Password: MIGRATION_PASSWORD ====================================================================================== NEXT STEPS: ====================================================================================== 1. Log into your new MySQL instance using the credentials above 2. Set up your schema manually 3. Once your schema is ready, run the start.sh script with the same identifier: sh start.sh --identifier \"$IDENTIFIER\" ``` Notably, this step does not actually begin the migration, but sets up the necessary AWS and DMS resources. > [!NOTE] > > If your database is on a PlanetScale cloud or managed plan, you will need to manually provide your IP addresses. > For this, use `--ips "manual"` and then give the script a comma-separated list of IPs as instructed by the script. ### 6. Copy your schema We have configured these scripts so that they do not automatically copy the schema from Postgres to MySQL. This is intentional, as DMS sometimes does not make good choices for how to convert Postgres types to MySQL. Therefore, we leave it up to you to copy the schema before we begin the migration via `import.sh`. There are several ways you can do this, but one option is to use `pg_dump` to get your schema, convert to MySQL types / syntax, the apply it to the MySQL target. First, you'd run a `pg_dump` command: ``` PGPASSWORD=${PG_PASSWORD} pg_dump -h ${PG_HOST} -U ${PG_USERNAME} ${PG_DATABASE} --schema-only --format=p > schema.sql ``` Next, modify `schema.sql`. Remove all excessive lines from the dump, and update all column types to use ones supported by MySQL. Finally, apply this schema to a new database in the MySQL target created by the `prepare.sh` script: ``` echo "CREATE DATABASE ${MYSQL_DATABASE}" | mysql -u ${MYSQL_USER} -p${MYSQL_PASSWORD} -h${MYSQL_HOST} cat schema.sql | mysql -u ${MYSQL_USER} -p${MYSQL_PASSWORD} -h${MYSQL_HOST} ${MYSQL_DATABASE} ``` Then, log in to the MySQL instance to confirm the schema was applied correctly. ### 7. Migration from Postgres to Aurora MySQL Next, we need to run `start.sh`. This initiates the migration between the Postgres source and the Aurora MySQL target. To start, just run: ``` start.sh --identifier "PGtoPSImport01" ``` You can monitor the progress of the migration by comparing row counts between the source and target database. The initial data copy may range from several minutes to several hours depending on the size of your database. After the initial copy, data changes to your Postgres database will replicate to Aurora MySQL. You do not need to connect to the Aurora MySQL database form your application. We are only using it as a temporary holding-place while moving into PlanetScale. ### 8. Import into PlanetScale To get your data into PlanetScale, we will [import](/docs/vitess/imports/database-imports) the data from the Aurora MySQL instance created in the previous steps into PlanetScale with the `pscale` CLI. First, create the PlanetScale database you're importing into. For large imports, we recommend using [Metal](/docs/metal) for improved import speed. ```bash theme={null} pscale database create --region --cluster-size --wait ``` Next, connect the Aurora MySQL instance to your production branch as an external keyspace. Use the credentials that the script printed out, and the username and password for the `migration_user`. Run the command with `--dry-run` first to check the connection and schema: ```bash theme={null} pscale keyspace create-external main aurora_source \ --host \ --source-database \ --username migration_user \ --password \ --ssl-mode required \ --dry-run ``` If you encounter any errors at this step, look carefully at the error message and address any connection or schema issues as needed. When the check passes, run the same command with `--wait` instead of `--dry-run`. Then start the import with a MoveTables workflow from the external keyspace into your PlanetScale keyspace, which has the same name as your database: ```bash theme={null} pscale branch vtctl move-tables create main \ --workflow import_postgres \ --source-keyspace aurora_source \ --target-keyspace \ --all-tables \ --defer-secondary-keys \ --format json ``` Track the import with `pscale branch vtctl move-tables status`, or on the **Workflows** page in the dashboard. ### 9. Complete the import This full import flow not only copies data, but does continuous replication of traffic from Postgres, to DMS, to Aurora MySQL, and finally to PlanetScale. Replication between these continues until you stop the DMS task using `cleanup.sh` and complete the PlanetScale import. It is up to you to determine how you want to cut over your application to use PlanetScale as your primary database instead of the old Postgres source. Before doing this, you should ensure all of your queries and/or your ORM are updated to work properly with PlanetScale. We also recommend doing some performance testing, and adding indexes if you encounter slow queries. When you're ready, [switch traffic and complete the import](/docs/vitess/imports/database-imports#step-7-switch-traffic), then delete the `aurora_source` external keyspace. ### 10. Clean up import After you have switched all of your traffic over to PlanetScale and are comfortable wrapping up the import, you can clean up the resources that the script created. This includes the DMS migration instance, the Aurora database, and the DMS source / targets. > [!WARNING] > > Do not run this until you are absolutely sure you no longer need the migration set up. To clean up the resources, you can run: ``` sh cleanup.sh --identifier "PGtoPSImport01" ``` We recommend double checking that all of the resources were properly cleaned up in your AWS console after running this. ## Resolving errors We have designed these scripts to run as generally as possible and have tested them on a variety of platforms. Even so, you may encounter errors for a variety of reasons. In some cases, the `aws` command that fails will produce a useful error message in the script output, which you can take action on as needed. If the script is able to set up all of the resources without error but the import is not working, we recommend you take a look at the migration task logs. You can view and search through these in the AWS web console. Log in to the web console, and go to the AWS DMS service page. In the sidebar, click on "Database migration tasks." ![AWS DMS landing page](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/imports/aws-dms.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=4fd7e3116d6ff25ce77b25e868e88dd9) You should see a migration task in the list with a name that corresponds to the identifier you chose. ![AWS DMS migration tasks](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/imports/aws-dms-migration-tasks.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=5a83fc179853103d6407e66e3b47fef0) Click on the migration task, and then click "View logs" in the top right corner. ![AWS DMS migration task choice](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/imports/aws-dms-migration-task-choice.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=1fa56d1a8c2d4edd7e0867768945c3b9) This will bring you to your CloudWatch logs for this replication task, and you can search through it for error and warning messages to help you pinpoint the issue. ![AWS DMS migration task logs](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/imports/aws-dms-migration-task-logs.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=e6718fc701f7354ed1c5bdfa0b7be24f) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Postgres to PlanetScale for Vitess migration guide Source: https://planetscale.com/docs/vitess/imports/postgres-planetscale-migration-guide > [!WARNING] > > **PlanetScale now supports Postgres**. This guide is for migrating from Postgres to PlanetScale's Vitess product. You can still use these scripts if you would like to utilize [Vitess](/docs/vitess). If you prefer to stay on Postgres, refer to the [Postgres import guides](/docs/postgres/imports/postgres-imports). This guide covers how to do an import directly between a Postgres source database and a PlanetScale for Vitess target. For this, we will be using the [postgres-planetscale scripts](https://github.com/planetscale/migration-scripts). This method of importing can be slow. If you have a large database, you might consider trying [an alternative, faster approach](/docs/vitess/imports/postgres-mysql-planetscale-migration-guide). You are encouraged to modify these scripts to suit your needs if need be. ## Methodology The scripts in this guide leverage [AWS Database Migration Service](https://aws.amazon.com/dms/) to handle conversions between Postgres and MySQL types. Even if you are migrating from a non-AWS Postgres provider, such as Neon or Supabase, you will still need an AWS account to perform the migration. DMS acts as a middle-man between your Postgres source database and PlanetScale. All of your data will pass through the DMS server which will handle type conversions before passing the data on to your PlanetScale database. After the initial data copy, DMS will continue to replicate changes from your source to the target until you stop it. It is up to you to determine how to handle the cut over between the two in your application. ![Import data flow](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/postgres-planetscale-darkmode.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=0bafdb11255e7a49f85e424a9636a558) ### Prerequisites * An AWS account * An empty PlanetScale database as the target * The [AWS CLI](https://aws.amazon.com/cli/) > [!WARNING] > > These import scripts create and modify resources in your AWS account. > Before executing, you should read through the scripts to ensure you are comfortable with the actions they will take. > You will also be billed in AWS for the resources it creates, which include: > > * 1 DMS replication task > * 1 DMS replication instance > * 1 DMS replication subnet group > * 2 DMS endpoints ## Importing a database ### 1. Prepare Postgres for migration Before beginning a migration to PlanetScale, you should ensure the following flags are set on your Postgres database. | flag name | flag value | | - | - | | logical\_replication | 1 | | shared\_preload\_libraries | pglogical | Given the variety of Postgres options and versions, there may be additional flags that you will need to adjust to make the import work. If you encounter errors when using the script below, it may be caused by other Postgres options not shown here. You can either update your Postgres configuration based on the error you see, or [contact support](https://planetscale.com/contact?initial=support) for additional assistance. > [!WARNING] > > You should not make any schema changes to the source database during an import. ### 2. Install the AWS CLI The migration scripts we provide rely on [AWS Database Migration Service](https://aws.amazon.com/dms/). To use the scripts, you need to download and install the [AWS CLI](https://aws.amazon.com/cli/). You will also need to authenticate into the AWS account that you would like to run the migration from. This step is necessary, even if you are importing from a non-AWS Postgres provider. Go ahead and download and install the AWS CLI. On **MacOS**, you can run: ``` curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg" sudo installer -pkg AWSCLIV2.pkg -target / ``` On **Linux**, you run: ``` curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip sudo ./aws/install ``` Full instructions can be found on the [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). ### 3. Authenticate into AWS After installing the CLI, you must authenticate into the AWS account that you intend to run the import scripts in. There are several ways to authenticate. You can find instructions in the [AWS documentation](https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html). We recommend you authenticate with [short-term credentials](https://docs.aws.amazon.com/cli/v1/userguide/cli-authentication-short-term.html). The authenticated account will need permissions to both create and modify DMS resources, security groups, and parameter groups. ### 4. Customizing import of large tables If you have large tables that are in need of a [parallel import](https://aws.amazon.com/blogs/database/speed-up-database-migration-by-using-aws-dms-with-parallel-load-and-filter-options/), you should specify custom rules for these to give to DMS. For example, perhaps we have a large table named `items` that contains 1 million rows. We would like DMS to import this in parallel with multiple threads. To configure this, create a json file named `custom-table-mappings.json` and place this in there: ```json expandable theme={null} { "rules": [ { "object-locator": { "schema-name": "public", "table-name": "items" }, "parallel-load": { "boundaries": [ ["250000"], ["500000"], ["750000"] ], "columns": ["item_id"], "type": "ranges" }, "rule-id": "3", "rule-name": "parallel-load-settings", "rule-type": "table-settings" } ] } ``` These rules will be used to tell DMS that it can load the table in 4 parallel threads. It can load rows with `item_id` ranges 0-250k, 250k-500k, 500k-750k, and 750k+ in separate threads. You can add multiple rules here if there are multiple large tables. You do not need to specify custom rules for small tables. If you use custom rules, ensure you pass this file to the `import.sh` script in the next step via `--table-mappings custom-table-mappings.json`. Learn more about the options available to you here in the [AWS table mappings documentation](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TableMapping.html). ### 5. Begin the import Check out the [migration-scripts](https://github.com/planetscale/migration-scripts) repository, and navigate to the `postgres-planetscale` directory. ```bash theme={null} git clone https://github.com/planetscale/migration-scripts cd migration-scripts/postgres-planetscale ``` Since these scripts will be running in your AWS account, it is important that you understand what the scripts are doing before executing them. Before running, look through the scripts and confirm what they will be doing in your AWS account is acceptable to you and your organization. At a high level, the `import.sh` script does the following: **Step 1** Creates a new DMS source and target using the credentials you provide **Step 2** Creates a DMS import instance (a server to handle the migration) **Step 3** Creates / modifies the subnets so the databases can communicate with DMS **Step 4** Sets up rules for how to handle the migration and how to map tables between the source and target **Step 5** Begins the DMS migration task to copy data between the two instances **Step 6** After initial data copy is complete, continues to replicate the data between the instances until you are ready to stop the task If there are any concerns, you should modify the scripts to suit your needs. If you are comfortable proceeding, the next step is to execute the `import.sh` script. You will need to provide this with a unique identifier for this import, as well as the connection credentials for both the source Postgres database and the target PlanetScale database. > [!NOTE] > > If you are importing from **Supabase**, the scripts will not work with a transaction pooler or session pooler connection. > You must use a direct connection over ipv4. > In order to use this, you must be on the pro plan or greater, and pay for the ipv4 connection upgrade. > After doing so, use the direct connection credentials and host when using `import.sh`. > > If you are importing from **Neon**, You must use `--tls` mode when importing from Neon. > This sets `SSL_MODE="require"` on the connection, a necessity for Neon. Here's an example of executing this: ``` sh import.sh --identifier "PGtoPSImport01" \ --source "${PG_USER}:${PG_PASSWORD}@${PG_HOST}/${PG_DB}/${PG_SCHEMA}" \ --target "${PS_USER}:${PS_PASSWORD}@${PS_HOST}/${PS_DB}" ``` You can choose whatever identifier you want in place of `PGtoPSImport01`. The variables prefixed with `PG_` are for the Postgres source, and the ones prefixed with `PS_` are for the PlanetScale target. Running the script like this will give you occasional log messages indicating which phase of the import process it is at. If you want full debug mode, including each command the script executes, add the `--debug` flag: ``` sh import.sh --identifier "PGtoPSImport01" \ --source "${PG_USER}:${PG_PASSWORD}@${PG_HOST}/${PG_DB}/${PG_SCHEMA}" \ --target "${PS_USER}:${PS_PASSWORD}@${PS_HOST}/${PS_DB}" \ --debug ``` If you are running the script for the first time, we recommend using `--debug` in case you encounter any issues needing debugging. ### 6. Completing the import The `import.sh` script can take upwards of 20 minutes to prepare all of the resources before even beginning the import. After this, the time for the import itself varies widely depending on the size and load of your database. You should monitor the progress of the migration by comparing row counts in PlanetScale to ones from the source. Once the migration is complete, it is up to you to determine when you want to cut over your application to use PlanetScale as your primary database instead of the old Postgres source. Before doing this, you should ensure all of your queries are updated to work properly with PlanetScale and do some performance testing. If you encounter performance issues, you likely need to add indexes. Use [PlanetScale Query Insights](/docs/vitess/monitoring/query-insights) to discover and improve poor performing queries. ### 7. Cleaning up the import When you have fully switched all traffic over to PlanetScale and are comfortable stopping the replication between Postgres and PlanetScale, you can use the `cleanup.sh` script to delete the DMS resources that `import.sh` created. All you have to provide is the `--identifier` that you used when starting the import. ``` sh cleanup.sh --identifier "PGtoPSImport01" ``` We recommend double checking that all of the resources were properly cleaned up in your AWS console after running this. ## Resolving errors We have designed these scripts to run as generally as possible and have tested them on a variety of platforms. Even so, you may encounter errors for a variety of reasons. In some cases, the `aws` command that fails will produce a useful error message in the script output, which you can take action on as needed. If the script is able to set up all of the resources without error but the import is not working, we recommend you take a look at the migration task logs. You can view and search through these in the AWS web console. Log in to the web console, and go to the AWS DMS service page. In the sidebar, click on "Database migration tasks." ![AWS DMS landing page](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/imports/aws-dms.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=4fd7e3116d6ff25ce77b25e868e88dd9) You should see a migration task in the list with a name that corresponds to the identifier you chose. ![AWS DMS migration tasks](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/imports/aws-dms-migration-tasks.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=5a83fc179853103d6407e66e3b47fef0) Click on the migration task, and then click "View logs" in the top right corner. ![AWS DMS migration task choice](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/imports/aws-dms-migration-task-choice.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=1fa56d1a8c2d4edd7e0867768945c3b9) This will bring you to your CloudWatch logs for this replication task, and you can search through it for error and warning messages to help you pinpoint the issue. ![AWS DMS migration task logs](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/imports/aws-dms-migration-task-logs.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=e6718fc701f7354ed1c5bdfa0b7be24f) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Airbyte integration Source: https://planetscale.com/docs/vitess/integrations/airbyte With PlanetScale Connect, you can extract data from your PlanetScale database and safely load it into other destinations for analysis, transformation, and more. We implemented an [Airbyte](https://airbyte.com/) connector as the pipeline between your PlanetScale source and selected destination. This document will walk you through how to connect your PlanetScale database to Airbyte. ## Connect to Airbyte Only [Airbyte Open Source](https://docs.airbyte.com/quickstart/deploy-airbyte) supports the PlanetScale data source. In this section, you'll learn how to set up Airbyte and connect your PlanetScale source. ### Requirements * A PlanetScale database * [Docker Desktop](https://www.docker.com/products/docker-desktop/) (Docker terms apply) ### Set up Airbyte locally **Step 1** Install [Docker Desktop](https://www.docker.com/products/docker-desktop/). **Step 2** Follow the related installation instructions included within the [Airbyte Quickstart Documentation](https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart). **Step 3** Open Airbyte in the browser at [http://localhost:8000](http://localhost:8000). ### Set up PlanetScale source Now that Airbyte is running locally, let's set up the custom PlanetScale source. **Step 1** In the Airbyte dashboard, click "**Settings**" on the bottom left. **Step 2** Click "**Sources**" on the left sidebar. **Step 3** Click the "**New connector**" button. **Step 4** Click the "**Add a new Docker connector**" option. Fill in the connector values as follows: * **Connector display name**: PlanetScale * **Docker repository name**: planetscale/airbyte-source * **Docker image tag**: `latest` * **Connector Documentation URL**:(/vitess/integrations/airbyte You can find the [PlanetScale Airbyte Source Dockerhub release page here](https://hub.docker.com/r/planetscale/airbyte-source). ![Airbyte new PlanetScale connector](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/modal.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=9c89d059bcd4372b9a95336817af9a6f) ### Fill in PlanetScale connection information You're now ready to connect your PlanetScale database to Airbyte. **Step 1** Click on the database and branch you want to connect to. **Step 2** Click "**Connect**", select "**General**" from the "**Connect with**" dropdown. **Step 3** Leave this tab open, as you'll need to copy these credentials shortly. **Step 4** Back in Airbyte, click "**Sources**" in the main left sidebar > "**New source**". **Step 5** Select the new PlanetScale source you created from the dropdown. **Step 6** Fill in the "**Set up the source**" values as follows: * **Name**: Any name of your choice * **Source type**: Select "PlanetScale" * **Host**: Paste in the copied value for `host` * **Database**: Paste in the copied value for `database` * **Username**: Paste in the copied value for `username` * **Password**: Paste in the copied value for `password` ![Airbyte - PlanetScale source setup](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/db-info.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=53101c9baf86f5b9ceabe6db28caa8e4) **Step 7** You can also provide some optional values: * **Replicas**: Select whether or not you want to collect data from replica nodes. * **Shards**: Map your shards. * **Starting GTIDs**: Start replication from a specific GTID per keyspace shard. ![Airbyte - PlanetScale optional setup](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/optional.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=fe5f352db1557ac739cc0939060529ae) You can see the [PlanetScale airbyte-source README](https://github.com/planetscale/airbyte-source/blob/main/README.md) for more details on these options. **Step 8** Click "**Set up source**" to connect. You should get a success message that the connection test passed. ### Choose your destination With the connection complete, you can now choose your destination. **Step 1** Click "**Destinations**" in the sidebar or the "**New destination**" button on the source connection page. **Step 2** Set up the destination you want to sync your data to. Each destination should have a Setup Guide linked on its destination setup page. ### Configure a connection Now to get the connection fully set up. Click on "Connections" on the left side bar. If you have not yet set up any connectors, you should see this: ![Airbyte - New connection](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/create.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=1983287a414b7bb7f90844ffddbc38ab) Click the button to set up a connection. Otherwise, click "**New Connection**" in the top right corner. From here, follow these steps: **Step 1** On the "**Define source**" page, choose your PlanetScale source as the **source**. ![Airbyte - Source](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/source.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=7ad44bdfc5e175db692cbf8d8fbfc4b5) **Step 2** On the "**Define destination**" page, select the **destination** you want to sync your PlanetScale data to. For this demo, we are using a CSV destination. ![Airbyte - Source](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/destination.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=2db7f41a6ef6f7e006d91fbb1691587d) **Step 3** On the "**Select streams**" page, select a sync mode. ![Airbyte - Source](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/streams.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=2c8f5bcc55fc8ddbe190835ff5fb7a94) **Step 4** Also on this page, you will need to select the specific tables and columns you want to sync. For each, choose what type of sync mode you'd like to use for each source table. ![Airbyte - Sync](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/sync.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=ba24aaa02dcc0ee19eeefd58dfa03420) * **Incremental** — Incremental sync pulls *only* the data that has been modified/added since the last sync. We use [Vitess VStream](https://vitess.io/docs/concepts/vstream/) to track the stopping point of the previous sync and only pull any changes since then. * **Full refresh** — Full refresh pulls *all* data at every scheduled sync frequency. **Step 5** On the "**Configure connection**" page, choose a sync frequency, which is how often we will connect to your PlanetScale database to download data. ![Airbyte - Connection](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/connection.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=8eca240239c85b56d2a9388ec0b6a9e9) **Step 6** Click "**Finish and sync**". Everything is now configured to pull your PlanetScale data into Airbyte and sync it to the selected destination on the schedule you chose. To run the connection, click "**Connections**" > "**Launch**". ## Handling schema changes Airbyte will not automatically detect when you make schema changes to your PlanetScale database. If you drop a column, your sync should throw an error as it looks for a column that doesn't exist. However, if you add a column, the sync will continue without any errors. Airbyte will be unaware of the new column altogether. This is known as schema drift. Whenever you perform a schema change, you need to notify Airbyte of it: **Step 1** In the Airbyte dashboard, click "**Connections**", select the connection, then navigate to the "**Schema**" tab. **Step 2** Click "**Refresh source schema**". **Step 3** Click "**Save changes**". Keep in mind, this might delete all data for the connection and start a new sync from scratch. ## Stopping Airbyte At any point, you can disable any incremental or full syncs by going to the 'Connection' settings page and clicking 'Delete this connection'. This will not touch any of the source or destination data, but will prevent Airbyte from doing any further operations. ![Airbyte - PlanetScale disconnection](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/airbyte/delete.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=ddfa9b8ec7f6025cf29bba29b4d076d0) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Artie integration Source: https://planetscale.com/docs/vitess/integrations/artie Connect PlanetScale Vitess to Artie so you can extract data and safely load it into other destinations for analysis, transformation, and more. Artie continuously replicates inserts, updates, and deletes from PlanetScale into supported warehouses, lakehouses, and object stores. Artie connects through Vitess VStream rather than standard MySQL binary-log replication, so you do not need to configure binlog replication, GTID, or MySQL replication privileges. In this section, you'll learn how to set up Artie and connect your PlanetScale source. ## Connect to Artie ### Requirements * A PlanetScale Vitess database * An Artie account * A destination configured in Artie. See Supported destinations below. #### Connect using private networking If you are using private networking and not connecting directly to your PlanetScale database, the Artie integration supports AWS PrivateLink and SSH tunnels. See the Artie AWS PrivateLink setup guide or Artie SSH tunnel setup guide for details. ### Set up Artie 1. In PlanetScale, navigate to the database you want to connect to Artie and click the "**Connect**" button. 2. Create a new password for the branch you want to replicate with read-only permissions. The read-only role provides the `SELECT` access Artie needs for the initial backfill and the VStream access it needs for ongoing replication. The password you create determines which branch Artie replicates. To replicate another branch, create a separate Artie source with a password for that branch. 3. Copy the connection values from the password details. 4. In the Artie dashboard, create a new pipeline and select "**PlanetScale**" as the source. Fill in the connection information: * **Host**: Paste in the copied value for `host` * **Port**: `3306` * **Username**: Paste in the copied value for `username` * **Password**: Paste in the copied value for `password` * **Database**: The keyspace to replicate. This is usually the database name. The password that you create sets the branch. TLS is enforced automatically. No additional TLS configuration is required. 5. Allow traffic on both ports Artie uses. The backfill and metadata connection uses port `3306`. Change capture (VStream) uses port `443`. If your password has IP restrictions, or if you use firewall rules, let traffic through on both ports for the Artie fixed IP addresses in the region that runs your pipeline. For other network setups, contact Artie support. 6. Select the tables you want to replicate. Tables are identified by keyspace and table name. Tables from different keyspaces are replicated to separate destination tables by default. See Multiple keyspaces below. 7. Select your destination, review the pipeline settings, and click "**Deploy**." Artie validates read access and the VStream endpoint, begins the initial backfill, and starts capturing changes. **Note:** If validation reports a VStream credential, connectivity, or timeout error, confirm that the password has the read-only role and that Artie can reach your PlanetScale endpoint on ports `3306` and `443`. ### Supported destinations Artie can replicate Vitess data to the following destinations. See the Artie documentation for destination-specific setup. * Data warehouses: Snowflake, BigQuery, Databricks, Redshift, ClickHouse, MotherDuck * Lakehouses: Apache Iceberg (including Amazon S3 Tables and REST Catalog deployments), Delta Lake * Object storage: Amazon S3, Google Cloud Storage ## Sync overview Artie starts the initial copy and change capture together: 1. **Backfill**: Artie reads the selected tables and loads their existing rows into your destination. 2. **Streaming**: Artie subscribes to VStream and captures inserts, updates, and deletes that occur while the backfill is running and afterward. Changes captured during the backfill are queued. When a table's backfill finishes, Artie applies those changes in order so the destination does not retain stale backfill values. VStream is part of Vitess. Because Artie receives row-level changes through VStream, this integration does not use the MySQL binlog and does not require `REPLICATION CLIENT` or `REPLICATION SLAVE` privileges, binlog retention settings, GTID, or a replica. A read-only password provides all the access Artie needs. ### Multiple keyspaces Vitess namespaces tables by keyspace. In Artie, you select and browse one keyspace at a time. A table's source identity includes its keyspace, so tables with the same name in different keyspaces are replicated separately by default. Artie can unify a table across keyspaces so the same logical table lands in one destination table. Enable "**Unify across keyspaces**" in the table's pipeline settings. You can select the keyspaces in two ways: * **Static**: A list of keyspaces. Artie backfills the selected table in each keyspace and then captures its changes through VStream. To include a new keyspace, add it to the list and run a backfill. * **Dynamic**: A regular expression, such as `^customer_(us|eu)$`. Artie finds new keyspaces that match and replicates from them. Discovery runs at most every five minutes. Only unify tables that have compatible schemas and the same merge-key semantics. ## Schema information Artie creates a destination table for each selected table and maps source column names and types to the closest supported destination types. See Artie's typing library for type-mapping details. ### Artie-generated columns Artie can add columns prefixed with `__artie_` to destination tables. Optional columns are configured in the pipeline's table or destination settings. | Column | Type | Description | | - | - | - | | `__artie_delete` | `BOOLEAN` | `true` when the row was deleted in your database. Added when soft deletes are enabled. | | `__artie_operation` | `STRING` | `r` for backfill, `c` for create, `u` for update, and `d` for delete. Optional in normal mode and included in history records in history mode. | | `__artie_updated_at` | `TIMESTAMP_TZ` | When Artie processed the change. Added in history mode. | | `__artie_db_updated_at` | `TIMESTAMP_TZ` | When the change occurred in the source database. Added in history mode. | | `__artie_full_source_table_name` | `STRING` | Fully qualified source table name. Optional. | | `__artie_source_metadata` | `JSON` | Source-specific event metadata. Optional. | See System columns for the complete list and destination-specific availability. ### Adding and dropping columns Artie detects new source columns. For eligible pipelines, a company admin can enable automatic schema reconciliation to add missing source columns to an existing destination table. By default, Artie does not rename or change the type of an existing destination column. Dropping destination columns requires the opt-in hard-delete setting and its safety checks. See schema evolution for supported changes and configuration. ## Excluding source data Choose the tables to replicate when you create the pipeline, then add or remove tables at any time. Within a table, use column inclusion and exclusion to choose which columns Artie writes to the destination. You can also hash selected columns before Artie writes them. ## Initial sync When you deploy a pipeline, Artie backfills each selected table, then continues applying VStream changes. You can run a new backfill for a table from the Artie dashboard without recreating the pipeline. See Backfills for behavior and controls. ## Updating data After the initial sync, Artie applies VStream inserts, updates, and deletes to the destination using the table's merge key. Artie batches changes and flushes them to the destination on a configurable interval. See flush rules to balance freshness against destination load. ### Deleted rows By default, a row deleted in your database is deleted from the destination. Enable **soft deletes** to retain the row and set `__artie_delete = true`. Enable **history mode** to create a separate `{TABLE}__HISTORY` table that records every insert, update, and delete with operation and timestamp columns. ## Need help? For help with Artie, contact Artie support or see the [Artie documentation](https://www.artie.com/docs/quickstart). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cloudflare Workers database integration Source: https://planetscale.com/docs/vitess/integrations/cloudflare-workers Create a new Vitess database and integrate it with Cloudflare Workers and Hyperdrive. [Cloudflare Workers](https://developers.cloudflare.com/workers/) is a serverless platform that allows you to run your code at the edge, close to your users. [Hyperdrive](https://developers.cloudflare.com/hyperdrive/) accelerates queries you make to existing databases, including PlanetScale. Want to create and pay for your database from Cloudflare without adding billing to PlanetScale? See [Create PlanetScale databases from Cloudflare](/docs/connect/cloudflare). Already created a PlanetScale Vitess database? [Jump straight to integration instructions](#integrate-with-cloudflare-workers). We'll cover: * Creating a new Vitess database * Cluster configuration options * Connecting to your database ## Prerequisites Before you begin, make sure you have a [PlanetScale account](https://auth.planetscale.com/sign-up). After you create an account, you'll be prompted to create a new organization, which is essentially a container for your databases, settings, and members. After creating your organization, it's important to understand the relationship between databases, branches, and clusters. * **Database**: Your overall project (e.g., "my-ecommerce-app") * **Branch**: Isolated database deployments that provide you with separate environments for development and testing, as well as restoring from backups - [learn more about branching](/docs/vitess/schema-changes/branching) * **Cluster**: The underlying compute and storage infrastructure that powers each branch PlanetScale Vitess clusters use [Vitess](https://vitess.io/), a database clustering system for horizontal scaling of MySQL. ## Create a new database **Dashboard** [Create a database on PlanetScale](https://www.youtube-nocookie.com/embed/6BBrgJcpTBY?rel=0) ### Step 1: Navigate to database creation **Step 1** Log in to your [PlanetScale dashboard](https://app.planetscale.com) **Step 2** Select your organization from the dropdown **Step 3** Click **"New database"** button or navigate to `/new` ### Step 2: Choose database engine **Step 1** On the database creation form, you'll see two engine options: * **Vitess** (MySQL-compatible) * **Postgres** (PostgreSQL-compatible) **Step 2** Select **Vitess** to create a MySQL-compatible database ### Step 3: Configure your database cluster **Step 1** **Database name**: Enter a unique name for your database > [!NOTE] > > Your database name can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. **Step 2** **Region**: Choose the primary region where your database will be hosted. For the lowest latency, select a region near you or your application's hosting location. **Step 3** **Cluster configuration**: Select your preferred [cluster and storage size](/docs/plans/planetscale-skus) for your database. ### Step 4: Create the database cluster **Step 1** Review your configuration settings **Step 2** Click **"Create database"** to provision your Vitess database **Step 3** Your database will be created with a `main` branch by default **CLI** If you are creating an automation, or are an LLM, you may prefer to create new databases using the PlanetScale CLI. ### Step 1: Install the CLI **Step 1** Check to see if the PlanetScale CLI is installed already by running: ```bash theme={null} pscale --version ``` **Step 2** Alternatively, follow the instructions in the [PlanetScale CLI GitHub repository](https://github.com/planetscale/cli#installation) ### Step 2: Log in or sign up **Step 1** If you do not already have a PlanetScale account, [sign up for one](https://auth.planetscale.com/sign-up) by running: ```bash theme={null} pscale signup ``` **Step 2** Log in to the PlanetScale CLI by running: ```bash theme={null} pscale auth login ``` You'll be taken to a screen in the browser where you'll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the **"Confirm code"** button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Step 3: Create a database **Step 1** Configure the CLI to use the **organization** in which you want to create the database if you have more than one. List organizations by running: ```bash theme={null} pscale org list ``` Switch organizations by running: ```bash theme={null} pscale org switch ``` **Step 2** Find the **region** closest to your application's hosting. List available regions by running: ```bash theme={null} pscale region list ``` > [!NOTE] > > If you do not specify a **region**, your database will automatically be deployed to `us-east` (US East — Northern Virginia). **Step 3** Create a new Vitess database by running: ```bash theme={null} pscale database create --region --engine mysql ``` > [!NOTE] > > Your **database name** can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. ## What happens during creation When you create a Vitess database cluster, PlanetScale automatically: * Provisions a Vitess cluster in your selected region * Creates the initial `main` branch * Sets up monitoring and metrics collection * Configures backup and high availability settings ## Integrate with Cloudflare Workers Don't have a Workers project yet? [Create a Workers project from the MySQL Hyperdrive template](https://developers.cloudflare.com/workers/get-started/quickstarts/#mysql-hyperdrive-template). ```bash Terminal theme={null} npm create cloudflare@latest -- --template=cloudflare/templates/mysql-hyperdrive-template ``` ### Step 1: Create a Hyperdrive connection You can automatically create a connection from the PlanetScale dashboard when creating a new role, or use one of the methods below. **Dashboard** **Step 1** Log into the Cloudflare dashboard and navigate to **"Compute & AI"** > **"Workers & Pages"**. You should see your Worker in the list. ![PlanetScale Cloudflare integration wizard - step 1](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_1.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=21239d9ba0e43e1616b020c77fe617b7) **Step 2** Select the **"Storage & databases"** tab, then **"Hyperdrive"**, and finally **"Create configuration"** in the top right corner. ![PlanetScale Cloudflare integration wizard - step 2](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_2.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=a6ca84d0b9f0f7a54dc37911e5d9f0b5) **Step 3** Click **"Connect to PlanetScale database"** then click **Next**. ![PlanetScale Cloudflare integration wizard - step 3](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_3.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=c42406780d8a671ed5c8d01b9be867f0) **Step 4** Click **Login to PlanetScale** to log in to your PlanetScale account. ![PlanetScale Cloudflare integration wizard - step 4](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_4.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=ee9e19bd44f298fe8a85958eee20a965) **Step 5** The next screen will allow you to grant access to your organization, database, and branch. Start by selecting your organization from the list. ![PlanetScale Cloudflare integration wizard - step 5](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_5.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=f22610297689c9d3b6ae4d9a651c1f0b) **Step 6** Select the database you created, the "main" branch and then click **Next**. You must include a **database name**. For MySQL/Vitess databases, the default database name is the same as your project name (for example, `my-project`). For Postgres databases, the default database name is `postgres`. ![PlanetScale Cloudflare integration wizard - step 6](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_6_unedited.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=5135047d924040fb2ce714b86799c146) **Step 7** You should see a success screen with the connection binding details to add to your `wrangler.json` file. ![PlanetScale Cloudflare integration wizard - step 7](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_7.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=83cca64caf0b9e48d33761e8f1551359) **Step 8** You should now have metrics within your Cloudflare dashboard in the Hyperdrive page. ![PlanetScale Cloudflare integration wizard - step 8](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/integrations/cloudflare-workers/pscf_step_8.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=6e2001b97e976e539176075edc476dfb) **CLI** Create a Hyperdrive connection by running the `wrangler` command below: * Replace the `--connection-string` placeholders with your database credentials * Replace the `` placeholder with a name for your Hyperdrive connection **For MySQL/Vitess databases:** ```bash Terminal theme={null} npx wrangler hyperdrive create --connection-string="mysql://:@/" ``` **For Postgres databases:** ```bash Terminal theme={null} npx wrangler hyperdrive create --connection-string="postgresql://:@:/?sslmode=verify-full" ``` If successful, the command will output your new Hyperdrive configuration, for example: ```json Terminal theme={null} { "hyperdrive": [ { "binding": "HYPERDRIVE", "id": "" } ] } ``` ### Configure Worker placement By default, Workers run at the edge close to your users. For database-heavy workloads with multiple round trips, you can use [placement hints](https://developers.cloudflare.com/workers/configuration/smart-placement/) to run your Worker closer to your PlanetScale database and reduce latency. Add a `placement` configuration to your `wrangler.jsonc` file with the region that matches your PlanetScale database region: ```json wrangler.json theme={null} { "placement": { "region": "aws:us-east-1" } } ``` > **PlanetScale region to placement hint mapping** > > | PlanetScale Region | Placement Hint | > | - | - | > | AWS ap-northeast-1 (Tokyo) | `aws:ap-northeast-1` | > | AWS ap-south-1 (Mumbai) | `aws:ap-south-1` | > | AWS ap-southeast-1 (Singapore) | `aws:ap-southeast-1` | > | AWS ap-southeast-2 (Sydney) | `aws:ap-southeast-2` | > | AWS ca-central-1 (Montreal) | `aws:ca-central-1` | > | AWS eu-central-1 (Frankfurt) | `aws:eu-central-1` | > | AWS eu-west-1 (Dublin) | `aws:eu-west-1` | > | AWS eu-west-2 (London) | `aws:eu-west-2` | > | AWS sa-east-1 (Sao Paulo) | `aws:sa-east-1` | > | AWS us-east-1 (N. Virginia) | `aws:us-east-1` | > | AWS us-east-2 (Ohio) | `aws:us-east-2` | > | AWS us-west-2 (Oregon) | `aws:us-west-2` | > | GCP asia-northeast3 (Seoul) | `gcp:asia-northeast3` | > | GCP europe-west1 (Belgium) | `gcp:europe-west1` | > | GCP europe-west4 (Netherlands) | `gcp:europe-west4` | > | GCP northamerica-northeast1 (Montreal) | `gcp:northamerica-northeast1` | > | GCP us-central1 (Iowa) | `gcp:us-central1` | > | GCP us-east1 (South Carolina) | `gcp:us-east1` | > | GCP us-east4 (Virginia) | `gcp:us-east4` | Your Worker will run in the Cloudflare data center with the lowest latency to the specified region. If your Worker connects to multiple back-end services and you're unsure which region to specify, use `"mode": "smart"` for automatic placement based on measured latency: ```json wrangler.json theme={null} { "placement": { "mode": "smart" } } ``` ### Step 2: Deploy your Worker Run the following to deploy your Worker. ```bash Terminal theme={null} npx wrangler deploy ``` For more information on using Cloudflare Workers and Hyperdrive, [refer to the Cloudflare documentation](https://developers.cloudflare.com/hyperdrive/). ## What's next? Once you're done with development, it is highly recommended that [safe migrations](/docs/vitess/schema-changes/safe-migrations) be turned on for your `main` production branch to protect from accidental schema changes and enable zero-downtime deployments. When you're ready to make more schema changes, you'll [create a new branch](/docs/vitess/schema-changes/branching) off of your production branch. Branching your database creates an isolated copy of your production schema so that you can easily test schema changes in development. Once you're happy with the changes, you'll open a [deploy request](/docs/vitess/schema-changes/deploy-requests). This will generate a diff showing the changes that will be deployed, making it easy for your team to review. Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Datadog integration Source: https://planetscale.com/docs/vitess/integrations/datadog PlanetScale can push metrics to Datadog to assist your team with understanding your database usage and performance. > [!WARNING] > > This Datadog integration is no longer receiving updates or new metric additions. You should instead use our [Datadog Agent Integration](/docs/vitess/tutorials/prometheus-metrics-datadog), which provides more metrics than this native integration. > > If you have any questions about migrating to the Datadog Agent Integration, [reach out to support](https://planetscale.com/contact?initial=support). ## Prerequisites * A [Datadog](https://www.datadoghq.com/) account ## Configuring the Datadog integration **Step 1** In Datadog, install the [PlanetScale integration](https://app.datadoghq.com/integrations/planetscale). **Step 2** Create a Datadog API key in your [Datadog Organization Settings](https://app.datadoghq.com/organization-settings/api-keys) and copy the key. **Step 3** In PlanetScale, go to your organization's [Integrations settings](https://app.planetscale.com/settings/integrations), and select **Configure** for the Datadog integration. Paste your Datadog API key into the field. Once complete, a "PlanetScale" dashboard will be available with incoming metrics from PlanetScale. ![PlanetScale Default Dashboard in Datadog](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/datadog/dashboard.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=7870ae02484c84e746a1d907f9be79bb) ## Metrics We Collect Once configured, PlanetScale collects the following metrics from every branch in your organization. | **Metric name** | **Metric type** | **Description** | | :- | :- | :- | | planetscale.connections | gauge | Number of active connections to a database branch. *Shown as connection.* | | planetscale.primary.cpu\_usage | gauge | Percentage of CPU utilized on a database branch's primary. *Shown as percent.* | | planetscale.primary.memory\_usage | gauge | Percentage of memory utilized on a database branch's primary. *Shown as percent.* | | planetscale.queries.latency | gauge | Query times in the p50, p95, p99 and p999 percentiles. *Shown as millisecond.* | | planetscale.replication\_lag | gauge | Replication lag in seconds between a database branch's primary and each replica. *Shown as second.* | | planetscale.rows\_read | count | Number of rows read from a database branch. *Shown as row.* | | planetscale.rows\_written | count | Number of rows written to a database branch. *Shown as row.* | | planetscale.tables.cumulative\_query\_time | count | Cumulative active query time in a database branch, by table and statement. *Shown as nanosecond.* | | planetscale.tables.queries | count | Number of queries issued to a database branch, by table and statement. *Shown as query.* | | planetscale.tables.rows\_deleted | count | Number of rows deleted from a database branch, by table. *Shown as row.* | | planetscale.tables.rows\_inserted | count | Number of rows inserted into a database branch, by table. *Shown as row.* | | planetscale.tables.rows\_selected | count | Number of rows selected in a database branch, by table. *Shown as row.* | | planetscale.tables.rows\_updated | count | Number of rows updated in a database branch, by table. *Shown as row.* | | planetscale.tables.storage | gauge | Total bytes stored in a database branch, by table. *Shown as byte.* | | planetscale.vtgate.errors | count | Number of errors encountered by a database branch's vtgate. *Shown as error.* | | planetscale.vttablet.mem\_util.max | gauge | Maximum memory utilization of a database branch's vttablet. *Shown as percent.* | | planetscale.vttablet.mem\_util.avg | gauge | Average memory utilization of a database branch's vttablet. *Shown as percent.* | | planetscale.vttablet.iops | gauge | Number of IOPS performed by a database branch's vttablet. *Shown as operation.* | ## Billing The Datadog integration is available on all of our [paid plans](https://planetscale.com/pricing). ## Frequently asked questions ### How do I track replication lag in Datadog? You can use the following formula to set alerts for replication lag: ```bash theme={null} (max:planetscale.replication_lag{ps_database: ps_tablet_type:replica, ps_branch:
}) ``` Make sure you replace `` with your PlanetScale database name and `
` with the name of the branch for which you'd like to track replication lag. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Debezium connector for PlanetScale Source: https://planetscale.com/docs/vitess/integrations/debezium The Debezium connector for PlanetScale is a connector for Debezium, an open-source distributed platform for change data capture. The [Debezium connector for PlanetScale](https://github.com/planetscale/debezium-connector-planetscale) is based on the [Debezium Vitess connector](https://debezium.io/documentation/reference/stable/connectors/vitess.html) and packages the PlanetScale-specific connector classes, patches, and runtime dependencies for Kafka Connect and Debezium Server. It streams change events from a PlanetScale Vitess database through [VStream](https://vitess.io/docs/concepts/vstream/), including the initial copy of existing rows. This page covers which connector release to pick, how to connect it to PlanetScale, the deployment options we test, and what each release is verified against. > [!NOTE] > > Looking for the 2.x connector? See [Debezium connector for PlanetScale (archived)](/docs/vitess/integrations/debezium-archived). ## Pick a release We publish one connector release line per upstream Debezium minor version, so you can match the Debezium version already running in your environment. Each line tracks the latest upstream patch release of that minor. Release tags look like `v-r`, where `r` counts PlanetScale releases on top of the same upstream version. | Debezium line | Connector release | Debezium Server | Kafka Connect / Confluent | | - | - | - | - | | 3.2 | [v3.2.7.Final-r2](https://github.com/planetscale/debezium-connector-planetscale/releases/tag/v3.2.7.Final-r2) | ✅ | ✅ | | 3.3 | [v3.3.2.Final-r1](https://github.com/planetscale/debezium-connector-planetscale/releases/tag/v3.3.2.Final-r1) | ✅ | ✅ | | 3.4 | [v3.4.3.Final-r1](https://github.com/planetscale/debezium-connector-planetscale/releases/tag/v3.4.3.Final-r1) | ✅ | ✅ | | 3.5 | [v3.5.2.Final-r1](https://github.com/planetscale/debezium-connector-planetscale/releases/tag/v3.5.2.Final-r1) | ✅ | ✅ | | 3.6 | [v3.6.3.Final-r1](https://github.com/planetscale/debezium-connector-planetscale/releases/tag/v3.6.3.Final-r1) | ✅ | ✅ | If you have no existing Debezium deployment, start with the **3.6 line**, the newest one available. ## Download the connector Every release on the [releases page](https://github.com/planetscale/debezium-connector-planetscale/releases) ships the same set of files: | File | Use it for | | - | - | | `planetscale-debezium-adapter-.jar` | Debezium Server: drop into the server's `lib/` directory | | `planetscale-debezium-connector-planetscale-.zip` | Kafka Connect and Confluent Cloud: the connector plugin archive | | `SHA256SUMS.txt` | Checksums for all files | | `*.sigstore.json`, `release.spdx.json` | Sigstore signatures and an SBOM for supply-chain verification | Verify what you downloaded before deploying it: ```bash theme={null} # in the directory with the downloaded files (on macOS use `shasum -a 256 -c -`) grep -E '\.(jar|zip)$' SHA256SUMS.txt | sha256sum -c - ``` ## Connection settings These settings are the same for every deployment option. In Kafka Connect and Confluent they are plain connector properties; in Debezium Server they carry the `debezium.source.` prefix. | Property | Value | | - | - | | `connector.class` | `com.planetscale.debezium.PlanetscaleConnector` | | `database.hostname` | The hostname from your PlanetScale connection string, for example `aws.connect.psdb.cloud` | | `database.port` | `443` | | `database.user` / `database.password` | A [PlanetScale password](/docs/vitess/connecting/connection-strings) for the branch. The connector connects over TLS automatically. | | `vitess.keyspace` | Your PlanetScale database name | | `vitess.tablet.type` | `MASTER` to stream from the primary, or `REPLICA` to stream from a replica | | `table.include.list` | Comma-separated `.
` names to capture. Omit to capture every table. | | `topic.prefix` | Prefix for the Kafka topics (or event names): events for a table land in `..
` | | `snapshot.mode` | `initial` (default) copies existing rows first, then streams; `never` streams only new changes | Notes: * The connector talks to PlanetScale over gRPC on port 443 with TLS. Outbound access to your connection-string hostname on port 443 is all it needs. * On a sharded PlanetScale database the connector streams all shards; leave `vitess.shard` unset. * Keep `tasks.max=1`. The Vitess connector runs a single task per keyspace. ## Deploy We test every release in the deployment options below. ### Debezium Server with the official image The quickest way to run the connector standalone. Mount the connector jar into the official Debezium Server image for the matching line and mount your configuration. ```properties theme={null} # application.properties debezium.format.value=json debezium.sink.type=http debezium.sink.http.url= debezium.source.connector.class=com.planetscale.debezium.PlanetscaleConnector debezium.source.database.hostname= debezium.source.database.port=443 debezium.source.database.user= debezium.source.database.password= debezium.source.vitess.keyspace= debezium.source.vitess.tablet.type=MASTER debezium.source.topic.prefix=connector-test debezium.source.offset.storage=org.apache.kafka.connect.storage.FileOffsetBackingStore debezium.source.offset.storage.file.filename=data/offsets.dat debezium.source.offset.flush.interval.ms=0 debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory debezium.source.schema.history.internal.file.filename=data/schema_history.dat ``` ```bash theme={null} # pick the Debezium Server image that matches the connector line, e.g. 3.6.3.Final for v3.6.3.Final-r1 docker run --rm \ -v "$PWD/planetscale-debezium-adapter-3.6.3.Final-r1.jar:/debezium/lib/planetscale-debezium-adapter-3.6.3.Final-r1.jar:ro" \ -v "$PWD/application.properties:/debezium/config/application.properties:ro" \ -v "$PWD/data:/debezium/data" \ quay.io/debezium/server:3.6.3.Final ``` Mount `data/` to persist offsets and schema history across restarts. Existing rows arrive as events first (the snapshot), followed by changes as they happen. ### Debezium Server on a VM or your laptop Download the Debezium Server distribution for the same line from Maven Central, unpack it, copy the connector jar into `lib/` and your configuration into `config/application.properties`, then start it with `run.sh`. Requires Java 21. ```bash theme={null} V=3.6.3.Final curl -fLO https://repo1.maven.org/maven2/io/debezium/debezium-server-dist/$V/debezium-server-dist-$V.tar.gz tar -xzf debezium-server-dist-$V.tar.gz cp planetscale-debezium-adapter-$V-r1.jar debezium-server/lib/ cp application.properties debezium-server/config/ cd debezium-server && ./run.sh ``` The repository's [`server/`](https://github.com/planetscale/debezium-connector-planetscale/tree/main/server) helper (`make -C server run`) automates these steps for a local build. ### Self-hosted Kafka Connect Unzip the plugin archive into a directory on the worker's `plugin.path`, restart the worker, and register the connector: ```bash theme={null} unzip planetscale-debezium-connector-planetscale-3.6.3.Final-r1.zip -d /opt/connect/plugins/ ``` ```json theme={null} { "name": "planetscale", "config": { "connector.class": "com.planetscale.debezium.PlanetscaleConnector", "tasks.max": "1", "database.hostname": "", "database.port": "443", "database.user": "", "database.password": "", "vitess.keyspace": "", "vitess.tablet.type": "MASTER", "topic.prefix": "connector-test", "table.include.list": ".
", "schema.history.internal.kafka.bootstrap.servers": "kafka:9092", "schema.history.internal.kafka.topic": "schema-changes.planetscale" } } ``` ```bash theme={null} curl -s -X POST -H 'Content-Type: application/json' --data @planetscale.json http://connect:8083/connectors curl -s http://connect:8083/connectors/planetscale/status ``` Both the connector and its task should report `RUNNING`, and topics named `..
` appear as events arrive. ### Confluent Cloud custom connector Upload the plugin archive as a custom connector plugin, then create a connector from it through the Confluent wizard: pick the `com.planetscale.debezium.PlanetscaleConnector` class, fill in the PlanetScale connection settings (port `443`, Shard left empty), and allow-list `:443` as a connection endpoint. The full walkthrough with screenshots is on its own page. > **[Debezium connector on Confluent Cloud](/docs/vitess/integrations/debezium-confluent)** ## What each release is tested against **Build-time tests** (run in CI on every change): * Value conversion for the Vitess types that differ from MySQL defaults: `BIT` columns, `TIMESTAMP`/`DATETIME` precision modes and zero dates, unsigned `BIGINT` handling, and spatial (`GEOMETRY`) columns. * Connection handling: TLS to PlanetScale, mutual TLS, and the gRPC authentication interceptor that carries the PlanetScale credentials. * Connector configuration validation and start-up, embedded-engine start-up, and the Kafka Connect plugin service registration. * An integration test against a Vitess cluster in a container that exercises VStream end to end. **Release validation** against a live PlanetScale database, using the downloaded release files (checksums verified against `SHA256SUMS.txt`): * **Debezium Server** on the official `quay.io/debezium/server` image of the same line, with the connector jar in `lib/`. * **Debezium Server distribution** (the Maven Central tarball) on a plain JVM, the way a VM or laptop deployment runs it. * **Self-hosted Kafka Connect** with the plugin archive, connector and task reaching `RUNNING`. * **Confluent Cloud** as a custom connector, uploaded from the plugin archive. Each run captures a fixed set of tables and checks that the initial copy and the following inserts, updates and deletes all arrive at the sink with the expected content: a plain table, a table of `TIMESTAMP`/`DATETIME` columns including zero-date values, and a table of `BIT` columns. ## Known limitations * **Zero dates**: a `0000-00-00 00:00:00` value is emitted as `null`, with a warning in the log. This follows Debezium's handling of invalid dates. * **`bigint.unsigned.handling.mode`**: the Vitess connector supports `precise` and `long` only. If your configuration sets `string`, the connector logs a warning and falls back to `precise`. * **Topic naming**: events for a table go to `..
`. Vitess shard names are not part of the topic name. ## Upgrading from the 2.x connector The 2.x connector (archived) was a fork of the Debezium 2.4 Vitess connector shipped as a single jar with dependencies. The 3.x lines are a different artifact: the connector class name changes from `io.debezium.connector.planetscale.PlanetScaleConnector` to `com.planetscale.debezium.PlanetscaleConnector`, the Kafka Connect plugin is a zip archive, and the Debezium Server jar must match the server's Debezium line. Offsets carry over. The 3.x connector reads the offsets written by the 2.x connector (same partition key and the same `vgtid` offset entry), skips the snapshot, and resumes the VStream from the last recorded VGTID. We verify this for both runtimes: the archived connector on Debezium Server 2.4.1 followed by the 3.6 line on the same offsets file, and the archived connector on a Kafka Connect worker followed by the 3.6 line registered under the same connector name against the same offsets topic. In both cases no rows are re-sent and streaming continues from the stored position. To keep the offsets matched after the upgrade: * Keep the same connector name (`name` in Kafka Connect, `debezium.source.name` in Debezium Server) and the same `topic.prefix`. Both are part of the key under which offsets are stored. * Point the new connector at the same offset storage: the same `offset.storage.file.filename` in Debezium Server, or the same offsets topic in Kafka Connect. * Change `connector.class` to the new class name. Columns of type `BIT` were not supported by the 2.x connector and were dropped from events with an `unknown column type` warning. The 3.x lines emit them, so tables with `BIT` columns gain those fields after the upgrade. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Debezium connector on Confluent Cloud Source: https://planetscale.com/docs/vitess/integrations/debezium-confluent Run the Debezium connector for PlanetScale as a custom connector on Confluent Cloud. The [Debezium connector for PlanetScale](/docs/vitess/integrations/debezium) runs on Confluent Cloud as a [custom connector](https://docs.confluent.io/cloud/current/connectors/bring-your-connector/overview.html): you upload the connector's Kafka Connect plugin archive once, then create connectors from it like any managed connector. This page walks through the upload and the connector wizard. Before you start, you need: * A Confluent Cloud cluster in a [region that supports custom connectors](https://docs.confluent.io/cloud/current/connectors/bring-your-connector/custom-connector-fands.html). * The plugin archive `planetscale-debezium-connector-planetscale-.zip` from the [releases page](https://github.com/planetscale/debezium-connector-planetscale/releases). Pick the release line as described on the [main page](/docs/vitess/integrations/debezium#pick-a-release). * A PlanetScale password for the branch you want to capture. ## Upload the plugin and create the connector Upload the plugin archive as a custom connector and configure it like a self-hosted Kafka Connect connector. **Step 1** In Confluent Cloud, go to **Connectors → Add plugin**. In the **Upload custom connector plugin** dialog choose **Create new plugin**, give it a name such as `PlanetScale 3.6.3.Final-r1`, pick the cloud provider of your Kafka cluster, and upload `planetscale-debezium-connector-planetscale-.zip` (zip or jar, 250 MB maximum). Take a look at our connector releases [on GitHub](https://github.com/planetscale/debezium-connector-planetscale/releases). ![Confluent Cloud: Upload custom connector plugin dialog with the PlanetScale connector zip](https://mintcdn.com/planetscale-2/WPOFVUxRXVcpW_lU/images/assets/docs/integrations/debezium/confluent-1-upload-plugin.png?fit=max\&auto=format\&n=WPOFVUxRXVcpW_lU\&q=85\&s=7df1eecf204e61c44e0adad5ffe52bb6) When a new connector release comes out, use **Add version to an existing plugin** on the same dialog instead of creating a new plugin. **Step 2** Click **Continue**. Confluent reads the plugin manifest from the archive and shows the plugin details: the version, the type (**Source**), the documentation link, and the connector classes it found. ![Confluent Cloud: connector plugin details listing the four connector classes found in the archive](https://mintcdn.com/planetscale-2/WPOFVUxRXVcpW_lU/images/assets/docs/integrations/debezium/confluent-3-connector-classes.png?fit=max\&auto=format\&n=WPOFVUxRXVcpW_lU\&q=85\&s=240f5f6a9eec0864467036c4a45e0257) The archive contains four connector classes. Use **`com.planetscale.debezium.PlanetscaleConnector`**. The other three (`com.planetscale.labs.io.debezium.connector.vitess…`, `com.planetscale.labs.io.debezium.connector.mysql…`, and `io.debezium.connector.mysql.MySqlConnector`) are internal dependencies bundled with the plugin and are not meant to be run on their own. Click **Create connector** to go straight to creating a connector from the new plugin. You can also come back later: the plugin stays listed under **Connectors → Connector Plugins** (filter on **Custom**). From there, **Get started** creates a connector from it, and **Add version** uploads a newer release of the archive next to the existing one. ![Confluent Cloud: Connector Plugins page showing the PlanetScale custom plugin with Get started and Add version](https://mintcdn.com/planetscale-2/P43mbon7mBrqbRCV/images/assets/docs/integrations/debezium/confluent-8-plugin-list.png?fit=max\&auto=format\&n=P43mbon7mBrqbRCV\&q=85\&s=26341c15838be37feb93ca116b31eaca) **Step 3** The **Add connector** wizard opens on **Connector details**. Select the plugin version, keep the connector type on **Source**, and under **Select class** choose `com.planetscale.debezium.PlanetscaleConnector`. Under **Kafka details**, the defaults work: the connector targets Java 17, so **JDK version** 17 or newer is fine, and any offered **Kafka version** from 3.6 up is supported. ![Confluent Cloud: Add connector wizard, connector details step with the PlanetScale class selected](https://mintcdn.com/planetscale-2/P43mbon7mBrqbRCV/images/assets/docs/integrations/debezium/confluent-4-connector-details.png?fit=max\&auto=format\&n=P43mbon7mBrqbRCV\&q=85\&s=313872d44f7905644966310cf009006b) **Step 4** On **Kafka access**, pick or create the API key the connector uses to write to your cluster. **Step 5** On **Configuration**, Confluent shows the connector's settings as a form. Fill in: * **Topic prefix**: the prefix for the topics the connector creates. * **Keyspace**: your PlanetScale database name. * **Shard**: leave empty to capture all shards. Ignore the field's tooltip example of `"0"` for an unsharded keyspace: an unsharded Vitess keyspace's shard is named `-`, and a `0` here makes the connector fail to start. * **vgtid** and **gtid**: leave at `current` to start streaming from the current position, after the initial snapshot. * **Vitess database hostname** and **port**: the hostname from your PlanetScale connection string, and `443`. The port field is pre-filled with the connector's default `15991`. Change it to `443`, the only port PlanetScale listens on. * **User** and **Password**: your PlanetScale password credentials. The password field is masked and stored encrypted by Confluent. ![Confluent Cloud: Add connector wizard, configuration step with the PlanetScale connection settings](https://mintcdn.com/planetscale-2/P43mbon7mBrqbRCV/images/assets/docs/integrations/debezium/confluent-5-configuration.png?fit=max\&auto=format\&n=P43mbon7mBrqbRCV\&q=85\&s=49fa6528d96394a85ae8f8f16e13ecdc) Confluent manages the offset and schema-history storage for you; there is nothing to configure for them. **Step 6** On **Networking**, add your PlanetScale hostname **with port 443** (for example `aws.connect.psdb.cloud:443`) as a **connection endpoint**. Confluent allow-lists outbound traffic to it; without this the connector cannot reach PlanetScale. Confluent's [endpoint syntax](https://docs.confluent.io/cloud/current/connectors/bring-your-connector/custom-connector-qs.html) is `:`, so include the port. ![Confluent Cloud: Add connector wizard, networking step with the PlanetScale hostname as connection endpoint](https://mintcdn.com/planetscale-2/P43mbon7mBrqbRCV/images/assets/docs/integrations/debezium/confluent-6-networking.png?fit=max\&auto=format\&n=P43mbon7mBrqbRCV\&q=85\&s=e2cf8b700be38c7cf958d8ae9421854a) **Step 7** Choose a size on **Sizing** (one task is enough; the connector runs a single task), then **Review and launch**. The connector shows **Provisioning** for a minute or two and then **Running** with one task; the topics named `..
` fill with the initial snapshot followed by live changes. ![Confluent Cloud: PlanetScale connector overview showing Running with one task running](https://mintcdn.com/planetscale-2/P43mbon7mBrqbRCV/images/assets/docs/integrations/debezium/confluent-7-running.png?fit=max\&auto=format\&n=P43mbon7mBrqbRCV\&q=85\&s=5f287d7e16c5567ed5e5fbf96ba17254) Under **Topics**, filter on your topic prefix to see one topic per captured table. Without a **Table include list**, every table in the database gets a topic. ![Confluent Cloud: Topics page listing one topic per PlanetScale table under the connector's topic prefix](https://mintcdn.com/planetscale-2/P43mbon7mBrqbRCV/images/assets/docs/integrations/debezium/confluent-9-topics.png?fit=max\&auto=format\&n=P43mbon7mBrqbRCV\&q=85\&s=c84f399461c9e2ee61f58348e943e07b) Your consumers subscribe to these topics; each message is a Debezium change event for one row. ## Related > **[Debezium connector for PlanetScale](/docs/vitess/integrations/debezium)** > **[Confluent custom connectors](https://docs.confluent.io/cloud/current/connectors/bring-your-connector/custom-connector-qs.html)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Fivetran integration Source: https://planetscale.com/docs/vitess/integrations/fivetran With PlanetScale Connect, you can extract data from your PlanetScale database and safely load it into other destinations for analysis, transformation, and more. We implemented a [Fivetran](https://fivetran.com/) connector as the pipeline between your PlanetScale source and selected destination. This document will walk you through connecting your PlanetScale database to Fivetran. ## Connect to Fivetran ### Sign up for the PlanetScale connector private preview Currently, the PlanetScale connector is in private preview mode in Fivetran. To get access: **Step 1** Log into your [Fivetran dashboard](https://fivetran.com/dashboard/) **Step 2** Click on "**Add connector**" and set up your destination (if needed) **Step 3** On the "**Browse our data sources**" page, type `planetscale` in the search box **Step 4** Click on the "**Contact us**" button in the `PlanetScale` search result **Step 5** Fill out the form and submit The Fivetran team will reach out to grant you access to the private preview. ### Requirements * A PlanetScale database * A Fivetran account * Access to the private preview of the PlanetScale connector in Fivetran #### Connect using private networking > [!NOTE] > > You must be using the [PlanetScale Enterprise single-tenant deployment option](/docs/plans/deployment-options#single-tenancy-deployment-on-planetscale) and on a [Fivetran Business Critical plan](https://resources.fivetran.com/datasheets/fivetran-business-critical-product-overview) to use private networking. If you are using private networking and not connecting directly to your PlanetScale database, the Fivetran integration supports AWS PrivateLink and GCP Private Service Connect. See the [Fivetran AWS Private Link setup guide](https://fivetran.com/docs/databases/connection-options#awsprivatelink) or [Fivetran GCP Private Service Connect setup guide](https://fivetran.com/docs/databases/connection-options#googlecloudprivateserviceconnect) for details. ### Set up Fivetran **Step 1** In the [Fivetran dashboard](https://fivetran.com/dashboard/), click on "**Add connector**," set up your destination (if needed), and search for `planetscale` on the “**Browse our data sources**” page. Once selected, you should see the PlanetScale connector settings page. **Step 2** In [PlanetScale](https://app.planetscale.com), navigate to the database you want to connect to Fivetran and click the "**Connect**" button. **Step 3** Create a new password for your main branch with [read-only permissions](/docs/vitess/security/password-roles#overview). **Step 4** Select "**General**" from the "**Connect with**" dropdown and leave this tab open, as you'll need to copy these credentials shortly. **Step 5** Back in Fivetran, in your [connector setup form](https://fivetran.com/docs/using-fivetran/fivetran-dashboard/connectors#addanewconnector), enter the connector values as follows: * **Destination schema**: This prefix applies to each replicated schema and cannot be changed once your connector is created. Note: Each replicated schema is appended with `_planetscale` at the end of your chosen name. * **Database host name**: Paste in the copied value for `host` * **Database name**: Paste in the copied value for `database` * **Database username**: Paste in the copied value for `username` * **Database password**: Paste in the copied value for `password` * **Comma-separated list of shards to sync (optional)**: If your PlanetScale database is *not* sharded, ignore this field. If the database is sharded, by default, the PlanetScale connector will download rows from all shards in the database. To pick which shards are synced by the connector, you can optionally provide a comma-separated list of shards in the connector configuration. * **Use replica?**: In PlanetScale, VStream will connect to the primary tablet for your database, which also serves queries to your database. To lessen the load on the primary tablet, set this to `true` to make Vstream read from a replica of your database. * **Treat tinyint(1) as boolean (optional)**: You can choose to have the connector transform tinyint(1) type columns in your database to either `true` or `false`. * **Fivetran IPs (optional)**: If your connection string was created with [IP restrictions](/docs/vitess/connecting/connection-strings#ip-restrictions), ensure that the [Fivetran IP ranges](https://fivetran.com/docs/using-fivetran/ips) are added to the password. **Step 6** Click "**Save & Test**". Fivetran tests and validates our connection to your PlanetScale database. Upon successfully completing the setup tests, you can sync your data using Fivetran. ## Sync overview Once Fivetran is connected to your PlanetScale primary or read replica, we pull a complete dump of all selected data from your database. The connector then connects to your database's [VStream](https://vitess.io/docs/concepts/vstream/) to pull all your new and changed data at regular intervals. VStream is Vitess' change tracking mechanism that is underneath every PlanetScale database. If data in the source changes (for example, you add new tables or change a data type), the connector automatically detects and persists these changes into your destination. ### Syncing empty tables and columns Fivetran can sync empty tables and columns for your PlanetScale connector. For more information, see the [Fivetran features documentation](https://fivetran.com/docs/using-fivetran/features#syncingemptytablesandcolumns). ## Schema information Fivetran tries to replicate the exact schema and tables from your PlanetScale source database to your destination according to our [standard database update strategies](https://fivetran.com/databases#transformationandmappingoverview). For every schema in the PlanetScale database you connect, we create a schema in your destination that maps directly to its native schema. This ensures that the data in your destination is in a familiar format to work with. ### Fivetran-generated columns Fivetran adds the following columns to every table in your destination: * `_fivetran_deleted` (BOOLEAN) marks deleted rows in the source database. * `_fivetran_synced` (UTC TIMESTAMP) indicates when Fivetran last successfully synced the row. * `_fivetran_index` (INTEGER) shows the order of updates for tables that do not have a primary key. * `_fivetran_id` (STRING) is the hash of the non-Fivetran values of each row. It's a unique ID that Fivetran uses to avoid duplicate rows in tables that do not have a primary key. We add these columns to give you insight into the state of your data and the progress of your data syncs. ### Type transformations and mapping As the connector extracts your data, the connector matches MySQL data types in your PlanetScale database to types that Fivetran supports. If the connector doesn't support a specific data type, the connector automatically changes that type to the closest supported type or, for some types, does not load that data at all. Our system automatically skips columns with data types we do not accept or transform. The following table illustrates how we transform your MySQL data types into Fivetran-supported types: | MySQL Type | Fivetran Data Type | Fivetran Supported | Notes | | :-: | :-: | :-: | :-: | | BINARY | BINARY | True | | | BIGINT | LONG | True | | | BIT | BOOLEAN | True | BIT type with a single digit is supported. | | BLOB | BINARY | True | | | CHAR | STRING | True | | | DATE | DATE | True | Invalid values will be loaded as NULL or EPOCH if the type is a primary key. | | DATETIME | TIMESTAMP\_NTZ | True | Invalid values will be loaded as NULL or EPOCH if the type is a primary key. | | DECIMAL/ NUMERIC | BIGDECIMAL | True | | | DOUBLE | DOUBLE | True | | | ENUM | STRING | True | | | FLOAT | DOUBLE | True | | | GEOMETRY | JSON | True | | | GEOMETRYCOLLECTION | JSON | True | | | JSON | JSON | True | | | INT | INTEGER | True | | | LINESTRING | JSON | True | | | LONGBLOB | BINARY | True | | | LONGTEXT | STRING | True | | | MEDIUMBLOB | BINARY | True | | | MEDIUMINT | INTEGER | True | | | MEDIUMTEXT | STRING | True | | | MULTILINESTRING | JSON | True | | | MULTIPOINT | JSON | True | | | MULTIPOLYGON | JSON | True | | | POINT | JSON | True | | | POLYGON | JSON | True | | | SET | JSON | True | | | SMALLINT | INTEGER | True | | | TIME | STRING | True | | | TIMESTAMP | TIMESTAMP | True | MYSQL always stores timestamps in UTC. Invalid values will be loaded as NULL or EPOCH if the type is a primary key. | | TINYBLOB | BINARY | True | | | TINYINT | BOOLEAN | True | If you select `Treat TinyInt(1) as boolean` in the connector configuration, we will enforce that the tinyint is either 1 or 0 and return true/false accordingly. | | TINYINT | INTEGER | True | In all other cases, the destination type for TINYINT columns will be INTEGER. If the width isn't specified to be exactly 1 (either no specification or a value other than 1), the destination type will be INTEGER, even if the column contains only 1s or 0s. | | TINYTEXT | STRING | True | | | UNSIGNED BIGINT | BIGDECIMAL | True | | | UNSIGNED INT | LONG | True | | | UNSIGNED SMALLINT | INTEGER | True | | | VARCHAR | STRING | True | | | VARBINARY | BINARY | True | | | YEAR | INTEGER | True | | > [!NOTE] > > If the connector is missing an important data type that you need, please [contact us](https://planetscale.com/contact). In some cases, when loading data into your destination, the connector may need to convert Fivetran data types into data types supported by the destination. For more information, see the [individual data destination pages](https://fivetran.com/destinations). ### Unparsable values When the connector encounters [an unparsable value](https://fivetran.com/docs/connectors/databases/mysql#unparsablevalues) of one of the following data types, the connector substitutes it with a default value. The default value the connector uses depends on whether the unparsable value is in a primary key column or non-primary key column: | MySQL Type | Primary Key Value | Non-Primary Key Value | | :- | :- | :- | | DATE | 1970-01-01 | null | | DATETIME | 1970-01-01T00:00:00 | null | | TIMESTAMP | 1970-01-01T00:00:00Z | null | Although we may be able to read some values outside the supported DATE, DATETIME, and TIMESTAMP ranges as defined by [MySQL's documentation](https://dev.mysql.com/doc/refman/8.0/en/datetime.html), there is no guarantee. Additionally, the special zero value 0000-00-00 00:00:00 is subject to this rule. ### Excluding source data If you don’t want to sync all the data from your database, you can exclude schemas, tables, or columns from your syncs on your Fivetran dashboard. To do so, go to your connector details page and uncheck the objects you want to omit from syncing. For more information, see the [Fivetran column Blocking documentation](https://fivetran.com/docs/using-fivetran/features/column-blocking-hashing). ## Initial sync When Fivetran connects to a new database, the connector first copies all rows from every table in every schema for which we have `SELECT` permission (except those you have excluded in your Fivetran dashboard) and add [Fivetran-generated columns](https://fivetran.com/docs/connectors/databases/mysql#fivetrangeneratedcolumns). Tables are copied in ascending size order (from smallest to largest). The connector copies rows by performing a `SELECT` statement on each table. For large tables, we copy a limited number of rows at a time so that we don't have to start the sync again from the beginning if our connection is lost midway. The duration of initial syncs can vary depending on the number and size of tables to be imported. We, therefore, interleave incremental updates with the table imports during the initial sync. ## Updating data Fivetran performs incremental updates of any new or modified data from your source database. The connector uses Vitess's inbuilt VStream VGtids, which allows Fivetran to update only the data that has changed since our last sync. ### Deleted rows The connector does not delete rows from the destination. It handles deletes as part of streaming changes from VStream. Note: We only process `DELETE` events from the stream. ### Deleted columns The connector does not delete columns from your destination. When a column is deleted from the source table, it replaces the existing values in the corresponding destination column with `NULL` values. ### Adding and dropping columns When you add or drop a column, the connector attempts to migrate your destination schema to the new table structure automatically. In some cases, it will be unable to do this and instead perform an automatic re-sync of the changed table. In the following scenarios, Fivetran will re-sync your table instead of automatically migrating it: * Changing column order * Changing primary keys * Modifying `ENUM` or `SET` columns ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale GitHub Actions Source: https://planetscale.com/docs/vitess/integrations/github-actions See our [tech talk on Databases + CI/CD](https://planetscale.com/blog/databases-ci-cd-pipeline) to see pscale + GitHub Actions used in a real application. With GitHub Actions, you can automate the creation of branches and deploy requests all within your CI workflow. > **[Getting Started](#getting-started)** > **[Authentication](#authentication)** > **[Convert GitHub branch name to PlanetScale branch name](#convert-github-branch-name-to-planetscale-branch-name)** > **[Create a PlanetScale branch](#create-a-planetscale-branch)** > **[Recreate the branch when the pull request changes](#recreate-the-branch-when-the-pull-request-changes)** > **[Create a password for a branch](#create-a-password-for-a-branch)** > **[Open a deploy request](#open-a-deploy-request)** > **[Get deploy request by branch name](#get-deploy-request-by-branch-name)** > **[Get deploy request diff and comment on pull request](#get-deploy-request-diff-and-comment-on-pull-request)** > **[Check for dropped columns](#check-for-dropped-columns)** > **[Submit a deploy request by branch name](#submit-a-deploy-request-by-branch-name)** ## Getting started The best way to use PlanetScale within GitHub Actions is via the `pscale` CLI. Use [`planetscale/setup-pscale-action`](https://github.com/planetscale/setup-pscale-action) to make pscale available within your GitHub Actions. ```yaml theme={null} - name: Setup pscale uses: planetscale/setup-pscale-action@v1 ``` The action works with Linux, Windows, and Mac runners. Once installed it will be added to your tool cache for subsequent runs. ## Authentication Authentication for pscale is via service token environment variables. You will need to [create a service token](/docs/cli/service-token). Make sure to give your service token the proper permissions to the database you'll be using in your workflow. Add your `PLANETSCALE_SERVICE_TOKEN_ID` and `PLANETSCALE_SERVICE_TOKEN` to your [Actions secrets](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository). In your Actions workflow, you will need to make the secrets available as environment variables. ```yaml theme={null} - name: Run pscale command env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: pscale database list ``` ## Examples The following examples show how to accomplish common Actions workflows with PlanetScale. In each example, notice that we use secrets for the service token, database and organization names. You will need to set them in your GitHub repository to target your own database. ``` PLANETSCALE_ORG_NAME PLANETSCALE_DATABASE_NAME PLANETSCALE_SERVICE_TOKEN_ID PLANETSCALE_SERVICE_TOKEN ``` ### Convert GitHub branch name to PlanetScale branch name PlanetScale branch names must be lowercase, alphanumeric characters and hyphens are allowed. Since git branch names allow more possibilities, you can use the following code to transform a git branch name into an acceptable PlanetScale branch name. ```yaml theme={null} - name: Rename branch name run: | branch_name=$(printf '%s' "$HEAD_REF" | tr -cd '[:alnum:]-' | tr '[:upper:]' '[:lower:]') echo "PSCALE_BRANCH_NAME=$branch_name" >> "$GITHUB_ENV" env: HEAD_REF: ${{ github.head_ref }} ``` This makes `${{ env.PSCALE_BRANCH_NAME }}` available for use in the rest of the workflow. This is useful to run in any scenario where you are creating a PlanetScale branch to correspond with a git branch. ### Create a PlanetScale branch You can use `pscale branch create` to create a branch that matches your GitHub branch name. ```yaml theme={null} - name: Create branch env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | set +e pscale branch show ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} exit_code=$? set -e if [ $exit_code -eq 0 ]; then echo "Branch exists. Skipping branch creation." else echo "Branch does not exist. Creating." pscale branch create ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --wait --org ${{ secrets.PLANETSCALE_ORG_NAME }} fi ``` Notice that we first check if the branch exists. If it does, we do nothing. Otherwise we create it and pass the `--wait` flag. This is useful when running in CI, as the workflow may run multiple times and you'll want the branch ready if you are running schema migrations immediately after creating the branch. ### Recreate the branch when the pull request changes Skipping creation when the branch exists reuses whatever schema that branch already has. If a migration in the pull request is edited after the first workflow run, the branch and any open deploy request go stale and no longer match the code under review. Deleting and recreating the branch on every run guarantees it reflects the current pull request. Before deleting, close any open deploy requests for the branch and leave a comment on them pointing back to the pull request. ```yaml expandable theme={null} - name: Recreate branch env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | set +e pscale branch show ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} exists=$? set -e if [ $exists -eq 0 ]; then echo "Branch exists. Closing any open deploy requests and recreating." pr_url="https://github.com/${{ github.repository }}/pull/${{ github.event.pull_request.number }}" stale_numbers=$(pscale deploy-request list ${{ secrets.PLANETSCALE_DATABASE_NAME }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} -f json \ | jq -r --arg branch "${{ env.PSCALE_BRANCH_NAME }}" '.[] | select(.state == "open" and .branch == $branch) | .number') for number in $stale_numbers; do echo "Commenting on and closing deploy request $number" jq -n --arg body "This deploy request is stale. The schema in ${pr_url} changed, so this DR is being closed and a new one will be opened." '{body: $body}' \ | pscale api organizations/${{ secrets.PLANETSCALE_ORG_NAME }}/databases/${{ secrets.PLANETSCALE_DATABASE_NAME }}/deploy-requests/"$number"/comments --org ${{ secrets.PLANETSCALE_ORG_NAME }} --input - \ || echo "Warning: could not comment on deploy request $number" pscale deploy-request close ${{ secrets.PLANETSCALE_DATABASE_NAME }} "$number" --org ${{ secrets.PLANETSCALE_ORG_NAME }} done if [ -n "$stale_numbers" ]; then echo "STALE_DR_NUMBERS=$(echo $stale_numbers | tr '\n' ' ')" >> "$GITHUB_ENV" fi pscale branch delete ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} --force fi - name: Create branch env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: pscale branch create ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} --wait ``` The numbers of any closed deploy requests are saved to `${{ env.STALE_DR_NUMBERS }}`, so a later step can mention which deploy request was replaced when commenting on the pull request. Pair this with a `concurrency` group keyed on the git branch so two pushes in quick succession don't delete and create the same PlanetScale branch at the same time. ```yaml theme={null} concurrency: group: ps-migrations-${{ github.head_ref }} cancel-in-progress: true ``` ### Create a password for a branch You can use `pscale password create` to generate credentials for your database branch. ```yaml expandable theme={null} - name: Generate password for branch env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | response=$(pscale password create ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} "" -f json --org ${{ secrets.PLANETSCALE_ORG_NAME }}) id=$(echo "$response" | jq -r '.id') host=$(echo "$response" | jq -r '.access_host_url') username=$(echo "$response" | jq -r '.username') password=$(echo "$response" | jq -r '.plain_text') ssl_mode="verify_identity" # Assuming a default value for ssl_mode ssl_ca="/etc/ssl/certs/ca-certificates.crt" # Assuming a default value for ssl_ca # Set the password ID, allows us to later delete it if wanted. echo "PASSWORD_ID=$id" >> $GITHUB_ENV # Create the DATABASE_URL database_url="mysql://$username:$password@$host/${{ secrets.PLANETSCALE_DATABASE_NAME }}?sslmode=$ssl_mode&sslca=$ssl_ca" echo "DATABASE_URL=$database_url" >> $GITHUB_ENV echo "::add-mask::$DATABASE_URL" - name: Use the DATABASE_URL in a subsequent step run: | echo "Using DATABASE_URL: $DATABASE_URL" ``` This example shows creating the password and getting back a response in json. The json is then parsed to create a `DATABASE_URL` which can be used in later steps. > [!NOTE] > > `pscale password create` can also accept a `ttl` flag which lets you limit the number of minutes the password is valid for. ### Open a deploy request You can use `pscale deploy-request create` to open a new deploy request from GitHub Actions. This can be useful after running migrations against a branch. ```yaml theme={null} - name: Open DR if migrations env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | pscale deploy-request create ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} \ --org ${{ secrets.PLANETSCALE_ORG_NAME }} \ --auto-delete-branch \ --notes "https://github.com/${{ github.repository }}/pull/${{ github.event.pull_request.number }}" ``` `--auto-delete-branch` deletes the PlanetScale branch once the deploy request completes, so CI branches don't accumulate. `--notes` links the deploy request back to the pull request that created it. ### Get deploy request by branch name You can use `pscale deploy-requests show` to grab the latest deploy request by branch name. This can be useful when deploying your application. You can first check if there are any deploy requests open for the branch being deployed. If there are, you can trigger the deploy request to run before you deploy your application. ```yaml theme={null} - name: Get Deploy Requests env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | deploy_request_number=$(pscale deploy-request show ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} -f json | jq -r '.number') echo "DEPLOY_REQUEST_NUMBER=$deploy_request_number" >> $GITHUB_ENV ``` This example also makes the deploy request number available as an `env` var so that it can be used in later steps. ### Get deploy request diff and comment on pull request We can use `pscale deploy-request diff` to see the full schema diff of a deploy request. This example is useful when combined with opening a deploy request for a git branch. You can then automatically comment the diff back to the GitHub pull request. ```yaml theme={null} - name: Comment on PR env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | echo "Deploy request opened: https://app.planetscale.com/${{ secrets.PLANETSCALE_ORG_NAME }}/${{ secrets.PLANETSCALE_DATABASE_NAME }}/deploy-requests/${{ env.DEPLOY_REQUEST_NUMBER }}" >> migration-message.txt echo "" >> migration-message.txt echo "\`\`\`diff" >> migration-message.txt pscale deploy-request diff ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.DEPLOY_REQUEST_NUMBER }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} -f json | jq -r '.[].raw' >> migration-message.txt echo "\`\`\`" >> migration-message.txt - name: Comment PR - db migrated uses: thollander/actions-comment-pull-request@v2 with: filePath: migration-message.txt ``` This writes the diff to the `migration-message.txt` file. And then creates a comment on the pull request that triggered the workflow. ### Check for dropped columns PlanetScale sets a `can_drop_data` boolean for any schema change that drop a column or table. We can make use of this to emit a warning into our pull requests. In this example, we first wait for the deployment check to be `ready`. During this time, PlanetScale is examining the schema change, verifying that it is safe and generating the DDL statements to make the change. Once it's done, we then use this information to put a comment on the deploy request with tips on how to deploy it safely. ```yaml expandable theme={null} - name: Check deployment state id: check-state env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | for i in {1..10}; do deployment_state=$(pscale deploy-request show ${{ secrets.PLANETSCALE_ORG_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} --format json | jq -r '.deployment_state') echo "Deployment State: $deployment_state" if [ "$deployment_state" = "ready" ]; then echo "Deployment state is ready. Continuing." break fi echo "Deployment state is not ready. Waiting 2 seconds before checking again." sleep 2 done - name: Comment PR - db migrated if: ${{ env.DR_OPENED }} env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | deploy_data=$(pscale api organizations/${{ secrets.PLANETSCALE_ORG_NAME }}/databases/${{ secrets.PLANETSCALE_DATABASE_NAME }}/deploy-requests/${{ env.DEPLOY_REQUEST_NUMBER }}/deployment --org planetscale) can_drop_data=$(echo "$deploy_data" | jq -r '.deploy_operations[] | select(.can_drop_data == true) | .can_drop_data') echo "Deploy request opened: https://app.planetscale.com/${{ secrets.PLANETSCALE_ORG_NAME }}/${{ secrets.PLANETSCALE_DATABASE_NAME }}/deploy-requests/${{ env.DEPLOY_REQUEST_NUMBER }}" >> migration-message.txt echo "" >> migration-message.txt if [ "$can_drop_data" = "true" ]; then echo ":rotating_light: You are dropping a column. Before running the migration make sure to do the following:" >> migration-message.txt echo "" >> migration-message.txt echo "1. [ ] Deploy app changes to ensure the column is no longer being used." >> migration-message.txt echo "2. [ ] Once you've verified it's no used, run the deploy request." >> migration-message.txt echo "" >> migration-message.txt else echo "When adding to the schema, the Deploy Request must be run **before** the code is deployed." >> migration-message.txt echo "Please ensure your schema changes are compatible with the application code currently running in production." >> migration-message.txt echo "" >> migration-message.txt echo "1. [ ] Successfully run the Deploy Request" >> migration-message.txt echo "2. [ ] Deploy this PR" >> migration-message.txt echo "" >> migration-message.txt fi echo "\`\`\`diff" >> migration-message.txt pscale deploy-request diff ${{ secrets.PLANETSCALE_ORG_NAME }} ${{ env.DEPLOY_REQUEST_NUMBER }} -f json | jq -r '.[].raw' >> migration-message.txt echo "\`\`\`" >> migration-message.txt - name: Comment PR - db migrated uses: thollander/actions-comment-pull-request@v2 if: ${{ env.DR_OPENED }} with: filePath: migration-message.txt ``` ### Submit a deploy request by branch name To trigger a deploy, we can use `pscale deploy-request deploy`. This command will accept either the deploy request number, or the name of the branch that the deploy request was created from. When using with GitHub Actions, it's often easier to use the branch name. The `--wait` flag will let the command run until the deployment is complete. This is important if you want your schema change to run before the next step in your workflow. ```yaml theme={null} - name: Deploy schema migrations env: PLANETSCALE_SERVICE_TOKEN_ID: ${{ secrets.PLANETSCALE_SERVICE_TOKEN_ID }} PLANETSCALE_SERVICE_TOKEN: ${{ secrets.PLANETSCALE_SERVICE_TOKEN }} run: | pscale deploy-request deploy ${{ secrets.PLANETSCALE_DATABASE_NAME }} ${{ env.PSCALE_BRANCH_NAME }} --org ${{ secrets.PLANETSCALE_ORG_NAME }} --wait ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Hightouch integration Source: https://planetscale.com/docs/vitess/integrations/hightouch Hightouch is a platform that allows you to sync data between systems using a simple and intuitive UI. ## Overview Hightouch supports PlanetScale databases as a data source and a destination to send data to. This article will provide an overview of setting up a PlanetScale database as a source and destination. Connecting your PlanetScale database to a Hightouch workflow can be used in a number of different ways, but here are a few ideas to get you started: * Automatically update customer records in your CRM when they are created or changed in your database. * Send a notification to your Slack workspace when data is changed in your PlanetScale database. * Execute asynchronous operations by sending data from your PlanetScale database to a data queue like AWS SQS. * Automate paid ad targeting, suppression, and conversion uploads by syncing audiences from your database to ad platforms. * Automatically sync account and user health scores to your customer success platforms. Before you can proceed, you’ll need to create a set of credentials as covered in our [Connection strings doc](/docs/vitess/connecting/connection-strings). Take note of the hostname, username, password, and database name. ## PlanetScale as a source If you wish to sync data from your PlanetScale database to another system, you’ll need to configure your PlanetScale database as a source in Hightouch. I will be using a database named `bookings_db` in this demo. ### 1. Create the source In Hightouch, start by selecting **"Sources"** from the sidebar. ![Sources](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/sources.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=f68e372e3ce499be3486ef0804ff5031) Click **"Add Source"**. ![Add source](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/add-source.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=0b8b300ce80853b70dfe57c81e5e1436) Select **"PlanetScale"** from the list of supported data sources. ![PlanetScale as a data source](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/planetscale-as-a-data-source.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=db6df618980080dd3cdf43278dc262a1) On the Configure your PlanetScale source, populate Step 1 of the form using a set of credentials generated for your database. The **Port** will always be **3306**. > [!NOTE] > > As a reminder, PlanetScale credentials can be generated by clicking **"Connect"** in the PlanetScale dashboard overview of your database. Review our [Connection strings doc](/docs/vitess/connecting/connection-strings) for more details. ![Connect to PlanetScale data source](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/connect-to-planetscale-data-source.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=43d35bf4caa90866e721c06ecada2fcf) Scroll down to Step 2, and populate your **User** and **Password** before clicking **"Continue"**. ![Connect to PlanetScale data source 2](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/connect-to-planetscale-data-source-2.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=b8671e41cfa1a6a8a4461273ef17d41a) In the next view, Hightouch will ensure that it can properly connect to your database. Provided it connects successfully, click **"Continue"**. ![Validate connection](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/validate-connection.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=1d1e33552f2b4a8a98bcae3d8bbad386) Finally, name your source and click **"Finish"**. ![Finalize source settings](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/finalize-source-settings.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=2ac9835e9beac42bdbb418f4a5d8bc19) ### 2. Create a model Before you can start syncing data, you need to tell Hightouch how the data looks with a Model. Start by selecting **"Models"** from the navigation. ![Create a model](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/create-a-model.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=9fed2336170daa18157a7f7b5431385c) Select the Source you created in the previous step. ![PlanetScale data source for the model](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/planetscale-data-source-for-the-model.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=90f591d3560f484193fa3ed30b5d157a) There are two available methods to model your data from a PlanetScale database: * SQL query - allows you to write a SQL query that returns the data you want to sync. * Table selector - allows you to select a table and read every row in it. I’ll use **"Table selector"** for this demo. ![Query vs table selector](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/query-vs-table-selector.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=ec4f1bd085ce46c5b4743107619623f2) Use the search to find the `hotels` table, select it from the list, and click **"Preview results"** to validate the data in that table. Once satisfied with the results, click **"Continue"**. ![New model from hotels table](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/new-model-from-hotels-table.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=4ac58e7b2a49508ccad5029269701864) Finally, name the model “Hotel” and set the Primary key to `id` before clicking **"Finish"**. ![Configure model](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/configure-model.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=a4df28fecaaf7cd212b1ccebe6680b42) ### 3. Create a sync Now that the Source and Model are configured, a Sync can be created which will sync data from PlanetScale to another system. For this demo, I’ll be using a spreadsheet in my Google Drive account that I’ve already configured as a destination. Select **"Syncs"** from the left and click **"Add sync"**. ![Create a sync](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/create-a-sync.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=13e32b82e974a12af2f71668ddc905d0) Select the “Hotel” model created in the previous step. ![Select the hotel model for the sync](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/select-the-hotel-model-for-the-sync.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=1ac749c64a5ce52461382d2b4d3df972) For my demo, I’ll select my Google Sheets destination. ![The Google Sheets destination](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/the-google-sheets-destination.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=bc730771162998ccacacd9e67ab54450) I’m mirroring data into the default sheet in a Sheets doc named “Hightouch demo” for the Google Sheets settings. ![Mirror data to the Google Sheet](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/mirror-data-to-the-google-sheet.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=ad8f90e1024e32a845a4c2082b9daa73) A sync can be performed using a number of different methods. For the demo, I’ll leave **"Manual"** selected and click **"Finish"**. ![Finalize settings for first sync](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/finalize-settings-for-first-sync.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=a71c4c9a5b24563f5adf331d16cce8fb) Since this is a manual sync, I’ll also need to click **"Run sync"** to execute it. ![Manually run the first sync](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/manually-run-first-sync.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=b6c05a3d483b972423471903bc63a0e0) ### 4. Review the synced data If I log into my Google Drive account and review that spreadsheet, there are now three rows that exist which match the data from the `hotels` table of my PlanetScale database. ![The synced data in Google Sheets](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/the-synced-data-in-google-sheets.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=0730c48413e264b97d1daf9045edeb00) ## PlanetScale as a destination Hightouch can also be configured to send data to a PlanetScale database. In this demo, I’ll perform the inverse of the above demo and configure the `new hotels` sheet from my Google Sheets spreadsheet to add new rows to my PlanetScale database. Here is that sheet with a single new hotel added to it. ![The new hotels sheet in Google Sheets](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/the-new-hotels-sheet-in-google-sheets.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=6b67a82b979d7ea91d9fb54f2a4c4ad5) ### 1. Create the Destination Configuring the source and model for Google Sheets will not be covered in this demo. I’ll start by creating a new destination for my PlanetScale database by selecting **"Destinations"** from the left navigation, then **"Add destination"**. ![Add Destination](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/add-destination.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=24ab5291bb949a4ef5077aee13585719) Search for the PlanetScale destination, select it, and click **"Continue"**. ![Select PlanetScale as the destination](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/select-planetscale-as-the-destination.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=edbc3c3f113a98a5ed7c5ea0dbffd201) In the next view, populate the details in the form using the connection details for your PlanetScale database. Click **"Continue"**. ![Connect to PlanetScale as a destination](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/connect-to-planetscale-as-a-destination.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=fb7b91f3cc2a9ed8aec76188ed02b12d) Click **"Finish"** to save the Destination. ![Finalize PlanetScale destination](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/finalize-planetscale-destination.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=cdce985a615ba7fce4c336d6c2d4c7e5) ### 2. Create the Sync To create a new Sync, select **"Syncs"** from the left navigation and click **"Add sync"**. ![Add second sync](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/add-second-sync.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=ed2e8a42d8197dc9622424dd42c9e91a) The model name for my Google Sheets worksheet is `new_hotel` so I’ll select that from the list. ![Select Google Sheets as a source](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/select-google-sheets-as-a-source.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=0be3694d8a9218e148dc6bba0492807d) Select the Destination created in the previous step. ![Select the PlanetScale destination](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/select-the-planetscale-destination.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=ec314c8d7e7841bebc07a573059e7a71) Next, I can select the table I wish to sync data to. I’ll choose `hotels` from the dropdown. Hightouch only supports upserting data at this time, so that option is preselected. I’ll also need to map the fields between my Google Sheets worksheet and my PlanetScale database, starting with the unique identifiers. Both have an `id` column so I can select that for both sides of the sync. Since the other column names match, I can click **"Suggest mappings"** to let Hightouch automatically figure out which fields map from the Source to the Destination. Once that’s done, I’ll click **"Continue"** to move forward. ![Configure the field mapping](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/configure-the-field-mapping.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=a9330e7866039d6a949cd2f71feea479) Finally, I can specify the Schedule type for when my data sync should occur. I’ll leave it set to **"Manual"** and click **"Continue"**. ![Finalize the second sync](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/hightouch/finalize-the-second-sync.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=01ef881807024fd08fa2dec71d649ef8) Since this Sync is configured to execute manually, I’ll click **"Run sync"**. ![Run the second sync](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/run-the-second-sync.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=7ddc3c970faf8ea494878a4d3d0e5a45) ### 3. Review the synced data I can verify that the sync was successful by querying my PlanetScale database to ensure the new hotel was added successfully. ![The select query from the PlanetScale console](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/hightouch/the-select-query-from-the-planetscale-console.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=6aa14b36bf32a9dfb3891ccb2539eb98) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Netlify Source: https://planetscale.com/docs/vitess/integrations/netlify ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Prometheus Source: https://planetscale.com/docs/vitess/integrations/prometheus PlanetScale exposes Prometheus-compatible metrics endpoints for scraping metrics about your database branches. This, along with our API-driven service discovery, allow you to automatically get in-depth information about all of the databases in your organization. In order to collect and store these, you will need to use Prometheus or a Prometheus-compatible metrics engine (such as VictoriaMetrics) that is capable of using the [HTTP SD](https://prometheus.io/docs/prometheus/latest/http_sd/) protocol. ## Prerequisites This document assumes we'll be configuring a Prometheus 3.x instance via a configuration file running on our local machine. If you are using managed Prometheus via AWS, GCP or another provider, you will have to deploy Prometheus to scrape and forward metrics via `remote_write`, as these services do not support scraping metrics. ## Getting Started First, provision a new PlanetScale [Service token](/docs/api/reference/service-tokens) in your Organization settings. Make sure to save the ID and token, as they will not be visible after they've been generated. When that's created, grant the token `read_metrics_endpoints` permissions and click "Save permissions". Your token should look like the following: ![Service Token configuration for Metrics Exporting](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-service-token-configuration.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=7154912fcc404c404843cdbed160db0f) ## Configuring Prometheus Now that we have a Service Token, we can add a scrape configuration for your PlanetScale organization. A minimal Prometheus configuration should look like the following: ```yaml theme={null} scrape_configs: - job_name: "${ORG}" http_sd_configs: - url: https://api.planetscale.com/v1/organizations/${ORG}/metrics authorization: type: "token" credentials: "${TOKEN_ID}:${TOKEN}" refresh_interval: 10m ``` Fill in your organization name in the `job_name` and `url`, and place the Service Token and ID that we created in the previous step for the credentials. Save this file to `prometheus.yml` in your working directory. ## Start Prometheus Run Prometheus pointed at this configuration file: ```bash theme={null} $ prometheus --config.file=prometheus.yml ``` By default, Prometheus will listen at `0.0.0.0:9090`, which means you can access it in your browser at [http://127.0.0.1:9090](http://127.0.0.1:9090). ### Validating Service Discovery First, let's make sure that Prometheus is properly querying the PlanetScale API for the right branches. If you go to `http://127.0.0.1:9090/service-discovery` you should see the job that we created earlier, with all of your branches listed under `Discovered labels`. In this example, our organization is called `nick`, so it looks like the following: ![Prometheus Target List](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-prometheus-targets.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=96f64a28bdbaac27e83310a751941169) Here, I have two branches that have been discovered. I can confirm that this matches what's in my organization: ```bash theme={null} $ pscale branch list test --org nick ID NAME PARENT BRANCH REGION PRODUCTION SAFE MIGRATIONS READY CREATED AT UPDATED AT -------------- ------------ --------------- --------- ------------ ----------------- ------- ------------- --------------- 7wxuxewx4l0p main n/a us-east Yes Yes Yes 2 years ago 7 minutes ago 6o0rr27785fl partitions main us-east No No Yes 1 month ago 9 minutes ago ``` Now, if I go to my list of targets I should see each branch as an Endpoint: ![Prometheus Endpoint List](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-prometheus-endpoints.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=6830ae6e1a43d6667ccde7f6784d8e5c) This screenshot shows that they're being correctly scraped, and I can start to query my Prometheus instance. ## Querying Prometheus Now that we're collecting metrics for my branches, our [reference guide](/docs/vitess/integrations/prometheus-metrics) has a list of everything that we export. If I want to see how many `vtgate` pods are running per AZ for my branch, I can query: ``` planetscale_vtgate_total_pods{planetscale_database_branch_id="7wxuxewx4l0p"} ``` Make sure the graph is set to stacked, and it should look like this: ![Querying Prometheus for VTGate Count](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-prometheus-querying.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=5271cb0215290d804c5c71d29cea7759) ## Next Steps If you keep this Prometheus instance running, it will collect metrics every 30 seconds, and refresh the list of branches every 10 minutes. For more information, see: * [Metrics reference](/docs/vitess/integrations/prometheus-metrics) for a list of metrics we expose * [Grafana and Prometheus](/docs/vitess/tutorials/prometheus-metrics-grafana) tutorial for using PlanetScale's provided dashboard to visualize these metrics in Grafana. * [Sending metrics to New Relic](/docs/vitess/tutorials/prometheus-metrics-newrelic) tutorial for using Prometheus to forward metrics to New Relic. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Prometheus metrics Source: https://planetscale.com/docs/vitess/integrations/prometheus-metrics PlanetScale exposes a Prometheus-compatible endpoint per-branch that allows you to scrape metrics for your database. ## Overview See our [Prometheus integration](/docs/vitess/integrations/prometheus) documentation for how to set Prometheus up to automatically discover and scrape metrics for your database branches. If you're using Datadog, see our [Datadog tutorial](/docs/vitess/tutorials/prometheus-metrics-datadog) for how to setup your Datadog agent to scrape metrics for your branch. ## Metrics PlanetScale emits the following metrics to be scraped. ## Database metrics | **Name & Description** | **Type** | **Tags** | | :- | :- | :- | | **planetscale\_edge\_active\_connections** The number of active MySQL connections to the branch | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_region | | **planetscale\_edge\_mysql\_queries\_by\_branch\_total** Total number of queries processed by Edge for the branch, per Edge instance | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_instance, planetscale\_region | | **planetscale\_mysql\_bytes\_received\_total** Total number of bytes received from MySQL clients | Counter | cluster, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_bytes\_sent\_total** Total number of bytes sent to MySQL clients | Counter | cluster, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_data\_writes\_total** Total number of InnoDB data write operations | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_row\_lock\_time\_total** Total time spent acquiring row locks in InnoDB | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_row\_lock\_waits\_total** Number of times InnoDB had to wait for a row lock | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_row\_operations\_total** Total number of row operations (inserts/updates/deletes) in InnoDB | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_mysql\_operation\_type, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_auto\_increment\_util\_percentage** The percentage of auto increment size being utilized | Gauge | cluster, planetscale\_column, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard, planetscale\_table | | **planetscale\_mysql\_replica\_lag\_seconds** Replica lag in fine-grained seconds from MySQL | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_slow\_queries\_total** Number of queries that exceeded the slow query threshold | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_threads\_running** Current number of threads executing in MySQL | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_pods\_container\_last\_terminated\_reason** Reason a container was last terminated (1 = the current reason). Join against **planetscale\_pods\_container\_status\_restarts\_total** to attribute restarts to a reason | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_restart\_reason, planetscale\_shard | | **planetscale\_pods\_container\_status\_restarts\_total** Total container restarts, across all restart reasons. Join **planetscale\_pods\_container\_last\_terminated\_reason** to attribute them | Counter | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard | | **planetscale\_pods\_container\_waiting\_reason** Container waiting reason (CrashLoopBackOff, ImagePullBackOff, etc) | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type, planetscale\_waiting\_reason | | **planetscale\_pods\_cpu\_util\_percentages** CPU utilization percentage of database pods | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_pods\_iops\_total** Total IOPS (Input/Output Operations Per Second) of database pods | Counter | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_pods\_mem\_util\_percentages** Memory utilization percentage of database pods | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_pods\_status\_phase** Pod status phase (Running, Pending, Failed, Succeeded, Unknown) | Gauge | cluster, planetscale\_cell, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_phase, planetscale\_pod, planetscale\_keyspace, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vtgate\_affected\_rows\_total** Number of rows affected by queries through vtgate | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_pod | | **planetscale\_vtgate\_errors\_total** Total number of errors encountered by vtgate | Counter | cluster, planetscale\_vtgate\_code, planetscale\_database\_branch\_id, planetscale\_tablet\_type, planetscale\_keyspace, planetscale\_cell, planetscale\_pod, planetscale\_vtgate\_operation | | **planetscale\_vtgate\_queries\_duration** Distribution of query execution times through vtgate | Histogram | cluster, le, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_tablet\_type, planetscale\_vtgate\_operation | | **planetscale\_vtgate\_query\_executions\_total** Number of queries executed by vtgate. Replaces planetscale\_vtgate\_commands\_total | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_query\_type, planetscale\_tablet\_type, planetscale\_vtgate\_plan\_type | | **planetscale\_vtgate\_returned\_rows\_total** Number of rows returned by queries through vtgate | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_pod, planetscale\_tablet\_type | | **planetscale\_vtgate\_total\_pods** Total number of vtgate pods | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id | | **planetscale\_vtgate\_v\_streams\_count** Number of active vstreams | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vtgate\_v\_streams\_created\_total** Number of vstreams created | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vtgate\_v\_streams\_ended\_with\_errors\_total** Number of vstreams that ended with errors | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vtgate\_v\_streams\_events\_streamed\_total** Number of events sent across all vstreams | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vtgate\_v\_streams\_lag\_seconds** Difference in seconds between the current time when the vstream event was sent and the time when the binlog event occurred | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vtorc\_detected\_problems** Problems VTOrc currently detects in the cluster (1 = the problem is active). Join against the recovery counters to see what VTOrc did about it | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_vtorc\_analysis | | **planetscale\_vtorc\_failed\_recoveries\_total** Number of failed replication/recovery attempts by vtorc | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace (Vitess 23+), planetscale\_pod, planetscale\_shard (Vitess 23+), planetscale\_vtorc\_recovery\_type | | **planetscale\_vtorc\_successful\_recoveries\_total** Number of successful replication/recovery attempts by vtorc | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace (Vitess 23+), planetscale\_pod, planetscale\_shard (Vitess 23+), planetscale\_vtorc\_recovery\_type | | **planetscale\_vttablet\_connection\_pool\_active** Number of active connections in the vttablet connection pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_connection\_pool\_capacity** Total capacity of the vttablet connection pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_connection\_pool\_get\_total** Number of connection get operations from the vttablet pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_connection\_pool\_in\_use** Number of connections currently in use in the vttablet pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_connection\_pool\_wait\_time\_total** Total time spent waiting for connections from the vttablet pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_connection\_pool\_wait\_total** Number of waits for an available connection in the vttablet pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_errors\_total** Number of errors encountered by vttablet | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type, planetscale\_vttablet\_code | | **planetscale\_vttablet\_found\_rows\_pool\_active** Number of active connections in the vttablet found rows pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_found\_rows\_pool\_capacity** Total capacity of the vttablet found rows pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_found\_rows\_pool\_get\_total** Number of connection get operations from the found rows pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_found\_rows\_pool\_in\_use** Number of connections currently in use in the found rows pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_found\_rows\_pool\_wait\_time\_total** Total time spent waiting for connections from the found rows pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_found\_rows\_pool\_wait\_total** Number of waits for an available connection in the found rows pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_is\_serving** If a vttablet is or is not serving traffic | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_queries\_affected\_rows\_total** Total number of rows affected by queries through vttablet | Counter | cluster, planetscale\_command, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_table, planetscale\_tablet\_type | | **planetscale\_vttablet\_queries\_duration** Distribution of query execution times through vttablet | Histogram | cluster, le, planetscale\_command, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_queries\_duration\_by\_table\_total** Total of query execution times through vttablet, broken down by table | Counter | cluster, planetscale\_command, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard, planetscale\_table, planetscale\_tablet\_type | | **planetscale\_vttablet\_queries\_read\_rows\_total** Total number of rows read by queries through vttablet | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_queries\_returned\_rows\_total** Total number of rows returned by queries through vttablet | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_table, planetscale\_tablet\_type | | **planetscale\_vttablet\_queries\_total** Total number of queries processed by vttablet | Counter | cluster, planetscale\_command, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_table, planetscale\_tablet\_type | | **planetscale\_vttablet\_replication\_lag** MySQL replication lag in seconds | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_stream\_connection\_pool\_active** Number of active connections in the vttablet stream connection pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_stream\_connection\_pool\_capacity** Total capacity of the vttablet stream connection pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_stream\_connection\_pool\_get\_total** Number of connection get operations from the stream connection pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_stream\_connection\_pool\_in\_use** Number of connections currently in use in the stream connection pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_stream\_connection\_pool\_wait\_time\_total** Total time spent waiting for connections from the stream pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_stream\_connection\_pool\_wait\_total** Number of waits for an available connection in the stream pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_table\_storage\_all\_bytes** Storage size of tables in bytes for all tablet types | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_table, planetscale\_tablet\_type | | **planetscale\_vttablet\_transaction\_pool\_active** Number of active transactions in the vttablet transaction pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_transaction\_pool\_capacity** Total capacity of the vttablet transaction pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_transaction\_pool\_get\_total** Number of transaction get operations from the transaction pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_transaction\_pool\_in\_use** Number of transactions currently in use in the transaction pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_transaction\_pool\_wait\_time\_total** Total time spent waiting for transactions from the transaction pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_transaction\_pool\_wait\_total** Number of waits for an available transaction in the transaction pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_volume\_available\_bytes** Available storage space in bytes on vttablet volumes | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_volume\_capacity\_bytes** Total storage capacity in bytes on vttablet volumes | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vttablet\_vstreamer\_errors\_total** Total number of vttablet vstreamer errors. | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard, planetscale\_vstreamer\_code | | **planetscale\_vttablet\_vstreamer\_events\_total** Total number of vttablet vstreamer streamed events. | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard | | **planetscale\_vttablet\_vstreamer\_throttled\_total** Number of times the vstreamer was throttled by the tablet throttler. | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard | | **planetscale\_workflow\_vreplication\_copy\_row\_count** Number of rows copied so far for a VReplication workflow stream | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard, planetscale\_source\_keyspace, planetscale\_source\_shard, planetscale\_workflow | | **planetscale\_workflow\_vreplication\_lag** VReplication lag in seconds for workflow operations | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard, planetscale\_source\_keyspace, planetscale\_source\_shard, planetscale\_workflow | | **planetscale\_workflow\_vreplication\_stream\_state** State of the VReplication stream for workflow operations | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard, planetscale\_state, planetscale\_workflow | | **planetscale\_workflow\_vreplication\_table\_copy\_row\_count** Number of rows copied so far per table for a VReplication workflow | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard, planetscale\_source\_keyspace, planetscale\_source\_shard, planetscale\_table, planetscale\_workflow | | **planetscale\_workflow\_vreplication\_throttled\_total** Number of times VReplication was throttled for a workflow, by component and throttler | Counter | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard, planetscale\_throttled\_component, planetscale\_throttler, planetscale\_workflow | The planetscale\_keyspace and planetscale\_shard tags on **planetscale\_vtorc\_failed\_recoveries\_total** and **planetscale\_vtorc\_successful\_recoveries\_total** are only present on databases running Vitess 23 or later. On earlier versions these counters are reported per pod and recovery type only, and queries that group by keyspace or shard return nothing. ## Edge metrics (Single Tenant & Managed only) | **Name & Description** | **Type** | **Tags** | | - | - | - | | **planetscale\_edge\_cpu\_util\_percentages** CPU utilization percentage of Edge instances | Gauge | cluster, planetscale\_component, planetscale\_instance | | **planetscale\_edge\_instance\_connections** Number of active connections to Edge instances | Gauge | cluster, planetscale\_component, planetscale\_instance, planetscale\_region | | **planetscale\_edge\_mem\_util\_percentages** Memory utilization percentage of Edge instances | Gauge | cluster, planetscale\_component, planetscale\_instance | | **planetscale\_edge\_mysql\_bytes\_received\_total** Total number of MySQL bytes received by Edge instances | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_instance, planetscale\_region | | **planetscale\_edge\_mysql\_bytes\_sent\_total** Total number of MySQL bytes sent by Edge instances | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_instance, planetscale\_region | | **planetscale\_edge\_mysql\_http\_bytes\_received\_total** Total number of MySQL HTTP bytes received by Edge instances | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_instance, planetscale\_region | | **planetscale\_edge\_mysql\_http\_bytes\_sent\_total** Total number of MySQL HTTP bytes sent by Edge instances | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_instance, planetscale\_region | | **planetscale\_edge\_queries\_total** Total number of queries processed by Edge instances | Counter | cluster, planetscale\_component, planetscale\_instance, planetscale\_region | | **planetscale\_psdb\_api\_cpu\_util\_percentages** CPU utilization percentage of PSDB API instances | Gauge | cluster, planetscale\_availability\_zone, planetscale\_component, planetscale\_pod | | **planetscale\_psdb\_api\_mem\_util\_percentages** Memory utilization percentage of PSDB API instances | Gauge | cluster, planetscale\_availability\_zone, planetscale\_component, planetscale\_pod | | **planetscale\_psdb\_api\_requests\_total** Total number of requests processed by PSDB API instances | Counter | cluster, planetscale\_availability\_zone, planetscale\_component, planetscale\_pod, planetscale\_psdb\_api\_operation | ## Deprecated metrics These metrics are deprecated and will be removed eventually. | **Name & Description** | **Type** | **Tags** | | - | - | - | | **planetscale\_pods\_container\_restarts\_total** Total container restart events detected. The reason label is attached to an all-reasons total and the counter cannot be observed at zero, so rates and increases over it are wrong in both directions. Replaced by planetscale\_pods\_container\_status\_restarts\_total. | Counter | cluster, planetscale\_cell, planetscale\_component, planetscale\_container, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_restart\_reason, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vtgate\_commands\_total** Number of commands processed by vtgate. Replaced by planetscale\_vtgate\_query\_executions\_total. | Counter | cluster, planetscale\_cell, planetscale\_command, planetscale\_database\_branch\_id, planetscale\_pod | | **planetscale\_vttablet\_table\_storage\_bytes** Storage size of tables in bytes. Replaced by planetscale\_vttablet\_table\_storage\_all\_bytes. | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_shard, planetscale\_table | ## Experimental metrics These metrics are experimental and may be subject to changes to their name, type, or tags. | **Name & Description** | **Type** | **Tags** | | - | - | - | | **planetscale\_mysql\_innodb\_history\_list\_length** InnoDB transaction history list length (purge lag indicator) | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_redo\_log\_capacity\_resized** Current InnoDB redo log capacity in bytes | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_redo\_log\_current\_lsn** Current InnoDB redo log sequence number | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_redo\_log\_checkpoint\_lsn** InnoDB redo log checkpoint sequence number | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_buffer\_pool\_read\_requests\_total** Total logical read requests to buffer pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_buffer\_pool\_reads\_total** Number of reads that required disk I/O (buffer pool misses) | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_buffer\_pool\_bytes\_dirty** Total bytes of dirty pages in buffer pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_buffer\_pool\_bytes\_data** Total bytes of data in buffer pool | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_buffer\_pool\_wait\_free\_total** Number of times InnoDB had to wait for free pages in buffer pool | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_log\_waits\_total** Number of times log buffer was too small and required flushing | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_os\_log\_written\_total** Total bytes written to InnoDB redo log files | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_data\_written\_total** Total bytes of data written by InnoDB | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_innodb\_buffer\_pool\_read\_ahead\_evicted\_total** Read-ahead pages evicted without being accessed | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_handlers\_total** Total handler operations by type | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_mysql\_handler\_type, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_sort\_merge\_passes\_total** Number of merge passes the sort algorithm has had to do | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_sort\_range\_total** Number of sorts done with ranges | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_sort\_scan\_total** Number of sorts done by scanning the table | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_threads\_connected** Number of currently open connections | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_connections\_total** Total number of connection attempts | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_max\_connections** Maximum permitted number of simultaneous connections | Gauge | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_created\_tmp\_tables\_total** Number of internal temporary tables created | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_created\_tmp\_disk\_tables\_total** Number of internal on-disk temporary tables created | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_table\_locks\_immediate\_total** Number of times a table lock was acquired immediately | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_table\_locks\_waited\_total** Number of times a table lock could not be acquired immediately | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_binlog\_cache\_use\_total** Number of transactions that used the binary log cache | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_binlog\_cache\_disk\_use\_total** Number of transactions that used temporary binary log cache but exceeded binlog\_cache\_size | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_binlog\_stmt\_cache\_use\_total** Number of non-transactional statements that used the binary log statement cache | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_mysql\_binlog\_stmt\_cache\_disk\_use\_total** Number of non-transactional statements that exceeded binlog\_stmt\_cache\_size | Counter | cluster, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_tablet\_type | | **planetscale\_vitess\_emergency\_reparents\_total** Number of emergency reparents attempted on a shard, counting both successes and failures. An emergency reparent means the primary was promoted to recover from a failure rather than as part of planned work | Counter | cluster, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_reparent\_result, planetscale\_shard | | **planetscale\_vitess\_planned\_reparents\_total** Number of planned reparents attempted on a shard, counting both successes and failures. A planned reparent is a graceful promotion, such as the one that happens during maintenance or when you request one | Counter | cluster, planetscale\_component, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_reparent\_result, planetscale\_shard | | **planetscale\_vttablet\_storage\_info** An info metric for the storage type of a vttablet | Gauge | cluster, planetscale\_cell, planetscale\_database\_branch\_id, planetscale\_keyspace, planetscale\_pod, planetscale\_shard, planetscale\_storage\_type, planetscale\_tablet\_type | ## Attributing container restarts `planetscale_pods_container_status_restarts_total` counts every restart and carries no role or reason label. Join the metric that carries the one you want. To break restarts down by role, join `planetscale_pods_status_phase`: ```promql theme={null} sum by (planetscale_tablet_type) ( increase(planetscale_pods_container_status_restarts_total[1h]) * on (planetscale_pod) group_left(planetscale_tablet_type) group by (planetscale_pod, planetscale_tablet_type) (planetscale_pods_status_phase) ) ``` To break restarts down by reason as well, filter against `planetscale_pods_container_last_terminated_reason` one minute at a time and join the role onto the result: ```promql theme={null} sum by (planetscale_tablet_type) ( sum_over_time(( increase(planetscale_pods_container_status_restarts_total[1m]) and on (planetscale_pod, planetscale_container) planetscale_pods_container_last_terminated_reason{planetscale_restart_reason="OOMKilled"} )[1h:1m]) * on (planetscale_pod) group_left(planetscale_tablet_type) group by (planetscale_pod, planetscale_tablet_type) (planetscale_pods_status_phase) ) ``` The gauge only reports a container's latest termination reason, so the query attributes each minute's restarts to the reason current in that minute, then sums over the hour. Change `1h` to your own window and leave the `1m` alone. A container that restarts twice inside one minute under two different reasons has both restarts attributed to the later one. ## Tag glossary | **Tag Name** | **Description** | | - | - | | cluster | The database cluster identifier | | le | Histogram bucket upper bound (less than or equal to) | | planetscale\_availability\_zone | AWS availability zone where the component is running | | planetscale\_cell | Availability zone or cell where the component is running | | planetscale\_column | Auto-increment column name | | planetscale\_command | VTGate command type (e.g., Select, Insert, Update) | | planetscale\_component | Vitess component type (vtgate, vttablet, vtctld, vtorc). On the reparent metrics it says which component ran the reparent: vtorc means VTOrc recovered the shard on its own, vtctld means the reparent was requested | | planetscale\_container | Kubernetes container name within the pod | | planetscale\_database\_branch\_id | Unique identifier for the database branch | | planetscale\_instance | Edge instance identifier | | planetscale\_keyspace | Database keyspace (logical database name) | | planetscale\_mysql\_handler\_type | MySQL handler operation type (e.g., read\_first, read\_key, write) | | planetscale\_mysql\_operation\_type | MySQL row operation type (inserted, read, updated, deleted) | | planetscale\_phase | Pod status phase (Running, Pending, Failed, Succeeded, Unknown) | | planetscale\_pod | Kubernetes pod name where the component is running | | planetscale\_psdb\_api\_operation | PSDB API operation type | | planetscale\_query\_type | Query statement type (SELECT, INSERT, UPDATE, DELETE, etc.) | | planetscale\_region | Geographic region where the database is hosted | | planetscale\_reparent\_result | Whether a reparent attempt succeeded (success, failure) | | planetscale\_restart\_reason | Reason for a container's **most recent** termination (OOMKilled, Error, Completed). Carried by **planetscale\_pods\_container\_last\_terminated\_reason** | | planetscale\_shard | Database shard identifier | | planetscale\_source\_keyspace | Source keyspace for VReplication workflow operations | | planetscale\_source\_shard | Source shard for VReplication workflow operations | | planetscale\_state | VReplication stream state (e.g., Running, Stopped) | | planetscale\_storage\_type | Storage type backing a vttablet (network-attached, metal) | | planetscale\_table | Database table name | | planetscale\_tablet\_type | Vitess tablet type (primary, replica, rdonly). Absent on components that have none, such as vtgate | | planetscale\_throttled\_component | VReplication component that was throttled (e.g., vcopier, vplayer, binlogplayer) | | planetscale\_throttler | Which throttler applied the throttle (tablet throttler name) | | planetscale\_vstreamer\_code | VTTablet VStreamer error code (StreamEnded, etc.) | | planetscale\_vtgate\_code | VTGate error code (INVALID\_ARGUMENT, etc.) | | planetscale\_vtgate\_operation | VTGate operation type (Execute, ExecuteBatch, etc.) | | planetscale\_vtgate\_plan\_type | VTGate query plan type (Passthrough, Scatter, MultiShard, Join, etc.) | | planetscale\_vtorc\_analysis | The problem VTOrc detected (DeadPrimary, ReplicaSemiSyncMustBeSet, etc.) | | planetscale\_vtorc\_recovery\_type | The recovery VTOrc performed (RecoverDeadPrimary, ElectNewPrimary, FixReplica, etc.) | | planetscale\_vttablet\_code | VTTablet error code (NOT\_FOUND, etc.) | | planetscale\_waiting\_reason | Container waiting reason (CrashLoopBackOff, ImagePullBackOff, etc.) | | planetscale\_workflow | VReplication workflow identifier | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Stitch integration Source: https://planetscale.com/docs/vitess/integrations/stitch With PlanetScale Connect, you can extract data from your PlanetScale database and safely load it into other destinations for analysis, transformation, and more. We implemented an [Stitch Singer Tap](https://stitchdata.com/) as the pipeline between your PlanetScale source and selected destination. This document will walk you through how to connect your PlanetScale database to Stitch. ## How to connect ### Step 1 : Setup an Import API integration in Stitch. PlanetScale's Stitch tap outputs records and metadata to stdout so that the `http tap` can import them into Stitch via [Stitch Import API](https://www.stitchdata.com/docs/developers/import-api/). **Step 1** Sign up for a [StitchData](https://app.stitchdata.com/signup) account **Step 2** Once you've signed up, create an Integration by clicking on **Add Integration**. **Step 3** On the marketplace screen, type in **import** to narrow the list down to the **Import API** ![Add Stitch Integration](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/stitch/integration.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=be1a33ff18aa41d7efa0df7d057e8293) **Step 4** On the next screen, configure your integration name and destination. ![Configure Stitch Integration](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/stitch/configure.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=35bc876bf5c8f251be4ac6bc6cf67a5d) **Step 5** Once the integration is created, save the access token for use with the PlanetScale tap. ![Save Stitch API Token](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/stitch/api-token.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=3605a6820d1265d306b1c9d1cfd60f36) ### Step 2 : Configure the PlanetScale Stitch Tap In this step, we will connect your PlanetScale database to the PlanetScale Singer Tap. **Step 1** Click on the database and branch you want to connect to. **Step 2** Click "Connect", and select "Stitch source" from the "Connect with" dropdown. **Step 3** Leave this tab open, as you'll need to copy these credentials shortly. ![Stitch Source config](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/stitch/connect.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=8d0374a8f67f7e3e4963c8ca8d7254cb) **Step 4** Copy the contents of `source.json` as a file on your local file system, and save it as `source.json`. This will now act as the `PlanetScale source config` when connecting the `PlanetScale Stitch Tap` to your database. ### Step 3: Run the PlanetScale Stitch Tap **Step 1** Install the PlanetScale Singer tap by running: ```bash theme={null} brew install planetscale/tap/ps-singer-tap ``` **Step 2** Install the PlanetScale Http tap by running ```bash theme={null} brew install planetscale/tap/ps-http-tap ``` **Step 3** Save the schema for your PlanetScale database. ```bash theme={null} ps-singer-tap --config source.json --discover > schema.json ``` **Step 4** The `schema.json` file you saved in the previous step is a JSON document that describes all tables & columns available in your PlanetScale database. By default, no tables/columns are selected. You can select a column or table by setting its `selected` property in the table's `metadata` element in the JSON document to be true. Here's an example of selecting the `dept_no` property in a table. ```json theme={null} { "metadata": { "selected": true, "inclusion": "available", "breadcrumb": ["properties", "dept_no"] } } ``` **Step 5** Sync your PlanetScale database to Stitch by running the following command: ```bash theme={null} ps-singer-tap --config source.json --catalog schema.json | ps-http-tap --api-token $(cat access_token) ``` **Step 6** You should see an output similar to this: ```bash expandable theme={null} PlanetScale Tap : INFO : Syncing records for PlanetScale database : import-on-scaler PlanetScale Tap : INFO : syncing rows from stream "departments" from shard "-" PlanetScale Tap : INFO : [departments shard : -] peeking to see if there's any new rows PlanetScale Tap : INFO : new rows found, syncing rows for 1m0s PlanetScale Tap : INFO : [departments shard : -] syncing rows with cursor [shard:"-" keyspace:"import-on-scaler"] PlanetScale Tap : INFO : Syncing with cursor position : [], using last known PK : false, stop cursor is : [MySQL56/e42292e8-e28f-11ec-9c5b-d680f5d655b3:1-705,e4e20f06-e28f-11ec-8d20-8e7ac09cb64c:1-26,eba743a8-e28f-11ec-9227-62aa711d33c6:1-20] PlanetScale Tap : INFO : [departments shard : -] Continuing with cursor after server timeout PlanetScale Tap : INFO : [departments shard : -] peeking to see if there's any new rows HTTP Tap : INFO : flushing [20] messages for stream "departments" PlanetScale Tap : INFO : [departments shard : -] no new rows found, exiting HTTP Tap : INFO : Server response status : "OK", message : "Batch accepted" HTTP Tap : INFO : flushing [1] messages for stream "departments" HTTP Tap : INFO : Server response status : "OK", message : "Batch accepted" HTTP Tap : INFO : saving state to path : state/state-1656850746251.json ``` **Step 7** Any state outputted by the PlanetScale Tap will be saved and you can look at the logs for the location. Here is an example of outputted state: ```bash theme={null} HTTP Tap : INFO : saving state to path : state/state-1656850746251.json ``` **Step 8** In this example, you should see that Stitch loaded `21` records to be replicated. ![Completed Stitch import](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/stitch/success.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=b85556b691d67b684879617e081bbd50) **Step 9** To incrementally sync from this last sync position, pass the path to last saved state in step 7 as the `--state` argument when you run sync. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vantage integration Source: https://planetscale.com/docs/vitess/integrations/vantage With [Vantage](https://www.vantage.sh/), you can set up [PlanetScale cost management](https://vantage.sh/integrations/planetscale) to report on PlanetScale costs alongside other infrastructure providers, such as AWS or GCP. After integrating, you can create [cost reports](https://docs.vantage.sh/cost_reports) to break down costs per database and branch. Beyond reporting, set up budget alerts, forecast usage, and view active database costs in Vantage. Vantage connects to your PlanetScale organizations using an OAuth flow. ## Prerequisites * The [Organization Admin role](/docs/security/access-control) in PlanetScale * A [Vantage account](https://console.vantage.sh/signup) > [!NOTE] > > Database cost reporting in Vantage is not available for [PlanetScale Managed](/docs/plans/managed) customers via the integration. ## Configure the Vantage integration **Step 1** From the Vantage console, navigate to the [Integrations page](https://console.vantage.sh/settings/integrations). **Step 2** Select **PlanetScale**, then click **Connect PlanetScale Account**. **Step 3** The PlanetScale login screen is displayed. Log in to your PlanetScale account and select the organizations you want to connect with. **Step 4** Click **Authorize access**. **Step 5** On the [PlanetScale Settings](https://console.vantage.sh/settings/planetscale/) page in Vantage, you should see the status of your connection change to **Importing**. Costs will be ingested and processed in Vantage once you add the integration. It typically takes less than 15 minutes to ingest PlanetScale costs. The costs will be available on your **All Resources** Cost Report in Vantage as soon as they are processed. > [!NOTE] > > PlanetScale data refreshes daily in Vantage. ## View PlanetScale costs in Vantage In Vantage, you can create cost reports to drill down into your costs. Vantage displays PlanetScale costs by Organization, Service, Category, and Resource. ![Image of a PlanetScale Cost Report in Vantage showing costs per database](https://mintcdn.com/planetscale-2/UzFO5Pe10M0-W-uW/images/assets/docs/integrations/vantage/vantage-console.png?fit=max\&auto=format\&n=UzFO5Pe10M0-W-uW\&q=85\&s=6018bb6488c79a20268f102876b221a8) In the graphic above, PlanetScale costs are grouped by database for the month. For complete cost reporting dimensions and more information, see the [PlanetScale documentation](https://docs.vantage.sh/connecting_planetscale) for Vantage. ## Billing The Vantage integration is available on all our [plans](/docs/planetscale-plans). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Anomalies Source: https://planetscale.com/docs/vitess/monitoring/anomalies Anomalies are defined as periods with a substantially elevated percentage of slow-running queries. ## Overview PlanetScale Insights continuously analyzes your query performance to establish a baseline for expected performance. When a high enough percentage of queries are running more slowly than the baseline expectation, we call this an anomaly. ## Using the Anomalies graph The graph shown on the Anomalies page shows the percentage of queries executing slower than the 97.7th (2-sigma) percentile baseline on the y-axis and the period of time on the x-axis. The "expected" line shows the percent of queries that are statistically expected in a database with uniform query performance over time. Slight deviations from the expected value are normal. Only substantial and sustained deviations from the expected value are considered an anomaly. ![Database health graph showing two anomalies](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/anomalies/database-health-graph.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=5b6c409b85f1638ae8a318e4d3c64453) Any periods where your database was unhealthy will be highlighted with a red icon representing a performance anomaly. Each anomaly on the graph is clickable. Clicking on it will pull up more details about it in the table below the graph, such as: duration, percentage of increase, and when the anomaly occurred. We also overlay any deploy requests that happened during that period over the anomaly graph. On top of this, we also surface any impact to the following: * The query that triggered the anomaly * CPU utilization * Memory * IOPS * Queries per second * Rows written per second * Rows read per second * Errors per second ## Anomalies vs query latency You may notice a correlation between some areas in the query latency graph and the anomalies graph. Conversely, in some cases, you may see a spike in query latency, but no corresponding anomaly. Increased query latency *can* be indicative of an anomaly, but not always. Query latency may increase and decrease in ways that don't always indicate an actual problem with your database. For example, you may run a weekly report that consists of a few slow-running queries. These queries are always slow. Every week, you'll see a spike on your query latency graph during the time that your weekly report is generated, but not on your anomaly violations graph. The queries are running at their *expected* latency, so this is not considered an anomaly. ## What should I do if my database has an anomaly? The purpose of the Anomalies page is to show you relevant information so you can determine what caused an anomaly and correct the issue. Let's look at an example scenario. You deploy a feature in your application that contains a new query. This query is slow, running frequently, and is hogging database resources. This new slow query is running so often that it's slowing down the rest of your database. Because your other queries are now running slower than expected, an anomaly is triggered. In this case, we will surface the new slow-running query so that you can find ways to optimize it to free up some of the resources it's using. Adding an index will often solve the problem. You can test this by adding the index, creating a deploy request, and deploying it. If it's successful, you'll quickly see the anomaly end. On the other hand, an anomaly does not necessarily mean you need to take any action. One common example where you may see an anomaly is in the case of large active-running backups. In this case, we will tell you that a backup was running during the time of the anomaly. > [!NOTE] > > Even if it causes an anomaly, we do not recommend you turn off backups to prevent possible data loss. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Inspect live Vitess connections Source: https://planetscale.com/docs/vitess/monitoring/connections Watch the live MySQL process list on a Vitess primary tablet, and cancel a query or terminate a connection from the PlanetScale CLI. ## Overview The `pscale branch connections` command shows what is running on a Vitess database branch right now: the active sessions on a primary tablet, what each is executing, and how long it has been running. If a session is stuck, you can cancel its query or terminate the connection directly. [See what's happening in your database RIGHT NOW](https://www.youtube-nocookie.com/embed/j67zO-MixK0?rel=0) It is the live equivalent of running `SHOW FULL PROCESSLIST` and `KILL` against a primary, without opening a MySQL shell or holding database credentials. The command detects the branch engine automatically; this guide covers Vitess branches. For Postgres, see [Inspect live Postgres connections](/docs/postgres/monitoring/connections). There are two ways to use it: * **Interactively**, with `pscale branch connections top` — a terminal UI for watching activity live and taking action from the keyboard. * **From scripts or AI agents**, with `pscale branch connections show` plus the `kill` command — one-shot, structured output designed to be safe to automate. The full command and flag reference lives in the [`connections` CLI reference](/docs/cli/connections). ## Scope and limitations For Vitess, `connections` targets one keyspace and one shard. If a branch has more than one keyspace, or a keyspace is sharded, pass `--keyspace` and `--shard` to choose the tablet. An interactive `top` prompts you to pick when a target is missing. ## Watch live activity with `top` Run `top` in an interactive terminal to launch the live view: ```bash theme={null} pscale branch connections top ``` Target a specific keyspace and shard: ```bash theme={null} pscale branch connections top --keyspace --shard ``` The view refreshes about once per second; press `space` to pause. You can scrub back through the recent in-memory history without leaving the live view: `[` and `]` step one sample at a time, `{` and `}` jump to the oldest or newest, and `}` returns to live-follow. ### Reading the table Each row is one session on the primary tablet, sorted by duration with the longest-running first. The columns are: | Column | Meaning | | :- | :- | | (marker) | `▶` marks the selected row. | | `PID` | The MySQL thread ID — the connection's identity. | | `TABLET` | The tablet the session is running on (shown on wider terminals). | | `STATE` | The session's command/state, for example `Query` or `Query/running`. | | `DURATION` | How long the session has been running its current work. | | `USER` | The session's MySQL user. | | `DB` | The session's current database. | | `QUERY` | A preview of the current statement. | ### Inspecting a session Press `enter` on a row to open the detail view. It shows the session's full metadata — `pid`, `tablet`, `state`, `duration`, `user`, `database`, `client_addr`, `connection_id`, and `query_id` — followed by the complete statement text, scrollable and never truncated. Press `esc` to go back, and `?` at any time for the in-app key reference. ### Capture and replay You can record a session to a trace file and review it later: ```bash theme={null} # record while you watch pscale branch connections top --capture trace.jsonl # or record headlessly, e.g. during an incident pscale branch connections top --capture trace.jsonl --duration 30m # replay the trace in the same UI pscale branch connections top --replay trace.jsonl ``` In the live view, press `C` to start or stop capturing on the fly. The same `[ ] { }` history keys work in replay to move through the recording. ## Take action on a connection Vitess sessions support two actions: | Action | TUI key | What it does | | :- | :- | :- | | **Cancel query** | `c` | Runs `KILL QUERY` against the session's `query_id`. The connection stays open; only the running statement is interrupted. | | **Terminate** | `shift+K` | Runs `KILL` against the session's `connection_id`. The whole connection is dropped. | In the TUI, the action applies to the selected session and asks for confirmation (`y`/`N`) before it runs. There is no transaction-level kill for Vitess. ## Inspect and act from scripts or AI agents For automation — including AI agents that cannot drive an interactive UI — use `show` with `--format json`, then act with the IDs it returns. ```bash theme={null} pscale branch connections show --keyspace commerce --shard -80 --format json ``` For a Vitess branch, the envelope reports `"database_kind": "mysql"` and a `topology` object with the `keyspace`, `shard`, and `tablet` the rows came from. Each connection carries tablet-qualified `connection_id` and `query_id` values (for example `zone1-1001-101`). Pass those to the action commands: ```bash theme={null} pscale branch connections kill --query # KILL QUERY pscale branch connections kill # KILL ``` > [!WARNING] > > The action sub-commands take effect immediately and have **no confirmation prompt** — the confirmation only exists in > the interactive TUI. Always target a session by the `query_id` or `connection_id` from a recent `show`. ## Permissions Access to Connections is tied to your user account and your role on the database, and what you can do depends on whether the branch is a [production branch](/docs/vitess/schema-changes/branching). It is available to signed-in users (for the CLI, authenticate with `pscale auth login`), not to service tokens. There are two levels of access: * **View** — see the connection list and the `top` view. * **Act** — cancel a query or terminate a connection. Production branches are gated more tightly than development branches: | Role | View dev | Act dev | View production | Act production | | :- | :-: | :-: | :-: | :-: | | Organization Member | ✓ | ✓ | | | | Organization Analyst | ✓ | ✓ | ✓ | | | Organization Administrator | ✓ | ✓ | ✓ | ✓ | | Database Administrator (for that database) | ✓ | ✓ | ✓ | ✓ | In short: any organization member can view and act on development branches; viewing a production branch requires Analyst or higher; acting on a production branch requires Administrator. `Database Administrator` access can be granted directly or through a [team](/docs/security/teams) — everyone on a team that includes the database has `Database Administrator` access to it. If you do not have the required role, the command returns a permission error (HTTP 403) for the action you attempted. See [Access control](/docs/security/access-control) for the full role and team model. ## How fresh the data is `connections` polls about once per second. Each refresh runs `SHOW FULL PROCESSLIST` on the primary tablet, so on a very busy primary you can raise `--interval` to poll less often. The `captured_at` timestamp, and the freshness indicator in the TUI header, always tell you how old the current snapshot is. ## What the data maps to If you know MySQL, these map to what you would expect: * A **connection** (or session) is a MySQL thread, identified by its `PID`. * A **query** is the statement that connection is currently running. Cancel it with `kill --query` (`KILL QUERY`). * A **tablet** is the primary serving the keyspace and shard you targeted. The `connection_id` and `query_id` are qualified by it. ## Related documentation > **[connections CLI reference](/docs/cli/connections)** > **[Query Insights](/docs/vitess/monitoring/query-insights)** > **[PlanetScale CLI environment setup](/docs/cli/planetscale-environment-setup)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Datadog integration Source: https://planetscale.com/docs/vitess/monitoring/datadog PlanetScale can push metrics to Datadog to assist your team with understanding your database usage and performance. > [!WARNING] > > This Datadog integration is no longer receiving updates or new metric additions. You should instead use our [Datadog Agent Integration](/docs/vitess/tutorials/prometheus-metrics-datadog), which provides more metrics than this native integration. > > If you have any questions about migrating to the Datadog Agent Integration, [reach out to support](https://planetscale.com/contact?initial=support). ## Prerequisites * A [Datadog](https://www.datadoghq.com/) account ## Configuring the Datadog integration **Step 1** In Datadog, install the [PlanetScale integration](https://app.datadoghq.com/integrations/planetscale). **Step 2** Create a Datadog API key in your [Datadog Organization Settings](https://app.datadoghq.com/organization-settings/api-keys) and copy the key. **Step 3** In PlanetScale, go to your organization's [Integrations settings](https://app.planetscale.com/settings/integrations), and select **Configure** for the Datadog integration. Paste your Datadog API key into the field. Once complete, a "PlanetScale" dashboard will be available with incoming metrics from PlanetScale. ![PlanetScale Default Dashboard in Datadog](https://mintcdn.com/planetscale-2/89X51wIXzJwNfurq/images/assets/docs/integrations/datadog/dashboard.png?fit=max\&auto=format\&n=89X51wIXzJwNfurq\&q=85\&s=7870ae02484c84e746a1d907f9be79bb) ## Metrics We Collect Once configured, PlanetScale collects the following metrics from every branch in your organization. | **Metric name** | **Metric type** | **Description** | | :- | :- | :- | | planetscale.connections | gauge | Number of active connections to a database branch. *Shown as connection.* | | planetscale.primary.cpu\_usage | gauge | Percentage of CPU utilized on a database branch's primary. *Shown as percent.* | | planetscale.primary.memory\_usage | gauge | Percentage of memory utilized on a database branch's primary. *Shown as percent.* | | planetscale.queries.latency | gauge | Query times in the p50, p95, p99 and p999 percentiles. *Shown as millisecond.* | | planetscale.replication\_lag | gauge | Replication lag in seconds between a database branch's primary and each replica. *Shown as second.* | | planetscale.rows\_read | count | Number of rows read from a database branch. *Shown as row.* | | planetscale.rows\_written | count | Number of rows written to a database branch. *Shown as row.* | | planetscale.tables.cumulative\_query\_time | count | Cumulative active query time in a database branch, by table and statement. *Shown as nanosecond.* | | planetscale.tables.queries | count | Number of queries issued to a database branch, by table and statement. *Shown as query.* | | planetscale.tables.rows\_deleted | count | Number of rows deleted from a database branch, by table. *Shown as row.* | | planetscale.tables.rows\_inserted | count | Number of rows inserted into a database branch, by table. *Shown as row.* | | planetscale.tables.rows\_selected | count | Number of rows selected in a database branch, by table. *Shown as row.* | | planetscale.tables.rows\_updated | count | Number of rows updated in a database branch, by table. *Shown as row.* | | planetscale.tables.storage | gauge | Total bytes stored in a database branch, by table. *Shown as byte.* | | planetscale.vtgate.errors | count | Number of errors encountered by a database branch's vtgate. *Shown as error.* | | planetscale.vttablet.mem\_util.max | gauge | Maximum memory utilization of a database branch's vttablet. *Shown as percent.* | | planetscale.vttablet.mem\_util.avg | gauge | Average memory utilization of a database branch's vttablet. *Shown as percent.* | | planetscale.vttablet.iops | gauge | Number of IOPS performed by a database branch's vttablet. *Shown as operation.* | ## Billing The Datadog integration is available on all of our [paid plans](https://planetscale.com/pricing). ## Frequently asked questions ### How do I track replication lag in Datadog? You can use the following formula to set alerts for replication lag: ```bash theme={null} (max:planetscale.replication_lag{ps_database: ps_tablet_type:replica, ps_branch:
}) ``` Make sure you replace `` with your PlanetScale database name and `
` with the name of the branch for which you'd like to track replication lag. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Errors Source: https://planetscale.com/docs/vitess/monitoring/errors The Errors page surfaces any errors that have been captured on your database in a 24 hour period. ## Overview The Errors page surfaces any errors that have been captured on your database in a 24 hour period. Underneath the graph, you'll find a list of database error messages that have been captured over the selected period. You can click on any of the error messages on the Errors page to open a more detailed view. This view shows you the individual queries that produced the error, when they ran, how long they ran, and any query tags attached to them. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sending Prometheus Metrics to Datadog Source: https://planetscale.com/docs/vitess/monitoring/prometheus-metrics-datadog If you're looking for more metrics than PlanetScale's native Datadog integration provides, this tutorial will show how to configure your [Datadog agent](https://docs.datadoghq.com/agent/) to scrape PlanetScale's [Prometheus infrastructure](/docs/vitess/integrations/prometheus) automatically, allowing you to collect detailed metrics for all of your PlanetScale branches. ## Overview In this tutorial, we'll assume that you have a Datadog Agent Version 7 running. For more information on what the Datadog Agent is and how to install it, start with the [Datadog Agent documentation](https://docs.datadoghq.com/getting_started/agent/). For the purposes of this guide, we'll be using a Datadog agent running with the recommended installation steps on a Linux system. ## Prerequisites You'll need a working Datadog agent and access to add a [Custom Agent Check](https://docs.datadoghq.com/developers/custom_checks/) to that instance. This may require `root` or `sudo` access on the machine running the Datadog agent. You'll also need a [Service token](/docs/api/reference/service-tokens) in your Organization, with the `read_metrics_endpoints` permission granted. ## Adding the Plugin to the Datadog Agent Go to [https://github.com/planetscale/planetscale-datadog](https://github.com/planetscale/planetscale-datadog), which is the repository that has our custom OpenMetrics Check. Place the unedited `planetscale.py` in the `checks.d` directory of your Datadog Agent. * On Linux, that is `/etc/datadog-agent/checks.d/` * On macOS, that is `/opt/datadog-agent/etc/checks.d/` Make sure that it belongs to the appropriate user. If you're using the recommended Linux installation steps, it will have created a `dd-agent` user: ``` $ pwd /etc/datadog-agent/checks.d $ ls -al planetscale.py -rw-r--r-- 1 dd-agent dd-agent 9261 Apr 2 22:54 planetscale.py ``` This file is owned by the `dd-agent` user and group in the `/etc/datadog-agent/checks.d` directory. If you're on macOS, it will depend on whether you installed the agent as a 'Single User Agent' or a 'Systemwide Agent'. If you picked Single User, there should be no additional permission changes needed. If you installed it as a Systemwide agent, make sure the user and group you installed the agent with as ownership of the file. ## Configuring the Datadog Agent Now that we have the plugin installed, we need to configure it. In the `conf.d` directory of the Datadog agent take the `conf.d/planetscale.yaml.example` file and edit it with your organization name and Service Token information. It should look like this: ```bash expandable theme={null} instances: - planetscale_organization: 'nick' # Required: Your PlanetScale organization ID ps_service_token_id: '${TOKEN_ID}' # Required: Your PlanetScale Service Token ID ps_service_token_secret: '${TOKEN}' # Required: Your PlanetScale Service Token Secret. Consider using Datadog secrets management: https://docs.datadoghq.com/agent/guide/secrets-management/ namespace: 'planetscale' # Required: Namespace for the metrics metrics: # Required: List of metrics to collect. Use mapping for renaming/type overrides. - planetscale_vtgate_queries_duration: vtgate_query_duration min_collection_interval: 60 send_distribution_buckets: true collect_counters_with_distributions: true ``` This configures the integration to look for all of the branches in the `"nick"` PlanetScale organization, only collect the `planetscale_vtgate_queries_duration` metric, which it will rename `vtgate_query_duration` and put it inside of the `planetscale` namespace. ### Collecting all metrics If you want to collect all available PlanetScale metrics instead of specifying individual metrics, you can use a wildcard pattern: ```yaml theme={null} instances: - planetscale_organization: 'your-org' ps_service_token_id: '${TOKEN_ID}' ps_service_token_secret: '${TOKEN}' namespace: 'planetscale' metrics: [".*"] min_collection_interval: 60 send_distribution_buckets: true collect_counters_with_distributions: true ``` Using `'.*'` in the metrics list tells the Datadog agent to scrape all metrics exposed by PlanetScale. This is useful when you want comprehensive monitoring without manually maintaining a list of specific metrics. Save the file at `planetscale.yaml`, making sure to double check permissions: ``` $ pwd /etc/datadog-agent/conf.d $ ls -al planetscale.yaml -rw-r--r-- 1 root root 1518 Apr 2 22:57 planetscale.yaml ``` ## Restart the Datadog Agent Now that this is configured and installed, restart the Agent: ``` $ sudo systemctl restart datadog-agent ``` ## Validating the PlanetScale Plugin Now that the Datadog Agent is running the PlanetScale plugin, metrics should start flowing into Datadog within a couple of minutes. To validate, we can ask the Datadog Agent: ``` sudo -u dd-agent -- datadog-agent check planetscale ``` If the plugin is installed successfuly, this should output the scrape targets for your branches, as well as metadata about when it was last run and how many metrics were emitted: ```bash expandable theme={null} $ sudo -u dd-agent -- datadog-agent check planetscale === Service Checks === [ { "check": "planetscale.api.can_connect", "host_name": "ubuntu", "timestamp": 1743638192, "status": 0, "message": "", "tags": [ "planetscale_org:nick" ] }, { "check": "planetscale.prometheus.health", "host_name": "ubuntu", "timestamp": 1743638192, "status": 0, "message": "", "tags": [ "endpoint:https://metrics.psdb.cloud/metrics/branch/7wxuxewx4l0p?..." ] }, { "check": "planetscale.prometheus.health", "host_name": "ubuntu", "timestamp": 1743638192, "status": 0, "message": "", "tags": [ "endpoint:https://metrics.psdb.cloud/metrics/branch/6o0rr27785fl?..." ] } ] Running Checks ============== planetscale (unversioned) ------------------------- Instance ID: planetscale:planetscale:8d4d64f696d967be [OK] Configuration Source: file:/etc/datadog-agent/conf.d/planetscale.yaml Total Runs: 1 Metric Samples: Last Run: 0, Total: 0 Events: Last Run: 0, Total: 0 Service Checks: Last Run: 3, Total: 3 Histogram Buckets: Last Run: 77, Total: 77 Average Execution Time : 809ms Last Execution Date : 2025-04-02 23:56:32 UTC (1743638192000) Last Successful Execution Date : 2025-04-02 23:56:32 UTC (1743638192000) Metadata ======== config.hash: planetscale:planetscale:8d4d64f696d967be config.provider: file ``` The Service Checks show that it has successfully connected to the PlanetScale API to request information about how to scrape for the branches in my organization, and it has successfully scraped both of what it discovered. We can also see that it successfully executed at `2025-04-02 23:56:32 UTC` and produced 77 Histogram Buckets. ## Adding Metrics In our earlier configuration, we only added one metric. For a complete list of what PlanetScale exposes, please take a look at our [Metrics Reference Documentation](/docs/vitess/integrations/prometheus-metrics). Note that the Datadog agent [normalizes metrics with certain suffixes starting in v7.32.0](https://github.com/DataDog/integrations-core/blob/master/openmetrics/README.md): > Starting in Datadog Agent v7.32.0, in adherence to the OpenMetrics specification standard, counter names ending in \_total must be specified without the \_total suffix. For example, to collect promhttp\_metric\_handler\_requests\_total, specify the metric name promhttp\_metric\_handler\_requests. This submits to Datadog the metric name appended with .count, promhttp\_metric\_handler\_requests.count. This means that to scrape a metric such as `planetscale_mysql_bytes_received_total`, you would configure the Datadog agent for `planetscale_mysql_bytes_received`. If I want to collect additional metrics, I can add them to the list: ``` metrics: # Required: List of metrics to collect. Use mapping for renaming/type overrides. - planetscale_vtgate_queries_duration: vtgate_query_duration - planetscale_edge_active_connections: active_connections ``` Then, restart the Datadog Agent: ``` $ sudo systemctl restart datadog-agent ``` If I check the status of the PlanetScale Plugin, I can see our last run added a Metric Sample: ```bash theme={null} Running Checks ============== planetscale (unversioned) ------------------------- Instance ID: planetscale:planetscale:fde586b60a54a38f [OK] Configuration Source: file:/etc/datadog-agent/conf.d/planetscale.yaml Total Runs: 1 Metric Samples: Last Run: 1, Total: 1 Events: Last Run: 0, Total: 0 Service Checks: Last Run: 3, Total: 3 Histogram Buckets: Last Run: 77, Total: 77 Average Execution Time : 826ms Last Execution Date : 2025-04-03 00:01:45 UTC (1743638505000) Last Successful Execution Date : 2025-04-03 00:01:45 UTC (1743638505000) ``` In the Datadog UI, I can see data for the `planetscale.active_connections` metric: ![Datadog Connections Metric](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/prometheus-datadog-graph.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=a27906622e857038e5b7e7f6c34c5598) ## What's Next? Now that you're sending a couple of metrics from PlanetScale to Datadog, take a look at our [full list](/docs/vitess/integrations/prometheus-metrics) and start building dashboards! ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Grafana Dashboard for PlanetScale Branches Source: https://planetscale.com/docs/vitess/monitoring/prometheus-metrics-grafana In this tutorial, you'll learn how to set up Grafana and connect it to a Prometheus instance to see metrics about your PlanetScale database. ## Introduction This guide requires that you've set up a Prometheus instance from our documentation. If you're already running Grafana in production and you're just looking for our standard dashboard template, you can find it [on GitHub](https://github.com/planetscale/grafana-dashboard). ## Install Grafana Grafana's [installation documentation](https://grafana.com/docs/grafana/latest/setup-grafana/installation/) contains information for their supported platforms. For this guide, we'll be setting this up locally on a macOS machine. If you're using a hosted Grafana option such as [Grafana Cloud](https://grafana.com/products/cloud/) or [AWS Managed Grafana](https://aws.amazon.com/grafana/) you can skip this step. On macOS, Grafana is availabile via [homebrew](https://brew.sh/), and I can install it with: ```bash theme={null} $ brew install grafana ``` This will download and install Grafana, and I can start it with: ```bash theme={null} $ brew services start grafana ``` When that succeeds, I can go to `http://localhost:3000/` and I should see the Grafana welcome page: ![Grafana Welcome Page](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-grafana-welcome.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=7eb0e85d512da6800feaaeed72519fe8) The default username and password for a new install is `admin` and `admin`. Grafana will ask you to change the password the first time you log in, please pick something more secure than `admin`. ### Adding a Prometheus Endpoint You can skip this step as well if you're already running a managed Prometheus or have added your datasource to Grafana already. If you're running Prometheus locally, you'll need to add that as a datasource. To do this: **Step 1** Open the menu in the top left and click "Connections" **Step 2** Search for "Prometheus" and pick the plain "Prometheus" option **Step 3** Click "Add new data source" in the top right of the page Now, you should look see a page that looks like this: ![Grafana Add Datasource](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-add-prometheus-connection.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=ae460e439d809b5724a517a615ae9271) You can call this whatever you want, we'll use the following: * Name: "PlanetScale" * Prometheus server URL: `http://localhost:9090/` ![Grafana Prometheus Configuration](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-prometheus-configuration.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=bd74325add64e87dd54c2398069f6c98) Because this is running on your local machine, we do not need to use any Authentication or TLS Scroll down to the "Interval behaviour" section and set the "Scrape interval" to `1m`. Finally, scroll to the bottom and click "Save & test". ## Import the PlanetScale Dashboard Now that we have our datasource added, let's import the PlanetScale Dashboard. This is a starter dashboard that PlanetScale has produced which shows an overview of your branch with the metrics that we expose. From the Grafana homepage, go to the top left menu and pick "Dashboards". In the top right, click "New" and then Import": PlanetScale maintains the latest version of the dashboard located here: [https://github.com/planetscale/grafana-dashboard/blob/main/overview.json](https://github.com/planetscale/grafana-dashboard/blob/main/overview.json) Download this file to your computer, and then click "Upload dashboard JSON file". Find the JSON file you downloaded in the previous step, and configure it with the Prometheus datasource that we added in an earlier: ![Grafana Prometheus Configuration](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-prometheus-configuration.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=bd74325add64e87dd54c2398069f6c98) Click 'Import' and you should be directed to the dashboard, configured to query your local Prometheus with the data it's been scraping! ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sending Prometheus Metrics to New Relic Source: https://planetscale.com/docs/vitess/monitoring/prometheus-metrics-newrelic If you're looking for your PlanetScale database metrics in your New Relic account, this tutorial will show how to configure a [Prometheus](https://prometheus.io/) instance to scrape PlanetScale's [Prometheus infrastructure](/docs/vitess/integrations/prometheus) automatically, allowing you to collect detailed metrics for all of your PlanetScale branches. While this tutorial is written for New Relic, using Prometheus' remote write is a common pattern for sending metrics to [AWS Managed Prometheus](https://aws.amazon.com/prometheus/), [Google Cloud Managed Service for Prometheus](https://cloud.google.com/stackdriver/docs/managed-prometheus), [Grafana hosted Prometheus](https://grafana.com/products/cloud/metrics/) and many other tools. For more information on Prometheus Remote Write and New Relic, see the [New Relic documentation on sending Prometheus metric data](https://docs.newrelic.com/docs/infrastructure/prometheus-integrations/get-started/send-prometheus-metric-data-new-relic/). ## Overview In this tutorial, we will be using an instance of [Prometheus](https://prometheus.io/) running on a Linux VM to scrape metrics from PlanetScale and then forward them to New Relic using [Remote Write](https://prometheus.io/docs/specs/prw/remote_write_spec/). We will make sure that Prometheus stays running by creating a [Systemd Unit File](https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files). The default configuration we will create will send all PlanetScale metrics to New Relic, and we will cover how to filter to drop certain metrics that may not be desired. In order to proceed, you'll need: * A [New Relic API Key](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/), make sure it is the `Ingest - License` type. * PlanetScale [Service token](/docs/api/reference/service-tokens) with `read_metrics_endpoints` permissions. ## Prometheus Installation First, let's download the latest release of Prometheus and create our user that is going to run it. We'll be using the latest 3.x release from the [GitHub Releases Page](https://github.com/prometheus/prometheus/releases). Create a `prometheus` user: ```bash theme={null} $ sudo useradd -M -U prometheus ``` ```bash theme={null} $ wget https://github.com/prometheus/prometheus/releases/download/v3.2.1/prometheus-3.2.1.linux-amd64.tar.gz $ tar xf prometheus-3.2.1.linux-amd64.tar.gz $ sudo mv prometheus-3.2.1.linux-amd64/ /opt/prometheus $ sudo chown prometheus:prometheus -R /opt/prometheus ``` This has put the Prometheus binary in `/opt/prometheus` along with the example configuration file that we can use. ## Create our Systemd Unit File Now that we have the binary in place, let's setup Systemd to run Prometheus by creating a Unit File in `/etc/systemd/system/prometheus.service` with the following contents: ```ini expandable theme={null} [Unit] Description=Prometheus Agent Documentation=https://prometheus.io/ After=network-online.target [Service] User=prometheus Group=prometheus Restart=on-failure ExecStart=/opt/prometheus/prometheus \ --config.file=/opt/prometheus/prometheus.yml \ --storage.agent.path=/opt/prometheus/data \ --web.listen-address=0.0.0.0:9091 \ --agent [Install] WantedBy=multi-user.target ``` ## Configure Prometheus Now that we've got Prometheus installed and a unit file present, let's configure Prometheus. We will be borrowing some of our configuration from the [Prometheus Guide](/docs/vitess/integrations/prometheus), and adding some New Relic specific configuration. Edit `/opt/prometheus/prometheus.yml` in your editor of choice so that it contains this, making sure to replace your org name, service token information, and New Relic API key: ```yaml theme={null} global: scrape_interval: 1m scrape_configs: - job_name: "${ORG}" http_sd_configs: - url: https://api.planetscale.com/v1/organizations/${ORG}/metrics authorization: type: "token" credentials: "${TOKEN_ID}:${TOKEN}" refresh_interval: 10m remote_write: - url: https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscale authorization: credentials: ${NEW_RELIC_API_TOKEN} ``` This configuration file does the following: * Configures Prometheus to discover scraping endpoints from the PlanetScale API using a service token * Points Prometheus to write the metrics it scrapes from PlanetScale to the New Relic API ## Starting Prometheus Now that we have a Systemd unit file and a configured Prometheus, let's run it! ```bash theme={null} $ sudo systemctl daemon-reload $ sudo systemctl start prometheus.service ``` We can also tell Systemd to run Prometheus when my VM boots: ```bash theme={null} $ sudo systemctl enable prometheus.service ``` Now, let's check to make sure everything is running properly: ```bash expandable theme={null} $ sudo systemctl status prometheus.service ● prometheus.service - Prometheus Agent Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; preset: enabled) Active: active (running) since Fri 2025-04-04 17:36:57 UTC; 38s ago Docs: https://prometheus.io/ Main PID: 745542 (prometheus) Tasks: 9 (limit: 9486) Memory: 21.2M (peak: 21.9M) CPU: 264ms CGroup: /system.slice/prometheus.service └─745542 /opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml --storage.agent.path=/opt/prometheus/data --web.listen-address=0.0.0.0:9091 --agent Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.804Z level=INFO source=main.go:1305 msg=EXT4_SUPER_MAGIC Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.804Z level=INFO source=main.go:1308 msg="Agent WAL storage started" Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.804Z level=INFO source=main.go:1437 msg="Loading configuration file" filename=/opt/prometheus/prometheus.yml Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.805Z level=INFO source=watcher.go:225 msg="Starting WAL watcher" component=remote remote_name=fbe64a url="https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscal> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.805Z level=INFO source=metadata_watcher.go:90 msg="Starting scraped metadata watcher" component=remote remote_name=fbe64a url="https://metric-api.newrelic.com/prometheus/v1/write?prometh> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.805Z level=INFO source=watcher.go:277 msg="Replaying WAL" component=remote remote_name=fbe64a url="https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscale" queu> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.806Z level=INFO source=main.go:1476 msg="updated GOGC" old=100 new=75 Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.806Z level=INFO source=main.go:1486 msg="Completed loading of configuration file" db_storage=791ns remote_storage=610.171µs web_handler=897ns query_engine=301ns scrape=517.175µs scrape_s> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.806Z level=INFO source=main.go:1213 msg="Server is ready to receive web requests." ``` This reports that prometheus is `active (running)`, and I can see the logs showing that it started successfully. Great! ## Querying on New Relic After a couple of minutes, head over to your New Relic dashboard and we can query for your database metrics. First, let's get a list of the database branches in the `nick` organization that I'm using to test: ```bash theme={null} $ pscale branch list test --org nick ID NAME PARENT BRANCH REGION PRODUCTION SAFE MIGRATIONS READY CREATED AT UPDATED AT -------------- ------------ --------------- --------- ------------ ----------------- ------- -------------- ---------------- 7wxuxewx4l0p main n/a us-east Yes No Yes 2 years ago 50 minutes ago 6o0rr27785fl partitions main us-east No No Yes 2 months ago 7 minutes ago ``` For this, we'll use the `7wxuxewx4l0p` branch. Using New Relic's NRQL, we can visualize the memory usage of my VTTablet instances with the following query: ```sql theme={null} FROM Metric SELECT average(planetscale_pods_cpu_util_percentages) WHERE planetscale_database_branch_id = '7wxuxewx4l0p' AND planetscale_component='vttablet' SINCE 30 minutes AGO TIMESERIES FACET planetscale_pod ``` Because my `main` branch is production, we will see the memory usage for my primary and both my replicas over the last 30 minutes: ![New Relic Memory Query](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/tutorials/metrics-new-relic-dashboard.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=5ae095318cdab9644ca82f12f11ad58f) ## Filtering Metrics If you don't want to ingest every metric into New Relic, you can tell Prometheus to drop certain metrics. For more information, see the [New Relic Documentation](https://docs.newrelic.com/docs/infrastructure/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration/#allow-deny). If we adjust our Prometheus configuration that we have in `/opt/prometheus/prometheus.yml` we can instruct Prometheus to drop all metrics unless they match a certain naming convention: ```yaml theme={null} remote_write: - url: https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscale authorization: credentials: ${NEW_RELIC_API_TOKEN} - source_labels: [__name__] regex: "planetscale_pods_(.*)" action: keep ``` If you replace your `remote_write` block with what's above, Prometheus will only forward the timeseries that match the `planetscale_pods_*` name. For a full list of metrics, see our [Metric List](/docs/vitess/integrations/prometheus-metrics). ## What's Next? Now that you have your branch metrics in New Relic, you can create dashboards and alerts for conditions such as high CPU or replication delay. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Query Insights Source: https://planetscale.com/docs/vitess/monitoring/query-insights PlanetScale Insights gives you a detailed look into **all active queries** running against your database. ## Overview This in-dashboard tool allows you to identify queries that are running too often, too long, returning too much data, producing errors, and more. You can scroll through the performance graph to detect the time that a query was impacted and, if applicable, the [Deploy Request](/docs/vitess/schema-changes/deploy-requests) that affected it. You can also see a [list of all queries](#queries-overview) performed on your database in the last 24 hours. For further analysis, you can sort these by metrics like amount of rows read, time per query, and more. With this built-in tool, you can easily diagnose issues with your queries, allowing you to optimize individual queries without much digging. We will also alert you of any active issues your database may be having in the [Anomalies](/docs/vitess/monitoring/anomalies) page. This feature flags queries that are running significantly slower than expected. Insights will also automatically recommend schema changes to improve database performance, reduce memory and storage, and improve your schema based on production database traffic. Read more about the supported recommendations and how to use them in the [schema recommendations documentation](/docs/vitess/monitoring/schema-recommendations). ## Insights page overview To view Insights for your database, head to the [PlanetScale dashboard](https://app.planetscale.com), select your database, and click the "**Insights**" tab. The dropdown on the top left lets you select which branch you want to analyze. You can also choose which servers you want to view insights for: primary or replicas. ![PlanetScale Insights overview page](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/query-insights/query-insights-overview.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=3f1d4d7528a976ec1fc943fc96bf7172) You can click the dates listed above the graph to scroll through the past seven days. To further narrow down query analysis, you can select a time range by clicking on the graph and dragging the cursor across. This will zoom in on the selected timeframe. You also have the option to save a screenshot of the graph by clicking "Save". If any deploy requests were deployed during the selected period, you will also see an overlay with a link to the deploy request. This can help you quickly assess any impact a deploy request had on your database. ### Queries overview table The table underneath the graph shows all queries performed on your database in the selected timeframe (last 24 hours by default). For more information about how to read and interpret this data, see the [Queries overview](#queries-overview) section. ### Insights graphs Once you have selected the branch and server you want to analyze, you can begin exploring the insights for them in the following sections: > **[Query latency](#query-latency)** > **[Queries](#queries)** > **[Rows read](#rows-read)** > **[Rows written](#rows-written)** > **[Any query metric](#any-query-metric)** The remaining sections of this doc walk through how to interpret and act on the data in each graph. If you'd like to see a practical example of how to use Insights to debug a performance issue, check out our [Announcing Insights blog post](https://planetscale.com/blog/introducing-planetscale-insights-advanced-query-monitoring) or [this YouTube video](https://www.youtube.com/watch?v=kkjAxSViOAA) walking you through an example. ## Query latency The default graph depicts your database's query latency in milliseconds over the last 24 hours. By default, the graph contains two line charts showing `p50` and `p95` latency. This means 50% and 95% of requests, respectively, completed faster than the time listed. You can also click on the `p99` and `p99.9` pills to toggle those on, or click `p50` or `p95` to toggle those off. ## Queries The Queries graph displays insights about all active running queries in your database. The graph displays total queries per second against the specified time period. ## Rows read The Rows read graph displays the total number of rows read per second across the selected time period. ## Rows written The Rows written graph displays the total number of rows written per second across the selected time period. ## Any query metric The last graph tab plots any single query metric you choose across the selected time period. This tab is named after the metric you have selected, so it reads "Total time" or "Bytes returned" rather than a fixed title. To graph a metric: 1. Click the chevron on the last graph tab to open the metric list. 2. Select a metric. The graph redraws with that metric plotted over the time period you already have selected. The list offers the same metrics you can add as columns in the queries table, described in [Available query statistics](#available-query-statistics). It includes only the metrics your branch collects, so the options vary from branch to branch. Sparklines in the queries table lead to the same graphs. Clicking a sparkline opens that query pattern's [deep dive](#query-deep-dive) with the matching metric already selected, which takes you from every query on the database to one query pattern in a single click. ## Queries overview The table underneath the graph shows queries performed on your database in the selected timeframe (last 24 hours by default). > [!NOTE] > > The queries table does not show following statements types: `BEGIN`, `COMMIT`, `RELEASE`, `ROLLBACK`, `SAVEPOINT`, `SAVEPOINT_ROLLBACK`, `SET`. You may see some placeholder values in the queries, such as `:v1`. This is because we consider the actual data private and normalize it away. > [!NOTE] > > You have the option to [opt in to complete query collection](#complete-query-collection) to see the full SQL statements. You may also see one or more orange icons next to some queries. * A shard icon indicates that the query requires execution across multiple shards. * An exclamation point icon indicates that the query is not currently using an index and requires a full table scan. Hovering over the icon will show a tooltip with information about the meaning of the icon. > [!NOTE] > > The queries table may also show internal PlanetScale queries that are used to monitor the state of your cluster. This query overviews table shows the same data for all graphs on this page. For information about [Anomalies](/docs/vitess/monitoring/anomalies) and [Errors](/docs/vitess/monitoring/errors), refer to those pages. ### Available query statistics You can customize the metrics that show up on the Queries list by selecting columns in the "View options" dropdown. * **Query** - The query that was run. * **Keyspace** — The default keyspace associated with the connection that issued the query, if set. * **Qualified table** — The table(s) referenced in the query, in the format `keyspace_name.table_name`. * **Table keyspace** — The keyspace(s) associated with the tables referenced in the query. (This may differ from the connection keyspace.) * **Table** — The table(s) being queried or modified. * **% of runtime** — The percent of the total runtime the query pattern is responsible for (query pattern time divided by the cumulative time of all query patterns on your database). * **Count** — The number of times this query has run. * **Total time (s)** — The total time the query has run in seconds. * **`p50` latency** — The `p50` latency for the query in milliseconds. This means that 50% of requests completed faster than the time listed. * **`p99` latency** — The `p99` latency for the query in milliseconds. This means that 99% of requests completed faster than the time listed. * **Max latency** — The maximum observed latency for the query in milliseconds. * **Rows returned** — The total number of rows fetched by a `SELECT` statement. This includes all times the query has run in the displayed time frame. * **Rows read** — The total number of rows read. This includes all times the query has run in the displayed time frame. * **Rows read/rows returned** — The result of dividing total rows read by rows returned in a query. A high number can indicate that your database is reading unnecessary rows, and the query may be improved by adding an index. * **Rows affected** — The total number of rows modified by an `INSERT`, `UPDATE`, or `DELETE` statement. This includes all times the query has run in the displayed time frame. * **Bytes returned** — The total number of bytes returned to clients in query responses. This includes all times the query has run in the displayed time frame. * **Bytes returned per query** — The average number of bytes returned to clients per query execution. * **Max bytes returned per query** — The largest response, in bytes, returned by any single execution of the query. * **Bytes received** — The total number of bytes received from clients, which includes the statement and any parameters sent with it. This includes all times the query has run in the displayed time frame. * **Bytes received per query** — The average number of bytes received from clients per query execution. * **Max bytes received per query** — The largest request, in bytes, received from a client for any single execution of the query. * **Tablet calls per query** (only visible for databases with at least one sharded keyspace) — The average number of VTTablet calls/queries for each execution of the query pattern. Calculated as the sum of VTTablet calls divided by the number of queries issued by the client in the selected time period. Query patterns where this value is equal to the number of shards are known as "scatter gather queries". * **Last run** — The last time a query was run. The bytes columns cover both directions of network traffic for a query pattern: "Bytes returned" measures the responses your database sends back to clients, and "Bytes received" measures the requests clients send to your database. Together they help you find the queries responsible for the most network traffic. You can also sort the columns for quick analysis by clicking on the title at the top of each column. If `Show sparklines` is selected, numeric columns in the queries table show a time series graph of the value within the selected time period. Click a sparkline to open that query pattern's [deep dive](#query-deep-dive) with the same metric graphed full size. ### Query filtering The search bar above the table allows you to filter queries as needed. You can filter for query SQL, keyspace (connection keyspace, and/or keyspace of tables referenced by the query), table name, query count, query latency, multisharded queries, index name, and if the query was indexed. Click on the `?` next to the search bar for the full list of search syntax. ### Query deep dive Clicking on a query in the Queries list will open a new page with more information about that query. You'll first see the full query pattern, which displays the query with data normalized away. This query may run several times with different values, which Insights combines into a single query pattern. You can display an LLM-generated summary of the query by clicking "Summarize query." [Learn more about how PlanetScale uses AI here](/docs/how-we-use-ai). You can also display a [query `EXPLAIN` plan](https://planetscale.com/blog/how-read-mysql-explains) by clicking "Show explain plan", which generates the [execution plan](https://dev.mysql.com/blog-archive/mysql-explain-analyze/) for the selected query. You may have to fill in some sample values designated with placeholders like `:v1`. We use placeholders in the patterns both so you can look at whole patterns at once and so the literal values remain private. Note, if you're viewing the `EXPLAIN` plan on a production branch, this button will be disabled unless you enable production web console access in your database Settings page. If you'd like to further interact with the query, click "Open query in web console", and you'll be taken to your in-dashboard web console, where you can run the `EXPLAIN` plan. #### Additional query information Beneath the query pattern is a graph with more information about the query. The set of available metrics/tabs include: Query latency, Queries, Rows read, Rows written, Errors and Indexes, along with a final tab that graphs [any query metric](#any-query-metric) you select. A Vindexes tab is also available for queries against a [sharded keyspace](/docs/vitess/sharding). The Indexes graph (which is not shown on the database-level page) shows the percentage of queries that used each of the listed indexes in each time bucket. (Currently only `SELECT` queries show index usage information.) The Vindexes graph (also not shown on the database-level page) shows the percentage of queries that used each of the listed [vindexes](/docs/vitess/sharding/vindexes) in each time bucket. Beneath the time series graphs you will see summary statistics for the query pattern. These data are scoped to the same time period shown in the main query pattern graphs. The available metrics have the same definitions as the query statistics listed in the main insights page. `SELECT` queries include a horizontal bar graph that shows the cumulative usage of each index over the complete time period shown in the main query pattern graphs. Additionally, for *all* queries against a sharded keyspace, another horizontal bar graph shows the cumulative percentage of queries that used each vindex over the time period. To change the time period reflected in the graphs and summary statistics, click and drag to restrict the time window, or click on one of the day icons above the graph to select a different day. #### Notable queries Underneath the graph, you'll see a table with more information about notable instances of the query, which are defined as queries that took longer than 1s, read more than 10,000 rows, or produced an error. If any of the selected queries have [query tags](/docs/vitess/monitoring/query-tags) attached, you'll see the key-value pairs in the table under `Tags`. See [Query tags](/docs/vitess/monitoring/query-tags) for how to add tags to your queries. The table also surfaces when the query started, rows returned, rows read, rows affected, the time it took the query to run (in ms), and the user associated with the query. ### Complete query collection If you would prefer to view the raw query data using Insights, you can enable this option in your database settings page. In the dashboard, select your database, click "**Settings**", scroll down until you see "**Complete query collection**", and click "**Enable**" to opt in. With this enabled, Insights will gather the complete raw SQL statements and display them when a query deep dive is selected and in the `EXPLAIN` plan. For example, if you select a query from the table on the Insights page, the query pattern and the selected queries below it will display the full query. Enabling complete query collection is beneficial when performance varies significantly within the same query pattern, and you need to see the full SQL statement, without placeholders, to identify the correct source of the performance issue. However, full queries may contain personally identifiable information, so it's important to consider this before opting in to the feature. Because of this, only [Organization Administrators](/docs/security/access-control#organization-administrator) can choose to opt a database into complete query collection. Please read our [Privacy Policy](https://planetscale.com/legal/privacy) for more information on how we collect, process, and disclose your personally identifiable information. #### Complete query collection and prepared statements With prepared statements, complete query collection will not display the query values. That is because the MySQL query that is collected is parameterized at the database level, and we cannot extract the bind variables. PlanetScale does not recommend using prepared statements, we recommend disabling them in order to use complete query collection. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Tags Source: https://planetscale.com/docs/vitess/monitoring/query-tags Attach key-value metadata to SQL statements to filter performance data in Insights. ![Insights tags](https://mintcdn.com/planetscale-2/Ez_PNQOyZBhNfDDQ/vitess/monitoring/insights-tags-light.png?fit=max\&auto=format\&n=Ez_PNQOyZBhNfDDQ\&q=85\&s=fa09e8d424f13817db98b2528dd1fc7f) ## Overview Query tags are key-value pairs embedded in SQL comments written into each statement to the database. PlanetScale parses these tags automatically and uses them to show filtered tag-based performance data alongside individual query executions in [Insights](/docs/vitess/monitoring/query-insights). Tags follow the [SQLCommenter](https://google.github.io/sqlcommenter/) specification, an open standard created by Google for attaching metadata to SQL statements as comments. ## Tag format A tag comment is appended to the SQL statement inside `/* */` delimiters. Each tag is a `key='value'` pair, and multiple tags are separated by commas: ```sql theme={null} SELECT * FROM orders WHERE status = 'pending' /*app='web',route='api-export'*/; ``` This example would create the following tags in the PlanetScale dashboard: * **app:** `web` * **route:** `api-export` Values are URL-encoded and wrapped in single quotes, following the [SQLCommenter specification](https://google.github.io/sqlcommenter/spec/). > [!WARNING] > > Tags must appear **before** the semicolon that ends the SQL statement. Tags placed after the semicolon are not recognized by PlanetScale. > > ```sql theme={null} > -- Correct: tag before semicolon > SELECT 1 /*category='analytics'*/; > > -- Incorrect: tag after semicolon > SELECT 1; /*category='analytics'*/ > ``` > > When a tag appears after the semicolon, the database treats it as a separate statement (a no-op comment), so Insights sees only the original query without any tags. > [!NOTE] > > If you're sending queries with comments using the MySQL shell, make sure you have [enabled comments with the `-c` flag](https://dev.mysql.com/doc/refman/en/mysql-command-options.html#option_mysql_comments). ## Where tags appear in PlanetScale ### Insights Queries in the Insights page can be filtered by tags by using the search bar. For example, to filter for queries with the tag `action='analytics'`, use the following syntax: ``` tag:action:analytics ``` ### Insights → Tags "Tags" is a top-level item under [Insights](/docs/vitess/monitoring/query-insights). Select one or many key/value pairs to filter the performance data. Or see the tags of all queries within the currently selected time range. You can also see the tags associated with a query on the query details page (click on the query text from the list in Insights). ### Tag cardinality limits You can attach tags with any cardinality. When a tag produces too many distinct values, PlanetScale may collapse the value to keep tag data usable in Insights. Tag values may be collapsed when either of these limits is exceeded: * Any query pattern has more than 20 values for the tag * Any query pattern has more than 50 distinct combinations of tag values within a 15-second period When a tag value is collapsed, PlanetScale removes the original value and reports it as `Collapsed` in the query details page. If you search for a tag with collapsed values, or open the Tags view for a tag with collapsed values, PlanetScale shows a warning with the percentage of results affected by collapsing. ## Adding tags to your queries There are two approaches to tagging queries: using an SQLCommenter integration that adds tags automatically, or appending tag comments manually. ### SQLCommenter integrations The [SQLCommenter project](https://google.github.io/sqlcommenter/) provides middleware and plugins for several frameworks and ORMs that automatically attach tags to every query: | Language | Supported frameworks / ORMs | | - | - | | Python | Django, SQLAlchemy, Flask | | Java | Hibernate, Spring | | Node.js | Knex.js, Sequelize.js, Express.js | | Go | `database/sql`, gorilla/mux, `net/http` | | PHP | Laravel | | Ruby | Ruby on Rails | [Drizzle](https://orm.drizzle.team/docs/sql-comments) and Prisma support SQLCommenter natively. These integrations typically inject framework-level context like the route, controller, and framework version. Some support adding custom key-value pairs. > [!NOTE] > > Several popular ORMs and query builders used with PlanetScale do not have official SQLCommenter support, including **Bun** and **Kysely**. If you use one of these, see [manual tagging](#manual-tagging) below. ### Manual tagging If your ORM or query builder does not have an SQLCommenter plugin, you can append the tag comment directly to your SQL statements. The format is straightforward: add `/*key='value'*/` before the semicolon. #### Raw SQL ```sql theme={null} SELECT * FROM orders WHERE user_id = ? /*app='checkout',feature='order-history'*/; ``` #### With a query builder ```typescript theme={null} // Example using Kysely const result = await db .selectFrom('orders') .where('user_id', '=', userId) .selectAll() .execute({ // Append the tag comment to the generated SQL queryId: "/*app='checkout',feature='order-history'*/" }); ``` > [!NOTE] > > The exact mechanism for appending a comment varies by library. Some query builders support a comment or annotation option, while others require you to modify the SQL string directly. Check your library's documentation for the best approach. #### Encoding rules When adding tags manually, follow these rules from the SQLCommenter specification: * **URL-encode** values that contain special characters. For example, `/api/export` becomes `%2Fapi%2Fexport`. * **Escape** single quotes within values with a backslash: `name='O\'Brien'`. * **Sort** key-value pairs lexicographically by key if you want to match the specification exactly, though PlanetScale does not require a specific order. ## Troubleshooting ### Tags are not appearing in Insights The most common cause is **tag placement**. Tags must appear before the semicolon. See the [warning above](#tag-format) for the correct format. If your tags are correctly placed and still not appearing: * Verify that your queries are being executed and appear in the [Insights queries table](/docs/vitess/monitoring/query-insights) * Check that you are searching with the correct syntax: `tag:key:value` (for example, `tag:category:analytics`) * If you're using the MySQL shell, confirm you have [enabled comments with the `-c` flag](https://dev.mysql.com/doc/refman/en/mysql-command-options.html#option_mysql_comments) ### ORM is stripping comments Some ORMs and connection poolers may strip SQL comments before sending queries to the database. If you have added tags but they are not appearing: * Check whether your ORM has a setting to preserve SQL comments * If you use a connection pooler, verify that it is not configured to strip comments ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Schema recommendations Source: https://planetscale.com/docs/vitess/monitoring/schema-recommendations With schema recommendations inside of PlanetScale Insights, you will automatically receive recommendations to improve database performance, reduce memory and storage, and improve your schema based on production database traffic. ## Overview Schema recommendations uses query-level telemetry to generate tailored recommendations in the form of DDL statements that can be applied directly to a database branch and then deployed to production. Schema recommendations are also available through the [PlanetScale MCP server](/docs/mcp-server). AI coding agents can evaluate and implement recommendations on a recurring schedule; see [automated database optimization](/docs/self-improving-database). > [!NOTE] > > If you are a PlanetScale Enterprise customer, please get in touch with your account manager to learn how you can fully benefit from schema recommendations. ## How to use schema recommendations To find the schema recommendations for your database, go to the “**Insights**” tab in your PlanetScale database and click “**View recommendations**.” You will see the current open recommendations that may help improve database performance. Select a recommendation to learn more. Each recommendation will have the following: * An explanation of the recommended changes, including some of the benefits of the recommended change (E.g., reduced memory and storage, decreased execution time, prevent ID exhaustion) * The schema or query that it will affect * The exact DDL that will apply the recommendation * The option to apply the recommended change to a branch for testing and a safe migration > [!NOTE] > > Schema recommendations that depend on your database traffic run **once per day**. Recommendations that depend only on database schema are run whenever the the schema of your default branch is modified. Schema recommendations are generated only for the database's default branch. ![Example of a recommendation to add an index](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/schema-recommendations/add-index-recommendation.jpg?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=82cc03a57deadff4ce0449e042167f35) ### Applying a recommendation Once you better understand the recommendation, you can apply the recommendation by either: * Applying it directly through a database branch * Making the schema change directly in your application or ORM code We recommend making the schema change directly in your application or ORM code if it is where you manage your schema outside of PlanetScale. We look for different migration framework’s table names in your schema to alert you when you might want to change the schema directly in your application or ORM code. If you manage the schema directly in the database, you can use the following method. #### Applying a recommendation directly through a database branch When directly applying through a database branch, you can apply the recommendation by creating and applying it to a new or existing branch. After applying the recommendation, click on the branch name to see the recommended schema changes. At this point, you can connect to this branch to do your own testing of the change. You can create a connection string for this branch using the “**Connect**” button. Once ready, you can use the “**Create deploy request**” button on the branch page to open a deploy request to merge the changes with your base branch. Once it is reviewed and ready to be deployed, you can deploy the changes to production. If you realize you have made a mistake, you can still [revert the schema change](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change) in the deploy request for up to 30 minutes. #### Applying a recommendation directly in your application or ORM code When directly applying through your application or ORM code, you can take the recommended change from PlanetScale and make the equivalent change in your code. The exact DDL provided in each recommendation will help you know what change to make. The exact change will depend on how you manage your schema in your application code. Then, once you have made the change, you will run the same migration process you would run for any other schema changes with PlanetScale. We recommend doing this inside a development branch with [safe migrations](/docs/vitess/schema-changes/safe-migrations). This will allow you to do your own testing of the change inside the isolated branch. You can create a connection string for this branch using the “**Connect**” button. Once ready, you can use the “**Create deploy request**” button on the branch page to open a deploy request to merge the changes with your base branch. Once it is reviewed and ready to be deployed, you can deploy the changes to production. If you realize you have made a mistake, you can still [revert the schema change](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change) in the deploy request for up to 30 minutes. ### Closing a recommendation Recommendations are automatically closed when: * The changes have been deployed to the default branch in a deploy request * The SQL is applied to the default branch by some other means * Other schema changes to the default branch make the recommendation unnecessary Once a recommendation is closed, PlanetScale will never re-suggest it. ## Supported schema recommendations The following are the currently supported schema recommendations: > **[Adding indexes for inefficient queries](#adding-indexes-for-inefficient-queries)** > **[Removing redundant indexes](#removing-redundant-indexes)** > **[Preventing primary key ID exhaustion](#preventing-primary-key-id-exhaustion)** > **[Dropping unused tables](#dropping-unused-tables)** > **[Upgrading legacy character sets and collations](#upgrading-legacy-character-sets-and-collations)** The impact of schema recommendations can vary by recommendation. In the following sections, we will inform you of each recommendation’s potential impacts and explain the recommendation further. > [!NOTE] > > Schema recommendations may not be in line with your desired outcomes. PlanetScale shall not be held liable for any actions you take based on these recommendations. ### Adding indexes for inefficient queries Indexes are crucial for relational database performance. With no indexes or suboptimal indexes, MySQL may have to scan a large number of rows to satisfy queries that only match a few records. This results in slow queries and poor database performance. The right index can reduce query execution time from hours to milliseconds. You can read more about [how database indexes work in this blog post](https://planetscale.com/blog/how-do-database-indexes-work). #### How PlanetScale recommends adding indexes To find missing indexes, Insights scans your query performance data to identify queries over the past 24 hours that consume significant resources and have a high aggregate ratio of rows read compared to rows returned. It will then parse the query to extract indexable columns, estimate each column’s cardinality (number of unique values) to determine optimal column order, and suggest a suitable index. In multi-column index suggestions, we first order columns with the most selective (highest cardinality). Our index recommendation engine doesn’t yet support all queries types, and some queries cannot benefit from an index. #### Caveats * Indexes do require both more memory and storage. Evaluating these additional costs against your application’s improvement in read performance is always a good idea. This is also why we offer recommendations to [remove redundant indexes](#removing-redundant-indexes). * Indexing is a complicated topic and depends on many factors, such as the distribution of values in your database and the particular queries your database receives. Every effort is made to ensure our suggestions improve performance, but verifying and measuring is important. * If you’re unsure about the impact of adding an index, we recommend benchmarking the index in a non-production environment. If you can simulate production-level traffic, you can do this inside a development branch. * If you decide to deploy a suggested index to production, it is a good idea to use Insights to verify that your index has the desired effect on relevant queries. If you realize it is not the desired effect, you can still [revert the schema change](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change) in the deploy request for up to 30 minutes. ### Removing redundant indexes While indexes can drastically improve query performance, having unnecessary indexes slows down writes and consumes additional storage and memory. #### How PlanetScale recommends removing indexes Insights scans your schema every time it is changed to find redundant indexes. We suggest removing two types of indexes: * Exact duplicate indexes - an index that has the same columns in the same order * Left prefix duplicate indexes - an index that has the same columns in the same order as the prefix of another index There are differences between the two, so note your exact recommendation and the following caveats. #### Caveats Removing redundant indexes is more nuanced than adding an index. * Exact duplicate indexes are *always* safe to remove. * Left prefix duplicate indexes are *almost always* safe to remove, but in some cases can lead to a performance regression. Usually, the larger index can be used instead of the left prefix duplicate indexes. Read the following section for more details on how this works. #### Left prefix duplicate indexes Since MySQL can use the leftmost elements of a multi-column index to efficiently find rows, an index that is a left-prefix duplicate of another index can *usually* be removed. To understand this, consider the following table. ```sql theme={null} CREATE TABLE `t` ( `id` bigint unsigned NOT NULL AUTO_INCREMENT, `a` bigint, `b` bigint, `c` bigint, PRIMARY KEY (`id`), KEY idx_a_b (`a`, `b`), KEY idx_a_b_c(`a`, `b`, `c`) ) ``` Consider the following query: ```sql theme={null} SELECT * FROM t WHERE a = ? and b = ? ``` This query can use either `idx_a_b` or `id_a_b_c` to quickly find results because both indexes have `a` and `b` as the leftmost columns. In general, removing left prefix duplicates, `idx_a_b` in our case, will result in lower memory and storage usage and improve the performance of inserts, updates, and deletes to this table because there is one less index to be maintained. However, there are a few important caveats. ##### Queries that use the primary key MySQL implicitly appends the primary key to the end of indexes. An index declared as `KEY idx (a, b)` can be thought of as `KEY idx (a, b, id)`, assuming `id` is the primary key. This has the potential to affect queries that make use of the indexed columns and the primary key. For example: ```sql theme={null} SELECT * FROM t WHERE a = ? AND b = ? ORDER BY id ``` This query can be served very efficiently by an `idx_a_b` because the `ORDER` clause is satisfied by the implicitly appended primary key. Since `idx_a_b_c` includes an additional column (`c`), MySQL must perform additional work to return rows in the correct order when using this index. ##### Index size Even though MySQL can use either index in our table to efficiently look up rows by a and b, searching through the larger `idx_a_b_c` index may take longer simply because the index is larger and will require more pages to be scanned. In most cases, you can safely remove redundant left-prefix indexes, resulting in better performance. However, ensure that your application doesn’t issue performance critical queries that the above mentioned issues will significantly impact. If you’re unsure about this, we recommend looking through your Insights dashboard to find queries that may be affected. ### Preventing primary key ID exhaustion As new rows are inserted, it’s possible for auto-incremented primary keys to exceed [maximum allowable value](https://planetscale.com/blog/mysql-data-types-integers#different-integer-data-types) for the underlying column type. When the column reaches the maximum value, subsequent inserts into the table will fail, which can cause a severe outage to your application. #### How PlanetScale detects primary key ID exhaustion Insights scans all of the `AUTO INCREMENT` primary keys in your database schema and checks the current `AUTO INCREMENT` value daily to identify where you might be approaching primary key ID exhaustion. If Insights detects that one of the columns is above 60% of the maximum allowable type, it will recommend changing the underlying column to a larger type. Additionally, Insights scans queries to parse joins and correlated subqueries to find foreign keys and suggests increasing the column size for those columns. #### Caveats * It is always a good idea to manually check your database and application for foreign key references to the column you are increasing to ensure none were missed. ### Dropping unused tables Dropping unused tables can help clean up data that is no longer needed and reduce storage. If the table is large, it can also decrease backup and restore time. If you are unsure if a table should be retained but decide to drop the table, make sure to [create a manual backup](/docs/vitess/backups#create-manual-backups) of your database before you deploy the change. If you realize after dropping the table that it is needed, you can still [revert the schema change](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change) in the deploy request for up to 30 minutes with no data loss. If you are outside the 30-minute period, you must [restore and recover the data inside a database backup branch](/docs/vitess/backups#restore-from-a-backup). If you determine that the table should be retained, close the recommendation, preventing the suggestion from being remade. #### How PlanetScale recommends dropping unused tables Insights scans your query performance data daily to identify if any tables are more than four weeks old and haven’t been queried in the last four weeks. #### Caveats * Only you can know if the table’s data is no longer needed. Ensure that the table is never used (even infrequently) and does not contain important data before removing it. * Once a drop unused table recommendation is opened, it will remain open even if it is subsequently queried. Check your Insights data to verify that the table is still unused before permanently dropping it. ### Upgrading legacy character sets and collations All non-binary string columns, such as `CHAR`, `VARCHAR` and `TEXT`, have an associated character set and collation. The character set represents the range of valid characters and how they are stored. The collation controls how values are compared, such as in `ORDER BY` clauses and `UNIQUE KEY`s. Over time MySQL has added support for new character sets and collations. The most recent character sets and collations have numerous advantages over their legacy counterparts. * Full Unicode support including emoji * Improved performance * Improved sort order for multi-byte characters * Awareness of trailing spaces in comparisons For these reasons we suggest using the `uftmb4` character set and the `utf8mb4_0900_*` collations. Unless you need byte-level comparisons (`utf8mb4_0900_bin`) or language-specific comparisons (e.g. `utf8mb4_0900_es_*` for spanish), we recommend using MySQL's default collation `utf8mb4_0900_ai_ci`. > [!NOTE] > > `0900` in the name of MySQL's modern collations [refers to](https://dev.mysql.com/blog-archive/mysql-character-sets-unicode-and-uca-compliant-collations/) version 9.0.0 of the Unicode Collation Algorithm. `ai` and `ci` stand for accent insensitive and case insensitive, respectively. Accent and case sensitive collations are also available. To see a list of all available character sets and collations, connect to your database and run `show collation;`. When in doubt, use `utf8mb4_0900_ai_ci`. > [!NOTE] > > The `utf8` character set is, for compatibility reasons, an alias for `utf8mb3`. We suggest the `utf8mb4` character set instead. #### Caveats ##### Joins Before upgrading from a legacy character set/collation, it is important to ensure there are no joins on string columns that will become incompatible after the upgrade. For example, if you issue the query ```sql theme={null} select * from t1 inner join t2 on t1.name = t2.name ``` both `t1.name` and `t2.name` need to have have identical character sets and collations. If the character sets are the same but the collations are different, the query will fail. If the character sets are different, the query will succeed but will be unable to make use of indexes, which can cause unexpected table scans and dramatically degrade performance for large tables. Character set/collation upgrade recommendations are not created for tables with a recent history of joins on string columns. However, we cannot automatically detect all join types so it is important to verify that your application does not join on a table's string columns prior to deploying a recommendation that would alter its character set or encoding. To upgrade the character sets or collations for tables that have string column joins, we recommend upgrading both tables to the same character set and collation in a single deploy request to minimize disruption. ##### Length limits > [!NOTE] > > Length limit issues only apply when upgrading from `utf8mb3` to `utf8mb4` character sets. Collation-only changes are unaffected. When upgrading from `utf8mb3` to `utf8mb4`, it is possible that some adjustments will have to be made to column types. Because `utf8mb3` is a subset of `utf8mb4`, existing data will not increase in size. However, `utf8mb4` requires *up to* four bytes per character instead of *up to* three. Because of the this, some column type changes may be required. Some data type changes occur automatically when applying the recommendation. `TEXT`-type columns increase to the next largest size (`TEXT` becomes `MEDIUMTEXT`, `MEDIUMTEXT` becomes `LONGTEXT` etc). Because `utf8mb4` possibly requires an extra byte per character, and `TEXT`-type columns are defined by the maximum number of *bytes*, not characters, these columns are upgraded to ensure the ability to store at least the same number of *characters* in the column before and after the `CONVERT TABLE` command in the recommendation. If storing fewer maximum-length characters is acceptable, you can alter the deploy request produced by the recommendation to restore the original `TEXT`-type column definitions. Other data type changes may need to be made manually. For example, the column definition `VARCHAR(20000) CHARACTER SET utf8mb3` is legal, because 20,000 characters \* 3 bytes per character = 60,000 bytes and `VARCHAR` columns can accommodate up to 65,535 bytes. However, when attempting to convert this column's character set to `utf8mb4`, the number of bytes required to store 20,000 4-byte characters (80,000 bytes) is over the `VARCHAR` byte limit. In this case, the `VARCHAR` length can be decreased, or the column can be changed to `MEDIUMTEXT`. Migrating from `utf8mb3` to `utf8mb4` may cause indexes on string columns to exceed the maximum number of bytes per entry (the maximum is dependent on the row format). If this happens, an index prefix length can be added or the existing prefix length can be reduced to bring the number of bytes under the limit. For more details, see the [MySQL docs for utf8mb3 to utf8mb4 conversion](https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-conversion.html). ##### Padding Legacy MySQL collations ignore trailing whitespace. For example `SELECT * FROM t WHERE a = 'a'` and `SELECT * FROM t where a = 'a '` (note trailing whitespace) are functionally identical if column `a` has a legacy MySQL character set. Usually trailing space-aware comparisons are more in line with developer expectations, but it is important to verify that this change in behavior won't adversely affect your application. For more details, wee the [MySQL docs for collation padding](https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-sets.html#charset-unicode-sets-pad-attributes) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # MySQL compatibility Source: https://planetscale.com/docs/vitess/mysql-compatibility PlanetScale Vitess is built on open-source Vitess, a database clustering system for horizontal scaling of MySQL. ## Overview PlanetScale databases run on MySQL 8. If you're [importing an existing database](/docs/vitess/imports/database-imports), PlanetScale supports MySQL database versions `5.7` through `8.0`. New PlanetScale databases are created on MySQL 8 with character set `utf8mb4_0900_ai_ci`. PlanetScale supports `utf8`, `utf8mb4`, and `utf8mb3`, character sets. We also support `latin1` and `ascii` character sets, but do not recommend them. ## MySQL compatibility limitations The following reference guide will cover some MySQL syntax, features, and more that PlanetScale either does not support or has limitations around. We are actively working on driving up compatibility, but it's an ongoing effort and will take some time to complete. See this [project board on GitHub](https://github.com/vitessio/vitess/projects/4) to learn what the Vitess team is currently focusing on. If you're importing a database, there are some additional requirements that you can find in our [Database imports documentation](/docs/vitess/imports/database-imports). ### Queries, functions, syntax, data types, and SQL modes > [!NOTE] > > \= *Limitations in support* > > \= *Not supported* | Statement | Support | Description | | - | - | - | | `ALTER TABLE...RENAME COLUMN` | | Renaming columns and tables may be destructive. See our [guide for column rename recommendations](/docs/vitess/schema-changes/handling-table-and-column-renames). | | `CREATE DATABASE` | | You cannot `CREATE` a logical database within a PlanetScale Vitess database. | | `DROP DATABASE` | | You cannot `DROP` a logical database within a PlanetScale Vitess database. | | `JSON_TABLE` | | The [`JSON_TABLE` function](https://dev.mysql.com/doc/refman/8.0/en/json-table-functions.html#function_json-table) is not yet supported. All other [JSON SQL functions](https://dev.mysql.com/doc/refman/8.0/en/json-function-reference.html) should work. | | `PROCEDURE` | | We do not support any form of [stored routines](https://dev.mysql.com/doc/refman/8.0/en/stored-routines.html). | | `FUNCTION` | | We do not support any form of [stored routines](https://dev.mysql.com/doc/refman/8.0/en/stored-routines.html). | | `TRIGGER` | | We do not support any form of [stored routines](https://dev.mysql.com/doc/refman/8.0/en/stored-routines.html). | | `EVENT` | | We do not support any form of [stored routines](https://dev.mysql.com/doc/refman/8.0/en/stored-routines.html). | | `LOAD DATA INFILE` | | Loading data via [`LOAD DATA INFILE` is not supported](https://github.com/vitessio/vitess/issues/2976). | | `LOCK TABLES` | | [`LOCK TABLES`](https://dev.mysql.com/doc/refman/8.0/en/lock-tables.html) is not supported. | | `KILL` | | We do not support killing queries or shards from the command line. | | `:=` | | The `:=` assignment operator is not yet supported. | | `SET GLOBAL time_zone` | | The global time zone is set to UTC and can not be modified. | | `SET GLOBAL sql_mode` | | The global SQL mode can not be changed permanently. Set each new session's mode instead with `SET sql_mode`. | | `PIPES_AS_CONCAT` | | Enabling this SQL mode can interfere with Vitess' evalengine parsing the SQL queries so enabling it may result in incorrect or unexpected results. Please use MySQL's standard dialect instead, e.g. `CONCAT()`. | | `ANSI_QUOTES` | | Enabling this SQL mode can interfere with Vitess' evalengine parsing the SQL queries so enabling it may result in incorrect or unexpected results. Please use MySQL's standard quotation instead. | | `WITH RECURSIVE` | | Experimental support for recursive common table expressions (CTEs) was introduced in Vitess 21 for `SELECT` queries. | ## Miscellaneous | Action | Support | Description | | - | - | - | | **Empty schemas** | | Databases with empty schemas are invalid. You cannot deploy a schema change to production if no tables exist. | | **Non-InnoDB Storage engine** | | We only support [InnoDB](https://dev.mysql.com/doc/refman/8.0/en/innodb-storage-engine.html) storage engine. | | **No applicable unique key** | | We require all tables have a [unique, non-null key](/docs/vitess/schema-changes/onlineddl-change-unique-keys) and that respective covered columns are shared between old and new schema. | | **Direct DDL** | | We do [not allow Direct DDL](/docs/vitess/schema-changes/how-online-schema-change-tools-work) on production branches when [safe migrations](/docs/vitess/schema-changes/safe-migrations) is enabled. This includes `TRUNCATE` statements. | | **Binary log access** | | PlanetScale does not currently support binlog replication to external databases. | | **Large JSON documents** | | MySQL supports JSON documents up to 1 GB in size. However, we do not recommend to store more than a few MB in a JSON document for performance reasons. | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Online DDL change unique keys Source: https://planetscale.com/docs/vitess/onlineddl-change-unique-keys It is possible to modify or replace a table's `PRIMARY KEY`, or any other `UNIQUE KEY`s according to the limitation described below, followed by examples. ## Overview To migrate data safely and [without downtime](/docs/vitess/schema-changes), PlanetScale requires that all tables have a unique, not-null key. Note that a `PRIMARY KEY` satisfies this condition, and it is generally recommended to always have a `PRIMARY KEY` on all tables. When you modify a table, both the old and the new schema must have a unique key as described, and the columns covered by those keys must exist in both the old and the new schema. Essentially this makes it possible for PlanetScale to unambiguously identify and correlate a row between the two schemas. If you attempt to deploy a schema change which does not comply with the above restriction, the deploy request will fail with the error `Table ... has no shared columns covered by non-null unique keys between both branches.`. ## Examples: allowed changes In our examples, we assume the base schema to be: ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`id`) ); ``` The following are all valid changes to the schema: ### Expanding the PRIMARY KEY ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`id`, `other_info`) ); ``` In the above we modified the `PRIMARY KEY` to include `other_info`. This is allowed since both `id` and `other_info` columns exist in both the old and the new schema. ### Moving PRIMARY KEY to a different column ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`email`) ); ``` Since both `id` and `email` columns exist in both old and new schema, the deploy request will be allowed. The success of the operation depends on whether `email` actually contains unique values. If there's duplication in `email` values, the deployment will fail with error. ### Moving PRIMARY KEY to different columns Likewise, there is no problem if the new `PRIMARY KEY` covers multiple columns. Again, the success of the operation depends on the actual uniqueness of the combination of columns. ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`username`, `other_info`) ); ``` ### Changing PRIMARY KEY and adding/removing other UNIQUE KEYs ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`username`, `other_info`), UNIQUE KEY `email` (`email`) ); ``` ## Examples: invalid changes Consider the next scenarios and the ways to work around them: ### Changing a PRIMARY KEY to include a new column ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), `new_info` int, PRIMARY KEY (`username`, `new_info`) ); ``` This is an invalid change because in the new schema, the `PRIMARY KEY` covers the `new_info` column. But this column does not exist in the old schema. Consider splitting into two distinct schema changes and deploy requests: 1. First, introduce the `new_info` column. 2. Next, change the `PRIMARY KEY`. ### Changing a PRIMARY KEY and also dropping the old covered column ```sql theme={null} CREATE TABLE `users` ( `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`email`) ); ``` The above is invalid because `id` column, covered by the `PRIMARY KEY` in the old schema, does not exist in the new schema. Again, consider splitting into two distinct changes: 1. First, change the `PRIMARY KEY`. 2. Next, drop the `id` column. ## Summary We've seen how, in many scenarios, it's straightforward to modify your table's `PRIMARY KEY` or other keys. For some scenarios, it might take two or more steps to achieve the new schema. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Operating without foreign key constraints Source: https://planetscale.com/docs/vitess/operating-without-foreign-key-constraints A **foreign key** is a logical association of rows between two tables, in a parent-child relationship. ## Overview A row in a "parent" table may be referenced by one or more rows in a "child" table. A foreign key typically suggests how you should `JOIN` tables in most queries. A table can also refer to itself, as a special case. A **`FOREIGN KEY` *constraint* is a database construct**, an implementation that *forces* the foreign key relationship's integrity (referential integrity). Namely, it ensures that a child table can only reference a parent table when the appropriate row *exists* in the parent table. A constraint also prevents the existence of "orphaned rows" in different methods, as you'll see described soon. At PlanetScale, we don't *recommend* using foreign key constraints. However, if you still want to use them, you can enable [support for foreign key constraints](/docs/vitess/foreign-key-constraints) in your database settings page. We still encourage you to use the relational model and associate tables by "pointing" rows from one table to another with foreign keys, just not with the `CONSTRAINT ... FOREIGN KEY` definition. We'll soon cover an example of a schema with and without foreign key constraints to clarify this slight difference. ## Why does PlanetScale not recommend constraints? There are a few significant technical reasons why we do not recommend foreign key constraints for some applications: * MySQL foreign keys introduce increased locking for data and metadata changes. Expect degraded performance, especially in high concurrency workloads. * Once two tables engage in a foreign key constraint relationship, it is not possible to change the data types for the columns used by the foreign key. For example, it is impossible to change a column from `INT` to `BIGINT`, neither on a parent nor child table. * As applications evolve, there is a need for schema refactoring. It is a more complex process involving more steps to refactor a schema containing foreign key constraints. * `FOREIGN KEY` constraints are difficult to maintain once your data grows and is split over multiple database servers. This typically happens when you introduce functional partitioning/sharding and/or horizontal sharding. > [!NOTE] > > You can still think in terms of foreign key relationships; of parent tables and child tables; of rows referencing each > other. You can structure your tables in the exact same way without FOREIGN KEY constraints as you would with the > constraints. It's just how these relationships are enforced that changes. To understand how to work without foreign key constraints, we must first understand foreign key functionality. ## Types and behavior of FOREIGN KEY constraints Consider the following trivial parent-child table relationship: ```sql theme={null} CREATE TABLE parent_table ( id INT NOT NULL, PRIMARY KEY (id) ); CREATE TABLE child_table ( id INT NOT NULL, parent_id INT, PRIMARY KEY (id), KEY parent_id_idx (parent_id), CONSTRAINT `child_parent_fk` FOREIGN KEY (parent_id) REFERENCES parent_table(id) ON DELETE NO ACTION ); ``` > [!TIP] > > In MySQL, foreign keys are only implemented by the storage engine layer, namely the default and popular InnoDB engine. > A FOREIGN KEY constraint isn't a separate entity. It lives within the child's table space. `FOREIGN KEY` constraints are available for row deletion (`ON DELETE`) and row updates (`ON UPDATE`). In this document, we discuss `ON DELETE` as it is the more impactful of the two. The discussion is relevant to `ON UPDATE` constraints, as well. Foreign keys further support three types of action (illustrated below for `ON DELETE`): ### `ON DELETE CASCADE` This is the most greedy, or ambitious action type. If you `DELETE` a row from a parent table, any referencing rows in a child table are subsequently deleted within the same transaction. This operation runs recursively for all of a parent's children, as well as for their children, should they also employ `ON DELETE CASCADE`. `ON DELETE CASCADE` is a risky and resource-consuming action. You intend to `DELETE` a single row, but end up deleting hundreds, thousands, or more, rows in multiple tables. What seemed like a simple transaction now turns into a massive operation, that involves excessive locking, increased MVCC overhead, impact on replication lag, and more. But perhaps the greatest danger is the potential unexpected loss of data. Whether an unsuspecting developer simply assumes a `DELETE FROM parent_table WHERE id=3` will at most delete one row, down to surprising behavior such as in `REPLACE INTO` queries, which actually run an implicit `DELETE`, leading to mass destruction of data. The use of `ON DELETE CASCADE` is controversial. Use it with great care. Consider using `NO ACTION` instead. ### `ON DELETE SET NULL` With this setup, a `DELETE` on a parent (e.g. `DELETE FROM parent_table WHERE id=3`) will set the referencing column on children (e.g. `parent_id` column in `child_table`) to `NULL` for matching rows. It effectively leads to orphaned rows, not very differently from having no foreign key constraints at all. One advantage is that it's easy to identify the orphan rows: those, and only those rows, will have `NULL` for parent-referencing columns. Like `CASCADE`, a single row deletion on the parent may lead to multiple rows updated on child tables. This again may cause large transactions, excessive locking, and replication lag. ### `ON DELETE NO ACTION` Possibly the single most important feature of foreign keys, a `DELETE` on a parent will fail if child rows exist that reference the parent's row. To `DELETE` a row from a parent, the app/user must first `DELETE` the referencing rows from all children. Recursively, if those are further referenced by other tables, the app/user must first `DELETE` rows from grandchildren, and so forth. This action (or lack thereof) type forces the app to have stronger ownership of its data. An app written to work with `ON DELETE NO ACTION` will organically evolve *knowing* which tables reference which other tables, and will have established `DELETE`/`UPDATE` flows that iterate through tables in the correct order to satisfy referential integrity. ## How does your schema look without FOREIGN KEY constraints? The above schema would look exactly the same, minus the `CONSTRAINT` clause: ```sql theme={null} CREATE TABLE parent_table ( id INT NOT NULL, PRIMARY KEY (id) ); CREATE TABLE child_table ( id INT NOT NULL, parent_id INT, PRIMARY KEY (id), KEY parent_id_idx (parent_id) ); ``` > [!TIP] > > Each FOREIGN KEY constraint requires an index covering the referenced column(s) on both sides of the connection. The index `_parent_id_idx` is **required** by the constraint. We can drop that key in our constraint-free table, depending on the type of queries we might use to retrieve data from the tables. ## Developing an app with no FOREIGN KEY constraints Consider an app's behavior in an `ON DELETE NO ACTION` setup again: the app *knows* the table's dependencies, handles iteration order for `DELETE`/`UPDATE`. It does everything right. What happens if we take away the `CONSTRAINT` at this point? The app remains unchanged. It already runs the proper actions in the proper order. The app operation that ends up running the `DELETE FROM parent_table WHERE id=3` succeeds the app operations that `DELETE` the child table(s). While we lose the database's safety guarantee enforcing this behavior, we are perfectly capable of managing it on our own. Likewise, an app that grows with a constraint-less schema organically learns to handle `DELETE` and `UPDATE` scenarios. It is in fact given some extra freedom because the *order* of operations is not enforced. This is an advantage because the app is not forced to `DELETE` thousands of dependent rows for each parent row deletion *at that same transaction*. The app may well postpone deletion as we discuss shortly. Referential integrity is but one of many logical data integrity constraints. It just happens to be one that databases can enforce. Any sizeable application will maintain the integrity of its data with rules the database is unaware of. ## Cleaning up orphaned rows Consider an `ON DELETE SET NULL` constraint. What happens to the child rows that were set to `NULL`? Typically, there is little or no damage in keeping them around. A query can, for example, `SELECT` rows `WHERE parent_id IS NOT NULL`. Better yet, child rows are often `JOIN`ed with their respective parent rows. Any query running such a `JOIN` will return empty since the parent row does not exist. Eventually, those rows will pile up, and you will want to reclaim the space. Without `FOREIGN KEY` constraints, the situation is very much the same, It is possible to `DELETE` a parent row without deleting its dependent children rows. A child row `JOIN`ed with a respective (deleted) parent row comes out empty. There is no `IS NOT NULL` to help you, but identifying those rows is still trivial. Similarly, there is little or no damage in keeping those rows around for a while. And similarly, those rows eventually pile up. A common practice, where appropriate, is to `DELETE` rows on a parent table, or perhaps also on a subset of children, but leave some other tables for offline batch processing. At some convenient time, such as low traffic hours, the app or some batch job will purge orphaned rows. Consider this simplified query: ```sql theme={null} DELETE FROM child_table LEFT JOIN parent_table ON (child_table.parent_id=parent_table.id) WHERE parent_table.id IS NULL ``` A single `DELETE` is likely a massive operation, which is to be avoided. A good practice is to break the statement into multiple small-scope statements, e.g. deleting `100` rows at a time. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Pricing Source: https://planetscale.com/docs/vitess/pricing A single Vitess database cluster includes the resources equivalent to 5 always-on instances within the base monthly plan cost. Vitess clusters are charged based on the following: * Cluster size and [region](#region-pricing) * Storage size (for network-attached storage clusters) * [Branch](/docs/vitess/schema-changes/branching) hours beyond the included 1440 hours * Number of [replicas](/docs/vitess/scaling/replicas) * Number and size of [VTGates](/docs/vitess/scaling/vtgates) * [Read-only regions](/docs/vitess/scaling/read-only-regions) ## Query live prices with SQL Cluster SKU prices are available in a public, read-only Postgres table: `public.planetscale_prices`. You and your agent can query live prices with `SELECT`. The role can only `SELECT` on that table. SSL is required. Connect through PgBouncer: ``` postgresql://pscale_api_hhdcchpiodja.dzken160d27q:pscale_pw_wa09rWwuQZQEWGS2eL0v7BlSGriZ6xgv@aws-us-east-1-3.pg.psdb.cloud:6432/postgres?sslmode=verify-full&sslrootcert=system ``` Each row is one priced cluster. Filter on `product` (`Postgres` or `Vitess`), `provider`, `region`, `machine_sku`, `replication_factor`, and `disk_sku` (empty string for `PS-*` network-attached SKUs). `price` is the monthly cluster cost in whole USD. It does not include extra storage, backups, egress, dedicated PgBouncers, or extra replicas. ```sql theme={null} -- Postgres, 3-node HA PS-160 ARM in us-east-1 SELECT machine_sku, replication_factor, price FROM planetscale_prices WHERE product = 'Postgres' AND machine_sku = 'PS-160-ARM' AND provider = 'AWS' AND region = 'us-east-1' AND disk_sku = '' AND replication_factor = 3; ``` ```sql theme={null} -- Vitess, 3-node HA PS-160 in us-east-1 SELECT machine_sku, replication_factor, price FROM planetscale_prices WHERE product = 'Vitess' AND machine_sku = 'PS-160' AND provider = 'AWS' AND region = 'us-east-1' AND disk_sku = '' AND replication_factor = 3; ``` > [!NOTE] > > Not every SKU exists in every region. If a query returns no rows, that configuration is not sold there. Do not invent a price. Vitess SKUs in this catalog are mostly x86; there is no `PS-160-ARM` Vitess row. Column meanings, more example queries, and an agent-friendly catalog are in [pricing.md](https://planetscale.com/pricing.md). The connection URI is also on the [pricing page](https://planetscale.com/pricing#pricing-database). ## Branches Each database includes one production branch that provides a primary instance and two replica instances spread across availability zones. The primary serves all queries by default, but the replicas can be used to serve read traffic, and are also used to maintain high-availability of your cluster. In addition to the three instances used by your production branch, \~1440 hours of [development branch](/docs/planetscale-plans#development-branches) time per month is included. The development branch time works out to two "always on" database instances, though we do recommend spinning them up and down and as needed during your development cycle. These development branches can replace your existing development or staging environment on other providers. Instead of purchasing 2-3 databases on PlanetScale to recreate your environment, you can just purchase one database cluster and use [branching](/docs/vitess/schema-changes/branching) for development. You can upsize and downsize your cluster at any time. Pricing is prorated to the millisecond, so if you temporarily upsize, you will only be charged for the larger cluster size for the time that it was running. Billing for a new cluster size begins as soon as you begin the resize. You can also spin up additional production branches at any timing for additional cost. The pricing for these is also prorated. ## Read-only regions [Read-only regions](/docs/vitess/scaling/read-only-regions) are an additional cost. Each read-only region is configured per keyspace with its own cluster size and replica count: the read-only region rate for the selected cluster size covers a single replica, and each additional replica is billed at the standard additional replica rate. Storage for read-only regions is billed as standalone storage and does not count toward your plan's included storage. See the [read-only regions pricing documentation](/docs/vitess/scaling/read-only-regions#availability-and-pricing) for per-region rates and storage costs. > [!NOTE] > > Cluster size options are capped at `PS-160` until you have a successfully paid an invoice of at least \$100. If you need larger sizes immediately, please [contact us](https://planetscale.com/contact) to unlock all sizes. You can find the full list of cluster sizes in our [Plans documentation](/docs/planetscale-plans#base-plan). ## Region pricing ### Amazon Web Services > **[ap-northeast-1 (Tokyo)](https://planetscale.com/pricing?region=ap-northeast)** > **[ap-south-1 (Mumbai)](https://planetscale.com/pricing?region=ap-south)** > **[ap-southeast-1 (Singapore)](https://planetscale.com/pricing?region=ap-southeast)** > **[ap-southeast-2 (Sydney)](https://planetscale.com/pricing?region=aws-ap-southeast-2)** > **[eu-central-1 (Frankfurt)](https://planetscale.com/pricing?region=eu-central)** > **[ca-central-1 (Montreal)](https://planetscale.com/pricing?region=aws-ca-central-1)** > **[eu-west-1 (Dublin)](https://planetscale.com/pricing?region=eu-west)** > **[eu-west-2 (London)](https://planetscale.com/pricing?region=aws-eu-west-2)** > **[sa-east-1 (Sao Paulo)](https://planetscale.com/pricing?region=aws-sa-east-1)** > **[us-east-1 (N. Virginia)](https://planetscale.com/pricing?region=us-east)** > **[us-east-2 (Ohio)](https://planetscale.com/pricing?region=us-east-2)** > **[us-west-2 (Oregon)](https://planetscale.com/pricing?region=us-west)** ### Google Cloud > **[asia-northeast3 (Seoul, South Korea)](https://planetscale.com/pricing?region=gcp-asia-northeast3)** > **[europe-west4 (Eemshaven, Netherlands)](https://planetscale.com/pricing?region=gcp-europe-west4)** > **[northamerica-northeast1 (Montréal, Québec)](https://planetscale.com/pricing?region=gcp-northamerica-northeast1)** > **[us-central1 (Council Bluffs, Iowa)](https://planetscale.com/pricing?region=gcp-us-central1)** > **[us-east1 (Moncks Corner, South Carolina)](https://planetscale.com/pricing?region=gcp-us-east1)** > **[us-east4 (Ashburn, Virginia)](https://planetscale.com/pricing?region=gcp-us-east4)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Database replicas Source: https://planetscale.com/docs/vitess/replicas A replica is a continuously updated copy of your Vitess database. ## Overview Replicas serve two main purposes: * They provide an additional source for your data to be queried. * They increase database availability by enabling fast failovers for maintenance or unexpected failure. > [!WARNING] > > Before utilizing replicas for reducing load on the primary, it's important to understand the trade-offs. For more information, see the [Data consistency and replication lag](#data-consistency-and-replication-lag) section. For information on replicas running in additional regions, see: [Read-only regions](/docs/vitess/scaling/read-only-regions). ## How to query replicas PlanetScale replicas can be used to read data and reduce load on the primary. PlanetScale does not automatically route queries to replicas unless you explicitly use a replica credential or tell your application to do so. ### 1. Create a global replica credential (recommended) With global replica credentials, you can have one credential that will automatically route queries to your branch's replicas and read-only regions. If you do not have any read-only regions, it will route to the included replicas on [your plan](/docs/planetscale-plans). If you do have any read-only regions, the queries will route to the closest replica out of your read-only region replicas and included failover replicas. To use this, create a new credential in the PlanetScale dashboard and select "**Replica**" as the connection type. You can then use the new credential's connection string to setup a replica-only database connection. If you have `pscale` installed locally, you can create a replica credential with the following command: ```shell theme={null} pscale password create --replica ``` All queries made using this credential will be routed to the nearest replica, even as you add and remove read-only regions. To pin traffic to a specific read-only region instead, create a region-scoped password: ```shell theme={null} pscale password create --read-only-region ``` See [Read-only regions](/docs/vitess/scaling/read-only-regions#region-scoped-credentials) for details. ### 2. `USE @replica` (not recommended) > [!WARNING] > > We do not recommend using this method to query replicas. If you are currently using this method, we recommend switching to method 1. If you are not using a global replica credential, then, by default, all queries are served by the primary. However you may route any `SELECT` queries to replicas by issuing the following command on a connection: ```sql theme={null} USE @replica ``` Once this command is run, all subsequent `SELECT` queries on that connection will be served by your branch's replicas instead of the primary node in your cluster. Please note that this does not send queries to any read-only regions, if you want to route queries all replicas, including read-only regions, you should use a global replica credential (method 1 above). > [!TIP] > > For querying replicas in separate regions, see: [Read-only regions](/docs/vitess/scaling/read-only-regions). We highly recommend using a global replica credential to ensure that your queries are always routed to the nearest replica. ## High availability Replicas within PlanetScale are used to enable high availability of your database. This is a part of the reason all production branches in PlanetScale are provided at least one replica. In situations where the underlying hardware or service hosting the primary MySQL node fails, our system will automatically elect a new primary node from the available replicas and reroute traffic to that new primary. This process is know as **reparenting** and typically is all handled within milliseconds or seconds. Querying the primary during a reparent typically goes unnoticed, other than a bit of additional query latency. This is because Vitess [buffers queries](https://vitess.io/docs/reference/features/vtgate-buffering) during this time. You can incorporate retry logic into your application to handle the rare instances where an issue arises during reparenting. When querying a replica during a reparent, you may encounter this error: ``` no healthy tablet available for 'keyspace:"${keyspace}" shard:"${shard}" tablet_type:REPLICA' ``` Incorporating retry logic into your application can help with this scenario as well. ## Multiple availability zones In cloud architecture, regions are further broken down into data centers known as availability zones (or AZs for short). For example, the `us-east-1` region on AWS contains 6 default AZs available to customers starting with `us-east-1a` through `us-east-1f`. The infrastructure for all Base plan and Enterprise PlanetScale databases are distributed across 3 availability zones. In the instance of an AZ failure, your database will auto failover to an available AZ. ## Data consistency and replication lag Whenever data is updated (`INSERT`, `UPDATE`, `DELETE`) on the primary node, those changes are synchronized to the replicas shortly after. The delay between when a primary is updated and the changes are applied to the replica is known as `replication lag`. Your databases replication lag is viewable on your [database dashboard](/docs/vitess/architecture#replication-lag-at-a-glance). Replication lag can also be viewed on Datadog if you set up the [PlanetScale - Datadog integration](/docs/vitess/integrations/datadog). In Datadog, you can use the following formula to set alerts for replication lag: ``` (max:planetscale.replication_lag{ps_database: ps_tablet_type:replica, ps_branch:
}) ``` Make sure you replace `` with your PlanetScale database name and `
` with the name of the branch for which you'd like to track replication lag. It is important to be aware of replication lag whenever querying data from your replicas. For example, if you make an update and then immediately try to query for that updated data via a replica, it may not be available yet due to replication lag. ## When should you use replicas in your application? Replicas are useful for offloading read-heavy workloads from the primary node. By using replicas, you can distribute the read load across multiple nodes, which can help improve the performance of your application. Some examples of where you might want to query a replica are scheduled jobs, analytics jobs, search features, or aggregate queries. Replicas can also be used to provide a read-only view of your data to users or applications that do not need to write data, such as when a user is logged out or writing one-off queries for debugging purposes. ## When are replicas used in PlanetScale? We use replicas for every production database branch. The number of replicas for a given database depends on the selected plan for that database: * **Base plan:** Includes 2 replicas per production branch distributed across multiple AZs in a given region. * **Enterprise:** Customizable to fit the needs of your organization, and as such can have as many replicas as needed. [Read-only regions](/docs/vitess/scaling/read-only-regions) also run replicas, hosted in a different geographical region. Each read-only region has its own cluster size and replica count, configured per keyspace independently of the primary region. It is important to note that the MySQL nodes in read-only regions are replicas intended only for reading data and are not eligible for failover if the primary node experiences an outage. > [!WARNING] > > Development branches DO NOT have replicas as they are intended for development only and are not designed with the same resiliency as production branches. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster configuration Source: https://planetscale.com/docs/vitess/scaling/cluster-configuration The Clusters page in your PlanetScale dashboard allows you to configure your PlanetScale cluster. From here, you can: * Adjust the instance sizes for [keyspaces](/docs/vitess/sharding/keyspaces) * Enable [per-shard sizing](/docs/vitess/cluster-configuration/per-shard-sizing) on sharded keyspaces * Create sharded or unsharded keyspaces * Create [external keyspaces](/docs/vitess/cluster-configuration#create-an-external-keyspace) that connect a production branch to an existing MySQL database * Switch between [Metal](/docs/plans/planetscale-skus#metal) and [network-attached storage](/docs/plans/planetscale-skus#network-attached-storage) * Adjust the number of replicas for each keyspace * Add and configure [read-only regions](/docs/vitess/scaling/read-only-regions) for each keyspace * Adjust the VSchema * Adjust the number and size of [VTGates](/docs/vitess/scaling/vtgates) * Turn on [VTGate autoscaling](/docs/vitess/scaling/vtgates#autoscaling-vtgates) * View the CPU and memory utilization of your VTGates * View any changes to your VTGates * Adjust the [memory allocated to vector indexes](/docs/vitess/vectors#resource-requirements) and the InnoDB buffer pool, for any branch with [vectors](/docs/vitess/vectors) enabled * Configure [VReplication settings](#vreplication-settings) * Configure [replication durability constraints](#replication-durability-constraints) * Enable or disable the [tablet throttler](#throttler) and set its replication lag threshold * Tune [VTGate, VTTablet, and MySQL parameters](/docs/vitess/cluster-configuration/parameters) This documentation will cover how to use everything on this Clusters page. For a full walkthrough with an example of setting up a sharded keyspace, refer to the [Sharding quickstart](/docs/vitess/sharding/sharding-quickstart). If you would like additional support configuring your sharded keyspaces, our [Enterprise plan](/docs/planetscale-plans#enterprise-plan) may be a good fit. [Get in touch](https://planetscale.com/contact) for a quick assessment. ## Adjust your cluster size To adjust your cluster size, first select the [keyspace](/docs/vitess/sharding/keyspaces) you'd like to adjust. If you only have one keyspace, it will be selected by default. Click on the "Cluster size" dropdown. Select the new cluster size, ([Metal](/docs/metal) or network-attached storage). If you select Metal, make sure you select the correct storage size. Finally. click "Save". A network-attached storage cluster (Amazon Elastic Block Storage or Google Persistent Disk) may take a few minutes to change sizes. And a Metal cluster can take hours, depending on the size. You can check the status of the resize from the [database homepage](/docs/vitess/architecture#resizing) in the dashboard, [CLI](/docs/cli/keyspace), or [API](/docs/api/reference/get_keyspace_rollout_status). There will be no downtime or locking during this process. For more information about selecting a cluster size, see the [Cluster sizing documentation](/docs/plans/cluster-sizing). ## Per-shard sizing On a sharded keyspace, enable [per-shard sizing](/docs/vitess/cluster-configuration/per-shard-sizing) from the Shards tab, then set a cluster size on individual shards. Shards you don't set keep the keyspace default. ## Configure read-only regions Each keyspace has a "Read-only regions" section where you can add [read-only regions](/docs/vitess/scaling/read-only-regions) and choose the cluster size and number of replicas for each region. The monthly cost of each region is shown as you configure it. See the [Read-only regions documentation](/docs/vitess/scaling/read-only-regions) for more information. ## Create a keyspace > [!WARNING] > > Misconfiguration to keyspaces can cause availability issues. We recommend thoroughly reading through the documentation > in the [Sharding section](/docs/vitess/sharding) of the docs prior to adding additional keyspaces. If you have any > questions, please [reach out to our support team](https://planetscale.com/contact?initial=support). > [!WARNING] > > If you are using [Vitess global routing](https://vitess.io/docs/reference/features/global-routing/), you must take extra care when adding a sharded keyspace to your database (for example, if you are using `@primary`). > Before creating one, you must ensure that all tables from your first *unsharded keyspace* are added to the `VSchema` of that *unsharded keyspace*. Eg: > > ``` > { > "tables": { > "users": { } > ... > } > } > ``` > > Otherwise, queries will fail. [Learn more about VSchema.](/docs/vitess/sharding/vschema) > [!NOTE] > > Sharded keyspaces are not currently supported on databases that have foreign key constraints enabled. To create a new [keyspace](/docs/vitess/sharding/keyspaces): **Step 1** Select the database you want to configure. **Step 2** Click "Clusters" in the left nav. **Step 3** You should see the existing unsharded keyspace for your database here. **Step 4** Click "New keyspace". **Step 5** Enter the keyspace name. For example, if your existing unsharded keyspace is named `metal`, you may create a sharded keyspace named `metal-sharded`. **Step 6** Select whether you want to keep it unsharded, or, if not, select the number of shards you want to exist in this keyspace. In most cases, you will be adding a new sharded keyspace. Adding a new unsharded keyspace is not a common use case, but is an option if you're looking to do [vertical sharding](https://planetscale.com/learn/courses/database-scaling/sharding/vertical-sharding). > [!NOTE] > > The cost of adding this additional keyspace largely depends on the number of shards you choose, the cluster size, and if you'd like to add additional replicas. **Step 1** Choose the cluster sizes you would like to use for this keyspace. Keep in mind, if you are creating a sharded keyspace, this will spin up multiple clusters of the selected size. For example, if you are creating 4 shards and choose the `PS-80` cluster size, we will create 4 `PS-80`s, each with 1 primary and 2 replicas. **Step 2** Select the number of *additional* replicas, if any, that you'd like to add to each cluster. Each cluster comes with 2 replicas by default, so any number you choose will be in addition to those 2. **Step 3** Review the new monthly cost for this keyspace below. This is in addition to your existing unsharded keyspace, as well as any other keyspaces you add. **Step 4** Once satisfied, click "Create keyspace". To attach an existing MySQL database as a keyspace instead of creating an internal one, see [Create an external keyspace](/docs/vitess/cluster-configuration#create-an-external-keyspace). ## Modify the VSchema of a keyspace via Clusters page > [!NOTE] > > You can modify the VSchema on your development branch either on the Clusters page , using the [`ALTER VSCHEMA` command](/docs/vitess/sharding/vschema#modifying-vschema), or with the pscale CLI using [`pscale keyspace vschema update`](/docs/cli/keyspace). Once you have created your keyspace, you will see a new tab: **VSchema**. The VSchema contains information about how the keyspace is sharded, sequence tables, and other Vitess schema information. The VSchema tab allows you to configure the Vschema for your new keyspace or modify it for existing keyspaces. We do not recommend modifying the VSchema directly on your production branch. In fact, it is not possible to do if you have [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled (as recommended). Instead, to modify the VSchema, you should first [create a new development branch](/docs/vitess/schema-changes/branching). Once you have your branch ready, follow these steps: **Step 1** To update the VSchema on the Clusters page, select your new development branch from the dropdown at the top, and then select the keyspace below that has the VSchema you'd like to modify. **Step 2** Next, click the tab labeled "VSchema". **Step 3** Modify the VSchema configuration JSON file as needed. Refer to the [VSchema documentation](/docs/vitess/sharding/vschema) for more information about the available options. **Step 4** When finished, click "Save changes". We will validate your VSchema, and if it is valid, the changes will be saved. If there are errors, we will warn you here to change them before saving. **Step 5** Go back to your "Branches" tab and click on the development branch that you modified. You should see a note on the right that says "Updated VSchema configuration" which lets you know the VSchema(s) for this branch has been modified. **Step 6** From here, go through the normal [deploy request process](/docs/vitess/schema-changes/deploy-requests) to deploy this change to production. Once your change is deployed to production, you can come back to the Clusters page, switch to your production branch, and view the updates to your VSchema. You can also click the "Changes" tab to see information, such as the resize event, status, and start/end time for any previous changes to the VSchema. ## Modify routing rules This configuration setting is currently only available for some Enterprise customers. To modify your routing rules, click "Manage routing rules" on the bottom left of the keyspace configuration panel. Again, you will need to create a new branch to modify routing rules, as described in the "Modify the VSchema of a keyspace" section above. ## Modify keyspace settings There are a number of keyspace-specific settings you can use to customize keyspace behavior. ### Replication durability constraints By default, replication to replica and read-only VTTablets is configured to maximize safety and data integrity. This behavior can be relaxed for performance improvements and reduced replication lag. * **Maximum** — Default setting; use maximum durability constraints. * **Dynamic** — Use maximum durability constraints when replication lag is under 5s, and automatically relax durability constraints when replication lag exceeds 5s. Durability constraints are automatically set back to maximum when replication lag reduces to under 5s. * **Minimum** — Reduce durability constraints. Optimizes for replica and read-only performance, but has highest risk of data loss on crashed instances. ### VReplication settings These settings improve performance during [VReplication](https://vitess.io/docs/reference/vreplication/vreplication/) processes like deploy requests and workflows. * **Optimize inserts** — Enabled by default. When enabled, during binlog replication catch-up, skip sending insert events for rows that have yet to be copied. For more technical details, see the corresponding [Vitess implementation](https://github.com/vitessio/vitess/pull/7708). * **Allow NOBLOB binlog row image** — Enabled by default. When enabled, then we support enabling MySQL’s NOBLOB binlog mode, to omit unchanged BLOB and TEXT columns from replication events, reducing binlog size. For more technical details, see the corresponding [Vitess Implementation](https://github.com/vitessio/vitess/pull/14502). * **Batch binlog statements** — Disabled by default. If enabled, batches binlog statements and transactions to limit the number of round-trips to MySQL. For more technical details, see the corresponding [Vitess implementation](https://github.com/vitessio/vitess/pull/14502). ### Throttler The [Vitess tablet throttler](https://vitess.io/docs/reference/features/tablet-throttler/) pauses schema migrations and VReplication workflows when replication lag rises above a threshold. You can enable or disable it per keyspace, and set the lag threshold, from the same keyspace settings form. * **Enable** — On by default. When enabled, migrations and workflows pause while replication lag is above the threshold. This setting applies across all shards in the keyspace. * **Replication lag threshold** — Seconds of lag that trip the throttler. The default is 5 seconds. You can also change these settings from the CLI with `pscale keyspace update-settings`. See the [keyspace CLI docs](/docs/cli/keyspace). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Cluster sizing Source: https://planetscale.com/docs/vitess/scaling/cluster-sizing You can easily upsize and downsize your database cluster from within the PlanetScale dashboard. This documentation covers some information about selecting a cluster size upon database creation as well as how to upsize and downsize. > [!NOTE] > > If you are on a consumption commitment plan, please be aware that any changes in cluster size will be reflected against your monthly or annual consumption commitment amount. Changes to the originally selected cluster size may cause you to utilize this amount either more quickly or slowly. If you have further questions, please reach out to your account manager or our [Support](https://planetscale.com/contact) team. [Resize your database](https://www.youtube-nocookie.com/embed/y94VExata3A?rel=0) ## Selecting a cluster size Selecting the correct cluster size for your database can have a dramatic impact on how it performs and how much it costs. A good rule of thumb is when you notice CPU usage is consistently at or close to 100% for an extended period of time, you may benefit from [upsizing your cluster](/docs/plans/planetscale-skus/#upsizing-and-downsizing-clusters). Conversely, if your CPU usage is consistently below 50%, you may be able to downsize. You can monitor your CPU usage by clicking on your database, clicking "Primary" in your architecture diagram, and referencing the chart under "Metrics and performance". > [!WARNING] > > For Metal instances, you have to consider both the compute and the storage, as storage does not autoscale. For more information about adjusting a Metal instance, see [Upgrading an existing database to Metal](/docs/metal/create-a-metal-database#upgrading-an-existing-database-to-metal). There are also special cases where you may want to temporarily upsize out of caution if you're anticipating a large spike in traffic, such as during a launch or event. In these cases, you can easily [upsize](/docs/plans/planetscale-skus#upsizing-and-downsizing-clusters) ahead of your event, and then downsize after. Changing cluster sizes is a seamless operation that requires no downtime. If you are switching between [network-attached storage](/docs/plans/planetscale-skus#network-attached-storage) (Amazon Elastic Block Storage or Google Persistent Disk) and [Metal](/docs/plans/planetscale-skus#metal), or changing the size of your Metal instance, be aware this switch takes additional time. Again, no downtime is required. ### Comparing PlanetScale to other database providers If you are migrating from an existing cloud provider with resource-based pricing, be sure to compare your currently selected instance with our available cluster sizes. Keep in mind, each database comes with a production branch with two replicas, as well as 1,440 hours worth of development branches. The development branches essentially equate to two extra "always on" databases. In many cases, you can deprecate your dev/staging databases that you pay extra for with other providers in favor of the development branches. In the end, this usually results in significant cost savings. Databases in PlanetScale also come with additional beneficial infrastructure that is not easily configured or available in other hosted database solutions. For more information on what is provisioned with each database, read our [Architecture](/docs/vitess/architecture) doc. If you are unsure which plan or cluster size is right for your application, [contact us](https://planetscale.com/contact) to get further assistance. Our self-serve plans are flexible enough to handle the majority of customers. However, there are several use cases where you may need a more custom plan. This is where our Enterprise offerings shine. ### Upsizing and downsizing clusters As your application scales, upgrading or downgrading your database cluster is a seamless operation that does not involve any downtime. To change cluster sizes, go to your PlanetScale dashboard, click on your database, click ["Clusters"](/docs/vitess/cluster-configuration), select the new cluster size for the [keyspace](/docs/vitess/sharding/keyspaces) you wish to configure, and click "Update". On a sharded keyspace, you can also enable [per-shard sizing](/docs/vitess/cluster-configuration/per-shard-sizing) and set a cluster size on individual shards. > [!NOTE] > > If you have [maintenance schedules](/docs/plans/managed/maintenance-schedules) enabled, changes to cluster size will roll out during your scheduled maintenance window. The time it takes to change sizes depends on the size and region of your database. Larger databases may take 20 minutes to upsize/downsize. However, this is all done online, so you will not experience any downtime. Keep in mind, once you update your cluster size, you cannot change sizes again until the first size change completes. When you choose to change cluster size on network-attached storage drives, we upgrade each of your replicas one by one: delete the tablet container, create a new tablet container of the new size, attach the persistent volume, start it up, and connect it to the primary. Once that's complete, we fail the primary over to one of those new replicas, and do the same thing to the old primary. On Metal, this works a little differently. Instead of upgrading replicas one by one, we spin up 3 new Metal instances in addition to your existing replicas. We catch the three new instances up by restoring a backup and then catching them up via MySQL replication. When those replicas are fully caught up, we reparent the primary to one of them, and begin to remove the old tablets. For this reason, you may notice a minimum of 5 replicas as you're upgrading your Metal instance size. ## Sharding You can create sharded keyspaces on any plan by adding a new sharded keyspace using the [Clusters page](/docs/vitess/cluster-configuration) and moving tables into it with [MoveTables](/docs/vitess/sharding/sharding-quickstart). If you would like additional support from our expert team, our [Enterprise plan](/docs/planetscale-plans#enterprise-plan) may be a good fit. [Get in touch](https://planetscale.com/contact) for a quick assessment. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vitess system limits Source: https://planetscale.com/docs/vitess/scaling/planetscale-system-limits > [!NOTE] > > We can sometimes manually adjust limits on a per-database level. If you are facing issues or have questions, the best course of action is to [open a support ticket](https://planetscale.com/contact). > > Additionally, you can find some solutions to specific error codes in the [Errors documentation](/docs/vitess/troubleshooting/errors). ## Table limits Database schemas are limited to a total of `2048` tables, including views. Individual tables are limited to a maximum of `1017` columns each. ## Development branch limits Each database allows `100` development branches by default. Change the limit on the database **Settings** page. The limit can be any number from 1 to 5,000. When a database is at its limit, delete development branches you no longer need, or raise the limit, before creating another one. For a limit above 5,000, [contact support](https://planetscale.com/contact). ## Disk limits For [**network-attached storage**](/docs/plans/planetscale-skus#network-attached-storage) databases, the disk size will auto-scale up to a maximum of 4 TB. For [**Metal** databases](/docs/plans/planetscale-skus#metal), the disk size is dictated by what you choose when the database is created. Disk sizes range from 110 GB to 7.4 TB. Which sizes are available depends on the compute instance chosen. This space is used both for user data as well as system data required to run database operations. If your database is near this limit, some deploy request operations may not be available as they require adequate disk space to complete. When near this limit, we recommend either [sharding your data](/docs/vitess/sharding) or distributing tables across multiple keyspaces. For Metal databases, you can try upgrading to a larger disk size. Please [contact us](https://planetscale.com/contact) to discuss options. ## Connection lifetime limits Database client connections that are held open longer than 24 hours may be terminated unexpectedly. We recommend that long-running database connections are closed and reconnected at least once per day. ## Simultaneous transaction limits Each database has a limit on the number of simultaneous *transactions* it can process, also known as the *transaction pool*. If you exceed the *transaction pool* setting for your database, you may encounter this error: ``` vttablet: rpc error: code = ResourceExhausted desc = transaction pool connection limit exceeded ``` This can often be mitigated by trying one of the following solutions: A) Reduce the amount of parallelism in the requests being sent to the database. \ B) Shorten lengthy transactions by reducing batch sizes or making some other application-level adjustment. If you cannot make such changes to your application, consider choosing a larger instance type with a larger transaction pool. The exact limit varies depending on the instance type of your database. For details, see the [plans page](/docs/plans/planetscale-skus). ## Query limits PlanetScale has enforced some system limits to prevent long-running queries or transactions from: * Piling up and consuming all available resources. * Blocking other important, short-lived queries and transactions from completing. * Overloading the database to the point where it is not recoverable without a restart. * Blocking planned failovers and critical upgrades. The following table details these limits: | Type | Limit | | :- | :- | | Per-query rows returned, updated, or deleted | 100k | | Per-query result set total size | 64 MiB | | Per-query autocommit timeout | 900s | | Per-transaction timeout | 20s | ### Recommendations for handling query limits These limits are enforced for the safety of your database. However, we do understand you may run into a situation where the limits are a blocker. Here are some best practices for solving common issues presented by the limits: **What should I do if I have a query that returns more than 100,000 records?** We recommend trying to break up large queries, e.g. through [pagination](https://planetscale.com/blog/mysql-pagination). **What should I do if I have a query that returns more than 64 MiB of data?** If your schema currently relies on storing large amounts of variable length data within `JSON`, `BLOB`, or `TEXT` type columns that is regularly over a few MiB in size, you will want to strongly consider storing that variable length data outside of your database, such as within an object storage solution, instead. If large values are stored within variable length data columns, it can limit the number of rows you can return. For example, above we described a 100K limit for result sets, but if your result set's size exceeds the 64 MiB limit then you may receive an error message like the following: `resource_exhausted: grpc: received message larger than max ( vs. 67108864)` when retrieving much less than 100K rows. Similarly, when generating a large `INSERT` or `UPDATE` query for these types of columns you may run into the `grpc: trying to send message larger than max ( vs. 67108864)` error message which would require you to reduce the query's size. **What should I do if I have a transaction that runs longer than 20 seconds?** For transactions that are running for longer than 20 seconds, we recommend breaking these up into multiple shorter transactions. For analytics queries that are not possible to break up, see "How should I handle analytical queries?" below. Keep your transactions small and short-lived. For transactional workloads that are inherently long-lived, consider replacing or complementing MySQL transactions with other techniques: * For simple workloads (e.g. a `SELECT`, followed by an HTTP request, followed by an `UPDATE`), consider using optimistic locking instead of transactions. * For complex workloads, consider using sagas to coordinate a series of small, short-lived transactions that possibly span multiple microservices. **What should I do if I have a select query that needs to timeout faster than 900 seconds?** For select statements only, you can provide an inline comment to kill the query if it has not returned results within the specified time in milliseconds. `select /*vt+ QUERY_TIMEOUT_MS=2000 */ sleep(3);` Please note, you can set this value shorter than 900 seconds but not greater than the 900 seconds default. **How should I handle analytical queries?** We recommend using [PlanetScale Connect](https://planetscale.com/blog/extract-load-and-transform-your-data-with-planetscale-connect) (via [Airbyte](/docs/vitess/integrations/airbyte) or [Stitch](/docs/vitess/integrations/stitch)) to extract your data to any compatible destination (e.g. BigQuery, Redshift, etc.). The analytics queries should then be performed at those destinations. ### OLAP mode While it is possible to bypass these safety limits using `OLAP` mode (`SET workload = OLAP`), we do not recommend this for the reasons listed at the beginning of this doc. In OLAP mode, you may return more than 100k rows from a single query, but the 100k row limit will still apply to updates and deletes. When the use of OLAP queries is strictly unavoidable, we recommend: * Where possible, sending those queries to a replica. Every PlanetScale database comes with at least two replicas. Learn how to send queries to replicas using [global replica credentials](/docs/vitess/scaling/replicas#how-to-query-replicas). * Carefully and regularly reviewing the performance of those queries with [PlanetScale Insights](/docs/vitess/monitoring/query-insights). > [!NOTE] > > If you choose to use OLAP mode, be prepared to handle errors that may occur if the connection to PlanetScale gets interrupted. ## Reducing table size without `OPTIMIZE TABLE` If you delete several rows in a table, you may wish to reclaim that storage space. The MySQL [`OPTIMIZE TABLE` statement](https://dev.mysql.com/doc/refman/en/optimize-table.html) allows you to reorganize the physical storage of table data to reduce its storage requirements. However, `OPTIMIZE TABLE` is a locking operation, so you cannot use it unless you disable [safe migrations](/docs/vitess/schema-changes/safe-migrations) in PlanetScale, which is not recommended. Any time you create and deploy a deploy request with safe migrations on, our schema change process uses [online DDL](/docs/vitess/schema-changes/how-online-schema-change-tools-work), and in doing so, recreates the table that you are modifying — thus reclaiming the storage space. But there are often cases where you do not need to make a schema change for a while, but you'd like to free up the space. In those cases, we suggest you do the following: **Step 1** Create a new branch. **Step 2** Run the following command: ```sql theme={null} ALTER TABLE COMMENT 'Optimize table size via DR - '; ``` Where `` is the name of your table and `` is the current date. **Step 3** Create a deploy request, and merge it into production. If you use this option you will likely want to [throttle the deploy request](/docs/vitess/schema-changes/throttling-deploy-requests#managing-throttler-settings-per-deploy-request) running the `ALTER TABLE`. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Read-only regions Source: https://planetscale.com/docs/vitess/scaling/read-only-regions ## Overview Replicate your production database across the globe by creating read-only regions in any available [PlanetScale region](/docs/vitess/regions). This feature supports globally distributed applications by enabling your database to perform low latency reads in the regions closest to your applications and users. Read-only regions are configured per [keyspace](/docs/vitess/sharding/keyspaces). Each read-only region runs a single replica by default, and you can choose the cluster size and number of replicas for each region independently of your primary region. ## How to create a read-only region **Step 1** In the [PlanetScale dashboard](https://app.planetscale.com), select the database you want to add a read-only region to. **Step 2** Click "**Clusters**" in the left nav and make sure your production branch is selected. **Step 3** Select the keyspace you want to add a read-only region to. If you only have one keyspace, it will be selected by default. **Step 4** In the "**Read-only regions**" section, select the desired region from the dropdown of [available regions](/docs/vitess/regions). **Step 5** Choose a cluster size and the number of replicas for the region. The monthly cost of the region is shown alongside your selections. **Step 6** Click "**Save**" and wait for your data to finish initially replicating across regions. To add more than one region, click "**Add another region**" and configure each region before saving. ## How to change a read-only region's cluster size or replica count Each read-only region has its own cluster size and replica count. For example, a region that serves less read traffic than your primary region can run a smaller cluster size. **Step 1** Go to the "**Clusters**" page for your production branch and select the keyspace. **Step 2** In the "**Read-only regions**" section, choose a new cluster size or number of replicas for the region. **Step 3** Review the updated monthly cost and click "**Save**". If a read-only region runs a single replica, that replica is replaced during a cluster size change, and reads routed to the region will fail until the replacement completes. With two or more replicas, the region continues to serve reads while its replicas are replaced. We recommend running at least two replicas in any read-only region where uninterrupted reads are important. ## How to remove a read-only region **Step 1** Go to the "**Clusters**" page for your production branch and select the keyspace. **Step 2** In the "**Read-only regions**" section, click "**Remove region**" next to the region you want to remove. **Step 3** Click "**Save**". Once you remove a region, you will no longer be charged for the storage or row reads associated with that region. If you were using global replica credentials, you do not need to take any additional action. Read queries will still be sent to the closest replica for any queries that are using global replica credentials. ## How to query a read-only region ### Global replica credentials A [global replica credential](/docs/vitess/scaling/replicas) routes queries to the nearest replica among your branch's primary-region replicas and read-only regions. **Step 1** Go to your database's production branch. **Step 2** Click on the "Connect" button in the top right **Step 3** On the "Connect" page, select "Replica" as the connection type. **Step 4** Click "Create password" to generate a new username and password pair. Alternatively, create the credential from the CLI: ```bash theme={null} pscale password create --replica ``` You can also create a connection string by going to your database settings page > "**Passwords**" > "**New password**". ### Region-scoped credentials To always connect to a specific read-only region, create a region-scoped password from the CLI: ```bash theme={null} pscale password create --read-only-region eu-west ``` Pass a region slug, display name, or id. List the regions configured for a keyspace with: ```bash theme={null} pscale keyspace read-only-regions ``` `--read-only-region` cannot be combined with `--replica`. Use `--replica` when you want nearest-replica routing across regions. ### Direct host selection If you already have a password, open it on the "**Passwords**" page within your database's settings. Under "**Database endpoint**", select "**Direct**" and choose your desired host from the "**Host**" dropdown. ## Concepts ### Read-only regions and keyspaces Read-only regions are configured independently for each keyspace on your production branch. If your branch has multiple keyspaces, each keyspace has its own set of read-only regions, with its own cluster size and replica count in each region. One keyspace can replicate to a region that another keyspace does not. For sharded keyspaces, the configuration applies to every shard in the keyspace. For example, a read-only region with 2 replicas in a 4-shard keyspace runs 2 replicas of each shard in that region, and the cost shown when configuring the region is per shard. ### Replication across regions PlanetScale replicates your data across regions with an asynchronous strategy, first storing your changes in the primary region and then forwarding them to your read-only region(s). The time that it takes those changes to propagate to your read-only region can be defined as "replication lag" and be measured by issuing the following statement to your read-only regions: ```sql theme={null} SELECT max_repl_lag(); ``` The `max_repl_lag()` function will return an instantaneous measurement of the maximum amount of seconds it has been since your read-only region has stored changes made to your primary region. ### Read-only connections Connecting to a read-only region will allow you to query your data, but will not allow you to insert, update, or delete it. ## Availability and pricing Read-only regions are available on the [Base and Enterprise plans](/docs/planetscale-plans). Read-only regions are priced differently depending on the selected region. ### Replica costs The prices in the tables below cover a read-only region running a single replica at the selected cluster size. Each replica beyond the first is billed at the standard additional replica rate for that cluster size. This is the same rate as additional [replicas](/docs/vitess/scaling/replicas) in your primary region. The total monthly cost for a region is shown in the app as you configure it. For sharded keyspaces, read-only region charges apply to each shard in the keyspace. As with all cluster charges, read-only region charges are prorated, so changing a region's cluster size or replica count mid-month only bills the new configuration for the time it was running. ### Storage costs Your storage costs will increase linearly with the amount of read-only regions added. Adding new read-only regions will always be billed as standalone storage and will not count toward your included storage. As an example, let's say you're on our Base plan with 10 GB of included storage and your primary contains 7 GB of data. If you have two read-only regions, each one will be charged at our additional storage rate, for a total of 14 GB. The read-only region storage rate is \$0.75 per GB, and in this case would lead to an additional storage charge of \$10.50. For more information on storage billing costs, see our [Billing documentation](/docs/planetscale-plans). ## Pricing Below we include the costs for read-only regions in each available region up to `400`-level instances. Larger sizes are available in the app. ### AWS ap-northeast-1 (Tokyo) | Read-only region | Price | | - | - | | PS-10RR | \$19 | | PS-20RR | \$28 | | PS-40RR | \$48 | | PS-80RR | \$86 | | PS-160RR | \$168 | | PS-320RR | \$336 | | PS-400RR | \$480 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$296 | | M-160RR | 460 GB | \$324 | | M-320RR | 229 GB | \$588 | | M-320RR | 929 GB | \$640 | ### AWS ap-south-1 (Mumbai) | Read-only region | Price | | - | - | | PS-10RR | \$10 | | PS-20RR | \$15 | | PS-40RR | \$26 | | PS-80RR | \$46 | | PS-160RR | \$91 | | PS-320RR | \$182 | | PS-400RR | \$260 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$256 | | M-160RR | 460 GB | \$312 | | M-320RR | 229 GB | \$508 | | M-320RR | 929 GB | \$620 | ### AWS ap-southeast-1 (Singapore) | Read-only region | Price | | - | - | | PS-10RR | \$19 | | PS-20RR | \$28 | | PS-40RR | \$48 | | PS-80RR | \$86 | | PS-160RR | \$168 | | PS-320RR | \$336 | | PS-400RR | \$480 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$296 | | M-160RR | 460 GB | \$332 | | M-320RR | 229 GB | \$588 | | M-320RR | 929 GB | \$656 | ### AWS ap-southeast-2 (Sydney) | Read-only region | Price | | - | - | | PS-10RR | \$19 | | PS-20RR | \$28 | | PS-40RR | \$48 | | PS-80RR | \$86 | | PS-160RR | \$168 | | PS-320RR | \$336 | | PS-400RR | \$480 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$280 | | M-160RR | 460 GB | \$332 | | M-320RR | 229 GB | \$560 | | M-320RR | 929 GB | \$656 | ### AWS eu-central-1 (Frankfurt) | Read-only region | Price | | - | - | | PS-10RR | \$19 | | PS-20RR | \$28 | | PS-40RR | \$48 | | PS-80RR | \$86 | | PS-160RR | \$168 | | PS-320RR | \$336 | | PS-400RR | \$480 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$292 | | M-160RR | 460 GB | \$328 | | M-320RR | 229 GB | \$584 | | M-320RR | 929 GB | \$652 | ### AWS eu-west-1 (Dublin) | Read-only region | Price | | - | - | | PS-10RR | \$18 | | PS-20RR | \$26 | | PS-40RR | \$44 | | PS-80RR | \$80 | | PS-160RR | \$156 | | PS-320RR | \$313 | | PS-400RR | \$448 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$272 | | M-160RR | 460 GB | \$304 | | M-320RR | 229 GB | \$540 | | M-320RR | 929 GB | \$604 | ### AWS eu-west-2 (London) | Read-only region | Price | | - | - | | PS-10RR | \$18 | | PS-20RR | \$28 | | PS-40RR | \$46 | | PS-80RR | \$84 | | PS-160RR | \$163 | | PS-320RR | \$327 | | PS-400RR | \$468 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$288 | | M-160RR | 460 GB | \$320 | | M-320RR | 229 GB | \$568 | | M-320RR | 929 GB | \$632 | ### AWS sa-east-1 (Sao Paulo) | Read-only region | Price | | - | - | | PS-10RR | \$25 | | PS-20RR | \$38 | | PS-40RR | \$63 | | PS-80RR | \$114 | | PS-160RR | \$223 | | PS-320RR | \$447 | | PS-400RR | \$639 | ### AWS us-east-1 (N. Virginia) | Read-only region | Price | | - | - | | PS-10RR | \$16 | | PS-20RR | \$24 | | PS-40RR | \$40 | | PS-80RR | \$72 | | PS-160RR | \$140 | | PS-320RR | \$280 | | PS-400RR | \$400 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$244 | | M-160RR | 460 GB | \$276 | | M-160RR | 1,241 GB | \$404 | | M-320RR | 229 GB | \$484 | | M-320RR | 929 GB | \$584 | | M-320RR | 2,490 GB | \$804 | ### AWS us-east-2 (Ohio) | Read-only region | Price | | - | - | | PS-10RR | \$16 | | PS-20RR | \$24 | | PS-40RR | \$40 | | PS-80RR | \$72 | | PS-160RR | \$140 | | PS-320RR | \$280 | | PS-400RR | \$400 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$244 | | M-160RR | 460 GB | \$276 | | M-160RR | 1,241 GB | \$404 | | M-320RR | 229 GB | \$484 | | M-320RR | 929 GB | \$584 | | M-320RR | 2,490 GB | \$804 | ### AWS us-west-2 (Oregon) | Read-only region | Price | | - | - | | PS-10RR | \$16 | | PS-20RR | \$24 | | PS-40RR | \$40 | | PS-80RR | \$72 | | PS-160RR | \$140 | | PS-320RR | \$280 | | PS-400RR | \$400 | | Metal read-only region | Storage | Price | | - | - | - | | M-160RR | 110 GB | \$244 | | M-160RR | 460 GB | \$276 | | M-160RR | 1,241 GB | \$404 | | M-320RR | 229 GB | \$484 | | M-320RR | 929 GB | \$584 | | M-320RR | 2,490 GB | \$804 | ### GCP asia-northeast3 (Seoul, South Korea) | Read-only region | Price | | - | - | | PS-10RR | \$19 | | PS-20RR | \$28 | | PS-40RR | \$48 | | PS-80RR | \$86 | | PS-160RR | \$168 | | PS-320RR | \$336 | | PS-400RR | \$480 | ### GCP northamerica-northeast1 (Montréal, Québec) | Read-only region | Price | | - | - | | PS-10RR | \$16 | | PS-20RR | \$24 | | PS-40RR | \$40 | | PS-80RR | \$72 | | PS-160RR | \$140 | | PS-320RR | \$280 | | PS-400RR | \$400 | ### GCP us-central1 (Council Bluffs, Iowa) | Read-only region | Price | | - | - | | PS-10RR | \$16 | | PS-20RR | \$24 | | PS-40RR | \$40 | | PS-80RR | \$72 | | PS-160RR | \$140 | | PS-320RR | \$280 | | PS-400RR | \$400 | ### GCP us-east4 (Ashburn, Virginia) | Read-only region | Price | | - | - | | PS-10RR | \$16 | | PS-20RR | \$24 | | PS-40RR | \$40 | | PS-80RR | \$72 | | PS-160RR | \$140 | | PS-320RR | \$280 | | PS-400RR | \$400 | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Database replicas Source: https://planetscale.com/docs/vitess/scaling/replicas ## Overview A replica is a continuously updated copy of your Vitess database. Replicas serve two main purposes: * They provide an additional source for your data to be queried. * They increase database availability by enabling fast failovers for maintenance or unexpected failure. > [!WARNING] > > Before utilizing replicas for reducing load on the primary, it's important to understand the trade-offs. For more information, see the [Data consistency and replication lag](#data-consistency-and-replication-lag) section. For information on replicas running in additional regions, see: [Read-only regions](/docs/vitess/scaling/read-only-regions). ## How to query replicas PlanetScale replicas can be used to read data and reduce load on the primary. PlanetScale does not automatically route queries to replicas unless you explicitly use a replica credential or tell your application to do so. ### 1. Create a global replica credential (recommended) With global replica credentials, you can have one credential that will automatically route queries to your branch's replicas and read-only regions. If you do not have any read-only regions, it will route to the included replicas on [your plan](/docs/planetscale-plans). If you do have any read-only regions, the queries will route to the closest replica out of your read-only region replicas and included failover replicas. To use this, create a new credential in the PlanetScale dashboard and select "**Replica**" as the connection type. You can then use the new credential's connection string to setup a replica-only database connection. If you have `pscale` installed locally, you can create a replica credential with the following command: ```shell theme={null} pscale password create --replica ``` All queries made using this credential will be routed to the nearest replica, even as you add and remove read-only regions. To pin traffic to a specific read-only region instead, create a region-scoped password: ```shell theme={null} pscale password create --read-only-region ``` See [Read-only regions](/docs/vitess/scaling/read-only-regions#region-scoped-credentials) for details. ### 2. `USE @replica` (not recommended) > [!WARNING] > > We do not recommend using this method to query replicas. If you are currently using this method, we recommend switching to method 1. If you are not using a global replica credential, then, by default, all queries are served by the primary. However you may route any `SELECT` queries to replicas by issuing the following command on a connection: ```sql theme={null} USE @replica ``` Once this command is run, all subsequent `SELECT` queries on that connection will be served by your branch's replicas instead of the primary node in your cluster. Please note that this does not send queries to any read-only regions, if you want to route queries all replicas, including read-only regions, you should use a global replica credential (method 1 above). > [!TIP] > > For querying replicas in separate regions, see: [Read-only regions](/docs/vitess/scaling/read-only-regions). We highly recommend using a global replica credential to ensure that your queries are always routed to the nearest replica. ## High availability Replicas within PlanetScale are used to enable high availability of your database. This is a part of the reason all production branches in PlanetScale are provided at least one replica. In situations where the underlying hardware or service hosting the primary MySQL node fails, our system will automatically elect a new primary node from the available replicas and reroute traffic to that new primary. This process is know as **reparenting** and typically is all handled within milliseconds or seconds. Querying the primary during a reparent typically goes unnoticed, other than a bit of additional query latency. This is because Vitess [buffers queries](https://vitess.io/docs/reference/features/vtgate-buffering) during this time. You can incorporate retry logic into your application to handle the rare instances where an issue arises during reparenting. When querying a replica during a reparent, you may encounter this error: ``` no healthy tablet available for 'keyspace:"${keyspace}" shard:"${shard}" tablet_type:REPLICA' ``` Incorporating retry logic into your application can help with this scenario as well. ## Multiple availability zones In cloud architecture, regions are further broken down into data centers known as availability zones (or AZs for short). For example, the `us-east-1` region on AWS contains 6 default AZs available to customers starting with `us-east-1a` through `us-east-1f`. The infrastructure for all Base plan and Enterprise PlanetScale databases are distributed across 3 availability zones. In the instance of an AZ failure, your database will auto failover to an available AZ. ## Data consistency and replication lag Whenever data is updated (`INSERT`, `UPDATE`, `DELETE`) on the primary node, those changes are synchronized to the replicas shortly after. The delay between when a primary is updated and the changes are applied to the replica is known as `replication lag`. Your databases replication lag is viewable on your [database dashboard](/docs/vitess/architecture#replication-lag-at-a-glance). Replication lag can also be viewed on Datadog if you set up the [PlanetScale - Datadog integration](/docs/vitess/integrations/datadog). In Datadog, you can use the following formula to set alerts for replication lag: ``` (max:planetscale.replication_lag{ps_database: ps_tablet_type:replica, ps_branch:
}) ``` Make sure you replace `` with your PlanetScale database name and `
` with the name of the branch for which you'd like to track replication lag. It is important to be aware of replication lag whenever querying data from your replicas. For example, if you make an update and then immediately try to query for that updated data via a replica, it may not be available yet due to replication lag. ## When should you use replicas in your application? Replicas are useful for offloading read-heavy workloads from the primary node. By using replicas, you can distribute the read load across multiple nodes, which can help improve the performance of your application. Some examples of where you might want to query a replica are scheduled jobs, analytics jobs, search features, or aggregate queries. Replicas can also be used to provide a read-only view of your data to users or applications that do not need to write data, such as when a user is logged out or writing one-off queries for debugging purposes. ## When are replicas used in PlanetScale? We use replicas for every production database branch. The number of replicas for a given database depends on the selected plan for that database: * **Base plan** — Base plan databases include 2 replicas per production branch distributed across multiple AZs in a given region. * **Enterprise** — The Enterprise plan is customizable to fit the needs of your organization, and as such can have as many replicas as needed. [Read-only regions](/docs/vitess/scaling/read-only-regions) also run replicas, hosted in a different geographical region. Each read-only region has its own cluster size and replica count, configured per keyspace independently of the primary region. It is important to note that the MySQL nodes in read-only regions are replicas intended only for reading data and are not eligible for failover if the primary node experiences an outage. > [!WARNING] > > Development branches DO NOT have replicas as they are intended for development only and are not designed with the same resiliency as production branches. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # VTGates Source: https://planetscale.com/docs/vitess/scaling/vtgates A VTGate, or Vitess Gateway, is the layer of Vitess that acts as a proxy between your application servers and the MySQL instances. When an application needs to connect to a Vitess cluster, it does not make connections directly to MySQL. Instead, connections are made to our [Global Edge Network](https://planetscale.com/blog/introducing-global-replica-credentials#building-planetscale-global-network), which then routes them to a [VTGate](https://vitess.io/docs/concepts/vtgate/). The VTGate layer acts as the entry point to the cluster, proxies connections, and handles routing of incoming queries to the appropriate [keyspace](/docs/vitess/sharding/keyspaces) and/or [shard](/docs/vitess/sharding). ## Managing your VTGate layer From the Clusters page: * Adjust the size of your VTGates, e.g. `VTG-5` to `VTG-10` * Adjust the number of VTGates your cluster has (only configurable for `VTG-320` and larger). By default, 1 VTGate is deployed per availability zone (for a total of 3 VTGates). You can adjust this up to 24 VTGates per AZ, for a total of 72 VTGates. * Turn on VTGate autoscaling (for `VTG-320` and larger) * View your VTGate CPU and memory utilization for the past hour up through the past week * View any past changes to your VTGates ![VTGate page in PlanetScale dashboard](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/vtgates/vtgates-panel.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=b27702d0cc3e50403d226b468fe5b07d) ## Adjusting the size and number of VTGates To modify the size or number of VTGates, click on your database in the dashboard, "Clusters" in the sidebar, choose the branch you want to modify from the dropdown, and then click on the "VTGates" tab at the top. From here, you can see the current VTGate utilization, adjust your VTGate size, and modify the number of VTGates per availability zone (for `VTG-320` and larger). Note the price difference on this page after you adjust. This will be added to your monthly bill going forward. When you're satisfied, click "Save changes". It will take some time to update the VTGate configuration. This is an online operation and does not cause downtime. ### Configuring VTGates with the CLI You can also view and resize VTGates with the [PlanetScale CLI](/docs/cli/branch). Only production branches can be resized. ```bash theme={null} # Show the current VTGate configuration pscale branch vtgate show # Queue a VTGate resize pscale branch vtgate resize \ --vtgate-size VTG_320 \ --vtgate-autoscaling \ --vtgate-count 2 \ --vtgate-max-count 8 \ --vtgate-target-cpu-utilization 50 # Check on or cancel a resize request pscale branch vtgate resize status pscale branch vtgate resize cancel ``` CLI size SKUs use underscores (for example `VTG_320`). Hyphens are accepted and converted. See the [`branch` command reference](/docs/cli/branch) for all flags. ## VTGate default sizes By default, every PlanetScale cluster comes with three VTGates, each in a different availability zone. The default size of your VTGates depends on the size of your cluster's largest keyspace: | Cluster size | Default VTGate size | | - | - | | `PS-10` | `VTG-5` | | `PS-20` | `VTG-5` | | `PS-40` | `VTG-5` | | `PS-80` | `VTG-5` | | `PS-160` | `VTG-10` | | `PS-320` | `VTG-20` | | `PS-400` | `VTG-40` | | `PS-640` | `VTG-40` | | `PS-700` | `VTG-80` | | `PS-900` | `VTG-80` | | `PS-1280` | `VTG-80` | | `PS-1400+` | `VTG-320` | PlanetScale will automatically adjust your VTGate size whenever you resize your cluster. For example, if you resize your cluster from `PS-10` to `PS-160`, your VTGates will automatically resize from `VTG-5` to `VTG-10`. If you have made any modifications to the default VTGate configuration, then PlanetScale will not automatically resize your VTGates during a cluster size change. ## VTGate credits and pricing Every PlanetScale cluster includes VTGate credits that cover the cost of a VTGate configuration that will work well for most workloads. These credits automatically scale with your cluster size and complexity: * Larger cluster sizes include more VTGate credits * Additional keyspaces increase your VTGate credits * Additional shards increase your VTGate credits If your specific workload requires more powerful VTGates, you can upgrade them manually through the dashboard or [CLI](/docs/cli/branch). Any VTGate usage that exceeds your included credits will be billed as an additional charge. Here are some examples of how VTGate credits work: **Example: Single keyspace** * Cluster: One unsharded `PS-10` (1/8 vCPU, 1GB memory) * Included credits cover: `VTG-5` VTGates (1/16 vCPU, 128MB memory each) **Example: Multiple keyspaces** * Cluster: Two unsharded `PS-10` clusters across two keyspaces * The additional keyspace doubles your VTGate credit allowance **Example: Sharded cluster** * Cluster: Two keyspaces * One unsharded `PS-10` cluster * One sharded keyspace with two `PS-10` shards * Credits are tripled due to the additional keyspace and shards ## Autoscaling VTGates You have the option to automatically scale the number of your VTGates based on VTGate CPU utilization. To turn on autoscaling, go to the Clusters page, click "VTGates", click the "Autoscaling" tab, and check the "Use horizontal autoscaling" box. You can choose from 40%, 50%, 60%, and 70% utilization. When the average CPU utilization of your VTGates exceeds the chosen threshold, the system will automatically increase the number of VTGates to accommodate. When CPU utilization falls below this target, it will scale in — downsizing the number of VTGates. VTGates in each availability zone will scale independently based on their individual utilization. This means that if one zone experiences higher load, it may scale up while others remain at their current number. | VTGate size | Maximum number of VTGates per availability zone | | - | - | | `VTG-320` | 16 | | `VTG-640` | 16 | | `VTG-1280` | 128 | > [!NOTE] > > You will be billed for the actual VTGate usage during autoscaling periods. If your VTGates frequently scale up to handle load, this can result in significant additional costs beyond your included VTGate credits. ## When to increase the number of VTGates There are several indicators that your VTGates may need more resources: ### High resource utilization Monitor your VTGate CPU and memory usage in the PlanetScale dashboard. If you consistently see: * CPU utilization above 70% * Memory utilization above 80% This indicates your VTGates are under heavy load and you should consider either increasing the VTGate size or adding more VTGates to your cluster. ### Connection errors If you encounter either of these errors, it typically means your VTGates are overwhelmed and unable to handle incoming connections: ``` ERROR HY000 (1105): unavailable: vtgate connection error: no endpoints, after 1 attempts ``` ``` Error 1105 (HY000): unavailable: vtgate connection error: no healthy endpoints, after 1 attempts ``` These errors occur when VTGates are resource-constrained and cannot accept new connections. To resolve this: 1. First, check your VTGate CPU and memory metrics 2. Consider upgrading to a larger VTGate size 3. For `VTG-320` and larger, you can also increase the number of VTGates per availability zone 4. Enable autoscaling if your workload has variable load patterns ## VTGate Resources Each VTGate size has varying levels of compute power and memory provisioned. Because an individual VTGate is run in every availability zone, your total amount of available resources is three times larger than the resources allocated to each instance. | | **Processor** | **Memory** | | - | - | - | | **VTG-5** | 1/16 vCPU | 128 MB RAM | | **VTG-10** | 1/8 vCPU | 256 MB RAM | | **VTG-20** | 1/4 vCPU | 512 MB RAM | | **VTG-40** | 1/2 vCPU | 1 GB RAM | | **VTG-80** | 1 vCPU | 2 GB RAM | | **VTG-320** | 4 vCPU | 8 GB RAM | | **VTG-640** | 8 vCPU | 16 GB RAM | | **VTG-1280** | 16 vCPU | 32 GB RAM | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # What are workflows? Source: https://planetscale.com/docs/vitess/scaling/workflows Vitess MoveTables workflows for sharding tables, reorganizing keyspaces, and importing external MySQL databases with no downtime. Workflows are [Vitess MoveTables](https://vitess.io/docs/user-guides/migration/move-tables/) workflows built on [Vitess VReplication](https://vitess.io/docs/reference/vreplication/vreplication/). A workflow copies tables from a source keyspace to a target keyspace, keeps them in sync with real-time replication, and lets you verify the data and switch traffic when you're ready. You create and drive workflows with [`pscale branch vtctl move-tables`](/docs/cli/move-tables) or the MoveTables API. The dashboard shows the live state of each workflow, read directly from Vitess. There are two ways to use workflows: > **[Move tables between keyspaces](#move-tables-between-keyspaces)** > > Move tables between keyspaces within your PlanetScale database. Used for sharding, resharding, and reorganizing data. > **[Import a database](#import-a-database)** > > Import data from an external MySQL database into PlanetScale with no downtime. ## Move tables between keyspaces Move tables from one [keyspace](/docs/vitess/sharding/keyspaces) to another within your PlanetScale database: * **Unsharded to sharded**: Move tables from an unsharded keyspace to a new sharded keyspace. This is the primary way to shard existing tables. See the [Sharding quickstart](/docs/vitess/sharding/sharding-quickstart). * **Sharded to sharded**: Move tables between sharded keyspaces to change the number of shards. See [Modifying the number of shards](/docs/vitess/sharding/sharding-a-sharded-keyspace). * **Unsharded to unsharded**: Move tables from one unsharded keyspace to another unsharded keyspace. ## Import a database To import an **external internet-accessible MySQL database**, you first connect it to your production branch as an [external keyspace](/docs/vitess/cluster-configuration#create-an-external-keyspace). Then you run a workflow with the external keyspace as the source and your PlanetScale keyspace as the target. The import copies your data, replicates new changes in real time, and switches traffic to PlanetScale with no downtime. For a full walkthrough, see the [Database imports documentation](/docs/vitess/imports/database-imports). ## Workflow lifecycle Every workflow follows the same lifecycle: 1. **Create**: `move-tables create` copies the table schemas to the target keyspace and starts copying rows. 2. **Copy**: Vitess copies the existing rows from the source to the target. Streams are `Copying`. 3. **Replicate**: Once the copy finishes, Vitess keeps the target up to date with every new write on the source. Streams are `Running`. 4. **Verify**: Optionally compare the source and target with a [VDiff](https://vitess.io/docs/reference/vreplication/vdiff/). 5. **Switch traffic**: `move-tables switch-traffic` moves replica traffic, then primary traffic, to the target keyspace. `move-tables reverse-traffic` moves it back. 6. **Complete**: `move-tables complete` stops replication. Its required flags decide whether the moved tables are dropped from the source keyspace and whether the routing rules are removed. You can also `cancel` a workflow at any point before it is complete. Use `move-tables status` between steps. It shows each table's copy progress, each stream's state, and the traffic state. JSON output includes a `next_steps` field with the command to run next. See the [`move-tables` reference](/docs/cli/move-tables) for every command and flag. ## View workflows in the dashboard Select your database and click **Workflows** in the left nav. Pick a branch to see the MoveTables workflows on it, across all keyspaces or filtered to one keyspace. Open a workflow to see: * The source and target keyspaces, replication lag, and when the workflow last updated * Which keyspace is serving reads and writes * Each stream's state, rows copied, and any error message * Each table's copy progress * The results of the latest VDiff The Workflows page is read-only. To create, switch, complete, or cancel a workflow, use [`pscale branch vtctl move-tables`](/docs/cli/move-tables). ## Earlier workflows Workflows created with `pscale workflow create`, the earlier dashboard workflow pages, or the `/workflows` API endpoints are managed separately from MoveTables workflows. [`pscale workflow`](/docs/cli/workflow) and the `/workflows` API endpoints are deprecated. Use `pscale branch vtctl move-tables` and the MoveTables API for new work. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Non-blocking schema changes Source: https://planetscale.com/docs/vitess/schema-changes **Non-blocking schema changes** in PlanetScale provide a schema change workflow that allows users to update database tables without locking or causing downtime for production databases. ## Overview > [!TIP] > > To make non-blocking schema changes in PlanetScale, you'll first need a basic understanding of > [branching](/docs/vitess/schema-changes/branching), the core PlanetScale feature that provides schema changes. Our > branching page is a great place to start. PlanetScale makes it safe to deploy schema changes to production databases via *development* and *production branches with [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled*. Branches with safe migrations enabled can only be updated using deploy requests, and default branches cannot be deleted. Development branches are a separate database with a copy of the source branch's schema. Developers can make schema changes in development branches, test locally, and open a deploy request for deploying their changes to the production database. Developers can also comment on deploy requests and request reviewers to approve a deploy request before its schema changes can deploy into base branches. Currently, requiring approval is a per-database setting that is turned off by default. With the setting turned off, developers do not need approval to merge a deploy request. ## Adding columns to large tables with PlanetScale is safe! *Create*, *drop*, and *alter* statements, also known as Data Definition Language (DDL), are used for making schema changes in a database table. PlanetScale enables developers to make schema changes without the fear of dropping columns, locking tables, causing downtime in their app, etc. PlanetScale also prevents schema changes with conflicts from being migrated and handles schema changes from multiple teammates. A user doesn't have to wait to find out if their changes will be rejected, they learn as they add the change to the queue. ## How do I make non-blocking schema changes with PlanetScale? In order to make non-blocking schema changes, you **must** enable [safe migrations](/docs/vitess/schema-changes/safe-migrations) on your production branch. Without safe migrations enabled, your schema changes will run directly on your production branch, which can lead to table locking. When safe migrations are enabled on a branch, all schema changes must occur on a database branch. *(A database branch is a separate database with a copy of the production branch's schema.)* At a high level, this is what happens during the *non-blocking schema change* process in PlanetScale: **Step 1** You create a development branch. **Step 2** You test your changes on this branch before attempting to apply the changes to the production branch. *(i.e., You made some changes to the database you wish to deploy to the production database.)* **Step 3** You open a request to deploy your changes to the base branch, the production branch. **Step 4** PlanetScale verifies that your schema changes are safe to be deployed to production. If there are any issues or schema conflicts, you'll be shown the errors. **Step 5** You click `Deploy changes`. Your deploy is added to a queue and run immediately or when existing deploys are complete. **Step 6** Your deployment makes it to the base branch, and you can now see your schema changes in production. > [!NOTE] > > PlanetScale makes sure not to exhaust your resources; the deployment may be throttled to avoid any impact on > production queries. ![PlanetScale non-blocking schema changes diagram](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/nonblocking-schema-changes/diagram.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=9e5e4b360a9af1f69992a766fae62a49) ## PlanetScale workflow The PlanetScale command-line tool (CLI) runs an interactive shell equipped with many commands designed to make the database management workflow easier for developers. A basic non-blocking schema change workflow in PlanetScale might look like this: **Step 1** Create a database: ```bash theme={null} pscale database create ``` **Step 2** Create a development branch: ```bash theme={null} pscale branch create ``` **Step 3** Make a schema change on this branch: ```bash theme={null} pscale shell ``` > [!TIP] > > A schema change is any change you make to the tables in your database environment created within the PlanetScale branch. (i.e., create, drop, and alter statements) > [!WARNING] > > You can only apply direct schema changes to branches without safe migrations enabled. Here is a sample CREATE table schema change you could try using: ```sql theme={null} CREATE TABLE `reminders` ( `id` bigint NOT NULL AUTO_INCREMENT, `body` varchar(1024) NOT NULL, `created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6), `updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6), `is_done` tinyint(1) NOT NULL DEFAULT '0', PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; ``` **Step 4** Test changes on branch locally. **Step 5** Create a deployment request by running: ```bash theme={null} pscale deploy-request create ``` **Step 6** Fix any schema conflicts. PlanetScale displays the difference between what is currently in the base branch and your development branch. Go back to *Step 3* of the workflow and test out new schema changes to fix the schema conflict. If you did not encounter any schema conflicts, you're ready for *Step 7*. **Step 7** Deploy the deploy request. * To *deploy* the **deploy request** created in *Step 5*, run the following command: ```bash theme={null} pscale deploy-request deploy ``` * To find your `deploy-request-number`, simply run: ```bash theme={null} pscale deploy-request list ``` Copy the value from `NUMBER` and use that digit as your `deploy-request-number`. ## Limitations If you want to make schema changes containing foreign key constraints, enable foreign key constraint support for your database in the database settings page. PlanetScale doesn't support direct `RENAME` for columns and tables. Learn why and how to rename tables or columns in [this tutorial](/docs/vitess/schema-changes/handling-table-and-column-renames). A single deploy request can alter up to 10 tables. To make more changes you'll want to split up your migration over multiple deploy requests or temporarily disable safe-migrations and apply the changes directly. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Aggressive deploy request cutover Source: https://planetscale.com/docs/vitess/schema-changes/aggressive-cutover ## Overview Cutover is the final step in an online schema migration where Vitess atomically swaps the original table with a newly created shadow table containing the updated schema. Completing the swap requires a brief metadata lock (MDL) on the table. Long-running queries or transactions holding locks on the table can prevent Vitess from acquiring the MDL, delaying cutover. Force cutover only applies after copy has finished. If the deploy failed while copying, see [Long-running transactions](/docs/vitess/schema-changes/long-running-transactions). ## Force cutover When cutover is blocked, Vitess retries for up to **1 hour**, then automatically runs force cutover and kills the blocking queries and transactions. To complete the cutover sooner, use the **Force cutover now** button on the deploy request page, or run: ```bash theme={null} pscale deploy-request force-cutover ``` This triggers force cutover immediately without waiting for the 1 hour timeout. Pass `--force` to skip the confirmation prompt. > [!WARNING] > > Force cutover terminates active queries and transactions holding locks on the table. Understand which queries will be affected before using this option. Having retry logic or a strategy to handle re-running killed queries is advised. ## Aggressive cutover for a single deploy request If you expect one deploy request to be blocked, opt it into aggressive cutover when you create it or any time before you deploy it. Vitess kills blocking queries and transactions on the first cutover attempt for that deploy request only. The database setting and other deploy requests are unchanged. ```bash theme={null} pscale deploy-request create --aggressive-cutover pscale deploy-request update --aggressive-cutover pscale deploy-request update --aggressive-cutover=false ``` Through the API, pass `aggressive_cutover: true` when creating a deploy request, or `PUT /organizations/{organization}/databases/{database}/deploy-requests/{number}/aggressive-cutover` with `{ "enable": true }`. The deployment object reports the opt-in as `aggressive_cutover`. ## Aggressive cutover setting For databases that regularly hit cutover delays, admins can enable **aggressive cutover** on the database **Settings** page under **Advanced settings** or with [`pscale database aggressive-cutover enable`](/docs/cli/database#database-level-aggressive-cutover). When enabled, Vitess kills blocking queries and transactions on the first cutover attempt instead of retrying for up to an hour. The setting applies to all future deploy requests. ### When to enable aggressive cutover 1. **Migration delayed notice**: Deploy requests showing "Migration delayed due to long-running transactions" 2. **Slow or long-running workloads**: Applications that consistently block cutover with slow queries or open transactions 3. **Frequent manual force cutover**: Regular use of **Force cutover now** on deploy requests ### Normal vs aggressive cutover behavior **Normal cutover behavior:** * Vitess attempts to acquire a metadata lock (MDL) on the table * If blocked, it waits and retries for up to 1 hour * After 1 hour, Vitess automatically runs force cutover **Aggressive cutover behavior:** * Vitess kills blocking queries and transactions on the first cutover attempt * The cutover proceeds without waiting for blocking operations to complete ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Branching Source: https://planetscale.com/docs/vitess/schema-changes/branching PlanetScale allows you to branch database schemas the same way you branch your code. ## What are branches on PlanetScale Database branches on PlanetScale are isolated database instances that give you flexibility when developing your application. When your database is first initialized, a single production branch is created called `main` and acts as the default branch. When you create additional branches, the schema of the source branch is copied to the new branch, giving you an isolated MySQL instance to develop with. Changes made in one branch, whether to the schema or the data, do not affect any other branches for a given database. [Data drop: Branching and deploy requests](https://www.youtube-nocookie.com/embed/EbG6-GUUIfc?rel=0) ## Development and production branches PlanetScale provides two types of database branches: * **Development branches** — Development branches provide isolated copies of your production database schema where you can make changes, experiment, or run CI. Please note that only the schema is copied. A new development branch will not have any data stored in it unless you [restore from a backup](/docs/vitess/backups#restore-from-a-backup). To automatically create a development branch with data from another branch, see the [Data Branching® feature](/docs/vitess/schema-changes/data-branching). * **Production branches** — Production branches are highly available databases intended for production traffic. They are automatically provided with an additional [replica](/docs/vitess/scaling/replicas) to resist outages, enabling zero-downtime failovers. The Base plan includes two additional replicas, and Enterprise plans are customizable. Both types of branches also offer [safe migrations](/docs/vitess/schema-changes/safe-migrations) as an optional feature, which helps protect the branch from accidental schema changes and enables non-blocking schema migrations. We highly recommend that all production database branches have the safe migrations setting turned on. If you want to set up a workflow with a ”staging” branch, see more the [staging branch documentation](/docs/vitess/schema-changes/safe-migrations#staging-branches) below. ## Promote a branch to production PlanetScale provides the ability to **promote any development branch with a valid schema to production**. Promoting a branch to production will automatically set up the database replica(s) to make your database highly available. Note: Only one production branch is included in the Base plan, but you can add more production branches for an additional fee. Only administrators can promote database branches. ### Promote a development branch to production Every new PlanetScale database is created with a production branch named `main`. This branch is intended as the starting point for building your database on PlanetScale and has increased performance and resilience by default when compared to development branches. That said, you don't have to use the default `main` branch as your production branch. **Any development branch can be promoted to production**. Once you are satisfied with the changes you've made to a given development branch, you can promote it to production. Going forward, you can continue to make new development branches off of this production branch to experiment with changes as needed. A branch can be promoted from the branch overview page in the PlanetScale app or by using the [PlanetScale CLI](/docs/cli/branch), as shown below: ``` pscale branch promote ``` It's possible to run multiple production branches simultaneously, within [your plan's limits](/docs/planetscale-plans). Keep in mind, PlanetScale does not provide data syncing between production branches. ### Demote a production branch to development Sometimes, you might need to demote a production branch to a development branch. This can be done from the production branch's overview page in the PlanetScale app (located in the right column of the page). Be aware when demoting a production branch to a development branch: * Databases must have at least one production branch at all times * Development branches are not meant to be used with production workloads * The branch will no longer have high-availability features * Existing scheduled production branch backup policies will no longer run * Any read-only regions will need to be removed If you are on an Enterprise plan, only an administrator for your organization can request to demote a branch, and the demotion request will need to be approved by another administrator. Once the first administrator requests to demote a production branch, the second administrator can approve the demotion on the production branch's overview page. You will see the request from the first administrator and a **Demote to development branch** button to complete the demotion. ## Create a development branch If you need to experiment with schema changes, you can create a new development branch. This will copy the schema from the base branch into a new branch where you can create and test your changes. Development branches **will not** copy over data, just the schema. To create a development branch with data from another branch, see the [Data Branching® feature](/docs/vitess/schema-changes/data-branching) section. **How to create a development branch**: **Step 1** On the database dashboard page, click the "**New branch**" button. **Step 2** Give your development branch a name and select the region closest to your or your application. **Step 3** Select the base branch you want to branch off of. **Step 4** Click "**Create branch**". **Step 5** (*Optional*) You can also create a new branch from the [PlanetScale CLI](/docs/cli/branch) with: ``` pscale branch create ``` ## Safe migrations [Safe migrations](/docs/vitess/schema-changes/safe-migrations) is an optional, but recommended, feature that can be enabled on branches. Branches with safe migrations enabled are restricted from accepting DDL directly. This prevents accidental changes to the database schema, and also enables non-blocking schema migrations. To make changes to branches with safe migrations enabled, you must create a new branch, then merge changes using a [deploy request](/docs/vitess/schema-changes/deploy-requests). Using this method, you get to see a schema diff before merging changes, have the option to have your team review changes, and receive [additional checks and warnings](https://planetscale.com/blog/deploy-requests-now-alert-on-potential-unwanted-changes) prior to making a production schema change. To enable safe migrations on a branch, select the branch you want to modify from the branch dropdown and click the **”cog”** in the upper right of the infrastructure card on the ”**Dashboard**” tab of the database. In the modal, toggle the option labeled **”Enable safe migrations”**, then click the **”Enable safe migrations”** button to save and close the modal. ### Staging branches You can use a development branch with safe migrations enabled to set up a workflow with a “staging” branch. First, make sure you have safe migration enabled for your main production branch. Then, create a “staging” branch with your main production branch as the base and turn on safe migrations. All new branches created for development can use this “staging” branch as the base branch. You can then open a deploy request against either the main production or “staging” branch. Once it is deployed to “staging,” you can open a deploy request against the main production branch. The main production branch must be set as the [default branch](/docs/vitess/schema-changes/branching#default-branches) to open a deploy request against it. > [!NOTE] > > In this setup, the “staging” branch is still a development branch. Compared to your main production branch (additional production branches are an additional cost), it will have reduced resources, similar to other development branches. ![View of the Branches tab with main \< staging \< dev branches](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/branching/branches-with-staging-branch.jpg?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=7f26070434f6e5fe07dd777c206a52c0) ## How to make schema changes on a branch with safe migrations enabled Since DDL is restricted on branches with safe migrations enabled to prevent accidental changes and enable zero-downtime migrations, you'll need to perform the following steps in order to make changes to a safe migrations enabled branch: ![PlanetScale Branching Flow Diagram](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/branching/diagram.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=70edd8ce6763ac4a1a0c70528a5aae19) > [!TIP] > > You'll see a `ERROR 1105 (HY000): direct DDL is disabled` message if you attempt to make schema changes in a branch with safe migrations enabled. Instead, create a development branch, and make your changes there. ### 1. Create a development branch The first step is to create a new development branch off of the branch you want to make changes to. This will make a copy of the source branches schema into the newly created development branch where you can perform the necessary changes to the schema. ### 2. Create a deploy request If you are working in a team, the [deploy request](/docs/vitess/schema-changes/deploy-requests) creates an opportunity for your teammates to review the changes you have made before they are deployed to production. **Step 1** To create the deploy request, go to the branch overview page for the branch you want to deploy. **Step 2** Select the base branch you want to deploy to from the "**Deploy to**" dropdown. **Step 3** (*Optional*) Add a comment describing your deploy request. . Click "**Create deploy request**". ![PlanetScale deploy request example](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/branching/deploy-request-page.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=abd052cb235c898d9ffb1f0690283bee) Alternatively, you can create a deploy request from the [PlanetScale CLI](/docs/cli/deploy-request) with the following command: ``` pscale deploy-request create ``` ### 3. Review the deploy request The deploy request includes a schema diff so that you can review the schema changes introduced by the deploy request against the base branch. **Step 1** On the deploy request page, click the "**Schema changes**" tab. **Step 2** Schema additions are highlighted in green, and deletions are highlighted in red. Alternatively, you can run the command below to see the [schema diff in the PlanetScale CLI](/docs/cli/deploy-request): ``` pscale deploy-request diff ``` Another benefit of the deploy request review process is that **PlanetScale will determine if certain requests aren't deployable**. For example, if you try to deploy a branch with no unique key, PlanetScale will block the deployment, as the unique key is required. ### 4. Add changes to the deploy queue Once a deploy request has been created and, optionally, approved, you need to add the changes to the deploy queue. Schema changes are deployed to a database in the order in which they are received. PlanetScale analyzes the schema changes in a deploy request when they are added to the deploy queue to ensure that the changes do not conflict with any of the queued schema changes. PlanetScale also provides insight on the deploy queue, listing all of the schema changes in the queue with their completion status. **Step 1** Organizations have the option to require approval before a deploy request can be added to the queue. If this is enabled, first make sure the request is approved. You can control this option in the database's "**Settings**" tab. **Step 2** To add a deploy request to the deploy queue, click “**Add changes to the deploy queue**” on the deploy request page. **Step 3** If successful, you'll get the message "These changes have been deployed". **Step 4** Your deploy request has now been merged into production. You can click on the "Deploy requests" tab of the database to see the list of previous deploys. Alternatively, you can run the following command with the [PlanetScale CLI](/docs/cli/deploy-request): ``` pscale deploy-request deploy ``` ## Default branches The `main` branch is automatically set as the default branch when the database is initialized. However, you can change the default branch if needed. **How to change the default branch**: There are two ways to change the default branch. **Step 1** Go to the "Branches" page in your dashboard. **Step 2** Click on the three dots "..." next to the branch whose name you'd like to change. **Step 3** Click "Set as default branch". If the current default branch has child branches, you also have the option to move those under the new default branch from here by checking the "Replace the current default branch" checkbox. Alternatively, you can do this from your Settings page. **Step 1** Go to your database dashboard page and click the "**Settings**" tab. **Step 2** Under "**General**" in the sidebar, you'll find the "**Default branch**" dropdown. **Step 3** Select the branch you want to be the default branch. It does not have to be a production branch. **Step 4** Scroll down and click "**Save database settings**". > [!NOTE] > > If you change the default branch, you will also need to update your credentials in your application, or wherever else you use credentials, to reference the new default branch (if desired). ## Renaming a branch You can change the name of a branch from the Branches tab in your PlanetScale dashboard. Click on the three dots ("...") next to the branch name that you would like to change, type in the new name, and click "Save changes". Renaming a branch does not affect that branch's credentials. You do not need to regenerate credentials if you rename a branch. ## Delete a branch You can delete a branch from the Branches overview page or by running the following command in the [PlanetScale CLI](/docs/cli/branch): ``` pscale branch delete ``` We recommend deleting branches after a deploy request is complete or if you are no longer using the branch for testing. Base plan development branches are billed only for the time that they are used, prorated to the millisecond, so it is beneficial to delete them when they are no longer in use. See the [billing documentation](/docs/planetscale-plans#development-branches) for more info. > [!NOTE] > > Only [Organization Administrators](/docs/security/access-control#organization-administrator) have permission to delete production branches. You cannot delete a branch that's [set as default](#default-branches). To delete, it set another branch as the default first. ## Resolve a schema conflict Schema conflicts occur when your branch has conflicting changes with the base branch. To resolve a schema conflict, create a new branch from the base branch, which will have the most up-to-date schema, and apply the necessary schema changes to the new branch before repeating the deploy process. ## Automatically copy migration data Many frameworks and migration tools keep track of data schema changes in a migration table. When turned on, PlanetScale will automatically copy migration table metadata from your development branches to the production branch as part of our deployment process. **Turn on automatic copying of migration data**: **Step 1** On your database dashboard page, click the "**Settings**" tab. **Step 2** Check the "**Automatically copy migration data**" box. **Step 3** Select one of the listed frameworks: Rails, Phoenix, Laravel, Django, .NET, Sequelize, or Other, which allows you to specify a custom table name. You can see how this works in this [Rails migration tutorial](/docs/vitess/tutorials/automatic-rails-migrations). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Column cannot be null Source: https://planetscale.com/docs/vitess/schema-changes/column-cannot-be-null Fix a deploy that fails while copying rows because a NOT NULL column still contains NULL. A deploy request can fail while copying a table because existing rows do not fit the new schema. PlanetScale copies the table in the background and inserts those rows into a new table that uses the deployed schema. If that schema makes a column `NOT NULL` and a row still has `NULL` in the column, the copy stops: ```text theme={null} vreplication: terminal error: task error: failed inserting rows: Column 'email' cannot be null (errno 1048) (sqlstate 23000) ``` The column name in quotes is the one to fix. The rest of the message is the insert that failed. The copy runs while your application keeps writing to the table. If the application inserts or updates a row with `NULL` in that column during the copy, the deploy fails with a slightly different message: ```text theme={null} vreplication: terminal error: error applying event: Column 'email' cannot be null (errno 1048) (sqlstate 23000) ``` In that case the application is still writing `NULL`. Update it to always set the column before retrying. ## How to fix it Update the rows that still have `NULL` in that column, then [retry the failed tables](/docs/vitess/schema-changes/deploy-requests#retry-a-partially-failed-deploy). ```sql theme={null} SELECT id FROM users WHERE email IS NULL; UPDATE users SET email = 'unknown@example.com' WHERE email IS NULL; ``` Use a value that is valid for the new column. Retrying before those rows are updated again will cause another failure. Retry starts the migration for the failed table over again. Tables in the same deploy that already finished copying are left as they are. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Column has no default value Source: https://planetscale.com/docs/vitess/schema-changes/column-has-no-default Fix a deploy that fails while copying rows because a new NOT NULL column has no DEFAULT. A deploy request can fail while copying a table because a column added by the deploy has no value for the existing rows. PlanetScale copies the table in the background and inserts existing rows into a new table that uses the deployed schema. Only the columns that exist in both tables are copied. If the new table adds a column that is `NOT NULL` and has no `DEFAULT`, the insert has nothing to put in it and MySQL rejects the row: ```text theme={null} vreplication: terminal error: task error: failed inserting rows: Field 'updatedAt' doesn't have a default value (errno 1364) (sqlstate HY000) ``` The column name in quotes is the one to fix. In plain MySQL, `ALTER TABLE ... ADD COLUMN` fills existing rows with an implicit default such as `0` or `''`. The copy uses `INSERT` statements under strict SQL mode instead, so there is no implicit default to fall back on. This also happens when a column is renamed. PlanetScale treats a rename as dropping the old column and adding a new one, so the new column has no values to copy. See [Handling table and column renames](/docs/vitess/schema-changes/handling-table-and-column-renames). ## How to fix it Change the column definition on your branch so existing rows have a value to copy: ```sql theme={null} -- Give the column a default ALTER TABLE users ADD COLUMN updatedAt datetime NOT NULL DEFAULT CURRENT_TIMESTAMP; -- Or allow NULL for now and tighten it later ALTER TABLE users ADD COLUMN updatedAt datetime NULL; ``` Then open a new deploy request. If the failed deploy is still in progress, cancel it first. Changing the schema on your branch does not update a deploy that has already started. If you allowed `NULL`, backfill the column after the deploy, then make it `NOT NULL` in a later deploy request. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Constraint not found Source: https://planetscale.com/docs/vitess/schema-changes/constraint-not-found A deploy fails because it drops a CHECK or FOREIGN KEY constraint by a name the production table no longer has. A deploy request can fail before copying starts because it drops a constraint by a name that does not exist on the production table: ```text theme={null} Found DROP CONSTRAINT: DROP CHECK `orders_amount_chk`, but could not find constraint name in map ``` MySQL requires `CHECK` and `FOREIGN KEY` constraint names to be unique across the whole schema. Because an online schema change builds a copy of the table alongside the original, PlanetScale gives every constraint on the copy a new name: the original name plus a generated suffix such as `orders_amount_chk_5vtaqz7kepok6wa91vryrkrje`. The suffix changes every time the table goes through a deploy. See [Foreign key constraint names change on every deployment](/docs/vitess/foreign-key-constraints#foreign-key-constraint-names-change-on-every-deployment). When the deploy runs, the `DROP` must name the constraint exactly as it exists on the production table at that moment. If the table was deployed again after this deploy request's changes were computed, or the name recorded for the production branch is out of date, the names no longer match and the deploy stops. ## How to fix it Check the current name on the production branch: ```sql theme={null} SHOW CREATE TABLE orders\G ``` Then open a new deploy request: **Step 1** Close or cancel the failed deploy request. **Step 2** Create a fresh branch from the production branch so it picks up the current constraint names. **Step 3** Apply your schema change on the new branch and open a new deploy request. If the new deploy request fails with the same error, contact support with the deploy request number. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Data Branching® Source: https://planetscale.com/docs/vitess/schema-changes/data-branching ## Overview The PlanetScale Data Branching® feature allows you to create isolated copies of your database that include both the schema and data. This differs from our [regular branching feature](/docs/vitess/schema-changes/branching), which only includes the schema. ## Enable the Data Branching® feature for your database. Before you can use the feature, you have to enable it in your database settings page. **Step 1** Navigate to the database that you'd like to enable. **Step 2** Click on the "**Settings**" link in the header navigation bar. **Step 3** Scroll to the option with the text "**Enable Data Branching®**" **Step 4** Enable this option and click "**Save database settings**". ## Create a new seeded branch **Step 1** After enabling the Data Branching® feature, navigate to the dashboard page of the database. **Step 2** Clicking on "**New Branch**" should now offer an option to select "**Seed Data**". * **None**: Creates a database branch with only the **schema** copied to the new branch. * **From most recent backup**: Creates a database branch with both the **schema and data** from the latest backup of the Base branch. ![PlanetScale dashboard new branch dialog with seed option](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/data-branching/branch.jpg?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=2f5a3a7f8ab31860712f14f90758ffd8) > [!WARNING] > > Branching off of a production branch and seeding it with data can incur additional charges, as branches seeded from production are treated as production branches instead of development branches. See [question 5](#faq) in the FAQ below for more information. 3. Once you've selected an option, click "**Create branch**" to create a new branch. [Data Branching](https://player.vimeo.com/video/830572488?dnt=true) ### FAQ > **Can I pick which backup is used for the new branch?** > > PlanetScale picks the latest backup so that your new branch has the latest dataset to work with. > **Can I pick which tables are copied into the new branch?** > > PlanetScale seeds the new branch with **all the schema & data** from the base branch. We do not offer a way to filter out data or schema when creating a new branch. > **Is data in the new branch kept up to date with changes to the base branch?** > > PlanetScale does not provide data syncing between a production branch and a development branch. > **When I merge my deploy request, will any data changes be merged back to the base branch?** > > PlanetScale does not provide data syncing between a production branch and a development branch. > **Will enabling the Data Branching® feature affect my billing?** > > Yes, it can. If you are branching and seeding from a *development* branch, the new branch will count towards your [development branch usage hours](/docs/planetscale-plans#development-branches) in the same way that regular branching does, but because you're seeding it with data, it will also be billed for storage. > > However, if you branch and seed from a *production* branch, the new branch will be created as a production branch at the same resource size as the source production branch, and will therefore be billed at that size. It will also be seeded with the production data and include the 2 default production replicas, so you will be charged for the storage as well for however long the branch stays running. > > For example, if you are branching off a `PS-80` production branch with 100 GB of storage, and you seed the new branch with that source production branch's data, you will be billed for the new seeded branch as follows: > > * `PS-80 — $179` > * `100 GB — ($0.50 * 100 GB) * 3 replicas = $135` > > We spin the new branch up using the same resource size as the production branch to ensure that large datasets initialize correctly and efficiently. After initialization, you may downgrade the branch size on the [Clusters page](/docs/vitess/cluster-configuration#adjust-your-cluster-size). All billing is prorated, so you will only be charged for the time your seeded branch is live. To avoid incurring extra charges, be sure to downsize your seeded branch, and delete it once you're done using it ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Deploy requests Source: https://planetscale.com/docs/vitess/schema-changes/deploy-requests Deploy requests are an integral part of the [PlanetScale workflow](/docs/vitess/best-practices). ## Overview Database branching, coupled with deploy requests, allows you to **deploy non-blocking schema changes to your production database with zero downtime**. You can also [undo deployments](#revert-a-schema-change) without losing any data that was written during that time. [Deploy requests overview](https://player.vimeo.com/video/830571933?dnt=true) ## Create a deploy request > [!NOTE] > > Before you can create a deploy request, the branch you are merging into must have [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled. **Step 1** Click on "**Branches**". **Step 2** Select the development branch you want to deploy into the base branch. **Step 3** This page shows you a diff of the schema against its base branch. **Step 4** To the right of the page, you'll see a dropdown that says "**Deploy to**". If you don't see this option, you likely don't have [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled on the branch you're deploying into. Go to the branch page for the branch you'd like to merge into, and enable safe migrations. **Step 5** Select the branch you want to deploy to. Again, if the branch doesn't show in the dropdown, you need to enable [safe migrations](/docs/vitess/schema-changes/safe-migrations). **Step 6** Optionally, add a comment about the deploy request. **Step 7** Click "**Create deploy request**". ![Example of deploy request on branch page](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/deploy-requests/deploy-request-page-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=470c972c0b7188a5b2f66ebfadbe02b0) ## Review a deploy request Once you create a deploy request, you or your team can review it and, optionally, approve it before deploying it. PlanetScale will check if the request is deployable. This process includes checking for issues like: * [Incompatible unique keys](/docs/vitess/schema-changes/onlineddl-change-unique-keys) * [Invalid charsets](/docs/vitess/schema-changes/schema-lint-errors/invalid-charset) (PlanetScale supports `utf8`, `utf8mb4`, `utf8mb3`, `latin1`, and `ascii`) * [Invalid foreign key constraints](/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-disallowed) * [Other schema lint errors](/docs/vitess/schema-changes/schema-lint-errors) that would prevent a successful schema change We will also warn you about potential data loss or inconsistencies and check if there are any known conflicts with the production schema that could prevent a clean merge. While we attempt to find all possible conflicts, it is ultimately up to you to confirm merge details. **Step 1** Click the "**Deploy requests**" tab on the database dashboard page. **Step 2** Select the open deploy request you want to review. **Step 3** Under "**Summary**", you'll see if the request is deployable. **Step 4** To review the schema changes, click the "**Schema changes**" tab. **Step 5** You'll see the proposed changes here. New additions are highlighted in green, and deletions are highlighted in red. **Step 6** If you have required deploy requests to be approved before deployment, other users in your Organization will see the option to "**Approve changes**" or "**Leave a comment**" on the "**Schema changes**" tab. > [!NOTE] > > If you are the only administrator in your Organization and you enable the "Require administrator approval for deploy requests" setting, you can self-approve your own deploy requests. If there is more than one administrator, self-approval is not allowed. ### Approvals dismissed on schema changes To ensure only approved schema changes are applied to production, an approval is automatically dismissed if the proposed changes are updated after the deploy request is approved. When this occurs, `planetscale-bot` comments on the deploy request with the list of changes since the last approval. If the deploy request is already in the deploy queue when the schema changes are updated, it will be removed from the queue. It must be re-approved and added to the queue again before it can deploy. ### Reviewing changes across shards If your deploy request contains changes to a sharded keyspace, you can see the affected shards by clicking the arrow next to each changed table. This will show the SQL that will run, and in the next tab, each shard that will be affected. ![PlanetScale deploy request - changes on sharded keyspace](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/deploy-requests/sharded-deploy-request.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=a083c1db1d2cd4c5fd283eef19cb6782) ## Deploy a deploy request By default, deploy requests on a branch run through a **serial deploy queue**: only one deploy runs at a time, and later requests wait until earlier ones finish. When a queue is already running, you can also [deploy in parallel](#parallel-deployments) alongside it. A deploy that cannot lock the table is usually due to long-running transactions. See [Long-running transactions](/docs/vitess/schema-changes/long-running-transactions). A copy can also fail because existing rows do not fit the new schema. See [Column cannot be null](/docs/vitess/schema-changes/column-cannot-be-null), [Column has no default value](/docs/vitess/schema-changes/column-has-no-default), [Duplicate entry](/docs/vitess/schema-changes/duplicate-entry), and [Value does not fit the new column](/docs/vitess/schema-changes/value-does-not-fit-column). If one migration in the deploy fails, later migrations stop until that one succeeds. See [Earlier migration failed](/docs/vitess/schema-changes/earlier-migration-failed). A deploy can fail before copying starts when the schema it was computed against no longer matches production, or when MySQL rejects the new table definition. See [Constraint not found](/docs/vitess/schema-changes/constraint-not-found) and [Foreign key missing unique key](/docs/vitess/schema-changes/foreign-key-missing-unique-key). If some tables fail while others are still copying or ready, the deploy stays in progress. See [Retry a partially failed deploy](#retry-a-partially-failed-deploy). **Step 1** Once the request is approved, if required, it's ready to deploy. Click on the "Summary" tab, and you'll see the option to deploy. **Step 2** Here you'll have the option to choose to "Deploy changes" or to "Deploy changes instantly". If [Prefer instant](#prefer-instant) is enabled and the request is instantly deployable, the button defaults to "Deploy changes instantly": ![PlanetScale deploy request - deploy options](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/deploy-requests/deploy.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=15afd661edfa9b23aa6ac7efc09fa32e) ### Deploy changes **Step 1** (Optional) You have the option to enable [**gated deployments**](#gated-deployments), which gives you the power to control exactly when the migration cuts over. You'll see an "**Auto-apply changes**" checkbox, which is checked by default. If you uncheck this, you will get the option to apply the changes once the schema changes are complete. If you leave it checked, it will auto-deploy as soon as it's ready. **Step 2** (Optional) You also have the option to run the migration using MySQL's **ALGORITHM=INSTANT**. Click the arrow next to "Deploy changes" and select "Deploy changes instantly". If [Prefer instant](#prefer-instant) is enabled, eligible deploy requests default to this option. Gated deployments are not available for this option. See the [Deploy changes instantly](#deploy-changes-instantly) section below for more information. **Step 3** When you're ready to deploy, click "**Deploy changes**". The deployment will begin immediately, or join the serial deploy queue if other deployments are already pending. If a queue is already running, you can instead choose [**Deploy in parallel**](#parallel-deployments) to run alongside it. You will still have a short window after deploying to enable [gated deployments](#gated-deployments) in this step. **Step 4** If you enabled gated deployments (step 2), you can click "**Apply changes**" to merge the deployment to production once it completes. **Step 5** After you deploy, you have **30 minutes to "undo"** it using our [schema revert feature](#revert-a-schema-change). **Step 6** If you are deploying changes to a sharded keyspace, you'll see the progress for each shard here as well. ### Deploy changes instantly You also have the option to use MySQL's **ALGORITHM=INSTANT** to instantly deploy the schema change. Learn more in the [**Instant Deployments** section](#instant-deployments). If [Prefer instant](#prefer-instant) is enabled, eligible deploy requests default the deploy button to this option. 1. When you're ready to deploy, click "**Deploy changes instantly**". The deployment will begin immediately, or join the serial deploy queue if other deployments are already pending. * If it joins the serial queue, it deploys instantly once it reaches the front of that queue. * You can also deploy instantly [in parallel](#parallel-deployments) alongside an existing queue. * Instant deployments **cannot be reverted**. If you would like to require an administrator's approval before a request can be deployed, go to the "**Settings**" page for your database and check the "**Require administrator approval for deploy requests**" box. You must be an Organization Administrator to enable this restriction. Please note you will not be able to approve your own deploy requests. ## Retry a partially failed deploy Online schema changes copy each affected table before cutover. If some tables fail while others are still copying or ready, the deploy stays in progress. The deploy page shows that it has partially failed and cannot continue. You can retry the failed tables, or cancel the deploy request. ### Retry the failed tables Retry recopies only the failed tables, in migration order. Tables that already finished stay as they are. **Step 1** Open the deploy request that has partially failed. **Step 2** Click "**Retry**". **Step 3** The button changes to "**Retry requested**". If a table fails again after that, "**Retry**" is available to try again. From the [PlanetScale CLI](/docs/cli/deploy-request): ```bash theme={null} pscale deploy-request retry ``` ### Cancel the deploy request Cancel the deploy request on the deploy page. This discards every table in the deploy, including copies that already finished. From the CLI: ```bash theme={null} pscale deploy-request cancel ``` ## Close a deploy request If you decide you don't want to proceed with a deploy request, you can easily close it. **Step 1** Click the "**Deploy requests**" tab on the database dashboard page. **Step 2** Select the request you want to close. **Step 3** Click on the "**Close deploy request**" button on the right-hand side. ## Deploy requests and foreign key constraints In most cases, deploy requests should work as expected when your schema changes have [foreign key constraints](/docs/vitess/foreign-key-constraints). > [!WARNING] > > A deploy request that modifies a table with many foreign key child tables may cause queries to block when the InnoDB adaptive hash index (AHI) is enabled. See [Adaptive hash index](/docs/vitess/foreign-key-constraints#adaptive-hash-index). There are some cases where a deploy request will not be deployable. This includes cases where there is a mismatched column type or when a foreign key constraint references a deleted column. For example, if we open a deploy request to add a foreign key constraint `t1_id` with type `BIGINT` on a table `t2` that references a column `id` on table `t1`, where `t1.id`'s type is `BIGINT`, the following cases would produce a linting error in the deploy request because it is not deployable: * if, while the previously mentioned deploy request is open, someone else updates `t1.id` to a different column type, i.e., `int`. * if, while the previously mentioned deploy request is open, someone else deletes `t1.id`. * if, while the previously mentioned deploy request is open, someone else deletes all indexes that cover `t1.id` as their prefix. (Because in a foreign key relationship, the referenced columns on the parent table must be indexed, usually by a dedicated index, but they can be the first columns in an otherwise wider index.) These are all cases where another user changes schema, causing the initial user's definition to be invalid MySQL. There are also two cases where a revert would cause orphaned rows that you can read about in this document's [revert section](#when-a-revert-can-result-in-orphaned-rows). ### Validating referential integrity of existing columns Deploy requests do not validate the referential integrity of *existing* columns. `ALTER TABLE… ADD FOREIGN KEY…` does not validate existing row relations within the context of a deploy request. Unlike standard MySQL, it is possible to add the foreign key constraint to a table with orphaned rows, and they will remain orphaned. In standard MySQL, adding a foreign key is a blocking operation, and it fails if any orphaned rows are found. ## Instant deployments Instant deployments give you the option to run schema changes using MySQL's **ALGORITHM=INSTANT**. This is different than how our [**online schema migrations**](/docs/vitess/schema-changes/how-online-schema-change-tools-work) work. Instant deployments will apply schema changes faster, however, these schema changes must be **auto-applied** and **cannot be reverted**. ### Who should use instant deployments? Instant deployments are well-suited to experienced users who want their schema change to take effect immediately rather than going through the online schema migration process. > [!WARNING] > > Although instant deployments complete near-instantaneously in most cases, they require briefly acquiring an exclusive metadata lock on the affected tables. To prevent the deployment from being blocked, instant deployments will immediately terminate any queries and transactions holding locks on the target table when they run. > > On a busy table this can mean a burst of failed application queries at the moment of deployment. Ensure your application has retry logic for terminated queries. > > In rare cases, if something prevents the instant deployment from acquiring or releasing the metadata lock, application writes to the table will be blocked until the lock is released. > > For details on which operations support `ALGORITHM=INSTANT` and their limitations, see the [MySQL InnoDB online DDL operations documentation](https://dev.mysql.com/doc/refman/en/innodb-online-ddl-operations.html). ### Supported operations In order for a deploy request to be instantly deployed, *all* schema changes in the deploy request must be instantly deployable. Some of those changes include: * Adding or dropping a column (with some exceptions) * Changing or dropping a column's default value * Changing an `ENUM` or `SET` definition The following changes are examples of changes that are **not** instantly deployable: * Changing a column's data type * Adding a column with a non-literal default value * Adding or dropping an index * Altering the visibility of an index * Adding or dropping a foreign key constraint * Extending a `VARCHAR` column size * Updating a column to `NULL` or `NOT NULL` To know whether or not a deploy request is instantly deployable, look for the "Instantly deployable" badge on your deploy request. This badge will only be visible on deploy requests that can be deployed instantly. ![PlanetScale deploy request - deploy instantly badge](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/deploy-requests/deploy-instantly.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=6c60531060cf8770ddcee94ebfb7a422) We recommend reading [MySQL's Online DDL documentation](https://dev.mysql.com/doc/refman/en/innodb-online-ddl-operations.html) for the full list of operations that can be deployed instantly. ### Prefer instant Admins can enable **Prefer instant** on the database **Settings** page under **Advanced settings**. When enabled, deploy requests that are instantly deployable default the deploy button to **Deploy changes instantly**. You can still choose **Deploy changes** from the dropdown. The setting applies to all future deploy requests that have not started deploying. It does not change in-progress regular deploys. CLI and API deploys are unchanged. Pass [`--instant`](/docs/cli/deploy-request) with `pscale deploy-request deploy`, or set `instant_ddl` when [queueing a deploy request](/docs/api/reference/queue_deploy_request). Before enabling this as a default, review [supported operations](#supported-operations) and [who should use instant deployments](#who-should-use-instant-deployments). Instant deployments cannot be reverted and can terminate queries holding locks on the table. ## Gated deployments Gated deployments give you more control over when a migration goes live after the deployment process completes. As part of our non-blocking schema change process, instead of directly modifying table(s) when you deploy a deploy request, we make a copy of the affected table(s) and apply changes to the copy. We get the data from the original table and the copy table in sync, and once complete, initiate a quick cutover where we swap the tables. > [!NOTE] > > If a deploy request includes changes to multiple tables, all tables cut over at the same time — unless there is a sequential dependency. With gated deployments, you can initiate the deployment, but once the table syncing is complete, we'll hold off on the cutover and let you click a button to swap the tables and complete the deployment. Gated deployments can be enabled on each deploy request by unchecking the "Auto-apply changes" box before you deploy. This feature is helpful if you have long-running migrations. For very large or complex databases, deploying a schema change can take several hours to complete. In those scenarios, you don't want the cutover to happen while you're offline. With gated deployments, you can start the deployment process by adding your deploy request to the queue, and once it's done, you'll be able to click a button to merge it in and complete the deployment while you're there to monitor it. ### Enable gated deployments in the dashboard **Step 1** When you open a deploy request, uncheck the "**Auto-apply changes**" box. **Step 2** ![PlanetScale deploy request - Auto-apply changes checkbox unchecked](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/deploy-requests/gated-deployments-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=934d81816eec779827ee80ffebc44491) Once your deploy requests begins running, you'll also have the option to uncheck the box here. **Step 3** When your deploy request has completed and is ready for cutover, the "**Apply changes**" button will appear. You can now complete the deployment at any time by clicking this button. ### Enable gated deployments via CLI You can also manage auto-apply settings using the [PlanetScale CLI](/docs/cli/deploy-request). To create a deploy request with auto-apply disabled: ```bash theme={null} pscale deploy-request create --disable-auto-apply ``` To disable auto-apply on an existing deploy request: ```bash theme={null} pscale deploy-request edit --disable-auto-apply ``` Once your deploy request has completed and is ready for cutover, trigger the swap: ```bash theme={null} pscale deploy-request apply ``` > [!NOTE] > > If neither `--enable-auto-apply` nor `--disable-auto-apply` is provided when creating a deploy request, the setting is inherited from the previous deploy request. ### Limitations * An open gated deployment holds the **serial** deploy queue until you apply (or cancel) it. Later serial deploy requests wait behind it. You can still [deploy in parallel](#parallel-deployments) alongside a gated deployment if the changes touch different tables. * Deploy requests that are [instantly deployed](#instant-deployments) cannot be gated. For more information about this process and why we built it, check out the [Gated Deployments: Addressing the complexity of schema deployments at scale](https://planetscale.com/blog/gated-deployments-addressing-the-complexity-of-schema-deployments-at-scale) blog post. ## Parallel deployments By default, deploy requests on a branch run one at a time through the serial deploy queue. Each deploy waits for the one ahead of it to finish before it starts. **Parallel deployments** let you run two deploy requests at once: one in the serial queue, and one alongside it in a parallel lane. Put a slow change in the parallel lane and keep shipping faster ones through the queue, or leave slow work in the queue and run a quick change in parallel. Either way works, as long as the changes touch different tables. ### Deploy in parallel When a deploy queue is already running on the branch, the deploy request Summary tab shows a deployment strategy choice: * **Add to deploy queue**: joins the serial queue and waits its turn (default) * **Deploy in parallel**: starts now, alongside the queue Select **Deploy in parallel**, then confirm. The deploy request page and the database deploy queue view show parallel deploys in a separate lane from the serial queue. You can also deploy in parallel from the [PlanetScale CLI](/docs/cli/deploy-request): ```bash theme={null} pscale deploy-request deploy --strategy parallel ``` Use `--strategy serial` (or omit `--strategy`) to join the serial queue. If a deploy request is already waiting in the serial queue, remove it from the queue and deploy it again with the parallel strategy. Everything else about the deploy (review, schema diff, gated cutover, and revert) works the same way as a serial deploy. Reverting a parallel deploy only undoes that deploy's changes. It does not rewind work the serial queue already completed, and it does not pause the serial queue. ### Limits * **One parallel deploy at a time per branch.** If a parallel deploy is already running, you must wait for it to finish before starting another. * **No overlapping table changes across lanes.** A parallel deploy and any deploy in the serial queue cannot modify the same table at the same time. The deploy is rejected if there is overlap. ### Performance considerations > [!WARNING] > > Two deploys at once can **increase database load**. Impact depends on the database, available resources, and workload. Avoid running two resource-intensive deploy requests at the same time. For example, two large but quiet tables may be fine together, while busy tables or deploy requests that each change many tables can put more pressure on the database. You can independently adjust the [throttler settings](/docs/vitess/schema-changes/throttling-deploy-requests) for each deploy request to control the impact on the database. ## Artifact tables During schema changes, Vitess creates artifact tables to facilitate non-blocking migrations. For detailed information about artifact table behavior, storage implications, and cleanup processes, see the [Artifact tables in schema changes](/docs/vitess/schema-changes/artifact-tables) documentation. ## Revert a schema change If you ever merge a deploy request, only to realize you need to undo it, PlanetScale can handle that! You have the option to revert a recently deployed schema change while maintaining data that was written to the original schema during that time. > [!NOTE] > > Deploy requests that are instantly deployed *cannot* be reverted. ### How to revert a schema change You can revert a deployment for **up to 30 minutes** after the deploying. After the 30 minute period is up, the deployment becomes permanent, and you will no longer have the option to revert. [Revert a schema change](https://player.vimeo.com/video/830571822?dnt=true) **Step 1** Select the deploy request you want to revert. **Step 2** To revert the schema changes made with the deploy request, click "**Revert changes**" and confirm. **Step 3** We will immediately revert the base branch back to its previous schema. **Step 4** Any data that was written to the original schema in the time between deploying and reverting will remain in your database after the revert. **Step 5** The deploy request will be closed, but the branch will remain for you to continue development on if you choose. ### When is data not retained There are some scenarios where some data is not retained when you revert your changes. 1. You add a table or column to your schema and then revert it. If any data was written to those newly introduced fields between deployment and reverting, that data will not be retained upon revert, as the fields will no longer exist. ### When a revert can result in orphaned rows In some cases, when you are using foreign key constraints, a revert of a deploy request can result in orphaned rows. These can happen when your schema change is: * Dropping a foreign key constraint: Once a foreign key constraint is dropped, new data written to the table is less constrained. Reverting this change may result in data that is inconsistent with the dropped foreign key constraint. * Dropping a table with foreign key constraints: When a table with foreign key constraints is dropped, the parent table(s) will continue to be written to. If this change is reverted, data in the table that was dropped may no longer be consistent with its foreign key constraints. > [!NOTE] > > You must enable [foreign key constraint](/docs/vitess/foreign-key-constraints) support in the database settings page before using them. ### When are you unable to revert a schema change There are also some edge cases where reverting a schema change is not possible. We will always attempt to revert, but if there are scenarios where your data integrity is at risk, we will not proceed with the revert. The following are some cases where a revert will fail: 1. If you deploy a schema change that expands the length of some column, such as changing from `VARCHAR(10)` to `VARCHAR(50)`, and add new data larger than 10 characters to it, a revert attempt may fail. This is to protect your data. You may have written data to the `VARCHAR(50)` field in that time that will not fit in the smaller 10 character space. If no data is added between deployment and revert, the revert process can proceed. 2. Some examples of other similar scenarios where revert won't be possible (again, only if larger sized data is added between deployment and revert) are: * `INT` to `BIGINT` * `NOT NULL` to `NULL` * `TIMESTAMP` to `TIMESTAMP(6)` * `utf8` to `utf8mb4` * Any other operation that expands the size of a field 3. If you deploy a schema change that removes a unique key or relaxes a unique constraint, and in the time between deployment and attempting to revert, you insert rows that would otherwise conflict with that constraint, the revert may fail. 4. Another uncommon but possible scenario: you deploy a schema change that has a `NOT NULL` column without a `DEFAULT` value, combined with an `ALTER TABLE DROP COLUMN` statement for that column. If you insert some rows between the deployment and the revert attempt, the revert will fail. We will not be able to re-add that column for the newly inserted rows and will not know how to populate it. For an in-depth look at how this process works, check out our [Behind the scenes: how schema reverts work](https://planetscale.com/blog/behind-the-scenes-how-schema-reverts-work) blog post. ### Schema revert and migration data If you've selected a migration framework or specified a table with migration data in the settings tab of your database, the data within the table that tracks migrations will be moved to the production branch only after the revert window has been closed. This is to ensure that if the deploy request is reverted, the production branch has the correct log of applied migrations. ### Billing considerations You may see some temporary `_vt` tables in your database. These are called [artifact tables](/docs/vitess/schema-changes/artifact-tables) and are used to facilitate the deployment and revert process. They do not count toward your storage costs if you're using [network-attached storage](/docs/plans/planetscale-skus#network-attached-storage). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Duplicate entry for unique key Source: https://planetscale.com/docs/vitess/schema-changes/duplicate-entry Fix a deploy that fails while copying rows because existing rows violate a new unique key. A deploy request can fail while copying a table because existing rows do not satisfy a unique key in the new schema. PlanetScale copies the table in the background and inserts existing rows into a new table that uses the deployed schema. If that schema adds a `UNIQUE` key, or changes the primary key, and two existing rows have the same value for those columns, the second insert is rejected: ```text theme={null} vreplication: terminal error: task error: failed inserting rows: Duplicate entry 'acme-1' for key 'uniq_org_priority' (errno 1062) (sqlstate 23000) ``` The first quoted value is one of the duplicates. The key name tells you which columns are in conflict. The key name may be shown with a `_vt_vrp_...` prefix, which is the name of the table being copied into. ## How to fix it Find the rows that share a value for the columns in the key, then remove or change them so each combination is unique: ```sql theme={null} SELECT org_id, priority, COUNT(*) FROM packaging_rules GROUP BY org_id, priority HAVING COUNT(*) > 1; ``` Then [retry the failed tables](/docs/vitess/schema-changes/deploy-requests#retry-a-partially-failed-deploy). Retrying before every duplicate is resolved will cause another failure. If the duplicates are expected, change the key on your branch so it is no longer unique, or add a column to it that makes each row distinct, and open a new deploy request. > [!NOTE] > > The copy runs while your application keeps writing to the table. If the application inserts a duplicate during the copy, the deploy fails with the same error. Make sure new writes also satisfy the unique key before retrying. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Earlier migration failed Source: https://planetscale.com/docs/vitess/schema-changes/earlier-migration-failed A later table in a deploy request stops when an earlier migration in that deploy has failed. PlanetScale runs the migrations in one deploy request in order. If an earlier migration fails or is cancelled, every later migration in that deploy stops: ```text theme={null} migration bbbbbbbb_bbbb_bbbb_bbbb_bbbbbbbbbbbb cannot run because prior migration aaaaaaaa_aaaa_aaaa_aaaa_aaaaaaaaaaaa in same context has failed/was cancelled ``` This table did not fail on its own. The migration named after `prior migration` is the one that failed. Later tables stay blocked until that earlier migration succeeds. ## How to fix it Find the first failed table in the deploy request and fix that error. Then [retry the failed tables](/docs/vitess/schema-changes/deploy-requests#retry-a-partially-failed-deploy). Retry recopies failed tables in migration order. Tables that already finished stay as they are. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Foreign key missing unique key Source: https://planetscale.com/docs/vitess/schema-changes/foreign-key-missing-unique-key A deploy fails because a foreign key references columns that are not covered by a unique key. A deploy request can fail before copying starts because a [foreign key constraint](/docs/vitess/foreign-key-constraints) references columns on the parent table that are not covered by a unique key: ```text theme={null} Failed to add the foreign key constraint. Missing unique key for constraint 'fk_assessments_group' in the referenced table 'student_groups' (errno 6125) (sqlstate HY000) ``` MySQL 8.0 allowed a foreign key to reference any indexed column, even when the index was not unique or the foreign key covered only a prefix of it. MySQL 8.4 rejects both by default. The referenced columns must be the `PRIMARY KEY` or a `UNIQUE` key on exactly those columns. ## How to fix it Add a unique key on the referenced columns of the parent table, then deploy again: ```sql theme={null} ALTER TABLE student_groups ADD UNIQUE KEY uniq_student_groups_id (id); ``` If the referenced column is not unique by design, point the foreign key at the parent table's primary key instead, or drop the constraint and enforce the relationship in your application. See [Strategies for maintaining referential integrity](/docs/vitess/strategies-for-maintaining-referential-integrity). Make the change on your branch, then open a new deploy request. If the failed deploy is still in progress, cancel it first. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Handling table and column renames Source: https://planetscale.com/docs/vitess/schema-changes/handling-table-and-column-renames ## Overview In SQL, it is possible to `RENAME` either a column or a table using an [`ALTER` statement](https://dev.mysql.com/doc/refman/8.0/en/alter-table.html). This keeps the data and type of the column or entire table the same and changes only the name. PlanetScale does not support this method of table and column renames. If you do need to rename a column or table, the following section covers how you should do it with PlanetScale using [Deploy Requests](/docs/vitess/schema-changes/deploy-requests). ## How to rename a column on PlanetScale **Step 1** Create a new column with the new name. **Step 2** Update the application to write to both columns with new data. **Step 3** Backfill all the data in the new column for rows that are still missing that information. **Step 4** Optionally, add constraints like `NOT NULL` to the new column once all the data is backfilled. **Step 5** Update the application to only use the new column, and remove any references to the old column name. **Step 6** Drop the old column. This means at least two deploy requests are needed (potentially more if you want to enforce `NOT NULL` without a `DEFAULT`), where you first add the newly named column and then drop the old one. ## Why not support renames? There are two reasons why renames are not supported. The first reason is that PlanetScale uses a declarative model for determining the changes between schemas. This means that there is no way to know for sure if something is a rename or if it was an add and drop column. The second reason is that safely performing a rename requires downtime for your application. ### Declarative model for schema migrations PlanetScale uses a declarative model to determine schema changes. This means that we look at the end state of two branches and compare them to find the difference. When you rename a column, there is no way to know whether there was a rename or if a column was added and dropped in a development branch. It would be possible to apply heuristics to determine if something looks like a rename, but this is not 100% guaranteed to be correct. For example, if a column is renamed and a second column is dropped adjacent to the one that was renamed, it is impossible to determine which column to rename with certainty. Because of this, we only detect simple cases that look like a rename and alert the user on that in a deploy request. While it would be possible to generate a `RENAME` in such a case, we don't know if that is correct, and do not want to execute any schema changes that are not requested. ### Downtime with renames When using a `RENAME`, downtime for your application can only be avoided with significant and complex logic on the application side. While the application is running, suddenly a column disappears and another one appears that contains the same data. This means that the application both needs to be able to handle the state when this happens and know that the data is the same. Usually the only way to handle a column rename is to simultaneously deploy the application, but that is very difficult as it races with the database changes and is not perfectly atomic and synchronized with the schema change. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # How Online Schema Change tools work Source: https://planetscale.com/docs/vitess/schema-changes/how-online-schema-change-tools-work ## The schema change problem Changing a table's schema is one of the most challenging problems in relational databases, and in MySQL in particular. In today's accelerated and rapid development cycles, engineers find that they need to make schema changes sometimes on a daily basis. Direct `ALTER TABLE` is a blocking operation, which renders the table completely inaccessible, even for reads. MySQL's InnoDB engine supports Online DDL for most operations. However, it is only *online* on the primary. On replicas, it executes sequentially to other statements, leading to unacceptable replication lag. On the primary, it is aggressive and uncontrollable. It will compete with production traffic over resources. InnoDB also offers Instant DDL, which is available for a small subset of schema change operations. See the full breakdown on the [MySQL documentation](https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html). ## Schema change solutions In today's world, the vast majority of users use one of the Online Schema Change tools. These are external, small tools or scripts, that hook onto the database and run a migration through emulation. Common external tools are [gh-ost](https://github.com/github/gh-ost), [pt-online-schema-change](https://www.percona.com/doc/percona-toolkit/3.0/pt-online-schema-change.html) and [Facebook's OSC](https://web.archive.org/web/2023/http://bazaar.launchpad.net/~mysqlatfacebook/mysqlatfacebook/tools/annotate/head:/osc/OnlineSchemaChange.php). This document describes the general technical operation of these tools. It's worth noting an alternative to using Online Schema Change tools, via rolling changes on replicas. In this technique, the user runs either straight or Online DDL on replicas, typically one or a few replicas at a time. Then, once all replicas have been updated with the new schema, the user demotes the primary and promotes one of the replicas. The demoted primary, now a replica, can then be updated, as well. This technique requires no external tooling. However, it takes substantially longer to complete the overall operation and requires running a promotion, which is typically disruptive to the applications. ## The general logic behind Online Schema Change tools The Online Schema Change tools have many differences, covered in our [Comparison of Online Schema change tools doc](/docs/vitess/schema-changes/online-schema-change-tools-comparison). However, they all share a similar fundamental logic and flow. All tools work by *emulating* a schema change. The general flow is: * Create a new, empty table, in the likeness of the original table. We title this the *ghost* table. * `ALTER` the ghost table. Since the table is empty, there is no overhead to this operation. * Validate the structural change is compatible with tooling requirements. * Analyze the diff. * Begin a long running process of copying existing rows from the original tables to the ghost table. Rows are copied in small batches. * Capture or react to ongoing changes to the original table, and continuously apply them onto the ghost table. * Monitor general database and replication metric, and throttle so as to prioritize production traffic as needed. * When the existing data copy is complete, the migration is generally considered as ready to cut-over, give or take some small backlog or state of the replication topology. * Final step is the cut-over: renaming away of the original table, and renaming the ghost table in its place. Up to some locking or small table outage time, the users and apps are largely ignorant that the table has been swapped under their feet. In the following sections, we break down these steps to understand better the requirements, limitations, and correctness of the flow. ## Initializing the ghost table schema The schema migration flow never changes or updates the original table. We only ever operate on the *ghost* table. We create a ghost table with an exact copy of the original table's schema, then we run the user's `ALTER TABLE` command on the ghost table. First and foremost, we are able to validate that the statement is valid. For example, if the user requests an `ALTER TABLE foo DROP COLUMN bar`, executing that statement on the ghost table *successfully* validates that the column `bar` did, in fact, exist in the first place. Similarly we can catch any lower level errors such as an oversized index, invalid `DEFAULT` values, etc. This is also a place to check whether we want to enforce a change to `AUTO_INCREMENT`. ## Validating the change On top of validating the raw MySQL change, all Online Schema Change tools come with their own restrictions. A couple notable restrictions are: * **Existence of relevant keys** — All tools copy data from the original table onto the ghost table by slow iteration. The iteration utilizes a `UNIQUE KEY`, ideally the `PRIMARY KEY`. The different tools have somewhat different constraints, but they all require some form of `UNIQUE KEY` or `PRIMARY KEY` to exist on the ghost table, maybe even require a *shared key* between the two tables. The existence of a key on the target table is required to satisfy the following: * When applying an ongoing change on the original table, for example an `UPDATE`, how do we efficiently find the row on the target table? * When copying a chunk of data from the original table onto the ghost table, how do we resolve conflicts? By way of example, by the time we copy the last chunk of data, we may have already captured and applied the `INSERT` change, so that the rows already exists on the ghost table. More on that logic later on, but it is the `UNIQUE KEY` that helps us resolve the conflict. * **Existence/addition of `FOREIGN KEY` constraints** — Foreign keys do not play well with the Online Schema Change flow, and there are either severe limitations on, or rejection of, tables that have foreign key constraints. ## Analyzing the diff The tools compare the two tables to determine what makes them similar and what makes them different. Most importantly, the tools need to understand which columns are shared between the two tables that should be copied from the original table to the ghost table. The tools need to take into consideration a potential rename of a column, or whether columns are `GENERATED`, or turned from `GENERATED` to normal columns or vice versa. ## Copying rows This is the single most time consuming part of the migration. Transferring all rows from one table to another can be a long running task even if running on a completely stale server. On large tables in production, the row copy could take as long as hours, days or even weeks. Copy is done via a statement of the form: ```sql theme={null} INSERT [IGNORE] INTO () SELECT FROM WHERE ``` It is impractical, or inefficient, to copy all rows from the original table to the ghost table in one pass. The tools work by iterating the original table start-to-end using some chosen `UNIQUE KEY` (typically the `PRIMARY KEY`). They only copy `n` rows at a time, where common values are around `100` or `1000` rows per chunk, though based on row data size and on workload, these can also be on the range `10...10000`. We reasonably know where the table starts. But where does it "end"? Consider a table with an `AUTO_INCREMENT PRIMARY KEY` column. As we work to copy the table rows, the app writes more rows into the table, pushing the end farther and farther. This creates an illusion of a never ending flow. However, this is not the case. The tools evaluate the first and last rows (as ordered by the iteration key), and only copy rows in that range. Any excessive rows beyond that range will be captured by following the ongoing changes. It is imperative then to begin capturing changes *before* evaluating row range. Copying, say, `100` rows at a time will very likely yield with an image inconsistent with a single snapshot of all rows. As we copy the first `100` rows, new rows can be added, removed or updated in the next `100` rows range. As we advance in our copy phase, the data gets more and more skewed from the original imaginary snapshot from the migration starting point. That is in fact not a problem, as the skew serves us to reflect an even more updated image of the data. We are interested not in the data snapshot at the *start* of the migration, but at the *end* of it. We want the ghost table to be aligned with the original table at such point when we can swap the two without data loss. Applying the ongoing changes is of course the key to achieving that. More on the interaction of row copy and ongoing changes later on. It's simple to consider how we may iterate a table with, say, a `AUTO_INCREMENT PRIMARY KEY`. It's easy to imagine iterating rows `[1..100]`, `[101..200]`, `[201..300]` and so forth. But we must then also consider real life situations: * We might have huge punctures in the `AUTO_INCREMENT` sequence. Rows `200..5000000` may be altogether missing, and iterating that range could be extremely wasteful. * Can we miss out on unforseen gaps? If we copied row `100`, does it follow that the next row is `101`? Could there be some `100.5`? Not if the data type is an `INT`, but what if it's another type? * Our iteration key may not necessarily be numeric. It may in fact contain multiple columns. How can you iterate a `PRIMARY KEY (file_name, submitted_at)` where `name` is a `VARCHAR` and `submitted_at` is a `TIMESTAMP`? We illustrate below the *pseudo code* used by Online Schema Change tools to iterate a table using a key with any number and type of columns. For clarity, let's assume two columns `(file_name, submitted_at)`; the logic remains similar, though somewhat more detailed, the more columns we iterate by. **Step 1** Grab the minimum value via: ```sql theme={null} SELECT file_name, submitted_at FROM original_table ORDER BY file_name, submitted_at ASC LIMIT 1 INTO @first_entry_file_name, @first_entry_submitted_at` ``` **Step 2** Grab the maximum value via: ```sql theme={null} SELECT file_name, submitted_at FROM original_table ORDER BY file_name DESC, submitted_at DESC LIMIT 1 INTO @last_entry_file_name, @last_entry_submitted_at` ``` **Step 3** Assign `(@previous_iteration_end_entry_file_name, @previous_iteration_end_entry_submitted_at) := (@first_entry_file_name, @first_entry_submitted_at)` and repeat. **Step 4** For any iteration: evaluate the next range: ```sql theme={null} SELECT file_name, submitted_at FROM ( SELECT file_name, submitted_at FROM original_table WHERE (file_name, submitted_at) > (@previous_iteration_end_entry_file_name, @previous_iteration_end_entry_submitted_at) AND (file_name, submitted_at) <>= (@last_entry_file_name, @last_entry_submitted_at) ORDER BY file_name, submitted_at LIMIT 100 ) ORDER BY file_name DESC, submitted_at DESC LIMIT 1 INTO @this_iteration_end_entry_file_name, @this_iteration_end_entry_submitted_at ``` In words, get the last `file_name, submitted_at` values counting `100` rows starting immediately after end of previous iteration, but also don't exceed our "end" position. **Step 5** Follow up with copy: ```sql theme={null} INSERT INTO ghost_table () SELECT FROM original_table WHERE (file_name, submitted_at) > (@previous_iteration_end_entry_file_name, @previous_iteration_end_entry_submitted_at) AND (file_name, submitted_at) <= (@this_iteration_end_entry_file_name, @this_iteration_end_entry_submitted_at) ORDER BY file_name, submitted_at ``` The two queries could have been executed as one, but by first evaluating the range using only the key columns, this process is actually more efficient. **Step 6** Assign `(@previous_iteration_end_entry_file_name, @previous_iteration_end_entry_submitted_at) := (@this_iteration_end_entry_file_name, @this_iteration_end_entry_submitted_at)` and repeat. > [!NOTE] > > The above pseudo-code is just a simplification of the process. There are several extra considerations that go into the full process, such as: * Care needed to capture the very first row in iteration. * Missing from the above is identifying the stop condition (empty row set found). * MySQL actually performs poorly with tuple comparison. The condition `(a, b) > (x, y)` must actually be phrased as `a > x OR a = x AND b > y`. This explodes fast the more columns are at play. * Most implementations will actually use a `INSERT IGNORE` rather than `INSERT`, to play well with ongoing updates. ## Ongoing changes Tools will capture ongoing changes by either utilizing database triggers or by hooking into the replication stream. This way or another, the changes must be applied to the ghost table. As with row copy, we are only interested in copying a subset of columns that are shared between the tables. Again, care needed to consider renamed or `GENERATED` columns. The ongoing changes always represent the latest and most up-to-date image of the data. As such, they must override any prior data already captured on the ghost table. As an example, consider we have just copied our first iteration of `100` rows, and then observe an `UPDATE` on, say, row `32`. This update must override whatever value we have already copied into the ghost table for row `32`. To this effect, row copy and applying changes take lower and higher priorities, respectively, over conflicts. Some of the tools achieve this like so: * Row copy only issues a `INSERT IGNORE INTO ... SELECT ... FROM ` * Changes applied via `REPLACE INTO ...` Consider a counter example to our previous scenario. Imagine we're now intercepting a change to row `1024`, but row copy has not reached that row yet. We apply the change. When we finally reach and copy the range that includes row `1024`, that specific row does *not* get overwritten. Now, what if by that time of row copy the row has been updated again? Isn't the row copy now even more up-to-date than the existing `1024` row on the ghost table? It is, in fact. But if that is the case, then there must have been a change, an `UPDATE` to row `1024`, that we just haven't applied yet, and are yet to discover and apply via `REPLACE INTO`. Different tools tackle this issue differently. A synchronous solution like `pt-online-schema-change` is *always* up-to-date with ongoing changes, while [Vitess's VReplication](https://vitess.io/docs/reference/vreplication/vreplication/) takes a completely different approach by marking GTID positions that have been or not have been already covered. The above should serve as a good illustration to the flow, regardless of the actual implementation. ## Monitor and throttle Most tools run some form of monitoring over the affected server or of its replicas, or both. Either sequentially to the copying process or concurrently, the tools may choose to throttle based on one or more metrics. The single most important metric, by far, is replication lag. How the tools discover the replicas, choose which replicas to consider, respond to topology changes, is a matter of implementation and of a different discussion. However, the tools that do observe replicas all try to maintain as low replication lag as possible, and will step back once that lag grows. What the threshold is, is configurable. On extremely busy and sensitive systems this may be as low as `1` second. Other environments may tolerate *minutes* of lag. Not all tools can throttle completely. Specifically, trigger based tools can only throttle the row copy process, but not the ongoing changes tracking. Non-trigger based tools are able to completely halt any processing, or choose to take lower precedence over production traffic. For more information about trigger based vs non-trigger based tools, check out our doc on [Comparison of online schema change tools](/docs/vitess/schema-changes/online-schema-change-tools-comparison#trigger-based-vs-triggerless). ## Determining when cut-over is possible Generally speaking, in a normal and successful migration flow, the migration is ready to cut-over once row-copy is complete. Different tools may still have some backlog to process. Under heavy workload, the backlog may take substantial time to process. However, the general state of the replication topology can also be a factor. It is a good idea to only cut-over when replication lag is minimal, to avoid schema confusion between primary and replicas. Also, the current workload on the primary plays an important part, as discussed in the next section. ## Cutting over By far the most critical point of the migration, this is where the changes take effect and the new, changed schema, tastes first production traffic. It's first important to realize some migrations do not go as planned. Some drop the wrong column, or change the wrong index, and such is life. The cut-over point is where the user might expect trouble. When some queries suddenly fail, or others suddenly perform poorly. Some of the tools will cut-over automatically when they deem the migration is complete, and others will allow the user to choose the timing. Replacing the table is also a point of contention. MySQL-wise, we need to acquire a table lock, also known as a metadata lock. As internal MySQL implementation goes, one cannot acquire a metadata lock while a query still runs on the table. This means a long running query may block the cut-over for a substantial time. This is why it's good to either consult with the running workload, or let the user choose the cut-over time. Once metadata lock is acquired, application connections start noticing something goes on. Depending on the cut-over implementation, the metadata lock can be held up to a few seconds. During that time more queries will pile up, leading to increased number of connections on the server. It's not unreasonable that the number of connections will surpass the configured database connections threshold, at which time no new connections and queries are allowed. This is again obviously noticeable to the apps. Trigger based tools suffer from an even increased metadata lock, and under heavy workloads this can cause seconds or minutes long stalls. Depending on the implementation, the tables will either swap atomically, such that the connections are suddenly unblocked and "magically" proceed to run on the new (previously *ghost*) table; or, there will be a "puncture" in time, where the table ceases to exist, which leads to query errors for a brief time. At long last, the migration is complete with the *ghost* table assuming the role of the original table. Production traffic proceeds to run on the new table. The old table is then normally left for a while, to be dropped later on by the database engineer. ## Beyond cut-over Most tools stop their operation and their interest at cut-over. Vitess, in particular, goes beyond that point to: * allow the migration to be *reverted* * clean up the left over tables For more information about how Vitess handles schema changes, check out the [Vitess Schema Changes documentation](https://vitess.io/docs/user-guides/schema-changes/). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # How to make different types of schema changes Source: https://planetscale.com/docs/vitess/schema-changes/how-to-make-different-types-of-schema-changes Your database has to grow and change with your application and product. ## Overview So schema changes are inevitable. Luckily, PlanetScale has features like [branching](/docs/vitess/schema-changes/branching), [deploy requests](/docs/vitess/schema-changes/deploy-requests), and [revert](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change) to help give you more confidence when making schema changes. But you still have to consider though how to make the schema change. Some schema changes are more complex than others. This document will help you understand how to make different types of schema changes and associated risks. > [!NOTE] > > Most of the following applies to any relational database and not only to PlanetScale. ## Adding a table Adding a table is a low risk since it does not affect the existing schema. Once you add the table to your schema and deploy the database to production, if you want to backfill the data, now would be the time. Otherwise, you add it to your application code after completing your database deployment. Deploying the database first and then changing your application code to avoid downtime is best. ## Adding a column Adding a column without a `NOT NULL` constraint is a low risk and can be handled similarly to [adding a table](#adding-a-table). The one exception is if you want to add a column with a `NOT NULL` constraint and/or no `DEFAULT` value. The risk increases because the previously inserted rows will be expected to be `NOT NULL` or have no default values; this will cause possible errors in your application. To work around this, you can do the following: **Step 1** Add the new column without defining a DEFAULT value, allowing NULL values. **Step 2** Write a script to backfill the new column for all missing data rows. **Step 3** Add the `NOT NULL` constraint once the data is backfilled. ## Changing a column or table Some examples of changing a column or table include: * Renaming an existing column or table * Changing the data type of an existing column * Splitting and other modifications to the data of an existing column or table This includes modifying your application to use a different column or table in existing code. For example, if you are using `username` in the code but instead want to start using a new `user_id` column in the existing code. Changing a column or table your current application uses is a high risk. It can cause disruptions to users and possible downtime. If you want to change a column or table, we recommend following the expand, migrate, and contract pattern described in the [backward compatible schema changes](https://planetscale.com/blog/backward-compatible-databases-changes) blog post. The blog post walks you through each step in both your database and application code and gives a detailed example. ## Removing a column or table ### In use Removing a column or table your current application uses is a high risk. Dropping a column or table can cause disruptions to users and possible downtime. If you want to remove a column or table in use, we recommend following the expand, migrate, and contract pattern described in the [backward compatible schema changes](https://planetscale.com/blog/backward-compatible-databases-changes) blog post. The blog post walks you through each step in both your database and application code and gives a detailed example. ### Not in use Removing a column or table that is no longer in use by your current application is a low risk. The only danger is potentially needing the removed data in the future. ## Other schema changes This is not an exhaustive list of schema changes. Remember, making smaller incremental schema changes is always safer than larger, more complex ones. If you want to read more on making safer schema changes with PlanetScale, read this blog post on [safely making database schema changes](https://planetscale.com/blog/safely-making-database-schema-changes). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Long-running transactions Source: https://planetscale.com/docs/vitess/schema-changes/long-running-transactions Long-running transactions on your database can cause deploy requests to fail. When you deploy a schema change, PlanetScale copies the table in the background so the original stays available. That copy needs a brief lock on the table: once at the start, and again as copying continues. The last step, [cutover](/docs/vitess/schema-changes/aggressive-cutover), needs a lock too. A deploy that cannot get the lock fails with a lock wait timeout. The statement after `during query` shows which step was waiting: ```text theme={null} Lock wait timeout exceeded; try restarting transaction (errno 1205) (sqlstate HY000) during query: lock tables orders read ``` * `lock tables ... read` is the lock at the start of the copy. * `RENAME TABLE ...` is the cutover. * `CREATE TABLE ...` or `ALTER TABLE ...` on a `_vt_vrp_...` table is PlanetScale building the copy. These wait on a lock when the table has [foreign key constraints](/docs/vitess/foreign-key-constraints) and a transaction is open on the parent or child table. ## How to fix long-running transactions If Vitess is unable to get a lock on the table, it will fail the deploy. This is most commonly caused by a long-running transaction that is still open on the table. For example, if your app opens a transaction, updates the table, and then doesn't commit or rollback, Vitess will be unable to get a lock on the table. Having this query pattern in low volume is generally not a problem. But if you have a high volume of these queries, it can cause Vitess to not be able to get a lock on the table. To fix this, you need to: **Commit or rollback the transaction as soon as possible** ### External service calls Another common anti-pattern is to open a transaction, update the table, and then wait on additional API calls (such as calls to an external service). This can cause the transaction to be held open for a long time, and can prevent the deploy request from completing. In these cases, we recommend moving the external API calls to outside of the transaction. ## Additional tips * The [aggressive cutover setting](/docs/vitess/schema-changes/aggressive-cutover) only helps after the copy phase has finished. If your deploy request is stuck in the copy phase, this setting will not help. * If the change is [instantly deployable](/docs/vitess/schema-changes/deploy-requests#instant-deployments), Instant deploy skips the copy, so it doesn't need this lock. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Comparison of Online Schema Change tools Source: https://planetscale.com/docs/vitess/schema-changes/online-schema-change-tools-comparison Online Schema Change tools for MySQL [share a similar basic design](/docs/vitess/schema-changes/how-online-schema-change-tools-work), but differ in implementation. Their differences imply different capabilities, different impact to production, different versatility, and different user experience. In this document, we break down several Online Schema Change tools to see how their design choices lead to different behavior. ## Tools reviewed We will compare these solutions: * [pt-online-schema-change](https://www.percona.com/doc/percona-toolkit/3.0/pt-online-schema-change.html) as part of Percona Toolkit. * [gh-ost](https://github.com/github/gh-ost) * [Facebook's OSC](https://web.archive.org/web/2023/http://bazaar.launchpad.net/~mysqlatfacebook/mysqlatfacebook/tools/annotate/head:/osc/OnlineSchemaChange.php) * [Vitess](https://vitess.io/docs/user-guides/schema-changes/) Some notes before we take off: * All reviewed solutions are free and open source * `pt-online-schema-change`, `gh-ost`, and `Facebook's OSC` are designated (command line) tools for making schema changes. * Vitess is a full blown sharding and infrastructure framework, where online schema changes (termed Online DDL) is one of its functions. As such, it is not an "online schema change tool" per se. In the context of Vitess's Online DDL, we assume all production traffic goes through Vitess's MySQL interface (the `VTGate` proxy), and never directly to the underlying MySQL servers. * It's noteworthy that Vitess can manage and run `gh-ost` and `pt-online-schema-change` migrations. In this comparison we look at the native Vitess Online DDL solution, based on [VReplication](https://vitess.io/docs/user-guides/schema-changes/ddl-strategies/#onlinevreplication). * We discuss the original Facebook online schema change tool. Facebook (now Meta) have since [redesigned and re-implemented](https://engineering.fb.com/2017/05/05/production-engineering/onlineschemachange-rebuilt-in-python/) the tool. * The original online schema change tool, [oak-online-alter-table](https://shlomi-noach.github.io/openarkkit/oak-online-alter-table.html), as part of the OpenArk kit, is discontinued and is not reviewed here. `pt-online-schema-change` derives its design from it and the two share many properties. * Another notable tool not covered here is [Rails LHM](https://github.com/soundcloud/lhm). For simplicity of naming, we will refer to `pt-online-schema-change` as `pt-osc`, and to Facebook's OSC as `fb-osc`. ## Comparison by design This document compares the online schema change solutions by key design elements, and shows how those affect capabilities and usability. We do not compare benchmarks or specific performance metrics. We will share some production experiences, while taking care not to make blanket statements. Each tool behaves differently under different workloads. Our breakdown compares these design principles: * Synchronous vs asynchronous * Trigger-based vs. triggerless * Coarse vs. fine * Atomic vs. punctured cut-over * Controlled vs. arbitrary cut-over schedule * Primary vs. shared vs. flexible key iteration * Managed vs. unmanaged * Auditable/controllable vs. not * Resumable after failure vs. not * Revertible after completion vs. not * Declarative vs. imperative Make sure to first read [How Online Schema Change tools work](/docs/vitess/schema-changes/how-online-schema-change-tools-work). ## Synchronous vs asynchronous In a synchronous design, the ghost table is updated within the same transaction that updates the original table. If a new row is `INSERT`ed to the original table, then within that transaction we also `INSERT` a row on the ghost table, and similarly for `UPDATE` and `DELETE`. By virtue of transactions, either both statements apply, or neither does. This design simplifies much of the logic. Once the table copy is complete, the synchronous design ensures the tables are in perfect sync, and will continue to be so if we were to keep the process going. The only way to achieve this design is by delegating work to the database itself, so that we're able to hook into a user's transaction. It is impossible for an external tool to apply changes to the ghost table while still operating under the user's transaction. The only means to do that in current MySQL is using table triggers (see next design comparison). In our reviewed solutions list, `pt-online-schema-change` is the only one to use the synchronous approach. In an asynchronous design, we apply changes to the ghost table asynchronously to the user's transaction. This means there is some *lag* between the user's `COMMIT` and the time at which the reflected change is applied to the ghost table. That lag can be small or high, and is always non-zero. Even at the moment when table copy is complete, there is no certainty that the two tables will be in complete sync. Ongoing production traffic may keep pushing the lag. The asynchronous approach implies we need to capture the changelog. We need some buffer where we can track this `INSERT` or that `DELETE`, which can sustain a growing lag. The two approaches to achieving asynchronous schema changes are: * Capture data change via triggers and use a helper table for capturing the changelog — used by `fb-osc`. * Tail the MySQL binary logs, which *are* the changelog — used by `gh-ost` and `Vitess`. However the changelog is captured, a background process needs to iterate the incoming changes, to then apply them onto the ghost table. ## Trigger-based vs. triggerless Triggers are MySQL constructs based on stored routines, which allow us to respond to `INSERT`, `UPDATE` and `DELETE` statements, either before or after they have been applied to the table. A trigger is invoked per row changed (created, updated or deleted) on a specific table. The invocation takes place in the same transaction space as the query which operated on the table. Triggers make it easy to keep things in the database level. Applying an `UPDATE` to the ghost table does not leave the server's space. This means we can let MySQL handle the way data is copied. For example, if the schema change converts a column type from `INT` to `BIGINT`, or from `enum` to `VARCHAR`, we don't need to worry about the type conversion, and we let MySQL do its natural conversion. Stored routines were introduced in MySQL `5.0` and have changed little since. A stored routine (procedure, function, event, trigger) is interpreted by the database. As opposed to other RDBMS, MySQL does not compile or pre-compile stored routines. Every time a stored routine is invoked, the code is interpreted by the database server. This means a user's transaction gets overloaded, or padded, with interpreter logic, and on high workloads this is noticeable to the app. Another limitation of the trigger-based design is that the triggers must be active throughout the lifetime of the migration. Online schema change solutions attempt to keep migration impact to production as low as possible, and throttle based on production load. However, the triggers must keep running, since they are the means to capturing changes, or else we have data loss. They cannot be throttled. Both `pt-osc` and `fb-osc` use triggers, but in different approaches: * `pt-osc` uses the synchronous approach. Therefore, an `INSERT` on the original table uses the trigger to also `INSERT` into the ghost table in that same transaction. While this maintains perfect sync between the two tables, it also introduces excessive locking. While normal transactions only intend to compete over writes to the original table, with the added triggers they also have to compete over writes to the ghost table. On high workloads this can bring down application's ability to write to the table to a considerable slowdown or even to a grinding halt. Again, this is workload dependent and for some workloads this works well. * `fb-osc` uses the asynchronous approach. The tool creates a changelog table, an append-only buffer. Each `INSERT`, `UPDATE` or `DELETE` on the original table translates to an `INSERT` on the changelog table. This reduces somewhat the locking contention since all changes map to new (and sequentially ascending) rows in the changelog table rather than existing (and random order) rows on the ghost table. Even though `fb-osc` uses the asynchronous approach, it still keeps the data in the database itself, and again enjoys MySQL's ability to reliably copy the data from the changelog table onto the ghost table. In the triggerless approach, the changelog cannot be acquired directly from the running transactions, and must be collected elsewhere. Both `Vitess` and `gh-ost` use a triggerless design, and both get the changelog from MySQL's replication stream. With binary logs enabled in MySQL, once a transaction completes its effects are written to the binary logs, and are made available to the replication stream. Notes: * How the transaction is written in the binary logs depends on the binlog format. Both `gh-ost` and `Vitess` require a `ROW` binlog format. `gh-ost` is more flexible in allowing a `STATEMENT` format on a primary with `ROW` format on a replica. * Depending on MySQL's configuration, the transaction and binary log entry couple be written atomically, or, there could be lag between the `COMMIT` and the binlog entry flush. The binary log is, in essence, the changelog of operations made on the MySQL server. It is the means for replication, where replicas replay those changes to achieve consistency with the primary MySQL server. It is possible to read the binary logs directly from disk, but both `Vitess` and `gh-ost` choose to hook to the MySQL server imposing as replicas, and requesting the binary log events over the wire. This allows flexibility in where the schema change logic can run. Reading data from the binary logs / stream means the schema change solution owns the interpretation or mutation of the data. It is no longer possible to delegate that work to MySQL. For example, texts are always written in UTF in the binary logs, regardless of the character set they originated with in the table. Conversions between types (again illustrating an `enum` to `varchar` change) need to be taken care of by the solution's logic. This makes a triggerless design more complex in handling of data and the numerous ways with which it can mutate. A triggerless design is inherently asynchronous. Capturing binlog changes from MySQL's replication stream, or even from disk, is not coupled with transaction commit time. There will be some lag between the time where an `UPDATE` is in the binary log stream, and time at which it is read from the stream. Read more on why `gh-ost` took a [triggerless approach](https://github.com/github/gh-ost/blob/master/doc/why-triggerless.md). ## Coarse vs. fine `pt-osc`, `fb-osc` and `gh-ost` all work in a coarse approach. To some extent it's a brute-force method of copying the data: * They iterate the original table and copy chunks of rows to the ghost table. * They respond to ongoing changes (whether sycnhronously or asynchronously) and apply them to the ghost table. The two operations could conflict. What if we applied a row by capturing the changelog, before we copied it via table iteration? What do we do when we then try to copy that chunk of data? In a synchronous operation, what if that actually happens concurrently in competing transactions? The coarse approach gives the changelog a higher priority over the table copy, and lets both run independently of each other. In a normal migration, there will be many conflicts between the row-copy and the changelog. They are resolved using database queries, simplified to: * `INSERT IGNORE INTO ghost_table (...) SELECT ... FROM original_table` for table copy, and * `REPLACE INTO ghost_table (...) ...` for changelog capture. Noteably `gh-ost` only ever runs table copy or applies changelog at a time, and there is never a competition between two concurrent transactions. `Vitess` takes a fine approach where it keeps track of: * **Executed `GTID`** — `Vitess` uses `GTID` while writing to the ghost table. It keeps a very small progress table to which it audits the `GTID` for the latest write to the ghost table. This is done atomically within the same transaction. * **Last copied range** — `Vitess` evaluates, programmatically, which key range has already been copied. The key range is again persisted in a helper table, atomically with the write to the ghost table. Vitess operates in a three step loop: * **Copy** — opens a transaction with `CONSISTENT SNAPSHOT` while atomically grabbing the `GTID` at time of the transaction. Begins streaming rows from the table. * **Catchup** — scans the changelog. * Ignore anything not related to the original table * Ignores anything that is already covered by the `GTID` captured above. * Ignores operations on rows which have not been iterated yet. * **Fast forward** — prepare the next Copy phase, providing it the next unhandled row range, and, since takes non-zero time to achieve, backfill the remaining changelog events as with the Catchup change. With this fine grained approach, Vitess is able to avoid superfluous writes to the ghost table, and only keep to the minimum necessary data copy. ## Atomic vs. punctured cut-over The cut-over is the most critical part of the migration process. It is when the new schema takes effect, and is a source of concern: * If we did anything wrong (dropped the wrong index? Wrong column?), that's when the application may break or degrade performance. * There is non-zero impact time for instating the new table. Queries will either stall, break or both for a hopefully brief period. `pt-online-schema-change` and `gh-ost` both utilize an atomic cut-over. In this approach, the app or user's queries on the original table are briefly *blocked*, but not rejected, during cut-over time. When the ghost table is renamed in place of the original table, the queries get *unblocked* and proceed to operate on the new table. On high workloads this is typically noticeable to the app. There will be some delays to queries. Moreover, there will be some pileup of connections. This is because an open connection on MySQL can only run a single query at a time. A `1000` blocked queries will mean a `1000` open connections. The busier the workload is, the higher number of piled up connections will be. Eventually, the database may run out of open connection limit (either by its own configuration or by OS limits), at which time apps and users will experience rejected connections or queries. `pt-osc`'s cut-over is the simplest of all. Becuase of its synchronous design, it is able to cut-over with a single two-table rename statement, such as `RENAME TABLE original TO _original_old, ghost_table TO original`. However, the metadata lock acquired also applies to the triggers, and under heavy workload the cut-over becomes a dangerous point of contention that can lock for long seconds or minutes. `gh-ost` uses the asynchronous approach, where there is some lag between the original and ghost tables. It implements an [elaborate](https://github.com/github/gh-ost/blob/master/doc/cut-over.md) logic to block queries on the original table while still applying last backlog changes to the ghost tables. Users have shown that under very specific query sequences, in particular monitoring-like queries that happen to access the ghost table just before/at cut-over time, there are scenarios that introduce data loss. `gh-ost` also supports a two-step cut-over, as illustrated next. `fb-osc`, also using an asynchronous approach, tackles the problem by creating a puncture. It runs a two-step flow where it first renames the original table away, and then, after processing the backlog, renames the ghost table in its place. There's a period in time where the table just does not exist. During that period in time, queries will fail, unexpectedly to the app/users. It is a simple and effective solution to the asycnhronous complexity, at the cost of user/app inconvenience. `Vitess` utilizes the fact that traffic goes through its own `VTGate` and `VTTablet` components. Behind the scenes, it creates a puncture, much like `fb-osc`. But during that brief time, it buffers any read/write queries that operate on the migrated table, up to a threshold. The user and app will experience an atomic, blocking cut-over. Queries will see increased latency and will block until cut-over is complete. Beyond some concurrent connections threshold, Vitess will reject queries, and the apps will notice an error. ## Controlled vs. arbitrary cut-over schedule In an arbitrary cut-over schedule design, the Online Schema Change solution decides when to cut-over, and it does so in the earliest possible opportunity. `pt-osc` an `fb-osc` take that approach. `pt-osc` will cut-over as soon as table copy is complete, and `fb-osc` will do the same, allowing the extra brief time for processing the remaining backlog. Since cut-over time is the most critical part of the migration, there is advantage to having people at their console and in their comfortable working hours when it takes place. Waking up at `2:00am` on a Saturday night is less than ideal for dealing with outages and incidents. `gh-ost` and `Vitess` support arbitrary schedule, but both also allow the user to decide when to cut-over. In this controlled approach, the migration process does not terminate voluntarily by the tool/solution. After table data is copied, and even when all seems to be in order, migration simply continues running. The tools keep tracking and applying the changelog to keep the ghost table in near sync (both `gh-ost` and `Vitess` use the asynchronous approach). At some point, the user may declare they are ready. Assuming conditions allow, or as soon as possible afterwards, the tools will proceed to run the cut-over. * `gh-ost` provides a [-postpone-cut-over-flag-file](https://github.com/github/gh-ost/blob/master/doc/command-line-flags.md#postpone-cut-over-flag-file) command line flag, and cut-over either when the file is removed or the user interactively commands the cut-over. * `Vitess` migrations [can be postponed](https://vitess.io/docs/user-guides/schema-changes/postponed-migrations/) with `-postpone-completion` flag, and completed via `ALTER VITESS_MIGRATION COMPLETE`. Controlled cut-over is a means for higher confidence and peace of mind for the operational user. ## Primary vs. shared vs. flexible key iteration Original table's data is copied to the ghost table by slowly iterating all rows. That takes place according to some order, and, in particular, according to some `UNIQUE KEY` order. Ideally, we'd always iterate by `PRIMARY KEY`, but sometimes the `PRIMARY KEY` itself is being modified by the schema change, which requires us to look for alternatives. Similarly, when we capture, say, an `UPDATE` event in the changelog, we need to uniquely identify the relevant row in the ghost table. There must be some way to ensure we're addressing the correct row, even in face of a change in `PRIMARY KEY`. The Online Schema Change tools differ in implementation as follows: * `fb-osc` will only ever iterate on `PRIMARY KEY` order. It allows original and ghost tables to have different `PRIMARY KEY`s, but required the `PRIMARY KEY` on the ghost table to only cover columns which exist in the original table. * `pt-osc` allows any type of `UNIQUE KEY`. Any change of `PRIMARY KEY` requires the user to add `--no-check-alter`. It allows the ghost table to have `UNIQUE KEY`s on columns not present in the original table, and will in such case use unindexed columns for `DELETE` operations, which is a major performance hit on large tables. * `gh-ost` requires some [shared key](https://github.com/github/gh-ost/blob/master/doc/shared-key.md). There must be any `UNIQUE KEY` (`PRIMARY` included) on the original table, which covers exact set of columns, and in same order, as some `UNIQUE KEY` on the ghost table. The name of the key does not matter, just the identity and order of covered columns. * `Vitess` requires the original table to have some `UNIQUE KEY` and the ghost table to have some `UNIQUE KEY`. the two do not have to be the same. Vitess requires columns covered by the original table's key to exist in the ghost table, and columns covered by the ghost table's key to exist in the original table. ## Auditable/controllable vs. not Both `fb-osc` and `pt-osc` are non-auditable and uncontrollable. Once they start running, they're on auto-mode and will pursue the migration to completion or until it fails. It's impossible to meanwhile control the tools, change their behavior, query for information etc. `Vitess` and `gh-ost` are both controllable and auditable. `gh-ost` opens a Unix socket, by which the user may communicate with the running migration. It is possible to reconfigure `gh-ost` while running (by e.g. setting new throttling thresholds), manually throttling it, getting status, cutting-over and more. `Vitess` lets the user communicate with the migration via SQL, with such commands as `SHOW VITESS_MIGRATION LIKE '...'` (get information and status), `ALTER VITESS_MIGRATION '...' CANCEL` (abort a running migration), `ALTER VITESS_MIGRATION '...' COMPLETE` (cut-over a postponed migration) and more. Both `gh-ost` and `Vitess` provide extra information about the migration, such as estimated progress percent as well as ETA (expected time till completion). ## Managed vs. unmanaged The tools `pt-osc`, `fb-osc` and `gh-ost` are concerned with making an `ALTER TABLE` (or emulation thereof) happen, and are not concerned about how the migration was made or scheduled, or what its consequences might be. Management of the migration flow must take place externally to the tools. The tools usually require multiple command line flags to pass information such as the server location, credentials, throttling information and thresholds, etc. `Vitess`, as a framework, manages many aspects of the migration: * **Scheduling** — Vitess schedules migrations to run, identifies migrations that conflict with each other, moves migration to the next state, etc. * **Discovery** — Vitess automatically knows where the migration should execute. * **Credentials** — Vitess can create and destroy credentials per migration (and does so for managed `gh-ost` and `pt-osc`), or use its own internal credentials for the task. * **Throttling** — built in Vitess, and again based on internal discovery. * **Cleanup** — safely and timely removing the migration's artifacts. * **History** — tracking all present and past migrations. ## Resumable after failure vs. not `gh-ost`, `fb-osc`, `pt-osc` are single process tools and do not keep track of their own progress outside the process' scope. If the tools break for any reason, or killed by the operating system, the migration is lost. They cannot be resumed from point of interruption. Someone will then also need to clean up the artifact tables. With `pt-osc` and `fb-osc`, there is furthermore the issue of dropping the leftover triggers. Moreover, they are bound to the MySQL primary the migration was issued on. If the primary fails, the migration is lost. Even if, for example, a replica is promoted as a new primary and the MySQL cluster resumes operation, it is impossible to resume the migration from point of breakage. [Vitess migrations survive](https://vitess.io/docs/user-guides/schema-changes/recoverable-migrations/) both failure of the Vitess process as well as failure of the MySQL server. This is the result of Vitess keeping track of the migrations' progress in helper tables, transactionally committed along with corresponding range and GTID values. Upon failure of either MySQL or Vitess itself, a new Vitess/VTTablet process is able to pick up the migration from the exact place where it left off. This works even if a new MySQL replica/server is promoted as primary. Vitess resumes the flow automatically and the user does not need to take action. ## Revertible after completion vs. not Some migrations turn bad. Perhaps dropping the wrong column or the wrong index. This is why the cut-over is the most critical part of the migration. In such situations, the operator may want to revert the migration, i.e. restore the table to its original format. Once a migration is complete, `pt-osc`, `fb-osc` and `gh-ost` exit. What information they had about the migration or the table's structure is lost. Since they all have no sense of migration management, there is no functionality to assist with reverting the migration. The user must contrive a new `ALTER TABLE` statement and run a new migration, or alternatively find what the original table was renamed to, and rename it back, losing some data accumulated since migration completed. Vitess migrations are [revertible](https://vitess.io/docs/user-guides/schema-changes/revertible-migrations/). Vitess is able to restore the table onto its original schema, without losing accumulated data. Furthermore, it requires no additional information from the user. It provides a `REVERT VITESS_MIGRATION ` SQL interface, which creates a new migration. That migration is expected to run quickly, as it does not need to copy table data. It only needs to hook onto the binary log stream from the point of previous migration termination and apply data from that point on. ## Declarative vs. imperative `pt-osc`, `fb-osc` and `gh-ost` require an `ALTER TABLE` statement. This statement is pre-evaluated by the user (or an ORM) and must be valid. The tools have no notion of the purpose nor semantics of the change. `Vitess` likewise supports `ALTER TABLE` statements. In addition, it also supports a declarative approach. In this approach, the user issues a `CREATE TABLE` to indicate what the table *should look like*. When the migration is scheduled to run, `Vitess` compares existing schema with desired schema, computes the required `ALTER TABLE` and runs the migration. Possibly the table is already in the desired format and no changes are needed. Possibly the table does not exist in the first place and needs to be created. `Vitess` will change the schema as needed to get to the desired state. [Declarative migrations](https://vitess.io/docs/user-guides/schema-changes/declarative-migrations/) are available via the `-declarative` strategy flag. ## Conclusion As you can see, there are several crucial differences when it comes to the implementation of Online Schema Change tools. Hopefully this guide gave you insight into the reasoning behind the design choices of the tools we reviewed. If you're interested in learning more about Vitess schema changes, be sure to check out the [Vitess documentation](https://vitess.io/docs/user-guides/schema-changes/). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Online DDL change unique keys Source: https://planetscale.com/docs/vitess/schema-changes/onlineddl-change-unique-keys It is possible to modify or replace a table's `PRIMARY KEY`, or any other `UNIQUE KEY`s according to the limitation described below, followed by examples. ## Overview To migrate data safely and [without downtime](/docs/vitess/schema-changes), PlanetScale requires that all tables have a unique, not-null key. Note that a `PRIMARY KEY` satisfies this condition, and it is generally recommended to always have a `PRIMARY KEY` on all tables. When you modify a table, both the old and the new schema must have a unique key as described, and the columns covered by those keys must exist in both the old and the new schema. Essentially this makes it possible for PlanetScale to unambiguously identify and correlate a row between the two schemas. If you attempt to deploy a schema change which does not comply with the above restriction, the deploy request will fail with the error `Table ... has no shared columns covered by non-null unique keys between both branches.`. ## Examples: allowed changes In our examples, we assume the base schema to be: ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`id`) ); ``` The following are all valid changes to the schema: ### Expanding the PRIMARY KEY ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`id`, `other_info`) ); ``` In the above we modified the `PRIMARY KEY` to include `other_info`. This is allowed since both `id` and `other_info` columns exist in both the old and the new schema. ### Moving PRIMARY KEY to a different column ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`email`) ); ``` Since both `id` and `email` columns exist in both old and new schema, the deploy request will be allowed. The success of the operation depends on whether `email` actually contains unique values. If there's duplication in `email` values, the deployment will fail with error. ### Moving PRIMARY KEY to different columns Likewise, there is no problem if the new `PRIMARY KEY` covers multiple columns. Again, the success of the operation depends on the actual uniqueness of the combination of columns. ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`username`, `other_info`) ); ``` ### Changing PRIMARY KEY and adding/removing other UNIQUE KEYs ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`username`, `other_info`), UNIQUE KEY `email` (`email`) ); ``` ## Examples: invalid changes Consider the next scenarios and the ways to work around them: ### Changing a PRIMARY KEY to include a new column ```sql theme={null} CREATE TABLE `users` ( `id` int, `other_info` int, `username` varchar(128), `email` varchar(128), `new_info` int, PRIMARY KEY (`username`, `new_info`) ); ``` This is an invalid change because in the new schema, the `PRIMARY KEY` covers the `new_info` column. But this column does not exist in the old schema. Consider splitting into two distinct schema changes and deploy requests: 1. First, introduce the `new_info` column. 2. Next, change the `PRIMARY KEY`. ### Changing a PRIMARY KEY and also dropping the old covered column ```sql theme={null} CREATE TABLE `users` ( `other_info` int, `username` varchar(128), `email` varchar(128), PRIMARY KEY (`email`) ); ``` The above is invalid because `id` column, covered by the `PRIMARY KEY` in the old schema, does not exist in the new schema. Again, consider splitting into two distinct changes: 1. First, change the `PRIMARY KEY`. 2. Next, drop the `id` column. ## Summary We've seen how, in many scenarios, it's straightforward to modify your table's `PRIMARY KEY` or other keys. For some scenarios, it might take two or more steps to achieve the new schema. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Safe migrations Source: https://planetscale.com/docs/vitess/schema-changes/safe-migrations Safe migrations is an optional but highly recommended feature for branches in PlanetScale. With safe migrations enabled on a branch, you’ll gain zero-downtime schema migrations, schema reverts, and protection against accidental schema changes. The safe migrations setting is recommended for all production database branches to prevent downtime and unintentional data loss during schema migrations. ## Zero-downtime schema migrations Safe migrations enable the [PlanetScale workflow](/docs/vitess/best-practices) on a given branch and allow your team to create deploy requests to merge schema changes into that branch. When changes are merged using deploy requests, a ghost table will be created with the desired schema changes. Your data will be continuously synchronized with that table until you decide to apply the changes. ## Schema revert Safe migrations and deploy requests provide the option to quickly [revert schema changes](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change) if you discover that they are not compatible with your application. With schema revert enabled for a database, the old table is retained. You’re provided a 30-minute window where data will still be synchronized as writes occur on the new table. If you decide to revert your changes, the status of the two tables is flipped, bringing the former table back. ## Protection against accidental schema changes To prevent accidental changes to the database schema, which may cause downtime, safe migrations enforce the use of [branching](/docs/vitess/schema-changes/branching) and [deploy requests](/docs/vitess/schema-changes/deploy-requests). This requires that changes be made safely and allows all team members to check and comment on schema changes before they are applied. With safe migrations enabled, Data Definition Language (DDL) statements issued to branches with safe migrations enabled will automatically be rejected by PlanetScale. Any `CREATE`, `ALTER`, or `DELETE` commands, whether sent using the PlanetScale built-in console, terminal, or MySQL GUI, will fail when we receive them. ## Staging branches You can use a development branch with safe migrations enabled to set up a workflow with a “staging” branch. First, make sure you have safe migration enabled for your main production branch. Then, create a “staging” branch with your main production branch as the base and turn on safe migrations. All new branches created for development can use this “staging” branch as the base branch. You can then open a deploy request against either the main production or “staging” branch. Once it is deployed to “staging,” you can open a deploy request against the main production branch. The main production branch must be set as the default branch (found in your database's “Settings” page) to open a deploy request against it. > [!NOTE] > > In this setup, the “staging” branch is still a development branch. Compared to your production branch, it will have reduced resources, similar to other development branches. ![View of the Branches tab with main \<- staging \<- dev branches](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/safe-migrations/branches-with-staging-branch.jpg?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=f7ddf5ad9f99a86f2121909ef57e6783) ## How to enable safe migrations Safe migrations can be enabled using the PlanetScale dashboard or the pscale CLI. ### Using the PlanetScale dashboard To enable safe migrations on a branch, select the branch you want to modify from the branch dropdown and click the **”cog”** in the upper right of the infrastructure card on the ”**Dashboard**” tab of the database. ![The production branch UI card.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/safe-migrations/production-branch-card-with-sm-disabled-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=e69aef44ce888729b22a81aa59cca2d2) In the modal, toggle the option labeled **”Enable safe migrations”**, then click the **”Enable safe migrations”** button to save and close the modal. The UI card will reflect the status of the safe migrations for that branch. ![Branch UI card with safe migrations enabled.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/safe-migrations/production-branch-card-with-sm-enabled-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=7cbcf6b5ea23eeceacc43d1443bf9553) You can also access the same settings from the **”cog”** on a branch overview page (from the **”Branches”** tab, then select the branch you want to view or modify). ### Using the pscale CLI To enable safe migrations on a branch using the pscale CLI, use the following command in your terminal: ``` pscale branch safe-migrations enable ``` ## How to disable safe migrations There are two ways to disable safe migrations: the PlanetScale dashboard and the CLI. ### Using the PlanetScale dashboard To disable safe migrations, click the **”cog”** in the upper right of the infrastructure card on the ”**Dashboard**” tab of the database. ![Branch UI with enabled with cog highlighted.](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/safe-migrations/prod-card-cog-2.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=da093e115a9159f7d76cad61ca1ecc11) In the modal, toggle the option labeled **”Enable safe migrations,”** then click the **”Disable safe migrations”** button to save and close the modal. You can also access the same settings from the **”cog”** on a branch overview page (from the **”Branches”** tab, then select the branch you want to view or modify). ### Using the pscale CLI To disable safe migrations on a branch using the pscale CLI, use the following command in your terminal: ``` pscale branch safe-migrations disable ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Schema lint errors Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors Schema lint errors explain why a PlanetScale Vitess schema change, deploy request, or import cannot be safely applied. ## Overview PlanetScale validates MySQL schema, Vitess VSchema, and routing rule changes before applying them to a production branch. These checks protect deploy requests and imports from changes that Vitess cannot apply safely or that would make data inaccessible after a merge. When a check fails, PlanetScale returns a schema lint error with an error code, a description, and context such as the keyspace, table, column, VSchema JSON path, or vindex name. Use the error-specific pages below with the exact error description shown in the dashboard or API response. ## Error reference * [`UNSPECIFIED`](/docs/vitess/schema-changes/schema-lint-errors/unspecified) * [`NO_UNIQUE_KEY`](/docs/vitess/schema-changes/schema-lint-errors/no-unique-key) * [`FOREIGN_KEYS_DISALLOWED`](/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-disallowed) * [`INVALID_CHARSET`](/docs/vitess/schema-changes/schema-lint-errors/invalid-charset) * [`UNSUPPORTED_FEATURE`](/docs/vitess/schema-changes/schema-lint-errors/unsupported-feature) * [`UNSUPPORTED_SHARDED_FEATURE`](/docs/vitess/schema-changes/schema-lint-errors/unsupported-sharded-feature) * [`INVALID_VSCHEMA`](/docs/vitess/schema-changes/schema-lint-errors/invalid-vschema) * [`DUPLICATE_ENUM_VALUE`](/docs/vitess/schema-changes/schema-lint-errors/duplicate-enum-value) * [`UNSUPPORTED_PARTITIONING_FEATURE`](/docs/vitess/schema-changes/schema-lint-errors/unsupported-partitioning-feature) * [`INVALID_TABLE_REFERENCED_IN_VIEW`](/docs/vitess/schema-changes/schema-lint-errors/invalid-table-referenced-in-view) * [`INVALID_COLUMN_REFERENCED_IN_VIEW`](/docs/vitess/schema-changes/schema-lint-errors/invalid-column-referenced-in-view) * [`BLOB_IN_UNIQUE_KEY`](/docs/vitess/schema-changes/schema-lint-errors/blob-in-unique-key) * [`PARTIAL_KEY_IN_UNIQUE_KEY`](/docs/vitess/schema-changes/schema-lint-errors/partial-key-in-unique-key) * [`FOREIGN_KEYS_CIRCULAR_DEPENDENCY`](/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-circular-dependency) * [`FOREIGN_KEYS_NONEXISTENT_TABLE`](/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-nonexistent-table) * [`FOREIGN_KEYS_REFERENCED_VIEW`](/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-referenced-view) * [`FOREIGN_KEYS_UNRESOLVED`](/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-unresolved) * [`INVALID_ROUTING_RULES`](/docs/vitess/schema-changes/schema-lint-errors/invalid-routing-rules) ## General troubleshooting flow 1. Read the error description first. It usually names the specific table, column, vindex, JSON path, or routing rule that failed validation. 2. Fix the schema or VSchema on a development branch. 3. If the fix depends on another object, such as a referenced table or a sequence table, deploy that dependency first. 4. Create a new deploy request after the branch schema is valid. For broader context, see [deploy requests](/docs/vitess/schema-changes/deploy-requests), [safe migrations](/docs/vitess/schema-changes/safe-migrations), [VSchema](/docs/vitess/sharding/vschema), and [Vindexes](/docs/vitess/sharding/vindexes). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # BLOB_IN_UNIQUE_KEY schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/blob-in-unique-key Fix primary keys that include BLOB or TEXT columns. ## What this error means `BLOB_IN_UNIQUE_KEY` means a primary key includes a `BLOB` or `TEXT` column. A common error message is: ```text theme={null} table "documents" contains BLOB or TEXT column "body" in the primary key. ``` ## Why PlanetScale rejects it PlanetScale needs a stable and efficient unique row identity for online schema changes. `BLOB` and `TEXT` columns are not accepted for that row identity because they can be large and are typically indexed through prefixes rather than complete values. ## How to fix it Use a surrogate primary key or a bounded column type that can be indexed fully. Invalid: ```sql theme={null} CREATE TABLE documents ( body text NOT NULL, PRIMARY KEY (body(255)) ); ``` Recommended: ```sql theme={null} CREATE TABLE documents ( id bigint unsigned NOT NULL, body text NOT NULL, PRIMARY KEY (id) ); ``` If the text value must be unique, consider adding a generated or application-managed hash column with a unique key after evaluating collision risk. ```sql theme={null} CREATE TABLE documents ( id bigint unsigned NOT NULL, body text NOT NULL, body_sha256 binary(32) NOT NULL, PRIMARY KEY (id), UNIQUE KEY documents_body_sha256_idx (body_sha256) ); ``` ## Related docs * [Changing primary and unique keys](/docs/vitess/schema-changes/onlineddl-change-unique-keys) * [MySQL compatibility](/docs/vitess/mysql-compatibility) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # DUPLICATE_ENUM_VALUE schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/duplicate-enum-value Fix MySQL enum definitions that repeat the same enum value. ## What this error means `DUPLICATE_ENUM_VALUE` means an `ENUM` column definition contains the same value more than once. This can include duplicate empty-string values. A common error message is: ```text theme={null} enum value '' occurs more than once. ``` ## Why PlanetScale rejects it Enum values are stored internally by ordinal position. Duplicate values make the schema ambiguous and can cause unexpected behavior when comparing schemas, generating diffs, or applying deploy requests. ## How to fix it Remove the duplicate enum value from the column definition. Invalid: ```sql theme={null} CREATE TABLE shirts ( id bigint unsigned NOT NULL, size enum('small', 'medium', 'medium', 'large') NOT NULL, PRIMARY KEY (id) ); ``` Valid: ```sql theme={null} CREATE TABLE shirts ( id bigint unsigned NOT NULL, size enum('small', 'medium', 'large') NOT NULL, PRIMARY KEY (id) ); ``` If the duplicate values were meant to represent different labels, store distinct enum values and map display names in your application. ## Related docs * [How to make different types of schema changes](/docs/vitess/schema-changes/how-to-make-different-types-of-schema-changes) * [Deploy requests](/docs/vitess/schema-changes/deploy-requests) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # FOREIGN_KEYS_CIRCULAR_DEPENDENCY schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-circular-dependency Fix circular foreign key dependencies in PlanetScale Vitess schemas. ## What this error means `FOREIGN_KEYS_CIRCULAR_DEPENDENCY` means foreign key relationships form a cycle. This can be a self-referencing table or a group of tables that reference each other in a loop. ## Why PlanetScale rejects it Foreign key cycles can make schema dependency ordering ambiguous. They can also produce unsupported cascaded behavior, especially when all participating constraints use `CASCADE` or `SET NULL` actions. ## How to fix it Break the cycle where possible: * Remove one database-enforced foreign key constraint and enforce that relationship in application code. * Replace cascading actions with explicit application workflows. * Split schema changes so parent tables and child constraints are introduced in a predictable order. * For self-references, check whether the constraint is necessary or whether application validation is enough. Example cycle: ```sql theme={null} CREATE TABLE teams ( id bigint unsigned NOT NULL, owner_user_id bigint unsigned NOT NULL, PRIMARY KEY (id), CONSTRAINT teams_owner_fk FOREIGN KEY (owner_user_id) REFERENCES users(id) ); CREATE TABLE users ( id bigint unsigned NOT NULL, team_id bigint unsigned NOT NULL, PRIMARY KEY (id), CONSTRAINT users_team_fk FOREIGN KEY (team_id) REFERENCES teams(id) ); ``` One common fix is to remove one constraint and keep an index for the relationship: ```sql theme={null} CREATE TABLE users ( id bigint unsigned NOT NULL, team_id bigint unsigned NOT NULL, PRIMARY KEY (id), KEY users_team_idx (team_id) ); ``` ## Related docs * [Foreign key constraints](/docs/vitess/foreign-key-constraints#cyclic-foreign-keys) * [Operating without foreign key constraints](/docs/vitess/operating-without-foreign-key-constraints) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # FOREIGN_KEYS_DISALLOWED schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-disallowed Fix schemas that define foreign key constraints when foreign key support is disabled. ## What this error means `FOREIGN_KEYS_DISALLOWED` means the schema contains a `FOREIGN KEY` constraint, but the database is not configured to allow foreign key constraints. A common error message is: ```text theme={null} table "posts" has a foreign key constraint: foreign key constraints are not supported. ``` The error may also include the local column names used by the foreign key. ## Why PlanetScale rejects it PlanetScale databases can run with foreign key constraints disabled or managed. When constraints are disabled, Vitess rejects DDL that creates them so the database does not enter a mixed or unsupported foreign key mode. ## How to fix it Choose one of these approaches: * Enable foreign key constraint support for the database if your workload needs database-enforced referential integrity and the database meets the prerequisites. * Remove the `CONSTRAINT ... FOREIGN KEY` clause and keep ordinary indexes for query performance. * Enforce referential integrity in application code or background jobs instead of with database constraints. Invalid when foreign keys are disabled: ```sql theme={null} CREATE TABLE posts ( id bigint unsigned NOT NULL, author_id bigint unsigned NOT NULL, PRIMARY KEY (id), CONSTRAINT posts_author_fk FOREIGN KEY (author_id) REFERENCES users(id) ); ``` Constraint-free version: ```sql theme={null} CREATE TABLE posts ( id bigint unsigned NOT NULL, author_id bigint unsigned NOT NULL, PRIMARY KEY (id), KEY posts_author_idx (author_id) ); ``` ## Related docs * [Foreign key constraints](/docs/vitess/foreign-key-constraints) * [Operating without foreign key constraints](/docs/vitess/operating-without-foreign-key-constraints) * [Strategies for maintaining referential integrity](/docs/vitess/strategies-for-maintaining-referential-integrity) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # FOREIGN_KEYS_NONEXISTENT_TABLE schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-nonexistent-table Fix foreign keys that reference tables missing from the schema. ## What this error means `FOREIGN_KEYS_NONEXISTENT_TABLE` means a foreign key references a table that does not exist in the resulting schema. A common underlying message is: ```text theme={null} table `posts` foreign key references nonexistent table `users` ``` ## Why PlanetScale rejects it MySQL foreign keys must reference an existing base table. During a deploy request, PlanetScale validates the final schema. If the parent table is missing, the child table cannot be created or preserved with that constraint. ## How to fix it Make sure the referenced table exists before the foreign key is created. * Fix spelling or keyspace/schema qualification mistakes. * Create the parent table before creating the child foreign key. * Do not drop a parent table while a child foreign key still references it. * Split complex changes into multiple deploy requests. Invalid: ```sql theme={null} CREATE TABLE posts ( id bigint unsigned NOT NULL, author_id bigint unsigned NOT NULL, PRIMARY KEY (id), CONSTRAINT posts_author_fk FOREIGN KEY (author_id) REFERENCES users(id) ); ``` If `users` is missing, create it first or remove the constraint until the parent exists. ## Related docs * [Foreign key constraints](/docs/vitess/foreign-key-constraints) * [Deploy requests](/docs/vitess/schema-changes/deploy-requests) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # FOREIGN_KEYS_REFERENCED_VIEW schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-referenced-view Fix foreign keys that reference views instead of base tables. ## What this error means `FOREIGN_KEYS_REFERENCED_VIEW` means a foreign key references a view. A common underlying message is: ```text theme={null} table `posts` foreign key references view `active_users` ``` ## Why PlanetScale rejects it MySQL foreign key constraints must reference real base tables. A view is a stored query, not a table that can enforce referential integrity. ## How to fix it Reference the underlying table instead of the view, or remove the foreign key constraint and enforce the relationship in application code. Invalid: ```sql theme={null} CREATE VIEW active_users AS SELECT id FROM users WHERE active = 1; CREATE TABLE posts ( id bigint unsigned NOT NULL, author_id bigint unsigned NOT NULL, PRIMARY KEY (id), CONSTRAINT posts_author_fk FOREIGN KEY (author_id) REFERENCES active_users(id) ); ``` Valid table reference: ```sql theme={null} CREATE TABLE posts ( id bigint unsigned NOT NULL, author_id bigint unsigned NOT NULL, PRIMARY KEY (id), CONSTRAINT posts_author_fk FOREIGN KEY (author_id) REFERENCES users(id) ); ``` ## Related docs * [Foreign key constraints](/docs/vitess/foreign-key-constraints) * [Operating without foreign key constraints](/docs/vitess/operating-without-foreign-key-constraints) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # FOREIGN_KEYS_UNRESOLVED schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/foreign-keys-unresolved Fix foreign key dependencies that Vitess cannot resolve. ## What this error means `FOREIGN_KEYS_UNRESOLVED` means Vitess could not resolve the dependency ordering for a table's foreign keys. A common underlying message is: ```text theme={null} table `posts` has unresolved foreign key dependencies ``` ## Why PlanetScale rejects it PlanetScale needs to know the order in which tables and constraints can be applied. Foreign keys introduce dependencies between tables. If those dependencies are missing, conflicting, cyclic, or changed together in an unsafe order, Vitess cannot safely build the final schema. ## How to fix it Check the table named in the error and each foreign key it defines. * Confirm every referenced table exists in the final schema. * Confirm every referenced column exists and is indexed as required by MySQL. * Create parent tables before child constraints. * Drop child constraints before dropping referenced parent tables or columns. * Split dependent changes into separate deploy requests. * Break cycles where possible. For example, deploy parent table changes first, then add child constraints in a later deploy request. ## Related docs * [Foreign key constraints](/docs/vitess/foreign-key-constraints) * [Deploy requests](/docs/vitess/schema-changes/deploy-requests) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # INVALID_CHARSET schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/invalid-charset Fix unsupported table or column character sets in a PlanetScale Vitess schema. ## What this error means `INVALID_CHARSET` means a table or column uses a character set that PlanetScale does not allow for schema linting. Common messages include: ```text theme={null} table "users" has an invalid table charset utf16: only utf8, utf8mb4, and utf8mb3 charsets are supported. table "users" has an invalid column charset utf16: only utf8, utf8mb4, and utf8mb3 charsets are supported. ``` PlanetScale supports `utf8mb4`, `utf8mb3`, `utf8`, `latin1`, and `ascii`. We recommend `utf8mb4` for new schemas. ## Why PlanetScale rejects it Vitess must be able to parse, compare, and apply schemas consistently across branches and tablets. Unsupported charsets can make schema diffs and data movement unsafe or incompatible with the target MySQL configuration. ## How to fix it Convert the table or column to a supported character set, preferably `utf8mb4`. ```sql theme={null} ALTER TABLE users CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci; ``` For a single column override, update the column definition: ```sql theme={null} ALTER TABLE users MODIFY display_name varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci NOT NULL; ``` If the table contains existing data, test the conversion on a development branch first and confirm your application handles the new collation rules. ## Related docs * [MySQL compatibility](/docs/vitess/mysql-compatibility) * [Deploy requests](/docs/vitess/schema-changes/deploy-requests) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # INVALID_COLUMN_REFERENCED_IN_VIEW schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/invalid-column-referenced-in-view Fix views that reference missing, ambiguous, or unsafe columns. ## What this error means `INVALID_COLUMN_REFERENCED_IN_VIEW` means a view references a column that Vitess cannot resolve in the final schema. This can also happen when a view uses a star expression that cannot be safely expanded. Common underlying messages include: ```text theme={null} view `active_users` references unqualified but non-existent column `email` view `active_users` references unqualified but non unique column `id` view `active_users` has invalid star expression ``` ## Why PlanetScale rejects it A view must remain valid after the deploy request is applied. If a column is dropped, renamed, ambiguous across joined tables, or hidden behind `SELECT *`, Vitess cannot reliably generate a safe schema diff. ## How to fix it Use explicit column lists and update the view before dropping or renaming columns. Avoid this: ```sql theme={null} CREATE VIEW active_users AS SELECT * FROM users WHERE active = 1; ``` Prefer this: ```sql theme={null} CREATE VIEW active_users AS SELECT id, email, created_at FROM users WHERE active = 1; ``` If you need to rename a column used by a view, split the migration: 1. Add the new column and update application writes. 2. Update the view to reference the new column. 3. Drop the old column after the view and application no longer use it. ## Related docs * [Handling table and column renames](/docs/vitess/schema-changes/handling-table-and-column-renames) * [Deploy requests](/docs/vitess/schema-changes/deploy-requests) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # INVALID_ROUTING_RULES schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/invalid-routing-rules Fix invalid Vitess routing rules in PlanetScale. ## What this error means `INVALID_ROUTING_RULES` means the Vitess routing rules for the schema are invalid or conflict with the target branch during a merge. Common messages include: ```text theme={null} table not found keyspace `target` not found in schema table "t" must be qualified table `source.t` has no targets table `source.t` has more than one target table "t@rdonly" already routes to different table in destination routing rules and conflicts ``` ## Why PlanetScale rejects it Routing rules tell Vitess how to route queries from one table name to another table. Invalid rules can send traffic to a missing table, an ambiguous keyspace, multiple targets, or a conflicting destination rule. ## How to fix it Use the routing rule location and the full error message to find the failing rule. * Qualify table names as `.
`. * Verify the target keyspace exists. * Verify the target table exists in the SQL schema. * Use exactly one target table for each rule. * Remove duplicate source table entries. * Use only valid tablet-type suffixes, such as `@primary`, `@replica`, or `@rdonly`. * If the error is a merge conflict, reconcile the destination branch's routing rules first, then recreate the deploy request. Example shape: ```json theme={null} { "rules": [ { "from_table": "source.orders", "to_tables": ["target.orders"] } ] } ``` ## Related docs * [VSchema](/docs/vitess/sharding/vschema) * [Sharding quickstart](/docs/vitess/sharding/sharding-quickstart) * [Sharding workflow state reference](/docs/vitess/sharding/sharding-workflow-state-reference) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # INVALID_TABLE_REFERENCED_IN_VIEW schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/invalid-table-referenced-in-view Fix views that reference missing tables or unresolved view dependencies. ## What this error means `INVALID_TABLE_REFERENCED_IN_VIEW` means a view references a table or entity that Vitess cannot resolve in the resulting schema. The error usually includes the view name and the missing referenced table. ## Why PlanetScale rejects it PlanetScale compares the final schema state when building a deploy request. A view must be valid in that final schema. If the referenced table is missing, renamed, dropped, or involved in an unresolved dependency loop, Vitess cannot safely create or preserve the view. ## How to fix it Check the view definition and make sure every referenced table exists after the deploy request is applied. Invalid: ```sql theme={null} CREATE VIEW active_users AS SELECT id, email FROM users_old WHERE active = 1; ``` If `users_old` no longer exists, update the view: ```sql theme={null} CREATE VIEW active_users AS SELECT id, email FROM users WHERE active = 1; ``` When renaming or replacing a table used by a view, split the work into multiple deploy requests: 1. Create the replacement table or compatibility view. 2. Update dependent views to reference the new object. 3. Drop the old table after no views depend on it. ## Related docs * [Handling table and column renames](/docs/vitess/schema-changes/handling-table-and-column-renames) * [Deploy requests](/docs/vitess/schema-changes/deploy-requests) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # INVALID_VSCHEMA schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/invalid-vschema Fix VSchema JSON and semantic validation errors in PlanetScale Vitess. ## What this error means `INVALID_VSCHEMA` means the Vitess VSchema is invalid JSON, does not match the expected VSchema shape, or is inconsistent with the SQL schema. The error usually points to the affected VSchema path, table, or vindex. Use that context to find the exact VSchema entry that failed validation. Common messages include: ```text theme={null} vindex not found column "customer_id" not found sequence table in vschema needs to be marked as sequence type wrong type of vindex defined in column `customer_id`, expected `hash` or `xxhash`, got `unicode_loose_md5` table orders_lookup has a changed column vindex. This will lead to inaccessible data ``` ## Why PlanetScale rejects it Vitess uses VSchema to route queries, map rows to shards, define vindexes, configure lookup vindexes, and manage sequences. If VSchema is inconsistent with the SQL schema, Vitess may route queries incorrectly or make data inaccessible. ## How to fix it Use the highlighted VSchema path or the table and vindex named in the message to find the failing VSchema entry. Then check these common causes: * The table named in VSchema also exists in the SQL schema. * Every `column_vindexes` entry references an existing vindex. * Every column used by a vindex exists in the SQL table. * The vindex type matches the column type. Numeric columns should use `hash` or `xxhash`; text columns should use `unicode_loose_md5` or `unicode_loose_xxhash`; binary columns should use `binary_md5`. * Sequence tables exist in SQL, are declared in VSchema as `type: "sequence"`, and contain integer `id`, `next_id`, and `cache` columns. * Existing primary column vindexes are not changed in a way that would make existing rows inaccessible. Example column vindex definition: ```json theme={null} { "sharded": true, "vindexes": { "xxhash": { "type": "xxhash" } }, "tables": { "orders": { "column_vindexes": [ { "column": "customer_id", "name": "xxhash" } ] } } } ``` ## Related docs * [VSchema](/docs/vitess/sharding/vschema) * [Vindexes](/docs/vitess/sharding/vindexes) * [Sequence tables](/docs/vitess/sharding/sequence-tables) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # NO_UNIQUE_KEY schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/no-unique-key Fix tables that do not have a unique, not-null key for PlanetScale online schema changes. ## What this error means `NO_UNIQUE_KEY` means a table does not have a usable unique, not-null key. A `PRIMARY KEY` satisfies this requirement. A `UNIQUE KEY` can also satisfy it if every indexed column is `NOT NULL` and the key does not use a text/blob column or a prefix index. A common error message is: ```text theme={null} table "orders" has no non-null unique key: all tables must have at least one unique, not-null key without using text / blob columns or partial indexes. ``` ## Why PlanetScale rejects it PlanetScale uses Vitess online schema change workflows to apply changes without downtime. Those workflows need a stable row identity while copying and reconciling data. Nullable unique keys do not qualify because MySQL allows multiple `NULL` values in a unique index. ## How to fix it Add a primary key or a unique key over columns that are all declared `NOT NULL`. ```sql theme={null} CREATE TABLE orders ( id bigint unsigned NOT NULL, customer_id bigint unsigned NOT NULL, created_at datetime NOT NULL, PRIMARY KEY (id) ); ``` If the table already exists and the intended key column contains `NULL` values, fix the data first, then make the column `NOT NULL`, then add the unique key. For large tables, split those steps into separate deploy requests. ```sql theme={null} ALTER TABLE orders MODIFY id bigint unsigned NOT NULL; ALTER TABLE orders ADD UNIQUE KEY orders_id_idx (id); ``` If your only candidate key uses a `TEXT`, `BLOB`, or prefix index, use a surrogate key such as a bigint or UUID column instead. ## Related docs * [Changing primary and unique keys](/docs/vitess/schema-changes/onlineddl-change-unique-keys) * [How online schema change tools work](/docs/vitess/schema-changes/how-online-schema-change-tools-work) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PARTIAL_KEY_IN_UNIQUE_KEY schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/partial-key-in-unique-key Fix primary keys that use prefix indexes. ## What this error means `PARTIAL_KEY_IN_UNIQUE_KEY` means a primary key uses a prefix length, such as `name(10)`. A common error message is: ```text theme={null} table "customers" contains partial index on column "email" in the primary key. ``` ## Why PlanetScale rejects it A prefix index only indexes part of a column value. PlanetScale needs a unique key that can identify the full row during online schema changes. A partial key is not accepted as that identity. ## How to fix it Use a full column in the primary key, add a surrogate primary key, or add a separate full unique key over `NOT NULL` columns. Invalid: ```sql theme={null} CREATE TABLE customers ( email varchar(255) NOT NULL, PRIMARY KEY (email(32)) ); ``` Recommended: ```sql theme={null} CREATE TABLE customers ( id bigint unsigned NOT NULL, email varchar(255) NOT NULL, PRIMARY KEY (id), UNIQUE KEY customers_email_idx (email) ); ``` If the full column is too large for your indexing strategy, use a shorter bounded column type or a separate stable identifier. ## Related docs * [Changing primary and unique keys](/docs/vitess/schema-changes/onlineddl-change-unique-keys) * [MySQL compatibility](/docs/vitess/mysql-compatibility) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # UNSPECIFIED schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/unspecified How to troubleshoot an unclassified PlanetScale schema lint error. ## What this error means `UNSPECIFIED` is a fallback schema lint error. PlanetScale found a schema, VSchema, or routing rule problem, but the failure did not map to a more specific lint code. The message shown with the error is the most important part of this error. It may include the table, keyspace, statement, or lower-level validation message that caused the check to fail. ## Why PlanetScale rejects it Deploy requests and imports must produce a schema Vitess can safely apply. If the validator cannot classify the failure, PlanetScale still blocks the change instead of applying an unknown or unsafe schema state. ## How to fix it 1. Read the full error message carefully and identify the referenced keyspace, table, column, VSchema path, or routing rule. 2. Check whether another page in this error reference matches the same message. 3. Reproduce the change on a development branch and make the smallest schema change needed to clear the message. 4. If the message is not actionable, contact support with the database, branch, deploy request, and full lint error details. ## Related docs * [Schema lint errors](/docs/vitess/schema-changes/schema-lint-errors) * [Deploy requests](/docs/vitess/schema-changes/deploy-requests) * [MySQL compatibility](/docs/vitess/mysql-compatibility) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # UNSUPPORTED_FEATURE schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/unsupported-feature Resolve PlanetScale schema lint errors for unsupported MySQL or Vitess features. ## What this error means `UNSUPPORTED_FEATURE` is used for schema features PlanetScale cannot safely apply through Vitess. The exact fix depends on the full error message and any context shown with it, such as the affected table, storage engine, or check constraint. Known cases include: * Non-InnoDB storage engines, such as `MyISAM`. * Unsupported SQL features such as `JSON_TABLE`. * Too many columns in one table. * Too many tables or views in one schema. * Check constraint names that are too long for older MySQL/Vitess rename behavior. * Generic precondition failures that do not have a more specific lint code. ## Why PlanetScale rejects it Deploy requests are applied by Vitess. Some MySQL features either are not supported by Vitess, cannot be represented safely in the online schema change flow, or exceed limits that protect branch creation and deploy reliability. ## How to fix it Start with the exact error description. If the error names a storage engine, use InnoDB: ```sql theme={null} ALTER TABLE archived_orders ENGINE = InnoDB; ``` If the error mentions too many columns or too many tables/views, split the schema or reduce the object count before deploying. If the error mentions a check constraint name, shorten the constraint name. For older MySQL versions, avoid long generated check names and avoid names with more than three characters after a `
_chk_` prefix. ```sql theme={null} ALTER TABLE accounts ADD CONSTRAINT balance_chk CHECK (balance >= 0); ``` If the error mentions `JSON_TABLE`, rewrite the query or schema design to avoid relying on `JSON_TABLE` in PlanetScale Vitess. ## Related docs * [MySQL compatibility](/docs/vitess/mysql-compatibility) * [PlanetScale system limits](/docs/vitess/troubleshooting/planetscale-system-limits) * [Deploy requests](/docs/vitess/schema-changes/deploy-requests) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # UNSUPPORTED_PARTITIONING_FEATURE schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/unsupported-partitioning-feature Fix unsupported MySQL table partitioning features in PlanetScale Vitess. ## What this error means `UNSUPPORTED_PARTITIONING_FEATURE` means a table uses a MySQL partitioning feature that PlanetScale does not support in schema linting. Common messages include: ```text theme={null} only RANGE, HASH, and KEY based partitioning is supported SUBPARTITION is not supported partition "p0" has unsupported storage engine MyISAM SUBPARTITION for partition "p0" is not supported ``` ## Why PlanetScale rejects it Vitess already provides horizontal sharding at the keyspace level. Some MySQL table partitioning features are difficult to apply safely through online schema changes or are not compatible with the schema deployment path. ## How to fix it Use only supported partitioning types: `RANGE`, `HASH`, or `KEY`. Remove `LIST` partitioning, subpartitioning, and non-InnoDB partition engines. Invalid: ```sql theme={null} CREATE TABLE events ( id bigint unsigned NOT NULL, region_id int NOT NULL, PRIMARY KEY (id) ) PARTITION BY LIST(region_id) ( PARTITION us VALUES IN (1, 2), PARTITION eu VALUES IN (3, 4) ); ``` A supported partitioning type: ```sql theme={null} CREATE TABLE events ( id bigint unsigned NOT NULL, created_at date NOT NULL, PRIMARY KEY (id, created_at) ) PARTITION BY RANGE COLUMNS(created_at) ( PARTITION p2026 VALUES LESS THAN ('2027-01-01') ); ``` If you are using partitioning to distribute data at scale, consider Vitess sharding instead of MySQL table partitioning. ## Related docs * [Sharding](/docs/vitess/sharding) * [Vindexes](/docs/vitess/sharding/vindexes) * [MySQL compatibility](/docs/vitess/mysql-compatibility) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # UNSUPPORTED_SHARDED_FEATURE schema lint error Source: https://planetscale.com/docs/vitess/schema-changes/schema-lint-errors/unsupported-sharded-feature Fix features that are not supported in sharded PlanetScale Vitess keyspaces. ## What this error means `UNSUPPORTED_SHARDED_FEATURE` means the schema uses a feature that is valid in some MySQL or unsharded Vitess schemas but not in a sharded keyspace. Common messages include: ```text theme={null} `recent_orders` is a view: sharded databases cannot have views table "orders" has an autoincrement column "id": sharded databases cannot have autoincrement columns, use a sequence instead ``` ## Why PlanetScale rejects it In a sharded keyspace, rows are distributed across multiple MySQL instances. Features that require one local MySQL instance to understand the entire table, such as `AUTO_INCREMENT`, cannot coordinate safely across shards. Views are also not supported in sharded keyspaces. ## How to fix it For `AUTO_INCREMENT` columns, use a Vitess sequence table instead: ```sql theme={null} CREATE TABLE orders ( id bigint unsigned NOT NULL, customer_id bigint unsigned NOT NULL, PRIMARY KEY (id) ); ``` Then create a sequence table in an unsharded keyspace and update the VSchema to use it. ```sql theme={null} CREATE TABLE orders_seq ( id bigint, next_id bigint, cache bigint, PRIMARY KEY (id) ) COMMENT 'vitess_sequence'; ``` For views in sharded keyspaces, remove the view from the sharded keyspace. Move the logic into application queries, materialize the data into a table, or use an unsharded keyspace when appropriate. ## Related docs * [Sequence tables](/docs/vitess/sharding/sequence-tables) * [VSchema](/docs/vitess/sharding/vschema) * [Vindexes](/docs/vitess/sharding/vindexes) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Throttling deploy requests Source: https://planetscale.com/docs/vitess/schema-changes/throttling-deploy-requests ## Overview Deploy requests, though non-blocking, do consume some resources as the online schema change process occurs. Normally, you don't need to get involved, as the [Vitess tablet throttler](https://vitess.io/docs/reference/features/tablet-throttler/) automatically identifies when replication lag is high on your database and [slows down migration progress](https://vitess.io/docs/user-guides/schema-changes/audit-and-control/#controlling-throttling). For long-running schema changes that take several minutes or hours, you may wish to increase throttling on your deploy request to mitigate load on your database. ![Deploy request throttling settings page](https://mintcdn.com/planetscale-2/xsX1e-5IXCYXbX59/vitess/schema-changes/throttling-deploy-requests/deploy-request-throttling.png?fit=max\&auto=format\&n=xsX1e-5IXCYXbX59\&q=85\&s=ca78fce20d99ca8c7025570ffda96ef6) ## How it works The throttle setting allows you to adjust the underlying Vitess tablet throttler for deploy requests. The throttle setting is a value between 0 and 95, where 0 means throttling is effectively disabled, and 95 is nearly fully throttled, which will drastically slow down migrations. 95 is currently the max throttle setting available. For example, if you set the throttle value to 25, deploy requests will run at about 75% of their normal speed. The lower the number, the less throttling. ## Managing throttler settings There are three ways to adjust the throttler for deploy requests: * At the database level * At the keyspace level * At the deploy request level You must be an [Organization Administrator](/docs/security/access-control#organization-administrator) or [Database Administrator](/docs/security/access-control#database-administrator) for the database that you're configuring in order to change these settings at the database level. However, any organization member who has permission to deploy a deploy request is able to adjust the configuration at the deploy request level. ### Managing throttler settings at the database or keyspace level For the first two options, first select your database, click "Settings" in the left nav, and scroll down to "Advanced settings". Here, you'll see the options to configure deploy request throttling. You can adjust the slider or manually type in the value you'd like to set throttling to. If you'd like to set throttling at the [keyspace](/docs/vitess/sharding/keyspaces) level so that each keyspace is throttled differently, check "Set throttling per keyspace". This will give you the option to individually adjust the sliders for each keyspace. Click "Save throttling settings". Going forward, these throttler settings will apply to every deploy request as configured. ### Managing throttler settings per deploy request You also have the option to manually configure throttler settings on each deploy request. Once you click "Deploy changes" on a deploy request, click the arrow next to "throttling" to show the settings to adjust the throttle value for that deploy request. From here, you'll see the same options to adjust the value for each keyspace or for all keyspaces. Changes to throttler configuration on the deploy request page will only apply to that individual deploy request and will not affect the database level throttler configurations. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Value does not fit the new column Source: https://planetscale.com/docs/vitess/schema-changes/value-does-not-fit-column Fix a deploy that fails while copying rows because existing values do not fit a changed column type. A deploy request can fail while copying a table because existing values do not fit the column's new type. PlanetScale copies the table in the background and inserts existing rows into a new table that uses the deployed schema. The copy runs under strict SQL mode, so a value that would be silently truncated or rounded in a lenient MySQL session is rejected instead. The column name in quotes is the one to fix. | Error | Common cause | | - | - | | `Data truncated for column 'status'` (errno 1265) | Changing a column to `ENUM` or `SET` when some rows hold a value that is not in the list. Converting text to a number, or lowering `DECIMAL` scale. | | `Data too long for column 'title'` (errno 1406) | Shortening a `VARCHAR` or `CHAR`, or changing `TEXT` to a smaller type, when some rows are longer than the new length. | | `Out of range value for column 'score'` (errno 1264) | Changing to a smaller integer type, from `SIGNED` to `UNSIGNED` when negative values exist, or lowering `DECIMAL` precision. | | `Incorrect integer value: 'abc' for column 'id'` (errno 1366) | Changing text to a numeric type when some rows are not numeric. Also `Incorrect string value` when changing to a character set that cannot store some existing characters. | | `Incorrect datetime value: '1785419821' for column 'created'` (errno 1292) | Changing a text or numeric column to `DATE`, `DATETIME`, or `TIMESTAMP` when some rows are not valid dates. | | `Invalid JSON text ... for column 'payload'` (errno 3140) | Changing a text column to `JSON` when some rows are not valid JSON. | ```text theme={null} vreplication: terminal error: task error: failed inserting rows: Data truncated for column 'visibility' at row 98 (errno 1265) (sqlstate 01000) ``` ## How to fix it Find the rows whose values do not fit the new definition, then update or delete them. For example: ```sql theme={null} -- Values not in the new ENUM list SELECT id, visibility FROM groups WHERE visibility NOT IN ('public', 'private'); -- Values longer than the new VARCHAR(100) SELECT id FROM posts WHERE CHAR_LENGTH(title) > 100; -- Values outside the range of the new SMALLINT UNSIGNED SELECT id FROM scores WHERE score < 0 OR score > 65535; -- Values that are not valid JSON SELECT id FROM events WHERE NOT JSON_VALID(payload); ``` Then [retry the failed tables](/docs/vitess/schema-changes/deploy-requests#retry-a-partially-failed-deploy). Retrying before every offending row is updated will cause another failure. If the existing values are correct and the new type is too strict, change the column definition on your branch instead, and open a new deploy request. > [!NOTE] > > The copy runs while your application keeps writing to the table. If the application writes a value that does not fit during the copy, the deploy fails with the same error. Make sure new writes also fit the new type before retrying. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connection strings Source: https://planetscale.com/docs/vitess/security/connection-strings ## Creating a password **Step 1** To create a password, head to your database dashboard page at `https://app.planetscale.com//` and click on the "**Connect**" button. **Step 2** On the **Connect page**, select the branch you wanted to create a password for, pick a [password role](/docs/vitess/security/password-roles), and provide a recognizable name for the new credentials. Clicking `Create password` will then generate a **unique username and password pair** that can only be used to access the designated branch of your database. Take note of this password, as you won't be able to see it again. **Step 3** Once created, you can browse the connection string in different framework formats by selecting framework in the "Select your language or framework" section. This will also show you all of the files you need to modify to get connected with PlanetScale in your framework or language of choice. > [!NOTE] > > Passwords support three connection types: `Primary`, `Replica`, and `Read-only region`. `Primary` connects to the primary region. `Replica` routes queries to your branch's replicas and read-only regions (nearest replica). `Read-only region` scopes the password to a specific read-only region and can be created with `pscale password create ... --read-only-region`. [Read more about replicas](/docs/vitess/scaling/replicas) and [read-only regions](/docs/vitess/scaling/read-only-regions). > [!TIP] > > Make sure you copy the credentials for your application and the "Other" format. We do not save the password in plaintext, so there will be no way to retrieve the password after you leave this page. ## Managing passwords Once you've created the password, you can head over to the "**Passwords**" settings page available at `Organization > Database > Settings > Passwords` to manage them. > [!TIP] > > You can also create passwords for branches other than `main` on this page. ![Manage passwords page](https://mintcdn.com/planetscale-2/pncBnOWnwAfGxG1f/images/assets/docs/concepts/connection-strings/manage-2.png?fit=max\&auto=format\&n=pncBnOWnwAfGxG1f\&q=85\&s=880492f6d69d4772c79eac4f05291cfe) Clicking on the `...` icon on the row for your password allows you rename or delete the password. ## Renaming a password Since the **username & password** pair is unique, the only metadata you can edit is the `display name` of the password. ## Deleting a password Deleting a password will invalidate the username & password pair and **disconnect any active clients using this password**. > [!NOTE] > > Any open database connections authenticated with a deleted password will be disconnected within five minutes. ## Native MySQL authentication support Use the tools you're familiar with to connect to PlanetScale databases. PlanetScale supports both [MySQL native authentication](https://dev.mysql.com/doc/refman/8.0/en/native-pluggable-authentication.html), which is widely used to provide a secure connection to MySQL servers, and [MySQL Caching SHA-2 authentication](https://dev.mysql.com/doc/refman/8.0/en/caching-sha2-pluggable-authentication.html), which is the most secure authentication mechanism to connect to MySQL. Based on your application needs and platform support, you can switch between the authentication modes, with the same password. For a list of tested MySQL GUI clients, review our article on [how to connect MySQL GUI applications](/docs/vitess/tutorials/connect-mysql-gui). ## Strong security model PlanetScale Passwords are created for use with a single database branch. This strong security model allows you to generate passwords that are tied to a branch, and cannot access data/schema from another branch. ## IP restrictions You can restrict database connections to specific IP ranges for a single password by updating its IP restrictions. For example, if you have a database for a web application and you create a password for use in the deployed application, you can restrict usage of that specific password to the IP ranges of the deployed application. If somebody attempts to connect to the database from outside of the deployed application, the connection will be refused. IP restrictions work on a per-password basis, so if you want to use the same restriction across passwords, they must be applied to each password separately. Some passwords are incompatible with IP restrictions, and you will need to create a new password to use IP restrictions. Examples of when you may want to use IP restrictions: * You want to segment database access so that the production database can only be connected to from production environments or development branches. * You use a bastion in production and want to ensure that all database connections originate or pass through the bastion. * You want to allow a single client to be able to access your database (e.g., for debugging) and want to provide the least amount of access for them to do so. * You have compliance requirements that require implementing a more stringent access control list in your database. ### Updating the IP restrictions for a password 1. Go to your database's "**Settings**" tab. 2. Click "**Passwords**." 3. You can update the IP restrictions for a password in two different ways: The first way is by opening the dropdown menu to the right of any password on the Passwords page and clicking "**Manage IP restrictions**." The second way is by clicking on the password and scrolling to the bottom of its page to update IP restrictions. 4. Add the IP ranges that you want to allow to connect using the selected password. > [!NOTE] > > If your password has no IP restrictions, it is set to **allow all traffic**. Similarly, when you add a new IP range to the restrictions, all IP addresses out of this range cannot connect to your database using that password. ## Disconnect clients by deleting passwords PlanetScale automatically disconnects clients that are using a deleted password. Head on over to the `Organization > Database > Settings > Passwords` page on your database branch to delete passwords for that branch. It may take up to five minutes for all active clients to be disconnected. ## No plain text password storage PlanetScale only stores hashes and metadata about your database passwords. To add an extra layer of security to your database, we do not store any passwords in plaintext. > [!NOTE] > > In the event that you lose a password, we cannot recover it for you. We recommend creating a new password with the > same access level. ## GitHub Secret Scanning integration All passwords and service tokens generated for use with PlanetScale databases are part of [GitHub's Secret Scanning](https://docs.github.com/en/code-security/secret-security/about-secret-scanning) program. If any database passwords or service tokens are committed in plaintext to any public GitHub repository, we will be notified and take corrective action to delete the access tokens and cut off their access. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Password roles Source: https://planetscale.com/docs/vitess/security/password-roles PlanetScale allows you to [create and manage passwords](/docs/vitess/connecting/connection-strings) for each branch of your database. ## Overview > [!NOTE] > > The "roles" on this page are the four **access levels of a Vitess/MySQL branch > password** (Read-only, Write-only, Read/Write, Admin). They are **not** the same > as Postgres **roles**, which are the credential system for PlanetScale Postgres > databases. If your database is Postgres, see [Managing > roles](/docs/postgres/connecting/roles) instead. PlanetScale passwords can be created with one of four roles: * **Read-only** — Can query rows * **Write-only** — Can modify rows * **Read/Write** — Can query and modify rows * **Admin** — All read/write permissions and can modify schema\* \* *This does not apply to production branches with [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled, as we [do not allow direct DDL](/docs/vitess/schema-changes/how-online-schema-change-tools-work) on those branches, even if your password has the `Admin` role.* ## Create a password with custom role **Step 1** Go to your database settings page. **Step 2** Click "**Passwords**" > "**New password**". **Step 3** Give it a name, select the role from the dropdown, select the branch, and click "**Generate password**". ![PlanetScale password roles priority](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/password-roles/roles.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=10bccb8f7e7678d0ff93f9779dff17d8) Once a password is created, **its role cannot be changed**. The access level available to these roles is shown in the table below. | Role name | Can create/edit schema | Can insert/update/delete rows | Can query rows | | :- | :- | :- | :- | | Read-only | | | | | Write-only | | | | | Read/write | | | | | Admin | | | | > [!NOTE] > > The default role for all passwords created by the **Connect** button is `Administrator`. Passwords with custom roles > must be created from your database settings page. ## Troubleshooting The following errors indicate that you do not have the permissions needed to perform an action. You must create a new password with a more privileged role to proceed. **SELECT DENIED** `Select command denied to user ‘planetscale-writer-only for table ‘customers’ (ACL check error) (CallerID: planetscale-writer-only)` **INSERT DENIED** `Insert command denied to user ‘planetscale-reader’ for table ‘customers’ (ACL check error) (CallerID: planetscale-reader)` **DELETE DENIED** `Delete command denied to user ‘planetscale-reader’ for table ‘customers’ (ACL check error) (CallerID: planetscale-reader)` **DDL DENIED** `DDL command denied to user ‘planetscale-writer' for table my-new-table’ (ACL check error) (CallerID: planetscale-writer)` > [!NOTE] > > If your pscale CLI version is less than 0.94.0, please upgrade your installation by following [this > document](/docs/cli/planetscale-environment-setup) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sharding with PlanetScale Source: https://planetscale.com/docs/vitess/sharding > [!NOTE] > > **Looking for Postgres sharding?** [Neki](/docs/neki) is horizontal sharding for Postgres. Start with the [Neki quickstart](/docs/neki/tutorials/planetscale-quick-start-guide) or [when to shard](/docs/neki/when-to-shard). > [!NOTE] > > You can create sharded keyspaces on any plan by adding a new sharded keyspace using the [Clusters page](/docs/vitess/cluster-configuration) and moving tables into it with [MoveTables](/docs/vitess/sharding/sharding-quickstart). > > If you would like additional support from our expert team, our [Enterprise plan](/docs/planetscale-plans#enterprise-plan) may be a good fit. [Get in touch](https://planetscale.com/contact) for a quick assessment. ## Sharding with PlanetScale PlanetScale allows you to break up a large monolithic database and spread the data out across multiple servers. This [reduces the load on a single database](https://planetscale.com/blog/one-million-queries-per-second-with-mysql) by distributing it across many. PlanetScale achieves this by building our product on top of [Vitess](https://vitess.io/), which provides a transparent [sharding solution](/docs/vitess/sharding) for MySQL databases. ### Sharding without application changes Sharding is a proven database architecture used by many organizations to help them scale up when database demand grows. When reaching this point, many organizations choose to scale and shard their database manually. This typically involves adding a bunch of sharding-specific logic to the application layer and/or creating a whole new proxy component to manage the shards. In addition to this, managing a large fleet of disparate database servers is a significant operational challenge. When using PlanetScale, customers can keep sharding logic out of their applications and let us take the burden of infrastructure management. This is because Vitess allows you to create sharded databases and have them appear to the application layer as a single, unified database. This means simpler application architecture, allowing your developers to worry less about the database and more on their work. PlanetScale abstracts away all of this complexity using the **VTGate** layer of Vitess. The VTGates act as the entryway into a Vitess database cluster. They handle incoming connections and route queries to the appropriate MySQL instances. When you're at the point where you've maxed out your vertical scaling efforts and you know you need to shard your database, PlanetScale allows you to do so *without* the burden of rearchitecting your entire application. ## How does our sharding process work? When it comes time to shard your database, we recommend following our [sharding quickstart](/docs/vitess/sharding/sharding-quickstart) guide. If you need assistance identifying the best [sharding scheme](https://vitess.io/docs/reference/features/sharding/#sharding-scheme) for your database, or are interested in expert-level support, our [Enterprise plan](https://planetscale.com/enterprise) may be a better option for you. [Get in touch](https://planetscale.com/contact) for more information. PlanetScale uses an explicit sharding system. This means that, if you are going to horizontally shard your data, we have to tell Vitess which sharding strategy to use for each sharded table. This involves some conversations to understand what your application does and the size, schema, and queries of your database. Once we have this complete view of your application, our next objective is to identify a sharding key. The sharding key, or [Primary Vindex](https://vitess.io/docs/reference/features/vindexes) is what determines how the rows of your sharded tables will be distributed across servers. For each table you want to shard, we must choose which column to use for the Vindex, and what [type of vindex](https://vitess.io/docs/reference/features/vindexes/#predefined-vindexes) to use with it. Each shard will cover a range of keyspace ID values, so this mapping is used to identify which shard a row is in. To determine a Primary Vindex, our team will analyze your schema and query patterns. We generally will ask you for the following information: 1. A copy of your schema. 2. Some indication of the size of each table in your database. We can typically gather this information by looking at `AUTO_INCREMENT` values, but may require additional context in some cases. 3. Information about your common query patterns — typically your most frequently used 50-100 queries. Using this holistic view of your database, we can work to determine a good candidate for the Primary Vindex. During this analysis, we also begin to determine which tables should be sharded, whether you'll require secondary Vindexes ([Lookup Vindexes](https://vitess.io/docs/reference/features/vindexes/#functional-and-lookup-vindex)), the strategy for tables that don't contain our chosen Primary Vindex, and more. While no sharding strategy can ever optimize for *all* query patterns, this deep analysis makes the strategy as efficient as possible for the majority of queries, especially the most frequently used queries. And, again, this is all done without requiring you to rearchitect your application. ## When should you consider sharding a table We generally recommend sharding when you're running into issues with: * Application performance due to **data size** (this can vary widely, but when your database exceeds 250 GB or a particular table becomes especially large, sharding may be something to explore) * Hitting vertical limits with **write throughput** * Hitting replica limits with **read throughput** We often see customers running into other infrastructure challenges before they really see their application performance impacted by large amounts of data. As your data grows, you may find your backups are becoming unreliable and time-consuming. Or you may have no way to test those large backups. If it feels like some things are starting to break down in your infrastructure, but you aren't sure if sharding is the solution, we can still help you identify if it's the correct strategy for scaling your database. If you would like to explore whether or not sharding is the right solution for you, [don't hesitate to reach out](https://planetscale.com/contact), and we'll be in touch. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # ALTER VSCHEMA syntax reference Source: https://planetscale.com/docs/vitess/sharding/alter-vschema A complete reference for the ALTER VSCHEMA statements you can use to modify the VSchema of a keyspace. The [VSchema](/docs/vitess/sharding/vschema) describes how your data is organized across keyspaces and shards: which tables exist in each keyspace, how sharded tables map rows to shards via [Vindexes](/docs/vitess/sharding/vindexes), and how IDs are generated with [sequence tables](/docs/vitess/sharding/sequence-tables). `ALTER VSCHEMA` statements let you modify the VSchema using SQL over a regular MySQL connection to your database branch, as an alternative to editing the VSchema JSON directly in the [Clusters page](/docs/vitess/cluster-configuration) or with the [pscale CLI](/docs/cli/keyspace). > [!WARNING] > > `ALTER VSCHEMA` statements do not work on a branch with [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled, which is the recommended setup for production branches. > Instead, run them on a [development branch](/docs/vitess/schema-changes/branching) and promote the changes to production with a [deploy request](/docs/vitess/schema-changes/deploy-requests). > See [Modifying VSchema](/docs/vitess/sharding/vschema#modifying-vschema) for the full workflow. > [!NOTE] > > `ALTER VSCHEMA` only changes Vitess routing metadata. > It never creates, alters, or drops the underlying MySQL tables — you manage those with regular DDL. ## Statement overview | Statement | Purpose | | - | - | | [`ALTER VSCHEMA ADD TABLE`](#add-a-table) | Add a table to an unsharded keyspace's VSchema | | [`ALTER VSCHEMA DROP TABLE`](#drop-a-table) | Remove a table from a keyspace's VSchema | | [`ALTER VSCHEMA ADD SEQUENCE`](#add-a-sequence) | Register a table as a sequence table | | [`ALTER VSCHEMA DROP SEQUENCE`](#drop-a-sequence) | Remove a sequence table from the VSchema | | [`ALTER VSCHEMA ... ADD AUTO_INCREMENT`](#add-an-auto-increment-column) | Back a column with a sequence table | | [`ALTER VSCHEMA ... DROP AUTO_INCREMENT`](#drop-an-auto-increment-column) | Remove the sequence backing from a table | | [`ALTER VSCHEMA CREATE VINDEX`](#create-a-vindex) | Define a Vindex in a keyspace | | [`ALTER VSCHEMA DROP VINDEX`](#drop-a-vindex) | Remove a Vindex definition from a keyspace | | [`ALTER VSCHEMA ... ADD VINDEX`](#add-a-vindex-to-a-table) | Attach a Vindex to one or more columns of a table | | [`ALTER VSCHEMA ... DROP VINDEX`](#drop-a-vindex-from-a-table) | Detach a Vindex from a table | ## How keyspaces are resolved Every `ALTER VSCHEMA` statement operates on exactly one keyspace. Vitess determines which one as follows: 1. If the table name is qualified (`keyspace.table`), the qualifier wins. 2. Otherwise, the keyspace you are currently connected to (or selected with `USE`) is used. 3. If neither is available, the statement fails with an error. ```sql theme={null} -- Explicit keyspace qualifier ALTER VSCHEMA ADD TABLE product.product; -- Uses the current keyspace USE product; ALTER VSCHEMA ADD TABLE product; ``` > [!NOTE] > > Keyspace, table, and column names that contain special characters (such as hyphens) or collide with reserved words (such as `binary`) must be quoted with backticks, just like in regular MySQL statements: > > ```sql theme={null} > ALTER VSCHEMA ON `my-sharded-keyspace`.users ADD VINDEX shard_index(id) USING xxhash; > ``` ## Managing tables ### Add a table ```sql theme={null} ALTER VSCHEMA ADD TABLE [.]; ``` Adds a table to the VSchema of an **unsharded** keyspace. In an unsharded keyspace, tables carry no Vindex information, so the VSchema entry is simply the table name. ```sql theme={null} -- Example ALTER VSCHEMA ADD TABLE product.product; ``` This statement fails if: * The keyspace is sharded. In a sharded keyspace, every table needs a primary Vindex, so use [`ALTER VSCHEMA ... ADD VINDEX`](#add-a-vindex-to-a-table) instead — it creates the table entry for you. * The table is already present in the VSchema. ### Drop a table ```sql theme={null} ALTER VSCHEMA DROP TABLE [.]; ``` Removes a table from the keyspace's VSchema. Unlike `ADD TABLE`, this works on both sharded and unsharded keyspaces. The underlying MySQL table is not touched. ```sql theme={null} -- Example ALTER VSCHEMA DROP TABLE product.product; ``` This statement fails if the table is not present in the VSchema. ## Managing sequences [Sequence tables](/docs/vitess/sharding/sequence-tables) provide auto-increment-style ID generation for sharded tables. The sequence table itself must live in an **unsharded** keyspace. ### Add a sequence ```sql theme={null} ALTER VSCHEMA ADD SEQUENCE [.]; ``` Registers a table in an unsharded keyspace as a sequence table (the VSchema entry gets `"type": "sequence"`). ```sql theme={null} -- Example ALTER VSCHEMA ADD SEQUENCE product.customer_seq; ``` The backing MySQL table must be created separately, with the required schema and the `vitess_sequence` comment, and seeded with a single row: ```sql theme={null} CREATE TABLE customer_seq ( id BIGINT, next_id BIGINT, cache BIGINT, PRIMARY KEY (id) ) COMMENT 'vitess_sequence'; INSERT INTO customer_seq (id, next_id, cache) VALUES (0, 1, 1000); ``` This statement fails if: * The keyspace is sharded. * A table with that name is already present in the VSchema. ### Drop a sequence ```sql theme={null} ALTER VSCHEMA DROP SEQUENCE [.]; ``` Removes a sequence table from the keyspace's VSchema. The backing MySQL table is not touched. ```sql theme={null} -- Example ALTER VSCHEMA DROP SEQUENCE product.customer_seq; ``` This statement fails if: * The keyspace is sharded. * The sequence is not present in the VSchema. ## Managing auto-increment columns ### Add an auto-increment column ```sql theme={null} ALTER VSCHEMA ON [.] ADD AUTO_INCREMENT USING [.]; ``` Configures a column to be populated from a [sequence table](/docs/vitess/sharding/sequence-tables) whenever an `INSERT` does not provide a value for it — typically the primary key of a sharded table. The sequence table is usually in a different (unsharded) keyspace, in which case it must be qualified. ```sql theme={null} -- Example ALTER VSCHEMA ON customer.customer ADD AUTO_INCREMENT customer_id USING product.customer_seq; ``` This statement fails if: * The table is not present in the keyspace's VSchema. Add its primary Vindex first with [`ALTER VSCHEMA ... ADD VINDEX`](#add-a-vindex-to-a-table). * The table already has an auto-increment column configured. A table can only have one; drop the existing one first. ### Drop an auto-increment column ```sql theme={null} ALTER VSCHEMA ON [.] DROP AUTO_INCREMENT; ``` Removes the sequence backing from a table. Inserts will no longer pull values from the sequence table. ```sql theme={null} -- Example ALTER VSCHEMA ON customer.customer DROP AUTO_INCREMENT; ``` This statement fails if: * The table is not present in the keyspace's VSchema. * The table has no auto-increment column configured. ## Managing Vindexes A [Vindex](/docs/vitess/sharding/vindexes) definition lives at the keyspace level and consists of a name, a type, and optional parameters. Tables then reference a Vindex by name, binding it to one or more of their columns. The first Vindex on a table is its **primary Vindex** — the one that decides which shard each row lives on. You will usually use [`ALTER VSCHEMA ... ADD VINDEX`](#add-a-vindex-to-a-table), which defines the Vindex and attaches it to a table in one statement. `CREATE VINDEX` exists for defining a Vindex ahead of time, without attaching it to any table. ### Create a Vindex ```sql theme={null} ALTER VSCHEMA CREATE VINDEX [.] USING [WITH =, =, ...]; ``` Defines a Vindex in the keyspace without attaching it to a table. `` is one of the [predefined Vindex types](https://vitess.io/docs/reference/features/vindexes/#predefined-vindexes), such as `xxhash`, `hash`, `unicode_loose_xxhash`, `binary`, `numeric`, `consistent_lookup`, or `consistent_lookup_unique`. ```sql theme={null} -- Example ALTER VSCHEMA CREATE VINDEX customer.shard_index USING xxhash; ``` This statement fails if a Vindex with that name already exists in the keyspace. > [!WARNING] > > Defining a Vindex is what makes a keyspace sharded: if this is the first Vindex in the keyspace, Vitess flips the keyspace to sharded in its VSchema. > There is no such thing as a Vindex on an unsharded keyspace, so only run Vindex statements against keyspaces that are meant to be sharded. ### Drop a Vindex ```sql theme={null} ALTER VSCHEMA DROP VINDEX [.]; ``` Removes a Vindex definition from the keyspace. ```sql theme={null} -- Example ALTER VSCHEMA DROP VINDEX customer.shard_index; ``` This statement fails if: * The Vindex does not exist in the keyspace. * Any table still uses the Vindex. Detach it from every table first with [`ALTER VSCHEMA ... DROP VINDEX`](#drop-a-vindex-from-a-table). ### Add a Vindex to a table ```sql theme={null} ALTER VSCHEMA ON [.] ADD VINDEX ( [, ...]) [USING [WITH =, =, ...]]; ``` Attaches a Vindex to one or more columns of a table. This is the statement that makes a table sharded: the first Vindex added to a table becomes its **primary Vindex**. If the table is not yet in the VSchema, the entry is created automatically. The `USING` clause is optional and controls whether the Vindex definition is created along the way: * **With `USING`**: if no Vindex named `vindex_name` exists in the keyspace, it is created with the given type and parameters. If it already exists, the type, owner, and parameters must match the existing definition exactly. * **Without `USING`**: the Vindex named `vindex_name` must already exist in the keyspace (created earlier via `CREATE VINDEX` or a previous `ADD VINDEX ... USING`). ```sql theme={null} -- Add a primary Vindex (creates both the Vindex and the table entry) ALTER VSCHEMA ON customer.customer ADD VINDEX shard_index(customer_id) USING xxhash; -- Reuse an existing Vindex on another table (no USING clause) ALTER VSCHEMA ON customer.corder ADD VINDEX shard_index(customer_id); -- A multi-column lookup Vindex, defined inline (note the backtick-quoted -- comma-separated column list in the from parameter) ALTER VSCHEMA ON customer.corder ADD VINDEX corder_region_idx(region, corder_id) USING consistent_lookup WITH owner=corder, table=`product.corder_region_idx`, from=`region,corder_id`, to=`keyspace_id`; ``` This statement fails if: * A Vindex with the same name is already attached to the table. * `USING` is given but the existing Vindex definition has a different type, owner, or parameters. * `USING` is omitted and no Vindex with that name exists in the keyspace. > [!WARNING] > > The order in which Vindexes are attached matters: the first one is the primary Vindex. > There is no statement to reorder Vindexes on a table, so add the primary Vindex first. > If you get the order wrong, detach the table's Vindexes with [`ALTER VSCHEMA ... DROP VINDEX`](#drop-a-vindex-from-a-table) and re-add them, primary first. > [!WARNING] > > Like `CREATE VINDEX`, adding the first Vindex to any table in a keyspace marks that keyspace as sharded in its VSchema. > Most keyspaces become sharded through this statement rather than through a standalone `CREATE VINDEX`. ### Drop a Vindex from a table ```sql theme={null} ALTER VSCHEMA ON [.] DROP VINDEX ; ``` Detaches a Vindex from a table. The keyspace-level Vindex definition is kept; use [`ALTER VSCHEMA DROP VINDEX`](#drop-a-vindex) to remove it entirely. If this was the last Vindex on the table, the table entry is removed from the VSchema as well. ```sql theme={null} -- Example ALTER VSCHEMA ON customer.corder DROP VINDEX corder_region_idx; ``` This statement fails if: * The table is not present in the keyspace's VSchema. * No Vindex with that name is attached to the table. ## Vindex parameters The `WITH` clause passes key-value parameters to the Vindex. Which parameters are valid depends on the Vindex type — most functional Vindexes such as `xxhash` take none, while [lookup Vindexes](https://vitess.io/docs/reference/features/vindexes/#lookup-vindex-types) require several: | Parameter | Meaning | | - | - | | `owner` | The table that owns the lookup Vindex. Vitess keeps the lookup table up to date as rows are inserted, updated, and deleted in the owner table. | | `table` | The lookup table that stores the mapping, as `keyspace.table`. | | `from` | Comma-separated list of column(s) in the lookup table that hold the input value(s). | | `to` | The column in the lookup table that holds the keyspace ID. | Only parameter values that contain special characters need backticks — such as the dot in a qualified `table` reference or the comma in a multi-column `from` list. Plain values like `owner=corder` are left unquoted: ```sql theme={null} ALTER VSCHEMA ON customer.corder ADD VINDEX corder_keyspace_idx(corder_id) USING consistent_lookup_unique WITH owner=corder, table=`product.corder_keyspace_idx`, from=corder_id, to=keyspace_id; ``` See the [Vitess Vindex documentation](https://vitess.io/docs/reference/features/vindexes/) for the full list of Vindex types and their parameters. ## A complete example This example shards a `customer` keyspace while keeping reference data and sequence tables in an unsharded `product` keyspace. ```sql theme={null} -- Register a table in the unsharded keyspace ALTER VSCHEMA ADD TABLE product.product; -- Shard the customer table on customer_id ALTER VSCHEMA ON customer.customer ADD VINDEX shard_index(customer_id) USING xxhash; -- Give it sequence-backed IDs (the customer_seq MySQL table must already exist -- and be seeded — see "Add a sequence" above for the CREATE TABLE and INSERT) ALTER VSCHEMA ADD SEQUENCE product.customer_seq; ALTER VSCHEMA ON customer.customer ADD AUTO_INCREMENT customer_id USING product.customer_seq; -- Shard the corder table on customer_id too, so a customer's orders -- live on the same shard as the customer ALTER VSCHEMA ON customer.corder ADD VINDEX shard_index(customer_id); ALTER VSCHEMA ADD SEQUENCE product.corder_seq; ALTER VSCHEMA ON customer.corder ADD AUTO_INCREMENT corder_id USING product.corder_seq; -- Add a unique lookup Vindex so corder rows can also be found by corder_id ALTER VSCHEMA ADD TABLE product.corder_keyspace_idx; ALTER VSCHEMA ON customer.corder ADD VINDEX corder_keyspace_idx(corder_id) USING consistent_lookup_unique WITH owner=corder, table=`product.corder_keyspace_idx`, from=corder_id, to=keyspace_id; ``` ## Verifying your changes `ALTER VSCHEMA` changes take effect immediately. You can inspect the resulting VSchema over the same MySQL connection: ```sql theme={null} -- List all keyspaces and whether they are sharded SHOW VSCHEMA KEYSPACES; -- List all tables in the current keyspace's VSchema SHOW VSCHEMA TABLES; -- List all Vindexes in the current keyspace SHOW VSCHEMA VINDEXES; -- List the Vindexes attached to a specific table SHOW VSCHEMA VINDEXES ON customer.corder; ``` > [!NOTE] > > `SHOW VSCHEMA TABLES` and `SHOW VSCHEMA VINDEXES` (without `ON`) report on the keyspace your connection is currently using — there is no keyspace-qualifier form. Switch keyspaces with `USE ` to inspect a different one. You can also view the full VSchema JSON in the [Clusters page](/docs/vitess/cluster-configuration) or fetch it with the [pscale CLI](/docs/cli/keyspace). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Avoiding cross-shard queries Source: https://planetscale.com/docs/vitess/sharding/avoiding-cross-shard-queries When designing your database sharding scheme, it's important to think about your common query patterns with the goal of avoiding cross-shard queries. You might start this exercise by deciding which table(s) you wish to shard. You probably already have some good candidates in mind which you know need to be sharded. Once you have these base table(s) down that you need to shard, the next step is to think about what tables you frequently `JOIN` with these sharded tables. Let's run through an example. If you prefer video content, you can watch the video on avoiding cross-shard queries here: [Avoiding cross-shard queries](https://www.youtube-nocookie.com/embed/4gzv5YP0D9A?rel=0) ## Sharding design example I have an unsharded database, `metal`, in a single [keyspace](/docs/vitess/sharding/keyspaces). This is the database for our gym tracker application. The database schema looks like this: **`users` table:** Stores the users that sign up to track their gym sessions. There are currently 200,000 users. ``` -- users +--------------------+-----------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +--------------------+-----------------+------+-----+---------+----------------+ | id | bigint unsigned | NO | PRI | NULL | auto\_increment | | name | varchar(255) | NO | | NULL | | | email | varchar(255) | NO | | | | | created_at | datetime(6) | NO | | NULL | | | encrypted_password | varchar(255) | NO | | | | | username | varchar(255) | NO | | NULL | | +--------------------+-----------------+------+-----+---------+----------------+ ``` **`exercises` table:** Stores all of the exercises that our users can track. This table is managed by the application owners. Users cannot add their own exercises. There are currently 200 exercises in the database. ``` -- exercises +-------------+-----------------+------+------+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+-----------------+------+------+---------+----------------+ | id | bigint unsigned | NO | PRI | NULL | auto_increment | | name | varchar(255) | NO | NULL | | | | created_at | datetime(6) | NO | NULL | | | | description | varchar(1000) | YES | NULL | | | +-------------+-----------------+------+------+---------+----------------+ ``` **`exercise_logs` table:** Stores all of the exercises that a user completes. Each `exercise_log` record holds all of the sets and reps you do for a single exercise at a specific weight. For example, if you do 5 sets of 3 reps of squats at a weight of 225 lbs, that is stored in a single `exercise_log`. If you do some warmup sets of squats at a weight of 135 lbs, that's stored in a new `exercise_log`. There are currently 65,155,000 exercise logs. ``` -- exercise_logs +-------------+-------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+-------------------+------+-----+---------+----------------+ | id | bigint | NO | PRI | NULL | auto_increment | | created_at | datetime(6) | NO | | NULL | | | user_id | bigint unsigned | NO | MUL | NULL | | | exercise_id | bigint unsigned | NO | MUL | NULL | | | reps | smallint unsigned | YES | | NULL | | | sets | smallint unsigned | YES | | NULL | | | notes | varchar(1024) | YES | | NULL | | | weight | smallint unsigned | YES | | NULL | | +-------------+-------------------+------+-----+---------+----------------+ ``` **`programs` table:** Stores the available pre-created gym programs that a user can optionally choose. These are created by the application owners. A user cannot create their own programs. There are currently 23 programs. ``` -- programs +---------------+------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +---------------+------------------+------+-----+---------+----------------+ | id | bigint unsigned | NO | PRI | NULL | auto_increment | | weeks | tinyint | YES | | NULL | | | days_per_week | tinyint unsigned | YES | | NULL | | | name | varchar(255) | NO | | NULL | | | description | varchar(255) | YES | | NULL | | +---------------+------------------+------+-----+---------+----------------+ ``` ## Selecting candidate(s) for sharding Based on the information above, you can see that `exercise_logs` will be the first natural candidate for sharding. Each user adds several `exercise_logs` records every time they track a workout, so this table grows much faster than the other three tables. We have decided to shard this table to spread the data across two shards/clusters. ## Common query pattern analysis Now that we've identified the table (or in some cases, tables) we want to shard, the next step is to look at our common query patterns to see if we frequently join this table to other tables. Again, here is the schema for `exercise_logs`: ``` -- exercise_logs +-------------+-------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+-------------------+------+-----+---------+----------------+ | id | bigint | NO | PRI | NULL | auto_increment | | created_at | datetime(6) | NO | | NULL | | | user_id | bigint unsigned | NO | MUL | NULL | | | exercise_id | bigint unsigned | NO | MUL | NULL | | | reps | smallint unsigned | YES | | NULL | | | sets | smallint unsigned | YES | | NULL | | | notes | varchar(1024) | YES | | NULL | | | weight | smallint | YES | | NULL | | +-------------+-------------------+------+-----+---------+----------------+ ``` Given that this table has two referential ids: `user_id` and `exercise_id`, it is likely that we often join this table to both `users` and `exercises`. In fact, some of our most executed queries involve joining all of these tables. The `programs` table, however, is never joined with `exercise_logs`. Given this information, we now must decide how to architect our sharding scheme such that we can avoid cross-shard queries as often as possible. ## A closer look at our sharded table Before we dive into the other tables, let's look a little closer at `exercise_logs`. Here is an example of a common query we run to display all of the exercises a user has completed today: ```sql theme={null} SELECT exercise_log.*, users.name AS user_name, users.email, exercises.name AS exercise_name FROM exercise_log JOIN users ON exercise_log.user_id = users.id JOIN exercises ON exercise_log.exercise_id = exercises.id WHERE exercise_log.user_id = 5 AND DATE(exercise_log.created_at) = CURDATE(); ``` This joins both the `exercises` table and the `users` table on `exercise_logs`. For the sake of this example, let's assume this is the most commonly executed query in our application. Now, we know we are going to shard `exercise_logs`, which means we need to choose a sharding key, or [primary Vindex](/docs/vitess/sharding/vindexes), for this table. ## Choosing a Vindex for `exercise_logs` Using the primary key as the primary Vindex can sometimes be the most natural choice to shard on. Let's start by considering `id` as the shard key for `exercise_logs`. For every row in the table, the `id` will be hashed with [xxHash64](https://vitess.io/docs/reference/features/vindexes/), resulting in some hexadecimal value between `0x00000000000000000` and `0xFFFFFFFFFFFFFFFF`. These values will be evenly distributed across all shards. With this solution, if we run the above query to get all of the exercises done by a particular user in a given day, we'll have to look across all shards because there's no guarantee that the user's exercise logs all ended up on the same shard. Instead, we might want to consider sharding on `exercise_logs.user_id`. If we do this, every `exercise_log` record with the same `exercise_logs.user_id` will hash to the same value. So let's say the hashed `user_id` of `2` comes out to `0x30F419900AA88B20`. Every `exercise_log` for our user with `id`=`2` will have a Vindex value of `0x30F419900AA88B20`. When Vitess distributes this data across our 2 shards, the distrubtion might look like this: **Shard 1**: Vindexes with values `0x0000000000000000` through `0x7FFFFFFFFFFFFFFF` **Shard 2**: Vindexes with values `0x8000000000000000` through `0xFFFFFFFFFFFFFFFF` In this case, our user whose `id` hashes to `0x30F419900AA88B20` will always end up on Shard 1. This means that ultimately, with this primary Vindex, there will not be a case where a user's `exercise_logs` records live on two different shards. When a request comes in to grab all of the exercise logs for a particular user on a given day, we only have to access a single shard. This is exactly what we want. ## Handling frequently `JOIN`ed tables With that out of the way, let's again look at how we handle joining the other relevant tables. With our current setup, we have `exercises`, `users`, and `programs` on the unsharded keyspace and the `exercise_logs` table on the sharded keyspace, as shown below: ![Example of cross-shard joins](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/sharding/cross-shard-queries/cross-shard-tables.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=9f193955b350e7e98da1859e9039b3a7) This means that every time we run the above query, we're doing cross-keyspace `JOIN`s. In this case, we'll see a massive hit to performance, and application speed will feel slow to the end user. Now that we have a good grasp on what we'd like to avoid, let's come up with some solutions. The main thing we need to solve is how to avoid cross-keyspace / cross-shard joins between `exercise_logs`, `users`, and `exercises`. ### The `users` table Let's start by looking at the `users` table. We already know we're using `exercise_logs.user_id` as the primary Vindex, so all exercise logs for a particular user will end up on the same shard. However, when we join that `user_id` on the `users` table, we have to jump back over to the `metal` keyspace to access the `users` table. To avoid this, we should move the `users` table to the `metal-sharded` keyspace and shard that as well. We'll need to choose a primary Vindex for `users` in order to shard it. Because we sharded `exercise_logs` on the `user_id`, we now have a great option for the `users` primary vindex: `users.id`. Hashing on `users.id` will guarantee that for every user, both their user record and exercise logs all end up on the same shard. Our cluster now looks like this: ![Example of cross-shard joins](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/sharding/cross-shard-queries/cross-shard-tables-2.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=a3fe34175c5d88e26a814994be4a7a8b) ### The `exercises` table The final table we need to deal with is the `exercises` table. This is a very small table with only 200 records. Users are not allowed to modify this table, so we have a predictable and slow growth rate with this one. Let's say we expect it to never exceed 1000 records. We could shard this table, but given that each record here could be associated with any user or any exercise log, we don't have a great path to ensure there won't be any cross-shard queries. An alternative option in this case is to use a [reference table](https://vitess.io/docs/reference/vreplication/reference_tables/) to make a copy of this table on every shard. This way, any time you want to join `exercise_logs` to the `exercises` table, the entire table already exists on the same shard as the exercise log. Reference tables can be extremely useful in scenarios like this where the table is small and not frequently updated. If, however, this table frequently modified, this could be a poor solution. Every time a record is updated in the table, it must be updated across all shards as well. This is not a problem in our scenario, but keep this tradeoff in mind when choosing to use reference tables. ## A look at our final cluster setup Here is a recap of what we've chosen for our `metal` database cluster: * Sharded `exercise_logs` and `users` * Used `exercise_logs.user_id` as the primary Vindex for `exercise_logs` * Used `users.id` as the primary Vindex for `users` * Used a reference table to copy `exercises` to every shard in our `sharded-metal` keyspace ![Example of avoiding cross-shard queries](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/sharding/cross-shard-queries/avoiding-cross-shard-joins.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=b2b873b1de5c4e8fef88b3ad4b56ea8b) With this setup, running our most common query does not involve any cross-keyspace or cross-shard queries: ```sql theme={null} SELECT exercise_log.\*, users.name AS user_name, users.email, exercises.name AS exercise_name FROM exercise_log JOIN users ON exercise_log.user_id = users.id JOIN exercises ON exercise_log.exercise_id = exercises.id WHERE exercise_log.user_id = 5 AND DATE(exercise_log.created_at) = CURDATE(); ``` ## What next? This was a simple example meant to get you thinking about how to design your sharding scheme. You likely have several commonly executed queries. It's of course nearly impossible to optimize for every single query, so what you want to do is optimize for the **most common** queries with the goal of avoiding cross-shard and cross-keyspace queries. If you're on the PlanetScale Enterprise Support plan, we do some of this query analysis alongside you to come up with the best sharding scheme for your database. You can learn more about that process in our [Proof of concept documentation](/docs/plans/managed/proof-of-concept). If you'd like more information about our Enterprise Support, don't hesitate to [reach out](https://planetscale.com/contact). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # What is a keyspace Source: https://planetscale.com/docs/vitess/sharding/keyspaces In Vitess, a keyspace is a logical database that maps to one or many MySQL instances. Keyspaces are used to group MySQL instances, typically to shard certain tables in your database cluster. While a keyspace may contain several MySQL instances across many shards, to your application, it will appear as a single database. All PlanetScale Vitess clusters have one or more keyspaces. ## Unsharded keyspaces If you only have a single, unsharded keyspace in your database cluster, then your keyspace maps directly to your database cluster in PlanetScale. For example, if you have an unsharded PlanetScale database named `gymtracker` with the default primary and 2 replicas, you likely only have a single keyspace. You can reference this keyspace in the same way you normally would your database: ```sql theme={null} use gymtracker; show tables; ``` Unless your database cluster is sharded or you have created multiple unsharded keyspaces (uncommon), you don't necessarily need to understand the concept of a keyspace. Keyspaces become important once you start to consider sharding. ## Sharded keyspaces As your database grows, you may wish to shard some tables in your database cluster. To do this, you will create a new keyspace and add the shards to the sharded keyspace. The following diagram depicts a PlanetScale database with 2 keyspaces: one unsharded and one sharded. The unsharded keyspace has the default 1 primary and 2 replicas. The sharded keyspace contains two shards, each with 1 primary and 2 replicas. ![Keyspace diagram](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/sharding/keyspace-diagram.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=2df1ab1c1dd28419f3ec9293e554d6a8) When a new request comes in, it first goes through our Global Edge Network layer. This Edge layer manages the connection and sends it to the correct Vitess cluster in PlanetScale. From here, the VTGates will parse the incoming query and determine the correct keyspace and shard to route it to. If you want to directly target a keyspace, you can do that with the `use` syntax, but this time, specify the keyspace name. For example, if we created a sharded keyspace for `gymtracker`, we may call it `gymtracker_sharded`. You can directly target the keyspace with the following: ```sql theme={null} use gymtracker_sharded; show tables; ``` ## Routing queries in your application One of the many perks of Vitess/PlanetScale is that you can distribute your data across hundreds of MySQL instances without complicating your application code. As mentioned earlier, all of these keyspaces and shards appear as a single MySQL instance to your application. Once you add multiple keyspaces, you no longer need to specify a database name in your application's database connection configuration code. Our Global Edge Network will correctly route you to the correct Vitess cluster, and the VTGates in your cluster will handle routing the request to the correct keyspace and shard. Some frameworks and ORMs require a database name is specified. In these scenarios, you can set the database name to `@primary`, and your requests will be automatically routed to the correct keyspace/shard. Alternatively, if you have specific queries that you wish to send to replicas, you can use `@replica`. ## Modifying keyspaces in PlanetScale Having 1 unsharded keyspace and 1 sharded is a typical setup for a database that needs sharding. On the [Clusters page](/docs/vitess/cluster-configuration), you are able to customize the number of shards in the sharded keyspace. You can also adjust the instance size for each primary and replica, and you can add additional replicas beyond the default of two if needed. On a sharded keyspace, you can enable [per-shard sizing](/docs/vitess/cluster-configuration/per-shard-sizing) to size individual shards separately. To get a better sense of this, or to configure your keyspaces, click on your [Clusters](/docs/vitess/cluster-configuration) tab in your dashboard. If you have an existing unsharded database, you'll see that database listed there as an unsharded keyspace. If you click "New keyspace", you're able to configure a brand new keyspace here. Production branches can also add an [external keyspace](/docs/vitess/cluster-configuration#create-an-external-keyspace) that connects to an existing MySQL database. The most common use case for creating a new keyspace is to shard one or multiple tables. All of your keyspaces are separate databases. And, again, sharded keyspaces hold multiple databases. However, with the power of Vitess, your application views these all as a single database. It uses the VTGate load balancer to route queries to the correct keyspace, and then the correct shard, and finally the correct primary or replica, as configured. For more information about modifying a keyspace, please see the [Cluster configuration documentation](/docs/vitess/cluster-configuration). ## Limitations PlanetScale currently supports a maximum of 2048 tables per keyspace. If you exceed this limit, you may want to consider sharding — [distributing your tables across multiple keyspaces](/docs/vitess/sharding). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Pre-sharding checklist Source: https://planetscale.com/docs/vitess/sharding/pre-sharding-checklist Replace `AUTO_INCREMENT` columns with sequence tables by hand before you move tables from an unsharded keyspace to a sharded keyspace. > [!NOTE] > > If you create your workflow with `--sharded-auto-increment-handling REPLACE` and `--global-keyspace`, as in the [Sharding quickstart](/docs/vitess/sharding/sharding-quickstart), MoveTables removes `AUTO_INCREMENT` from the target tables and creates the sequence tables for you. You can skip this page. > > Follow the steps below if you would rather create and name the sequence tables yourself. ## Remove `AUTO_INCREMENT` from the sharded tables When a table is spread across multiple shards, using `AUTO_INCREMENT` on your primary key can cause problems. Because each shard is its own separate MySQL instance, the shards do not have the context to know whether or not a primary key for a table entry is already in use on other shards. This means you risk two different table entries being assigned the same primary key. To avoid this, it is a best practice to use [sequence tables](/docs/vitess/sharding/sequence-tables) instead. When you create the workflow, MoveTables copies the schema of each table you're moving to the target keyspace. By default (`--sharded-auto-increment-handling REMOVE`), it removes `AUTO_INCREMENT` from those tables as it copies them, so you don't need to create the target tables yourself. The rest of this page sets up the sequence tables that replace it. This example moves the `users` and `notifications` tables. ## Add sequence tables to unsharded keyspace As mentioned earlier, you should use [sequence tables](/docs/vitess/sharding/sequence-tables) in place of `AUTO_INCREMENT` for your sharded tables. Your sequence tables will live in the source unsharded keyspace. **Step 1** Switch back to your original unsharded keyspace. ```sql theme={null} use `metal`; ``` **Step 2** Create 2 new sequence tables: one for `notifications` and one for `users`. ```sql theme={null} CREATE TABLE `notifications_seq` ( `id` bigint NOT NULL, `next_id` bigint DEFAULT NULL, `cache` bigint DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci COMMENT='vitess_sequence'; CREATE TABLE `users_seq` ( `id` bigint NOT NULL, `next_id` bigint DEFAULT NULL, `cache` bigint DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci COMMENT='vitess_sequence'; ``` ## Add the sequence tables to the VSchema The following will add the sequence tables to the source keyspace VSchema (`metal`): ```sql theme={null} alter vschema add sequence `metal`.notifications_seq; alter vschema add sequence `metal`.users_seq; ``` Next, add the following to specify that those sequence tables should be used as the sequence tables for the sharded tables in the new target keyspace VSchema (`metal-sharded`): ```sql theme={null} alter vschema on metal-sharded.notifications add auto_increment id using `metal`.notifications_seq; alter vschema on metal-sharded.users add auto_increment id using `metal`.users_seq; ``` The resulting VSchema for `metal` will look like this: ```json theme={null} { "tables": { "notifications_seq": { "type": "sequence" }, "users_seq": { "type": "sequence" } } } ``` ## Add the tables to the source keyspace VSchema (`metal`) > [!NOTE] > > If you are using Vitess global routing you may have already completed this. > If so, you can skip this step. You now need to add all tables to your source keyspace (`metal` for this example) VSchema. The VSchema is used to route queries to the proper keyspace. When you only had one keyspace, you didn't need to worry about this. But now that you've added a new sharded keyspace, Vitess will need to check the VSchema of each keyspace to route queries. For more information, see the [VSchema documentation](/docs/vitess/sharding/vschema). For this step, it's often easier to do from the UI instead of with an `ALTER` statement. **Step 1** On the Clusters page, click on your source unsharded keyspace (`metal`). **Step 2** Select the branch you created in the previous step. **Step 3** Click "VSchema". **Step 4** Add in **all tables** that exist in this keyspace. This is what our `metal` keyspace looks like: ```json theme={null} { "tables": { "exercises": {}, "exercise_log": {}, "programs": {}, "users": {} } } ``` **Step 5** Click "Save changes" ## Initialize the sequences when you switch traffic When you switch primary traffic, pass `--initialize-target-sequences` so that each sequence table starts above the highest ID already in its table: ```bash theme={null} pscale branch vtctl move-tables switch-traffic \ --workflow \ --target-keyspace metal-sharded \ --tablet-types PRIMARY \ --initialize-target-sequences ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sequence Tables Source: https://planetscale.com/docs/vitess/sharding/sequence-tables In MySQL, it is common to have an integer primary key that uses the `AUTO_INCREMENT` feature for assigning IDs. However, if you have a horizontally-sharded table, you will not be able to use `AUTO_INCREMENT` for your ID. In such a setup, the rows of the table are distributed across many instances of MySQL. The separate instances of MySQL do not have a built-in way to coordinate which IDs are in use and which are not. Instead, you will need to use a **sequence table**. A [sequence table](https://vitess.io/docs/reference/features/vitess-sequences) is a special table that contains metadata for managing the incrementing ID values for the column of a horizontally sharded table. Each time you create a new horizontally sharded table, you should create the corresponding sequence table and update the VSchema. ## Creating a horizontally-sharded table We recommend keeping [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled for all production databases. Thus, the first step to make schema modifications is to create a new [branch](/docs/vitess/schema-changes/branching), and connect to it via the [command line](/docs/cli). Next, to create a horizontally-sharded table, switch to your desired sharded keyspace. Create a new table in this keyspace, and do *not* use `AUTO_INCREMENT` for your ID column. For example, to create a table in the `test_sharded` keyspace, run: ```sql theme={null} USE test_sharded; CREATE TABLE test(id BIGINT UNSIGNED PRIMARY KEY, data JSON); ``` Next, switch over to the unsharded keyspace that you want to use for sequence tables. Here, you'll create a sequence table. It is good practice to use the same name as the sharded table with `_seq` or `_sequence` appended. Being consistent with this naming will help maintain a clear association between your data tables and sequence tables. ```sql theme={null} USE test_unsharded; CREATE TABLE test_seq(id bigint, next_id bigint, cache bigint, primary key(id)) comment 'vitess_sequence'; ``` We also need to update the [VSchema](/docs/vitess/sharding/vschema) of our database. We need to tell Vitess about this new `SEQUENCE`, let it know to use the `id` column as the shard key, and tell it to use the `test_seq` table for fetching auto incrementing IDs. ```sql theme={null} ALTER VSCHEMA ADD SEQUENCE `test_unsharded`.`test_seq`; ALTER VSCHEMA ON `test_sharded`.`test` ADD VINDEX xxhash(id) USING xxhash; ALTER VSCHEMA ON `test_sharded`.`test` ADD auto_increment id USING `test_unsharded`.`test_seq`; ``` When you are comfortable with your schema changes, create a [deploy request](/docs/vitess/schema-changes/deploy-requests) and merge. ## Sequence table values Unlike vanilla Vitess, PlanetScale will automatically populate the single required row into any sequence table created with the above steps. After merging your deploy request, you should be able to query the sequence table as follows: ```sql theme={null} SELECT * FROM test_unsharded.test_seq; +----+---------+-------+ | id | next_id | cache | +----+---------+-------+ | 0 | 1 | 1000 | +----+---------+-------+ 1 row in set (0.04 sec) ``` * `id` Should always be 0. * `next_id` represents the next ID in the sequence to be fetched. You typically want this to start as 1. * `cache` represents the number of IDs that can be fetched and cached by a VTTablet. For good performance, this should be set to a large number like 1000 or more. We can check that the sequence table is working in assigning IDs by inserting a new row and then querying for the row with ID `1`. ```sql theme={null} INSERT INTO test (data) VALUES ('{"errors": [{"message": "Error message", "code": 10}]}'); Query OK, 1 row affected (0.06 sec) SELECT data FROM test WHERE id=1; +--------------------------------------------------------+ | data | +--------------------------------------------------------+ | {"errors": [{"message": "Error message", "code": 10}]} | +--------------------------------------------------------+ 1 row in set (0.05 sec) ``` > [!NOTE] > > Check out the [Vitess documentation on sequences](https://vitess.io/docs/reference/features/vitess-sequences/) for more information. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Change the number of shards with MoveTables Source: https://planetscale.com/docs/vitess/sharding/sharding-a-sharded-keyspace Add or remove shards in a sharded keyspace by moving its tables to a new sharded keyspace. Useful when a sharded keyspace grows significantly and needs additional shards. Adding or removing shards from a sharded keyspace requires some rebalancing, as you may be moving data from one MySQL instance to another. In PlanetScale, this is done with a Vitess MoveTables [workflow](/docs/vitess/scaling/workflows) from one sharded keyspace to another. This involves creating a new sharded (target) keyspace with the new desired number of shards and transferring all of the data from your current (source) keyspace to the new target. This operation is done with no downtime. > [!NOTE] > > The steps in this documentation are similar to those in the [Sharding quickstart](/docs/vitess/sharding/sharding-quickstart), however, there is one important addition that you cannot skip. Be sure to review [Step 7: Remove `"require_explicit_routing": true`](/docs/vitess/sharding/sharding-a-sharded-keyspace#step-7-remove-require_explicit_routing-true), as it is a crucial step that differs from sharding an unsharded keyspace. * If you are sharding an existing table in an *unsharded* keyspace, follow the instructions in the [Sharding quickstart documentation](/docs/vitess/sharding/sharding-quickstart). * If you are creating a new table that you want in your existing sharded keyspace, follow the instructions in the [Sharding new tables documentation](/docs/vitess/sharding/sharding-new-tables). * If you simply need to adjust the size of each shard, and not the number of shards, you can do so from the [Clusters](/docs/vitess/cluster-configuration) page in the dashboard. > [!WARNING] > > These are advanced configuration settings that expose some of the underlying Vitess configuration of your cluster. > Misconfiguration can cause availability issues. We recommend thoroughly reading through the documentation in the [Sharding section](/docs/vitess/sharding) of the docs prior to making any changes. If you have any questions, please [reach out to our support team](https://planetscale.com/contact?initial=support). Throughout this guide, we will refer to the source keyspace and target keyspace, which are defined as follows: * **Source keyspace**: The original sharded keyspace from which you are moving the tables you wish to shard. * **Target keyspace**: The new sharded keyspace that you are moving the selected tables to. You run the move with [`pscale branch vtctl move-tables`](/docs/cli/move-tables), so make sure you have the [`pscale` CLI installed and authenticated](/docs/cli/planetscale-environment-setup). You must be an [Organization Administrator](/docs/security/access-control#organization-administrator) or a [Database Administrator](/docs/security/access-control#database-administrator) of the database. This guide also assumes that you either are already using `@primary` in your application code to [target your keyspaces](/docs/vitess/sharding/targeting-correct-keyspace) or you do not directly set a database name in your application code. ## Pre-sharding checklist There is a small amount of upfront work that needs to happen prior to sharding your table(s) again. ### 1. Prepare to move all table(s) from source keyspace Some common signals that a table may benefit from further sharding include: * The table has become very large and query performance has degraded due to this * Schema changes to the table take hours * You expect the table to grow quickly and want to shard it further before it becomes a problem We strongly recommend moving all table(s) from the source keyspace to the target keyspace, so that by the end of this process your database has at most two keyspaces: one unsharded keyspace, and one sharded keyspace. ### 2. Create another sharded keyspace To set up another sharded keyspace: **Step 1** Go to the "**Clusters**" tab in the left nav in the PlanetScale dashboard. **Step 2** Click "**New keyspace**". **Step 3** Enter the keyspace name (for example, `metal-sharded-2`). **Step 4** Select the **shard count** and choose the **cluster size** for this keyspace. Keep in mind, creating a sharded keyspace will use the selected size for *each* shard. For example, if you are creating 4 shards and choose the `PS-80` cluster size, we will create 4 `PS-80`s, each with 1 primary and 2 replicas. **Step 5** Select the number of *additional* replicas, if any, that you'd like to add to each cluster. Each cluster comes with 2 replicas by default, so any number you choose will be in addition to those 2. **Step 6** Review the new monthly cost for this keyspace below. This is in addition to your existing unsharded keyspace, as well as any other keyspaces you add. **Step 7** Once satisfied, click "**Create keyspace**". ### 3. Add `"require_explicit_routing": true` If you are using [Vitess global routing](https://vitess.io/docs/reference/features/global-routing/) (for example, if you are using `@primary`), you will get ambiguous table errors once you add Vindexes to your new keyspace. To prevent this error, you **must** temporarily add `require_explicit_routing` to your new keyspace's VSchema: ``` { "sharded": true, "require_explicit_routing": true, } ``` This will instruct Vitess's global routing to exclude your new keyspace from routing until explicitly targeted. This is just temporary. You will remove this later in this tutorial *before* completing the workflow. You can choose to do this step, and the following step, by one of the following methods: * **Safe migrations off**: modify the target keyspace VSchema directly on the Clusters page * **Safe migrations on**: modify the target keyspace VSchema using deploy requests ### 4. Copy Vindexes and auto-increment VSchema settings Assuming your sharding scheme will remain the same, once you've completed the previous step of adding `"require_explicit_routing": true`, you can copy the relevant parts of your source keyspace VSchema into your target keyspace VSchema. Since we recommend moving all tables from the source to the target keyspace, your target VSchema will look exactly the same as your source VSchema, with the addition of `"require_explicit_routing": true`. For example, if you are moving the tables `users` and `exercise_logs`, and your source keyspace VSchema looks like this: ```json expandable theme={null} { "sharded": true, "vindexes": { "xxhash": { "type": "xxhash" } }, "tables": { "exercise_logs": { "column_vindexes": [ { "name": "xxhash", "columns": ["user_id"] } ], "auto_increment": { "column": "id", "sequence": "`unsharded`.`exercise_logs_seq`" } }, "users": { "column_vindexes": [ { "name": "xxhash", "columns": ["id"] } ], "auto_increment": { "column": "id", "sequence": "`unsharded`.`users_seq`" } } } } ``` Your target keyspace VSchema will look like this: ```json expandable theme={null} { "sharded": true, "require_explicit_routing": true, "vindexes": { "xxhash": { "type": "xxhash" } }, "tables": { "exercise_logs": { "column_vindexes": [ { "name": "xxhash", "columns": ["user_id"] } ], "auto_increment": { "column": "id", "sequence": "`unsharded`.`exercise_logs_seq`" } }, "users": { "column_vindexes": [ { "name": "xxhash", "columns": ["id"] } ], "auto_increment": { "column": "id", "sequence": "`unsharded`.`users_seq`" } } } } ``` ## Move the tables with MoveTables The examples below use a database named `mydb`, the `main` production branch, the current `metal-sharded` keyspace as the source, and the new `metal-sharded-2` keyspace as the target. ### Step 1: Create the workflow ```bash theme={null} pscale branch vtctl move-tables create mydb main \ --workflow reshard_metal \ --source-keyspace metal-sharded \ --target-keyspace metal-sharded-2 \ --all-tables \ --defer-secondary-keys \ --format json ``` This creates a MoveTables workflow named `reshard_metal` that moves every table from the source keyspace, and starts it right away. To move only some tables, pass `--tables` with a comma-separated list instead of `--all-tables`. `--defer-secondary-keys` creates secondary indexes after the copy finishes, which makes the copy much faster. ### Step 2: Watch the copy As soon as the workflow starts, Vitess copies rows of the tables from your source keyspace to your target keyspace. It uses a combination of `SELECT * FROM table` and binlog-based replication, and redistributes the rows across the target keyspace's shards using the Vindexes in its VSchema. Check progress with `status`: ```bash theme={null} pscale branch vtctl move-tables status mydb main \ --workflow reshard_metal \ --target-keyspace metal-sharded-2 \ --format json ``` Once every table is copied, the streams move to `Running`: Vitess keeps replicating every new write on the source keyspace to the target keyspace, while the source keyspace still serves all primary and replica traffic. You can also follow the workflow from the dashboard. Click "**Workflows**" in the left nav, select your branch, and open the workflow to see its streams, per-table copy progress, replication lag, and traffic routing. ### Step 3: Verify data consistency Once the streams are `Running`, verify that the source and target keyspaces hold the same data with a VDiff: ```bash theme={null} pscale branch vtctl vdiff create mydb main \ --workflow reshard_metal \ --target-keyspace metal-sharded-2 \ --format json ``` Pass the `uuid` from the output to `vdiff show`, and repeat until the VDiff completes: ```bash theme={null} pscale branch vtctl vdiff show mydb main \ --workflow reshard_metal \ --target-keyspace metal-sharded-2 \ --uuid \ --format json ``` If the VDiff reports mismatches, do not switch traffic until you have resolved them. ### Step 4: Switch replica traffic Switch replica traffic first to test reads from the new keyspace: ```bash theme={null} pscale branch vtctl move-tables switch-traffic mydb main \ --workflow reshard_metal \ --target-keyspace metal-sharded-2 \ --tablet-types REPLICA,RDONLY \ --format json ``` You can skip this step and switch primary traffic directly. Pass `--dry-run` to see what a switch would do without applying it. ### Step 5: Switch primary traffic When reads from replicas look good, switch primary traffic: ```bash theme={null} pscale branch vtctl move-tables switch-traffic mydb main \ --workflow reshard_metal \ --target-keyspace metal-sharded-2 \ --tablet-types PRIMARY \ --format json ``` After primary traffic switches, Vitess replicates writes from the target keyspace back to the source keyspace, so both keep the same data until you complete the workflow. ### Step 6: Check traffic in your application You should now go check out your production application that uses this database to make sure everything is running as expected. You can also check the [Insights](/docs/vitess/monitoring/query-insights) tab for errors or slow queries. If you notice an issue, switch traffic back to the source keyspace: ```bash theme={null} pscale branch vtctl move-tables reverse-traffic mydb main \ --workflow reshard_metal \ --target-keyspace metal-sharded-2 \ --format json ``` ### Step 7: Remove `"require_explicit_routing": true` You should have added `require_explicit_routing` to your target keyspace's VSchema in step 3 of the "Pre-sharding checklist": ``` { "require_explicit_routing": true, ... } ``` You'll need to remove it before completing the workflow. > [!WARNING] > > If you don't remove this, you may start to see "table not found" errors once the workflow is completed. ### Step 8: Complete the workflow Up until now, you can cancel the workflow or reverse traffic. Completing the workflow is not reversible: it stops replication between the keyspaces and, with the flags below, drops the moved tables from the source keyspace and removes the routing rules. Preview what `complete` will do with `--dry-run` first: ```bash theme={null} pscale branch vtctl move-tables complete mydb main \ --workflow reshard_metal \ --target-keyspace metal-sharded-2 \ --keep-data=false \ --keep-routing-rules=false \ --dry-run \ --format json ``` If you have removed `"require_explicit_routing": true` from your target keyspace VSchema, and you are sure you want to proceed, run the same command without `--dry-run`. * `--keep-data=false` drops the moved tables from the source keyspace. Add `--rename-tables` to rename them instead of dropping them. `--keep-data=true` leaves them in place. * `--keep-routing-rules=false` removes the routing rules. Pass `--keep-routing-rules=true` to keep them if some queries still name the source keyspace. Write both flags with an `=`: a space-separated value such as `--keep-data false` is read as `--keep-data=true`. ### Step 9: Check that your production application is working as expected Finally, check your production application to make sure everything is working as expected. You can check your [Insights](/docs/vitess/monitoring/query-insights) tab to see if queries are being properly routed to your new keyspace. Insights will also show you any errors, query performance issues, and more. If you moved every table and the source keyspace has no tables left, you can delete it with [`pscale keyspace delete`](/docs/cli/keyspace) or from the **Clusters** page. That's it! The tables you selected at the beginning are now being served by the new sharded keyspace. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sharding new tables Source: https://planetscale.com/docs/vitess/sharding/sharding-new-tables This tutorial shows you how to create new tables in a sharded keyspace. If you have an existing table that you want to shard, follow the [Sharding quickstart](/docs/vitess/sharding/sharding-quickstart) instead. > [!WARNING] > > Misconfiguration can cause availability issues. We recommend thoroughly reading through the documentation in the [Sharding section](/docs/vitess/sharding) of the docs prior to making any changes. If you have any questions, please [reach out to our support team](https://planetscale.com/contact?initial=support). Before you begin, we recommend the following reading: > **[What is a keyspace?](/docs/vitess/sharding)** > **[Vindexes](/docs/vitess/sharding/vindexes)** > **[Sharding quickstart](/docs/vitess/sharding/sharding-quickstart)** > **[Avoiding cross-shard queries](/docs/vitess/sharding/avoiding-cross-shard-queries)** > **[Sequence tables](/docs/vitess/sharding/sequence-tables)** > [!NOTE] > > Sharded keyspaces are not supported on databases with foreign key constraints enabled. When adding a new table to your database cluster, you have the option to shard it from the start. This can be a good option if you expect the table to grow incredibly quickly, or if you already have some other sharded tables that you know you will frequently join with this new table and want to avoid cross-shard queries. Before you get started, you need a sharded keyspace. If you already have one, you can skip the next section. ## 1. Add a sharded keyspace > [!WARNING] > > If you are using [Vitess global routing](https://vitess.io/docs/reference/features/global-routing/), you must take extra care when adding a sharded keyspace to your database (for example, if you are using `@primary`). > Before creating one, you must ensure that all tables from your first *unsharded keyspace* are added to the `VSchema` of that *unsharded keyspace*. Eg: > > ``` > { > "tables": { > "users": { } > ... > } > } > ``` > > Otherwise, queries will fail. [Learn more about VSchema.](/docs/vitess/sharding/vschema) Navigate to the [Clusters page](/docs/vitess/cluster-configuration). **Step 1** Click "New keyspace". **Step 2** Enter the keyspace name. For example, if your existing unsharded keyspace is named `metal`, you may create a sharded keyspace named `metal-sharded`. **Step 3** Select the number of shards you want to create in this keyspace. > [!NOTE] > > The cost of adding this additional keyspace largely depends on the number of shards you choose, the cluster size, and if you'd like to add additional replicas. **Step 4** Choose the cluster sizes you would like to use for this keyspace. Keep in mind, if you are creating a sharded keyspace, this will spin up multiple clusters of the selected size. For example, if you are creating 4 shards and choose the `PS-80` cluster size, we will create 4 `PS-80`s, each with 1 primary and 2 replicas. **Step 5** Select the number of *additional* replicas, if any, that you'd like to add to each cluster. Each cluster comes with 2 replicas by default, so any number you choose will be in addition to those 2. **Step 6** Review the new monthly cost for this keyspace below. This is in addition to your existing unsharded keyspace, as well as any other keyspaces you add. **Step 7** Once satisfied, click "Create keyspace". ## 2. Copy the sharded tables to the new keyspace and remove `AUTO_INCREMENT` Without considering sharding, you may create a new table that looks like this: ```sql theme={null} -- Don't create a sharded table with `AUTO_INCREMENT` on the primary key CREATE TABLE exercise_logs ( id BIGINT UNSIGNED AUTO_INCREMENT, -- Problem! user_id BIGINT UNSIGNED, exercise_id BIGINT UNSIGNED, created_at DATETIME, reps SMALLINT UNSIGNED, sets SMALLINT UNSIGNED, weight SMALLINT UNSIGNED, notes VARCHAR(1024), PRIMARY KEY(id) ); ``` When a table is spread across multiple shards, using `AUTO_INCREMENT` on your primary key can cause problems. Because each shard is its own separate MySQL instance, the shards do not have the context to know whether or not a primary key for a table entry is already in use on other shards. This means you risk two different table entries being assigned the same primary key. To avoid this, it is a best practice to use [sequence tables](/docs/vitess/sharding/sequence-tables) instead. We will cover how to set these up shortly. First, make sure you remove any `AUTO_INCREMENT`s from the tables you're sharding. ```sql theme={null} -- Switch to the new keyspace USE `metal-sharded`; -- Create the table with no `AUTO_INCREMENT` CREATE TABLE exercise_logs ( id BIGINT UNSIGNED, -- Better! user_id BIGINT UNSIGNED, exercise_id BIGINT UNSIGNED, created_at DATETIME, reps SMALLINT UNSIGNED, sets SMALLINT UNSIGNED, weight SMALLINT UNSIGNED, notes VARCHAR(1024), PRIMARY KEY(id) ); ``` ## 3. Add sequence tables to unsharded keyspace As mentioned earlier, you should use [sequence tables](/docs/vitess/sharding/sequence-tables) in place of `AUTO_INCREMENT` for your sharded tables. Your sequence tables will live in the source unsharded keyspace. **Step 1** Switch back to your original unsharded keyspace. ```sql theme={null} USE `metal`; ``` **Step 2** Create the new sequence table for `exercise_logs`. If you're adding multiple new tables, create a sequence table for each: ```sql theme={null} CREATE TABLE exercise_logs_seq ( id BIGINT, next_id BIGINT, cache BIGINT, PRIMARY KEY(id) -- 👇 Comment is required ) COMMENT='vitess_sequence'; ``` > [!TIP] > > Note `COMMENT='vitess_sequence'` at the end. This must be added for every sequence table you create. ## 4. Choose your Vindexes Every sharded table needs a [Vindex](/docs/vitess/sharding/vindexes) (Vitess index) to determine how incoming rows are distributed across shards. Similar to how every MySQL table must have a primary key, every sharded table must additionally have a **primary Vindex**. The primary Vindex determines which shard each row of data will reside on. For more information about choosing a Vindex, see the [Vindexes documentation](/docs/vitess/sharding/vindexes). You can also see a practical example in the [Avoiding cross-shard queries](/docs/vitess/sharding/avoiding-cross-shard-queries) documentation. In this example, we'll shard `exercise_logs` by `user_id` using the predefined [`xxhash` Vindex function](https://vitess.io/docs/reference/features/vindexes/#predefined-vindexes), which is a common choice: ```sql theme={null} ALTER VSCHEMA ON `metal-sharded`.exercise_logs ADD VINDEX xxhash(user_id) USING xxhash; ``` This registers the table in the sharded keyspace's VSchema and configures how rows are distributed across shards. ## 5. Add the sequence tables to the VSchema Now that the table exists in the sharded keyspace's VSchema, you can configure the sequence tables. The following will add the sequence table to the source keyspace VSchema (`metal`): ```sql theme={null} ALTER VSCHEMA ADD SEQUENCE `metal`.exercise_logs_seq; ``` Next, add the following to specify that the sequence table should be used for the sharded table's primary key in the target keyspace VSchema (`metal-sharded`): ```sql theme={null} ALTER VSCHEMA ON `metal-sharded`.exercise_logs ADD AUTO_INCREMENT id USING `metal`.exercise_logs_seq; ``` ### Verify with pscale CLI You can verify the resulting VSchema for each keyspace with the [`pscale` CLI](/docs/cli/keyspace). Check the `metal` keyspace VSchema: ```bash theme={null} pscale keyspace vschema show metal ``` ```json theme={null} { "tables": { "exercise_logs_seq": { "type": "sequence" } } } ``` Check the `metal-sharded` keyspace VSchema: ```bash theme={null} pscale keyspace vschema show metal-sharded ``` ```json theme={null} { "sharded": true, "vindexes": { "xxhash": { "type": "xxhash" } }, "tables": { "exercise_logs": { "column_vindexes": [ { "name": "xxhash", "columns": ["user_id"] } ], "auto_increment": { "column": "id", "sequence": "metal.exercise_logs_seq" } } } } ``` The `column_vindexes` entry tells Vitess to use the `xxhash` Vindex on the `user_id` column to determine shard placement. The `auto_increment` entry configures the sequence table for generating unique primary keys across shards. ## 6. Add the tables to the source keyspace VSchema (`metal`) You now need to add all tables to your source keyspace (`metal` for this example) VSchema. The VSchema is used to route queries to the proper keyspace. When you only had one keyspace, you didn't need to worry about this. But now that you've added a new sharded keyspace, Vitess will need to check the VSchema of each keyspace to route queries. For more information, see the [VSchema documentation](/docs/vitess/sharding/vschema). For this step, it's often easier to do from the UI instead of with an `ALTER` statement. **Step 1** On the Clusters page, click on your source unsharded keyspace (`metal`). **Step 2** Select the branch you created in the previous step. **Step 3** Click "VSchema". **Step 4** Add in **all tables** that exist in this keyspace. This is what our `metal` keyspace looks like: ```json theme={null} { "tables": { "exercises": {}, "exercise_log": {}, "programs": {}, "users": {} } } ``` **Step 5** Click "Save changes" ## 7. Targeting the correct keyspace Once you have more than one keyspace, with tables distributed across both keyspaces, your application may not know how to properly route queries to the correct keyspace. If you originally set up your application configuration code with something like `DATABASE_NAME=your_database_name`, where `your_database_name` is the name of your original unsharded keyspace, you will need to update your configuration code so that all queries don't go straight to that keyspace. The preferred way to do this is to just leave off the database name completely in your application configuration code. PlanetScale will be able to route traffic correctly just using the connection username and password. While this is the preferred way, it's sometimes not possible. For example, many frameworks and ORMs require that you include a database name. In those cases, you should use `@primary`. This will send any incoming queries first to our [Global Edge Network](https://planetscale.com/blog/introducing-global-replica-credentials#building-planetscale-global-network), which will see that you're targeting a primary. Edge will then send the request to the VTGate(s)/load balancer. We typically will use [Vitess's Global Routing](https://vitess.io/docs/reference/features/global-routing/) to direct the query to the correct keyspace and, optionally, correct shard. > [!NOTE] > > If you explicitly wish to target a replica for some or all reads, using `@replica` will have the same effect as `@primary` in that it will automatically route the request to the correct keyspace. [Global Replica Credentials](/docs/vitess/scaling/replicas#1-create-a-global-replica-credential-recommended) are not currently supported in this context. You can still target replicas instead of your primary with `@replica`, but it will not automatically route the query to the *closest* replica. For more information, refer to the [Targeting the correct keyspace documentation](/docs/vitess/sharding/targeting-correct-keyspace). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Shard existing tables with MoveTables Source: https://planetscale.com/docs/vitess/sharding/sharding-quickstart Move **existing tables** in your PlanetScale database from an unsharded keyspace to a sharded keyspace with a Vitess MoveTables [workflow](/docs/vitess/scaling/workflows). * If you are creating a new table that you want in a sharded keyspace, follow the instructions in the [Sharding new tables doc](/docs/vitess/sharding/sharding-new-tables). * If you are moving tables from one sharded keyspace to another sharded keyspace, follow the instructions in the [Sharding a sharded keyspace doc](/docs/vitess/sharding/sharding-a-sharded-keyspace). Before you begin, we recommend the following reading: > **[Workflows overview](/docs/vitess/scaling/workflows)** > **[What is a keyspace?](/docs/vitess/sharding)** > **[Vindexes](/docs/vitess/sharding/vindexes)** > **[Avoiding cross-shard queries](/docs/vitess/sharding/avoiding-cross-shard-queries)** > [!NOTE] > > Sharded keyspaces are not supported on databases with foreign key constraints enabled. > [!WARNING] > > These are advanced configuration settings that expose some of the underlying Vitess configuration of your cluster. Misconfiguration can cause availability issues. We recommend thoroughly reading through the documentation in the [Sharding section](/docs/vitess/sharding) of the docs prior to making any changes. If you have any questions, please [reach out to our support team](https://planetscale.com/contact?initial=support). Throughout this guide, we will refer to the source keyspace and target keyspace, which are defined as follows: * **Source keyspace**: The original unsharded keyspace from which you are moving the tables you wish to shard. * **Target keyspace**: The new sharded keyspace that you are moving the selected tables to. * **Global keyspace**: An unsharded keyspace that holds the [sequence tables](/docs/vitess/sharding/sequence-tables) that replace `AUTO_INCREMENT` on the sharded tables. This is usually the source keyspace. You run the move itself with [`pscale branch vtctl move-tables`](/docs/cli/move-tables). Make sure you have the [`pscale` CLI installed and authenticated](/docs/cli/planetscale-environment-setup) before you start. You must be an [Organization Administrator](/docs/security/access-control#organization-administrator) or a [Database Administrator](/docs/security/access-control#database-administrator) of the database to create and change workflows. ## Prepare to shard There is a small amount of upfront work that needs to happen before you move your table(s). ### 1. Decide which table(s) you want to shard First and foremost, decide which table(s) you want to shard. Some common signals that a table may benefit from sharding include: * The table has become very large (>100 GB) and query performance has degraded due to this * Schema changes to the table take hours * You expect the table to grow quickly and want to shard it before it becomes a problem ### 2. Identify tables you frequently `JOIN` Once you know the tables that you are going to move to a sharded keyspace, you also need to think about which other tables you frequently join with the tables you are going to shard. We recommend that tables you frequently join together all live in the same keyspace. As an example, if you have an `exercise_logs` table that has become extremely large and continues to grow, you may decide to move this to a sharded keyspace. Perhaps this table is frequently joined with the `users` table. In this scenario, we recommend moving both the `exercise_logs` and `users` tables to the new sharded keyspace and sharding both tables. The goal here is to avoid cross-keyspace or cross-shard queries. For more information about this, see the [Avoiding cross-shard queries](/docs/vitess/sharding/avoiding-cross-shard-queries) documentation. ### 3. Create a sharded keyspace > [!WARNING] > > If you are using [Vitess global routing](https://vitess.io/docs/reference/features/global-routing/) (for example, if you are using `@primary`), you must take extra care when adding a second keyspace to a database. Before creating one, you must ensure that all tables from your *initial keyspace* are added to the `VSchema` of that *initial keyspace*. Eg: > > ``` > { > "tables": { > "users": { } > ... > } > } > ``` > > Otherwise, queries will fail. [Learn more about VSchema.](/docs/vitess/sharding/vschema) To set up the sharded keyspace: **Step 1** Go to the "**Clusters**" tab in the left nav in the PlanetScale dashboard. **Step 2** Click "**New keyspace**". **Step 3** Enter the keyspace name (for example, `metal-sharded`). **Step 4** Select the **shard count** and choose the **cluster size** for this keyspace. Keep in mind, creating a sharded keyspace will use the selected size for *each* shard. For example, if you are creating 4 shards and choose the `PS-80` cluster size, we will create 4 `PS-80`s, each with 1 primary and 2 replicas. **Step 5** Select the number of *additional* replicas, if any, that you'd like to add to each cluster. Each cluster comes with 2 replicas by default, so any number you choose will be in addition to those 2. **Step 6** Review the new monthly cost for this keyspace below. This is in addition to your existing unsharded keyspace, as well as any other keyspaces you add. **Step 7** Once satisfied, click "**Create keyspace**". You can also create the keyspace with [`pscale keyspace create`](/docs/cli/keyspace): ```bash theme={null} pscale keyspace create main metal-sharded --shards 4 --cluster-size PS_80 --wait ``` ### 4. Choose your Vindexes > [!WARNING] > > If you are using [Vitess global routing](https://vitess.io/docs/reference/features/global-routing/) (for example, if you are using `@primary`), you may get ambiguous table errors once you add Vindexes to your new, second keyspace, if those tables also exist in the first keyspace. > > To prevent this error, you can add `require_explicit_routing` to your new, second keyspace's VSchema: > > ``` > { > "require_explicit_routing": true, > ... > } > ``` > > This will instruct Vitess's global routing to exclude your second keyspace from routing until explicitly targeted. Make sure to remove this field *before* completing the workflow. When configuring a sharded keyspace, you must think about *how* to distribute the data across shards. This is done by selecting a [Vindex](/docs/vitess/sharding/vindexes) (Vitess index) for each table. A Vindex provides a way to map incoming rows of data to the appropriate shard in your keyspace. Similar to how every MySQL table must have a primary key, every sharded table must additionally have a **primary Vindex**. The primary Vindex is the Vindex that determines which shard each row of data will reside on. For more information about choosing a Vindex, see the [Vindexes documentation](/docs/vitess/sharding/vindexes). You can also see an example in the [Avoiding cross-shard queries](/docs/vitess/sharding/avoiding-cross-shard-queries) documentation. To specify the vindex for the tables you want to shard: **Step 1** Create a new branch. **Step 2** Once on your new branch, switch to your sharded keyspace. You can do this in the PlanetScale console if preferred by clicking "Console" in the left nav. Continuing the previous example, we'll switch to the new `metal-sharded` keyspace: ``USE \`metal-sharded\`;`` **Step 3** Alter the VSchema (Vitess schema) of the tables you have chosen to shard to add your chosen Vindex. VSchema is provided to Vitess in JSON. For example, let's say we are sharding a table called `exercise_logs`, and we determined `user_id` to be the best option. We are also using the predefined [`xxhash` Vindex function](https://vitess.io/docs/reference/features/vindexes/#predefined-vindexes), which is a common choice. ```sql theme={null} ALTER VSCHEMA ON exercise_logs ADD VINDEX xxhash(user_id) USING xxhash; ALTER VSCHEMA ON users ADD VINDEX xxhash(id) USING xxhash; ``` You do not need to create sequence tables yourself. MoveTables creates them in the global keyspace when you create the workflow in the next section. See the [pre-sharding checklist](/docs/vitess/sharding/pre-sharding-checklist) if you would rather set them up by hand. ### 5. Deploy the changes to production Once you're finished with these pre-sharding steps, you can go ahead and deploy the changes to production. **Step 1** Click "Branches". **Step 2** Select the branch that has your keyspace updates. **Step 3** Select the sharded keyspace in the dropdown. You should see a diff that shows edits to your VSchema. **Step 4** Click "Create deploy request". **Step 5** If everything looks good, deploy your changes. If you go back to your Clusters tab, click your sharded keyspace, and click the VSchema tab, you'll see those changes reflected there. ## Move the tables with MoveTables Now that the prep work is done, it's time to move the tables you chose into your sharded keyspace. The examples below use a database named `mydb`, the `main` production branch, the unsharded `metal` keyspace, and the new `metal-sharded` keyspace. ### Step 1: Create the workflow ```bash theme={null} pscale branch vtctl move-tables create mydb main \ --workflow shard_users \ --source-keyspace metal \ --target-keyspace metal-sharded \ --tables users,exercise_logs \ --sharded-auto-increment-handling REPLACE \ --global-keyspace metal \ --defer-secondary-keys \ --format json ``` This creates a MoveTables workflow named `shard_users` and starts it right away. You will use the workflow name and the target keyspace in every later command. * `--sharded-auto-increment-handling REPLACE` removes `AUTO_INCREMENT` from the target tables and replaces it with Vitess [sequence tables](/docs/vitess/sharding/sequence-tables). Because you passed `--global-keyspace metal`, MoveTables creates those sequence tables in the `metal` keyspace. * `--defer-secondary-keys` creates the target tables' secondary indexes after the copy finishes instead of during it, which makes the copy much faster. By default, the workflow stops if a schema change runs on a moved table in the source keyspace while the workflow is running, so you can review it before continuing. See [`move-tables`](/docs/cli/move-tables) for `--on-ddl` and the other options. ### Step 2: Watch the copy As soon as the workflow starts, Vitess copies rows of the tables you selected from the source keyspace to the target keyspace. It uses a combination of `SELECT * FROM table` and binlog-based replication. Check progress with `status`: ```bash theme={null} pscale branch vtctl move-tables status mydb main \ --workflow shard_users \ --target-keyspace metal-sharded \ --format json ``` While the copy is running, `table_copy_state` lists each table's progress. Once every table is copied, the streams move to `Running`: Vitess keeps replicating every new write on the source tables to the target keyspace. At this point: * Your source keyspace is still serving all primary and replica traffic for the tables you're moving. * All existing data has been copied to the target keyspace. * VReplication keeps the target tables up to date with new writes. You can also follow the workflow from the dashboard. Click "**Workflows**" in the left nav, select your branch, and open the workflow to see its streams, per-table copy progress, replication lag, and traffic routing. The dashboard view is read-only: you drive every step from the CLI. Each command's JSON output includes a `next_steps` field with the command to run next. ### Step 3: Verify data consistency Once the streams are `Running`, verify that the source and target keyspaces hold the same data with a [VDiff](https://vitess.io/docs/reference/vreplication/vdiff/): ```bash theme={null} pscale branch vtctl vdiff create mydb main \ --workflow shard_users \ --target-keyspace metal-sharded \ --format json ``` The output includes the VDiff's `uuid`. Pass it to `vdiff show` and repeat until the VDiff completes: ```bash theme={null} pscale branch vtctl vdiff show mydb main \ --workflow shard_users \ --target-keyspace metal-sharded \ --uuid \ --format json ``` If the VDiff reports mismatches, do not switch traffic until you have resolved them. VDiff is optional, but we recommend running it before you switch traffic on a production branch. ### Step 4: Switch traffic to the target keyspace You are now able to switch traffic so that the moved tables are served from the target keyspace instead of the source keyspace. You can switch replica traffic first to test reads, then switch primary traffic. Switch replica traffic: ```bash theme={null} pscale branch vtctl move-tables switch-traffic mydb main \ --workflow shard_users \ --target-keyspace metal-sharded \ --tablet-types REPLICA,RDONLY \ --format json ``` Once reads from replicas look good in your application, switch primary traffic: ```bash theme={null} pscale branch vtctl move-tables switch-traffic mydb main \ --workflow shard_users \ --target-keyspace metal-sharded \ --tablet-types PRIMARY \ --initialize-target-sequences \ --format json ``` `--initialize-target-sequences` sets each sequence table's next value above the highest ID already in the table, so new rows don't collide with copied rows. Pass `--dry-run` to either command to see what would happen without switching. After primary traffic switches, Vitess starts a reverse workflow that replicates writes from the target keyspace back to the source keyspace. Both keyspaces keep the same data until you complete the workflow. ### Step 5: Check traffic in your application You should now go check out your production application that uses this database to make sure everything is running as expected. You can also check the [Insights](/docs/vitess/monitoring/query-insights) tab in the dashboard for errors or slow queries on the moved tables. If something looks wrong, switch traffic back to the source keyspace with `reverse-traffic`: ```bash theme={null} pscale branch vtctl move-tables reverse-traffic mydb main \ --workflow shard_users \ --target-keyspace metal-sharded \ --format json ``` You can reverse and switch traffic again as many times as you need until you complete the workflow. ### Step 6: Update your application code to serve from `@primary` When you switched traffic, Vitess applied [schema routing rules](https://vitess.io/docs/reference/features/schema-routing-rules/). Routing rules are responsible for routing traffic to the correct keyspace and/or shard. The configuration code in your application likely says something like `database_name = your_database_name`, where `your_database_name` is your original unsharded keyspace. This was fine when you only had one keyspace, but now that you have multiple keyspaces, your application won't know that the other ones exist with this current configuration. The routing rules applied during this workflow point incoming queries from your unsharded keyspace to the sharded keyspace, where necessary. However, when you complete the workflow in the next step with `--keep-routing-rules=false`, **these routing rules are removed**. That means if your application is still configured to explicitly send traffic to your original unsharded keyspace, `database_name = your_database_name`, Vitess will not know how to correctly route the queries for the tables that moved to the sharded keyspace. The fix for this is simple: update your application to route traffic to your primary instance. You can do that by setting database name to `@primary`. For example, in Rails, it would look like this: ```yaml theme={null} # database.yml production: <<: *default username: <%= Rails.application.credentials.planetscale&.fetch(:username) %> password: <%= Rails.application.credentials.planetscale&.fetch(:password) %> database: "@primary" host: <%= Rails.application.credentials.planetscale&.fetch(:host) %> ssl_mode: verify_identity ``` You can safely update and deploy this application code before completing the workflow. When you only have one keyspace, `@primary` will automatically route queries to that keyspace. Likewise, if you have queries that you specifically send to replicas or read-only regions, you can use `@replica` for those to have them automatically routed to the correct keyspace/shard. For more framework-specific examples, see [Targeting the correct keyspace documentation](/docs/vitess/sharding/targeting-correct-keyspace). Once you deploy this code change, double check that everything in your production application is working correctly. ### Step 7: Complete the workflow > [!WARNING] > > If you added `require_explicit_routing` to your target keyspace's VSchema in step 4: > > ``` > { > "require_explicit_routing": true, > ... > } > ``` > > You'll need to remove it before completing the workflow. Before you proceed with this step, **it is extremely important that you complete step 6**. This requires changes to your application code. Up until now, you can cancel the workflow or reverse traffic. Completing the workflow is not reversible: it stops replication between the keyspaces and, with the flags below, drops the moved tables from the source keyspace and removes the routing rules. Preview what `complete` will do with `--dry-run` first: ```bash theme={null} pscale branch vtctl move-tables complete mydb main \ --workflow shard_users \ --target-keyspace metal-sharded \ --keep-data=false \ --keep-routing-rules=false \ --dry-run \ --format json ``` Once you have reviewed the output, run the same command without `--dry-run`. `--keep-data` and `--keep-routing-rules` are required, so you always choose what happens to the source tables and routing rules: * `--keep-data=false` drops the moved tables from the source keyspace. Add `--rename-tables` to rename them instead of dropping them. `--keep-data=true` leaves them in place. * `--keep-routing-rules=false` removes the routing rules. Pass `--keep-routing-rules=true` to keep them if some queries still name the source keyspace. Write boolean flags as `--keep-data=false`, with an `=`. A space-separated value such as `--keep-data false` is read as `--keep-data=true`. ### Step 8: Check that your production application is working as expected Finally, check your production application to make sure everything is working as expected. You can check your [Insights](/docs/vitess/monitoring/query-insights) tab to see if queries are being properly routed to your new keyspace. Insights will also show you any errors, query performance issues, and more. That's it! The tables you selected at the beginning are now being served by the sharded keyspace. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sharding workflow state reference Source: https://planetscale.com/docs/vitess/sharding/sharding-workflow-state-reference The states of workflows created with the deprecated `pscale workflow` command and the earlier dashboard workflow pages. > [!WARNING] > > This page describes the states of workflows created with the deprecated [`pscale workflow`](/docs/cli/workflow) command, the earlier dashboard workflow pages, or the `/workflows` API endpoints. > > Workflows created with [`pscale branch vtctl move-tables`](/docs/cli/move-tables) report their state directly from Vitess. Run `move-tables status` to see each stream's state (`Copying`, `Running`, `Lagging`, `Stopped`, or `Error`) and the workflow's `traffic_state`. See the [Sharding quickstart](/docs/vitess/sharding/sharding-quickstart) for the full lifecycle. ## `copying` During this state, Vitess is copying rows from your source keyspace to your target keyspace using a combination of `SELECT * FROM table` and binlog-based replication. ## `running` Once copying is complete, we switch to `running` state. Replication lag must be low enough for VReplication to advance into the `running` state. During this state, we are running pure binlog-based replication. At this point, your original source keyspace tables are serving all primary and replica traffic. VReplication is replicating all writes to the tables in the target keyspace. Routing rules are routing all queries/writes from the target keyspace to the source keyspace. There are some additional substates you may enter during the `running` state: * `running` with not `verified data`: You can verify replicated data is accurate before switching any traffic * `running` with `verified data`: You have run the verification process in the last 24 hours * `running` with `verification stale`: You have run the verification process over 24 hours ago ## `switched_replicas` The `switched_replicas` state is an optional intermediate step between `running` and `switched_primaries`. During the `switched_replicas` state, the new target keyspace is serving replica traffic, and the source keyspace continues serving all primary traffic. Because writes are still doing to the source keyspace, data is still being replicated to the target keyspace. ## `switched_primaries` During the `switched_primaries` state, the target keyspace is serving replica *and* primary traffic. Vitess has now created a *reverse* workflow that replicates from the target keyspace back to the source keyspace. At this point: * The original, "forward" workflow is paused (stopped) * Vitess has initialized the sequence tables * The routing rules are in place, as they will be critical for `reversed_cutover` state. If you click "reverse traffic", here's what happens: * The workflow goes back to a `running` state * Depending on which way you're reversing, "forward" workflow is started again or the "reverse" workflow is stopped ## `cutover` * This is tantamount to `MoveTables Complete --keep-data=false --keep-routing-rules=false` * User **cannot** cancel the workflow at this point. This is because the underlying workflow is actually complete, and the source tables have been deleted * The underlying Vitess workflow is finished. * We reset the routing rules to what they were before the workflow started * User **must have** updated their application code to no longer target a keyspace in order for their app to still work at this step. * This is because the routing rules have been cleaned up, and can no longer route queries from `source_keyspace.table` to `target_keyspace.table` and we must rely on Vitess global routing. * If the user's app is failing at this point, have no fear! There is a `reverse cutover` button `reversed_cutover` * User has pressed "reverse cutover" * We put back the routing rules we saved during `switched_primaries` * These rules route queries to the source keyspace table, to the target keyspace table so that queries work again * User now has more time to update/fix their client application code `complete` * Does nothing except say, "I'm all done and I don't need to reverse cutover anymore". * Allows you to do another workflow. since we limit to 1 active workflow per branch at a time `cancelled` * User clicked "cancel" * We issued a `MoveTables Cancel --keep-data=false --keep-routing-rules=false` * target tables will be deleted * routing rules will be removed * Vitess workflow deleted `error` * Some sort of error happened. User can click "retry" to try again. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Targeting the correct keyspace Source: https://planetscale.com/docs/vitess/sharding/targeting-correct-keyspace Once you have more than one keyspace, with tables distributed across both keyspaces, your application may not know how to properly route queries to the correct keyspace. If you originally set up your application configuration code with something like `DATABASE_NAME=your_database_name`, where `your_database_name` is the name of your original unsharded keyspace, you will need to update your configuration code so that all queries don't go straight to that keyspace. The preferred way to do this is to just leave off the database name completely in your application configuration code. PlanetScale will be able to route traffic correctly just using the connection username and password. While this is the preferred way, it's sometimes not possible. For example, many frameworks and ORMs require that you include a database name. In those cases, you should use `@primary`. This will send any incoming queries first to our [Global Edge Network](https://planetscale.com/blog/introducing-global-replica-credentials#building-planetscale-global-network), which will see that you're targeting a primary. Edge will then send the request to the VTGate(s)/load balancer. We typically will use [Vitess's Global Routing](https://vitess.io/docs/reference/features/global-routing/) to direct the query to the correct keyspace and, optionally, correct shard. > [!NOTE] > > If you explicitly wish to target a replica for some or all reads, using `@replica` will have the same effect as `@primary` in that it will automatically route the request to the correct keyspace. [Global Replica Credentials](/docs/vitess/scaling/replicas#1-create-a-global-replica-credential-recommended) are not currently supported in this context. You can still target replicas instead of your primary with `@replica`, but it will not automatically route the query to the *closest* replica. ## Framework examples Using `@primary` is simple, but there are slight variations for each framework. The following code snippets show how to target `@primary` in some of the popular languages/frameworks. If you don't see your framework on here and are unsure of how to proceed, please [reach out to support](https://planetscale.com/contact?initial=support). ### MySQL CLI ```sql theme={null} mysql -h aws.connect.psdb.cloud -D @primary -u your_username -p pscale_pw_xxxxxxxxxxxxxxx --ssl-mode=VERIFY_IDENTITY --ssl-ca=/etc/ssl/cert.pem ``` ### Rails ```ruby theme={null} # database.yml production: <<: *default username: <%= Rails.application.credentials.planetscale&.fetch(:username) %> password: <%= Rails.application.credentials.planetscale&.fetch(:password) %> database: "@primary" host: <%= Rails.application.credentials.planetscale&.fetch(:host) %> ssl_mode: verify_identity ``` ### Django ```python theme={null} # Connect to the database connection = MySQLdb.connect( host=os.getenv("DATABASE_HOST"), user=os.getenv("DATABASE_USERNAME"), passwd=os.getenv("DATABASE_PASSWORD"), db="@primary", autocommit=True, ssl_mode="VERIFY_IDENTITY", ssl={"ca": os.getenv('SSL_CA')} ) ``` ### Laravel ```php expandable theme={null} 'mysql' => [ 'driver' => 'mysql', 'url' => env('DB_URL'), 'host' => env('DB_HOST', '127.0.0.1'), 'port' => env('DB_PORT', '3306'), 'database' => '@primary', 'username' => env('DB_USERNAME', 'root'), 'password' => env('DB_PASSWORD', ''), 'unix_socket' => env('DB_SOCKET', ''), 'charset' => env('DB_CHARSET', 'utf8mb4'), 'collation' => env('DB_COLLATION', 'utf8mb4_unicode_ci'), 'prefix' => '', 'prefix_indexes' => true, 'strict' => true, 'engine' => null, 'options' => extension_loaded('pdo_mysql') ? array_filter([ PDO::MYSQL_ATTR_SSL_CA => env('MYSQL_ATTR_SSL_CA'), ]) : [], ], ``` ### PHP MySQLi ```php expandable theme={null} # .env DATABASE_HOST=aws.connect.psdb.cloud DATABASE=@primary DATABASE_USERNAME=username DATABASE_PASSWORD=password # index.php // Connect to PlanetScale using credentials stored in environment variables $mysqli = mysqli_init(); $mysqli->ssl_set(NULL, NULL, "/etc/ssl/cert.pem", NULL, NULL); $mysqli->real_connect( $_ENV["DATABASE_HOST"], $_ENV["DATABASE_USERNAME"], $_ENV["DATABASE_PASSWORD"], "@primary", ); ``` ### PHP PDO ```php theme={null} # .env DATABASE_HOST=aws.connect.psdb.cloud DATABASE=@primary DATABASE_USERNAME=username DATABASE_PASSWORD=password # index.php // Use env variables to connect to the database $dsn = "mysql:host={$_ENV["DATABASE_HOST"]};dbname={$_ENV["DATABASE"]}"; $options = array( PDO::MYSQL_ATTR_SSL_CA => "/etc/ssl/cert.pem", ); $pdo = new PDO($dsn, $_ENV["DATABASE_USERNAME"], $_ENV["DATABASE_PASSWORD"], $options); ``` ### Elixir ```elixir expandable theme={null} defmodule Connect do def main do hostname = "aws.connect.psdb.cloud" {:ok, pid} = MyXQL.start_link(username: System.get_env("DATABASE_USERNAME"), database: "@primary", hostname: System.get_env("DATABASE_HOST"), password: System.get_env("DATABASE_PASSWORD"), ssl: true, ssl_opts: [ verify: :verify_peer, cacertfile: CAStore.file_path(), server_name_indication: String.to_charlist(hostname), customize_hostname_check: [ match_fun: :public_key.pkix_verify_hostname_match_fun(:https) ] ] ) {:ok, res} = MyXQL.query(pid, "select * from users limit 1") IO.input(res) IO.puts "Successfully connected to PlanetScale!" end end Connect.main ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vindexes Source: https://planetscale.com/docs/vitess/sharding/vindexes When configuring a horizontally [sharded](/docs/vitess/sharding) keyspace, one of the most important decisions to make is _how_ to distribute the data across shards. > [!NOTE] > > You can create sharded keyspaces on any plan by adding a new sharded keyspace using the [Clusters page](/docs/vitess/cluster-configuration) and moving tables into it with [MoveTables](/docs/vitess/sharding/sharding-quickstart). > > If you would like additional support from our expert team, our [Enterprise plan](/docs/planetscale-plans#enterprise-plan) may be a good fit. [Get in touch](https://planetscale.com/contact) for a quick assessment. To do this, we must select a **Vindex** (Vitess index) for each table. A **[Vindex](https://vitess.io/docs/reference/features/vindexes/)** provides a way to map incoming rows of data to the appropriate shard in your keyspace. Similar to how every MySQL table must have a primary key, every sharded table must additionally have a **primary Vindex**. The primary Vindex is the Vindex that determines which shard each row of data will reside on. Additional **secondary Vindexes** can be created to help speed up queries in a sharded environment. This is analogous to using a secondary index in MySQL. ## How do Vindexes work? Vindexes come in two main types: **functional** and **lookup**. ### Functional Vindex A functional Vindex is a Vindex that takes one or more column value(s) from an incoming row, executes a function, and produces a 64 bit number as output (keyspace ID). The input column will typically be something like your `user_id` or `tenant_id` column in your sharded table. Each time a row is inserted into the table, Vitess will extract this column value and run it through the Vindex function. This could be the identity function (a function that returns its input), but there are also a number of [built-in functions](https://vitess.io/docs/reference/features/vindexes/#predefined-vindexes) to choose from. `xxhash` is a commonly used one, as it does a good job of keeping data spread out evenly amongst shards. The function will return a keyspace ID, which tells Vitess which shard the row should be saved to. ### Lookup Vindex A lookup Vindex is a Vindex that uses a lookup table to store the mapping between the column value and the keyspace ID. In this setup, you would create a dedicated lookup table with at least two columns: `from` and `to`. The `from` column should contain the possible column values that we need to map to shards, and the `to` column contains the corresponding `keyspace ID` for that shard. Each time a new row needs to be inserted, Vitess looks up the mapping in the table and uses the resulting `keyspace ID` to send it to the appropriate shard. Lookup indexes tend to be slower than functional ones, therefore functional is preferred when possible. ## Keyspace IDs In Vitess terminology, "keyspace ID" does not mean "The ID of a keyspace." Rather, it is an identifier used to determine which shard a row should be stored on *within* a keyspace. When a single keyspace contains multiple shards (horizontal sharding), each shard must follow the [sharding naming conventions](https://vitess.io/docs/concepts/shard/#shard-naming). Each will be named according to the range of keyspace IDs it is responsible for, formatted like `BEGIN_ID-END_ID`. The first shard will have an empty `BEGIN_ID` and the last one an empty `END_ID`. For example, if we wanted to set up a keyspace with four shards, and each one is responsible for exactly one quarter of the keyspace IDs, we should name them: `-40`, `40-80`, `80-c0`, `c0-`. You can find more information about shard naming on the [Vitess website](https://vitess.io/docs/concepts/shard/). ## Primary Vindexes Every horizontally sharded table must have a primary Vindex. Let's consider the following `exercise_log` table and determine which column we should use for our primary Vindex. ```sql theme={null} +-------------+-----------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+-----------------+------+-----+---------+----------------+ | log_id | bigint unsigned | NO | PRI | | auto_increment | | user_id | bigint unsigned | NO | | | | | exercise_id | bigint unsigned | NO | | | | | gym_id | bigint unsigned | NO | | | | | reps | smallint | YES | | | | | created_at | datetime | YES | | | | | edited_at | datetime | YES | | | | | deleted_at | datetime | YES | | | | | notes | varchar(1024) | YES | | | | +-------------+-----------------+------+-----+---------+----------------+ ``` Each `exercise_log` row has an auto incrementing `log_id`. Each row will also have other IDs acting as foreign keys to other tables: > [!NOTE] > > You can't use an `auto_increment` primary key in a sharded keyspace. > Instead you'll need to use a [sequence table](/docs/vitess/sharding/sequence-tables). * `user_id` is a foreign key to a `user` table * `exercise_id` is a foreign key to an `exercise` table * `gym_id` is a foreign key to a `gym` table In some cases, using the `primary key` as the primary Vindex can be helpful, so let's start by considering `log_id`. For each row inserted into this table, we could take the `log_id`, run it through an `xxhash` functional Vindex, and then the result would be used by Vitess to assign the row to a shard. This would cause all of the rows to get evenly distributed across all shards, which is good for insert performance. However, when considering what to choose for the primary Vindex, it is important to consider the types of queries your database is expected to fulfill. We may have queries that need to fetch sequences of logs for a given user. Something along the lines of: ```sql theme={null} SELECT * FROM exercise_log WHERE user_id = $SOME_USER_ID AND created_at BETWEEN $START_CREATED_AT AND $END_CREATED_AT; ``` If we use `log_id` as the primary Vindex, the rows will be seemingly "randomly" distributed across shards. Each one of these queries would have to search all shards for the log messages of a single user. This would hurt performance severely if this query is executed frequently. Instead, we could choose `user_id` for our primary Vindex. A given `user_id` will always produce the same output hash, meaning all of the logs for user X will end up on the same shard. This means that when a user wants to view their exercise log history, a single shard will be able to fulfill the query, leading to improved performance. Queries are more performant when performed on a single shard. In other words, cross-shard queries are *less* performant than single-shard queries. ## Secondary Vindexes Secondary indexes are optional, but may be created to help speed up lookups for queries in your workload. Consider the case where we want to look at sequences of the log that happened for all users at a specific gym. Such a query would look something like this: ```sql theme={null} SELECT reps, notes, created_at FROM exercise_log WHERE gym_id = $SOME_GYM_ID ORDER BY created_at DESC LIMIT 1000; ``` In this case, we are not doing any filtering by the primary Vindex, and instead using `gym_id`. Without a secondary index, Vitess has no way of knowing which shard(s) to look at to get the rows. To execute this, it would have to send a query to all shards, aggregate the results, and then return to the client. We can set up a secondary lookup Vindex to help with this. To keep the example simple, we will make an over-simplification and assume that each user goes to exactly one gym and never changes gyms. We will need to create a lookup table in MySQL to store the mapping between gyms and logs. The table will have two columns, `from` and `to`. ```sql theme={null} CREATE TABLE gym_id_exercise_log_lookup_vindex ( from BIGINT UNSIGNED, to BINARY(64), primary key(from) ); ``` The `from` column can be populated with the `gym_id`s from the `gym` table, and the `to` colum will store the corresponding keyspace ID. This could then be used with a Vindex function like `consistent_lookup_unique` to fulfill queries and send requests to the appropriate shards when executing the query. ## Built-in functional Vindexes Vitess supports a [many predefined functional Vindexes](https://vitess.io/docs/reference/features/vindexes/#predefined-vindexes). Here we'll cover a subset, and in what scenarios you might consider using them. ### `hash` and `xxhash` These are popular choices, particularly for use as a primary Vindex. They are especially useful when creating a Vindex from an incrementing ID. This is because a sequence of numbers that are close to each other will be assigned wildy different outputs, spreading them out nicely across shards. For example, user IDs `1`, `2`, `3`, `4`... might hash to a sequence like `3e8a`, `9b11`, `de1e`, `781b`... `hash` should generally be avoided in favor of `xxhash` `hash` uses the DES hashing algorithm whereas `xxhash` uses the xxhash64 algorithm. DES was designed to be cryptographically secure and xxhash64 is not, leading to xxhash64 having better performance. If you do not need cryptographic security for your keyspace ID, use `xxhash`. ### `numeric` This acts as the identify function. Since keyspace IDs need to be 64 bit numeric values, this only works for a column that are 64 bits or smaller. A numeric Vindex function has the advantace of being the most performant. This Vindex can be useful when you already have a good numeric key to hash on inherent in the schema of your database. This might be a good choice if you have a database with multiple tenants and you want to have one tenant per-shard. If you already have a `tenant_id`, you can use this value directly as your key to shard on. ### `consistent_lookup` and `consistent_lookup_unique` These are used for lookup Vindexes, and should be used in favor of the older `lookup` and `lookup_unique` Vindexes. These provide better performance, as they do not require two-phase commit to look up a value. If you need to do lookup-table based Vindexing, we recommend one of these two options. ### `numeric_static_map` This Vindex allows you to specify a static mapping between input column values and keyspace IDs via a JSON file. This is a good option if you have a small set of mappings that can be easily managed in JSON. If the set of mappings is large, you can opt for something like `consistent_lookup` instead. ## Next steps This page mostly focuses on Vindexes conceptually. If you want to see an example of how to update your Vschema to use a Vindex, check out the [sharding walkthrough](/docs/vitess/sharding/sharding-quickstart). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # VSchema Source: https://planetscale.com/docs/vitess/sharding/vschema PlanetScale databases are powered by Vitess. ## VSchema overview Each Vitess cluster can have one or more [keyspaces](https://vitess.io/docs/concepts/keyspace/). For unsharded databases, there is a 1:1 relationship between a keyspace and a database within MySQL. For sharded databases, a single keyspace can map to multiple MySQL databases under the hood. Each keyspace in your PlanetScale database has an associated [VSchema](https://vitess.io/docs/reference/features/vschema/). The VSchema contains information about how the keyspace is sharded, sequence tables, and other Vitess schema information. ## Viewing VSchema In order to view your VSchema, first go to the "Branches" tab in the PlanetScale app. ![PlanetScale app tab bar](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/enterprise/tabs.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=999eb75e681ce17244962c3cc222a095) Click on the branch you would like to view the VSchema for. Then, select the keyspace and expand out the "Configuration Files" drop-down. ![PlanetScale keyspace selection and configuration files drop down](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/enterprise/keyspace.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=a2884cf910a9a84f11ac23beac862596) From here, you can inspect your VSchema configuration JSON file. ![VSchema JSON view](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/enterprise/vschema.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=98286e12ad8804178e1204c3b4a757b4) ## Modifying VSchema You must have a sharded keyspace in order to make VSchema changes. If you have a database with at least one sharded keyspace, you can modify its VSchema either in the [Clusters](/docs/vitess/cluster-configuration) tab in the dashboard, from the [pscale CLI](/docs/cli/keyspace), or using `ALTER VSCHEMA ...` commands. ### Using the Clusters page We do not recommend modifying the VSchema directly on your production branch. In fact, it is not possible to do if you have [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled (as recommended). Instead, to modify the VSchema, you should first [create a new development branch](/docs/vitess/schema-changes/branching). Once you have your branch ready, follow these steps: **Step 1** To update on the Clusters page, select your new development branch from the dropdown at the top, and then select the keyspace below that has the VSchema you'd like to modify. **Step 2** Next, click the tab labeled "VSchema". **Step 3** Modify the VSchema configuration JSON file as needed. Refer to the [VSchema documentation](/docs/vitess/sharding/vschema) for more information about the available options. **Step 4** When finished, click "Save changes". We will validate your VSchema, and if it is valid, the changes will be saved. If there are errors, we will warn you here to change them before saving. **Step 5** Go back to your "Branches" tab and click on the development branch that you modified. You should see a note on the right that says "Updated VSchema configuration" which lets you know the VSchema(s) for this branch has been modified. **Step 6** From here, go through the normal [deploy request process](/docs/vitess/schema-changes/deploy-requests) to deploy this change to production. Once your change is deployed to production, you can come back to the Clusters page, switch to your production branch, and view the updates to your VSchema. You can also click the "Changes" tab to see information, such as the resize event, status, and start/end time for any previous changes to the VSchema. ### Using `ALTER VSchema` You can modify the VSchema with `ALTER VSCHEMA ...` statements over a regular MySQL connection. For a complete list of the supported statements, see the [`ALTER VSCHEMA` syntax reference](/docs/vitess/sharding/alter-vschema). PlanetScale recommends making all such modifications in a development branch. When ready, you can make a deploy request to get the changes into production. Consider the following database with two keyspaces. ![Sharded keyspace](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/enterprise/sharded-keyspace.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=37565511e5d1e6b0c9c7f7ea89d0697e) `sharded` is a sharded keyspace with two shards and `tweeter` is unsharded. Also note that safe migrations are enabled. In order to make a VSchema change for the production branch in this configuration, we first must create a new branch. We'll call it `add-tweets` ![New branch](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/enterprise/new-branch.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=37d7be81e67eb81ea6a34a3ae16a253e) On this branch you can make your VSchema and schema changes. In this case, we'll create a new table called `tweets` in the sharded keyspace and also update the VSchema. ![Create the tweets table](https://mintcdn.com/planetscale-2/TneybaJ6MA8SGyM3/images/assets/docs/enterprise/tweets-table.png?fit=max\&auto=format\&n=TneybaJ6MA8SGyM3\&q=85\&s=4b576310285299ee264c929474cf01bf) We will also create a sequence table in the unsharded keyspace, and update the VSchema accordingly: ![Create the sequence table](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/enterprise/sequence-table.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=004856cf8a48f41477528c2ac2d712e7) We have now made updates both to our Vitess VSchema and MySQL schema. To get these changes into production, navigate to the "Branches" page and select the `add-tweets` branch. Here, you will be presented with a diff of both the VSchema and schema changes: ![Schema diff](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/enterprise/schema-diff.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=655ba0a94ea0631f4849990e37e4aca9) Click "Create deploy request." The deploy request should indicate that it is going to apply both VSchema and Schema changes: ![Deploy request](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/enterprise/deploy-request.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=78abdfd7f4fb6ebd51c48708eff38d5c) Click "Deploy changes." Once complete, the Vschema and schema changes will be applied to your production branch. > [!NOTE] > > `ALTER VSCHEMA ...` commands also work on unsharded keyspaces, but they are rarely useful there. > An unsharded keyspace has only one shard, so Vitess can route every query to it without any VSchema metadata. > Most of what `ALTER VSCHEMA` manages — Vindexes, auto-increment columns, and sequence tables — only comes into play when a keyspace is sharded, or is about to be. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Strategies for maintaining referential integrity Source: https://planetscale.com/docs/vitess/strategies-for-maintaining-referential-integrity If you choose to not use foreign key constraints, cascading actions need to be addressed via code instead of letting the database engine handle them for you. This document outlines some recommended strategies for building a system that handles those actions for you. > [!NOTE] > > If you want to use foreign key constraints instead for referential integrity, enable [foreign key constraint](/docs/vitess/foreign-key-constraints) support in your database settings page. If you are unsure if you should use them, the [foreign key constraints documentation](/docs/vitess/foreign-key-constraints) covers some of the advantages and disadvantages. The following examples will use the concept of a recipe manager with the following schema. Samples will be provided in SQL, so they are relatively universal regardless of the language or framework used. The same scenario will be used for each suggestion below: deleting a recipe and its associated records. ## Executing actions inline The first method you could use to address stale records is to write the necessary code to handle cascading actions in line with the main execution. For example, if you are deleting a recipe using an API, you could extend the route handler to delete the associated ingredients and steps as well. ```sql theme={null} -- Main action DELETE FROM recipes WHERE id = 123 -- Secondary actions DELETE FROM ingredients WHERE recipe_id = 123 DELETE FROM steps WHERE recipe_id = 123 ``` The benefit this approach provides is that it’s relatively straightforward using the systems that are already in place. The downside is in situations where a large number of records need to be deleted or updated, you might make your users wait for longer than needed as the cascading actions are handled. ## Scheduled jobs The second approach is using a system that will scan the database at regular intervals for cascading actions that need to be performed. For example, you might set up Cron with a script that will execute queries against the database to find records that are soft-deleted, and run the necessary SQL to delete the ingredients and steps, then the recipe itself. The main action would flag the record in the `recipes` table as being soft-deleted: ```sql theme={null} -- Flag the record as requiring deletion. UPDATE recipes SET is_deleted = true WHERE id = 123 ``` A scheduled task could then execute the following statements to find the records requiring cleanup, and delete those records: ```sql theme={null} -- Find the recipes that have been deleted. SELECT id FROM recipes WHERE is_deleted = true -- Then for each recipe delete the necessary records. DELETE FROM ingredients WHERE recipe_id = 123 DELETE FROM steps WHERE recipe_id = 123 DELETE FROM recipes WHERE id = 123 ``` The benefit this approach provides is that the main execution point of your application (ex: an API) would return quicker as you are simply updating a single value in the database. This prevents the user from waiting for those records to be cleaned up. The downside is that this requires an additional system with its own associated code that would require maintenance. While `DELETE` operations are relatively straightforward, addressing `UPDATE` operations may be a challenge. There is also an inherent delay where stale records would exist in the database between script executions. You can see a full example of how to handle this in Rails in our [Ruby on Rails: 3 tips for deleting data at scale blog post](https://planetscale.com/blog/ruby-on-rails-3-tips-for-deleting-data-at-scale). ## Asynchronous cleanup using queues The final recommended strategy would be to use a message queue like AWS SQS or RabbitMQ. The application would send messages to the queue so that a handler can perform the cascading operations asynchronously. For example, you may set up an AWS SQS queue to receive messages from your API on what actions were performed on a particular recipe or what data was changed. When a message is dropped on the queue, a Lambda function is triggered to perform the required operations. In this scenario, your API may start by deleting the recipe record: ```sql theme={null} DELETE FROM recipes WHERE id = 123 ``` The API might then drop a message into the queue in the following format: ```json theme={null} { "id": 123, "action": "deleted" } ``` The Lambda function that receives the message from the API could then perform the cleanup operations: ```sql theme={null} DELETE FROM ingredients WHERE recipe_id = 123 DELETE FROM steps WHERE recipe_id = 123 DELETE FROM recipes WHERE id = 123 ``` This approach provides the greatest number of benefits. Cascading operations are asynchronous, so the caller wouldn’t need to wait. It scales well since any number of serverless functions may be started to handle the load. Cleanup operations would be executed in near real-time, as long as it takes for a message to enter the queue and the serverless function to pick it up. It is also decoupled and easily fits the serverless model very well. The primary downside is that it increases the complexity of the overall application and requires the largest number of moving parts, as well as the knowledge to support these systems. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale terminology Source: https://planetscale.com/docs/vitess/terminology Here we provide definitions for technologies and concepts you will see throughout our documentation and product. Some of these are common terms in the databases space, while others are Vitess or PlanetScale specific. ## Database technologies ### MySQL MySQL is the world's most popular relational database. Many of the worlds largest web applications are powered by MySQL behind the scenes including Slack, X, GitHub, JD.com, and many others. MySQL is open-source, has existed for 30+ years, and is currently maintained by Oracle. ### Vitess [Vitess](https://vitess.io) is an open-source project created by engineers at YouTube in the early 2010's. It was created to solve their MySQL scalability challenges. Vitess works in tandem with MySQL and provides proxying, orchestration, monitoring, and sharding capabilities. A Vitess-powered MySQL database can scale to petabytes of data and millions of queries per second. ### PlanetScale PlanetScale is a complete database platform that emphasizes reliability, scalability, and developer productivity. PlanetScale offers three database engines: [Vitess](/docs/vitess) (MySQL-compatible), [Neki](/docs/neki) (sharded Postgres), and [Postgres](/docs/postgres). PlanetScale Vitess databases are powered by Vitess and MySQL. PlanetScale makes it easy to spin up, resize, manage, and work with databases both for small organizations and large enterprises. Additionally, PlanetScale employs the majority of the Vitess core maintainers. ## PlanetScale concepts ### Database In the PlanetScale app, users can create one or more **Databases**. Each database belongs to an **Organization**. Each database uses [Vitess](/docs/vitess), [Neki](/docs/neki), or [Postgres](/docs/postgres). Vitess databases use MySQL under the hood. If you've used vanilla MySQL in the past, you may be used to one MySQL server managing multiple logical databases. In a PlanetScale Vitess database, you cannot use `CREATE DATABASE` to create multiple logical databases. Instead, you can achieve a similar outcome by creating multiple **keyspaces**. [PlanetScale Postgres](/docs/postgres) databases do not have this limitation. You can use `CREATE DATABASE` to create multiple PostgreSQL databases within a single PlanetScale Postgres database. ### Keyspace A **Keyspace** represents a single, logical database. When you create a new PlanetScale database, it contains a single, default keyspace of the same name as your database. More keyspaces can be added using the [Clusters](/docs/vitess/cluster-configuration) page. Each keyspace has its own primary and replicas. The "keyspace" terminology [comes from Vitess](https://vitess.io/docs/concepts/keyspace/). [Read more about keyspaces](/docs/vitess/sharding/keyspaces). ### Branch A typical flow on GitHub is to work on new feature in branches. When development is complete, the developer can create a pull request which then gets merged into `main`. PlanetScale allows you to work with your database in a similar fashion via **branches**. Every database has a default branch, often named `main`. You can create branches of your database, in which you can make schema changes without affecting your production tables and data. ### Deploy request When you are ready to bring the changes from a branch into your production database, you can create a **Deploy request** (or **DR** for short). This is akin to a pull request on GitHub. Deploy requests can be created, reviewed by peers, and deployed to the main database. Deploy requests give you the ability to change the schema of your database with 0 downtime. We also give you the ability to revert deploy requests for a short window of time after being deployed, in case you encounter an issue with your application. Deploy requests are currently limited to Vitess databases. ## Vitess concepts ### VTGate Every PlanetScale database comes with at least 3 **VTGates**. A VTGate is the layer of Vitess that acts as a proxy between your application servers and the MySQL instances. VTGates play an important role in allowing your applications to treat the database as a single server, even if your data is actually spread across many keyspaces and shards. VTGates handle query buffering, distributing your queries across shards, and other important functions for high-availability. ### Shard Vitess keyspaces can be unsharded or [sharded](/docs/vitess/sharding). A sharded keyspace is one where the data of tables is spread out across multiple primaries, rather than all going to one. The way data is sharded is specified via the [Vindex](/docs/vitess/sharding/vindexes). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale database error reference Source: https://planetscale.com/docs/vitess/troubleshooting/errors This documentation covers some commonly encountered errors and approaches for addressing them. > [!NOTE] > > If you are facing issues or have questions that were not answered in the documentation, the best course of action is to [open a support ticket](https://planetscale.com/contact). > > Additionally, you can find some broader limitations in the [PlanetScale system limits documentation](/docs/vitess/troubleshooting/planetscale-system-limits). ## MySQL and Vitess errors PlanetScale Vitess databases are powered by Vitess and MySQL. Many errors you encounter will contain a substring of the form `SQL Error [XXXX]` where `XXXX` is some integer number. Often, these are errors that are passed along back to the client from MySQL. There are hundreds of MySQL error codes documented on the [MySQL error codes page](https://dev.mysql.com/doc/mysql-errors/8.0/en/server-error-reference.html). For example, say you encounter this error: ``` Error synchronizing data with database Reason: SQL Error [1364] [HY000]: target: unigate.-.primary: vttablet: rpc error: code = Unknown desc = (errno 1364) (sqlstate HY000) (CallerID: unsecure_grpc_client): Sql: ... BindVars: {} ``` The second line says `SQL Error [1364]`, which corresponds to the [ER\_NO\_DEFAULT\_FOR\_FIELD](https://dev.mysql.com/doc/mysql-errors/8.0/en/server-error-reference.html#error_er_no_default_for_field) error in MySQL. Knowing this, you can make the appropriate changes to your query or schema to mitigate the error. You can also search the MySQL docs and other online sources using the error code, as there is a long history of MySQL questions and answers online. One specific code you may come across is `SQL Error [1105]`, which represents an [unknown error](https://dev.mysql.com/doc/mysql-errors/8.0/en/server-error-reference.html#error_er_unknown_error) in MySQL. On PlanetScale, it's likely that such an error is actually coming from Vitess, which also has an [error documentation page](https://vitess.io/docs/reference/errors/query-serving/). Note that a number of older Vitess errors used the `1105` code, but there are many other errors documented there as well. We recommend you reference this for Vitess-specific errors. Below, we document more common errors from our customers, and provide suggestions for how to address them. | **Error** | **Reason** | **How to address** | | - | - | - | | `ResourceExhausted desc = transaction pool connection limit exceeded` | Every PlanetScale database has limit on the number of concurrent transactions it can process. This message indicates you are hitting the limit. | This can be resolved in several ways. One option is to make changes to your application code that reduce the number of long-running transactions. If this is not feasible, you can also [size up your database](/docs/vitess/cluster-configuration#adjust-your-cluster-size). This can allow long-running transactions to execute quicker, and you also may get a higher concurrent transaction limit. | | `primary is not serving, there is a reparent operation in progress` | Your primary database server is unavailable. This often happens due to an OOM error (Out Of Memory). You also may see this on dev branches if you happen to query them during an upgrade. | In the short term, if this error message persists, we recommend you [reach out to support](https://planetscale.com/contact?initial=support). If this was caused by an OOM, you should see if there are changes you can make to your queries to reduce memory pressure on your database. This could mean consolidating queries or building indexes to reduce the number of pages needing to be brought into memory during query execution. If this is not possible, you will likely need to [upgrade](/docs/vitess/cluster-configuration#adjust-your-cluster-size) to a larger cluster size. | | `vttablet: (errno 2013) due to context deadline exceeded, elapsed time: ...` | Your query or transaction exceeded the default [20 second per-transaction timeout](/docs/vitess/troubleshooting/planetscale-system-limits#query-limits). | Consider options to reduce the time needed for the query/transaction in question to execute. Things that could help include: adding an index to speed up the query, breaking a large multi-query transaction up into multiple shorter ones. If you need some long-running transactions for analytical purposes, consider offloading those to a tool like [Airbyte](/docs/vitess/integrations/airbyte) or [Stitch](/docs/vitess/integrations/stitch). | | `SQL Error [1105] [HY000]: target: platform.-.primary: vttablet: rpc error: code = Aborted desc = transaction ... (exceeded timeout: 20s)` | Max transaction time exceeded. | See row above. | | `vttablet: rpc error: code = Aborted desc = Row count exceeded 100000` | You have hit the [100k limit](/docs/vitess/troubleshooting/planetscale-system-limits#query-limits) for number of rows returned, updated, or deleted in a single query. | If this is coming from a `SELECT` statement, consider narrowing the search or paginating results. If it is coming from an `UPDATE` or `DELETE`, consider performing updates or deletions in smaller batches. | | `vttablet: rpc error: code = ResourceExhausted desc = Out of sort memory, consider increasing server sort buffer size (errno 1038)` | MySQL has a [buffer specifically used for sorting](https://dev.mysql.com/doc/refman/8.4/en/server-system-variables.html#sysvar_sort_buffer_size). This error is passed up from MySQL and indicates the buffer has been exhausted. | Buffer sizes for MySQL servers on PlanetScale come pre-configured, and we do not allow you to modify the system `sort_buffer_size` parameter. You may be able to get around this error by reducing the size of the result sets being sorted, adding a covering index to avoid performing a filesort during query execution, or increasing the sort buffer size for the individual sessions the query is running in. | | `unavailable: vtgate connection error: no endpoints, after 1 attempts` | A connection could not be made to one of your VTGates. These show up as your "load balancers" in the PlanetScale UI. | This can be caused by running queries that return very large result sets, exhausting the available memory on the load balancer. Please [contact support](https://planetscale.com/contact?initial=support) to discuss options for mitigating this. | ## PlanetScale-specific errors You also may encounter error messages in the PlanetScale UI or on specific PlanetScale features such as [safe migrations](/docs/vitess/schema-changes/safe-migrations) or [workflows](/docs/vitess/scaling/workflows). Here, we document a selection of those errors you may run into. | **Error** | **Reason** | **How to address** | | - | - | - | | `This deploy request is not deployable` | Too many schema changes in one deploy request. If you are making a large number of schema changes in a single branch, you may encounter this error. | Divide up the schema changes into multiple incremental changes, and create separate branches and deploy requests for each. | | `Data Definition Language is not supported on branches with safe migrations enabled.` | You attempted to run DDL on a branch with [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled. | We recommend you keep safe migrations enabled. If you need to change schema in such a branch, you can create a new branch, modify the schema, and then use a deploy request to bring that change into the original branch. | | `not_found: branch is missing or sleeping: branch_id` | The branch you are targeting is either [sleeping](/docs/plans/database-sleeping#what-is-database-sleeping) or has been deleted. | We recommend double checking that you are targeting the correct branch. Additionally, ensure the credentials are not tied to a deleted branch. [Reach out to support](https://planetscale.com/contact?initial=support) if you need further assistance. | ## Other errors If you are encountering an error not listed here, we recommend you use the [MySQL](https://dev.mysql.com/doc/mysql-errors/8.0/en/server-error-reference.html) and [Vitess](https://vitess.io/docs/reference/errors/query-serving/) error documentation pages to narrow down your issue. If you are unable to identify it on your own, please [reach out to support](https://planetscale.com/contact?initial=support). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # MySQL compatibility Source: https://planetscale.com/docs/vitess/troubleshooting/mysql-compatibility PlanetScale Vitess is built on open-source Vitess, a database clustering system for horizontal scaling of MySQL. ## Overview PlanetScale databases run on MySQL 8. If you're [importing an existing database](/docs/vitess/imports/database-imports), PlanetScale supports MySQL database versions `5.7` through `8.0`. New PlanetScale databases are created on MySQL 8 with character set `utf8mb4_0900_ai_ci`. PlanetScale supports `utf8`, `utf8mb4`, and `utf8mb3`, character sets. We also support `latin1` and `ascii` character sets, but do not recommend them. ## MySQL compatibility limitations The following reference guide will cover some MySQL syntax, features, and more that PlanetScale either does not support or has limitations around. We are actively working on driving up compatibility, but it's an ongoing effort and will take some time to complete. See this [project board on GitHub](https://github.com/vitessio/vitess/projects/4) to learn what the Vitess team is currently focusing on. If you're importing a database, there are some additional requirements that you can find in our [Database imports documentation](/docs/vitess/imports/database-imports). ### Queries, functions, syntax, data types, and SQL modes > [!NOTE] > > \= *Limitations in support* > > \= *Not supported* | Statement | Support | Description | | - | - | - | | `ALTER TABLE...RENAME COLUMN` | | Renaming columns and tables may be destructive. See our [guide for column rename recommendations](/docs/vitess/schema-changes/handling-table-and-column-renames). | | `CREATE DATABASE` | | You cannot `CREATE` a logical database within a PlanetScale Vitess database. | | `CREATE USER` | | Vitess does not support MySQL's native user and privilege management. Create database credentials through the PlanetScale dashboard, API, CLI, or [password roles](/docs/vitess/security/password-roles). | | `DROP DATABASE` | | You cannot `DROP` a logical database within a PlanetScale Vitess database. | | `JSON_TABLE` | | The [`JSON_TABLE` function](https://dev.mysql.com/doc/refman/8.0/en/json-table-functions.html#function_json-table) is not yet supported. All other [JSON SQL functions](https://dev.mysql.com/doc/refman/8.0/en/json-function-reference.html) should work. | | `PROCEDURE` | | We do not support any form of [stored routines](https://dev.mysql.com/doc/refman/8.0/en/stored-routines.html). | | `FUNCTION` | | We do not support any form of [stored routines](https://dev.mysql.com/doc/refman/8.0/en/stored-routines.html). | | `TRIGGER` | | We do not support any form of [stored routines](https://dev.mysql.com/doc/refman/8.0/en/stored-routines.html). | | `EVENT` | | We do not support any form of [stored routines](https://dev.mysql.com/doc/refman/8.0/en/stored-routines.html). | | `LOAD DATA INFILE` | | Loading data via [`LOAD DATA INFILE` is not supported](https://github.com/vitessio/vitess/issues/2976). | | `KILL` | | We do not support killing queries or shards from the command line. | | `:=` | | The `:=` assignment operator is not yet supported. | | `SET GLOBAL time_zone` | | The global time zone is set to UTC and can not be modified. | | `SET GLOBAL sql_mode` | | The global SQL mode can not be changed permanently. Set each new session's mode instead with `SET sql_mode`. | | `PIPES_AS_CONCAT` | | Enabling this SQL mode can interfere with Vitess' evalengine parsing the SQL queries so enabling it may result in incorrect or unexpected results. Please use MySQL's standard dialect instead, e.g. `CONCAT()`. | | `ANSI_QUOTES` | | Enabling this SQL mode can interfere with Vitess' evalengine parsing the SQL queries so enabling it may result in incorrect or unexpected results. Please use MySQL's standard quotation instead. | | `WITH RECURSIVE` | | Experimental support for recursive common table expressions (CTEs) was introduced in Vitess 21 for `SELECT` queries. | ## Miscellaneous | Action | Support | Description | | - | - | - | | **Empty schemas** | | Databases with empty schemas are invalid. You cannot deploy a schema change to production if no tables exist. | | **Non-InnoDB Storage engine** | | We only support [InnoDB](https://dev.mysql.com/doc/refman/8.0/en/innodb-storage-engine.html) storage engine. | | **No applicable unique key** | | We require all tables have a [unique, non-null key](/docs/vitess/schema-changes/onlineddl-change-unique-keys) and that respective covered columns are shared between old and new schema. | | **Direct DDL** | | We do [not allow Direct DDL](/docs/vitess/schema-changes/how-online-schema-change-tools-work) on production branches when [safe migrations](/docs/vitess/schema-changes/safe-migrations) is enabled. This includes `TRUNCATE` statements. | | **Binary log access** | | PlanetScale does not currently support binlog replication to external databases. | | **Large JSON documents** | | MySQL supports JSON documents up to 1 GB in size. However, we do not recommend to store more than a few MB in a JSON document for performance reasons. | ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Out of memory (OOM) Source: https://planetscale.com/docs/vitess/troubleshooting/out-of-memory Understanding and resolving out of memory events on PlanetScale (Vitess) clusters. An out of memory (OOM) event occurs when a process on a tablet consumes more memory than is available on the cluster. When this happens, the affected process is automatically restarted to free up memory. Each tablet runs two main processes in separate containers: **`vttablet`** (the query-serving proxy) and **`mysqld`** (the MySQL server). Either can run out of memory independently. It is possible for `vttablet` to be OOM-killed under load while `mysqld` memory looks stable, so overall pod memory can appear healthy even during an OOM. ## Impact and risks When an OOM event occurs: * **Process restart required**: The affected container (`vttablet` or `mysqld`) is restarted to recover. Repeated OOMs show up as a container that keeps restarting (`CrashLoopBackOff`). * **Brief unavailability**: The affected tablet is unavailable while it restarts. If a shard's primary is repeatedly OOM-killed, writes to that shard can be disrupted until it recovers or is resized. * **In-flight queries fail**: Queries and uncommitted transactions running on the affected tablet at the time of the restart are terminated. ## Primary resolution: Upgrade cluster size The most direct solution to OOM events is to increase your cluster size to provide more memory. See [cluster sizing](/docs/vitess/scaling/cluster-sizing) for the available sizes and their memory, and [cluster configuration](/docs/vitess/scaling/cluster-configuration) for how to resize. Resizing is performed with minimal downtime. ## Common causes of high memory usage ### Heavy write or bulk operations Large writes, deletes, or backfills can drive `vttablet` memory up sharply. A single large `DELETE`, in particular, can hold a lot of state in memory while it runs, so breaking these into smaller batches keeps memory bounded. **Recommendations:** * Break large `DELETE`/`UPDATE`/backfill operations into smaller batches. * Avoid running several large bulk operations concurrently. ### Memory-intensive queries Certain query patterns can consume large amounts of memory, for example large `ORDER BY` sorts, `GROUP BY`/`DISTINCT` over large result sets, hash joins on large tables, and queries that return very large result sets. **Recommendations:** * Add indexes so sorts and lookups don't have to be done in memory. * Use `LIMIT` and pagination instead of returning large result sets at once. ### Too many connections Each connection consumes memory. A large number of open connections, whether intentional or due to connection leaks, increases memory pressure, especially on smaller cluster sizes. **Recommendations:** * Ensure your application closes connections and uses a connection pool. * Reduce the number of concurrent connections where possible. ## Monitoring and prevention * **Watch for the OOM banner**: When a branch experiences OOM kills, a banner on the branch dashboard shows how many occurred in the selected period. * **Scrape it into your own monitoring**: `planetscale_pods_container_status_restarts_total` counts **all** container restarts, not only out-of-memory kills. To count OOM kills, filter it against `planetscale_pods_container_last_terminated_reason`: ```promql theme={null} sum by (planetscale_keyspace, planetscale_shard, planetscale_container) ( sum_over_time(( increase(planetscale_pods_container_status_restarts_total[1m]) and on (planetscale_pod, planetscale_container) planetscale_pods_container_last_terminated_reason{planetscale_restart_reason="OOMKilled"} )[1h:1m]) ) ``` The gauge only reports a container's latest termination reason, so the query attributes each minute's restarts to the reason current in that minute, then sums over the hour. Change `1h` to your own window and leave the `1m` alone. Matching the gauge against the whole window in one step gives the wrong count. A container OOM-killed once and then restarted five times for other reasons reports six kills, or none, depending on which way you write it. The older `planetscale_pods_container_restarts_total` is deprecated and should not be used for this. See the [Prometheus metrics](/docs/vitess/integrations/prometheus-metrics) reference. > [!NOTE] > > Some workloads spike memory so quickly that the increase is not captured on the memory graph, which is sampled periodically. An OOM kill is recorded as an event, so it remains visible even when no corresponding spike appears on the memory graph. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vitess system limits Source: https://planetscale.com/docs/vitess/troubleshooting/planetscale-system-limits > [!NOTE] > > We can sometimes manually adjust limits on a per-database level. If you are facing issues or have questions, the best course of action is to [open a support ticket](https://planetscale.com/contact). > > Additionally, you can find some solutions to specific error codes in the [Errors documentation](/docs/vitess/troubleshooting/errors). ## Table limits Database schemas are limited to a total of `2048` tables, including views. Individual tables are limited to a maximum of `1017` columns each. ## Development branch limits Each database allows `100` development branches by default. Change the limit on the database **Settings** page. The limit can be any number from 1 to 5,000. When a database is at its limit, delete development branches you no longer need, or raise the limit, before creating another one. For a limit above 5,000, [contact support](https://planetscale.com/contact). ## Disk limits For [**network-attached storage**](/docs/plans/planetscale-skus#network-attached-storage) databases, the disk size will auto-scale up to a maximum of 4 TB. For [**Metal** databases](/docs/plans/planetscale-skus#metal), the disk size is dictated by what you choose when the database is created. Disk sizes range from 110 GB to 7.4 TB. Which sizes are available depends on the compute instance chosen. This space is used both for user data as well as system data required to run database operations. If your database is near this limit, some deploy request operations may not be available as they require adequate disk space to complete. When near this limit, we recommend either [sharding your data](/docs/vitess/sharding) or distributing tables across multiple keyspaces. For Metal databases, you can try upgrading to a larger disk size. Please [contact us](https://planetscale.com/contact) to discuss options. ## Connection lifetime limits Database client connections that are held open longer than 24 hours may be terminated unexpectedly. We recommend that long-running database connections are closed and reconnected at least once per day. ## Simultaneous transaction limits Each database has a limit on the number of simultaneous *transactions* it can process, also known as the *transaction pool*. If you exceed the *transaction pool* setting for your database, you may encounter this error: ``` vttablet: rpc error: code = ResourceExhausted desc = transaction pool connection limit exceeded ``` This can often be mitigated by trying one of the following solutions: 1. Reduce the amount of parallelism in the requests being sent to the database. 2. Shorten lengthy transactions by reducing batch sizes or making some other application-level adjustment. If you cannot make such changes to your application, consider choosing a larger instance type with a larger transaction pool. The exact limit varies depending on the instance type of your database. For details, see the [plans page](/docs/plans/planetscale-skus). ## Query limits PlanetScale has enforced some system limits to prevent long-running queries or transactions from: * Piling up and consuming all available resources. * Blocking other important, short-lived queries and transactions from completing. * Overloading the database to the point where it is not recoverable without a restart. * Blocking planned failovers and critical upgrades. The following table details these limits: | Type | Limit | | :- | :- | | Per-query rows returned, updated, or deleted | 100k | | Per-query result set total size | 64 MiB | | Per-query autocommit timeout | 900s | | Per-transaction timeout | 20s | ### Recommendations for handling query limits These limits are enforced for the safety of your database. However, we do understand you may run into a situation where the limits are a blocker. Here are some best practices for solving common issues presented by the limits: **What should I do if I have a query that returns more than 100,000 records?** We recommend trying to break up large queries, e.g. through [pagination](https://planetscale.com/blog/mysql-pagination). **What should I do if I have a query that returns more than 64 MiB of data?** If your schema currently relies on storing large amounts of variable length data within `JSON`, `BLOB`, or `TEXT` type columns that is regularly over a few MiB in size, you will want to strongly consider storing that variable length data outside of your database, such as within an object storage solution, instead. If large values are stored within variable length data columns, it can limit the number of rows you can return. For example, above we described a 100K limit for result sets, but if your result set's size exceeds the 64 MiB limit then you may receive an error message like the following: `resource_exhausted: grpc: received message larger than max ( vs. 67108864)` when retrieving much less than 100K rows. Similarly, when generating a large `INSERT` or `UPDATE` query for these types of columns you may run into the `grpc: trying to send message larger than max ( vs. 67108864)` error message which would require you to reduce the query's size. **What should I do if I have a transaction that runs longer than 20 seconds?** For transactions that are running for longer than 20 seconds, we recommend breaking these up into multiple shorter transactions. For analytics queries that are not possible to break up, see "How should I handle analytical queries?" below. Keep your transactions small and short-lived. For transactional workloads that are inherently long-lived, consider replacing or complementing MySQL transactions with other techniques: * For simple workloads (e.g. a `SELECT`, followed by an HTTP request, followed by an `UPDATE`), consider using optimistic locking instead of transactions. * For complex workloads, consider using sagas to coordinate a series of small, short-lived transactions that possibly span multiple microservices. **What should I do if I have a select query that needs to timeout faster than 900 seconds?** For select statements only, you can provide an inline comment to kill the query if it has not returned results within the specified time in milliseconds. `select /*vt+ QUERY_TIMEOUT_MS=2000 */ sleep(3);` Please note, you can set this value shorter than 900 seconds but not greater than the 900 seconds default. **How should I handle analytical queries?** We recommend using [PlanetScale Connect](https://planetscale.com/blog/extract-load-and-transform-your-data-with-planetscale-connect) (via [Airbyte](/docs/vitess/integrations/airbyte) or [Stitch](/docs/vitess/integrations/stitch)) to extract your data to any compatible destination (e.g. BigQuery, Redshift, etc.). The analytics queries should then be performed at those destinations. ### OLAP mode While it is possible to bypass these safety limits using `OLAP` mode (`SET workload = OLAP`), we do not recommend this for the reasons listed at the beginning of this doc. In OLAP mode, you may return more than 100k rows from a single query, but the 100k row limit will still apply to updates and deletes. When the use of OLAP queries is strictly unavoidable, we recommend: * Where possible, sending those queries to a replica. Every PlanetScale database comes with at least two replicas. Learn how to send queries to replicas using [global replica credentials](/docs/vitess/scaling/replicas#how-to-query-replicas). * Carefully and regularly reviewing the performance of those queries with [PlanetScale Insights](/docs/vitess/monitoring/query-insights). > [!NOTE] > > If you choose to use OLAP mode, be prepared to handle errors that may occur if the connection to PlanetScale gets interrupted. ## Reducing table size without `OPTIMIZE TABLE` If you delete several rows in a table, you may wish to reclaim that storage space. The MySQL [`OPTIMIZE TABLE` statement](https://dev.mysql.com/doc/refman/en/optimize-table.html) allows you to reorganize the physical storage of table data to reduce its storage requirements. However, `OPTIMIZE TABLE` is a locking operation, so you cannot use it unless you disable [safe migrations](/docs/vitess/schema-changes/safe-migrations) in PlanetScale, which is not recommended. Any time you create and deploy a deploy request with safe migrations on, our schema change process uses [online DDL](/docs/vitess/schema-changes/how-online-schema-change-tools-work), and in doing so, recreates the table that you are modifying — thus reclaiming the storage space. But there are often cases where you do not need to make a schema change for a while, but you'd like to free up the space. In those cases, we suggest you do the following: **Step 1** Create a new branch. **Step 2** Run the following command: ```sql theme={null} ALTER TABLE COMMENT 'Optimize table size via DR - '; ``` Where `` is the name of your table and `` is the current date. **Step 3** Create a deploy request, and merge it into production. If you use this option you will likely want to [throttle the deploy request](/docs/vitess/schema-changes/throttling-deploy-requests#managing-throttler-settings-per-deploy-request) running the `ALTER TABLE`. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Automatic Prisma migrations Source: https://planetscale.com/docs/vitess/tutorials/automatic-prisma-migrations > [!NOTE] > > This document has been updated to include the recommended Prisma and PlanetScale workflow, specifically the > recommendation to use `prisma db push` instead of `prisma migrate dev` with shadow branches. Also, you previously > needed to turn on the ability to automatically copy the Prisma migration metadata. You no longer need to do this. Read > more below. ## Introduction In this tutorial, we're going to learn how to do Prisma migrations in PlanetScale as part of your deployment process using `prisma db push`. ### Quick introduction to Prisma's db push From a high level, [Prisma's `db push`](https://www.prisma.io/docs/orm/reference/prisma-cli-reference#db-push) introspects your PlanetScale database to infer and execute the changes required to make your database schema reflect the state of your Prisma schema. When `prisma db push` is run, it will ensure the schema in the PlanetScale branch you are currently connected to matches your current Prisma schema. We recommend `prisma db push` over `prisma migrate dev` for the following reasons: PlanetScale provides [Online Schema Changes](/docs/vitess/schema-changes/how-online-schema-change-tools-work) that are deployed automatically when you merge a deploy request and prevents [blocking schema changes](/docs/vitess/schema-changes) that can lead to downtime. This is different from the typical Prisma workflow which uses `prisma migrate` in order to generate SQL migrations for you based on changes in your Prisma schema. When using PlanetScale with Prisma, the responsibility of applying the changes is on the PlanetScale side. Therefore, there is little value to using `prisma migrate` with PlanetScale. Also, the migrations table created when `prisma migrate` runs can also be misleading since PlanetScale does the actual migration when the deploy request is merged, not when `prisma migrate` is run which only updates the schema in the development database branch. You can still see the history of your schema changes in PlanetScale. ## Prerequisites * Add Prisma to your project using `npm install prisma --save-dev` or `yarn add prisma --dev` (depending on what package manager you prefer). * Run `npx prisma init` inside of your project to create the initial files needed for Prisma. * Install the [PlanetScale CLI](https://github.com/planetscale/cli). * Authenticate the CLI with the following command: ```bash theme={null} pscale auth login ``` ## Execute your first Prisma db push Prisma migrations follow the PlanetScale [non-blocking schema change](/docs/vitess/schema-changes) workflow. First, the schema is applied to a *development* branch and then the development branch is merged into the `main` production database. Let's begin with an example flow for running Prisma migrations in PlanetScale: **Step 1** Create a new *prisma-playground* database: ```bash theme={null} pscale db create prisma-playground ``` **Step 2** Connect to the database branch: ```bash theme={null} pscale connect prisma-playground main --port 3309 ``` > [!NOTE] > > This step assumes you created a new PlanetScale database and have not yet enabled [Safe Migrations](/docs/vitess/schema-changes/safe-migrations) on the `main` branch. You will need to create a new development branch otherwise. **Step 3** Update your `prisma/schema.prisma` file with the following schema: > [!NOTE] > > In Prisma `4.5.0`, `referentialIntegrity` changed to `relationMode` and became generally available in `4.7.0`. The following schema reflects this change. > > You can learn more about Prisma's Relation mode in the > [Prisma docs](https://www.prisma.io/docs/orm/prisma-schema/data-model/relations/relation-mode). ```js expandable theme={null} datasource db { provider = "mysql" url = env("DATABASE_URL") relationMode = "prisma" } generator client { provider = "prisma-client-js" } model Post { id Int @default(autoincrement()) @id createdAt DateTime @default(now()) updatedAt DateTime @updatedAt title String @db.VarChar(255) content String? published Boolean @default(false) author User @relation(fields: [authorId], references: [id]) authorId Int } model Profile { id Int @default(autoincrement()) @id bio String? user User @relation(fields: [userId], references: [id]) userId Int @unique } model User { id Int @default(autoincrement()) @id email String @unique name String? posts Post[] profile Profile? } ``` **Step 4** Update your `.env` file: ```shell theme={null} DATABASE_URL="mysql://root@127.0.0.1:3309/prisma-playground" ``` **Step 5** In another terminal, use the `db push` command to push the schema defined in `prisma/schema.prisma`: ```bash theme={null} npx prisma db push ``` Unlike the `prisma migrate dev` command, it will not create a migrations folder containing a SQL file with the SQL used to update the schema in your PlanetScale database. PlanetScale will be tracking your migrations in this workflow. > [!TIP] > > You can learn more about the `prisma db push` command in the > [Prisma docs](https://www.prisma.io/docs/orm/reference/prisma-cli-reference#db-push). After `db push` is successful, you can see the table created in your terminal. For example, to see the `Post` table: ```bash theme={null} pscale shell prisma-playground main ``` ```sql theme={null} describe Post; ``` > [!TIP] > > Use the `exit` command to exit the MySQL shell. Or you can see it in the PlanetScale UI under the Schema tab in your `main` branch. **Step 6** Finally, turn on safe migrations on the `main` branch to enable non-blocking schema changes: ```bash theme={null} pscale branch safe-migrations enable prisma-playground main ``` ## Execute succeeding Prisma migrations in PlanetScale Our first example migration flow went well, but what happens when you need to run further changes to your schema? Let's take a look: **Step 1** Create a new *development* branch from `main` called `add-subtitle-to-posts`: ```bash theme={null} pscale branch create prisma-playground add-subtitle-to-posts ``` **Step 2** Close the proxy connection to your `main` branch (if still open) and connect to the new `add-subtitle-to-posts` development branch: ```bash theme={null} pscale connect prisma-playground add-subtitle-to-posts --port 3309 ``` **Step 3** In the `prisma/schema.prisma` file, update the `Post` model: Add a new `subtitle` field to `Post`: ``` subtitle String @db.VarChar(255) ``` **Step 4** Run `db push` again to update the schema in PlanetScale: ```bash theme={null} npx prisma db push ``` **Step 5** Open a deploy request for your `add-subtitle-to-posts` branch, so that you can deploy these changes to `main`. You can complete the deploy request either in the web app or with the `pscale deploy-request` command. ```bash theme={null} pscale deploy-request create prisma-playground add-subtitle-to-posts ``` ```bash theme={null} pscale deploy-request deploy prisma-playground 1 ``` **Step 6** Once the deploy request is merged, you can see the results in your main branch's `Post` table: ```bash theme={null} pscale shell prisma-playground main ``` ```sql theme={null} describe Post; ``` ## What's next? Now that you've successfully conducted your first automatic Prisma migration in PlanetScale and know how to handle future migrations, it's time to deploy your application with a PlanetScale database! Let's learn how to [deploy an application with a PlanetScale database to Vercel](/docs/vitess/tutorials/deploy-to-vercel). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Automatic Rails migrations Source: https://planetscale.com/docs/vitess/tutorials/automatic-rails-migrations > [!TIP] > > If you are using PlanetScale with a Rails application, go to your database's Settings page in the web app and enable > "Automatically copy migration data." Select "Rails/Phoenix" as the migration framework. When enabled, this setting > updates the *schema\_migrations* table each time you branch with the latest migration. If disabled, running > *rake db:migrate* will try to run all migrations every time, instead of only the latest one. ## Introduction In this tutorial, you're going to learn how Rails migrations work with the PlanetScale branching and deployment workflows. > [!NOTE] > > Migration tracking works with any migration tool, not just Rails. For other frameworks, specify the migration table name on your database's Settings page. ## Prerequisites Follow the [Connect a Rails app](/docs/vitess/tutorials/connect-rails-app) tutorial first. By the end, you will have: * Installed the [PlanetScale CLI](https://github.com/planetscale/cli), Ruby, and the Rails gem * Created a PlanetScale database named `blog` * Started a new Rails app named `blog` with a migration creating a `Users` table * Run the first Rails migration ### A quick introduction to Rails migrations Rails tracks an application's migrations in an internal table called `schema_migrations`. At a high level, running `rake db:migrate` does the following: * Rails looks at all of the migration files in your `db/migrate` directory. * Rails queries the `schema_migrations` table to see which migrations have and haven't been run. * Any migration that doesn’t appear in the `schema_migrations` table is considered pending and is executed by this task. > [!TIP] > > When you merge a deploy request in PlanetScale, the *schema\_migrations* table in *main* is > automatically updated with the migration data from your branch. ## Execute a Rails migration on PlanetScale Rails migrations follow the PlanetScale [non-blocking schema change](/docs/vitess/schema-changes) workflow. First, the migration is applied to a *development* branch, and then the development branch is merged into the `main` production branch with [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled. Let's add another table to your existing `blog` schema: **Step 1** Create an `add-posts-table` development branch from `main` in your database *blog*: ```bash theme={null} pscale branch create blog add-posts-table ``` When the branch is ready, you can verify that the `schema_migrations` table is up-to-date with `main` by checking for the timestamp of your `Create Users` migration file. Your migration will have a different timestamp than the one shown here. Check the timestamp in your codebase: ```bash theme={null} ls db/migrate 20211014210422_create_users.rb ``` Connect to the new branch: ```bash theme={null} pscale shell blog add-posts-table ``` Query the migration table: ```sql theme={null} blog/add-posts-table> select * from schema_migrations; +----------------+ | version | +----------------+ | 20211014210422 | +----------------+ ``` **Step 2** Connect your development environment to the new branch: One way to do this is to create a new password for the `add-posts-table` branch and update `config/database.yml` with the new username, password, and host. Another is to use `pscale connect` to establish a secure connection on a local port. Since the `add-posts-table` branch won't be needed after the migration, let's use the `pscale connect` proxy. In a separate terminal, establish the connection: ```bash theme={null} pscale connect blog add-posts-table --port 3309 ``` Then, update `config/database.yml` to connect through the proxy: ```yaml theme={null} development: <<: *default adapter: trilogy database: blog host: 127.0.0.1 port: 3309 ``` **Step 3** Create the second Rails migration and call it `CreatePosts`: ```bash theme={null} rails generate migration CreatePosts ``` Find the new migration file in `db/migrate` and add a few details for the new Posts table: ```ruby theme={null} class CreatePosts < ActiveRecord::Migration[7.0] def change create_table :posts do |t| t.string :title t.text :content t.bool :published t.references :user t.timestamps end end end ``` **Step 4** Run the CreatePosts migration: ```bash theme={null} rake db:migrate ``` This command runs the new migration against your `add-posts-table` *development* branch. At this point, Rails creates the `posts` table and inserts another `timestamp` into the `schema_migrations` table on your development branch. You can verify the change in `schema_migrations` yourself: ```sql theme={null} blog/add-posts-table> select * from schema_migrations; +----------------+ | version | +----------------+ | 20211014210422 | | 20220224221753 | +----------------+ ``` **Step 5** Open a deploy request for your `add-posts-table` branch, and deploy your changes to `main`. You can complete the deploy request either in the web app or with the `pscale deploy-request` command. ```bash theme={null} pscale deploy-request create blog add-posts-table ``` ```bash theme={null} pscale deploy-request deploy blog 1 ``` To create the deploy request, PlanetScale looks at the differences between the schemas of `main` and `add-posts-table` and plans a `create table` statement to add the new table to `main`. When you deploy, PlanetScale runs that ` create table` statement and copies the second row from `schema_migrations` in `add-posts-table` to the `schema_migrations` table in `main`.\` **Step 6** Verify the changes in your `main` branch: In a `pscale` shell for `main` you can verify that the changes from `add-posts-table` were deployed successfully. ```bash theme={null} pscale shell blog main ``` ```sql theme={null} blog/|⚠ main ⚠|> show tables; +----------------------+ | Tables_in_blog | +----------------------+ | posts | | schema_migrations | | users | +----------------------+ blog/|⚠ main ⚠|> select * from schema_migrations; +----------------+ | version | +----------------+ | 20220223232425 | | 20220224221753 | +----------------+ ``` ## Summary In this tutorial, we learned how to use the PlanetScale deployment process with the Rails migration workflow. ## What's next? Learn more about how PlanetScale allows you to make [schema changes](/docs/vitess/schema-changes) to your production databases without downtime or locking tables. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # AWS Lambda connection strings Source: https://planetscale.com/docs/vitess/tutorials/aws-lambda-connection-strings In this guide, you'll learn how to properly store and use PlanetScale MySQL connection strings for use in AWS Lambda Functions. ## Introduction We'll use a [pre-built NodeJS](https://github.com/planetscale/aws-connection-strings-example) app for this example, but you can follow along using your own application as well. ## Prerequisites * An AWS account * A [PlanetScale account](https://auth.planetscale.com/sign-up) ## Set up the database > [!NOTE] > > If you already have a database with a production branch, skip to [the next section](#configure-the-lambda-function). Let's start by creating the database. In the PlanetScale dashboard, click the "**New database**" button followed by "**Create new database**". Name the database **lambda-connection-strings,** or any other name that you prefer. Click "**Create database**". Once your database has finished initializing, you'll need to enable the web console on production branches. To do this, go to the "**Settings**" tab, check "**Allow web console access to production branches**", and click "**Save database settings**". Now, access the console of the main branch by clicking "**Console**", then "**Connect**". ![The console](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/console-3.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=8255d9deaa6855225cb9a041be6c5a89) Create a simple table & insert some data using the following script: ```sql theme={null} CREATE TABLE Tasks( Id int PRIMARY KEY AUTO_INCREMENT, Name varchar(100), IsDone bit ); INSERT INTO Tasks (Name) VALUES ('Clean the kitchen'); INSERT INTO Tasks (Name) VALUES ('Fold the laundry'); INSERT INTO Tasks (Name) VALUES ('Watch the sportsball game'); ``` You may run `SELECT * FROM Tasks` to ensure the data was properly added from the console. ![Records from the console](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/select.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=d48ac40df1cfbb67394c9b55937c8f31) Now we need to enable [**safe migrations**](/docs/vitess/schema-changes/safe-migrations) on the **main** branch. Click the **Dashboard** tab, then click the **cog** icon in the upper right of the infrastructure card. ![The option to promote a branch](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/production-2.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=3eac808438c88fbc3b87fc273742d818) Toggle on the "**Enable safe migrations**" option and click the "**Enable safe migrations**" button. ![Enable safe migrations](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/safe-migrations-2.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=e85f41d66261fe91d36a6d081456cbec) Before moving on from the PlanetScale dashboard, grab the connection details to be used in the next step. Click on the **Connect** button to go to the Connect page. Enter a name for your password, and click the **Create password** button to generate a new password. ![The dashboard after the database has been promoted](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/promoted-2.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=9761608fc3826672a130048a7a232274) In the **Select your language or framework** section, select **Node.js** and note the details in the `.env` section of the guide. These details will be required to connect to the database. ## Configure the Lambda function Secrets in AWS Lambda functions, which include database connection strings, are often stored as environment variables with the Lambda function. We’ll be uploading a sample NodeJS app that has been provided and storing the connection string from the previous section as an environment variable to test. Start by cloning the following Git repository: ```bash theme={null} git clone https://github.com/planetscale/aws-connection-strings-example.git ``` Log into the AWS Console, use the universal search to search for ‘**Lambda**’, and select it from the list of services. ![Search for Lambda in the AWS Console](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/aws.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=1b04f3e1c89e89f739ea09a30f17bde8) Create a new function using the **Create function** button in the upper right of the console. ![The default view of Lambda functions](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/functions.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=04926ae1d4bc634b5d0f12fc58877fb0) Name your function **lambda-connection-strings** (or any other name that suits you) and select **NodeJS** under **Runtime**. The other fields can be left as default. Click **Create function** to finish the initial setup of your Lambda. ![The view to create a Lambda function](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/create-function.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=8f1b1d8d1416735fac6c69ba6de610ec) On the next view, about halfway down the page you’ll see a section called **Code source**. Click the **Upload from** button, then **.zip file**. ![The default NodeJS Lambda function](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/node.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=c4cb6bcaaaa6e2088f882c1303a5376f) Click the **Upload** button which will display a file browser. Select the **aws-connection-strings-example.zip** file from the **dist** folder of the provided repository. Click **Save** once it’s been selected. ![The modal to upload code](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/upload.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=0b71f632247356949c022b826bb2ef84) The contents of the code editor under **Code source** should have updated to show the code stored in the zip file. ![The code of the Lambda function that was uploaded](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/source.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=b0e9c090ece970d4bd110b82a0420ca9) ### Configure environment variables Next, you need to set the PlanetScale `DATABASE_URL` environment variable that you copied earlier. Select the **Configuration** tab, and click **Edit**. ![The configuration tab](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/configuration.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=27df1ca8b7351d0dcf49f7da946f70fe) You’ll be presented with a view to add or update environment variables. Click **Add environment variable** and the view will update with a row to add an environment variable. Set the **Key** field to **DATABASE\_URL** and the **Value** to the connection string taken from the previous section. Click **Save** once finished. ![The view to manage environment variables](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/environment-variables.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=faaf49570dc93d29ef979d9ee04053ab) Finally, test the function by selecting the **Test** tab, and then clicking the **Test** button. ![The test tab](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/test.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=395b8104f5c86c154d0b6ae5ef3c08f6) An **Execution results** box will display above the **Test event** section. If the box is green, it likely means everything executed as expected. Click the dropdown next to **Details** to see the results of the query. Since the results of the query were logged out to the console, they will be displayed in the **Log output** section. ![The execution results](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/aws-lambda-connection-strings/success.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=e2bc56e236240c8ec2faf45ae5db578d) ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect any application to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-any-application ## Introduction In this tutorial, you'll learn how to connect any application to your PlanetScale database. If you're just getting started and still need to set up a database, we recommend starting with the [PlanetScale quick start guide](/docs/vitess/tutorials/planetscale-quick-start-guide) first. We also have language/framework-specific guides under "**Integration guides**" if you prefer a more detailed walk-through. PlanetScale uses [database branches](/docs/vitess/schema-changes/branching) to create a development-friendly workflow. Your database is initially created with a default branch, `main`, which is meant to serve as a production database branch. Production branches are highly available databases intended for production traffic. They are automatically provided with an additional replica to resist outages, enabling zero-downtime failovers. You can connect to a *production* or *development* database branch. We recommend creating and connecting to a *development* branch while in *development*, as it allows you to make schema changes without affecting production. Your production application, however, should be connected to your production database branch. Check out our [Branching guide](/docs/vitess/schema-changes/branching) for more information about the branching workflow. > [!NOTE] > > By default, production branches have safe migrations turned off. This means that any schema changes you make will be applied immediately. Once you are ready to go to production, we recommend turning on safe migrations if you want to make non-blocking schema changes. Check out our [Safe migrations documentation](/docs/vitess/schema-changes/safe-migrations) for more information. There are two ways to connect your app to PlanetScale. Both are covered below. ## Option 1: Connect with username and password (Recommended) This section will show you how to create a username and password for your branch and use those credentials to connect to your database. This is the recommended way to connect. There are two ways to generate a new username and password for your branch: * In the PlanetScale dashboard * With the PlanetScale CLI ### Generate credentials in the PlanetScale dashboard **Step 1** On the database dashboard page, click the **Connect** button. **Step 2** Select the database branch that you want to create a password for. **Step 3** Click the **Create password** button. **Step 4** Select the applicable language or framework for your application or click "Other". **Step 5** Copy the credentials. **Step 6** Paste them in your application's MySQL configuration file (often just a `.env` file). The layout and name of this file will vary depending on the application language, but it may look something like this: ```bash theme={null} DATABASE= USERNAME= HOST= PASSWORD= SSL= // more information about this in next step ``` Check out our language integration guides in the side navigation for more explicit instructions. **Step 7** To ensure a secure connection, you must validate the server-side certificate from PlanetScale. This configuration depends on your application, but often it just means adding a line to your `.env` file similar to this: ```bash theme={null} MYSQL_ATTR_SSL_CA=/etc/ssl/cert.pem ``` The path to the certificate depends on your system. The above example shows the path for macOS, but you can find others in our [Secure connections documentation](/docs/vitess/connecting/secure-connections#ca-root-configuration). Again, the variable name here, `MYSQL_ATTR_SSL_CA`, is just an example. The actual name and location for it will depend on the application. If you're **unsure what to put here**, we recommend selecting your application's language or framework from the 2nd section of the Connect page (see step 4 above) and copy the credentials from there. This includes the necessary SSL configuration variables and shows what files they belong in. Additionally, we show you the correct certificate path by default based on your system. ### Generate credentials in the PlanetScale CLI If you prefer working from the CLI, you can quickly spin up new credentials there. Make sure you have the [CLI set up](/docs/cli/planetscale-environment-setup) first. **Step 1** Run the following command in the CLI to create a new username and password for your branch. ```bash theme={null} pscale password create ``` > [!NOTE] > > The `PASSWORD_NAME` value represents the name of the username and password being generated. You can have multiple credentials for a branch, so this gives you a way to categorize them. To manage your passwords in the dashboard, go to your database dashboard page, click "Settings", and then click "Passwords". **Step 2** Take note of the values returned. You won't be able to see this password again. ``` Password production-password was successfully created. Please save the values below, as they will not be shown again. NAME USERNAME ACCESS HOST URL ROLE PASSWORD --------------------- ------------- --------------------------------- ------------------ -------------------------------- production-password xxxxxxxxxx xxxxxxxxxx.us-east-2.psdb.cloud Can Read & Write pscale_pw_xxxxxx_xxxxxxxxxxxxx ``` **Step 3** Paste the values from the console output into your application's MySQL configuration file. The layout and name of this file will vary depending on the application language, but it may look something like this: ```bash theme={null} DATABASE= USERNAME= HOST= PASSWORD= SSL= // This is covered in the next step ``` **Step 4** To ensure a secure connection, you must validate the server-side certificate from PlanetScale. This configuration depends on your application, but often it just means adding a line to your `.env` file similar to this: ```bash theme={null} MYSQL_ATTR_SSL_CA=/etc/ssl/cert.pem ``` The path to the certificate depends on your system. The above example shows the path for macOS, but you can find others in our [Secure connections documentation](/docs/vitess/connecting/secure-connections#ca-root-configuration). Again, the variable name here, `MYSQL_ATTR_SSL_CA`, is just an example. The actual name and location for it will depend on the application. If you're **unsure what to put here**, we recommend selecting your application's language from the dropdown in the PlanetScale dashboard (see step 3 from the previous section) and copying the credentials from there. This includes the necessary SSL configuration variables and shows what files they belong in. Additionally, we show you the correct certificate path by default based on your system. ## Option 2: Connect using the PlanetScale proxy Another way to connect your application to your PlanetScale database *during development* is using the PlanetScale proxy. You won't have to fiddle with configuring any credential details, as that's handled by PlanetScale. It's as simple as a single CLI command. You'll use the CLI to establish a secure connection to PlanetScale. It will listen on a local port that your application can connect to. The main benefit of this method is you won't have to generate and remember multiple passwords every time you're creating or switching to a new branch. **Step 1** Make sure you have [the CLI set up](/docs/cli/planetscale-environment-setup), and then run the following command: ```bash theme={null} pscale connect ``` This establishes a secure connection and opens a port on your local machine that you can use to connect to any MySQL client. **Step 2** Take note of the address it returns to you. By default, it is `127.0.0.1:3306`. The CLI will use a different port if `3306` is unavailable. **Step 3** In your application's MySQL configuration file, use the following to connect: ```bash theme={null} DATABASE= HOST=127.0.0.1 PORT=3306 // use the value that was returned in the console ``` Your application should now be connected to the specified PlanetScale database branch! ## What's next? Once your application is connected to a development database branch, you can make schema changes in an isolated development environment without worrying about affecting production. Additionally, we recommend that [safe migrations](/docs/vitess/schema-changes/safe-migrations) be enabled on your production database branch, which allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) without locking or causing downtime. ### PlanetScale workflow Here's the general workflow that you'll go through to get schema changes from development to production: **Step 1** Follow this guide to connect to a development branch. **Step 2** Modify your schema as needed. **Step 3** Test them locally or in your staging environment. **Step 4** Once satisfied and ready to deploy your changes to production, [create a deploy request](/docs/vitess/schema-changes/deploy-requests). **Step 5** You or your team can review and approve the schema changes. **Step 6** Deploy your deploy request to production. **Step 7** Bonus: If you realize you made a mistake, you can click "Revert changes" to [undo a schema change](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change). > [!NOTE] > > Note: you must already have [a production branch](/docs/vitess/schema-changes/branching#promote-a-branch-to-production) in place to create a deploy request. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Django application to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-django-app ## Introduction In this tutorial, you'll learn how to connect a Django application to a PlanetScale MySQL database using a pre-built Django application. > [!TIP] > > Already have a Django application and just want to connect to PlanetScale? Check out the [Django quick connect repo](https://github.com/planetscale/connection-examples/tree/main/python). ## Prerequisites * Python — This tutorial uses `v3.6` * A [PlanetScale account](https://auth.planetscale.com/sign-up) * (Optional) [PlanetScale CLI](https://github.com/planetscale/cli) — This isn't required, but it can make setup much faster ## Set up the Django application This guide integrates a simple Django application with PlanetScale. The application has one endpoint that displays a list of products and categories pulled from the database. If you have an existing application, you can also use that. **Step 1** Clone the starter Django application and switch into the project folder: ```bash theme={null} git clone https://github.com/planetscale/django-example.git cd django-example ``` **Step 2** Start the virtual environment: ```bash theme={null} python3 -m venv env source env/bin/activate ``` For Windows, use `env/Scripts/activate`. **Step 3** Install the required packages: ```bash theme={null} pip install -r ./requirements.txt ``` ## Set up the database Next, you need to set up your PlanetScale database and connect to it in the Django application. You can create a database either in the [PlanetScale dashboard](https://app.planetscale.com) or from the PlanetScale CLI. This guide will use the CLI, but you can follow the database setup instructions in the [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide#getting-started-%E2%80%94-planetscale-dashboard) if you prefer the dashboard. Authenticate the CLI with the following command: ```bash theme={null} pscale auth login ``` Create a new database with a default `main` branch with the following command: ```bash theme={null} pscale database create --region ``` This tutorial uses `django_example` for `DATABASE_NAME`, but you can use any name with lowercase, alphanumeric characters, or underscores. You can also use dashes, but we don't recommend them, as they may need to be escaped in some instances. For `REGION_SLUG`, choose a region closest to you from the [available regions](/docs/vitess/regions#available-regions) or leave it blank. That's it! Your database is ready to use. Next, let's connect it to the Django application and then add some data. ## Connect to the Django application There are **two ways to connect** your Django application to PlanetScale: * With an auto-generated username and password * Using the PlanetScale proxy with the CLI Both options are covered below. ### Option 1: Connect with username and password (Recommended) **Step 1** Create a username and password with the PlanetScale CLI by running: ```bash theme={null} pscale password create ``` > [!NOTE] > > The `PASSWORD_NAME` value represents the name of the username and password being generated. You can have multiple credentials for a branch, so this gives you a way to categorize them. To manage your passwords in the dashboard, go to your database dashboard page, click "Settings", and then click "Passwords". You can also get these exact values to copy/paste from your [PlanetScale dashboard](https://app.planetscale.com). In the dashboard, click on the database > "**Connect**" > "**Connect with**" language dropdown > "**Django**". If the password is blurred, click "**New password**". Take note of the values returned to you, as you won't be able to see this password again. **Step 2** Open the .env file in your Django app, find the database connection section, and fill it in as follows: ```bash theme={null} DB_HOST= DB_PORT=3306 DB_NAME= DB_USER= DB_PASSWORD= MYSQL_ATTR_SSL_CA=/etc/ssl/cert.pem ``` The value for `MYSQL_ATTR_SSL_CA` may differ [depending on your operating system](/docs/vitess/connecting/secure-connections#ca-root-configuration). **Step 3** Next, in the `mysite/settings.py` file, scroll down and look for the `DATABASES` object. Replace it with the following: ```python theme={null} DATABASES = { 'default': { 'ENGINE': 'django_psdb_engine', 'NAME': os.environ.get('DB_NAME'), 'HOST': os.environ.get('DB_HOST'), 'PORT': os.environ.get('DB_PORT'), 'USER': os.environ.get('DB_USER'), 'PASSWORD': os.environ.get('DB_PASSWORD'), 'OPTIONS': {'ssl': {'ca': os.environ.get('MYSQL_ATTR_SSL_CA')}} } } ``` ### Option 2: Connect with PlanetScale proxy To connect with the PlanetScale proxy, you'll need the [PlanetScale CLI](https://github.com/planetscale/cli). **Step 1** Open a connection by running the following: ```bash theme={null} pscale connect ``` If you're following this guide exactly and haven't created any new branches, you'll use the default branch, `main`, for `BRANCH_NAME`. **Step 2** A secure connection to your database will be established and you'll see a local address you can use to connect to your application. **Step 3** Open the `.env` file in your Django app and update it as follows: ```bash theme={null} DB_HOST=127.0.0.1 DB_PORT=3306 DB_NAME= ``` The connection uses port `3306` by default, but if that's being used, it will pick a random port. Make sure you paste in whatever port is returned in the terminal. ## Optional — Bring in PlanetScale custom database wrapper This next step is only necessary if you're using your own application to go through this guide **and** [do not want to use foreign key constraints](/docs/vitess/operating-without-foreign-key-constraints). If you cloned the sample app, this already exists in the repo. Foreign key constraint support is not enabled by default in PlanetScale. If you'd like to use foreign key constraints in your Django application with PlanetScale, you must first enable foreign key constraint support in your database settings page. Django uses foreign key constraint syntax by default. So, if you want to proceed without foreign key constraints, you need to pull in the PlanetScale database wrapper for Django to disable foreign key syntax in the Django migrations. **Step 1** Run the following to pull it in: ```bash theme={null} git clone https://github.com/planetscale/django_psdb_engine.git ``` **Step 2** In your `settings.py` file, add `django_psdb_engine` as the database engine. ```python theme={null} DATABASES = { 'default': { 'ENGINE': 'django_psdb_engine', } } ``` ## Run migrations and seeder Now that you're connected, let's add some data to see everything in action. You can find the migrations file in `mysite/store/migrations/0002_auto_20220919_0058.py` that references the `Category` and `Product` models to create the schema. It also contains seed data to create two products and two categories. Run this migration with: ```bash theme={null} python manage.py migrate ``` This will also run the default Django migrations. ## Display the data Finally, let's display the data to confirm that everything worked correctly. This Django starter application has a pre-built endpoint, `/products`, that will grab and display all of the product data. To view the data, start the server: ```bash theme={null} python manage.py runserver ``` Then go to [`localhost:8000/products`](http://localhost:8000/products) and you'll see a list of the data from the products table. ## Add data manually If you want to continue playing around with adding data on the fly, you have a few options: * PlanetScale CLI shell * PlanetScale dashboard console * Your favorite MySQL client (for a list of tested MySQL clients, review our article on [how to connect MySQL GUI applications](/docs/vitess/tutorials/connect-mysql-gui)) The first two options are covered below. ### Add data with PlanetScale CLI You can use the PlanetScale CLI to open a MySQL shell to interact with your database. You may need to [install the MySQL command line client](/docs/cli/planetscale-environment-setup) if you haven't already. Run the following command in your terminal: ```bash theme={null} pscale shell ``` This will open up a MySQL shell connected to the specified database and branch. > [!NOTE] > > A branch, `main`, was automatically created when you created your database, so you can use that for `BRANCH_NAME`. Add a record to the `store_product` table: ```sql theme={null} INSERT INTO `store_product` (name, description, image, category_id) VALUES ('Product 3', 'Product 3 description', 'https://via.placeholder.com/300.png?text=Product1', 1); ``` The value `id` will be filled with a default value. Type `exit` to exit the shell. Refresh the Django homepage to see the new record. You can also verify it was added in the PlanetScale CLI MySQL shell with: ```sql theme={null} select * from store_product; ``` ### Add data with PlanetScale dashboard console If you don't care to install MySQL client or the PlanetScale CLI, another quick option using the MySQL console built into the PlanetScale dashboard. **Step 1** Go to your [PlanetScale dashboard](https://app.planetscale.com) and select your Django database. **Step 2** Click on "**Console**". **Step 3** Select the `main` branch and click "**Connect**". **Step 4** Add a new record to the `store_product` table with: ```sql theme={null} INSERT INTO `store_product` (name, description, image, category_id) VALUES ('Product 3', 'Product 3 description', 'https://via.placeholder.com/300.png?text=Product1', 1); ``` **Step 5** You can confirm that it was added by running: ```sql theme={null} select * from store_product; ``` You can also head to the [`/products`](http://localhost:8000/products) endpoint in your Django application to see the new data. ## Foreign key constraints with PlanetScale If you'd like to use foreign key constraints in your Django application with PlanetScale, you must first enable foreign key constraint support in your database settings page. If you prefer to enforce referential integrity at the application level, you will have to do some extra configuration with Django. You can disable foreign key constraint checks at the model level in Django, but if you're running the default migrations, you'll need to turn them off globally. The [PlanetScale custom database backend](https://github.com/planetscale/django_psdb_engine) manages this, but if you want to do it manually in each model. For example, in the `models.py` file for the example in this document, we define the foreign key on the `category` table with the following: ```python theme={null} category = models.ForeignKey(Category, on_delete=models.DO_NOTHING, db_constraint=False) ``` This isn't necessary to do in every model if you're pulling in the `django_psdb_engine` because it overrides the setting globally anyway, but this will work if you want to do it per model. ## What's next? Once you're done with initial development, you can enable [safe migrations](/docs/vitess/schema-changes/safe-migrations) to protect from accidental schema changes and enable zero-downtime deployments. Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Go application to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-go-app [Connect to PlanetScale with Go](https://player.vimeo.com/video/759188218?dnt=true) ## Introduction In this guide, you’ll learn how to connect to a PlanetScale MySQL database with Go by exploring a sample API built using the Gin routing framework. **Prerequisites:** * [Go](https://go.dev/doc/install) * [A PlanetScale account](https://auth.planetscale.com/sign-up) * [VS Code](https://code.visualstudio.com/download) (optional) * The [VS Code Rest Client plugin](https://marketplace.visualstudio.com/items?itemName=humao.rest-client) (optional) > [!TIP] > > Already have a Go application and just want to connect to PlanetScale? Check out the [Go quick connect repo](https://github.com/planetscale/connection-examples/tree/main/go). ## Create the database Start in PlanetScale by creating a new database. From the dashboard, click "**New Database**", then "**Create new database**". Name the database `products_db`, select the desired [Plan type](/docs/planetscale-plans), and click "**Create database**". By default, web console access to production branches is disabled to prevent accidental deletion. From your database's dashboard page, click on the "**Settings**" tab, check the box labelled "**Allow web console access to production branches**", and click "**Save database settings**". Then, click on the **"Console"** tab, then "**Connect**". ![The Console tab](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/connect-go-app/console-2.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=47ce9ab0dec9d1aa926fdbfd3e2e3f3d) Run the following two commands to create a sample table and insert some data: ```sql theme={null} CREATE TABLE `products` ( `id` int PRIMARY KEY AUTO_INCREMENT, `name` varchar(100) NOT NULL, `price` int NOT NULL ); INSERT INTO `products` (name, price) VALUES ('Cyberfreak 2076', 40), ('Destination 2: Shining Decline', 20), ('Edge Properties 3', 15); ``` Finally, head to the **"Dashboard"** tab and click **"Connect"**. ![The location of the Connect button](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/connect-go-app/connect-2.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=3dafe11e53a351c65575f30f0a246c84) On the following page, click **"Create password"** to generate a new password for your database. Then click **Go** in the **Select your language or framework** section, and copy the contents of the `.env` file. You'll need it for the next section. ## Run the demo project Start by opening a terminal on your workstation and clone the sample repository provided. ```bash theme={null} git clone https://github.com/planetscale/golang-example-gin.git ``` Open the project in VS Code and add a new file in the root of the project named `.env`, Populate the file with the contents taken from the Connect modal in the previous section. ```sql theme={null} DSN=****************:************@tcp(us-east.connect.psdb.cloud)/products_db?tls=true&interpolateParams=true ``` Now open an integrated terminal in VS Code and run the project using the following commands: ```bash theme={null} go mod tidy go run . ``` The terminal should update with the following output. ![The output of the GET test](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/connect-go-app/go-run-output.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=d39f619c6cadddb3e9330b6087de3539) ## Exploring the code Now that the project is running, let’s explore the code to see how everything works. All of the code is stored in `main.go`, with each of the core SQL operations mapped by HTTP method in the `main` function: | HTTP Method Name | Query Type | | :- | :- | | get | SELECT | | post | INSERT | | put | UPDATE | | delete | DELETE | ```go theme={null} func main() { // Load in the `.env` file err := godotenv.Load() if err != nil { log.Fatal("failed to load env", err) } // Open a connection to the database db, err = sql.Open("mysql", os.Getenv("DSN")) if err != nil { log.Fatal("failed to open db connection", err) } // Build router & define routes router := gin.Default() router.GET("/products", GetProducts) router.GET("/products/:productId", GetSingleProduct) router.POST("/products", CreateProduct) router.PUT("/products/:productId", UpdateProduct) router.DELETE("/products/:productId", DeleteProduct) // Run the router router.Run() } ``` Open the `tests.http` file, which contains HTTP requests that can be sent to test the API. Running the `get {{hostname}}/products` test is the equivalent of running `SELECT * FROM products` in SQL and returning the results as JSON. > [!WARNING] > > If you do not wish to use VS Code with the Rest Client plugin, you may use `tests.http` as a reference for your preferred IDE and API testing software. ![The terminal output of the go run command](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/connect-go-app/go-run-output.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=d39f619c6cadddb3e9330b6087de3539) This is the `GetProducts` function defined in `main.go`. Notice how the `query` variable is the `SELECT` statement, which is passed into `db.Query` before being scanned into a slice of `Product` structs. ```go expandable theme={null} func GetProducts(c *gin.Context) { query := "SELECT * FROM products" res, err := db.Query(query) defer res.Close() if err != nil { log.Fatal("(GetProducts) db.Query", err) } products := []Product{} for res.Next() { var product Product err := res.Scan(&product.Id, &product.Name, &product.Price) if err != nil { log.Fatal("(GetProducts) res.Scan", err) } products = append(products, product) } c.JSON(http.StatusOK, products) } ``` To pass parameters into queries, you may use a `?` as a placeholder for the parameter. For example, `GetSingleProduct` uses a query with a `WHERE` clause that is passed into the `db.QueryRow` function along with the query string. ```go expandable theme={null} func GetSingleProduct(c *gin.Context) { productId := c.Param("productId") productId = strings.ReplaceAll(productId, "/", "") productIdInt, err := strconv.Atoi(productId) if err != nil { log.Fatal("(GetSingleProduct) strconv.Atoi", err) } var product Product // `?` is a placeholder for the parameter query := `SELECT * FROM products WHERE id = ?` // `productIdInt` is passed in with the query err = db.QueryRow(query, productIdInt).Scan(&product.Id, &product.Name, &product.Price) if err != nil { log.Fatal("(GetSingleProduct) db.Exec", err) } c.JSON(http.StatusOK, product) } ``` Parameters in queries are populated in the order they are passed into the respective `db` function, as demonstrated in `CreateProduct`. ```go expandable theme={null} func CreateProduct(c *gin.Context) { var newProduct Product err := c.BindJSON(&newProduct) if err != nil { log.Fatal("(CreateProduct) c.BindJSON", err) } // This query has multiple `?` parameter placeholders query := `INSERT INTO products (name, price) VALUES (?, ?)` // The `Exec` function takes in a query, as well as the values for // the parameters in the order they are defined res, err := db.Exec(query, newProduct.Name, newProduct.Price) if err != nil { log.Fatal("(CreateProduct) db.Exec", err) } newProduct.Id, err = res.LastInsertId() if err != nil { log.Fatal("(CreateProduct) res.LastInsertId", err) } c.JSON(http.StatusOK, newProduct) } ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Go application using GORM to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-go-gorm-app ## Introduction In this tutorial, you'll learn how to connect a Go application to a PlanetScale MySQL database using a sample Go starter app with GORM. > [!TIP] > > Already have a Go application and just want to connect to PlanetScale? Check out the [Go quick connect > repo](https://github.com/planetscale/connection-examples/tree/main/go). ## Prerequisites * [Go](https://go.dev/doc/install) * A [PlanetScale account](https://auth.planetscale.com/sign-up) * [PlanetScale CLI](https://github.com/planetscale/cli) — You can also follow this tutorial in the PlanetScale admin dashboard, but the CLI will make setup quicker. ## Set up the Go app This guide will integrate [a simple Go (Golang) app](https://github.com/planetscale/golang-example) with PlanetScale that will display a list of products stored in the database. If you have an existing application, you can also use that. **Step 1** Clone the starter Go application: ```bash theme={null} git clone https://github.com/planetscale/golang-example.git ``` **Step 2** Enter into the folder: ```bash theme={null} cd golang-example ``` **Step 3** Copy the `.env.example` file into `.env`: ```bash theme={null} cp .env.example .env ``` ## Set up the database Next, you need to set up your PlanetScale database and connect to it in the Go application. You can create a database in the [PlanetScale dashboard](https://app.planetscale.com) or from the PlanetScale CLI. This guide will use the CLI, but you can follow the database setup instructions in the [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide) if you prefer the dashboard. **Step 1** Authenticate the CLI with the following command: ```bash theme={null} pscale auth login ``` **Step 2** Create a new database with a default `main` branch with the following command: ```bash theme={null} pscale database create --region ``` For `DATABASE_NAME`, you can use any name with lowercase, alphanumeric characters, or underscores. You can also use dashes, but we don't recommend them, as they may need to be escaped in some instances. For `REGION_SLUG`, choose a region closest to you from the [available regions](/docs/vitess/regions#available-regions) or leave it blank. That's it! Your database is ready to use. Next, let's connect it to the Go application and then add some data. ## Connect to the Go app There are **two ways to connect** your Go app to PlanetScale: * With an auto-generated username and password * Using the PlanetScale proxy with the CLI Both options are covered below. ### Option 1: Connect with username and password (Recommended) **Step 1** Create a username and password with the PlanetScale CLI by running: ```bash theme={null} pscale password create ``` A default branch, `main`, is created when you create the database, so you can use that for `BRANCH_NAME`. > [!NOTE] > > The `PASSWORD_NAME` value represents the name of the username and password being generated. You can have multiple > credentials for a branch, so this gives you a way to categorize them. To manage your passwords in the dashboard, go to > your database dashboard page, click "Settings", and then click "Passwords". Take note of the values returned to you, as you won't be able to see this password again. **Step 2** Open the `.env` file in your Go app and update `DSN` as follows: ```bash theme={null} DSN=":@tcp()/?tls=true&interpolateParams=true" ``` Fill in `USERNAME`, `PASSWORD`, `ACCESS HOST URL`, and `DATABASE_NAME` with the appropriate values from the CLI output above. Do not remove the parentheses around the access host URL. You can also get these exact values to copy/paste from your PlanetScale dashboard. In the dashboard, click on the database > "**Connect**" > "**Connect with**" language dropdown > "**Go**". ### Option 2: Connect with the PlanetScale proxy To connect with the PlanetScale proxy, you need the [PlanetScale CLI](https://github.com/planetscale/cli). **Step 1** Open a connection by running the following: ```bash theme={null} pscale connect ``` If you're following this guide exactly and haven't created any branches, you can use the default branch, `main`. **Step 2** A secure connection to your database will be established, and you'll see a local address you can use to connect to your application. **Step 3** Open the `.env` file in your Go app and update it as follows: ```bash theme={null} DSN="mysql://root@tcp(127.0.0.1:)/?interpolateParams=true" ``` The connection uses port `3306` by default, but if that's being used, it will pick a random port. Make sure you paste in whatever port is returned in the terminal. Fill in the database name as well. ## Run migrations and seeder Now that you're connected let's add some data to see it in action. The sample application has an endpoint that you can use to run migrations to create your `categories` and `products` tables. It will seed your database with sample product and category data. You can find this in `main.go`. Let's run those now. **Step 1** First, start your Go app with: ```bash theme={null} go run . ``` **Step 2** Next, navigate to [`localhost:8080/seed`](http://localhost:8080/seed) to run the migrations and the seeder. **Step 3** You can now see the products and categories: * Get all products — [`localhost:8080/products`](http://localhost:8080/products) * Get all categories — [`localhost:8080/categories`](http://localhost:8080/categories) * Get a single product — [`localhost:8080/product/{id}`](http://localhost:8080/products/1) * Get a single category — [`localhost:8080/category/{id}`](http://localhost:8080/categories/1) ### Foreign key constraints If you're using GORM in your Go application and [do not want to use foreign key constraints](/docs/vitess/operating-without-foreign-key-constraints), you can turn them off with this line in the `main.go` file of the Go starter application: ```go theme={null} // ... DisableForeignKeyConstraintWhenMigrating: true, // ... ``` If you prefer to use foreign key constraints in your Go application, you can skip the previous step. However, you need to first enable [foreign key constraint](/docs/vitess/foreign-key-constraints) support in your database settings page. ## Add data manually If you want to continue to play around with adding data on the fly, you have a few options: * PlanetScale CLI shell * PlanetScale dashboard console * Your favorite MySQL client (for a list of tested MySQL clients, review our article on [how to connect MySQL GUI applications](/docs/vitess/tutorials/connect-mysql-gui)) The first two options are covered below. ### Add data with PlanetScale CLI You can use the PlanetScale CLI to open a MySQL shell to interact with your database. You may need to install the MySQL command line client if you haven't already. **Step 1** Run the following command in your terminal: ```bash theme={null} pscale shell ``` This will open up a MySQL shell connected to the specified database and branch. > [!NOTE] > > A branch, `main`, was automatically created when you created your database, so you can use that for `BRANCH_NAME`. **Step 2** Add a record to the `products` table: ```sql theme={null} INSERT INTO `products` (name, description, image, category_id) VALUES ('Spaceship', 'Get ready for the trip of a lifetime', 'https://via.placeholder.com/300.png', 2); ``` The value `id` will be filled with a default value. **Step 3** You can verify it was added in the PlanetScale CLI MySQL shell with: ```sql theme={null} SELECT * FROM products; ``` **Step 4** Type `exit` to exit the shell. You can now navigated the [Go products page](http://localhost:8080/products) to see the new record. ### Add data with PlanetScale dashboard console If you don't care to install MySQL client or the PlanetScale CLI, another quick option is using the MySQL console built into the PlanetScale dashboard. ![PlanetScale console insert and select example](https://mintcdn.com/planetscale-2/Lta43VIYjNTnQ47e/images/assets/docs/tutorials/connect-go-gorm-app/console-2.png?fit=max\&auto=format\&n=Lta43VIYjNTnQ47e\&q=85\&s=660a882657ddb3d00d20aa7f7291a43f) **Step 1** Go to your [PlanetScale dashboard](https://app.planetscale.com) and select your Go database. **Step 2** Click on "**Console**". **Step 3** Select the `main` branch and click "**Connect**". **Step 4** Add a new record to the `product` table with: ```sql theme={null} INSERT INTO `products` (name, description, image, category_id) VALUES ('Spaceship', 'Get ready for the trip of a lifetime', 'https://via.placeholder.com/300.png', 2); ``` **Step 5** You can confirm that it was added by running: ```sql theme={null} SELECT * FROM products; ``` You can now refresh the [Go products page](http://localhost:8080/products) to see the new record. ## What's next? Once you're done with initial development, you can enable [safe migrations](/docs/vitess/schema-changes/safe-migrations) on your `main` production branch to protect it against direct schema changes and enable zero-downtime schema migrations. When you're reading to make more schema changes, you'll [create a new branch](/docs/vitess/schema-changes/branching) off of your production branch. Branching your database creates an isolated copy of your production schema so that you can easily test schema changes in development. Once you're happy with the changes, you'll open a [deploy request](/docs/vitess/schema-changes/deploy-requests). This will generate a diff showing the changes that will be deployed, making it easy for your team to review. Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Laravel application to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-laravel-app In this tutorial, you'll learn how to connect a Laravel 12 application to a PlanetScale MySQL database using a sample Laravel starter app. ## Prerequisites * [PHP](https://www.php.net/manual/en/install.php) — This tutorial uses `v8.2` * [Composer](https://getcomposer.org/) * A [PlanetScale account](https://auth.planetscale.com/sign-up) ## Set up the Laravel app This guide will integrate [a simple Laravel 12 app](https://github.com/planetscale/planetscale-laravel-mysql) with PlanetScale. The application displays a list of users from your PlanetScale database. The sample repo contains migrations and seed data to create and populate the `users` table. If you have an existing application, you can also use that. **Step 1** Clone the starter Laravel application: ```bash theme={null} git clone https://github.com/planetscale/planetscale-laravel-mysql.git ``` **Step 2** Enter into the folder and install the dependencies: ```bash theme={null} cd planetscale-laravel-mysql composer install ``` You may need to run `composer update` if you haven't updated in a while. **Step 3** Copy the `.env.example` file into `.env` and generate the app key: ```bash theme={null} cp .env.example .env php artisan key:generate ``` ## Set up the database Next, you need to set up your PlanetScale database and connect to it in the Laravel application. > [!NOTE] > > If you have an existing cloud-hosted MySQL database, you can import it into PlanetScale instead. See the [Database imports documentation](/docs/vitess/imports/database-imports). If this is your first time in the dashboard, you'll be prompted to create an organization and go through the database creation walkthrough. Otherwise, click "**New database**" > "**Create new database**". * **Name** — You can use any name with lowercase, alphanumeric characters, or underscores. We also permit dashes, but don't recommend them, as they may need to be escaped in some instances. * **Region** — Choose the [region](/docs/vitess/regions#available-regions) closest to you or your application. It's important to note if you intend to make this branch a production branch, you will not be able to change the region later, so choose the region with this in mind. * **Storage option** — Choose a storage option. You can choose between network-attached storage or [Metal](/docs/metal) for storage. For more information, see the [plans documentation](/docs/vitess/tutorials/connect-laravel-app). * **Cluster size** — Select the [desired cluster size](/docs/planetscale-plans) for your database. Finally, click "**Create database**". A [production branch](/docs/vitess/schema-changes/branching), `main`, is automatically created when you create your database. [Safe migrations](/docs/vitess/schema-changes/safe-migrations) are turned off by default, so you can make schema changes directly to this branch. Once you're ready for production, you can turn on safe migrations to protect from accidental schema changes and enable zero-downtime deployments. That's it! Your database is ready to use. Next, let's connect it to the Laravel application and then add some data. ## Connect to the Laravel app There are **two ways to connect** to PlanetScale: * With an auto-generated username and password * Using the PlanetScale proxy with the CLI Both options are covered below. ### Option 1: Connect with username and password (Recommended) First, you need to generate a database username and password so that you can use it to connect to your application. You'll be presented with this option after creating your database. You can also access the password creation page by clicking "**Connect**" -> "**Create password**". As long as you're an organization administrator, this will generate a username and password that has administrator privileges to the database. > [!TIP] > > If the password value is blurred, you need to click "**New password**" to generate a new one. Click "Laravel" as the framework, then copy the contents of the `.env` tab and paste them into your own `.env` file in your Laravel application. The structure will look like this: ```bash theme={null} DB_CONNECTION=mysql DB_HOST= DB_PORT=3306 DB_DATABASE= DB_USERNAME= DB_PASSWORD= MYSQL_ATTR_SSL_CA=/etc/ssl/cert.pem ``` For `DB_DATABASE`, you can use your PlanetScale database name directly if you have a *single unsharded keyspace*. If you have a sharded keyspace, you'll need to use `@primary`. This will automatically direct incoming queries to the correct keyspace/shard. For more information, see the [Targeting the correct keyspace documentation](/docs/vitess/sharding/targeting-correct-keyspace). The `MYSQL_ATTR_SSL_CA` value is platform-dependent. Please refer to our documentation around [how to connect to PlanetScale securely](/docs/vitess/connecting/secure-connections#ca-root-configuration) for the platform you're using. ### Option 2: Connect with the PlanetScale proxy To connect with the PlanetScale proxy, you need to install and use the [PlanetScale CLI](https://github.com/planetscale/cli). **Step 1** Open a connection by running the following: ```bash theme={null} pscale connect ``` If you're following this guide exactly and haven't created any branches, you can use the default branch, `main`. **Step 2** A secure connection to your database will be established and you'll see a local address you can use to connect to your application. **Step 3** Open the `.env` file in your Laravel app and update it as follows: ```bash theme={null} DB_CONNECTION=mysql DB_HOST=127.0.0.1 DB_PORT=3306 # Get this from the output of the previous step DB_DATABASE= DB_USERNAME= DB_PASSWORD= ``` The connection uses port `3306` by default, but if that's being used, it will pick a random port. Make sure you paste in whatever port is returned in the terminal. You can leave `DB_USERNAME` and `DB_PASSWORD` blank. ## Run migrations and seeder Now that you're connected, let's add some data to see it in action. The sample application comes with some default Laravel migration files, `database/migrations/`, to create the database schema. It also contains a user seeder to seed some mock user data. > [!NOTE] > > Laravel uses foreign key constraints by default. PlanetScale, however, has foreign key constraint support turned off by default. For this tutorial, we're keeping the Laravel defaults, so you need to enable [foreign key constraint](/docs/vitess/foreign-key-constraints) support in your database settings page. Click the checkbox next to "Allow foreign key constraints" and press "Save database settings". Let's migrate and seed the database now. **Step 1** In the root of the Laravel project, run the following to migrate and seed the database: ```bash theme={null} php artisan migrate --seed ``` **Step 2** Start the application: ```bash theme={null} php artisan serve ``` You can view the application at [http://localhost:8000](http://localhost:8000). 1. Refresh your Laravel homepage and you'll see a list of users. ![Laravel PlanetScale starter app homepage](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/laravel-users.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=a07e91e1736940969c60a8c872345e61) ## Add data manually If you want to continue to play around with adding data on the fly, you have a few options: * PlanetScale [dashboard console](/docs/vitess/web-console) * [Laravel Tinker](hhttps://laravel.com/12.x/artisan#tinker) * [PlanetScale CLI shell](/docs/cli/shell) * Your favorite MySQL client (for a list of tested MySQL clients, review our article on [how to connect MySQL GUI applications](/docs/vitess/tutorials/connect-mysql-gui)) The first option is covered below. ### Add data in PlanetScale dashboard console PlanetScale has a [built-in console](/docs/vitess/web-console) where you can run MySQL commands against your branches. By default, web console access to production branches is disabled to prevent accidental deletion. From your database's dashboard page, click on the "**Settings**" tab, check the box labelled "**Allow web console access to production branches**", and click "**Save database settings**". To access it, click "**Console**" > select your branch > "**Connect**". From here, you can run MySQL queries and DDL against your database branch. **Step 1** Add a record to the `users` table: ```sql theme={null} UPDATE users SET email = 'cyrus@planetscale.com' WHERE id=1; ``` **Step 2** Refresh the Laravel homepage to see the new record. You can also verify it was added in the console with: ```sql theme={null} SELECT * FROM users; ``` ![PlanetScale web console](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/laravel-web-console.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=a05304025d02458f95ab6eddf0be69ac) ## What's next? Once you're done with initial development, you can enable [safe migrations](/docs/vitess/schema-changes/safe-migrations) to protect from accidental schema changes and enable zero-downtime deployments. To learn more about PlanetScale, take a look at the following resources: * [PlanetScale workflow](/docs/vitess/best-practices) — Quick overview of the PlanetScale workflow: branching, non-blocking schema changes, deploy requests, and reverting a schema change. * [PlanetScale branching](/docs/vitess/schema-changes/branching) — Learn how to utilize branching to ship schema changes with no locking or downtime. * [PlanetScale CLI](/docs/cli) — Power up your workflow with the PlanetScale CLI. Every single action you just performed in this quickstart (and much more) can also be done with the CLI. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a MySQL GUI to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-mysql-gui ## Introduction In this tutorial, you'll learn how to connect to a PlanetScale database using a MySQL GUI. While this tutorial uses Sequel Ace as a demonstration, many applications that connect to MySQL databases will support connecting to and querying a PlanetScale database as long as the applicaton supports connecting over SSL. ## Gather the credentials To connect to a PlanetScale database, you'll need four pieces of information: * The database name * Host name * Username * Password The easiest way to gather this information is by selecting the **Connect** button from the **Dashboard** tab. Then, on the **Connect** page, select the branch that you wish to connect to and click the **Create password** button. Within the **Select a language or framework** section, select "Other" to display the connection details as a list instead of a language or framework-specific connection string. > [!NOTE] > > As a security best practice, passwords are only displayed when they are created. ## Connect to the database In the application you are using, enter the access information you gathered in the previous step into the appropriate fields. Make sure to check **"Require SSL"** as SSL is required to connect to a PlanetScale database. Click **"Connect"** once you are finished. ![The new connection window in Sequel Ace.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/connect-mysql-gui/ace-connect.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=917a6f3faaaa65eca280742f4db0e73e) If the connection is successful, you should be able to query your database and perform other [supported operations](/docs/vitess/troubleshooting/mysql-compatibility). ![A sample query in Sequel Ace.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/connect-mysql-gui/ace-query.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=e7b77e5ea7ca4d1b2f825004497b3910) ## Caveats While many standard MySQL statements are supported, there are a few caveats worth calling out: **Step 1** Each branch of a PlanetScale database is considered an isolated MySQL database. You'll need separate connection details per branch. **Step 2** Production branches with [safe migrations](/docs/vitess/schema-changes/safe-migrations) enforce the use of [branching](/docs/vitess/schema-changes/branching) and [deploy requests](/docs/vitess/schema-changes/deploy-requests) to safely make schema changes and do not support direct DDL as a result. However, DDL is supported on development branches and production branches without safe migrations enabled (not recommended). **Step 3** Creating new databases is not supported using any GUI tool. ## Tested GUIs The following MySQL GUI applications have been tested and confirmed to work with PlanetScale databases: > **[Sequel Ace](https://sequel-ace.com/)** > **[TablePlus](https://tableplus.com/)** > **[MySQL Workbench](https://www.mysql.com/products/workbench/)** > **[JetBrains DataGrip](https://www.jetbrains.com/datagrip/)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Next.js application to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-nextjs-app In this tutorial, you'll create a [Next.js](https://nextjs.org/) application that uses [Tailwind CSS](https://tailwindcss.com/) for styling and [Prisma](https://www.prisma.io/) to connect to a [PlanetScale](/docs/) database. ## Prerequisites * [Node.js](https://nodejs.org/en/download/) * A [PlanetScale account](https://auth.planetscale.com/sign-up) ## Set up the database If this is your first time in the dashboard, you'll be prompted to go through a database creation walkthrough where you'll create a new database. Otherwise, click "**New database**" > "**Create new database**". * **Name** — You can use any name with lowercase, alphanumeric characters, or underscores. We also permit dashes, but don't recommend them, as they may need to be escaped in some instances. * **Plan type** — Select the [desired plan](/docs/planetscale-plans) for your database. * **Region** — Choose the [region](/docs/vitess/regions#available-regions) closest to you or your application. It's important to note if you intend to make this branch a production branch, you will not be able to change the region later, so choose the region with this in mind. Finally, click "**Create database**". > [!NOTE] > > If you have an existing cloud-hosted MySQL database, you can import it into PlanetScale instead. See the [Database imports documentation](/docs/vitess/imports/database-imports). A [production branch](/docs/vitess/schema-changes/branching), `main`, is automatically created when you create your database. Production branches are highly available, protected database that you can connect your production application to. Once you are satisfied with your initial development, you may enable [safe migrations](/docs/vitess/schema-changes/safe-migrations) to enable zero-downtime migrations and protect the branch from accidental data deletion. ## Set up the starter Next.js app Now that you have your database, clone the [Next.js starter repository](https://github.com/planetscale/nextjs-starter), or grab your own project. ```sh theme={null} git clone https://github.com/planetscale/nextjs-starter ``` Install the dependencies with: ```sh theme={null} cd nextjs-starter npm install ``` Create your `.env` file by renaming the `.env.example` file to `.env`: ```sh theme={null} mv .env.example .env ``` ## Generate a connection string Next, you need to generate a database username and password so that you can use it to connect to your application. In your PlanetScale dashboard, select your database, click "**Connect**", and select "**Prisma**" from the "**Connect with**" dropdown. As long as you're an organization administrator, this will generate a username and password that has administrator privileges to the database. Copy the `DATABASE_URL` string from the `.env` tab and paste it into your own `.env` file. The structure will look like this: ```sh theme={null} DATABASE_URL='mysql://:@/?sslaccept=strict' ``` For `DATABASE_NAME`, you can use your PlanetScale database name directly if you have a *single unsharded keyspace*. If you have a sharded keyspace, you'll need to use `@primary`. This will automatically direct incoming queries to the correct keyspace/shard. For more information, see the [Targeting the correct keyspace documentation](/docs/vitess/sharding/targeting-correct-keyspace). Your PlanetScale database should now be connected to your application. ## Add the schema and data With the database connected, you're now ready to add a schema, and read/write data. The sample repo we shared above includes a pre-built schema and some seed data built with Prisma. Push the database schema to your PlanetScale database with: ```sh theme={null} npx prisma db push ``` Run the seed script. This will populate your database with `Product` and `Category` data: ```sh theme={null} npm run seed ``` ## Run the app Run the app with following command: ```sh theme={null} npm run dev ``` Open your browser at [localhost:3000](http://localhost:3000) to see the running application. ## Deploy your app After you have your application running locally, you may want to deploy it to production. Your database branch (`main` by default) is already a production database branch. You should also enable [safe migrations](/docs/vitess/schema-changes/safe-migrations), which protects your production branch from accidental schema changes. This can be done from the PlanetScale dashboard by clicking the same **"cog"** once the branch has been promoted to production. In the modal that will appear, toggle the option labeled **"Enable safe migrations"**, then click the **"Enable safe migrations"**. This will close the modal and save the setting. ![Enable safe migrations](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/connect-nextjs-app/prod-branch-options-modal.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=68804dcbb922b0c9653d0e19de7651c7) ### Deploy to Vercel If you'd like to deploy to Vercel, check out our [Deploy to Vercel documentation](/docs/vitess/tutorials/deploy-to-vercel). ### Deploy to Netlify If you'd like to deploy to Netlify, check out our [Deploy to Netlify documentation](/docs/vitess/tutorials/deploy-to-netlify). > [!NOTE] > > If you are deploying the `nextjs-starter` repo, the `Netlify.toml` file in this repository includes the configuration for you to customize the `PLANETSCALE_PRISMA_DATABASE_URL` property on the initial deploy. ## What's next? To learn more about PlanetScale, take a look at the following resources: * [PlanetScale workflow](/docs/vitess/best-practices) — Quick overview of the PlanetScale workflow: branching, non-blocking schema changes, deploy requests, and reverting a schema change. * [PlanetScale branching](/docs/vitess/schema-changes/branching) — Learn how to utilize branching to ship schema changes with no locking or downtime. * [PlanetScale CLI](/docs/cli) — Power up your workflow with the PlanetScale CLI. Every single action you just performed in this quickstart (and much more) can also be done with the CLI. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Node.js application to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-nodejs-app In this tutorial, you'll create a simple Node.js and Express.js application and connect it to a PlanetScale database. > [!TIP] > > Already have a Node.js application and just want to connect to PlanetScale? Check out the [Node.js quick connect repo](https://github.com/planetscale/connection-examples/tree/main/nodejs). ## Prerequisites > **[Node.js](https://nodejs.org/en/download/)** ## Set up the database First, create a new database with the following command: ```bash theme={null} pscale database create ``` Next, let's add some data to the database. You'll create a new table called `users` and add one record to it. To do this, use the PlanetScale CLI shell to open a MySQL shell where you can manipulate your database. You may need to [install the MySQL command line client](/docs/cli/planetscale-environment-setup) if you haven't already. ```bash theme={null} pscale shell ``` > [!NOTE] > > A branch, `main`, was automatically created when you created your database, so you can use that for `BRANCH_NAME`. Create the `users` table: ```sql theme={null} CREATE TABLE `users` ( `id` int NOT NULL AUTO_INCREMENT PRIMARY KEY, `email` varchar(255) NOT NULL, `first_name` varchar(255), `last_name` varchar(255) ); ``` Then, add a record to it with: ```sql theme={null} INSERT INTO `users` (email, first_name, last_name) VALUES ('jp@example.com', 'Jean', 'Pixy'); ``` You can verify it was added with: ```sql theme={null} select * from users; ``` ``` +----+----------------+------------+-----------+ | id | email | first_name | last_name | +----+----------------+------------+-----------+ | 1 | jp@example.com | Jean | Pixy | +----+----------------+------------+-----------+ ``` Next, you'll set up the Express starter application. ## Set up the starter Node.js app Clone the starter repository: ```bash theme={null} git clone https://github.com/planetscale/express-example.git ``` Enter into the folder and install the dependencies with: ```bash theme={null} cd express-example npm install ``` Now that your application is set up and the database is ready to be used, let's connect them. ## Connect to PlanetScale with Express.js There are **two ways to connect** to PlanetScale: * With an auto-generated username and password * Using the PlanetScale proxy with the CLI Both options are covered below. ### Option 1: Connect with username and password (Recommended) These instructions show you how to generate a set of credentials with [the PlanetScale CLI](/docs/cli/planetscale-environment-setup). You can also get these exact values to copy/paste from your [PlanetScale dashboard](https://app.planetscale.com). In the dashboard, click on the database > "**Connect**" > "**Connect with**" language dropdown > "**Node.js**". If the password is blurred, click "**New password**". Skip to step 3 once you have these credentials. 1. Authenticate the CLI with the following command: ```bash theme={null} pscale auth login ``` 2. Using the PlanetScale CLI, create a new username and password for the branch of your database: ```bash theme={null} pscale password create ``` > [!NOTE] > > The `PASSWORD_NAME` value represents the name of the username and password being generated. You can have multiple credentials for a branch, so this gives you a way to categorize them. To manage your passwords in the dashboard, go to your database dashboard page, click "Settings", and then click "Passwords". Take note of the values returned to you, as you won't be able to see this password again. ``` Password production-password was successfully created. Please save the values below as they will not be shown again NAME USERNAME ACCESS HOST URL ROLE PASSWORD --------------------- -------------- ----------------------------------- ------------------ ------------------------------------------------------- production-password xxxxxxxxxxxxx xxxxxx.us-east-2.psdb.cloud Can Read & Write pscale_pw_xxxxxxx ``` 3. Next, create your `.env` file by renaming the `.env.example` file to `.env`: ```bash theme={null} mv .env.example .env ``` 4. Use the values from the CLI output in step 1 to construct your connection string that will be used to connect your Node app to your PlanetScale database. Create your connection string in the following format: ``` mysql://:@/?ssl={"rejectUnauthorized":true} ``` 5. In the `.env` file, fill in the `DATABASE_URL` variable with the value you constructed above. It should look something like this: ```bash theme={null} DATABASE_URL=mysql://xxxxxxxxxxxxx:pscale_pw_xxxxxxx@xxxxxx.us-east-2.psdb.cloud/express_database?ssl={"rejectUnauthorized":true} ``` 6. Finally, run your Express application with: ```bash theme={null} node app.js ``` Navigate to [http://localhost:3000](http://localhost:3000) and you'll see the data from your `users` table! ### Option 2: Using the PlanetScale proxy with the CLI Use the following command to create a connection to your database and start the application: ```bash theme={null} pscale connect --execute 'node app.js' ``` > [!NOTE] > > Running `pscale connect` with the execute flag will pass a `DATABASE_URL` to the Node application, enabling it to connect to PlanetScale. Don't forget to look in `app.js` to see how the DATABASE\_URL is used. Navigate to [http://localhost:3000](http://localhost:3000) and you'll see the data from your `users` table! ## What's next? Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. If you're interested in learning how to secure your application when connecting to PlanetScale, please read [Connecting to PlanetScale securely](/docs/vitess/connecting/secure-connections). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a PHP application to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-php-app In this tutorial, you'll learn how to connect a PHP application to a PlanetScale MySQL database with a sample PHP starter app using [MySQLi](https://www.php.net/manual/en/book.mysqli.php). > [!TIP] > > Already have a PHP application and just want to connect to PlanetScale? Check out the [PHP quick connect repo](https://github.com/planetscale/connection-examples/tree/main/php). ## Prerequisites * [PHP](https://www.php.net/manual/en/install.php) — This tutorial uses `v8.1` * [Composer](https://getcomposer.org/) * A [PlanetScale account](https://auth.planetscale.com/sign-up) * [PlanetScale CLI](https://github.com/planetscale/cli) (Optional) — You can also follow this tutorial using just the PlanetScale admin dashboard, but the CLI will make setup quicker. ## Set up the PHP app This guide uses [a simple PHP app](https://github.com/planetscale/php-example) that displays a list of products stored in a PlanetScale database. If you have an existing application, you can also use that. ![PHP sample application homepage priority](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/connect-php-app/example.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=edd8e7396d7181d338fe8c4d2d5393af) **Step 1** Clone the starter PHP application: ```bash theme={null} git clone https://github.com/planetscale/php-example.git ``` **Step 2** Enter into the folder and install the dependencies: ```bash theme={null} cd php-example composer install ``` **Step 3** Rename the `.env.example` file to `.env`: ```bash theme={null} mv .env.example .env ``` **Step 4** Start the application: ```bash theme={null} php -S localhost:8000 ``` You can view the application at [http://localhost:8000](http://localhost:8000). ## Set up the database Next, you need to set up your PlanetScale database and connect it to the PHP application. You can create a database either in the [PlanetScale dashboard](https://app.planetscale.com) or from the PlanetScale CLI. This guide will use the CLI, but you can follow the database setup instructions in the [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide#create-a-database) if you prefer the dashboard. Just create the database and then come back here to continue. **Step 1** Install the [PlanetScale CLI](/docs/cli/planetscale-environment-setup). **Step 2** Authenticate in the CLI with the following command: ```bash theme={null} pscale auth login ``` **Step 3** Create a new database with the following command: ```bash theme={null} pscale database create --region ``` You can use any name with lowercase, alphanumeric characters, or underscores. You can also use dashes, but we don't recommend them, as they may need to be escaped in some instances. For `REGION_SLUG`, choose a region closest to you from the [available regions](/docs/vitess/regions#available-regions) or leave it blank. Your database is created with a default branch, `main`, which is meant to serve as your production database branch. That's it! Your database is ready to use. Next, let's connect it to the PHP application and then add some data. ## Connect to the PHP app There are **two ways to connect** to PlanetScale: * With an auto-generated username and password * Using the PlanetScale proxy with the CLI Both options are covered below. The environment variables you fill in next will be used in the [`db.php` file of the sample application](https://github.com/planetscale/php-example/blob/main/db.php): ```php theme={null} ssl_set(NULL, NULL, $ssl, NULL, NULL); $mysqli->real_connect($hostname, $username, $password, $dbName, $port); if ($mysqli->connect_error) { echo 'not connected to the database'; } else { echo "Connected successfully"; } ``` For `dbName`, you can use your PlanetScale database name directly if you have a *single unsharded keyspace*. If you have a sharded keyspace, you'll need to use `@primary`. This will automatically direct incoming queries to the correct keyspace/shard. For more information, see the [Targeting the correct keyspace documentation](/docs/vitess/sharding/targeting-correct-keyspace). ### Option 1: Connect with username and password (Recommended) If you're not using the CLI, you can get the exact values to copy/paste from your PlanetScale dashboard. In the dashboard, select the branch you want to connect to from the infrastructure card (we're using `main`), click "**Connect**", and select "**PHP**" from the language dropdown. Copy these credentials, and then skip to step 2 to fill them in. **Step 1** Create a username and password with the PlanetScale CLI by running: ```bash theme={null} pscale password create ``` A default branch, `main`, was created when you created the database, so you can use that for `BRANCH_NAME`. > [!NOTE] > > The `PASSWORD_NAME` value represents the name of the username and password being generated. You can have multiple > credentials for a branch, so this gives you a way to categorize them. To manage your passwords in the dashboard, go to > your database dashboard page, click "Settings", and then click "Passwords". Take note of the values returned to you, as you won't be able to see this password again. **Step 2** Open the `.env` file in your PHP app: ```bash theme={null} HOST= DATABASE= USERNAME= PASSWORD= MYSQL_ATTR_SSL_CA= ``` Fill in your database name. For `USERNAME`, `PASSWORD`, and `HOST`, use the corresponding values from the CLI output. **Step 3** For `MYSQL_ATTR_SSL_CA`, use our [CA root configuration doc](/docs/vitess/connecting/secure-connections#ca-root-configuration) to find the correct value for your system. For example, if you're on MacOS, it would be: ```bash theme={null} MYSQL_ATTR_SSL_CA=/etc/ssl/cert.pem ``` **Step 4** Refresh your PHP homepage, and you should see the message that you're connected to your database! ### Option 2: Connect with the PlanetScale proxy We recommend connecting with a username and password, but you can also open a quick connection with the PlanetScale proxy. You'll need the [PlanetScale CLI](https://github.com/planetscale/cli) for this option. **Step 1** Open a connection by running the following: ```bash theme={null} pscale connect ``` If you're following this guide exactly and haven't created any branches, you can use the default branch, `main`. **Step 2** A secure connection to your database will be established, and you'll see a local address you can use to connect to your application. **Step 3** Open the `.env` file in your PHP app and update it as follows: ```bash theme={null} HOST=127.0.0.1 PORT=3306 # Get this from the output of the previous step DATABASE= ``` The connection uses port `3306` by default, but we'll assign a random port if `3306` is in use. Make sure you paste in whatever port is returned in the terminal. Fill in the database name as well. **Step 4** Open `db.php` and replace it with the following: ```php expandable theme={null} ssl_set(NULL, NULL, $ssl, NULL, NULL); $mysqli->real_connect($hostname, '', '', $dbName, $port); if ($mysqli->connect_error) { echo 'not connected to the database'; } else { echo "Connected successfully"; } ``` This removes all references to `username`, `password`, and `ssl`. > [!NOTE] > > It's important to make sure that you add the SSL check back if you switch back to username and password credentials. > We're intentionally commenting it out instead of deleting it in case you switch back. **Step 5** Refresh your PHP homepage, and you should see the message that you're connected to your database! ## Add the schema and data Now that you're connected to the database let's create the `products` and `categories` tables and add some data. There are a few ways to do this: * PlanetScale CLI shell * PlanetScale dashboard console * Your favorite MySQL client (for a list of tested MySQL clients, review our article on [how to connect MySQL GUI applications](/docs/vitess/tutorials/connect-mysql-gui)) The first two options are covered below. ### Option 1: Add data with PlanetScale dashboard console If you don't care to install the MySQL client or the PlanetScale CLI, another quick option is using the MySQL console built into the PlanetScale dashboard. **Step 1** Go to your [PlanetScale dashboard](https://app.planetscale.com) and select your PHP database. **Step 2** On the "**Dashboard**" page, you will need to select the gear icon and demote your `main` branch by toggling the "Promote to production" option. This is so you can create tables directly on your `main` branch. **Step 3** Click on the "**Console**" and select the `main` branch (or whatever development branch you used). **Step 4** Create the `categories` table: ```sql theme={null} CREATE TABLE categories ( id INT AUTO_INCREMENT NOT NULL, name VARCHAR(255) NOT NULL, description VARCHAR(255) NOT NULL, PRIMARY KEY(id)) DEFAULT CHARACTER SET utf8mb4 COLLATE `utf8mb4_unicode_ci` ENGINE = InnoDB; ``` **Step 5** Create the `products` table: ```sql theme={null} CREATE TABLE products ( id INT AUTO_INCREMENT NOT NULL, name VARCHAR(255) NOT NULL, description VARCHAR(255) NOT NULL, image VARCHAR(255) NOT NULL, category_id INT NOT NULL, PRIMARY KEY(id)) DEFAULT CHARACTER SET utf8mb4 COLLATE `utf8mb4_unicode_ci` ENGINE = InnoDB; ``` > [!NOTE] > > If you are using foreign key constraints, you must first enable [foreign key constraints](/docs/vitess/foreign-key-constraints) support in your database settings page. **Step 6** Add data to the `products` table with: ```sql theme={null} INSERT INTO `products` (name, description, image, category_id) VALUES ('Shoes', 'Description for Shoes', 'https://via.placeholder.com/150.png', '1'), ('Hat', 'Description for Hats', 'https://via.placeholder.com/150.png', '1'), ('Bicycle', 'Description for Bicycle', 'https://via.placeholder.com/150.png', '4'); ``` **Step 7** Add data to the `categories` table with: ```sql theme={null} INSERT INTO `categories` (name, description) VALUES ('Clothing', 'Description for Clothing'), ('Electronics', 'Description for Electronics'), ('Appliances', 'Description for Appliances'), ('Health', 'Description for Health'); ``` **Step 8** You can confirm that it was added by running: ```sql theme={null} SELECT * FROM products; SELECT * FROM categories; ``` You can now refresh the [PHP homepage](http://localhost:8000) to see the new record. ### Option 2: Add data with PlanetScale CLI You can use the PlanetScale CLI to open a MySQL shell to interact with your database. You may need to [install the MySQL command line client](/docs/cli/planetscale-environment-setup) if you haven't already. **Step 1** Run the following command in your terminal: ```bash theme={null} pscale shell ``` This will open up a MySQL shell connected to the specified database and branch. > [!NOTE] > > A branch, `main`, was automatically created when you created your database, so you can use that for `BRANCH_NAME`. **Step 2** Create the `categories` table: ```sql theme={null} CREATE TABLE categories ( id INT AUTO_INCREMENT NOT NULL, name VARCHAR(255) NOT NULL, description VARCHAR(255) NOT NULL ); ``` **Step 3** Create the `products` table: ```sql theme={null} CREATE TABLE products ( id int NOT NULL AUTO_INCREMENT PRIMARY KEY, name VARCHAR(255) NOT NULL, description VARCHAR(255) NOT NULL, image VARCHAR(255) NOT NULL, category_id INT NOT NULL, KEY category_id_idx (category_id) ); ``` **Step 4** Add some records to the `products` table: ```sql theme={null} INSERT INTO `products` (name, description, image, category_id) VALUES ('Shoes', 'Description for Shoes', 'https://via.placeholder.com/150.png', '1'), ('Hat', 'Description for Hats', 'https://via.placeholder.com/150.png', '1'), ('Bicycle', 'Description for Bicycle', 'https://via.placeholder.com/150.png', '4'); ``` The value `id` will be filled with a default value. **Step 5** Add some data to the `categories` table: ```sql theme={null} INSERT INTO `categories` (name, description) VALUES ('Clothing', 'Description for Clothing'), ('Electronics', 'Description for Electronics'), ('Appliances', 'Description for Appliances'), ('Health', 'Description for Health'); ``` **Step 6** You can verify everything was added in the PlanetScale CLI MySQL shell with: ```sql theme={null} SELECT * FROM products; SELECT * FROM categories; ``` **Step 7** Type `exit` to exit the shell. You can now refresh the [PHP homepage](http://localhost:8000) to see the new records. ## What's next? Once you're done with initial development, you can promote your branch to production and enable [safe migrations](/docs/vitess/schema-changes/safe-migrations) on your `main` production branch to protect it against direct schema changes and enable zero-downtime schema migraions. When you're reading to make more schema changes, you'll [create a new branch](/docs/vitess/schema-changes/branching) off of your production branch. Branching your database creates an isolated copy of your production schema so that you can easily test schema changes in development. Once you're happy with the changes, you'll [open a deploy request](/docs/vitess/schema-changes/deploy-requests). This will generate a diff showing the changes that will be deployed, making it easy for your team to review. Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Rails application to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-rails-app ## Introduction In this tutorial, you’re going to create a simple Rails application named *blog* and connect it to a PlanetScale database. You’ll perform the initial migration from your local Rails application, and set up the database for future development. > [!TIP] > > Already have a Rails application and just want to connect to PlanetScale? Check out the [Rails quick connect repo](https://github.com/planetscale/connection-examples/tree/main/ruby). ## Prerequisites * Install [Ruby and the Rails gem](https://guides.rubyonrails.org/getting_started.html#creating-a-new-rails-project-installing-rails). * Install the [PlanetScale CLI](https://github.com/planetscale/cli). * Authenticate the CLI with the following command: ```bash theme={null} pscale auth login ``` ## Create a Rails project To connect a Rails application to a PlanetScale database, you'll first create a sample Rails project named *blog* and install the libraries needed to connect to your PlanetScale database. Open the command line and follow these steps: **Step 1** Create a Rails app named *blog* by running the following command: ```bash theme={null} rails new blog ``` **Step 2** Change into the directory you just created, the `blog` Rails app: ```bash theme={null} cd blog ``` **Step 3** Next, add the `trilogy` gem to your *Gemfile*: ```ruby theme={null} gem "trilogy" ``` **Step 4** Then run `bundle install` At this point, you have accomplished two things: you've created a Rails project called *blog* and installed the libraries that you'll need to connect to your PlanetScale database. Now, it’s time to create a PlanetScale database. ## Create a PlanetScale database and password Now you'll need to create credentials for your Rails application to use. ### Using the CLI to create a connection string **Step 1** Using the `pscale` CLI, create a new database also named *blog*: ```bash theme={null} pscale database create blog ``` **Step 2** Using the `pscale` CLI, create a new database password for the `main` branch of your database named *blog*: ```bash theme={null} pscale password create blog main ``` > [!NOTE] > > The `PASSWORD_NAME` value represents the name of the username and password being generated. You can have multiple credentials for a branch, so this gives you a way to categorize them. To manage your passwords in the dashboard, go to your database dashboard page, click "Settings", and then click "Passwords". **Step 3** Take note of the values returned to you, as they will not be shown again. ``` NAME BRANCH USERNAME ACCESS HOST URL ROLE ROLE DESCRIPTION PASSWORD --------------------- -------- -------------- ----------------------------------- -------- ------------------ ------------------------------------------------------- development-password main xxxxxxxx xxxxxxxxxx.us-east-3.psdb.cloud writer Can Read & Write pscale_pw_xxxxxxxxxxxxxxxxxxxxx ``` > [!NOTE] > > You can also create passwords in the PlanetScale dashboard, as documented [in our Creating a password documentation](/docs/vitess/connecting/connection-strings#creating-a-password). ## Configure Rails and PlanetScale Let's set up the Rails application to talk to the new database. Open `config/database.yml` and configure the `development` database settings with your new credentials from the output in the previous step: ```yaml theme={null} development: <<: *default adapter: trilogy database: blog username: host: password: ssl_mode: <%= Trilogy::SSL_VERIFY_IDENTITY %> ``` For `database` (database name), you can use your PlanetScale database name directly if you have a *single unsharded keyspace*. If you have a sharded keyspace, you'll need to use `@primary`. This will automatically direct incoming queries to the correct keyspace/shard. For more information, see the [Targeting the correct keyspace documentation](/docs/vitess/sharding/targeting-correct-keyspace). The correct `sslca` path depends on your operating system and distribution. See [CA root configuration](/docs/vitess/connecting/secure-connections#ca-root-configuration) for more information. > [!NOTE] > > You're configuring the **development** Rails environment here for the sake of expedience. In actual use, the **main** database branch would typically serve the **production** environment. Because this is a Rails app, you can also enable [Automatic Rails migrations](/docs/vitess/tutorials/automatic-rails-migrations) from the database's settings page. Select your database, click on the `main` branch, click "**Settings**", check the "**Automatically copy migration data**" box, and select "**Rails**" from the dropdown. ## Migrate your database Here comes the fun stuff! Now that your application is configured to talk to PlanetScale, you can create your first migration. **Step 1** Create a Rails migration and call it `CreateUsers`: ```bash theme={null} rails generate migration CreateUsers ``` This rails command begins the migration for your table that is currently empty and generates a Ruby file that’ll be named something like this: `db/migrate/20211014210422_create_users.rb` Fill in the body of this skeleton file with a few more relevant details, such as a user's **name** and **email**. ```ruby theme={null} class CreateUsers < ActiveRecord::Migration[6.1] def change create_table :users do |t| t.string :name t.string :email t.timestamps end end end ``` **Step 2** Run your migration: ```bash theme={null} bin/rails db:migrate ``` **Step 3** Now, give it a whirl to make sure you can query the new table with the `pscale` CLI: ```bash theme={null} pscale shell blog main ``` ```sql expandable theme={null} blog/main> show tables; +----------------------+ | Tables_in_blog | +----------------------+ | ar_internal_metadata | | schema_migrations | | users | +----------------------+ blog/main> describe users; +------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +------------+--------------+------+-----+---------+----------------+ | id | bigint | NO | PRI | NULL | auto_increment | | name | varchar(255) | YES | | NULL | | | email | varchar(255) | YES | | NULL | | | created_at | datetime(6) | NO | | NULL | | | updated_at | datetime(6) | NO | | NULL | | +------------+--------------+------+-----+---------+----------------+ ``` ## Enable safe migrations [Safe migrations](/docs/vitess/schema-changes/safe-migrations) is an optional but highly recommended feature for branches on PlanetScale. With safe migrations enabled, direct schema changes (`CREATE`, `ALTER`, and `DELETE`) are not allowed on production branches to prevent accidental data loss and must be applied via [deploy requests](/docs/vitess/best-practices). ```bash theme={null} pscale branch safe-migrations enable blog main ``` Congratulations! You're ready to develop your Rails application against PlanetScale. ## Summary In this tutorial, you created a simple Rails application named *blog* and connected it to a PlanetScale database. ## Further reading If you're interested in learning how to secure your application's connection to PlanetScale, please read [Connecting to PlanetScale securely](/docs/vitess/connecting/secure-connections). ## What's next? Now that you've successfully connected your Rails app to PlanetScale, it's time to make more schema changes to your tables! Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database, or how to keep your **schema\_migrations** table up-to-date between development and production branches with [automatic schema migrations](/docs/vitess/tutorials/automatic-rails-migrations). ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Connect a Symfony application to PlanetScale Source: https://planetscale.com/docs/vitess/tutorials/connect-symfony-app In this tutorial, you'll learn how to connect a Symfony application to a PlanetScale MySQL database using a sample Symfony starter app. ## Prerequisites * [PHP](https://www.php.net/manual/en/install.php) — This tutorial uses `v8.1` * [Composer](https://getcomposer.org/) * A [PlanetScale account](https://auth.planetscale.com/sign-up) * [PlanetScale CLI](https://github.com/planetscale/cli) — You can also follow this tutorial using just the PlanetScale admin dashboard, but the CLI will make setup quicker. ## Set up the Symfony app This guide will integrate [a simple Symfony app](https://github.com/planetscale/symfony-example) with PlanetScale that will display a list of products stored in the database. If you have an existing application, you can also use that. **Step 1** Clone the starter Symfony application: ```bash theme={null} git clone https://github.com/planetscale/symfony-example.git ``` **Step 2** Enter into the folder and install the dependencies: ```bash theme={null} cd symfony-example composer install ``` **Step 3** Rename the `.env.example` file to `.env.local`: ```bash theme={null} mv .env.example .env.local ``` Once you deploy to production, don't forget to update `.env.local` to `.env`. **Step 4** Start the application: ```bash theme={null} symfony serve ``` You can view the application at [http://localhost:8000](http://localhost:8000). ## Set up the database Next, you need to set up your PlanetScale database and connect to it in the Symfony application. You can create a database either in the [PlanetScale dashboard](https://app.planetscale.com) or from the PlanetScale CLI. This guide will use the CLI, but you can follow the database setup instructions in the [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide) if you prefer the dashboard. **Step 1** Authenticate the CLI with the following command: ```bash theme={null} pscale auth login ``` **Step 2** Create a new database with a default `main` branch with the following command: ```bash theme={null} pscale database create --region ``` This tutorial uses `symfony_example` for `DATABASE_NAME`, but you can use any name with lowercase, alphanumeric characters, or underscores. You can also use dashes, but we don't recommend them, as they may need to be escaped in some instances. For `REGION_SLUG`, choose a region closest to you from the [available regions](/docs/vitess/regions#available-regions) or leave it blank. That's it! Your database is ready to use. Next, let's connect it to the Symfony application and then add some data. ## Connect to the Symfony app There are **two ways to connect** to PlanetScale: * With an auto-generated username and password * Using the PlanetScale proxy with the CLI Both options are covered below. ### Option 1: Connect with username and password (Recommended) These instructions show you how to use the [PlanetScale CLI](/docs/cli/planetscale-environment-setup) to generate a set of credentials. You can also get these exact values to to copy/paste from your [PlanetScale dashboard](https://app.planetscale.com). In the dashboard, click on the database > "**Connect**" > "**Generate new password**" > "**General**" dropdown > "**Symfony**". If the password is blurred, click "**New password**". Skip to step 3 once you have these credentials. **Step 1** Create a username and password with the PlanetScale CLI by running: ```bash theme={null} pscale password create ``` A default branch, `main`, is created when you create the database, so you can use that for `BRANCH_NAME`. > [!NOTE] > > The `PASSWORD_NAME` value represents the name of the username and password being generated. You can have multiple > credentials for a branch, so this gives you a way to categorize them. To manage your passwords in the dashboard, go to > your database dashboard page, click "Settings", and then click "Passwords". Take note of the values returned to you, as you won't be able to see this password again. **Step 2** Open the `.env.local` file in your Symfony app, find the database connection section, and replace the existing `DATABASE_URL` value with: ```bash theme={null} DATABASE_URL="mysql://:@:3306/?serverVersion=8.0" ``` Fill in `USERNAME`, `PASSWORD`, `ACCESS HOST URL` and `DATABASE_NAME` with the appropriate values from the CLI output above. Refresh your Symfony homepage and you should see the message that you're connected to your database! ### Option 2: Connect with the PlanetScale proxy To connect with the PlanetScale proxy, you'll need the [PlanetScale CLI](https://github.com/planetscale/cli). **Step 1** Open a connection by running the following: ```bash theme={null} pscale connect ``` If you're following this guide exactly and haven't created any branches, you can use the default branch, `main`. **Step 2** A secure connection to your database will be established and you'll see a local address you can use to connect to your application. **Step 3** Open the `.env.local` file in your Symfony app and update it as follows: ```bash theme={null} DB_HOST=127.0.0.1 DB_PORT=3306 # Get this from the output of the previous step DB_NAME= DATABASE_URL=mysql://${DB_HOST}:${DB_PORT}/${DB_NAME}?serverVersion=5.7 ``` The connection uses port `3306` by default, but if that's being used, it will pick a random port. Make sure you paste in whatever port is returned in the terminal. Fill in the database name as well. **Step 4** Open up `config/packages/doctrine.yaml`. Under `option`, you'll see a line for the SSL certificate that was used to connect with username and password: ```php theme={null} !php/const:PDO::MYSQL_ATTR_SSL_CA: /etc/ssl/cert.pem ``` Delete that line and save. Refresh your Symfony homepage and you should see the message that you're connected to your database! ## Run migrations and seeder Now that you're connected, let's add some data to see it in action. The sample application comes with a migration file at `migrations/Version20220120102247.php` that will create `category` and `product` tables in the database. There are also two seeder files, `src/DataFixtures/CategoryFixtures.php` and `src/DataFixtures/ProductFixtures.php`, that will add ten random categories and products to the `category` and `product` tables, respectively. Let's run those now. **Step 1** Make sure your database connection has been established. You'll see the message "You are connected to your-database-name" on the [Symfony app homepage](http://localhost:8000/) if everything is configured properly. **Step 2** In your terminal in the root of the Symfony project, run the following to run the migrations: ```bash theme={null} symfony console doctrine:migrations:migrate ``` You will get a message asking you to confirm. Type "yes" and hit enter to proceed. **Step 3** Next, seed the database by running: ```bash theme={null} symfony console doctrine:fixtures:load ``` This will purge your database and load the placeholder data into it. **Step 4** Refresh your Symfony homepage and you'll see a list of products and their category printed out. The `templates/product/index.html.twig` file pulls this data from the `product` table with the help of the `src/Controller/ProductController.php` file. ![Symfony PlanetScale starter app homepage](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/connect-symfony-app/example.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=97eb3ecc7ec5ab2161d913ede7e14122) ## Add data manually If you want to continue to play around with adding data on the fly, you have a few options: * PlanetScale CLI shell * PlanetScale dashboard console * Your favorite MySQL client (for a list of tested MySQL clients, review our article on [how to connect MySQL GUI applications](/docs/vitess/tutorials/connect-mysql-gui)) The first two options are covered below. ### Add data with PlanetScale CLI You can use the PlanetScale CLI to open a MySQL shell to interact with your database. You may need to [install the MySQL command line client](/docs/cli/planetscale-environment-setup) if you haven't already. Run the following command in your terminal: ```bash theme={null} pscale shell ``` This will open up a MySQL shell connected to the specified database and branch. > [!NOTE] > > A branch, `main`, was automatically created when you created your database, so you can use that for `BRANCH_NAME`. Add a record to the `product` table: ```sql theme={null} INSERT INTO `store_product` (name, description, image, category_id) VALUES ('Spaceship', 'Get ready for the trip of a lifetime', 'https://via.placeholder.com/150.png', 2); ``` The value `id` will be filled with a default value. You can verify it was added in the PlanetScale CLI MySQL shell with: ```sql theme={null} select * from product; ``` Type `exit` to exit the shell. You can now refresh the [Symfony homepage](http://localhost:8000) to see the new record. ### Add data with PlanetScale dashboard console If you don't care to install MySQL client or the PlanetScale CLI, another quick option using the MySQL console built into the PlanetScale dashboard. By default, web console access to production branches is disabled to prevent accidental deletion. From your database's dashboard page, click on the "**Settings**" tab, check the box labelled "**Allow web console access to production branches**", and click "**Save database settings**". **Step 1** Go to your [PlanetScale dashboard](https://app.planetscale.com) and select your Symfony database. **Step 2** Click on the "**Branches** and select the `main` branch. **Step 3** Click on "**Console**" **Step 4** Add a new record to the `product` table with: ```sql theme={null} INSERT INTO `store_product` (name, description, image, category_id) VALUES ('Spaceship', 'Get ready for the trip of a lifetime', 'https://via.placeholder.com/150.png', 2); ``` **Step 5** You can confirm that it was added by running: ```sql theme={null} select * from product; ``` You can now refresh the [Symfony homepage](http://localhost:8000) to see the new record. ## What's next? Once you're done with initial development, you can enable [safe migrations](/docs/vitess/schema-changes/safe-migrations) on your `main` production branch to protect it against direct schema changes and enable zero-downtime schema migrations. Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Deploy a Django app to Heroku Source: https://planetscale.com/docs/vitess/tutorials/deploy-a-django-app-to-heroku ## Overview This article will describe how to deploy a Django app to Heroku, which includes the necessary setup in Heroku’s dashboard. ## Prerequisites * A PlanetScale database — If you haven't created a database, refer to our [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide) to get started. * A Heroku account. * A project deployed to Heroku — If you're just poking around and don't already have an application to deploy, you can use our [Django sample](https://github.com/planetscale/django-example). ## Set up the project for Heroku There are a few requirements for running a Django application in Heroku: * The `gunicorn` and `django-heroku` packages as requirements. * A properly setup [Procfile](https://devcenter.heroku.com/articles/procfile). * Proper Config Var setup in Heroku. > [!NOTE] > > This article will make use of the [django-example GitHub repository](https://github.com/planetscale/django-example) that is built for the [Connect a Django application to PlanetScale document](/docs/vitess/tutorials/connect-django-app) ### Set up the Heroku Config Vars It’s important to store the connection details for the PlanetScale database in **Config Vars** in Heroku so they are properly secured. These details can be obtained from the PlanetScale dashboard by clicking the "**Connect**" button. ![The location of the “Connect” button in the PlanetScale dashboard. priority](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-a-django-app-to-heroku/database-2.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=7f1c488c306d2d4c1e167565835364e5) In the following modal, choose Django from the “Connect with” dropdown. The .env tab will show all of the Config vars that need to be set up in Heroku. Take note of these and head to the Heroku dashboard. ![The connection details for the project.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-a-django-app-to-heroku/connect-2.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=93308b998c3e32c437650fb5708f646d) Select the **Settings** tab of your Heroku project and then “**Reveal Config Vars”** from the Config **Vars** section. You should see your current Config Vars or an empty set of inputs if there are none configured yet. ![The Settings tab of the Heroku dashboard.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-a-django-app-to-heroku/heroku.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=59783b2b08b30c0f9d8bfad1077e23bd) Set up a separate **Config Var** for each line you captured from the PlanetScale dashboard. The one exception is the `MYSQL_ATTR_SSL_CA`, which should be set to `/etc/ssl/certs/ca-certificates.crt` > [!NOTE] > > Heroku uses Ubuntu by default to run applications deployed to their systems, which is why the `MYSQL_ATTR_SSL_CA` value needs to be different than the default values provided by PlanetScale ![The Config Vars setup for the project.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-a-django-app-to-heroku/ssl.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=58e09b8514cdd80d1f27cd81c62368b4) ### Update the requirements Add `gunicorn` and `django-heroku` to your `requirements.txt` file. This will install the necessary packages when deploying to Heroku. If you are following along using the example provided, here is the updated `requirements.txt` file: ``` asgiref==3.4.1 Django==4.0.1 djangorestframework==3.13.1 mysqlclient==2.1.0 python-dotenv==0.19.2 pytz==2021.3 sqlparse==0.4.2 gunicorn django-heroku ``` ### Add a Procfile The **Procfile** in your project tells Heroku how it should start up the project. The file must be in the root of the project and not in a subdirectory. Here is the **Procfile** used to deploy the **django-example** project to Heroku: ``` web: gunicorn --chdir ./mysite mysite.wsgi --log-file - ``` After these steps have been completed, you may redeploy your application to Heroku. To view a complete example, please refer to the [heroku-deployment branch](https://github.com/planetscale/django-example/tree/heroku-deployment) of the sample repository. This concludes the guide on deploying a Django application to Heroku. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Deploy to Netlify Source: https://planetscale.com/docs/vitess/tutorials/deploy-to-netlify This guide will walk you through setting up and deploying your PlanetScale database on Netlify. > [!NOTE] > > This doc is intended for users that are manually storing a connection string in an environment variable in Netlify. If you want to use the Netlify integration, which handles this for you, see the [PlanetScale integration in the Netlify docs](https://docs.netlify.com/extend/install-and-use/setup-guides/planetscale-integration/). ## Prerequisites * A PlanetScale database — If you haven't created a database, refer to our [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide) to get started * A [Netlify account](https://netlify.com/) * A project deployed to Netlify — If you're just poking around and don't already have an application to deploy, you can use our [Next.js + PlanetScale sample](/docs/vitess/tutorials/connect-nextjs-app) ## Connecting your PlanetScale database to your Netlify application ### Get your connection string from PlanetScale **Step 1** In your [PlanetScale dashboard](https://app.planetscale.com), click on the database you want to connect to. **Step 2** Click "**Connect**". **Step 3** Create a new password. Make sure to copy the password, as you'll only be shown it once. **Step 4** Select the framework you're using from the "**Select your language or framework**" section. This will give you the exact environment variable names you need for your selected framework. If your framework is not listed, choose "Other". ![PlanetScale dashboard connect modal priority](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-netlify/prisma.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=835e84a4d3e55f73b4989fba9a697a93) **Step 5** Keep this page open, as you'll need to copy these to Netlify momentarily. **Step 6** If you navigate away from the page, and can no longer access the password, create a new password by repeating steps 1-5. ### Copy environment variables to Netlify **Step 1** Go to your Netlify dashboard. **Step 2** Click on your Netlify project. **Step 3** Click "**Site settings**". **Step 4** Click "**Build & deploy**," then "**Environment**". **Step 5** Click "**Edit variable**". **Step 6** Click "**New variable**" and copy each value from your PlanetScale dashboard into a new environment variable in Netlify. Once you're done with one, click "**Add**" and continue to the next, if applicable. For example, if you're using Prisma, your connection string will look similar to this: ```bash theme={null} DATABASE_URL='mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my-database?sslaccept=strict' ``` > [!NOTE] > > Your environment variable name will be the same in your application's code. We used `DATABASE_URL` as an example, but this can be given a different name. In Netlify, you'll set it as follows: * **Key** = `DATABASE_URL` * **Value** = `mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my-database?sslaccept=strict` > [!NOTE] > > The credentials are blurred for the example, but when you paste them in, use the actual values. ![Netlify dashboard - Environment variables](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-netlify/environment-variables.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=0c932e42819d03a2c1f098333b41c42a) After you have saved, you will need to rebuild the site with the new environment variable. ## What's next? Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Deploy to Vercel Source: https://planetscale.com/docs/vitess/tutorials/deploy-to-vercel This guide will walk you through setting up and deploying your PlanetScale database on Vercel. To use a PlanetScale database with Vercel, there are a few prerequisites: * A PlanetScale database — If you haven't created a database, refer to our [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide) to get started * A [Vercel account](https://vercel.com/) * A project deployed to Vercel — If you're just poking around and don't already have an application to deploy, you can use our [Next.js + PlanetScale sample](/docs/vitess/tutorials/connect-nextjs-app) ## Get your connection string from PlanetScale **Step 1** In your [PlanetScale dashboard](https://app.planetscale.com), click on the database you want to connect to. **Step 2** Click "**Connect**". **Step 3** Create a new password. Make sure to copy the password, as you'll only be shown it once. **Step 4** Select the framework you're using from the "**Select your language or framework**" section. This will give you the exact environment variable names you need for your selected framework. If your framework is not listed, choose "Other". ![PlanetScale dashboard connect modal \{priority}](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-vercel/prisma.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=a0e74c1e2d55b2b86a1eb506ec0a91d4) **Step 5** Keep this page open, as you'll need to copy these to Vercel momentarily. **Step 6** If you navigate away from the page, and can no longer access the password, create a new password by repeating steps 1-5. ## Copy environment variables to Vercel **Step 1** Go to your Vercel dashboard. **Step 2** Click on your Vercel project. **Step 3** Click "**Settings**". **Step 4** Click "**Environment variables**". **Step 5** Copy each value from your PlanetScale dashboard into a new environment variable in Vercel. Once you're done with one, click "**Add**" and continue to the next, if applicable. For example, if you're using Prisma, your connection string will look similar to this: ```bash theme={null} DATABASE_URL='mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my_database?sslaccept=strict' ``` In Vercel, you'll set it as follows: * **NAME** = `DATABASE_URL` * **VALUE** = `mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my_database?sslaccept=strict` > [!NOTE] > > The credentials are blurred for the example, but when you paste them in, use the actual values. ![Vercel dashboard - Environment variables](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-vercel/environment-variables.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=e66d4f8a5e6ab9c21f7be071a3086a1c) ## What's next? Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Deploy to Netlify Source: https://planetscale.com/docs/vitess/tutorials/deployments/deploy-to-netlify This guide will walk you through setting up and deploying your PlanetScale database on Netlify. > [!NOTE] > > This doc is intended for users that are manually storing a connection string in an environment variable in Netlify. If you want to use the Netlify integration, which handles this for you, see the [PlanetScale integration in the Netlify docs](https://docs.netlify.com/extend/install-and-use/setup-guides/planetscale-integration/). ## Prerequisites * A PlanetScale database — If you haven't created a database, refer to our [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide) to get started * A [Netlify account](https://netlify.com/) * A project deployed to Netlify — If you're just poking around and don't already have an application to deploy, you can use our [Next.js + PlanetScale sample](/docs/vitess/tutorials/connect-nextjs-app) ## Connecting your PlanetScale database to your Netlify application ### Get your connection string from PlanetScale **Step 1** In your [PlanetScale dashboard](https://app.planetscale.com), click on the database you want to connect to. **Step 2** Click "**Connect**". **Step 3** Create a new password. Make sure to copy the password, as you'll only be shown it once. **Step 4** Select the framework you're using from the "**Select your language or framework**" section. This will give you the exact environment variable names you need for your selected framework. If your framework is not listed, choose "Other". ![PlanetScale dashboard connect modal priority](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-netlify/prisma.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=835e84a4d3e55f73b4989fba9a697a93) **Step 5** Keep this page open, as you'll need to copy these to Netlify momentarily. **Step 6** If you navigate away from the page, and can no longer access the password, create a new password by repeating steps 1-5. ### Copy environment variables to Netlify **Step 1** Go to your Netlify dashboard. **Step 2** Click on your Netlify project. **Step 3** Click "**Site settings**". **Step 4** Click "**Build & deploy**," then "**Environment**". **Step 5** Click "**Edit variable**". **Step 6** Click "**New variable**" and copy each value from your PlanetScale dashboard into a new environment variable in Netlify. Once you're done with one, click "**Add**" and continue to the next, if applicable. For example, if you're using Prisma, your connection string will look similar to this: ```bash theme={null} DATABASE_URL='mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my-database?sslaccept=strict' ``` > [!NOTE] > > Your environment variable name will be the same in your application's code. We used `DATABASE_URL` as an example, but this can be given a different name. In Netlify, you'll set it as follows: * **Key** = `DATABASE_URL` * **Value** = `mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my-database?sslaccept=strict` > [!NOTE] > > The credentials are blurred for the example, but when you paste them in, use the actual values. ![Netlify dashboard - Environment variables](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-netlify/environment-variables.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=0c932e42819d03a2c1f098333b41c42a) After you have saved, you will need to rebuild the site with the new environment variable. ## What's next? Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Deploy to Vercel Source: https://planetscale.com/docs/vitess/tutorials/deployments/deploy-to-vercel This guide will walk you through setting up and deploying your PlanetScale database on Vercel. It will cover two options: > **[Manually adding PlanetScale environment variables to your Vercel project](#manually-connect-to-vercel)** > **[Connecting your PlanetScale database to your application using the Vercel integration](#deploy-with-the-planetscale-vercel-integration)** ## Prerequisites * A PlanetScale database — If you haven't created a database, refer to our [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide) to get started * A [Vercel account](https://vercel.com/) * A project deployed to Vercel — If you're just poking around and don't already have an application to deploy, you can use our [Next.js + PlanetScale sample](/docs/vitess/tutorials/connect-nextjs-app) ## Manually connect to Vercel ### Get your connection string from PlanetScale **Step 1** In your [PlanetScale dashboard](https://app.planetscale.com), click on the database you want to connect to. **Step 2** Click "**Connect**". **Step 3** Create a new password. Make sure to copy the password, as you'll only be shown it once. **Step 4** Select the framework you're using from the "**Select your language or framework**" section. This will give you the exact environment variable names you need for your selected framework. If your framework is not listed, choose "Other". ![PlanetScale dashboard connect modal \{priority}](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-vercel/prisma.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=a0e74c1e2d55b2b86a1eb506ec0a91d4) **Step 5** Keep this page open, as you'll need to copy these to Vercel momentarily. **Step 6** If you navigate away from the page, and can no longer access the password, create a new password by repeating steps 1-5. ### Copy environment variables to Vercel **Step 1** Go to your Vercel dashboard. **Step 2** Click on your Vercel project. **Step 3** Click "**Settings**". **Step 4** Click "**Environment variables**". **Step 5** Copy each value from your PlanetScale dashboard into a new environment variable in Vercel. Once you're done with one, click "**Add**" and continue to the next, if applicable. For example, if you're using Prisma, your connection string will look similar to this: ```bash theme={null} DATABASE_URL='mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my_database?sslaccept=strict' ``` In Vercel, you'll set it as follows: * **NAME** = `DATABASE_URL` * **VALUE** = `mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my_database?sslaccept=strict` > [!NOTE] > > The credentials are blurred for the example, but when you paste them in, use the actual values. ![Vercel dashboard - Environment variables](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-vercel/environment-variables.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=e66d4f8a5e6ab9c21f7be071a3086a1c) ## Deploy with the PlanetScale Vercel integration If you don't want to copy and paste the environment variables over to Vercel, you can use the [PlanetScale integration from the Vercel marketplace](https://vercel.com/integrations/planetscale). You can choose which database you want to connect to, and we'll automatically pull the necessary environment variables into your Vercel project. **Step 1** You must have an existing PlanetScale database to use the integration. You can create a database in the [PlanetScale dashboard](https://app.planetscale.com). **Step 2** Click "**Add integration**" on the [Vercel integrations page](https://vercel.com/integrations/planetscale). **Step 3** Select the Vercel account you want to connect with. **Step 4** On the left, you'll see the Vercel options, and on the right, the PlanetScale options. **Step 5** Select the Vercel project you want to connect to, and beneath that, select the framework you're using. If the framework isn't listed, select "**General**". This selection is what determines the names of the environment variables. **Step 6** On the right side, choose the [PlanetScale Organization](/docs/security/access-control) that the database is in. The integration will remain tied to this Organization and cannot be changed. **Step 7** Beneath that, select the database you want to connect to. **Step 8** Click "**Connect database**". **Step 9** Back in your Vercel dashboard, confirm the environment variables were added by going to your Vercel project > "**Settings**" > "**Environment variables**" ### Configure your connection After you set up the initial connection, you also have the option to configure the PlanetScale connection, add more databases to the project, or remove databases from the project. To access the configuration page: **Step 1** In your Vercel project dashboard, click "**Integrations**". **Step 2** Click the "**PlanetScale**" integration. **Step 3** Click the "**Configure**" button. > [!NOTE] > > If you're modifying an existing connection on a Vercel project, these environment variable values will be regenerated and overwritten. > [!NOTE] > > * Environment variables are removed from all projects tied to the integration. > * Your application will no longer be able to connect to your PlanetScale database. ## What's next? Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Next.js and PlanetScale Netlify template tutorial Source: https://planetscale.com/docs/vitess/tutorials/deployments/nextjs-planetscale-netlify-template ## Overview This guide will show you how to get up and running with the [Netlify, Next.js, and PlanetScale starter template](https://github.com/planetscale/nextjs-planetscale-starter). The template includes the following features: * Simple user admin dashboard * [PlanetScale](/docs/) database * [Prisma ORM](https://www.prisma.io/) integration * [Next.js authentication](https://nextjs.org/docs/app/building-your-application/authentication) * One-click [deploy to Netlify](https://netlify.com) * [Tailwind CSS](https://tailwindcss.com/) styling ![Example of the dashboard application](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/nextjs-planetscale-netlify-template/example.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=0dd367307a302e8c6ce3b50d729eb13e) > [!NOTE] > > If you're coming from the **[Netlify Template](https://github.com/planetscale/nextjs-planetscale-starter)** and you already clicked deploy, you're in the right place! This tutorial will walk you through how to set up your PlanetScale database so that you can fill in the environment variables in the Netlify dashboard. You'll also learn how to set up your local environment so you can continue to develop and extend this starter template. Just read through the prerequisites and then skip the first section to [go straight to the local setup](#set-up-the-project-locally). > > If you're **starting fresh and haven't deployed yet** (or don't want to deploy), you can start from the beginning of this tutorial. ### Prerequisites To follow along with this guide, you'll need the following: * A [free PlanetScale account](https://auth.planetscale.com/sign-up) * The [PlanetScale CLI](https://github.com/planetscale/cli#installation) * [Yarn](https://yarnpkg.com/getting-started/install) * [Node (LTS)](https://nodejs.org/en/) * A [free Netlify account](https://app.netlify.com/signup) ## One-click deploy to Netlify The one-click deploy button allows you to connect Netlify to your GitHub account to clone the `nextjs-planetscale-starter` repository and automatically deploy it. Be sure to [sign up for a Netlify account](https://app.netlify.com/signup) before clicking the deploy button. [![Deploy to Netlify button](https://www.netlify.com/img/deploy/button.svg)](https://app.netlify.com/start/deploy?repository=https://github.com/planetscale/nextjs-planetscale-starter) Once you click the button, you'll be taken to Netlify’s direct deploy page with the pre-built project’s repository passed as a parameter in the URL. Click the "**Connect to GitHub**" button to authorize access. Next, you'll be asked to configure your site variables. For the `Secret` value, navigate to [`https://generate-secret.now.sh/32`](https://generate-secret.now.sh/32) to generate a secret and then paste that in. You can leave the `Database URL` and `NextAuth URL` values blank for now. Click "Save & Deploy". Your site will take about a minute to build and then you'll be taken to a settings page. A unique Netlify URL will be generated for the project. You can click that now to see your live site! The next section will show you how to set the project up locally and create your PlanetScale database to connect to your live site. ## Set up the project locally If you already went through the [Netlify deployment](https://github.com/planetscale/nextjs-planetscale-starter), find the repository that was created for you in your GitHub account and clone it. If you didn't deploy and just want to run the template locally, you can clone the [`nextjs-planetscale-starter` repository](https://github.com/planetscale/nextjs-planetscale-starter). Enter into the folder and install the dependencies: ```bash theme={null} yarn install ``` Run the application with: ```bash theme={null} yarn next ``` Navigate to [http://localhost:3000/](http://localhost:3000/) in your browser to view the PlanetScale Next.js Starter app. ![Next.js PlanetScale Starter application homepage](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/nextjs-planetscale-netlify-template/starter.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=d5b438dfb74ffe78aa4c1c67a67eae1f) ## Database setup Next, you need to set up your PlanetScale database. If you don't already have a [PlanetScale account](/docs/planetscale-plans), you can [sign up for a free one here](https://auth.planetscale.com/sign-up). ### Create your database To begin, create a new database. You can either do this in the dashboard or using the [PlanetScale CLI](/docs/cli). In the dashboard, click on the "**Create a database**" button. Name your database "`netlify-starter`", or whatever name you wish. Select the region closest to you, and click "**Create database**". Alternatively, [sign in and create a database with the CLI](/docs/vitess/tutorials/planetscale-quick-start-guide#getting-started-%E2%80%94-planetscale-cli) by running the following: ```bash theme={null} pscale database create --region ``` The list of region slugs can be found in our [Regions documentation](/docs/vitess/regions#available-regions). ### Connect to your database branches locally To connect locally, make sure you've authenticated in the CLI. If not, follow the directions in the [sign-in section of our quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide#sign-in-to-your-account). Next, connect to your `main` branch locally by running the following in your terminal: ```bash theme={null} pscale connect main --port ``` Choose any unused port. This tutorial uses `3309`. You'll see the response "Secure connection to database `netlify-starter` and branch `main` is established!" along with the local proxy address for your database. Take note of that address for the next step. ## Set up local environment variables For the last part of setup, you need to fill in your environment variables. Make a copy of the `.env.example` file at the root of your project and rename it `.env`: ```bash theme={null} cp .env.example .env ``` The `DATABASE_URL` value comes from your PlanetScale database and will be in the following form: `mysql://@
:/` * `user` — the database user * `address` — the local address returned in the previous step * `port` — the port you specified in the previous step * `database` — the name of your database Below is an example of the `.env` file based on this tutorial: ```js theme={null} DATABASE_URL="mysql://root@127.0.0.1:3309/netlify-starter" NEXTAUTH_URL=http://localhost:3000 # Navigate to https://generate-secret.now.sh/32 to generate a secret for the variable below NEXTAUTH_SECRET= ``` Remember, these are the values for your **local environment**. The values needed for the Netlify environment variables will be covered shortly. ## Push your database schema to PlanetScale Now that your PlanetScale database is connected to your application, it's time to [push your database schema](https://www.prisma.io/docs/orm/reference/prisma-cli-reference#db-push). In your terminal, run: ```bash theme={null} yarn db:push ``` You can view the schema in your PlanetScale dashboard by clicking the `netlify-starter` database > "**Branches**" > `main` > "**Schema**" > "**Refresh schema**". You'll now see the following tables: * `Account` * `Session` * `User` * `VerificationToken` You can also verify it worked using the PlanetScale CLI. Run the following to start a MySQL shell (where `netlify-starter` is your database): ```bash theme={null} pscale shell netlify-starter main ``` Once in the shell, view the tables with: ```bash theme={null} SHOW tables; ``` Type `exit` to exit the MySQL shell. The database schema, `db/schema.prisma`, has been loaded and your PlanetScale database is now ready for data. ## Seed the database To seed the database with users, run the following: ```bash theme={null} yarn db:seed ``` This will add three mock users to the `User` database, as described in `db/seed.ts`. To verify that they were added, click on the "**Console**" tab in your PlanetScale dashboard on the `main` branch of your `netlify-starter` database. Run the following: ```bash theme={null} SELECT * from User; ``` You can also run this command from the CLI using the same `pscale shell` command mentioned above. ## Netlify environment variables Now that you have your site running locally, let's shift back to the live Netlify site. The final step in the site deployment is configuring your production environment variables. In the Netlify dashboard under your site's settings page, click on "**Build & deploy**" > "**Environment**" in the left nav. Click on the "**Edit variables**" button and enter in the following key/value pairs: * `DATABASE_URL` — To find this value, go back to your PlanetScale dashboard, click on the `netlify-starter` database, click "**Connect**". If the password is masked and you don't have one, you may click the "**Generate new password**" button to create a new one. Next, click on the "**General**" dropdown in that modal and select "**Prisma**". Copy the value for `url` and paste that back in the Netlify dashboard as the value for `DATABASE_URL`. Be sure to save your PlanetScale password somewhere as you won't be able to access it again after closing the modal. * `NEXTAUTH_SECRET` — You may have already filled this out, but if not generate a new secret at [https://generate-secret.now.sh/32](https://generate-secret.now.sh/32) and paste in the value that's returned. * `NEXTAUTH_URL` — Paste in the Netlify site name that was generated for you. For example, `https://stoic-lumiere-6df10.netlify.app` Click "**Save**". Now, redeploy the site with these new variables by going to "Deploys" in the top nav and clicking the "**Trigger deploy**" > "**Deploy site**" button. ## Set up your admin account Now that your site is live and connected to your PlanetScale database, you need to set up your admin account for your application. Go to `/admin/setup` on your live Netlify site (or locally if you didn't deploy), and fill in the form to set up your account. This will be automatically saved to your PlanetScale database. Creating an admin account gives you access to the `/admin` route in your application, which is the dashboard to manage your users. Once you're signed in as an admin, navigate to `/admin` to see a list of your users. ## Customize and extend And that's it! If you followed this tutorial completely, you now have a local and production version of your **Next.js + PlanetScale + Prisma admin dashboard**. So what's next? Now, it's time to extend it! Next.js Authentication is baked into this starter, so you can explore the [Next.js docs to manage authentication](https://nextjs.org/docs/app/building-your-application/authentication) in your application. You also have a fully functional PlanetScale MySQL database built for scale using the power of [Vitess](https://vitess.io/). You might have noticed that this tutorial uses the same database locally as it is in production. This was just for simplicity, but with PlanetScale, you can take advantage of our [powerful branching feature](/docs/vitess/schema-changes/branching) to create development branches of your database specifically for testing locally. All you have to do is create a new branch and swap out the `DATABASE_URL` environment variable in your local `.env` file. Finally, you have [Prisma ORM](https://www.prisma.io/) already configured in your application. If you want to add any more fields to your `User` table or create any new tables, Prisma makes it easy with the `schema.prisma` file. If you want to make any schema changes, it's a perfect time to try out the [PlanetScale branching feature](/docs/vitess/schema-changes/branching) feature. If you mess up, those changes won't touch production. And once you're satisfied with the changes, you can deploy to production without causing downtime thanks to PlanetScale [non-blocking schema changes](/docs/vitess/schema-changes). Hopefully this tutorial has been helpful. We'd love to hear how you're extending your starter application. [Tweet us @PlanetScale](https://twitter.com/planetscale) and share what you built! ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Deploy to Vercel Source: https://planetscale.com/docs/vitess/tutorials/nextjs-deploy-to-vercel This guide will walk you through setting up and deploying your PlanetScale database on Vercel. It will cover two options: > **[Manually adding PlanetScale environment variables to your Vercel project](#manually-connect-to-vercel)** > **[Connecting your PlanetScale database to your application using the Vercel integration](#deploy-with-the-planetscale-vercel-integration)** ## Prerequisites * A PlanetScale database — If you haven't created a database, refer to our [PlanetScale quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide) to get started * A [Vercel account](https://vercel.com/) * A project deployed to Vercel — If you're just poking around and don't already have an application to deploy, you can use our [Next.js + PlanetScale sample](/docs/vitess/tutorials/connect-nextjs-app) ## Manually connect to Vercel ### Get your connection string from PlanetScale **Step 1** In your [PlanetScale dashboard](https://app.planetscale.com), click on the database you want to connect to. **Step 2** Click "**Connect**". **Step 3** Create a new password. Make sure to copy the password, as you'll only be shown it once. **Step 4** Select the framework you're using from the "**Select your language or framework**" section. This will give you the exact environment variable names you need for your selected framework. If your framework is not listed, choose "Other". ![PlanetScale dashboard connect modal \{priority}](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-vercel/prisma.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=a0e74c1e2d55b2b86a1eb506ec0a91d4) **Step 5** Keep this page open, as you'll need to copy these to Vercel momentarily. **Step 6** If you navigate away from the page, and can no longer access the password, create a new password by repeating steps 1-5. ### Copy environment variables to Vercel **Step 1** Go to your Vercel dashboard. **Step 2** Click on your Vercel project. **Step 3** Click "**Settings**". **Step 4** Click "**Environment variables**". **Step 5** Copy each value from your PlanetScale dashboard into a new environment variable in Vercel. Once you're done with one, click "**Add**" and continue to the next, if applicable. For example, if you're using Prisma, your connection string will look similar to this: ```bash theme={null} DATABASE_URL='mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my_database?sslaccept=strict' ``` In Vercel, you'll set it as follows: * **NAME** = `DATABASE_URL` * **VALUE** = `mysql://xxxxxxxxx:************@xxxxxxxxxx.us-east-3.psdb.cloud/my_database?sslaccept=strict` > [!NOTE] > > The credentials are blurred for the example, but when you paste them in, use the actual values. ![Vercel dashboard - Environment variables](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/deploy-to-vercel/environment-variables.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=e66d4f8a5e6ab9c21f7be071a3086a1c) ## Deploy with the PlanetScale Vercel integration If you don't want to copy and paste the environment variables over to Vercel, you can use the [PlanetScale integration from the Vercel marketplace](https://vercel.com/integrations/planetscale). You can choose which database you want to connect to, and we'll automatically pull the necessary environment variables into your Vercel project. **Step 1** You must have an existing PlanetScale database to use the integration. You can create a database in the [PlanetScale dashboard](https://app.planetscale.com). **Step 2** Click "**Add integration**" on the [Vercel integrations page](https://vercel.com/integrations/planetscale). **Step 3** Select the Vercel account you want to connect with. **Step 4** On the left, you'll see the Vercel options, and on the right, the PlanetScale options. **Step 5** Select the Vercel project you want to connect to, and beneath that, select the framework you're using. If the framework isn't listed, select "**General**". This selection is what determines the names of the environment variables. **Step 6** On the right side, choose the [PlanetScale Organization](/docs/security/access-control) that the database is in. The integration will remain tied to this Organization and cannot be changed. **Step 7** Beneath that, select the database you want to connect to. **Step 8** Click "**Connect database**". **Step 9** Back in your Vercel dashboard, confirm the environment variables were added by going to your Vercel project > "**Settings**" > "**Environment variables**" ### Configure your connection After you set up the initial connection, you also have the option to configure the PlanetScale connection, add more databases to the project, or remove databases from the project. To access the configuration page: **Step 1** In your Vercel project dashboard, click "**Integrations**". **Step 2** Click the "**PlanetScale**" integration. **Step 3** Click the "**Configure**" button. > [!NOTE] > > If you're modifying an existing connection on a Vercel project, these environment variable values will be regenerated and overwritten. > [!NOTE] > > * Environment variables are removed from all projects tied to the integration. > * Your application will no longer be able to connect to your PlanetScale database. ## What's next? Learn more about how PlanetScale allows you to make [non-blocking schema changes](/docs/vitess/schema-changes) to your database tables without locking or causing downtime for production databases. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Next.js and PlanetScale Netlify template tutorial Source: https://planetscale.com/docs/vitess/tutorials/nextjs-planetscale-netlify-template ## Overview This guide will show you how to get up and running with the [Netlify, Next.js, and PlanetScale starter template](https://github.com/planetscale/nextjs-planetscale-starter). The template includes the following features: * Simple user admin dashboard * [PlanetScale](/docs/) database * [Prisma ORM](https://www.prisma.io/) integration * [Next.js authentication](https://nextjs.org/docs/app/building-your-application/authentication) * One-click [deploy to Netlify](https://netlify.com) * [Tailwind CSS](https://tailwindcss.com/) styling ![Example of the dashboard application](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/nextjs-planetscale-netlify-template/example.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=0dd367307a302e8c6ce3b50d729eb13e) > [!NOTE] > > If you're coming from the **[Netlify Template](https://github.com/planetscale/nextjs-planetscale-starter)** and you already clicked deploy, you're in the right place! This tutorial will walk you through how to set up your PlanetScale database so that you can fill in the environment variables in the Netlify dashboard. You'll also learn how to set up your local environment so you can continue to develop and extend this starter template. Just read through the prerequisites and then skip the first section to [go straight to the local setup](#set-up-the-project-locally). > > If you're **starting fresh and haven't deployed yet** (or don't want to deploy), you can start from the beginning of this tutorial. ### Prerequisites To follow along with this guide, you'll need the following: * A [free PlanetScale account](https://auth.planetscale.com/sign-up) * The [PlanetScale CLI](https://github.com/planetscale/cli#installation) * [Yarn](https://yarnpkg.com/getting-started/install) * [Node (LTS)](https://nodejs.org/en/) * A [free Netlify account](https://app.netlify.com/signup) ## One-click deploy to Netlify The one-click deploy button allows you to connect Netlify to your GitHub account to clone the `nextjs-planetscale-starter` repository and automatically deploy it. Be sure to [sign up for a Netlify account](https://app.netlify.com/signup) before clicking the deploy button. [![Deploy to Netlify button](https://www.netlify.com/img/deploy/button.svg)](https://app.netlify.com/start/deploy?repository=https://github.com/planetscale/nextjs-planetscale-starter) Once you click the button, you'll be taken to Netlify’s direct deploy page with the pre-built project’s repository passed as a parameter in the URL. Click the "**Connect to GitHub**" button to authorize access. Next, you'll be asked to configure your site variables. For the `Secret` value, navigate to [`https://generate-secret.now.sh/32`](https://generate-secret.now.sh/32) to generate a secret and then paste that in. You can leave the `Database URL` and `NextAuth URL` values blank for now. Click "Save & Deploy". Your site will take about a minute to build and then you'll be taken to a settings page. A unique Netlify URL will be generated for the project. You can click that now to see your live site! The next section will show you how to set the project up locally and create your PlanetScale database to connect to your live site. ## Set up the project locally If you already went through the [Netlify deployment](https://github.com/planetscale/nextjs-planetscale-starter), find the repository that was created for you in your GitHub account and clone it. If you didn't deploy and just want to run the template locally, you can clone the [`nextjs-planetscale-starter` repository](https://github.com/planetscale/nextjs-planetscale-starter). Enter into the folder and install the dependencies: ```bash theme={null} yarn install ``` Run the application with: ```bash theme={null} yarn next ``` Navigate to [http://localhost:3000/](http://localhost:3000/) in your browser to view the PlanetScale Next.js Starter app. ![Next.js PlanetScale Starter application homepage](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/nextjs-planetscale-netlify-template/starter.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=d5b438dfb74ffe78aa4c1c67a67eae1f) ## Database setup Next, you need to set up your PlanetScale database. If you don't already have a [PlanetScale account](/docs/planetscale-plans), you can [sign up for a free one here](https://auth.planetscale.com/sign-up). ### Create your database To begin, create a new database. You can either do this in the dashboard or using the [PlanetScale CLI](/docs/cli). In the dashboard, click on the "**Create a database**" button. Name your database "`netlify-starter`", or whatever name you wish. Select the region closest to you, and click "**Create database**". Alternatively, [sign in and create a database with the CLI](/docs/vitess/tutorials/planetscale-quick-start-guide#getting-started-%E2%80%94-planetscale-cli) by running the following: ```bash theme={null} pscale database create --region ``` The list of region slugs can be found in our [Regions documentation](/docs/vitess/regions#available-regions). ### Connect to your database branches locally To connect locally, make sure you've authenticated in the CLI. If not, follow the directions in the [sign-in section of our quickstart guide](/docs/vitess/tutorials/planetscale-quick-start-guide#sign-in-to-your-account). Next, connect to your `main` branch locally by running the following in your terminal: ```bash theme={null} pscale connect main --port ``` Choose any unused port. This tutorial uses `3309`. You'll see the response "Secure connection to database `netlify-starter` and branch `main` is established!" along with the local proxy address for your database. Take note of that address for the next step. ## Set up local environment variables For the last part of setup, you need to fill in your environment variables. Make a copy of the `.env.example` file at the root of your project and rename it `.env`: ```bash theme={null} cp .env.example .env ``` The `DATABASE_URL` value comes from your PlanetScale database and will be in the following form: `mysql://@
:/` * `user` — the database user * `address` — the local address returned in the previous step * `port` — the port you specified in the previous step * `database` — the name of your database Below is an example of the `.env` file based on this tutorial: ```js theme={null} DATABASE_URL="mysql://root@127.0.0.1:3309/netlify-starter" NEXTAUTH_URL=http://localhost:3000 # Navigate to https://generate-secret.now.sh/32 to generate a secret for the variable below NEXTAUTH_SECRET= ``` Remember, these are the values for your **local environment**. The values needed for the Netlify environment variables will be covered shortly. ## Push your database schema to PlanetScale Now that your PlanetScale database is connected to your application, it's time to [push your database schema](https://www.prisma.io/docs/orm/reference/prisma-cli-reference#db-push). In your terminal, run: ```bash theme={null} yarn db:push ``` You can view the schema in your PlanetScale dashboard by clicking the `netlify-starter` database > "**Branches**" > `main` > "**Schema**" > "**Refresh schema**". You'll now see the following tables: * `Account` * `Session` * `User` * `VerificationToken` You can also verify it worked using the PlanetScale CLI. Run the following to start a MySQL shell (where `netlify-starter` is your database): ```bash theme={null} pscale shell netlify-starter main ``` Once in the shell, view the tables with: ```bash theme={null} SHOW tables; ``` Type `exit` to exit the MySQL shell. The database schema, `db/schema.prisma`, has been loaded and your PlanetScale database is now ready for data. ## Seed the database To seed the database with users, run the following: ```bash theme={null} yarn db:seed ``` This will add three mock users to the `User` database, as described in `db/seed.ts`. To verify that they were added, click on the "**Console**" tab in your PlanetScale dashboard on the `main` branch of your `netlify-starter` database. Run the following: ```bash theme={null} SELECT * from User; ``` You can also run this command from the CLI using the same `pscale shell` command mentioned above. ## Netlify environment variables Now that you have your site running locally, let's shift back to the live Netlify site. The final step in the site deployment is configuring your production environment variables. In the Netlify dashboard under your site's settings page, click on "**Build & deploy**" > "**Environment**" in the left nav. Click on the "**Edit variables**" button and enter in the following key/value pairs: * `DATABASE_URL` — To find this value, go back to your PlanetScale dashboard, click on the `netlify-starter` database, click "**Connect**". If the password is masked and you don't have one, you may click the "**Generate new password**" button to create a new one. Next, click on the "**General**" dropdown in that modal and select "**Prisma**". Copy the value for `url` and paste that back in the Netlify dashboard as the value for `DATABASE_URL`. Be sure to save your PlanetScale password somewhere as you won't be able to access it again after closing the modal. * `NEXTAUTH_SECRET` — You may have already filled this out, but if not generate a new secret at [https://generate-secret.now.sh/32](https://generate-secret.now.sh/32) and paste in the value that's returned. * `NEXTAUTH_URL` — Paste in the Netlify site name that was generated for you. For example, `https://stoic-lumiere-6df10.netlify.app` Click "**Save**". Now, redeploy the site with these new variables by going to "Deploys" in the top nav and clicking the "**Trigger deploy**" > "**Deploy site**" button. ## Set up your admin account Now that your site is live and connected to your PlanetScale database, you need to set up your admin account for your application. Go to `/admin/setup` on your live Netlify site (or locally if you didn't deploy), and fill in the form to set up your account. This will be automatically saved to your PlanetScale database. Creating an admin account gives you access to the `/admin` route in your application, which is the dashboard to manage your users. Once you're signed in as an admin, navigate to `/admin` to see a list of your users. ## Customize and extend And that's it! If you followed this tutorial completely, you now have a local and production version of your **Next.js + PlanetScale + Prisma admin dashboard**. So what's next? Now, it's time to extend it! Next.js Authentication is baked into this starter, so you can explore the [Next.js docs to manage authentication](https://nextjs.org/docs/app/building-your-application/authentication) in your application. You also have a fully functional PlanetScale MySQL database built for scale using the power of [Vitess](https://vitess.io/). You might have noticed that this tutorial uses the same database locally as it is in production. This was just for simplicity, but with PlanetScale, you can take advantage of our [powerful branching feature](/docs/vitess/schema-changes/branching) to create development branches of your database specifically for testing locally. All you have to do is create a new branch and swap out the `DATABASE_URL` environment variable in your local `.env` file. Finally, you have [Prisma ORM](https://www.prisma.io/) already configured in your application. If you want to add any more fields to your `User` table or create any new tables, Prisma makes it easy with the `schema.prisma` file. If you want to make any schema changes, it's a perfect time to try out the [PlanetScale branching feature](/docs/vitess/schema-changes/branching) feature. If you mess up, those changes won't touch production. And once you're satisfied with the changes, you can deploy to production without causing downtime thanks to PlanetScale [non-blocking schema changes](/docs/vitess/schema-changes). Hopefully this tutorial has been helpful. We'd love to hear how you're extending your starter application. [Tweet us @PlanetScale](https://twitter.com/planetscale) and share what you built! ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale Vitess quickstart guide Source: https://planetscale.com/docs/vitess/tutorials/planetscale-quick-start-guide PlanetScale supports Vitess, [Neki](/docs/neki/tutorials/planetscale-quick-start-guide), and [Postgres](/docs/postgres/tutorials/planetscale-postgres-quickstart) databases. This guide covers getting started with a Vitess database. ## Overview The following guide will show you how to: * Create a database with PlanetScale * Make a schema change * Insert data * Promote your database branch to production If you already have your PlanetScale database set up, you may find the [Connecting your application](/docs/vitess/tutorials/connect-any-application) or [Branching](/docs/vitess/schema-changes/branching) guides more helpful. This guide is split up so that you can either follow it in the [PlanetScale dashboard](#getting-started-%E2%80%94-planetscale-dashboard) or using the [PlanetScale CLI](#getting-started-%E2%80%94-planetscale-cli). [PlanetScale quickstart guide](https://player.vimeo.com/video/830571983?dnt=true) ## Getting started — PlanetScale dashboard You'll need [a PlanetScale account](https://auth.planetscale.com/sign-up) to complete this guide. ### Create a database Follow these steps to create a database: **Step 1** Click "**New database**" > "**Create a database**" on your organization's overview page. **Step 2** Name your database. **Step 3** Select a [region](/docs/vitess/regions). For the lowest latency, select a region near you or your application's hosting location. **Step 4** Select the desired [cluster and storage size](/docs/planetscale-plans#base-plan) for your database. **Step 5** Enter a valid credit or debit card. **Step 6** Finally, click the "**Create database**" button to deploy your database. Your database is created with a [**default production branch**](/docs/vitess/schema-changes/branching), `main`, which you will use to apply a schema change and insert data. While this is just the first branch we create for you, you can always create new development branches (isolated copies of the production schema) off of production to use for development. ### Add a schema to your database This quickstart demonstrates how to create and use two relational tables: `categories` and `products`. **Step 1** By default, web console access to production branches is disabled to prevent accidental deletion. From your database's dashboard page, click on the "**Settings**" tab, check the box labelled "**Allow web console access to production branches**", and click "**Save database settings**". **Step 2** Click on the "**Console**" tab in the database navigation. This will open up a [web console](/docs/vitess/web-console) connected to your database branch. ![Branches](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/planetscale-quick-start-guide/the-console-tab.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=099f323651e81c65e32b469381a80cf3) **Step 3** By default the `main` branch is preselected. Click **"Connect"**. **Step 4** Create the `categories` and `products` tables by running the following commands in the web console: ```sql theme={null} CREATE TABLE categories ( id int NOT NULL AUTO_INCREMENT PRIMARY KEY, name varchar(255) NOT NULL ); ``` ```sql theme={null} CREATE TABLE products ( id int NOT NULL AUTO_INCREMENT PRIMARY KEY, name varchar(255) NOT NULL, image_url varchar(255), category_id INT, KEY category_id_idx (category_id) ); ``` > [!NOTE] > > If you want to make schema changes containing foreign key constraints, enable [foreign key constraints](/docs/vitess/foreign-key-constraints) support in your database settings page. **Step 5** You can confirm that the tables have been added by running: ```sql theme={null} SHOW TABLES; ``` ### Insert data into your database Now that you have created your tables, let's insert some data. Run the following commands to add a product and category to your tables: ```sql theme={null} INSERT INTO categories (name) VALUES ('Office supplies'); ``` ```sql theme={null} INSERT INTO products (name, image_url, category_id) VALUES ('Ballpoint pen', 'https://example.com/500x500', '1'); ``` You can confirm the data has been added with: ```sql theme={null} SELECT * FROM products; ``` ```sql theme={null} SELECT * FROM categories; ``` You can view the schema of your database by navigating to the "**Branches**" tab and selecting the database you want to view. For now, select the `main` database, and it will display the names of the two tables you just created. Click on the name of each table to see further schema details. ### Enable safe migrations on your `main` branch All of the work you've done so far has been on a default production branch, `main`, that was automatically created when you created the database. A production branch is a highly available database branch that includes an additional replica. It also has the option to enable [safe migrations](/docs/vitess/schema-changes/safe-migrations), which enables non-blocking schema changes and can protect your database from accidental schema changes. [Safe migrations](/docs/vitess/schema-changes/safe-migrations) is an optional, but highly recommended, feature that adds an additional layer of protection to your branch by preventing accidental schema modifications and enabling no-downtime schema changes. With safe migrations enabled, any DDL issued directly to the branch will not be accepted. Instead, changes must be made using the PlanetScale flow, where deploy requests are used to safely merge changes in a collaborative environment. ### To enable safe migrations: **Step 1** Click "Dashboard" in the navigation, and click the "**cog**" in the upper right of the infrastructure card to open a modal. ![Production UI card](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/planetscale-quick-start-guide/production-branch-card-with-sm-disabled-2.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=cb7b8a1aedcbe46563911f9647d6c21d) **Step 2** Toggle "**Enable safe migrations**", then click the "**Enable safe migrations**" button. The `main` branch now contains the `categories` and `products` tables you created, along with the data you inserted. In addition, it is highly available with an additional replica, and is enabled for zero-downtime migrations with [safe migrations](/docs/vitess/schema-changes/safe-migrations). ### What's next? Now that you've created a database, applied schema changes, added data, and enable safe migrations, it's time to connect to your application. You can use our [Connect Any Application tutorial](/docs/vitess/tutorials/connect-any-application) for a general step-by-step approach, one of our language-specific guides, or head straight to our [Connection Strings documentation](/docs/vitess/connecting/connection-strings) for more information about creating connection strings. When you want to continue development on your database: **Step 1** [Create a new branch](/docs/vitess/schema-changes/branching) off of your production branch **Step 2** Go through the same process described in this doc to make schema changes **Step 3** [Create a deploy request](/docs/vitess/schema-changes/deploy-requests) to merge the changes into your production branch > [!NOTE] > > When you branch off of a production branch, your development branch will have the same schema as production, but it > **will not** copy over any data from the production database. We suggest seeding development branches with mock > data. ## Getting started — PlanetScale CLI Make sure you first have [downloaded and installed the PlanetScale CLI](https://github.com/planetscale/cli#installation). You will also need a PlanetScale account. You can [sign up for a PlanetScale account here](https://auth.planetscale.com/sign-up) or run `pscale signup` to create an account straight from the CLI. ### Sign in to your account To authenticate with the PlanetScale CLI, enter the following: ```bash theme={null} pscale auth login ``` You'll be taken to a screen in the browser where you'll be asked to confirm the code displayed in your terminal. If the confirmation codes match, click the "**Confirm code**" button in your browser. You should receive the message "Successfully logged in" in your terminal. You can now close the confirmation page in the browser and proceed in the terminal. ### Create a database Run the following command to create a database: ```bash theme={null} pscale database create --region ``` * **DATABASE\_NAME** — Your database name can contain lowercase, alphanumeric characters, or underscores. We allow dashes, but don't recommend them, as they may need to be escaped in some instances. * **REGION\_SLUG** — For the lowest latency, choose the region closest to you or your application's hosting location. You can find our regions and their slugs on the [Regions page](/docs/vitess/regions#available-regions). > [!NOTE] > > If you do not specify a region, your database will automatically be deployed to **US East — Northern Virginia**. Your database is created with an [**initial branch**](/docs/vitess/schema-changes/branching), `main`, which you will use to apply a schema change and insert data. While this is just the first branch we create for you, you can always create new branches (isolated copies of the production schema) off of production to use for development. ### Add a schema to your database To add a schema to your database, you will need to connect to MySQL, so [make sure you `mysql-client` installed](/docs/cli/planetscale-environment-setup#setup-overview). **Step 1** Run the following command: ```bash theme={null} pscale shell main ``` You are now connected to your `main` branch and can run MySQL queries against it. **Step 2** Create the `categories` and `products` tables by running the following: ```sql theme={null} CREATE TABLE categories ( id int NOT NULL AUTO_INCREMENT PRIMARY KEY, name varchar(255) NOT NULL ); ``` ```sql theme={null} CREATE TABLE products ( id int NOT NULL AUTO_INCREMENT PRIMARY KEY, name varchar(255) NOT NULL, image_url varchar(255), category_id INT, KEY category_id_idx (category_id) ); ``` > [!NOTE] > > If you want to make schema changes containing foreign key constraints, enable [foreign key constraints](/docs/vitess/foreign-key-constraints) support in your database settings page. **Step 3** You can confirm that the table has been added by running: ```sql theme={null} SHOW TABLES; ``` **Step 4** To see the table schemas, run: ```sql theme={null} DESCRIBE categories; ``` ```sql theme={null} DESCRIBE products; ``` ### Insert data into your database Now that you have your schema set up, let's insert some data. **Step 1** Run the following commands to add one entry to each table: ```sql theme={null} INSERT INTO `categories` (name) VALUES ('Office supplies'); ``` ```sql theme={null} INSERT INTO `products` (name, image_url, category_id) VALUES ('Ballpoint pen', 'https://example.com/500x500', '1'); ``` **Step 2** You can confirm the data has been added with: ```sql theme={null} SELECT * FROM products; ``` ```sql theme={null} SELECT * FROM categories; ``` **Step 3** Exit the shell by typing `exit`. ### Enable safe migrations All of the work you've done so far has been on a default production branch, `main`, that was automatically created when you created the database. A production branch is a highly available database branch that includes an additional replica. It also has the option to enable [safe migrations](/docs/vitess/schema-changes/safe-migrations), which enables non-blocking schema changes and can protect your database from accidental schema changes. [Safe migrations](/docs/vitess/schema-changes/safe-migrations) is an optional, but highly recommended, feature that adds an additional layer of protection to your branch by preventing accidental schema modifications and enabling no-downtime schema changes. With safe migrations enabled, any DDL issued directly to the branch will not be accepted. Instead, changes must be made using the PlanetScale flow, where deploy requests are used to safely merge changes in a collaborative environment. To enable safe migrations on your branch, run: ```bash theme={null} pscale branch safe-migrations enable main ``` The `main` branch now contains the `categories` and `products` tables you created, along with the data you inserted. In addition, it is highly available with an additional replica, and is enabled for zero-downtime migrations with [safe migrations](/docs/vitess/schema-changes/safe-migrations). ### What's next? Now that you've created a database, applied schema changes, added data, and enabled safe migrations, it's time to connect to your application. You can use our [Connect Any Application tutorial](/docs/vitess/tutorials/connect-any-application) for a general step-by-step approach, one of our language-specific guides, or head straight to our [Connection Strings documentation](/docs/vitess/connecting/connection-strings) for more information about creating connection strings. When you want to continue development on your database: **Step 1** [Create a new branch](/docs/vitess/schema-changes/branching) off of your production branch **Step 2** Go through the same process described in this doc to make schema changes **Step 3** [Create a deploy request](/docs/vitess/schema-changes/deploy-requests) to merge the changes into your production branch > [!NOTE] > > When you branch off of a production branch, your development branch will have the same schema as production, but it > **will not** copy over any data from the production database. We suggest seeding development branches with mock > data. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # PlanetScale serverless driver for JavaScript Source: https://planetscale.com/docs/vitess/tutorials/planetscale-serverless-driver ## Why use the PlanetScale serverless driver Before learning how to use the PlanetScale serverless driver for JavaScript, it’s worth understanding why you should use this over other MySQL packages available in the directory. Some serverless and edge function hosts do not permit arbitrary outbound TCP connections, which is how many MySQL clients operate. Using the PlanetScale serverless driver for JavaScript provides a means of accessing your database and executing queries over an HTTP connection, which is generally not blocked by cloud providers. If you encounter issues using MySQL packages with PlanetScale, use the serverless driver instead. > [!NOTE] > > Be sure to check out our [F1 Championship Stats demo application](https://github.com/planetscale/f1-championship-stats) to find examples for use with Cloudflare Workers, Vercel Edge Functions, and Netlify Edge Functions. ## Add and use the PlanetScale serverless driver for JavaScript to your project To install the package in your project, run the following install command: ```bash theme={null} npm install @planetscale/database ``` ### Connect to the database The first step to using the PlanetScale serverless driver for JavaScript is to connect to your database. You can get your connection string in the PlanetScale dashboard by clicking on your database, clicking "**Connect**", and selecting `database-js` from the "Select your language or framework" section. ![Database-js selection priority](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/planetscale-serverless-driver/connect-serverless-credentials-database-js.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=5c2e3614d9bbca1970191da46306d547) Scroll down to the env variables. You'll need this to connect to your database. Use the `connect` function to create the connection and return it to an object. ```js theme={null} const config = { host: 'aws.connect.psdb.cloud', username: '', password: '' } const conn = await connect(config) ``` ### Executing queries To execute a query, use the `execute` function of the connection object, with the query passed as the first parameter. ```js theme={null} const results = await conn.execute('SELECT * FROM hotels') ``` Here is the content of the `results` object from the `SELECT` statement: ```js expandable theme={null} { headers: [ 'id', 'name', 'address', 'stars' ], types: { id: 'UINT32', name: 'VARCHAR', address: 'VARCHAR', stars: 'FLOAT32' }, rows: [ { id: 1, name: 'Four Seasons Resort Jackson hole', address: '7680 Granite Loop Rd, Teton Village, WY 83025', stars: 4.7 }, { id: 2, name: 'The Galt House', address: '140 N Fourth St, Louisville, KY 40202', stars: 4 }, // ...results removed for brevity ], rowsAffected: null, insertId: null, error: null, size: 5, statement: 'SELECT * FROM hotels', time: 136 } ``` For parameterized queries, there are two ways in which to pass data to the query. The first is by the order in which they appear in the query. The first step is to add a `?` in the specific locations you want the parameters passed into. ```js theme={null} const query = 'INSERT INTO hotels (`name`, `address`, `stars`) VALUES (?, ?, ?)' ``` Then you can pass your parameters as an array of values. The driver package will replace the `?` entries in the query with the values passed in the array, in the order in which they were placed. ```js theme={null} const params = ['The Galt House', '140 N Fourth St, Louisville, KY 40202', 4.2] const results = await conn.execute(query, params) ``` Here is the content of the `results` object for the `INSERT` statement: ```js theme={null} { headers: [], types: {}, rows: [], rowsAffected: 1, insertId: '6', error: null, size: 0, statement: "INSERT INTO hotels (`name`, `address`, `stars`) VALUES ('Montage Kapalua Bay 2', '1 Bay Dr, Lahaina, HI 96761', 4)", time: 102 } ``` Alternately, you can name your parameters using the `:param_name` format. ```js theme={null} const query = 'INSERT INTO hotels (`name`, `address`, `stars`) VALUES (:name, :address, :stars)' const params = { name: 'The Galt House', address: '140 N Fourth St, Louisville, KY 40202', stars: 4.2 } const results = await conn.execute(query, params) ``` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Node.js example using the PlanetScale serverless driver Source: https://planetscale.com/docs/vitess/tutorials/planetscale-serverless-driver-node-example This guide will cover how to use the provided Node.js sample application using the PlanetScale serverless driver for JavaScript. ## Overview > [!NOTE] > > This guide will be using VS Code as the IDE, but you may use your preferred IDE. ## Use the sample repository We offer a sample repository that can be used as an educational resource. It is an Express API that can be run locally with sample `SELECT`, `INSERT`, `UPDATE`, and `DELETE` statements mapped to the proper API endpoints. To follow along, you’ll need the following: * A PlanetScale account, as well as knowing how to create a database. * The PlanetScale CLI is installed on your computer, which will be used to seed data. Start by creating a database in PlanetScale by clicking **"New database"** > **"Create new database"**. ![How to create a new database. priority](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/planetscale-serverless-driver-node-example/how-to-create-a-new-database-2.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=beb148475e5dfe5db5f24934b11ad022) Name the database `travel_db`. Click **"Create database"**. Wait for the database to finish initializing before moving on. ![The travel\_db initializing.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/planetscale-serverless-driver-node-example/initializing.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=5fc78b39b71f92b2fe184b2478fb5a84) Generate a set of credentials by clicking the **"Connect"** button. ![The Connect button in the PlanetScale dashboard.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/planetscale-serverless-driver-node-example/the-connect-button-in-the-planetscale-dashboard.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=2fd16015214563dbbc2b6b06a1c2c402) Copy your password credentials first: ![The password details.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/planetscale-serverless-driver-node-example/the-serverlessjs-connect-modal.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=f73a60ddc00f0cb91e3cae3555e92e05) Scroll down and select **"database-js"** from the "Select your language or framework" options. Copy the text from the **".env"** section, as we'll be putting this in the project after it's pulled down from GitHub. ![The password env details.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/planetscale-serverless-driver-node-example/the-serverlessjs-connect-modal-env.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=a0eedb7b2b936dbf75a9d8bca97148b1) On your workstation, open a terminal and clone the repository to your computer by running the following command: ```bash theme={null} git clone https://github.com/planetscale/database-js-starter ``` Navigate to the `scripts` folder and run the `seed_database.sh` script to populate a small database simulating a travel agency. ```bash theme={null} cd database-js-starter/scripts ./seed-database.sh ``` > [!NOTE] > > If you are using Windows, run this command through the [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/) Create a new file named `.env` in the root of the project and paste in the sample provided from PlanetScale that you copied earlier. To run the project, run the following commands from the root of the project. ```bash theme={null} npm install npm start ``` If the project is running properly, you should receive a message stating that the API is running. The `tests.http` file is designed to work with the [VS Code Rest Client plugin](https://marketplace.visualstudio.com/items?itemName=humao.rest-client), but can be used as a reference when testing with the tool of your choosing. If you are using the plugin, you may click the **"Send request"** button that appears above each request to see the API in action. ![An example of a POST request to the sample project.](https://mintcdn.com/planetscale-2/AJPY38bILe2zenXX/images/assets/docs/tutorials/planetscale-serverless-driver-node-example/an-example-of-a-post-request-to-the-sample-project.png?fit=max\&auto=format\&n=AJPY38bILe2zenXX\&q=85\&s=d1eeca2575f13571f643ef7af27003e8) If you check the terminal where the API was started, the response from the `execute` function is logged out for review. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Using the PlanetScale serverless driver with Prisma Source: https://planetscale.com/docs/vitess/tutorials/planetscale-serverless-driver-prisma-example This document outlines how you can use the [PlanetScale serverless driver](/docs/vitess/tutorials/planetscale-serverless-driver) along with Prisma in your application. ## Set up To get started: **Step 1** Install the Prisma driver adapter for PlanetScale (`@prisma/adapter-planetscale`) and PlanetScale serverless driver (`@planetscale/database`) packages: ```shell theme={null} npm install @prisma/adapter-planetscale @planetscale/database ``` **Step 2** Update your `prisma/schema.prisma` file: ```javascript theme={null} // schema.prisma generator client { provider = "prisma-client" output = "../generated/prisma" } datasource db { provider = "mysql" url = env("DATABASE_URL") relationMode = "prisma" } ``` > [!NOTE] > > Ensure you update the host value in your connection string to `aws.connect.psdb.cloud`. You can learn more about this [here](/docs/vitess/tutorials/planetscale-serverless-driver#add-and-use-the-planetscale-serverless-driver-for-javascript-to-your-project). **Step 3** Generate Prisma Client: ```shell theme={null} npx prisma generate ``` **Step 4** Update your Prisma Client instance to use the PlanetScale serverless driver: ```javascript expandable theme={null} import "dotenv/config" import { PrismaPlanetScale } from '@prisma/adapter-planetscale' import { PrismaClient } from './generated/prisma/client.js' const adapter = new PrismaPlanetScale({ url: process.env.DATABASE_URL }) const prisma = new PrismaClient({ adapter }) async function main() { const posts = await prisma.post.findMany() console.log(posts) } ``` You can then use Prisma Client as you usually would with auto-completion and full type-safety. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Prisma best practices Source: https://planetscale.com/docs/vitess/tutorials/prisma-best-practices This document provides various best practices for getting the most out of Prisma, a next-generation ORM for Node.js and TypeScript, and PlanetScale. It also includes relevant links to Prisma's documentation. ## Referential actions and integrity with Prisma and PlanetScale When using Prisma with PlanetScale, you need to make sure to set `relationMode` to `prisma` in your Prisma schema: ```js theme={null} datasource db { provider = "mysql" url = env("DATABASE_URL") relationMode = "prisma" } ``` > [!NOTE] > > In Prisma `4.5.0`, `referentialIntegrity` changed to `relationMode` and generally became available in `4.7.0`. The `prisma` relation mode emulates some foreign key constraints and referential actions for each Prisma Client query to maintain referential integrity, using some additional database queries and logic. Read more about [Relation mode in Prisma's documentation](https://www.prisma.io/docs/orm/prisma-schema/data-model/relations/relation-mode). ### Creating an index for relation scalar fields When a Prisma client uses the `foreignKeys` relation mode, which does not work with PlanetScale, the database implicitly creates an index for the foreign key columns. Therefore, it is recommended that you create an index for your relation scalar fields with the `@@index` attribute (or the `@unique`, `@@unique` or `@@id` attributes, if applicable) when using `prisma` relation mode with PlanetScale. ```js theme={null} model Post { id Int @id userId Int user User @relation(fields: [userId], references: [id]) @@index([userId]) } ``` If you do not add the index, you might notice that some of your queries are slow or are performing full table scans and reading a lot of data. When you run `prisma format` or `prisma validate`, it will warn you about a missing index where you are using foreign key constraints. If you want to learn more about MySQL indexes, check out the [MySQL for Developers course section on indexes](https://planetscale.com/learn/courses/mysql-for-developers/indexes/introduction-to-indexes). ## Migration workflows using `prisma db push` With Prisma, there are two ways to apply schema changes to your database: `prisma migrate` and `prisma db push`. We recommend `prisma db push` over `prisma migrate dev` for the following reasons: PlanetScale automatically provides built-in [Online Schema Change tools](/docs/vitess/schema-changes/how-online-schema-change-tools-work) when you merge a deploy request and prevents [blocking schema changes](/docs/vitess/schema-changes) that can lead to downtime. This differs from the typical Prisma workflow, which uses `prisma migrate` to generate SQL migrations for you based on changes in your Prisma schema. When using PlanetScale with Prisma, the responsibility of applying the changes is on the PlanetScale side. Therefore, there is little value to using `prisma migrate` with PlanetScale. Also, the migrations table created when `prisma migrate` runs can be misleading since PlanetScale does the actual migration when the deploy request is merged, not when `prisma migrate` is run, which only updates the schema in the development database branch. You can still see the history of your schema changes in PlanetScale. If you want to read more about `prisma db push`, see the [Prisma documentation on making schema changes with `prisma db push`](https://www.prisma.io/docs/orm/overview/databases/planetscale). ## Connection management > [!NOTE] > > In Prisma v7, connection pooling is managed by the driver adapter (e.g. `@prisma/adapter-mariadb` or `@prisma/adapter-planetscale`) rather than through URL parameters like `connection_limit` and `pool_timeout`. If you are using Prisma v7, configure pool settings directly in your adapter options. See the [Prisma connection pool guide](https://www.prisma.io/docs/orm/prisma-client/setup-and-configuration/databases-connections/connection-pool) for details. When using Prisma with PlanetScale, you might encounter some specific error messages. It might seem like your PlanetScale database is down, but there are other reasons why it might appear this way. For both errors, if your serverless function or application servers and database are not in the same region, this can contribute to latency problems. If possible, try to move them closer together to decrease physical latency. Here are some of the common error messages with their possible causes and solutions: ### Prisma error P1001 ```bash theme={null} "Can't reach database server at {database_host}:{database_port} Please make sure your database server is running at {database_host}:{database_port}." ``` **Possible cause:** * The Prisma Client did not establish the connection within the `connect_timeout`. There are many possible reasons for this to occur, two of the common examples we see are DNS resolution issues and network latency. **Possible solutions:** * Increase the `connect_timeout` in your `DATABASE_URL`. Example `DATABASE_URL`: `mysql://USER:PASSWORD@HOST:PORT/DATABASE?connect_timeout=30` The default is `5`. The `connect_timeout` is the maximum number of seconds to wait for a new connection to be opened, `0` means no timeout. We suggest trying a higher number around `30` seconds if you have this issue. See the [Prisma connection URL argument documentation](https://www.prisma.io/docs/orm/reference/connection-urls) for more information. ### Prisma error P2024 ``` "Timed out fetching a new connection from the connection pool. (More info: http://pris.ly/d/connection-pool (Current connection pool timeout: {timeout}, connection limit: {connection_limit})" ``` **Possible cause:** * The connection is established, but the Prisma query engine is not able to process a query in the queue before the time limit. **Possible solutions:** * Increase the pool size by increasing the `connection_limit`. Example `DATABASE_URL`: `mysql://USER:PASSWORD@HOST:PORT/DATABASE?connection_limit=10` The default is `num_cpus * 2 + 1`. The `connection_limit` is the maximum size of the connection pool for each instance of the Prisma Client. `connection_limit` is not the total for all of your application servers or serverless functions. While PlanetScale can handle hundreds of thousands of connections at a time, we recommend incrementally increasing this number to tune your Prisma Client. See the [Prisma documentation on optimizing the connection pool size](https://www.prisma.io/docs/orm/prisma-client/setup-and-configuration/databases-connections#optimizing-the-connection-pool) for more info. * Increase the `pool_timeout`. Example `DATABASE_URL`: `mysql://USER:PASSWORD@HOST:PORT/DATABASE?pool_timeout=30` The default is `10`. The `pool_timeout` is the maximum number of seconds to wait for a new connection from the pool, `0` means no timeout. We recommend increasing this only after you've tuned the `connection_limit`. See the [Prisma documentation on optimizing the connection pool timeout](https://www.prisma.io/docs/orm/prisma-client/setup-and-configuration/databases-connections#optimizing-the-connection-pool) for more info. * Decrease query duration, which might include asking for less data in the query or improving performance with an index or other methods. Start by looking at your [Insights](/docs/vitess/monitoring/query-insights) page within your PlanetScale database to see which queries might be performing poorly. Lastly, this [discussion topic response on Prisma connection pooling](https://github.com/planetscale/discussion/discussions/188#discussioncomment-3808093) can be helpful if you are experiencing these errors. ## Other resources: > **[Using PlanetScale with Prisma](/docs/vitess/tutorials/using-planetscale-with-prisma)** > **[Prisma quickstart for adding Prisma to an existing project](https://www.prisma.io/docs/getting-started/quickstart)** > **[Prisma document on connection management](https://www.prisma.io/docs/orm/prisma-client/setup-and-configuration/databases-connections/connection-management)** > **[Prisma document on how the connection pool works in Prisma](https://www.prisma.io/docs/orm/prisma-client/setup-and-configuration/databases-connections/connection-pool#how-the-connection-pool-works)** > **[Video: Prisma & PlanetScale Best Practices](https://youtu.be/iaHt5_hg44c)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Using the PlanetScale serverless driver with Prisma Source: https://planetscale.com/docs/vitess/tutorials/prisma-example This document outlines how you can use the [PlanetScale serverless driver](/docs/vitess/tutorials/planetscale-serverless-driver) along with Prisma in your application. ## Set up To get started: **Step 1** Install the Prisma driver adapter for PlanetScale (`@prisma/adapter-planetscale`) and PlanetScale serverless driver (`@planetscale/database`) packages: ```shell theme={null} npm install @prisma/adapter-planetscale @planetscale/database ``` **Step 2** Update your `prisma/schema.prisma` file: ```javascript theme={null} // schema.prisma generator client { provider = "prisma-client" output = "../generated/prisma" } datasource db { provider = "mysql" url = env("DATABASE_URL") relationMode = "prisma" } ``` > [!NOTE] > > Ensure you update the host value in your connection string to `aws.connect.psdb.cloud`. You can learn more about this [here](/docs/vitess/tutorials/planetscale-serverless-driver#add-and-use-the-planetscale-serverless-driver-for-javascript-to-your-project). **Step 3** Generate Prisma Client: ```shell theme={null} npx prisma generate ``` **Step 4** Update your Prisma Client instance to use the PlanetScale serverless driver: ```javascript expandable theme={null} import "dotenv/config" import { PrismaPlanetScale } from '@prisma/adapter-planetscale' import { PrismaClient } from './generated/prisma/client.js' const adapter = new PrismaPlanetScale({ url: process.env.DATABASE_URL }) const prisma = new PrismaClient({ adapter }) async function main() { const posts = await prisma.post.findMany() console.log(posts) } ``` You can then use Prisma Client as you usually would with auto-completion and full type-safety. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Prometheus Source: https://planetscale.com/docs/vitess/tutorials/prometheus PlanetScale exposes Prometheus-compatible metrics endpoints for scraping metrics about your database branches. This, along with our API-driven service discovery, allow you to automatically get in-depth information about all of the databases in your organization. In order to collect and store these, you will need to use Prometheus or a Prometheus-compatible metrics engine (such as VictoriaMetrics) that is capable of using the [HTTP SD](https://prometheus.io/docs/prometheus/latest/http_sd/) protocol. ## Prerequisites This document assumes we'll be configuring a Prometheus 3.x instance via a configuration file running on our local machine. If you are using managed Prometheus via AWS, GCP or another provider, you will have to deploy Prometheus to scrape and forward metrics via `remote_write`, as these services do not support scraping metrics. ## Getting Started First, provision a new PlanetScale [Service token](/docs/api/reference/service-tokens) in your Organization settings. Make sure to save the ID and token, as they will not be visible after they've been generated. When that's created, grant the token `read_metrics_endpoints` permissions and click "Save permissions". Your token should look like the following: ![Service Token configuration for Metrics Exporting](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-service-token-configuration.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=7154912fcc404c404843cdbed160db0f) ## Configuring Prometheus Now that we have a Service Token, we can add a scrape configuration for your PlanetScale organization. A minimal Prometheus configuration should look like the following: ```yaml theme={null} scrape_configs: - job_name: "${ORG}" http_sd_configs: - url: https://api.planetscale.com/v1/organizations/${ORG}/metrics authorization: type: "token" credentials: "${TOKEN_ID}:${TOKEN}" refresh_interval: 10m ``` Fill in your organization name in the `job_name` and `url`, and place the Service Token and ID that we created in the previous step for the credentials. Save this file to `prometheus.yml` in your working directory. ## Start Prometheus Run Prometheus pointed at this configuration file: ```bash theme={null} $ prometheus --config.file=prometheus.yml ``` By default, Prometheus will listen at `0.0.0.0:9090`, which means you can access it in your browser at [http://127.0.0.1:9090](http://127.0.0.1:9090). ### Validating Service Discovery First, let's make sure that Prometheus is properly querying the PlanetScale API for the right branches. If you go to `http://127.0.0.1:9090/service-discovery` you should see the job that we created earlier, with all of your branches listed under `Discovered labels`. In this example, our organization is called `nick`, so it looks like the following: ![Prometheus Target List](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-prometheus-targets.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=96f64a28bdbaac27e83310a751941169) Here, I have two branches that have been discovered. I can confirm that this matches what's in my organization: ```bash theme={null} $ pscale branch list test --org nick ID NAME PARENT BRANCH REGION PRODUCTION SAFE MIGRATIONS READY CREATED AT UPDATED AT -------------- ------------ --------------- --------- ------------ ----------------- ------- ------------- --------------- 7wxuxewx4l0p main n/a us-east Yes Yes Yes 2 years ago 7 minutes ago 6o0rr27785fl partitions main us-east No No Yes 1 month ago 9 minutes ago ``` Now, if I go to my list of targets I should see each branch as an Endpoint: ![Prometheus Endpoint List](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-prometheus-endpoints.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=6830ae6e1a43d6667ccde7f6784d8e5c) This screenshot shows that they're being correctly scraped, and I can start to query my Prometheus instance. ## Querying Prometheus Now that we're collecting metrics for my branches, our [reference guide](/docs/vitess/integrations/prometheus-metrics) has a list of everything that we export. If I want to see how many `vtgate` pods are running per AZ for my branch, I can query: ``` planetscale_vtgate_total_pods{planetscale_database_branch_id="7wxuxewx4l0p"} ``` Make sure the graph is set to stacked, and it should look like this: ![Querying Prometheus for VTGate Count](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-prometheus-querying.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=5271cb0215290d804c5c71d29cea7759) ## Next Steps If you keep this Prometheus instance running, it will collect metrics every 30 seconds, and refresh the list of branches every 10 minutes. For more information, see: * [Metrics reference](/docs/vitess/integrations/prometheus-metrics) for a list of metrics we expose * [Grafana and Prometheus](/docs/vitess/tutorials/prometheus-metrics-grafana) tutorial for using PlanetScale's provided dashboard to visualize these metrics in Grafana. * [Sending metrics to New Relic](/docs/vitess/tutorials/prometheus-metrics-newrelic) tutorial for using Prometheus to forward metrics to New Relic. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sending Prometheus Metrics to Datadog Source: https://planetscale.com/docs/vitess/tutorials/prometheus-metrics-datadog If you're looking for more metrics than PlanetScale's native Datadog integration provides, this tutorial will show how to configure your [Datadog agent](https://docs.datadoghq.com/agent/) to scrape PlanetScale's [Prometheus infrastructure](/docs/vitess/integrations/prometheus) automatically, allowing you to collect detailed metrics for all of your PlanetScale branches. ## Overview In this tutorial, we'll assume that you have a Datadog Agent Version 7 running. For more information on what the Datadog Agent is and how to install it, start with the [Datadog Agent documentation](https://docs.datadoghq.com/getting_started/agent/). For the purposes of this guide, we'll be using a Datadog agent running with the recommended installation steps on a Linux system. ## Prerequisites You'll need a working Datadog agent and access to add a [Custom Agent Check](https://docs.datadoghq.com/developers/custom_checks/) to that instance. This may require `root` or `sudo` access on the machine running the Datadog agent. You'll also need a [Service token](/docs/api/reference/service-tokens) in your Organization, with the `read_metrics_endpoints` permission granted. ## Adding the Plugin to the Datadog Agent Go to [https://github.com/planetscale/planetscale-datadog](https://github.com/planetscale/planetscale-datadog), which is the repository that has our custom OpenMetrics Check. Place the unedited `planetscale.py` in the `checks.d` directory of your Datadog Agent. * On Linux, that is `/etc/datadog-agent/checks.d/` * On macOS, that is `/opt/datadog-agent/etc/checks.d/` Make sure that it belongs to the appropriate user. If you're using the recommended Linux installation steps, it will have created a `dd-agent` user: ``` $ pwd /etc/datadog-agent/checks.d $ ls -al planetscale.py -rw-r--r-- 1 dd-agent dd-agent 9261 Apr 2 22:54 planetscale.py ``` This file is owned by the `dd-agent` user and group in the `/etc/datadog-agent/checks.d` directory. If you're on macOS, it will depend on whether you installed the agent as a 'Single User Agent' or a 'Systemwide Agent'. If you picked Single User, there should be no additional permission changes needed. If you installed it as a Systemwide agent, make sure the user and group you installed the agent with as ownership of the file. ## Configuring the Datadog Agent Now that we have the plugin installed, we need to configure it. In the `conf.d` directory of the Datadog agent take the `conf.d/planetscale.yaml.example` file and edit it with your organization name and Service Token information. It should look like this: ```bash expandable theme={null} instances: - planetscale_organization: 'nick' # Required: Your PlanetScale organization ID ps_service_token_id: '${TOKEN_ID}' # Required: Your PlanetScale Service Token ID ps_service_token_secret: '${TOKEN}' # Required: Your PlanetScale Service Token Secret. Consider using Datadog secrets management: https://docs.datadoghq.com/agent/guide/secrets-management/ namespace: 'planetscale' # Required: Namespace for the metrics metrics: # Required: List of metrics to collect. Use mapping for renaming/type overrides. - planetscale_vtgate_queries_duration: vtgate_query_duration min_collection_interval: 60 send_distribution_buckets: true collect_counters_with_distributions: true ``` This configures the integration to look for all of the branches in the `"nick"` PlanetScale organization, only collect the `planetscale_vtgate_queries_duration` metric, which it will rename `vtgate_query_duration` and put it inside of the `planetscale` namespace. ### Collecting all metrics If you want to collect all available PlanetScale metrics instead of specifying individual metrics, you can use a wildcard pattern: ```yaml theme={null} instances: - planetscale_organization: 'your-org' ps_service_token_id: '${TOKEN_ID}' ps_service_token_secret: '${TOKEN}' namespace: 'planetscale' metrics: [".*"] min_collection_interval: 60 send_distribution_buckets: true collect_counters_with_distributions: true ``` Using `'.*'` in the metrics list tells the Datadog agent to scrape all metrics exposed by PlanetScale. This is useful when you want comprehensive monitoring without manually maintaining a list of specific metrics. Save the file at `planetscale.yaml`, making sure to double check permissions: ``` $ pwd /etc/datadog-agent/conf.d $ ls -al planetscale.yaml -rw-r--r-- 1 root root 1518 Apr 2 22:57 planetscale.yaml ``` ## Restart the Datadog Agent Now that this is configured and installed, restart the Agent: ``` $ sudo systemctl restart datadog-agent ``` ## Validating the PlanetScale Plugin Now that the Datadog Agent is running the PlanetScale plugin, metrics should start flowing into Datadog within a couple of minutes. To validate, we can ask the Datadog Agent: ``` sudo -u dd-agent -- datadog-agent check planetscale ``` If the plugin is installed successfuly, this should output the scrape targets for your branches, as well as metadata about when it was last run and how many metrics were emitted: ```bash expandable theme={null} $ sudo -u dd-agent -- datadog-agent check planetscale === Service Checks === [ { "check": "planetscale.api.can_connect", "host_name": "ubuntu", "timestamp": 1743638192, "status": 0, "message": "", "tags": [ "planetscale_org:nick" ] }, { "check": "planetscale.prometheus.health", "host_name": "ubuntu", "timestamp": 1743638192, "status": 0, "message": "", "tags": [ "endpoint:https://metrics.psdb.cloud/metrics/branch/7wxuxewx4l0p?..." ] }, { "check": "planetscale.prometheus.health", "host_name": "ubuntu", "timestamp": 1743638192, "status": 0, "message": "", "tags": [ "endpoint:https://metrics.psdb.cloud/metrics/branch/6o0rr27785fl?..." ] } ] Running Checks ============== planetscale (unversioned) ------------------------- Instance ID: planetscale:planetscale:8d4d64f696d967be [OK] Configuration Source: file:/etc/datadog-agent/conf.d/planetscale.yaml Total Runs: 1 Metric Samples: Last Run: 0, Total: 0 Events: Last Run: 0, Total: 0 Service Checks: Last Run: 3, Total: 3 Histogram Buckets: Last Run: 77, Total: 77 Average Execution Time : 809ms Last Execution Date : 2025-04-02 23:56:32 UTC (1743638192000) Last Successful Execution Date : 2025-04-02 23:56:32 UTC (1743638192000) Metadata ======== config.hash: planetscale:planetscale:8d4d64f696d967be config.provider: file ``` The Service Checks show that it has successfully connected to the PlanetScale API to request information about how to scrape for the branches in my organization, and it has successfully scraped both of what it discovered. We can also see that it successfully executed at `2025-04-02 23:56:32 UTC` and produced 77 Histogram Buckets. ## Adding Metrics In our earlier configuration, we only added one metric. For a complete list of what PlanetScale exposes, please take a look at our [Metrics Reference Documentation](/docs/vitess/integrations/prometheus-metrics). Note that the Datadog agent [normalizes metrics with certain suffixes starting in v7.32.0](https://github.com/DataDog/integrations-core/blob/master/openmetrics/README.md): > Starting in Datadog Agent v7.32.0, in adherence to the OpenMetrics specification standard, counter names ending in \_total must be specified without the \_total suffix. For example, to collect promhttp\_metric\_handler\_requests\_total, specify the metric name promhttp\_metric\_handler\_requests. This submits to Datadog the metric name appended with .count, promhttp\_metric\_handler\_requests.count. This means that to scrape a metric such as `planetscale_mysql_bytes_received_total`, you would configure the Datadog agent for `planetscale_mysql_bytes_received`. If I want to collect additional metrics, I can add them to the list: ``` metrics: # Required: List of metrics to collect. Use mapping for renaming/type overrides. - planetscale_vtgate_queries_duration: vtgate_query_duration - planetscale_edge_active_connections: active_connections ``` Then, restart the Datadog Agent: ``` $ sudo systemctl restart datadog-agent ``` If I check the status of the PlanetScale Plugin, I can see our last run added a Metric Sample: ```bash theme={null} Running Checks ============== planetscale (unversioned) ------------------------- Instance ID: planetscale:planetscale:fde586b60a54a38f [OK] Configuration Source: file:/etc/datadog-agent/conf.d/planetscale.yaml Total Runs: 1 Metric Samples: Last Run: 1, Total: 1 Events: Last Run: 0, Total: 0 Service Checks: Last Run: 3, Total: 3 Histogram Buckets: Last Run: 77, Total: 77 Average Execution Time : 826ms Last Execution Date : 2025-04-03 00:01:45 UTC (1743638505000) Last Successful Execution Date : 2025-04-03 00:01:45 UTC (1743638505000) ``` In the Datadog UI, I can see data for the `planetscale.active_connections` metric: ![Datadog Connections Metric](https://mintcdn.com/planetscale-2/g0AZZQkXmTSBYuKj/images/prometheus-datadog-graph.png?fit=max\&auto=format\&n=g0AZZQkXmTSBYuKj\&q=85\&s=6bb2c1f0801932630df74a2853140785) ## What's Next? Now that you're sending a couple of metrics from PlanetScale to Datadog, take a look at our [full list](/docs/vitess/integrations/prometheus-metrics) and start building dashboards! ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Grafana Dashboard for PlanetScale Branches Source: https://planetscale.com/docs/vitess/tutorials/prometheus-metrics-grafana In this tutorial, you'll learn how to set up Grafana and connect it to a Prometheus instance to see metrics about your PlanetScale database. ## Introduction This guide requires that you've set up a Prometheus instance from our documentation. If you're already running Grafana in production and you're just looking for our standard dashboard template, you can find it [on GitHub](https://github.com/planetscale/grafana-dashboard). ## Install Grafana Grafana's [installation documentation](https://grafana.com/docs/grafana/latest/setup-grafana/installation/) contains information for their supported platforms. For this guide, we'll be setting this up locally on a macOS machine. If you're using a hosted Grafana option such as [Grafana Cloud](https://grafana.com/products/cloud/) or [AWS Managed Grafana](https://aws.amazon.com/grafana/) you can skip this step. On macOS, Grafana is availabile via [homebrew](https://brew.sh/), and I can install it with: ```bash theme={null} $ brew install grafana ``` This will download and install Grafana, and I can start it with: ```bash theme={null} $ brew services start grafana ``` When that succeeds, I can go to `http://localhost:3000/` and I should see the Grafana welcome page: ![Grafana Welcome Page](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-grafana-welcome.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=d8fc1003e38cf51f2abe3d7b06aceec8) The default username and password for a new install is `admin` and `admin`. Grafana will ask you to change the password the first time you log in, please pick something more secure than `admin`. ### Adding a Prometheus Endpoint You can skip this step as well if you're already running a managed Prometheus or have added your datasource to Grafana already. If you're running Prometheus locally, you'll need to add that as a datasource. To do this: **Step 1** Open the menu in the top left and click "Connections" **Step 2** Search for "Prometheus" and pick the plain "Prometheus" option **Step 3** Click "Add new data source" in the top right of the page Now, you should look see a page that looks like this: ![Grafana Add Datasource](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-add-prometheus-connection.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=c4af401d1b96e7d7b9c8a4a08298560c) You can call this whatever you want, we'll use the following: * Name: "PlanetScale" * Prometheus server URL: `http://localhost:9090/` ![Grafana Prometheus Configuration](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-prometheus-configuration.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=fb82e6048d85fc8bb54a2151ea0c98e4) Because this is running on your local machine, we do not need to use any Authentication or TLS Scroll down to the "Interval behaviour" section and set the "Scrape interval" to `1m`. Finally, scroll to the bottom and click "Save & test". ## Import the PlanetScale Dashboard Now that we have our datasource added, let's import the PlanetScale Dashboard. This is a starter dashboard that PlanetScale has produced which shows an overview of your branch with the metrics that we expose. From the Grafana homepage, go to the top left menu and pick "Dashboards". In the top right, click "New" and then Import": PlanetScale maintains the latest version of the dashboard located here: [https://github.com/planetscale/grafana-dashboard/blob/main/overview.json](https://github.com/planetscale/grafana-dashboard/blob/main/overview.json) Download this file to your computer, and then click "Upload dashboard JSON file". Find the JSON file you downloaded in the previous step, and configure it with the Prometheus datasource that we added in an earlier: ![Grafana Prometheus Configuration](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-prometheus-configuration.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=fb82e6048d85fc8bb54a2151ea0c98e4) Click 'Import' and you should be directed to the dashboard, configured to query your local Prometheus with the data it's been scraping! ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Sending Prometheus Metrics to New Relic Source: https://planetscale.com/docs/vitess/tutorials/prometheus-metrics-newrelic If you're looking for your PlanetScale database metrics in your New Relic account, this tutorial will show how to configure a [Prometheus](https://prometheus.io/) instance to scrape PlanetScale's [Prometheus infrastructure](/docs/vitess/integrations/prometheus) automatically, allowing you to collect detailed metrics for all of your PlanetScale branches. While this tutorial is written for New Relic, using Prometheus' remote write is a common pattern for sending metrics to [AWS Managed Prometheus](https://aws.amazon.com/prometheus/), [Google Cloud Managed Service for Prometheus](https://cloud.google.com/stackdriver/docs/managed-prometheus), [Grafana hosted Prometheus](https://grafana.com/products/cloud/metrics/) and many other tools. For more information on Prometheus Remote Write and New Relic, see the [New Relic documentation on sending Prometheus metric data](https://docs.newrelic.com/docs/infrastructure/prometheus-integrations/get-started/send-prometheus-metric-data-new-relic/). ## Overview In this tutorial, we will be using an instance of [Prometheus](https://prometheus.io/) running on a Linux VM to scrape metrics from PlanetScale and then forward them to New Relic using [Remote Write](https://prometheus.io/docs/specs/prw/remote_write_spec/). We will make sure that Prometheus stays running by creating a [Systemd Unit File](https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files). The default configuration we will create will send all PlanetScale metrics to New Relic, and we will cover how to filter to drop certain metrics that may not be desired. In order to proceed, you'll need: * A [New Relic API Key](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/), make sure it is the `Ingest - License` type. * PlanetScale [Service token](/docs/api/reference/service-tokens) with `read_metrics_endpoints` permissions. ## Prometheus Installation First, let's download the latest release of Prometheus and create our user that is going to run it. We'll be using the latest 3.x release from the [GitHub Releases Page](https://github.com/prometheus/prometheus/releases). Create a `prometheus` user: ```bash theme={null} $ sudo useradd -M -U prometheus ``` ```bash theme={null} $ wget https://github.com/prometheus/prometheus/releases/download/v3.2.1/prometheus-3.2.1.linux-amd64.tar.gz $ tar xf prometheus-3.2.1.linux-amd64.tar.gz $ sudo mv prometheus-3.2.1.linux-amd64/ /opt/prometheus $ sudo chown prometheus:prometheus -R /opt/prometheus ``` This has put the Prometheus binary in `/opt/prometheus` along with the example configuration file that we can use. ## Create our Systemd Unit File Now that we have the binary in place, let's setup Systemd to run Prometheus by creating a Unit File in `/etc/systemd/system/prometheus.service` with the following contents: ```ini expandable theme={null} [Unit] Description=Prometheus Agent Documentation=https://prometheus.io/ After=network-online.target [Service] User=prometheus Group=prometheus Restart=on-failure ExecStart=/opt/prometheus/prometheus \ --config.file=/opt/prometheus/prometheus.yml \ --storage.agent.path=/opt/prometheus/data \ --web.listen-address=0.0.0.0:9091 \ --agent [Install] WantedBy=multi-user.target ``` ## Configure Prometheus Now that we've got Prometheus installed and a unit file present, let's configure Prometheus. We will be borrowing some of our configuration from the [Prometheus Guide](/docs/vitess/integrations/prometheus), and adding some New Relic specific configuration. Edit `/opt/prometheus/prometheus.yml` in your editor of choice so that it contains this, making sure to replace your org name, service token information, and New Relic API key: ```yaml theme={null} global: scrape_interval: 1m scrape_configs: - job_name: "${ORG}" http_sd_configs: - url: https://api.planetscale.com/v1/organizations/${ORG}/metrics authorization: type: "token" credentials: "${TOKEN_ID}:${TOKEN}" refresh_interval: 10m remote_write: - url: https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscale authorization: credentials: ${NEW_RELIC_API_TOKEN} ``` This configuration file does the following: * Configures Prometheus to discover scraping endpoints from the PlanetScale API using a service token * Points Prometheus to write the metrics it scrapes from PlanetScale to the New Relic API ## Starting Prometheus Now that we have a Systemd unit file and a configured Prometheus, let's run it! ```bash theme={null} $ sudo systemctl daemon-reload $ sudo systemctl start prometheus.service ``` We can also tell Systemd to run Prometheus when my VM boots: ```bash theme={null} $ sudo systemctl enable prometheus.service ``` Now, let's check to make sure everything is running properly: ```bash expandable theme={null} $ sudo systemctl status prometheus.service ● prometheus.service - Prometheus Agent Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; preset: enabled) Active: active (running) since Fri 2025-04-04 17:36:57 UTC; 38s ago Docs: https://prometheus.io/ Main PID: 745542 (prometheus) Tasks: 9 (limit: 9486) Memory: 21.2M (peak: 21.9M) CPU: 264ms CGroup: /system.slice/prometheus.service └─745542 /opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml --storage.agent.path=/opt/prometheus/data --web.listen-address=0.0.0.0:9091 --agent Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.804Z level=INFO source=main.go:1305 msg=EXT4_SUPER_MAGIC Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.804Z level=INFO source=main.go:1308 msg="Agent WAL storage started" Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.804Z level=INFO source=main.go:1437 msg="Loading configuration file" filename=/opt/prometheus/prometheus.yml Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.805Z level=INFO source=watcher.go:225 msg="Starting WAL watcher" component=remote remote_name=fbe64a url="https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscal> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.805Z level=INFO source=metadata_watcher.go:90 msg="Starting scraped metadata watcher" component=remote remote_name=fbe64a url="https://metric-api.newrelic.com/prometheus/v1/write?prometh> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.805Z level=INFO source=watcher.go:277 msg="Replaying WAL" component=remote remote_name=fbe64a url="https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscale" queu> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.806Z level=INFO source=main.go:1476 msg="updated GOGC" old=100 new=75 Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.806Z level=INFO source=main.go:1486 msg="Completed loading of configuration file" db_storage=791ns remote_storage=610.171µs web_handler=897ns query_engine=301ns scrape=517.175µs scrape_s> Apr 04 17:36:57 ubuntu prometheus[745542]: time=2025-04-04T17:36:57.806Z level=INFO source=main.go:1213 msg="Server is ready to receive web requests." ``` This reports that prometheus is `active (running)`, and I can see the logs showing that it started successfully. Great! ## Querying on New Relic After a couple of minutes, head over to your New Relic dashboard and we can query for your database metrics. First, let's get a list of the database branches in the `nick` organization that I'm using to test: ```bash theme={null} $ pscale branch list test --org nick ID NAME PARENT BRANCH REGION PRODUCTION SAFE MIGRATIONS READY CREATED AT UPDATED AT -------------- ------------ --------------- --------- ------------ ----------------- ------- -------------- ---------------- 7wxuxewx4l0p main n/a us-east Yes No Yes 2 years ago 50 minutes ago 6o0rr27785fl partitions main us-east No No Yes 2 months ago 7 minutes ago ``` For this, we'll use the `7wxuxewx4l0p` branch. Using New Relic's NRQL, we can visualize the memory usage of my VTTablet instances with the following query: ```sql theme={null} FROM Metric SELECT average(planetscale_pods_cpu_util_percentages) WHERE planetscale_database_branch_id = '7wxuxewx4l0p' AND planetscale_component='vttablet' SINCE 30 minutes AGO TIMESERIES FACET planetscale_pod ``` Because my `main` branch is production, we will see the memory usage for my primary and both my replicas over the last 30 minutes: ![New Relic Memory Query](https://mintcdn.com/planetscale-2/qp-ZKr_vKb0iLpeb/images/metrics-new-relic-dashboard.png?fit=max\&auto=format\&n=qp-ZKr_vKb0iLpeb\&q=85\&s=63d2958ae54d53e76643c09f0db60a66) ## Filtering Metrics If you don't want to ingest every metric into New Relic, you can tell Prometheus to drop certain metrics. For more information, see the [New Relic Documentation](https://docs.newrelic.com/docs/infrastructure/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration/#allow-deny). If we adjust our Prometheus configuration that we have in `/opt/prometheus/prometheus.yml` we can instruct Prometheus to drop all metrics unless they match a certain naming convention: ```yaml theme={null} remote_write: - url: https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=planetscale authorization: credentials: ${NEW_RELIC_API_TOKEN} - source_labels: [__name__] regex: "planetscale_pods_(.*)" action: keep ``` If you replace your `remote_write` block with what's above, Prometheus will only forward the timeseries that match the `planetscale_pods_*` name. For a full list of metrics, see our [Metric List](/docs/vitess/integrations/prometheus-metrics). ## What's Next? Now that you have your branch metrics in New Relic, you can create dashboards and alerts for conditions such as high CPU or replication delay. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Using PlanetScale with Prisma Source: https://planetscale.com/docs/vitess/tutorials/using-planetscale-with-prisma [Prisma](https://www.prisma.io/) ORM provides type-safe database access through an intuitive API, eliminating the need to write SQL queries manually. Prisma and PlanetScale together offer a powerful workflow. Prisma handles the object-relational mapping with full type safety and intuitive data modeling, while PlanetScale provides horizontal scaling, branching workflows, and zero-downtime schema changes, making it an ideal stack for building modern, scalable applications. This guide covers how to use Prisma and PlanetScale together, including how to: * Create a database with PlanetScale * Integrate it with Prisma * Use additional features like sharding and the serverless driver If you don't already have an application set up with Prisma, but want to test how it works with PlanetScale, we recommend grabbing a sample application from [Prisma's examples repository](https://github.com/prisma/prisma-examples/tree/latest/orm). ## Set up your PlanetScale database First, set up your PlanetScale database. **Step 1** Sign in to your PlanetScale account **Step 2** Click "Create a database" **Step 3** Give your database a name **Step 4** Select your preferred region **Step 5** Choose cluster and storage size **Step 6** Click "Create database" Your database will deploy with an initial production branch, `main`. ### Create a password After the database is created, you'll be prompted to generate credentials for it. You can come back to this later if needed. **Step 1** Give your password a name or leave the default **Step 2** Select a [password role](/docs/vitess/security/password-roles). We recommend `Admin` for your default password. **Step 3** Click "Create password" **Step 4** Select "Prisma" for "Select your language or framework" **Step 5** Do not navigate away from this page. The next section covers where to paste these credentials in your Prisma app. If you close without copying, you can regenerate new credentials. ## Connect to PlanetScale in your application Next, add your database credentials to your Prisma application. If you are migrating an existing database to PlanetScale, you can test the PlanetScale/Prisma integration locally or in staging first. Once you're ready to migrate the database from your existing provider, refer to our no downtime [migration guides](/docs/vitess/imports/database-imports). To connect PlanetScale to your Prisma application: **Step 1** Follow the steps in the above section if you navigated away from the page previously **Step 2** Update your `prisma/schema.prisma` file with the following: ```js theme={null} generator client { provider = "prisma-client" output = "../generated/prisma" } datasource db { provider = "mysql" url = env("DATABASE_URL") relationMode = "prisma" } ``` **Step 3** Create a `prisma.config.ts` file in the root of your project: ```js theme={null} import "dotenv/config"; import { defineConfig, env } from "prisma/config"; export default defineConfig({ schema: "prisma/schema.prisma", datasource: { url: env("DATABASE_URL"), }, }); ``` **Step 4** Update the `DATABASE_URL` value in your `.env` file with the value provided to you when you created a new password in the PlanetScale dashboard. It should look like this: ``` DATABASE_URL='mysql://:@aws.connect.psdb.cloud/?sslaccept=strict' ``` ## Foreign key constraints PlanetScale does not enable foreign key constraints by default. If you are not using foreign key constraints to enforce referential integrity at the database level, integrating with Prisma will still work, but it requires a couple additional steps (detailed below). If you do plan to use foreign key constraints, enable them on your PlanetScale database settings page. **Step 1** In your database dashboard, go to "Settings" **Step 2** Enable "Foreign key constraints" **Step 3** Save your changes ### Using Prisma without foreign key constraints If you are not using foreign key constraints, you can use Prisma's [`relationMode`](https://www.prisma.io/docs/orm/prisma-schema/data-model/relations/relation-mode) to emulate relations. You need to update `datasource db` in your `schema.prisma` file to include `relationMode = "prisma"`: ``` datasource db { provider = "mysql" url = env("DATABASE_URL") relationMode = "prisma" } ``` When emulating relations, you also need to manually create dedicated indexes on foreign keys to avoid performance issues. For example, if you have `Post` and `Comment` tables, where the `Comment` table references a `post`, you need to add an index to your `Post` model: ```js theme={null} model Post { id Int @id @default(autoincrement()) title String content String likes Int @default(0) comments Comment[] } model Comment { id Int @id @default(autoincrement()) comment String postId Int post Post @relation(fields: [postId], references: [id], onDelete: Cascade) @@index([postId]) // manually created index } ``` You can learn more about how to do this in Prisma in their [Emulating relations documentation](https://www.prisma.io/docs/orm/overview/databases/planetscale#option-1-emulate-relations-in-prisma-client). ## Push your Prisma schema to PlanetScale Push your schema to your PlanetScale branch with: ```bash theme={null} npx prisma db push ``` Then generate the Prisma Client: ```bash theme={null} npx prisma generate ``` The recommended workflow with using Prisma alongside PlanetScale is to use `prisma db push` instead of `prisma migrate`. You can read more about [`prisma db push` here](https://www.prisma.io/docs/orm/reference/prisma-cli-reference#db-push). Your PlanetScale database schema now matches the Prisma schema you configured in `prisma/schema.prisma`. To confirm this, go to your PlanetScale dashboard, click "Branches", and select the branch you generated credentials for. You should see your schema. ## Using the PlanetScale serverless driver with Prisma The [PlanetScale serverless driver](/docs/vitess/tutorials/planetscale-serverless-driver) allows you to execute queries over HTTP. You can use it with Prisma ORM via the `@prisma/adapter-planetscale` driver adapter. **Step 1** Install the Prisma `adapter-planetscale` and PlanetScale serverless driver packages: ```bash theme={null} npm install @prisma/adapter-planetscale @planetscale/database ``` **Step 2** Generate Prisma Client: ```bash theme={null} npx prisma generate ``` **Step 3** If you're not already using a direct connection string, update your host to `aws.connect.psdb.cloud` or `gcp.connect.psdb.cloud`, depending on your chosen region. **Step 4** Configure your Prisma Client with the adapter: ```js theme={null} import "dotenv/config" import { PrismaPlanetScale } from '@prisma/adapter-planetscale' import { PrismaClient } from './generated/prisma/client.js' const adapter = new PrismaPlanetScale({ url: process.env.DATABASE_URL }) const prisma = new PrismaClient({ adapter }) ``` For more information, see the [Prisma documentation](https://www.prisma.io/docs/getting-started/prisma-orm/quickstart/planetscale). ## Using the MariaDB adapter with Prisma (TCP) If you are running in a traditional server environment (Express, Fastify, etc.) where TCP connections are available, you can use `@prisma/adapter-mariadb` instead of the PlanetScale serverless driver. **Step 1** Install the MariaDB adapter: ```bash theme={null} npm install @prisma/adapter-mariadb ``` **Step 2** Configure your Prisma Client with the adapter: ```js theme={null} import "dotenv/config" import { PrismaMariaDb } from '@prisma/adapter-mariadb' import { PrismaClient } from './generated/prisma/client.js' const adapter = new PrismaMariaDb({ host: 'aws.connect.psdb.cloud', user: process.env.DB_USER, password: process.env.DB_PASSWORD, database: process.env.DB_NAME, port: 3306, ssl: true, }) const prisma = new PrismaClient({ adapter }) ``` > [!NOTE] > > The MariaDB adapter requires explicit SSL configuration when connecting to PlanetScale. Set `ssl: true` in the adapter options. Passing a connection string URL directly to `PrismaMariaDb` does not automatically enable SSL. ## Sharding with PlanetScale and Prisma If you are using a [sharded database](/docs/vitess/sharding) with PlanetScale, Prisma supports defining shard keys in your Prisma schema. This is currently available as a [Preview](https://www.prisma.io/docs/orm/more/releases#preview) feature in Prisma as of [v6.10.0](https://github.com/prisma/prisma/releases/tag/6.10.0). To use shard key attributes, specify the `shardKeys` Preview feature on your Prisma generator in `schema.prisma`: ```prisma theme={null} generator client { provider = "prisma-client" output = "../generated/prisma" previewFeatures = ["shardKeys"] } ``` This allows you to use `@shardKey` and `@@shardKey` attributes. * `@shardKey` — Used to define a single-column shard key * `@@shardKey` — Used to define a multi-column shard key For example, if you have a shard key on your `region` column in your `User` database, you can define that in your Prisma model with: ```js theme={null} model User { id String @default(uuid()) region String @shardKey } ``` ## Next steps * Make safe schema changes with [branching and deploy requests](/docs/vitess/schema-changes/branching) * Explore [PlanetScale Insights](/docs/vitess/monitoring/query-insights) for performance monitoring * Learn how to [target your replicas](/docs/vitess/scaling/replicas) * Explore our [migration guides](/docs/vitess/imports/database-imports) * Enable [PlanetScale vectors](/docs/vitess/vectors) * Implement [read replicas](/docs/vitess/scaling/replicas) for scale * Join the [PlanetScale Discord](https://pscale.link/community) community ## Additional resources > **[Prisma Documentation](https://www.prisma.io/docs)** > **[Prisma + PlanetScale Best Practices](https://www.prisma.io/docs/orm/overview/databases/planetscale)** > **[PlanetScale Support](https://planetscale.com/contact?initial=support)** > **[Foreign key constraints support](/docs/vitess/foreign-key-constraints)** ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vector search and storage Source: https://planetscale.com/docs/vitess/vectors Vectors are a data structure that captures opaque semantic meaning about something and allows a database to search for resources by similarity based on this opaque meaning. As a data type, a vector is just an array of floating-point numbers. Those numbers are generated by submitting some resource — a word, a string, a document, an image, audio, etc. — to an *embedding model*,¹ which converts the resource to a vector. A vector database stores those vector embeddings alongside other relational data. In practice, that might look like a table with columns for ID (a primary key), content (as a BLOB or VARCHAR), and a vector. Then it becomes possible to perform queries that find content similar to a search vector, like so: ```sql theme={null} SELECT id FROM t1 ORDER BY DISTANCE(TO_VECTOR('[1.2, 3.4, 5.6]'), embedding, 'L2_squared') LIMIT 10; ``` Possible applications include recommendation engines that show products similar to a user's purchase history, or search engines that find documents or other resources based on natural-language queries. Read our [applications of vector databases](/docs/vitess/vectors/use-cases) docs to learn more about how vector databases can be applied in the real world. PlanetScale has added support for vector columns, vector distance functions, and vector indexes, as described below. If at any point you experience issues with vectors, we highly encourage you to get in touch. Your feedback is extremely valuable as we continue to refine vectors support, so don’t hesitate to reach out. You can [submit a support ticket](https://planetscale.com/contact) to relay any feedback or issues. We also have a vectors channel in our [Discord](https://pscale.link/community) where you can ask questions and share feedback. \[¹]: PlanetScale does not currently provide an embedding service. You can find several good cloud-based options like OpenAI or AWS Titan, or local options like Python `sentence_transformers`. ## Enabling PlanetScale vectors PlanetScale has a custom version of MySQL that has been extended with vector support. Vector support can be enabled on a per-branch basis. The branch will be updated to the vectors-capable version of MySQL. Branches with vectors enabled [will repurpose half of the memory](#resource-requirements) in the InnoDB buffer pool for use with vectors. To enable the vector support on a branch: **Step 1** In the database where you want to use vectors, navigate to the “Branches” tab, and click on the branch where you would like to enable vector support. **Step 2** Click on the small gear icon underneath the “Connect” button on the right. **Step 3** Click the toggle next to “Enable vectors”. **Step 4** Click “Save branch settings”. **Step 5** The branch will upgrade asynchronously to the correct version of MySQL, which may take 30-60 minutes. While this happens, the database dashboard will show an "Enabling vectors" badge, which changes to a "Vector-enabled" badge when the upgrade is complete. ## Adding vector columns As a first step, create one or more columns with the VECTOR type and then insert some vectors. Here’s an example: ```sql theme={null} CREATE TABLE t1(id INT PRIMARY KEY auto_increment, embedding VECTOR(4)); INSERT INTO t1(embedding) VALUES (TO_VECTOR('[1, 2, 3, 4]')), (TO_VECTOR('[5, 6, 7, 8]')); ``` At some point, you’ll want to build an index on this vector column to facilitate fast similarity searches. Here’s how to create one: ```sql theme={null} CREATE /*vt+ QUERY_TIMEOUT_MS=0 */ VECTOR INDEX embedding_index ON t1(embedding); ``` The `QUERY_TIMEOUT_MS` comment is optional, but important for tables over 100,000 rows. Vector indexes take a significant amount of time to build, and the comment allows the `CREATE` statement to bypass Vitess’s usual timeout. Then, you can perform similarity searches like so: ```sql theme={null} SELECT id, DISTANCE(TO_VECTOR('[3, 3, 3, 3]'), embedding, 'L2_SQUARED') AS d FROM t1 ORDER BY d LIMIT 10; ``` Use an `EXPLAIN` query to confirm that the query uses the new index. This query actually won’t use the index until the table has around 50 rows in it. Vector indexes provide approximate results. An unindexed query with LIMIT 100 returns exactly the 100 rows closest to the reference vector, after performing a full table scan and a sort. An indexed query returns, on average, about 100 of the top 105 (around 95%) of the rows closest to the reference vector, but much faster than a full table scan. This is expected, because all efficient vector indexes, including PlanetScale’s vector indexes, perform approximate nearest neighbor (ANN) searches. If you are adding vectors to your database from an application, you may want to use prepared statements, although we do not recommend it. `TO_VECTOR` works in that setting, but serializing the vectors on the client side and uploading them as binary is faster. The serialized format is IEEE-754 32-bit floats, which you can serialize with code like this: * Python: `struct.pack(f'{len(float_array)}f', *float_array)` * Ruby: `float_array.pack(“f*”)` * Rust: `float_array.map(|f| f.to_ne_bytes()).flatten().collect()` You can use the resulting blob (which will be 4 bytes times the number of dimensions in the vector) in an `INSERT` statement like this: ```sql theme={null} INSERT INTO t1(embedding) VALUES (CAST(? AS CHAR CHARACTER SET binary)); ``` ## Vector index parameters Statements that create a vector index take optional parameters, which can be specified as JSON key-value pairs, like so: ```sql theme={null} CREATE VECTOR INDEX embedding_index ON t1(embedding) SECONDARY_ENGINE_ATTRIBUTE='{"type":"spann", "distance":"cosine"}'; ``` The `type` attribute specifies the algorithm used to build and query the vector index. It is optional, and the only valid value is `spann`, which is the default. The `distance` attribute specifies the distance metric that queries will use, and can be any of the following: * `dot` for the dot product * `cosine` for the cosine of the angle between the two vectors. * `l2` or `euclidean` for the length of a line between the ends of the vectors * `l2_squared` or `euclidean_squared` for the square of the Euclidean distance. This is the default. The `distance` metric specified at index creation time must match the distance metric used at query time, or the index cannot be used, and MySQL will perform a full-table scan instead. Other possible options include `fixed_quantization` and `product_quantization`, which configure [vector compression](/docs/vitess/vectors/terminology-and-concepts#quantization) in the index, resulting in lower storage requirements and faster query times. See the [Quantization](/docs/vitess/vectors/reference#quantization-options) section for more information. ## Filtered vector queries Vector data is stored in tables alongside any other relational data, and sometimes applications need to query, join, or filter based on that relational data. In particular, queries with `WHERE` clauses work, and MySQL chooses the index or indexes that allow the query to complete the fastest. For example, in a table of products for sale containing indexed columns for price and seller ID, here are how two queries might be executed: ```sql theme={null} SELECT id,price,seller_id FROM products WHERE price < 20.0 ORDER BY DISTANCE(TO_VECTOR('[1.2, 3.4, 5.6]'), embedding, 'L2_squared') LIMIT 10; ``` This query selects the ten products with a price under \$20 closest to some reference vector. If a non-trivial fraction of products cost less than \$20, then MySQL will use the vector index to produce more than ten results, and filter them down to exactly ten results that meet the price constraint. ```sql theme={null} SELECT id,price,seller_id FROM products WHERE seller_id=789 ORDER BY DISTANCE(TO_VECTOR('[1.2, 3.4, 5.6]'), embedding, 'L2_squared') LIMIT 10; ``` This query selects the ten products from a given seller that are closest to some reference vector. If that seller’s products are a small enough fraction of the rows in the table, then MySQL will use the index on seller\_id to find all products from that seller, and will sort those products by vector distance and return the top ten. The MySQL query planner chooses whether to use the vector index or some other index automatically based on the query and based on the contents of the table, to maximize query performance. Use `EXPLAIN` on any given query to see how it will execute. As we continue to refine vectors support, we’re looking for feedback on how well MySQL plans vector queries. If you believe you’ve hit an edge case or something looks wrong, please [open a support ticket](https://planetscale.com/contact) and let us know. ## Resource requirements A vector index with default parameters requires 3-4x as much disk space as the underlying data in the vector column, and it requires around 15% as much memory as the underlying data. For example, a vector column with 1536 dimensions and a million rows takes up 1536 \* 1000000 \* 4 = 5.7GB of space in the table. The corresponding index will be 18-24GB, and it will require around 800MiB of memory to build and query. Using [quantization settings](/docs/vitess/vectors/reference#quantization-options) other than the default will increase or decrease the required disk space and memory proportional to how much less or more each vector gets compressed relative to the 16-bit default. PlanetScale scales storage space automatically for any branch backed by EBS. On Metal, make sure the instance size you choose has enough storage for the vector indexes you plan to create. By default, vector indexes reserve half of the buffer-pool memory allocated to each instance, equal to about one third of overall instance memory. This tradeoff can be changed in the [cluster configuration](/docs/vitess/cluster-configuration) for any branch with vectors enabled. ## Usage recommendations * **Incremental indexes** (indexes that begin empty and update as new vectors are added) function correctly, but are significantly slower to build compared to a one-shot index (an index built on an existing set of vectors). Disk usage is much higher due to potentially very high InnoDB blob fragmentation issues, so it's much easier to run out of disk space. * Once you opt a branch into the vectors feature, that branch must continue to run a vectors-enabled version of MySQL. You can remove your vector columns/tables, but you cannot downgrade that branch to its prior version of MySQL. ## Feedback We want to make our vectors offering as reliable, fast, and feature-rich as possible. Feedback from our early users will help make this possible. If you encounter any issues, crashes, unexpected errors or poor performance, please [submit a support ticket](https://planetscale.com/contact). You are also welcome to reach out with general feedback and suggestions. We also have a [Discord](https://pscale.link/community) channel for the vectors feature where you can ask questions, share feedback, and discuss what you’re working on. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vector type and index reference Source: https://planetscale.com/docs/vitess/vectors/reference ## Vector type PlanetScale MySQL provides a `VECTOR(X)` type that can be used to store vectors. To add a vector column to a table, set it to type `VECTOR(X)` where `X` is the dimension of the vectors to be stored in this column. ### Example ```sql theme={null} CREATE TABLE t1 ( id INT PRIMARY KEY auto_increment, embedding VECTOR(4) ); ``` ## Vector index PlanetScale MySQL provides a new `VECTOR INDEX` to facilitate fast and scalable approximate nearest neighbor (ANN) search on vector data. Statements that create a vector index may take optional parameters, which can be specified as JSON key-value pairs, via the `SECONDARY_ENGINE_ATTRIBUTE` variable. There are four options that can be specified in the JSON. The first two are the following: * `type`: specifies the algorithm used to build and query the vector index. * Supported values: `spann` (more info on the [SPANN algorithm](/docs/vitess/vectors/terminology-and-concepts#space-partitioned-approximate-nearest-neighbors-spann)) * `distance` specifies the distance metric that queries will use. * Supported values: * `dot` for the dot product * `cosine` for the cosine of the angle between the two vectors. * `l2` or `euclidean` for the length of a line between the ends of the vectors * `l2_squared` or `euclidean_squared` for the square of the Euclidean distance. This is the default. The distance metric specified at index creation time must match the distance metric used at query time, or the index cannot be used, and MySQL will perform a full-table scan instead. ### Quantization options Vector indexes in PlanetScale MySQL can be configured to use quantization to reduce the size of the vectors stored in the index. Quantization is a compression technique that reduces the number of bits used to store each vector. This compression is only applied to the vectors as they're stored in the index: the vectors that you insert in your MySQL table are always stored losslessly. The default mode of quantization is `bfloat16`, which reduces the memory and disk space used by an index by half with little or no effect on recall. If you need full, 32-bit floating point numbers in a vector index, you must explicitly request `"fixed_quantization":"none"` when creating the index. The quantization options can be specified alongside the other index options in the `SECONDARY_ENGINE_ATTRIBUTE` variable. The following quantization algorithms are currently supported: * `product_quantization` (PQ) specifies that product quantization should be used for vector compression. * A value of the form `{"dimensions":X}` must be provided, where `X` is the number of dimensions to use for the quantization. `X` must be a divisor of the vector's dimension. As an example, vectors with 1536 dimensions can be quantized to 192 dimensions with minimal losses to recall, but overly small values of `X` may end up affecting the recall of the index. * Product quantization currently only works with the `l2` distance metric. > [!NOTE] > > Product Quantization is a learned quantization algorithm, which means that an existing vector dataset is required to train the quantization model. > To use Product Quantization on your indexes, begin by inserting a large number of vectors into your table. The more vectors you insert, the better the resulting quantization will be. Then, create the index with the Product Quantization options you want. > Once the index has been created, you can continuously insert, update, and delete vectors from your table, but beware that if the distribution of the new vectors changes significantly (e.g. because they come from a different source), the > recall on the index may be affected. > If you create a Product Quantized index on an empty table, you will need to insert at least 1000 vectors in a single transaction before the index is usable. * `"fixed_quantization" : "float16"` specifies that fixed quantization into [Float16](https://en.wikipedia.org/wiki/Half-precision_floating-point_format) should be used for vector compression. No extra arguments are required. * `"fixed_quantization" : "bfloat16"` specifies that fixed quantization into [Brain Float 16](https://en.wikipedia.org/wiki/Bfloat16_floating-point_format) should be used for vector compression. No extra arguments are required. This setting is the default. * `"fixed_quantization" : "none"` specifies that no quantization should occur; the index will use full, 32-bit floats. No extra arguments are required. * `"fixed_quantization" : "onebit"` specifies that fixed quantization into 1 bit should be used for vector compression. No extra arguments are required. * This is a lossy compression method that can be used to store vectors in a very compact form. It is not suitable for all use cases, as it can significantly affect the recall of the index. * This quantization method currently only works with the `l2` distance metric, which effectively calculates the Hamming Distance between the vectors. ### Examples ```sql theme={null} CREATE /*vt+ QUERY_TIMEOUT_MS=0 */ VECTOR INDEX embedding_index ON t1(embedding); ``` ```sql theme={null} CREATE /*vt+ QUERY_TIMEOUT_MS=0 */ VECTOR INDEX embedding_index ON t1(embedding) SECONDARY_ENGINE_ATTRIBUTE='{"type":"spann", "distance":"cosine"}'; ``` ```sql theme={null} CREATE /*vt+ QUERY_TIMEOUT_MS=0 */ TABLE t1( title VARCHAR(250), vec VECTOR(1536), VECTOR KEY k(vec) SECONDARY_ENGINE_ATTRIBUTE='{"type":"spann", "distance":"l2", "product_quantization":{"dimensions":96}}' ); ``` ## Vector functions PlanetScale MySQL includes several new functions for working with vectors. ## `TO_VECTOR(string)` or `STRING_TO_VECTOR(string)` Converts a text string to a binary vector value. The text string is an array of floating point numbers in JSON format. ### Example ```sql theme={null} SELECT TO_VECTOR('[1, 2.78, 3.14]'); -> 0x0000803F85EB3140C3F54840 ``` ## `FROM_VECTOR(string)` or `VECTOR_TO_STRING(vector)` Converts a binary vector to a human-readable string. ### Example ```sql theme={null} SELECT FROM_VECTOR(0x0000803F85EB3140C3F54840); -> [1.00000e+00,2.78000e+00,3.14000e+00] ``` ## `VECTOR_DIM(string)` Calculates the dimension of a vector. ### Example ```sql theme={null} SELECT VECTOR_DIM(TO_VECTOR('[1,2,3]')); -> 3 ``` ## `DISTANCE(vector1, vector2, [metric])` Calculates the distance between `vector1` and `vector2`. The optional third parameter specifies which distance metric is to be used: `dot`, `cosine`, `l2` (`euclidean`), or `l2_squared` (`euclidean_squared`). * `dot` means the dot product. * `cosine` means the cosine of the angle between the two vectors. This is mathematically defined as the dot product divided by the magnitude of the two vectors, which yields a value between `-1` and `1`. Some vector database vendors do additional math on the result to normalize the value to be between `0` and `1` or between `0` and `2`. Our implementation normalizes the output, and returns a value between `0` and `2`. A result of `0` means the vectors are proportional (point in the same direction) and `2` means the vectors are opposite. * `l2` (or `euclidean`) means the length of a line between the ends of the vectors. * `l2_squared` (or `euclidean_squared`) is the square of the Euclidean distance If the distance metric is omitted, it defaults to `dot`. ### Examples ```sql theme={null} SELECT DISTANCE(TO_VECTOR('[1,2]'), TO_VECTOR('[5,4]'), 'dot'); -> 13 ``` ```sql theme={null} SELECT DISTANCE(TO_VECTOR('[1,2]'), TO_VECTOR('[5,4]'), 'cosine'); -> 0.9079593845004517 ``` ```sql theme={null} SELECT DISTANCE(TO_VECTOR('[1,2]'), TO_VECTOR('[5,4]'), 'l2'); -> 4.47213595499958 ``` ```sql theme={null} SELECT DISTANCE(TO_VECTOR('[1,2]'), TO_VECTOR('[5,4]'), 'l2_squared'); -> 20 ``` ```sql theme={null} SELECT id, price, seller_id FROM products WHERE price < 20.0 ORDER BY DISTANCE(TO_VECTOR('[1.2, 3.4, 5.6]'), embedding, 'l2_squared') LIMIT 10; ``` ## `DISTANCE_DOT(vector1, vector2)` Is the same as `DISTANCE(vector1, vector2, 'dot')` ## `DISTANCE_COSINE(vector1, vector2)` Is the same as `DISTANCE(vector1, vector2, 'cosine')` ## `DISTANCE_L2(vector1, vector2)` Is the same as `DISTANCE(vector1, vector2, 'l2')` ## `DISTANCE_EUCLIDEAN(vector1, vector2)` Is the same as `DISTANCE(vector1, vector2, 'l2')` ## `DISTANCE_L2_SQUARED(vector1, vector2)` Is the same as `DISTANCE(vector1, vector2, 'l2_squared')` ## `DISTANCE_EUCLIDEAN_SQUARED(vector1, vector2)` Is the same as `DISTANCE(vector1, vector2, 'l2_squared')` ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vector database terminology and concepts Source: https://planetscale.com/docs/vitess/vectors/terminology-and-concepts There are many concepts, algorithms and data structures that are used when discussing vector databases. Here, we provide an overview of many of these concepts and describe the indexing technique that PlanetScale MySQL uses (SPANN). ## General terminology Common terms used when discussing vector databases and indexes. #### Vector (Embedding) Vectors (more specifically, Embeddings) are a data structure that captures opaque semantic meaning about something and allows a database to search for resources by similarity based on this opaque meaning. As a data type, a vector is just an array of floating-point numbers. Those numbers are generated by submitting some resource — a word, a string, a document, an image, audio, etc — to an *embedding model*, which converts the resource to a vector. #### K-Nearest Neighbors (KNN) KNN refers to the K-Nearest Neighbors to some point P in a high-dimensional space. With vector databases, we sometimes want to store millions or billions of high-dimensional vectors. A common query on a vector database is to find the other vectors in our data set that are similar to some input search vector. If the vectors are embeddings generated from an AI model, two vectors being similar (or near each other) in the high-dimensional space means that they have similar opaque meaning. We can ask the vector database to give us the 100 closest vectors, which is a form of KNN search, where K = 100. #### Approximate Nearest Neighbors (ANN) ANN It is similar to KNN, but instead of finding the *exact* K closest neighbors to some vector, we instead look for the *approximately* K closest neighbors in the high-dimensional space. This means that we might miss some of the *actual* closest neighbors. However, if we relax our requirement and use ANN instead of KNN, we can often get significant performance improvement using specialized vector search indexes, which are further discussed later on this page. We measure how good a job an algorithm does at producing nearest neighbors with ANN via a measure known as *recall*. #### Recall In the context of ANN search, the *recall* is the percentage of the actual nearest neighbors found in an ANN search. For example, say that we are performing a search on a large set of vectors and want to get the nearest 100 neighbors. If we perform KNN search, this will give us the exact 100 top searches. Instead, we may perform ANN search. If that ANN search returns 95 of the actual top 100 and then 5 results that are outside of the top 100, we would say the recall is 95%. If instead it returned 80 of the actual top 100 and then 20 results outside, we say the recall was 80%. Most vector indexes provide ANN search. Ideally, we'd like to perform these searches with high recall. Often, there is a trade-off between speed and recall %. #### Quantization Quantization is a technique for compressing a vector into a more compact representation. Many vector indexes use quantization to compress the vector data stored in the index to reduce its memory and disk footprint. Sometimes, the full vector representation will still be stored somewhere on disk, and sometimes only the quantized vector will be stored. With PlanetScale, the SQL table always stores the full, original vectors, and the index stores half-precision `bfloat16` values by default. You can use the different quantization options when creating a vector index to use other quantized formats or disable quantization entirely. #### Fixed Quantization Fixed Quantization is one of the simplest ways to quantize a vector, because it is stateless and does not require any training data: all the elements of the vector are compressed into a smaller datatype (e.g. from float32 to float16, or all the way down to a single bit). #### Product Quantization (PQ) PQ is a learned algorithm for compressing a vector into a more compact representation. Unlike fixed quantization methods, it requires an existing vector dataset to "train" before it can be used. ## Types of search algorithms and indexes There are many algorithms and indexes that have been developed over the years, each with their strengths and weaknesses. #### Brute-force search Brute-force search on an vector data set for nearest neighbors is one that does not use any index. Instead, it compares the search vector to all other vectors to find the nearest neighbors. The advantage of this is that is can produce exact KNN results. The disadvantage is that it does not scale well beyond a few hundred or a few thousand vectors. It is impractical for any large-scale data set. #### Inverted File Index (IVF) IVF is a type of index used to speed up ANN search. In an IVF index, all of the vectors are partitioned into chunks of similar indexes. The number of chunks to partition the data into is typically configurable. In the image below, we have our vectors (blue dots) partitioned into 5 chunks (colored regions). This shows a representation in 2D space, but is applicable to N-dimensional space as well. ![IVF Visual](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/vector-indexes/ivf.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=d419dd6efb6c688439d07541b11b88bc) When a vector search is performed, the index finds the partition(s) that are most likely to contain similar vectors, and only performs similarity search on vectors in those. This technique allows for much faster search compared to a full scan of all vectors, as we eliminate much of the vector data from the search early in the process. However, it might miss some similar vectors in the chunks that it skips, and therefore is only capable of performant ANN searches. * **Pros:** Speeds up searches, can be combined with other methods. * **Cons:** Large data sets either cause slowdown or reduced recall, or both. #### Hierarchical Navigable Small World (HNSW) HNSW is one of the most commonly implemented vector indexes in modern vector databases. This is because it provides efficient ANN search for small and medium-sized data sets, and is a widely-adopted algorithm. HNSW indexes map every vector in the data set onto a graph, with nodes representing each vector and edges between nodes that are near each-other in the high-dimensional space. HNSW builds multiple graph layers. The bottom layer is the full similarity graph, and each level above is a sparser version of the graph below (in other words, some nodes and edges get pruned). ![HNSW Visual](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/vector-indexes/hnsw.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=110175b3d464562ea04231bb8344f01b) HNSW begins its search for a vector at the top layer. Since the graph is sparse, it can quickly navigate through and find the nearest similar vector. It then drops down to the next level, and continues to search for the closest vector match in that region of the graph. This process continues until it finds the closest match(es) in the lowest level. HNSW works really well for small and medium sized vector data sets when the index can all fit in RAM. However, once the index no longer fits in RAM, performance takes a drastic hit. This is because the search on the graph produces a lot of non-sequential I/O. In memory this is fine, but this leads to poorly-optimized disk access patterns, even for SSDs. Another downside of HNSW indexes is that they cannot be updated incrementally, so they require periodically re-building the index with the underlying vector data. * **Pros:** Popular, easy to implement, fast searches. * **Cons:** Does not scale well for large data sets, requires periodically re-building the index #### DiskANN DiskANN is another graph-based ANN search algorithm akin to HNSW. However, DiskANN does not use the multi-layer graph approach like HNSW does. Instead, it builds a single large graph of the vector data in several phases. Initially, a graph node is created for each vector in the data set, and random edges are added, leading to a very "messy" graph. Then, two phases of optimization and pruning occur. The first optimization phase does significant adjustment and pruning to optimize similarity search with short edges. The second optimization phase build longer edges into the graph, allowing for faster graph traversals. This graph construction algorithm is known as *Vamana*. ![DiskANN Visual](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/vector-indexes/diskann.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=5d7d753bec7a71d185eee38eff416cd1) As the name suggests, DiskANN has better performance than HNSW when the index does not all fit into RAM. DiskANN uses a different technique for building the graph compared to HNSW. This technique leads to different neighbors for each node and a different memory / disk layout. When the index cannot fit into memory, this graph allows for a significant reduction in the number of disk read operations needed to fulfill a search compared to HNSW. DiskANN scales well, but suffers from worse query performance. While it can be modified to allow incremental updates, these are not particularly efficient and are hard to map to transactional SQL semantics. * **Pros:** Fast searches, scales better than HNSW or IVF * **Cons:** Relies on basic graph traversal, incremental index updates are expensive #### Space-Partitioned Approximate Nearest Neighbors (SPANN) SPANN is a hybrid vector indexing and search algorithm that uses both graph and tree structures, and was specifically designed to work well for indexes requiring SSD usage. A graph is created for the vector data, with edges representing nearby neighbors. The graph is partitioned into many small clusters called *posting lists*. In SPANN, nodes that are near the boundary between two posting lists may reside in multiple posting lists to help improve recall. The full set of posting lists are stored on disk. The center-most node of each posting list (known as the *centroid*) is stored in a special SPTAG index, which is designed to fit in memory. ![SPANN Visual](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/vector-indexes/spann.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=ed655d8c73eb2adf9f18bb48e4cab8e0) When an ANN search is performed, the search algorithm can quickly identify a small set of the nearest centroids to the search vector using the in-memory index. Then, a relatively small number of disk reads can take place to load only the relevant parts of the graph into memory, and then the search can be completed. According to the [SPANN research paper](https://www.microsoft.com/en-us/research/uploads/prod/2021/11/SPANN_finalversion1.pdf), this leads to a 2x performance improvement over DiskANN. * **Pros:** Fast search, scales to huge data sets, the SPFresh variant allows for efficient incremental updates * **Cons:** High complexity, high disk usage because of replicated data on-disk **PlanetScale MySQL uses a variant of the SPANN algorithm for vector indexes.** Though challenging to implement, we wanted to provide the best solution for our users to scale their vector data sets. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Common use cases for Vector search Source: https://planetscale.com/docs/vitess/vectors/use-cases The ability to store and search vector data in your MySQL database, right alongside the rest of your relational data, is a powerful tool when applied correctly. Some are already keenly familiar with the types of features and products that you can build with this. However, many are not. Here, we provide some examples of the types of features you can build with vector search when it is built directly in to your MySQL database. ## Semantic search We use search all the time, from getting answers to questions via Google, looking for products on Amazon, finding a video on YouTube, or even searching for a document in our local file system. However, not all search systems are created equal, and they can be implemented in a variety of ways. Let's say we have a corpus of blog posts that we want to be able to search through to find relevant articles to read. The table to store the blog posts in looks like this: ```sql theme={null} CREATE TABLE blog ( id BIGINT NOT NULL AUTO_INCREMENT, url VARCHAR(2048) NOT NULL, title VARCHAR(1024), subtitle VARCHAR(1024), author VARCHAR(128), PRIMARY KEY(id) ); ``` One way we could accomplish this is using plain-text matching. For our search feature, a user would type in a search string, and then the text of each blog post's title and subtitle is searched for exact string (or substring) matches of our search term. ```sql theme={null} SELECT title, url FROM blog WHERE title LIKE '%$SEARCH_TERM%' OR subtitle LIKE '%$SEARCH_TERM%' LIMIT 10; ``` (Note: we could also use a MySQL `FULLTEXT` index to perform faster text-matching search). This could give some good results, however there would be some instances where it would be problematic. Searching only for exact matches may miss some of the relevant results. For example, a user might search for the term "dogs" and end up with some posts about dogs. However, it would miss results that do not use the term dog in favor of words like "puppy" or "hound." It also might miss documents that are about "wolves" or "coyotes." This search knows nothing about the *meaning* of the word "dog." This is where **vector similarity search** comes into play. With this type of search, we would generate an [embedding](/docs/vitess/vectors/terminology-and-concepts#vector-embedding) for each blog post in our data set. An embedding is an N-dimensional vector that captures opaque meaning about some piece of data — in this case, the title + subtitle of a blog post. This vector would then be stored right along with the corresponding blog post row in the database. ```sql theme={null} CREATE TABLE blog ( id BIGINT NOT NULL AUTO_INCREMENT, url VARCHAR(2048) NOT NULL, title VARCHAR(1024), subtitle VARCHAR(1024), author VARCHAR(128), embedding VECTOR(384), /* <-- New column for the embedding */ PRIMARY KEY(id) ); ``` Whenever a search occurs on our database, we will also generate an embedding for that search term. Then, we can use vector similarity search to find the top 10 results that have the most similar *meaning* to the search term. ```sql theme={null} SELECT title, url FROM blog ORDER BY DISTANCE($SEARCH_TERM_VECTOR, embedding, 'L2_squared') LIMIT 10; ``` This type of search will be able to include those other posts, since we are now searching by meaning rather than text matches. ## Recommendation systems Recommendation systems are also common in many products. Amazon may recommend purchases similar to ones you view, and streaming services may recommend shows to you based on your watch history. Similar types of systems can be built using vector similarity search. Perhaps we have an e-commerce platform. In our database, we have a `product` table, a `user` table, and ` purchase` table to track which items each user purchases. ```sql expandable theme={null} CREATE TABLE product ( product_id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, title VARCHAR(256) NOT NULL, description VARCHAR(1024) NOT NULL, price INT NOT NULL, ... ); CREATE TABLE user ( user_id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, username VARCHAR(256) NOT NULL, email VARCHAR(256) NOT NULL, ... ); CREATE TABLE purchase ( purchase_id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, product_id BIGINT NOT NULL, user_id BIGINT NOT NULL, purchased_at DATETIME NOT NULL, ... ); ``` A great way to add a purchase recommendation feature to our application would be to use vectors. The first step would be to add a `VECTOR` column to the `product` table: ```sql theme={null} CREATE TABLE product ( product_id bigint not null auto_increment primary key, title varchar(256) not null, description varchar(1024) not null, price INT NOT NULL, embedding VECTOR(384), /* <-- New column for the embedding */ ... ); ``` We could generate an embedding for each row by feeding the title and description into an embedding model and storing the results in this new column. Whenever a user logs in to our platform, we want to show them a list of 5 recommended purchases, based on similarity to their most recently purchased item. To do this, we need a query that finds a user's most recent purchase, and then performs a KNN search for the 5 most similar products in the product table, based on vector similarity. This query would look something like this: ```sql expandable theme={null} SET @uid = 12345; SET @recentEmbedding = (SELECT product.embedding FROM product JOIN purchase ON product.product_id = purchase.product_id JOIN user ON purchase.user_id = user.user_id WHERE user.user_id = @uid ORDER BY purchase.purchased_at DESC LIMIT 1); SET @recommendationIDs = (SELECT product_id FROM product ORDER BY DISTANCE(@recentEmbedding, embedding, 'L2_squared') LIMIT 5); ``` We can then use the products in `@recommendationIDs` and display those to the user. ## Retrieval-Augmented Generation (RAG) RAG is a popular technique for augmenting and enhancing results produced by an LLM. LLMs such as GPT-4.0 or Sonnet-3.5 are extremely powerful, as they have been trained on immense data sets. However, these LLMs are not trained on the entire universe of data, and it is often useful to pass them additional context to help answer a query. Suppose we have a private question/answer platform, internal to our organization. None of the information on this platform is on the public internet and was not used to train any public LLMs. This platform stores questions and answers like so: ```sql theme={null} CREATE TABLE question ( question_id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, title VARCHAR(256) NOT NULL, text VARCHAR(2048) NOT NULL ); CREATE TABLE answer( answer_id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, question_id BIGINT NOT NULL, text VARCHAR(2048) NOT NULL ); ``` We would like to add a feature to this internal Q/A platform to allow a user to ask an LLM-powered chatbot questions instead of posting a question for other humans to answer. One way we could do this is allow the user to enter a question, send the prompt directly to the OpenAI API (or similar APIs), fetch the result, and display to the user. This may not work well for questions that are specific to our organization, as OpenAI may not have sufficient knowledge in this area. What we can do instead is leverage similarity search to augment the user's question with additional context before sending the prompt to OpenAI, which will allow it to produce better answers. To do this, we will yet again need to add a vector column to the `answer` table and populate them with embeddings: ```sql theme={null} CREATE TABLE answer( answer_id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, question_id BIGINT NOT NULL, text VARCHAR(2048) NOT NULL, embedding VECTOR(384) /* <-- New column for the embedding */ ); ``` Whenever a user enters a query for the chatbot, we will first generate an embedding for that question. Then, we will perform similarity search to find existing answers on our platform that are related to this: ```sql theme={null} SELECT answer.text FROM answer JOIN question ON question.question_id = answer.question_id ORDER BY DISTANCE($QUERY_EMBEDDING, question.embedding, 'L2_squared') LIMIT 5; ``` Now we have the user's initial question and several related answers. We will construct a string formatted something like this: ``` Here is some information that is related to a question that a user has: [RELATED ANSWERS] Using this information, please answer the following user question: [THE USER'S ORIGINAL QUERY] ``` This new larger string will get passed on to our AI API for a response. Since we have added context to the prompt, it will be able to do a better job at giving the user a satisfactory answer. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Vectors with an ORM Source: https://planetscale.com/docs/vitess/vectors/using-with-an-orm Many modern web development frameworks provide ORMs to make mapping data between your database and the objects in your application a seamless experience. Here, we show several examples of how you can use the `VECTOR` MySQL type with ORMs. These docs to not provide a comprehensive list of how to use vectors with all ORMs. Rather, small examples for several popular choices are provided (Drizzle, Prisma, Rails). If you use a different ORM and are having trouble getting it to work with vectors, please [reach out](https://planetscale.com/contact). Also note that for now, using PlanetScale MySQL vectors with these ORMs requires either a custom type or running raw MySQL queries. ## Drizzle Here, we'll show how you can create and insert rows into a table that has a `VECTOR` column using Drizzle. First, modify your `schema.ts` file to import the appropriate items and define two new types. The first type is for the distance functions, and the second is to represent a vector. ```typescript expandable theme={null} import { sql } from 'drizzle-orm' import { mysqlTable, customType, text, MySqlColumn, bigint } from 'drizzle-orm/mysql-core' export type DistanceFunction = 'COSINE' | 'DOT' | 'L2' | 'L2_SQUARED' export const vector = customType<{ data: ArrayBuffer config: { length: number } configRequired: true driverData: Buffer }>({ dataType(config) { return `VECTOR(${config.length})` }, fromDriver(value) { return value.buffer as ArrayBuffer }, toDriver(value) { return Buffer.from(value) } }) ``` With these in place, we can go ahead and declare our table, also in `schema.ts`. ```typescript theme={null} export const product = mysqlTable('product', { id: bigint('id', { mode: 'number', unsigned: true }).autoincrement().primaryKey(), name: text('name'), description: text('description'), embedding: vector('embedding', { length: 384 }) }) ``` This would be in addition to other schema declarations you need for your application. We now have an object to model a table with a vector embedding column. Elsewhere in our application, we can perform typical operations like inserts, deletes, searches, etc. For example, to insert a row into this table, do the following: ```typescript theme={null} const embedding = // generate an embedding with your preferred API const serializedEmbedding = Buffer.from((new Float32Array(embedding)).buffer) await db.insert(product).values({ description: 'hi there', embedding: serializedEmbedding }) ``` We can also perform searches on the data: ```typescript theme={null} const rankedItems = await db .select({ description: product.description }) .from(product) .orderBy(`DISTANCE(TO_VECTOR(${embedding}), ${product.embedding}, L2_SQUARED)`) .limit(10) ``` If you have a large data set, you'll want to make sure you create an index on this vector column. ## Prisma Next, let's look at what it takes to get a table with a `VECTOR` column working with Prisma. As of this writing, [Prisma does not support custom types](https://github.com/prisma/prisma/issues/5039). Until Prisma provides support, you can still use vectors in a Prisma-powered application by using the `Unsupported` function in your `schema.prisma` and then use raw queries to perform vector searches. We can add a new table with a `VECTOR(4)` column by adding the following to our `schema.prisma` file. ```typescript theme={null} model Product { id BigInt @id @default(autoincrement()) @db.UnsignedBigInt name String? @db.Text description String? @db.Text embedding Unsupported("vector(4)")? } ``` When running `prisma db push`, it will create the table with the `embedding` column having type `VECTOR(4)`, even though Prisma does not technically support vectors. From here, you can use Prisma's `queryRaw` feature to run raw SQL queries. For example, to insert a row into this table. ```typescript theme={null} const name = 'Pots' const description = 'For cooking' const embedding = '[0.5, 0.4, 0.3, 0.2]' await prisma.$queryRaw( Prisma.sql` INSERT INTO Product (name, description, embedding) VALUES(${name}, ${description}, TO_VECTOR(${embedding}))` ) ``` And to retrieve results from this table: ```typescript theme={null} const result = await prisma.$queryRaw( Prisma.sql`SELECT * FROM Product ORDER BY DISTANCE(TO_VECTOR(${query_vector}), embedding, 'l2_squared')` ) console.log(result) ``` If you have a large data set, you'll want to make sure you create an index on this vector column. ## Ruby on Rails Lets look at how you can work with a vector column in a Ruby on Rails application. Say you have an existing object that models rows in a table, and you are using ActiveRecord to manage the mapping between your objects and your database. For example, an object representing a tweet in `app/models/tweet.rb`. ```ruby theme={null} class Tweet < ApplicationRecord belongs_to :user has_one_attached :image default_scope -> { order(created_at: :desc) } validates :content, presence: true, length: { maximum: 140 } validates :user_id, presence: true end ``` Since vectors are a new type for MySQL, we will add a custom migration to handle adding and dropping a `VECTOR` column to this table in the database. To do this, add a new migration in `db/migrations` like so: ```ruby expandable theme={null} class AddTweetEmbeddings < ActiveRecord::Migration[7.0] def up execute <<-SQL ALTER TABLE tweets ADD COLUMN embedding VECTOR(1536); SQL execute <<-SQL CREATE VECTOR INDEX idx_tweet_embeddings ON tweets(embedding) '{"type":"spann","distance":"cosine"}'; SQL end def down execute <<-SQL ALTER TABLE tweets DROP COLUMN embedding; SQL end end ``` You'll also need to make the appropriate adjustments in your project to ensure that this gets executed when you run `rails db:migrate`. With the schema updated appropriately, you can run raw SQL queries via ActiveRecord to insert rows with vectors and perform search. For example, to insert a new row with a vector, you can do something like this: ```ruby expandable theme={null} embedding = [1.0, 0.5, 0.25, 0.125].to_json created_at = Time.now.to_s updated_at = Time.now.to_s content = 'A tweet!' user_id = 100 sql = <<-SQL INSERT INTO tweets (user_id, content, content_embeddings, created_at, updated_at) VALUES ( #{ActiveRecord::Base.connection.quote(user_id)}, #{ActiveRecord::Base.connection.quote(content)}, TO_VECTOR(#{ActiveRecord::Base.connection.quote(embedding)}), #{ActiveRecord::Base.connection.quote(created_at)}, #{ActiveRecord::Base.connection.quote(updated_at)} ) SQL ActiveRecord::Base.connection.execute(sql) ``` To perform a similarity search based on an input embedding, you can execute the following: ```ruby theme={null} embedding = [1.0, 0.5, 0.25, 0.125] sql = <<-SQL SELECT id, content, distance(content_embeddings, TO_VECTOR('#{embedding}'), 'COSINE') as d FROM tweets ORDER BY d LIMIT 10 SQL results = ActiveRecord::Base.connection.execute(sql) ``` ## Other ORMs If you have requests for other ORMs you'd like to see documented for using with vectors, please [reach out](https://planetscale.com/contact) with your questions. On the other hand, if you get PlanetScale MySQL vectors working with an ORM not listed here and would like to share your technique, please do as well. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # Web console Source: https://planetscale.com/docs/vitess/web-console The PlanetScale web console is an interactive interface for running MySQL queries and DDL (Create, Alter, and Delete) against your PlanetScale database branches. ## Get started The PlanetScale web console can be used to query to any database branch; however, it is [disabled for production branches](/docs/vitess/web-console#enable-for-production-branches) by default to protect production data. To access the web console, navigate to a database, and click on the "Console" tab in the page navigation. From here, you can select which branch you'd like to connect to by selecting it in the dropdown and clicking "Connect". Once you have accessed the web console, you can run queries against your database branch, or apply DDL to branches without [safe migrations](/docs/vitess/schema-changes/safe-migrations) enabled. The following are examples of MySQL statements you may find useful within the web console: Use `SHOW TABLES;` to see a list of the tables in your database branch. Use `DESCRIBE table_name;` to obtain information about a given table's structure. Use `EXPLAIN` in front of `SELECT`, `DELETE`, `INSERT`, `REPLACE` and `UPDATES` statements to learn how the database is executing a query. This can be useful for optimizing slow queries. ## Primary and replica connections After connecting to a branch, use the **Server** dropdown in the console header to choose which node to connect to: * **Primary** — Connect to the primary node. Use this for read and write queries. * **Replica** — Connect to a read replica. The console uses read-only credentials on replica connections. If a branch has no replicas, the console connects to the primary and the **Replica** option is unavailable. Switching between primary and replica reconnects the console and clears the current output. ## Supported console commands | Command | Description | | :- | :- | | ?, \\? | Synonym for `help` | | clear, \c | Clear the current input statement | | help, \h | Display list of commands | | ego, \G | Send command to server | | go, \g | Send command to server | ## Enable for production branches By default, the web console is disabled for production branches to prevent accidental data loss. You can enable the web console for production branches on the "Settings" page for the given database, `app.planetscale.com///settings`. Select the checkbox for "Allow web console access to production branches", then scroll down and click the "Save database settings" button to save your changes. This will enable the ability to use the web console to run queries against production branches for the given database. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale. # What is PlanetScale? Source: https://planetscale.com/docs/what-is-planetscale A fully managed relational database platform for Vitess, Neki, and Postgres, built for scale, performance, and reliability. PlanetScale is a fully managed relational database platform for [Vitess](/docs/vitess), [Neki](/docs/neki), and [Postgres](/docs/postgres), bringing you scale, performance, and reliability — without sacrificing developer experience. We don't call ourselves the world's fastest database for nothing. Benchmarks for both [Vitess](https://planetscale.com/benchmarks/vitess) and [Postgres](https://planetscale.com/blog/benchmarking-postgres) show PlanetScale in the lead over and over again for both queries per second and latency. With PlanetScale, you get blazing fast performance with our [locally-attached NVMe drives](/docs/metal), high availability 3 node clusters (1 primary and 2 replicas by default), automated failovers, connection pooling, online fully-managed version upgrades, and more. Our Vitess product offers unlimited scalability through explicit horizontal sharding. Vitess was [created at YouTube in 2010](https://vitess.io/overview/history/#:~:text=Vitess%20was%20created%20in%202010,exceed%20the%20database's%20serving%20capacity.) to solve the scaling issues they faced with their massive MySQL database. Vitess was later donated to the CNCF and continues to scale massive companies like [Slack](https://slack.engineering/scaling-datastores-at-slack-with-vitess/), [GitHub](https://github.blog/2021-09-27-partitioning-githubs-relational-databases-scale/), and more. For Postgres, [Neki](/docs/neki) provides the same model of horizontal sharding: applications connect to a router and query a logical database while data lives across independent shards. [Sign up for a PlanetScale account](https://auth.planetscale.com/sign-up) to use Neki. The team building PlanetScale is made up of passionate industry experts who have spent decades working on databases for some of the web's largest companies. Our team has directly felt the pain of overly-complicated, unintuitive database tools and came to PlanetScale to build the future of databases — the database they wished they had at their previous companies. ## PlanetScale features The fastest way to understand how PlanetScale is changing the database landscape is to take a peek inside the product. The following features create a powerful developer experience that enables teams to develop quickly and confidently. ### PlanetScale Metal When you deploy a PlanetScale database, you can choose from network-attached storage or Metal — locally-attached NVMe SSD drives. These blazing fast NVMe drives unlock unlimited IOPS, ultra-low latencies, and the [highest throughput](https://planetscale.com/blog/benchmarking-postgres) for your workloads while still running in the cloud (AWS, GCP, or Azure). ### Non-blocking schema changes With most businesses now operating online, downtime and maintenance windows are no longer acceptable. Not only does downtime hurt customer experience and trust, but even a small amount of downtime can result in thousands to millions of dollars lost for companies. Our [non-blocking schema change workflow](/docs/vitess/schema-changes) for Vitess means that you'll never experience costly table locking or downtime when running schema changes. This is a fundamental piece of PlanetScale and something that we think everyone should have access to, so there's no additional configuration required. Zero-downtime schema changes are baked into the product. ### Branching workflow Our [branching workflow](/docs/vitess/schema-changes/branching) paired with [safe migrations](/docs/vitess/schema-changes/safe-migrations) is what enables non-blocking schema changes on your production Vitess database. Instead of applying schema changes directly to your production database, we let you create branches, which are essentially copies of your database. When you create a new branch off of production, you have an isolated copy of your database that you can use for development to make schema changes. ![Branching workflow diagram - Create dev branch off of main, make schema changes, make deploy request, resolve schema conflicts, test, deploy to main](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/what-is-planetscale/branching-diagram.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=d54f0954cf67d1b8586eea998ae20c98) Development branches can serve as your staging environment, so you don't have to worry about spinning up a new testing database and constantly syncing it with production. We handle all of that for you. Once you're ready to deploy schema changes from your development branch to production, you [open a deploy request](/docs/vitess/schema-changes/deploy-requests). The deploy request allows your team to view a diff of the schema changes being made, comment, and approve before deploying the change to production. ![Example of a deploy request showing comments, approval, and deployment](https://mintcdn.com/planetscale-2/GA0k5H-MolPvBjDk/images/assets/docs/concepts/what-is-planetscale/deploy-request.png?fit=max\&auto=format\&n=GA0k5H-MolPvBjDk\&q=85\&s=b1048b95a92ac1c91750bcfca9b99a60) ### Revert a schema change The final piece of the non-blocking schema change workflow is the ability to [revert a recently deployed schema change](/docs/vitess/schema-changes/deploy-requests#revert-a-schema-change) without losing any data that was written since deploying. [Revert a schema change](https://player.vimeo.com/video/830571822?dnt=true) Despite all the safeguards we put in place, accidents can happen. If someone on your team deploys a schema change, only to realize afterward that it adversely affected the application, you can simply revert it in the PlanetScale dashboard with the click of a button. Perhaps the most impressive part is that when you revert, you won't lose any data that was written to your database during the time the updated schema was live. We keep track of it and apply it back to the original schema once you revert. No more fumbling around with snapshots or backups and restores. Just revert. ### Scale with sharding + unlimited connections With Vitess and [Neki](/docs/neki), we're able to offer horizontal scaling via sharding with minimal application changes. PlanetScale allows you to break up a monolithic database and partition the data across several databases. This [reduces the load on a single database](https://planetscale.com/blog/one-million-queries-per-second-with-mysql) by distributing it across several. Sharding can easily become a convoluted and hard-to-manage scenario, but because of our underlying architecture, we're able to keep this sharding logic largely out of the application. So, from the application's perspective, there only exists one database. Use [Vitess sharding](/docs/vitess/sharding) for MySQL-compatible workloads, or [Neki](/docs/neki) for Postgres. Another scenario that companies with massive databases often run into is connection limits due to MySQL. With PlanetScale, we can support [nearly infinite connections](https://planetscale.com/blog/one-million-connections). Vitess offers built-in [connection pooling](https://vitess.io/reference/features/connection-pools/), and we've built our own [edge infrastructure](https://planetscale.com/blog/introducing-the-planetscale-serverless-driver-for-javascript) into PlanetScale to ensure connection limits are never an issue. We generally recommend exploring horizontal sharding when your database exceeds 250 GB of data and you are beginning to feel some of the [pains associated with large scale](https://planetscale.com/blog/how-to-scale-your-database-and-when-to-shard-mysql). [Vitess sharding](/docs/vitess/sharding/sharding-quickstart) is offered on our Base plan. Horizontal sharding for Postgres is available with [Neki](/docs/neki). If you need assistance with setting up horizontal sharding, migrating to PlanetScale, or want enterprise-level SLAs, we offer this through our [Enterprise plan](https://planetscale.com/enterprise) option. [Please reach out](https://planetscale.com/contact) for more information. ### Insights [PlanetScale Insights](/docs/vitess/monitoring/query-insights) is our in-dashboard query performance analytics tool. What's unique about Insights is that you can track performance down to the individual query level. [PlanetScale Insights](https://player.vimeo.com/video/830571854?dnt=true) At a glance, the interactive graph shows you query latency, queries per second, rows read, and rows written charted against time. You'll also see any deploy requests on the graph, so you can quickly see the impact of those changes. If you notice your application is running slower than it should or you want to do a deep dive on your bill, you can come to the Insights dashboard and drill down at the individual query level to see: * number of times a query has run * total time the query has run * time per query * rows read, affected, and returned ### No downtime imports We understand changing database providers can be a pain, from dealing with downtime to complicated dumps and restores and endless compatibility issues. [Database imports](/docs/vitess/imports/database-imports) make moving to Vitess as pain-free as possible. You connect your internet-accessible MySQL database to PlanetScale as an [external keyspace](/docs/vitess/cluster-configuration#create-an-external-keyspace), then copy its tables into PlanetScale with a [Vitess MoveTables](/docs/cli/move-tables) workflow. During the import, your production database remains live, and every change to it keeps replicating into PlanetScale. When you're ready, you switch traffic to PlanetScale with no downtime and no data loss. ### Connect A common task companies need to handle is extracting data out of their database for transformation and analysis. [PlanetScale Connect](/docs/vitess/integrations/airbyte) provides you with an easy way to perform ELT. You can connect your PlanetScale database to Airbyte or Stitch, and select the destination from there. Both platforms support several data storage destinations, such as Snowflake, Google Big Query, and more. ### CLI Nearly every action you can take in the PlanetScale dashboard can also be done with our [`pscale` CLI](/docs/cli). With commands for branching, deploy requests, backups, service tokens, and more, the CLI allows teams to work quickly and efficiently. You can use the CLI to extend PlanetScale into your own DevOps workflow with [GitHub Actions](https://planetscale.com/blog/using-the-planetscale-cli-with-github-actions-workflows), [AWS CodeBuild](https://planetscale.com/blog/build-a-multi-stage-pipeline-with-planetscale-and-aws), and more. ### API Like the CLI, you can programmatically interact with PlanetScale using our [API](/docs/api/planetscale-api-oauth-applications). The API is useful for building PlanetScale into other developer tooling for faster development workflows. For example, you can programmatically create and delete database branches, open and merge deploy requests, and more. See the [PlanetScale API reference](/docs/api/reference/getting-started-with-planetscale-api) for more information. ### Replicas Every production PlanetScale branch comes with two [replicas](/docs/vitess/scaling/replicas). Replicas are read-only copies of your database that can be used to offload read traffic from your primary. With global replica credentials, you can have one credential that will automatically route queries to your branch's replicas and read-only regions. ### Read-only regions Spin up [read-only regions](/docs/vitess/scaling/read-only-regions) for Vitess with the click of a button. For globally distributed applications, read-only regions allow you to place a copy of your data close to your users. To query your read-only region, create [a replica credential](/docs/vitess/scaling/replicas) for your database. Replica queries will be automatically routed to the nearest read-only region or one of the branch's replicas, whichever has the lowest latency available. ## Get in touch Want to learn more about PlanetScale and how it can help your business prevent downtime and improve development speed? [Reach out to learn more or schedule a demo](https://planetscale.com/contact), and we'll be in touch shortly. ## Need help? Get help from [the PlanetScale Support team](https://planetscale.com/contact?initial=support), or join our [Discord community](https://pscale.link/community) to see how others are using PlanetScale.