Postgres Imports - Amazon DMS with CloudFormation

July 30, 2025

This method uses Infrastructure as Code with Step Functions workflow automation for a completely managed migration experience. Amazon Database Migration Service (DMS) provides a fully automated approach to migrate your PostgreSQL database to PlanetScale for Postgres.

Overview

This automated migration method:

1. Deploys DMS infrastructure via CloudFormation template
2. Configures source and target database endpoints automatically
3. Creates Step Functions workflow for automated migration orchestration
4. Provides built-in monitoring, notifications, and automated cleanup
5. Requires minimal manual intervention - mostly console clicks

General Prerequisites

Before starting the migration:

• Have an active AWS user with CloudFormation, EC2, DMS, SNS, and Step Functions permissions
• Source PostgreSQL database accessible from AWS VPC
• Connection details for your PlanetScale for Postgres database from the console
• VPC with at least 2 subnets in different Availability Zones

Source database prerequisites

The Task that AWS DMS runs will automatically perform these 7 validation checks before starting the migration. Confirm before starting this process that they will succeed.

Step 1: Check DMS IAM Roles

Before deploying, check if DMS service roles already exist in your AWS account:

1. Go to IAM Console
2. Click "Roles" in the left sidebar
3. Search for these role names:
   • dms-vpc-role
   • dms-cloudwatch-logs-role

• If neither role exists: Set CreateDMSRoles parameter to true in Step 3
• If both roles exist: Set CreateDMSRoles parameter to false in Step 3
• If one role exists but not the other: Consider manually creating the roles per guidance in the AWS DMS documentation and set CreateDMSRoles parameter to false in Step 3

Step 2: Download CloudFormation Template

Get the CloudFormation template:

1. Visit: https://github.com/planetscale/migration-scripts/tree/main/postgres-planetscale
2. Right-click on aurora-to-ps-dms.yaml → "Save link as"
3. Save the file to your computer

Step 3: Deploy CloudFormation Stack

1. Navigate to AWS CloudFormation Console
2. Click "Create stack" → "With new resources (standard)"
3. Select "Upload a template file"
4. Click "Choose file" and select the downloaded template
5. Click "Next"

Configure Stack Parameters

Stack name: postgres2planetscale or any name you want but note that overly-long names can cause resource creation issues

VPC Information

• VPC ID: Select your VPC from dropdown
• Subnet IDs: Select 2+ subnets in different AZs which are "public" subnets in that they have route tables through an Internet Gateway (IGW) and NACLs that allow outbound routing. See Subnet types in the AWS documentation for more information

Source Database (Your Aurora Postgres database)

• Source Endpoint Host: Your Aurora hostname (primary write endpoint is best, not a proxy)
• Source Port: 5432 (or your custom port)
• Source Database Name: Your database name
• Source Username: Your database username
• Source Password: Your database password

Target Database (PlanetScale for Postgres)

From your PlanetScale console connection details:

• Target Endpoint Host: PlanetScale host
• Target Port: 5432
• Target Database Name: PlanetScale database name
• Target Username: PlanetScale username
• Target Password: PlanetScale password

Additional Features

• DMS Instance Class: dms.c6i.large (recommended)
• Migration Type: full-load-and-cdc (recommended)
• Enable Automation: true ⭐ Important: This creates the Step Functions workflow
• Create DMS Roles: true or false (based on Step 1 findings)
• Notification Email: Your email for status updates

6. Click "Next" → "Next" → Check "I acknowledge that AWS CloudFormation might create IAM resources"
7. Click "Submit"

Step 4: Wait for Stack Completion

1. Stay on the CloudFormation console
2. Click on your stack name to view details
3. Watch the "Events" tab for progress
4. Wait for stack status to show CREATE_COMPLETE (typically 5-10 minutes)

Step 5: Confirm your email notification subscription

1. Check the inbox for the email used above
2. You will get an email from no-reply@sns.amazonaws.com "DMS Migration Workflow Notifications"
3. Click the link for Confirm Subscription

Note that after the migration task completes and the stack is deleted, you would receive no further communications from this AWS SNS Topic, but other SNS topics may use the same address.

Step 6: Get Workflow Input Configuration

1. In your completed CloudFormation stack, click the "Outputs" tab
2. Find the output key StepFunctionsPayloadTemplate
3. Copy the entire JSON value (this contains the configuration for your migration) The JSON should look like the following example:

{ 
   "replicationInstanceArn": "arn:aws:dms:us-east-2:1234567890:rep:YMZ2AH4YAJCRNJKOWBR7EEIRGE",
   "sourceEndpointArn": "arn:aws:dms:us-east-2:1234567890:endpoint:SIVNPTNFJZDCVK4ODTN6ZLONN4",
   "targetEndpointArn": "arn:aws:dms:us-east-2:1234567890:endpoint:MLSCVENBKVBWJKRVJ27EWB32IU",
   "replicationTaskArn": "arn:aws:dms:us-east-2:1234567890:task:QZCBNW565VH2JG2KE5UXX42LS4",
   "sourceEndpointHost": "prod-aurora-cluster.cluster-abc1234567.us-east-2.rds.amazonaws.com",
   "sourcePort": "5432" 
}

Step 7: Locate Step Functions Workflow

1. While still in the "Outputs" tab find the key StepFunctionsConsoleURL
2. Click on the URL link to open the AWS Step Functions console for the workflow created here

The workflow includes these automated steps:

• Network Connectivity Check: Tests connections to both source and target databases
• Security Group Auto-Fix: Automatically corrects Aurora security group settings if DMS connectivity fails
• Pre-Migration Validation: Validates database schemas, table structures, and data types
• Migration Task Start: Launches the DMS full-load and CDC replication
• Progress Monitoring: Continuously monitors migration progress and handles errors

Step 8: Start Migration Workflow

1. In the Step Functions state machine, click "Start execution"
2. Execution name: Leave as auto-generated
3. Input: Paste the JSON you copied from CloudFormation outputs
4. Click "Start execution"

The workflow will automatically:

• Test database connections
• Start the DMS migration task
• Monitor progress
• Send notifications
• Handle errors and retries

Step 9: Monitor Migration Progress

There are several tools you can use to monitor the progress and if necessary troubleshoot potential failures.

DMS Console

1. Go to DMS Console
2. Click "Tasks"
3. Select the task Identifier for this migration
4. View your task for detailed table-level progress

Step Functions Console

1. Watch the visual workflow progress in the Step Functions console
2. Each step will show green (success), red (failure), or blue (in progress)
3. Click on individual steps to see detailed logs

CloudWatch Dashboard

1. Navigate to CloudWatch Console
2. Click "Dashboards" → "Automatic Dashboards" → "DMS-Migration-Dashboard"
3. Monitor key metrics:
   • Full load progress percentage
   • CDC latency
   • Error counts
   • Throughput

Wait for automated emails

You will receive an email once the migration has reached a 100% full load and CDC replication is ongoing.

If the workflow does fail at any point, you will instead receive an email with where the failure occurred and then you can review the previously mentioned tools for more information.

Step 10: Application Cutover

When the Step Functions workflow or DMS task itself indicates migration is ready (Status is "Load completed, replication ongoing"), then you can begin your cutover process.

Pre-Cutover Checklist (Automated)

AWS DMS ensures:

• ✅ Full load is 100% complete
• ✅ CDC latency is under 5 seconds
• ✅ Data validation passes
• ✅ Both databases are synchronized

Cutover Process

1. Put application in maintenance mode, pause all writes from your application to the old database
2. Wait for DMS to confirm final sync
3. Update your application's database connection strings to use PlanetScale details
4. Restart or redeploy your application
5. Test critical functionality

Automated Cleanup (mostly)

1. Go to your CloudFormation stack

2. Click "Delete"

3. Click the "Delete" confirmation popup

   Note

   The first time you attempt to delete the stack, the full process will fail to delete some of the resources. Minimally the S3 bucket created to store the DMS pre-migration test reports will need to be manually emptied before it can be deleted. If the Step Functions workflow had to modify Aurora security group, then the rule added needs to be deleted as well. Both of these resources could be safely left behind, however the S3 bucket's data will incur ongoing charges.

4. In the CloudFormation stack "Resources" tab, find the resources where deletion failed

5. Find the Resource named PreMigrationAssessmentBucket

6. Click to open the link under the Physical ID heading for this resource

   This will take you to the Amazon S3 console for this bucket. The assessment-folder is a versioned object in S3, which means that directly deleting it here does not actually remove it, but instead places a deleted marker on the version.

7. To fully empty the bucket you will need to navigate to the main console page. Click "Buckets" from the top nav, or "General purpose buckets" from the left nav

8. Select the bucket used, it will start with the name of the CloudFormation stack

9. Click "Empty"

10. Follow the prompt to permanently delete the pre-migration reports

11. You can now re-attempt the stack deletion from the 1st step here

Troubleshooting

Stack Creation Issues

Permission Errors:

• Ensure your AWS user has CloudFormation, DMS, Step Functions, and IAM permissions
• Check that you acknowledged IAM resource creation during stack creation

Network Issues:

• Verify your VPC allows internet access for DMS to reach PlanetScale
• Check security groups allow port 5432 access
• Ensure subnets are in different Availability Zones

Step Functions Workflow Issues

Workflow Creation Fails:

• Verify you copied the complete JSON from CloudFormation outputs
• Check that the Step Functions execution role exists

Migration Task Fails:

• Check Step Functions execution details for specific error messages
• Verify database connection details are correct
• Ensure source database has logical replication enabled

Connection Problems

Source Database:

• Verify hostname, port, username, and password
• Check that source database allows connections from DMS subnet
• Ensure database has logical replication enabled (rds.logical_replication = 1 for RDS)

Target Database (PlanetScale):

• Double-check connection details from PlanetScale console
• Verify PlanetScale database is active and accessible
• Test connectivity from AWS region to PlanetScale

Step Functions Workflow Benefits

Using the automated Step Functions workflow provides:

• Visual Progress Tracking: See each migration phase in real-time
• Automatic Error Handling: Built-in retry logic and error notifications
• Audit Trail: Complete log of migration steps and timings

Advanced Options

Custom Migration Settings

Modify template parameters for:

• Different DMS instance sizes
• Custom migration types (full-load only, CDC only)
• Extended monitoring periods
• Custom notification settings

Multiple Database Migration

Deploy multiple stacks with different names to migrate multiple databases in parallel.

Migration Considerations

Before migration, review:

• PostgreSQL version compatibility
• Extension support limitations
• Third-party enhancement restrictions

Support and Resources

• Template Issues: GitHub repository
• AWS Documentation: CloudFormation and Step Functions
• PlanetScale Support: Contact support for migration assistance

For simpler migrations, consider pg_dump/restore, or WAL streaming methods.

Need help?

Get help from the PlanetScale Support team, or join our GitHub discussion board to see how others are using PlanetScale.