Attentive can confidently handle a 10× increase in volume on Black Friday/Cyber Monday thanks to PlanetScale.

Background

Attentive, the world’s #1 SMS marketing platform, drives billions of dollars in revenue for the top e-commerce brands and is experiencing rapid growth. In addition to supporting the platform’s organic growth, Attentive’s infrastructure must be prepared to service 10× spikes in volume around major e-commerce events, including Black Friday and Cyber Monday.

Michael Ndubuisi is a Senior Software Engineer on Attentive’s Targeting team who played a key role in testing, migrating, and productionizing PlanetScale instances at the company. Michael’s team was using Postgres via AWS RDS and was reaching the limits of its ability to scale beyond Attentive’s typical daily usage. With the team facing the ceiling of a costly vertical scaling strategy and Black Friday and Cyber Monday (BFCM) coming up, it was time to consider an alternative.

Challenges

In getting prepared for BFCM 2022, the team took a look at existing data infrastructure and realized that in order to be prepared to handle 10× the volume of typical usage, they were going to have to overcome some challenges:

• Strict data SLAs: Attentive’s business depends on sending timely information to customers – there is no room for error and delay when it comes to relaying SMS messages in an e-commerce context

• Poor support: Support from AWS was lacking, SLAs were poor, and support was hard to reach at critical times

• Large quantities of data: Some Attentive teams were close to maxing out the AWS instance sizes available for running Postgres in RDS

• Distractions of database issues: Issues with RDS installation has caused customer issues and wasted developer time, damaging team productivity

• BFCM can’t be moved: Everything needed to be tested and thoroughly vetted in time for BFCM, with no exceptions and no ability to compromise

Features and capabilities

Once it was decided that it was necessary to look beyond their RDS installation to meet the demands of the upcoming holiday season, it became clear that organization-wide the following were required in a new solution:

• Scale infinitely, rewrite nothing: There was a strong desire to not have to rewrite application code in order to migrate databases

• Managed horizontal sharding: Attentive didn’t just want something “serverless,” they wanted something modern that would be able to help them scale as necessary

• Unlimited connection pooling: No more limits on how many services can connect to the database at one time

• 99.99% availability: Not willing to sacrifice any customer SLAs in order to gain convenience or performance in other areas. Uptime is an absolute must

• Performance SLAs met: Strict time-based SLAs with customers must be upheld

• BFCM benchmarks beat: The entire system needed to be able to handle a 10× increase in volume, including the database

• Controlled costs: Understanding the costs involved via transparent pricing and not having to hire Database Administrators

Choosing PlanetScale

We considered sharding our Postgres installation, but that created hot spots and more issues than we wanted to deal with. We looked at CockroachDB, but it wasn’t the right fit for what we needed. We did the Proof of Concept, things looked great, and we were able to hit all of our latency targets.

When the team looked around for potential solutions, after working through a Proof of Concept and seeing how everything worked out, PlanetScale quickly moved to the top:

• Mission critical databases all eligible: The most important databases that Attentive relied on in the “hot path” were all going to be straightforward to migrate and operate with PlanetScale

• Horizontal, “infinite” scalability: PlanetScale unlocking the power of Vitess meant that Attentive had other options to scale beyond “simply buying a larger instance”

• Connection pooling not an issue: PlanetScale configuration could allow for the large number of simultaneous connections Attentive required

• Data SLA performance targets met: PlanetScale was fast enough at scale to meet Attentive’s stringent performance requirements

• Other products couldn’t meet targets: Other products considered, including CockroachDB, either couldn’t meet scalability requirements, performance requirements, or both

• Support and communication: Superior support SLAs and communication with the company proved to be much better than their experience with AWS

PlanetScale migration

We were concerned with how our table partitioning in RDS would translate to our PlanetScale installation, but that turned out to be a non-issue.

To recap, Michael’s team and the rest of Attentive’s engineering organization were facing a Black Friday/Cyber Monday weekend, the busiest days of the year for all of their customers. They had some AWS instances running very large RDS instances, performance issues wasting developer time and causing occasional outages, and on top of that, mandates to handle 10× throughput and stringent data performance requirements in order to be confident going into BFCM.

There was a period of adjustment as sharding strategies from Postgres were ironed out to work properly in PlanetScale and the MySQL world, but once the eligible databases were selected and migrated, including the largest and most critical in the organization, things ran very smoothly. Attentive was able to beat their benchmarks and gain confidence that when the time came, their infrastructure would be capable of handling the scaling and performance demands.

Experiences and impact

PlanetScale was not on our list of concerns during Black Friday/Cyber Monday. The new guidance for database selection internally is, if it’s a new project that’s high volume or large scale, we’re looking at PlanetScale first.

Michael’s team and several others at Attentive migrated to PlanetScale before Black Friday/Cyber Monday, one of their biggest days every year. How did it work out, and what are Attentive’s plans for PlanetScale in the future?

• Migration was delightfully uneventful and brownfield projects are a good fit for PlanetScale.

• Black Friday/Cyber Monday was a resounding success for Attentive’s customers, and therefore for Attentive. Over the 32 shard cluster, they were able to sustain 180k QPS all day, including peaks of up to 400k

• Surprise workload handled easily: Nearly 120k QPS were coming from an internal service that wasn’t anticipated, but PlanetScale handled the workload with ease

• PlanetScale is the choice for projects that need to scale and meet traffic and data SLA demands

• Support and product experience greatly improved over AWS

• Developer experience and productivity has been a blessing in disguise for teams switching over to PlanetScale. By taking advantage of features like PlanetScale Insights the team is less distracted by performance issues and putting out downtime fires

• Scalability goals have been met and teams at Attentive can confidently employ PlanetScale for their critical path services knowing that they can horizontally scale to meet even the most rigorous of demands